Diary studies

(Redirected from Diary studies in user research) Research method

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Diary studies" – news · newspapers · books · scholar · JSTOR (September 2012) (Learn how and when to remove this message)

Diary studies is a research method that collects qualitative information by having participants record entries about their everyday lives in a log, diary or journal about the activity or experience being studied. This collection of data uses a longitudinal technique, meaning participants are studied over a period of time. This research tool, although not being able to provide results as detailed as a true field study, can still offer a vast amount of contextual information without the costs of a true field study. Diary studies are also known as experience sampling or ecological momentary assessment (EMA) methodology.

Traditionally diary studies involved participants keeping a written diary of events. However the emergence of smartphones now enables participants to diary with photos, videos and text using a variety of online or offline apps and tools. Since the diary studies are recorded sequentially over time, it can be used to investigate time-based phenomena, temporal dynamics, and fluctuating phenomena such as moods.

Diary studies can also be employed together with other research techniques within a mixed method framework and is particularly useful in obtaining rich subjective data. For instance, experience sampling method (ESM) combines it with questionnaires to gather data and examine people's experiences in daily life.

History

An early example of a diary study was "How Workingmen Spend Their Time" (Bevans, 1913), which went unpublished by George Esdras Bevans.

Background

Diary studies originate from the fields of psychology and anthropology. In the field of human–computer interaction (HCI), diary studies have been adopted as one method of learning about user needs towards designing more appropriate technologies.

Temporal processes in diary studies

A key characteristic of diary studies is their ability to track daily events over time. Researchers have begun conducting studies that allow them to explore the connection between a previous day's events and a current day's outcome, or to what extent prior events make people responsive to current events. Researchers Robert E. Wickham and C. Raymond Knee have concluded that future research studies would benefit from evaluating temporal processes, or processes related to time, in diary studies. This would serve as a way for researchers to test unique questions and hypotheses.

Evaluating in-person change in diary

Researchers have been able to use diary studies to evaluate how people can change over time. Traditional diary studies have evaluated change between individuals, but new studies have been conducted to evaluate within-person changes using diary studies. Through a framework of the generalizability theory, researchers have used a condensed version of the Profile of Mood States (POMS) to study within-person emotional changes via diaries.

Types of diary studies

Feedback studies

Feedback studies involve participants answering predefined questions about the phenomenon of interest in a natural setting, with the answers acting as a diary entry. This is usually at assigned times, frequencies, or occurrences of the phenomenon, stated by the researcher. The most common method is using ‘paper and pencil’, although there are some studies that both utilise and suggest other technologies such as mobile phones or Psion Organisers. As such, feedback studies involve asynchronous communication between the participants and the researchers, as the participants’ data is recorded in their diary first, and then passed on to the researchers once complete.

Feedback studies are scalable - that is, a large-scale sample can be used, since it is mainly the participants themselves who are responsible for collecting and recording data.

Elicitation studies

In elicitation studies, participants capture media as soon as the phenomenon occurs - the media is usually in the form of a photograph, but can be in other different forms as well, and so the recording is generally quick and less effortful than feedback studies. These media are then used as prompts and memory cues - to elicit memories and discussion - in interviews that take place much later. As such, elicitation studies involve synchronous communication between the participants and the researchers, usually through interviews.

In these later interviews, the media and other memory cues (such as what activities were done before and after the event) can improve participants’ episodic memory. In particular, photos were found to elicit more specific recall than all other media types.

Comparisons

There are two prominent trade-offs between each type of study. Feedback studies involve answering questions more frequently and in situ, therefore enabling more accurate recall but more effortful recording. In contrast, elicitation studies involve quickly capturing media in situ but answering questions much later, therefore enabling less effortful recording but potentially inaccurate recall.

When to use diary studies

Diary studies are most often used when observing behavior over time in a natural environment. They can be beneficial when one is looking to find new qualitative and quantitative data. Diary studies aim to measure people's behavior over an extended period. They provide the opportunity for exploratory research, collecting large quantities of precise data that are both in-depth and contextual. What makes diary studies particularly unique is that these substantial amounts of data are collected at the micro-level. When the subject of research undergoes a change, utilizing diary studies becomes interesting because they allow for the measurement of change over an extended period and the observation of its effects on the individual; this is referred to as within-subject analysis. Additionally, it is possible to conduct between-subjects analysis to observe the different effects among respondents. A diary study offers the advantage over a traditional survey study in that it allows for the collection of data on a daily basis or even multiple times a day. In contrast, a survey study typically gathers data at a single point in time, or in the case of a longitudinal study, with time lags spanning months or years.

Advantages

Advantages of diary studies are numerous. They allow:

collecting longitudinal and temporal information;
reporting events and experiences in context and in-the-moment;
participants to diary their behaviours, thoughts and feelings in-the-moment thereby minimising the potential for post rationalisation;
determining the antecedents, correlations, and consequences of daily experiences and behaviors.
respondents are not coerced
regular reporting leads to a rich data collection
an autonomous report is created, with no influence of social desirability
it is unobtrusive: there is little distortion due to setting or investigations
it is suitable for Small-n, medium-n, or even large-n (depending on the purpose and analysis method)
it is a unique window on human phenomenology

Limitations

There are some limitations of diary studies, mainly due to their characteristics of reliance on memory and self-report measures.

There is low control, low participation and there is a risk of disturbing the action. In feedback studies, it can be troubling and disturbing to write everything down. This is called a respondent burden.

Inaccurate recall

The validity of diary studies rests on the assumption that participants will accurately recall and record their experiences. This is somewhat more easily enabled by the fact that diaries are completed & media is captured in a natural environment and closer, in real-time, to any occurrences of the phenomenon of interest. However, there are multiple barriers to obtaining accurate data, such as:

Social desirability bias - where participants may answer in a way that makes them appear more socially desirable. This may be more prominent in longitudinal studies where participants frequently communicate with researchers.
Recall bias - where participants recall events or feelings inaccurately due to systematic errors. Human memory is unreliable, as it is an active reconstruction which is susceptible to errors and biases. For example, after recording their emotions in daily diaries, older adults tended to recall more positive emotions than younger adults, while younger adults tended to recall more negative emotions than older adults.
The phenomenon of interest itself - participants may find it difficult to both record and experience the phenomenon accurately at the same time. For example, a diary study assessing the drinking urges of recently treated alcoholics found that, if/when participants began to drink alcohol, they tended to stop recording in their diary.
Reactive bias - As people are required to keep a diary, they may start thinking differently. This can lead to the Hawthorne effect.

Non-compliance and retrospective entries

Diary studies tend to dictate specific times or a frequency at which the participant should complete their diary entries or capture media. This is usually a time that is close to the event or phenomenon of interest to the researcher, and it is assumed that participants will comply with these instructions. However, researchers often find it difficult to verify the extent to which this assumption is met, and have observed various forms of non-compliance. One example of this is hoarding, where previously missed entries are 'backfilled' or completed all at once, therefore being completed retrospectively rather than at the assigned time. A further example of this is where entries are 'forward-filled', i.e. recorded for a future date.

Researchers have found significant rates of non-compliance and entries written retrospectively in feedback studies. A study by Hyland and colleagues (1993) estimated that the percentage of errors in paper and pencil diaries could be anywhere from 2-24%, due to diaries written retrospectively and/or due to inaccuracies in recording. Another study by Stone and colleagues (2003) compared paper diaries versus compliance-enhancing electronic diaries. The paper diaries contained a hidden instrument which detected when the diary was opened - from this, actual compliance rate was found to be only 11%, while 79% of entries were faked, i.e. not written within 30 minutes of the assigned time. In contrast, the electronic diaries created timestamps for each entry, which prevented the possibility of fake timestamped entries, and therefore yielded a compliance rate of 94%.

Notable diary studies

In 2015, a diary study was conducted at a Dutch University of Applied Sciences to evaluate how learning spaces affect students' learning activities. 52 business management students kept records of the learning activities that they worked on, where they worked on them, and why they worked there for a week. Through evaluating the diary entries of this study, researchers found a significant correlation between the spaces in which students chose to work and their learning activities.

Tools

PACO is an open source mobile platform for behavioral science.

References

Sullivan, Brian K. (August 9, 2012). "5 methods to collect data with diary studies". Big Design. Retrieved September 24, 2016.
Flaherty, Kim (June 5, 2016). "Diary Studies: Understanding Long-Term User Behavior and Experiences". Nielsen Norman Group. Retrieved September 24, 2016.
Wildemuth, Barbara (2016). Applications of Social Research Methods to Questions in Information and Library Science, 2nd Edition. Santa Barbara, CA: ABC-CLIO. p. 230. ISBN 9781440839047.
Tavakoli, Hossein (2012). A Dictionary of Research Methodology and Statistics in Applied Linguistics. Rahnama Press. p. 163. ISBN 9789643675080.
Allen, Mike (2017-01-15). The SAGE Encyclopedia of Communication Research Methods. SAGE Publications. ISBN 9781483381459.
"Bevans, G.E. (1913). How Workingmen Spend Their Time (Unpublished Doctoral Thesis). Columbia University, New York, NY"
^ Carter, Scott; Mankoff, Jennifer (2005). "When participants do the capturing". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Chi '05. Portland, Oregon, USA: ACM Press. pp. 899–908. doi:10.1145/1054972.1055098. ISBN 978-1-58113-998-3. S2CID 3338852.
Palen, L., & Salzman, M. (2002). Voice-mail diary studies for naturalistic data capture under mobile conditions. In Proceedings of Computer Supported Cooperative Work (CSCW '02), pp. 87–95. New York: ACM.
Wickham, Robert E.; Knee, C. Raymond (2013-06-24). "Examining Temporal Processes in Diary Studies". Personality and Social Psychology Bulletin. 39 (9): 1184–1198. doi:10.1177/0146167213490962. PMID 23798376. S2CID 6285145.
Cranford, James A.; Shrout, Patrick E.; Iida, Masumi; Rafaeli, Eshkol; Yip, Tiffany; Bolger, Niall (2016-07-02). "A Procedure for Evaluating Sensitivity to Within-Person Change: Can Mood Measures in Diary Studies Detect Change Reliably?". Personality and Social Psychology Bulletin. 32 (7): 917–929. doi:10.1177/0146167206287721. PMC 2414486. PMID 16738025.
^ Hyland, M E; Kenyon, C A; Allen, R; Howarth, P (1993-02-20). "Diary keeping in asthma: comparison of written and electronic methods". BMJ. 306 (6876): 487–489. doi:10.1136/bmj.306.6876.487. ISSN 0959-8138. PMC 1676776. PMID 8448458.
^ Gouveia, Rúben; Karapanos, Evangelos (2013). "Footprint tracker". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Chi '13. Paris, France: ACM Press. pp. 2921–2930. doi:10.1145/2470654.2481405. ISBN 978-1-4503-1899-0. S2CID 6343495.
^ Sohn, Timothy; Li, Kevin A.; Griswold, William G.; Hollan, James D. (2008). "A diary study of mobile information needs". Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Florence, Italy: ACM Press. pp. 433–442. doi:10.1145/1357054.1357125. ISBN 978-1-60558-011-1. S2CID 22118405.
"How to Do Diary Studies, a Great Alternative to Field Studies". www.userinterviews.com. Retrieved 2020-03-02.
^ Houtgraaf, Glenn (2023-08-03). "Public sector creativity: triggers, practices and ideas for public sector innovations. A longitudinal digital diary study". Public Management Review. 25 (8): 1610–1631. doi:10.1080/14719037.2022.2037015. hdl:2066/284675. ISSN 1471-9037.
Ohly, Sandra; Sonnentag, Sabine; Niessen, Cornelia; Zapf, Dieter (January 2010). "Diary Studies in Organizational Research". Journal of Personnel Psychology. 9 (2): 79–93. doi:10.1027/1866-5888/a000009. ISSN 1866-5888.
^ Olorunfemi, Doyin (2023-12-17). "Diary studies in research: More than a research method". International Journal of Market Research. 66 (4): 410–427. doi:10.1177/14707853231222139. ISSN 1470-7853.
Lallemand, C. (2012). Dear Diary: Using Diaries to Study User Experience UX Magazine 11.3, August 2012.
Kuniavsky, Observing the User Experience, A Practitioner's Guide to User Research September 21, 2012, ISBN 0123848695.
Legg, C.; Puri, A.; Thomas, N. (2000). "Dietary restraint and self-reported meal sizes: diary studies with differentially informed consent". Appetite. 34 (3): 235–243. doi:10.1006/appe.2000.0314. PMID 10888286. S2CID 12303779.
^ Stone, Arthur A; Shiffman, Saul; Schwartz, Joseph E; Broderick, Joan E; Hufford, Michael R (2003). "Patient compliance with paper and electronic diaries". Controlled Clinical Trials. 24 (2): 182–199. doi:10.1016/S0197-2456(02)00320-3. PMID 12689739.
Ready, Rebecca E.; Weinberger, Mark I.; Jones, Kelly M. (2007). "How happy have you felt lately? Two diary studies of emotion recall in older and younger adults". Cognition & Emotion. 21 (4): 728–757. doi:10.1080/02699930600948269. ISSN 0269-9931. S2CID 143775879.
Litt, Mark D.; Cooney, Ned L.; Morse, Priscilla (1998). "Ecological momentary assessment (EMA) with treated alcoholics: Methodological problems and potential solutions". Health Psychology. 17 (1): 48–52. doi:10.1037/0278-6133.17.1.48. ISSN 1930-7810. PMID 9459069.
Beckers, Ronald; Voordt, Theo van der; Dewulf, Geert (2016-01-02). "Why do they study there? Diary research into students' learning space choices in higher education". Higher Education Research & Development. 35 (1): 142–157. doi:10.1080/07294360.2015.1123230. ISSN 0729-4360. S2CID 146488726.
"Home". pacoapp.com.

External links

Categories:

Misplaced Pages