Dear Diary…

If diary is a piece of record to spell one’s heart out, then diary study attempts to extract one’s feeling towards interacting with a product / service.

Recently we conducted a diary study on a new mobile app. The participants were asked to download the app on their phones and use it as they may, for about 10 days. Along the way, they were given a couple of mini surveys where they could share their opinion and experience with the app.

Diary study lies heavily on self-report from the participants, with minimal supervision from the research analyst. However, when dealing with major Indonesian users — not the most independent explorers — it takes careful preparation so we don’t end up overworking the supervision bit.

1. Face-to-face briefing

Indonesians are very used to personal guidance. It applies to the diary setup, as well.

• What’s the study about?
• Signing the NDA?
• Downloading the app?
• Accessing the survey?
• Certain phone setting that affect the app usage?

It’s very very important to address all those questions, doubts, how-to’s upfront, in face-to-face meeting with the participants. Investing a day or two to tackle this at once will worth so much more than handling trivial questions from any participant every day along the study.

2. Clear deliverables and expected efforts

Participants signing up to the study may have variety understanding of what they are asked to deliver. Stating “a total of 4 mini surveys to fill out every 3 days” simply doesn’t cut out. We need to be more specific on the effort required from the participant’s side.

• What are the activities?

Whether it involves typing, recording, taking pictures, reporting bug, or else, list out as detailed as possible. If participants are not familiar with any of the activities, be prepared to provide step-by-step guidelines.

• How long does it take?

We’re asking participants to take some time out of their regular schedule, so we need to guarantee that “the daily 5-minutes diary” really takes no more than 5 minutes. Whenever possible, try piloting at least with the internal team.

• When do I need to fulfill it?

Still related to the previous point, participants appreciate knowing the exact time window to submit the diary. This will help them plan their days ahead. Create a calendar of activities to share with them.

• Any special circumstances?

If we shall ask participants to handle certain circumstances (activities, tools, time, location) out of their normal routine, we need to ensure it’s still relevant and they can do it as natural as possible.

The best way to set the expected efforts is to show participants a sample diary with clear items to deliver, ideally during face-to-face briefing.

3. One contact point along the study

Just like the case with customer service, nobody wants to deal with multiple contacts and being tossed around from one to another. Create a dedicated communication channel that works best for the participants.

In our study, we used email primarily for distributing general notification. Turned out there were so many questions raised from the participants, thus we decided to open a mobile contact to follow up more quickly. Since the particular study was for pairs, we created a WhatsApp group for each pair to contact both participants at once and get them to help each other.

Providing multiple channels, we need to align them so that participants feel they’re still in contact with the same ‘person’ regardless the channel. At one point during our study, participants were asked to submit a video. Some sent it through email, some through WhatsApp. We managed an online spreadsheet to track participants’ deliverables, so everyone in the team can see the progress realtime.

One thing we forgot to set in the beginning was the contact point’s ‘office hour’. A few participants called us at 10 pm asking how to submit the survey! Clearly state the time window of the contact point’s availability, align it with the time window of the diary submission. Tips: 2 hours before the submission deadline is typically the busiest time.

4. Final interview to wrap-up

Looking into the diary entry, there’s only so much we can tell about the participants’ experience. Final interview is the only chance to dig deeper and clarify any particular curiosity.

Before the interview, prepare a summary about the participants — profile, activities (if tracking is available), and the submitted diary entries. Note things to follow up, so we can maximize the time to uncover what hasn’t been told before.

Confirm the interview schedule with participants at least 3 days in advance, and remind them again the day before. Ask them to bring the study material, i.e. the app in their phones. Prepare the interview environment to get the most out of the sessions.

• Recording

We set up two cameras — one to record the participants’ facial expression and the other to record the participant’s phone. Be weary of the participant’s privacy, ask them to put their phones under the camera only for showing their interaction within the app.

• Note taking

Assign a person to take notes during the session, for easier debrief afterwards. Focus on which parts of the app that participants like, don’t like, and ideas for improvement.

• Power outlet

Make sure there are enough power outlets, and keep the devices plugged during the session — laptop, video camera, TV, whatever required. Set aside extra power outlets for participants’ devices as well.

Conclusion

Conducting diary study with Indonesian users gets us to think about the usability of the study setup itself. Understanding the participants and how putting the study in relevance to their daily activities is key for getting rich results. In many cases, it’s safer not to assume that users can do it on their own in first attempt.