A qualitative evaluation of a Nudgeathon event for the co-design of sexual health campaign images targeting overseas-born men who have sex with men

Keywords: Australia, behavioural economics, co-design, evaluation, HIV, MSM, nudge, pre-exposure prophylaxis, United Kingdom.

In settings of subsidised access to pre-exposure prophylaxis (PrEP) to prevent HIV, such as the United Kingdom (UK) and Australia, HIV rates have declined in men who have sex with men (MSM).^1,2 However, this decline has not been observed in overseas-born MSM living in Australia but who are not residents (hereafter referred to as overseas-born MSM). Unlike MSM who are Australian residents, overseas-born MSM do not pay a Medicare levy and therefore, some are ineligible for subsidised PrEP under the Pharmaceutical Benefits Scheme. Moreover, for some overseas-born MSM, PrEP-related costs are not covered by their health insurance providers.³ In addition to cost as a barrier, overseas-born MSM can face many other barriers that impact their access to PrEP, including cultural factors that influence their knowledge, attitudes, and beliefs about HIV risk and prevention.^4,5

Additional approaches to HIV prevention and management are needed to create interventions that positively impact not only PrEP-related knowledge, attitudes, and beliefs but also behaviour change that is conducive to efforts that aim to increase access to PrEP for overseas-born MSM living in Australia. A Nudgeathon, a type of co-design workshop, offers an approach to creating a behavioural nudge. A nudge is defined as any aspect of the choice architecture (i.e., organising the context in which people make decisions) that predictably alters behaviour without forbidding any options or significantly changing one’s economic incentives.⁶

While past human-centred design approaches have involved co-designing interventions for HIV prevention^7–9 and HIV care,¹⁰ rarely do they engage MSM using online platforms,⁹ and none have specifically focused on altering the choice architecture that may influence decision-making. Past Nudgeathons have been conducted as in-person events lasting 1–2 days where groups address issues, such as increasing hand hygiene¹¹ and reducing plastic waste.¹²

Given co-designing workshops have recently transitioned from face-to-face to online due to the coronavirus disease 2019 pandemic,¹³ evaluations of these online workshops are needed. Through qualitative interviews, we aimed to evaluate the processes of an online Nudgeathon, which aimed to co-design a digital advertisement campaign to improve awareness of and access to HIV PrEP among overseas-born MSM.

An online 4-h Nudgeathon was organised and conducted via the Zoom platform in September 2021. Twenty people attended, including four experts in PrEP and eight overseas-born MSM who lived or worked for less than 5 years in Australia (i.e. ‘recently arrived’). The eight overseas-born MSM were recruited by a clinician who regularly consults with overseas-born MSM, including prescribing PrEP. The four experts in PrEP included two clinicians, one sexual health researcher, and one leader from a local community-based organisation. In addition, four behavioural economists and four graphic designers attended from the UK. The four behavioural economists are part of the research team, including three who were the creators of Nudgeathons. The behavioural economists acted as the group facilitators. The graphic designers were recruited through the contacts of one of the behavioural economists.

Attendees were divided into four teams, each team consisting of: (1) one behavioural economist; (2) one graphic designer; (3) one PrEP expert; and (4) two overseas-born MSM. Before the Nudgeathon, participants were provided a problem brief with information about PrEP in Australia and the target audience: overseas-born MSM (see Supplementary Material File S1). The Nudgeathon event consisted of three parts: (1) a behavioural science Masterclass to inform participants of successful nudges and how to create nudges; (2) group work to co-design a digital advertisement campaign to improve awareness of and access to HIV PrEP among overseas-born MSM; and (3) group presentations.

After the Nudgeathon, 17 attendees were invited to be interviewed about their Nudgeathon experience and those who were interested were asked to provide their signed informed consent to participate in the interview. Participants were briefed about the evaluation study and were given the opportunity to ask questions to seek clarification before commencing their interview. Semi-structured interviews were guided by an open-ended schedule to explore their views and opinions of the Nudgeathon (see Supplementary Material File S2). One researcher (JTr) conducted the interviews via Zoom within 3 weeks after the Nudgeathon. Participants were reimbursed AU$50 for their participation. Audio recordings of the interviews were transcribed with any personally identifiable information redacted. Participants were provided access to their interview transcripts should they want to adjust their responses to the interview questions; however, none requested to make any adjustments. Audio recordings of the interviews were securely stored to check for accuracy during the analysis. This qualitative evaluation was approved by the Human Research Ethics Committee of the Alfred Hospital, Melbourne, Australia (447/21).

We conducted a qualitative content analysis¹⁴ of interview data on the experiences of the Nudgeathon attendees. We examined and compared themes or categories (hereafter referred to as themes) from the interview data by systematically classifying or coding themes, and in turn, identifying core consistencies and meanings within the qualitative data.¹⁵ Qualitative content analysis was a suitable analytical approach to identify themes in our interview data because our research questions were not highly open-ended.¹⁶

After reviewing a portion of the interview transcripts, JTr and JTa developed codes for themes inductively using initial codes (based on topics in the semi-structured interview guide) and additional codes (codes we identified).¹⁷ Where there were discrepancies, JTr and JTa reviewed these discrepancies and iteratively refined the codes. Some themes were merged into main themes after regular discussions between JTr and JTa.¹⁷ The transcribed interviews were imported into Atlas.ti software (ATLAS.ti Scientific Software Development GmbH) where the interviews were coded for themes. The codes identified themes as they were experienced by the interviewees, which were informed by their ideas, beliefs, and framing regarding the processes carried out during the Nudgeathon, their own contributions to their team, working with others, and satisfaction with their end-products.

Five overseas-born MSM, two behavioural economists, and one graphic designer consented to participate in the interview. The MSM were born in various countries, including Colombia, Indonesia, Japan, Malaysia and Vietnam. The interviewees were aged between 23 and 41 years old. The duration of the interviews ranged from 20 to 30 min. For most of the interviewees, it was their first Nudgeathon, and they found the Nudgeathon to be an enjoyable and positive learning experience.

We identified three major themes and eight subthemes from the interviews: (1) what worked well; (2) group dynamics; and (3) improvement and future directions.

Views and opinions varied on the positive aspects of or highlights from the Nudgeathon. Interviewees shared that, in general, the Nudgeathon ‘went well’ and that the goal for the day was clearly communicated and understood through the problem brief.

Most interviewees (n = 6) conveyed that attending the Nudgeathon was a good experience, where they learned about new ideas and concepts.

I thought that worked quite well because I’m not the expert…in PrEP and in medicine and in the gay community…I never knew that medicine has come so far that there is basically a pill. (Female, 41 years old, graphic designer, no previous experience with Nudgeathons)

For me, the highlight definitely was learning about what nudging is. (Male, 23 years old, MSM born in Indonesia, no previous experience with Nudgeathons)

There was an emphasis on the impact of team members with different professional backgrounds and personal experiences involved in the Nudgeathon. The grouping of overseas-born MSM, behavioural economists, and graphics designers in teams was positively received and achieved unexpected, pleasant outcomes.

We genuinely maxed out the diversity in this Nudgeathon … in some previous Nudgeathons usually you might have two or three teams who might come up with a similar idea. In this one that we had four different takes. That was quite nice actually. That really stood out for me. (Male, 33 years old, facilitator, experienced in Nudgeathons)

Among interviewees who had not attended Nudgeathons before, some commented that the 4-h commitment to the Nudgeathon was sufficient, and others commented that more time was needed, but none noted the event being hosted online. The online nature of the Nudgeathon provided allowed attendees to take part across time zones (the UK and Australia). It also rendered the event less costly to participants who otherwise would have been required to travel overseas to participate. In-person Nudgeathons with international attendees would have required costs for flights, accommodation, function room hire and catering.

This is one of those instances where doing it online has a lot of added value. We [UK participants] would not have been able to participate. Just for this we might not have travelled [as there are] lots of cost involved. Also, the designers who are from the UK [and] they certainly wouldn’t have been able to participate. Doing this online, you could involve people who we might not have otherwise. (Male, 33 years old, facilitator, experienced in Nudgeathons)

All interviewees were satisfied with the campaign images they helped co-design and with their team’s collaborations, despite some feeling initial pressure (n = 4).

I was very stressed, under pressure at the beginning. I did enjoy it because when I saw the ad my team formed…I feel we achieved.. we contributed ideas to create ads. When I saw it visually, I felt very satisfied. (MSM born in Japan, 36 years old, no previous experience with Nudgeathons)

I was thinking the creative part is going to sit on my shoulders and I have to like show something at the end, but I was really positively surprised that you can turn something around within 4 hours with people on two different continents. (Female, 41 years old, graphic designer, no previous experience with Nudgeathons)

The positive group dynamics were mostly due to how the behavioural economist, graphic designer and overseas-born MSM roles were somewhat predefined.

There was an overwhelming emphasis on the contributions of graphic designers to the co-design of the digital ad campaign. However, the contributions from the other team members were also considered valuable.

Having the freelance graphic designer was a good idea, because [otherwise] the visual aspect would have been really bad. I think, based on the team that I was put into, they were really great. There was another international student. He was very passionate about the issue. You can definitely tell that everyone is very into the topic. (MSM born in Indonesia, 23 years old, no previous experience with Nudgeathons)

All interviewees could freely express their ideas to contribute to the co-design of the campaign. From the perspective of a facilitator and graphic designer, they were aware of their roles and contributions. However, the personal experiences of overseas-born MSM regarding PrEP essentially helped shape the campaign images.

I talked about my personal experience with the PrEP and all those kinds of things and so many people around me who have had this experience, and those ideas I think were really helpful when constructing the concepts we were trying to shift a final result. (MSM born in Colombia, 27 years old, no previous experience with Nudgeathons)

A barrier to access PrEP is people might think you’re just taking this only for having sex or just you want to do orgy or you want to do bareback, those stereotypes. I think that’s a challenge. That’s why I suggested my idea. It doesn’t have to be a celebrity to be sexual. It doesn’t have to be gay. It can be accessed by anyone. (MSM born in Japan, 36 years old, no previous experience with Nudgeathons)

Despite the consensus on the Nudgeathon being a collaborative and productive event, some critiques and suggestions can be considered for future improvement and direction. These critiques and suggestions focused on the information about the issues regarding PrEP provided to the participants before the event, the Masterclass where the framework was presented, and the possibility of events conducted both in person and online.

There was expression for more information about PrEP to be provided to attendees before the Nudgeathon event, especially for the overseas-born MSM.

There could’ve been more information about what they [participants] would experience at the Nudgeathon to prepare in advance. There could’ve been information about what materials people might benefit from bringing with them to the event. (Female, 37 years old, facilitator, experienced in Nudgeathons).

The Masterclass presented most interviewees with interesting new ideas and concepts. However, some believed that the amount of information presented could be overwhelming and that the presentation could have been shorter than the designated full hour.

[The masterclass] was hugely interesting… It could have been cut a bit short…We could have probably done with one or two videos less, I think personally. (Female, 41 years old, graphic designer, no previous experience with Nudgeathons)

The possibility of hosting future Nudgeathons as ‘hybrid’ events was suggested. However, hybrid events are most likely inefficient, as those who attend events in person are likely to interact more with each other and less with those online.

I am not too convinced by the efficiency of hybrid because if you have half people in-person and half online, the half online will always be left out. Those who are in-person will be talking more to each other. (Male, 33 years old, facilitator, experienced in Nudgeathons)

Through semi-structured interviews, we explored the experiences of individuals who participated in a 4-h online Nudgeathon. The views and opinions shared represented three different lenses: the behavioural economist (who also acted as the facilitator), the graphic designer, and the overseas-born MSM. Placing facilitators, graphic designers and overseas-born MSM in the same team proved synergistic and fruitful, offering a collaborative environment and a space to share professional and personal experiences that essentially informed the campaign. While most past Nudgeathons have been hosted as in-person events of 1–2 days, the present evaluation suggests similar co-designs can emerge for online Nudgeathons in less than 1 day. For international collaborations, the online format may improve participation and minimise costs.

A key strength of the current evaluation is that the interviews about participants’ experiences were conducted a short time (up to 3 weeks) after the workshop, which allowed participants time to reflect on their experiences. This differed from a previous evaluation of an online workshop co-designing a project to prevent poor mental health among farmers, where activity cards were used to stimulate participants’ reflections right after their workshops, and these reflections were collated based on design principles.¹³ Previous reports on co-designing HIV interventions have appropriately conducted interviews prior to the actual workshops as formative research, but post-workshop evaluations of the co-design experience were not included.^7,8 Post-workshop interviews may be valuable to understand how the co-design process facilitates intervention design and to improve the conduct of future workshops.

Our evaluation found that interviewees who did not have previous experience with Nudgeathons did not mention the online nature of the Nudgeathon in which they participated. This suggests that they might have believed that online Nudgeathons are standard practice. There is a possibility that with more people interviewed, perhaps some may note differences in in-person and online modes. However, considering there were only four teams, it is unlikely that someone without prior experience with Nudgeathons will notice differences in delivery modes, even with equal representation of all three perspectives.

Several limitations of the present evaluation should be noted. Ideally, we would prefer more interviewees. Although the perspectives of the facilitator, the graphic designer and the overseas-born MSM were represented, there was an underrepresentation of graphic designers, and a few more voices from graphic designers could have offered different perspectives. Also, the interviews were conducted online through Zoom, and this may have limited opportunities to build better rapport that would have been otherwise possible with in-person face-to-face interviews.

PrEP as HIV prevention can be a controversial topic, as it carries stigma and shame associated with promiscuity, condomless anal sex, and chemsex (or sexualised drug use),^18–21 and therefore, not all Nudgeathon participants will share personal experiences regarding PrEP. Future Nudgeathons that focus on controversial or sensitive topics may want to allow more time for participants to build rapport with each other and to feel more comfortable with sharing their personal stories. Sharing personal stories, especially stories related to the workshop topic, can ideally help develop behavioural interventions.¹³

Conducting online workshops to co-design behavioural interventions to prevent HIV in overseas-born MSM may have some areas for improvement, such as more information about the topic before the Nudgeathon and shorter presentations during the Nudgeathon. However, our evaluation shows that online co-design workshops can be a fast and efficient option for developing potential solutions to complex and sensitive issues.

Supplementary material is available online.

The data that support this study cannot be publicly shared due to ethical or privacy reasons and may be shared upon reasonable request to the corresponding author if appropriate.

JJO and EPFC are editors of Sexual Health, but neither had a role in the editorial handling of this manuscript.

The project is funded by Gilead Pharmaceuticals. EPFC are JJO are supported by an Australian National Health and Medical Research Council (NHMRC) Emerging Leadership Investigator Grant (GNT1172873 for EPFC and GNT1104781 for JJO). JT is supported by the Australian Government Research Training Program (RTP) Scholarship. The funder was not involved in the data analysis, manuscript preparation, and decision to submit the manuscript for publication.

JJO conceived the idea for the Nudgeathon and its evaluation. JJO, IV, DR, KS, EPFC, and DL planned and conducted the Nudgeathon. JTr assisted with the construction of the semi-structured interview schedule, conducted the interviews, performed the thematic analysis, and wrote the first draft of the manuscript. JTa assisted with thematic analysis. All authors were involved in revising the manuscript for important intellectual content and approved the final version.