Free Standard AU & NZ Shipping For All Book Orders Over $80!
Register      Login
Sexual Health Sexual Health Society
Publishing on sexual health from the widest perspective
Shopping Cart: ( empty )
You are here: Home > Journals > SH > SH22016 > Fulltext
REVIEW
Contents Vol 19(5)

Effectiveness and acceptability of conversational agents for sexual health promotion: a systematic review and meta-analysis

Divyaa Balaji https://orcid.org/0000-0002-0654-467X A * , Linwei He B , Stefano Giani C , Tibor Bosse D , Reinout Wiers E and Gert-Jan de Bruijn F
+ Author Affiliations
- Author Affiliations

A Amsterdam School for Communication Research, University of Amsterdam, Amsterdam, the Netherlands.

B Department of Communication and Cognition, Tilburg University, Tilburg, the Netherlands.

C University Library, University of Amsterdam, Amsterdam, the Netherlands.

D Behavioural Science Institute, Radboud University, Nijmegen, the Netherlands.

E Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands.

F Department of Communication Science, University of Antwerp, Antwerp, Belgium.

* Correspondence to: d.balaji@uva.nl

Handling Editor: Matthew Hogben

Sexual Health 19(5) 391-405 https://doi.org/10.1071/SH22016
Submitted: 30 January 2022  Accepted: 16 June 2022   Published: 22 July 2022

© 2022 The Author(s) (or their employer(s)). Published by CSIRO Publishing

Abstract

Digital health interventions for sexual health promotion have evolved considerably alongside innovations in technology. Despite these efforts, studies have shown that they do not consistently result in the desired sexual health outcomes. This could be attributed to low levels of user engagement, which can hinder digital health intervention effectiveness, as users do not engage with the system enough to be exposed to the intervention components. It has been suggested that conversational agents (automated two-way communication systems e.g. Alexa) have the potential to overcome the limitations of prior systems and promote user engagement through the increased interactivity offered by bidirectional, natural language-based interactions. The present review, therefore, provides an overview of the effectiveness and user acceptability of conversational agents for sexual health promotion. A systematic search of seven databases provided 4534 records, and after screening, 31 articles were included in this review. A narrative synthesis of results was conducted for effectiveness and acceptability outcomes, with the former supplemented by a meta-analysis conducted on a subset of studies. Findings provide preliminary support for the effectiveness of conversational agents for promoting sexual health, particularly treatment adherence. These conversational agents were found to be easy to use and useful, and importantly, resulted in high levels of satisfaction, use and intentions to reuse, whereas user evaluations regarding the quality of information left room for improvement. The results can inform subsequent efforts to design and evaluate these interventions, and offer insight into additional user experience constructs identified outside of current technology acceptance models, which can be incorporated into future theoretical developments.

Keywords: chatbot, conversational agent, digital health intervention, HIV, meta-analysis, mHealth, review, sexual health.


References

[1]  Murray E, Hekler EB, Andersson G, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med 2016; 51 843–51.
| Evaluating digital health interventions: key questions and approaches.Crossref | GoogleScholarGoogle Scholar |

[2]  Bender SS, Fulbright YK. Content analysis: a review of perceived barriers to sexual and reproductive health services by young people. Eur J Contracept Reprod Health Care 2013; 18 159–67.
| Content analysis: a review of perceived barriers to sexual and reproductive health services by young people.Crossref | GoogleScholarGoogle Scholar |

[3]  Marlow LAV, Waller J, Wardle J. Barriers to cervical cancer screening among ethnic minority women: a qualitative study. J Fam Plann Reprod Health Care 2015; 41 248–54.
| Barriers to cervical cancer screening among ethnic minority women: a qualitative study.Crossref | GoogleScholarGoogle Scholar |

[4]  Harb CYW, Pass LE, De Soriano IC, Zwick A, Gilbert PA. Motivators and barriers to accessing sexual health care services for transgender/genderqueer individuals assigned female sex at birth. Transgender Health 2019; 4 58–67.
| Motivators and barriers to accessing sexual health care services for transgender/genderqueer individuals assigned female sex at birth.Crossref | GoogleScholarGoogle Scholar |

[5]  Edelman NL, Patel H, Glasper A, Bogen-Johnston L. Understanding barriers to sexual health service access among substance-misusing women on the South East coast of England. J Fam Plann Reprod Health Care 2013; 39 258–63.
| Understanding barriers to sexual health service access among substance-misusing women on the South East coast of England.Crossref | GoogleScholarGoogle Scholar |

[6]  MacAfee LK, Harfmann RF, Cannon LM, et al. Substance use treatment patient and provider perspectives on accessing sexual and reproductive health services: barriers, facilitators, and the need for integration of care. Subst Use Misuse 2020; 55 95–107.
| Substance use treatment patient and provider perspectives on accessing sexual and reproductive health services: barriers, facilitators, and the need for integration of care.Crossref | GoogleScholarGoogle Scholar |

[7]  Kurtz SP, Surratt HL, Kiley MC, Inciardi JA. Barriers to health and social services for street-based sex workers. J Health Care Poor Underserved 2005; 16 345–61.
| Barriers to health and social services for street-based sex workers.Crossref | GoogleScholarGoogle Scholar |

[8]  Ma PHX, Chan ZCY, Loke AY. The socio-ecological model approach to understanding barriers and facilitators to the accessing of health services by sex workers: a systematic review. AIDS Behav 2017; 21 2412–38.
| The socio-ecological model approach to understanding barriers and facilitators to the accessing of health services by sex workers: a systematic review.Crossref | GoogleScholarGoogle Scholar |

[9]  Statista. Internet users in the world. Statista; 2021. Available at https://www.statista.com/statistics/617136/digital-population-worldwide/ [accessed 30 April 2021]

[10]  Burns K, Keating P, Free C. A systematic review of randomised control trials of sexual health interventions delivered by mobile technologies. BMC Public Health 2016; 16 778
| A systematic review of randomised control trials of sexual health interventions delivered by mobile technologies.Crossref | GoogleScholarGoogle Scholar |

[11]  Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV treatment & prevention: a systematic review of the literature. Open AIDS J 2013; 7 17–41.
| mHealth for HIV treatment & prevention: a systematic review of the literature.Crossref | GoogleScholarGoogle Scholar |

[12]  L’Engle KL, Mangone ER, Parcesepe AM, Agarwal S, Ippoliti NB. Mobile phone interventions for adolescent sexual and reproductive health: a systematic review. Pediatrics 2016; 138 e20160884
| Mobile phone interventions for adolescent sexual and reproductive health: a systematic review.Crossref | GoogleScholarGoogle Scholar |

[13]  Lim MSC, Hocking JS, Hellard ME, Aitken CK. SMS STI: a review of the uses of mobile phone text messaging in sexual health. Int J STD AIDS 2008; 19 287–90.
| SMS STI: a review of the uses of mobile phone text messaging in sexual health.Crossref | GoogleScholarGoogle Scholar |

[14]  Taylor D, Lunny C, Lolić P, et al. Effectiveness of text messaging interventions on prevention, detection, treatment, and knowledge outcomes for sexually transmitted infections (STIs)/HIV: a systematic review and meta-analysis. Syst Rev 2019; 8 12
| Effectiveness of text messaging interventions on prevention, detection, treatment, and knowledge outcomes for sexually transmitted infections (STIs)/HIV: a systematic review and meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[15]  van Velthoven MHMMT, Brusamento S, Majeed A, Car J. Scope and effectiveness of mobile phone messaging for HIV/AIDS care: a systematic review. Psychol Health Med 2013; 18 182–202.
| Scope and effectiveness of mobile phone messaging for HIV/AIDS care: a systematic review.Crossref | GoogleScholarGoogle Scholar |

[16]  Wadham E, Green C, Debattista J, Somerset S, Sav A. New digital media interventions for sexual health promotion among young people: a systematic review. Sex Health 2019; 16 101–23.
| New digital media interventions for sexual health promotion among young people: a systematic review.Crossref | GoogleScholarGoogle Scholar |

[17]  Guse K, Levine D, Martins S, et al. Interventions using new digital media to improve adolescent sexual health: a systematic review. J Adolesc Health 2012; 51 535–43.
| Interventions using new digital media to improve adolescent sexual health: a systematic review.Crossref | GoogleScholarGoogle Scholar |

[18]  Bailey J, Mann S, Wayal S, et al. Sexual health promotion for young people delivered via digital media: a scoping review. Public Health Res 2015; 3 1–120.
| Sexual health promotion for young people delivered via digital media: a scoping review.Crossref | GoogleScholarGoogle Scholar |

[19]  Muessig KE, Pike EC, LeGrand S, Hightow-Weidman LB. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res 2013; 15 e1
| Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review.Crossref | GoogleScholarGoogle Scholar |

[20]  Muessig KE, Nekkanti M, Bauermeister J, Bull S, Hightow-Weidman LB. A systematic review of recent smartphone, internet and Web 2.0 interventions to address the HIV continuum of care. Curr HIV/AIDS Rep 2015; 12 173–90.
| A systematic review of recent smartphone, internet and Web 2.0 interventions to address the HIV continuum of care.Crossref | GoogleScholarGoogle Scholar |

[21]  Cao B, Gupta S, Wang J, et al. Social media interventions to promote HIV testing, linkage, adherence, and retention: systematic review and meta-analysis. J Med Internet Res 2017; 19 e394
| Social media interventions to promote HIV testing, linkage, adherence, and retention: systematic review and meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[22]  Gabarron E, Wynn R. Use of social media for sexual health promotion: a scoping review. Glob Health Action 2016; 9 32193
| Use of social media for sexual health promotion: a scoping review.Crossref | GoogleScholarGoogle Scholar |

[23]  DeSmet A, Shegog R, Van Ryckeghem D, Crombez G, De Bourdeaudhuij I. A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games. Games Health J 2015; 4 78–90.
| A systematic review and meta-analysis of interventions for sexual health promotion involving serious digital games.Crossref | GoogleScholarGoogle Scholar |

[24]  Hookham G, Nesbitt K. A systematic review of the definition and measurement of engagement in serious games. In: Proceedings of the Australasian computer science week multiconference. ACM; 2019: 1–10.
| Crossref |

[25]  Cole-Lewis H, Ezeanochie N, Turgiss J. Understanding health behavior technology engagement: pathway to measuring digital behavior change interventions. JMIR Form Res 2019; 3 e14052
| Understanding health behavior technology engagement: pathway to measuring digital behavior change interventions.Crossref | GoogleScholarGoogle Scholar |

[26]  Yardley L, Spring BJ, Riper H, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016; 51 833–42.
| Understanding and promoting effective engagement with digital behavior change interventions.Crossref | GoogleScholarGoogle Scholar |

[27]  Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med 2017; 7 254–67.
| Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis.Crossref | GoogleScholarGoogle Scholar |

[28]  Webb T, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010; 12 e4
| Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy.Crossref | GoogleScholarGoogle Scholar |

[29]  Brouwer W, Kroeze W, Crutzen R, et al. Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review. J Med Internet Res 2011; 13 e2
| Which intervention characteristics are related to more exposure to internet-delivered healthy lifestyle promotion interventions? A systematic review.Crossref | GoogleScholarGoogle Scholar |

[30]  Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 2019; 64 456–64.
| Chatbots and conversational agents in mental health: a review of the psychiatric landscape.Crossref | GoogleScholarGoogle Scholar |

[31]  Bérubé C, Schachner T, Keller R, et al. Voice-based conversational agents for the prevention and management of chronic and mental health conditions: systematic literature review. J Med Internet Res 2021; 23 e25933
| Voice-based conversational agents for the prevention and management of chronic and mental health conditions: systematic literature review.Crossref | GoogleScholarGoogle Scholar |

[32]  Tudor Car L, Dhinagaran DA, Kyaw BM, et al. Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res 2020; 22 e17158
| Conversational agents in health care: scoping review and conceptual analysis.Crossref | GoogleScholarGoogle Scholar |

[33]  Montenegro JLZ, da Costa CA, da Rosa Righi R. Survey of conversational agents in health. Expert Syst Appl 2019; 129 56–67.
| Survey of conversational agents in health.Crossref | GoogleScholarGoogle Scholar |

[34]  Kocaballi AB, Berkovsky S, Quiroz JC, et al. The personalization of conversational agents in health care: systematic review. J Med Internet Res 2019; 21 e15360
| The personalization of conversational agents in health care: systematic review.Crossref | GoogleScholarGoogle Scholar |

[35]  Gaffney H, Mansell W, Tai S. Conversational agents in the treatment of mental health problems: mixed-method systematic review. JMIR Ment Health 2019; 6 e14166
| Conversational agents in the treatment of mental health problems: mixed-method systematic review.Crossref | GoogleScholarGoogle Scholar |

[36]  Rapp A, Curti L, Boldi A. The human side of human-chatbot interaction: a systematic literature review of ten years of research on text-based chatbots. Int J Hum-Comput Stud 2021; 151 102630
| The human side of human-chatbot interaction: a systematic literature review of ten years of research on text-based chatbots.Crossref | GoogleScholarGoogle Scholar |

[37]  Sciuto A, Saini A, Forlizzi J, Hong JI. “Hey Alexa, what’s up?”: a mixed-methods studies of in-home conversational agent usage. In: Proceedings of the 2018 designing interactive systems conference. DIS ’18. Association for Computing Machinery; 2018: 857–68.
| Crossref |

[38]  Adam M, Wessel M, Benlian A. AI-based chatbots in customer service and their effects on user compliance. Electron Mark 2021; 31 427–45.
| AI-based chatbots in customer service and their effects on user compliance.Crossref | GoogleScholarGoogle Scholar |

[39]  Provoost S, Lau HM, Ruwaard J, Riper H. Embodied conversational agents in clinical psychology: a scoping review. J Med Internet Res 2017; 19 e151
| Embodied conversational agents in clinical psychology: a scoping review.Crossref | GoogleScholarGoogle Scholar |

[40]  Chiang N, Guo M, Amico KR, Atkins L, Lester RT. Interactive two-way mHealth interventions for improving medication adherence: an evaluation using the behaviour change wheel framework. JMIR MHealth UHealth 2018; 6 e87
| Interactive two-way mHealth interventions for improving medication adherence: an evaluation using the behaviour change wheel framework.Crossref | GoogleScholarGoogle Scholar |

[41]  Wald DS, Butt S, Bestwick JP. One-way versus two-way text messaging on improving medication adherence: meta-analysis of randomized trials. Am J Med 2015; 128 1139.e1–1139.e5.
| One-way versus two-way text messaging on improving medication adherence: meta-analysis of randomized trials.Crossref | GoogleScholarGoogle Scholar |

[42]  Baek H, Kim S, Lee S. Effects of interactivity and usage mode on user experience in chatbot interface. J HCI Soc Korea 2019; 14 35–43.
| Effects of interactivity and usage mode on user experience in chatbot interface.Crossref | GoogleScholarGoogle Scholar |

[43]  Go E, Sundar SS. Humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions. Comput Hum Behav 2019; 97 304–16.
| Humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions.Crossref | GoogleScholarGoogle Scholar |

[44]  Skjuve M, Brandtzæg PB. Chatbots as a new user interface for providing health information to young people; 2018. Available at https://sintef.brage.unit.no/sintef-xmlui/handle/11250/2576290. [accessed 30 April 2021]

[45]  Pereira J, Díaz Ó. Using health chatbots for behavior change: a mapping study. J Med Syst 2019; 43 135
| Using health chatbots for behavior change: a mapping study.Crossref | GoogleScholarGoogle Scholar |

[46]  Berendes S, Gubijev A, McCarthy OL, Palmer MJ, Wilson E, Free C. Sexual health interventions delivered to participants by mobile technology: a systematic review and meta-analysis of randomised controlled trials. Sex Transm Infect 2021; 97 190–200.
| Sexual health interventions delivered to participants by mobile technology: a systematic review and meta-analysis of randomised controlled trials.Crossref | GoogleScholarGoogle Scholar |

[47]  Swanton R, Allom V, Mullan B. A meta-analysis of the effect of new-media interventions on sexual-health behaviours. Sex Transm Infect 2015; 91 14–20.
| A meta-analysis of the effect of new-media interventions on sexual-health behaviours.Crossref | GoogleScholarGoogle Scholar |

[48]  Zaneva M, Philpott A, Singh A, Larsson G, Gonsalves L. What is the added value of incorporating pleasure in sexual health interventions? A systematic review and meta-analysis. PLoS ONE 2022; 17 e0261034
| What is the added value of incorporating pleasure in sexual health interventions? A systematic review and meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[49]  Becasen JS, Ford J, Hogben M. Sexual health interventions: a meta-analysis. J Sex Res 2015; 52 433–43.
| Sexual health interventions: a meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[50]  Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960; 20 37–46.
| A coefficient of agreement for nominal scales.Crossref | GoogleScholarGoogle Scholar |

[51]  Delone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst 2003; 19 9–30.
| The DeLone and McLean model of information systems success: a ten-year update.Crossref | GoogleScholarGoogle Scholar |

[52]  Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. John Wiley & Sons; 2021.

[53]  Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003; 327 557–60.
| Measuring inconsistency in meta-analyses.Crossref | GoogleScholarGoogle Scholar |

[54]  Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315 629–34.
| Bias in meta-analysis detected by a simple, graphical test.Crossref | GoogleScholarGoogle Scholar |

[55]  Duval S, Tweedie R. A nonparametric “Trim and Fill” method of accounting for publication bias in meta-analysis. J Am Stat Assoc 2000; 95 89–98.
| A nonparametric “Trim and Fill” method of accounting for publication bias in meta-analysis.Crossref | GoogleScholarGoogle Scholar |

[56]  Liu AY, Vittinghoff E, von Felten P, et al. Randomized controlled trial of a mobile health intervention to promote retention and adherence to preexposure prophylaxis among young people at risk for human immunodeficiency virus: the EPIC study. Clin Infect Dis 2019; 68 2010–7.
| Randomized controlled trial of a mobile health intervention to promote retention and adherence to preexposure prophylaxis among young people at risk for human immunodeficiency virus: the EPIC study.Crossref | GoogleScholarGoogle Scholar |

[57]  Chernick LS, Stockwell MS, Gonzalez A, et al. A user-informed, theory-based pregnancy prevention intervention for adolescents in the emergency department: a prospective cohort study. J Adolesc Health 2021; 68 705–12.
| A user-informed, theory-based pregnancy prevention intervention for adolescents in the emergency department: a prospective cohort study.Crossref | GoogleScholarGoogle Scholar |

[58]  Davis T, Diclemente RJ, Prietula M. Using ADAPT-ITT to modify a telephone-based HIV prevention intervention for SMS delivery: formative study. JMIR Form Res 2020; 4 e22485
| Using ADAPT-ITT to modify a telephone-based HIV prevention intervention for SMS delivery: formative study.Crossref | GoogleScholarGoogle Scholar |

[59]  Pot M, Paulussen TG, Ruiter RA, et al. Effectiveness of a web-based tailored intervention with virtual assistants promoting the acceptability of HPV vaccination among mothers of invited girls: randomized controlled trial. J Med Internet Res 2017; 19 e312
| Effectiveness of a web-based tailored intervention with virtual assistants promoting the acceptability of HPV vaccination among mothers of invited girls: randomized controlled trial.Crossref | GoogleScholarGoogle Scholar |

[60]  Rodrigues R, Shet A, Antony J, et al. Supporting adherence to antiretroviral therapy with mobile phone reminders: results from a cohort in South India. PLoS ONE 2012; 7 e40723
| Supporting adherence to antiretroviral therapy with mobile phone reminders: results from a cohort in South India.Crossref | GoogleScholarGoogle Scholar |

[61]  Rodrigues R, Poongulali S, Balaji K, Atkins S, Ashorn P, Costa AD. ‘The phone reminder is important, but will others get to know about my illness?’ Patient perceptions of an mHealth antiretroviral treatment support intervention in the HIVIND trial in South India. BMJ Open 2015; 5 e007574
| ‘The phone reminder is important, but will others get to know about my illness?’ Patient perceptions of an mHealth antiretroviral treatment support intervention in the HIVIND trial in South India.Crossref | GoogleScholarGoogle Scholar |

[62]  Garofalo R, Kuhns LM, Hotton A, Johnson A, Muldoon A, Rice D. A randomized controlled trial of personalized text message reminders to promote medication adherence among HIV-positive adolescents and young adults. AIDS Behav 2016; 20 1049–59.
| A randomized controlled trial of personalized text message reminders to promote medication adherence among HIV-positive adolescents and young adults.Crossref | GoogleScholarGoogle Scholar |

[63]  Moore DJ, Pasipanodya EC, Umlauf A, et al. Individualized texting for adherence building (iTAB) for methamphetamine users living with HIV: a pilot randomized clinical trial. Drug Alcohol Depend 2018; 189 154–60.
| Individualized texting for adherence building (iTAB) for methamphetamine users living with HIV: a pilot randomized clinical trial.Crossref | GoogleScholarGoogle Scholar |

[64]  Ingersoll KS, Dillingham RA, Hettema JE, et al. Pilot RCT of bidirectional text messaging for ART adherence among nonurban substance users with HIV. Health Psychol 2015; 34 1305–15.
| Pilot RCT of bidirectional text messaging for ART adherence among nonurban substance users with HIV.Crossref | GoogleScholarGoogle Scholar |

[65]  Dworkin MS, Lee S, Chakraborty A, et al. Acceptability, feasibility, and preliminary efficacy of a theory-based relational embodied conversational agent mobile phone intervention to promote HIV medication adherence in young HIV-positive African American MSM. AIDS Educ Prev 2019; 31 17–37.
| Acceptability, feasibility, and preliminary efficacy of a theory-based relational embodied conversational agent mobile phone intervention to promote HIV medication adherence in young HIV-positive African American MSM.Crossref | GoogleScholarGoogle Scholar |

[66]  Dworkin M, Chakraborty A, Lee S, et al. A realistic talking human embodied agent mobile phone intervention to promote HIV medication adherence and retention in care in young HIV-positive African American men who have sex with men: qualitative study. JMIR Mhealth Uhealth 2018; 6 e10211
| A realistic talking human embodied agent mobile phone intervention to promote HIV medication adherence and retention in care in young HIV-positive African American men who have sex with men: qualitative study.Crossref | GoogleScholarGoogle Scholar |

[67]  Christopoulos KA, Riley ED, Carrico AW, et al. A randomized controlled trial of a text messaging intervention to promote virologic suppression and retention in care in an urban safety-net human immunodeficiency virus clinic: the Connect4Care trial. Clin Infect Dis 2018; 67 751–9.
| A randomized controlled trial of a text messaging intervention to promote virologic suppression and retention in care in an urban safety-net human immunodeficiency virus clinic: the Connect4Care trial.Crossref | GoogleScholarGoogle Scholar |

[68]  Chavez K, Palfai TP. Feasibility of a mobile messaging-enhanced brief intervention for high risk heavy drinking MSM: a pre-pilot study. Alcohol Treat Q 2020; 38 87–105.
| Feasibility of a mobile messaging-enhanced brief intervention for high risk heavy drinking MSM: a pre-pilot study.Crossref | GoogleScholarGoogle Scholar |

[69]  Jack BW, Bickmore T, Yinusa-Nyahkoon L, et al. Improving the health of young African American women in the preconception period using health information technology: a randomised controlled trial. Lancet Digit Health 2020; 2 e475–85.
| Improving the health of young African American women in the preconception period using health information technology: a randomised controlled trial.Crossref | GoogleScholarGoogle Scholar |

[70]  Naar-King S, Outlaw AY, Sarr M, et al. Motivational enhancement system for adherence (MESA): pilot randomized trial of a brief computer-delivered prevention intervention for youth initiating antiretroviral treatment. J Pediatr Psychol 2013; 38 638–48.
| Motivational enhancement system for adherence (MESA): pilot randomized trial of a brief computer-delivered prevention intervention for youth initiating antiretroviral treatment.Crossref | GoogleScholarGoogle Scholar |

[71]  Fuchs JD, Stojanovski K, Vittinghoff E, et al. A mobile health strategy to support adherence to antiretroviral preexposure prophylaxis. AIDS Patient Care STDs 2018; 32 104–11.
| A mobile health strategy to support adherence to antiretroviral preexposure prophylaxis.Crossref | GoogleScholarGoogle Scholar |

[72]  Amith M, Cui L, Roberts K, Tao C. Towards an ontology-based medication conversational agent for PrEP and PEP. In: Proceedings of the first workshop on natural language processing for medical conversations. Association for Computational Linguistics (ACL); 2020; 31–40.
| Crossref |

[73]  Moore DJ, Jain S, Dubé MP, et al. Randomized controlled trial of daily text messages to support adherence to preexposure prophylaxis in individuals at risk for human immunodeficiency virus: the TAPIR study. Clin Infect Dis 2018; 66 1566–72.
| Randomized controlled trial of daily text messages to support adherence to preexposure prophylaxis in individuals at risk for human immunodeficiency virus: the TAPIR study.Crossref | GoogleScholarGoogle Scholar |

[74]  Bonnevie E, Lloyd TD, Rosenberg SD, Williams K, Goldbarg J, Smyser J. Layla’s got you: developing a tailored contraception chatbot for Black and Hispanic young women. Health Educ J 2021; 80 413–24.
| Layla’s got you: developing a tailored contraception chatbot for Black and Hispanic young women.Crossref | GoogleScholarGoogle Scholar |

[75]  van Heerden A, Ntinga X, Vilakazi K. The potential of conversational agents to provide a rapid HIV counseling and testing services. In: 2017 international conference on the frontiers and advances in data science (FADS). Institute of Electrical and Electronics Engineers Inc.; 2017. 2018: 80–5.
| Crossref |

[76]  Salvadori N, Adam P, Mary JY, et al. Appointment reminders to increase uptake of HIV retesting by at-risk individuals: a randomized controlled study in Thailand. J Int AIDS Soc 2020; 23 e25478
| Appointment reminders to increase uptake of HIV retesting by at-risk individuals: a randomized controlled study in Thailand.Crossref | GoogleScholarGoogle Scholar |

[77]  Moore DJ, Poquette A, Casaletto KB, et al. Individualized texting for adherence building (iTAB): improving antiretroviral dose timing among HIV-infected persons with co-occurring bipolar disorder. AIDS Behav 2015; 19 459–71.
| Individualized texting for adherence building (iTAB): improving antiretroviral dose timing among HIV-infected persons with co-occurring bipolar disorder.Crossref | GoogleScholarGoogle Scholar |

[78]  Amith M, Zhu A, Cunningham R, et al. Early usability assessment of a conversational agent for HPV vaccination. Stud Health Technol Inform 2019; 257 17–23.
| Early usability assessment of a conversational agent for HPV vaccination.Crossref | GoogleScholarGoogle Scholar |

[79]  Amith M, Lin R, Cunningham R, et al. Examining potential usability and health beliefs among young adults using a conversational agent for HPV vaccine counseling. AMIA Jt Summits Transl Sci Proc 2020; 2020 43–52.

[80]  Mendu S, Boukhechba M, Gordon JR, et al. Design of a culturally-informed virtual human for educating hispanic women about cervical cancer. In: Proceedings of the 12th EAI international conference on pervasive computing technologies for healthcare. PervasiveHealth ’18. Association for Computing Machinery; 2018: 360–6.
| Crossref |

[81]  Wells KJ, Vázquez-Otero C, Bredice M, et al. Acceptability of a virtual patient educator for hispanic women. Hisp Health Care Int 2015; 13 179–85.
| Acceptability of a virtual patient educator for hispanic women.Crossref | GoogleScholarGoogle Scholar |

[82]  Swendeman D, Fehrenbacher AE, Roy S, et al. A pilot randomized controlled trial (RCT) of daily versus weekly interactive voice response calls to support adherence among antiretroviral treatment patients in India. mHealth 2020; 6 35
| A pilot randomized controlled trial (RCT) of daily versus weekly interactive voice response calls to support adherence among antiretroviral treatment patients in India.Crossref | GoogleScholarGoogle Scholar |

[83]  King E, Kinvig K, Steif J, et al. Mobile text messaging to improve medication adherence and viral load in a vulnerable canadian population living with human immunodeficiency virus: a repeated measures study. J Med Internet Res 2017; 19 e190
| Mobile text messaging to improve medication adherence and viral load in a vulnerable canadian population living with human immunodeficiency virus: a repeated measures study.Crossref | GoogleScholarGoogle Scholar |

[84]  Dowshen N, Kuhns LM, Johnson A, Holoyda BJ, Garofalo R. Improving adherence to antiretroviral therapy for youth living with HIV/AIDS: a pilot study using personalized, interactive, daily text message reminders. J Med Internet Res 2012; 14 e51
| Improving adherence to antiretroviral therapy for youth living with HIV/AIDS: a pilot study using personalized, interactive, daily text message reminders.Crossref | GoogleScholarGoogle Scholar |

[85]  Harris LT, Lehavot K, Huh D, et al. Two-way text messaging for health behavior change among human immunodeficiency virus–positive individuals. Telemed e-Health 2010; 16 1024–9.
| Two-way text messaging for health behavior change among human immunodeficiency virus–positive individuals.Crossref | GoogleScholarGoogle Scholar |

[86]  Hardy H, Kumar V, Doros G, et al. Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy. AIDS Patient Care STDs 2011; 25 153–61.
| Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy.Crossref | GoogleScholarGoogle Scholar |

[87]  Baclic O, Tunis M, Young K, Doan C, Swerdfeger H, Schonfeld J. Challenges and opportunities for public health made possible by advances in natural language processing. Can Commun Dis Rep 2020; 46 161–8.
| Challenges and opportunities for public health made possible by advances in natural language processing.Crossref | GoogleScholarGoogle Scholar |

[88]  Bazzi AR, Drainoni M-L, Biancarelli DL, et al. Systematic review of HIV treatment adherence research among people who inject drugs in the United States and Canada: evidence to inform pre-exposure prophylaxis (PrEP) adherence interventions. BMC Public Health 2019; 19 31
| Systematic review of HIV treatment adherence research among people who inject drugs in the United States and Canada: evidence to inform pre-exposure prophylaxis (PrEP) adherence interventions.Crossref | GoogleScholarGoogle Scholar |

[89]  Geense WW, van de Glind IM, Visscher TLS, van Achterberg T. Barriers, facilitators and attitudes influencing health promotion activities in general practice: an explorative pilot study. BMC Fam Pract 2013; 14 20
| Barriers, facilitators and attitudes influencing health promotion activities in general practice: an explorative pilot study.Crossref | GoogleScholarGoogle Scholar |

[90]  Kelly S, Martin S, Kuhn I, Cowan A, Brayne C, Lafortune L. Barriers and facilitators to the uptake and maintenance of healthy behaviours by people at mid-life: a rapid systematic review. PLoS ONE 2016; 11 e0145074
| Barriers and facilitators to the uptake and maintenance of healthy behaviours by people at mid-life: a rapid systematic review.Crossref | GoogleScholarGoogle Scholar |

[91]  McGill T, Hobbs V, Klobas J. User developed applications and information systems success: a test of DeLone and McLean’s model. Inf Resour Manag J IRMJ 2003; 16 24–45.
| User developed applications and information systems success: a test of DeLone and McLean’s model.Crossref | GoogleScholarGoogle Scholar |

[92]  Ojo AI. Validation of the DeLone and McLean information systems success model. Healthc Inform Res 2017; 23 60
| Validation of the DeLone and McLean information systems success model.Crossref | GoogleScholarGoogle Scholar |

[93]  Petter S, McLean ER. A meta-analytic assessment of the DeLone and McLean IS success model: an examination of IS success at the individual level. Inf Manage 2009; 46 159–66.
| A meta-analytic assessment of the DeLone and McLean IS success model: an examination of IS success at the individual level.Crossref | GoogleScholarGoogle Scholar |

[94]  Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Manag Sci 2000; 46 186–204.
| A theoretical extension of the technology acceptance model: four longitudinal field studies.Crossref | GoogleScholarGoogle Scholar |

[95]  Turner M, Kitchenham B, Brereton P, Charters S, Budgen D. Does the technology acceptance model predict actual use? A systematic literature review. Inf Softw Technol 2010; 52 463–79.
| Does the technology acceptance model predict actual use? A systematic literature review.Crossref | GoogleScholarGoogle Scholar |

[96]  Huang J-H, Lin Y-R, Chuang S-T. Elucidating user behavior of mobile learning: a perspective of the extended technology acceptance model. Electron Libr 2007; 25 585–98.
| Elucidating user behavior of mobile learning: a perspective of the extended technology acceptance model.Crossref | GoogleScholarGoogle Scholar |

[97]  Wang X, Goh DH-L. Video game acceptance: a meta-analysis of the extended technology acceptance model. Cyberpsychology Behav Soc Netw 2017; 20 662–71.
| Video game acceptance: a meta-analysis of the extended technology acceptance model.Crossref | GoogleScholarGoogle Scholar |

[98]  Grice HP. Logic and conversation. In Speech Acts. Brill; 1975. pp. 41–58.
| Crossref |

[99]  Benjumea J, Ropero J, Rivera-Romero O, Dorronzoro-Zubiete E, Carrasco A. Assessment of the fairness of privacy policies of mobile health apps: scale development and evaluation in cancer apps. JMIR MHealth UHealth 2020; 8 e17134
| Assessment of the fairness of privacy policies of mobile health apps: scale development and evaluation in cancer apps.Crossref | GoogleScholarGoogle Scholar |

[100]  Price WN, Cohen IG. Privacy in the age of medical big data. Nat Med 2019; 25 37–43.
| Privacy in the age of medical big data.Crossref | GoogleScholarGoogle Scholar |

[101]  Sax M. Between empowerment and manipulation: the ethics and regulation of for-profit health apps; 2021. Available at https://dare.uva.nl/search?identifier=52225d37-e7e1-4883-9dab-a3f5d3a063d8. [accessed 4 May 2021]

[102]  Laumer S, Maier C, Gubler F. Chatbot acceptance in healthcare: explaining user adoption of conversational agents for disease diagnosis. In: Proceedings of the 27th European conference on information systems (ECIS). AISeL; 2019. Available at https://aisel.aisnet.org/ecis2019_rp/88

[103]  Hernandez JPT. Network diffusion and technology acceptance of a nurse chatbot for chronic disease self-management support : a theoretical perspective. J Med Invest 2019; 66 24–30.
| Network diffusion and technology acceptance of a nurse chatbot for chronic disease self-management support : a theoretical perspective.Crossref | GoogleScholarGoogle Scholar |

[104]  de Cosmo LM, Piper L, Di Vittorio A. The role of attitude toward chatbots and privacy concern on the relationship between attitude toward mobile advertising and behavioral intent to use chatbots. Ital J Mark 2021; 2021 83–102.
| The role of attitude toward chatbots and privacy concern on the relationship between attitude toward mobile advertising and behavioral intent to use chatbots.Crossref | GoogleScholarGoogle Scholar |

[105]  Kasilingam DL. Understanding the attitude and intention to use smartphone chatbots for shopping. Technol Soc 2020; 62 101280
| Understanding the attitude and intention to use smartphone chatbots for shopping.Crossref | GoogleScholarGoogle Scholar |

[106]  van der Goot MJ, Pilgrim T. Exploring age differences in motivations for and acceptance of chatbot communication in a customer service context. In: Følstad A, Araujo T, Papadopoulos S, et al., editors. Chatbot research and design. Lecture notes in computer science. Springer International Publishing; 2020: 173–86.
| Crossref |

[107]  Tsai W-HS, Liu Y, Chuan C-H. How chatbots’ social presence communication enhances consumer engagement: the mediating role of parasocial interaction and dialogue. J Res Interact Mark 2021; 15 460–82.
| How chatbots’ social presence communication enhances consumer engagement: the mediating role of parasocial interaction and dialogue.Crossref | GoogleScholarGoogle Scholar |

[108]  Araujo T. Living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput Hum Behav 2018; 85 183–9.
| Living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions.Crossref | GoogleScholarGoogle Scholar |

[109]  Park H, Lee J. Designing a conversational agent for sexual assault survivors: defining burden of self-disclosure and envisioning survivor-centered solutions. In: Proceedings of the 2021 CHI conference on human factors in computing systems. ACM; 2021: 1–17.
| Crossref |

[110]  Nadarzynski T, Puentes V, Pawlak I, et al. Barriers and facilitators to engagement with artificial intelligence (AI)-based chatbots for sexual and reproductive health advice: a qualitative analysis. Sex Health 2021; 18 385–93.
| Barriers and facilitators to engagement with artificial intelligence (AI)-based chatbots for sexual and reproductive health advice: a qualitative analysis.Crossref | GoogleScholarGoogle Scholar |

[111]  Croes EAJ, Antheunis ML. 36 questions to loving a chatbot: are people willing to self-disclose to a chatbot? In: Følstad A, Araujo T, Papadopoulos S, et al., editors. Chatbot research and design. Lecture notes in computer science. Springer International Publishing; 2021: 81–95.
| Crossref |

[112]  Diederich S, Brendel AB, Lichtenberg S, Kolbe L. Design for fast request fulfillment or natural interaction? Insights from an experiment with a conversational agent. In Proceedings of the 27th European Conference on Information Systems (ECIS), AIS Electronic Library; 2019. https://aisel.aisnet.org/ecis2019_rp/20

[113]  Jain M, Kumar P, Kota R, Patel SN. Evaluating and informing the design of chatbots. In: Proceedings of the 2018 designing interactive systems conference. DIS ’18. Association for Computing Machinery; 2018: 895–906.
| Crossref |

[114]  Zamora J. I’m sorry, Dave, I’m afraid I can’t do that: chatbot perception and expectations. In: Proceedings of the 5th international conference on human agent interaction. HAI ’17. Association for Computing Machinery; 2017: 253–260.
| Crossref |

[115]  Croes EAJ, Antheunis ML. Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot. J Soc Pers Relatsh 2021; 38 279–300.
| Can we be friends with Mitsuku? A longitudinal study on the process of relationship formation between humans and a social chatbot.Crossref | GoogleScholarGoogle Scholar |

[116]  Vardoulakis LP, Ring L, Barry B, Sidner CL, Bickmore T. Designing relational agents as long term social companions for older adults. In: Nakano Y, Neff M, Paiva A, Walker M, editors. Intelligent virtual agents. Lecture notes in computer science. Springer; 2012: 289–302.
| Crossref |

[117]  Bickmore T, Schulman D, Yin L. Maintaining engagement in long-term interventions with relational agents. Appl Artif Intell 2010; 24 648–66.
| Maintaining engagement in long-term interventions with relational agents.Crossref | GoogleScholarGoogle Scholar |

[118]  de Gennaro M, Krumhuber EG, Lucas G. Effectiveness of an empathic chatbot in combating adverse effects of social exclusion on mood. Front Psychol 2020; 10 3061
| Effectiveness of an empathic chatbot in combating adverse effects of social exclusion on mood.Crossref | GoogleScholarGoogle Scholar |

[119]  Liu B, Sundar SS. Should machines express sympathy and empathy? experiments with a health advice chatbot. Cyberpsychology Behav Soc Netw 2018; 21 625–36.
| Should machines express sympathy and empathy? experiments with a health advice chatbot.Crossref | GoogleScholarGoogle Scholar |

[120]  Crutzen R, Peters G-JY, Portugal SD, Fisser EM, Grolleman JJ. An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study. J Adolesc Health 2011; 48 514–9.
| An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study.Crossref | GoogleScholarGoogle Scholar |

[121]  Hussain SA, Ogundimu F, Bhattarai S. Mobile phone-based chatbot for family planning and contraceptive information. In: Duffy VG, editor. Digital human modeling and applications in health, safety, ergonomics and risk management. healthcare applications. Lecture notes in computer science. Springer International Publishing; 2019: 342–52.
| Crossref |

[122]  van Wezel MMC, Croes EAJ, Antheunis ML. “I’m here for you”: can social chatbots truly support their users? a literature review. In: Følstad A, Araujo T, Papadopoulos S, et al., editors. Chatbot research and design. Lecture notes in computer science. Springer International Publishing; 2021: 96–113.
| Crossref |