Barriers and facilitators to engagement with artificial intelligence (AI)-based chatbots for sexual and reproductive health advice: a qualitative analysis
Tom Nadarzynski A * , Vannesa Puentes B , Izabela Pawlak A , Tania Mendes A , Ian Montgomery C , Jake Bayley D and Damien Ridge AA School of Social Sciences, University of Westminster, London, UK.
B Science, Engineering and Computing Faculty, Kingston University, London, UK.
C Positive East, London, UK.
D Barts NHS Trust, London, UK.
Sexual Health 18(5) 385-393 https://doi.org/10.1071/SH21123
Submitted: 12 March 2021 Accepted: 19 July 2021 Published: 16 November 2021
© 2021 The Author(s) (or their employer(s)). Published by CSIRO Publishing. This is an open access article distributed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND)
Abstract
Background: The emergence of artificial intelligence (AI) provides opportunities for demand management of sexual and reproductive health services. Conversational agents/chatbots are increasingly common, although little is known about how this technology could aid services. This study aimed to identify barriers and facilitators for engagement with sexual health chatbots to advise service developers and related health professionals.
Methods: In January–June 2020, we conducted face-to-face, semi-structured and online interviews to explore views on sexual health chatbots. Participants were asked to interact with a chatbot, offering advice on sexually transmitted infections (STIs) and relevant services. Participants were UK-based and recruited via social media. Data were recorded, transcribed verbatim and analysed thematically.
Results: Forty participants (aged 18–50 years; 64% women, 77% heterosexual, 58% white) took part. Many thought chatbots could aid sex education, providing useful information about STIs and sign-posting to sexual health services in a convenient, anonymous and non-judgemental way. Some compared chatbots to health professionals or Internet search engines and perceived this technology as inferior, offering constrained content and interactivity, limiting disclosure of personal information, trust and perceived accuracy of chatbot responses.
Conclusions: Despite mixed attitudes towards chatbots, this technology was seen as useful for anonymous sex education but less suitable for matters requiring empathy. Chatbots may increase access to clinical services but their effectiveness and safety need to be established. Future research should identify which chatbots designs and functions lead to optimal engagement with this innovation.
Keywords: AI, artificial intelligence, chatbot, e-health, education, health promotion, health services, risk assessment.
Introduction
Sexual and reproductive health services (SRHSs) face significant challenges related to the increased demand for screening, treatment, partner notification and professional advice to their users. Before the COVID-19 pandemic, around 1 million people were acquiring a sexually transmitted infection (STI) each day, worldwide.1 In England, there were 468 342 diagnoses of STIs in 2019, with a 10% increase in syphilis and a 26% increase in gonorrhoea, the highest since records began in 1918.2 Young heterosexual people, men who have sex with men (MSM), and black minority ethnic groups continue to be disproportionally at risk of STIs and HIV. These groups also face multiple obstacles, such as embarrassment, low levels of knowledge about STIs, stigma and fear of discrimination, in accessing SRHSs.3,4
The COVID-19 pandemic and related physical distancing measures disrupted SRHSs. Reports demonstrated a reduction in the number of consultations, STI screening, vaccinations for MSM, STI diagnoses and treatment initiation such as for hepatitis C. In England, the pandemic had an impact on SRHS delivery with around 45% of all consultations in April–June 2020 being conducted over the Internet, compared to 26% in January–March 2020.5 A decline in service utilisation by 13% may reflect the general decrease in sexual activities in some at-risk groups during the first UK-wide lockdown, but also demonstrates the reduced availability of in-person services and a parallel rapid digitalisation aimed at improved accessibility of SRHS.6 However, little is known about patient acceptability, engagement and utilisation of novel remote SRHSs and online platforms for professional advice.
Digital interventions to promote self-care behaviours are increasingly common, although the research has mainly focused on adolescents and young adults. A review of 10 web-based interventions for adolescents showed that they had increased knowledge about STIs and condoms and increased positive attitudes towards screening and self-protective behaviours.7 However, studies have failed to link the increased knowledge with biological outcomes such as the rates of STIs. Similar findings were shown in a review of 19 trials examining digital interventions for sexual health promotion reporting a moderate effect on knowledge and self-efficacy, but no effect on safer sex intentions or biological outcomes.8 Nevertheless, a review of 51 studies on the use of social media for sexual health promotion found that interventions conducted on interactive channels such as Facebook or Twitter are capable of not only increasing knowledge and improving attitudes but also of having a potential impact on behaviours such as the uptake of STI screening.9 Also, two of the studies found a reduction in chlamydia and gonorrhoea cases as a result of an intervention on social media. These findings indicate that digital interventions that promote the exchange of health information may be more effective than static interventions that offer little interactivity. Although online interventions are capable of increasing knowledge and influencing some one-off behaviours, there is still a need to establish which components of digital services are the most engaging and effective at reducing STI rates.
Recent years have seen an expansion of innovative digital services that use automation, such as streamlining of repetitive and instructive tasks, and complex algorithms. Healthcare services that produce large amounts of data can now mine their datasets using artificial intelligence (AI); e.g. machine learning or deep learning, to predict patients at risk of HIV and their potential need for PrEP.10,11 Several AI applications have aimed at increasing patient self-care behaviours using automation. Chatbots or conversational agents are virtual digital systems that mimic human interaction using textual or voice input through ‘natural language processing’ and are typically delivered through websites, smartphone apps and communication exchange systems.12 A review of 47 studies showed that AI-led chatbots have been applied for general health diagnostics, treatment and monitoring, health services support, education and behaviour change.13 Additionally, a separate systematic review of 31 studies reported moderate evidence on the effectiveness, usability and positive user perceptions of chatbots in healthcare, indicating a potential for this technology to supplement current healthcare services.14 Chatbots have also been used for sexual and reproductive health providing information about HIV and AIDS via Facebook,15 educating adolescents about sex, drugs and alcohol,16 promoting fertility awareness and preconception health17,18 and promoting HIV medication adherence.19 The potential benefit of incorporating chatbots within SRHSs are their convenience, accessibility and increasing users’ levels of disclosure about intimate and potentially embarrassing topics that may be difficult to discuss with a healthcare professional.20 The conversational presentation of sexual health information via chatbots may also be preferred by patients with lower health literacy, facilitating their engagement with healthcare services.21
Our previous research on the acceptability of sexual health chatbots amongst clinic attendees showed a moderate rate of 40%, correlated with access to technology and its utilisation.22 However, motivations for such low acceptability were not explored qualitatively and there is a possibility that the acceptability would be higher among those struggling to access healthcare services. Therefore, there is a need to understand user perspectives on sexual health chatbots to inform the development of this technology to ensure optimal acceptability and uptake. This study aimed to explore barriers and facilitators to engagement with AI-led chatbots for sexual and reproductive health advice.
Methods
Design
Given that little is currently known about user perspectives in the area, our exploratory study used semi-structured interviews (guided by a topic guide) and thematic analysis to explore views on engagement with AI-led chatbots in sexual health. This study was approved by the University of Westminster Research Ethics Committee (reference: ETH1920-0381).
Participants and recruitment
The study aimed to gather diverse opinions of individuals at higher risk of poorer sexual health; i.e. young people, sexual and gender minorities as well as black and Asian minority ethnic groups. All participants needed to be at least 18 years old with no upper end limit, located in the UK, willing to interact with a sexual health chatbot and comprehend the English language to consent to the study and engage in interviews. No specific sampling framework was used for recruitment.
The participants were recruited through multiple sources between January and June 2020. Facebook, Twitter and Instagram were used to advertise the study inviting to discuss the usability of sexual health chatbots. Social media handles and hashtags were used to promote the study. The advert was also circulated amongst students of the most ethnically diverse University in London, to include the opinions of younger participants. Positive East, a London-based HIV support and prevention charity circulated the study advert within its networks, service users and social media platforms to seek views of people at risk of HIV and STIs or those currently accessing medical and their psycho-social support services to manage their HIV diagnosis.
Procedure
All those interested in the study were asked to click on a link that would direct them to an online information page and consent form. They were then asked for demographic questions (i.e. age, gender, sexual orientation and ethnicity), and to leave their contact details to be contacted by researchers, with a choice of an online or face-to-face interview. All participants that met inclusion criteria were invited to take part in the qualitative interviews.
Before the interview, participants provided signed consent and were requested to engage with a London-based chatbot called PAT (https://www.positiveeast.org.uk/chattopat). PAT is a sexual health chatbot developed and hosted by Positive East charity, funded through Public Health England HIV Innovation Fund. PAT aims to answer and sign-post for simple queries regarding sexual and reproductive health and HIV/STI prevention. It was selected as an example of a sexual health chatbot as it had the capability to interpret free text, through natural language processing, typical of this AI technology. The participants were asked to engage with the chatbot for at least 10 min to provide informed and experience-based views on this type of innovation. Participants were asked to consider how chatbots in general could be used to aid SRHSs, with PAT being used as a demonstration to allow participants better comprehension of chatbots. The study used a 13-item topic guide to explore potential barriers and facilitators to engagement with sexual health chatbots broadly (i.e. ‘What is your general opinion on talking about your sexual health to chatbots?’; ‘Would you consider a chatbot, like PAT, as a way of talking about your sex life?’, ‘What would you say was a limitation of your interaction with the chatbot?’). The interviews lasted approximately 20 min (range, 14–45 min), were audio-recorded, transcribed verbatim and anonymised by TM, IP and VP.
Data analysis
Thematic analysis was used to identify patterns and varying views on the data in line with the approach recommended by Braun and Clarke (2006).23 Both deductive approaches, guided by previous research on chatbots, as well as an inductive approach, grounded in interview data, were used for the analysis. Authors TM, IP and VP thoroughly familiarised themselves with the data by reading it through multiple times. Microsoft Excel spreadsheets were used to classify all the data into themes, sub-themes and exemplar quotes. Three researchers analysed the transcripts independently, coding sub-themes and themes. Next, the analyses were compared in group discussions to increase reflexivity, by debating and agreeing on final themes and subthemes in line with the research objectives.24 To further increase transparency and credibility of data analysis, all procedures, themes, sub-themes and quotes were scrutinised by an independent senior researcher (TN) who reported back to the authors, all of whom subsequently contributed to multiple iterations of the manuscript before it was finalised. A targeted sample size of 40 participants was set prior to data collection and deemed as sufficient for thematic analysis. Saturation, in which no new or additional issues were identified, was reached with the sample of 40 and no further recruitment was required.
Ethics approval
The study received the approval of the University of Westminster Research Governance and Ethics Committee (reference: ETH1920-0381).
Results
Forty participants (age 18–50 years; median age 27 years; 64% women; 77% self-identified heterosexual; and 58% white British or European) from south-east England took part in the study. Two major themes of barriers and facilitators, with seven sub-themes each, to engagement with sexual health chatbots were identified (Table 1).
Facilitators for sexual health chatbot use
The accessibility of chatbots and immediate provision of sexual health information, regardless of the location and time, were seen as advantageous (sub-theme: ‘Convenience’). Participants perceived chatbots that could incorporate interaction with users in the form of reminders, self-help tips, advice about healthier lifestyles as useful and potentially engaging. Chatbots that could reduce large volumes of text, typically seen on websites and webpages, to a single most relevant message were viewed as attractive having an impact on users’ time spent searching for relevant information. Chatbots were seen as potential hubs for links and information about STI/HIV screening, condom distribution or support groups (‘Enabling access to clinical services’). They were viewed as a virtual place where questions about STIs and treatment could be easily answered with directions to appropriate clinical services or relevant organisations offering professional help and support. Some participants felt that chatbots were free of moral judgement and unable to discriminate and marginalise users based on their characteristics and sexual practices (‘Neutral and non-judgemental tool’). A few reported that they would be more likely to disclose highly sensitive information about sexual behaviours such as condomless sex to a chatbot compared to certain health professionals, such as general practitioners and those without specific training in sexual health (‘Enabling disclosure of potentially embarrassing information’). The apparent lack of traceability of sexual health chatbots, where the information could be exchanged anonymously, was seen as an important factor promoting engagement, especially for users who did not wish to be identified (‘Anonymity’). Here, the participants emphasised that the ability to ask difficult questions about their sexual health without revealing their identity was advantageous over clinical visits or telephone conversations.
Sexual health chatbots were thought to be helpful in tasks such as symptom checking, clinic finding and as an information hub about STI risks. The interface, layout, design and appearance of chatbots were seen as essential for interaction and engagement with some highlighting the importance of vibrant and exciting graphics and short videos in addition to interactive messages (‘Ease and accessibility of health information’). Chatbots capable of reaching young people or specific minority groups via tailoring of information and design were seen as more effective (‘Reaching the “seldom heard”’). The best use of this technology was attributed to sex education, where sensitive questions about sex could be asked freely and openly. There were mixed views on whether chatbots should be linked with sexual health services, with some believing that they could be used to support the work of clinicians, and improve the communication between patients and clinics.
Barriers to sexual health chatbot use
Lack of awareness and previous experience of chatbots for sexual health were identified as major limitations, all affecting attitudes towards the technology (‘Awareness and understanding of chatbots’). The majority of participants were unfamiliar with chatbots specifically designed for sexual health advice, but most acknowledge its potential for helping users find relevant information. The participants emphasised their preferences for human-to-human contact when discussing their risk of STIs or contraception (‘Comparison to human interactions’). Here, chatbot competence was not perceived to be sufficient for meaningful consultations. For most participants, the interaction with the chatbot (PAT) was described as a novel and confusing experience, as the technology was perceived as still in development, and thus limited. Participants familiar with chatbots used for customer service or banking had especially negative attitudes due to the perception that this technology was unable to provide adequate and relevant information, especially in the sexual health context, which was thought to require the use of sensitive languages, such as due to the stigma associated with STIs.
Chatbots were seen as lacking important human traits, including empathy and the ability to process and understand emotions (‘Lacking cognitive and affective empathy’). The responses given by chatbots were seen as dry and generic. Interactions were perceived as limited in exploring individual issues and contexts, lacking sufficient depth to make clinical judgements and appropriate recommendations (‘Limited interactivity’). Chatbots were considered restricted in offering personalised advice, as participants had doubts about the effectiveness of an algorithm or computer pattern being able to provide advice on sex and the complexities associated with lifestyles and activities. Participants were also sceptical that chatbots were capable of helping users who felt anxious about their sexual health, specifically in the context of HIV, pregnancy and other aspects of health that are perceived as highly consequential, potentially severe and/or stigmatising.
Chatbots were also seen as lacking diverse content on a wide range of sex-related topics and issues; however, they were perceived as most useful for sign-posting to various services. Participants did not believe that users with specific needs, such as those concerned about ‘polycystic ovaries syndrome’, would find relevant and in-depth information using chatbots (‘Limited content’). The technology was seen as only providing advice about mainstream, easily accessible information, already available on the Internet. Subsequently, some struggled to understand the need for chatbots in sexual health. Instead, conversations with a computer were typically cast as frustrating, due to the lack of prompts and follow-ups. The chatbot language and the method of communication using simple phrases were seen by some users as too simplistic, unsophisticated and limiting.
Participants were concerned about the trustworthiness, data handling and privacy of chatbots. They worried about the lack of confidentiality when using chatbots, and that they did know who could read their responses. Most felt hesitant to answer highly sensitive questions such as HIV status or about engaging in condomless sex. Participants were uncertain how the data were collected, and where they were stored, being anxious that it could be misused against them (‘Concerns about confidentiality and privacy’). Hence, face-to-face interactions with health professionals were seen as safer and reliable. Chatbots’ clinical advice was seen as less accurate and relevant than that of trained professionals, whose training, knowledge and experience were viewed as essential in providing recommendations for STI screening, contraception and treatment (‘Limited credibility, competence and accuracy’). In summary, chatbots were not seen as competent or capable of responding to complex sexual health issues. Participants also perceived chatbots as inferior to the Internet search engines or NHS websites, in comparison to their familiarity with these platforms and the depth of information provided.
Discussion
To our knowledge, this is the first study offering insights on the barriers and facilitators for the engagement with AI-led chatbots in sexual health. Despite low levels of awareness about chatbots, participants had some positive views on this technology in general, following their engagement with a type of chatbot used as an example. They highlighted chatbots’ anonymity, privacy and the lack of judgement as potential advantages. There was a preference for user–chatbot interaction when enquiring about sensitive matters that were seen as difficult or embarrassing to disclose during face-to-face health consultations. These findings indicate a potential role of chatbots in facilitating clinic–patient communication, adopting this technology for pre-consultation sexual health history taking, or preparing users for documenting uncomfortable questions, which they might expect during live consultations with health professionals. However, technological limitations, restricted interactions between users and chatbots, as well as the lack of empathy were viewed negatively by some users. Engagement with chatbots was often compared to human interactions and deemed inferior in providing a whole and reliable sexual health advice. Therefore, the results indicate that this technology could be of use for sign-posting, such as on information about where to test for HIV/STIs. However, it may be especially unsuitable for matters that typically evoke high levels of anxiety such as risk behaviours for and symptoms of HIV infection. Chatbots could aid access and engagement with SRHSs, for example as a screening tool for patient needs,25 rather than as a service replacement.
There was a wide range of perceived barriers and facilitators to chatbot engagement. Some participants expressed low interest in sexual health chatbots due to their limited technological development, algorithm simplicity, limited keywords, restricted sexual health advice and constrained input options. There was a noticeable feeling of frustration and hesitation to engage with chatbots due to the perceived under-development of the technology, and the limited ability to provide advice on a wide range of health topics. Consistent with this finding, Vaira et al. (2018)26 reported that rule-based chatbots operating on a pre-established list of questions and answers were associated with user dissatisfaction, related to limits in expressing medical concerns. Perceived restricted capabilities to mimic human interactions have also been associated with hesitancy to use chatbots in healthcare in general, demonstrating widespread user comparison of chatbot abilities with those of trained health professionals.27 The limited capability for interaction, and the lack of flexibility to process a range of specific personal questions, had a negative impact on engagement, with most users indicating preferences for human-to-human interaction instead. Mierzwa et al. (2019)28 demonstrated low engagement and modest acceptability of medial chatbots, due to their inability to understand or display human emotion, highlighting the importance of cognitive and affective empathy in sexual health consultations. This is also reflected in the findings of Gao et al. (2020)29 showing that the absence of the human care aspect and the immaturity of AI technology and distrust of related companies were the main reasons users held negative attitudes about medical chatbots. As shown in previous research on AI, concerns over data protection and user privacy emerged as a barrier for some; numerous participants suggested a low inclination to interact with sexual health chatbots and to provide any private or personal information that might lead to identification.11 Several participants perceived chatbot technology as not secure enough and were concerned about any potential breach of confidentiality related to sexual health records. However, the anonymity offered by chatbots was seen as an important incentive for engagement, with the majority of participants expressed willingness to disclose information about their sexual behaviours if there were no traces of their online activity. As confidentiality is key to the successful provision of sexual health services,30,31 the potential for anonymity offered by chatbots could be attractive, especially for individuals experiencing barriers to accessing community-based services, including young people and sexual/gender minority groups. Chatbots could potentially raise awareness of available services and increase knowledge of STIs and screening services promoting self-care behaviours. However, the potential limitations to the traceability of chatbot users while offering an anonymous platform for sexual health advice needs further investigation. Such constraints to data collection are important when evaluating the effectiveness of chatbots services if most users disengage with them due to privacy concerns. Our findings indicate that anonymous chatbot services are more acceptable, and may lead to higher engagement. However, due to the lack of familiarity with this innovation, most users remained cautious about the technology and information provided.
Limitations
The present study enhances the understanding of engagement with sexual health chatbots as an emerging tool for health promotion and sex education. It offers novel knowledge on the potential applicability as well as limitations of this technology, highlighting the need for further research on chatbot effectiveness as a supplementary tool. Unlike acceptability studies based on hypothetical chatbots, this study used an existing sexual health chatbot enabling participants to discuss their experiences and provide a more experiential perspective. However, the findings could be influenced by the particular characteristics of the PAT chatbot used for demonstration, and different chatbot designs could evoke additional views. Future studies should offer a range of chatbots for participants to experience a broader and more objective perspective on this technology. It is also possible that participants with predetermined views on digital SRHSs in general self-referred for this study, thus we may have missed the views of those with lower levels of digital literacy or engagement with online services. Future studies could explore if health chatbots make sexual health advice more accessible for individuals that struggle to navigate through the Internet in search of reliable health information. This study took place in the first months of the COVID-19 pandemic, thus an online data collection method was implemented. As such, views on sexual health chatbots could be influenced by using digital technology for interviews (e.g. selecting those more technologically savvy), and social distancing measures, which include restricted access to SRHSs. Finally, it was not within the scope of this inquiry to measure the differences in views by any demographic characteristics; hence, there was no outline of views by age, gender, ethnicity or sexual orientation. Future quantitative studies need to measure if any particular demographic groups, such as ethnic minorities or those with limited access to the Internet, are more hesitant to this technology.
As face-to-face interactions are primarily the most preferred mode of communication regarding sexual health, a combination of chatbot and human-led services could be the way forward, facilitating access to professional advice and allowing contact with health professionals when required. For example, chatbots could triage online users to corresponding services or webpages containing reliable health information with an option to discuss concerns with live health advisors via webchat or similar facilities. As chatbots services may be a convenient and attractive tool for online sexual health advice, their effectiveness still needs to be established. Future studies should examine the impact of chatbots on individual knowledge, motivation and behaviours such as the uptake of STI and HIV testing. Also, it is important to understand the impact of chatbots on the provision of SRHSs, their demand and accessibility and future quantitative studies and trials should identify to what extent chatbots can be incorporated into SRHS. This technology may support sex education at schools allowing young people to ask often embarrassing questions about sex and sexuality. Service developers need to acknowledge user concerns and preferences to increase engagement and utilisation of this technology. As sexual health chatbots become more common, clear guidelines and regulations on their use are needed to prevent potential harms and unintended effects.
Data availability
Data available on request.
Conflicts of interest
The authors have no relevant financial or non-financial interests to disclose.
Declaration of funding
No funding was received for conducting this study.
Author contributions
TN, TM, IP and VP contributed to design, data collection, data analysis, interpretation of results and writing of the manuscript; IM, JB and DR contributed to data interpretation and manuscript editing.
Acknowledgements
We thank the Positive East charity for help with the recruitment of participants.
References
[1] World Health Organization. Global health sector strategy on sexually transmitted infections 2016–2021: toward ending STIs. Geneva: World Health Organization; 2016.[2] Mitchell H, Allen H, Sonubi T, Kuyumdzhieva G, Harb A, Shah A. Sexually transmitted infections and screening for chlamydia in England, 2019. London: Official Statistics of Public Health England; 2020.
[3] Deblonde J, De Koker P, Hamers FF, Fontaine J, Luchters S, Temmerman M. Barriers to HIV testing in Europe: a systematic review. Eur J Public Health 2010; 20 422–32.
| Barriers to HIV testing in Europe: a systematic review.Crossref | GoogleScholarGoogle Scholar | 20123683PubMed |
[4] McDonagh LK, Saunders JM, Cassell J, Curtis T, Bastaki H, Hartney T, Rait G. Application of the COM-B model to barriers and facilitators to chlamydia testing in general practice for young people and primary care practitioners: a systematic review. Implement Sci 2018; 13 130
| Application of the COM-B model to barriers and facilitators to chlamydia testing in general practice for young people and primary care practitioners: a systematic review.Crossref | GoogleScholarGoogle Scholar | 30348165PubMed |
[5] Public Health England. The impact of the COVID-19 pandemic on prevention, testing, diagnosis and care for sexually transmitted infections, HIV and viral hepatitis in England. London: Public Health England; 2020.
[6] Jacob L, Smith L, Butler L, Barnett Y, Grabovac I, McDermott D, Armstrong N, Yakkundi A, Tully MA. Challenges in the practice of sexual medicine in the time of COVID-19 in the United Kingdom. J Sex Med 2020; 17 1229–36.
| Challenges in the practice of sexual medicine in the time of COVID-19 in the United Kingdom.Crossref | GoogleScholarGoogle Scholar | 32411271PubMed |
[7] Guse K, Levine D, Martins S, Lira A, Gaarde J, Westmorland W, Gilliam M. Interventions using new digital media to improve adolescent sexual health: a systematic review. J Adolesc Health 2012; 51 535–43.
| Interventions using new digital media to improve adolescent sexual health: a systematic review.Crossref | GoogleScholarGoogle Scholar | 23174462PubMed |
[8] Bailey J, Mann S, Wayal S, Hunter R, Free C, Abraham C, Murray E. Sexual health promotion for young people delivered via digital media: a scoping review. Public Health Res 2015; 3 1–20.
| Sexual health promotion for young people delivered via digital media: a scoping review.Crossref | GoogleScholarGoogle Scholar |
[9] Gabarron E, Wynn R. Use of social media for sexual health promotion: a scoping review. Glob Health Action 2016; 9 32193
| Use of social media for sexual health promotion: a scoping review.Crossref | GoogleScholarGoogle Scholar | 27649758PubMed |
[10] Marcus JL, Hurley LB, Krakower DS, Alexeeff S, Silverberg MJ, Volk JE. Use of electronic health record data and machine learning to identify candidates for HIV pre-exposure prophylaxis: a modelling study. Lancet HIV 2019; 6 e688–95.
| Use of electronic health record data and machine learning to identify candidates for HIV pre-exposure prophylaxis: a modelling study.Crossref | GoogleScholarGoogle Scholar | 31285183PubMed |
[11] Marcus JL, Sewell WC, Balzer LB, Krakower DS. Artificial intelligence and machine learning for HIV prevention: emerging approaches to ending the epidemic. Curr HIV/AIDS Rep 2020; 17 171–9.
| Artificial intelligence and machine learning for HIV prevention: emerging approaches to ending the epidemic.Crossref | GoogleScholarGoogle Scholar | 32347446PubMed |
[12] Laranjo L, Dunn AG, Tong HL, Kocaballi AB, Chen J, Bashir R, Surian D, Gallego B, Magrabi F, Lau AYS, Coiera E. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018; 25 1248–58.
| Conversational agents in healthcare: a systematic review.Crossref | GoogleScholarGoogle Scholar | 30010941PubMed |
[13] Tudor Car L, Dhinagaran DA, Kyaw BM, Kowatsch T, Joty S, Theng YL, Atun R. Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res 2020; 22 e17158
| Conversational agents in health care: scoping review and conceptual analysis.Crossref | GoogleScholarGoogle Scholar | 32763886PubMed |
[14] Milne-Ives M, de Cock C, Lim E, Shehadeh MH, de Pennington N, Mole G, Normando E, Meinert E. The Effectiveness of artificial intelligence conversational agents in health care: systematic review. J Med Internet Res 2020; 22 e20346
| The Effectiveness of artificial intelligence conversational agents in health care: systematic review.Crossref | GoogleScholarGoogle Scholar | 33090118PubMed |
[15] Brixey J, Hoegen R, Lan W, Rusow J, Singla K, Yin X, Artstein R, Leuski A. Shihbot: a facebook chatbot for sexual health information on HIV/AIDS. Proceedings of the 18th annual SIGdial meeting on discourse and dialogue; 15–17 August 2017; Saarbrucken, Germany. Stroudsburg: Association for Computational Linguistics; 2017. p. 370–3.
[16] Crutzen R, Peters GJ, Portugal SD, Fisser EM, Grolleman JJ. An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study. J Adolesc Health 2011; 48 514–19.
| An artificially intelligent chat agent that answers adolescents’ questions related to sex, drugs, and alcohol: an exploratory study.Crossref | GoogleScholarGoogle Scholar | 21501812PubMed |
[17] Maeda E, Miyata A, Boivin J, Nomura K, Kumazawa Y, Shirasawa H, Saito H, Terada Y. Promoting fertility awareness and preconception health using a chatbot: a randomized controlled trial. Reprod Biomed Online 2020; 41 1133–43.
| Promoting fertility awareness and preconception health using a chatbot: a randomized controlled trial.Crossref | GoogleScholarGoogle Scholar | 33039321PubMed |
[18] Bickmore T, Zhang Z, Reichert M, Julce C, Jack B. Promotion of preconception care among adolescents and young adults by conversational agent. J Adolesc Health 2020; 67 S45–51.
| Promotion of preconception care among adolescents and young adults by conversational agent.Crossref | GoogleScholarGoogle Scholar | 32718515PubMed |
[19] Dworkin MS, Lee S, Chakraborty A, Monahan C, Hightow-Weidman L, Garofalo R, Qato DM, Liu L, Jimenez A. Acceptability, feasibility, and preliminary efficacy of a theory-based relational embodied conversational agent mobile phone intervention to promote HIV medication adherence in young HIV-positive African American MSM. AIDS Educ Prev 2019; 31 17–37.
| Acceptability, feasibility, and preliminary efficacy of a theory-based relational embodied conversational agent mobile phone intervention to promote HIV medication adherence in young HIV-positive African American MSM.Crossref | GoogleScholarGoogle Scholar | 30742481PubMed |
[20] Park H, Joonhwan L. Can a conversational agent lower sexual violence victims’ burden of self-disclosure?. CHI EA ’20: Extended Abstracts of the 2020 CHI Conference on human factors in computing systems; 25 April 2020; Honolulu, HI, USA. New York: Association for Computing Machinery; 2020. p. 1–8.
[21] Kocielnik R, Agapie E, Argyle A, Hsieh DT, Yadav K, Taira B, Hsieh G. HarborBot: a chatbot for social needs screening. AMIA Annu Symp Proc 2019; 552–61.
| 32308849PubMed |
[22] Nadarzynski T, Bayley J, Llewellyn C, Kidsley S, Graham CA. Acceptability of artificial intelligence (AI)-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice. BMJ Sex Reprod Health 2020; 46 210–7.
| Acceptability of artificial intelligence (AI)-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice.Crossref | GoogleScholarGoogle Scholar | 31964779PubMed |
[23] Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 77–101.
| Using thematic analysis in psychology.Crossref | GoogleScholarGoogle Scholar |
[24] Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet 2001; 358 483–488.
| Qualitative research: standards, challenges, and guidelines.Crossref | GoogleScholarGoogle Scholar | 11513933PubMed |
[25] Espinoza J, Crown K, Kulkarni O. A guide to chatbots for COVID-19 screening at pediatric health care facilities. JMIR Public Health Surveill 2020; 6 e18808
| A guide to chatbots for COVID-19 screening at pediatric health care facilities.Crossref | GoogleScholarGoogle Scholar | 32325425PubMed |
[26] Vaira L, Bochicchio MA, Conte M, Casaluci FM, Melpignano A. MamaBot: a System based on ML and NLP for supporting Women and Families during Pregnancy. In Proceedings of the 22nd International Database Engineering & Applications Symposium; June 2018. pp. 273–277.
[27] Nadarzynski T, Miles O, Cowie A, Ridge D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study. Digit Health 2019; 5 1–12.
| Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study.Crossref | GoogleScholarGoogle Scholar |
[28] Mierzwa S, Souidi S, Conroy T, Abusyed M, Watarai H, Allen T. On the potential, feasibility, and effectiveness of chat bots in public health research going forward. Online Journal of Public Health Informatics 2019; 11 e4
| On the potential, feasibility, and effectiveness of chat bots in public health research going forward.Crossref | GoogleScholarGoogle Scholar | 31632598PubMed |
[29] Gao S, He L, Chen Y, Li D, Lai K. Public perception of artificial intelligence in medical care: Content analysis of social media. J Med Internet Res 2020; 22 e16649
| Public perception of artificial intelligence in medical care: Content analysis of social media.Crossref | GoogleScholarGoogle Scholar | 32673231PubMed |
[30] Garside R, Ayres R, Owen M, Pearson VA, Roizen J. Anonymity and confidentiality: rural teenagers’ concerns when accessing sexual health services. BMJ Sex Reprod Health 2002; 28 23–26.
| Anonymity and confidentiality: rural teenagers’ concerns when accessing sexual health services.Crossref | GoogleScholarGoogle Scholar |
[31] Wilson A, Williams R. Sexual health services: what do teenagers want? Ambul Child Health 2000; 6 253–260.
| Sexual health services: what do teenagers want?Crossref | GoogleScholarGoogle Scholar |