A digital citizen science intervention to reduce HIV stigma and promote HIV testing: a randomized clinical trial among adolescents and young adults in Kazakhstan
Alissa Davis
A * , Laura Nyblade B , Yihang Sun A , Olga Balabekova C , Sara E. Landers A , Denis Gryazev C , Joseph D. Tucker D E , Valera Gulyaev C , Susan L. Rosenthal F G , Karsten Lunze H I , Weiming Tang D , Azamat Kuskulov A , Assel Terlikbayeva A C , Sholpan Primbetova A C , Mingway Chang A , Alfiya Y. Denebayeva J , Aikerim Utegulovna Akhmetova J , Ainur Absemetova K , Sholpan Karzhaubayeva L , Sairankul Kassymbekova M , Marina Maximova M , Zhannat Mussina M , Gulnar Bekenova N , Zhamilya Nugmanova O , Zhanna Kalmatayeva O , Ludmila Polyakova P , Zhanneta Kanaevna Zhazykbaeva Q , Vitaliy Vinogradov C , Amir Shaikezhanov C , Meruyert Darisheva C , Bella Orynbetova R , Elena Norakidze S , Mashirov Kozhakhmet T , Akbota Tolegenova U , Aknur Imadillda C # , Dariga Satkhozhina C # , Alikhan Kartamyssov C # , Zhamilya Kanieva C # , Albina Aleshina C # , Olzhas Makhan C # , Aida Muravyova C # , Adema Rahimzanova C # , Arman Duisenbayev C # , Zhanerke Tursynbek C # , Nurgazy Dias C # , Malika Beken C # , Miras Murzakhan C # , Zhandos Ali Brown C # , Daniyal Maitekov C # , Artur Li C # , Sandizaira Mergen C # , Dautali Mergenov C # , Amirali Kuanysh C # , Anuar Rakhimbekov C # , Yenlik Baisbay C # , Alibek Aruzhan C # , Zhasmina Kozhambet C # , Denis Grebenchishikov C # , Dmitriy Bekker C # , Sultan Kozhamberdiev C # , Dauren Salykov C # , Madina Sagimbayeva C # , Kamila Yussupova C # and Gaukhar Mergenova C
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
# These authors contributed equally to this paper
Handling Editor: Chunyan Li
Abstract
Kazakhstan has a high HIV incidence among adolescents and young adults (AYA), and high HIV stigma contributing to low HIV testing uptake. We examined whether an AYA-developed digital crowdsourced intervention reduced HIV stigma compared with conventional public health materials among AYA in Almaty, Kazakhstan.
A total of 216 AYA (females:116/males:110) aged 16–24 years were recruited to the online study cohort and randomized 1:1 to the intervention or control arm. AYA were exposed to the crowdsourced intervention or control materials once a week for 5 weeks, with equivalent exposures between arms. Outcomes included a total HIV stigma score assessed at baseline, immediately post-intervention and 2 months post-intervention from January to August 2023. We conducted multilevel mixed models to compare changes over time by arm and sex.
AYA in the intervention arm had significantly lower HIV testing stigma 2 months post-intervention (adjusted mean change (AMC): −0.73 (−1.07, −0.39)) than AYA in the control arm (AMC: −0.06 (−0.42, 0.30); P = 0.032). Female AYA in the intervention arm had significantly lower total HIV stigma immediately post-intervention (AMC: −4.91 (−7.25, −2.58)) and 2 months post-intervention (AMC: −5.16 (−7.48, −2.84)) than females in the control arm (immediately post-intervention AMC: −0.03 (−2.63, 2.57) and 2-months post-intervention AMC: −0.07 (−2.70, 2.56); P = 0.012, P = 0.012).
The AYA-developed crowdsourced intervention decreased HIV testing stigma, although this effect was moderated by sex, and decreased total HIV stigma among female AYA. Crowdsourced interventions may be a promising way to engage communities to develop interventions to decrease HIV stigma.
Keywords: adolescents and young adults, citizen science, crowdsourcing, digital intervention, HIV, Kazakhstan, stigma, testing.
Introduction
Adolescents and young adults (AYA) often have a high HIV incidence and face HIV stigma.1 Eastern Europe and Central Asia has the world’s fastest-growing HIV epidemic,2 and Kazakhstan has the largest increase within Eastern Europe and Central Asia2 (132.7% increase among AYA from 2018 to 2020).3 Increases in AYA HIV incidence are driven by condomless sex and substance use before/during sex, combined with low HIV testing rates, which result in low HIV status awareness and delayed linkage to antiretroviral therapy.4
A major barrier to HIV testing among AYA is HIV stigma.5–7 HIV testing in Kazakhstan has traditionally been available at specialized AIDS centers, where it is obvious individuals are receiving HIV services, and where unauthorized disclosure of HIV testing and status frequently occurs.6,8–10 AYA often avoid HIV testing, because individuals who attend AIDS centers are generally perceived as engaging in HIV risk behaviors, such as pre-marital/extra-marital sex and substance use, and because it raises the suspicion of being HIV-positive, which is also highly stigmatized. People with HIV in Kazakhstan often face discrimination from family, friends, employers, healthcare providers and others in the community.6,8,9 Although HIV testing is now available in health clinics and non-governmental organizations, and HIV self-test kits can be purchased online or at a pharmacy, many AYA are not aware of these HIV testing options. Additionally, the majority of HIV self-testing messaging is focused on men who have sex with men, and the messaging may not resonate with AYA. AYA-tailored messaging that addresses their stigma concerns is needed in Kazakhstan.
The majority of HIV stigma reduction interventions have been designed for adults.11,12 The small evidence-base on HIV stigma interventions among AYA has focused on those living with HIV, not those at-risk for future HIV acquisition.11 There are also few interventions focused on reducing HIV stigma as a main outcome. Studies examining HIV stigma interventions for AYA have mixed results,11 and UNAIDS and WHO have called for more HIV stigma reduction strategies.13,14 Further work is needed to develop AYA-tailored HIV stigma reduction interventions.15 Our study helps address this gap in the literature by presenting results of an HIV stigma reduction intervention designed specifically for HIV-negative AYA at-risk of future HIV acquisition.16
Citizen science uses participatory methods, such as crowdsourcing, to engage the local community to develop solutions to reduce HIV stigma and promote HIV testing.16,17 Conventional approaches to designing HIV interventions often do not sufficiently involve communities, frequently resulting in low community engagement.18 Crowdsourcing takes a ‘bottom up’ approach to HIV intervention design and has high involvement from local communities. We selected this approach over other participatory methods, such as community-based participatory research (CBPR), because of several key advantages. First, crowdsourcing can engage hundreds or even thousands over a larger geographic area, whereas CBPR methods typically work with smaller, localized community groups. This broader reach allows for more diverse perspectives. Second, crowdsourcing projects can be launched quickly and gather content rapidly through digital platforms. CBPR methods often require significantly more time to build community partnerships and develop projects. Third, crowdsourcing can be more cost-effective, as it often leverages existing platforms, and does not require the same level of sustained community engagement and support that CBPR demands.
The primary purpose of this study was to evaluate the effect of a digital crowdsourced intervention on HIV stigma compared with existing public health messaging among AYA in Almaty, Kazakhstan, using a randomized controlled trial. Additionally, we examined the effects of the digital crowdsourced intervention on HIV stigma subscales (e.g. HIV testing stigma, perceived community HIV stigma) and HIV self-testing uptake as secondary outcomes.
Methods
Study design
We conducted a randomized controlled trial among AYA in Almaty, Kazakhstan. Almaty is the former capital and largest city in Kazakhstan. Participants randomized to the intervention arm viewed AYA-crowdsourced digital materials (video, audio, images, text) over a 5-week period. Participants randomized to the control arm viewed existing multi-media materials obtained from Kazakhstani health organizations over the same period.
Randomization and masking
AYA were recruited online via websites and social media advertisements, and in-person at youth events in Almaty, Kazakhstan. Interested AYA completed an online screening survey. To be eligible, AYA had to: (1) be aged 16–24 years, (2) have had sex in the past year, (3) not be diagnosed with HIV, (4) reside in Almaty, Kazakhstan, (5) speak Russian or Kazakh, and (6) own a smartphone. Eligible AYA completed informed consent online. For adolescents aged <18 years, we were granted a waiver of parental permission, because the study was minimal risk and under Kazakh law, adolescents ages ≥16 years are allowed to obtain an HIV test without parental permission.
A research assistant assigned participants 1:1 to the intervention or control arm using a computer-generated randomization sequence, and scheduled the participants to receive surveys and their assigned intervention or control content. The randomization sequence was checked by two research assistants to ensure accuracy prior to participant assignment. AYA were blinded to their arm assignment and did not know whether they were receiving intervention or control materials. Intervention and control materials were programmed into separate Qualtrics surveys to ensure that participants could only view their assigned content (intervention or control).
Intervention and control materials
To develop digital intervention materials in Russian and Kazakh languages, we conducted an online crowdsourcing open call among AYA across Kazakhstan (Supplementary Fig. S1a, b). We received 96 entries from 77 AYA across Kazakhstan.16 Most submissions were video (n = 36), images (n = 28) or text entries (n = 24). Three entries were audio (e.g. songs, monologues), and five entries were in the ‘other’ category (i.e. online game, crossword puzzle, website, chatbot, online article with graphics). Submissions were judged by a panel of health experts and AYA based on four criteria: (1) potential to reduce HIV stigma to increase HIV testing, (2) innovation, (3) relevancy to youth, and (4) overall impression. Each entry was judged by two AYA and two health experts. Scores were averaged across the four judges, and top submissions were included in the intervention package (detailed description provided in Davis et al. 2024).16 Intervention content was creative and contained elements that would appeal to AYA specifically, such as visual or textual representations of youth, comics, memes, artwork, poems, interactive game components and/or references to pop culture. The conventional materials were obtained from the Kazakhstan Ministry of Healthcare, the Kazakh Scientific Center of Dermatology and Infectious Diseases, and city AIDS centers (Fig. S2a, b). These materials included videos, digital brochures, flyers and images used by Kazakh health organizations to promote HIV testing, and address stigma and fears associated with testing. AYA-specific content was generally not present in conventional materials. Intervention and conventional materials were administered online through weekly emails via Qualtrics for 5 weeks (after pre-intervention survey (Week 0) to immediately post-intervention survey (week 4)), with content scheduled to be delivered at the same time on a weekly basis (2–3 items shown per week). Viewing length of weekly content was approximately 5–10 min. We varied weekly content by type (video, image, text), and aimed to match control content by type and length to the intervention content. Engagement was high. All participants viewed all of their assigned intervention and control materials at Weeks 2 and 3. Only 14 participants at Week 0, four at Week 1 and six at Week 4 did not view all materials. There were no significant differences in viewing completion between arms.
Sampling and data collection
Data were from three sequential surveys administered online to AYA participants – pre-intervention assessment, immediately post-intervention assessment (1 month after the pre-intervention assessment) and 2 months post-intervention (with no intervention or control materials administered after the 1-month assessment). Data for all assessments were collected from 10 January to 22 August 2023.
Measures
Our primary dependent variable was a 17-item HIV stigma scale originally developed for community members not living with HIV in South Africa and Zambia.19 The scale originally included 12 items consisting of the following subscales: fear and judgement (three items), perceived stigma in the community (five items), perceived stigma in healthcare settings (two items), and HIV testing stigma (two items).19 This scale had not been validated in Central Asia, in Russian or Kazakh, or with AYA, thus, we consulted stigma and community experts to adapt HIV stigma items for potential inclusion in the questionnaire. Consultation with experts identified two additional fear and judgement items (i.e. ‘I would not buy fresh vegetables from a shopkeeper or vendor if I knew the person had HIV’ and ‘I would not attend school with someone who is living with HIV’), and one perceived community stigma item (‘People think children living with HIV should not be allowed to attend school with children who are not living with HIV’). AYA and community members on our Community Collaborative Research Board recommended two additional items (‘People living with HIV often lose their jobs when employers learn they have HIV’, and ‘People assume if you ask for an HIV test, you have had unprotected sex or injected drugs’). Item response options were on a 5-point Likert scale (1 – strongly disagree to 5 – strongly agree), with a maximum total score of 85.
We then conducted cognitive interviews with 10 Russian-speaking AYA and nine Kazakh-speaking AYA. AYA provided feedback on the cultural and age-appropriateness of the items, and recommendations on item modification. AYA recommended retaining all 17 items, but that the original wording ‘people thought to be living with HIV’ be modified to ‘people living with HIV’ to increase clarity.
Scale reliability was high in our sample (α = 0.871). We examined changes in Cronbach’s alpha if each of the items was removed; removal of any of the items would result in a decreased alpha. Thus, we retained all items in the scale and conducted analysis based on the total HIV stigma sum score, as well as secondary analyses examining subscale scores (fear and judgement, perceived stigma in the community, perceived stigma in healthcare settings, and HIV testing stigma).
HIV self-testing uptake – AYA who ordered an HIV self-test in the follow-up period after the pre-intervention assessment were coded as ‘1’. AYA who did not order an HIV self-test in the follow-up period were coded as ‘0’. Participants sent pictures of their self-test results via WhatsApp to a research assistant.
Arm – AYA received intervention or control content.
Timing of assessment – The survey was administered at three time points – pre-intervention (month 0), immediately post-intervention (month 1) and 2 months post-intervention (month 3).
Age – Participants reported their age in years on a continuous scale.
Sex at birth – Participants reported sex at birth (male/female). No participants reported being intersex. Participants also reported their gender identity, which aligned with reported sex at birth in most cases. Because we had only eight participants whose gender identity did not align with their sex at birth (four female at birth who identify as male, two male at birth who identify as female and two non-binary individuals), we did not have a large enough sample size to examine potential differences among individuals whose current gender identity did not align with sex at birth. We recognize the importance of gender identity. Thus, we conducted a sensitivity analysis using gender identity rather than sex at birth, which required that we drop the two non-binary individuals due to low sample size. The results did not significantly differ from those using sex at birth. Thus, recognizing that our non-binary participants gave of their time and experience, and to retain them in the analysis, we used sex at birth.
Sexual orientation – Participants reported whether they were heterosexual, gay/lesbian, bisexual or other orientation. Due to small sample sizes, any participant who reported a sexual orientation other than heterosexual was classified as a sexual minority.
Submitted material to JasSpark contest – Participants who had previously submitted material to the JasSpark contest were coded as ‘1’. Those who had not submitted material were coded as ‘0’.
Prior HIV testing – Participants who received an HIV test prior to the study were coded as ‘1’. Those who had not were coded as ‘0’.
Baseline HIV testing – Participants who ordered an HIV test at baseline were coded as ‘1’. Those who did not were coded as ‘0’.
Sample size
Power analyses were conducted with G*Power (v3.1.0) based on the primary outcome of HIV stigma reduction using a Wilcoxon–Mann–Whitney test approach and α = 0.05. With 108 participants per arm, we would have 80% power to detect a small-to-medium effect size (d = 0.39).
Statistical analysis
The primary aim was to compare mean total HIV stigma outcome changes from pre-intervention to the 2-month post-intervention assessment among AYA in the intervention arm versus in the control arm. A secondary aim assessed total HIV stigma changes from pre-intervention to immediately post-intervention between arms. We also examined differences in mean stigma outcome changes among the stigma subscales in secondary analyses. Separate multilevel linear mixed models were calculated for the overall stigma score and then for each of the stigma subscale scores. The multilevel linear mixed models incorporated fixed effects for time (categorical), study arm and their interaction, and included intercept as a random effect. Models were also adjusted for pre-intervention stigma levels, whether the participant had submitted content to the JasSpark study contest, prior HIV testing, age, sex at birth and sexual orientation. Cases with missing data at a particular timepoint were automatically dropped from the analysis using casewise deletion. Effect sizes were estimated using Cohen’s f2.20,21
Sex was marginally significant in the initial models, therefore, we examined potential moderation effects of sex on stigma changes. Fixed effects corresponding to the two-way interactions involving sex, time and/or arm, as well as the three-way interaction of sex, time and arm, were added to the models. To account for multiple comparisons arising from analyses of moderating effects and multiple subscales, the false discovery rate was controlled using Benjamini–Hochberg procedures in a tiered approach.22 First, the P-values associated with the three-way interaction tests across all outcomes were adjusted, followed by the P-values associated with the planned contrasts for just the interaction tests that were significant. Stratified planned contrasts associated with three-way interaction tests that were not significant at P ≤ 0.05 had P-values and 95% confidence intervals withheld.
For HIV self-testing uptake, we used a generalized linear model using a binomial distribution to examine whether the intervention was associated with increased HIV self-testing uptake in the follow-up period. The model was adjusted for history of HIV testing (pre-intervention testing and testing prior to the study), whether the participant had submitted content to the JasSpark contest and if the participant had a main intimate partner (e.g. girlfriend/boyfriend, spouse). All tests were two-sided at the 0.05 significance level. Analyses were performed using SAS v9.4 (SAS Institute, Cary, NC, USA).
Ethics approval
All study protocols received approval from the Institutional Review Board at Columbia University (protocol AAAT7871, approved 16 July 2021) and the Ethics Committee at Al-Farabi Kazakh National University (protocol A324, approved 22 June 2021). Research procedures conformed to the principles embodied in the Declaration of Helsinki.
Results
We had 216 participants complete the pre-intervention assessment, with 111 (51.4%) in the intervention arm and 105 (48.6%) in the control arm (Table 1; Supplementary Table S1). On average, participants were aged almost 20 years. Approximately half of participants were female (n = 116, 53.7%), nearly one- quarter identified as a sexual minority (n = 50, 23.1%) and a little over half had a main intimate partner (n = 120, 55.6%). Participants in the intervention arm were significantly older (20.1 ± 2.4) than participants in the control arm (19.4 ± 2.4, P = 0.022). Baseline HIV stigma levels among participants in both arms were above the 50th percentile (mean 47.6 (out of 85-points)).
| Characteristic | Overall (n = 216) n (%) | Intervention arm (n = 111) n (%) | Control arm (n = 105) n (%) | P-valueA | |
|---|---|---|---|---|---|
| Sex at birth | 0.14 | ||||
| Male | 100 (46.3%) | 46 (41.4%) | 54 (51.4%) | ||
| Female | 116 (53.7%) | 65 (58.6%) | 51 (48.6%) | ||
| Sexual orientation | 0.23 | ||||
| Heterosexual | 166 (76.9%) | 89 (80.2%) | 77 (73.3%) | ||
| Sexual minority | 50 (23.1%) | 22 (19.8%) | 28 (26.7%) | ||
| Ethnicity | 0.49 | ||||
| Kazakh | 121 (56.0%) | 58 (52.3%) | 63 (60.0%) | ||
| Russian | 65 (30.1%) | 37 (33.3%) | 28 (26.7%) | ||
| Other | 30 (14.0%) | 16 (14.4%) | 14 (13.3%) | ||
| Employment status | 0.67 | ||||
| Student | 121 (56.0%) | 59 (53.2%) | 62 (59.0%) | ||
| Employed full/part-time | 81 (37.5%) | 44 (39.6%) | 37 (35.2%) | ||
| Unemployed | 14 (6.5%) | 8 (7.2%) | 6 (5.7%) | ||
| Have a main intimate partner (girlfriend/boyfriend/spouse)? | 0.36 | ||||
| Yes | 120 (55.6%) | 65 (58.6%) | 46 (41.4%) | ||
| No | 96 (44.4%) | 55 (52.4%) | 50 (47.6%) | ||
| In the past 6 months, been homeless or without a regular place to sleep? | 17 (7.9%) | 10 (9.0%) | 7 (6.7%) | 0.52 | |
| Tested for HIV prior to the study? | 0.031 | ||||
| Yes | 63 (29.6%) | 40 (36.0%) | 23 (22.5%) | ||
| No | 150 (70.4%) | 71 (64.0%) | 79 (7.5%) | ||
| HIV test ordered at baseline | 0.027 | ||||
| Yes | 46 (21.3%) | 17 (15.3%) | 29 (27.6%) | ||
| No | 170 (78.7%) | 94 (84.7%) | 76 (72.4%) |
| Overall mean ± s.d. | Intervention arm mean ± s.d. | Control arm mean ± s.d. | |||
|---|---|---|---|---|---|
| Age (years) | 19.7 ± 2.4 | 20.1 ± 2.4 | 19.4 ± 2.4 | 0.022 | |
| HIV stigma scaleB | 47.6 ± 10.2 | 47.1 ± 10.1 | 48.0 ± 10.3 | 0.53 |
Of the 216 participants, 187 completed the survey immediately post-intervention, and 189 completed the 2-months post-intervention survey (see Fig. S3 for participant flow between arms). All AYA had HIV stigma outcome data at pre-intervention, <15% were missing data immediately post-intervention and 2 months post-intervention. The proportion of AYA with stigma outcome data compared with those without outcome data did not significantly differ between study arms at any timepoint.
From pre-intervention to immediately post-intervention, there were no significant differences between arms in changes in total HIV stigma or HIV stigma subscales (Table 2).
| Study arm | Intervention (Arm 1) | Control (Arm 2) | Between group difference in mean change (95% CI)E | P-valueF | |||||
|---|---|---|---|---|---|---|---|---|---|
| TimepointA | Time 1 | Time 2 | Within group mean change (95% CI)D | Time 1 | Time 2 | Within group mean change (95% CI)D | |||
| OutcomeB | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | |||||
| HIV stigma total | 47.48 (0.71) | 44.30 (0.76) | −3.17 (−4.98, −1.36)**** | 47.89 (0.74) | 47.10 (0.79) | −0.80 (−2.68, 1.08) | −2.37 (−4.98, 0.24) | 0.074 | |
| Perceived community HIV stigma | 24.65 (0.45) | 23.34 (0.48) | −1.31 (−2.39, −0.23)** | 25.05 (0.47) | 24.47 (0.50) | −0.58 (−1.71, 0.54) | −0.72 (−2.28, 0.84) | 0.36 | |
| Fear and judgement stigma | 11.12 (0.35) | 9.95 (0.37) | −1.16 (−1.92, −0.40)*** | 11.24 (0.36) | 11.09 (0.38) | −0.16 (−0.94, 0.63) | −1.01 (−2.10, 0.09) | 0.28 | |
| Perceived HIV healthcare stigma | 5.80 (0.14) | 5.45 (0.15) | −0.35 (−0.70, 0.001)* | 5.90 (0.15) | 5.92 (0.16) | 0.02 (−0.35, 0.38) | −0.36 (−0.87, 0.14) | 0.31 | |
| Perceived HIV test stigma | 5.92 (0.13) | 5.56 (0.14) | −0.35 (−0.70, −0.01)** | 5.70 (0.14) | 5.63 (0.14) | −0.07 (−0.43, 0.29) | −0.28 (−0.78, 0.21) | 0.35 | |
*P ≤ 0.1, **P ≤ 0.05, *** P ≤ 0.01, ****P ≤ 0.001.
From pre-intervention to 2 months post-intervention, intervention arm participants reported significantly greater decreases in HIV testing stigma (AMC −0.73 (95% CI −1.07, −0.39); 12.3% decrease) than those in the control arm (AMC −0.06 (95% CI −0.42, 0.30); 1.1% decrease), P = 0.032, f2 = 0.024, which corresponds to a small effect (Table 3).
| Study arm | Intervention (Arm 1) | Control (Arm 2) | Between group difference in mean change (95% CI)E | P-valueF | |||||
|---|---|---|---|---|---|---|---|---|---|
| Survey timepointA | Time 1 | Time 3 | Within group mean change (95% CI)D | Time 1 | Time 3 | Within group mean change (95% CI)D | |||
| OutcomeB | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | Adjusted mean (s.e.)C | |||||
| HIV stigma total | 47.48 (0.71) | 44.60 (0.75) | −2.87 (−4.67, −1.08)** | 47.89 (0.74) | 47.32 (0.79) | −0.58 (−2.46, 1.31) | −2.30 (−4.90, 0.30) | 0.083 | |
| Perceived community HIV stigma | 24.65 (0.45) | 23.28 (0.47) | −1.36 (−2.44, −0.29)* | 25.05 (0.47) | 24.74 (0.50) | −0.31 (−1.44, 0.81) | −1.05 (−2.60, 0.51) | 0.37 | |
| Fear and judgement stigma | 11.12 (0.35) | 10.56 (0.36) | −0.56 (−1.31, 0.20) | 11.24 (0.36) | 11.08 (0.38) | −0.16 (−0.95, 0.63) | −0.39 (−1.49, 0.70) | 0.48 | |
| Perceived HIV healthcare stigma | 5.80 (0.14) | 5.57 (0.15) | −0.23 (−0.58, 0.11) | 5.90 (0.15) | 5.89 (0.16) | −0.02 (−0.38, 0.35) | −0.21 (−0.72, 0.29) | 0.48 | |
| Perceived HIV test stigma | 5.92 (0.13) | 5.19 (0.14) | −0.73 (−1.07, −0.39)*** | 5.70 (0.14) | 5.64 (0.14) | −0.06 (−0.42, 0.30) | −0.67 (−1.16, −0.18) | 0.032 | |
*P ≤ 0.05, **P ≤ 0.01, ***P ≤ 0.001.
Subgroup analyses
Regarding moderating effects of sex from pre-intervention to immediately post-intervention, among female participants, those in the intervention arm reported a significantly larger decrease in total HIV stigma (AMC −4.91 (95% CI −7.25, −2.58); 10.3% decrease) than those in the control arm (AMC −0.03 (95% CI −2.63, 2.57); 0.06% decrease), P = 0.012, f2 = 0.077, which corresponds to a small-to-medium effect (Table 4). Males in the intervention arm did not see measurable change from pre-intervention to immediately post-intervention in levels of total HIV stigma compared with males in the control arm (P = 0.57).
| Outcome A | Potential moderator | Level | Intervention (Arm 1) | Control (Arm 2) | Adjusted difference in mean change (95% CI) D | P-value E | Test for moderation F | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Time 1 | Time 2 | Adjusted mean change (95% CI) C | Time 1 | Time 2 | Adjusted mean change (95% CI) C | |||||||
| Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | |||||||||
| Total HIV stigma | Sex | Male | 47.44 (1.09) | 46.73 (1.17) | −0.71 (−3.52, 2.09) | 47.63 (1.02) | 45.79 (1.14) | −1.84 (−4.53, 0.85) | 1.12 (−2.76, 5.01) | 0.57 | 0.0015 | |
| Female | 47.55 (0.91) | 42.63 (0.97) | −4.91 (−7.25, −2.58)*** | 48.24 (1.06) | 48.21 (1.07) | −0.03 (−2.63, 2.57) | −4.88 (−8.37, −1.39) | 0.012 | ||||
| Perceived community HIV stigma subscale | Sex | Male | 24.40 (0.68) | 24.63 (0.73) | 0.24 (−1.44, 1.91) | 24.35 (0.64) | 22.72 (0.71) | −1.63 (−3.25, −0.02)* | 1.87 (−0.45, 4.19) | 0.15 | 0.0024 | |
| Female | 24.88 (0.57) | 22.50 (0.60) | −2.38 (−3.77, −0.99)*** | 25.74 (0.66) | 25.97 (0.67) | 0.24 (−1.31, 1.79) | −2.62 (−4.70, −0.54) | 0.056 | ||||
| HIV fear and judgement stigma subscale | Sex | Male | 11.47 (0.54) | 10.64 (0.58) | −0.83 (−2.02, 0.36) | 11.90 (0.51) | 11.86 (0.55) | −0.04 (−1.17, 1.10) | −0.79 (−2.44, 0.85) | 0.80 | ||
| Female | 10.85 (0.45) | 9.44 (0.47) | −1.41 (−2.40, −0.42)** | 10.67 (0.53) | 10.41 (0.53) | −0.26 (−1.36, 0.83) | −1.15 (−2.63, 0.33) | |||||
| HIV healthcare stigma subscale | Sex | Male | 5.49 (0.22) | 5.75 (0.24) | 0.27 (−0.27, 0.81) | 5.64 (0.21) | 5.67 (0.23) | 0.03 (−0.49, 0.55) | 0.24 (−0.51, 0.99) | 0.53 | 0.0058 | |
| Female | 6.05 (0.18) | 5.27 (0.19) | −0.78 (−1.23, −0.33)*** | 6.15 (0.21) | 6.13 (0.22) | −0.01 (−0.51, 0.48) | −0.76 (−1.43, −0.09) | 0.052 | ||||
| HIV testing stigma subscale | Sex | Male | 6.10 (0.20) | 5.70 (0.22) | −0.39 (−0.93, 0.14) | 5.75 (0.19) | 5.56 (0.21) | −0.18 (−0.69, 0.33) | −0.21 (−0.95, 0.53) | 0.76 | 0.0375 | |
| Female | 5.78 (0.17) | 5.45 (0.18) | −0.33 (−0.77, 0.12) | 5.69 (0.19) | 5.70 (0.20) | 0.01 (−0.48, 0.51) | −0.34 (−1.00, 0.32) | 0.63 | ||||
*P ≤ 0.05, **P ≤ 0.01, ***P ≤ 0.001.
From pre-intervention to 2 months post-intervention, among female participants, those in the intervention arm reported a significantly larger decrease in total HIV stigma (AMC −5.16 (95% CI −7.48, −2.84); 10.9% decrease) than those in the control arm (AMC −0.07 (95% CI −2.70, 2.56); 0.1% decrease), P = 0.012, f2 = 0.07, which corresponds to a small-to-medium effect (Table 5). Female participants in the intervention arm also had a significantly larger decrease in HIV testing stigma (AMC −1.00 (95% CI −1.44, −0.56); 17.3% decrease) than those in the control arm (AMC 0.13 (95% CI −0.37, 0.63); 2.3% increase), P = 0.004, f2 = 0.098, which corresponds to a small-to-medium effect. Males in the intervention arm did not see measurable change from pre-intervention to 2 months post-intervention in levels of total HIV stigma compared with males in the control arm (P = 0.56), nor in levels of HIV testing stigma (P = 0.87).
| Outcome A | Potential moderator | Level | Intervention (Arm 1) | Control (Arm 2) | Adjusted difference in mean change (95% CI) D | P-value E | Test for moderation F | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Time 1 | Time 3 | Adjusted mean change (95% CI) C | Time 1 | Time 3 | Adjusted mean change (95% CI) C | |||||||
| Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | Adjusted mean (s.e.) B | |||||||||
| Total HIV stigma | Sex | Male | 47.44 (1.09) | 47.80 (1.16) | 0.36 (−2.43, 3.14) | 47.63 (1.02) | 46.41 (1.12) | −1.22 (−3.87, 1.43) | 1.58 (−2.26, 5.42) | 0.56 | 0.0015 | |
| Female | 47.55 (0.91) | 42.39 (0.96) | −5.16 (−7.48, −2.84)*** | 48.24 (1.06) | 48.17 (1.09) | −0.07 (−2.70, 2.56) | −5.09 (−8.59, −1.58) | 0.012 | ||||
| Perceived community HIV stigma subscale | Sex | Male | 24.40 (0.68) | 24.66 (0.72) | 0.27 (−1.40, 1.93) | 24.35 (0.64) | 24.03 (0.70) | −0.32 (−1.91, 1.26) | 0.59 (−1.71, 2.89) | 0.61 | 0.0024 | |
| Female | 24.88 (0.57) | 22.37 (0.60) | −2.51 (−3.89, −1.12)*** | 25.74 (0.66) | 25.37 (0.68) | −0.37 (−1.94, 1.20) | −2.14 (−4.23, −0.04) | 0.091 | ||||
| HIV fear and judgement stigma subscale | Sex | Male | 11.47 (0.54) | 11.46 (0.57) | −0.004 (−1.18, 1.18) | 11.90 (0.51) | 11.47 (0.55) | −0.43 (−1.56, 0.70) | 0.42 (−1.21, 2.06) | 0.80 | ||
| Female | 10.85 (0.45) | 9.89 (0.47) | −0.96 (−1.94, 0.02)* | 10.67 (0.53) | 10.75 (0.54) | 0.08 (−1.03, 1.19) | −1.04 (−2.52, 0.44) | |||||
| HIV healthcare stigma subscale | Sex | Male | 5.49 (0.22) | 5.90 (0.23) | 0.41 (−0.13, 0.94) | 5.64 (0.21) | 5.45 (0.23) | −0.19 (−0.70, 0.32) | 0.60 (−0.14, 1.33) | 0.15 | 0.0058 | |
| Female | 6.05 (0.18) | 5.36 (0.19) | −0.68 (−1.13, −0.24)** | 6.15 (0.21) | 6.27 (0.22) | 0.13 (−0.38, 0.63) | −0.81 (−1.48, −0.13) | 0.052 | ||||
| HIV testing stigma subscale | Sex | Male | 6.10 (0.20) | 5.77 (0.21) | −0.32 (−0.85, 0.20) | 5.75 (0.19) | 5.48 (0.21) | −0.26 (−0.77, 0.24) | −0.06 (−0.79, 0.67) | 0.87 | 0.0375 | |
| Female | 5.78 (0.17) | 4.77 (0.18) | −1.00 (−1.44, −0.56)*** | 5.69 (0.19) | 5.81 (0.20) | 0.13 (−0.37, 0.63) | −1.13 (−1.80, −0.46) | 0.004 | ||||
*P ≤ 0.1, **P ≤ 0.01, ***P ≤ 0.001.
To examine potential reasons for moderating effects by sex, we examined differences in time spent viewing intervention content between male and female participants. On average, participants spent approximately 6 min viewing intervention materials each week, with no significant sex differences overall in the amount of time spent viewing materials. A total of 13 intervention contents (seven developed by females, six developed by males) were administered over the 5-week intervention period. Participants were asked to rate each content viewed on a 1–5 scale (1 = low, 5 = high) about the: (1) quality of the content, (2) interest in the content, (3) how effective they think the content would be at reducing HIV stigma, and (4) how interested they think other AYA would be in the content. For 11 of the 13 entries, males rated the quality of the materials significantly lower than females, regardless of whether the entry was developed by a male or female. Males also rated they were significantly less interested in the content (for 10 of the 13 entries), thought other AYA would be less interested in the content (11 of the 13 entries) and thought the content would be less effective at reducing HIV stigma than females (9 of the 13 entries).
Out of 216 participants, 48 ordered an HIV self-test kit during the 3-month follow-up period (all tested negative). Participants in the intervention were marginally more likely to order an HIV self-test kit during the 3-month follow-up period than participants in the control arm (adjusted relative risk 1.13 (95% CI 0.98, 1.32), P = 0.099; Table S2). As HIV self-testing was a secondary outcome, we were not powered to detect effect sizes for self-testing, and did not have a sufficient sample size to examine sex differences in testing.
Discussion
Crowdsourcing is a novel approach in Central Asia, particularly with AYA. Our study provides evidence that youth-developed crowdsourced digital interventions may decrease HIV stigma, particularly among female AYA. AYA receiving the crowdsourced intervention had a significant decrease in HIV testing stigma from pre-intervention to 2 months post-intervention compared with AYA receiving standard HIV messaging materials, although this effect was moderated by sex. We also found a significant decrease in HIV stigma overall among female participants receiving the crowdsourced intervention compared with females receiving standard HIV messaging from pre-intervention to immediately post-intervention, and from pre-intervention to 2 months post-intervention. To our knowledge, this is the first study using crowdsourcing to develop a digital HIV stigma reduction intervention for AYA in Kazakhstan. Although crowdsourcing has been effective in increasing HIV/STI testing uptake among AYA in Nigeria,15 and in decreasing hepatitis B stigma among men who have sex with men in China,23 prior studies have not examined the effects of crowdsourced interventions on reducing HIV stigma among AYA.
Although we do not fully know the mechanisms of change, AYA-developed intervention content focused on how HIV stigma can be harmful and reduce HIV testing. Some entries addressed misperceptions about the spread of HIV, and promoted the idea that living with HIV is similar to living with other chronic health conditions, such as vision problems or asthma. They promoted the importance of HIV testing as part of managing one’s health. Other entries portrayed positive HIV testing experiences where the AYA were socially supported during the testing and diagnosis process, even if they tested positive for HIV. It is plausible that by countering narratives of stigma and fear through examples specific to AYA, their HIV stigma levels were reduced. Further research with larger sample sizes is needed to fully elucidate the mechanisms of change for citizen science HIV stigma reduction interventions.
Results from our study showed high acceptability and engagement among AYA compared with other studies using crowdsourcing.17,24 We received 96 submissions from 77 youth across every province in Kazakhstan, except one.16 Participation from AYA on our Youth Research Collaborative was greater than 90%, suggesting that citizen science approaches can effectively engage youth in developing solutions to topics important to them.16 Our citizen science approach and results were shared with the Kazakhstan Ministry of Health, as well as with health organizations of other Eastern Europe and Central Asian countries for use in future health campaigns. Representatives from the Kazakhstan Ministry of Health, city AIDS centers, non-governmental organizations, media and other local organizations were involved in designing and implementing the crowdsourcing open call, which likely contributed to our effective outreach strategies and high engagement rates. We recommend that those seeking to conduct citizen science projects in other countries involve a variety of community partners and stakeholders from national and local health organizations to facilitate the implementation, scale-up and sustainability of these projects. Crowdsourcing interventions are inexpensive and easy to deliver via digital platforms, making them feasible to scale-up and sustain.25 Further research with large-scale, national clinical trials is needed to determine if our results are replicable on a broad scale.
Interventions often do not work equally for all groups, yet the majority of studies examining HIV stigma interventions for AYA have not examined potential sex differences in effects.11,15,26 The few studies examining sex differences in stigma interventions have found mixed results. For example, a study examining the effects of a video intervention on transphobia among adolescents in the US found that the intervention had the greatest effect among males.27 Another study examining an HIV stigma reduction intervention in the US for AYA living with HIV found that the intervention was more effective among boys than girls.28 Conversely, a study examining the effects of community-based interventions on gender attitudes among adolescents in Indonesia found girls in Indonesia had a greater reduction in gender stereotypes than boys.29 In a mental illness stigma intervention study among adults in California, US, the intervention was more effective among females.30 These mixed results regarding sex differences indicate that, for many behavioural interventions, we do not yet have an in-depth understanding of the mechanisms of action. In our study, we found that females rated the content better than males, regardless of whether the content was created by a female or a male. These findings suggest that, to have an impact on reducing HIV stigma, content needs to be perceived as high quality and interesting to those viewing it. It may be that perceptions of higher content quality by females resulted in their greater reductions in HIV stigma. The content may not have been sufficiently appealing to males to decrease their stigma levels. It may be that different types of content or messaging are needed to impact male perceptions, or it may be that female stigma is more easily modified. It is also possible that other external factors, such as higher levels of gaming among males than females,31,32 result in differential content standards between males and females. It may be that males have higher standards for digital content that must be met for the content to be impactful. Further research is needed to determine underlying reasons for differences in intervention effects by sex.
This study has several limitations. First, the study used self-reported measures of HIV stigma, which could be subject to social desirability bias and result in under-reporting of stigma. However, this potential bias would not likely be different in the study arms. Second, the RCT only included AYA living in Almaty, Kazakhstan. Almaty is the biggest city in the country; thus, findings may not be fully applicable to AYA living in rural areas. Several studies have found rural–urban differences in stigma levels,33–35 and indicate there may be differential effects of interventions for reducing some stigma measures.36,37 Although Internet access is high in Kazakhstan (~93% of those <25 years own a smartphone),38 AYA living in rural areas of Kazakhstan may have greater difficulty accessing the Internet and participating in online interventions than AYA in urban areas. Third, researchers were not blinded in this study, which could introduce bias. However, the RAs conducting randomization processes were separate from the researchers conducting data analysis, and data analyses were not conducted until data collection was complete, which helped minimize researcher bias. Fourth, although missing data rates were low, it is possible missing data may have impacted our estimates. Finally, our study was powered to detect differences in HIV stigma levels, and HIV self-testing uptake was a secondary outcome. Additional research with larger sample sizes and longer follow-up periods are needed to determine whether crowdsourced interventions reducing HIV stigma can also increase HIV testing uptake.
This study demonstrated that crowdsourcing, a novel citizen science approach, could effectively reduce HIV stigma among female AYA in Kazakhstan. It is one of the first efforts to address HIV stigma among AYA in Kazakhstan, including among adolescents aged <18 years, contributing unique insights to the field. Implementing crowdsourced interventions in other communities impacted by growing HIV epidemics could help decrease HIV stigma, and increase engagement in the HIV care continuum in low- and middle-income countries, and other resource-constrained settings globally.
Participant consent
Eligible participants completed informed consent online. For adolescents aged <18 years, we were granted a waiver of parental permission, because the study was minimal risk and under Kazakh law, adolescents ages ≥16 years are allowed to obtain an HIV test without parental permission.
Data availability
We plan by December 2024 to make JasSpark Study methods, data and results available to the public to the extent that governing data use agreements allow. The data sharing plan will comply with the NIH Data Sharing Policy.
Conflicts of interest
Alissa Davis and Weiming Tang are Associate Editors of Sexual Health, and Joseph Tucker is co-Editor-in-Chief of Sexual Health but were not involved in the peer review or decision-making process for this paper. The authors have no further conflicts of interest to declare.
Declaration of funding
This research was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the Fogarty International Center (FIC) (R21TW012017). Dr Davis was supported by a career development award from the National Institute on Drug Abuse (K01DA044853). Dr Tucker is supported by a Mid-Career Award from the National Institute of Allergy and Infectious Diseases (K24AI143471). The Fogarty International Center (FIC), Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), and National Institute on Drug Abuse (NIDA) had no involvement in the collection, analysis and interpretation of data, or in the writing of this report. The content of this manuscript and decision to submit for publication is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Author contributions
All authors were involved in reviewing the manuscript and approved the submission of the manuscript for publication. AD had the final responsibility for submitting the manuscript. AD, OB, YS, SEL, VG, DG, MC and GM had access to the raw data. AD, LN, JDT, SLR, KL, WT, AT, SP and GM were involved in funding acquisition. AD was involved in study design and implementation, data analysis, writing the manuscript, and preparing the manuscript for submission. LN was involved in study design and implementation, writing sections of the manuscript, and reviewing and editing the manuscript. YS was involved in study design and implementation, data analysis, and reviewing and editing the manuscript. OB was involved in data collection, and reviewing and editing the manuscript. SEL was involved in study design and implementation, data analysis, and reviewing and editing the manuscript. DG was involved in data collection, and reviewing and editing the manuscript. JDT was involved in study design and implementation, and reviewing and editing the manuscript. VG was involved in data collection, and reviewing and editing the manuscript. SLR was involved in study design and implementation, and reviewing and editing the manuscript. KL was involved in study design and implementation, and reviewing and editing the manuscript. WT was involved in study design and implementation, and reviewing and editing the manuscript. AK was involved in data collection, and reviewing and editing the manuscript. AT was involved in study design and implementation, and reviewing and editing the manuscript. SP was involved in study design and implementation, and reviewing and editing the manuscript. MC was involved in data analysis, and reviewing and editing the manuscript. GM was involved in study design and implementation, data collection, and reviewing and editing the manuscript.
Acknowledgements
We acknowledge all members of our Community Collaborative Research Board, our Youth Advisory Board and AYA contestants participating in the open call for their assistance and collaboration on the study.
References
1 Pettifor A, Bekker LG, Hosek S, DiClemente R, Rosenberg M, Bull SS, et al. Preventing HIV among young people: research priorities for the future. J Acquir Immune Defic Syndr 2013; 63(Supplement 2): S155-60.
| Crossref | Google Scholar |
2 UNAIDS. UNAIDS data 2021. Geneva, Switzerland: UNAIDS; 2021. Available at https://www.unaids.org/en/resources/documents/2021/2021_unaids_data [cited 20 January 2021]
3 Bilibayeva G, Ospanova D, Nurkerimova A, Kussainova F, Tukeev M, Shokybaeva M, et al. Epidemiological analysis of HIV/AIDS in Kazakhstan during 2018–2020. J Res Health Sci 2023; 23(2): e00580.
| Crossref | Google Scholar |
5 Nyblade L, Mingkwan P, Stockton MA. Stigma reduction: an essential ingredient to ending AIDS by 2030. Lancet HIV 2021; 8(2): e106-13.
| Crossref | Google Scholar |
6 Davis A, Terlikbayeva A, Terloyeva D, Primbetova S, El-Bassel N. What prevents central Asian migrant workers from accessing HIV testing? Implications for increasing HIV testing uptake in Kazakhstan. AIDS Behav 2017; 21(8): 2372-80.
| Crossref | Google Scholar | PubMed |
7 Gesesew HA, Tesfay Gebremedhin A, Demissie TD, Kerie MW, Sudhakar M, Mwanri L. Significant association between perceived HIV related stigma and late presentation for HIV/AIDS care in low and middle-income countries: a systematic review and meta-analysis. PLoS ONE 2017; 12(3): e0173928.
| Crossref | Google Scholar |
8 Stringer KL, Mukherjee T, McCrimmon T, Terlikbayeva A, Primbetovac S, Darisheva M, et al. Attitudes towards people living with HIV and people who inject drugs: a mixed method study of stigmas within harm reduction programs in Kazakhstan. Int J Drug Policy 2019; 68: 27-36.
| Crossref | Google Scholar |
9 Boltaev AA, El-Bassel N, Deryabina AP, Terlikbaeva A, Gilbert L, Hunt T, et al. Scaling up HIV prevention efforts targeting people who inject drugs in Central Asia: a review of key challenges and ways forward. Drug Alcohol Depend 2013; 132: S41-7.
| Crossref | Google Scholar |
10 El-Bassel N, McCrimmon T, Wu E, Chang M, Terlikbayeva A, Hunt T, et al. Effectiveness of an intervention to improve HIV service delivery for people who inject drugs in Kazakhstan: a cluster trial. JAMA Netw Open 2022; 5(12): e2244734.
| Crossref | Google Scholar | PubMed |
11 Hartog K, Hubbard CD, Krouwer AF, Thornicroft G, Kohrt BA, Jordans MJD. Stigma reduction interventions for children and adolescents in low- and middle-income countries: systematic review of intervention strategies. Soc Sci Med 2020; 246: 112749.
| Crossref | Google Scholar |
12 Kane JC, Elafros MA, Murray SM, Mitchell EMH, Augustinavicius JL, Causevic S, et al. A scoping review of health-related stigma outcomes for high-burden diseases in low- and middle-income countries. BMC Med 2019; 17(1): 17.
| Crossref | Google Scholar | PubMed |
14 World Health Organization. HIV basic knowledge and stigma reduction in health care settings. World Health Organization; 2015. Available at https://applications.emro.who.int/dsaf/EMROPUB_2015_EN_1887.pdf?ua=1
15 Iwelunmor J, Ezechi O, Obiezu-Umeh C, Gbaja-Biamila T, Musa AZ, Nwaozuru U, et al. Enhancing HIV self-testing among Nigerian youth: feasibility and preliminary efficacy of the 4 youth by youth study using crowdsourced youth-led strategies. AIDS Patient Care STDS 2022; 36(2): 64-72.
| Crossref | Google Scholar |
16 Davis A, Rosenthal SL, Tucker JD, Balabekova O, Nyblade L, Sun Y, et al. A citizen science approach to develop a digital intervention to reduce HIV stigma and promote HIV self-testing among adolescents and young adults: a mixed methods analysis from Kazakhstan. J Int AIDS Soc 2024; 27(S3): e26314.
| Crossref | Google Scholar |
17 Wang C, Han L, Stein G, Day S, Bien-Gund C, Mathews A, et al. Crowdsourcing in health and medical research: a systematic review. Infect Dis Poverty 2020; 9(1): 8.
| Crossref | Google Scholar | PubMed |
18 Tucker JD, Day S, Tang W, Bayus B. Crowdsourcing in medical research: concepts and applications. PeerJ 2019; 7: e6762.
| Crossref | Google Scholar |
19 Stangl AL, Lilleston P, Mathema H, Pliakas T, Krishnaratne S, Sievwright K, et al. Development of parallel measures to assess HIV stigma and discrimination among people living with HIV, community members and health workers in the HPTN 071 (PopART) trial in Zambia and South Africa. J Int AIDS Soc 2019; 22(12): e25421.
| Crossref | Google Scholar |
20 Lorah J. Effect size measures for multilevel models: definition, interpretation, and TIMSS example. Large-Scale Assess Educ 2018; 6(1): 8.
| Crossref | Google Scholar |
22 Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 1995; 57(1): 289-300.
| Crossref | Google Scholar |
23 Shen K, Yang NS, Huang W, Fitzpatrick TS, Tang W, Zhao Y, et al. A crowdsourced intervention to decrease hepatitis B stigma in men who have sex with men in China: a cohort study. J Viral Hepat 2020; 27(2): 135-42.
| Crossref | Google Scholar | PubMed |
24 Tahlil KM, Rachal L, Gbajabiamila T, Nwaozuru U, Obiezu-Umeh C, Hlatshwako T, et al. Assessing engagement of adolescents and young adults (AYA) in HIV research: a multi-method analysis of a crowdsourcing open call and typology of AYA engagement in Sub-Saharan Africa. AIDS Behav 2023; 27(S1): 116-27.
| Crossref | Google Scholar |
25 Ong JJ, Booton RD, Tucker JD, Tang W, Vickerman P, Zhang L, et al. Economic evaluation of improving HIV self-testing among MSM in China using a crowdsourced intervention: a cost-effectiveness analysis. AIDS 2023; 37(4): 671-78.
| Crossref | Google Scholar | PubMed |
26 Tang W, Wei C, Cao B, Wu D, Li KT, Lu H, et al. Crowdsourcing to expand HIV testing among men who have sex with men in China: a closed cohort stepped wedge cluster randomized controlled trial. PLoS Med 2018; 15(8): e1002645.
| Crossref | Google Scholar |
27 Amsalem D, Halloran J, Penque B, Celentano J, Martin A. Effect of a brief social contact video on transphobia and depression-related stigma among adolescents: a randomized clinical trial. JAMA Netw Open 2022; 5(2): e220376.
| Crossref | Google Scholar | PubMed |
28 Harper GW, Lemos D, Hosek SG. Stigma reduction in adolescents and young adults newly diagnosed with HIV: findings from the project ACCEPT intervention. AIDS Patient Care STDS 2014; 28(10): 543-54.
| Crossref | Google Scholar |
29 Beckwith S, Li M, Barker KM, Gayles J, Kågesten AE, Lundgren R, et al. The impacts of two gender-transformative interventions on early adolescent gender norms perceptions: a difference-in-difference analysis. J Adolesc Health 2023; 73(1): S55-64.
| Crossref | Google Scholar |
30 Wong EC, Collins RL, Cerully JL, Yu JW, Seelam R. Effects of contact-based mental illness stigma reduction programs: age, gender, and Asian, Latino, and White American differences. Soc Psychiatry Psychiatr Epidemiol 2018; 53(3): 299-308.
| Crossref | Google Scholar | PubMed |
31 Gao Y-X, Wang J-Y, Dong G-H. The prevalence and possible risk factors of internet gaming disorder among adolescents and young adults: systematic reviews and meta-analyses. J Psychiatr Res 2022; 154: 35-43.
| Crossref | Google Scholar |
32 Müller KW, Janikian M, Dreier M, Wölfling K, Beutel ME, Tzavara C, et al. Regular gaming behavior and internet gaming disorder in European adolescents: results from a cross-national representative survey of prevalence, predictors, and psychopathological correlates. Eur Child Adolesc Psychiatry 2015; 24(5): 565-74.
| Crossref | Google Scholar | PubMed |
33 Davis A, Stringer KL, Drainoni M-L, Oser CB, Knudsen HK, Aldrich A, et al. Community-level determinants of stakeholder perceptions of community stigma toward people with opioid use disorders, harm reduction services and treatment in the HEALing Communities Study. Int J Drug Policy 2023; 122: 104241.
| Crossref | Google Scholar |
34 Kalichman S, Katner H, Banas E, Kalichman M. Population density and AIDS-related stigma in large-urban, small-urban, and rural communities of the southeastern USA. Prev Sci 2017; 18(5): 517-25.
| Crossref | Google Scholar | PubMed |
35 Schroeder S, Tan CM, Urlacher B, Heitkamp T. The role of rural and urban geography and gender in community stigma around mental illness. Health Educ Behav 2021; 48(1): 63-73.
| Crossref | Google Scholar | PubMed |
36 Davis A, Knudsen HK, Walker DM, Chassler D, Lunze K, Westgate PM, et al. Effects of the communities that heal (CTH) intervention on perceived opioid-related community stigma in the HEALing communities study: results of a multi-site, community-level, cluster-randomized trial. Lancet Reg Health Am 2024; 32: 100710.
| Crossref | Google Scholar |
37 Ashworth M, Thunström L, Clancy GL, Thompson RA, Johnson D, Fletcher E. Addressing rural and non-rural substance use disorder stigma: evidence from a national randomized controlled trial. Addict Behav Rep 2024; 19: 100541.
| Crossref | Google Scholar |
38 World Bank Group. Individuals using the internet (% of population) – Kazakhstan. 2022. Available at https://data.worldbank.org/indicator/IT.NET.USER.ZS
