Condom effectiveness: where are we now?

Additional keywords: misclassification bias, rigor, study design.

In vitro assays have consistently shown that the porosity of condoms is protective against even the smallest sexually transmissible pathogens (e.g. viral infections, including HIV).¹ Unfortunately, in vivo studies become quite complicated. In 2001, the United States Department of Health and Human Services issued a report on a workshop designed to assess the effectiveness of latex condoms for the prevention of sexually transmissible infections (STI).² The report cited in vivo evidence that condoms are effective in preventing HIV transmission and female-to-male transmission of gonorrhoea, but stated that empirical evidence was insufficient to evaluate the degree of risk reduction provided by condoms in regards to chlamydia, trichomoniasis, syphilis, chancroid, genital herpes and human papillomavirus (HPV). As stated in the report (page i), ‘In general, the Panel found the published epidemiology literature to be inadequate to definitively answer the question posed to the workshop participants. Most studies reviewed did not employ a prospective design, which is the optimal method to assess the effectiveness of condoms in preventing infection.’ An unfortunate outcome of this report was the lack of evidence being interpreted negatively to mean that condoms do not work rather than a more empirical appraisal.³

A decade later, a considerably larger body of evidence of condom effectiveness has emerged.^4–11 However, to the best of our knowledge, other large-scale prospective studies specifically funded to test condom effectiveness have not been published. Thus, for now, it is important to understand the value and overall meaning of the extant literature on this topic, most of which is based on secondary data analysis in studies not designed to test condom effectiveness. Accordingly, the purpose of this article is to provide a focussed assessment of the current knowledge base and methodology pertaining to condom effectiveness against STI, including HIV. To achieve this goal, it is first necessary to provide a commentary about the essential methodological principles involved in studies of condom effectiveness. Next, the article will provide a review of only prospective cohort evaluations. Finally, the article will discuss the need to greatly reduce error variance in studies of condom effectiveness.

Unlike evaluating the effectiveness of a medication or surgical procedure, determining the protective value of condoms against STI is an inherently complex endeavour. Two realities are apparent: (1) cross-sectional studies of condom effectiveness are highly problematic and thus the need for prospective cohort studies is quite strong; and (2) the assessment of condom use is reliant on the validity of self-report; this and related measurement issues inherent in the study of condom effectiveness have created a strong bias towards the null^6,7,10,11 (meaning that results unfairly favour the hypothesis that condoms are not effective).

A prospective cohort design with at least two biological assessments for the STI under consideration is essential. Indeed, the hallmark of studies demonstrating condom effectiveness against HIV infection has been the prospective design.^12,13 The purpose of the first assessment is to establish an infection-free cohort (this implies treatment when bacterial or parasitic STI are found). This assures investigators that true incidence measures are evaluated rather than pre-existing STI that may have been acquired before condom use began. Biological testing is conducted at the start and the conclusion of the observational period for condom use. If disease acquisition occurs between the two time points, then the recall period is essentially truncated to conclude at the date of infection (as measuring condom use after infection is a moot point relative to STI acquisition). Thus, the problem of pinpointing the date of the acquired infection remains critical. This leaves the length of time between the Time 1 assessment and disease acquisition as a highly problematic question. Here, it is worth noting that the hidden and often ‘silent’ nature of STI (especially in females) precludes easily pinpointing the event of acquisition. Therefore, interview questions at Time 2 should include items designed to assess probable dates of new STI acquisition. This task may be easy or difficult, depending on the number of sex partners and the condom use behaviours of the person. For example, a symptomatic person who had unprotected sex with a new partner 3 to 10 days before noticing symptoms most likely experienced acquisition during that encounter. Fig. 1 displays a visual depiction of an idealised observational interval.

Fig. 1.  Visual depiction of an observation interval in a condom effectiveness study.

Efforts to pinpoint the infection date are not required under two circumstances: (1) when condoms are used consistently and correctly throughout the recall period, and (2) when condoms are not used at all during the recall period. In either case, the problem of deciding if condom use frequency was either higher or lower before the infection date and after the infection date is not applicable simply because there is no variance. However, when frequency does vary (i.e. condoms were used but not consistently), it is important to know how the variance corresponds with the infection. Given this observation, there is a rationale for the often used dichotomies of ‘always use’ (versus the remainder) and ‘never use’ (versus the remainder), which are so often employed in research designs, such as those selected for inclusion in a review of condom effectiveness against the heterosexual transmission of HIV.¹⁴ However, in each dichotomy, the ‘remainder’ will inevitably contain people using condoms for selected sexual events. Therefore, determining if the non-use events occurred before or after infection is critical to avoid bias.

Given that some cases of STI acquisition cannot be pinpointed to a date, temporal ordering remains a problem even in prospective designs. This liability is especially problematic, given evidence suggesting that condom use may be a response to suspected infection.¹⁵ Consider an example in which a study participant completes a questionnaire or interview using a fairly typical 3-month recall period. Suppose this participant accurately reports having penile–vaginal sex 17 times during that period and using a condom for 15 of those times. Thus, unprotected sex occurred only twice and this person would be classified as a fairly consistent user of condoms. If this person tests positive for an STI, it is rather tempting to conclude that this ‘almost always’ use of condoms simply did not work. Of course, the temporal relationship between the participant’s date of infection and various events of condom-protected and condom-unprotected sex can never be known with certainty. This leaves open the possibility that all 15 acts of condom-protected sex occurred after disease acquisition rather than before, thereby preserving the possibility that condoms offer 100% protection against the STI under investigation. Science is a process of ruling out possibilities, and this example illustrates that such a process cannot operate easily. Although an argument could be made that this problem is overcome by daily or weekly testing for non-viral STI, a minimum of 21 days between DNA amplification tests is required to ensure specificity. As a rule, then, the time between the Time 1 assessment and the Time 2 assessment should be as brief as possible but at least 3 weeks in length.

Although imperfect, the prospective design clearly rules out STI acquisition before the recall period began. In this same regard, cross-sectional studies are problematic because it is not practical to use the very long recall periods that would be needed to accommodate the often ‘simmering’ nature of infections such as chlamydia. A key principle is that recall bias in condom use assessment is reduced – in part – by shorter recall periods such as 30 days. Further, when cross-sectional studies take the form of a case–control design, the potential for recall bias to influence condom use reporting among those testing positive for an STI may be quite high if a positive clinical or laboratory diagnosis was made previous to self-reported data collection. In contrast, prospective studies avoid this possibility and they have the capacity to employ daily diary assessments of condom use during the recall period, thereby minimising the odds of inaccurate reporting.

The science of collecting valid self-reported behavioural data, including data on condom use behaviours, has increased substantially in recent years.¹⁶ Advances in technology, particularly hand-held mobile electronic devices, allow for daily reporting, which may dramatically reduce recall bias. Evidence also suggests that electronic assessments may reduce overestimates of condom use stemming from social desirability bias.¹⁷ In parallel, technology in the form of biomarkers has been advanced. For example, the use of prostate-specific antigens as a marker for unprotected penile–vaginal sex in females may contribute to improved validity.¹⁸ The same is true of a recently developed assay that can detect the presence of Y-chromosomes in vaginal specimens.^5,19

Equally important as valid reporting, emerging evidence shows that assessing condom use must be a rather in-depth process that includes event-specific behaviours that may compromise the protective value of condoms.²⁰ This is best illustrated by answering a nagging question that frequently occurs in studies of condom effectiveness, ‘Given the premise of condom effectiveness, how is it possible for consistent users to be diagnosed with STI such as gonorrhoea, chlamydia and trichomoniasis?’ A good example was published by Zenilman and colleagues and again by Crosby and colleagues.^20,21 They prospectively observed frequent cases of STI acquisition among self-reported consistent condom users. However, neither study assessed whether condoms broke, slipped off, were applied after sex had begun or were removed before sex was concluded. Thus, neither study could adjust for the misclassification bias caused by these forms of incorrect condom use. A subsequent study clearly showed that non-significant associations between condom use and STI became significant upon adjusting for these errors and problems in condom use.¹¹ It is now widely apparent that people make multiple errors when using condoms and that they experience an array of problems.^22–32 Thus studies of condom effectiveness that fail to account for these errors and problems lack rigor. (A review of condom use errors and problems is included in this special issue of Sexual Health.)

Because STI are vastly different from one another, and because the range of available evidence stems from studies not specifically designed to assess condom effectiveness, this review is provided with commentary. More importantly, the review is designed to shed light on salient methodological issues rather than yield an updated review of condom effectiveness evaluations. As already noted in this article, prospective studies provide much more rigorous tests of condom effectiveness, and thus we will not review cross-sectional studies.

Winer and colleagues³³ analysed data from 444 college women who were assessed for various HPV types over a 2-year period and initially tested negative. Genital swab specimens were collected every 4 months. The purpose of this study was to predict risk factors for HPV acquisition rather than to determine the protective value of condoms. Nonetheless, the variable ‘condom use with new partners’ (dichotomised as always v. not always) produced a non-significant hazard ratio (HR) of 0.8 (95% confidence interval (CI) = 0.5–1.2). While it is tempting to consider the physiology of HPV transmission and conclude that the null finding is valid. It is vital to note, however, that several measurement issues handicapped this study. Subsequently, Winer and colleagues prospectively studied 82 young women for a mean of 33 months, again with HPV testing at 4-month intervals but measuring risk factors plus condom use behaviours every 2 weeks via web-based reporting system. They found only a significant effect by contrasting those reporting <5% use to those reporting 100% use (HR = 0.4, 95% CI = 0.2–0.95).³⁴

A study conducted in the Netherlands provided some intriguing evidence suggesting that condom use may promote the regression of cervical intraepithelial neoplasia (CIN). The design was novel because it involved actual assignment to condom use conditions rather than the more traditional approach of measuring use and subsequently classifying participants into groups. Women with CIN were randomised to one of two conditions (condom use or non-use of condoms) and followed for a median of ~15 months. Among 64 women assigned to the condom use group, the cumulative regression rate was 53% compared with only 35% among 61 women in non-use group (P = 0.03). A similarly large difference was found for HPV clearance (23% in the condom use group v. only 4% in the non-use group; P = 0.02). Unfortunately, the authors did not provide data about actual condom use (or non-use). Thus it cannot be known whether a misclassification bias had a potential role in detracting from the effect size. Nonetheless, the effect was quite substantial despite this potential bias towards the null hypothesis. Logically, the protective effect results from an interruption of persistent transmission of HPV between sex partners (sometimes known as a ‘ping-pong effect’). Although replications studies are needed, the findings are important because they suggest that public health efforts regarding cervical cancer should be triangulated: vaccination, Pap testing (colposcopy) and condom use.

Wald and colleagues used data from a vaccine trial to test the hypothesis that condoms would be protective against herpes simplex virus Type 2 (HSV-2) acquisition.³⁵ The trial enrolled 528 presumed monogamous heterosexual couples who were serodiscordant for HSV-2. Over an 18-month study period, 5 of 261 men acquired HSV-2 and 26 of 267 women acquired the virus. Those testing HSV-2 negative at baseline maintained a daily log of sexual activity that included condom use. Condom use records were cumulated from these logs and classified into four categories: 100% use, 51% to 99%, 26% to 50%, and 25% or less often. Condom use data for 502 participants were available and this represented 22 incident cases of HSV-2. Analyses were conducted using the dichotomy of 25% or less condom use versus the remaining three classifications. In adjusted analyses, condom use (at 26% of the time or greater) was protective against HSV-2 for women (HR = 0.085; 95% CI = 0.01–0.067) but not for men (HR = 2.02; 95% CI = 0.32–12.5). Of note, only 2 of the 22 incident cases were among men and thus the available statistical power for this analysis was extremely low, as is evident by the exceptionally large confidence interval that spanned a range of more than 13.0. The analyses for women were most likely underpowered as well, given the wide confidence interval; nonetheless, the HR (unaffected by power) was significant.

Two observations from the study by Wald and colleagues are warranted. First, the conversion of condom use data into percentages masks the level of STI risk. A 50% condom user, for example, may have had sex twice thereby meaning he or she had only one exposure to the HSV-2 positive partner. Alternatively, another 50% condom user may have had sex 30 times, meaning he or she had 15 exposures. Classifying these two very different people in the same category is a source of bias. Second, the findings for men and for women underestimate the protective value of condoms against HSV-2 acquisition. This is true because people who used condoms as infrequently as 26% of the time were counted as ‘condom users’, again creating a misclassification bias that favours the null hypothesis.

Although multiple cross-sectional studies have evaluated condom effectiveness against syphilis,³⁶ only two prospective studies were found. A prospective study of female sex workers reported in 1998 found a strong and significant protective effect for ‘always or almost always’ using condoms against syphilis acquisition (adjusted odds ratio (AOR) = 0.39; 95% CI = 0.23–0.64).³⁷ This finding is interesting in that ‘always use’ and ‘almost always use’ were collapsed into a single category, thereby weakening the contrast between users and non-users. Despite this handicap, the protective AOR was nonetheless quite strong. The other prospective study assessed 807 post partum women and compared those using condoms consistently to those never using condoms over a 6-month recall period.³⁸ Although the association was not significant among HIV-positive women, it is important to note that among HIV-negative women, no incident cases of syphilis were observed among consistent condom users versus nine cases being observed among those never using condoms. This study illustrates a typical conundrum in condom effectiveness studies – what should be done with data representing condom use ranging from 1% to 99% of the acts? Evidence suggests that the decision to classify these ‘occasional’ condom users with the ‘always users’ or with those never using condoms (e.g. lack of use) is vital and may require considerable analytic attention as described in extensive commentary on the subject.³⁹

Although numerous studies have reported associations between condom use and protection against chlamydia infection,⁵ the majority of the study designs were cross-sectional. Five prospective studies are noteworthy, with the study reported by Zenilman and colleagues in 1995 being one of these.²⁰ Their analysis of 322 male STI clinic patients compared those who had used condoms to the remainder and found a non-significant association with subsequent chlamydia infection. Their analysis of 275 female STI clinic patients also yielded a non-significant association. Similarly, for females, a study of 216 teenagers found a non-significant association between condom use (always used v. the remainder) and chlamydia,⁴⁰ as well as study of 3203 teenage females that yielded a non-significant association for chlamydia incidence.⁴¹ However, two studies of females did show a protective effect of condom use against incident chlamydia. Despite a very small sample size of 229 women, one study found a significant protective effect when comparing those always using condoms to those mainly using condoms (univariate odds ratio = 0.40; 95% CI = 0.30–0.60) and when comparing those always using condoms to those seldom using condoms (univariate odds ratio = 0.50; 95% CI = 0.30–0.80).⁴²

Several prospective studies of condom effectiveness against gonorrhoea have been published;^{20,36,43–46} however, most of these are substantially underpowered, making the standard use of 95% CI a severe handicap to testing the null hypothesis fairly. Nonetheless, two of these six studies observed a significant protective effect. For example, a study of 948 female sex workers compared consistent users to all other users and found a significant protective effect (AOR = 0.60; 95% CI = 0.40–0.80).⁴¹ Another study of female sex workers (n = 508) compared those using condoms ‘always and almost always’ to the remainder and found a significant protective effect (AOR = 0.63; 95% CI = 0.41–0.97).⁴²

In a study of 919 women, those using condoms consistently and correctly (no breakage or slippage) were compared with those not using condoms at all, and this yielded a protective effect against incident chlamydia and gonorrhoea after controlling for confounding variables (AOR = 0.49; 95% CI = 0.26–0.92).⁶ The analysis, however, was quite distinct from a traditional cohort study. The analysis used women as their own controls, meaning that intervals of observation (intervals were 1 month) that ended with chlamydia or gonorrhoea acquisition were contrasted with intervals not ending in disease acquisition using a within-subjects model including 183 women. Of note, the AOR was non-significant when contrasting women who reported consistent use with either breakage, slippage or both events against women reporting no use of condoms (AOR = 0.72; 95% CI = 0.31–1.64). Two points warrant attention here. First, the within-subjects design reduces error variance because it controls for differences between condom users and condom non-users that are probably impossible to assess otherwise. Specifically, the key difference is the likelihood that those using condoms do so because they have valid reason to suspect their sex partners may have an STI. Second, failure to control for condom breakage and slippage may produce the analytical equivalent of condom non-use. This second point is better illustrated by a prospective study of 132 clinic attendees tested for chlamydia and gonorrhoea at the beginning and end of a 3-month interval.⁴³ Although the small sample size precludes meaningful CI, the descriptive information from this study was intriguing. Among 18 people reporting consistent use of condoms in the recall period and a lack of problems (slippage, breakage, leaking, early removal, late application), none acquired either chlamydia or gonorrhoea. Among 15 people also reporting consistent use but having at least one problem with correct use, 13.3% acquired infections. Among 62 reporting inconsistent use, 17.7% acquired infection and among 35 reporting complete non-use, 22.9% acquired infections. The study provided evidence strongly suggesting that evaluations of condom effectiveness that fail to account for problems with correct use may grossly underestimate the protective effects.

A study of 442 adolescent females⁴⁴ provides further evidence supporting the imperative to account for problems with correct condom use. Despite being handicapped by a cross-sectional study design, the study found a strong protective effect for consistent and correct condom use against chlamydia (odds ratio (OR) = 0.40; 95% CI = 0.20–0.80) and gonorrhoea (OR = 0.10; 95% CI = 0.0–0.50). The extremely high prevalence of condom use problems (71%) among these young females was clearly a factor that could not be ignored analytically. Of interest, 35% reported consistent condom use over a 3-month recall period; however, only 16% of the sample reported consistent use that was also problem-free. Considering that 19% of the sample could have been labelled as ‘consistent users’ in an unadjusted analysis, the study nicely illustrates the previously introduced concept of misclassification bias.

To the best of our knowledge, only one study has investigated the protective effect of condoms against trichomoniasis.⁴⁵ Unfortunately, that study did not describe how condom use was assessed. The authors reported a protective OR of 0.34 (P = 0.016) for the interaction term of consistent condom use and the 4-month follow-up visit. However, the study’s definition of ‘consistent condom use’ was never articulated. It is unclear how consistent condom use was measured.

A prospective study of 355 men assessed the combined incidence of chlamydia, gonorrhoea and trichomoniasis over four testing intervals. Among men reporting they had always used condoms during the 6-month study, 3.7% tested positive for at least one incident infection compared with 8.6% among those indicating they had not always used condoms (AOR = 0.40; 95% CI = 0.20–0.80).⁴⁶ Using other estimates of condom use, the study also found significant protective effects against incident infections with one or more of the three STI. Of interest, in a model that estimated condom use based on extrapolation from a 7-day recall period, the AOR was non-significant when contrasting the middle classification of condom use (1% through 99%) with the referent category of no condom use at all. This is an important observation because it lends some credence to the practice of using a simple ‘always’ versus ‘not always’ dichotomy.

Based on the existence of so many monogamous serodiscordant couples now that the HIV epidemic is 30 years old, studies of condom effectiveness against this infection have the great advantage of being conducted in a near natural setting. Typical studies^12,13 are conducted with known HIV-positive patients who are connected to care and thus are available for participation in research studies. The HIV-negative sex partners of these patients can then be prospectively observed for their condom use behaviours and periodically tested to detect seroconversion. A recent meta-analysis of 14 studies meeting stringent requirements for rigor estimated condom effectiveness at approximately an 80% protective value.¹⁴ Despite the advantages of studying known serodiscordant couples for a virus that does not naturally clear (like HPV), these studies are nonetheless handicapped by the same misclassification biases already described in this article, including failure to account for the correct use of condoms.

Throughout this brief review of condom effectiveness in prospective studies, it is quite apparent that multiple issues confound the study designs in a way that dilutes the obtained effect sizes. In essence, effect size can be conceptually viewed as variance attributed to the actual correlation between condom use and incident STI divided by the sum of all variance obtained. The challenge is to reduce the amount of ‘other variance’ that is attributable to confounding sources. This reduction of error variance, in turn, creates a smaller denominator, and therefore yields a larger and more precise effect size. Fig. 2 displays examples of reductions in error variance for condom effectiveness studies.

Fig. 2.  Reducing error variance to uncover true effect size in condom effectiveness studies.

As shown in Fig. 2, one source of error variance involves known exposure to infected partners. The underlying principle here is that people using condoms with uninfected partners should be excluded from analyses estimating condom effectiveness. This is simply because grouping condom users who have exposure to infected partners with condom users who do not have this same exposure dilutes the value of condom use, given that condoms cannot possibly prevent a disease that is impossible to acquire. This observation was reported by Warner and colleagues, who found that the prevalence of chlamydia and gonorrhoea was significantly associated with consistent condom use for 429 clinic attendees with known exposure (AOR = 0.42; 95% CI = 0.18–0.99) but not among 4314 attendees with unknown exposure (AOR = 0.82; 95% CI = 0.66–1.01).⁷ Of note, the added precision derived from ‘known exposure’ may be the reason why studies of female sex workers previously noted in this article tend to achieve larger effect sizes. A study of 1455 men and women also illustrates this point quite well.⁴⁷ In that study, consistent condom use (v. less than consistent) did not have a significant association with prevalent chlamydia infection among 1303 people with unknown exposure to a chlamydia-infected sex partner (AOR = 1.34; 95% CI = 0.85–2.11). Despite a large decline in statistical power, when analysing 152 people documented to have known exposure to a chlamydia-infected sex partner, the association was significant and quite strong (AOR = 0.10; 95% CI = 0.01–0.83). Another source of error variance shown in Fig. 2 involves incomplete use of condoms during sex (as opposed to inconsistent use over a recall period of days, weeks or months). Incomplete use during sex occurs when condoms are put on after sex begins or removed before sex ends. In the study reported by Warner and colleagues⁶ (described under the subheading ‘Chlamydia and gonorrhoea’), it is noteworthy that their correction for condom breakage and slippage yielded a significant protective effect for consistent condom use in contrast to their measure of consistent use that was not adjusted for these two forms of condom failure. Given the relatively widespread practice of incomplete condom use,^29,32,48 it is intriguing to consider how much stronger the obtained protective effect would have been, given the use of a measure also that adjusts for this form of error variance.

As previously established in this article, Fig. 1 also shows that error variance from lack of temporal ordering and from poor measurement of condom use are common sources of error variance. Of great interest, we were unable to locate a published study (through to January 2011) that investigated condom effectiveness while controlling for all of the sources of error variance portrayed in Fig. 1. Studies that eloquently control one source of error variance such as known exposure to infected partners lack control for other sources such as temporal ordering and measurement issues, creating misclassification bias and problems with correct use. Fortunately, the National Institute of Allergies and Infectious Disease (USA) has funded a large multisite prospective evaluation of condom effectiveness against biologically assessed non-viral STI.⁴⁹ Results from that study are anticipated in the year 2013.

The science behind studies of condom effectiveness has lacked precision in a direction favouring the null hypothesis (i.e. that condoms do not protect against STI). Despite this lack of precision, prospective studies generally support condom effectiveness against even the most infectious pathogens (chlamydia and gonorrhoea) as well as providing some protection against non-HIV viral infections (in as much as these can be averted by covering the penile head and shaft). Because so many sources of error variance have not been controlled in studies of condom effectiveness, observed protective effects can be viewed as substantial underestimates of true effects. Studies specifically designed to evaluate condom effectiveness are needed, and these studies should seek to reduce error variance to zero. In addition, future studies should address the question of whether a true dose–response relationship exists between condom use and STI acquisition; however, doing so is somewhat academic because an affirmative answer would imply that intervention objectives advocate less than consistent use in exchange for partial protection (an intervention goal that will be confusing to the public). For now, the critical question is quite simple: what level of protective effect warrants the full support of public health efforts to bring and keep condoms to the forefront of STI prevention efforts? Evidence presented in this article suggests that the necessary threshold has probably been crossed if one considers the multiple sources of error variance inherent in studies not specifically funded to investigate condom effectiveness. Until more definitive findings are available, it appears wise to proceed under the assumption that condoms offer clinically significant protection against a host of sexually acquired infections.

None declared.

Supported by a grant from NIAID to Dr Crosby (RA1068119A).