Evaluation of between-assay consistency among laboratory testing methods for neurosyphilis: a systematic review

Introduction

Neurosyphilis is an infectious disease caused by Treponema pallidum, a spirochaete bacterium that invades the central nervous system. Neurosyphilis can occur at any stage of syphilis, and it can cause serious neurological damage or even death due to delayed diagnosis or belated treatment.¹ The clearly recommended laboratory diagnostic indicators that include the venereal disease research laboratory (VDRL) test, fluorescent treponemal antibody absorption (FTA-ABS) test, white blood cell count and protein quantification in cerebrospinal fluid (CSF) are in accordance with the most recent version of China’s ‘Sexually Transmitted Diseases Clinical Diagnosis, Treatment and Prevention Guidelines’ and the US Centers for Disease Control and Prevention’s guidelines for sexually transmitted disease diagnosis and treatment.^2,3 Of these tests, CSF-VDRL is recommended as one of the laboratory diagnostic indicators for neurosyphilis due to its high specificity.⁴ However, the laboratory conditions required to operate the CSF-VDRL test are of relatively high complexity, as the reagent needs to be prepared and must be used within 2 h, and a light microscope is required for detection, making it difficult for many laboratories to execute the test effectively. Therefore, the reactive plasma regain (RPR) test and the toluidine red serum unheated test (TRUST) may be reasonable alternative choices. RPR and TRUST are commonly used to detect non-T. pallidum-specific antibodies in peripheral blood, and they carry relatively high specificity and sensitivity.⁵ However, it is unknown whether the specificity and sensitivity remain high when detecting non-T. pallidum-specific antibodies in CSF or if they constitute appropriate alternatives to VDRL. Therefore, a number of controlled clinical studies have been conducted that focus on the differential sensitivity and detection rates of neurosyphilis between VDRL and RPR/TRUST in CSF; but these have produced inconsistent conclusions, and controversies persist that require resolution. We therefore searched studies over the past 30 years in which comparisons were made between VDRL and RPR or with TRUST in CSF, and we exploited kappa consistency analysis and McNemar’s chi-squared test to determine the consistency of the conclusions derived from various studies. We posit that CSF-RPR and CSF-TRUST can be systematically evaluated as to their feasibility as alternative tests to CSF-VDRL.

Methods

Results

Included articles

In our search, we identified 198 articles after removing 55 duplications. We then made a further elimination under the following orders: 25 were non-neurosyphilis studies, 123 were case reports, 20 conducted different tests among different groups of patients, 15 did not show sensitivity and specificity results, and nine did not provide diagnostic criteria. As a result, six met the inclusion criteria of the present study (Fig. 1).^6–11 These studies encompassed 429 patients who were classified as exhibiting definite neurosyphilis, 49 as having probable neurosyphilis and 1668 as not having neurosyphilis (Tables 1 and 2).

Fig. 1.

Flowchart of studies reviewed for inclusion.

Table 1.Comparisons of characteristics between CSF-RPR and CSF-VDRL in included studies.

Study	Criteria for definite neurosyphilis	Criteria for probable neurosyphilis	Cases of definite neurosyphilis	Cases of probable neurosyphilis	Cases without neurosyphilis	Cases with HIV co-Infection	Sensitivity by VDRL (%)	Specificity by VDRL (%)	Sensitivity by RPR (%)	Specificity by RPR (%)
Versiani et al. 6	Clinical suspicion and CSF-VDRL+	Clinical suspicion	21	49	50	U	100	100	100	100
Zhu et al. 7	(1) CSF-TPPA+ and clinical suspicion (2) CSF-TPPA+ and WBC ≥10/μL	NA	210	NA	922	N	81.4	90.3	76.2	93.4
Castro et al. 8	(1) RPR ≥1:8, MHA-TP ≥1:80 and CSF-FTA-ABS+ (2) CSF WBC ≥10/μL or CSF protein ≥45 mg/dL	NA	24	NA	290	Y	99	70.8	99.3	75
Marra et al. 9	CSF-FTA-ABS+ and CSF WBC >20/μL	NA	72	NA	77	Y	71.8/66.7A	98.3/80.2	56.4/51.5	100/89.7
Lin et al. 10	CSF-VDRL+	NA	61	NA	295	U	100	100	93.4	99.3

CSF, cerebrospinal fluid; RPR, rapid plasma reagin; VDRL, venereal disease research laboratory test; TPPA, Treponema pallidum particle agglutination; WBC, white blood cells; MHA-TP, microhaemagglutination assay for Treponema pallidum; U, unknown; N, no; NA, not applicable; Y, yes.

^ALaboratory-defined neurosyphilis/symptomatic neurosyphilis.

Table 2.Comparison of characteristics between CSF-TRUST and CSF-VDRL in included studies.

Study	Criteria for definite neurosyphilis	Criteria for probable neurosyphilis	Cases of definite neurosyphilis	Cases of probable neurosyphilis	Cases without neurosyphilis	Cases with HIV co-infection	Sensitivity by VDRL (%)	Specificity by VDRL (%)	Sensitivity by TRUST (%)	Specificity by TRUST (%)
Zhu et al. 7	(1) CSF-TPPA+ and clinical suspicion (2) CSF-TPPA+ and WBC ≥10/μL	NA	210	NA	922	N	81.4	90.3	76.2	93.1
Jiang et al. 11	(1) CSF-VDRL+ (2) CSF WBC >5/μL and CSF-TPPA+	NA	41	NA	34	N	93.1	100	94.7	100

CSF, cerebrospinal fluid; TRUST, toluidine red unheated serum test; TPPA, Treponema pallidum particle agglutination; VDRL, venereal disease research laboratory test; WBC, white blood cells; N, no; NA, not applicable.

Discussion

The CSF-VDRL has a high specificity and is currently considered the definitive test for the diagnosis of neurosyphilis globally. However, the antigen used in VDRL is combined with proportional cardiolipin, lecithin and cholesterol, and without any observable trace flags. Thus, the CSF-VDRL relies on a light microscope and skilled technicians, which made it difficult to popularise. In this review, we comprehensively analysed the consistency between CSF-VDRL and CSF-RPR, or CSF-TRUST, after collecting the data from previous studies. Our results indicated that CSF-RPR and CSF-TRUST were highly consistent with CSF-VDRL, showing that they may constitute substitute methods for the laboratory diagnosis of neurosyphilis.

Both RPR and VDRL are non-treponemal tests with similar experimental principles. RPR is posited to be an appropriate test for serum and plasma, as it uses charcoal particles as the stabiliser, and RPR can also be used today to detect antibodies against cardiolipin antigen (among other bodily fluids, such as CSF) due to technological advancements.⁴ Compared with VDRL, the RPR test can be conducted in laboratories and in less-developed areas with relatively lower requirements for experimental sophistication. There is still no definite conclusion whether CSF-RPR can substitute for CSF-VDRL, although several studies have uncovered a fairly similar performance between them.^6–8 Our study showed high consistency between CSF-RPR and CSF-VDRL in neurosyphilis diagnosis, supporting the view that CSF-RPR can provide an alternative choice to VDRL. According to this conclusion, the diagnostic efficiency of neurosyphilis is expected to improve significantly, particularly in areas where VDRL cannot be executed.

The TRUST method is a modified version of VDRL, and as the toluidine red particle-labelled antigen is stable at room temperature, the TRUST reagents have been successfully applied in China and in many other countries.^7,11 Similar to the case for RPR, TRUST can also detect antibodies against the cardiolipin antigen in CSF due to its technological improvements. In one study, the authors ascertained a 97.3% consistency between CSF-TRUST and CSF-VDRL after using McNemar’s chi-squared test (Gu et al.);¹² others found a kappa value of 0.709 between CSF-VDRL and other nontreponemal tests in CSF among symptomatic neurosyphilis patients, and a kappa value of 0.692 between CSF-VDRL and CSF-RPR or CSF-TRUST among asymptomatic patients (Zhu et al.).⁷ These results are in good agreement with our study, suggesting that CSF-TRUST and CSF-VDRL are highly consonant. Thus, the CSF-TRUST can also be employed in many geographic areas as an alternative test in neurosyphilis diagnosis. It is worth noting that the CSF-VDRL instructions call for diluting the VDRL antigen and using less antigen than in the serum version to account for lower antibody concentrations in CSF compared with serum, to avoid the prozone phenomenon and reduce false negative test results. Thus, if researchers can perform the RPR or TRUST on CSF using the method recommended for the CSF-VDRL, it may decrease the number of false negative results and further increase the consistency.

In our study, we also focused on other factors that might affect consistency between different types of tests. As a result, only a tiny difference was observed in kappa values (CSF-RPR vs CSF-VDRL) between HIV-positive and HIV-negative patients, suggesting that HIV infection may not influence the test comparisons. We additionally confirmed that symptoms may play an important role in the performance of different tests, as laboratorial physicochemical indices were originally different between symptomatic and asymptomatic patients – and this may then further explain our findings and remind researchers to consider this effect in future comparison studies.

We compare different CSF tests from the point of view of consistency analysis, which may lead to selection bias compared with meta-analysis. Another limitation is that the individual-level data were unavailable from these collected articles, which prevented us from further analysis. Furthermore, the final consistency analysis included a relatively small dataset due to the fact that only a portion of the analyses encompassed the complete datasets needed for calculation. Further, the literature we collected may not have provided enough comprehensive information on patients, such as symptoms and HIV status, which may have influenced testing performance to a degree. Finally, the studies included in the review used different definitions for neurosyphilis, and may affect test performance among different types of neurosyphilis, such as symptomatic and asymptomatic neurosyphilis.

To summarise, we have herein suggested that CSF-RPR or CSF-TRUST were consistent with CSF-VDRL, which would make it possible to substitute VDRL as the neurosyphilis diagnostic method of choice. Popularisation of CSF-RPR and CSF-TRUST may, therefore, facilitate the improvement in diagnostic ability regarding neurosyphilis, especially in those geographic areas where CSF-VDRL is hard to implement. It is worth noting that CSF-VDRL is still acknowledged as an imperfect test for neurosyphilis diagnosis, although it is the most recommended test nowadays. Thus, the best solution in the future is to evaluate novel tests that may possess superior performance and provide less difficulty in producing a diagnosis of neurosyphilis.

Supplementary material

Supplementary material is available online.

Data availability

Data and code are available upon reasonable request.

Conflicts of interest

The authors declare that they have no conflicts of interest.

Declaration of funding

This research was provided by funding from National Natural Science Foundation of China (81772209), CAMS Innovation Fund for Medical Sciences (CIFMS-2021-I2M-1-001), and Wuxi Science and technology development funds (N20192021).

Author contributions

X Zhang contributed to the data curation, formal analysis, software and original draft writing. FZ Du contributed to the data curation and formal analysis. QQ Wang and RL Zhang contributed to conceptualisation, and manuscript review and editing.