Table of Content

Archive

Spring 2015, Vol. 23 No. 1

Hong Kong J. Dermatol. Venereol. (2015) 23, 5-12


Original Article

A study of point-of-care test to diagnose syphilis in Hong Kong

定點照護測試於香港診斷梅毒的研究

CC Koh許招財, CK Ho何正綱, KM Ho何景文

Abstract

Background: In Hong Kong, a 'point-of-care' test (POC) syphilis test is used by some outreach teams of non-government organisations (NGO) for screening syphilis. In Hong Kong, in the public sector, sexually transmitted infections are seen at the outpatient clinics under the Social Hygiene Service. Objective: This study aimed to determine the performance of POC syphilis test among high risk people in Hong Kong. Methods: Patients were recruited from two Social Hygiene Clinics for the POC syphilis test. From October 2012 to February 2013, the following groups were recruited: (1) new attendees; (2) patients with newly diagnosed syphilis; (3) patients with a history of treated syphilis. Two different POC syphilis tests [Alere DetermineTM Syphilis TP ("Alere Ltd", USA), Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea)] were performed. Results: A total of 356 patients were recruited. Overall results for Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test was as follows: sensitivity 56.1%, specificity 100%. Overall results for Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) test was as follows: sensitivity 57.4%, specificity 100%. Conclusion: Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests have high specificity in this study but high sensitivity in previous studies cannot be reproduced. Validation of POC syphilis test may be required before its use as a standard screening test for syphilis.

背景:在香港,有一些非政府組織外展團隊會使用梅毒定點照護測試來篩查梅毒,而在公營醫療系統,性病診療多在社會衞生科診所內進行。目標:這項研究的目的是要測試梅毒定點照護測試在香港高風險人群中的準確性。方法:病人是從兩個社會衞生科診所選出,從2012年10月到 2013年2月,我們招募新到診病人、新患梅毒病人、舊有已確診梅毒病人,進行兩種不同定點梅毒照護測試[Alere DetermineTM Syphilis TP ("Alere Ltd", USA) 及Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea)]。結果:此研究招募了356個病人,Alere DetermineTM Syphilis TP ("Alere Ltd", USA) 梅毒定點照護測試結果如下:靈敏度56.1%,特異度100%。Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) 梅毒定點照護測試結果如下:靈敏度57.4%,特異度100%。結論:在這項研究中,Alere DetermineTM Syphilis TP ("Alere Ltd", USA)和 Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) 梅毒定點照護測試特異度高但靈敏度低於以往的研究。作為標準的篩選梅毒測試前,梅毒定點照護測試的有效性值得再作研究。

Keywords: Syphilis, point-of-care syphilis test, Social Hygiene Clinic, Hong Kong

關鍵詞: 梅毒、梅毒定點照護測試、社會衞生科、香港

Introduction

In recent years, a resurgence of syphilis has been observed in various parts of the world. In order to better control the transmission of syphilis, strategies such as "same day testing and treatment/referral" have been pioneered in outreach projects targeting high risk groups such as sex workers. "Point-of-care" (POC) testing of syphilis identifies positive cases in less than half an hour with same day treatment or referral to the proper health care facility. It enhances prevention via a single encounter with the target group and enables case finding, prompt treatment and hence shortens the duration of infectivity.

Objectives

About twenty POC syphilis diagnostic tests are globally available in screening syphilis. The sensitivity and specificity of various POC syphilis tests using archived serum range from 84.5%-97.7%, and 92.8%-98% respectively in a previous study (Table 1).1 Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests are widely used in Hong Kong but there are no local studies on the use of POC syphilis test in Hong Kong. This study aimed to: (a) study the utility of POC test in screening syphilis among high risk people in Hong Kong, (b) study the diagnostic performance of two different brands of POC syphilis tests commonly used in Hong Kong.

Table 1 Sensitivity and specificity results for different POC syphilis tests (using sera) in a previous study1
POC syphilis test Sensitivity Specificity
Abbott Determine Syphilis TP ("Abott Laboratories", UK) 97.2% 94.1%
Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) 95% 94.9%
Fujirebio Espline?TP (Fujirebio Inc, Japan) 97.7% 93.4%
Diesse Syphilis Fast (Diesse Diagnostica, Italy) 86% 92.8%
Omega Visitect Syphilis (Omega Diagnostics Ltd, UK) 85% 98%
Qualpro Syphicheck-WB (Qualpro Diagnostics, India) 84.5% 97.7%

Methods

Setting of study
POC syphilis tests were performed from October 2012 to January 2013 in two Social Hygiene Clinics (Yau Ma Tei and Yung Fung Shee Social Hygiene Clinics). Patients above 18 years old with newly diagnosed syphilis, treated syphilis and new patients attending these two social hygiene clinics were invited to join the study. Patients with syphilis were assigned to the "Syphilis group" and those without syphilis were assigned to the "Control group". Two POC syphilis tests were done for all patients: Alere DetermineTM Syphilis TP ("Alere Ltd", USA) Test and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) Test. The diagnosis of syphilis is made according to a standardised case definition used by Social Hygiene Service (SHS): positive dark-ground examination for spirochaete in genital ulcers or positive syphilis serology test (two positive specific Treponemal pallidum tests) with relevant clinical history and physical signs.

Procedures of POC syphilis test
Blood sample by venipuncture/finger stick (if no concomitant blood taking during the same visit) were collected and transferred to the sample well of the test kits by ethylenediaminetetraacetic acid (EDTA) capillary tube. Assay diluent was added and the test results were read at 15 minutes for both POC syphilis tests. Red bars may appear in both control windows (labelled "Control") and patient window (labelled "Patient") of the strip. A red bar in the control window validated the test and any visible red colour in the patient window was interpreted as positive. Absence of a red bar in patient window of the strip was interpreted as negative. If there was no red bar in the control window of the strip, the result was invalid/indeterminate.2,3 Results were read by a trained nurse and counterchecked by the medical officer if necessary.

Statistical analysis
Differences in basic demographic characteristics, between the "syphilis" and "control" groups were compared by using Chi-square test and Mann-Whitney U test. Any p value <0.05 was considered statistically significant. The sensitivities, specificities, positive predictive values and negative predictive values for the presence of syphilis detected by POC syphilis tests (Alere DetermineTM Syphilis TP and Standard Diagnostics Bioline Syphilis 3.0) were reported with reference to diagnostic criteria adopted in SHS. Calculation of the sample size showed that the required number of subjects should be 135 in the "syphilis group" and 135 in the "control group" for a statistical power of 80% in rejecting the null hypothesis that the sensitivity and specificity of POC syphilis test were both 90% of those of the reference test.

Results

A total of 356 patients underwent the POC syphilis test. Results are shown in Tables 2-6.

Table 2 Table of patients with "POC syphilis test" performed
  Categoryof patient Number of patients
New syphilis patients Syphilis group 41
Treated syphilis patients Syphilis group 121
New patients attending clinics without syphilis Control group 194
Total number of patients   356

Table 3 Age and gender distribution of all 356 patients in "POC syphilis test" study
Characteristics   Syphilis (n=162) Control (n=194) Total (n=356) P-value
Average age (years)   52 (35-65) 35 (25-48) 41 (29-57) 0.000
Sex Female 18 (11.11%) 14 (7.22%) 32 (8.99%) 0.201
  Male 144 (88.89%) 180 (92.78%) 324 (91.01%)  

Among 162 syphilis patients in the "POC syphilis test" study, 109 (67.3%) patients were diagnosed as late latent syphilis (LLS) and 13(8%) patients were diagnosed as early latent syphilis (ELS). Nineteen (11.7%) patients were diagnosed as secondary syphilis (SS) and 20 (12.3%) patients were diagnosed as primary syphilis (PS). One (0.006%) patient had neurosyphilis. The overall sensitivity and specificity of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test were 56.1% and 100% respectively. The sensitivity and specificity of the Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test for newly diagnosed syphilis without previous treatment was 64.1% and 100% respectively. The sensitivity and specificity of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test for patients with treated syphilis were 53.4% and 100% respectively. In patients with newly diagnosed syphilis, the sensitivities of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test for PS, ELS, LLS were 62.5%, 66.67 %, 58.61% respectively (Table 4). The overall sensitivity and specificity of Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) test were 57.4% and 100% respectively. The sensitivity and specificity of Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) test for newly diagnosed syphilis without treatment was 63.4% and 100% respectively. The sensitivity and specificity of Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) test for patients with treated syphilis was 55.4% and 100% respectively. In patients with newly diagnosed syphilis, the sensitivities of Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc.,Korea) test for PS, ELS and LLS were 62.5%, 66.67% and 62.07% respectively (Table 5).

Table 4 Performance of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test in this study
  Sensitivity Specificity
Overall 56.05% (CI 48.23-63.58%) 100% (CI 98.06-100%)
New/untreated 64.10% (CI 48.42-78.26%) 100% (CI 98.06-100%)
Old/treated 53.38% (CI 44.42-62.14%) 100% (CI 98.06-100%)
Stage of syphilis (Newly diagnosed)  
Primary syphilis 62.5% (CI 30.57-86.32%) 100% (CI 98.06-100%)
Secondary syphilis - -
ELS 66.67% (CI 20.77-93.85%) 100% (CI 98.06-100%)
LLS 58.61% (CI 40.74-74.49%) 100% (CI 98.06-100%)

Table 5 Performance of Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) test in this study
  Sensitivity Specificity
Overall 57.41% (CI 49.71-64.76%) 100% (CI 98.06-100%)
New/untreated 63.41% (CI 48.12-76.41%) 100% (CI 98.06-100%)
Old/treated 55.37% (CI 46.49-63.93%) 100% (CI 98.06-100%)
Stage of syphilis (Newly diagnosed)  
Primary syphilis 62.5% (CI 30.57-86.32%) 100% (CI 98.06-100%)
Secondary syphilis - -
ELS 66.67% (CI 20.77-93.85%) 100% (CI 98.06-100%)
LLS 62.07% (CI 44-77.31%) 100% (CI 98.06-100%)

Table 6 Summary of sensitivity of different POC syphilis tests in previous studies
Year Test Sensitivity (whole blood) Sensitivity (serum)
2003 Abbott Determine Syphilis TP ("Abott Laboratories", UK)1 - 97.2%
2004 Abbott Determine Syphilis TP ("Abott Laboratories", UK)7 59.6-88.5% 88.5-100%
2003 Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea)1 - 95%
2004 Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea)7 85.7-100% 90.2-100%
2003 Omega Visitect Syphilis (Omega Diagnostics Ltd, UK)1 - 85%
2008 Omega Visitect Syphilis (Omega Diagnostics Ltd, UK)8 57% -
2011 Omega Visitect Syphilis (Omega Diagnostics Ltd, UK)9 62.5% -
2003 Qualpro Syphicheck-WB (Qualpro Diagnostics, India)1 - 84.5%
2009 Qualpro Syphicheck-WB (Qualpro Diagnostics, India)10 70.8% -

Discussion

Principles of point-of-care syphilis test in screening syphilis
Both Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests are immunochromatographic assays for the qualitative detection of treponemal antibodies.4 Antibodies are transported by capillary flow to test line of strips with the help of a buffer. An EDTA capillary tube was used to transfer blood specimen to prevent coagulation. Antibodies in specimens bound to specific surface antigens of Treponemal pallidum in the test line and appear as a visible line due to dye bound to anti-immunoglobulin.1 Alere DetermineTM Syphilis TP ("Alere Ltd", USA) test contains a membrane strip pre-coated with recombinant Treponema pallidum antigens 47 kDa on the test band region and involeves the detection of antibodies of all isotypes (IgG, IgA and IgM) against Treponema pallidum.1 Assay diluent is a phosphate buffer.5 Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) contains a membrane strip pre-coated with recombinant Treponema pallidum antigens (15 kDa, 17 kDa) on the test band region and involves the detection of antibodies of all isotypes ( IgG, IgA and IgM) against Treponema pallidum. The assay diluent is Tris-HCl buffer.5 The implication of point-of-care syphilis test is that it allows health workers to provide immediate and confidential test results,6 and to provide a strategy of "Same-day Testing and Treatment" (STAT) for syphilis.

Performance of Determine and Bioline point-of-care syphilis test
A total of 356 patients underwent the POC syphilis test. The overall performance (sensitivity, specificity) of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests were similar. The specificity for both Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests were 100%. However, the sensitivity of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) (64.1%) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) (63.41%) syphilis tests for newly diagnosed syphilis patients were low in this study.

Other studies on the performance of point-of-care syphilis test
In contrast to a previous study of six POC syphilis tests done (with serum) in 2003 (sensitivity 85-98%, specificity 93-98% for six different POC syphilis tests) (Table 1), the sensitivity of the current POC syphilis study is lower than that expected in this study.1 In fact, different studies showed a discrepancy in the test performance (Table 6) and the sensitivity of POC syphilis tests (using whole blood) may actually be lower than expected. A multi-centre evaluation of four POC syphilis tests (Abbott Determine, Omega Visitect Syphilis, Qualpro Syphicheck-WB, Standard Diagnostics Bioline 3.0) was done with 2335 patients at four sites (Brazil, China, Haiti and Tanzania) in 2004.7 Tests were performed using whole blood in the clinic and serum in the laboratory. The specificity of each POC syphilis test was >95% at each site. Sensitivities varied from 64-100% and in most cases these were lower when whole blood instead of serum was used. With whole blood, the sensitivity and specificity of Abbott Determine Syphilis TP ("Abbott Ltd", Japan) test were 59.6-88.5% and 97.9-99.4% respectively. With serum, the sensitivity and specificity of Abbott Determine Syphilis TP ("Abbott Ltd", Japan) test were 88.5-100% and 95.7-98.9% respectively. With whole blood, the sensitivity and specificity of Standard Diagnostics Bioline syphilis 3.0 (Standard Diagnostics, Korea) were 85.7-100% and 98.1-99.4% respectively. With serum, the sensitivity and specificity of Standard Diagnostics Bioline syphilis 3.0 (Standard Diagnostics, Korea) were 90.2-100% and 95.5-99.4% respectively.7 The low sensitivity of whole blood POC syphilis test in this study is not totally inconsistent with the finding of a lower sensitivity of whole blood POC syphilis test in previous studies.7

In a Visitect POC syphilis test study in 2008, 510 patients attending sexually transmitted disease clinics in Manaus, Brazil, were screened using finger prick blood samples tested with Visitect POC syphilis test. The sensitivity, specificity, positive and negative predictive values of Visitect POC syphilis test were 57% (95% CI 45.8 to 66.7%), 99% (95% CI 97.0 to 99.6%), 91% (95% CI 80.0 to 96.7%) and 91% (95% CI 88.0 to 93.5%) respectively.8 Another Visitect POC syphilis test study was done in Brazil in 2011 for 712 pregnant women using whole blood obtained by finger prick. The sensitivity, specificity, positive and negative predictive values were 62.5% (95% CI: 38.6-81.5%), 99.1% (95% CI: 98.1-99.6%), 62.5% (95% CI: 38.6-81.5%) and 99.1% (95% CI: 98.1-99.6%) respectively.9 The sensitivity of Visitect POC syphilis test was 85% in the 2003 study,1 and Visitect POC syphilis test was found to have a low sensitivity in field use.9

In a study of Syphicheck-WB POC syphilis test done in 2009, 1614 female sex workers attending sexually transmitted disease clinics in Bangalore, India had POC syphilis screening using finger-prick whole blood.10 The sensitivity and specificity of the Syphicheck-WB POC syphilis test were 70.8% (95% CI 62.7 to 79.0%) and 97.8% (95% CI 97.1 to 98.5%) respectively.10 The sensitivity of Syphicheck-WB POC syphilis test was 84.5% in the 2003 study.1 In fact, the Syphicheck-WB POC syphilis test using finger-prick whole blood has a relatively low sensitivity in detecting syphilis.10

Evaluation of the performance of Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) tests
In contrast to the previous study of POC syphilis test in 2003, (sensitivity 85-98%, specificity 93-98%), the POC syphilis test in this study did not reproduce the high sensitivity results as quoted in previous studies.1 In previous studies, the better performance of these tests was mostly based on serum specimens rather than whole blood specimens used in this study. The concentration of antibody is lower in whole blood than serum. The lower antibody concentration could decrease the sensitivity of the test by diminishing the strength of the antigen-antibody interaction that is responsible for producing a positive test. Besides, the performance of these tests was harder to read than using serum, with the sensitivity generally being lower.5,11,12 In fact, there are previous studies showing that the sensitivity of POC syphilis test was actually low when testing whole blood rather than serum specimens.7 The finding of low sensitivity of POC syphilis test in this study is not totally inconsistent with the finding of previous studies.7

Other possibilities for the low sensitivity of the POC syphilis tests in this study include the dilution of antibodies if excess buffer was used and coagulation of blood before application to the test strip. Excess buffer is unlikely as the quantity of buffer used was standardised (as recommended by the package insert). Besides, the problem of coagulation should be solved by the use of anticoagulant-coated capillary tubes which can be used during blood collection resulting in enhanced test sensitivity, as recommended by the package insert.10 However, EDTA anticoagulant interfering with the antibody-antigen reaction is still possible.12 Furthermore, there may be variations in the quality control in antigen processing between brands of POC syphilis test kits from different countries and periods.1

"ASSURED" criteria for POC syphilis test
The Sexually Transmitted Diseases Diagnostics Initiative in the United Nations Children's Fund/United Nations Development Programme/World Bank/World Health Organization Special Programme for Research and Training in Tropical Diseases, recently initiated a program for the evaluation of tests known by the acronym "ASSURED". This acronym ensures that tests are: "Affordable," "Sensitive", "Specific", "User-friendly", "Rapid and Robust", "Equipment-free", "Deliverable" to the developing countries.13,14 A POC syphilis test with a good overall performance (sensitivity, specificity, positive predictive value, negative predictive value) is essential in screening syphilis. Patients may have a false sense of security and abandon regular standard laboratory screening of syphilis if the sensitivity of POC syphilis test is low. This indicates that validation of POC syphilis test may be required before its use as a standard screening test for syphilis. Local NGOs that are performing POC syphilis test should be made aware of the potential limitations.

Limitations of this study
These results may need further exploration in view of the small sample size in the study. POC syphilis tests were performed in patients with treated syphilis and the effect of syphilis treatment on the performance of the tests was unknown. Besides, technical error in performing the tests cannot be excluded in this study. The sensitivity results of previous POC syphilis studies were not reproduced in this study. However, the results of POC syphilis tests in this study may only be applicable in this study and may not represent the actual sensitivity of POC syphilis test or reflect the whole picture.

Conclusion and way forward
Alere DetermineTM Syphilis TP ("Alere Ltd", USA) and Standard Diagnostics Bioline Syphilis 3.0 (Standard Diagnostics, Inc., Korea) syphilis tests have high specificity in this study but the high sensitivity quoted in previous studies could not be reproduced. Further exploration and validation of POC syphilis test may be required before its use as a standard screening test for syphilis. POC syphilis tests should be used with caution in field projects in Hong Kong.

References

1. Herring AJ, Ballard RC, Pope V, Adegbola RA, Changalucha J, Fitzgerald DW, et al. A multi-centre evaluation of nine rapid, point-of-care syphilis tests using archived sera. Sex Transm Infect 2006;82:7-12.

2. Package Insert. Standard Diagnostics Bioline 3.0. http://www.standardia.com.

3. Package Insert. Alere Determine Syphilis treponema pallidum. http://www.determinetest.com.

4. Castro AR, Esfandiari J, Kumar S, Ashton M, Kikkert SE, Park MM, et al. Novel point-of-care test for simultaneous detection of nontreponemal and treponemal antibodies in patients with syphilis. J Clin Microbiol 2010;48:4615-9.

5. Campos PE, Buffardi AL, Chiappe M, Buendia C, Garcia PJ, Carcamo CP, et al. Utility of the Determine Syphilis Treponema pallidum rapid test in commercial sex venues in Peru. Sex Transm Infect 2006;82:22-5.

6. Tinajeros F, Grossman D, Richmond K, Steele M, Garcia SG, Zegarra L, et al. Diagnostic accuracy of a point-of-care syphilis test when used among pregnant women in Bolivia. Sex Transm Infect 2006;82:17-21.

7. Mabey D, Peeling RW, Ballard R, Benzaken AS, Galban E, Chan GC, et al. Prospective, multi-centre clinic-based evaluation of four rapid diagnostic tests for syphilis. Sex Transm Infect 2006;5:13-6.

8. Benzaken AS, Sabido M, Galban EG, Pedroza V, Vasquez F, Araujo A, et al. Field evaluation of the performance and testing costs of a rapid point-of-care test for syphilis in a red-light district of Manaus, Brazil. Sex Transm Infect 2008;84:297-302.

9. Benzaken AS, Sabido M, Galban E, Pedroza V, Araujo AJ, Peeling RW, et al. Field performance of a rapid point-of-care diagnostic test for antenatal syphilis screening in Amazon region, Brazil. Int J of STD AIDS 2011;22:15-8.

10. Mishra S, Naik B, Venugopal B, Kudur P, Washington R, Becker M, et al. Syphilis screening among female sex workers in Bangalore, India: comparison of point-of-care testing and traditional serological approaches. Sex Transm Infect 2010;86:193-8.

11. Cox DL. Progress in development of dual rapid syphilis test technology. Sex Transm Infect 2011;87(Suppl 1): A5-A6.

12. Siedner M, Zapitz V, Ishida M, De La Roca R, Klausner JD. Performance of rapid syphilis tests in venous and finger stick whole blood specimens. Sex Transm Dis 2004;31:557-60.

13. Rosanna WP, Htun YE. Diagnostic tools for preventing and managing maternal and congenital syphilis: an overview. Bull World Health Organ 2004;82:439-46.

14. Peeling RW, Holmes KK, Mabey D, Ronald A. Rapid tests for sexually transmitted infections (STIs): the way forward. Sex Transm Infect 2006;82 Suppl 5:v1-6.