Factors associated with children’s seasonal influenza diagnosis by their guardians: a prospective observational study in Japan

Keywords: diagnosis, family practice, influenza, parents, primary health care, rapid influenza diagnostic test, self-diagnosis, vaccination.

Seasonal influenza is an acute respiratory tract infection caused by the influenza virus and a common disease among children visiting primary care clinics in winter. In the 2017–18 winter season, approximately 0.86 million influenza outpatients aged <5 years were reported in Japan¹ and 2.88 million were reported in the United States.² The influenza virus is transmitted from person to person via respiratory secretions, and children are important vectors for the spread of the disease.^3,4 The viral-shedding period can be prolonged, especially in younger and immunocompromised children.^5,6 However, influenza symptoms in children are not always classic or characteristic,⁷ and younger children are often unable to accurately describe their symptoms to their guardians. Self-diagnosis and self-prescription using the internet are expected to increase with advancements in information and communication technology.⁸ If the diagnostic ability and treatment provided by guardians, as the children’s closest observers, are accurate, this could reduce need for visits to medical institutions and consequent risk of spreading the infection.

A retrospective observational study showed that a quantitative diagnosis using a continuous indicator (in contrast to a qualitative diagnosis, which is a binary evaluation of yes/no) of seasonal influenza in children by their guardians could help establish a confirmatory diagnosis when the probability was higher than the optimal cut-off point (80%), and an exclusionary diagnosis when the probability was low (50%).⁹ However, there have been no prospective observational studies conducted to confirm the quantitative diagnostic accuracy by guardians, and no studies of factors affecting the accuracy of a quantitative influenza diagnosis by guardians. Therefore, we conducted a prospective observational study to determine the accuracy of quantitative influenza diagnosis of children by their guardians, describe factors affecting guardians’ diagnosis, and analyse the additional clinical value of guardians’ influenza diagnosis.

This prospective observational study was conducted at Rokkasho Centre for Community and Family Medicine, a community clinic in Rokkasyo-mura, Aomori, Japan. Data were collected on factors potentially affecting the accuracy of influenza diagnosis by guardians. We then verified the guardians’ diagnostic data for accuracy and assessed the additional diagnostic value of the guardians’ diagnosis.

This study received ethical approval from the Medical Ethics Committee of Hirosaki University (approval number: 2017-1100). Written informed consent was obtained from the parents or guardians of all patients included in the study. All data were anonymised at the time of data collection, and confidentiality was ensured throughout the study.

Trained nurses who conducted preliminary medical interviews invited all accompanying persons (guardians) of the children with clinically suspected seasonal influenza (eg upper respiratory tract symptoms and fever) who visited the study clinic to complete a pre-examination checklist during two consecutive influenza seasons from December 2017 to April 2019. Patients who met all the following criteria were included: aged <12 years; the guardian had completed a pre-examination checklist before the patient was assessed by a doctor; and the patient underwent a rapid influenza diagnostic test. Patients were excluded from the study if guardians did not complete all items on the pre-examination checklist and did not provide written consent.

Data were collected using pre-examination checklists (Supplementary File S1, Supplementary File S2) and electronic medical records. Items included in both data sources reflected clinical symptoms and factors important for guardians’ diagnosis according to past studies,^9–15 or symptoms and factors that were considered clinically important. The pre-examination checklist included the following: the guardian’s relationship to the patient, the child’s influenza history, the child’s influenza vaccination status, the child’s medication intake before medical visits, and the guardian’s awareness of the child’s contact with persons with influenza. Clinical symptoms, the axillary temperature at home, time of symptom onset, the severity of the current symptoms, and the guardians’ diagnosis are presented as a percentage (%). The following data were collected from the patients’ medical records: age, sex, clinical signs (axillary temperature and pulse rate at the clinic), time of the rapid influenza diagnostic test, results of the rapid influenza diagnostic test, and final diagnosis by the physician.

Quantitative diagnosis was defined as a continuous variable and qualitative diagnosis as a categorical variable index such as yes/no. For quantitative diagnosis, guardians were asked to provide an estimate of the possibility that their children were suffering from influenza, reported as a percentage, on the pre-examination checklist. We considered that by converting the guardians’ estimation into a numerical value such as percentage, we could deal with it as if it were laboratory data.

As a reference standard for influenza diagnosis, a rapid diagnostic kit (Prime Check Flu, Alfresa Co., Ltd., Japan) was used. This kit was reported to have a sensitivity of 98.4% and a specificity of 97.6%, using virus isolation and culture as the reference standard. The decision to conduct the test was made by the physician who examined the patient. The test result was determined by a clinical laboratory technician, independent of the physician who examined the patient and the nurses who conducted the pre-examination.

Factors that affect guardians’ influenza diagnostic accuracy were classified as reliable or unreliable information. Reliable information included: objective findings, such as body temperature (fever) and duration of symptoms; child’s influenza vaccination history; the guardian’s relationship to the child; and awareness of the child having contact with anyone with influenza. Unreliable information included the child’s subjective symptoms, such as disease onset (acute or gradual) and pain, because guardians could not accurately recognise their children’s symptoms. In particular, younger children cannot always communicate their symptoms to their guardians. In this study, only reliable factors were included in the analysis.

The accuracy of the guardians’ quantitative influenza diagnosis (%) was analysed by receiver-operating characteristic (ROC) curve analysis to determine the sensitivity, specificity, and likelihood ratio of their diagnosis at various cut-off points and to determine the optimal cut-off point. The area under the curve (AUC) was used to evaluate the discriminatory power of guardians’ diagnosis. The AUC results were interpreted according to the following criteria:¹⁶ AUC = 0.5, non-informative; 0.5 < AUC ≤ 0.7, less accurate; 0.7 < AUC ≤ 0.9, moderately accurate; 0.9 < AUC ≤ 1, highly accurate; AUC = 1, perfect test. A subgroup analysis was performed to evaluate the differences between groups with and without a factor using the AUC as an index. The sensitivity and specificity of each cut-off point were calculated. Multivariable logistic regression was performed to validate the additional contribution of the guardians’ influenza diagnosis.

Sample size calculations were performed according to the following assumptions: AUC, 0.7 based on a previous study;⁹ influenza positive rate, 50%; α, 0.05, and power, 0.9. Using these assumptions, the required sample size for AUC analysis was estimated to be 66. The target sample size was set at ≥100 to enable subgroup analysis. All statistical analyses were performed using EZR version 1.40 (Saitama Medical Centre, Jichi Medical University, Saitama, Japan).¹⁷

In total, 162 patients met the inclusion criteria, and 112 pairs of patients (median age [interquartile range] 6 years [4, 8]) and guardians (mother, 81.2%; father, 16.1%; grandmother, 1.8%; other, 0.9%) were included in the analysis (Fig. 1, Table 1). In the ROC analysis, the AUC for the guardians’ influenza diagnosis was 0.69 (95% confidence interval [CI]: 0.59–0.79). The optimal cut-off point was 70%, and the sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio were 59.6% (95% CI: 45.8–72.4%), 69.1% (95% CI: 55.2–80.9%), 1.93 (95% CI: 1.23–3.02), and 0.58 (95% CI: 0.41–0.84), respectively (Fig. 2). The negative likelihood ratio at a 30% cut-off point was 0.12 (95% CI: 0.02–0.93), and the positive likelihood ratio at a 90% cut-off point was 2.77 (95% CI: 1.36–5.67). Table 2 shows the sensitivity and specificity of guardians’ influenza diagnosis at various cut-off points.

Fig. 1.  Flowchart of participant recruitment and study design.

Table 1.  Characteristics of the study patients (N = 112).

Fig. 2.  Receiver-operating characteristic (ROC) curve of guardians’ quantitative influenza diagnosis.

Table 2.  Receiver-operating characteristic curve analysis of the guardians’ influenza diagnosis.

In subgroup analyses, the AUCs were higher for: mothers (0.72), guardians of children with an influenza history (0.72), guardians who were aware of the epidemic (0.71), and guardians of unvaccinated children (0.76). AUCs were lower for guardians whose children were aged <4 years (0.52) or who were unaware of the epidemic (0.57); and were lowest for fathers (0.48) (Table 3). The ROC curve and sensitivity and specificity at major cut-off points for each subgroup are shown in Supplementary File S3.

Table 3.  Area under the receiver-operating characteristics curve of the guardians’ influenza diagnosis in each subgroup.

The AUC increased significantly from 0.79 to 0.85 (P = 0.04) when adding information provided by the guardian (the guardians' influenza diagnosis likelihood estimate (%), guardian type, patient’s history of contact with persons with influenza, patient’s history of influenza, and patient’s influenza vaccination status) and basic clinical information (age, sex, headache, nasal discharge, cough, joint and muscle pain, fatigue, and axillary temperature taken at home) (Fig. 3).

Fig. 3.  Receiver-operating characteristic (ROC) curves of guardians’ influenza diagnosis estimated according to a basic clinical information model vs a basic clinical information plus guardians’ diagnosis model. The basic clinical information included age, sex, headache, nasal discharge, cough, joint and muscle pain, fatigue, and axillary temperature taken at home. The guardians’ diagnosis model included guardians’ influenza diagnosis (%), guardian type, patient’s exposure to persons with influenza, patient’s history of influenza, and patient’s influenza vaccination status.

This prospective observational study confirmed that guardians’ quantitative influenza diagnosis was useful for both definitive and exclusionary diagnoses, especially when the guardians’ reports revealed an extremely high (85.5% specificity at 90% cut-off point) or low (98.2% sensitivity at 30% cut-off point) probability, and identified factors that improve guardians’ diagnostic accuracy. Multivariable logistic regression analysis showed that information on guardians’ diagnosis and factors that improve guardians’ diagnostic accuracy further increased overall clinical diagnostic accuracy when added to basic clinical information for an influenza diagnosis.

A previous report showed that guardians’ qualitative influenza diagnosis had a low diagnostic accuracy (sensitivity 42.4%, specificity 52.6%).¹⁸ In addition, an earlier retrospective observational study found that the guardians’ quantitative influenza diagnosis could change the cut-off point in the same manner as ordinary laboratory data. Therefore, a quantitative diagnosis was more useful than a qualitative diagnosis for diagnosing influenza (90.9% sensitivity at a 50% cut-off point, 92.3% specificity at an 80% cut-off point, AUC 0.82).¹⁰ This prospective observational study confirmed the accuracy of guardians’ quantitative influenza diagnosis in their children (98.2% sensitivity at 30% cut-off point, 85.5% specificity at 90% cut-off point, AUC 0.69), and the results are consistent with those of a previous retrospective observational study.⁹ The AUC of this study was lower than that reported previously, which may be due to random error. The accuracy of physicians’ qualitative clinical diagnosis of influenza in children (sensitivity 38%, specificity 91%) has been reported, which corresponds to a cut-off value of 90% for guardian diagnosis in our study.¹⁹

Influenza is a common disease, and many patients recover after symptomatic treatment. However, some patients develop severe disease; the influenza virus causes pandemics. Utilisation of guardians’ diagnoses could have some social benefits. First, it may reduce the need to visit medical institutions for an influenza diagnosis, thus reducing virus transmission. Aside from the examination performed at a medical institution, the guardians’ diagnosis can be by a remote medical interview.

Second, even when a patient visits a medical institution, a guardian’s diagnosis can improve a physician’s influenza diagnostic accuracy. Generally, influenza tends to be overlooked, especially in the early stage, which sometimes delays patients’ recovery.²⁰ This study showed that a guardian’s diagnosis is useful even in the early stage of influenza when the sensitivity of the rapid influenza diagnostic test is low.²¹ A more accurate diagnosis by physicians in the early stage of influenza can improve disease control in the community.

Guardians’ diagnoses of illnesses in their children are a useful social resource in emergencies, such as pandemics. During emergencies, it is reasonable and efficient to use the guardians’ diagnosis for infection control. For example, when the value of a guardian’s diagnosis is extremely high because of factors that can improve the diagnosis, treatment can commence without the child visiting a medical institution. In the future, the medical information obtained outside medical institutions will increase because of the innovations in information and communication technology.²² This may affect guardians’ diagnosis and the use of over-the-counter drugs.⁸ In the future, physicians may be increasingly required to resolve guardians’ concerns caused by excessive medical information and to utilise the guardians’ diagnoses.²³

This prospective study had less selection bias than the previous retrospective study⁹ and showed the usefulness of guardians’ quantitative influenza diagnosis of their children. The diagnostic accuracy tended to be lower in guardians of children aged <4 years and higher in guardians whose children had not been vaccinated or who had a history of influenza. This result indicates that diagnosis by guardians is difficult with children aged <4 years, as young children cannot express their subjective symptoms clearly. In contrast, symptoms in unvaccinated children may be typical of influenza and therefore, it may be relatively easy for guardians to diagnose influenza in their children. Prior experience of influenza in their children may improve the diagnostic accuracy because guardians can compare their children’s current and past influenza symptoms.

Second, we examined factors related to guardians’ characteristics. Diagnostic accuracy tended to be higher in mothers and lower in fathers. Among the developed countries, fathers in Japan spend the least time performing housework and childcare²⁴ and only 14.5–23.5% of fathers take time off from work to take care of their children when they are sick.²⁵ This may be related to Japan’s working culture rather than gender differences in diagnostic accuracy.

Finally, we reviewed factors related to the environment in which guardians made their diagnosis. The accuracy of diagnosis tended to be higher in guardians who were aware of the influenza epidemic situation. We considered this an environment-related factor rather than a factor related to the guardian’s characteristics. An environment where information on epidemics can be accurately shared, such as schools and nurseries, is likely to effectively improve a guardian’s diagnosis by comparing symptoms with other patients and considering the possibility of infectious exposure such as contact with other known patients.

Our study has some limitations. We did not include a sufficient number of younger children, such as infants. Younger children may have fewer symptoms than older children and are less able to communicate their symptoms to their guardian, which may negatively affect the guardians’ diagnostic accuracy. Larger studies with a sufficient number of patients in each age group for subgroup analysis are needed.

Second, to perform reliable research, more accurate reference standards should be used. In Japan, it is usually difficult to perform polymerase chain reaction tests or virus cultures that are more reliable for influenza diagnosis than rapid influenza diagnostic tests, especially in primary care clinics. Furthermore, the strains observed in the 2018–19 influenza season and the 2017–18 influenza season were different.²⁶ More accurate virus testing methods are needed to clarify the difference in the guardians’ diagnostic accuracy between different strains of the influenza virus.

As well, the study could not adequately analyse the guardians’ background, such as smartphone and internet use, educational background, work, family and friend structure, and medical history, because data were collected from the medical chart and pre-examination checklist of children who were suspected to have influenza. A more detailed assessment of the guardians’ diagnostic accuracy requires a different approach to data collection beyond general practice for influenza. In addition, the accuracy of the guardians’ diagnosis needs to be verified in other countries. Influenza is a common disease because it occurs repeatedly worldwide every year and the symptoms are relatively easy to understand. However, considering the differences in national healthcare systems and the people’s behaviour toward influenza, differences in guardians’ diagnostic accuracy should be assessed separately in each country.

We conducted a prospective observational study and confirmed that guardians’ influenza diagnosis of their children was useful for both definitive and exclusionary diagnoses, especially when their reports revealed an extremely high or low probability, and we identified some factors that could be used to improve the accuracy of guardians’ influenza diagnosis. Moreover, the results revealed that adding information about guardians’ diagnosis and factors that improve guardians’ diagnosis to basic clinical information can improve physicians’ clinical diagnostic accuracy. Guardians’ influenza diagnosis of their children could prevent the spread of the infection and reduce its social burden.

The data generated and analysed during this study are mostly included in this published article. Additional data are available from the corresponding author on reasonable request.

The authors have no competing interests to declare.

This study was conducted without external funding.

Some of the results contained in this manuscript were presented at the 11th Annual Conference of the Japan Primary Care Association, which was held online in Hiroshima from 23 July to 31 August 2020 (https://www.c-linkage.co.jp/jpca2020/en/).

Supplementary material is available online.