Co-developing ‘The CyberABIlity Scale’ to assess vulnerability to cyberscams for people with acquired brain injury: Delphi and cognitive interviews with clinicians and people with acquired brain injury
Jao-Yue J. Carminati
A B * , Jennie L. Ponsford A B and Kate Rachel Gould A B
A
B
Abstract
Although individuals with acquired brain injury (ABI) may be vulnerable to cyberscams, the lack of existing measures documenting cybersafety behaviours in people with ABI limits our understanding of ABI-specific risk factors, the frequency of this problem, and the ability to evaluate evidence-based interventions. The CyberABIlity Scale was developed to assess vulnerability in people with ABI via self-rated statements and practical scam-identification tasks. This study aimed to develop and refine The CyberABIlity Scale through feedback from clinicians and people with ABI.
Scale feedback was collected via three rounds of clinician surveys (n = 14) using Delphi methods and two rounds of cognitive interviews with participants with ABI (n = 8). Following each round, feedback was quantitatively and qualitatively summarised, and revisions were made accordingly.
Key revisions included removing 12 items deemed irrelevant. Instructions and rating scales were revised to improve clarity. Cognitive interviews identified 15 comprehension errors, with further revisions made to support response clarity for participants with ABI. Clinicians and participants with ABI endorsed the content and face validities of The CyberABIlity Scale.
Following further validation, The CyberABIlity Scale has the potential to be an effective screening measure for online vulnerability for people with ABI within clinical and research settings.
Keywords: acquired brain injury, cognitive interviewing, cyberscams, cybercrime, delphi method, measure development, validation.
References
Australian Bureau of Statistics (2008) ‘Personal fraud in Australia.’ (Australian Institute of Criminology) Available at https://www.aic.gov.au/publications/cfi/cfi180
Beatty PC, Willis GB (2007) Research Synthesis: The Practice of Cognitive Interviewing. Public Opinion Quarterly 71(2), 287-311.
| Crossref | Google Scholar |
Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL (2018) Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Frontiers in Public Health 6, 149.
| Crossref | Google Scholar | PubMed |
Boeije H, Willis G (2013) The Cognitive Interviewing Reporting Framework (CIRF): Towards the harmonization of cognitive testing reports. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 9(3), 97-95.
| Crossref | Google Scholar |
Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C (2011) Using and Reporting the Delphi Method for Selecting Healthcare Quality Indicators: A Systematic Review. PLoS One 6(6), e20476.
| Crossref | Google Scholar | PubMed |
Brunner M, Hemsley B, Palmer S, Dann S, Togher L (2015) Review of the literature on the use of social media by people with traumatic brain injury (TBI). Disability and Rehabilitation 37(17), 1511-1521.
| Crossref | Google Scholar | PubMed |
Brunner M, Palmer S, Togher L, Hemsley B (2019) ‘I kind of figured it out’: the views and experiences of people with traumatic brain injury (TBI) in using social media—self-determination for participation and inclusion online. International Journal of Language & Communication Disorders 54(2), 221-233.
| Crossref | Google Scholar | PubMed |
Brunner M, Palmer S, Togher L, Dann S, Hemsley B (2020) “If I knew what I was doing on Twitter then I would use it more”: Twitter experiences and networks of people with traumatic brain injury (TBI). Brain Impairment 21(1), 1-18.
| Crossref | Google Scholar |
Brunner M, Hemsley B, Togher L, Dann S, Palmer S (2021) Social Media and People With Traumatic Brain Injury: A Metasynthesis of Research Informing a Framework for Rehabilitation Clinical Practice, Policy, and Training. American Journal of Speech-Language Pathology 30(1), 19-33.
| Crossref | Google Scholar | PubMed |
Carpenter S (2018) Ten Steps in Scale Development and Reporting: A Guide for Researchers. Communication Methods and Measures 12(1), 25-44.
| Crossref | Google Scholar |
Carrier SL, Wong D, Lawrence K, McKay A (2022) Preliminary validation of a new competency tool for evaluating assessment skills in professional psychology trainees. Training and Education in Professional Psychology 16(2), 166-172.
| Crossref | Google Scholar |
Charters E, Gillett L, Simpson GK (2015) Efficacy of electronic portable assistive devices for people with acquired brain injury: a systematic review. Neuropsychological Rehabilitation 25(1), 82-121.
| Crossref | Google Scholar | PubMed |
Delbecq AL, Van de Ven AH, Gustafson DH (1975) ‘Group techniques for program planning: a guide to nominal group and delphi processes.’ (Scott Foresman) Available at https://eduq.info/xmlui/handle/11515/11368
Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, Wales PW (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology 67(4), 401-409.
| Crossref | Google Scholar | PubMed |
Fink A, Kosecoff J, Chassin M, Brook RH (1984) Consensus methods: characteristics and guidelines for use. American Journal of Public Health 74(9), 979-983.
| Crossref | Google Scholar | PubMed |
Flake JK (2021) Strengthening the foundation of educational psychology by integrating construct validation into open science reform. Educational Psychologist 56(2), 132-141.
| Crossref | Google Scholar |
Flake JK, Fried EI (2020) Measurement Schmeasurement: Questionable Measurement Practices and How to Avoid Them. Advances in Methods and Practices in Psychological Science 3(4), 456-465.
| Crossref | Google Scholar |
Gould KR, Brokenshire C (2017) Scams and brain impairment: a clinician’s treatment recommendations and a survivor’s perspective. Brain Impairment 18, 395.
| Crossref | Google Scholar |
Gould KR, Carminati JJ, Ponsford JL (2023a) “They just say how stupid I was for being conned”. Cyberscams and acquired brain injury: a qualitative exploration of the lived experience of survivors and close others. Neuropsychological Rehabilitation 33(2), 325-345.
| Crossref | Google Scholar | PubMed |
Gould KR, Carolan M, Ponsford JL (2023b) Do we need to know about cyberscams in neurorehabilitation? Brain Impairment 24, 229-244.
| Crossref | Google Scholar | PubMed |
Han SD, Boyle PA, James BD, Yu L, Bennett DA (2016) Mild Cognitive Impairment and Susceptibility to Scams in Old Age. Journal of Alzheimer’s Disease 49(3), 845-851.
| Crossref | Google Scholar | PubMed |
Hawthorne G, Gruen RL, Kaye AH (2009) Traumatic Brain Injury and Long-Term Quality of Life: Findings from an Australian Study. Journal of Neurotrauma 26, 1623-1633.
| Crossref | Google Scholar | PubMed |
Hinkin TR (1995) A Review of Scale Development Practices in the Study of Organizations. Journal of Management 21(5), 967-988.
| Crossref | Google Scholar |
Holmbeck GN, Devine KA (2009) Editorial: an author's checklist for measure development and validation manuscripts. Journal of Pediatric Psychology 34(7), 691-696.
| Crossref | Google Scholar |
James BD, Boyle PA, Bennett DA (2014) Correlates of Susceptibility to Scams in Older Adults Without Dementia. Journal of Elder Abuse & Neglect 26(2), 107-122.
| Crossref | Google Scholar | PubMed |
Jovanovski D, Zakzanis K, Ruttan L, Campbell Z, Erb S, Nussbaum D (2012) Ecologically Valid Assessment of Executive Dysfunction Using a Novel Virtual Reality Task in Patients with Acquired Brain Injury. Applied Neuropsychology: Adult 19(3), 207-220.
| Crossref | Google Scholar | PubMed |
Kilov AM, Togher L, Power E, Turkstra L (2010) Can teenagers with traumatic brain injury use Internet chatrooms? A systematic review of the literature and the Internet. Brain Injury 24(10), 1135-1172.
| Crossref | Google Scholar | PubMed |
Laconi S, Kaliszewska-Czeremska K, Gnisci A, Sergi I, Barke A, Jeromin F, Groth J, Gamez-Guadix M, Ozcan NK, Demetrovics Z, Király O, Siomos K, Floros G, Kuss DJ (2018) Cross-cultural study of Problematic Internet Use in nine European countries. Computers in Human Behavior 84, 430-440.
| Crossref | Google Scholar |
Li N, Kirkup G (2007) Gender and cultural differences in Internet use: a study of China and the UK. Computers & Education 48(2), 301-317.
| Crossref | Google Scholar |
Lichtenberg PA, Tocco M, Moray J, Hall L (2021) Examining the Validity of the Financial Exploitation Vulnerability Scale. Clinical Gerontologist 44(5), 585-593.
| Crossref | Google Scholar | PubMed |
Miller H, Lawson D, Power E, das Nair R, Sathananthan N, Wong D (2022) How do people with acquired brain injury interpret the Valued Living Questionnaire? A cognitive interviewing study. Journal of Contextual Behavioral Science 23, 125-136.
| Crossref | Google Scholar |
Modic D, Anderson R, Palomäki J (2018) We will make you like our research: The development of a susceptibility-to-persuasion scale. PLoS One 13(3), e0194119.
| Crossref | Google Scholar | PubMed |
Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, Bouter LM, de Vet HC (2010) The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Medical Research Methodology 10(1), 22.
| Crossref | Google Scholar |
Okoli C, Pawlowski SD (2004) The Delphi method as a research tool: An example, design considerations and applications. Information & Management 42(1), 15-29.
| Crossref | Google Scholar |
Ponsford JL, Downing MG, Olver J, Ponsford M, Acher R, Carty M, Spitz G (2014) Longitudinal Follow-Up of Patients with Traumatic Brain Injury: Outcome at Two, Five, and Ten Years Post-Injury. Journal of Neurotrauma 31(1), 64-77.
| Crossref | Google Scholar | PubMed |
Stanyon MR, Goldberg SE, Astle A, Griffiths A, Gordon AL (2017) The competencies of Registered Nurses working in care homes: a modified Delphi study. Age and Ageing 46, 582-588.
| Crossref | Google Scholar | PubMed |
Trevelyan EG, Robinson PN (2015) Delphi methodology in health research: How to do it? European Journal of Integrative Medicine 7(4), 423-428.
| Crossref | Google Scholar |
Tsaousides T, Matsuzawa Y, Lebowitz M (2011) Familiarity and prevalence of Facebook use for social networking among individuals with traumatic brain injury. Brain Injury 25(12), 1155-1162.
| Crossref | Google Scholar | PubMed |
Vaccaro M, Hart T, Whyte J, Buchhofer R (2007) Internet use and interest among individuals with traumatic brain injury: a consumer survey. Disability and Rehabilitation: Assistive Technology 2(2), 85-95.
| Crossref | Google Scholar | PubMed |
Vakil E, Greenstein Y, Weiss I, Shtein S (2019) The Effects of Moderate-to-Severe Traumatic Brain Injury on Episodic Memory: A Meta-Analysis. Neuropsychology Review 29(3), 270-287.
| Crossref | Google Scholar | PubMed |
Whiting DL, Deane FP, Ciarrochi J, McLeod HJ, Simpson GK (2015) Validating measures of psychological flexibility in a population with acquired brain injury. Psychological Assessment 27(2), 415-423.
| Crossref | Google Scholar | PubMed |
Whitty MT, Buchanan T (2016) The online dating romance scam: The psychological impact on victims – both financial and non-financial. Criminology & Criminal Justice 16(2), 176-194.
| Crossref | Google Scholar |
Wong D, Grace N, Baker K, McMahon G (2019) Measuring clinical competencies in facilitating group-based rehabilitation interventions: development of a new competency checklist. Clinical Rehabilitation 33(6), 1079-1087.
| Crossref | Google Scholar | PubMed |
