Adaptation of autism spectrum screening questionnaire (ASSQ) for use in Georgian school settings

Abstract

Purpose

Even though 95% of children with neurodevelopmental disorders, including autism spectrum disorder (ASD), live in low- and middle-income countries, there is a dearth of studies from these countries, including the Republic of Georgia. Several ASD screening tools are available, but few are validated for use in Georgian or other smaller countries. This study aims to adapt and validate the autism spectrum screening questionnaire (ASSQ) for use in Georgia.

Design/methodology/approach

The ASSQ was administered for all third-grade students in 402 schools in the five main Georgian cities, n = 27,336. Prior to use, the 27-item ASSQ was translated, back-translated and adapted for use in Georgia. A total of 16,556 students (approximately 61%) were assessed by a parent and/or teacher. Optimal cutoff scores were estimated. Randomly chosen children who screened positive (n = 173) and negative (n = 127) were offered comprehensive assessment using standardized diagnostic procedures.

Findings

Data from 15,510 parents- and 13,517 teachers-administered ASSQ revealed statistically significant differences in median and cutoff scores between parents and teachers: 7 versus 4 and 9 versus 6, respectively. Cutoff score = 14, on either parent or teacher ASSQ, had sensitivity of 0.94, indicating that it can be used in school settings.

Originality/value

The Georgian adaptation of the ASSQ creates opportunity for further ASD research, while also providing a valid screening tool for clinicians. Data from Georgia will add to the growing understanding of the broader ASD phenotype.

Citation

Zirakashvili, M., Gabunia, M., Mebonia, N., Mikiashvili, T., Lomidze, G., Bishop, S., Leventhal, B. and Kim, Y.S. (2022), "Adaptation of autism spectrum screening questionnaire (ASSQ) for use in Georgian school settings", Journal of Public Mental Health, Vol. 21 No. 4, pp. 309-322. https://doi.org/10.1108/JPMH-03-2022-0028

Autism spectrum disorder (ASD) is a heterogeneous neurodevelopmental disorder (NDD) characterized by deficits in social communication/interactions, and the presence of restricted, repetitive patterns of behavior, interests or activities, that usually persists throughout the lifespan (American Psychiatric Association, 2013). Estimated ASD prevalence varies with population and settings studied, and methods of ascertainment, as well as sampling and case definition (Lord et al., 2020). Recently, the Autism and Developmental Disabilities Monitoring (ADDM) Network reported US ASD prevalence as one in 44 children (2.27%), aged eight years (Maenner et al., 2021). This is in contrast to World Health Organization ASD prevalence estimates of 1% worldwide (WHO, 2022).

It is estimated that 95% of all individuals < five years of age with NDDs, including ASD, live in low- and middle-income countries (LMICs) (Olusanya et al., 2018). Yet, the vast majority of ASD research has been conducted in high-income countries (Durkin et al., 2015), resulting in an information gap with respect to children in LMICs (Kim et al., 2011; Olusanya et al., 2018; Lord et al., 2022; WHO, 2022), including the Republic of Georgia, who may have significant variations in phenotypes, genotypes and access to assessment and treatment services. As a result, there is a need for research that assesses the manifestations and the impact of NDDs in different world regions, including prevalence, diagnosis and management.

Before 2009, ASD was rarely diagnosed in Georgia, and there were no specific policies related to ASD and other NDDs (cerebral palsies and other motor disabilities, Down syndrome, epilepsies) with respect to health care and educational services. More recently, NDD-focused outpatient centers use standard instruments, and there is municipal governmental support for rehabilitation therapy, specifically for children diagnosed with ASD. There is still no state program for children with NDDs with mental health problems. Despite the fact that educational system underwent reforms, and inclusive education is now available for all children in Georgia, many children with NDDs are still undiagnosed and untreated (UNICEF & National Statistics Office of Georgia, Tblisi, 2020). NDD diagnoses still lead to considerable stigmatization. Public awareness is still low, and multidisciplinary evaluation/treatment teams are available only in some regional centers in Georgia (UNICEF & National Statistics Office of Georgia, Tblisi, 2020). There are no data on the prevalence of NDDs, particularly ASD, in Georgia. The state guidelines for the identification and management of ASD were approved recently (MOH, 2018), recommending use of ASD screening tools, but, at this time, this is available only for toddlers. There is a substantial need for screening tools to be used in general population screening of school-age children.

The autism spectrum screening questionnaire (ASSQ) is one of the screening tools for ASD (Bernie et al., 2021; Chakraborty et al., 2021; DuBay et al., 2021; Lord et al., 2022; Scott et al., 2021; Yan et al., 2021). It was originally designed by Ehlers and Gillberg, with further refinement in collaboration with Lorna Wing for the purpose of estimating prevalence of Asperger’s syndrome (Ehlers et al., 1999). Subsequently, it has become one of the most widely used autism screening tools for screening populations, and for school-age children, worldwide (Adachi et al., 2018; Guo et al., 2011; Mattila et al., 2012; Posserud et al., 2009). It is short and suitable for completion by both parents and teachers in a school setting, free of charge. The ASSQ is also useful for screening in clinical settings, as well as in population-based epidemiological studies.

Objective

The goal of this study was to adapt and validate the ASSQ for use in Georgian school settings, using an epidemiologically ascertained, representative population sample. We chose third-grade children as a target population, because teachers have been with the same children for a minimum of two years, several hours per day; as a result, they are experienced enough to answer the questionnaire reliably.

Methods

In Georgia, compulsory education is available for every child aged 6–18 years. The target population (N = 22,553) included all third-grade children in the 264 schools in the five main cities in Georgia (Tbilisi – registered 33.9% of all third-grade children, Kutaisi – 5.5%, Batumi – 5.3%, Zugdidi – 1.5% and Telavi – 0.9%). This sample represents 47% of all third-grade children in Georgia (N = 47,997). For the randomization of study participants, we used the single-step cluster sampling procedure – a method of probability sampling; The list of public schools was obtained from authorized institutions. Afterward, we performed a random selection of individual institutions that were considered clusters; for further identification of individual respondents, we used simple random sampling. Then, parents and teachers of all children satisfying the selection criteria were invited to participate. Because of the COVID-19 pandemic restrictions, assessment tools that did not need direct observation of a child were completed using teleconsultation.

The study was conducted in two steps:

1. Phase I: ASSQ screening (April–December 2019), ascertainment of screen positive and negative children (January–March 2020);

2. Phase II: Confirmative diagnostic assessments (March 2020–June 2021), and best-estimate diagnoses (July–November 2021).

In Phase II, children from the upper 10 percentiles were contacted and invited for final diagnosis of ASD. Those who responded were evaluated using standardized diagnostic assessments, including the Autism Diagnostic Schedule-II (ADOS-II), Autism Diagnostic Interview-Revised (ADI-R), Test of Nonverbal Intelligence, Fourth Edition (TONI-4), the Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime Version (K-SADS-PL), Vineland Adaptive Behavior Scales, second edition (Vineland II) and speech, language and communication tests. Consensus, best estimate diagnoses were made by two independent, experienced, GAn clinicians, after reviewing all the relevant documents.

We have accepted some methodological elements for this study from the article of Mattila and others published in 2012 (Mattila et al., 2012).

Eligibility criteria: All Georgian-speaking parents and teachers of third-grade children from the selected schools were included. Parents with cognitive disabilities that could interfere with the proper completion of the ASSQ were excluded from the study.

The study was approved by the Ethics Committee of the National Disease Control Center. Approval for study and official permission was requested from the Ministry of Education of Georgia and each school principal. Written informed consent was obtained from parents.

Screening

The ASSQ consists of 27 items/statements, each rated on a three-point scale: “not true” (0 points), “somewhat true” (1 point) and “certainly true” (2 points). All item scores are added to produce a total score (maximum of 54 points). No training is required for the use of the ASSQ, which, for each child, takes approximately 10 min to complete.

The study team provided informative talks about ASD and NDDs for the teachers and school principals. At that time, research materials and questionnaires were given to the teachers who then distributed materials to parents. All parents and teachers were asked to complete the ASSQ. In total, 16,654 children were assessed by parents and/or teachers, with a 74% participation rate (16,556/22,553) (Table 1). After initial cleaning (dropping duplicate observations and observations with missing child identification information), 15,510 and 13,517 records were included in the parent and teacher data sets, respectively.

All screened children were assigned to the top first, second, third, fourth, fifth and tenth percentiles, based on their ASSQ scores (Table 1) from parents and/or teachers.

Diagnostic assessment

We endeavored to contact all eligible children for whom there was consent for diagnostic evaluation. Diagnostic assessments were conducted from May 2020 to June 2021. The K-SADS-PL, ADI-R and Vineland-II questionnaires were completed using teleconsultation. The ADOS-2, TONI-4, the Peabody Picture Vocabulary Test, Fifth Edition – PPVT-5, Expressive Vocabulary test, Third Edition – EVT-3 and Children’s Communication Checklist-2 – CCC-2 were performed in periods of milder restrictions at the centers proximal to the schools attended by each child. All diagnostic instruments were translated, back-translated and piloted for use with Georgian children in the current study; however, these versions were not validated and the sensitivity and specificity of those tests were not established.

The International Classification of Diseases and Related Health Problems tenth revision – ICD-10 [World Health Organization (WHO), 2015] and Diagnostic and Statistical Manual of Mental Disorders – DSM-5 (American Psychiatric Association, 2013) were used when arriving at final diagnoses.

Cultural consideration

The research team carefully addressed potential cultural biases in case identification (Figure 2). All instruments, including the ASSQ, were translated from English to Georgian. The translations were done by two bilingual translators for whom Georgian was their mother language. Discrepancies between the translators was discussed and resolved with the addition of a bilingual translator who was not involved in the previous translations. A corrected version was then back-translated by a person who was unaware of the English version. The translated questionnaire was compared to original questionnaire.

A panel of qualified experts (multidisciplinary team of mental health, including parents and special educators) reviewed all items and determined whether the translated and original versions achieve semantic, idiomatic, experiential and conceptual equivalence. Discrepancies were resolved, and a consensus was achieved on all items.

For the ASSQ, the final version was pilot tested, with a focus group of intended respondents: ten teachers and 12 parents. Using open-ended questions, respondents were asked what they thought of each item; their responses confirmed that translated items retained the same meaning as the original items.

To minimize possible cultural biases in diagnoses, a diagnostic team was composed of Georgian professionals with a minimum of five years of clinical and two years of research diagnostic experience. Moreover, a random sample of diagnoses (10%) were validated by US ASD diagnostic experts.

Statistical methods

Descriptive statistics were used for demographic variables. Pearson χ² test (or Fisher’s exact test, when appropriate) was used to detect associations between categorical variables. The Kolmogorov–Smirnov and the Shapiro–Wilk tests were used to assess the normality of the distribution of continuous variables. A Mann–Whitney U was used to detect differences between independent means. The Wilcoxon test was used to assess associations between paired samples.

Observations missing both name, surname and contact person variables were excluded from analyses due to the impossibility of identifying the child. Questionnaires with more than five missing values were also dropped. For the management of missing data, we provided multiple imputation assessments. Analyzing the missing values showed random patterns of missing data across variables. After that, missing values for every ASSQ variable were substituted with the corresponding value based on logistic regression computation of the data variable, and then total ASSQ scores were calculated.

Internal consistency of the Georgian versions of ASSQ was assessed with Cronbach’s α. A value above 0.7 is considered acceptable.

The receiver operational characteristic (ROC) analysis was used to assess the performance of the screening questionnaires. The area under the ROC curve (AUC) with a 95% confidence interval (95% CI) was calculated (Figure.1). Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were assessed for parent-administered, teacher-administered and maximum ASSQ scores (choosing the higher score from among parent and teacher ASSQ scores for the child). We calculated the Youden index as the sum of sensitivity and specificity subtracted by 100% (Mackrides and Ryherd, 2011).

Statistical analyses were performed using SPSS (IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp).

Results

Of the 16,654 individuals completing the ASSQ, 11,264 (67.6%) were from Tbilisi (capital city), 2,162 (13.0%) in Kutaisi, 2,283 (13.7%) in Batumi, 545 (3.3%) in Zugdidi and 400 (2.4%) in Telavi (Table 2); this reflects the distribution of all third-grade children in five Georgian research sites. Participation rates were 69% in Tbilisi, 82% in Kutaisi, 90% in Batumi, 75% in Zugdidi and 98% in Telavi.

Overall, 15,431 (92.7% from total respondents) parent-administered ASSQs were completed: 7,735 (50.1%) for females and 7,696 (49.9%) for male. Of the 13,500 teacher-completed ASSQs, 6,646 (49.2%) were females and 6,854 (50.8%) males.

The overall analysis of data for parent-administered ASSQ showed Cronbach’s α = 0.807, indicating that questionnaire to has a good internal consistency. As for teacher-administered ASSQ, Cronbach’s α was 0.826, which is an acceptable parameter for internal consistency. The median values of both the parent-administered ASSQ for males (median 8 (IQR 9) were higher than those for females (median 7 (IQR 8); the same is observed for teacher-administered ASSQ where males showed higher median value (median 5, (IQR 6)) compared to females (median 4, (IQR 5)); overall, parent-administered ASSQ scores were higher than those administered by teachers (Tables 3 and 4). These differences between males and females, as well as between parents and teachers, were statistically significant (p < 0.001).

The upper 95th percentile for parent-administered ASSQ scores was 22, which included 871 individuals. For the teacher-administered ASSQ, the same parameter was set at 18 and included 742 individuals. The upper 95th percentile of maximum ASSQ score was 23 and included 922 completed questionnaires. The maximum ASSQ score was calculated by combining the higher scores of either parents’ or teachers’ ratings on each of 27 items. See Table 1 for detailed data and distribution of ASSQ, based on data collection.

In total, 300 individuals completed further assessment for the final diagnosis of ASD (Tables 5, 6, 7). Among them, 53 (17.7%) children were diagnosed with ASD. The remaining 247 (82.3%) individuals had no signs of ASD. However, among children without ASD, various developmental disorders were detected; the most frequent was attention deficit hyperactivity disorder, which was revealed in 57 (23.1%) individuals; learning disability was detected in 21 (8.5%) children; conduct disorder was detected in 20 (8.1%) individuals; 19 (7.7%) children were diagnosed with anxiety disorder and another 19 (7.7%) speech and language disorders were found; remaining 14 (5.6%) children were diagnosed with tics, enuresis or unspecified developmental disorders. Children with final diagnoses were used for the assessment of the screening property of the ASSQ.

ROC analyses was performed to assess the discriminative validity of the ASSQ in distinguishing ASD from non-ASD cases. Figure 1 presents ROC curves for parent-administered ASSQ, teacher-administered ASSQ and for the maximum ASSQ.

The AUC indicates good discriminant ability of the ASSQ questionnaire: 0.671 (95% CI 0.593–0.790) for parent; 0.706 (95% CI 0.627–0.789) for teacher; and 0.724 (95% CI (0.654–0.794) for maximum ASSQ score. Table 8 shows the sensitivity, specificity, PPV and NPV for the optimal cutoff value from the ROC analyses.

For parent-administered ASSQ cutoff value, 13 or more yielded the highest Youden index (24%), setting sensitivity as 0.830 and specificity as 0.418. As for teacher ASSQ cutoff value, eight or more showed Youden index as 30%, setting sensitivity as 0.833 and specificity 0.468. The highest Youden index for maximum ASSQ score was 40%, with sensitivity of 0.792 and specificity of 0.611 at the cutoff value of 20 or more.

We performed a sex-specific assessment of the screening ability of ASSQs. Table 9 shows the results for boys. It should be mentioned that ROC is slightly higher compared to overall data. However, we did not find statistically significant discriminating abilities of ASSQs for girls. Table 10 provides more details.

Discussion

Numerous screening instruments have been developed for the efficient identification of individuals at high risk for ASD. The ASSQ (Ehlers and Gillberg, 1993), initially developed as a screening tool specifically for Asperger syndrome, later was validated for use in children with ASD (Ehlers et al., 1999). Further studies of the ASSQ determined the importance of establishing cutoff scores for different languages and cultures (Guo et al., 2011; Mattila et al., 2012; Posserud et al., 2009).

In Ehlers’ and Gillberg’s 1993 study, teacher-administered ASSQ was studied among 7–16-year-old children from the general population and among the five definite Asperger syndrome cases in this community sample. In the same study, for teacher ASSQ, the total scores showed excellent results regarding test–retest reliability over an eight-month period (Pearson r = 0.90; p = 0.001) and the good inter-rater reliability for two different teachers’ ratings for the same child (Pearson r = 0.79; p = 0.001), indicating satisfactory reliability and discriminating abilities for the questionnaire (Ehlers and Gillberg, 1993).

Cronbach’s α measures the internal consistency of the questionnaire by identifying similarities of coded responses for a closely related set of questions within the domain. In our cases, Cronbach’s α for parent-administered ASSQ was 0.807, and for teacher-administered ASSQ – 0.826, indicating the good internal consistency of the questionnaires assessed.

Sensitivity and specificity are the main parameters and defining an optimal cutoff value are critical to maximizing the discriminating ability of the screening questionnaires. Adachi and colleagues found a sensitivity of 93% and specificity of 84% for the parent-administered ASSQ at the cutoff value of 7 (Adachi et al., 2018). Guo and co-authors detected a cutoff of 12 having a maximum sensitivity of 95.7% and a specificity of 82.5 for parent-administered ASSQ (Guo et al., 2011).

In defining the cutoff value, we have taken into consideration the Youden index and recommended sensitivity value for screening tools (Mackrides and Ryherd, 2011). High sensitivity is recognized as a priority for screening questionnaires and a decision about the cutoff value is made in favor of sensitivity. On the other hand, specificity is also important to avoid false-positive results and unnecessary admittance of those cases for full-scale evaluation. Stemming from this, we recommend sensitivity of the Georgian version of parent-administered ASSQ cutoff value ≥ 13 that yields sensitivity of 83.0% and specificity of 41.8%; for teacher-administered ASSQ, the cutoff value ≥ 8 that produces sensitivity of 83.3% and specificity of 46.8%; and for maximum ASSQ, the cutoff value was set as ≥ 20, with sensitivity 79.2% and specificity 61.1%.

Sensitivity describes the ability of the screening questionnaire to correctly identify cases. Instead, specificity stands for the ability of a questionnaire to correct the identification of healthy individuals (or individuals without condition of interest). According to our data, sensitivity of parent-administered ASSQ at the cutoff value ≥ 13 was 0.830, indicating that out of 100 ASD cases, a questionnaire will recognize 83 of them as having ASD. Similarly, the specificity of parent-administered ASSQ was 0.418, which implies that out of 100 persons without ASD, the questionnaire will correctly identify 42 of them as not having ASD.

As for PPV and NPV, they reflect the test’s ability to predict ASD cases and healthy individuals based on the results of the screening questionnaire. In our case, parent-administered ASSQ showed PPV to be 0.232, indicating that from 100 individuals identified as possible ASD cases after assessment with ASSQ, 24 of them will have a final ASD diagnosis. Conversely, parent-administered ASSQ showed an NPV of 0.922, indicating that only out of 100 individuals identified as not having ASD through ASSQ assessment, 93% of them will not be diagnosed as having ASD after final assessment. It should be mentioned that PPV and NPV are highly dependent on the prevalence of the disorder when sensitivity and specificity are independent of the population parameters of the disorder.

Our study revealed that the ASSQ is a valid and easy-to-use tool for use in school settings for the identification of children with high risk for ASD, using the maximum ASSQ scores among parent and teacher ASSQ scores, giving combined information from different surroundings (i.e. school and home) according to Posserud et al. (2009) and Mattila et al. (2012). For our study, the combined cutoff score is equal to, or above the cutoff score of 14, giving higher sensitivity (0.943) than only parents (0.91) or teachers (0.90) score.

The study found that ASSQ scores for most children (84.9%) diagnosed with ASD were in upper 90th percentile (45 out of 53), and 64.2% were in upper 95th percentile (34 out 53). ROC analysis also revealed that the ASSQ has better predictive value when using maximum ASSQ scores among parent and teacher questionnaires (Table 8). This finding is consistent with Norwegian and Finnish studies (Mattila et al., 2012; Posserud et al., 2009).

Our study revealed a statistically significant difference between parent and teacher ASSQ scores, with parents scoring higher (Tables 3 and 4). The complexity of child behaviors, and their variations across different settings, require multiple informants to reflect the respondents’ attributions and perspectives on reporting children’s behaviors, facilitating comprehensive and accurate understanding of the child (Stadnick et al., 2017). As in our study, several prior studies observed discrepancies in ASSQ reports between questionnaire respondents (Ehlers et al., 1999; Mattila et al., 2012; Posserud et al., 2009). Stadnick and colleagues demonstrated that caregivers and teachers perceive the intensity of child behavioral problems differently (Stadnick et al., 2017). Such differences may be the result of parents being more sensitive to their children’s externalizing difficulties (Cheng et al., 2018). A child’s individual characteristics can also contribute to such differences. For example, Azad and coauthors (Azad et al., 2016) indicated that parent and teacher ratings were statistically significantly correlated at the beginning and end of the academic year and with the degree to which children were affected. Dickson et al. also reported that the concordance of parent–teacher rating was significantly impacted by autism severity and cognitive abilities of a child (Dickson et al., 2018).

Low specificity and PPV may be explained by low ASD prevalence in general population (Ehlers et al., 1999) and, specifically, in this sample. However, sensitivity is not dependent on the prevalence of ASD, as it is calculated using affected and unaffected children in the sample, resulting in optimal sensitivity of ASSQ in our study population.

We suppose that the low specificity of the ASSQs was due to the fact that 300 children for the final assessment were invited from the upper 90th percentile of the ASSQ score of the whole sample. Besides, in children from the final assessment who were not diagnosed with ASD, a significant number were diagnosed with different NDDs, which share some behavioral features with ASD, that could contribute to low specificity. We expect that in real conditions, discriminating abilities of the ASSQs will be better than we have observed during the study; however, this will need further observation in real conditions.

We failed to find statistically significant screening ability for both, parent- and teacher-administered questionnaires for girls. The reason for this can be a relatively small number of female respondents with final assessment for ASD. Besides, it may be due to socio-cultural peculiarities as it is described in the work of Lord and coauthors (Lord et al., 2022) where the presentation of autism in girls is somewhat different from that seen in boys. It is clear that this is problematic in terms of practical use of the questionnaires. More efforts should be done to refine ASSQ questionnaires to become valid for girls. In general, the data about sex-specific discrimination ability are scarce. The authors identified problems in the detection of ASD symptoms among girls and women (Lord et al., 2020). Kopp and Gillberg elaborated a new version of ASSQ for more refined detection of possible ASD among girls (Kopp and Gillberg, 2011). Furthermore, they developed additional domains for the ASSQs that are specifically oriented to reveal character traits among girls. It is obvious that ASSQ validity for female respondents is questionable as it seems to be problematic for the Georgian version of the ASSQs. More efforts are needed to be done to improve the screening ability of the Georgian version of ASSQs for use among girls.

Cultural context and parents’ attitudes toward their child’s behaviors could significantly impact responses on questionnaire items. Georgian children, especially in more remote regions, are expected to be shy around adults, a behavior that is reinforced by parents, especially in unfamiliar settings. Parents and teachers may consider child behavior as atypical when the child behaves in a culturally inappropriate manner. So, cultural context should be considered in the validation of any instruments (Zirakashvili et al., 2018).

The ASSQ appears to have great utility in the Georgian school population, in which 75% of screen positive children received their first ASD diagnosis in the comprehensive assessment.

Strengths and limitations

The study is population-based and covers 47% of all Georgian third-grade children, which should be considered as a strength of the study. The completed screening questionnaires were obtained from both, parents and teachers. All screen-positive children were invited for standardized, comprehensive diagnostic assessments using seven different diagnostic instruments; however, the lack of screen-negative individuals in the final assessment may contribute to the low specificity of the ASSQs.

Limitations include some of the instruments being translated and adapted for the Georgian language; they are not yet validated in Georgian children. Further, final best estimate diagnoses were made by clinical experts, not based on the scores and algorithms of the instruments. For female respondents, both parent- and teacher-administered ASSQs were insufficient to adequately discriminate possible ASD cases; this should be considered at the time of use of the Georgian version of ASSQ in practical settings.

Conclusions

The ASSQ could be used as a screening instrument for ASD in the Georgian school setting, but for the best screening results, the maximum ASSQ scores from parents and teachers should be used. However, low discriminating capabilities of both parent- and teacher-administered ASSQs for girls should be taken into account and should be used with great caution. Our results also suggest that the ASSQ, with appropriate cultural adaptations for use in LMICs, may also play an important role in identifying children with ASD, as well as helping to guide resource allocation and development of educational services in these countries.

Figures

Figure 1

ROC curve demonstrates ASSQ performance against diagnosis of ASD

Figure 2

Flow chart of case identification

Table 1

Number of individuals with ASSQ scores by percentiles and places of investigation

Percentile	ASSQs types	Tbilisi	Kutaisi	Batumi	Zugdidi	Telavi	Total
		n (%)	n (%)	n (%)	n (%)	n (%)	n (%)
90	Parent ASSQ	1,039 (9.9)	175 (8.8)	319 (15.1)	59 (10.8)	36 (10.3)	1,628 (10.5)
	Teacher ASSQ	904 (10.7)	182 (9.4)	285 (13.2)	74 (13.6)	22 (5.5)	1,467 (10.9)
	Maximum ASSQ	1,273 (11.3)	203 (9.4)	383 (16.8)	65 (11.9)	41 (10.3)	1,965 (11.8)
95	Parent ASSQ	505 (4.8)	76 (3.8)	160 (7.6)	29 (5.3)	19 (5.5)	789 (5.1)
	Teacher ASSQ	449 (5.3)	66 (3.4)	127 (5.9)	22 (4)	14 (3.5)	678 (5)
	Maximum ASSQ	519 (4.9)	68 (3.4)	143 (6.8)	18 (3.3)	16 (4.6)	764 (4.9)
96	Parent ASSQ	420 (4)	56 (2.8)	133 (6.3)	20 (3.7)	17 (4.9)	646 (4.2)
	Teacher ASSQ	377 (4.4)	58 (3)	100 (4.6)	17 (3.1)	14 (3.5)	566 (4.2)
	Maximum ASSQ	426 (4)	54 (2.7)	115 (5.4)	13 (2.4)	13 (3.7)	621 (4)
97	Parent ASSQ	355 (3.4)	45 (2.3)	109 (5.2)	15 (2.8)	14 (4)	359 (2.3)
	Teacher ASSQ	321 (3.8)	51 (2.6)	89 (4.1)	14 (2.6)	10 (2.5)	485 (3.6)
	Maximum ASSQ	296 (2.8)	34 (1.7)	84 (4)	10 (1.8)	6 (1.7)	430 (2.8)
98	Parent ASSQ	193 (1.7)	24 (1.1)	65 (2.9)	6 (1.1)	5 (1.3)	293 (2.1)
	Teacher ASSQ	234 (2.2)	33 (1.7)	76 (3.6)	8 (1.5)	8 (2.3)	359 (2.2)
	Maximum ASSQ	235 (2.2)	26 (1.3)	73 (3.5)	8 (1.5)	6 (1.7)	348 (2.2)
99	Parent ASSQ	129 (1.2)	19 (1)	45 (2.1)	5 (0.9)	5 (1.4)	203 (1.3)
	Teacher ASSQ	102 (1.2)	10 (0.5)	29 (1.3)	4 (0.7)	2 (0.5)	147 (1.1)
	Maximum ASSQ	117 (1.1)	22 (1.1)	37 (1.7)	5 (0.9)	4 (1.1)	185 (1.2)

Table 2

Numbers of children in the survey data sets

City	Numbers of students							Participation rates
	Registered	Total in parent Dataset	Total in teacher data set	Appeared in both data sets	Appeared in parent data set only	Appeared in teacher data set only	Total	Parents (%)	Teachers (%)	Overall (%)
Tbilisi	16,247	10,695	8,548	7,979	2,716	569	11,264	66	53	69
Kutaisi	2,638	2,054	1,956	1,848	206	108	2,162	78	74	82
Batumi	2,529	2,164	2,189	2,070	94	119	2,283	86	87	90
Zugdidi	730	545	545	545	0	0	545	75	75	75
Telavi	409	348	400	348	0	52	400	85	98	98
Total	22,553	15,806	13,638	12,790	3,016	848	16,654	70	60	74

Table 3

Differences in respondents scores by child sex and by percentiles

Percentiles	90th	95th	96th	97th	98th	99th
Parents score: female/male	17.0/18.0	21.0/22.0	22.0/23.0	24.0/25.0	26.0/27.0	29.0/30.0
Teachers score: female/male	12.0/14.0	16.0/19.0	17.0/20.0	18.0/22.0	20.0/25.0	24.0/30.0

Table 4

Mean, median and mode for teachers and parents ASSQ scores stratified by sex

Respondents	Mean (SD)	Median (IQR)	Mode
Parents administered ASQ score: female/male	8.3 (6.4)/9.1 (6.7)	7 (4; 12)/8 (4; 13)	4/4
Teachers score: female/male	5.3 (5.1)/6.4 (6.2)	4 (2; 7)/5 (2; 8)	2/2

Table 5

Parent-administered ASSQ percentiles according to final diagnosis of ASD

Diagnosis	99th to 100% n (%)	98th to 99th% n (%)	97th to 98th% n (%)	96th to 97th% n (%)	95h to 96th% n (%)	90th to 95th% n (%)	0 to 89th% n (%)	Total n (%)
No ASD	14 (5.74)	11 (4.5)	20 (8.2)	6 (2.5)	7 (2.9)	37 (15.2)	149 (61.1)	244 (100)
ASD	8 (15.1)	3 (5.66)	9 (17.0)	3 (5.7)	2 (3.8)	8 (15.1)	20 (37.7)	53 (100)
Total	22 (7.4)	14 (4.7)	29 (9.8)	9 (3)	9 (3)	45 (15.2)	169 (56.9)	297 (100)

Table 6

Teacher-administered ASSQ percentiles according to final diagnosis of ASD

Diagnosis	99th to 100% n (%)	98th to 99th% n (%)	97th to 98th% n (%)	96th to 97th%, n (%)	95h to 96th% n (%)	90th to 95th% n (%)	0 to 89th% n (%)	Total n (%)
No ASD	8 (3.6)	12 (5.4)	24 (10.8)	4 (1.8)	8 (3.6)	28 (12.6)	138 (62.2)	222 (100)
ASD	9 (18.8)	6 (12.5)	4 (8.3)	3 (6.3)	2 (4.2)	10 (20.8)	14 (29.2)	48 (100)
Total	17 (6.3)	18 (6.7)	28 (10.4)	7 (2.6)	10 (3.7)	38 (14.1)	152 (56.3)	270 (100)

Table 7

Maximum ASSQ percentiles according to final diagnosis of ASD

Diagnosis	99th to 100% n (%)	98th to 99th% n (%)	97th to 98th% n (%)	96th to 97th% n (%)	95th to 96th% n (%)	90th to 95th% n (%)	0 to 89th% n (%)	Total n (%)
No ASD	10 (4.0)	9 (3.6)	15 (6.1)	21 (8.5)	8.(3.2)	61 (24.7)	123 (49.8)	247 (100)
ASD	9 (17.0)	7 (13.2)	1 (1.9)	9 (17.0)	3 (5.7)	13 (24.5)	11 (20.8)	53 (100)
Total	19 (6.3)	16 (5.3)	16 (5.3)	30 (10.0)	11 (3.7)	74 (24.7)	134 (44.7)	300 (100)

Table 8

Sensitivity, specificity, PPV and NPV of the ASSQ

ASSQ type	ROC AUC (95% CI)	Cutoff value	Sensitivity	Specificity	PPV	NPV	Youden index
Parent ASSQ	0.671 (0.593–0.750)	≥ 10	0.906	0.279	0.211	0.929	0.18
		≥ 11	0.868	0.328	0.215	0.916	0.18
		≥ 12	0.849	0.361	0.229	0.927	0.23
		≥ 13	0.830	0.418	0.232	0.922	0.24
Teacher ASSQ	0.706 (0.627–0.786)	≥ 6	0.917	0.333	0.229	0.949	0.25
		≥ 7	0.875	0.383	0.235	0.934	0.26
		≥ 8	0.833	0.468	0.253	0.929	0.30
		≥ 9	0.771	0.527	0.261	0.914	0.30
Maximum ASSQ	0.724 (0.654–0.794)	≥ 15	0.906	0.389	0.241	0.950	0.29
		≥ 16	0.868	0.409	0.240	0.935	0.28
		≥ 17	0.830	0.462	0.249	0.927	0.29
		≥ 18	0.811	0.502	0.259	0.925	0.31
		≥ 19	0.792	0.571	0.284	0.928	0.36
		≥ 20	0.792	0.611	0.298	0.931	0.40

Notes:

ROC – receiver operating characteristics; CI – confidence interval; PPV – positive predictive value; NPV – negative predictive value

Table 9

Sensitivity, specificity, PPV and NPV of the ASSQ stratified by sex of respondents (males; n = 206)

ASSQ type	ROC AUC (95% CI)	Cutoff value	Sensitivity	Specificity	PPV	NPV	Youden index
Parent ASSQ	0.691 (0.607–0.776)	≥ 10	0.915	0.318	0.283	0.923	0.23
		≥ 11	0.872	0.357	0.289	0.903	0.23
		≥ 12	0.872	0.401	0.304	0.913	0.27
		≥ 13	0.851	0.427	0.308	0.905	0.28
Teacher ASSQ	0.690 (0.602–0.779)	≥ 6	0.930	0.293	0.278	0.935	0.22
		≥ 7	0.884	0.340	0.281	0.909	0.22
		≥ 8	0.837	0.422	0.298	0.899	0.26
		≥ 9	0.767	0.483	0.303	0.877	0.25
Max ASSQ	0.734 (0.658–0.810)	≥ 15	0.915	0.409	0.314	0.942	0.32
		≥ 16	0.872	0.428	0.311	0.919	0.30
		≥ 17	0.830	0.491	0.325	0.907	0.32
		≥ 18	0.809	0.516	0.330	0.901	0.32
		≥ 19	0.787	0.591	0.363	0.904	0.38
		≥ 20	0.787	0.648	0.394	0.911	0.44

Notes:

ROC – receiver operating characteristics; CI – confidence interval; PPV – positive predictive value; NPV – negative predictive value

Table 10

Sensitivity, specificity, PPV and NPV of the ASSQ stratified by sex of respondents (females; n = 94)

ASSQ type	ROC AUC (95% CI)	Cutoff value	Sensitivity	Specificity	PPV	NPV	Youden index
Parent ASSQ	0.657 (0.420–0.894)*	≥ 10	0.833	0.207	–	–	0.04
		≥ 11	0.833	0.230	–	–	0.06
		≥ 12	0.833	0.299	–	–	0.13
		≥ 13	0.833	0.322	–	–	0.16
Teacher ASSQ	0.676 (0.421–0.931)**	≥ 6	0.800	0.413	–	–	0.21
		≥ 7	0.800	0.467	–	–	0.27
		≥ 8	0.800	0.560	–	–	0.36
		≥ 9	0.800	0.613	–	–	0.41
Max ASSQ	0.735 (0.517–0.953)***	≥ 15	0.833	0.352	–	–	0.19
		≥ 16	0.833	0.375	–	–	0.21
		≥ 17	0.833	0.409	–	–	0.24
		≥ 18	0.833	0.477	–	–	0.31
		≥ 19	0.833	0.534	–	–	0.37
		≥ 20	0.833	0.545	–	–	0.38

Notes: ROC – receiver operating characteristics; CI – confidence interval; PPV – positive predictive value; NPV – negative predictive value. *Asymptotic p-value – 0.2; ** asymptotic p-value – 0.19; *** Asymptotic p-value – 0.055