Evaluation of a pilot school-based physical activity challenge for primary students

Key words: Aboriginal and Torres Strait Islanders, children, program evaluation.

Physical inactivity during childhood is a key risk factor for the development of overweight and obesity^1,2 and chronic diseases including type 2 diabetes.³ In the Australian context, improving physical activity levels among Aboriginal^† people is of particular importance. Chronic disease occurs more often among Aboriginal people than non-Aboriginal people and at a much younger age.^4,5 Despite widespread attempts to promote physical activity and healthy lifestyles for children in Australia, many children are not sufficiently active, with only one-third of non-Aboriginal children (35%) and half of Aboriginal children (48%) meeting physical activity recommendations.⁶

Sedentary behaviour, a specific health behaviour which is more than the inverse of physical activity,⁷ is also a growing health concern among Australian children. Sedentary behaviour in adults is associated with the development of chronic disease⁸ and is linked to increased risk of obesity in children.^9–11

There is evidence that physical activity and sedentary behaviours adopted during childhood continue into adult life;¹² this highlights the importance of establishing healthy lifestyle behaviours during childhood. Schools are recognised as an important setting for health promotion and education for both Aboriginal and non-Aboriginal children.^13,14 Children and young people spend a substantial proportion of their waking hours at school and the school setting provides opportunities to increase physical activity, through both formal learning (e.g. physical education classes¹⁵) and during break times.¹⁶ A range of school-based interventions have been shown to be effective in increasing physical activity among students.¹⁷ These include changes to school physical education curricula,¹⁸ short activity breaks¹⁹ and provision of equipment (e.g. skipping ropes) to promote physical activity during recess and lunch breaks.²⁰ However, to our knowledge no studies have assessed the impacts of school-based interventions on physical activity levels among Aboriginal children in Australia.

To address the need for physical activity programs that are accessible and appropriate for Aboriginal children, the New South Wales (NSW) Government developed and piloted the Culture Health Communities Activity Challenge (hereafter known as the Challenge). The Challenge is a school-based pedometer program that aims to encourage regular physical activity in primary school students, and is part of the Culture Health Communities strategy promoting healthy lifestyles among Aboriginal people. An evaluation of this pilot program was undertaken to explore students’ and teachers’ experiences of the Challenge and to assess the impact of the Challenge on students’ physical activity levels.

Twenty-five schools across NSW were invited to participate in the Challenge. Invited schools were located in towns that had previously participated in the NSW Knockout Health Challenge (a community-based healthy lifestyle and weight loss challenge for Aboriginal people) with high numbers of Aboriginal students. Fifteen Year 5 and 6 classes from nine schools agreed to participate. Participating schools were located in a mix of metropolitan, regional and remote areas. In each school between one and three classes participated (Table 1). Three hundred and twenty-two students registered to take part in the Challenge (Table 1). Four students did not provide any demographic or outcome data and were excluded from analysis.

Table 1.  Profile of schools participating in the Culture Health Communities Activity Challenge
ASGC, Australian Standard Geographical Classification

The Challenge is an 8-week school-based pedometer program, in which classes compete to achieve the highest class average daily steps (of all participating classes) across the Challenge period. The Challenge ran from May to June 2014. The Challenge was developed for both Aboriginal and non-Aboriginal students, with a focus on being engaging and appropriate for Aboriginal students. The design and implementation of the Challenge was led by Aboriginal staff employed by NSW Health and overseen by an Advisory Committee comprising representatives from the Aboriginal community-controlled health sector, NSW Health, NSW Rugby League, Country Rugby League of NSW, and the Royal Australian College of General Practitioners. Implementation was led by the class teacher, with support from Aboriginal Education Assistants and the state-wide program implementation team. Teachers received a 2-hour training session and a resource book to support implementation of the Challenge. The resource book contained instructions for implementing the Challenge, using the Challenge website and suggested strategies for increasing students’ physical activity during the school day. In addition, an Aboriginal member of the state implementation team visited each participating school at least once during the Challenge to support implementation.

As part of the Challenge, each student was given a pedometer on each school day to record their steps during school hours. Each student also received a printed guidebook that contained goal setting exercises related to physical activity, tables for students to record their daily pedometer readings and other physical activity, instructions for using the pedometers, and nutrition and physical activity messages. Students were instructed to record their pedometer readings in the guidebook at the end of each school day. The following day, the class calculated their class’ average number of steps for the previous day and entered this data on the Challenge website. Students could also create individual profiles, where they could enter their individual daily pedometer data, their minutes of moderate-to-vigorous physical activity and also create personalised avatars. Via the website the class’ pedometer data were linked to an online ‘journey’, with a course map showing destinations around the world and the number of steps (calculated as a class average) the class needed to attain to ‘reach’ this destination. When the class achieved average step targets the class’ position on the course map was updated and videos unlocked. The videos featured students’ personalised avatars visiting locations around Australia and the world with well-known Aboriginal National Rugby League players as their guides. The guides spoke about the cultural significance of the location and delivered key health messages.

The average daily steps for all participating classes were updated live on the website; this enabled classes to track their placing relative to other participating classes. At the end of the Challenge, monetary prizes for the purchase of school equipment were awarded to first-, second- and third-placed classes.

An evaluation of the Challenge was undertaken with all participating schools from May to August 2014. The evaluation was approved by the Aboriginal Health and Medical Research Council Ethics Committee and the NSW Department of Education and Communities. The evaluation utilised a mixed-methods design and had three components: a pre- and post-Challenge physical activity survey; qualitative interviews; and classroom discussion.

In the week before the Challenge and again at the end of the Challenge students completed a survey assessing their self-reported levels of physical activity and sedentary behaviour in the previous week. The survey was self-complete, with teachers providing assistance only. Physical activity was assessed using the Physical Activity Questionnaire for Older Children (PAQ-C).²¹ The PAQ-C is a self-administered questionnaire designed to measure children’s moderate to vigorous physical activity (MVPA) over the preceding 7 days. The PAQ-C comprises 9 items, which are used to compute a summary score of 1 (low level of moderate-vigorous physical activity) to 5 (high level of moderate-vigorous physical activity). The PAQ-C has been validated for use with children aged 9–14 years, with demonstrated good internal consistency, suitable test-retest reliability and moderate correlation with perceptions of athletic competence (r = 0.48), teachers’ rating of physical activity (r = 0.45) and physical activity measured by accelerometry (r = 0.39).²² Screen time was assessed as a proxy for sedentary behaviour, using the validated screen time question from the NSW Schools Physical Activity and Nutrition Survey 2010,²³ in which screen time includes watching TV, watching DVDs/videos, using a computer for fun and using a smartphone or tablet for fun. Although screen time does not cover all sedentary behaviour, recreational screen use is the primary contributor to total sedentary behaviours outside school hours among young people.²³ Paired t-tests and McNemar’s Chi-square tests were conducted for the outcomes of interest, using data for students who completed both pre- and post-Challenge surveys. A significance level of P < 0.05 was used. Data were also compared with national screen time guidelines.²⁴

After the Challenge, semi-structured interviews were conducted with class teachers (n = 11). Teachers were interviewed face-to-face (n = 7) or by telephone (n = 4). Interview questions related to how teachers implemented the Challenge in their classroom, the class response to the Challenge, elements that worked well and those that did not, positive and negative impacts, adequacy of support provided to teachers for implementation and possible future improvements. Interviews were audio-recorded and transcribed with interviewee consent. Two teachers declined to be recorded; for these teachers, comprehensive notes were taken during the interview and subsequently analysed. Braun and Clarke’s²⁵ approach to thematic analysis was utilised, with analysis conducted by two members of the research team (CDH and MR). Initial coding and thematic mapping was undertaken by an analyst (MR) not involved in the interviewing; these were then collaboratively reviewed and refined with the second analyst (CDH), with final analysis occurring during report writing by CDH.

In the last week of the Challenge, students’ experiences and perceptions of the Challenge were explored in a 30-min classroom-based discussion led by an Aboriginal researcher. Classroom discussions were conducted in 10 classes at five 5 schools (Table 1: schools A, D, E, F, I), selected to reflect varying levels of remoteness and levels of participation in the Challenge (indicated by class average pedometer readings). The discussion took place during regular class hours and all students present in class that day took part. Students were asked about their experiences of the Challenge, what they liked most and least, suggestions for improvement and how they thought the program impacted their level of physical activity. Notes were taken during the discussions and analysed thematically.

In total 322 students registered for the Challenge. Of these, 318 provided demographic and health outcome data and are included in subsequent analyses. All students (100%) in each participating class provided parental consent to participate in the Challenge. There were an equal number of boys and girls (159 each). The average age of students was 10.9 years (s.d. = 0.88) and 57% of students identified as an Aboriginal person (Table 2).

Table 2.  Culture Health Communities Activity Challenge participant profile

Two hundred and nine students (66% of participating students) completed both pre- and post-Challenge surveys. Survey results for screen time and physical activity are presented below.

Results from the pre- and post-Challenge student survey indicate average daily screen time on weekdays decreased slightly, from 3.0 h pre-Challenge to 2.7 h post-Challenge, however this was not statistically significant (Table 3).

Table 3.  Average screen time (hours per day) on a usual weekday and weekend day, by demographic characteristics

For all students, average daily screen time on weekend days decreased significantly from 3.9 h pre-Challenge to 3.2 h post-Challenge (t = 2.64, 204 degrees of freedom (d.f.), P = 0.009, 95% confidence interval (CI) 0.17–1.21). When stratified by gender, the decrease in weekend screen time was statistically significant for girls but not boys (girls: t = 3.07, 112 d.f., P = 0.003, 95% CI 0.36–1.69). When stratified by Aboriginality, the change in weekend screen time was not statistically significant for either Aboriginal or non-Aboriginal students. The proportion of students spending more than 4 hours on screen time on weekend days reduced significantly from 35% pre-Challenge to 26% post-Challenge (McNemar’s χ² = 5.3, P = 0.021), and on weekdays decreased from 31% to 25% (not statistically significant). However, there was no change in the proportion of students meeting screen time guidelines of no more than 2 hours of screen time on a usual day (pre-Challenge 50%, post-Challenge 49%, not statistically significant).

For all students, there was a slight but statistically significant increase in students’ mean PAQ-C score, from 3.2 pre-Challenge to 3.3 post-Challenge (t = 2.30, 207 d.f., P = 0.023) (Table 4). When stratified by gender, pre-Challenge PAQ-C scores were higher for boys compared with girls and there was a statistically significant increase in PAQ-C scores pre- to post-Challenge for girls but not boys (girls: pre-Challenge 3.1, post-Challenge 3.3, t = 2.51, 114 d.f., P = 0.014, mean change 0.16, 95% CI 0.03–0.29). When stratified by Aboriginality, pre-Challenge PAQ-C scores were higher among Aboriginal students compared with non-Aboriginal students and there was a statistically significant increase in PAQ-C scores pre- to post-Challenge for non-Aboriginal students but not for Aboriginal students (non-Aboriginal students: pre-Challenge 3.0, post-Challenge 3.2, t = 3.0, 87 d.f., P = 0.004, 95% CI 0.07–0.34).

Table 4.  PAQ-C scores by demographic characteristics

The PAQ-C includes individual items asking children how often they were physically active during school time and out of school hours. Notable changes from pre- to post-Challenge include increases in the proportion of students who, in the previous 7 days, were always active during physical education classes (40% pre vs 52% post, n = 206); were very active ≥4 days per week in the afternoon after school (43% vs 52%, n = 205); and were physically active in their free time ≥5 times (28% vs 36%, n = 182). There was very little change in the proportion of students who reported they ‘ran and played hard most of the time’ at recess (38% pre vs 40% post) or lunchtime (39% pre vs 36% post).

Qualitative feedback from students and teachers indicated high levels of engagement and satisfaction with the Challenge. All teachers reported that engagement was high and that most students were highly motivated to participate in the Challenge. Illustrative quotes from teachers are provided below. Teachers reported that the Challenge led to a greater focus on incorporating physical activity into daily class routines. Teachers reported boosting physical activity among students by increasing in-class physical activity (e.g. through additional games, outdoor breaks and extra sport sessions), organising lunchtime activities (such as walking groups) and generally encouraging their students to be active. Teachers utilised opportunities to integrate the Challenge into lesson plans, including Personal Development, Health and Physical Education, Human Society and its Environment and mathematics lessons:

The kids have learnt how to calculate averages just through looking at the steps from the Challenge.

Students reported increasing their physical activity during recess and lunchtime to increase their number of steps, to beat their friends and improve their personal best. Competition among friends and between classes in schools where more than one class participated, was identified by students and teachers as a key motivator:

They knew they had to keep moving and playing the games to be able to keep up with the others in the class.

Students and teachers reported benefits including increased motivation to be active, increased knowledge of the benefits of physical activity and increased physical activity. Teachers reported the Challenge was particularly popular amongst girls and students who were usually less active:

Kids are forming little groups – ‘Oh let’s go off and do this to get our steps at lunchtime’. So socially it was good. Especially girls, there’s quite a few girls that got together in groups.

One [student] in particular, he does not do any sport at all and he wanted to be involved and he’d put [the pedometer] on and he’d walk around the oval, and that’s the most sport I think he’s done since he’s been in kindergarten.

Teachers reported the physical activities performed during class time also addressed fatigue and helped students re-engage with classroom learning. Teachers also reported improvements in student engagement during the Challenge, including increased attendance, improved behaviour and improved social interaction and teamwork linked to students’ working together to increase their steps:

One [example] that springs to mind is I had attendance issues with one of my students and when we started the Challenge those attendance issues were almost eliminated. This person was coming to school late in the mornings and stuff but then they started coming well before class time to pick up their pedometer and stuff.

I have a group of boys that are troubled and normally cause trouble in the playground and the Challenge motivated them to play differently and to organise activities in the playground to increase their steps.

Although all teachers reported that they were able to implement the Challenge successfully with their class, some difficulties were identified. Some students complained of pedometers restarting and breaking and of some students falsely accumulating steps by shaking or twirling their pedometer. Some teachers reported that recording and entering pedometer data online was sometimes a challenge, related to the time required to enter data and slow internet connection speeds. A few teachers noted that the level of numeracy and literacy required to complete the printed guidebook was too high for some students.

Results indicate participation in the Activity Challenge was associated with a significant reduction in weekend screen time during the Challenge period and a slight but statistically significant increase in physical activity using validated self-reported measures.

The quantitative findings suggest participation in the Challenge was associated with increased physical activity among girls; this finding was reinforced by qualitative evidence that the Challenge appealed to girls. Girls in this age group are significantly less likely than boys to meet Australian guidelines for levels of physical activity²³ and are less likely to participate in organised sport.²⁶ Participation in walking-based activities that offer flexibility for students to choose how they increase their steps, such as the Activity Challenge, may be an alternative activity that appeals to girls in this age group.

Although the slight increase in physical activity scores was statistically significant, both the pre- and post-Challenge scores were indicative of intermediate levels of moderate-vigorous physical activity. The PAQ-C provides a physical activity ‘score’ scale of 1–5 and to our knowledge no studies have assessed the clinical significance of changes in PAQ-C score. However, when triangulated with other study findings (i.e. the observed decrease in weekend screen time and qualitative feedback from students and teachers), the findings suggest the Challenge had a meaningful positive impact on students’ activity levels. This is also consistent with program implementation data which indicated students’ individual average daily steps during school hours increased from 9368 in the first week of the Challenge to 10 315 in the final week of the Challenge (data not shown).

Teachers and students reported a range of educational benefits including increased school attendance, improved teamwork and improved classroom behaviour. Although these benefits were self-reported and not objectively verified, these benefits were highly valued by teachers and are consistent with previous research indicating school-based physical activity interventions can facilitate positive classroom behaviour.¹⁹ Future research may seek to objectively measure the educational benefits of this program.

The lack of rigorous evaluation of healthy lifestyle interventions among Aboriginal people is well recognised, despite many programs having been implemented in Aboriginal communities.^27,28 To our knowledge this is the first pilot of a pedometer program targeting schools with high proportions of Aboriginal students. The high rate of participation of Aboriginal students and the positive feedback from teachers about the benefits for all students involved, points to the success of the program in engaging Aboriginal students and highlights the importance of the school setting for delivering health interventions for Aboriginal children.²⁹ While participation of Aboriginal students in the Challenge was high, the Challenge did not have a significant impact on levels of physical activity or sedentary time among Aboriginal students. This may be because Aboriginal students reported being more active and having less screen time before the Challenge. The findings may also be related to the use of the PAQ-C to measure physical activity and screen time, which has not been validated for use with Aboriginal children in Australia; therefore, it may not be a reliable and culturally relevant measure for use with Aboriginal children.

The study has several limitations, related to both the evaluation design and the intervention itself. In terms of the evaluation design, first, the quantitative assessment of physical activity had no comparison group. This was considered appropriate given this was a small-scale pilot program; however, it is possible the observed changes in physical activity and screen time may be related to external factors (e.g. weather, participation in extracurricular sport activities). A second limitation relates to the use of the PAQ-C to measure physical activity and screen time. As noted above, the PAQ-C is a self-report measure, it has not been validated with Aboriginal students, and the clinical significance of changes in PAQ-C score are unclear. However, the evaluation findings are strengthened by the mixed-methods design, with the triangulated data suggesting the Challenge encouraged students to be more active. A third limitation is the low response rate to the post-Challenge survey, with only 66% of participating students completed both the pre- and post-Challenge surveys. This is primarily because post-Challenge surveys were not completed by 3 of the 15 participating classes; the teachers indicated this was due to classroom time constraints. However, it remains possible that classes and teachers that were more engaged in the Challenge were more likely to complete the post-Challenge surveys, which may have inflated our estimates of the impact of the Challenge on physical activity and screen time.

In terms of the strengths and limitations of the intervention, key strengths of the Challenge model are that it is flexible and reasonably easy for teachers to implement. However, a potential limitation is that the Challenge is moderately resource intensive, requiring teacher engagement, physical resources (e.g. pedometers, computers) and coordination by a central implementation team.

Another strength of the study was the use of pedometers as an intervention component. Pedometers are inexpensive, easy to use and are considered to provide a valid measure of physical activity in children and young people.³⁰ Pedometers may also be particularly useful in school settings, as numerical pedometer counts can be used not only in physical education, but across a wide range of other subjects.³¹ However, although pedometers are an effective motivational tool to encourage physical activity,³² this may limit their utility as a measurement tool for intervention research, as observed changes in pedometer readings may reflect the motivational aspects of pedometer usage, rather than the effect of the intervention.³³

Other limitations include the inability of pedometers to provide a measure of the intensity of physical activity or total physical activity, and the fact that they are vulnerable to measurement error (e.g. students can falsely accumulate steps by shaking the pedometer).

Despite these limitations, the findings suggest pedometer-based physical activity programs are feasible to implement in primary school settings and may promote physical activity in students. Based on the findings of this pilot, it would be appropriate for the Challenge to be evaluated on a larger scale, utilising an appropriate comparison group, more rigorous measurement methods and potentially assessment of less resource-intensive modes of implementation.

The findings suggest school-based physical activity programs can be engaging and appropriate for classes with high proportions of Aboriginal students, and may increase physical activity levels and decrease sedentary time among some students during the program period.