The Effect of Test Setting Upon Test-Retest Reliability of ImPACT Baseline Tests in High School Athletes

Objective To determine the test-retest reliability of ImPACT baseline tests across different schools within the same larger concussion management program.

Background ImPACT is the most widely used concussion management cognitive testing tool. Baseline testing is often required for high school sports participation. Typically, testing occurs every 2 years based on test-retest reliability statistics in previous studies. Demographic and environmental factors, such as age, sex, number of participants, and supervision, all impact baseline performance. Studies to date have not examined test-retest differences across testing sites, such as between school differences.

Design/Methods Cross-sectional retrospective design. Valid baseline tests from high school athletes over a 2-year interval were included. Participants who experienced concussions prior to or between tests were excluded. A total of 979 student athletes from 5 schools were included. The Intra-class correlations were determined over a 2-year period for each ImPACT composite score and school.

Results ICC estimates averaged between schools reflected good reliability for visual-motor speed (0.833), visual memory (0.673) and reaction time (0.615) over the two-year period. Verbal memory (0.586) and impulse control (0.556) were less reliable. Between schools a greater range of composite reliability was observed for reaction time (0.484–0.730) and impulse control (0.461–0.655) compared to verbal memory (0.534–0.637), visual memory (0.61–0.719), and visual-motor speed (0.769–914).

Conclusions As previously established, reliability of ImPACT baselines vary by composite. This study revealed that reliability also varies by setting, as different schools yielded different ICCs. Consistent with the literature, the most reliable measure was visual-motor speed. The greatest difference in reliability between schools was for reaction time. These results suggest that test setting and environment affect reliability of ImPACT baseline scores, with varying effects per composite. Attention must be paid to environmental setting to improve reliability of baseline cognitive test performance to maximize athlete safety.