Research
Essay · Field comparison

How to tell if a concussion test is gaming-resistant.

SCAT5, ImPACT, Sway, and King-Devick compared on the one property that matters: can a motivated athlete defeat the test.

May 18, 20269 min read

I faked impact tests in high school and college. I gamed the balance screens and I gamed the Sway protocol on my phone. Most of the guys I played with did some version of the same thing. The athletic trainers running those tests were good people doing the job right. The problem was the test itself. It asked teenagers who loved a sport to flag themselves as too hurt to play, and we lied. Of course we lied.

That is the part of concussion testing nobody likes to say out loud. The tools that became standard between the mid-nineties and today (SCAT, ImPACT, Sway, the balance error count, the symptom checklist) all share one design assumption: ask the brain to grade the brain. Press a button, hold a stance, rate your headache from zero to six. The athlete decides how to answer. The athlete has every reason to answer one way at baseline and a different way after a hit. The literature has been clocking the result for a decade. Around half of injuries slip past.[1][2] Sub-concussive impacts compound over a career without anyone keeping count. And most of what we know about chronic traumatic encephalopathy still comes from the autopsy table.[3]

So when I use the word "gaming-resistant" in this essay, that is the problem I mean. A test that still works on the only athlete a concussion test actually has to work on: the one who does not want to come out of the game. Below: what gaming-resistance is in measurement terms, how the four tools you have probably seen handle it (or fail to), and what to ask for when the next one lands on your school, your team, or your kid.

What "gaming-resistant" really means

The question comes down to which physiological signals the test actually reads. Some you can control. Some you cannot.

Breathing. The speed of a button press. How long you hold a stance. What you tell the trainer when she reads you the symptom list. All voluntary. In a playoff game where the line of scrimmage is the rest of your senior year, you have plenty of motivation to move those numbers in whichever direction helps.

The latency of a saccade to a flashing target on a phone screen. The speed at which your pupil constricts when a light hits the retina. The small corrections your body makes underneath you while you try to stand still. The beat-to-beat variability of your heart rate. Those are involuntary. They run on brainstem reflex circuits the conscious mind has no wire into. The part of you that wants to stay in the game does not get a vote.

A gaming-resistant test anchors its decision on the second list, not the first. The rest of the design (how you capture the signal, how you score it, how you compare it back to a baseline) is downstream of that one choice.

One question

For any tool that calls itself a concussion screen, ask whether a motivated athlete can deliberately tank the baseline and walk through the post-injury comparison, or run the baseline hot and tank after a hit. If the answer is yes, it is not gaming-resistant. Sandbagging and post-injury concealment have been documented in the peer-reviewed literature for more than a decade.[1][2] A tool either measures around it or it does not.

SCAT5

The Sport Concussion Assessment Tool, fifth edition, is the document most athletic trainers reach for on a sideline. It is peer-reviewed, freely available, and in essentially universal use at the high school level and above. Just about every input on the form is voluntary.

The Symptom Evaluation is twenty-two self-rated items scored zero through six. The athlete sits down, the trainer reads the symptoms aloud, the athlete grades each one. Nothing about that section is objective. The Cognitive Screening (orientation questions, recall of five words, reciting months backwards) is voluntary task performance scored by the observer. The Balance Examination is the modified BESS, where the trainer counts errors by eye while the athlete holds three stances. Everything the athlete does on that page, she chose to do.

Used as a sideline checklist, SCAT5 is genuinely useful. It gives a trainer a structured way to slow down and pay attention. As an instrument with any gaming-resistant anchor, it has none. A practiced athlete can underrate the symptoms, overperform the cognitive piece through rehearsal, and hold the stances long enough to pass the count. I did all three, more than once. Teammates of mine did the same.

What SCAT5 measures

Symptom checklist, orientation, immediate memory, concentration, modified BESS balance, neck examination, coordination. All voluntary except the brief neurological screen.

How it is gamed

Underrate the symptoms. Memorize the symptom categories so nothing reads as new after a hit. Practice the cognitive items. Tense up in stance and shift weight off the supporting leg. None of this is subtle. All of it works.

ImPACT

Immediate Post-Concussion Assessment and Cognitive Testing is a computer battery that became the standard baseline-versus- post-injury comparison across U.S. high school and college athletics in the 2000s. It is FDA-cleared and used by most professional leagues. It is also the cleanest example in the sports medicine literature of what happens to a voluntary-input tool once an incentive walks into the room.

The battery measures verbal and visual memory, processing speed, and reaction time. Every input is a mouse click or a key press. The baseline gets recorded preseason. After a suspected hit, the post-injury run gets compared against that same athlete's preseason numbers. The size of the deviation drives return to play.

The "sandbagging" literature on ImPACT goes back to at least Erdal in 2012 and Schatz and Glatts in 2013.[1][2] Athletes sit down at the computer in August knowing exactly what they are doing. They underperform on reaction time without bottoming it out, so the software validity index does not flag them. Then in November, after a hit, they ride that same low baseline back through the comparison and clear.[1] Peer-reviewed review work has put the sensitivity of computerized concussion testing in the same ballpark as a coin flip, around fifty-two percent in some samples.[4]

What ImPACT measures

Verbal memory composite, visual memory composite, motor processing speed, reaction time composite, impulse control composite. All computer-administered, all voluntary inputs.

How it is gamed

Slow reaction time at baseline just enough to shift the distribution down, but not so far that the validity index screams. After a hit, run the post-injury comparison against that same lower baseline and clear "normal." The trainers I have talked to can name two or three kids per team who do it every season.

Sway Medical

Sway is the FDA-cleared mobile balance and reaction-time tool that turned the smartphone into a sideline device for concussion testing. It is in widespread use across high school, collegiate, and youth sports. Sway is the product we get compared to most often. Sensitivity by both the company's own published research and by independent peer review sits in the same band as the rest of the voluntary-input class. A meaningful share of real injuries pass the test.

Sway uses the accelerometer and gyroscope built into the phone to read body sway during a sequence of stances. It also has a reaction-time component where the athlete taps the screen on a visual stimulus. The protocol is rigorous, the engineering is clean, and the cost of deployment is essentially zero. The gaming problem has nothing to do with any of that. Sway's balance reading is a stance the athlete chose to hold. Sway's reaction time is a tap the athlete chose how fast to make. At baseline, the athlete can wobble more than necessary. After a hit, the athlete can tense up and stand stiller than she feels. The accelerometer does what accelerometers do. It measures whatever happens to be there.

What Sway measures

Sway path and velocity during stances via the phone's IMU. Reaction time to visual stimulus. Both voluntary.

How it is gamed

Move during the baseline stances enough to inflate the personal norm. Suppress motion on the post-injury test by tensing the postural muscles. Tap the screen a few milliseconds slower at baseline so the post-injury reaction time still falls inside the normal band. None of this needs equipment or coaching. It needs motivation, which the athlete has by definition.

King-Devick

King-Devick is a one-minute rapid number-naming test. The athlete reads sequences of numbers aloud as quickly as she can. After a concussion, the timing slows because the saccadic eye movements between numbers and the vocalization timing both depend on neural circuits the injury perturbs. King-Devick is FDA-cleared and in wide use.

Of the four tools in this comparison, King-Devick is the hardest to game, and it is worth knowing why. The saccades that step the eye from number to number are partly involuntary. You can hesitate before saying each number, sure, but the latency from one fixation to the next saccade is buried in a brainstem and frontal-cortex loop that the conscious mind only partly reaches. The output is still spoken aloud (so a determined kid can pace himself at baseline and accelerate afterward), but the underlying signal sits closer to the involuntary side of the line than the other three tests. That is the reason King-Devick has held up better than the rest in head-to-head comparison work.[5]

What King-Devick measures

Rapid number-naming time, dependent on saccadic eye movement chains and the speech-production circuit timing.

How it is gamed

Read slower at baseline to inflate the personal norm. The room for gaming is smaller than the other three, but it is not zero.

What gaming-resistant actually looks like

Five physiological signals you cannot fake, and one short note on why each one ends up on the list.

  • Pupillary light reflex. The pretectal olivary nucleus drives bilateral Edinger-Westphal output to the iris sphincter through parasympathetic fibers.[7] There is no voluntary motor pathway to the iris sphincter. You cannot will your pupil to constrict slower or faster. Concussion, brainstem dysfunction, and acute pharmacology all measurably shift constriction amplitude, latency, and peak velocity.[6]
  • Saccadic eye movement latency and peak velocity. Saccades are ballistic and pre-programmed. Once a saccade is initiated it cannot be aborted mid-flight.[9] Their latency tracks frontal-eye-fields and superior-colliculus integrity. Their peak velocity tracks brainstem motor circuit health. Slowing or impairment is a documented signature of mild traumatic brain injury.[8]
  • Postural micro-corrections during stance. The cerebellar vermis and the vestibular nuclei run a continuous predictive control loop to keep the center of mass over the base of support. The voluntary part is "stand here." The involuntary part is the thousand small corrections per minute your body makes to do that. The high-frequency band of those corrections shifts after concussion.[10] The athlete who stands still cannot still that.
  • Heart rate variability. Vagal autonomic tone measured between beats is the most-validated non-invasive proxy for autonomic balance in clinical research. Concussion blunts it.[11][12] Post-traumatic stress further blunts it. The athlete does not consciously control beat-to-beat variability, and tools that read it from facial-skin video (remote photoplethysmography) capture it without contact.[13]
  • Spontaneous blink rate. Driven by striatal dopamine tone.[14] Shifts with concussion-related autonomic and attentional change. Athletes notice they are being measured on the timed cognitive task; they do not notice they are being measured on how often they blink between stimuli.

Fuse enough of these signals into one calibrated index, compare the athlete back to herself over time, and you get a number that tracks her neurology and ignores her preferences about playing time. That is what we have been building for the past year and a half.

A note on us

Chronic Trace is research-stage. The platform is Software-as-a-Medical-Device in development. It is not FDA-cleared and it is not a diagnostic device. Nothing in this essay should be read as a clinical claim about our product. The argument is about the design property the rest of the field is missing. The clinical validation work that backs our specific implementation is still in flight.

What to ask for

Whether the test in front of you is ours, theirs, or something nobody has heard of yet, four questions will tell you most of what you need.

  1. Which involuntary biomarker channels does this assessment actually read? "We measure heart rate" is not enough if the rate is the only output. "We measure beat-to-beat heart-rate variability and the latency of pupillary constriction to a light pulse" is the kind of answer that counts.
  2. How does it tell when a baseline has been deliberately tanked? Specifically, what property of the data flags it? A tool with no answer to that question has no defense against the most common manipulation pattern in youth and collegiate sport.
  3. What is the inter-rater and inter-device variability? If the result depends on whether the trainer is generous with error counts, or on how the kid happens to hold the phone, the test is measuring the rater and the device, not the brain.
  4. What is the head-to-head sensitivity against a reference standard, and at what false-positive rate? Anything below a documented clinical reference is incomplete. Anything claimed without a reference is marketing.

Most of the tools in widest use today do not have clean answers to all four. That is fine. Asking is the point.

The bigger point

Every sport that has tried to take concussion seriously has ended up in the same place, with the same gap between what the assessment reports and what the biology is doing. It is not because the trainers, the clinicians, or the league officials are cutting corners. Most of the people I have worked with on sidelines care a lot. The gap exists because the tools they were handed depend on voluntary inputs from athletes who have every reason to lie. Move the assessment onto channels that cannot be lied to and the gap closes. Leave it alone and the gap stays exactly where it is.

This is not pessimism about concussion safety. The fix is engineering, not enforcement, and the engineering is no longer speculative. Decades of neuroscience tell us which circuits produce uncontrollable outputs. The hardware that captures them sits in everyone's pocket. The math that fuses them into one comparable index is well-trodden. What is in front of us is integration, validation, and deployment. That is the next few years of our work.

References

  1. Erdal K. Neuropsychological testing for sports-related concussion: how athletes can sandbag their baseline testing without detection. Arch Clin Neuropsychol. 2012;27(4):473-479. PubMed 22543569
  2. Schatz P, Glatts C. "Sandbagging" baseline test performance on ImPACT, without detection, is more difficult than it appears. Arch Clin Neuropsychol. 2013;28(3):236-244. PubMed 23299178
  3. Mez J, Daneshvar DH, Kiernan PT, et al. Clinicopathological evaluation of chronic traumatic encephalopathy in players of American football. JAMA. 2017;318(4):360-370. JAMA
  4. Resch JE, McCrea MA, Cullum CM. Computerized neuropsychological assessment tools in the management of sports-related concussion. Neuropsychol Rev. 2013;23(4):335-349. PubMed 24306202
  5. Galetta KM, Brandes LE, Maki K, et al. The King-Devick test and sports-related concussion: study of a rapid visual screening tool in a collegiate cohort. J Neurol Sci. 2011;309(1-2):34-39. PubMed 21849171
  6. Master CL, Podolak OE, Ciuffreda KJ, et al. Utility of pupillary light reflex metrics as a physiologic biomarker for adolescent sport-related concussion. JAMA Ophthalmol. 2020;138(11):1135-1141. JAMA Ophthalmology
  7. Belliveau AP, Somani AN, Dossani RH. Pupillary light reflex. In: StatPearls. Treasure Island (FL): StatPearls Publishing; 2024. NCBI Bookshelf
  8. Heitger MH, Jones RD, Macleod AD, Snell DL, Frampton CM, Anderson TJ. Impaired eye movements in post-concussion syndrome indicate suboptimal brain function beyond the influence of depression, malingering or intellectual ability. Brain. 2009;132(Pt 10):2850-2870. PubMed 19772996
  9. Sparks DL. The brainstem control of saccadic eye movements. Nat Rev Neurosci. 2002;3(12):952-964. PubMed 12461552
  10. Guskiewicz KM. Assessment of postural stability following sport-related concussion. Curr Sports Med Rep. 2003;2(1):24-30. PubMed 12831673
  11. La Fountaine MF, Heffernan KS, Gossett JD, Bauman WA, De Meersman RE. Transient suppression of heart rate complexity in concussed athletes. Auton Neurosci. 2009;148(1-2):101-103. PubMed 19345153
  12. Senthinathan A, Mainwaring LM, Hutchison M. Heart rate variability of athletes across concussion recovery milestones: a preliminary study. Clin J Sport Med. 2017;27(3):288-295. PubMed 27347863
  13. Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Opt Express. 2008;16(26):21434-21445. PubMed 19104573
  14. Jongkees BJ, Colzato LS. Spontaneous eye blink rate as predictor of dopamine-related cognitive function - a review. Neurosci Biobehav Rev. 2016;71:58-82. PubMed 27555290