Most fairness evaluation happens once, offline, before a model ships. ARIA audits every single response, inline, the moment it's generated.
Almost every fairness benchmark works the same way: run the model against a fixed test set once, report a score, ship. ARIA treats fairness as something to check continuously instead — on the actual, free-form text a model produces in the wild, not just on a curated benchmark it may have been tuned against.
A one-time benchmark score tells you nothing about the response a model gives a real user tomorrow. ARIA closes that gap by generating counterfactual variants of the input — swapping demographic references while holding everything else constant — and comparing how the model's output changes. If the response shifts in ways unrelated to the actual question, that's flagged before the user ever sees it.
Disparate impact and equalized-odds-style metrics are computed inline, per response, across the counterfactual set — entirely on-device, with no call to an external API and no stored benchmark to game.
Evaluated once, offline, against a fixed test set — blind to anything that drifts after deployment.
Audits the actual response, every time, in ~1.2s — catching failures a one-time benchmark structurally cannot see.
Same model, same deployment — the difference is checking once versus checking continuously
ARIA is also the audit engine running live inside the ARIA Assistant voice agent — the same methodology studied here on paper runs inline, in real time, on consumer hardware. Fairness becomes a property the system checks on itself continuously, not a certificate issued once and forgotten.