Signer-Dialect Robustness in Landmark-Only Sign Recognition: A 21-Fold Leave-One-Signer-Out Study

Abstract

A single cross-signer accuracy number hides who a sign-recognition model actually serves. We run a 21-fold leave-one-signer-out study on the Google ISLR 250-sign dataset, holding out each of the 21 signers in turn and reporting the full per-signer distribution rather than a pooled average. Top-1 accuracy ranges from 25.6% on the worst-served signer to 64.2% on the best — a 38.6 percentage-point spread — with a mean of 41.7% and a per-signer standard deviation of 11.4 points, more than ten times the seed-to-seed deviation of the same architecture. Five of twenty-one signers fall below 30%. We pre-registered three hypotheses: that the spread would exceed 25 points (confirmed, 38.6), that the worst signer would fall below 30% (confirmed, 25.6%), and that handshape complexity would predict per-signer degradation (refuted, r-squared 0.008). The degradation is signer-driven, not sign-feature-driven: the same sign swings 60 to 82 points between the best and worst signers. We argue the per-signer distribution, not the pooled mean, is the honest unit of report for sign recognition, and that signer-level metadata must be captured in any future data collection.