Enterprises deploying large language models (LLMs) at scale face a costly trade-off: safety classifiers that screen inputs for jailbreak attempts almost exclusively rely on GPU-based models, such as fine-tuned transformers and LLM-as-a-judge pipelines. According to a new preprint by researchers Majhi, Vasudev, Gupta, Dhruv, Singh, Advait, Barker, and Kumar, this GPU-centric assumption may be unnecessary for the majority of traffic.
The Research Setup
The study, published on arXiv (ID 2512.19011), evaluates five CPU classifier families (including support vector machines and gradient-boosted trees trained on TF-IDF features) against GPU-based models: Mamba-130M as an SSM-based GPU classifier, and transformer models DeBERTa-v3 and Gemma-2B with LoRA. Testing covered nine jailbreak sources across three regimes: in-distribution (D1), out-of-distribution (D2), and adversarially obfuscated (D3).
Key Findings
The results reveal complementary failure modes between CPU and GPU classifiers. On D1, the best CPU classifier matches the best transformer GPU model at roughly one-fifth the deployment cost. However, on D2, CPU classifiers fail via "confident miscalibration," producing high-confidence false negatives that bypass escalation entirely. On D3, CPU classifiers outperform transformer GPU models by more than 26 percentage points in F1.
| Regime | CPU Classifier Performance vs. GPU | Key Observation |
|---|---|---|
| In-distribution (D1) | Matches best transformer at ~1/5 cost | CPU is cost-effective |
| Out-of-distribution (D2) | Fails via confident miscalibration | High-confidence false negatives |
| Adversarially obfuscated (D3) | Outperforms GPU by >26 F1 points | CPU more robust |
GuardChain: A Multi-Stage Pipeline
Based on these failure modes, the authors designed GuardChain, a three-stage safety pipeline: Regex -> CPU -> GPU. The pipeline routes each prompt to the cheapest stage capable of a confident decision. According to the paper, the CPU stage alone resolves 80% of in-distribution prompts at near-peak accuracy, while the GPU stage recovers the out-of-distribution failures.
Implications for Enterprise Deployment
For practitioners deploying LLM safety at scale, this work provides evidence that GPU-class infrastructure is unnecessary for the majority of traffic. The GuardChain approach could significantly reduce infrastructure costs for enterprises running high-volume LLM applications, where only a fraction of prompts require the computational expense of GPU inference. The study suggests that CPU-based classifiers, when integrated into a staged pipeline, offer a practical and economical alternative without sacrificing overall safety accuracy.
The research did not disclose specific cost savings per query, but the one-fifth deployment cost ratio on in-distribution data implies substantial savings for organizations handling millions of daily prompts. The authors' results indicate that a mix of CPU and GPU stages, rather than a monolithic GPU model, can achieve robust safety enforcement across diverse threat scenarios.
