Rankings / Meta
Llama-4-Maverick-17B-128E-Instruct (open weights, self-hosted reference)
Usage share 0.06% · OpenRouter rankings API (daily token share, 2026-08-04)
Graded as the self-hosted reference deployment. The Mistral objection to company-level safety indexes applies here with full force: the deployer controls fine-tuning, guardrails, and data handling. Vectors that are deployer properties are marked as such.
Missing for Tier 2: external pre-deployment testing, third-party certification. The tier is computed from this checklist — satisfying these requirements moves the badge, automatically.
What happens to your data: training-on-customer-data defaults, retention windows, residency options, and the enterprise-versus-consumer terms gap. A legal property, not a capability — it survives every model generation.
Deployer property, not a property of the weights: in the self-hosted reference deployment, prompts and outputs never leave deployer infrastructure. Hosted providers' terms vary and are not graded here.
Whether it changes without warning: versioning discipline, changelog quality, deprecation policy, and observed silent changes. The signal no one else tracks.
Published immutable weights are the strongest possible versioning discipline: a given checkpoint cannot change silently, ever. Releases are explicit and diffable.
“We're making Llama 4 Scout and Llama 4 Maverick available for download today on llama.com and Hugging Face so everyone can continue to build new experiences using our latest technology.”
Whether an attacker can make it misbehave — direct jailbreaks against the model's own policies and indirect prompt injection in agentic tool use. Graded to the weaker of the two, because an attacker takes the easier path.
The bare model jailbreaks readily in independent testing. Meta ships Llama Guard and Prompt Guard as separate optional layers, but the graded artifact — the weights as released — carries little built-in resistance.
No built-in agentic injection hardening; Prompt Guard exists as an add-on classifier but is off by default and trivially omitted by deployers.
“Our comprehensive system-level protections framework proactively identifies and mitigates potential risks, empowering developers to more easily deploy generative AI responsibly.”
“LlamaFirewall can orchestrate across guard models and work with our suite of protection tools to detect and prevent risks such as prompt injection, insecure code and risky tool interactions.”
Whether you can see how it was built and tested: model cards, published safety evals, external pre-deployment testing, and disclosure of changes. The mechanism behind the tier ladder.
Model card and benchmark disclosures exist, but no external pre-deployment testing, and the launch-time LM Arena benchmark episode (an experimental variant submitted to the public leaderboard) damaged disclosure credibility.
“Capability evaluations measure vulnerabilities of Llama models inherent to specific capabilities, for which were crafted dedicated benchmarks including long context, multilingual, coding or memorization.”
Whether it is certified and compliant: SOC 2, ISO/IEC 42001, HIPAA eligibility, EU AI Act readiness, and audit availability.
Deployer property: no certification attaches to open weights; compliance posture is entirely inherited from whichever deployer or host runs the model.
No regional pinning — the provider chooses where data is processed.
Self-hosted reference deployment: data never leaves deployer infrastructure; hosted providers' terms vary and are not graded here.
No consumer tier exists for the self-hosted reference deployment; data handling is deployer-controlled.
Level tiers can mean both are clean, or that the API tier is itself weak with nothing better to compare against.
One tracked change, on 2026-08-04 — 1 days before the as-of date (2026-08-05). A single event cannot establish a cadence, so days-since is shown without a baseline.
No dated score readings recorded for this model yet. Readings are only entered where multiple real, dated third-party values exist — never interpolated.
retrieved 2026-08-03 — 2026-08-05
Meta's leaderboard submission was an experimental chat-tuned variant, not the released weights, inflating public perception of the released model. Recorded under transparency: disclosure practices are part of the risk surface buyers rely on.
Outcome: LM Arena updated its policies; Meta acknowledged the variant difference.
Tiers are now derived from a per-requirement checklist rather than assigned, and not-applicable / under-review became first-class grade states excluded from the composite score. Deployer-property vectors on open-weight models (data handling, compliance) moved from graded to not-applicable, changing the composite scores of the tagged models.
Compare this model: Llama 4 Maverick + open compare view →