Five checks before you trust a model card

Every model launch comes with a benchmark table where the new model is in bold. Before you believe it, run five checks. I'll use DeepSeek's V4.1-Flash-1017 card as the example, because it lists its setup.

Who ran the numbers? Most tables are self-reported. Independent leaderboards and your own tests count for more.

Which harness? That card notes Terminal-Bench 4.0 ran on the Claude Code harness and DeepSWE on Mini-SWE-V2. Change the harness and scores move.

Which competitors, and which versions? The table compares against 'Opus 5' and 'GPT-5.6 Sol', and Opus 5.5 is out now.

Where does it lose? The same card shows 42.6 on HLE without tools against 56.3 for Opus 5. Read the losing rows first.

Does the benchmark look like your work? A repo-level code benchmark says little about summarizing contracts.

Then do the only test that matters: run 10 to 20 real tasks from your own work, with the same prompts and tools, and compare cost and failures as well as accuracy. A model that wins your tasks beats a model that wins a table.

What's your quick test for a new model before it touches real work?