A model card that wins some benchmarks and loses others

DeepSeek's newest model card claims wins over Opus 5 on some benchmarks and big losses on others.

DeepSeek-V4.1-Flash-1017 is on Hugging Face under an MIT license. It updates V4.1-Flash, and the card says it cuts task time and output length while improving on reasoning, coding and agent tasks.

Three rows from the card's own table (all self-reported):

DeepSWE v1.1: 77.0 for DeepSeek, 74.0 for Opus 5, 73.0 for GPT-5.6 Sol Terminal-Bench 4.0: 42.3 for DeepSeek, 51.8 for Opus 5, 39.9 for GPT-5.6 Sol HLE without tools: 42.6 for DeepSeek, 56.3 for Opus 5, 44.5 for GPT-5.6 Sol

Same model, three different stories. The footnote matters too. Terminal-Bench ran on the Claude Code harness and DeepSWE on a different one, Mini-SWE-V2. Change the harness and the scores move.

My read: strong at repo-level code fixes, weaker on hard reasoning. But a vendor table is a starting point, not a verdict. Run it on your own tasks before you switch anything.

Which matters more for your work: a lead on code fixes, or the broad reasoning score?

Sources

#DeepSeek#AI#ArtificialIntelligence#MachineLearning

New notes go out on LinkedIn first. Building something with AI? We can help.

Diagram card: A model card that wins some benchmarks and loses others