What Beat Claude Wasn't the Model

Security scanning company Semgrep posted an article titled 'We Have Mythos at Home.' Mythos is the name of Anthropic's top-tier model, and the title references a self-deprecating meme about having a cheap knockoff of an expensive frontier model at home. The gist: Chinese company Z.ai's open-weight model GLM-5.2 beat Claude Opus with nothing but a bare prompt, no tools. But if you read the same article to the end, the conclusion changes.

The test task was detecting a vulnerability called IDOR. It's a hole where an app exposes internal identifiers like user IDs in requests but doesn't actually verify whether that user has access permissions. They measured the ability to find this in real open-source app code using F1 scores. Looking at results alone, GLM-5.2 scored 39%, ahead of Claude Code's Opus 4.6 (37%) and 4.8 (28%). The cost to find one vulnerability was approximately $0.17, one-sixth that of frontier models.

That's the headline. What flips the story are the top two rows of the same table. First and second place went to GPT-5.5 (61%) and Opus 4.8 (53%), both run by Semgrep with endpoint exploration structure attached to their own harness. Claude didn't lose. It only trailed GLM under the specific condition of bare prompts with all tools stripped away. The same Opus 4.8 jumped from 28% to 53% depending on the harness. That's why the author named 'harness matters more than model' as the first conclusion.

GLM-5.2 itself is a serious piece of work. It's an open-weight model with parameters released under MIT license. Out of 750 billion total parameters, it uses an MoE structure that activates only 40 billion per token, and extended context to 1 million tokens. However, open-weight is not open source. Training data and pipeline remain proprietary. Z.ai itself disclosed that GLM-5.2 exhibits more reward hacking behavior than the previous version.

For Busan founders, these results read in two directions. One is freedom. For teams considering their own infrastructure like domestic NPU data centers, the option to run MIT-licensed open-weight on-premises has opened with real measurements, throwing off both API dependency and data leakage concerns at once. GLM-5.2's structure extends context to 1 million tokens, making it suitable for exhaustive analysis of local codebases. The other is warning. Just swapping in a cheaper model doesn't bring performance along. Teams that code their own validation harnesses and scoring pipelines create the gap. The place to attach isn't the model itself but the thin tool layer above it.

The author summarizes three conclusions: harness matters more than model, don't put all eggs in one basket, and open-weight has reached a practical threshold. The last line is honest. It notes this is one task, one dataset, one run—don't generalize. But the direction itself is clear. In a world where model prices have dropped to one-sixth, the battleground shifts from which model you choose to what you build on top of it. In an era when cheap engines are everywhere, what's valuable is the hand that builds the chassis.