Original accuracy
95% confidence intervals
Correct answers did not always stay correct after renaming. 53.3%–81.5% of initially correct answers stayed correct after degree-reversed renaming, depending on the configuration.
Hard v1 · results snapshot 2026-11-03 · 16,384-token cap; costs include recovery attempts.
New: Claude’s triangle accuracy fell from 86.5% to 62.5% after renaming → 96 fresh graphs · zero cutoffs. Separate from Hard v1.
5 evaluated configurations · full release · original accuracy, highest first
Scroll sideways to see all scores →
| Model / provider | Original accuracy | Renaming survival | Cutoffs | Coverage | Cost / 1,000 |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flashsail-research/fp4 · FP4 | 73.3%n=120 | 73.9%n=88 initially correct answers | 32.1%308/960 | 960/960 | $1.61 |
| MiniMax M3.1 Flashtenstorrent/fp8 · FP8 | 67.5%n=120 | 81.5%n=81 initially correct answers | 21.5%206/960 | 960/960 | $1.58 |
| GLM-5.3baidu/fp8 · FP8 | 63.3%n=120 | 80.3%n=76 initially correct answers | 38.8%372/960 | 960/960 | $9.42 |
| Ling 3.1 Flashnovita · UNKNOWN | 39.2%n=120 | 76.6%n=47 initially correct answers | 62.2%597/960 | 960/960 | $3.07 |
| Granite 4.2 8Bcoreweave/bf16 · BF16 | 12.5%n=120 | 53.3%n=15 initially correct answers | 89.8%862/960 | 960/960 | $2.49 |
These are the full Hard v1 results. Enable JavaScript to use the filters and charts. Metric definitions and intervals · All scores (JSON)
Scroll sideways to see all scores →
| Model / provider | Originally correct → still correct | Coverage |
|---|
95% confidence intervals
Only the labels change in this illustration. The edges and correct answer stay the same.
Protocol hard-v1-16k-8db578dd652c
SHA-256 8db578dd652caa37fcbe7ac745bfde5f18b111dc64d5156c209949c63315b4d7
We use high reasoning effort where supported, default sampling and no tools. Each model uses a fixed provider. Reasoning settings and token counts work differently across providers.
Download settings and file hashesHard v1 · results snapshot . Use this citation when referring to these results.
@misc{hu2026misgraphhardv1,
author = {Hu, Lifan},
title = {{MisGraph} Hard v1: Graph Reasoning under Node Renaming},
year = {2026},
howpublished = {\url{https://anormalm.com/misgraph/}},
note = {Hard v1, 16,384-token protocol. Results snapshot 2026-11-03. Protocol SHA-256: 8db578dd652caa37fcbe7ac745bfde5f18b111dc64d5156c209949c63315b4d7}
}