Free, standard and frontier by category
How each of the three models phantom-1 picks from did on its own, answering every task in the October 5, 2026 runs.
Main set: 295 everyday tasks
Each model answered all 295 tasks once. The table shows the tasks it passed and what they cost a Phantom caller.
| Category | FreeLing 3.1 Flash Free | StandardDeepSeek V4 Flash | Standard nowDeepSeek V4.1 Flash | FrontierClaude Sonnet 5.5 | ||||
|---|---|---|---|---|---|---|---|---|
| Passed | Cost | Passed | Cost | Passed | Cost | Passed | Cost | |
Formatting Generated extraction and reshaping jobs | 40/40 | $0 | 31/40 | $0.00408 | 34/40 | $0.00650 | 40/40 | $0.082 |
Small functions HumanEval and MBPP | 68/70 | $0 | 67/70 | $0.015 | 67/70 | $0.023 | 68/70 | $0.194 |
Bug fixing HumanEvalFix, and bugs written for the benchmark | 39/40 | $0 | 37/40 | $0.00687 | 40/40 | $0.032 | 40/40 | $0.125 |
Algorithms APPS interview and competition problems | 26/45 | $0 | 27/45 | $0.035 | 32/45 | $0.201 | 44/45 | $0.556 |
Concurrency Questions written for the benchmark | 33/35 | $0 | 33/35 | $0.00858 | 35/35 | $0.021 | 35/35 | $0.092 |
Math GSM8K, and MATH at level 5 | 35/40 | $0 | 38/40 | $0.020 | 40/40 | $0.049 | 40/40 | $0.121 |
Long context Generated Python modules and standard library files | 25/25 | $0 | 25/25 | $0.017 | 25/25 | $0.021 | 25/25 | $2.819 |
| All tasks | 266/295 | $0 | 258/295 | $0.107 | 273/295 | $0.353 | 292/295 | $3.990 |
What stands out
The free model passed as many tasks as DeepSeek V4 Flash or more in 5 of 7 categories, at no cost. Of its 29 failed tasks, 25 were replies cut off at the output limit, with no answer.
Frontier passed at least as many tasks as the other two in every category. Its lead was widest in algorithms: 44 of 45, against 26 for free and 27 for standard.
All three passed every long-context task. Frontier charged $2.819 for them, standard $0.017 and free nothing.
DeepSeek V4.1 Flash passed 273 of 295 tasks, against 258 for DeepSeek V4 Flash, and cost $0.353 against $0.107. It gained most in algorithms, from 27 to 32.
Hard set: 357 tasks from public benchmarks
Problems from BigCodeBench-Hard, LiveCodeBench, AIME and MMLU-Pro. Each model answered every task once.
| Benchmark | FreeLing 3.1 Flash Free | StandardDeepSeek V4 Flash | FrontierClaude Sonnet 5.5 | Only frontier | None passed | |||
|---|---|---|---|---|---|---|---|---|
| Passed | Cost | Passed | Cost | Passed | Cost | |||
BigCodeBench-Hard Python functions that call libraries | 19/60 | $0 | 22/60 | $0.035 | 36/60 | $0.459 | 14 | 22 |
LiveCodeBench AtCoder and LeetCode contest problems | 63/125 | $0 | 54/125 | $0.173 | 113/125 | $4.596 | 34 | 9 |
AIME The 2025 and 2026 exams | 40/60 | $0 | 31/60 | $0.191 | 59/60 | $1.248 | 10 | 1 |
MMLU-Pro Multiple choice across 14 subjects | 86/112 | $0 | 92/112 | $0.029 | 99/112 | $0.487 | 4 | 8 |
| All tasks | 208/357 | $0 | 199/357 | $0.428 | 307/357 | $6.790 | 62 | 40 |
What stands out
Frontier passed the most tasks on every benchmark. It led by the most on LiveCodeBench, with 113 of 125 against 63 for free and 54 for standard, and next on AIME, with 59 of 60 against 40 and 31.
The free model passed more than the standard model on LiveCodeBench and AIME, and fewer on BigCodeBench-Hard and MMLU-Pro. It also failed to answer: its last attempt ended in an error on 79 tasks, and those count as fails.
On 62 tasks only frontier passed. On 40, no model did.
Each model answered each task once, at the default temperature. A pass means the answer passed the task’s tests or matched the expected answer exactly. Cost is what a Phantom caller would pay for the same tokens. The benchmark report has the method and scores each routing strategy on these answers.
Source: Phantom’s benchmark runs of 5 October 2026. This page reads the results when it is built.