Fikra Eval 500: 11 open models ranked.

Fikra Eval 500 is a deterministic 500-task suite covering arithmetic, numeric reasoning, knowledge, coding-light tasks, data manipulation, language, extraction and instruction following. We ran it across 11 open-model chat SKUs on the Fikra catalog. Every result is stored with per-category scores and latency.

What the numbers say

• The top is close. MiMo, GLM 5.3 Flash and Kimi sit within 1 point. Model choice should come down to price, latency and task fit, not the headline score alone.
• Arithmetic is solved by most. Ten of the eleven models score near 100 on arithmetic. Step 5 Preview is the outlier at 0.
• Extraction is the weak category. Most models score between 15 and 36 on extraction. This is exactly where fixed-price tools and a stricter schema help.
• Latency is not chat TTFT. The suite allows 512 output tokens per task and some models spend most of that on reasoning. Average task latency is useful for batch and agent work, not for first-token chat speed.
• Step 5 Preview is not production-ready. 8.2/100 and 546 seconds average. It stays visible as a preview and a failure, not a recommendation.

Method

Fixed seed, 500 tasks, temperature 0, 512 max output tokens, eight parallel workers, direct upstream calls. Scoring is deterministic per task. The full per-category results are on each model page. The suite is a Fikra benchmark, not MMLU, GPQA or SWE-bench, and it should not be presented as one.