nvidia-nemotron-3.5-super-vl

Overall

78.0

/ 100

Tasks

500/500

task suite

Avg latency

27648ms

round trip

Tokens used

92929

during evaluation

Per-category results

arithmetic 100.0 / 100

coding_light 75.0 / 100

data_manipulation 94.29 / 100

extraction 30.0 / 100

instruction_following 100.0 / 100

knowledge 85.0 / 100

language 50.0 / 100

Strongest category: language

About the suite

Fikra Eval 500 is one deterministic 500-task platform evaluation generated from a fixed seed: 110 arithmetic, 60 knowledge, 80 coding-light, 70 data manipulation, 70 language, 60 extraction and 50 instruction-following tasks. Same tasks every model, automatic scoring, published exactly when 500/500 completes.

suite :: fikra-eval-500 / v1

nvidia-nemotron-3.5-super-vl model page →

Sample task outputs (first 60 of 500)

#	Category	Result	Latency	Reply
1	arithmetic	✓	1615 ms	4195
2	arithmetic	✓	1680 ms	365
3	arithmetic	✓	1767 ms	3900
4	arithmetic	✓	2760 ms	2122
5	arithmetic	✓	1664 ms	19430
6	arithmetic	✓	1454 ms	236
7	arithmetic	✓	1517 ms	4185
8	arithmetic	✓	1710 ms	12502
9	arithmetic	✓	2571 ms	42693
10	arithmetic	✓	2555 ms	27216
11	arithmetic	✓	2564 ms	4583
12	arithmetic	✓	2658 ms	15634
13	arithmetic	✓	2571 ms	3756
14	arithmetic	✓	2561 ms	35
15	arithmetic	✓	2636 ms	326
16	arithmetic	✓	3377 ms	12029
17	arithmetic	✓	3399 ms	82
18	arithmetic	✓	3356 ms	239
19	arithmetic	✓	3105 ms	2727
20	arithmetic	✓	3194 ms	30590
21	arithmetic	✓	3478 ms	3001
22	arithmetic	✓	3442 ms	137
23	arithmetic	✓	3280 ms	1393
24	arithmetic	✓	3704 ms	23650
25	arithmetic	✓	4022 ms	4050
26	arithmetic	✓	3722 ms	5
27	arithmetic	✓	4758 ms	30848
28	arithmetic	✓	4008 ms	7341
29	arithmetic	✓	4057 ms	140
30	arithmetic	✓	4886 ms	7747
31	arithmetic	✓	4480 ms	1778
32	arithmetic	✓	4589 ms	2828
33	arithmetic	✓	4507 ms	1096
34	arithmetic	✓	5021 ms	140
35	arithmetic	✓	4825 ms	12240
36	arithmetic	✓	4821 ms	9612
37	arithmetic	✓	5297 ms	302
38	arithmetic	✓	5055 ms	5063
39	arithmetic	✓	5232 ms	3272
40	arithmetic	✓	6156 ms	334
41	arithmetic	✓	5422 ms	24960
42	arithmetic	✓	6149 ms	263
43	arithmetic	✓	5808 ms	38
44	arithmetic	✓	6324 ms	14989
45	arithmetic	✓	6153 ms	4069
46	arithmetic	✓	6490 ms	3412
47	arithmetic	✓	6229 ms	7215
48	arithmetic	✓	6147 ms	261
49	arithmetic	✓	7560 ms	108
50	arithmetic	✓	6956 ms	231
51	arithmetic	✓	6867 ms	24060
52	arithmetic	✓	7131 ms	1443
53	arithmetic	✓	7018 ms	13617
54	arithmetic	✓	6978 ms	19228
55	arithmetic	✓	7238 ms	18760
56	arithmetic	✓	7870 ms	5286
57	arithmetic	✓	7694 ms	281
58	arithmetic	✓	7796 ms	42816
59	arithmetic	✓	7674 ms	4326
60	arithmetic	✓	7733 ms	8552