Grok 4.6 tops BBEH Mini benchmark, scoring 0.67 on 460 tasks

3 models · 460 tasks · suite: BBEH Mini

In a 460-task BBEH Mini evaluation, SpaceXAI’s Grok 4.6 led the field with a 0.67 score at $0.0034 per task. Anthropic’s Claude Opus 4.8 followed at 0.62, while OpenAI’s GPT-5.6 Sol Pro posted 0.59.

Rank	Model	Mean score (0-1)	Est. cost/task
1	SpaceXAI: Grok 4.6	0.665	$0.0034
2	Anthropic: Claude Opus 4.8	0.620	$0.05
3	OpenAI: GPT-5.6 Sol Pro	0.587	$0.0089

## Leaderboard | Rank | Model | Mean score | Est. cost/task | | --- | --- | --- | --- | | 1 | SpaceXAI: Grok 4.6 | 0.67 | $0.0034 | | 2 | Anthropic: Claude Opus 4.8 | 0.62 | $0.0485 | | 3 | OpenAI: GPT-5.6 Sol Pro | 0.59 | $0.0089 | ## How we scored it Every model answered the same 460-task battery from **BBEH Mini** (BBEH — Google DeepMind (Apache-2.0)), one task at a time, with no tools and no retries on content. 460 tasks carry ground truth (exact, numeric, multiple-choice, or the benchmark's official matcher) and were scored deterministically against the reference answer — correct is 1, incorrect is 0. A model's **mean score** averages its graded tasks; a generation failure counts as 0. **Cost per task** is estimated from each model's published per-token pricing ("—" where pricing isn't public), so treat it as directional, not billing-exact. --- Explore every prompt, answer, and per-task grade in the [interactive leaderboard](/benchmarks/bbeh-mini-leaderboard).

Tasks

bbeh_spatial_reasoning #001 (bbeh)
bbeh_zebra_puzzles #002 (bbeh)
bbeh_dyck_languages #003 (bbeh)
bbeh_sportqa #004 (bbeh)
bbeh_zebra_puzzles #005 (bbeh)
bbeh_dyck_languages #006 (bbeh)
bbeh_multistep_arithmetic #007 (bbeh)
bbeh_shuffled_objects #008 (bbeh)
bbeh_web_of_lies #009 (bbeh)
bbeh_object_counting #010 (bbeh)
bbeh_geometric_shapes #011 (bbeh)
bbeh_disambiguation_qa #012 (bbeh)
bbeh_causal_understanding #013 (bbeh)
bbeh_boolean_expressions #014 (bbeh)
bbeh_linguini #015 (bbeh)
bbeh_nycc #016 (bbeh)
bbeh_multistep_arithmetic #017 (bbeh)
bbeh_nycc #018 (bbeh)
bbeh_sportqa #019 (bbeh)
bbeh_shuffled_objects #020 (bbeh)
bbeh_geometric_shapes #021 (bbeh)
bbeh_object_properties #022 (bbeh)
bbeh_web_of_lies #023 (bbeh)
bbeh_boardgame_qa #024 (bbeh)
bbeh_nycc #025 (bbeh)
bbeh_zebra_puzzles #026 (bbeh)
bbeh_shuffled_objects #027 (bbeh)
bbeh_movie_recommendation #028 (bbeh)
bbeh_sportqa #029 (bbeh)
bbeh_time_arithmetic #030 (bbeh)
bbeh_multistep_arithmetic #031 (bbeh)
bbeh_nycc #032 (bbeh)
bbeh_hyperbaton #033 (bbeh)
bbeh_multistep_arithmetic #034 (bbeh)
bbeh_linguini #035 (bbeh)
bbeh_sportqa #036 (bbeh)
bbeh_time_arithmetic #037 (bbeh)
bbeh_object_properties #038 (bbeh)
bbeh_boolean_expressions #039 (bbeh)
bbeh_sarc_triples #040 (bbeh)
bbeh_hyperbaton #041 (bbeh)
bbeh_geometric_shapes #042 (bbeh)
bbeh_dyck_languages #043 (bbeh)
bbeh_buggy_tables #044 (bbeh)
bbeh_zebra_puzzles #045 (bbeh)
bbeh_boardgame_qa #046 (bbeh)
bbeh_boolean_expressions #047 (bbeh)
bbeh_shuffled_objects #048 (bbeh)
bbeh_object_counting #049 (bbeh)
bbeh_dyck_languages #050 (bbeh)
bbeh_temporal_sequence #051 (bbeh)
bbeh_web_of_lies #052 (bbeh)
bbeh_buggy_tables #053 (bbeh)
bbeh_movie_recommendation #054 (bbeh)
bbeh_causal_understanding #055 (bbeh)
bbeh_sportqa #056 (bbeh)
bbeh_web_of_lies #057 (bbeh)
bbeh_sarc_triples #058 (bbeh)
bbeh_shuffled_objects #059 (bbeh)
bbeh_movie_recommendation #060 (bbeh)
bbeh_web_of_lies #061 (bbeh)
bbeh_causal_understanding #062 (bbeh)
bbeh_multistep_arithmetic #063 (bbeh)
bbeh_buggy_tables #064 (bbeh)
bbeh_causal_understanding #065 (bbeh)
bbeh_disambiguation_qa #066 (bbeh)
bbeh_object_properties #067 (bbeh)
bbeh_boolean_expressions #068 (bbeh)
bbeh_object_counting #069 (bbeh)
bbeh_multistep_arithmetic #070 (bbeh)
bbeh_multistep_arithmetic #071 (bbeh)
bbeh_sarc_triples #072 (bbeh)
bbeh_boardgame_qa #073 (bbeh)
bbeh_sportqa #074 (bbeh)
bbeh_movie_recommendation #075 (bbeh)
bbeh_boardgame_qa #076 (bbeh)
bbeh_hyperbaton #077 (bbeh)
bbeh_movie_recommendation #078 (bbeh)
bbeh_shuffled_objects #079 (bbeh)
bbeh_temporal_sequence #080 (bbeh)
bbeh_object_counting #081 (bbeh)
bbeh_disambiguation_qa #082 (bbeh)
bbeh_temporal_sequence #083 (bbeh)
bbeh_word_sorting #084 (bbeh)
bbeh_object_properties #085 (bbeh)
bbeh_time_arithmetic #086 (bbeh)
bbeh_dyck_languages #087 (bbeh)
bbeh_word_sorting #088 (bbeh)
bbeh_linguini #089 (bbeh)
bbeh_object_properties #090 (bbeh)
bbeh_spatial_reasoning #091 (bbeh)
bbeh_spatial_reasoning #092 (bbeh)
bbeh_buggy_tables #093 (bbeh)
bbeh_multistep_arithmetic #094 (bbeh)
bbeh_geometric_shapes #095 (bbeh)
bbeh_sportqa #096 (bbeh)
bbeh_boardgame_qa #097 (bbeh)
bbeh_boardgame_qa #098 (bbeh)
bbeh_shuffled_objects #099 (bbeh)
bbeh_word_sorting #100 (bbeh)
bbeh_temporal_sequence #101 (bbeh)
bbeh_geometric_shapes #102 (bbeh)
bbeh_boolean_expressions #103 (bbeh)
bbeh_causal_understanding #104 (bbeh)
bbeh_web_of_lies #105 (bbeh)
bbeh_temporal_sequence #106 (bbeh)
bbeh_time_arithmetic #107 (bbeh)
bbeh_buggy_tables #108 (bbeh)
bbeh_object_counting #109 (bbeh)
bbeh_multistep_arithmetic #110 (bbeh)
bbeh_sarc_triples #111 (bbeh)
bbeh_nycc #112 (bbeh)
bbeh_hyperbaton #113 (bbeh)
bbeh_disambiguation_qa #114 (bbeh)
bbeh_sportqa #115 (bbeh)
bbeh_nycc #116 (bbeh)
bbeh_movie_recommendation #117 (bbeh)
bbeh_dyck_languages #118 (bbeh)
bbeh_dyck_languages #119 (bbeh)
bbeh_time_arithmetic #120 (bbeh)
bbeh_zebra_puzzles #121 (bbeh)
bbeh_linguini #122 (bbeh)
bbeh_geometric_shapes #123 (bbeh)
bbeh_linguini #124 (bbeh)
bbeh_spatial_reasoning #125 (bbeh)
bbeh_boolean_expressions #126 (bbeh)
bbeh_object_counting #127 (bbeh)
bbeh_boardgame_qa #128 (bbeh)
bbeh_boardgame_qa #129 (bbeh)
bbeh_movie_recommendation #130 (bbeh)
bbeh_nycc #131 (bbeh)
bbeh_geometric_shapes #132 (bbeh)
bbeh_disambiguation_qa #133 (bbeh)
bbeh_web_of_lies #134 (bbeh)
bbeh_nycc #135 (bbeh)
bbeh_causal_understanding #136 (bbeh)
bbeh_zebra_puzzles #137 (bbeh)
bbeh_dyck_languages #138 (bbeh)
bbeh_causal_understanding #139 (bbeh)
bbeh_boardgame_qa #140 (bbeh)
bbeh_multistep_arithmetic #141 (bbeh)
bbeh_dyck_languages #142 (bbeh)
bbeh_geometric_shapes #143 (bbeh)
bbeh_hyperbaton #144 (bbeh)
bbeh_spatial_reasoning #145 (bbeh)
bbeh_nycc #146 (bbeh)
bbeh_causal_understanding #147 (bbeh)
bbeh_word_sorting #148 (bbeh)
bbeh_boolean_expressions #149 (bbeh)
bbeh_movie_recommendation #150 (bbeh)
bbeh_boolean_expressions #151 (bbeh)
bbeh_zebra_puzzles #152 (bbeh)
bbeh_zebra_puzzles #153 (bbeh)
bbeh_hyperbaton #154 (bbeh)
bbeh_buggy_tables #155 (bbeh)
bbeh_spatial_reasoning #156 (bbeh)
bbeh_disambiguation_qa #157 (bbeh)
bbeh_linguini #158 (bbeh)
bbeh_movie_recommendation #159 (bbeh)
bbeh_nycc #160 (bbeh)
bbeh_multistep_arithmetic #161 (bbeh)
bbeh_boolean_expressions #162 (bbeh)
bbeh_spatial_reasoning #163 (bbeh)
bbeh_word_sorting #164 (bbeh)
bbeh_shuffled_objects #165 (bbeh)
bbeh_boardgame_qa #166 (bbeh)
bbeh_dyck_languages #167 (bbeh)
bbeh_multistep_arithmetic #168 (bbeh)
bbeh_disambiguation_qa #169 (bbeh)
bbeh_geometric_shapes #170 (bbeh)
bbeh_boardgame_qa #171 (bbeh)
bbeh_web_of_lies #172 (bbeh)
bbeh_object_properties #173 (bbeh)
bbeh_nycc #174 (bbeh)
bbeh_shuffled_objects #175 (bbeh)
bbeh_geometric_shapes #176 (bbeh)
bbeh_movie_recommendation #177 (bbeh)
bbeh_movie_recommendation #178 (bbeh)
bbeh_sportqa #179 (bbeh)
bbeh_sportqa #180 (bbeh)
bbeh_word_sorting #181 (bbeh)
bbeh_web_of_lies #182 (bbeh)
bbeh_temporal_sequence #183 (bbeh)
bbeh_movie_recommendation #184 (bbeh)
bbeh_word_sorting #185 (bbeh)
bbeh_sarc_triples #186 (bbeh)
bbeh_multistep_arithmetic #187 (bbeh)
bbeh_web_of_lies #188 (bbeh)
bbeh_spatial_reasoning #189 (bbeh)
bbeh_shuffled_objects #190 (bbeh)
bbeh_zebra_puzzles #191 (bbeh)
bbeh_object_counting #192 (bbeh)
bbeh_sarc_triples #193 (bbeh)
bbeh_word_sorting #194 (bbeh)
bbeh_spatial_reasoning #195 (bbeh)
bbeh_dyck_languages #196 (bbeh)
bbeh_time_arithmetic #197 (bbeh)
bbeh_buggy_tables #198 (bbeh)
bbeh_object_counting #199 (bbeh)
bbeh_buggy_tables #200 (bbeh)
… and 260 more tasks

See every leaderboard on the Benchmarks index.