🔍 Read the full analysis: Using ThinkingBox To Shape Agent Rewards From Database Grades on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Microsoft and Hugging Face say their ThinkingBox-Training stack is available on GitHub, using executable checks of final database states as rewards for agent training. In one reported 50-step GRPO run, Qwen3.8-27B’s average success on a 507-task benchmark rose from 51.70% to 60.78%, but the model solved seven fewer tasks at least once.
Microsoft and Hugging Face say their ThinkingBox-Training stack is now available on GitHub, using executable checks of an agent’s final database state as rewards during post-training, as described in the original analysis. In one reported 50-step GRPO run, Qwen3.8-27B raised its average success rate on a 507-task benchmark from 51.70% to 60.78%, while the number of tasks it passed in all 20 attempts increased; it solved seven fewer tasks at least once.
The teams trained Qwen3.8-27B with a pool of ThinkingBox-format practice tasks separate from the evaluation benchmark. At the end of each task, executable checks inspected whether the expected records were present in the backend. The authors say the reward is based on the resulting database state, rather than requiring a specific sequence of tool calls.
On the benchmark’s 507 tasks, each evaluated 20 times, successful attempts rose from 5,242 of 10,140 before training to 6,163 afterward. That is the reported change in pass@1, or the share of individual attempts that succeeded: 51.70% to 60.78%. The count of tasks passed in all 20 attempts rose from 38 (7.50%) to 107 (21.10%).
Coverage moved in the other direction. Pass@20—the number of tasks solved at least once across their 20 attempts—fell from 453 tasks (89.35%) to 446 (87.97%). The reported results therefore combine a higher average success rate and more repeatable performance on some tasks with a modest reduction in the number of tasks the model managed to solve at least once.
Reliability Gains Come With Less Coverage
For agents that update business records, a convincing answer or plausible sequence of tool calls does not prove that the requested change was completed correctly. Checking the final database state gives a training system a way to reward the outcome itself, including whether necessary records were created or whether a transaction left incorrect data behind.
The benchmark figures highlight a distinction that matters to teams evaluating agents: average attempt success is not the same as task coverage. The reported rise in pass@1 and in tasks passed every time suggests more dependable results on part of the benchmark. The lower pass@20 figure means the model did not expand the set of tasks it could solve at least once. The single reported run does not establish how the approach would perform in live operations.
As an affiliate, we earn on qualifying purchases.
From Database Benchmark to Reward
ThinkingBox was initially described as an evaluation method for agents that interact with databases. Its benchmark has 507 tasks, with each task set up around a backend, MCP tools and a simulated user holding private context. Executable checks compare the final database state with the expected outcome.
In their joint account, Microsoft and Hugging Face say they extended that grading approach to training. They selected practice tasks using an estimate of how difficult each was for the base model, then kept the training pool fixed for the run. The authors’ stated rationale was to focus practice on tasks that were neither already easy nor beyond the model’s current ability. They trained for 50 GRPO steps and evaluated a selected checkpoint on the separate benchmark, which they describe as out of distribution because its domains, databases and most tools were not encountered during training.
“For most tasks, an executable check can return 1 or 0 directly from the final database state. That is not just a grader. This can be a reward function.”
— Microsoft and Hugging Face, in their joint post
As an affiliate, we earn on qualifying purchases.
One Run Leaves Key Questions Open
The reported results come from one 50-step run. The supplied account does not give enough information to judge sensitivity to random seeds, choices of training pool or run length, nor does it provide a full comparison with other reward designs or training methods. Those gaps make it difficult to isolate how much of the measured change came specifically from database-state rewards.
The authors describe the evaluation benchmark as out of distribution, but the reported figures do not establish performance in a live production setting. Production reliability, generalization and training cost remain unreported in the supplied material. It also does not explain why pass@20 fell by seven tasks or whether that tradeoff would hold across further runs.
As an affiliate, we earn on qualifying purchases.
Reproduction and Broader Testing
With the stack available on GitHub, researchers and developers can inspect the implementation and attempt their own training runs. The next useful evidence would include replicated results across random seeds, other models and different practice-task pools, along with comparisons against alternative reward methods.
Before drawing conclusions about deployment, teams would also need tests in workflows resembling their intended operating environment. Those evaluations should check both whether final records are correct and whether the agent avoids incomplete or unauthorized changes, as illustrated by the authors’ hotel-booking example. No production deployment results or further milestones are specified in the supplied account.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is ThinkingBox-Training?
It is a training stack from Microsoft and Hugging Face that uses executable checks of an agent’s final database state as rewards. The teams say it is available on GitHub.
What changed in the reported Qwen3.8-27B run?
After 50 GRPO steps, average success on a 507-task benchmark rose from 51.70% to 60.78%. Tasks passed in all 20 recorded attempts increased from 38 to 107.
Did the model solve more benchmark tasks overall?
No. Pass@20, the number of tasks solved at least once across 20 attempts, declined from 453 to 446. The report describes more consistent results on more tasks alongside slightly lower overall task coverage.
Do these results show the method works in production?
No production results are included in the supplied account. The reported figures come from one benchmark evaluation after a single 50-step run; performance in live workflows remains unknown.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
