← Research

RuleSmith: Multi-Agent LLMs for Automated Game Balancing

Ziyao Zeng, Hao Wang, Chen Liu, Youheng Yao, Jingcheng Ni, Tianyu Liu, Xiatao Sun, Fengyu Yang, Chenyu You, Xiaofeng Liu, Daniel Rakita, Ronald R. Coifman, Yuval Kluger, Zhiwen Fan

Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, a framework that couples multi-agent LLM self-play with surrogate-based optimization over a parameterized rule space. As a proof of concept, we build CivMini, a simplified civilization-style game with two asymmetric factions governed by 12 tunable parameters. LLM agents interpret textual rulebooks and game states to propose actions that are validated by the game engine, yielding noisy estimates of faction win rates. Our primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates. RuleSmith consistently finds near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent 200- and 300-game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components. Cross-model evaluations and small human studies provide preliminary evidence of transfer across player populations. RuleSmith produces interpretable numerical rule adjustments without task-specific policy training. Project and code: https://adonis-galaxy.github.io/RuleSmith-website/

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace