Reinforcement Learning Playtest Agents That Find Stuck Spots and Exploits in Indie Game Levels

The problem: bugs that only players find

Indie games ship with bugs that frustrate players and fill review pages: a corner where the character gets stuck, a gap in a wall that lets you fall out of the world, a ledge that skips half a level, a key placed where nobody can reach it. Large studios find many of these with big QA teams and automated testing. A two-person studio in Pakistan or elsewhere usually finds them after launch, from angry players.

Scripted tests help only partly. They replay paths someone already wrote down, so they miss the places nobody thought to try. Random bots wander but rarely get far into a level. Manual playtesting by friends finds feel and fun problems, but nobody walks into every wall of every level after each change.

What already exists and the gap

• Unity ML-Agents and Godot RL Agents are open source toolkits for training agents inside game engines. They are built for training game AI and research, not as QA tools with bug reports.
• modl.ai offers commercial AI-driven game testing for studios.
• Engine test frameworks support scripted and unit tests, but not open-ended exploration.

The gap is an open, practical toolkit that turns reinforcement learning into a QA step for small teams: plug in a build, run exploration, get a heatmap and a reproducible bug list.

Inspiration

"Augmenting Automated Game Testing with Deep Reinforcement Learning" (Bergdahl, Gordillo, Tollmar and Gisslén, IEEE Conference on Games 2020) from SEED at Electronic Arts showed that agents trained with deep reinforcement learning can explore game levels to find exploits and areas where players get stuck, complementing scripted tests. Your project brings that approach to indie-sized games, with a focus on cheap training, clear reports and running after every build.

What you build

• Level designers push a new build; agents explore overnight; a report shows coverage heatmaps and a list of suspicious places with clips.
• Developers use the action sequence and seed in each report to reproduce a bug in the editor.
• Teams track coverage and bug counts across builds to see whether a level is getting more or less robust.

The technically interesting part

• Rewarding exploration. Agents need a reason to explore. Use rewards for new grid cells, curiosity bonuses, or count-based methods, and compare them on the same levels.
• Defining bugs. "Stuck" means no progress in position or state for a period while the agent is acting; "out of bounds" means below the floor or outside the level volume; "unintended shortcut" means reaching a later checkpoint without passing an earlier one. Make these definitions configurable.
• Training on a budget. Run builds headless, speed up time, train several environments in parallel, and reuse trained agents across small level changes instead of training from scratch each time.
• Reproducibility. A bug report is useless if nobody can repeat it. Record seeds, action sequences and physics settings, and verify that replays hit the same bug.
• Observations that generalise. Give agents simple observations, such as raycasts and position, rather than pixels, so training is cheap and agents transfer between similar levels.
• Comparison. Reinforcement learning is not always needed. Show where it beats random and scripted agents, and where it does not.

How to evaluate it

• Seeded bugs: add 30 known bugs (holes, stuck corners, unreachable items) to test levels and measure how many each method finds, and how fast.
• Real bugs: run on two real games and have the developers confirm which reports are real.
• Coverage: share of reachable level area visited by each agent type.
• Cost: training time and compute per level, and time per rerun after a small change.
• Reproducibility: share of reports that reproduce exactly in a replay.

Suggested plan

• Weeks 1 to 4: read the SEED paper, set up ML-Agents or Godot RL Agents with a simple level.
• Weeks 5 to 10: headless runs, exploration rewards and coverage heatmaps.
• Weeks 11 to 15: bug detectors, recording and replay.
• Weeks 16 to 20: seeded-bug study, comparison with random and scripted agents.
• Weeks 21 to 26: build pipeline integration, real-game study, write-up and demo.

Path to a product

Game testing is labour-intensive, and small studios have little budget for it. An open source toolkit with a hosted runner for studios, or a testing service sold per project, are both realistic. Game studios in Pakistan increasingly work for international publishers, where quality matters. The project is also a strong portfolio piece for both machine learning and game development roles.

Scope warning

Pick one engine and one game genre (for example 3D platformer or top-down action) and do it well. Do not train agents to play the game well; train them to explore. Keep levels small at first, because training cost grows quickly with level size.

Learn what you need on Sefism

• Machine learning roadmap: the foundations behind reinforcement learning.
• Game development roadmap: level design and engine workflows.
• PyTorch: Learn the Basics: training the agents.
• Unity Manual and Scripting API: if you build on Unity and ML-Agents.
• Godot Engine Documentation: if you build on Godot RL Agents.
• QA and test automation roadmap: writing bug reports developers can act on.

Get help with this idea

The risk is spending the whole year on training and never producing a useful report. Tauseef can help you choose the engine and genre, design the seeded-bug study and keep training costs under control. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session