On-Device Language Model NPCs with Memory and Lore Consistency Checks for Indie Games

The problem: talking NPCs that forget, contradict and cost money

Players have seen what language models can do, and many indie developers want characters that respond to anything a player says. The first attempts are usually disappointing. The blacksmith who died in act one is suddenly alive in conversation. The innkeeper reveals the quest solution because the player asked politely. A guard invents a town that does not exist. And if the model runs in the cloud, every conversation costs money and the game stops working offline.

For a small studio in Lahore, Karachi or anywhere else, these problems are enough to drop the idea. What they need is a way to run characters locally and keep them inside the story.

What already exists and the gap

• Inworld AI and Convai offer cloud platforms for AI-driven characters, with tools for personalities and knowledge. They are aimed at studios willing to depend on a hosted service.
• NVIDIA ACE provides models and tools for digital humans, including smaller models that can run on the player's PC with suitable hardware.
• Local model runtimes such as llama.cpp make it possible to run small models on ordinary laptops, but they give you text generation, not characters that respect a game's rules.

The gap is an open, engine-friendly framework focused on what goes wrong: memory, lore contradictions and quest spoilers, measured on a standard test set, with models small enough to run on the player's own machine.

Inspiration

"Generative Agents: Interactive Simulacra of Human Behavior" (Park et al., UIST 2023, Stanford and Google) populated a small town with 25 characters powered by a language model. Each agent stored its experiences as a memory stream, retrieved relevant memories, reflected on them and planned its day. That architecture is a good starting point for NPC memory. Your project adapts it to the constraints of a real game: a small local model instead of a large cloud model, designer-written lore that must not be broken, and quests that must not be spoiled.

What you build

• Game designers write lore facts, character sheets and quest rules in simple files, and mark what each NPC knows.
• Players talk to NPCs by typing; replies appear quickly, reflect past events and stay consistent with the world.
• Developers drop the plugin into a Godot or Unity project and tune memory size, model and guard rules.

The technically interesting part

• Small models, tight budgets. A game must keep running smoothly while the model generates. Run inference on a separate thread, stream tokens, cap reply length and measure the frame rate impact.
• Memory that fits. Small models have small context windows. Retrieve only relevant memories and lore, and summarise older conversations.
• Checking consistency. After the model drafts a reply, extract its claims and check them against the lore store and quest state, using rules for hard facts (who is alive, what the player owns) and a small model for softer checks. If a check fails, regenerate or fall back to a written line.
• Spoiler and exploit guards. Players will try to talk NPCs into revealing answers or giving free items. Treat this like prompt injection testing and build a set of attack messages.
• Measuring believability. Contradiction rates are measurable; "feels alive" is not. Combine automatic metrics with a small player study.

How to evaluate it

• Contradiction rate: share of replies that contradict lore or quest state on an evaluation set of 300 or more player messages, with and without the checker.
• Spoiler and exploit rate: share of attack messages that succeed.
• Latency and frame rate: time to first token and full reply, and frame rate during generation, on two hardware classes.
• Player study: ten or more players rate believability and enjoyment against a version with scripted dialogue trees.

Suggested plan

• Weeks 1 to 4: read the Generative Agents paper, run small models locally, write the demo's lore and quest.
• Weeks 5 to 10: engine plugin, streaming replies, lore store and memory.
• Weeks 11 to 16: consistency checker, guard rules and the evaluation set.
• Weeks 17 to 21: demo game polish, performance tuning, model comparison.
• Weeks 22 to 26: player study, write-up and demo.

Path to a product

Indie studios want characters that respond to players without a monthly bill per conversation. An open source plugin with paid support, or a paid designer tool for lore and testing, is a realistic path. The evaluation set itself could become a public benchmark for NPC consistency, and the project is a strong portfolio piece for both game and AI engineering roles.

Scope warning

Do not train or fine-tune a model in the first version; pick a small existing one and focus on memory, checking and guards. Keep the demo small: one location, four characters, one quest. A short demo that never contradicts itself is far more impressive than a large one that does.

Learn what you need on Sefism

• Game development roadmap: engine plugins, threads and game loops.
• AI engineering roadmap: running small models, retrieval and evaluation.
• LLM APIs cheat sheet: prompts, context windows and streaming.
• Ollama: quick experiments with small local models before you embed one.
• OWASP Top 10 for LLM Applications: prompt injection, which is how players will attack your NPCs.
• Godot Engine Documentation: building the plugin and demo.

Get help with this idea

The risk is a demo that talks well for five minutes and falls apart under real players. Tauseef can help you design the evaluation set, choose a model size and keep the scope realistic. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session