The Concept

Make my characters come to life with LLMs.

For the past several years I have been worldbuilding as a hobby to work out the creative wrinkles of my brain that were unable to be reasonably flexed at my job. I had created hundred upon hundreds of text files and maps and stories and character descriptions to sate my thirst for a creative outlet.

(Inspired by the reddit post here)

After a great deal of worldbuilding, what are you left with? A world, of course! One that is ready to be explored, experience, and expanded.

A few months ago (years ago?), amid the sewerous wave of the AI/LLM/Machine Learning craze, I decided that I wanted to try running my own models locally. I began with the Mistral 7b Q5_K_M model, since it’s what my Nvidia 3060 could handle at the time. I compiled llama.cpp, loaded up the model, and was off the the races.

(I have since been unable to compile llama.cpp with ROCm after changing GPUs due to reasons I do not have enough time to figure out, and have since then moved to lmstudio)

The Problem

Playing around with language models was fun, don’t get me wrong. But it lacked the zhuzh, the Je ne sais quoi, if you would. I wanted to talk to characters from my worldbuilding, and each time I spun up the LLM, it was like resetting the world to a fresh copy. I wanted a way to load custom characters via some sort of card, perhaps similar to CharacterHub, but closer to home (and less slimy). I also wanted a better way to manage context windows, since loading in the entirety of my worldbuilding wasn’t going to fit in my itsy-bitsy 4096 token window.

The Architecture

I know I wanted some sort of retrieval to manage context (a simple home-grown RAG), ease of loading characters, and a rudimentary memory system so that NPCs could remember our conversations after they were done (i.e. after the model was unloaded). I arrived at a system that works like this:

1. Create a series of NPC cards defined by YAML e.g.

Name: Muzzlepaw
Race: Bjorni
Age: 78
Personality:
  - Angry
  - Violent
  - Grumpy
  - Guarded
  - Bearlike
  - Often lost in thought
  - Secretly fearful of mice
Residence: Thoradus
Player Attitude: 0
Knowledge:
  - Bjorni: Those who are south of Sirenia turn gray with age
  - The_Compound: dark magic is researched here
Memories:

2. Load the NPC YAML into the system prompt MINUS the knowledge fields with some structural prompting:

You are an NPC in a fantasy world, described by the following YAML:

Name: Muzzlepaw
Race: Bjorni
Age: 78
Personality:
  - Angry
  - Violent
  - Grumpy
  - Guarded
  - Bearlike
  - Often lost in thought
  - Secretly fearful of mice
Residence: Thoradus
Player Attitude: 0
Knowledge:
Memories:

You will roleplay and do everything you can to not break character...

3. Prompt the user as usual.

User: Hello there friend!
Muzzlepaw: You dare speak to me? A Bjorni of my status? 
           I would tear you asunder in the flash of an eye!

4. If the user says something that matches a knowledge of the NPC (seen in the first YAML above), insert it into the system prompt before sending it off to the model.

System prompt:
You are an NPC in a fantasy world, described by the following YAML:

Name: Muzzlepaw
Race: Bjorni
Age: 78
Personality:
  - Angry
  - Violent
  - Grumpy
  - Guarded
  - Bearlike
  - Often lost in thought
  - Secretly fearful of mice
Residence: Thoradus
Player Attitude: 0
Knowledge:
  - The_Compound: dark magic is researched here
Memories:

You will roleplay and do everything you can to not break character...

-----------

User: Hello there friend!
Muzzlepaw: You dare speak to me? A Bjorni of my status? 
           I would tear you asunder in the flash of an eye!
User: I am looking for The Compound. Do you have any information 
      that would help me find it?
Muzzlepaw: Your ignorance is a weakness I shall not tolerate. 
           You shall only find dark magics in that place.

Nice, he remembered what he’s suppose to know about The Compound!

5. When the user says goodbye, summarize the conversation and append it to the character’s memories.

User: Hello there friend!
Muzzlepaw: You dare speak to me? A Bjorni of my status? 
           I would tear you asunder in the flash of an eye!
User: I am looking for The Compound. Do you have any information 
      that would help me find it?
Muzzlepaw: Your ignorance is a weakness I shall not tolerate. 
           You shall only find dark magics in that place.
User: Fine. Goodbye.
Muzzlepaw: Bye.
** Takes this conversation and sends it back to the LLM to summarize **
** Mem added: Muzzlepaw remembers talking to User about The Compound **

Some of these memories were a bit verbose and perhaps not-too-helpful:

Borin Stonebeard remembers no new information from the brief exchange, as the stranger declined his ale offer and departed without further conversation.

A work in progress, for sure.

The results are quite neat, in my opinion. With this, I am able to:

• Create a plethora of characters and store them in a structured format
• Have the ability to talk to any of my characters without having to swap out models, reprompt, or edit code
• Limit context size by pulling knowledge only when needed (via a fuzzy find)
• Have the characters remember conversations that we’ve had before

That last point was the most exciting for me. I can tell the NPC that my favorite fruit are bananas, and they will remember it every time I talk to them afterwards. This holds true even if it is months later since I store the memories in the YAML file.

This was all done in about 190 lines of bash with a wall of seds, greps and dumb bashisms I’ll probably never fix.

The filesystem looks like this:

├── histories <--------------- Chat Histories
│   ├── npc_borin.history
│   ├── npc_muzzlepaw.history
│   └── npc_willnt.history
├── live <-------------------- Live YAML files with knowledge/mem added
│   ├── npc_borin.live
│   ├── npc_muzzlepaw.live
│   └── npc_willnt.live
├── npc_chat.sh <------------- The bash script
├── npc_yaml <---------------- Clean YAML
│   ├── npc_borin.yaml
│   ├── npc_elara.yaml
│   ├── npc_muzzlepaw.yaml
│   ├── npc_snitch.yaml
│   ├── npc_TEMPLATE.yaml
│   ├── npc_torin.yaml
│   └── npc_willnt.yaml

Anyways, the next thing on my list is getting it fully integrated into a terminal program. I have a working demo done, but there is a lot to do in terms of actually making it fun.

Why?

Because it’s cool! Look how neat!