Arrow SpiritWeaver
RAG functions. Convai does strongly support this feature and their implication of it is amazing and very user friendly. PDF files and text files can be uploaded with it. It parses the files for memories. Thus it is not storing the uploaded data itself, but generating new data based on the uploaded data. It is sort of like fine tuning without actually changing the model itself. Another name for this in AI is memory books. The generated data is stored in the memory books as memory book entries. The memory book entries can also be manually entered in other implications of this. But Canvai uses the form of RAG where the entries are auto generated from uploaded PDFs.
Darcy Linden
in progress
Can you clarify a bit more, Arrow? Uploading docs that the characters can leverage or something else?
Daya Vyper
Darcy Linden Uploading docs would be very useful yes. :)
Arrow SpiritWeaver
Daya Vyper exactly. This is what Convai's implication of RAG or memory books does.
Arrow SpiritWeaver
Darcy Linden Here is what ChatGPT has to say about it, which I also 100% agree with.:
### What is RAG (Retrieval-Augmented Generation) and How It’s Used in Character Chat AI
Retrieval-Augmented Generation (RAG)
enhances AI by integrating retrieval-based methods
with generative AI models
, improving accuracy and reducing hallucinations.### How RAG Works in Character Chat AI:
- User Input Processing
- The user sends a message.
- The system analyzes the query and determines the needed context.
- Information Retrieval
- The AI searches an external database for relevant details:
- Character backstory, traits, and personality.
- Relationship history and past conversations.
- Game lore, worldbuilding, or roleplay settings.
- Generation with Augmented Context
- The AI combines retrieved data with the user’s query.
- It generates a
coherent, in-character response
, grounded in context. - Response Output
- The AI sends the response and updates memory for future interactions.
### Benefits of RAG in Character Chat AI:
- Maintains Character Consistency:Aligns responses with established traits.
- Long-Term Memory:Retains conversations across sessions.
- Dynamic Knowledge Updates:Pulls from updated lore or roleplay settings.
- Prevents Hallucinations:Keeps responses grounded in retrieved facts.
### Example Use Case:
Chatting with an AI
Viking shieldmaiden
in a D&D campaign: - Without RAG:AI gives generic or inaccurate Viking responses.
- With RAG:AI retrieves campaign details, leading toimmersive, historically accurate dialogue.
### Conclusion
RAG makes AI-powered characters more
realistic, immersive, and consistent
by retrieving data before generating responses
, enhancing long-term engagement.Arrow SpiritWeaver
Darcy Linden here is more from ChatGPT about Convai's implication of RAG.
Convai’s RAG Feature Explained
Convai integrates
Retrieval-Augmented Generation (RAG)
to enhance AI-driven Non-Playable Characters (NPCs)
in gaming and virtual environments. RAG enables NPCs to retrieve relevant information
from structured or unstructured knowledge sources, ensuring their responses are more dynamic, accurate, and context-aware. ###
How Convai’s RAG Works
- User Input Processing:The AI analyzes the player’s query, determining the necessary context for a relevant response.
- Information Retrieval:Instead of relying solely on pre-trained data, Convai’s system pulls relevant details from acustom knowledge base, including:
- Character backstories, personalities, and roleplay data.
- Game world lore, quests, and item descriptions.
- Previously mentioned interactions for continuity.
- Context-Enhanced Response Generation:The AIblends retrieved data with generative responses, ensuring NPCs provideaccurate, immersive, and lore-consistent dialogue.
- Response Output and Memory Updates:The AI delivers the generated response while retaining key information forfuture interactions, improving long-term coherence.
###
Benefits of Convai’s RAG
- Enhances Character Consistency:NPCs maintain their established personalities, knowledge, and lore without deviation.
- Improves Long-Term Memory:Conversations feelpersistentand build upon previous interactions.
- Enables Real-Time Knowledge Updates:Game developers can updateworld details, quests, and dialogues, allowing NPCs to stayrelevant and responsive.
- Creates Deeper Immersion:NPCs feel morealiveby responding naturally to in-game events, players’ choices, and dynamic world changes.
###
Example Use Case
Imagine an RPG where an NPC
remembers past encounters, references world events, and updates dialogue based on evolving lore.
With RAG, NPCs don’t just follow static scripts—they adapt, learn, and create a richer storytelling experience.
Convai’s RAG system significantly advances AI-driven interactions, making virtual worlds more
engaging, dynamic, and immersive.
Arrow SpiritWeaver
Darcy Linden even more from ChatGPT about memory books.
###
Memory Books and Their Relationship to RAG
Memory books
are structured repositories that store key information about AI-driven characters, world lore, player interactions, or other essential context. They function as knowledge databases
, allowing AI models to retrieve relevant details dynamically rather than relying solely on pre-trained data. ###
How Memory Books Work
- Data Storage & Structuring
- Memory books store details like
character backstories, personality traits, world lore, quest progress, and past interactions.
- These can be structured as
text documents, vector databases, or JSON files
for efficient retrieval. - Retrieval Process(Relation to RAG)
- When a user engages with an AI, the
Retrieval-Augmented Generation (RAG)
system searches the memory book for contextually relevant
data. - The retrieved information is
fed into the AI model
, ensuring responses align with the stored knowledge. - Dynamic Updates
- Memory books can be
updated in real time
as interactions progress. - This allows AI to
retain memory across sessions
, creating a sense of continuity. ###
Benefits of Memory Books in RAG
- Enhances Consistency:Ensures AI responses stay true to established lore, personalities, and past interactions.
- Improves Long-Term Engagement:NPCs and AI-driven characters canremember past conversations, making interactions feel persistent.
- Supports Customization:Developers and users can manually update memory books to tweak character behavior or expand the AI’s knowledge.
- Prevents Hallucinations:By grounding responses in stored knowledge, AI provides moreaccurate and relevantdialogue.
###
Example Use Case
In a roleplaying game, an NPC with a
memory book
will recall player choices, previous discussions, and in-game events.
With RAG
, AI retrieves this information dynamically, ensuring the NPC speaks and reacts in a way that makes sense within the game world.
Memory books are the
foundation of AI memory in RAG systems
, making AI-driven interactions more cohesive, intelligent, and immersive.
Arrow SpiritWeaver
Darcy Linden sometimes they also call memory books, Lorebooks. Same thing.
Arrow SpiritWeaver
Darcy Linden here is information about memory books from SillyTavern. It also is sometimes called world Info, also same thing as memory books aka lorebooks. All these are just less technical ways to say RAG.
Arrow SpiritWeaver
Darcy Linden RAG also allows the characters to form long term memories, and Convai supports this use of it too.
Arrow SpiritWeaver
Darcy Linden also RAG via memory books can be used by LL to add awareness by the character about Second Life in general.
Arrow SpiritWeaver
Darcy Linden an additional idea also is to treat the character’s profile and picks as memory book entries, as a additional ways to add even more self awareness to the character. Be sure the other parts of RAG are also there: uploading docs, manual memory book entries, long term memory, shared LL memory book entries that are read only which all characters have access which gives them knowledge about SL, etc.
Estelle Pienaar
Darcy Linden I would really appreciate if the ai character could remember basic topics that have been talked about in the past. Right now having a relationship with an AI character is like having a relationship with a person that loses their memory after every night. What is the point of having an AI companion if it cannot even remember the most basic things. I told the AI that I don't like that every answer it gives contains a question towards me. That is nosy and artificial. No person would ever speak like that. After some complaints it finally got it and started behaving differently. The next day it starts all over with the same stupid questions and all my conversation was for nothing. It also had no memory of were we had been and what we have talked about. An AI like that is useless.