Upsy: A RAG-Powered Bot That Learns Your Slack and Discord History

February 17, 2024

|repo-review

by Florian Narr

Upsy: A RAG-Powered Bot That Learns Your Slack and Discord History

Upsy is a Discord and Slack bot that indexes your channel history into an Upstash Vector database and answers questions by doing RAG over that history. Mention it, DM it, or drop a ❓ emoji on a message — it threads a reply. It also learns continuously: every non-question message sent after setup gets embedded and stored.

Why I starred it

Most teams have years of institutional knowledge buried in Slack threads. Search helps if you know what to search for. Upsy takes a different angle: treat the channel history as a knowledge base, embed it, and let people ask questions in natural language.

The interesting part isn't the bot interface — it's the feedback loop. The bot doesn't just answer from a static snapshot. It keeps updating the vector store as new messages come in, filtering out questions (so it doesn't embed unanswered queries as facts) and storing answers and context. That distinction is worth reading the source for.

How it works

The repo splits into three folders: discord/, slack/, and shared/. The shared module has a single file, shared/llm.ts, which was the original lower-level implementation using LangChain directly with ConversationChain and @upstash/vector. The Discord bot has since migrated to @upstash/rag-chat, a higher-level wrapper that handles session-aware history and context retrieval in one call.

The core query path in discord/src/llm/llm.ts is now about fifteen lines:

export const ragChat = new RAGChat({
    model: openai('gpt-4-turbo'),
    promptFn: ({ context, question, chatHistory }) =>
        `You are an AI assistant with access to a Vector Store.
          Use the provided context and chat history to answer the question.
          If the answer isn't available, then try to answer yourself.
          ...
          Question: ${question}
          Answer:`,
});

export async function query(question: string, sessionId: string): Promise<string> {
    const strippedQuestion = question.replace('upsy', '');
    let response = await ragChat.chat(strippedQuestion, {
        historyLength: 7,
        sessionId: sessionId
    });
    return response.output;
}

The sessionId maps to a channel or DM, so history is scoped correctly. @upstash/rag-chat handles the embedding lookup, context injection, and Redis-based history storage — the older shared/llm.ts did all of this manually with embeddings.embedDocuments, index.query, and redis.lrange.

The message routing lives in discord/src/events/trigger-handler.ts (not shown in source, but called from message-handler.ts). Two entry points trigger a query: a direct mention of "upsy" in channel, or a DM. A separate path handles the ❓/🤔 emoji reactions via reaction-handler.ts, which dispatches through a Reaction interface with requireGuild, requireSentByClient, and requireEmbedAuthorTag guards before executing.

The write path in the Slack implementation (slack/src/index.ts) shows how the bot learns. On every non-question message it enriches the content string before storing it:

msg += ", Date: " + new Date().toLocaleDateString();
msg += ", Author: " + authorInfo?.user?.real_name;

await addDocument({ id: client_msg_id, type: "slack-message", ... }, msg);

The date and author get baked into the text that gets embedded — not as separate metadata fields that the retriever would need to handle separately. That means a query like "what did Ana say about the deployment last month" has a chance of working because that information is literally in the embedded string.

File uploads (PDF and DOCX) are also handled: the bot extracts text via pdf-parse and mammoth, attaches the filename and description, and stores the full text as a document.

When the bot joins a Slack channel it backfills the last 12 months of history in paginated batches via saveChannelHistory, bulk-embedding with addDocuments.

Using it

Deploy to Fly.io with the included Dockerfile and fly.example.toml:

fly launch
fly deploy
fly scale count 1  # important  single instance to avoid duplicate embeds

You need three Upstash resources: a Redis database (for chat history and deduplication), a Vector index (for embeddings), and an OpenAI key. The Discord setup also registers slash commands at build time:

npm run commands:register

The deduplication logic in the Slack bot is worth noting. It uses redis.setnx on the client_msg_id to skip duplicate deliveries — Slack's event API can fire the same event more than once under load.

Rough edges

The isQuestion function makes a full GPT-4 round trip per message just to classify it:

export async function isQuestion(question: string): Promise<boolean> {
    const resp = await ragChat.chat(
        "Does the following sentence look like a question semantically? Say just yes or no " + question,
        { disableRAG: true, disableHistory: true }
    );
    return resp.output.toLowerCase() === 'yes';
}

That's an extra LLM call on every single message. At any real message volume that adds up. A local classifier or even a regex heuristic (ends in ?, starts with "what/why/how/is/are") would handle 90% of cases without the latency or cost.

The test directory in discord/test contains only a data/ folder with no actual tests. The Slack implementation imports moment for the 12-month history window but moment is not in package.json — it's presumably a transitive dependency that happens to resolve.

The project is also explicitly a community project — not officially maintained by Upstash — and the last five commits are all README updates from September 2024. The shared/llm.ts LangChain version is still in the repo but looks superseded by the @upstash/rag-chat rewrite in the Discord bot; the Slack side hasn't been updated to match.

Bottom line

Upsy is a solid starting point for a self-hosted RAG bot that indexes your team's chat history. If you're already in the Upstash ecosystem and want to avoid building the vector retrieval and session history plumbing yourself, the current @upstash/rag-chat-based Discord implementation gets you there fast. Go in with eyes open on the LLM-call-per-message classification cost and the stale Slack codebase.

upstash/upsy on GitHub
upstash/upsy