WhatsApp Knowledge Agent
Multimodal WhatsApp brain — text, images, PDF, and Excel into a vector store, then answers from the private knowledge base.
Problem
WhatsApp groups dump PDFs and product photos. Replies still come from whoever saw the chat last.
What we built
Vision + document loaders + embeddings + agent. The WhatsApp thread is the UI; the vector store is the memory.
How it runs in real time
Customer or staff sends a WhatsApp. If it is a photo of a product or a board, Vision describes it. If it is a PDF/xlsx, loaders extract text and upsert into the vector store. The next question (“what is the GST on that SKU?”) retrieves the chunk and answers in the same thread. Operators do not re-upload files to a CMS; the chat is the ingest.
Pipeline
01
Trigger + switch
Incoming WhatsApp classified as text, image, or document.
02
Understand media
Vision captions images. Document loaders flatten PDF/Excel to text.
03
Index + retrieve
Embeddings land in the vector store. The agent queries it with the user’s question.
04
Reply
WhatsApp Send Response returns a sourced answer, not a hallucinated brochure.
In the canvas
- ▸ WhatsApp Trigger as the only frontend
- ▸ OpenAI Vision on images
- ▸ PDF + Excel loaders
- ▸ Embeddings → vector store retrieval
- ▸ Agent reply back on WhatsApp
This is the marketing/support WhatsApp agent: not a menu bot. WhatsApp Trigger receives the message. A Switch splits Text / Image / Document. Images go to OpenAI Vision for a description. PDFs and Excel are loaded to text. Everything is embedded and upserted into a vector store so the next question can retrieve. A Chain/Agent node mixes the live query + retrieved chunks + vision captions and replies on WhatsApp. Real-time use: a client drops a rate card, a product photo, or a policy PDF in chat; the agent answers from that file without a human opening Drive.
Proof
Results
- ▸ Text, image, and PDF/Excel on one Switch
- ▸ Private knowledge base searchable in chat
- ▸ Reply stays inside WhatsApp — no extra portal