Private knowledge system

Dharma Sky

Turning 1,000+ files into an AI-searchable knowledge system.

I wanted to bring together Buddhist texts I kept returning to — the Buddha's tales of previous lives, sutras from public-domain archives, and other material scattered across the web or saved as PDFs. I designed and built Dharma Sky: a private library where those files can be browsed, searched, and asked about through an AI interface grounded in the source material.

Role: Product Designer · AI Product Builder

AI Product Design
AI-assisted Development
Knowledge Retrieval
Product Building

The challenge

Knowledge trapped in many files

The material already existed freely online or in files — across formats and webpages. Finding a specific article or topic on a theme, comparing related passages, or answering a precise question meant manual hunting through an archive no human interface could reasonably navigate.

This was an information architecture problem: how to make a large, unstructured body of knowledge accessible without asking someone to remember filenames, dates, or which document held which idea.

The idea

A retrieval layer between people and the archive

Instead of asking users to search documents one by one, Dharma Sky processes the corpus into an indexed knowledge layer — then offers a calm product surface for browsing, thematic exploration, and grounded questions.

The system

From unstructured files to grounded answers

The core product story is the indexing pipeline. Everything the user experiences — search, Ask, topics — depends on what happens here.

Indexing pipeline

Files

A private corpus of 1,000+ teachings arrives in many formats — Word, PDF, RTF, HTML, scanned images, nested archives.

Processing

Each file is expanded, text is extracted (including OCR for images), and the content is normalized into readable teaching text.

Indexing

The teaching is split into retrieval chunks, embedded for vector search, and also given a document-level index: key terms, a faithful synopsis, and a routing embedding.

Retrieval

A question first shortlists the most relevant teachings, then searches passages inside those teachings — so answers stay grounded in the right sources.

AI response

The model answers only from retrieved passages, with citations back to the original teachings — never inventing content beyond the library.

Chunking

Teachings are split into overlapping retrieval units sized for embedding — keeping page context when available.

Embeddings

Chunks and document indexes are embedded so meaning — not just keywords — can be searched.

Hierarchical retrieval

Shortlist teachings first, then retrieve passages inside them. Diversify sources so one file doesn’t dominate the answer.

The point for a recruiter who has never built AI infrastructure: I understand what happens between a user asking a question and an AI producing an answer — and I designed that path as a product.

Architecture evolution

A hybrid system built for speed, cost, and control

As the corpus grew, I moved retrieval away from a single Supabase dependency. Each service now does the job it is best at, while the product keeps one consistent experience.

System of record

Supabase

Authentication, private file storage, structured metadata, topics, and user data remain in Supabase.

Retrieval engine

Qdrant

Document and chunk embeddings live in Qdrant. Passage text and source metadata travel with each vector result, removing an extra database round trip from Ask.

Experience layer

Next.js cache

Catalog, teaching, and topic reads use tagged server caches. Storage changes trigger signed revalidation, so repeated browsing stays efficient without serving stale content.

Staged migration

A shared retrieval interface can route to Qdrant or the existing Postgres vector functions by configuration. This separated product behavior from storage choice and made the migration incremental.

UX effect

People keep the same simple model — browse, ask, verify sources — while caching and payload-rich vector results reduce avoidable backend work.

Scale

1,000+

files

At this scale, manually searching the source material is no longer a practical interface.

The product needed a structured way to process, index, retrieve, and surface the right knowledge — so a question or theme could open the archive without requiring someone to know where each teaching lived.

Product & UX

A simple surface over a complex system

The interface is intentionally quiet: library, browse, topics, Ask. Complexity stays in the pipeline; the product asks only for intent.

Ask

Natural-language questions become a two-stage search: find the right teachings, then quote from matching passages.

Browse

Semantic search ranked by aboutness — synopses and key terms make a large library scannable without opening every file.

Library

Catalog filters by year, event, type, and topic. Detail pages open the full file with optional AI summaries.

Topics

Themes gathered across teachings, with Topic Builder linking each theme back to supporting passages.

How people move through it

Search by meaning — not only exact titles — using the document index and semantic ranking.
Open a teaching to read the file, scan source metadata, and optionally generate a structured summary.
Explore curated topics that pull passages from across the corpus instead of forcing a single-file mental model.
Ask in natural language when the goal is an answer with sources — not a file hunt.

Browse

Anonymized preview

The jealous monk's vow

The jealous monk's vow · Jataka tale

A monk lets jealousy drive him to slander a companion who earned respect. The tale turns on confession, making amends, and choosing a path toward awakening.

Confession and redemption · Jataka tale

After admitting his wrongdoing, the monk offers what he can, renews his vow, and receives encouragement — while others in the story make vows of their own.

AI interaction

Conversational — but purpose-built

Ask Dharma is not a generic chat clone. It is designed around one job: answer from this library, show the passages that support the answer, and refuse to invent teachings that are not there.

What can jealousy make someone do?

From the library

In the jealous monk's vow, jealousy pushes a monk to slander a companion who earned more respect — falsely accusing him instead of facing his own envy. The retrieved sources show how admiration can twist into accusation when jealousy goes unchecked.

Supporting passages

“Jealousy led him to slander a companion who had earned respect, falsely accusing him of wrongdoing.”

Jataka tale · The jealous monk's vow

Illustrative exchange — private corpus content is not shown.

Under the surface: the question is embedded, candidate teachings are shortlisted from the document index (and lexical key-term overlap), matching chunks are retrieved, sources are diversified, and only then does the model write — constrained to those passages.

The model does not "know" the files. It answers from what retrieval returns.

Building the product

Designed. Built. Shipped.

I used AI-assisted tools to move from product concept and UX into a working application — schema, ingestion pipeline, retrieval, and interface.

The important part is not that AI wrote code. It is that I could direct the full loop: define the problem, design the experience, decide how knowledge should be indexed and retrieved, and ship a private product people can actually use.

Multi-format ingestion pipeline
Supabase + Qdrant hybrid architecture
Document-level + chunk-level indexes
Grounded Ask flow with citations
Tagged cache + event-driven invalidation
Authenticated private library shell

Design decisions

Product choices that shaped the system

Hide the infrastructure

Users never see embeddings, chunk boundaries, or vector scores. They see search, answers, and sources.

Retrieve before you answer

The model is not treated as if it "knows" the corpus. Passages are retrieved first; the answer is constrained to those passages.

Cite every claim path

Supporting passages appear under each answer, linked to teaching titles — trust comes from provenance, not polish.

Index at two levels

Document-level aboutness finds the right teachings; chunk-level search finds the exact lines to quote.

Design for private knowledge

Authenticated access, private storage, and no public registration — the corpus stays a closed library.

States that match the system

Loading (“Searching the teachings…”), empty results, and retrieval failures are designed as product moments, not generic errors.

Split the system by responsibility

Supabase manages private content and relational data; Qdrant handles semantic retrieval and returns passage payloads with each result.

Cache without losing freshness

Stable library reads are cached by tag, then automatically revalidated when teaching files change.

What I learned

Building AI products is systems design

1
Designing an AI product means designing the system around the AI — retrieval, grounding, and citations — not dropping a chatbot onto a document pile.
2
Retrieval quality is product quality. If the wrong passages surface, the answer feels broken even when the UI is calm.
3
Good AI UX makes complex infrastructure invisible. Users should feel guided, not impressed by jargon.
4
Architecture choices should map to user value: fewer backend round trips and deliberate cache invalidation support a faster-feeling, dependable library.
5
AI-assisted development changed how quickly I could move from product concept and UX into a working system I could iterate on.