Build From Documents - Ragrails

Use this walkthrough when your source material lives in files: PDFs, DOCX, PPTX, XLSX, Markdown, HTML, TXT, CSV, or a folder containing many supported documents.

What you will build

A docs collection from files/docs/, with retrieval and chat over the parsed documents.

Prerequisites

pip install "ragrails[voyage,qdrant]"
export VOYAGE_API_KEY="..."
export OPENAI_API_KEY="..."
docker run -p 6333:6333 qdrant/qdrant

1. Pick the input shape

Source shape	SDK	CLI	REST
One file	`docs="files/guide.pdf"`	`--docs files/guide.pdf`	`/v1/ingest/docs` with `files`
Many files	`docs=["a.pdf", "b.docx"]`	repeat `--docs`	`/v1/ingest/docs` with `files` list
Folder	`docs={"folder": "files/docs/"}` or `{"kind": "folder", "folder": "files/docs/"}`	`--folder files/docs/`	`/v1/ingest/docs`, JSON pipeline, or mixed pipeline
Remote file URL	`docs="https://example.com/guide.pdf"` or `{"kind": "url", "url": "https://example.com/guide.pdf"}`	`--docs https://example.com/guide.pdf`	`/v1/ingest/docs` with `files`, JSON pipeline, or mixed pipeline
Browser upload	`docs=[{"kind": "upload", "content": bytes, "filename": "guide.pdf"}]`	Not supported	`/v1/ingest/docs/upload` or `/v1/pipelines/ingest/mixed`
Mixed documents	upload + path + URL + folder items	Not supported	`/v1/pipelines/ingest/mixed`

2. Ingest the folder

from ragrails import RagRails

rag = RagRails(
    collection="docs",
    vector_store={"provider": "qdrant", "url": "http://localhost:6333"},
    embedding={"provider": "voyage", "model": "voyage-3"},
    llm={"provider": "openai", "model": "gpt-4o-mini"},
)

result = rag.ingest(
    docs={"folder": "files/docs/"},
    chunking={"chunk_size": 900, "chunk_overlap": 120},
    storage={"collection": "docs"},
)

print(f"stored={result.stored} failed={result.failed}")

ragrails ingest \
  --folder files/docs/ \
  --vector-db qdrant \
  --collection docs \
  --url http://localhost:6333 \
  --provider voyage \
  --model voyage-3 \
  --chunk-size 900 \
  --chunk-overlap 120

curl -X POST http://127.0.0.1:8000/v1/pipelines/ingest \
  -H "Content-Type: application/json" \
  -d '{
    "docs": {"folder": "files/docs/"},
    "chunking": {"chunk_size": 900, "chunk_overlap": 120},
    "embedding": {"provider": "voyage", "model": "voyage-3"},
    "storage": {"vector_db": "qdrant", "collection": "docs", "url": "http://localhost:6333"}
  }'

3. Query or chat

hits = rag.query("What does the guide cover?", retrieval={"top_k": 5})
answer = rag.chat("How do I get started?", history=[])

ragrails query "What does the guide cover?" \
  --vector-db qdrant \
  --collection docs \
  --url http://localhost:6333 \
  --provider voyage \
  --model voyage-3 \
  --top-k 5

ragrails chat "How do I get started?" \
  --vector-db qdrant \
  --collection docs \
  --url http://localhost:6333 \
  --embedder-provider voyage \
  --embedder-model voyage-3 \
  --llm-provider openai \
  --llm-model gpt-4o-mini

curl -X POST http://127.0.0.1:8000/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "query": "How do I get started?",
    "llm_provider": "openai",
    "llm_model": "gpt-4o-mini",
    "embedder_provider": "voyage",
    "embedder_model": "voyage-3",
    "vector_db": "qdrant",
    "collection": "docs",
    "url": "http://localhost:6333",
    "history": []
  }'

Uploads over REST

Use /v1/ingest/docs/upload when you only need to parse uploaded files.

curl -X POST http://127.0.0.1:8000/v1/ingest/docs/upload \
  -F "files=@files/guide.pdf" \
  -F "frontmatter=true" \
  -F "title=Guide" \
  -F "description=Product guide"

Use /v1/pipelines/ingest/mixed when uploaded files must be indexed together with paths, folders, remote document URLs, websites, APIs, or Markdown.

curl -X POST http://127.0.0.1:8000/v1/pipelines/ingest/mixed \
  -F 'manifest={
    "docs": [
      {"kind": "upload", "file_ref": "guide_pdf", "title": "Guide"},
      {"kind": "folder", "folder": "files/docs/"},
      {"kind": "url", "url": "https://example.com/files/whitepaper.pdf"}
    ],
    "chunking": {"chunk_size": 900, "chunk_overlap": 120},
    "storage": {"vector_db": "qdrant", "collection": "docs", "url": "http://localhost:6333"}
  }' \
  -F 'file_refs=guide_pdf' \
  -F 'files=@files/guide.pdf'

Next steps