Build From an API

Use this walkthrough when your source material comes from an API: product catalogs, tickets, changelogs, help articles, accounts, or internal records exposed as JSON.

What you will build

A products collection from a paginated API endpoint, then retrieval and chat over the fetched JSON content.

Prerequisites

pip install "ragrails[voyage,qdrant]"
export VOYAGE_API_KEY="..."
export OPENAI_API_KEY="..."
docker run -p 6333:6333 qdrant/qdrant

1. Configure pagination

API style	Pagination config
Page numbers	`{"type": "page", "param": "page", "size_param": "per_page", "size": 100}`
Offset	`{"type": "offset", "param": "offset", "size_param": "limit", "size": 100}`
Cursor	`{"type": "cursor", "param": "cursor", "cursor_path": "meta.next_cursor"}`

Always set max_pages as a safety cap.

2. Ingest the API

from ragrails import RagRails

rag = RagRails(
    collection="products",
    vector_store={"provider": "qdrant", "url": "http://localhost:6333"},
    embedding={"provider": "voyage", "model": "voyage-3"},
    llm={"provider": "openai", "model": "gpt-4o-mini"},
)

result = rag.ingest(
    api={
        "url": "https://api.example.com/products",
        "title": "Products",
        "headers": {"Authorization": "Bearer sk_live_..."},
        "pagination": {"type": "page", "param": "page", "size_param": "per_page", "size": 100},
        "max_pages": 10,
    },
    storage={"collection": "products"},
)

print(result.sources, result.stored, result.failed)

ragrails ingest \
  --api-url https://api.example.com/products \
  --vector-db qdrant \
  --collection products \
  --url http://localhost:6333 \
  --provider voyage \
  --model voyage-3

curl -X POST http://127.0.0.1:8000/v1/pipelines/ingest \
  -H "Content-Type: application/json" \
  -d '{
    "api": {
      "url": "https://api.example.com/products",
      "title": "Products",
      "headers": {"Authorization": "Bearer sk_live_..."},
      "pagination": {"type": "page", "param": "page", "size_param": "per_page", "size": 100},
      "max_pages": 10
    },
    "embedding": {"provider": "voyage", "model": "voyage-3"},
    "storage": {"vector_db": "qdrant", "collection": "products", "url": "http://localhost:6333"}
  }'

The CLI pipeline command accepts --api-url for simple GET endpoints. Use SDK or REST when you need headers, request bodies, or pagination config.

3. Query or chat

hits = rag.query("Which products are waterproof?", retrieval={"top_k": 5})
answer = rag.chat("Recommend a waterproof jacket under $200", history=[])

ragrails query "Which products are waterproof?" \
  --vector-db qdrant \
  --collection products \
  --url http://localhost:6333 \
  --provider voyage \
  --model voyage-3 \
  --top-k 5

ragrails chat "Recommend a waterproof jacket under $200" \
  --vector-db qdrant \
  --collection products \
  --url http://localhost:6333 \
  --embedder-provider voyage \
  --embedder-model voyage-3 \
  --llm-provider openai \
  --llm-model gpt-4o-mini

curl -X POST http://127.0.0.1:8000/v1/pipelines/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Which products are waterproof?",
    "embedding": {"provider": "voyage", "model": "voyage-3"},
    "retrieval": {"vector_db": "qdrant", "collection": "products", "url": "http://localhost:6333", "top_k": 5}
  }'

Handling failures

API request failures are returned with isRetryable and retry_input when they can be retried. See Resilient Ingestion.

Get Started

Features

Capabilities

Use Cases

Walkthroughs

Build From an API

What you will build

Prerequisites

2. Ingest the API

3. Query or chat

Handling failures

Next steps

​What you will build

​Prerequisites

​1. Configure pagination

​2. Ingest the API

​3. Query or chat

​Handling failures

​Next steps

What you will build

Prerequisites

1. Configure pagination

2. Ingest the API

3. Query or chat

Handling failures

Next steps