> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ragrails.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat

> Answer questions grounded in your data.

Chat is part of the [Query](/features/query) workflow. It retrieves relevant chunks, builds grounded context, calls an LLM, and returns an answer with sources, updated history, confidence metadata, and any stage errors.

Use this page to understand chat behavior. Use the interface-specific pages when you need exact parameters, flags, request fields, or response shapes.

* [SDK chat](/usage/sdk/chat)
* [CLI chat](/usage/cli/chat)
* [REST chat](/usage/server/chat)

<Note>Chat is stateless. Ragrails does not persist conversations. Pass `history` into each turn and store the returned `history` wherever your application keeps session state.</Note>

## Basic chat

<CodeGroup>
  ```python SDK theme={null}
  from ragrails import RagRails

  rag = RagRails(
      collection="docs",
      vector_store={"provider": "qdrant", "url": "http://localhost:6333"},
      embedding={"provider": "voyage", "model": "voyage-3"},
      llm={"provider": "openai", "model": "gpt-4o-mini"},
  )

  history = []
  result = rag.chat("How do I authenticate?", history=history)
  print(result.answer)

  history = result.history
  ```

  ```bash CLI theme={null}
  ragrails chat "How do I authenticate?"   --vector-db qdrant   --collection docs   --url http://localhost:6333   --embedder-provider voyage   --embedder-model voyage-3   --llm-provider openai   --llm-model gpt-4o-mini   --history-file files/chat-history.json
  ```

  ```bash REST API theme={null}
  curl -X POST http://127.0.0.1:8000/v1/chat   -H "Content-Type: application/json"   -d '{
      "query": "How do I authenticate?",
      "llm_provider": "openai",
      "llm_model": "gpt-4o-mini",
      "embedder_provider": "voyage",
      "embedder_model": "voyage-3",
      "vector_db": "qdrant",
      "collection": "docs",
      "url": "http://localhost:6333",
      "history": []
    }'
  ```
</CodeGroup>

Result:

```json theme={null}
{
  "answer": "Create an API key and include it as a Bearer token in the Authorization header.",
  "sources": [
    {
      "id": "auth#overview",
      "title": "Authentication",
      "score": 0.84,
      "metadata": {"source": "docs/auth.md"}
    }
  ],
  "history": [
    {"role": "user", "content": "How do I authenticate?"},
    {"role": "assistant", "content": "Create an API key and include it as a Bearer token in the Authorization header."}
  ],
  "retrieval": {
    "query": "How do I authenticate?",
    "search_query": "How do I authenticate?",
    "retrieved": 3,
    "failed": 0,
    "outputs": [],
    "errors": []
  },
  "llm": {"provider": "openai", "model": "gpt-4o-mini"},
  "errors": [],
  "retrieval_quality": {"status": "passed", "passed_chunks": 3},
  "answer_confidence": {"level": "high", "reason": "retrieval_quality_pass"},
  "compacted": false,
  "intent": "rag"
}
```

| Key                 | Meaning                                                                                                      |
| ------------------- | ------------------------------------------------------------------------------------------------------------ |
| `answer`            | Generated response. Empty when retrieval or generation fails before an answer can be produced.               |
| `sources`           | Source chunks used to build the grounded prompt. Use this for citations.                                     |
| `history`           | Updated chat history. Persist it and pass it into the next turn.                                             |
| `retrieval`         | Raw retrieval stage summary, including `search_query`, counts, outputs, and retrieval errors.                |
| `llm`               | LLM provider metadata returned by the generation stage.                                                      |
| `errors`            | Chat, retrieval, quality, or generation errors.                                                              |
| `retrieval_quality` | Quality gate status after retrieval.                                                                         |
| `answer_confidence` | UI-friendly confidence level and reason.                                                                     |
| `compacted`         | Whether old history was summarized during the turn.                                                          |
| `intent`            | `"rag"` for retrieval-backed answers, or a direct intent such as greeting/thanks when retrieval is bypassed. |

## Streaming

`chat_stream()` yields progress events, token events, error events, and one final event containing the same `ChatResult` shape.

<CodeGroup>
  ```python SDK theme={null}
  for event in rag.chat_stream("How do I authenticate?", history=[]):
      if event["type"] == "token":
          print(event["data"]["text"], end="")
      elif event["type"] == "final":
          result = event["data"]
  ```

  ```bash CLI theme={null}
  # The CLI chat command prints the completed answer.
  ragrails chat "How do I authenticate?"   --vector-db qdrant   --collection docs   --url http://localhost:6333
  ```

  ```bash REST API theme={null}
  curl -N -X POST http://127.0.0.1:8000/v1/chat/stream   -H "Content-Type: application/json"   -d '{
      "query": "How do I authenticate?",
      "collection": "docs",
      "url": "http://localhost:6333",
      "history": []
    }'
  ```
</CodeGroup>

Example event sequence:

```json theme={null}
{"type":"progress","stage":"chat","message":"Chat started","data":{"query":"How do I authenticate?"},"sequence":1}
{"type":"progress","stage":"retrieval","message":"Retrieval complete","data":{"retrieved":3,"failed":0,"search_query":"How do I authenticate?"},"sequence":3}
{"type":"token","stage":"generation","message":"","data":{"text":"Create"},"sequence":5}
{"type":"final","stage":"complete","message":"Chat complete","data":{"answer":"Create an API key..."},"sequence":32}
```

## Conversation features

| Feature            | Default  | What it does                                                                                 |
| ------------------ | -------- | -------------------------------------------------------------------------------------------- |
| History compaction | Enabled  | Summarizes older turns when history grows past `history_limit`, keeping recent turns intact. |
| Query rewriting    | Disabled | Rewrites follow-ups into standalone search queries before retrieval.                         |
| Intent routing     | Enabled  | Bypasses retrieval for simple greetings, thanks, farewells, and acknowledgements.            |
| Retrieval quality  | Enabled  | Filters weak context and controls low-confidence answer behavior.                            |
| Reranking          | Disabled | Reorders retrieved chunks before the answer prompt is built.                                 |

## Query rewriting

Use query rewriting when follow-up questions need previous context to search correctly.

<CodeGroup>
  ```python SDK theme={null}
  from ragrails import QueryRewriteConfig

  result = rag.chat(
      "What about the second step?",
      history=history,
      query_rewrite=QueryRewriteConfig(
          enabled=True,
          session_context="The user is asking about onboarding setup.",
      ),
  )
  ```

  ```bash CLI theme={null}
  ragrails chat "What about the second step?"   --history-file files/chat-history.json   --rewrite-query   --rewrite-session-context "The user is asking about onboarding setup."   --vector-db qdrant   --collection docs   --url http://localhost:6333
  ```

  ```bash REST API theme={null}
  curl -X POST http://127.0.0.1:8000/v1/chat   -H "Content-Type: application/json"   -d '{
      "query": "What about the second step?",
      "query_rewrite": {
        "enabled": true,
        "session_context": "The user is asking about onboarding setup."
      },
      "collection": "docs",
      "url": "http://localhost:6333",
      "history": []
    }'
  ```
</CodeGroup>

## Reranked chat

Reranking retrieves a wider candidate set, reorders it, and keeps the top reranked chunks for answer generation.

<CodeGroup>
  ```python SDK theme={null}
  from ragrails.core.stg_05_retriever import RetrieverConfig

  result = rag.chat(
      "What is the refund window?",
      history=history,
      retrieval_config=RetrieverConfig(use_rerank=True, rerank_top_k=5),
  )
  ```

  ```bash CLI theme={null}
  ragrails chat "What is the refund window?"   --vector-db qdrant   --collection docs   --url http://localhost:6333   --rerank   --reranker voyage   --reranker-model rerank-2-lite   --rerank-top-k 5
  ```

  ```bash REST API theme={null}
  curl -X POST http://127.0.0.1:8000/v1/chat   -H "Content-Type: application/json"   -d '{
      "query": "What is the refund window?",
      "collection": "docs",
      "url": "http://localhost:6333",
      "rerank": true,
      "reranker": "voyage",
      "reranker_model": "rerank-2-lite",
      "rerank_top_k": 5,
      "history": []
    }'
  ```
</CodeGroup>

## Retrieval quality

Retrieval quality decides how chat behaves when the retrieved context is weak.

<CodeGroup>
  ```python SDK theme={null}
  from ragrails import ChatRetrievalQualityConfig

  result = rag.chat(
      "Can I get a refund after 30 days?",
      history=history,
      retrieval_quality=ChatRetrievalQualityConfig(
          min_retrieval_score=0.35,
          min_rerank_score=0.50,
          low_confidence_mode="answer_with_caution",
          max_context_chunks=5,
      ),
  )
  ```

  ```bash CLI theme={null}
  # The CLI does not expose retrieval-quality thresholds yet.
  ragrails chat "Can I get a refund after 30 days?"   --vector-db qdrant   --collection docs   --url http://localhost:6333
  ```

  ```bash REST API theme={null}
  curl -X POST http://127.0.0.1:8000/v1/chat   -H "Content-Type: application/json"   -d '{
      "query": "Can I get a refund after 30 days?",
      "collection": "docs",
      "url": "http://localhost:6333",
      "retrieval_quality": {
        "min_retrieval_score": 0.35,
        "min_rerank_score": 0.50,
        "low_confidence_mode": "answer_with_caution",
        "max_context_chunks": 5
      },
      "history": []
    }'
  ```
</CodeGroup>

Low-confidence modes:

| Mode                      | Behavior                                                                   |
| ------------------------- | -------------------------------------------------------------------------- |
| `answer_with_caution`     | Generate an answer, but confidence is low and the prompt asks for caution. |
| `ask_clarifying_question` | Ask the user for a narrower question.                                      |
| `refuse_grounded_answer`  | Refuse to answer from weak context.                                        |
| `return_no_answer`        | Return an empty answer and a quality error.                                |

## Possible errors

Most stage-level failures are returned in the result `errors` list instead of raising immediately. REST validation and setup failures may return an exception envelope instead. See [Errors](/reference/errors) for the shared shapes.

<CodeGroup>
  ```json Validation theme={null}
  {
    "answer": "",
    "sources": [],
    "history": [],
    "retrieval": {"retrieved": 0, "failed": 0, "outputs": [], "errors": []},
    "llm": {},
    "errors": [
      {
        "source": "",
        "source_kind": "chat",
        "stage": "validate",
        "error": "query must be a non-empty string",
        "isRetryable": false,
        "attempts": 1
      }
    ],
    "retrieval_quality": {"status": "not_evaluated"},
    "answer_confidence": {"level": "none", "reason": "errors"},
    "compacted": false,
    "intent": "rag"
  }
  ```

  ```json Retrieval failure theme={null}
  {
    "answer": "",
    "sources": [],
    "history": [],
    "retrieval": {
      "retrieved": 0,
      "failed": 1,
      "outputs": [],
      "errors": [
        {
          "source": "How do I authenticate?",
          "source_kind": "query",
          "stage": "retrieve",
          "error": "Collection not found: docs",
          "isRetryable": true,
          "attempts": 1
        }
      ]
    },
    "llm": {},
    "errors": [
      {
        "source": "How do I authenticate?",
        "source_kind": "query",
        "stage": "retrieve",
        "error": "Collection not found: docs",
        "isRetryable": true,
        "attempts": 1
      }
    ],
    "retrieval_quality": {"status": "not_evaluated"},
    "answer_confidence": {"level": "none", "reason": "errors"},
    "compacted": false,
    "intent": "rag"
  }
  ```

  ```json Quality failure theme={null}
  {
    "answer": "",
    "sources": [],
    "history": [{"role": "user", "content": "Can I get a refund after 30 days?"}],
    "retrieval": {"retrieved": 1, "failed": 0, "outputs": [], "errors": []},
    "llm": {},
    "errors": [
      {
        "source": "",
        "source_kind": "chat",
        "stage": "quality",
        "error": "retrieval quality below configured threshold",
        "isRetryable": false,
        "attempts": 1
      }
    ],
    "retrieval_quality": {"status": "low_confidence", "mode": "return_no_answer", "passed_chunks": 0},
    "answer_confidence": {"level": "none", "reason": "retrieval_quality_low_confidence"},
    "compacted": false,
    "intent": "rag"
  }
  ```
</CodeGroup>

## Next steps

* Use [Retrieval](/features/retrieval) when you want chunks without answer generation.
* Use [Pipeline Overview](/features/pipeline-overview) to see how chat fits into the full RAG flow.
* Use [Chat Tuning](/capabilities/chat-tuning) for deeper quality and prompt behavior.