Chunking - Ragrails

ragrails chunk reads extraction output JSON from --input-dir and writes chunks.json to --output-dir. Use it when you want to inspect or version chunks before embedding. If you do not need intermediate files, use ragrails ingest instead.

Command

ragrails chunk \
  --input-dir files/output/docs/ \
  --output-dir files/output/chunks/ \
  --chunk-size 2000 \
  --chunk-overlap 200 \
  --min-chunk-length 100

Output:

Documents : 12
Chunks    : 87
Failed    : 0
Saved to  : files/output/chunks/chunks.json

Choosing Chunk Size

Start at --chunk-size 2000 with --chunk-overlap 200. Lower the size for dense factual content like FAQs, specs, or policies. Raise it when answers need more surrounding context.

Options

Option	Default	Description
`--input-dir`	required	Folder containing extraction output JSON files.
`--output-dir`	required	Folder to write `chunks.json`.
`--chunk-size`	`2000`	Target max characters per chunk.
`--chunk-overlap`	`200`	Character overlap between adjacent chunks.
`--min-chunk-length`	`100`	Drop chunks shorter than this.

Next Stage

ragrails embed --input-dir files/output/chunks/ --output-dir files/output/embedded/

Extraction

Create document JSON.

Embedding

Embed chunk JSON.