LCORE-2505: Add migration guide and deprecation notes for BYOK config refactoring

[are-ces]

Description

Adds a v0.7.0 migration guide and deprecation notices for the RAG configuration refactoring (PR #1843 / LCORE-1426). The migration guide documents the move from four separate top-level sections (byok_rag, rag, okp, reranker) to a unified nested rag section, including:

• Field-by-field mapping table (old path → new path)
• Full before/after YAML examples with inline comments
• Chunk limit constants → config fields migration table

Deprecation callouts are added to byok_guide.md and rag_guide.md linking to the migration guide.

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up service version
• Bump-up dependent library
• Bump-up library or tool used for development (does not change the final image)
• CI configuration change
• Konflux configuration change
• Unit tests improvement
• Integration tests improvement
• End to end tests improvement
• Benchmarks improvement

Tools used to create PR

• Assisted-by: Claude Code
• Generated by: Claude Opus 4.6

Related Tickets & Documents

• Closes LCORE-2505
• Related: PR LCORE-1426: BYOK Config refactoring #1843 (LCORE-1426)

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Docs-only change; verify markdown renders correctly (tables, callouts, YAML blocks)
• Confirm field mapping table matches the changes in PR LCORE-1426: BYOK Config refactoring #1843

Summary by CodeRabbit

• Documentation
  • Added v0.7.0 migration guide with configuration restructuring details and before/after YAML examples.
  • Added deprecation warnings to configuration guides, directing users to updated setup procedures and the migration guide for version 0.7.0.

[coderabbitai]

Walkthrough

This PR adds comprehensive v0.7.0 migration documentation for RAG configuration restructuring, consolidating four top-level configuration sections into a unified rag section with nested subsections. The PR creates a new migration guide and updates existing documentation with deprecation warnings pointing to the new guide.

Changes

RAG Configuration Restructuring Documentation

Layer / File(s)	Summary
RAG Configuration Restructuring Overview docs/migrations/v0.7.0.md	Introduces the v0.7.0 migration guide describing consolidation of byok_rag, rag, okp, and reranker into a single unified rag section. Includes a detailed field-by-field mapping table showing old-to-new configuration paths, the rag_type → backend rename, and removal of inline::/remote:: prefixes.
Configuration Transformation Examples docs/migrations/v0.7.0.md	Provides complete before/after YAML examples demonstrating the full v0.6.x-to-v0.7.0 transformation, showing BYOK store backends, retrieval inline/tool sources, reranker placement, and configurable max-chunk fields within the new nested structure.
Chunk Limits Configuration Migration docs/migrations/v0.7.0.md	Documents the migration of hardcoded chunk-limit constants (RAG_CHUNK_LIMIT, OKP_CHUNK_LIMIT, etc.) to configurable fields in lightspeed-stack.yaml with equivalent defaults mapped to rag.byok.max_chunks, rag.okp.max_chunks, rag.retrieval.inline.max_chunks, and rag.retrieval.tool.max_chunks.
Deprecation Warnings in Existing Guides docs/byok_guide.md, docs/rag_guide.md	Adds two deprecation callouts in existing documentation warning that top-level RAG configuration sections and chunk-limit constants are deprecated as of v0.7.0, describing the new unified rag structure and linking to the migration guide.

---

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

---

Possibly related PRs

• lightspeed-core/lightspeed-stack#1809: Both PRs touch the documented RAG/reranker configuration shape; this PR adds v0.7.0 deprecation and migration guidance for the unified rag section structure that includes reranker, while the other PR regenerates the configuration schema and docs with a reranker configuration section.

---

Suggested reviewers

• tisnik
• syedriko

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically identifies the main change: adding migration guide and deprecation notes for BYOK config refactoring, which matches the file changes and PR objectives.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

✨ Simplify code

• Create PR with simplified code

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[are-ces]

Should we include deprecation warning in logs? My concern is that if we do, users cannot already migrate to the new config format which will be confusing.

What do you think? @syedriko @tisnik

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/migrations/v0.7.0.md`:
- Around line 11-27: The migration doc describes moving fields into rag.byok,
rag.okp and rag.retrieval.* but the runtime schema still exposes top-level
byok_rag, okp, reranker and rag.inline/rag.tool and code
(validate_reranker_auto_enable) reads self.reranker and self.rag.inline; update
the doc to match the actual v0.7.0 schema by changing the Field Mapping and
examples to show the current top-level paths (byok_rag, okp, reranker,
rag.inline/rag.tool) OR explicitly mark this migration as targeting a future
schema and add version gating guidance; also mention
validate_reranker_auto_enable, self.reranker, self.rag.inline and the top-level
config names so maintainers can reconcile examples with the runtime model.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 45cf17c1-0c9a-4824-ae83-763a7dddb35c

📥 Commits

Reviewing files that changed from the base of the PR and between 2606037 and f9f362b.

📒 Files selected for processing (3)

• docs/byok_guide.md
• docs/migrations/v0.7.0.md
• docs/rag_guide.md

📜 Review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)

• GitHub Check: E2E Tests for Lightspeed Evaluation job
• GitHub Check: E2E: library mode / ci / group 1
• GitHub Check: mypy
• GitHub Check: black
• GitHub Check: unit_tests (3.13)
• GitHub Check: unit_tests (3.12)
• GitHub Check: bandit
• GitHub Check: Pylinter
• GitHub Check: build-pr
• GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request

🔇 Additional comments (2)

docs/byok_guide.md (1)

116-123: LGTM!

docs/rag_guide.md (1)

66-73: LGTM!

Also applies to: 323-328

[tisnik]

LGTM