feat(rag): Introduce embedding model status checks and UI feedback - #671
Conversation
- Allows RAG projects to display a banner warning if an embedding model is not configured for the current AI provider. - Provides centralized event handling to monitor changes in AI provider configuration status. - Improves user experience by clearly indicating RAG capability status in project views. - Exposes embedding model configuration status through AppContext. Refs: #123
There was a problem hiding this comment.
Pull request overview
This pull request adds embedding-model readiness checks, reactive status updates, localized warning banners, and navigation to AI provider settings for RAG projects.
Changes:
- Tracks embedding configuration through
AppContextand project view models. - Adds provider-save events, UI banners, and localized messages.
- Three critical issues remain: prevent indexing errors when models are unavailable, invalidate/rebuild indexes after model changes, and invalidate cached models after instance-setting changes.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Summary |
|---|---|
shared/src/main/kotlin/io/askimo/core/context/AppContext.kt |
Embedding status check |
desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt |
Provider-save event emission |
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectViewModel.kt |
Detail-view status tracking |
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt |
Detail-view warning banner |
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsViewModel.kt |
List-view status tracking |
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsView.kt |
List-view warning banner |
desktop/src/main/kotlin/io/askimo/desktop/Main.kt |
Settings navigation wiring |
desktop-shared/src/main/resources/i18n/messages.properties |
English strings |
desktop-shared/src/main/resources/i18n/messages_zh_TW.properties |
Traditional Chinese strings |
desktop-shared/src/main/resources/i18n/messages_zh_CN.properties |
Simplified Chinese strings |
desktop-shared/src/main/resources/i18n/messages_vi_VN.properties |
Vietnamese strings |
desktop-shared/src/main/resources/i18n/messages_pt_BR.properties |
Brazilian Portuguese strings |
desktop-shared/src/main/resources/i18n/messages_ko_KR.properties |
Korean strings |
desktop-shared/src/main/resources/i18n/messages_ja_JP.properties |
Japanese strings |
desktop-shared/src/main/resources/i18n/messages_fr.properties |
French strings |
desktop-shared/src/main/resources/i18n/messages_es.properties |
Spanish strings |
desktop-shared/src/main/resources/i18n/messages_de.properties |
German strings |
Suppressed comments (3)
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt:214
- The status is also false when the active factory reports
supportsEmbedding() == false(for example, Anthropic or xAI), but those providers' settings cards do not render an embedding-model selector. The banner still offers “Configure embedding model” and navigates to a screen where that action is impossible; use unsupported-provider copy and a provider-switch action, or hide the link for unsupported providers.
if (!viewModel.embeddingModelConfigured && onNavigateToAiProviderSettings != null) {
embeddingModelNotConfiguredBanner(onConfigureClick = onNavigateToAiProviderSettings)
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsView.kt:318
- This message is also rendered when the active factory reports
supportsEmbedding() == false(for example Anthropic or xAI), not just when a supported provider has a blank model. In that state the settings screen hides the embedding selector, so “Configure embedding model” cannot be completed for the current provider and the guidance is misleading. Use a separate unsupported-provider message/action (such as “Change provider”) or only show this banner for supported providers with a missing model.
text = stringResource("projects.rag.embedding.not.configured"),
shared/src/main/kotlin/io/askimo/core/context/AppContext.kt:408
- The documentation says a sensible default can make this return
true, but the implementation only accepts a non-blank per-instance value;getEmbeddingModel()likewise throws when that value is blank, and no default is consulted. Remove the default claim or implement the same fallback in both status and model creation paths.
* @return true if there is an active instance, its factory supports embeddings, and an
* embedding model has been configured (either explicitly or via a sensible default)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.
Suppressed comments (8)
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt:154
- This effect is keyed by the availability Boolean rather than the embedding identity. Switching from configured model A to configured model B leaves the key
true, so no indexing request is posted and the identity-mismatch check is never reached while this view remains mounted; retrieval can then embed queries with B against A's vectors. Track the embedding identity (or handle the model-save event) and trigger a full re-index when it changes.
LaunchedEffect(currentProject.id, viewModel.embeddingModelConfigured) {
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectViewModel.kt:423
- This handler only refreshes the boolean status. When the old and new providers/models are both embedding-capable, the value remains
true, so ProjectView's LaunchedEffect (keyed by project ID and this boolean) does not enqueue a new index and the existing coordinators continue serving vectors from the old model. Compare the previous embedding identity and request a full re-index when it changes.
refreshEmbeddingModelStatus()
desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt:166
- Changing a utility, vision, image, or embedding override through this card only emits
ProviderInstanceSavedEvent.ChatSessionServiceinvalidates its cachedSessionChatContext/retriever only onModelChangedEvent, so an already-open project session can continue using the old model after this setting changes. Emit the existing model-change invalidation event for the active instance as well (or have the service consume the saved event).
EventBus.emit(
ProviderInstanceSavedEvent(
instanceId = instanceId,
displayName = instance.displayName,
isNewInstance = false,
),
)
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:613
- When
event.knowledgeSources != null, the branch below indexes onlynewSources. If this model-ID mismatch path runs for such a request,cleanupIndexDatahas already deleted all existing vectors, mappings, and metadata, so the old knowledge sources are never rebuilt and disappear from the index. A mismatch must force a full re-index fromproject.knowledgeSourcesinstead of the targeted append path.
if (storedDimension != null && (dimensionMismatch || modelIdMismatch)) {
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:613
- On this mismatch path the coordinators are closed, but the existing
embeddingStores[projectId]is not removed or closed. The later assignment inperformIndexingoverwrites the only reference, potentially orphaning a JVector store and its resources on every model switch. Close/remove the stored embedding store before deleting and replacing the index data, asremoveCoordinatordoes.
if (storedDimension != null && (dimensionMismatch || modelIdMismatch)) {
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:523
handleReIndexRequestcallsremoveCoordinatorbeforegetEmbeddingModel. If a queued re-index runs after the embedding model is cleared, this cleanup removes the existing usable index and the new catch then silently returns, leaving the project unindexed with no failure state. Check model availability before cleanup, or preserve the old index for this recoverable condition.
} catch (e: EmbeddingModelNotConfiguredException) {
// No embedding model configured — this is a normal, recoverable state surfaced
// via the "configure embedding model" banner in ProjectView/ProjectsView, not an
// unexpected error. Skip quietly: no AppErrorEvent (global dialog) and no
// IndexingFailedEvent (which would additionally show a red FAILED indicator in
// the knowledge sources panel, duplicating the banner's messaging).
log.debug("Skipping re-index for project ${event.projectId}: ${e.message}")
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:689
- This quiet path handles only a blank embedding-model field.
isEmbeddingModelConfigured()is also false for an unsupported provider or no active instance, for whichgetEmbeddingModel()throwsUnsupportedOperationExceptionorProviderNotConfiguredException; an already queued request then reaches the generic catch and still shows the global error/FAILED state despite the banner. Treat all configuration-unavailable cases consistently or cancel pending requests when the provider changes.
} catch (e: EmbeddingModelNotConfiguredException) {
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:611
- This identity check protects only indexing requests. Project chat creation independently opens the existing store with the current embedding model (
ChatSessionService.kt:303-323) and does not inspectindex.meta; after switching from a valid model A to B, a resumed session can query A's vectors with B before or without a rebuild. Prevent retrieval until the stored identity matches, or ensure the model-change flow rebuilds the project before exposing its retriever.
val modelIdMismatch = storedModelId != null && currentModelId != null && storedModelId != currentModelId
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Suppressed comments (5)
Previously missed (1) — in code that hasn't changed since the last review.
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:686
- When the model/dimension mismatch branch has just deleted the complete old index, this
newSourcespath still indexes only the newly added sources withappendCoordinators = true. All pre-existing project sources are then absent from the rebuilt index. If cleanup was caused by a mismatch, rebuild fromproject.knowledgeSources(and replace coordinators) instead of appending onlynewSources.
performIndexing(
project = project,
knowledgeSources = newSources,
embeddingStore = embeddingStore,
embeddingModel = embeddingModel,
embeddingModelId = currentModelId,
desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt:165
- This is the only event emitted when an embedding override changes, but
ChatSessionServicecurrently invalidates its cached per-session clients/retrievers only forModelChangedEvent(shared/.../ChatSessionService.kt:193-216). A session opened before this save can therefore keep a retriever built with the old embedding model while the project is reindexed with the new one, producing incompatible retrieval results. Handle this event in the session service or emit a cache-invalidation event that it consumes.
EventBus.emit(
ProviderInstanceSavedEvent(
instanceId = instanceId,
displayName = instance.displayName,
isNewInstance = false,
),
shared/src/main/kotlin/io/askimo/core/context/AppContext.kt:451
- This identity does not include the provider instance or endpoint. Two
OPENAI_COMPATIBLEinstances can point at different servers while using the same model name, yet both produceOPENAI_COMPATIBLE:<model>. After switching instances, the indexer can therefore pass the duplicate guard and reuse vectors from the old endpoint with queries embedded by the new endpoint. Include at least the instance identity and the endpoint/configuration fingerprint so such a switch forces a rebuild.
fun activeEmbeddingModelIdentity(): String? {
val instance = getActiveInstance() ?: return null
return "${instance.providerType}:${instance.settings.embeddingModel}"
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:711
getEmbeddingModel()throwsProviderNotConfiguredExceptionwhen there is no active instance andUnsupportedOperationExceptionwhen the active factory does not support embeddings. The new banner explicitly represents both states as normal RAG-unavailable states, but this catch handles only a blank model; creating a project with sources throughProjectServicestill emits an indexing event and reaches the generic error path, showing an error dialog and FAILED state. Treat all expected unavailable-embedding cases as the same recoverable path, preferably with a dedicated exception.
} catch (e: EmbeddingModelNotConfiguredException) {
// No embedding model configured — this is a normal, recoverable state surfaced
// via the "configure embedding model" banner in ProjectView/ProjectsView, not an
// unexpected error. Skip quietly: no AppErrorEvent (global dialog) and no
// IndexingFailedEvent (which would additionally show a red FAILED indicator in
// the knowledge sources panel, duplicating the banner's messaging).
log.debug("Skipping indexing for project ${event.projectId}: ${e.message}")
shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:620
- The model and
currentModelIdare read at different times, with network embedding work occurring between them. A provider switch during that work can pair the oldembeddingModelwith the new identity, so the subsequent rebuild may tag old vectors as the new model and suppress future mismatch detection. Capture both values atomically before running the checks.
val currentDimension = RagUtils.getDimensionForModel(embeddingModel)
val currentModelId = appContext.activeEmbeddingModelIdentity()
Refs: #123