Skip to content

feat(rag): Introduce embedding model status checks and UI feedback - #671

Merged
haiphucnguyen merged 4 commits into
mainfrom
feature/check-embedding-model-status-and-show-banner
Aug 24, 2026
Merged

feat(rag): Introduce embedding model status checks and UI feedback#671
haiphucnguyen merged 4 commits into
mainfrom
feature/check-embedding-model-status-and-show-banner

Conversation

@haiphucnguyen

Copy link
Copy Markdown
Collaborator
  • Allows RAG projects to display a banner warning if an embedding model is not configured for the current AI provider.
  • Provides centralized event handling to monitor changes in AI provider configuration status.
  • Improves user experience by clearly indicating RAG capability status in project views.
  • Exposes embedding model configuration status through AppContext.

Refs: #123

- Allows RAG projects to display a banner warning if an embedding model is not configured for the current AI provider.
- Provides centralized event handling to monitor changes in AI provider configuration status.
- Improves user experience by clearly indicating RAG capability status in project views.
- Exposes embedding model configuration status through AppContext.

Refs: #123
Copilot AI lite review requested due to automatic review settings August 23, 2026 23:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds embedding-model readiness checks, reactive status updates, localized warning banners, and navigation to AI provider settings for RAG projects.

Changes:

  • Tracks embedding configuration through AppContext and project view models.
  • Adds provider-save events, UI banners, and localized messages.
  • Three critical issues remain: prevent indexing errors when models are unavailable, invalidate/rebuild indexes after model changes, and invalidate cached models after instance-setting changes.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Summary
shared/src/main/kotlin/io/askimo/core/context/AppContext.kt Embedding status check
desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt Provider-save event emission
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectViewModel.kt Detail-view status tracking
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt Detail-view warning banner
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsViewModel.kt List-view status tracking
desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsView.kt List-view warning banner
desktop/src/main/kotlin/io/askimo/desktop/Main.kt Settings navigation wiring
desktop-shared/src/main/resources/i18n/messages.properties English strings
desktop-shared/src/main/resources/i18n/messages_zh_TW.properties Traditional Chinese strings
desktop-shared/src/main/resources/i18n/messages_zh_CN.properties Simplified Chinese strings
desktop-shared/src/main/resources/i18n/messages_vi_VN.properties Vietnamese strings
desktop-shared/src/main/resources/i18n/messages_pt_BR.properties Brazilian Portuguese strings
desktop-shared/src/main/resources/i18n/messages_ko_KR.properties Korean strings
desktop-shared/src/main/resources/i18n/messages_ja_JP.properties Japanese strings
desktop-shared/src/main/resources/i18n/messages_fr.properties French strings
desktop-shared/src/main/resources/i18n/messages_es.properties Spanish strings
desktop-shared/src/main/resources/i18n/messages_de.properties German strings
Suppressed comments (3)

desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt:214

  • The status is also false when the active factory reports supportsEmbedding() == false (for example, Anthropic or xAI), but those providers' settings cards do not render an embedding-model selector. The banner still offers “Configure embedding model” and navigates to a screen where that action is impossible; use unsupported-provider copy and a provider-switch action, or hide the link for unsupported providers.
                    if (!viewModel.embeddingModelConfigured && onNavigateToAiProviderSettings != null) {
                        embeddingModelNotConfiguredBanner(onConfigureClick = onNavigateToAiProviderSettings)

desktop/src/main/kotlin/io/askimo/desktop/project/ProjectsView.kt:318

  • This message is also rendered when the active factory reports supportsEmbedding() == false (for example Anthropic or xAI), not just when a supported provider has a blank model. In that state the settings screen hides the embedding selector, so “Configure embedding model” cannot be completed for the current provider and the guidance is misleading. Use a separate unsupported-provider message/action (such as “Change provider”) or only show this banner for supported providers with a missing model.
                    text = stringResource("projects.rag.embedding.not.configured"),

shared/src/main/kotlin/io/askimo/core/context/AppContext.kt:408

  • The documentation says a sensible default can make this return true, but the implementation only accepts a non-blank per-instance value; getEmbeddingModel() likewise throws when that value is blank, and no default is consulted. Remove the default claim or implement the same fallback in both status and model creation paths.
     * @return true if there is an active instance, its factory supports embeddings, and an
     *         embedding model has been configured (either explicitly or via a sensible default)

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Suppressed comments (8)

desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt:154

  • This effect is keyed by the availability Boolean rather than the embedding identity. Switching from configured model A to configured model B leaves the key true, so no indexing request is posted and the identity-mismatch check is never reached while this view remains mounted; retrieval can then embed queries with B against A's vectors. Track the embedding identity (or handle the model-save event) and trigger a full re-index when it changes.
    LaunchedEffect(currentProject.id, viewModel.embeddingModelConfigured) {

desktop/src/main/kotlin/io/askimo/desktop/project/ProjectViewModel.kt:423

  • This handler only refreshes the boolean status. When the old and new providers/models are both embedding-capable, the value remains true, so ProjectView's LaunchedEffect (keyed by project ID and this boolean) does not enqueue a new index and the existing coordinators continue serving vectors from the old model. Compare the previous embedding identity and request a full re-index when it changes.
                refreshEmbeddingModelStatus()

desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt:166

  • Changing a utility, vision, image, or embedding override through this card only emits ProviderInstanceSavedEvent. ChatSessionService invalidates its cached SessionChatContext/retriever only on ModelChangedEvent, so an already-open project session can continue using the old model after this setting changes. Emit the existing model-change invalidation event for the active instance as well (or have the service consume the saved event).
            EventBus.emit(
                ProviderInstanceSavedEvent(
                    instanceId = instanceId,
                    displayName = instance.displayName,
                    isNewInstance = false,
                ),
            )

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:613

  • When event.knowledgeSources != null, the branch below indexes only newSources. If this model-ID mismatch path runs for such a request, cleanupIndexData has already deleted all existing vectors, mappings, and metadata, so the old knowledge sources are never rebuilt and disappear from the index. A mismatch must force a full re-index from project.knowledgeSources instead of the targeted append path.
            if (storedDimension != null && (dimensionMismatch || modelIdMismatch)) {

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:613

  • On this mismatch path the coordinators are closed, but the existing embeddingStores[projectId] is not removed or closed. The later assignment in performIndexing overwrites the only reference, potentially orphaning a JVector store and its resources on every model switch. Close/remove the stored embedding store before deleting and replacing the index data, as removeCoordinator does.
            if (storedDimension != null && (dimensionMismatch || modelIdMismatch)) {

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:523

  • handleReIndexRequest calls removeCoordinator before getEmbeddingModel. If a queued re-index runs after the embedding model is cleared, this cleanup removes the existing usable index and the new catch then silently returns, leaving the project unindexed with no failure state. Check model availability before cleanup, or preserve the old index for this recoverable condition.
        } catch (e: EmbeddingModelNotConfiguredException) {
            // No embedding model configured — this is a normal, recoverable state surfaced
            // via the "configure embedding model" banner in ProjectView/ProjectsView, not an
            // unexpected error. Skip quietly: no AppErrorEvent (global dialog) and no
            // IndexingFailedEvent (which would additionally show a red FAILED indicator in
            // the knowledge sources panel, duplicating the banner's messaging).
            log.debug("Skipping re-index for project ${event.projectId}: ${e.message}")

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:689

  • This quiet path handles only a blank embedding-model field. isEmbeddingModelConfigured() is also false for an unsupported provider or no active instance, for which getEmbeddingModel() throws UnsupportedOperationException or ProviderNotConfiguredException; an already queued request then reaches the generic catch and still shows the global error/FAILED state despite the banner. Treat all configuration-unavailable cases consistently or cancel pending requests when the provider changes.
        } catch (e: EmbeddingModelNotConfiguredException) {

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:611

  • This identity check protects only indexing requests. Project chat creation independently opens the existing store with the current embedding model (ChatSessionService.kt:303-323) and does not inspect index.meta; after switching from a valid model A to B, a resumed session can query A's vectors with B before or without a rebuild. Prevent retrieval until the stored identity matches, or ensure the model-change flow rebuilds the project before exposing its retriever.
            val modelIdMismatch = storedModelId != null && currentModelId != null && storedModelId != currentModelId

Comment thread shared/src/main/kotlin/io/askimo/core/context/AppContext.kt
Comment thread shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Suppressed comments (5)

Previously missed (1) — in code that hasn't changed since the last review.

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:686

  • When the model/dimension mismatch branch has just deleted the complete old index, this newSources path still indexes only the newly added sources with appendCoordinators = true. All pre-existing project sources are then absent from the rebuilt index. If cleanup was caused by a mismatch, rebuild from project.knowledgeSources (and replace coordinators) instead of appending only newSources.
                    performIndexing(
                        project = project,
                        knowledgeSources = newSources,
                        embeddingStore = embeddingStore,
                        embeddingModel = embeddingModel,
                        embeddingModelId = currentModelId,

desktop/src/main/kotlin/io/askimo/desktop/settings/AIProviderViewModel.kt:165

  • This is the only event emitted when an embedding override changes, but ChatSessionService currently invalidates its cached per-session clients/retrievers only for ModelChangedEvent (shared/.../ChatSessionService.kt:193-216). A session opened before this save can therefore keep a retriever built with the old embedding model while the project is reindexed with the new one, producing incompatible retrieval results. Handle this event in the session service or emit a cache-invalidation event that it consumes.
            EventBus.emit(
                ProviderInstanceSavedEvent(
                    instanceId = instanceId,
                    displayName = instance.displayName,
                    isNewInstance = false,
                ),

shared/src/main/kotlin/io/askimo/core/context/AppContext.kt:451

  • This identity does not include the provider instance or endpoint. Two OPENAI_COMPATIBLE instances can point at different servers while using the same model name, yet both produce OPENAI_COMPATIBLE:<model>. After switching instances, the indexer can therefore pass the duplicate guard and reuse vectors from the old endpoint with queries embedded by the new endpoint. Include at least the instance identity and the endpoint/configuration fingerprint so such a switch forces a rebuild.
    fun activeEmbeddingModelIdentity(): String? {
        val instance = getActiveInstance() ?: return null
        return "${instance.providerType}:${instance.settings.embeddingModel}"

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:711

  • getEmbeddingModel() throws ProviderNotConfiguredException when there is no active instance and UnsupportedOperationException when the active factory does not support embeddings. The new banner explicitly represents both states as normal RAG-unavailable states, but this catch handles only a blank model; creating a project with sources through ProjectService still emits an indexing event and reaches the generic error path, showing an error dialog and FAILED state. Treat all expected unavailable-embedding cases as the same recoverable path, preferably with a dedicated exception.
        } catch (e: EmbeddingModelNotConfiguredException) {
            // No embedding model configured — this is a normal, recoverable state surfaced
            // via the "configure embedding model" banner in ProjectView/ProjectsView, not an
            // unexpected error. Skip quietly: no AppErrorEvent (global dialog) and no
            // IndexingFailedEvent (which would additionally show a red FAILED indicator in
            // the knowledge sources panel, duplicating the banner's messaging).
            log.debug("Skipping indexing for project ${event.projectId}: ${e.message}")

shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt:620

  • The model and currentModelId are read at different times, with network embedding work occurring between them. A provider switch during that work can pair the old embeddingModel with the new identity, so the subsequent rebuild may tag old vectors as the new model and suppress future mismatch detection. Capture both values atomically before running the checks.
            val currentDimension = RagUtils.getDimensionForModel(embeddingModel)
            val currentModelId = appContext.activeEmbeddingModelIdentity()

Comment thread desktop/src/main/kotlin/io/askimo/desktop/project/ProjectView.kt
Comment thread shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt Outdated
Comment thread shared/src/main/kotlin/io/askimo/core/rag/ProjectIndexer.kt
@haiphucnguyen
haiphucnguyen merged commit 3d6af35 into main Aug 24, 2026
10 checks passed
@haiphucnguyen
haiphucnguyen deleted the feature/check-embedding-model-status-and-show-banner branch August 24, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants