[Epic] Component Marketplace

[Mbeaulne]

Problem

Finding components in Tangle should not depend on users knowing exact component names. Weak discovery creates duplicate components, fragments the ecosystem, and makes it harder for users and AI agents to reuse existing work.

This epic tracks the path from basic component search to a component marketplace: better semantic discovery, clearer trust/lifecycle signals, curated libraries, sharing, and enough context to choose the right component before adding it to a pipeline.

Direction

The default Tangle-UI experience should remain open-source-compatible and work without a hosted/private semantic search endpoint.

The frontend should provide a strong local discovery baseline over the component sources it can already see. Backend-powered semantic search can be added later as an optional provider when an OSS-compatible API exists.

Current direction:

• Keep deterministic local search as the baseline retrieval layer.
• Search across all frontend-visible component sources.
• Add optional AI enhancements that improve ranking or descriptions, but do not make the LLM discover components from scratch.
• Add an API adapter later so hosted/backend search can be used when configured, while preserving local fallback behavior.
• Avoid showing AI results when local search has no signal.

Success criteria

Quantitative

• Backend search adoption exceeds legacy frontend search usage once an OSS-compatible backend search API exists.
• Net-new unique component creation decreases, indicating reuse over duplication.
• Component result clicks and add-to-pipeline actions increase from search surfaces.

Qualitative

• Users can search by description, tags/metadata, behavior, inputs, and outputs, not just title.
• Components show clear lifecycle state: active, deprecated, superseded.
• Teams can curate, organize, and share libraries within their domain.
• Components are shareable by link.
• Component detail views show enough context before adding: usage, examples, compatible components, source, and lifecycle signals.
• Search can surface both components and curated collections, for example ML Team - 26 components.
• Humans and AI agents can discover and reuse existing components instead of creating duplicates.

Current implementation stack

• Component search index service
• Components V2 route shell
• Multi-source Components V2 search and filters
• Component detail view in Components V2
• BYOK/settings hook for AI provider configuration
• Settings UI for AI provider configuration
• Natural-language component rerank service
• AI rerank actions in Components V2 and editor component search
• AI-generated component descriptions in Components V2
• Frontend-only no-match behavior: AI search does not return fallback results when local search has no matches
• Deep AI search hidden behind an off-by-default feature flag

Frontend-only, OSS-compatible work

These items can be implemented in Tangle-UI without requiring a new backend API.

Search quality and local discovery

• Frontend component search index over component name, description, inputs, outputs, metadata, implementation text, source label, and publisher where available
• Multi-source component search UI
• Source grouping and source filters
• Filters and preview cards
• Component detail panel
• Shareable/copyable component references where possible
• Identifier normalization for snake_case, kebab-case, camelCase, plurals, and simple stems
• Curated synonym expansion for common ML/component terms
• Bounded typo tolerance
• Negative query constraints, for example upload a file but not to storage
• Search quality tests for expected queries/results
• Continue tuning scoring against real component examples
• Make why this matched explanations consistent across dashboard and editor surfaces
• Improve empty states and guidance for no-result queries

Optional local semantic layer

• Optional client-side embedding ranking using the configured OpenAI-compatible embeddings provider
• IndexedDB embedding cache for repeated query/component text
• Safe fallback to lexical search when embeddings are unavailable or fail
• Add settings/help copy explaining when local semantic ranking is available
• Add cache management UX if needed

Optional AI enhancement layer

• BYOK provider settings: API base URL, API key, model id
• Provider connection test
• Natural-language reranking over bounded frontend search candidates
• AI-derived component descriptions
• AI unavailable/error states
• Manual and/or beta-flagged automatic description generation
• Relevance score/reason display for AI-ranked results
• Prevent nonsense/no-match queries from returning AI fallback results
• Deep AI candidate cap and no-match guard
• Deep AI feature flag, default off
• Split loading states if Deep AI is enabled broadly later

Marketplace-shaped frontend UX

• Component detail route or URL-addressable selected component state
• Copy link action for selected component/search state
• Lifecycle badges from existing metadata when available
• Deprecation/supersession warnings from existing metadata when available
• Collection/library cards in search results using registered libraries as a frontend collection model
• Collection/library detail views
• Heuristic compatible-component suggestions based on input/output types
• Component YAML/spec preview improvements

Frontend API adapter

• Define a search provider interface that supports local search first and backend search later
• Add capability detection for backend search when configured
• Preserve local fallback when backend search is unavailable
• Keep result shape compatible across local and backend providers

Stretch goals / backend-required work

These require backend/API support and cannot be fully solved in frontend-only OSS code.

OSS-compatible semantic search API

• Semantic search API endpoint in the OSS backend
• Backend search index over name, description, tags/metadata, inputs, outputs, and lifecycle fields
• Backend semantic/KNN search over component text/spec fields
• Schema/capability endpoint for frontend discovery
• Pagination and stable result IDs
• Frontend integration with backend search when configured
• Search telemetry to compare backend search adoption against legacy/local search

Publishing and lifecycle

• Component/published component data model suitable for marketplace use
• Publish workflow API
• Publish flow UI
• Lifecycle state machine: active, deprecated, superseded
• Deprecation reason and migration guidance
• Superseded-by relationship and replacement resolution
• Lifecycle badges in search and detail views
• Warnings/actions when adding deprecated components

Libraries, ownership, and sharing

• Library/collection data model
• Team-scoped ownership and permissions model
• Owner/co-owner management
• Ownership transfer/reassignment for orphaned components
• Library management UI
• Ownership transfer UI
• Add/remove component from library UI
• Shareable component links backed by stable IDs
• Shareable library/collection links
• Collections surfaced in search results

Detail context and recommendations

• Pipeline usage data for a component
• Example pipelines using a component
• Compatible upstream/downstream component recommendations
• Similar/duplicate component detection
• Popularity/reuse/trust signals
• Component detail panel enhancements using backend context

Agent/MCP support

• MCP/API support for component search
• MCP/API support for component detail lookup
• MCP/API support for curated library lookup
• MCP/API support for lifecycle status and replacement suggestions
• Agent guidance to prefer active/non-deprecated reusable components

Deferred / removed from the frontend-only scope

• Frontend will not hard-depend on a hosted/backend-specific experimental semantic search endpoint.
• Component trust/quality scoring should not be invented solely in the frontend unless backed by available metadata. Prefer explicit lifecycle, ownership, usage, and curation signals when backend support exists.
• Deep AI is not a default UX. Keep it feature-flagged until there is stronger product confidence.
• Backend search adoption metrics are deferred until there is an OSS-compatible backend search path to adopt.