fix: allow multi-connection to Any-typed pipeline sockets - #12362
fix: allow multi-connection to Any-typed pipeline sockets#12362Harsh23Kashyap wants to merge 1 commit into
Conversation
`Pipeline.connect` rejected a second connection to an `Any`-typed input socket (e.g. `PromptBuilder.documents`, where every template variable is registered as `Any` via `component.set_input_type(self, var, Any)`) with a `PipelineConnectError`. The previous behaviour was the gap left after deepset-ai#10783 — its commit message claimed `Any`-type support, but `_make_socket_auto_variadic` only handled `list`, `Optional[list]`, and union-of-list receivers. The fix adds a dedicated branch for `Any`-typed receivers: when the second connection arrives, the socket is marked lazy variadic. The framework has no element-type information for an `Any`, so `wrap_input_in_list` is left at the default `True` and the receiver gets `[sender_1, sender_2, ...]` at runtime — a list of the per-sender values. The component decides how to handle the type-erased inputs (e.g. `PromptBuilder` can flatten the list of lists in its own `run()` if desired). Tests: added `TestMakeSocketAutoVariadic.test_successful_any_typed_socket` (direct unit test on the new branch) and `TestAnyTypedMultiConnection.test_prompt_builder_documents_accepts_multiple_retrievers` (end-to-end via `PromptBuilder`). Updated the error message regex in `TestValidateInput.test_validate_multiple_connections_to_non_variadic_input` to match the extended message ("or Any" appended). This is the connection-eligibility half of #10721; rendering-side aggregation (e.g. flattening inside `PromptBuilder`) is a natural follow-up since `Any` carries no list-element type to flatten against. Fixes #10721
|
@Harsh23Kashyap is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @Harsh23Kashyap, thanks for your interest in contributing to Haystack! 🙏 This is an automated message to help us keep the review queue healthy. |
|
|
|
Hi @Harsh23Kashyap, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the |
HaystackBot
added
the
cla-pending
4 participants
Related Issues
Proposed Changes:
Pipeline.connectrejected a second connection to anAny-typed input socket (e.g.PromptBuilder.documents, where every template variable is registered asAnyviacomponent.set_input_type(self, var, Any)) with aPipelineConnectError:This was the gap left after #10783. The merged PR's commit message explicitly claimed
Any-type support ("We now allow a component's whose input type is typed asAnyto allow multiple input connections as long as the input connections are all of type list"), but the actual_make_socket_auto_variadiconly handledlist/Optional[list]/ union-of-list receivers — theAnycase fell through to theraise error_type(...)branch. (The same branch that produced the error message above.)The fix is one new branch in
_make_socket_auto_variadic: when the receiver socket's type isAny, mark the socketis_lazy_variadic = Trueand leavewrap_input_in_listat the defaultTrue. The framework has no element-type information forAny, so it cannot safely flatten one level; the component receives[sender_1, sender_2, ...]and decides how to handle the type-erased inputs (aPromptBuilder-style component can flatten in its ownrun()if it always aggregates lists of lists).This is the connection-eligibility half of deepset-ai/haystack-private#556. The rendering-side aggregation (flattening
[[d1, d2], [d3, d4]]to a flat list inPromptBuilder.run()) is a natural follow-up — I left it out of this PR to keep the scope minimal: the framework now allows the multi-connection that the issue is fundamentally about, and a component that knows the intent can flatten internally.How did you test it?
TestMakeSocketAutoVariadic.test_successful_any_typed_socket— the newAnybranch is exercised with the exactInputSocket(name="documents", type=Any, senders=["retriever_1"])shape thatPromptBuilderproduces. Assertsis_variadic,is_lazy_variadic, and thatwrap_input_in_liststaysTrue(the safe default for an unknown element type).TestAnyTypedMultiConnection.test_prompt_builder_documents_accepts_multiple_retrievers— spins up a realInMemoryDocumentStore, twoInMemoryBM25Retrievers, and aPromptBuilder, then asserts bothconnectcalls succeed. This is the exact reproducer from the issue.TestValidateInput.test_validate_multiple_connections_to_non_variadic_inputregex to match the extended error message ("or Any"appended).TestMakeSocketAutoVariadiccases (list / union-of-lists / Optional[list]) all still pass — the newAnybranch is added before the existing dispatch, not in place of it.test/core/pipeline/suite: 416 passed, no regressions. Broadertest/core/+test/components/builders/(which exercises the component side of multi-connection): 1904 passed, no regressions.hatch run fmtclean.hatch run test:typesreports 17 pre-existing errors in 9 unrelated files (azure, openai_counter, etc.) — confirmed pre-existing viagit stashof this branch.Checklist
releasenotes/notes/)hatch run fmtandhatch run test:typeslocally