Feature: source dt4acc integration lattice/config from pyaml-test-lattice, pin companion deps

[gupichon]

Description, motivation and use case
The dt4acc integration pipeline (.github/workflows/dt4acc-integration.yml +
tests/integration/test_dt4acc_twin_smoke.py) currently vendors its own hand-maintained copy of
the FODO_1GeV_6D test lattice and its pyaml/dt4acc/catalog configuration fragments directly under
tests/integration/data/. This is the same synthetic FODO lattice that has since been extracted
into its own dedicated package, pyaml-test-lattice,
intended as the shared source of truth for pyAML integration-test/tutorial fixtures across the
ecosystem (pyaml, tango-pyaml, pyaml-cs-oa, dt4acc).

Keeping a hand-maintained duplicate here means:

• lattice/config fixes made in pyaml-test-lattice don't propagate to this repo automatically, so
  the FODO fixtures can silently drift from what companion packages test against;
• the pipeline installs its control-system companions (tango-pyaml, pyaml-cs-oa) unpinned from
  git main, so a breaking change there (e.g. to the readback()/get() return contract) can land
  in this CI without warning — this is no longer hypothetical, see the PR Fix typing for _fill_device() methods #393 incident under
  Additional context below.

Proposed solution

1. Add pyaml-test-lattice as a test/integration dependency, pinned via a git VCS URL to a
   specific tag or commit (it has no PyPI release yet — only alpha tags 0.1.0a1/0.1.0a2 — and a
   git pin is wanted regardless, so the same setup works for local dev/integration runs, not just CI).
2. This depends on pyaml-test-lattice first shipping the accelerator/catalog configuration
   fragments this pipeline actually needs (lattice, dt4acc accelerator-setup, tango-pyaml
   pyaml+catalog config, pyaml-cs-oa pyaml+catalog config) — that authoring work, including the
   choice of static vs. dynamic Tango catalog, is happening in parallel in that repo and isn't
   pyaml's call to make. Track/coordinate with that work rather than migrating unilaterally.
3. Once available, replace tests/integration/data/fodo_1gev_6d/* with references resolved
   through pyaml_test_lattice.lattices / pyaml_test_lattice.configurations, updating
   test_dt4acc_twin_smoke.py and the --lattice-file/--accelerator-setup-file arguments in
   dt4acc-integration.yml accordingly.
4. Pin tango-pyaml and pyaml-cs-oa in dt4acc-integration.yml to a specific tag/commit instead
   of tracking main unpinned, so a breaking companion-package change can't silently break this
   pipeline; bump the pin deliberately when a companion release is meant to be picked up.

Describe alternatives you've considered

• Keep vendoring the lattice/config here and only fix the immediate pinning gap: rejected, it
  doesn't address the duplication/drift with pyaml-test-lattice and companion repos that the new
  shared package exists to solve.
• Pin tango-pyaml/pyaml-cs-oa to a main commit SHA updated manually on every merge: a possible
  short-term stop-gap, but doesn't scale; prefer tagged releases on those repos with the pin bumped
  deliberately.

Example

from pyaml_test_lattice import lattices, configurations

lattice_path = lattices["fodo_1gev_6d.json"]
accel_config_path = configurations["pyaml/tango/tango-pyaml/fodo_1gev_6d_pyaml.yaml"]

used in place of the current hardcoded Path(__file__).parent / "data" / "fodo_1gev_6d" / ... in
tests/integration/test_dt4acc_twin_smoke.py, and equivalently to build the --lattice-file /
--accelerator-setup-file arguments consumed by tests/integration/start_dt4acc_twin.sh in
dt4acc-integration.yml. (Exact registry keys to use depend on what pyaml-test-lattice ends up
shipping — see point 2 above.)

Additional context

• pyaml-test-lattice is still Alpha with only pre-release tags (0.1.0a1, 0.1.0a2); a stable
  0.1.0 release + PyPI publish would be a nice follow-up but isn't a blocker since we pin via git
  regardless.
• The registry key path in pyaml-test-lattice's own README example
  (pyaml/tango/fodo_1gev_6d_pyaml.yaml) doesn't match the actual on-disk path
  (pyaml/tango/tango-pyaml/fodo_1gev_6d_pyaml.yaml) — worth a heads-up/PR on that repo too.
• Two smaller issues surfaced in tango-pyaml while investigating this (not blockers here, but
  worth filing separately in that repo): a dormant AttributeError in
  TangoControlSystem.get_device_access()'s error-handling path (it calls catalog.get_name(),
  which no catalog class defines), and a stale agent-notes.md there still describing the
  pre-rename get_device()/get_devices() API.
• Concrete incident confirming the need for point 4 (2026-09-10): on
  PR #393 ("Fix typing for
  _fill_device() methods" — a pyaml-only typing change, unrelated to control-system backends),
  Jean-Luc Pons reported the dt4acc smoke test failing only for the tango-pyaml parametrization:

  test_dt4acc_twin_accelerator_instantiates_and_reads_live_values[tango-pyaml] FAILED
  test_dt4acc_twin_accelerator_instantiates_and_reads_live_values[pyaml-cs-oa] PASSED
  ...
  for attribute in self._attributes:
  E       TypeError: 'NoneType' object is not iterable
  .../tango/pyaml/multi_attribute.py:60: TypeError
  

  and confirmed it reproduces locally and worked with his previous tango-pyaml release. Since the
  PR itself touches no control-system code, this is a breaking change that landed on tango-pyaml
main and was picked up silently by dt4acc-integration.yml's unpinned install — exactly the
  failure mode point 4 is meant to prevent.
• Related constraints already noted in this repo's agent-notes.md: dt4acc tests must run from
  WSL/Linux on Windows dev machines (not Windows Conda Python), and the twin can take ~8 min to
  start (CI timeout kept above 7 min).

Checklist

• I've assigned this issue to a project
• I've @-mentioned relevant people

[TeresiaOlsson]

I think starting to pin versions for tests is a good way to go. Until now we could maybe manage by testing directly against main but it is starting to become increasingly difficult to manage the dependencies (and the ecosystem is still quite small so I'm guessing it will become worse as it grows...).

Testing against stable versions (even if they are just pre-releases) at least makes it possible to control when you are making a version jump so you are prepared for potential bug fixes.

[gupichon]

Yes, we should rethink our branching strategy for feature development.

• dev/main branch: Acts as our staging branch, merged into main on a regular basis.

• Feature branching hierarchy:

  • Create a root feature branch: dev/feature.
  • Branch sub-tasks off it for iterative work: dev/feature_part_1, dev/feature_part_2, etc.

• Integration workflow:

1. Merge sub-parts incrementally back into dev/feature.
2. Prepare and merge dev/feature into dev/main.
3. Merge dev/main into main to trigger the production release.

[TeresiaOlsson]

Yes, we should rethink our branching strategy for feature development.

* **`dev/main` branch:** Acts as our staging branch, merged into `main` on a regular basis.

* **Feature branching hierarchy:**
  
  * Create a root feature branch: `dev/feature`.
  * Branch sub-tasks off it for iterative work: `dev/feature_part_1`, `dev/feature_part_2`, etc.

* **Integration workflow:**


1. Merge sub-parts incrementally back into `dev/feature`.

2. Prepare and merge `dev/feature` into `dev/main`.

3. Merge `dev/main` into `main` to trigger the production release.

@gubaidulinvadim Can you perhaps add this to a future maintainer meeting?