Skip to content

Merge dev: VLM, accessibility, run history, analyzer fixes#175

Merged
JE-Chen merged 14 commits intomainfrom
dev
Apr 24, 2026
Merged

Merge dev: VLM, accessibility, run history, analyzer fixes#175
JE-Chen merged 14 commits intomainfrom
dev

Conversation

@JE-Chen
Copy link
Copy Markdown
Member

@JE-Chen JE-Chen commented Apr 24, 2026

Summary

  • Roll up 10 commits on dev into main, covering new features and a full analyzer clean-up.
  • Feature work: VLM element locator, cross-platform accessibility-tree locator, SQLite-backed run history with REST /history and error-screenshot artifacts, per-platform hotkey backends, clipboard/cron/triggers/CLI/plugins/i18n, GUI tab trimming.
  • Quality pass: close the current SonarCloud queue (43 issues) and Codacy backlog (166 issues) — reflected-XSS in /execute, Thread.run override bug, redundant except subclasses, cognitive-complexity refactors, defusedxml adoption, argv-only subprocess calls, README TOC anchor fixes, bandit config to stop pytest B101 noise.

Test plan

  • CI AutoControl Stable CI passes on all Python versions
  • python -m pytest test/unit_test/headless test/unit_test/flow_control passes (139 tests green locally)
  • python -m ruff check je_auto_control/ test/ is clean
  • python -m bandit -r je_auto_control/ -c pyproject.toml reports 0 issues
  • SonarCloud + Codacy show no new findings on the merge commit
  • import je_auto_control stays Qt-free (no PySide6 in sys.modules)

JE-Chen added 10 commits April 18, 2026 18:21
Inner labels, buttons, group titles and placeholders previously kept
their original strings when the Language menu changed — only tab titles
updated. Add a TranslatableMixin registry (_tr/retranslate) and wire it
through every tab (main builders, shell/report mixin, class-based tabs,
script builder) so all widgets retranslate together. Add the missing
translation keys across English / Traditional Chinese / Simplified
Chinese / Japanese.
The Traditional Chinese pack had U+FFFD replacement characters where
"幕" should close "截取螢幕" — the button rendered as ??? in the GUI.
Locate GUI elements by role/name/app via UIA on Windows and AX on macOS,
falling back to a null backend with a clear error elsewhere. Exposes a
headless API (list/find/click), AC_a11y_* executor commands, and an
Accessibility GUI tab so scripts and users can target widgets without
pixel coordinates.
These tabs don't fit the automation-GUI role: shell execution
duplicates the OS terminal with added injection risk, a network
server tied to a transient GUI process dies with the window, and
screen-recording UX is better served by dedicated OS tools. The
headless APIs, AC_* executor commands, and language keys stay so
scripts and the REST/TCP entry points remain unchanged.
Failed scheduler/trigger/hotkey runs now grab a screenshot into
~/.je_auto_control/artifacts/ and record the path on the row so the
GUI Run History tab can open it. Clear/prune delete the files too.
Capture is best-effort: if the screenshot call fails the run is
still logged with a plain error.
Address the full current SonarCloud queue (43 issues) and the Codacy
backlog (166 issues) so new analyzer runs start from a clean baseline.

- rest_server: stop echoing user-controlled paths/errors; log server-side
  and return generic messages (fixes reflected-XSS blocker)
- KeypressHandler: stop inheriting Thread and rename the callback to
  handle_reply; Thread.run signature mismatch was a latent override bug
- Drop NotImplementedError / NotADirectoryError / UnicodeDecodeError /
  JSONDecodeError from except tuples that already catch their parent
- change_xml_structure: split the two recursive helpers by responsibility
  to bring cognitive complexity back under the 15-line limit
- Replace stdlib xml.etree/xml.dom.minidom parsing with defusedxml across
  xml_file, change_xml_structure, and generate_xml_report
- Extract repeated string literals (JSON filter, {temp} placeholder,
  keycode error, calculator image paths) into module constants
- Validate package names before importlib.import_module; use argv lists
  and timeouts for every subprocess call; drop shell=True
- Add .codacy.yaml excluding test/** from bandit and [tool.bandit] in
  pyproject.toml so pytest assertions stop tripping B101
- Fix stale README TOC anchors (EN/zh-CN/zh-TW) and the Sphinx copyright
  builtin rebind
- Iterate dict.values() directly in socket_server, use slice copy for
  language listeners, tighten regex to \w, drop stray list()/dict()
- Refresh test fixtures (unused imports, loop-once, opposite-operator,
  positional-args mismatch) so the tests stop generating findings
@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented Apr 24, 2026

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 556 complexity · 8 duplication

Metric Results
Complexity 556
Duplication 8

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

JE-Chen added 4 commits April 25, 2026 01:14
- english/japanese language dicts: extract _SCRIPT_LABEL / _REMOVE_SELECTED /
  _SELECT_SCRIPT / _SCRIPT to drop S1192 duplicate-literal warnings
- accessibility/vision backend factories: type the module-level cache as
  Optional[...] instead of assigning None against a non-optional hint (S5890)
- macos_backend: rename the ApplicationServices param from AS to ax_module
  so it passes S117 snake_case
- hotkey linux_backend._sync: split into _ungrab_stale / _sync_one /
  _ungrab_masked / _grab_masked helpers to bring cognitive complexity
  from 21 back under the 15-line limit
- hotkey windows_backend: iterate the dict directly in the finally block
  instead of wrapping in list() (S7504)
- HotkeyDaemon._snapshot is now an alias of list_bindings so Sonar stops
  flagging the identical body (S4144)
- change_xml_structure: lift the nested conditional into _initial_body
  (S3358)
- package_manager regex uses concise \w (S6353); history_store uses the
  _IN_MEMORY_DB constant instead of repeating ":memory:" (S1192)
- Add nosemgrep suppressions for defusedxml imports, the run_history
  SELECT builder, and the openai SDK guardrails false positive
- vision/_parse.py: rewrite the coord regex as a single unambiguous
  alternation (``\s*,\s*`` | ``\s+``) so Sonar S5852 stops flagging
  polynomial-backtracking risk; behaviour unchanged on covered cases
- test_run_history: route artifact paths through the ``tmp_path``
  fixture instead of hard-coded ``/tmp/*`` literals, eliminating the
  five S5443 "publicly writable directory" hotspots without changing
  test intent
- utils/xml/__init__.py: call ``defusedxml.defuse_stdlib()`` on import
  so every stdlib xml parser used anywhere in the package is replaced
  by the safe variant — closes the remaining Bandit B314/B318 nags
- platform_wrapper: add an explicit ``__all__`` listing the facade
  re-exports so Prospector/pyflakes stop reporting ``keyboard_check``
  as unused (F401)
- .codacy.yaml: also exclude ``test/**`` from Prospector and drop the
  manual/gui smoke scripts from analysis so pytest-style imports and
  assertions stop lighting up PR runs
- mss 10.2.0 is the current release; pin matches pyproject, requirements,
  and dev_requirements so editable installs resolve the same wheel
- Sphinx 7 is EOL for security fixes; lift docs/requirements.txt to
  >=8.1.3 to match the Read the Docs build and pull in the current
  release line
Opengrep's ``python_sql_rule-hardcoded-sql-expression`` can't prove the
dynamically built SQL string in ``list_runs`` is safe even with the
parameters bound, so it keeps firing as a false positive on PR runs.
Branch the two cases (with/without source_type filter) into two fully
literal queries — no string concatenation, just placeholders — so the
rule has nothing to flag. Behaviour and test coverage unchanged.
@sonarqubecloud
Copy link
Copy Markdown

@JE-Chen JE-Chen merged commit 241b812 into main Apr 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant