Skip to content

Simplifies local integration-test development - #2

Open
rzaitov wants to merge 5 commits into
mainfrom
simplify-local-development
Open

Simplifies local integration-test development#2
rzaitov wants to merge 5 commits into
mainfrom
simplify-local-development

Conversation

@rzaitov

@rzaitov rzaitov commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Simplifies local integration-test development and adds end-to-end coverage for Ansible template working directories.

Local tests can now consume uncommitted fixture changes without pushing them to a remote repository.

Local development workflow

Before this change, fixture updates had to be committed and pushed before a Semaphore stand could consume them. This slowed the test-and-fixture feedback loop and became a barrier to adding new integration tests.

run-local.sh provides a self-contained local workflow instead. Its only prerequisite is building the local Semaphore image with task docker:build:server tag=local. The script then starts a stand from semaphoreui/semaphore:local, waits for it to become ready, runs either the complete API suite or a selected test class, and tears the stand down afterward. A different image can still be selected through SEMAPHORE_IMAGE.

The script also exposes uncommitted files under test-environment/fixtures as a temporary Git repository through the fixture-init helper service. Semaphore and the persistent runner clone it locally through file:///repository.

The test framework now accepts the fixture repository URL and branch as configuration instead of hardcoding a GitHub URL in fixture definitions. run-local.sh supplies file:///repository and main, while external runs default to https://github.com/semaphoreui/integration-tests.git. Using a Git file:// URL rather than an absolute path is important because it preserves clone and branch checkout behavior, keeping local development network-independent while still exercising successful and missing branch scenarios.

Ansible working-directory coverage

The template request and response models now expose working_directory and arguments. This allows the integration tests to configure the Ansible process directory, pass relative command-line files, and verify that both values are preserved by the template API.

AnsibleWorkingDirectoryTest adds three end-to-end scenarios backed by fixtures under test-environment/fixtures/ansible/working-directory-fixture:

  1. Ansible configuration and role discovery — runs a repository-rooted playbook with a repository-rooted file inventory while setting the process directory to working-dir. The ansible.cfg in that directory configures roles_path = roles, and successful role loading emits a deterministic marker.
  2. Relative extra-vars resolution — passes --extra-vars @vars.yml through the template arguments field and verifies that Ansible resolves vars.yml from the configured working directory.
  3. Relative private-key resolution — passes --private-key key.pem and verifies that the key path is resolved from the configured working directory.

Each scenario creates a real Semaphore template and task, checks the persisted working_directory and arguments values, waits for successful execution, and asserts a scenario-specific output marker.

Local usage

Build the local Semaphore image:

cd ../semaphore
task docker:build:server tag=local

Run all API tests:

./run-local.sh

- point the default fixture source at the integration-tests repository
- build a fresh local Git snapshot from test-environment/fixtures
- preserve repository-relative fixture paths for local and external runs
- move Bash fixtures out of the Ansible directory
- share file:///repository with the Semaphore server and runner
- add working_directory and arguments to template API models
- add typed resources and executable Ansible fixtures
- verify ansible.cfg and role discovery from the working directory
- verify relative extra-vars and private-key path resolution
- confirm working-directory persistence in template responses
@rzaitov
rzaitov requested a review from fiftin September 1, 2026 13:16
@rzaitov rzaitov self-assigned this Sep 1, 2026
@rzaitov

rzaitov commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

A failure is unfortunate, because test stand was set up not from an appropriate feature branch. So I added integration tests for working directory feature, but that feature is not merged yet, so test stand doesn't have it.

AnsibleWorkingDirectoryTest > Working directory loads repository-local Ansible configuration and role FAILED
    java.lang.IllegalStateException at TaskSteps.java:122

AnsibleWorkingDirectoryTest > Relative extra-vars path resolves from the working directory FAILED
    java.lang.IllegalStateException at TaskSteps.java:122

@dantro86 dantro86 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The working-directory coverage and local fixture-repository direction are useful, but this branch needs a rebase after #4 and two runtime blockers fixed. The existing CI artifact confirms that the three new scenarios fail before exercising the Semaphore feature: the cloned main branch does not contain their playbooks.

boolean teardownFailOnError();

@Key("semaphore.repository.url")
@DefaultValue("https://github.com/semaphoreui/integration-tests.git")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This default makes PR CI clone the remote main branch instead of the fixture repository assembled from the checked-out PR. In run 33514596281 all three new tests consequently fail with the playbook ... could not be found; they never reach the working-directory behavior. Please make the profile/CI path use file:///repository (or explicitly inject the checked-out ref), reserving a remote URL for an explicitly selected external mode.

Comment thread run-local.sh

run_integration_tests() {
docker run --rm \
--network host \

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test container uses localhost URLs below, so this depends on Docker host networking. Host networking is optional on Docker Desktop and unavailable in several common/rootless setups, while the README says the local image is the only prerequisite. Please attach the runner to the Compose network and address Semaphore by service name, or provide a portable host.docker.internal/host-gateway path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants