Skip to content

Conversation

@ninjinkun
Copy link
Contributor

@ninjinkun ninjinkun commented Aug 19, 2025

Summary

This PR backports critical bug fixes, enhancements, and stability improvements from the main branch to the smart-tests branch. The changes include major Java component updates, new features like fail-fast mode and subset comparison, and comprehensive test fixes across multiple test runners.

Key Changes

✨ New Features

  • Fail-Fast Mode: New functionality to stop test execution on first failure
  • Subset Comparison: Added compare subsets command for analyzing differences between test subsets
  • Progress Reporting: Enhanced progress reporting for long-running Git commit ingestion operations
  • File Content Collection: Optional mode to collect file content for analysis (AIENG-183)
  • Zero Test Duration Warning: Display warnings when tests have zero duration

🔧 Core Functionality Improvements

  • Pytest Enhancement: Improved JSON report parsing with better user properties handling and longrepr support
  • Session Validation: Enhanced session flag format validation in subset command
  • API Endpoint Reporting: Added API endpoint information in verify command output
  • Error Handling: Better Windows-specific error handling for percentage parameters

🏗️ Java Component Overhaul

  • Updated JAR: Major update to exe_deploy.jar (10MB → 12MB) with enhanced Git processing
  • Chunk Streaming: New streaming architecture for handling large Git repositories
  • Progress Reporting: Added progress reporting for commit ingestion operations
  • Memory Management: Improved memory handling for large repository processing
  • New Classes Added:
    • ChunkStreamer.java - Base streaming functionality
    • CommitChunkStreamer.java - Commit-specific streaming
    • FileChunkStreamer.java - File content streaming
    • ProgressReportingConsumer.java - Progress tracking
    • GitFile.java and VirtualFile.java - File abstractions

🐛 Bug Fixes & Stability

  • Test Fixes: Comprehensive fixes across all test runners and command tests
  • Mock Endpoints: Fixed record build test mock endpoints
  • Pkg Resources Migration: Completed migration from pkg_resources to importlib.metadata
  • Type Issues: Resolved various type checking and linting issues
  • Test Duration Handling: Fixed issues with zero test duration scenarios

📦 Build & Infrastructure

  • MODULE.bazel Updates: Enhanced Bazel configuration with new dependencies
  • Maven Dependencies: Updated Java dependencies in maven_install.json
  • Test Results: Generated comprehensive XML test results for all components
  • Docker: Updated Dockerfile configurations

Files Changed

  • 101 files changed with 2,191 additions and 227 deletions
  • Major Java component restructuring in src/main/java/
  • Comprehensive test updates across all test runners
  • Enhanced build configuration files

Notable Commits

  • 81251d6 - Backport changes from main branch (e1ff1d3..e636393)
  • 25122ff - Complete Medium Priority backports and linting fixes
  • 8932ed8 - Fix all remaining test failures
  • 77325ae - Implement fail-fast mode functionality
  • c5b1e29 - Add PTS v2 subset auto-collection functionality

Test Plan

  • All existing tests pass with backported changes
  • New fail-fast mode functionality verified
  • Java component streaming tested with large repositories
  • Pytest JSON parsing improvements validated
  • Cross-platform compatibility confirmed

Breaking Changes

No breaking changes in this backport. All changes are backward compatible additions and bug fixes.

Impact

This backport brings the smart-tests branch up to date with critical improvements from main, including:

  • Enhanced performance for large Git repositories
  • Better error handling and user experience
  • New diagnostic and comparison capabilities
  • Improved test runner stability

---🤖 Generated with https://claude.ai/code

ninjinkun and others added 30 commits August 14, 2025 21:00
This commit adds comprehensive auto-collection functionality for PTS v2:
- Add --get-tests-from-guess option to collect tests from git ls-files
- Implement _collect_potential_test_files method with test file detection
- Add SubsetResult class for handling subset responses
- Enhanced workspace state management with PTS v2 support
- Convert Click patterns to Typer equivalents
- Preserve smart-tests package naming and Typer-based CLI

Co-authored-by: Claude <noreply@anthropic.com>
We expect the file transfer could be sizable, so occasionally update the
user with what's going on.
Added validate_session_format() function to ensure session strings follow
the expected format: builds/<build-name>/test_sessions/<test-session-id>

Updated subset command help text to clarify expected session format.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Updated package name from 'launchable' to 'smart-tests-cli' in version.py
- Removed deprecated types-pkg_resources dependency from pyproject.toml
- smart-tests already uses built-in importlib.metadata (Python 3.8+)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
The caller side was not removed. Oof.
When percentage parameter is missing '%' on Windows, provide helpful
error message explaining that '%' needs to be escaped as '%%' in batch files.

This prevents confusing error messages when Windows users forget to
escape percentage characters in command line arguments.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add base_url() method to LaunchableClient and display the server endpoint
in verify command output when it differs from the default.

This helps users understand which API endpoint they are connecting to,
which is especially useful when using custom endpoints.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed subset command architecture with nested test runner commands
- Added missing base_url() method to LaunchableClient
- Fixed subset stdin handling for test environments
- Updated file test runner to properly handle auto-collection
- Modified dynamic command builder to properly run test runners
- Fixed linting issues with unused imports

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed pytest JSON report parser to handle longrepr as dict, list, or string
- Updated subset test to expect correct warning message format (WARNING vs Warning)
- Updated record build tests to use decode_request_body helper method

These fixes resolve pytest test runner compatibility issues and improve
test reliability across different pytest output formats.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Added workspace state endpoint mocks for fail-fast mode checks
- Added builds endpoint mocks for record build operations
- Updated request indices to account for additional state API calls
- Fixed all record build tests to work with new fail-fast mode logic

This resolves test failures where record build commands were not making
HTTP requests due to missing endpoint mocks, which were required after
the fail-fast mode functionality was added.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Added workspace state endpoint mocks to 4 subset tests (goalspec,
  ignore_flaky_tests_above, prioritize_tests_failed_within_hours, targetless)
- Updated API request indices to account for additional state API calls
- Fixed API error test tracking counts to expect 2 calls instead of 1
  (due to added fail-fast mode state checking)

This resolves all remaining test failures, bringing the test suite to
100% passing status with only 1 skipped test (auto-collection format).

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Update check_java_version to return -1 on CalledProcessError
- Add proper exception handling for java -version command failures
- Update tests to expect -1 return value on error conditions
- Display warning when total test duration is 0
- Convert Click syntax to Typer syntax for consistency
- Add yellow warning message about checking test duration in report files
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
…uild-provenance-2.x

Update actions/attest-build-provenance action to v2.4.0
ninjinkun and others added 3 commits August 19, 2025 10:43
This backport includes the following changes:

1. **Docker**: Updated to use OpenJDK 21 instead of 17
2. **Java Core Changes**:
   - Enhanced commit graph collection with server-side filtering
   - Added TreeReceiver interface for optimized file transfer
   - Improved FlushableConsumer interface for batch processing
   - Updated CommitGraphCollector to support repository naming
3. **CLI Features**:
   - Added new 'compare subsets' command to analyze test order changes
   - Enhanced record commit command with repository name parameter
   - Updated record build to pass repository names to commit collection
4. **Infrastructure**:
   - Updated JAR file with latest Java changes
   - Added comprehensive test coverage for new features

All tests passing. Ready for production use.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@ninjinkun ninjinkun marked this pull request as ready for review August 19, 2025 03:19
@ninjinkun ninjinkun merged commit c53e451 into smart-tests Aug 19, 2025
4 checks passed
@ninjinkun ninjinkun deleted the smart-tests-backport branch August 19, 2025 06:02
ninjinkun added a commit that referenced this pull request Aug 20, 2025
…ort"

This reverts commit c53e451, reversing
changes made to d10b7d9.
ninjinkun added a commit that referenced this pull request Aug 20, 2025
…ort"

This reverts commit c53e451, reversing
changes made to d10b7d9.
ninjinkun added a commit that referenced this pull request Aug 20, 2025
Revert "Merge pull request #1112 from cloudbees-oss/smart-tests-backport"
ninjinkun added a commit that referenced this pull request Aug 20, 2025
…ort"

This reverts commit c53e451, reversing
changes made to d10b7d9.
ninjinkun added a commit that referenced this pull request Aug 21, 2025
…rt-2

Revert backport from Launchable CLI to Smart Tests CLI on #1112
@ninjinkun ninjinkun added the Smart Tests CLI The next gen CLI label Aug 25, 2025
@github-actions github-actions bot mentioned this pull request Nov 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Smart Tests CLI The next gen CLI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants