diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 3b4e00cb5..165ff864f 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -4,6 +4,44 @@ CHANGELOG ========= +.. _changelog-v10.4.0: + +v10.4.0 (2025-09-08) +==================== + +✨ Features +----------- + +* **config**: Add ``conventional-monorepo`` as valid ``commit_parser`` type (`PR#1143`_, `e18f866`_) + +* **parser**: Add new conventional-commits standard parser for monorepos, closes `#614`_ + (`PR#1143`_, `e18f866`_) + +📖 Documentation +---------------- + +* Add configuration guide for monorepo use with PSR (`PR#1143`_, `e18f866`_) + +* **commit-parsers**: Introduce conventional commit monorepo parser options & features (`PR#1143`_, + `e18f866`_) + +* **configuration**: Update ``commit_parser`` option with new ``conventional-monorepo`` value + (`PR#1143`_, `e18f866`_) + +💡 Additional Release Information +--------------------------------- + +* **config**: This release introduces a new built-in parser type that can be utilized for monorepo + projects. The type value is ``conventional-monorepo`` and when specified it will apply the + conventional commit parser to a monorepo environment. This parser has specialized options to help + handle monorepo projects as well. For more information, please refer to the `Monorepo Docs`_. + +.. _#614: https://github.com/python-semantic-release/python-semantic-release/issues/614 +.. _e18f866: https://github.com/python-semantic-release/python-semantic-release/commit/e18f86640a78b374a327848b9e2ba868003d1a43 +.. _Monorepo Docs: /configuration/configuration-guides/monorepos.html +.. _PR#1143: https://github.com/python-semantic-release/python-semantic-release/pull/1143 + + .. _changelog-v10.3.2: v10.3.2 (2025-09-06) diff --git a/docs/concepts/commit_parsing.rst b/docs/concepts/commit_parsing.rst index 163927c39..296169c52 100644 --- a/docs/concepts/commit_parsing.rst +++ b/docs/concepts/commit_parsing.rst @@ -49,6 +49,7 @@ Built-in Commit Parsers The following parsers are built in to Python Semantic Release: - :ref:`ConventionalCommitParser ` +- :ref:`ConventionalCommitMonorepoParser ` *(available in v10.4.0+)* - :ref:`AngularCommitParser ` *(deprecated in v9.19.0)* - :ref:`EmojiCommitParser ` - :ref:`ScipyCommitParser ` @@ -65,7 +66,7 @@ Conventional Commits Parser A parser that is designed to parse commits formatted according to the `Conventional Commits Specification`_. The parser is implemented with the following -logic in relation to how PSR's core features: +logic in relation to PSR's core features: - **Version Bump Determination**: This parser extracts the commit type from the subject line of the commit (the first line of a commit message). This type is matched against @@ -127,13 +128,98 @@ logic in relation to how PSR's core features: If no commit parser options are provided via the configuration, the parser will use PSR's built-in -:py:class:`defaults `. +:py:class:`defaults `. .. _#402: https://github.com/python-semantic-release/python-semantic-release/issues/402 .. _Conventional Commits Specification: https://www.conventionalcommits.org/en/v1.0.0 ---- +.. _commit_parser-builtin-conventional-monorepo: + +Conventional Commits Monorepo Parser +"""""""""""""""""""""""""""""""""""" + +*Introduced in v10.4.0* + +.. important:: + In order for this parser to be effective, please review the section titled + :ref:`monorepos` for details on file structure, configurations, and release actions. + +This parser is an extension of the :ref:`commit_parser-builtin-conventional`, designed specifically +for monorepo environments. A monorepo environment is defined as a single source control repository +that contains multiple packages, each of which can be released independently and may have different +version numbers. + +This parser introduces two new configuration options that determine which packages are affected +by a commit. These options control whether a commit is considered for version determination, +changelog generation, and other release actions for the relevant packages. The 2 new +configuration options are +:py:class:`path_filters ` +and +:py:class:`scope_prefix `. + +**Features**: + +- **Package Specific Commit Filtering**: For monorepo support, this parser uses 2 filtering rules + to determine if a commit should be considered for a specific package. The first rule is based on + file paths that are changed in the commit and the second rule is based on the optional scope + prefix defined in the commit message. If either rule matches, then the commit is considered + relevant to that package and will be used in version determination, changelog generation, etc, + for that package. If neither rule matches, then the commit is ignored for that package. File + path filtering rules are applied first and are the primary way to determine package relevance. The + :py:class:`path_filters ` + option allows for specifying a list of file path patterns and will also support negated patterns + to ignore specific paths that otherwise would be selected from the file glob pattern. Negated + patterns are defined by prefixing the pattern with an exclamation point (``!``). File path + filtering is quite effective by itself but to handle the edge cases, the parser has the + :py:class:`scope_prefix ` + configuration option to allow the developer to specifically define when the commit is relevant + to the package. In monorepo setups, there are often shared files between packages (generally at + the root project level) that are modified occasionally but not always relevant to the package + being released. Since you do not want to define this path in the package configuration as it may + not be relevant to the release, then this parser will look for a match with the scope prefix. + The scope prefix is a regular expression that is used to match the text inside the scope field + of a Conventional Commit. The scope prefix is optional and is used only if file path filtering + does not match. Commits that have matching files in the commit will be considered relevant to + the package **regardless** if a scope prefix exists or if it matches. + +- **Version Bump Determination**: Once package-specific commit filtering is applied, the relevant + commits are passed to the Conventional Commits Parser for evaluation and then used for version + bump determination. See :ref:`commit_parser-builtin-conventional` for details. + +- **Changelog Generation**: Once package-specific commit filtering is applied, the relevant + commits are passed to the Conventional Commits Parser for evaluation and then used for + changelog generation. See :ref:`commit_parser-builtin-conventional` for details. + +- **Pull/Merge Request Identifier Detection**: Once package-specific commit filtering is applied, + the relevant commits are passed to the Conventional Commits Parser for pull/merge request + identifier detection. See :ref:`commit_parser-builtin-linked_merge_request_detection` for details. + +- **Linked Issue Identifier Detection**: Once package-specific commit filtering is applied, the + relevant commits are passed to the Conventional Commits Parser for linked issue identifier + detection. See :ref:`commit_parser-builtin-issue_number_detection` for details. + +- **Squash Commit Evaluation**: Squashed commits are separated out into individual commits with + the same set of changed files **BEFORE** the package-specific commit filtering is applied. + Each pseudo-commit is then subjected to the same filtering rules as regular commits. See + :ref:`commit_parser-builtin-squash_commit_evaluation` for details. + +- **Release Notice Footer Detection**: Once package-specific commit filtering is applied, the + relevant commits are passed to the Conventional Commits Parser for release notice footer + detection. See :ref:`commit_parser-builtin-release_notice_footer_detection` for details. + +**Limitations**: + +- ``revert`` commit type is NOT supported, see :ref:`commit_parser-builtin-conventional`'s + limitations for details. + +If no commit parser options are provided via the configuration, the parser will use PSR's +built-in +:py:class:`defaults `. + +---- + .. _commit_parser-builtin-angular: Angular Commit Parser diff --git a/docs/configuration/automatic-releases/github-actions.rst b/docs/configuration/automatic-releases/github-actions.rst index c7d4b0bfb..57cd7b507 100644 --- a/docs/configuration/automatic-releases/github-actions.rst +++ b/docs/configuration/automatic-releases/github-actions.rst @@ -933,14 +933,14 @@ to the GitHub Release Assets as well. - name: Action | Semantic Version Release id: release # Adjust tag with desired version if applicable. - uses: python-semantic-release/python-semantic-release@v10.3.2 + uses: python-semantic-release/python-semantic-release@v10.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} git_committer_name: "github-actions" git_committer_email: "actions@users.noreply.github.com" - name: Publish | Upload to GitHub Release Assets - uses: python-semantic-release/publish-action@v10.3.2 + uses: python-semantic-release/publish-action@v10.4.0 if: steps.release.outputs.released == 'true' with: github_token: ${{ secrets.GITHUB_TOKEN }} @@ -1039,7 +1039,7 @@ The equivalent GitHub Action configuration would be: - name: Action | Semantic Version Release # Adjust tag with desired version if applicable. - uses: python-semantic-release/python-semantic-release@v10.3.2 + uses: python-semantic-release/python-semantic-release@v10.4.0 with: github_token: ${{ secrets.GITHUB_TOKEN }} force: patch @@ -1098,14 +1098,14 @@ Publish Action. - name: Release submodule 1 id: release-submod-1 - uses: python-semantic-release/python-semantic-release@v10.3.2 + uses: python-semantic-release/python-semantic-release@v10.4.0 with: directory: ${{ env.SUBMODULE_1_DIR }} github_token: ${{ secrets.GITHUB_TOKEN }} - name: Release submodule 2 id: release-submod-2 - uses: python-semantic-release/python-semantic-release@v10.3.2 + uses: python-semantic-release/python-semantic-release@v10.4.0 with: directory: ${{ env.SUBMODULE_2_DIR }} github_token: ${{ secrets.GITHUB_TOKEN }} @@ -1117,7 +1117,7 @@ Publish Action. # ------------------------------------------------------------------- # - name: Publish | Upload package 1 to GitHub Release Assets - uses: python-semantic-release/publish-action@v10.3.2 + uses: python-semantic-release/publish-action@v10.4.0 if: steps.release-submod-1.outputs.released == 'true' with: directory: ${{ env.SUBMODULE_1_DIR }} @@ -1125,7 +1125,7 @@ Publish Action. tag: ${{ steps.release-submod-1.outputs.tag }} - name: Publish | Upload package 2 to GitHub Release Assets - uses: python-semantic-release/publish-action@v10.3.2 + uses: python-semantic-release/publish-action@v10.4.0 if: steps.release-submod-2.outputs.released == 'true' with: directory: ${{ env.SUBMODULE_2_DIR }} diff --git a/docs/configuration/configuration-guides/index.rst b/docs/configuration/configuration-guides/index.rst index 70024dd11..d094496dd 100644 --- a/docs/configuration/configuration-guides/index.rst +++ b/docs/configuration/configuration-guides/index.rst @@ -11,4 +11,5 @@ more specific configurations. .. toctree:: :maxdepth: 1 + Monorepos UV Project Setup diff --git a/docs/configuration/configuration-guides/monorepos-ex-easy-before-release.png b/docs/configuration/configuration-guides/monorepos-ex-easy-before-release.png new file mode 100644 index 000000000..6c9bc6113 Binary files /dev/null and b/docs/configuration/configuration-guides/monorepos-ex-easy-before-release.png differ diff --git a/docs/configuration/configuration-guides/monorepos-ex-easy-post-release.png b/docs/configuration/configuration-guides/monorepos-ex-easy-post-release.png new file mode 100644 index 000000000..76693ecd3 Binary files /dev/null and b/docs/configuration/configuration-guides/monorepos-ex-easy-post-release.png differ diff --git a/docs/configuration/configuration-guides/monorepos.rst b/docs/configuration/configuration-guides/monorepos.rst new file mode 100644 index 000000000..1f173998c --- /dev/null +++ b/docs/configuration/configuration-guides/monorepos.rst @@ -0,0 +1,358 @@ +.. _monorepos: + +Releasing Packages from a Monorepo +================================== + + +A monorepo (mono-repository) is a software development strategy where code for multiple projects is stored in a single source control system. This approach streamlines and consolidates configuration, but introduces complexities when using automated tools like Python Semantic Release (PSR). + +Previously, PSR offered limited compatibility with monorepos. As of v10.4.0, PSR introduces the :ref:`commit_parser-builtin-conventional-monorepo`, designed specifically for monorepo environments. To fully leverage this new parser, you must configure your monorepo as described below. + +.. _monorepos-config: + +Configuring PSR +--------------- + +.. _monorepos-config-example_simple: + +Example: Simple +""""""""""""""" + + +**Directory Structure**: PSR does not yet support a single, workspace-level configuration definition. This means each package in the monorepo requires its own PSR configuration file. A compatible and common monorepo file structure looks like: + +.. code:: + + project/ + ├── .git/ + ├── .venv/ + ├── packages/ + │ ├── pkg1/ + │ │ ├── docs/ + │ │ │ └── source/ + │ │ │ ├── conf.py + │ │ │ └── index.rst + │ │ │ + │ │ ├── src/ + │ │ │ └── pkg1/ + │ │ │ ├── __init__.py + │ │ │ ├── __main__.py + │ │ │ └── py.typed + │ │ │ + │ │ ├── CHANGELOG.md + │ │ ├── README.md + │ │ └── pyproject.toml <-- PSR Configuration for Package 1 + │ │ + │ └── pkg2/ + │ ├── docs/ + │ │ └── source/ + │ │ ├── conf.py + │ │ └── index.rst + │ ├── src/ + │ │ └── pkg2/ + │ │ ├── __init__.py + │ │ ├── __main__.py + │ │ └── py.typed + │ │ + │ ├── CHANGELOG.md + │ ├── README.md + │ └── pyproject.toml <-- PSR Configuration for Package 2 + │ + ├── .gitignore + └── README.md + + +This is the most basic monorepo structure, where each package is self-contained with its own configuration files, documentation, and CHANGELOG files. To release a package, change your current working directory to the package directory and execute PSR's :ref:`cmd-version`. PSR will automatically read the package's ``pyproject.toml``, looking for the ``[tool.semantic_release]`` section to determine the package's versioning and release configuration, then search up the file tree to find the Git repository. + +Because there is no workspace-level configuration, you must duplicate any common PSR configuration in each package's configuration file. Customize each configuration for each package to specify how PSR should distinguish between commits. + +With the example file structure above, here is an example configuration file for each package: + +.. code-block:: toml + + # FILE: pkg1/pyproject.toml + [project] + name = "pkg1" + version = "1.0.0" + + [tool.semantic_release] + commit_parser = "conventional-monorepo" + commit_message = """\ + chore(release): pkg1@{version}` + + Automatically generated by python-semantic-release + """ + tag_format = "pkg1-v{version}" + version_toml = ["pyproject.toml:project.version"] + + [tool.semantic_release.commit_parser_options] + path_filters = ["."] + scope_prefix = "pkg1-" + +.. code-block:: toml + + # FILE: pkg2/pyproject.toml + [project] + name = "pkg2" + version = "1.0.0" + + [tool.semantic_release] + commit_parser = "conventional-monorepo" + commit_message = """\ + chore(release): pkg2@{version}` + + Automatically generated by python-semantic-release + """ + tag_format = "pkg2-v{version}" + version_toml = ["pyproject.toml:project.version"] + + [tool.semantic_release.commit_parser_options] + path_filters = ["."] + scope_prefix = "pkg2-" + + +These are the minimum configuration options required for each package. Note the use of :ref:`config-tag_format` to distinguish tags between packages. The commit parser options are specific to the new :ref:`commit_parser-builtin-conventional-monorepo` and play a significant role in identifying which commits are relevant to each package. Since you are expected to change directories to each package before releasing, file paths in each configuration file should be relative to the package directory. + +Each package also defines a slightly different :ref:`config-commit_message` to reflect the package name in each message. This helps clarify which release number is being updated in the commit history. + + +Release Steps +''''''''''''' + +Given the following Git history of a monorepo using a GitHub Flow branching strategy (without CI/CD): + +.. image:: ./monorepos-ex-easy-before-release.png + +To manually release both packages, run: + +.. code-block:: bash + + cd packages/pkg1 + semantic-release version + # 1.0.1 (tag: pkg1-v1.0.1) + + cd ../pkg2 + semantic-release version + # 1.1.0 (tag: pkg2-v1.1.0) + +After releasing both packages, the resulting Git history will look like: + +.. image:: ./monorepos-ex-easy-post-release.png + +.. seealso:: + + - :ref:`GitHub Actions with Monorepos ` + + +Considerations +'''''''''''''' + +1. **Custom Changelogs**: Managing changelogs can be tricky depending on where you want to write the changelog files. In this simple example, the changelog is located within each package directory, and the changelog template does not have any package-specific formatting or naming convention. You can use one shared template directory at the root of the project and configure each package to point to the shared template directory. + +.. code-block:: toml + + # FILE: pkg1/pyproject.toml + [tool.semantic_release] + template_dir = "../../config/release-templates" + +.. code-block:: toml + + # FILE: pkg2/pyproject.toml + [tool.semantic_release] + template_dir = "../../config/release-templates" + +.. code:: + + project/ + ├── .git/ + ├── config/ + │ └── release-templates/ + │ ├── CHANGELOG.md.j2 + │ └── .release_notes.md.j2 + ├── packages/ + │ ├── pkg1/ + │ │ ├── CHANGELOG.md + │ │ └── pyproject.toml + │ │ + │ └── pkg2/ + │ ├── CHANGELOG.md + │ └── pyproject.toml + │ + ├── .gitignore + └── README.md + +.. seealso:: + + - For situations with more complex documentation needs, see our :ref:`Advanced Example `. + + +2. **Package Prereleases**: Creating pre-releases is possible, but it is recommended to use package-prefixed branch names to avoid collisions between packages. For example, to enable alpha pre-releases for new features in both packages, use the following configuration: + +.. code-block:: toml + + # FILE: pkg1/pyproject.toml + [tool.semantic_release.branches.alpha-release] + match = "^pkg1/feat/.+" # <-- note pkg1 prefix + prerelease = true + prerelease_token = "alpha" + +.. code-block:: toml + + # FILE: pkg2/pyproject.toml + [tool.semantic_release.branches.alpha-release] + match = "^pkg2/feat/.+" # <-- note pkg2 prefix + prerelease = true + prerelease_token = "alpha" + +---- + +.. _monorepos-config-example_advanced: + +Example: Advanced +""""""""""""""""" + + +If you want to consolidate documentation into a single top-level directory, the setup becomes more complex. In this example, there is a common documentation folder at the top level, and each package has its own subfolder within the documentation folder. + +Due to naming conventions, PSR cannot automatically accomplish this with its default changelog templates. For this scenario, you must copy the internal PSR templates into a custom directory (even if you do not modify them) and add custom scripting to prepare for each release. + +The directory structure looks like: + +.. code:: + + project/ + ├── .git/ + ├── docs/ + │ ├── source/ + │ │ ├── pkg1/ + │ │ │ ├── changelog.md + │ │ │ └── README.md + │ │ ├── pkg2/ + │ │ │ ├── changelog.md + │ │ │ └── README.md + │ │ └── index.rst + │ │ + │ └── templates/ + │ ├── .base_changelog_template/ + │ │ ├── components/ + │ │ │ ├── changelog_header.md.j2 + │ │ │ ├── changelog_init.md.j2 + │ │ │ ├── changelog_update.md.j2 + │ │ │ ├── changes.md.j2 + │ │ │ ├── first_release.md.j2 + │ │ │ ├── macros.md.j2 + │ │ │ ├── unreleased_changes.md.j2 + │ │ │ └── versioned_changes.md.j2 + │ │ └── changelog.md.j2 + │ ├── .gitignore + │ └── .release_notes.md.j2 + │ + ├── packages/ + │ ├── pkg1/ + │ │ ├── src/ + │ │ │ └── pkg1/ + │ │ │ ├── __init__.py + │ │ │ └── __main__.py + │ │ └── pyproject.toml + │ │ + │ └── pkg2/ + │ ├── src/ + │ │ └── pkg2/ + │ │ ├── __init__.py + │ │ └── __main__.py + │ └── pyproject.toml + │ + └── scripts/ + ├── release-pkg1.sh + └── release-pkg2.sh + + +Each package should point to the ``docs/templates/`` directory to use a common release notes template. PSR ignores hidden files and directories when searching for template files to create, allowing you to hide shared templates in the directory for use in your release setup script. + +Here is our configuration file for package 1 (package 2 is similarly defined): + +.. code-block:: toml + + # FILE: pkg1/pyproject.toml + [project] + name = "pkg1" + version = "1.0.0" + + [tool.semantic_release] + commit_parser = "conventional-monorepo" + commit_message = """\ + chore(release): Release `pkg1@{version}` + + Automatically generated by python-semantic-release + """ + tag_format = "pkg1-v{version}" + version_toml = ["pyproject.toml:project.version"] + + [tool.semantic_release.commit_parser_options] + path_filters = [ + ".", + "../../../docs/source/pkg1/**", + ] + scope_prefix = "pkg1-" + + [tool.semantic_release.changelog] + template_dir = "../../../docs/templates" + mode = "update" + exclude_commit_patterns = [ + '''^chore(?:\([^)]*?\))?: .+''', + '''^ci(?:\([^)]*?\))?: .+''', + '''^refactor(?:\([^)]*?\))?: .+''', + '''^style(?:\([^)]*?\))?: .+''', + '''^test(?:\([^)]*?\))?: .+''', + '''^Initial [Cc]ommit''', + ] + + [tool.semantic_release.changelog.default_templates] + # To enable update mode: this value must set here because the default is not the + # same as the default in the other package & must be the final destination filename + # for the changelog relative to this file + changelog_file = "../../../docs/source/pkg1/changelog.md" + + +Note: In this configuration, we added path filters for additional documentation files related to the package so that the changelog will include documentation changes as well. + + +Next, define a release script to set up the common changelog templates in the correct directory format so PSR will create the desired files at the proper locations. Following the :ref:`changelog-templates-template-rendering` reference, you must define the folder structure from the root of the project within the templates directory so PSR will properly lay down the files across the repository. The script cleans up any previous templates, dynamically creates the necessary directories, and copies over the shared templates into a package-named directory. Now you are prepared to run PSR for a release of ``pkg1``. + +.. code-block:: bash + + #!/bin/bash + # FILE: scripts/release-pkg1.sh + + set -euo pipefail + + PROJECT_ROOT="$(dirname "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")")" + VIRTUAL_ENV="$PROJECT_ROOT/.venv" + + PACKAGE_NAME="pkg1" + + cd "$PROJECT_ROOT" || exit 1 + + # Setup documentation template + pushd "docs/templates" >/dev/null || exit 1 + + rm -rf docs/ + mkdir -p "docs/source/" + cp -r .base_changelog_template/ "docs/source/$PACKAGE_NAME" + + popd >/dev/null || exit 1 + + # Release the package + pushd "packages/$PACKAGE_NAME" >/dev/null || exit 1 + + printf '%s\n' "Releasing $PACKAGE_NAME..." + "$VIRTUAL_ENV/bin/semantic-release" -v version --no-push + + popd >/dev/null || exit 1 + + +That's it! This example demonstrates how to set up a monorepo with shared changelog templates and a consolidated documentation folder for multiple packages. + +.. seealso:: + + - Advanced Example Monorepo: `codejedi365/psr-monorepo-poweralpha `_ diff --git a/docs/configuration/configuration.rst b/docs/configuration/configuration.rst index d9c154a93..2b2278382 100644 --- a/docs/configuration/configuration.rst +++ b/docs/configuration/configuration.rst @@ -796,6 +796,7 @@ within the Git repository. Built-in parsers: * ``angular`` - :ref:`AngularCommitParser ` *(deprecated in v9.19.0)* * ``conventional`` - :ref:`ConventionalCommitParser ` *(available in v9.19.0+)* + * ``conventional-monorepo`` - :ref:`ConventionalCommitMonorepoParser ` *(available in v10.4.0+)* * ``emoji`` - :ref:`EmojiCommitParser ` * ``scipy`` - :ref:`ScipyCommitParser ` * ``tag`` - :ref:`TagCommitParser ` *(deprecated in v9.12.0)* diff --git a/pyproject.toml b/pyproject.toml index ce10f324b..22640474d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta" [project] name = "python-semantic-release" -version = "10.3.2" +version = "10.4.0" description = "Automatic Semantic Versioning for Python projects" requires-python = "~= 3.8" license = { text = "MIT" } diff --git a/src/gh_action/requirements.txt b/src/gh_action/requirements.txt index 9d0c27391..e35c59aae 100644 --- a/src/gh_action/requirements.txt +++ b/src/gh_action/requirements.txt @@ -1 +1 @@ -python-semantic-release == 10.3.2 +python-semantic-release == 10.4.0 diff --git a/src/semantic_release/cli/config.py b/src/semantic_release/cli/config.py index 1d2057a48..37b86a811 100644 --- a/src/semantic_release/cli/config.py +++ b/src/semantic_release/cli/config.py @@ -39,6 +39,7 @@ from semantic_release.commit_parser import ( AngularCommitParser, CommitParser, + ConventionalCommitMonorepoParser, ConventionalCommitParser, EmojiCommitParser, ParseResult, @@ -71,9 +72,10 @@ class HvcsClient(str, Enum): GITEA = "gitea" -_known_commit_parsers: Dict[str, type[CommitParser]] = { - "conventional": ConventionalCommitParser, +_known_commit_parsers: dict[str, type[CommitParser[Any, Any]]] = { "angular": AngularCommitParser, + "conventional": ConventionalCommitParser, + "conventional-monorepo": ConventionalCommitMonorepoParser, "emoji": EmojiCommitParser, "scipy": ScipyCommitParser, "tag": TagCommitParser, diff --git a/src/semantic_release/commit_parser/__init__.py b/src/semantic_release/commit_parser/__init__.py index 740f4ae7f..15a96c176 100644 --- a/src/semantic_release/commit_parser/__init__.py +++ b/src/semantic_release/commit_parser/__init__.py @@ -7,6 +7,8 @@ AngularParserOptions, ) from semantic_release.commit_parser.conventional import ( + ConventionalCommitMonorepoParser, + ConventionalCommitMonorepoParserOptions, ConventionalCommitParser, ConventionalCommitParserOptions, ) @@ -28,3 +30,24 @@ ParseResult, ParseResultType, ) + +__all__ = [ + "CommitParser", + "ParserOptions", + "AngularCommitParser", + "AngularParserOptions", + "ConventionalCommitParser", + "ConventionalCommitParserOptions", + "ConventionalCommitMonorepoParser", + "ConventionalCommitMonorepoParserOptions", + "EmojiCommitParser", + "EmojiParserOptions", + "ScipyCommitParser", + "ScipyParserOptions", + "TagCommitParser", + "TagParserOptions", + "ParsedCommit", + "ParseError", + "ParseResult", + "ParseResultType", +] diff --git a/src/semantic_release/commit_parser/conventional/__init__.py b/src/semantic_release/commit_parser/conventional/__init__.py new file mode 100644 index 000000000..dd7d57d63 --- /dev/null +++ b/src/semantic_release/commit_parser/conventional/__init__.py @@ -0,0 +1,17 @@ +from semantic_release.commit_parser.conventional.options import ( + ConventionalCommitParserOptions, +) +from semantic_release.commit_parser.conventional.options_monorepo import ( + ConventionalCommitMonorepoParserOptions, +) +from semantic_release.commit_parser.conventional.parser import ConventionalCommitParser +from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, +) + +__all__ = [ + "ConventionalCommitParser", + "ConventionalCommitParserOptions", + "ConventionalCommitMonorepoParser", + "ConventionalCommitMonorepoParserOptions", +] diff --git a/src/semantic_release/commit_parser/conventional/options.py b/src/semantic_release/commit_parser/conventional/options.py new file mode 100644 index 000000000..6bdb9739c --- /dev/null +++ b/src/semantic_release/commit_parser/conventional/options.py @@ -0,0 +1,72 @@ +from __future__ import annotations + +from itertools import zip_longest +from typing import Tuple + +from pydantic.dataclasses import dataclass + +from semantic_release.commit_parser._base import ParserOptions +from semantic_release.enums import LevelBump + + +@dataclass +class ConventionalCommitParserOptions(ParserOptions): + """Options dataclass for the ConventionalCommitParser.""" + + minor_tags: Tuple[str, ...] = ("feat",) + """Commit-type prefixes that should result in a minor release bump.""" + + patch_tags: Tuple[str, ...] = ("fix", "perf") + """Commit-type prefixes that should result in a patch release bump.""" + + other_allowed_tags: Tuple[str, ...] = ( + "build", + "chore", + "ci", + "docs", + "style", + "refactor", + "test", + ) + """Commit-type prefixes that are allowed but do not result in a version bump.""" + + allowed_tags: Tuple[str, ...] = ( + *minor_tags, + *patch_tags, + *other_allowed_tags, + ) + """ + All commit-type prefixes that are allowed. + + These are used to identify a valid commit message. If a commit message does not start with + one of these prefixes, it will not be considered a valid commit message. + """ + + default_bump_level: LevelBump = LevelBump.NO_RELEASE + """The minimum bump level to apply to valid commit message.""" + + parse_squash_commits: bool = True + """Toggle flag for whether or not to parse squash commits""" + + ignore_merge_commits: bool = True + """Toggle flag for whether or not to ignore merge commits""" + + @property + def tag_to_level(self) -> dict[str, LevelBump]: + """A mapping of commit tags to the level bump they should result in.""" + return self._tag_to_level + + def __post_init__(self) -> None: + self._tag_to_level: dict[str, LevelBump] = { + str(tag): level + for tag, level in [ + # we have to do a type ignore as zip_longest provides a type that is not specific enough + # for our expected output. Due to the empty second array, we know the first is always longest + # and that means no values in the first entry of the tuples will ever be a LevelBump. We + # apply a str() to make mypy happy although it will never happen. + *zip_longest(self.allowed_tags, (), fillvalue=self.default_bump_level), + *zip_longest(self.patch_tags, (), fillvalue=LevelBump.PATCH), + *zip_longest(self.minor_tags, (), fillvalue=LevelBump.MINOR), + ] + if "|" not in str(tag) + } diff --git a/src/semantic_release/commit_parser/conventional/options_monorepo.py b/src/semantic_release/commit_parser/conventional/options_monorepo.py new file mode 100644 index 000000000..58bcaf47d --- /dev/null +++ b/src/semantic_release/commit_parser/conventional/options_monorepo.py @@ -0,0 +1,90 @@ +from __future__ import annotations + +from pathlib import Path +from re import compile as regexp, error as RegExpError # noqa: N812 +from typing import TYPE_CHECKING, Any, Iterable, Tuple + +from pydantic import Field, field_validator +from pydantic.dataclasses import dataclass + +# typing_extensions is for Python 3.8, 3.9, 3.10 compatibility +from typing_extensions import Annotated + +from semantic_release.commit_parser.conventional.options import ( + ConventionalCommitParserOptions, +) + +if TYPE_CHECKING: # pragma: no cover + pass + + +@dataclass +class ConventionalCommitMonorepoParserOptions(ConventionalCommitParserOptions): + # TODO: add example into the docstring + """Options dataclass for ConventionalCommitMonorepoParser.""" + + path_filters: Annotated[Tuple[str, ...], Field(validate_default=True)] = (".",) + """ + A set of relative paths to filter commits by. Only commits with file changes that + match these file paths or its subdirectories will be considered valid commits. + + Syntax is similar to .gitignore with file path globs and inverse file match globs + via `!` prefix. Paths should be relative to the current working directory. + """ + + scope_prefix: str = "" + """ + A prefix that will be striped from the scope when parsing commit messages. + + If set, it will cause unscoped commits to be ignored. Use this in tandem with + the `path_filters` option to filter commits by directory and scope. This will + be fed into a regular expression so you must escape any special characters that + are meaningful in regular expressions (e.g. `.`, `*`, `?`, `+`, etc.) if you want + to match them literally. + """ + + @classmethod + @field_validator("path_filters", mode="before") + def convert_strs_to_paths(cls, value: Any) -> tuple[Path, ...]: + values = value if isinstance(value, Iterable) else [value] + results: list[Path] = [] + + for val in values: + if isinstance(val, (str, Path)): + results.append(Path(val)) + continue + + raise TypeError(f"Invalid type: {type(val)}, expected str or Path.") + + return tuple(results) + + @classmethod + @field_validator("path_filters", mode="after") + def resolve_path(cls, dir_paths: tuple[Path, ...]) -> tuple[Path, ...]: + return tuple( + ( + Path(f"!{Path(str_path[1:]).expanduser().absolute().resolve()}") + # maintains the negation prefix if it exists + if (str_path := str(path)).startswith("!") + # otherwise, resolve the path normally + else path.expanduser().absolute().resolve() + ) + for path in dir_paths + ) + + @classmethod + @field_validator("scope_prefix", mode="after") + def validate_scope_prefix(cls, scope_prefix: str) -> str: + if not scope_prefix: + return "" + + # Allow the special case of a plain wildcard although it's not a valid regex + if scope_prefix == "*": + return ".*" + + try: + regexp(scope_prefix) + except RegExpError as err: + raise ValueError(f"Invalid regex {scope_prefix!r}") from err + + return scope_prefix diff --git a/src/semantic_release/commit_parser/conventional.py b/src/semantic_release/commit_parser/conventional/parser.py similarity index 74% rename from src/semantic_release/commit_parser/conventional.py rename to src/semantic_release/commit_parser/conventional/parser.py index 3cd50d9c7..5cab34c56 100644 --- a/src/semantic_release/commit_parser/conventional.py +++ b/src/semantic_release/commit_parser/conventional/parser.py @@ -1,16 +1,25 @@ from __future__ import annotations -import re from functools import reduce -from itertools import zip_longest -from re import compile as regexp +from logging import getLogger +from re import ( + DOTALL, + IGNORECASE, + MULTILINE, + Match as RegexMatch, + Pattern, + compile as regexp, + error as RegexError, # noqa: N812 +) from textwrap import dedent -from typing import TYPE_CHECKING, Tuple +from typing import TYPE_CHECKING, ClassVar from git.objects.commit import Commit -from pydantic.dataclasses import dataclass -from semantic_release.commit_parser._base import CommitParser, ParserOptions +from semantic_release.commit_parser._base import CommitParser +from semantic_release.commit_parser.conventional.options import ( + ConventionalCommitParserOptions, +) from semantic_release.commit_parser.token import ( ParsedCommit, ParsedMessageResult, @@ -25,16 +34,10 @@ ) from semantic_release.enums import LevelBump from semantic_release.errors import InvalidParserOptions -from semantic_release.globals import logger from semantic_release.helpers import sort_numerically, text_reducer -if TYPE_CHECKING: # pragma: no cover - from git.objects.commit import Commit - - -def _logged_parse_error(commit: Commit, error: str) -> ParseError: - logger.debug(error) - return ParseError(commit, error=error) +if TYPE_CHECKING: + pass # TODO: Remove from here, allow for user customization instead via options @@ -53,69 +56,6 @@ def _logged_parse_error(commit: Commit, error: str) -> ParseError: } -@dataclass -class ConventionalCommitParserOptions(ParserOptions): - """Options dataclass for the ConventionalCommitParser.""" - - minor_tags: Tuple[str, ...] = ("feat",) - """Commit-type prefixes that should result in a minor release bump.""" - - patch_tags: Tuple[str, ...] = ("fix", "perf") - """Commit-type prefixes that should result in a patch release bump.""" - - other_allowed_tags: Tuple[str, ...] = ( - "build", - "chore", - "ci", - "docs", - "style", - "refactor", - "test", - ) - """Commit-type prefixes that are allowed but do not result in a version bump.""" - - allowed_tags: Tuple[str, ...] = ( - *minor_tags, - *patch_tags, - *other_allowed_tags, - ) - """ - All commit-type prefixes that are allowed. - - These are used to identify a valid commit message. If a commit message does not start with - one of these prefixes, it will not be considered a valid commit message. - """ - - default_bump_level: LevelBump = LevelBump.NO_RELEASE - """The minimum bump level to apply to valid commit message.""" - - parse_squash_commits: bool = True - """Toggle flag for whether or not to parse squash commits""" - - ignore_merge_commits: bool = True - """Toggle flag for whether or not to ignore merge commits""" - - @property - def tag_to_level(self) -> dict[str, LevelBump]: - """A mapping of commit tags to the level bump they should result in.""" - return self._tag_to_level - - def __post_init__(self) -> None: - self._tag_to_level: dict[str, LevelBump] = { - str(tag): level - for tag, level in [ - # we have to do a type ignore as zip_longest provides a type that is not specific enough - # for our expected output. Due to the empty second array, we know the first is always longest - # and that means no values in the first entry of the tuples will ever be a LevelBump. We - # apply a str() to make mypy happy although it will never happen. - *zip_longest(self.allowed_tags, (), fillvalue=self.default_bump_level), - *zip_longest(self.patch_tags, (), fillvalue=LevelBump.PATCH), - *zip_longest(self.minor_tags, (), fillvalue=LevelBump.MINOR), - ] - if "|" not in str(tag) - } - - class ConventionalCommitParser( CommitParser[ParseResult, ConventionalCommitParserOptions] ): @@ -128,14 +68,57 @@ class ConventionalCommitParser( # TODO: Deprecate in lieu of get_default_options() parser_options = ConventionalCommitParserOptions + # GitHub & Gitea use (#123), GitLab uses (!123), and BitBucket uses (pull request #123) + mr_selector = regexp(r"[\t ]+\((?:pull request )?(?P[#!]\d+)\)[\t ]*$") + + issue_selector = regexp( + str.join( + "", + [ + r"^(?:clos(?:e|es|ed|ing)|fix(?:es|ed|ing)?|resolv(?:e|es|ed|ing)|implement(?:s|ed|ing)?):", + r"[\t ]+(?P.+)[\t ]*$", + ], + ), + flags=MULTILINE | IGNORECASE, + ) + + notice_selector = regexp(r"^NOTICE: (?P.+)$") + + common_commit_msg_filters: ClassVar[dict[str, tuple[Pattern[str], str]]] = { + "typo-extra-spaces": (regexp(r"(\S) +(\S)"), r"\1 \2"), + "git-header-commit": ( + regexp(r"^[\t ]*commit [0-9a-f]+$\n?", flags=MULTILINE), + "", + ), + "git-header-author": ( + regexp(r"^[\t ]*Author: .+$\n?", flags=MULTILINE), + "", + ), + "git-header-date": ( + regexp(r"^[\t ]*Date: .+$\n?", flags=MULTILINE), + "", + ), + "git-squash-heading": ( + regexp( + r"^[\t ]*Squashed commit of the following:.*$\n?", + flags=MULTILINE, + ), + "", + ), + } + def __init__(self, options: ConventionalCommitParserOptions | None = None) -> None: super().__init__(options) + self._logger = getLogger( + str.join(".", [self.__module__, self.__class__.__name__]) + ) + try: commit_type_pattern = regexp( r"(?P%s)" % str.join("|", self.options.allowed_tags) ) - except re.error as err: + except RegexError as err: raise InvalidParserOptions( str.join( "\n", @@ -167,45 +150,11 @@ def __init__(self, options: ConventionalCommitParserOptions | None = None) -> No r"(?:\n\n(?P.+))?", # commit body ], ), - flags=re.DOTALL, + flags=DOTALL, ) - # GitHub & Gitea use (#123), GitLab uses (!123), and BitBucket uses (pull request #123) - self.mr_selector = regexp( - r"[\t ]+\((?:pull request )?(?P[#!]\d+)\)[\t ]*$" - ) - self.issue_selector = regexp( - str.join( - "", - [ - r"^(?:clos(?:e|es|ed|ing)|fix(?:es|ed|ing)?|resolv(?:e|es|ed|ing)|implement(?:s|ed|ing)?):", - r"[\t ]+(?P.+)[\t ]*$", - ], - ), - flags=re.MULTILINE | re.IGNORECASE, - ) - self.notice_selector = regexp(r"^NOTICE: (?P.+)$") - self.filters = { - "typo-extra-spaces": (regexp(r"(\S) +(\S)"), r"\1 \2"), - "git-header-commit": ( - regexp(r"^[\t ]*commit [0-9a-f]+$\n?", flags=re.MULTILINE), - "", - ), - "git-header-author": ( - regexp(r"^[\t ]*Author: .+$\n?", flags=re.MULTILINE), - "", - ), - "git-header-date": ( - regexp(r"^[\t ]*Date: .+$\n?", flags=re.MULTILINE), - "", - ), - "git-squash-heading": ( - regexp( - r"^[\t ]*Squashed commit of the following:.*$\n?", - flags=re.MULTILINE, - ), - "", - ), + self.filters: dict[str, tuple[Pattern[str], str]] = { + **self.common_commit_msg_filters, "git-squash-commit-prefix": ( regexp( str.join( @@ -215,17 +164,20 @@ def __init__(self, options: ConventionalCommitParserOptions | None = None) -> No commit_type_pattern.pattern + r"\b", # prior to commit type ], ), - flags=re.MULTILINE, + flags=MULTILINE, ), # move commit type to the start of the line r"\1", ), } - @staticmethod - def get_default_options() -> ConventionalCommitParserOptions: + def get_default_options(self) -> ConventionalCommitParserOptions: return ConventionalCommitParserOptions() + def log_parse_error(self, commit: Commit, error: str) -> ParseError: + self._logger.debug(error) + return ParseError(commit, error=error) + def commit_body_components_separator( self, accumulator: dict[str, list[str]], text: str ) -> dict[str, list[str]]: @@ -267,14 +219,20 @@ def commit_body_components_separator( return accumulator def parse_message(self, message: str) -> ParsedMessageResult | None: - if not (parsed := self.commit_msg_pattern.match(message)): - return None + return ( + self.create_parsed_message_result(match) + if (match := self.commit_msg_pattern.match(message)) + else None + ) - parsed_break = parsed.group("break") - parsed_scope = parsed.group("scope") or "" - parsed_subject = parsed.group("subject") - parsed_text = parsed.group("text") - parsed_type = parsed.group("type") + def create_parsed_message_result( + self, match: RegexMatch[str] + ) -> ParsedMessageResult: + parsed_break = match.group("break") + parsed_scope = match.group("scope") or "" + parsed_subject = match.group("subject") + parsed_text = match.group("text") + parsed_type = match.group("type") linked_merge_request = "" if mr_match := self.mr_selector.search(parsed_subject): @@ -322,7 +280,7 @@ def is_merge_commit(commit: Commit) -> bool: def parse_commit(self, commit: Commit) -> ParseResult: if not (parsed_msg_result := self.parse_message(force_str(commit.message))): - return _logged_parse_error( + return self.log_parse_error( commit, f"Unable to parse commit message: {commit.message!r}", ) @@ -342,7 +300,7 @@ def parse(self, commit: Commit) -> ParseResult | list[ParseResult]: will be returned as a list of a single ParseResult. """ if self.options.ignore_merge_commits and self.is_merge_commit(commit): - return _logged_parse_error( + return self.log_parse_error( commit, "Ignoring merge commit: %s" % commit.hexsha[:8] ) diff --git a/src/semantic_release/commit_parser/conventional/parser_monorepo.py b/src/semantic_release/commit_parser/conventional/parser_monorepo.py new file mode 100644 index 000000000..18a938c43 --- /dev/null +++ b/src/semantic_release/commit_parser/conventional/parser_monorepo.py @@ -0,0 +1,467 @@ +from __future__ import annotations + +import os +from fnmatch import fnmatch +from logging import getLogger +from pathlib import Path, PurePath, PurePosixPath, PureWindowsPath +from re import DOTALL, compile as regexp, error as RegexError # noqa: N812 +from typing import TYPE_CHECKING + +from semantic_release.commit_parser._base import CommitParser +from semantic_release.commit_parser.conventional.options import ( + ConventionalCommitParserOptions, +) +from semantic_release.commit_parser.conventional.options_monorepo import ( + ConventionalCommitMonorepoParserOptions, +) +from semantic_release.commit_parser.conventional.parser import ConventionalCommitParser +from semantic_release.commit_parser.token import ( + ParsedCommit, + ParsedMessageResult, + ParseError, + ParseResult, +) +from semantic_release.commit_parser.util import force_str +from semantic_release.errors import InvalidParserOptions + +if TYPE_CHECKING: # pragma: no cover + from git.objects.commit import Commit + + +class ConventionalCommitMonorepoParser( + CommitParser[ParseResult, ConventionalCommitMonorepoParserOptions] +): + # TODO: Remove for v11 compatibility, get_default_options() will be called instead + parser_options = ConventionalCommitMonorepoParserOptions + + def __init__( + self, options: ConventionalCommitMonorepoParserOptions | None = None + ) -> None: + super().__init__(options) + + try: + commit_scope_pattern = regexp( + r"\(" + self.options.scope_prefix + r"(?P[^\n]+)?\)", + ) + except RegexError as err: + raise InvalidParserOptions( + str.join( + "\n", + [ + f"Invalid options for {self.__class__.__name__}", + "Unable to create regular expression from configured scope_prefix.", + "Please check the configured scope_prefix and remove or escape any regular expression characters.", + ], + ) + ) from err + + try: + commit_type_pattern = regexp( + r"(?P%s)" % str.join("|", self.options.allowed_tags) + ) + except RegexError as err: + raise InvalidParserOptions( + str.join( + "\n", + [ + f"Invalid options for {self.__class__.__name__}", + "Unable to create regular expression from configured commit-types.", + "Please check the configured commit-types and remove or escape any regular expression characters.", + ], + ) + ) from err + + # This regular expression includes scope prefix into the pattern and forces a scope to be present + # PSR will match the full scope but we don't include it in the scope match, + # which implicitly strips it from being included in the returned scope. + self._strict_scope_pattern = regexp( + str.join( + "", + [ + r"^" + commit_type_pattern.pattern, + commit_scope_pattern.pattern, + r"(?P!)?:\s+", + r"(?P[^\n]+)", + r"(?:\n\n(?P.+))?", # commit body + ], + ), + flags=DOTALL, + ) + + self._optional_scope_pattern = regexp( + str.join( + "", + [ + r"^" + commit_type_pattern.pattern, + r"(?:\((?P[^\n]+)\))?", + r"(?P!)?:\s+", + r"(?P[^\n]+)", + r"(?:\n\n(?P.+))?", # commit body + ], + ), + flags=DOTALL, + ) + + file_select_filters, file_ignore_filters = self._process_path_filter_options( + self.options.path_filters + ) + self._file_selection_filters: list[str] = file_select_filters + self._file_ignore_filters: list[str] = file_ignore_filters + + self._logger = getLogger( + str.join(".", [self.__module__, self.__class__.__name__]) + ) + + self._base_parser = ConventionalCommitParser( + options=ConventionalCommitParserOptions( + **{ + k: getattr(self.options, k) + for k in ConventionalCommitParserOptions().__dataclass_fields__ + } + ) + ) + + def get_default_options(self) -> ConventionalCommitMonorepoParserOptions: + return ConventionalCommitMonorepoParserOptions() + + @staticmethod + def _process_path_filter_options( # noqa: C901 + path_filters: tuple[str, ...], + ) -> tuple[list[str], list[str]]: + file_ignore_filters: list[str] = [] + file_selection_filters: list[str] = [] + unique_selection_filters: set[str] = set() + unique_ignore_filters: set[str] = set() + + for str_path in path_filters: + str_filter = str_path[1:] if str_path.startswith("!") else str_path + filter_list = ( + file_ignore_filters + if str_path.startswith("!") + else file_selection_filters + ) + unique_cache = ( + unique_ignore_filters + if str_path.startswith("!") + else unique_selection_filters + ) + + # Since fnmatch is not too flexible, we will expand the path filters to include the name and any subdirectories + # as this is how gitignore is interpreted. Possible scenarios: + # | Input | Path Normalization | Filter List | + # | ---------- | ------------------ | ------------------------- | + # | / | / | /** | done + # | /./ | / | /** | done + # | /** | /** | /** | done + # | /./** | /** | /** | done + # | /* | /* | /* | done + # | . | . | ./** | done + # | ./ | . | ./** | done + # | ././ | . | ./** | done + # | ./** | ./** | ./** | done + # | ./* | ./* | ./* | done + # | .. | .. | ../** | done + # | ../ | .. | ../** | done + # | ../** | ../** | ../** | done + # | ../* | ../* | ../* | done + # | ../.. | ../.. | ../../** | done + # | ../../ | ../../ | ../../** | done + # | ../../docs | ../../docs | ../../docs, ../../docs/** | done + # | src | src | src, src/** | done + # | src/ | src | src/** | done + # | src/* | src/* | src/* | done + # | src/** | src/** | src/** | done + # | /src | /src | /src, /src/** | done + # | /src/ | /src | /src/** | done + # | /src/** | /src/** | /src/** | done + # | /src/* | /src/* | /src/* | done + # | ../d/f.txt | ../d/f.txt | ../d/f.txt, ../d/f.txt/** | done + # This expansion will occur regardless of the negation prefix + + os_path: PurePath | PurePosixPath | PureWindowsPath = PurePath(str_filter) + + if r"\\" in str_filter: + # Windows paths were given so we convert them to posix paths + os_path = PureWindowsPath(str_filter) + os_path = ( + PureWindowsPath( + os_path.root, *os_path.parts[1:] + ) # drop any drive letter + if os_path.is_absolute() + else os_path + ) + os_path = PurePosixPath(os_path.as_posix()) + + path_normalized = str(os_path) + if path_normalized == str( + Path(".").absolute().root + ) or path_normalized == str(Path("/**")): + path_normalized = "/**" + + elif path_normalized == str(Path("/*")): + pass + + elif path_normalized == str(Path(".")) or path_normalized == str( + Path("./**") + ): + path_normalized = "./**" + + elif path_normalized == str(Path("./*")): + path_normalized = "./*" + + elif path_normalized == str(Path("..")) or path_normalized == str( + Path("../**") + ): + path_normalized = "../**" + + elif path_normalized == str(Path("../*")): + path_normalized = "../*" + + elif path_normalized.endswith(("..", "../**")): + path_normalized = f"{path_normalized.rstrip('*')}/**" + + elif str_filter.endswith(os.sep): + # If the path ends with a separator, it is a directory, so we add the directory and all subdirectories + path_normalized = f"{path_normalized}/**" + + elif not path_normalized.endswith("*"): + all_subdirs = f"{path_normalized}/**" + if all_subdirs not in unique_cache: + unique_cache.add(all_subdirs) + filter_list.append(all_subdirs) + # And fall through to add the path as is + + # END IF + + # Add the normalized path to the filter list if it is not already present + if path_normalized not in unique_cache: + unique_cache.add(path_normalized) + filter_list.append(path_normalized) + + return file_selection_filters, file_ignore_filters + + def logged_parse_error(self, commit: Commit, error: str) -> ParseError: + self._logger.debug(error) + return ParseError(commit, error=error) + + def parse(self, commit: Commit) -> ParseResult | list[ParseResult]: + if self.options.ignore_merge_commits and self._base_parser.is_merge_commit( + commit + ): + return self._base_parser.log_parse_error( + commit, "Ignoring merge commit: %s" % commit.hexsha[:8] + ) + + separate_commits: list[Commit] = ( + self._base_parser.unsquash_commit(commit) + if self.options.parse_squash_commits + else [commit] + ) + + # Parse each commit individually if there were more than one + parsed_commits: list[ParseResult] = list( + map(self.parse_commit, separate_commits) + ) + + def add_linked_merge_request( + parsed_result: ParseResult, mr_number: str + ) -> ParseResult: + return ( + parsed_result + if not isinstance(parsed_result, ParsedCommit) + else ParsedCommit( + **{ + **parsed_result._asdict(), + "linked_merge_request": mr_number, + } + ) + ) + + # TODO: improve this for other VCS systems other than GitHub & BitBucket + # Github works as the first commit in a squash merge commit has the PR number + # appended to the first line of the commit message + lead_commit = next(iter(parsed_commits)) + + if isinstance(lead_commit, ParsedCommit) and lead_commit.linked_merge_request: + # If the first commit has linked merge requests, assume all commits + # are part of the same PR and add the linked merge requests to all + # parsed commits + parsed_commits = [ + lead_commit, + *map( + lambda parsed_result, mr=lead_commit.linked_merge_request: ( # type: ignore[misc] + add_linked_merge_request(parsed_result, mr) + ), + parsed_commits[1:], + ), + ] + + elif isinstance(lead_commit, ParseError) and ( + mr_match := self._base_parser.mr_selector.search( + force_str(lead_commit.message) + ) + ): + # Handle BitBucket Squash Merge Commits (see #1085), which have non angular commit + # format but include the PR number in the commit subject that we want to extract + linked_merge_request = mr_match.group("mr_number") + + # apply the linked MR to all commits + parsed_commits = [ + add_linked_merge_request(parsed_result, linked_merge_request) + for parsed_result in parsed_commits + ] + + return parsed_commits + + def parse_message( + self, message: str, strict_scope: bool = False + ) -> ParsedMessageResult | None: + if ( + not (parsed_match := self._strict_scope_pattern.match(message)) + and strict_scope + ): + return None + + if not parsed_match and not ( + parsed_match := self._optional_scope_pattern.match(message) + ): + return None + + return self._base_parser.create_parsed_message_result(parsed_match) + + def parse_commit(self, commit: Commit) -> ParseResult: + """Attempt to parse the commit message with a regular expression into a ParseResult.""" + # Multiple scenarios to consider when parsing a commit message [Truth table]: + # ======================================================================================================= + # | || INPUTS || | + # | # ||------------------------+----------------+--------------|| Result | + # | || Example Commit Message | Relevant Files | Scope Prefix || | + # |----||------------------------+----------------+--------------||-------------------------------------| + # | 1 || type(prefix-cli): msg | yes | "prefix-" || ParsedCommit | + # | 2 || type(prefix-cli): msg | yes | "" || ParsedCommit | + # | 3 || type(prefix-cli): msg | no | "prefix-" || ParsedCommit | + # | 4 || type(prefix-cli): msg | no | "" || ParseError[No files] | + # | 5 || type(scope-cli): msg | yes | "prefix-" || ParsedCommit | + # | 6 || type(scope-cli): msg | yes | "" || ParsedCommit | + # | 7 || type(scope-cli): msg | no | "prefix-" || ParseError[No files & wrong scope] | + # | 8 || type(scope-cli): msg | no | "" || ParseError[No files] | + # | 9 || type(cli): msg | yes | "prefix-" || ParsedCommit | + # | 10 || type(cli): msg | yes | "" || ParsedCommit | + # | 11 || type(cli): msg | no | "prefix-" || ParseError[No files & wrong scope] | + # | 12 || type(cli): msg | no | "" || ParseError[No files] | + # | 13 || type: msg | yes | "prefix-" || ParsedCommit | + # | 14 || type: msg | yes | "" || ParsedCommit | + # | 15 || type: msg | no | "prefix-" || ParseError[No files & wrong scope] | + # | 16 || type: msg | no | "" || ParseError[No files] | + # | 17 || non-conventional msg | yes | "prefix-" || ParseError[Invalid Syntax] | + # | 18 || non-conventional msg | yes | "" || ParseError[Invalid Syntax] | + # | 19 || non-conventional msg | no | "prefix-" || ParseError[Invalid Syntax] | + # | 20 || non-conventional msg | no | "" || ParseError[Invalid Syntax] | + # ======================================================================================================= + + # Initial Logic Flow: + # [1] When there are no relevant files and a scope prefix is defined, we enforce a strict scope + # [2] When there are no relevant files and no scope prefix is defined, we parse scoped or unscoped commits + # [3] When there are relevant files, we parse scoped or unscoped commits regardless of any defined prefix + has_relevant_changed_files = self._has_relevant_changed_files(commit) + strict_scope = bool( + not has_relevant_changed_files and self.options.scope_prefix + ) + pmsg_result = self.parse_message( + message=force_str(commit.message), + strict_scope=strict_scope, + ) + + if pmsg_result and (has_relevant_changed_files or strict_scope): + self._logger.debug( + "commit %s introduces a %s level_bump", + commit.hexsha[:8], + pmsg_result.bump, + ) + + return ParsedCommit.from_parsed_message_result(commit, pmsg_result) + + if pmsg_result and not has_relevant_changed_files: + return self.logged_parse_error( + commit, + f"Commit {commit.hexsha[:7]} has no changed files matching the path filter(s)", + ) + + if strict_scope and self.parse_message(str(commit.message), strict_scope=False): + return self.logged_parse_error( + commit, + str.join( + " and ", + [ + f"Commit {commit.hexsha[:7]} has no changed files matching the path filter(s)", + f"the scope does not match scope prefix '{self.options.scope_prefix}'", + ], + ), + ) + + return self.logged_parse_error( + commit, + f"Format Mismatch! Unable to parse commit message: {commit.message!r}", + ) + + def unsquash_commit_message(self, message: str) -> list[str]: + return self._base_parser.unsquash_commit_message(message) + + def _has_relevant_changed_files(self, commit: Commit) -> bool: + # Extract git root from commit + git_root = ( + Path(commit.repo.working_tree_dir or commit.repo.working_dir) + .absolute() + .resolve() + ) + + cwd = Path.cwd().absolute().resolve() + + rel_cwd = cwd.relative_to(git_root) if git_root in cwd.parents else Path(".") + + sandboxed_selection_filters: list[str] = [ + str(file_filter) + for file_filter in ( + ( + git_root / select_filter.rstrip("/") + if Path(select_filter).is_absolute() + else git_root / rel_cwd / select_filter + ) + for select_filter in self._file_selection_filters + ) + if git_root in file_filter.parents + ] + + sandboxed_ignore_filters: list[str] = [ + str(file_filter) + for file_filter in ( + ( + git_root / ignore_filter.rstrip("/") + if Path(ignore_filter).is_absolute() + else git_root / rel_cwd / ignore_filter + ) + for ignore_filter in self._file_ignore_filters + ) + if git_root in file_filter.parents + ] + + # Check if the changed files of the commit that match the path filters + for full_path in iter( + str(git_root / rel_git_path) for rel_git_path in commit.stats.files + ): + # Check if the filepath matches any of the file selection filters + if not any( + fnmatch(full_path, select_filter) + for select_filter in sandboxed_selection_filters + ): + continue + + # Pass filter matches, so now evaluate if it is supposed to be ignored + if not any( + fnmatch(full_path, ignore_filter) + for ignore_filter in sandboxed_ignore_filters + ): + # No ignore filter matched, so it must be a relevant file + return True + + return False diff --git a/tests/const.py b/tests/const.py index c7cc0a8b4..69a7ca778 100644 --- a/tests/const.py +++ b/tests/const.py @@ -11,6 +11,9 @@ class RepoActionStep(str, Enum): CONFIGURE = "CONFIGURE" + CONFIGURE_MONOREPO = "CONFIGURE_MONOREPO" + CREATE_MONOREPO = "CREATE_MONOREPO" + CHANGE_DIRECTORY = "CHANGE_DIRECTORY" WRITE_CHANGELOGS = "WRITE_CHANGELOGS" GIT_CHECKOUT = "GIT_CHECKOUT" GIT_COMMIT = "GIT_COMMIT" diff --git a/tests/e2e/cmd_changelog/test_changelog.py b/tests/e2e/cmd_changelog/test_changelog.py index 757e1316b..0ff1bc710 100644 --- a/tests/e2e/cmd_changelog/test_changelog.py +++ b/tests/e2e/cmd_changelog/test_changelog.py @@ -76,7 +76,7 @@ from requests_mock import Mocker - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/e2e/cmd_changelog/test_changelog_custom_parser.py b/tests/e2e/cmd_changelog/test_changelog_custom_parser.py index 72b430875..5aa199bcb 100644 --- a/tests/e2e/cmd_changelog/test_changelog_custom_parser.py +++ b/tests/e2e/cmd_changelog/test_changelog_custom_parser.py @@ -18,7 +18,7 @@ if TYPE_CHECKING: from pathlib import Path - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) diff --git a/tests/e2e/cmd_version/bump_version/conftest.py b/tests/e2e/cmd_version/bump_version/conftest.py index 3af0678df..04a5209ce 100644 --- a/tests/e2e/cmd_version/bump_version/conftest.py +++ b/tests/e2e/cmd_version/bump_version/conftest.py @@ -18,13 +18,22 @@ from tests.conftest import RunCliFn from tests.fixtures.example_project import UpdatePyprojectTomlFn - from tests.fixtures.git_repo import BuildRepoFromDefinitionFn, RepoActionConfigure + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + RepoActionConfigure, + RepoActionConfigureMonorepo, + RepoActionCreateMonorepo, + ) class InitMirrorRepo4RebuildFn(Protocol): def __call__( self, mirror_repo_dir: Path, - configuration_steps: Sequence[RepoActionConfigure], + configuration_steps: Sequence[ + RepoActionConfigure + | RepoActionCreateMonorepo + | RepoActionConfigureMonorepo + ], files_to_remove: Sequence[Path], ) -> Path: ... @@ -43,7 +52,9 @@ def init_mirror_repo_for_rebuild( ) -> InitMirrorRepo4RebuildFn: def _init_mirror_repo_for_rebuild( mirror_repo_dir: Path, - configuration_steps: Sequence[RepoActionConfigure], + configuration_steps: Sequence[ + RepoActionConfigure | RepoActionCreateMonorepo | RepoActionConfigureMonorepo + ], files_to_remove: Sequence[Path], ) -> Path: # Create the mirror repo directory diff --git a/tests/e2e/cmd_version/bump_version/github_flow_monorepo/__init__.py b/tests/e2e/cmd_version/bump_version/github_flow_monorepo/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_1_channel.py b/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_1_channel.py new file mode 100644 index 000000000..fd5fb5ff2 --- /dev/null +++ b/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_1_channel.py @@ -0,0 +1,251 @@ +from __future__ import annotations + +from pathlib import Path +from typing import TYPE_CHECKING, cast + +import pytest +from freezegun import freeze_time + +from semantic_release.version.version import Version + +from tests.const import RepoActionStep +from tests.fixtures.monorepos.github_flow import ( + monorepo_w_github_flow_w_default_release_channel_conventional_commits, +) +from tests.util import temporary_working_directory + +if TYPE_CHECKING: + from typing import Literal, Sequence + from unittest.mock import MagicMock + + from requests_mock import Mocker + + from tests.e2e.cmd_version.bump_version.conftest import ( + InitMirrorRepo4RebuildFn, + RunPSReleaseFn, + ) + from tests.e2e.conftest import GetSanitizedChangelogContentFn + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildSpecificRepoFn, + CommitConvention, + GetGitRepo4DirFn, + RepoActionConfigure, + RepoActionConfigureMonorepo, + RepoActionCreateMonorepo, + RepoActionRelease, + RepoActions, + SplitRepoActionsByReleaseTagsFn, + ) + + +@pytest.mark.parametrize( + "repo_fixture_name", + [ + pytest.param(repo_fixture_name, marks=pytest.mark.comprehensive) + for repo_fixture_name in [ + monorepo_w_github_flow_w_default_release_channel_conventional_commits.__name__, + ] + ], +) +def test_githubflow_monorepo_rebuild_1_channel( + repo_fixture_name: str, + run_psr_release: RunPSReleaseFn, + build_monorepo_w_github_flow_w_default_release_channel: BuildSpecificRepoFn, + split_repo_actions_by_release_tags: SplitRepoActionsByReleaseTagsFn, + init_mirror_repo_for_rebuild: InitMirrorRepo4RebuildFn, + example_project_dir: ExProjectDir, + git_repo_for_directory: GetGitRepo4DirFn, + build_repo_from_definition: BuildRepoFromDefinitionFn, + mocked_git_push: MagicMock, + post_mocker: Mocker, + get_sanitized_md_changelog_content: GetSanitizedChangelogContentFn, + get_sanitized_rst_changelog_content: GetSanitizedChangelogContentFn, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_rst_file: Path, +): + # build target repo into a temporary directory + target_repo_dir = example_project_dir / repo_fixture_name + commit_type = cast( + "CommitConvention", repo_fixture_name.split("commits", 1)[0].split("_")[-2] + ) + target_repo_definition = build_monorepo_w_github_flow_w_default_release_channel( + repo_name=repo_fixture_name, + commit_type=commit_type, + dest_dir=target_repo_dir, + ) + target_git_repo = git_repo_for_directory(target_repo_dir) + + # split repo actions by release actions + release_tags_2_steps = split_repo_actions_by_release_tags(target_repo_definition) + + configuration_steps = cast( + "Sequence[RepoActionConfigure | RepoActionCreateMonorepo | RepoActionConfigureMonorepo]", + release_tags_2_steps.pop(None), + ) + + release_versions_2_steps = cast( + "dict[Version | Literal['Unreleased'], list[RepoActions]]", + release_tags_2_steps, + ) + + # Create the mirror repo directory + mirror_repo_dir = init_mirror_repo_for_rebuild( + mirror_repo_dir=(example_project_dir / "mirror"), + configuration_steps=configuration_steps, + files_to_remove=[], + ) + + mirror_git_repo = git_repo_for_directory(mirror_repo_dir) + + # rebuild repo from scratch stopping before each release tag + for curr_release_key, steps in release_versions_2_steps.items(): + curr_release_str = ( + curr_release_key.as_tag() + if isinstance(curr_release_key, Version) + else curr_release_key + ) + + # make sure mocks are clear + mocked_git_push.reset_mock() + post_mocker.reset_mock() + + # Extract expected result from target repo + if curr_release_str != "Unreleased": + target_git_repo.git.checkout(curr_release_str, detach=True, force=True) + + expected_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + expected_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + expected_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + expected_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + expected_pkg1_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + expected_pkg2_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + expected_pkg1_version_file_content = ( + target_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + expected_pkg2_version_file_content = ( + target_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + expected_release_commit_text = target_git_repo.head.commit.message + + # In our repo env, start building the repo from the definition + build_repo_from_definition( + dest_dir=mirror_repo_dir, + # stop before the release step + repo_construction_steps=steps[ + : -1 if curr_release_str != "Unreleased" else None + ], + ) + + release_directory = mirror_repo_dir + + for step in steps[::-1]: # reverse order + if step["action"] == RepoActionStep.CHANGE_DIRECTORY: + release_directory = ( + mirror_repo_dir + if str(Path(step["details"]["directory"])) + == str(mirror_repo_dir.root) + else Path(step["details"]["directory"]) + ) + + release_directory = ( + mirror_repo_dir / release_directory + if not release_directory.is_absolute() + else release_directory + ) + + if mirror_repo_dir not in release_directory.parents: + release_directory = mirror_repo_dir + + break + + # Act: run PSR on the repo instead of the RELEASE step + if curr_release_str != "Unreleased": + release_action_step = cast("RepoActionRelease", steps[-1]) + + with freeze_time( + release_action_step["details"]["datetime"] + ), temporary_working_directory(release_directory): + run_psr_release( + next_version_str=release_action_step["details"]["version"], + git_repo=mirror_git_repo, + config_toml_path=Path("pyproject.toml"), + ) + else: + # run psr changelog command to validate changelog + pass + + # take measurement after running the version command + actual_release_commit_text = mirror_git_repo.head.commit.message + actual_pkg1_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + actual_pkg2_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + actual_pkg1_version_file_content = ( + mirror_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + actual_pkg2_version_file_content = ( + mirror_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + actual_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + actual_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + actual_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + actual_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + + # Evaluate (normal release actions should have occurred as expected) + # ------------------------------------------------------------------ + # Make sure version file is updated + assert ( + expected_pkg1_pyproject_toml_content == actual_pkg1_pyproject_toml_content + ) + assert ( + expected_pkg2_pyproject_toml_content == actual_pkg2_pyproject_toml_content + ) + assert expected_pkg1_version_file_content == actual_pkg1_version_file_content + assert expected_pkg2_version_file_content == actual_pkg2_version_file_content + + # Make sure changelog is updated + assert expected_pkg1_md_changelog_content == actual_pkg1_md_changelog_content + assert expected_pkg2_md_changelog_content == actual_pkg2_md_changelog_content + assert expected_pkg1_rst_changelog_content == actual_pkg1_rst_changelog_content + assert expected_pkg2_rst_changelog_content == actual_pkg2_rst_changelog_content + + # Make sure commit is created + assert expected_release_commit_text == actual_release_commit_text + + if curr_release_str != "Unreleased": + # Make sure tag is created + assert curr_release_str in [tag.name for tag in mirror_git_repo.tags] + + # Make sure publishing actions occurred + assert mocked_git_push.call_count == 2 # 1 for commit, 1 for tag + assert post_mocker.call_count == 1 # vcs release creation occurred diff --git a/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_2_channels.py b/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_2_channels.py new file mode 100644 index 000000000..57d6cd3fb --- /dev/null +++ b/tests/e2e/cmd_version/bump_version/github_flow_monorepo/test_monorepo_2_channels.py @@ -0,0 +1,250 @@ +from __future__ import annotations + +from pathlib import Path +from typing import TYPE_CHECKING, cast + +import pytest +from freezegun import freeze_time + +from semantic_release.version.version import Version + +from tests.const import RepoActionStep +from tests.fixtures.monorepos.github_flow import ( + monorepo_w_github_flow_w_feature_release_channel_conventional_commits, +) +from tests.util import temporary_working_directory + +if TYPE_CHECKING: + from typing import Literal, Sequence + from unittest.mock import MagicMock + + from requests_mock import Mocker + + from tests.e2e.cmd_version.bump_version.conftest import ( + InitMirrorRepo4RebuildFn, + RunPSReleaseFn, + ) + from tests.e2e.conftest import GetSanitizedChangelogContentFn + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildSpecificRepoFn, + CommitConvention, + GetGitRepo4DirFn, + RepoActionConfigure, + RepoActionConfigureMonorepo, + RepoActionCreateMonorepo, + RepoActionRelease, + RepoActions, + SplitRepoActionsByReleaseTagsFn, + ) + + +@pytest.mark.parametrize( + "repo_fixture_name", + [ + pytest.param(repo_fixture_name, marks=pytest.mark.comprehensive) + for repo_fixture_name in [ + monorepo_w_github_flow_w_feature_release_channel_conventional_commits.__name__, + ] + ], +) +def test_githubflow_monorepo_rebuild_2_channels( + repo_fixture_name: str, + run_psr_release: RunPSReleaseFn, + build_monorepo_w_github_flow_w_feature_release_channel: BuildSpecificRepoFn, + split_repo_actions_by_release_tags: SplitRepoActionsByReleaseTagsFn, + init_mirror_repo_for_rebuild: InitMirrorRepo4RebuildFn, + example_project_dir: ExProjectDir, + git_repo_for_directory: GetGitRepo4DirFn, + build_repo_from_definition: BuildRepoFromDefinitionFn, + mocked_git_push: MagicMock, + post_mocker: Mocker, + get_sanitized_md_changelog_content: GetSanitizedChangelogContentFn, + get_sanitized_rst_changelog_content: GetSanitizedChangelogContentFn, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_rst_file: Path, +): + # build target repo into a temporary directory + target_repo_dir = example_project_dir / repo_fixture_name + commit_type = cast( + "CommitConvention", repo_fixture_name.split("commits", 1)[0].split("_")[-2] + ) + target_repo_definition = build_monorepo_w_github_flow_w_feature_release_channel( + repo_name=repo_fixture_name, + commit_type=commit_type, + dest_dir=target_repo_dir, + ) + target_git_repo = git_repo_for_directory(target_repo_dir) + + # split repo actions by release actions + release_tags_2_steps = split_repo_actions_by_release_tags(target_repo_definition) + + configuration_steps = cast( + "Sequence[RepoActionConfigure | RepoActionCreateMonorepo | RepoActionConfigureMonorepo]", + release_tags_2_steps.pop(None), + ) + + release_versions_2_steps = cast( + "dict[Version | Literal['Unreleased'], list[RepoActions]]", + release_tags_2_steps, + ) + + # Create the mirror repo directory + mirror_repo_dir = init_mirror_repo_for_rebuild( + mirror_repo_dir=(example_project_dir / "mirror"), + configuration_steps=configuration_steps, + files_to_remove=[], + ) + mirror_git_repo = git_repo_for_directory(mirror_repo_dir) + + # rebuild repo from scratch stopping before each release tag + for curr_release_key, steps in release_versions_2_steps.items(): + curr_release_str = ( + curr_release_key.as_tag() + if isinstance(curr_release_key, Version) + else curr_release_key + ) + + # make sure mocks are clear + mocked_git_push.reset_mock() + post_mocker.reset_mock() + + # Extract expected result from target repo + if curr_release_str != "Unreleased": + target_git_repo.git.checkout(curr_release_str, detach=True, force=True) + + expected_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + expected_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + expected_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + expected_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + expected_pkg1_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + expected_pkg2_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + expected_pkg1_version_file_content = ( + target_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + expected_pkg2_version_file_content = ( + target_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + expected_release_commit_text = target_git_repo.head.commit.message + + # In our repo env, start building the repo from the definition + build_repo_from_definition( + dest_dir=mirror_repo_dir, + # stop before the release step + repo_construction_steps=steps[ + : -1 if curr_release_str != "Unreleased" else None + ], + ) + + release_directory = mirror_repo_dir + + for step in steps[::-1]: # reverse order + if step["action"] == RepoActionStep.CHANGE_DIRECTORY: + release_directory = ( + mirror_repo_dir + if str(Path(step["details"]["directory"])) + == str(mirror_repo_dir.root) + else Path(step["details"]["directory"]) + ) + + release_directory = ( + mirror_repo_dir / release_directory + if not release_directory.is_absolute() + else release_directory + ) + + if mirror_repo_dir not in release_directory.parents: + release_directory = mirror_repo_dir + + break + + # Act: run PSR on the repo instead of the RELEASE step + if curr_release_str != "Unreleased": + release_action_step = cast("RepoActionRelease", steps[-1]) + + with freeze_time( + release_action_step["details"]["datetime"] + ), temporary_working_directory(release_directory): + run_psr_release( + next_version_str=release_action_step["details"]["version"], + git_repo=mirror_git_repo, + config_toml_path=Path("pyproject.toml"), + ) + else: + # run psr changelog command to validate changelog + pass + + # take measurement after running the version command + actual_release_commit_text = mirror_git_repo.head.commit.message + actual_pkg1_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + actual_pkg2_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + actual_pkg1_version_file_content = ( + mirror_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + actual_pkg2_version_file_content = ( + mirror_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + actual_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + actual_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + actual_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + actual_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + + # Evaluate (normal release actions should have occurred as expected) + # ------------------------------------------------------------------ + # Make sure version file is updated + assert ( + expected_pkg1_pyproject_toml_content == actual_pkg1_pyproject_toml_content + ) + assert ( + expected_pkg2_pyproject_toml_content == actual_pkg2_pyproject_toml_content + ) + assert expected_pkg1_version_file_content == actual_pkg1_version_file_content + assert expected_pkg2_version_file_content == actual_pkg2_version_file_content + + # Make sure changelog is updated + assert expected_pkg1_md_changelog_content == actual_pkg1_md_changelog_content + assert expected_pkg2_md_changelog_content == actual_pkg2_md_changelog_content + assert expected_pkg1_rst_changelog_content == actual_pkg1_rst_changelog_content + assert expected_pkg2_rst_changelog_content == actual_pkg2_rst_changelog_content + + # Make sure commit is created + assert expected_release_commit_text == actual_release_commit_text + + if curr_release_str != "Unreleased": + # Make sure tag is created + assert curr_release_str in [tag.name for tag in mirror_git_repo.tags] + + # Make sure publishing actions occurred + assert mocked_git_push.call_count == 2 # 1 for commit, 1 for tag + assert post_mocker.call_count == 1 # vcs release creation occurred diff --git a/tests/e2e/cmd_version/bump_version/trunk_based_dev_monorepo/__init__.py b/tests/e2e/cmd_version/bump_version/trunk_based_dev_monorepo/__init__.py new file mode 100644 index 000000000..e69de29bb diff --git a/tests/e2e/cmd_version/bump_version/trunk_based_dev_monorepo/test_monorepo_trunk.py b/tests/e2e/cmd_version/bump_version/trunk_based_dev_monorepo/test_monorepo_trunk.py new file mode 100644 index 000000000..ec4ccd60a --- /dev/null +++ b/tests/e2e/cmd_version/bump_version/trunk_based_dev_monorepo/test_monorepo_trunk.py @@ -0,0 +1,251 @@ +from __future__ import annotations + +from pathlib import Path +from typing import TYPE_CHECKING, cast + +import pytest +from freezegun import freeze_time + +from semantic_release.version.version import Version + +from tests.const import RepoActionStep +from tests.fixtures.monorepos.trunk_based_dev import ( + monorepo_w_trunk_only_releases_conventional_commits, +) +from tests.util import temporary_working_directory + +if TYPE_CHECKING: + from typing import Literal, Sequence + from unittest.mock import MagicMock + + from requests_mock import Mocker + + from tests.e2e.cmd_version.bump_version.conftest import ( + InitMirrorRepo4RebuildFn, + RunPSReleaseFn, + ) + from tests.e2e.conftest import GetSanitizedChangelogContentFn + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildSpecificRepoFn, + CommitConvention, + GetGitRepo4DirFn, + RepoActionConfigure, + RepoActionConfigureMonorepo, + RepoActionCreateMonorepo, + RepoActionRelease, + RepoActions, + SplitRepoActionsByReleaseTagsFn, + ) + + +@pytest.mark.parametrize( + "repo_fixture_name", + [ + pytest.param(repo_fixture_name, marks=pytest.mark.comprehensive) + for repo_fixture_name in [ + monorepo_w_trunk_only_releases_conventional_commits.__name__, + ] + ], +) +def test_trunk_monorepo_rebuild_1_channel( + repo_fixture_name: str, + run_psr_release: RunPSReleaseFn, + build_trunk_only_monorepo_w_tags: BuildSpecificRepoFn, + split_repo_actions_by_release_tags: SplitRepoActionsByReleaseTagsFn, + init_mirror_repo_for_rebuild: InitMirrorRepo4RebuildFn, + example_project_dir: ExProjectDir, + git_repo_for_directory: GetGitRepo4DirFn, + build_repo_from_definition: BuildRepoFromDefinitionFn, + mocked_git_push: MagicMock, + post_mocker: Mocker, + get_sanitized_md_changelog_content: GetSanitizedChangelogContentFn, + get_sanitized_rst_changelog_content: GetSanitizedChangelogContentFn, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_rst_file: Path, +): + # build target repo into a temporary directory + target_repo_dir = example_project_dir / repo_fixture_name + commit_type = cast( + "CommitConvention", repo_fixture_name.split("commits", 1)[0].split("_")[-2] + ) + target_repo_definition = build_trunk_only_monorepo_w_tags( + repo_name=repo_fixture_name, + commit_type=commit_type, + dest_dir=target_repo_dir, + ) + target_git_repo = git_repo_for_directory(target_repo_dir) + + # split repo actions by release actions + release_tags_2_steps = split_repo_actions_by_release_tags(target_repo_definition) + + configuration_steps = cast( + "Sequence[RepoActionConfigure | RepoActionCreateMonorepo | RepoActionConfigureMonorepo]", + release_tags_2_steps.pop(None), + ) + + release_versions_2_steps = cast( + "dict[Version | Literal['Unreleased'], list[RepoActions]]", + release_tags_2_steps, + ) + + # Create the mirror repo directory + mirror_repo_dir = init_mirror_repo_for_rebuild( + mirror_repo_dir=(example_project_dir / "mirror"), + configuration_steps=configuration_steps, + files_to_remove=[], + ) + + mirror_git_repo = git_repo_for_directory(mirror_repo_dir) + + # rebuild repo from scratch stopping before each release tag + for curr_release_key, steps in release_versions_2_steps.items(): + curr_release_str = ( + curr_release_key.as_tag() + if isinstance(curr_release_key, Version) + else curr_release_key + ) + + # make sure mocks are clear + mocked_git_push.reset_mock() + post_mocker.reset_mock() + + # Extract expected result from target repo + if curr_release_str != "Unreleased": + target_git_repo.git.checkout(curr_release_str, detach=True, force=True) + + expected_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + expected_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + expected_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + expected_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=target_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + expected_pkg1_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + expected_pkg2_pyproject_toml_content = ( + target_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + expected_pkg1_version_file_content = ( + target_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + expected_pkg2_version_file_content = ( + target_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + expected_release_commit_text = target_git_repo.head.commit.message + + # In our repo env, start building the repo from the definition + build_repo_from_definition( + dest_dir=mirror_repo_dir, + # stop before the release step + repo_construction_steps=steps[ + : -1 if curr_release_str != "Unreleased" else None + ], + ) + + release_directory = mirror_repo_dir + + for step in steps[::-1]: # reverse order + if step["action"] == RepoActionStep.CHANGE_DIRECTORY: + release_directory = ( + mirror_repo_dir + if str(Path(step["details"]["directory"])) + == str(mirror_repo_dir.root) + else Path(step["details"]["directory"]) + ) + + release_directory = ( + mirror_repo_dir / release_directory + if not release_directory.is_absolute() + else release_directory + ) + + if mirror_repo_dir not in release_directory.parents: + release_directory = mirror_repo_dir + + break + + # Act: run PSR on the repo instead of the RELEASE step + if curr_release_str != "Unreleased": + release_action_step = cast("RepoActionRelease", steps[-1]) + + with freeze_time( + release_action_step["details"]["datetime"] + ), temporary_working_directory(release_directory): + run_psr_release( + next_version_str=release_action_step["details"]["version"], + git_repo=mirror_git_repo, + config_toml_path=Path("pyproject.toml"), + ) + else: + # run psr changelog command to validate changelog + pass + + # take measurement after running the version command + actual_release_commit_text = mirror_git_repo.head.commit.message + actual_pkg1_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg1_pyproject_toml_file + ).read_text() + actual_pkg2_pyproject_toml_content = ( + mirror_repo_dir / monorepo_pkg2_pyproject_toml_file + ).read_text() + actual_pkg1_version_file_content = ( + mirror_repo_dir / monorepo_pkg1_version_py_file + ).read_text() + actual_pkg2_version_file_content = ( + mirror_repo_dir / monorepo_pkg2_version_py_file + ).read_text() + actual_pkg1_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_md_file + ) + actual_pkg2_md_changelog_content = get_sanitized_md_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_md_file + ) + actual_pkg1_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg1_changelog_rst_file + ) + actual_pkg2_rst_changelog_content = get_sanitized_rst_changelog_content( + repo_dir=mirror_repo_dir, changelog_file=monorepo_pkg2_changelog_rst_file + ) + + # Evaluate (normal release actions should have occurred as expected) + # ------------------------------------------------------------------ + # Make sure version file is updated + assert ( + expected_pkg1_pyproject_toml_content == actual_pkg1_pyproject_toml_content + ) + assert ( + expected_pkg2_pyproject_toml_content == actual_pkg2_pyproject_toml_content + ) + assert expected_pkg1_version_file_content == actual_pkg1_version_file_content + assert expected_pkg2_version_file_content == actual_pkg2_version_file_content + + # Make sure changelog is updated + assert expected_pkg1_md_changelog_content == actual_pkg1_md_changelog_content + assert expected_pkg2_md_changelog_content == actual_pkg2_md_changelog_content + assert expected_pkg1_rst_changelog_content == actual_pkg1_rst_changelog_content + assert expected_pkg2_rst_changelog_content == actual_pkg2_rst_changelog_content + + # Make sure commit is created + assert expected_release_commit_text == actual_release_commit_text + + if curr_release_str != "Unreleased": + # Make sure tag is created + assert curr_release_str in [tag.name for tag in mirror_git_repo.tags] + + # Make sure publishing actions occurred + assert mocked_git_push.call_count == 2 # 1 for commit, 1 for tag + assert post_mocker.call_count == 1 # vcs release creation occurred diff --git a/tests/fixtures/__init__.py b/tests/fixtures/__init__.py index d9e987f57..fcf471c9b 100644 --- a/tests/fixtures/__init__.py +++ b/tests/fixtures/__init__.py @@ -1,5 +1,6 @@ from tests.fixtures.commit_parsers import * from tests.fixtures.example_project import * from tests.fixtures.git_repo import * +from tests.fixtures.monorepos import * from tests.fixtures.repos import * from tests.fixtures.scipy import * diff --git a/tests/fixtures/example_project.py b/tests/fixtures/example_project.py index 6b804fdae..41664f75a 100644 --- a/tests/fixtures/example_project.py +++ b/tests/fixtures/example_project.py @@ -24,6 +24,9 @@ EmojiCommitParser, ScipyCommitParser, ) +from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, +) from semantic_release.hvcs import Bitbucket, Gitea, Github, Gitlab import tests.conftest @@ -83,7 +86,7 @@ def __call__( class UseParserFn(Protocol): def __call__( - self, toml_file: Path | str = ... + self, toml_file: Path | str = ..., monorepo: bool = ... ) -> type[CommitParser[ParseResult, ParserOptions]]: ... class UseReleaseNotesTemplateFn(Protocol): @@ -606,13 +609,16 @@ def use_conventional_parser( """Modify the configuration file to use the Conventional parser.""" def _use_conventional_parser( - toml_file: Path | str = pyproject_toml_file, + toml_file: Path | str = pyproject_toml_file, monorepo: bool = False ) -> type[CommitParser[ParseResult, ParserOptions]]: update_pyproject_toml( - pyproject_toml_config_option_parser, "conventional", toml_file=toml_file + pyproject_toml_config_option_parser, + f"conventional{'-monorepo' if monorepo else ''}", + toml_file=toml_file, ) return cast( - "type[CommitParser[ParseResult, ParserOptions]]", ConventionalCommitParser + "type[CommitParser[ParseResult, ParserOptions]]", + ConventionalCommitMonorepoParser if monorepo else ConventionalCommitParser, ) return _use_conventional_parser @@ -627,8 +633,14 @@ def use_emoji_parser( """Modify the configuration file to use the Emoji parser.""" def _use_emoji_parser( - toml_file: Path | str = pyproject_toml_file, + toml_file: Path | str = pyproject_toml_file, monorepo: bool = False ) -> type[CommitParser[ParseResult, ParserOptions]]: + if monorepo: + raise ValueError( + "The Emoji parser does not support monorepo mode. " + "Use the conventional parser instead." + ) + update_pyproject_toml( pyproject_toml_config_option_parser, "emoji", toml_file=toml_file ) @@ -646,8 +658,14 @@ def use_scipy_parser( """Modify the configuration file to use the Scipy parser.""" def _use_scipy_parser( - toml_file: Path | str = pyproject_toml_file, + toml_file: Path | str = pyproject_toml_file, monorepo: bool = False ) -> type[CommitParser[ParseResult, ParserOptions]]: + if monorepo: + raise ValueError( + "The Scipy parser does not support monorepo mode. " + "Use the conventional parser instead." + ) + update_pyproject_toml( pyproject_toml_config_option_parser, "scipy", toml_file=toml_file ) diff --git a/tests/fixtures/git_repo.py b/tests/fixtures/git_repo.py index f76f83455..e4051183f 100644 --- a/tests/fixtures/git_repo.py +++ b/tests/fixtures/git_repo.py @@ -64,6 +64,9 @@ from semantic_release.commit_parser.conventional import ( ConventionalCommitParser, ) + from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, + ) from semantic_release.commit_parser.emoji import EmojiCommitParser from semantic_release.commit_parser.scipy import ScipyCommitParser from semantic_release.commit_parser.token import ParsedMessageResult, ParseResult @@ -73,6 +76,7 @@ GetParserFromConfigFileFn, UpdateVersionPyFileFn, ) + from tests.fixtures.monorepos.git_monorepo import BuildMonorepoFn try: # Python 3.8 and 3.9 compatibility @@ -155,6 +159,7 @@ def __call__( extra_configs: dict[str, TomlSerializableTypes] | None = None, mask_initial_release: bool = True, # Default as of v10 package_name: str = ..., + monorepo: bool = False, ) -> tuple[Path, HvcsBase]: ... class CommitNReturnChangelogEntryFn(Protocol): @@ -301,6 +306,34 @@ class RepoActionConfigureDetails(DetailsBase): mask_initial_release: bool extra_configs: dict[str, TomlSerializableTypes] + class RepoActionConfigureMonorepo(TypedDict): + action: Literal[RepoActionStep.CONFIGURE_MONOREPO] + details: RepoActionConfigureMonorepoDetails + + class RepoActionConfigureMonorepoDetails(DetailsBase): + package_dir: Path | str + package_name: str + tag_format_str: str | None + mask_initial_release: bool + extra_configs: dict[str, TomlSerializableTypes] + + class RepoActionCreateMonorepo(TypedDict): + action: Literal[RepoActionStep.CREATE_MONOREPO] + details: RepoActionCreateMonorepoDetails + + class RepoActionCreateMonorepoDetails(DetailsBase): + commit_type: CommitConvention + hvcs_client_name: str + hvcs_domain: str + origin_url: NotRequired[str] + + class RepoActionChangeDirectory(TypedDict): + action: Literal[RepoActionStep.CHANGE_DIRECTORY] + details: RepoActionChangeDirectoryDetails + + class RepoActionChangeDirectoryDetails(DetailsBase): + directory: Path | str + class RepoActionMakeCommits(TypedDict): action: Literal[RepoActionStep.MAKE_COMMITS] details: RepoActionMakeCommitsDetails @@ -381,6 +414,7 @@ def __call__( commit_spec: CommitSpec, commit_type: CommitConvention, parser: CommitParser[ParseResult, ParserOptions], + monorepo: bool = ..., ) -> CommitDef: ... class GetRepoDefinitionFn(Protocol): @@ -415,6 +449,7 @@ def __call__( commits: Sequence[CommitSpec], commit_type: CommitConvention, parser: CommitParser[ParseResult, ParserOptions], + monorepo: bool = ..., ) -> Sequence[CommitDef]: ... class BuildSpecificRepoFn(Protocol): @@ -423,7 +458,10 @@ def __call__( ) -> Sequence[RepoActions]: ... RepoActions: TypeAlias = Union[ + RepoActionChangeDirectory, RepoActionConfigure, + RepoActionConfigureMonorepo, + RepoActionCreateMonorepo, RepoActionGitCheckout, RepoActionGitMerge[RepoActionGitMergeDetails], RepoActionGitMerge[RepoActionGitFFMergeDetails], @@ -611,6 +649,43 @@ def _get_commit_def(msg: str, parser: ConventionalCommitParser) -> CommitDef: return _get_commit_def +@pytest.fixture(scope="session") +def get_commit_def_of_conventional_commit_monorepo() -> ( + GetCommitDefFn[ConventionalCommitMonorepoParser] +): + def _get_commit_def( + msg: str, parser: ConventionalCommitMonorepoParser + ) -> CommitDef: + if not (parsed_result := parser.parse_message(msg)): + return { + "cid": "", + "msg": msg, + "type": "unknown", + "category": "Unknown", + "desc": msg, + "brking_desc": "", + "scope": "", + "mr": "", + "sha": NULL_HEX_SHA, + "include_in_changelog": False, + } + + return { + "cid": "", + "msg": msg, + "type": parsed_result.type, + "category": parsed_result.category, + "desc": str.join("\n\n", parsed_result.descriptions), + "brking_desc": str.join("\n\n", parsed_result.breaking_descriptions), + "scope": parsed_result.scope, + "mr": parsed_result.linked_merge_request, + "sha": NULL_HEX_SHA, + "include_in_changelog": True, + } + + return _get_commit_def + + @pytest.fixture(scope="session") def get_commit_def_of_emoji_commit() -> GetCommitDefFn[EmojiCommitParser]: def _get_commit_def_of_emoji_commit( @@ -1078,6 +1153,7 @@ def _build_configured_base_repo( # noqa: C901 extra_configs: dict[str, TomlSerializableTypes] | None = None, mask_initial_release: bool = True, # Default as of v10 package_name: str = EXAMPLE_PROJECT_NAME, + monorepo: bool = False, ) -> tuple[Path, HvcsBase]: if not cached_example_git_project.exists(): raise RuntimeError("Unable to find cached git project files!") @@ -1094,6 +1170,7 @@ def _build_configured_base_repo( # noqa: C901 extra_configs=extra_configs, mask_initial_release=mask_initial_release, package_name=package_name, + monorepo=monorepo, ) return _build_configured_base_repo @@ -1131,16 +1208,19 @@ def _configure_base_repo( # noqa: C901 extra_configs: dict[str, TomlSerializableTypes] | None = None, mask_initial_release: bool = True, # Default as of v10 package_name: str = EXAMPLE_PROJECT_NAME, + monorepo: bool = False, ) -> tuple[Path, HvcsBase]: # Make sure we are in the dest directory with temporary_working_directory(dest_dir): # Set parser configuration if commit_type == "conventional": - use_conventional_parser(toml_file=pyproject_toml_file) + use_conventional_parser( + toml_file=pyproject_toml_file, monorepo=monorepo + ) elif commit_type == "emoji": - use_emoji_parser(toml_file=pyproject_toml_file) + use_emoji_parser(toml_file=pyproject_toml_file, monorepo=monorepo) elif commit_type == "scipy": - use_scipy_parser(toml_file=pyproject_toml_file) + use_scipy_parser(toml_file=pyproject_toml_file, monorepo=monorepo) else: use_custom_parser(commit_type, toml_file=pyproject_toml_file) @@ -1290,12 +1370,16 @@ def _separate_squashed_commit_def( @pytest.fixture(scope="session") def convert_commit_spec_to_commit_def( get_commit_def_of_conventional_commit: GetCommitDefFn[ConventionalCommitParser], + get_commit_def_of_conventional_commit_monorepo: GetCommitDefFn[ + ConventionalCommitMonorepoParser + ], get_commit_def_of_emoji_commit: GetCommitDefFn[EmojiCommitParser], get_commit_def_of_scipy_commit: GetCommitDefFn[ScipyCommitParser], stable_now_date: datetime, ) -> ConvertCommitSpecToCommitDefFn: message_parsers = { "conventional": get_commit_def_of_conventional_commit, + "conventional-monorepo": get_commit_def_of_conventional_commit_monorepo, "emoji": get_commit_def_of_emoji_commit, "scipy": get_commit_def_of_scipy_commit, } @@ -1304,8 +1388,12 @@ def _convert( commit_spec: CommitSpec, commit_type: CommitConvention, parser: CommitParser[ParseResult, ParserOptions], + monorepo: bool = False, ) -> CommitDef: - parse_msg_fn = cast("GetCommitDefFn[Any]", message_parsers[commit_type]) + parse_msg_fn = cast( + "GetCommitDefFn[Any]", + message_parsers[f"{commit_type}{'-monorepo' if monorepo else ''}"], + ) # Extract the correct commit message for the commit type return { @@ -1330,9 +1418,12 @@ def _convert( commits: Sequence[CommitSpec], commit_type: CommitConvention, parser: CommitParser[ParseResult, ParserOptions], + monorepo: bool = False, ) -> Sequence[CommitDef]: return [ - convert_commit_spec_to_commit_def(commit, commit_type, parser=parser) + convert_commit_spec_to_commit_def( + commit, commit_type, parser=parser, monorepo=monorepo + ) for commit in commits ] @@ -1342,6 +1433,8 @@ def _convert( @pytest.fixture(scope="session") def build_repo_from_definition( # noqa: C901, its required and its just test code build_configured_base_repo: BuildRepoFn, + build_base_monorepo: BuildMonorepoFn, + configure_monorepo_package: BuildRepoFn, default_tag_format_str: str, create_release_tagged_commit: CreateReleaseFn, create_squash_merge_commit: CreateSquashMergeCommitFn, @@ -1386,6 +1479,7 @@ def _build_repo_from_definition( # noqa: C901, its required and its just test c ) repo_dir = Path(dest_dir).resolve().absolute() + commit_type: CommitConvention = "conventional" hvcs: Github | Gitlab | Gitea | Bitbucket commit_cache: dict[str, CommitDef] = {} current_repo_def: dict[Version | Literal["Unreleased"], RepoVersionDef] = {} @@ -1418,6 +1512,64 @@ def _build_repo_from_definition( # noqa: C901, its required and its just test c }, ) + elif action == RepoActionStep.CREATE_MONOREPO: + cfg_mr_def = cast( + "RepoActionCreateMonorepoDetails", step_result["details"] + ) + build_base_monorepo(dest_dir=repo_dir) + hvcs = get_hvcs( + hvcs_client_name=cfg_mr_def["hvcs_client_name"], + origin_url=cfg_mr_def.get("origin_url") + or example_git_https_url, + hvcs_domain=cfg_mr_def["hvcs_domain"], + ) + commit_type = cfg_mr_def["commit_type"] + + elif action == RepoActionStep.CONFIGURE_MONOREPO: + cfg_mr_pkg_def = cast( + "RepoActionConfigureMonorepoDetails", step_result["details"] + ) + configure_monorepo_package( + dest_dir=cfg_mr_pkg_def["package_dir"], + commit_type=commit_type, + hvcs_client_name=hvcs.__class__.__name__.lower(), + hvcs_domain=str(hvcs.hvcs_domain), + tag_format_str=cfg_mr_pkg_def["tag_format_str"], + extra_configs=cfg_mr_pkg_def["extra_configs"], + mask_initial_release=cfg_mr_pkg_def["mask_initial_release"], + package_name=cfg_mr_pkg_def["package_name"], + monorepo=True, + ) + + elif action == RepoActionStep.CHANGE_DIRECTORY: + change_dir_def = cast( + "RepoActionChangeDirectoryDetails", step_result["details"] + ) + if not ( + new_cwd := Path(change_dir_def["directory"]) + .resolve() + .absolute() + ).exists(): + msg = f"Directory {change_dir_def['directory']} does not exist." + raise NotADirectoryError(msg) + + # Helpful Transform to find the project root repo without needing to pass it around (ie '/' => repo_dir) + new_cwd = ( + repo_dir if str(new_cwd) == str(repo_dir.root) else new_cwd + ) + + if not new_cwd.is_dir(): + msg = f"Path {change_dir_def['directory']} is not a directory." + raise NotADirectoryError(msg) + + # TODO: 3.9+, use is_relative_to + # if not new_cwd.is_relative_to(repo_dir): + if repo_dir != new_cwd and repo_dir not in new_cwd.parents: + msg = f"Cannot change directory to '{new_cwd}' as it is outside the repo directory '{repo_dir}'." + raise ValueError(msg) + + os.chdir(str(new_cwd)) + elif action == RepoActionStep.MAKE_COMMITS: mk_cmts_def = cast( "RepoActionMakeCommitsDetails", step_result["details"] @@ -1571,19 +1723,19 @@ def _build_repo_from_definition( # noqa: C901, its required and its just test c ) elif action == RepoActionStep.GIT_MERGE: - this_step = cast("RepoActionGitMerge", step_result) + this_step = cast( + "RepoActionGitMerge[RepoActionGitFFMergeDetails | RepoActionGitMergeDetails]", + step_result, + ) with Repo(repo_dir) as git_repo: if this_step["details"]["fast_forward"]: - ff_merge_def = cast( - "RepoActionGitFFMergeDetails", this_step["details"] + git_repo.git.merge( + this_step["details"]["branch_name"], ff=True ) - git_repo.git.merge(ff_merge_def["branch_name"], ff=True) else: - merge_def = cast( - "RepoActionGitMergeDetails", this_step["details"] - ) + merge_def = this_step["details"] # Update the commit definition with the repo hash merge_def["commit_def"] = create_merge_commit( @@ -1735,7 +1887,14 @@ def _split_repo_actions_by_release_tags( # Loop through all actions and split them by release tags for step in repo_definition: - if any(step["action"] == action for action in [RepoActionStep.CONFIGURE]): + if any( + step["action"] == action + for action in [ + RepoActionStep.CONFIGURE, + RepoActionStep.CREATE_MONOREPO, + RepoActionStep.CONFIGURE_MONOREPO, + ] + ): releasetags_2_steps[None].append(step) continue @@ -1754,6 +1913,7 @@ def _split_repo_actions_by_release_tags( insignificant_actions = [ RepoActionStep.GIT_CHECKOUT, + RepoActionStep.CHANGE_DIRECTORY, ] # Remove Unreleased if there are no significant steps in an Unreleased section diff --git a/tests/fixtures/monorepos/__init__.py b/tests/fixtures/monorepos/__init__.py new file mode 100644 index 000000000..e3546c89f --- /dev/null +++ b/tests/fixtures/monorepos/__init__.py @@ -0,0 +1,4 @@ +from tests.fixtures.monorepos.example_monorepo import * +from tests.fixtures.monorepos.git_monorepo import * +from tests.fixtures.monorepos.github_flow import * +from tests.fixtures.monorepos.trunk_based_dev import * diff --git a/tests/fixtures/monorepos/example_monorepo.py b/tests/fixtures/monorepos/example_monorepo.py new file mode 100644 index 000000000..6150e0635 --- /dev/null +++ b/tests/fixtures/monorepos/example_monorepo.py @@ -0,0 +1,523 @@ +from __future__ import annotations + +from pathlib import Path +from textwrap import dedent +from typing import TYPE_CHECKING + +import pytest + +# NOTE: use backport with newer API +import tests.conftest +import tests.const +import tests.fixtures.example_project +import tests.util +from tests.const import ( + EXAMPLE_PROJECT_NAME, + EXAMPLE_PROJECT_VERSION, + EXAMPLE_PYPROJECT_TOML_CONTENT, + EXAMPLE_RELEASE_NOTES_TEMPLATE, +) +from tests.util import copy_dir_tree, temporary_working_directory + +if TYPE_CHECKING: + from typing import Any, Protocol, Sequence + + from tests.conftest import ( + BuildRepoOrCopyCacheFn, + GetMd5ForSetOfFilesFn, + ) + from tests.fixtures.example_project import ( + UpdatePyprojectTomlFn, + UpdateVersionPyFileFn, + ) + from tests.fixtures.git_repo import RepoActions + + # class GetWheelFileFn(Protocol): + # def __call__(self, version_str: str) -> Path: ... + + class UpdatePkgPyprojectTomlFn(Protocol): + def __call__(self, pkg_name: str, setting: str, value: Any) -> None: ... + + class UseCommonReleaseNotesTemplateFn(Protocol): + def __call__(self) -> None: ... + + +@pytest.fixture(scope="session") +def deps_files_4_example_monorepo() -> list[Path]: + return [ + # This file + Path(__file__).absolute(), + # because of imports + Path(tests.const.__file__).absolute(), + Path(tests.util.__file__).absolute(), + # because of the fixtures + Path(tests.conftest.__file__).absolute(), + Path(tests.fixtures.example_project.__file__).absolute(), + ] + + +@pytest.fixture(scope="session") +def build_spec_hash_4_example_monorepo( + get_md5_for_set_of_files: GetMd5ForSetOfFilesFn, + deps_files_4_example_monorepo: list[Path], +) -> str: + # Generates a hash of the build spec to set when to invalidate the cache + return get_md5_for_set_of_files(deps_files_4_example_monorepo) + + +@pytest.fixture(scope="session") +def cached_example_monorepo( + build_repo_or_copy_cache: BuildRepoOrCopyCacheFn, + monorepo_pkg1_dir: Path, + monorepo_pkg2_dir: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, + build_spec_hash_4_example_monorepo: str, + update_version_py_file: UpdateVersionPyFileFn, + update_pyproject_toml: UpdatePyprojectTomlFn, +) -> Path: + """ + Initializes the example monorepo project. DO NOT USE DIRECTLY + + Use the `init_example_monorepo` fixture instead. + """ + + def _build_project(cached_project_path: Path) -> Sequence[RepoActions]: + # purposefully a relative path + # example_dir = version_py_file.parent + gitignore_contents = dedent( + f""" + *.pyc + /{monorepo_pkg1_version_py_file} + /{monorepo_pkg2_version_py_file} + dist/ + """ + ).lstrip() + init_py_contents = dedent( + ''' + """An example package with a very informative docstring.""" + from ._version import __version__ + + def hello_world() -> None: + print("{pkg_name} Hello World") + ''' + ).lstrip() + + with temporary_working_directory(cached_project_path): + update_version_py_file( + version=EXAMPLE_PROJECT_VERSION, + version_file=monorepo_pkg1_version_py_file, + ) + update_version_py_file( + version=EXAMPLE_PROJECT_VERSION, + version_file=monorepo_pkg2_version_py_file, + ) + + file_2_contents: list[tuple[str | Path, str]] = [ + ( + monorepo_pkg1_version_py_file.parent / "__init__.py", + init_py_contents.format(pkg_name="Pkg 1:"), + ), + ( + monorepo_pkg2_version_py_file.parent / "__init__.py", + init_py_contents.format(pkg_name="Pkg 2:"), + ), + (".gitignore", gitignore_contents), + (monorepo_pkg1_pyproject_toml_file, EXAMPLE_PYPROJECT_TOML_CONTENT), + (monorepo_pkg2_pyproject_toml_file, EXAMPLE_PYPROJECT_TOML_CONTENT), + ] + + for file, contents in file_2_contents: + abs_filepath = cached_project_path.joinpath(file).resolve() + # make sure the parent directory exists + abs_filepath.parent.mkdir(parents=True, exist_ok=True) + # write file contents + abs_filepath.write_text(contents) + + config_updates: list[tuple[str, Any, Path]] = [ + ( + "tool.poetry.name", + "pkg-1", + cached_project_path / monorepo_pkg1_pyproject_toml_file, + ), + ( + "tool.poetry.name", + "pkg-2", + cached_project_path / monorepo_pkg2_pyproject_toml_file, + ), + ( + "tool.semantic_release.version_variables", + [ + f"{monorepo_pkg1_version_py_file.relative_to(monorepo_pkg1_dir)}:__version__" + ], + cached_project_path / monorepo_pkg1_pyproject_toml_file, + ), + ( + "tool.semantic_release.version_variables", + [ + f"{monorepo_pkg2_version_py_file.relative_to(monorepo_pkg2_dir)}:__version__" + ], + cached_project_path / monorepo_pkg2_pyproject_toml_file, + ), + ] + + for setting, value, toml_file in config_updates: + update_pyproject_toml( + setting=setting, + value=value, + toml_file=toml_file, + ) + + # This is a special build, we don't expose the Repo Actions to the caller + return [] + + # End of _build_project() + + return build_repo_or_copy_cache( + repo_name="example_monorepo", + build_spec_hash=build_spec_hash_4_example_monorepo, + build_repo_func=_build_project, + ) + + +@pytest.fixture +def init_example_monorepo( + example_project_dir: tests.fixtures.example_project.ExProjectDir, + cached_example_monorepo: Path, + change_to_ex_proj_dir: None, +) -> None: + """This fixture initializes the example project in the current test's project directory.""" + if not cached_example_monorepo.exists(): + raise RuntimeError( + f"Unable to find cached project files for {EXAMPLE_PROJECT_NAME}" + ) + + # Copy the cached project files into the current test's project directory + copy_dir_tree(cached_example_monorepo, example_project_dir) + + +@pytest.fixture +def monorepo_project_w_common_release_notes_template( + init_example_monorepo: None, + monorepo_use_common_release_notes_template: UseCommonReleaseNotesTemplateFn, +) -> None: + monorepo_use_common_release_notes_template() + + +@pytest.fixture(scope="session") +def monorepo_pkg1_name() -> str: + return "pkg1" + + +@pytest.fixture(scope="session") +def monorepo_pkg2_name() -> str: + return "pkg2" + + +@pytest.fixture(scope="session") +def monorepo_pkg_dir_pattern() -> str: + return str(Path("packages", "{package_name}")) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_dir( + monorepo_pkg1_name: str, + monorepo_pkg_dir_pattern: str, +) -> str: + return monorepo_pkg_dir_pattern.format(package_name=monorepo_pkg1_name) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_dir( + monorepo_pkg2_name: str, + monorepo_pkg_dir_pattern: str, +) -> str: + return monorepo_pkg_dir_pattern.format(package_name=monorepo_pkg2_name) + + +@pytest.fixture(scope="session") +def monorepo_pkg_version_py_file_pattern(monorepo_pkg_dir_pattern: str) -> str: + return str(Path(monorepo_pkg_dir_pattern, "src", "{package_name}", "_version.py")) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_version_py_file( + monorepo_pkg1_name: str, + monorepo_pkg_version_py_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_version_py_file_pattern.format(package_name=monorepo_pkg1_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_version_py_file( + monorepo_pkg2_name: str, + monorepo_pkg_version_py_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_version_py_file_pattern.format(package_name=monorepo_pkg2_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg_pyproject_toml_file_pattern( + monorepo_pkg_dir_pattern: str, + pyproject_toml_file: str, +) -> str: + return str(Path(monorepo_pkg_dir_pattern, pyproject_toml_file)) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_pyproject_toml_file( + monorepo_pkg1_name: str, + monorepo_pkg_pyproject_toml_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_pyproject_toml_file_pattern.format(package_name=monorepo_pkg1_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_pyproject_toml_file( + monorepo_pkg2_name: str, + monorepo_pkg_pyproject_toml_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_pyproject_toml_file_pattern.format(package_name=monorepo_pkg2_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg_dist_dir_pattern(monorepo_pkg_dir_pattern: str) -> str: + return str(Path(monorepo_pkg_dir_pattern, "dist")) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_dist_dir( + monorepo_pkg1_name: str, + monorepo_pkg_dist_dir_pattern: str, +) -> Path: + return Path(monorepo_pkg_dist_dir_pattern.format(package_name=monorepo_pkg1_name)) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_dist_dir( + monorepo_pkg2_name: str, + monorepo_pkg_dist_dir_pattern: str, +) -> Path: + return Path(monorepo_pkg_dist_dir_pattern.format(package_name=monorepo_pkg2_name)) + + +@pytest.fixture(scope="session") +def monorepo_pkg_changelog_md_file_pattern(monorepo_pkg_dir_pattern: str) -> str: + return str(Path(monorepo_pkg_dir_pattern, "CHANGELOG.md")) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_changelog_md_file( + monorepo_pkg1_name: str, + monorepo_pkg_changelog_md_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_changelog_md_file_pattern.format(package_name=monorepo_pkg1_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_changelog_md_file( + monorepo_pkg2_name: str, + monorepo_pkg_changelog_md_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_changelog_md_file_pattern.format(package_name=monorepo_pkg2_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg_changelog_rst_file_pattern(monorepo_pkg_dir_pattern: str) -> str: + return str(Path(monorepo_pkg_dir_pattern, "CHANGELOG.rst")) + + +@pytest.fixture(scope="session") +def monorepo_pkg1_changelog_rst_file( + monorepo_pkg1_name: str, + monorepo_pkg_changelog_rst_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_changelog_rst_file_pattern.format(package_name=monorepo_pkg1_name) + ) + + +@pytest.fixture(scope="session") +def monorepo_pkg2_changelog_rst_file( + monorepo_pkg2_name: str, + monorepo_pkg_changelog_rst_file_pattern: str, +) -> Path: + return Path( + monorepo_pkg_changelog_rst_file_pattern.format(package_name=monorepo_pkg2_name) + ) + + +# @pytest.fixture(scope="session") +# def get_wheel_file(dist_dir: Path) -> GetWheelFileFn: +# def _get_wheel_file(version_str: str) -> Path: +# return dist_dir / f"{EXAMPLE_PROJECT_NAME}-{version_str}-py3-none-any.whl" + +# return _get_wheel_file + + +@pytest.fixture +def example_monorepo_pkg_dir_pattern( + tmp_path: Path, + monorepo_pkg_dir_pattern: Path, +) -> str: + return str(tmp_path.resolve() / monorepo_pkg_dir_pattern) + + +@pytest.fixture +def example_monorepo_pkg1_dir( + monorepo_pkg1_name: str, + example_monorepo_pkg_dir_pattern: str, +) -> Path: + return Path( + example_monorepo_pkg_dir_pattern.format(package_name=monorepo_pkg1_name) + ) + + +@pytest.fixture +def example_monorepo_pkg2_dir( + monorepo_pkg2_name: str, + example_monorepo_pkg_dir_pattern: str, +) -> Path: + return Path( + example_monorepo_pkg_dir_pattern.format(package_name=monorepo_pkg2_name) + ) + + +@pytest.fixture +def monorepo_use_common_release_notes_template( + example_project_template_dir: Path, + changelog_template_dir: Path, + update_pyproject_toml: UpdatePyprojectTomlFn, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, +) -> UseCommonReleaseNotesTemplateFn: + config_setting_template_dir = "tool.semantic_release.changelog.template_dir" + + def _use_release_notes_template() -> None: + update_pyproject_toml( + setting=config_setting_template_dir, + value=str( + Path( + *( + "../" + for _ in list(Path(monorepo_pkg1_pyproject_toml_file).parents)[ + :-1 + ] + ), + changelog_template_dir, + ) + ), + toml_file=monorepo_pkg1_pyproject_toml_file, + ) + + update_pyproject_toml( + setting=config_setting_template_dir, + value=str( + Path( + *( + "../" + for _ in list(Path(monorepo_pkg2_pyproject_toml_file).parents)[ + :-1 + ] + ), + changelog_template_dir, + ) + ), + toml_file=monorepo_pkg2_pyproject_toml_file, + ) + + example_project_template_dir.mkdir(parents=True, exist_ok=True) + release_notes_j2 = example_project_template_dir / ".release_notes.md.j2" + release_notes_j2.write_text(EXAMPLE_RELEASE_NOTES_TEMPLATE) + + return _use_release_notes_template + + +# @pytest.fixture +# def example_pyproject_toml( +# example_project_dir: ExProjectDir, +# pyproject_toml_file: Path, +# ) -> Path: +# return example_project_dir / pyproject_toml_file + + +# @pytest.fixture +# def example_dist_dir( +# example_project_dir: ExProjectDir, +# dist_dir: Path, +# ) -> Path: +# return example_project_dir / dist_dir + + +# @pytest.fixture +# def example_project_wheel_file( +# example_dist_dir: Path, +# get_wheel_file: GetWheelFileFn, +# ) -> Path: +# return example_dist_dir / get_wheel_file(EXAMPLE_PROJECT_VERSION) + + +# Note this is just the path and the content may change +# @pytest.fixture +# def example_changelog_md( +# example_project_dir: ExProjectDir, +# changelog_md_file: Path, +# ) -> Path: +# return example_project_dir / changelog_md_file + + +# Note this is just the path and the content may change +# @pytest.fixture +# def example_changelog_rst( +# example_project_dir: ExProjectDir, +# changelog_rst_file: Path, +# ) -> Path: +# return example_project_dir / changelog_rst_file + + +# @pytest.fixture +# def example_project_template_dir( +# example_project_dir: ExProjectDir, +# changelog_template_dir: Path, +# ) -> Path: +# return example_project_dir / changelog_template_dir + + +@pytest.fixture(scope="session") +def update_pkg_pyproject_toml( + update_pyproject_toml: UpdatePyprojectTomlFn, + monorepo_pkg_pyproject_toml_file_pattern: str, +) -> UpdatePkgPyprojectTomlFn: + """Update the pyproject.toml file with the given content.""" + + def _update_pyproject_toml(pkg_name: str, setting: str, value: Any) -> None: + toml_file = Path( + monorepo_pkg_pyproject_toml_file_pattern.format(package_name=pkg_name) + ).resolve() + + if not toml_file.exists(): + raise FileNotFoundError( + f"pyproject.toml file for package {pkg_name} not found at {toml_file}" + ) + + update_pyproject_toml( + setting=setting, + value=value, + toml_file=toml_file, + ) + + return _update_pyproject_toml diff --git a/tests/fixtures/monorepos/git_monorepo.py b/tests/fixtures/monorepos/git_monorepo.py new file mode 100644 index 000000000..c88fbdb29 --- /dev/null +++ b/tests/fixtures/monorepos/git_monorepo.py @@ -0,0 +1,206 @@ +from __future__ import annotations + +from pathlib import Path +from shutil import rmtree +from typing import TYPE_CHECKING + +import pytest +from git import Repo + +import tests.conftest +import tests.const +import tests.fixtures.git_repo +import tests.util +from tests.const import ( + DEFAULT_BRANCH_NAME, + EXAMPLE_HVCS_DOMAIN, + EXAMPLE_PROJECT_NAME, +) +from tests.util import copy_dir_tree + +if TYPE_CHECKING: + from typing import Protocol, Sequence + + from git import Actor + + from semantic_release.hvcs import HvcsBase + + from tests.conftest import ( + BuildRepoOrCopyCacheFn, + GetMd5ForSetOfFilesFn, + RepoActions, + ) + from tests.fixtures.git_repo import ( + BuildRepoFn, + CommitConvention, + TomlSerializableTypes, + ) + + class BuildMonorepoFn(Protocol): + def __call__(self, dest_dir: Path | str) -> Path: ... + + +@pytest.fixture(scope="session") +def deps_files_4_example_git_monorepo( + deps_files_4_example_monorepo: list[Path], +) -> list[Path]: + return [ + *deps_files_4_example_monorepo, + # This file + Path(__file__).absolute(), + # because of imports + Path(tests.const.__file__).absolute(), + Path(tests.util.__file__).absolute(), + # because of the fixtures + Path(tests.conftest.__file__).absolute(), + Path(tests.fixtures.git_repo.__file__).absolute(), + ] + + +@pytest.fixture(scope="session") +def build_spec_hash_4_example_git_monorepo( + get_md5_for_set_of_files: GetMd5ForSetOfFilesFn, + deps_files_4_example_git_monorepo: list[Path], +) -> str: + # Generates a hash of the build spec to set when to invalidate the cache + return get_md5_for_set_of_files(deps_files_4_example_git_monorepo) + + +@pytest.fixture(scope="session") +def cached_example_git_monorepo( + build_repo_or_copy_cache: BuildRepoOrCopyCacheFn, + build_spec_hash_4_example_git_monorepo: str, + cached_example_monorepo: Path, + example_git_https_url: str, + commit_author: Actor, +) -> Path: + """ + Initializes an example monorepo project with git. DO NOT USE DIRECTLY. + + Use a `repo_*` fixture instead. This creates a default + base repository, all settings can be changed later through from the + example_project_git_repo fixture's return object and manual adjustment. + """ + + def _build_repo(cached_repo_path: Path) -> Sequence[RepoActions]: + if not cached_example_monorepo.exists(): + raise RuntimeError("Unable to find cached monorepo files") + + # make a copy of the example monorepo as a base + copy_dir_tree(cached_example_monorepo, cached_repo_path) + + # initialize git repo (open and close) + # NOTE: We don't want to hold the repo object open for the entire test session, + # the implementation on Windows holds some file descriptors open until close is called. + with Repo.init(cached_repo_path) as repo: + rmtree(str(Path(repo.git_dir, "hooks"))) + # Without this the global config may set it to "master", we want consistency + repo.git.branch("-M", DEFAULT_BRANCH_NAME) + with repo.config_writer("repository") as config: + config.set_value("user", "name", commit_author.name) + config.set_value("user", "email", commit_author.email) + config.set_value("commit", "gpgsign", False) + config.set_value("tag", "gpgsign", False) + + repo.create_remote(name="origin", url=example_git_https_url) + + # make sure all base files are in index to enable initial commit + repo.index.add(("*", ".gitignore")) + + # This is a special build, we don't expose the Repo Actions to the caller + return [] + + # End of _build_repo() + + return build_repo_or_copy_cache( + repo_name=cached_example_git_monorepo.__name__.split("_", maxsplit=1)[1], + build_spec_hash=build_spec_hash_4_example_git_monorepo, + build_repo_func=_build_repo, + ) + + +@pytest.fixture(scope="session") +def file_in_pkg_pattern(file_in_repo: str, monorepo_pkg_dir_pattern: str) -> str: + return str(Path(monorepo_pkg_dir_pattern) / file_in_repo) + + +@pytest.fixture(scope="session") +def file_in_monorepo_pkg1( + monorepo_pkg1_name: str, + file_in_pkg_pattern: str, +) -> Path: + return Path(file_in_pkg_pattern.format(pkg_name=monorepo_pkg1_name)) + + +@pytest.fixture(scope="session") +def file_in_monorepo_pkg2( + monorepo_pkg2_name: str, + file_in_pkg_pattern: str, +) -> Path: + return Path(file_in_pkg_pattern.format(pkg_name=monorepo_pkg2_name)) + + +@pytest.fixture(scope="session") +def build_base_monorepo( # noqa: C901 + cached_example_git_monorepo: Path, +) -> BuildMonorepoFn: + """ + This fixture is intended to simplify repo scenario building by initially + creating the repo but also configuring semantic_release in the pyproject.toml + for when the test executes semantic_release. It returns a function so that + derivative fixtures can call this fixture with individual parameters. + """ + + def _build_configured_base_monorepo(dest_dir: Path | str) -> Path: + if not cached_example_git_monorepo.exists(): + raise RuntimeError("Unable to find cached git project files!") + + # Copy the cached git project the dest directory + copy_dir_tree(cached_example_git_monorepo, dest_dir) + + return Path(dest_dir) + + return _build_configured_base_monorepo + + +@pytest.fixture(scope="session") +def configure_monorepo_package( # noqa: C901 + configure_base_repo: BuildRepoFn, +) -> BuildRepoFn: + """ + This fixture is intended to simplify repo scenario building by initially + creating the repo but also configuring semantic_release in the pyproject.toml + for when the test executes semantic_release. It returns a function so that + derivative fixtures can call this fixture with individual parameters. + """ + + def _configure( # noqa: C901 + dest_dir: Path | str, + commit_type: CommitConvention = "conventional", + hvcs_client_name: str = "github", + hvcs_domain: str = EXAMPLE_HVCS_DOMAIN, + tag_format_str: str | None = None, + extra_configs: dict[str, TomlSerializableTypes] | None = None, + mask_initial_release: bool = True, # Default as of v10 + package_name: str = EXAMPLE_PROJECT_NAME, + monorepo: bool = True, + ) -> tuple[Path, HvcsBase]: + if not monorepo: + raise ValueError("This fixture is only for monorepo packages!") + + if not Path(dest_dir).exists(): + raise RuntimeError(f"Destination directory {dest_dir} does not exist!") + + return configure_base_repo( + dest_dir=dest_dir, + commit_type=commit_type, + hvcs_client_name=hvcs_client_name, + hvcs_domain=hvcs_domain, + tag_format_str=tag_format_str, + extra_configs=extra_configs, + mask_initial_release=mask_initial_release, + package_name=package_name, + monorepo=monorepo, + ) + + return _configure diff --git a/tests/fixtures/monorepos/github_flow/__init__.py b/tests/fixtures/monorepos/github_flow/__init__.py new file mode 100644 index 000000000..3b951b378 --- /dev/null +++ b/tests/fixtures/monorepos/github_flow/__init__.py @@ -0,0 +1,2 @@ +from tests.fixtures.monorepos.github_flow.monorepo_w_default_release import * +from tests.fixtures.monorepos.github_flow.monorepo_w_release_channels import * diff --git a/tests/fixtures/monorepos/github_flow/monorepo_w_default_release.py b/tests/fixtures/monorepos/github_flow/monorepo_w_default_release.py new file mode 100644 index 000000000..f42789953 --- /dev/null +++ b/tests/fixtures/monorepos/github_flow/monorepo_w_default_release.py @@ -0,0 +1,954 @@ +from __future__ import annotations + +from datetime import timedelta +from itertools import count +from pathlib import Path +from textwrap import dedent +from typing import TYPE_CHECKING, cast + +import pytest + +from semantic_release.cli.config import ChangelogOutputFormat +from semantic_release.commit_parser.conventional.options_monorepo import ( + ConventionalCommitMonorepoParserOptions, +) +from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, +) +from semantic_release.version.version import Version + +import tests.conftest +import tests.const +import tests.util +from tests.const import ( + DEFAULT_BRANCH_NAME, + EXAMPLE_HVCS_DOMAIN, + INITIAL_COMMIT_MESSAGE, + RepoActionStep, +) + +if TYPE_CHECKING: + from typing import Sequence + + from semantic_release.commit_parser._base import CommitParser, ParserOptions + from semantic_release.commit_parser.token import ParseResult + + from tests.conftest import ( + GetCachedRepoDataFn, + GetMd5ForSetOfFilesFn, + GetStableDateNowFn, + ) + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildRepoOrCopyCacheFn, + BuildSpecificRepoFn, + BuiltRepoResult, + CommitConvention, + CommitSpec, + ConvertCommitSpecsToCommitDefsFn, + ConvertCommitSpecToCommitDefFn, + ExProjectGitRepoFn, + FormatGitHubSquashCommitMsgFn, + GetRepoDefinitionFn, + RepoActionChangeDirectory, + RepoActions, + RepoActionWriteChangelogsDestFile, + TomlSerializableTypes, + ) + + +@pytest.fixture(scope="session") +def deps_files_4_github_flow_monorepo_w_default_release_channel( + deps_files_4_example_git_monorepo: list[Path], +) -> list[Path]: + return [ + *deps_files_4_example_git_monorepo, + # This file + Path(__file__).absolute(), + # because of imports + Path(tests.const.__file__).absolute(), + Path(tests.util.__file__).absolute(), + # because of the fixtures + Path(tests.conftest.__file__).absolute(), + ] + + +@pytest.fixture(scope="session") +def build_spec_hash_4_github_flow_monorepo_w_default_release_channel( + get_md5_for_set_of_files: GetMd5ForSetOfFilesFn, + deps_files_4_github_flow_monorepo_w_default_release_channel: list[Path], +) -> str: + # Generates a hash of the build spec to set when to invalidate the cache + return get_md5_for_set_of_files( + deps_files_4_github_flow_monorepo_w_default_release_channel + ) + + +@pytest.fixture(scope="session") +def get_repo_definition_4_github_flow_monorepo_w_default_release_channel( + convert_commit_specs_to_commit_defs: ConvertCommitSpecsToCommitDefsFn, + convert_commit_spec_to_commit_def: ConvertCommitSpecToCommitDefFn, + format_squash_commit_msg_github: FormatGitHubSquashCommitMsgFn, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg2_changelog_rst_file: Path, + monorepo_pkg1_name: str, + monorepo_pkg2_name: str, + monorepo_pkg1_dir: Path, + monorepo_pkg2_dir: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + monorepo_pkg1_pyproject_toml_file: Path, + monorepo_pkg2_pyproject_toml_file: Path, + stable_now_date: GetStableDateNowFn, + default_tag_format_str: str, +) -> GetRepoDefinitionFn: + """ + Builds a Monorepo with the GitHub Flow branching strategy and a squash commit merging strategy + for a single release channel on the default branch. + + Implementation: + - The monorepo contains two packages, each with its own internal changelog but shared template. + - The repository implements the following git graph: + + ``` + * chore(release): pkg1@1.1.0 [skip ci] (tag: pkg1-v1.1.0, branch: main, HEAD -> main) + * feat(pkg1): file modified outside of pkg 1, identified by scope (#5) + | + | * feat(pkg1): file modified outside of pkg 1, identified by scope (branch: pkg1/feat/pr-4) + |/ + * chore(release): pkg2@1.1.1 [skip ci] (tag: pkg2-v1.1.1) + * fix(pkg2-cli): file modified outside of pkg 2, identified by scope (#4) + | + | * fix(pkg2-cli): file modified outside of pkg 2, identified by scope (branch: pkg2/fix/pr-3) + |/ + * chore(release): pkg2@1.1.0 [skip ci] (tag: pkg2-v1.1.0) + * feat: no pkg scope but file in pkg 2 directory (#3) # Squash merge of pkg2/feat/pr-2 + * chore(release): pkg1@1.0.1 [skip ci] (tag: pkg1-v1.0.1) + * fix: no pkg scope but file in pkg 1 directory (#2) # Squash merge of pkg1/fix/pr-1 + | + | * docs(cli): add cli documentation + | * test(cli): add cli tests + | * feat: no pkg scope but file in pkg 2 directory (branch: pkg2/feat/pr-2) + |/ + | * fix: no pkg scope but file in pkg 1 directory (branch: pkg1/fix/pr-1) + |/ + * chore(release): pkg2@1.0.0 [skip ci] (tag: pkg2-v1.0.0) # Initial release of pkg 2 + * chore(release): pkg1@1.0.0 [skip ci] (tag: pkg1-v1.0.0) # Initial release of pkg 1 + * Initial commit # Includes core functionality for both packages + ``` + """ + + def _get_repo_from_definition( + commit_type: CommitConvention, + hvcs_client_name: str = "github", + hvcs_domain: str = EXAMPLE_HVCS_DOMAIN, + tag_format_str: str | None = default_tag_format_str, + extra_configs: dict[str, TomlSerializableTypes] | None = None, + mask_initial_release: bool = True, + ignore_merge_commits: bool = True, + ) -> Sequence[RepoActions]: + stable_now_datetime = stable_now_date() + commit_timestamp_gen = ( + (stable_now_datetime + timedelta(seconds=i)).isoformat(timespec="seconds") + for i in count(step=1) + ) + pr_num_gen = (i for i in count(start=2, step=1)) + + pkg1_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg1_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg1_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + pkg2_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg2_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg2_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + change_to_pkg1_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg1_dir, + }, + } + + change_to_pkg2_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg2_dir, + }, + } + + change_to_example_project_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": "/", + }, + } + + if commit_type != "conventional": + raise ValueError(f"Unsupported commit type: {commit_type}") + + pkg1_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=True, + ignore_merge_commits=ignore_merge_commits, + scope_prefix=f"{monorepo_pkg1_name}-?", + path_filters=(".",), + ) + ) + + pkg2_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=pkg1_commit_parser.options.parse_squash_commits, + ignore_merge_commits=pkg1_commit_parser.options.ignore_merge_commits, + scope_prefix=f"{monorepo_pkg2_name}-?", + path_filters=(".",), + ) + ) + + common_configs: dict[str, TomlSerializableTypes] = { + # Set the default release branch + "tool.semantic_release.branches.main": { + "match": r"^(main|master)$", + "prerelease": False, + }, + "tool.semantic_release.allow_zero_version": False, + "tool.semantic_release.changelog.exclude_commit_patterns": [r"^chore"], + "tool.semantic_release.commit_parser": f"{commit_type}-monorepo", + "tool.semantic_release.commit_parser_options.parse_squash_commits": pkg1_commit_parser.options.parse_squash_commits, + "tool.semantic_release.commit_parser_options.ignore_merge_commits": pkg1_commit_parser.options.ignore_merge_commits, + } + + mr1_pkg1_fix_branch_name = f"{monorepo_pkg1_name}/fix/pr-1" + mr2_pkg2_feat_branch_name = f"{monorepo_pkg2_name}/feat/pr-2" + mr3_pkg2_fix_branch_name = f"{monorepo_pkg2_name}/fix/pr-3" + mr4_pkg1_feat_branch_name = f"{monorepo_pkg1_name}/feat/pr-4" + + pkg1_new_version = Version.parse( + "1.0.0", tag_format=f"{monorepo_pkg1_name}-{tag_format_str}" + ) + pkg2_new_version = Version.parse( + "1.0.0", tag_format=f"{monorepo_pkg2_name}-{tag_format_str}" + ) + + repo_construction_steps: list[RepoActions] = [ + { + "action": RepoActionStep.CREATE_MONOREPO, + "details": { + "commit_type": commit_type, + "hvcs_client_name": hvcs_client_name, + "hvcs_domain": hvcs_domain, + "post_actions": [ + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg1_dir, + "package_name": monorepo_pkg1_name, + "tag_format_str": pkg1_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg1_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg1_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + "tool.semantic_release.commit_parser_options.scope_prefix": pkg1_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg1_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg2_dir, + "package_name": monorepo_pkg2_name, + "tag_format_str": pkg2_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg2_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg2_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + "tool.semantic_release.commit_parser_options.scope_prefix": pkg2_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg2_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_c1_initial := "c1_initial_commit" + ), + "conventional": INITIAL_COMMIT_MESSAGE, + "emoji": INITIAL_COMMIT_MESSAGE, + "scipy": INITIAL_COMMIT_MESSAGE, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": bool( + commit_type == "emoji" + ), + }, + ], + commit_type, + # this parser does not matter since the commit is common + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + ], + }, + } + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_c1_initial], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [cid_c1_initial], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg1_fix_branch_commits: Sequence[CommitSpec] = [ + { + "cid": "pkg1-fix-1-squashed", + "conventional": "fix: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "emoji": ":bug: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "scipy": "MAINT: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "datetime": next(commit_timestamp_gen), + }, + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr1_pkg1_fix_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + } + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg1_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + **commit, + "include_in_changelog": False, + } + for commit in pkg1_fix_branch_commits + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # simulate separate work by another person at same time as the fix branch + pkg2_feat_branch_commits: Sequence[CommitSpec] = [ + { + "cid": "pkg2-feat-1-squashed", + "conventional": "feat: no pkg scope but file in pkg 2 directory", + "emoji": ":sparkles: no pkg scope but file in pkg 2 directory", + "scipy": "ENH: no pkg scope but file in pkg 2 directory", + "datetime": next(commit_timestamp_gen), + }, + { + "cid": "pkg2-feat-2-squashed", + "conventional": "test(cli): add cli tests", + "emoji": ":checkmark: add cli tests", + "scipy": "TST: add cli tests", + "datetime": next(commit_timestamp_gen), + }, + { + "cid": "pkg2-feat-3-squashed", + "conventional": "docs(cli): add cli documentation", + "emoji": ":memo: add cli documentation", + "scipy": "DOC: add cli documentation", + "datetime": next(commit_timestamp_gen), + }, + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr2_pkg2_feat_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg2_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + **commit, + "include_in_changelog": False, + } + for commit in pkg2_feat_branch_commits + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg1_new_version = Version.parse( + "1.0.1", tag_format=pkg1_new_version.tag_format + ) + + all_commit_types: list[CommitConvention] = ["conventional", "emoji", "scipy"] + fix_branch_pr_number = next(pr_num_gen) + fix_branch_squash_commit_spec: CommitSpec = { + "cid": "mr1-pkg1-fix", + **{ # type: ignore[typeddict-item] + cmt_type: format_squash_commit_msg_github( + # Use the primary commit message as the PR title + pr_title=pkg1_fix_branch_commits[0][cmt_type], + pr_number=fix_branch_pr_number, + squashed_commits=[ + cmt[commit_type] for cmt in pkg1_fix_branch_commits[1:] + ], + ) + for cmt_type in all_commit_types + }, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + } + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": {"branch": DEFAULT_BRANCH_NAME}, + }, + { + "action": RepoActionStep.GIT_SQUASH, + "details": { + "branch": mr1_pkg1_fix_branch_name, + "strategy_option": "theirs", + "commit_def": convert_commit_spec_to_commit_def( + fix_branch_squash_commit_spec, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + "config_file": monorepo_pkg1_pyproject_toml_file, + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [ + f'{fix_branch_squash_commit_spec["cid"]}-{index + 1}' + for index in range(len(pkg1_fix_branch_commits)) + ], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + feat_branch_pr_number = next(pr_num_gen) + feat_branch_squash_commit_spec: CommitSpec = { + "cid": "mr2-pkg2-feat", + **{ # type: ignore[typeddict-item] + cmt_type: format_squash_commit_msg_github( + # Use the primary commit message as the PR title + pr_title=pkg2_feat_branch_commits[0][cmt_type], + pr_number=feat_branch_pr_number, + squashed_commits=[ + cmt[commit_type] for cmt in pkg2_feat_branch_commits[1:] + ], + ) + for cmt_type in all_commit_types + }, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + } + + pkg2_new_version = Version.parse( + "1.1.0", tag_format=pkg2_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_SQUASH, + "details": { + "branch": mr2_pkg2_feat_branch_name, + "strategy_option": "theirs", + "commit_def": convert_commit_spec_to_commit_def( + feat_branch_squash_commit_spec, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + "config_file": monorepo_pkg2_pyproject_toml_file, + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [ + f'{feat_branch_squash_commit_spec["cid"]}-{index + 1}' + for index in range( + len(pkg2_feat_branch_commits) + ) + ], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg2_fix_branch_commits: Sequence[CommitSpec] = [ + { + "cid": "pkg2-fix-1-squashed", + "conventional": "fix(pkg2-cli): file modified outside of pkg 2, identified by scope\n\nResolves: #123\n", + "emoji": ":bug: (pkg2-cli) file modified outside of pkg 2, identified by scope\n\nResolves: #123\n", + "scipy": "MAINT:pkg2-cli: file modified outside of pkg 2, identified by scope\n\nResolves: #123\n", + "datetime": next(commit_timestamp_gen), + }, + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr3_pkg2_fix_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + } + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + **commit, + "include_in_changelog": False, + } + for commit in pkg2_fix_branch_commits + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + }, + }, + ] + ) + + pkg2_new_version = Version.parse( + "1.1.1", tag_format=pkg2_new_version.tag_format + ) + + fix_branch_pr_number = next(pr_num_gen) + fix_branch_squash_commit_spec = { + "cid": "mr3-pkg2-fix", + **{ # type: ignore[typeddict-item] + cmt_type: format_squash_commit_msg_github( + # Use the primary commit message as the PR title + pr_title=pkg2_fix_branch_commits[0][cmt_type], + pr_number=fix_branch_pr_number, + squashed_commits=[ + cmt[commit_type] for cmt in pkg2_fix_branch_commits[1:] + ], + ) + for cmt_type in all_commit_types + }, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + } + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": {"branch": DEFAULT_BRANCH_NAME}, + }, + { + "action": RepoActionStep.GIT_SQUASH, + "details": { + "branch": mr3_pkg2_fix_branch_name, + "strategy_option": "theirs", + "commit_def": convert_commit_spec_to_commit_def( + fix_branch_squash_commit_spec, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + "config_file": monorepo_pkg2_pyproject_toml_file, + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [ + f'{fix_branch_squash_commit_spec["cid"]}-{index + 1}' + for index in range(len(pkg2_fix_branch_commits)) + ], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg1_feat_branch_commits: Sequence[CommitSpec] = [ + { + "cid": "pkg1-feat-1-squashed", + "conventional": "feat(pkg1): file modified outside of pkg 1, identified by scope", + "emoji": ":sparkles: (pkg1) file modified outside of pkg 1, identified by scope", + "scipy": "ENH:pkg1: file modified outside of pkg 1, identified by scope", + "datetime": next(commit_timestamp_gen), + } + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr4_pkg1_feat_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + **commit, + "include_in_changelog": False, + } + for commit in pkg1_feat_branch_commits + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + ] + ) + + feat_branch_pr_number = next(pr_num_gen) + feat_branch_squash_commit_spec = { + "cid": "mr4-pkg1-feat", + **{ # type: ignore[typeddict-item] + cmt_type: format_squash_commit_msg_github( + # Use the primary commit message as the PR title + pr_title=pkg1_feat_branch_commits[0][cmt_type], + pr_number=feat_branch_pr_number, + squashed_commits=[ + cmt[commit_type] for cmt in pkg1_feat_branch_commits[1:] + ], + ) + for cmt_type in all_commit_types + }, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + } + + pkg1_new_version = Version.parse( + "1.1.0", tag_format=pkg1_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": {"branch": DEFAULT_BRANCH_NAME}, + }, + { + "action": RepoActionStep.GIT_SQUASH, + "details": { + "branch": mr4_pkg1_feat_branch_name, + "strategy_option": "theirs", + "commit_def": convert_commit_spec_to_commit_def( + feat_branch_squash_commit_spec, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + "config_file": monorepo_pkg1_pyproject_toml_file, + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [ + f'{feat_branch_squash_commit_spec["cid"]}-{index + 1}' + for index in range( + len(pkg1_feat_branch_commits) + ) + ], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + return repo_construction_steps + + return _get_repo_from_definition + + +@pytest.fixture(scope="session") +def build_monorepo_w_github_flow_w_default_release_channel( + build_repo_from_definition: BuildRepoFromDefinitionFn, + get_repo_definition_4_github_flow_monorepo_w_default_release_channel: GetRepoDefinitionFn, + get_cached_repo_data: GetCachedRepoDataFn, + build_repo_or_copy_cache: BuildRepoOrCopyCacheFn, + build_spec_hash_4_github_flow_monorepo_w_default_release_channel: str, +) -> BuildSpecificRepoFn: + def _build_specific_repo_type( + repo_name: str, commit_type: CommitConvention, dest_dir: Path + ) -> Sequence[RepoActions]: + def _build_repo(cached_repo_path: Path) -> Sequence[RepoActions]: + repo_construction_steps = ( + get_repo_definition_4_github_flow_monorepo_w_default_release_channel( + commit_type=commit_type, + ) + ) + return build_repo_from_definition(cached_repo_path, repo_construction_steps) + + build_repo_or_copy_cache( + repo_name=repo_name, + build_spec_hash=build_spec_hash_4_github_flow_monorepo_w_default_release_channel, + build_repo_func=_build_repo, + dest_dir=dest_dir, + ) + + if not (cached_repo_data := get_cached_repo_data(proj_dirname=repo_name)): + raise ValueError("Failed to retrieve repo data from cache") + + return cached_repo_data["build_definition"] + + return _build_specific_repo_type + + +# --------------------------------------------------------------------------- # +# Test-level fixtures that will cache the built directory & set up test case # +# --------------------------------------------------------------------------- # + + +@pytest.fixture +def monorepo_w_github_flow_w_default_release_channel_conventional_commits( + build_monorepo_w_github_flow_w_default_release_channel: BuildSpecificRepoFn, + example_project_git_repo: ExProjectGitRepoFn, + example_project_dir: ExProjectDir, + change_to_ex_proj_dir: None, +) -> BuiltRepoResult: + repo_name = ( + monorepo_w_github_flow_w_default_release_channel_conventional_commits.__name__ + ) + commit_type: CommitConvention = repo_name.split("_")[-2] # type: ignore[assignment] + + return { + "definition": build_monorepo_w_github_flow_w_default_release_channel( + repo_name=repo_name, + commit_type=commit_type, + dest_dir=example_project_dir, + ), + "repo": example_project_git_repo(), + } diff --git a/tests/fixtures/monorepos/github_flow/monorepo_w_release_channels.py b/tests/fixtures/monorepos/github_flow/monorepo_w_release_channels.py new file mode 100644 index 000000000..1e25cdb4f --- /dev/null +++ b/tests/fixtures/monorepos/github_flow/monorepo_w_release_channels.py @@ -0,0 +1,888 @@ +from __future__ import annotations + +from datetime import timedelta +from itertools import count +from pathlib import Path +from textwrap import dedent +from typing import TYPE_CHECKING, cast + +import pytest + +from semantic_release.cli.config import ChangelogOutputFormat +from semantic_release.commit_parser.conventional.options_monorepo import ( + ConventionalCommitMonorepoParserOptions, +) +from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, +) +from semantic_release.version.version import Version + +import tests.conftest +import tests.const +import tests.util +from tests.const import ( + DEFAULT_BRANCH_NAME, + EXAMPLE_HVCS_DOMAIN, + INITIAL_COMMIT_MESSAGE, + RepoActionStep, +) + +if TYPE_CHECKING: + from typing import Sequence + + from semantic_release.commit_parser._base import CommitParser, ParserOptions + from semantic_release.commit_parser.token import ParseResult + + from tests.conftest import ( + GetCachedRepoDataFn, + GetMd5ForSetOfFilesFn, + GetStableDateNowFn, + ) + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildRepoOrCopyCacheFn, + BuildSpecificRepoFn, + BuiltRepoResult, + CommitConvention, + ConvertCommitSpecsToCommitDefsFn, + ConvertCommitSpecToCommitDefFn, + ExProjectGitRepoFn, + FormatGitHubMergeCommitMsgFn, + GetRepoDefinitionFn, + RepoActionChangeDirectory, + RepoActionGitMerge, + RepoActionGitMergeDetails, + RepoActions, + RepoActionWriteChangelogsDestFile, + TomlSerializableTypes, + ) + + +@pytest.fixture(scope="session") +def deps_files_4_github_flow_monorepo_w_feature_release_channel( + deps_files_4_example_git_monorepo: list[Path], +) -> list[Path]: + return [ + *deps_files_4_example_git_monorepo, + # This file + Path(__file__).absolute(), + # because of imports + Path(tests.const.__file__).absolute(), + Path(tests.util.__file__).absolute(), + # because of the fixtures + Path(tests.conftest.__file__).absolute(), + ] + + +@pytest.fixture(scope="session") +def build_spec_hash_4_github_flow_monorepo_w_feature_release_channel( + get_md5_for_set_of_files: GetMd5ForSetOfFilesFn, + deps_files_4_github_flow_monorepo_w_feature_release_channel: list[Path], +) -> str: + # Generates a hash of the build spec to set when to invalidate the cache + return get_md5_for_set_of_files( + deps_files_4_github_flow_monorepo_w_feature_release_channel + ) + + +@pytest.fixture(scope="session") +def get_repo_definition_4_github_flow_monorepo_w_feature_release_channel( + convert_commit_specs_to_commit_defs: ConvertCommitSpecsToCommitDefsFn, + convert_commit_spec_to_commit_def: ConvertCommitSpecToCommitDefFn, + format_merge_commit_msg_github: FormatGitHubMergeCommitMsgFn, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg2_changelog_rst_file: Path, + monorepo_pkg1_name: str, + monorepo_pkg2_name: str, + monorepo_pkg1_dir: Path, + monorepo_pkg2_dir: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + stable_now_date: GetStableDateNowFn, + default_tag_format_str: str, +) -> GetRepoDefinitionFn: + """ + Builds a Monorepo with the GitHub Flow branching strategy and a merge commit merging strategy + for alpha feature releases and official releases on the default branch. + + Implementation: + - The monorepo contains two packages, each with its own internal changelog but shared template. + - The repository implements the following git graph: + + ``` + * chore(release): pkg2@1.1.0 [skip ci] (tag: pkg2-v1.1.0) + * Merge pull request #3 from 'pkg2/feat/pr-2' + |\ + | * chore(release): pkg2@1.1.0-alpha.2 [skip ci] (tag: pkg2-v1.1.0-alpha.2, branch: pkg2/feat/pr-2) + | * fix(pkg2-cli): file modified outside of pkg 2, identified by scope + | * chore(release): pkg2@1.1.0-alpha.1 [skip ci] (tag: pkg2-v1.1.0-alpha.1) + | * docs: add cli documentation + | * test: add cli tests + | * feat: no pkg scope but file in pkg 2 directory + |/ + * chore(release): pkg1@1.0.1 [skip ci] (tag: pkg1-v1.0.1) + * Merge pull request #2 from 'pkg1/fix/pr-1' + |\ + | * chore(release): pkg1@1.0.1-alpha.2 [skip ci] (tag: pkg1-v1.0.1-alpha.2, branch: pkg1/fix/pr-1) + | * fix(pkg1-cli): file modified outside of pkg 1, identified by scope + | * chore(release): pkg1@1.0.1-alpha.1 [skip ci] (tag: pkg1-v1.0.1-alpha.1) + | * fix: no pkg scope but file in pkg 1 directory + |/ + * chore(release): pkg2@1.0.0 [skip ci] (tag: pkg2-v1.0.0) # Initial release of pkg 2 + * chore(release): pkg1@1.0.0 [skip ci] (tag: pkg1-v1.0.0) # Initial release of pkg 1 + * Initial commit # Includes core functionality for both packages + ``` + """ + + def _get_repo_from_definition( + commit_type: CommitConvention, + hvcs_client_name: str = "github", + hvcs_domain: str = EXAMPLE_HVCS_DOMAIN, + tag_format_str: str | None = default_tag_format_str, + extra_configs: dict[str, TomlSerializableTypes] | None = None, + mask_initial_release: bool = True, + ignore_merge_commits: bool = True, + ) -> Sequence[RepoActions]: + stable_now_datetime = stable_now_date() + commit_timestamp_gen = ( + (stable_now_datetime + timedelta(seconds=i)).isoformat(timespec="seconds") + for i in count(step=1) + ) + pr_num_gen = (i for i in count(start=2, step=1)) + + pkg1_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg1_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg1_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + pkg2_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg2_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg2_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + change_to_pkg1_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg1_dir, + }, + } + + change_to_pkg2_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg2_dir, + }, + } + + change_to_example_project_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": "/", + }, + } + + if commit_type != "conventional": + raise ValueError(f"Unsupported commit type: {commit_type}") + + pkg1_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=True, + ignore_merge_commits=ignore_merge_commits, + scope_prefix=f"{monorepo_pkg1_name}-?", + path_filters=(".",), + ) + ) + + pkg2_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=pkg1_commit_parser.options.parse_squash_commits, + ignore_merge_commits=pkg1_commit_parser.options.ignore_merge_commits, + scope_prefix=f"{monorepo_pkg2_name}-?", + path_filters=(".",), + ) + ) + + common_configs: dict[str, TomlSerializableTypes] = { + # Set the default release branch + "tool.semantic_release.branches.main": { + "match": r"^(main|master)$", + "prerelease": False, + }, + "tool.semantic_release.allow_zero_version": False, + "tool.semantic_release.changelog.exclude_commit_patterns": [r"^chore"], + "tool.semantic_release.commit_parser": f"{commit_type}-monorepo", + "tool.semantic_release.commit_parser_options.parse_squash_commits": pkg1_commit_parser.options.parse_squash_commits, + "tool.semantic_release.commit_parser_options.ignore_merge_commits": pkg1_commit_parser.options.ignore_merge_commits, + } + + mr1_pkg1_fix_branch_name = f"{monorepo_pkg1_name}/fix/pr-1" + mr2_pkg2_feat_branch_name = f"{monorepo_pkg2_name}/feat/pr-2" + + pkg1_new_version = Version.parse( + "1.0.0", tag_format=f"{monorepo_pkg1_name}-{tag_format_str}" + ) + pkg2_new_version = Version.parse( + "1.0.0", tag_format=f"{monorepo_pkg2_name}-{tag_format_str}" + ) + + repo_construction_steps: list[RepoActions] = [ + { + "action": RepoActionStep.CREATE_MONOREPO, + "details": { + "commit_type": commit_type, + "hvcs_client_name": hvcs_client_name, + "hvcs_domain": hvcs_domain, + "post_actions": [ + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg1_dir, + "package_name": monorepo_pkg1_name, + "tag_format_str": pkg1_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg1_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg1_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + # package branches "feat/" & "fix/" has prerelease suffix of "alpha" + "tool.semantic_release.branches.alpha-release": { + "match": rf"^{monorepo_pkg1_name}/(feat|fix)/.+", + "prerelease": True, + "prerelease_token": "alpha", + }, + "tool.semantic_release.commit_parser_options.scope_prefix": pkg1_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg1_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg2_dir, + "package_name": monorepo_pkg2_name, + "tag_format_str": pkg2_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg2_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg2_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + # package branches "feat/" & "fix/" has prerelease suffix of "alpha" + "tool.semantic_release.branches.alpha-release": { + "match": rf"^{monorepo_pkg2_name}/(feat|fix)/.+", + "prerelease": True, + "prerelease_token": "alpha", + }, + "tool.semantic_release.commit_parser_options.scope_prefix": pkg2_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg2_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_c1_initial := "c1_initial_commit" + ), + "conventional": INITIAL_COMMIT_MESSAGE, + "emoji": INITIAL_COMMIT_MESSAGE, + "scipy": INITIAL_COMMIT_MESSAGE, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": bool( + commit_type == "emoji" + ), + }, + ], + commit_type, + # this parser does not matter since the commit is common + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + ], + }, + } + ] + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_c1_initial], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [cid_c1_initial], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Make a fix in package 1 and release it as an alpha release + pkg1_new_version = Version.parse( + "1.0.1-alpha.1", tag_format=pkg1_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr1_pkg1_fix_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + } + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg1_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_pkg1_fib1_c1_fix + := "pkg1_fix_branch_1_c1_fix" + ), + "conventional": "fix: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "emoji": ":bug: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "scipy": "MAINT: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_pkg1_fib1_c1_fix], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Update the fix in package 1 and release another alpha release + pkg1_new_version = Version.parse( + "1.0.1-alpha.2", tag_format=pkg1_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_pkg1_fib1_c2_fix + := "pkg1_fix_branch_1_c2_fix" + ), + "conventional": "fix(pkg1-cli): file modified outside of pkg 1, identified by scope\n\n", + "emoji": ":bug: (pkg1-cli) file modified outside of pkg 1, identified by scope", + "scipy": "MAINT:pkg1-cli: file modified outside of pkg 1, identified by scope", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_pkg1_fib1_c2_fix], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Merge the fix branch into the default branch and formally release it + pkg1_new_version = Version.parse( + "1.0.1", tag_format=pkg1_new_version.tag_format + ) + + merge_def_type_placeholder: RepoActionGitMerge[RepoActionGitMergeDetails] = { + "action": RepoActionStep.GIT_MERGE, + "details": { + "branch_name": mr1_pkg1_fix_branch_name, + "fast_forward": False, + "commit_def": convert_commit_spec_to_commit_def( + { + "cid": (cid_pkg1_fib1_merge := "pkg1_fix_branch_1_merge"), + "conventional": ( + merge_msg := format_merge_commit_msg_github( + pr_number=next(pr_num_gen), + branch_name=mr1_pkg1_fix_branch_name, + ) + ), + "emoji": merge_msg, + "scipy": merge_msg, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": not ignore_merge_commits, + }, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + } + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": {"branch": DEFAULT_BRANCH_NAME}, + }, + merge_def_type_placeholder, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_pkg1_fib1_merge], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Make a feature branch and release it as an alpha release + pkg2_new_version = Version.parse( + "1.1.0-alpha.1", tag_format=pkg2_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": { + "create_branch": { + "name": mr2_pkg2_feat_branch_name, + "start_branch": DEFAULT_BRANCH_NAME, + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg2_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_pkg2_feb1_c1_feat + := "pkg2_feat_branch_1_c1_feat" + ), + "conventional": "feat: no pkg scope but file in pkg 2 directory\n", + "emoji": ":sparkles: no pkg scope but file in pkg 2 directory", + "scipy": "ENH: no pkg scope but file in pkg 2 directory", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + { + "cid": ( + cid_pkg2_feb1_c2_test + := "pkg2_feat_branch_1_c2_test" + ), + "conventional": "test: add cli tests", + "emoji": ":checkmark: add cli tests", + "scipy": "TST: add cli tests", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + { + "cid": ( + cid_pkg2_feb1_c3_docs + := "pkg2_feat_branch_1_c3_docs" + ), + "conventional": "docs: add cli documentation", + "emoji": ":memo: add cli documentation", + "scipy": "DOC: add cli documentation", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [ + cid_pkg2_feb1_c1_feat, + cid_pkg2_feb1_c2_test, + cid_pkg2_feb1_c3_docs, + ], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Update the feat with a fix in package 2 and release another alpha release + pkg2_new_version = Version.parse( + "1.1.0-alpha.2", tag_format=pkg2_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_pkg2_feb1_c4_fix + := "pkg2_feat_branch_1_c4_fix" + ), + "conventional": "fix(pkg2-cli): file modified outside of pkg 2, identified by scope", + "emoji": ":bug: (pkg2-cli) file modified outside of pkg 2, identified by scope", + "scipy": "MAINT:pkg2-cli: file modified outside of pkg 2, identified by scope", + "datetime": next(commit_timestamp_gen), + "include_in_changelog": True, + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [cid_pkg2_feb1_c4_fix], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + # Merge the feat branch into the default branch and formally release a package 2 + pkg2_new_version = Version.parse( + "1.1.0", tag_format=pkg2_new_version.tag_format + ) + + merge_def_type_placeholder = { + "action": RepoActionStep.GIT_MERGE, + "details": { + "branch_name": mr2_pkg2_feat_branch_name, + "fast_forward": False, + "commit_def": convert_commit_spec_to_commit_def( + { + "cid": (cid_pkg2_feb1_merge := "pkg2_feat_branch_1_merge"), + "conventional": ( + merge_msg := format_merge_commit_msg_github( + pr_number=next(pr_num_gen), + branch_name=mr2_pkg2_feat_branch_name, + ) + ), + "emoji": merge_msg, + "scipy": merge_msg, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": not ignore_merge_commits, + }, + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + }, + } + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.GIT_CHECKOUT, + "details": {"branch": DEFAULT_BRANCH_NAME}, + }, + merge_def_type_placeholder, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [cid_pkg2_feb1_merge], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + return repo_construction_steps + + return _get_repo_from_definition + + +@pytest.fixture(scope="session") +def build_monorepo_w_github_flow_w_feature_release_channel( + build_repo_from_definition: BuildRepoFromDefinitionFn, + get_repo_definition_4_github_flow_monorepo_w_feature_release_channel: GetRepoDefinitionFn, + get_cached_repo_data: GetCachedRepoDataFn, + build_repo_or_copy_cache: BuildRepoOrCopyCacheFn, + build_spec_hash_4_github_flow_monorepo_w_feature_release_channel: str, +) -> BuildSpecificRepoFn: + def _build_specific_repo_type( + repo_name: str, commit_type: CommitConvention, dest_dir: Path + ) -> Sequence[RepoActions]: + def _build_repo(cached_repo_path: Path) -> Sequence[RepoActions]: + repo_construction_steps = ( + get_repo_definition_4_github_flow_monorepo_w_feature_release_channel( + commit_type=commit_type, + ) + ) + return build_repo_from_definition(cached_repo_path, repo_construction_steps) + + build_repo_or_copy_cache( + repo_name=repo_name, + build_spec_hash=build_spec_hash_4_github_flow_monorepo_w_feature_release_channel, + build_repo_func=_build_repo, + dest_dir=dest_dir, + ) + + if not (cached_repo_data := get_cached_repo_data(proj_dirname=repo_name)): + raise ValueError("Failed to retrieve repo data from cache") + + return cached_repo_data["build_definition"] + + return _build_specific_repo_type + + +# --------------------------------------------------------------------------- # +# Test-level fixtures that will cache the built directory & set up test case # +# --------------------------------------------------------------------------- # + + +@pytest.fixture +def monorepo_w_github_flow_w_feature_release_channel_conventional_commits( + build_monorepo_w_github_flow_w_feature_release_channel: BuildSpecificRepoFn, + example_project_git_repo: ExProjectGitRepoFn, + example_project_dir: ExProjectDir, + change_to_ex_proj_dir: None, +) -> BuiltRepoResult: + repo_name = ( + monorepo_w_github_flow_w_feature_release_channel_conventional_commits.__name__ + ) + commit_type: CommitConvention = repo_name.split("_")[-2] # type: ignore[assignment] + + return { + "definition": build_monorepo_w_github_flow_w_feature_release_channel( + repo_name=repo_name, + commit_type=commit_type, + dest_dir=example_project_dir, + ), + "repo": example_project_git_repo(), + } diff --git a/tests/fixtures/monorepos/trunk_based_dev/__init__.py b/tests/fixtures/monorepos/trunk_based_dev/__init__.py new file mode 100644 index 000000000..c84f42bc8 --- /dev/null +++ b/tests/fixtures/monorepos/trunk_based_dev/__init__.py @@ -0,0 +1 @@ +from tests.fixtures.monorepos.trunk_based_dev.monorepo_w_tags import * diff --git a/tests/fixtures/monorepos/trunk_based_dev/monorepo_w_tags.py b/tests/fixtures/monorepos/trunk_based_dev/monorepo_w_tags.py new file mode 100644 index 000000000..9687e088f --- /dev/null +++ b/tests/fixtures/monorepos/trunk_based_dev/monorepo_w_tags.py @@ -0,0 +1,623 @@ +from __future__ import annotations + +from datetime import timedelta +from itertools import count +from pathlib import Path +from textwrap import dedent +from typing import TYPE_CHECKING, cast + +import pytest + +from semantic_release.cli.config import ChangelogOutputFormat +from semantic_release.commit_parser.conventional.options_monorepo import ( + ConventionalCommitMonorepoParserOptions, +) +from semantic_release.commit_parser.conventional.parser_monorepo import ( + ConventionalCommitMonorepoParser, +) +from semantic_release.version.version import Version + +import tests.conftest +import tests.const +import tests.util +from tests.const import ( + EXAMPLE_HVCS_DOMAIN, + INITIAL_COMMIT_MESSAGE, + RepoActionStep, +) + +if TYPE_CHECKING: + from typing import Sequence + + from semantic_release.commit_parser._base import CommitParser, ParserOptions + from semantic_release.commit_parser.token import ParseResult + + from tests.conftest import ( + GetCachedRepoDataFn, + GetMd5ForSetOfFilesFn, + GetStableDateNowFn, + ) + from tests.fixtures.example_project import ExProjectDir + from tests.fixtures.git_repo import ( + BuildRepoFromDefinitionFn, + BuildRepoOrCopyCacheFn, + BuildSpecificRepoFn, + BuiltRepoResult, + CommitConvention, + ConvertCommitSpecsToCommitDefsFn, + ExProjectGitRepoFn, + GetRepoDefinitionFn, + RepoActionChangeDirectory, + RepoActions, + RepoActionWriteChangelogsDestFile, + TomlSerializableTypes, + ) + + +@pytest.fixture(scope="session") +def deps_files_4_trunk_only_monorepo_w_tags( + deps_files_4_example_git_monorepo: list[Path], +) -> list[Path]: + return [ + *deps_files_4_example_git_monorepo, + # This file + Path(__file__).absolute(), + # because of imports + Path(tests.const.__file__).absolute(), + Path(tests.util.__file__).absolute(), + # because of the fixtures + Path(tests.conftest.__file__).absolute(), + ] + + +@pytest.fixture(scope="session") +def build_spec_hash_4_trunk_only_monorepo_w_tags( + get_md5_for_set_of_files: GetMd5ForSetOfFilesFn, + deps_files_4_trunk_only_monorepo_w_tags: list[Path], +) -> str: + # Generates a hash of the build spec to set when to invalidate the cache + return get_md5_for_set_of_files(deps_files_4_trunk_only_monorepo_w_tags) + + +@pytest.fixture(scope="session") +def get_repo_definition_4_trunk_only_monorepo_w_tags( + convert_commit_specs_to_commit_defs: ConvertCommitSpecsToCommitDefsFn, + monorepo_pkg1_changelog_md_file: Path, + monorepo_pkg1_changelog_rst_file: Path, + monorepo_pkg2_changelog_md_file: Path, + monorepo_pkg2_changelog_rst_file: Path, + monorepo_pkg1_name: str, + monorepo_pkg2_name: str, + monorepo_pkg1_dir: Path, + monorepo_pkg2_dir: Path, + monorepo_pkg1_version_py_file: Path, + monorepo_pkg2_version_py_file: Path, + stable_now_date: GetStableDateNowFn, + default_tag_format_str: str, +) -> GetRepoDefinitionFn: + """ + Builds a Monorepo with trunk-based development only with official releases. + + Implementation: + - The monorepo contains two packages, each with its own internal changelog but shared template. + - The repository implements the following git graph: + + ``` + * chore(release): pkg1@0.1.0 [skip ci] (tag: pkg1-v0.1.0, branch: main) + * feat(pkg1): file modified outside of pkg 1, identified by scope + * chore(release): pkg2@0.1.1 [skip ci] (tag: pkg2-v0.1.1) + * fix(pkg2-cli): file modified outside of pkg 2, identified by scope + * chore(release): pkg2@0.1.0 [skip ci] (tag: pkg2-v0.1.0) + * docs(pkg2-cli): common docs modified outside of pkg 2, identified by scope + * test: no pkg scope but add tests to package 2 directory + * feat: no pkg scope but file in pkg 2 directory + * chore(release): pkg1@0.0.1 [skip ci] (tag: pkg1-v0.0.1) + * fix: no pkg scope but file in pkg 1 directory + * Initial commit # Includes core functionality for both packages + ``` + """ + + def _get_repo_from_definition( + commit_type: CommitConvention, + hvcs_client_name: str = "github", + hvcs_domain: str = EXAMPLE_HVCS_DOMAIN, + tag_format_str: str | None = default_tag_format_str, + extra_configs: dict[str, TomlSerializableTypes] | None = None, + mask_initial_release: bool = True, + ignore_merge_commits: bool = True, + ) -> Sequence[RepoActions]: + stable_now_datetime = stable_now_date() + commit_timestamp_gen = ( + (stable_now_datetime + timedelta(seconds=i)).isoformat(timespec="seconds") + for i in count(step=1) + ) + + pkg1_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg1_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg1_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + pkg2_changelog_file_definitions: Sequence[RepoActionWriteChangelogsDestFile] = [ + { + "path": monorepo_pkg2_changelog_md_file, + "format": ChangelogOutputFormat.MARKDOWN, + "mask_initial_release": True, + }, + { + "path": monorepo_pkg2_changelog_rst_file, + "format": ChangelogOutputFormat.RESTRUCTURED_TEXT, + "mask_initial_release": True, + }, + ] + + change_to_pkg1_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg1_dir, + }, + } + + change_to_pkg2_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": monorepo_pkg2_dir, + }, + } + + change_to_example_project_dir: RepoActionChangeDirectory = { + "action": RepoActionStep.CHANGE_DIRECTORY, + "details": { + "directory": "/", + }, + } + + if commit_type != "conventional": + raise ValueError(f"Unsupported commit type: {commit_type}") + + pkg1_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=True, + ignore_merge_commits=ignore_merge_commits, + scope_prefix=f"{monorepo_pkg1_name}-?", + path_filters=(".",), + ) + ) + + pkg2_commit_parser = ConventionalCommitMonorepoParser( + options=ConventionalCommitMonorepoParserOptions( + parse_squash_commits=pkg1_commit_parser.options.parse_squash_commits, + ignore_merge_commits=pkg1_commit_parser.options.ignore_merge_commits, + scope_prefix=f"{monorepo_pkg2_name}-?", + path_filters=(".",), + ) + ) + + common_configs: dict[str, TomlSerializableTypes] = { + # Set the default release branch + "tool.semantic_release.branches.main": { + "match": r"^(main|master)$", + "prerelease": False, + }, + "tool.semantic_release.allow_zero_version": True, + "tool.semantic_release.changelog.exclude_commit_patterns": [r"^chore"], + "tool.semantic_release.commit_parser": f"{commit_type}-monorepo", + "tool.semantic_release.commit_parser_options.parse_squash_commits": pkg1_commit_parser.options.parse_squash_commits, + "tool.semantic_release.commit_parser_options.ignore_merge_commits": pkg1_commit_parser.options.ignore_merge_commits, + } + + pkg1_new_version = Version.parse( + "0.0.1", tag_format=f"{monorepo_pkg1_name}-{tag_format_str}" + ) + pkg2_new_version = Version.parse( + "0.1.0", tag_format=f"{monorepo_pkg2_name}-{tag_format_str}" + ) + + repo_construction_steps: list[RepoActions] = [ + { + "action": RepoActionStep.CREATE_MONOREPO, + "details": { + "commit_type": commit_type, + "hvcs_client_name": hvcs_client_name, + "hvcs_domain": hvcs_domain, + "post_actions": [ + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg1_dir, + "package_name": monorepo_pkg1_name, + "tag_format_str": pkg1_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg1_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg1_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + "tool.semantic_release.commit_parser_options.scope_prefix": pkg1_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg1_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.CONFIGURE_MONOREPO, + "details": { + "package_dir": monorepo_pkg2_dir, + "package_name": monorepo_pkg2_name, + "tag_format_str": pkg2_new_version.tag_format, + "mask_initial_release": mask_initial_release, + "extra_configs": { + **common_configs, + "tool.semantic_release.commit_message": ( + pkg2_cmt_msg_format := dedent( + f"""\ + chore(release): {monorepo_pkg2_name}@{{version}} [skip ci] + + Automatically generated by python-semantic-release + """ + ) + ), + "tool.semantic_release.commit_parser_options.scope_prefix": pkg2_commit_parser.options.scope_prefix, + "tool.semantic_release.commit_parser_options.path_filters": pkg2_commit_parser.options.path_filters, + **(extra_configs or {}), + }, + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": ( + cid_c1_initial := "c1_initial_commit" + ), + "conventional": INITIAL_COMMIT_MESSAGE, + "emoji": INITIAL_COMMIT_MESSAGE, + "scipy": INITIAL_COMMIT_MESSAGE, + "datetime": next(commit_timestamp_gen), + "include_in_changelog": bool( + commit_type == "emoji" + ), + }, + ], + commit_type, + # this parser does not matter since the commit is common + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + ], + }, + } + ] + + # Make initial release for package 1 + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg1_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": (cid_c2_pkg1_fix := "c2_pkg1_fix"), + "conventional": "fix: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "emoji": ":bug: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "scipy": "MAINT: no pkg scope but file in pkg 1 directory\n\nResolves: #123\n", + "datetime": next(commit_timestamp_gen), + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_c1_initial, cid_c2_pkg1_fix], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "pre_actions": [change_to_pkg2_dir], + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": (cid_c4_pkg2_feat := "c4_pkg2_feat"), + "conventional": "feat: no pkg scope but file in pkg 2 directory", + "emoji": ":sparkles: no pkg scope but file in pkg 2 directory", + "scipy": "ENH: no pkg scope but file in pkg 2 directory", + "datetime": next(commit_timestamp_gen), + }, + { + "cid": (cid_c5_pkg2_test := "c5_pkg2_test"), + "conventional": "test: no pkg scope but add tests to package 2 directory", + "emoji": ":checkmark: no pkg scope but add tests to package 2 directory", + "scipy": "TST: no pkg scope but add tests to package 2 directory", + "datetime": next(commit_timestamp_gen), + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + "post_actions": [change_to_example_project_dir], + }, + }, + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": (cid_c6_pkg2_docs := "c6_pkg2_docs"), + "conventional": "docs(pkg2-cli): common docs modified outside of pkg 2, identified by scope", + "emoji": ":book: (pkg2-cli) common docs modified outside of pkg 2, identified by scope", + "scipy": "DOC:pkg2-cli: common docs modified outside of pkg 2, identified by scope", + "datetime": next(commit_timestamp_gen), + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [ + cid_c1_initial, + cid_c4_pkg2_feat, + cid_c5_pkg2_test, + cid_c6_pkg2_docs, + ], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg2_new_version = Version.parse( + "0.1.1", tag_format=pkg2_new_version.tag_format + ) + + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": (cid_c8_pkg2_fix := "c8_pkg2_fix"), + "conventional": "fix(pkg2-cli): file modified outside of pkg 2, identified by scope", + "emoji": ":bug: (pkg2-cli) file modified outside of pkg 2, identified by scope", + "scipy": "MAINT:pkg2-cli: file modified outside of pkg 2, identified by scope", + "datetime": next(commit_timestamp_gen), + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg2_commit_parser, + ), + monorepo=True, + ), + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg2_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg2_new_version.tag_format, + "version_py_file": monorepo_pkg2_version_py_file.relative_to( + monorepo_pkg2_dir + ), + "commit_message_format": pkg2_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg2_new_version, + "dest_files": pkg2_changelog_file_definitions, + "commit_ids": [cid_c8_pkg2_fix], + }, + }, + change_to_pkg2_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + pkg1_new_version = Version.parse( + "0.1.0", tag_format=pkg1_new_version.tag_format + ) + + # Add a feature to package 1 and release + repo_construction_steps.extend( + [ + { + "action": RepoActionStep.MAKE_COMMITS, + "details": { + "commits": convert_commit_specs_to_commit_defs( + [ + { + "cid": (cid_c10_pkg1_feat := "c10_pkg1_feat"), + "conventional": "feat(pkg1): file modified outside of pkg 1, identified by scope", + "emoji": ":sparkles: (pkg1) file modified outside of pkg 1, identified by scope", + "scipy": "ENH:pkg1: file modified outside of pkg 1, identified by scope", + "datetime": next(commit_timestamp_gen), + }, + ], + commit_type, + parser=cast( + "CommitParser[ParseResult, ParserOptions]", + pkg1_commit_parser, + ), + monorepo=True, + ), + }, + }, + { + "action": RepoActionStep.RELEASE, + "details": { + "version": str(pkg1_new_version), + "datetime": next(commit_timestamp_gen), + "tag_format": pkg1_new_version.tag_format, + "version_py_file": monorepo_pkg1_version_py_file.relative_to( + monorepo_pkg1_dir + ), + "commit_message_format": pkg1_cmt_msg_format, + "pre_actions": [ + { + "action": RepoActionStep.WRITE_CHANGELOGS, + "details": { + "new_version": pkg1_new_version, + "dest_files": pkg1_changelog_file_definitions, + "commit_ids": [cid_c10_pkg1_feat], + }, + }, + change_to_pkg1_dir, + ], + "post_actions": [change_to_example_project_dir], + }, + }, + ] + ) + + return repo_construction_steps + + return _get_repo_from_definition + + +@pytest.fixture(scope="session") +def build_trunk_only_monorepo_w_tags( + build_repo_from_definition: BuildRepoFromDefinitionFn, + get_repo_definition_4_trunk_only_monorepo_w_tags: GetRepoDefinitionFn, + get_cached_repo_data: GetCachedRepoDataFn, + build_repo_or_copy_cache: BuildRepoOrCopyCacheFn, + build_spec_hash_4_trunk_only_monorepo_w_tags: str, +) -> BuildSpecificRepoFn: + def _build_specific_repo_type( + repo_name: str, commit_type: CommitConvention, dest_dir: Path + ) -> Sequence[RepoActions]: + def _build_repo(cached_repo_path: Path) -> Sequence[RepoActions]: + repo_construction_steps = get_repo_definition_4_trunk_only_monorepo_w_tags( + commit_type=commit_type, + ) + return build_repo_from_definition(cached_repo_path, repo_construction_steps) + + build_repo_or_copy_cache( + repo_name=repo_name, + build_spec_hash=build_spec_hash_4_trunk_only_monorepo_w_tags, + build_repo_func=_build_repo, + dest_dir=dest_dir, + ) + + if not (cached_repo_data := get_cached_repo_data(proj_dirname=repo_name)): + raise ValueError("Failed to retrieve repo data from cache") + + return cached_repo_data["build_definition"] + + return _build_specific_repo_type + + +# --------------------------------------------------------------------------- # +# Test-level fixtures that will cache the built directory & set up test case # +# --------------------------------------------------------------------------- # + + +@pytest.fixture +def monorepo_w_trunk_only_releases_conventional_commits( + build_trunk_only_monorepo_w_tags: BuildSpecificRepoFn, + example_project_git_repo: ExProjectGitRepoFn, + example_project_dir: ExProjectDir, + change_to_ex_proj_dir: None, +) -> BuiltRepoResult: + repo_name = monorepo_w_trunk_only_releases_conventional_commits.__name__ + commit_type: CommitConvention = repo_name.split("_")[-2] # type: ignore[assignment] + + return { + "definition": build_trunk_only_monorepo_w_tags( + repo_name=repo_name, + commit_type=commit_type, + dest_dir=example_project_dir, + ), + "repo": example_project_git_repo(), + } diff --git a/tests/fixtures/repos/git_flow/repo_w_1_release_channel.py b/tests/fixtures/repos/git_flow/repo_w_1_release_channel.py index 4c12f6a72..a36a5eb3f 100644 --- a/tests/fixtures/repos/git_flow/repo_w_1_release_channel.py +++ b/tests/fixtures/repos/git_flow/repo_w_1_release_channel.py @@ -24,7 +24,7 @@ from typing import Any, Generator, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/git_flow/repo_w_2_release_channels.py b/tests/fixtures/repos/git_flow/repo_w_2_release_channels.py index 59aa7541a..ba6bd820a 100644 --- a/tests/fixtures/repos/git_flow/repo_w_2_release_channels.py +++ b/tests/fixtures/repos/git_flow/repo_w_2_release_channels.py @@ -24,7 +24,7 @@ from typing import Any, Generator, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/git_flow/repo_w_3_release_channels.py b/tests/fixtures/repos/git_flow/repo_w_3_release_channels.py index cc8b6b42d..bfddc1c82 100644 --- a/tests/fixtures/repos/git_flow/repo_w_3_release_channels.py +++ b/tests/fixtures/repos/git_flow/repo_w_3_release_channels.py @@ -24,7 +24,7 @@ from typing import Any, Generator, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/git_flow/repo_w_4_release_channels.py b/tests/fixtures/repos/git_flow/repo_w_4_release_channels.py index 4cb3ac1f4..1495b6dbd 100644 --- a/tests/fixtures/repos/git_flow/repo_w_4_release_channels.py +++ b/tests/fixtures/repos/git_flow/repo_w_4_release_channels.py @@ -24,7 +24,7 @@ from typing import Any, Generator, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/github_flow/repo_w_default_release.py b/tests/fixtures/repos/github_flow/repo_w_default_release.py index 612dac16f..4a7cb1064 100644 --- a/tests/fixtures/repos/github_flow/repo_w_default_release.py +++ b/tests/fixtures/repos/github_flow/repo_w_default_release.py @@ -24,7 +24,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/github_flow/repo_w_default_release_w_branch_update_merge.py b/tests/fixtures/repos/github_flow/repo_w_default_release_w_branch_update_merge.py index b6ca93954..a224aad47 100644 --- a/tests/fixtures/repos/github_flow/repo_w_default_release_w_branch_update_merge.py +++ b/tests/fixtures/repos/github_flow/repo_w_default_release_w_branch_update_merge.py @@ -24,7 +24,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/github_flow/repo_w_release_channels.py b/tests/fixtures/repos/github_flow/repo_w_release_channels.py index 4e3fb87be..ab2f9effa 100644 --- a/tests/fixtures/repos/github_flow/repo_w_release_channels.py +++ b/tests/fixtures/repos/github_flow/repo_w_release_channels.py @@ -24,7 +24,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/repo_initial_commit.py b/tests/fixtures/repos/repo_initial_commit.py index cef6eacfe..baed8fe97 100644 --- a/tests/fixtures/repos/repo_initial_commit.py +++ b/tests/fixtures/repos/repo_initial_commit.py @@ -20,7 +20,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support.py b/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support.py index a0b6f8b16..cd8761e58 100644 --- a/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support.py +++ b/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support.py @@ -24,7 +24,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support_w_prereleases.py b/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support_w_prereleases.py index 557f9f38b..9b1c4c527 100644 --- a/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support_w_prereleases.py +++ b/tests/fixtures/repos/trunk_based_dev/repo_w_dual_version_support_w_prereleases.py @@ -24,7 +24,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/trunk_based_dev/repo_w_no_tags.py b/tests/fixtures/repos/trunk_based_dev/repo_w_no_tags.py index dc2362f39..201cc9c87 100644 --- a/tests/fixtures/repos/trunk_based_dev/repo_w_no_tags.py +++ b/tests/fixtures/repos/trunk_based_dev/repo_w_no_tags.py @@ -22,7 +22,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/trunk_based_dev/repo_w_prereleases.py b/tests/fixtures/repos/trunk_based_dev/repo_w_prereleases.py index f9536183d..0f958aff3 100644 --- a/tests/fixtures/repos/trunk_based_dev/repo_w_prereleases.py +++ b/tests/fixtures/repos/trunk_based_dev/repo_w_prereleases.py @@ -23,7 +23,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser diff --git a/tests/fixtures/repos/trunk_based_dev/repo_w_tags.py b/tests/fixtures/repos/trunk_based_dev/repo_w_tags.py index 6121b3699..488e6c28c 100644 --- a/tests/fixtures/repos/trunk_based_dev/repo_w_tags.py +++ b/tests/fixtures/repos/trunk_based_dev/repo_w_tags.py @@ -23,7 +23,7 @@ from typing import Any, Sequence from semantic_release.commit_parser._base import CommitParser, ParserOptions - from semantic_release.commit_parser.conventional import ( + from semantic_release.commit_parser.conventional.parser import ( ConventionalCommitParser, ) from semantic_release.commit_parser.emoji import EmojiCommitParser