Skip to content

feat(integrations): Add Sonar connector support with git submodule #873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
aaronsteers wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

feat(integrations): Add Sonar connector support with git submodule #873

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ac23073
feat(integrations): Add Sonar connector support
devin-ai-integration[bot] Nov 20, 2025
c85fae9
Add sonar as git submodule and configure dependencies
devin-ai-integration[bot] Nov 20, 2025
1b2883d
Add Octavia Bot auth for submodule checkout in pytest workflows
devin-ai-integration[bot] Nov 20, 2025
930434c
Add Octavia Bot auth and submodule checkout to all workflows
devin-ai-integration[bot] Nov 20, 2025
b86455e
Fix deptry and test patches for Sonar integration
devin-ai-integration[bot] Nov 20, 2025
06c07d5
Fix deptry configuration for Sonar integration
devin-ai-integration[bot] Nov 20, 2025
8513158
Add poe task for deptry and update workflow to use it
devin-ai-integration[bot] Nov 20, 2025
313f142
Add deptry ignores for self-imports and connector-sdk
devin-ai-integration[bot] Nov 21, 2025
1361ab1
Fix test patch for SecretStr - remove unnecessary mock
devin-ai-integration[bot] Nov 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/autofix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,19 @@ jobs:
python-autofix:
runs-on: ubuntu-latest
steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/fix-pr-command.yml
View file
Open in desktop
Copy link

@github-actions github-actions bot Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:2:27: Double quote to prevent globbing and word splitting [shellcheck]

PyAirbyte/.github/workflows/fix-pr-command.yml

Line 102 in 930434c

run: |

Copy link

@github-actions github-actions bot Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:4:26: Double quote to prevent globbing and word splitting [shellcheck]

PyAirbyte/.github/workflows/fix-pr-command.yml

Line 102 in 930434c

run: |

Copy link

@github-actions github-actions bot Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:2:27: Double quote to prevent globbing and word splitting [shellcheck]

PyAirbyte/.github/workflows/fix-pr-command.yml

Line 131 in 930434c

run: |

Copy link

@github-actions github-actions bot Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 [actionlint] reported by reviewdog 🐶
shellcheck reported issue in this script: SC2086:info:4:26: Double quote to prevent globbing and word splitting [shellcheck]

PyAirbyte/.github/workflows/fix-pr-command.yml

Line 131 in 930434c

run: |

Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout Airbyte
Expand All @@ -49,6 +49,7 @@ jobs:
# Important that this is set so that CI checks are triggered again
# Without this we would be forever waiting on required checks to pass
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive

- name: Checkout PR (${{ github.event.inputs.pr }})
uses: dawidd6/action-checkout-pr@a7598e18433a763b784f17d666372913d8bd4205 # v1.2.0
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/pydoc_preview.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,19 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/pydoc_publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,19 @@ jobs:
url: ${{ steps.deployment.outputs.page_url }}

steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand Down
33 changes: 33 additions & 0 deletions .github/workflows/python_lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,19 @@ jobs:
runs-on: ubuntu-latest
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Poetry
uses: Gr1N/setup-poetry@48b0f77c8c1b1b19cb962f0f00dff7b4be8f81ec # v9
with:
Expand All @@ -43,8 +54,19 @@ jobs:
runs-on: ubuntu-latest
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand All @@ -65,8 +87,19 @@ jobs:
runs-on: ubuntu-latest
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Poetry
uses: Gr1N/setup-poetry@48b0f77c8c1b1b19cb962f0f00dff7b4be8f81ec # v9
with:
Expand Down
47 changes: 45 additions & 2 deletions .github/workflows/python_pytest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,19 @@ jobs:
runs-on: ubuntu-latest
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
Comment on lines +28 to +40
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

Verify GitHub App token is available and properly scoped for fork PRs.

The GitHub App authentication pattern looks correct for accessing the private sonar submodule, but the pytest-no-creds job runs on fork PRs where the OCTAVIA_BOT_APP_ID and OCTAVIA_BOT_PRIVATE_KEY secrets won't be available. This will cause the create-github-app-token step to fail on fork PRs.

Do you want to add a conditional check (similar to the pytest job's fork logic) to skip the GitHub App token creation on forks, or does the action gracefully handle missing secrets?

I'll verify this concern by examining the workflow file and checking how the GitHub App token action handles missing secrets.

🏁 Script executed:

#!/bin/bash
# Inspect the full workflow file to understand pytest-no-creds job config
# and check for existing fork protection conditionals
cat .github/workflows/python_pytest.yml | head -n 350

Length of output: 10302

🏁 Script executed:

#!/bin/bash
# Also check if there are any conditional checks (if:) in the workflow
rg -n "if:" .github/workflows/python_pytest.yml -C2

Length of output: 2982

🏁 Script executed:

#!/bin/bash
# Search for pytest-no-creds job definition
rg -n "pytest-no-creds" .github/workflows/python_pytest.yml -A15

Length of output: 1924

🌐 Web query:

actions/create-github-app-token missing secrets behavior fork PR

💡 Result:

Short answer: actions/create-github-app-token needs the app private key (a secret) to create an installation token — GitHub will NOT provide repository secrets to workflows triggered by a pull request from a fork, so the action will be unable to read the PRIVATE_KEY (and will fail or not produce a usable token) when the workflow runs on a fork PR. [1][2]

What to do (brief):

  • If you must run with secrets for fork PRs, use pull_request_target (runs in base repo and can access secrets) — but avoid checking out or running untrusted PR code in that job because of security risks. [3]
  • Alternatively require contributors to open branches in the same repo (not a fork) or have a maintainer re-run/approve the workflow so secrets are available.
  • Or design the workflow to run a safe, unprivileged check for fork PRs and only run secret-requiring jobs after merge or via a maintainer-triggered workflow_run. [3][1]

Sources:

  • actions/create-github-app-token repo (usage/inputs/notes). [1]
  • GitHub docs: secrets are not passed to workflows triggered by a pull request from a fork. [2]
  • GitHub security docs on pull_request vs pull_request_target and associated risks. [3]

References:
[1] actions/create-github-app-token (GitHub). (github.com)
[2] Understanding GitHub secret types / Secrets limitations (GitHub Docs). (docs.github.com)
[3] GitHub Security Lab / docs on pull_request_target and secrets (GitHub). (securitylab.github.com)

Citations:

Add fork protection conditionals to pytest-fast, pytest-no-creds, and dependency-analysis jobs.

Your concern is spot-on—but the issue affects more than pytest-no-creds. GitHub does NOT provide repository secrets to workflows triggered by a pull request from a fork, so all three jobs that use create-github-app-token will fail when triggered on fork PRs.

This is a design contradiction: the workflow comments explicitly state that pytest-no-creds exists to run tests "on forks, where secrets are not available," yet it now requires secrets for GitHub App authentication. The pytest job correctly protects against this with a fork conditional at lines 170–172, but pytest-fast, pytest-no-creds, and dependency-analysis lack this protection.

You'll want to add the same conditional to those three jobs:

if: >
  (github.event_name == 'push' && github.ref == 'refs/heads/main') ||
  (github.event.pull_request.head.repo.fork == false)

Wdyt? Should pytest-no-creds perhaps have different logic if it's meant to be fork-friendly, or is the intent to require these jobs to only run on the main repo going forward?

🤖 Prompt for AI Agents 
.github/workflows/python_pytest.yml lines 28-40: the jobs pytest-fast,
pytest-no-creds, and dependency-analysis call create-github-app-token and will
fail on forked PRs because secrets are not provided; add the same
fork-protection conditional used by the pytest job (the check that allows only
pushes to main or pull_request where github.event.pull_request.head.repo.fork ==
false) to each of those three job definitions so they are skipped for forked PR
runs.

- name: Set up Poetry
uses: Gr1N/setup-poetry@48b0f77c8c1b1b19cb962f0f00dff7b4be8f81ec # v9
with:
Expand Down Expand Up @@ -90,8 +101,19 @@ jobs:
runs-on: ubuntu-latest
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Poetry
uses: Gr1N/setup-poetry@48b0f77c8c1b1b19cb962f0f00dff7b4be8f81ec # v9
with:
Expand Down Expand Up @@ -168,8 +190,19 @@ jobs:
PYTHONIOENCODING: utf-8
steps:
# Common steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand Down Expand Up @@ -241,8 +274,19 @@ jobs:
name: Dependency Analysis with Deptry
runs-on: ubuntu-latest
steps:
- name: Authenticate as GitHub App
uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2.1.4
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
token: ${{ steps.get-app-token.outputs.token }}
submodules: recursive
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
Expand All @@ -256,5 +300,4 @@ jobs:

# Job-specific step(s):
- name: Run Deptry
run: |
poetry run deptry .
run: poetry run poe check-deps
3 changes: 2 additions & 1 deletion .github/workflows/test-pr-command.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ jobs:
id: get-app-token
with:
owner: "airbytehq"
repositories: "PyAirbyte"
repositories: "PyAirbyte,sonar"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}

Expand All @@ -90,6 +90,7 @@ jobs:
with:
token: ${{ secrets.GITHUB_TOKEN }}
ref: ${{ needs.start-workflow.outputs.commit-sha }}
submodules: recursive

# Post "In Progress" status to the PR.
# This is required because otherwise slash commands won't automatically
Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "sonar"]
path = sonar
url = https://github.com/airbytehq/sonar.git
6 changes: 4 additions & 2 deletions .ruff.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@
target-version = "py310"
preview = true
line-length = 100
exclude = [
"sonar", # Git submodule, not part of PyAirbyte codebase
]

[lint]
select = [
# For rules reference, see https://docs.astral.sh/ruff/rules/
"A", # flake8-builtins
Expand Down Expand Up @@ -61,8 +65,6 @@ select = [
"W", # pycodestyle (warnings)
"YTT", # flake8-2020
]

[lint]
ignore = [
# For rules reference, see https://docs.astral.sh/ruff/rules/

Expand Down
6 changes: 6 additions & 0 deletions airbyte/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@
from airbyte.datasets import CachedDataset
from airbyte.destinations.base import Destination
from airbyte.destinations.util import get_destination
from airbyte.integrations.base import Integration
from airbyte.integrations.util import get_integration
from airbyte.records import StreamRecord
from airbyte.registry import get_available_connectors
from airbyte.results import ReadResult, WriteResult
Expand All @@ -154,6 +156,7 @@
documents,
exceptions, # noqa: ICN001 # No 'exc' alias for top-level module
experimental,
integrations,
logs,
mcp,
records,
Expand All @@ -175,6 +178,7 @@
"documents",
"exceptions",
"experimental",
"integrations",
"logs",
"mcp",
"records",
Expand All @@ -187,6 +191,7 @@
"get_colab_cache",
"get_default_cache",
"get_destination",
"get_integration",
"get_secret",
"get_source",
"new_local_cache",
Expand All @@ -195,6 +200,7 @@
"CachedDataset",
"Destination",
"DuckDBCache",
"Integration",
"ReadResult",
"SecretSourceEnum",
"Source",
Expand Down
2 changes: 1 addition & 1 deletion airbyte/_connector_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
class ConnectorBase(abc.ABC):
"""A class representing a destination that can be called."""

connector_type: Literal["destination", "source"]
connector_type: Literal["destination", "source", "integration"]

def __init__(
self,
Expand Down
Loading
Loading