Skip to content

Commit abd8149

Browse files
adamantiketomasfarias
authored andcommitted
fix: Conditionally import required hook
This change makes the library to only import the hook it's going to need based on the remote scheme. As it is currently implemented, an environment that uses a S3 remote fails if it doesn't have the `ssh` Airflow provider installer: ``` Traceback (most recent call last): File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/dbt.py", line 126, in get_remote return self.remotes[(scheme, conn_id)] KeyError: ('s3', None) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/dbt.py", line 300, in dbt_directory project_dir, profiles_dir = self.prepare_directory( File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/dbt.py", line 345, in prepare_directory project_dir_path = self.download_dbt_project( File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/dbt.py", line 158, in download_dbt_project remote = self.get_remote(scheme, self.project_conn_id) File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/dbt.py", line 128, in get_remote remote = get_remote(scheme, conn_id) File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/remote.py", line 147, in get_remote from .git import DbtGitRemoteHook File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow_dbt_python/hooks/git.py", line 5, in <module> from airflow.providers.ssh.hooks.ssh import SSHHook ModuleNotFoundError: No module named 'airflow.providers.ssh' ``` The same happens when the environment uses a remote that requires the SSH hook, and doesn't install the `aws` Airflow provider.
1 parent ef1449d commit abd8149

File tree

1 file changed

+6
-4
lines changed

1 file changed

+6
-4
lines changed

airflow_dbt_python/hooks/remote.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -144,15 +144,17 @@ def get_remote(scheme: str, conn_id: Optional[str] = None) -> DbtRemoteHook:
144144
In the future we should make our hooks discoverable and package ourselves as a
145145
proper Airflow providers package.
146146
"""
147-
from .git import DbtGitRemoteHook
148-
from .localfs import DbtLocalFsRemoteHook
149-
from .s3 import DbtS3RemoteHook
150-
151147
if scheme == "s3":
148+
from .s3 import DbtS3RemoteHook
149+
152150
remote_cls: Type[DbtRemoteHook] = DbtS3RemoteHook
153151
elif scheme in ("https", "git", "git+ssh", "ssh", "http"):
152+
from .git import DbtGitRemoteHook
153+
154154
remote_cls = DbtGitRemoteHook
155155
elif scheme == "":
156+
from .localfs import DbtLocalFsRemoteHook
157+
156158
remote_cls = DbtLocalFsRemoteHook
157159
else:
158160
raise NotImplementedError(f"Backend {scheme} is not supported")

0 commit comments

Comments
 (0)