Commit ed205ca
fix: Avoid heavy imports in operator constructor
After a previous commit [1] improving import time on operator import,
this commit fixes the import time issues when instantiating any of the
dbt operators.
The main issue still present was the enum imports in the operator
constructors, which make DAGs slow to import any time they instantiated
one of them.
Profiling can be easily run locally, with the following command:
```shell
python -X importtime -c "from airflow_dbt_python.operators.dbt import DbtRunOperator; DbtRunOperator(task_id='test')" 2>import-times.log
```
[1] 67c8d781 parent 3ca0fa2 commit ed205ca
File tree
4 files changed
+39
-37
lines changed- airflow_dbt_python
- hooks
- operators
- utils
4 files changed
+39
-37
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | 9 | | |
11 | 10 | | |
12 | 11 | | |
| |||
50 | 49 | | |
51 | 50 | | |
52 | 51 | | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
| 52 | + | |
67 | 53 | | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
80 | | - | |
81 | | - | |
82 | | - | |
83 | | - | |
84 | | - | |
85 | | - | |
| 54 | + | |
86 | 55 | | |
87 | 56 | | |
88 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
18 | 20 | | |
19 | 21 | | |
20 | 22 | | |
| |||
110 | 112 | | |
111 | 113 | | |
112 | 114 | | |
113 | | - | |
114 | | - | |
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
| |||
618 | 618 | | |
619 | 619 | | |
620 | 620 | | |
621 | | - | |
622 | | - | |
623 | 621 | | |
624 | 622 | | |
625 | 623 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
0 commit comments