Keep scheduling remaining tasks when a running Dag is paused

[NihalRadhakrishna]

1. Problem

Pausing a Dag stopped the scheduler from examining already-running Dag runs and from enqueueing remaining scheduled tasks. In-flight work froze until unpaused, even though running tasks were allowed to finish. As a result,
a running Dag could remain running indefinitely because its downstream tasks would never be queued.

closes: #71381

2. Solution

• A Dag run that is already running continues while the Dag is paused: remaining tasks are still scheduled and queued, and the run can reach a terminal state.
• Queued Dag runs stay queued until unpause.
• New Dag runs are not created.
• Pause is not the way to stop in-flight work. If you need a running Dag run to stop, mark the run or its tasks as failed.

Cleanup

• Removed _update_dag_run_state_for_paused_dags (and its 60s timer). Paused running Dag runs are now examined in the main scheduler loop, so a separate update_state pass was redundant.

Tests

uv run --project airflow-core pytest \
  airflow-core/tests/unit/models/test_dagrun.py::TestDagRun::test_next_dagruns_to_examine_paused \
  airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_execute_task_instances_paused_running_dag_still_enqueues \
  airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_paused_running_dag_run_reaches_terminal_state \
  airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_paused_running_dag_still_schedules_and_enqueues_downstream \
  airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_paused_queued_dag_run_stays_queued_until_unpause \
  -xvs

---

Was generative AI tooling used to co-author this PR?

• Yes — Cursor Grok 4.6

Generated-by: Cursor Grok 4.6 following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[raphaelauv]

did you read this -> #71384 (comment) ?

[NihalRadhakrishna]

Apologies for not addressing your earlier comment.

If we don't want to change the behavior of the current Pause DAG button, as you suggested, we could give users two options when they click pause:

1. Pause all DAG runs (current and new) - default behaviour which exists currently
2. Pause only new DAG runs - new feature

Unpause would resume current and new dag runs.
For this, I think we would need an additional attribute, such as pause_type, in DagModel, since the existing is_paused attribute only indicates whether the DAG is paused or not.

Open to suggestions on the best way to approach this.