fix(worker): event loop should run until yield (#53)

<!-- Describe what has changed in this PR -->
**What changed?**

* added a new method in DeterministicEventLoop to run workflow
coroutines until no callbacks available

<!-- Tell your future self why have you made these changes -->
**Why?**

Existing EventLoop API doesn't have functionality of waiting until
callback queue is empty. This is required for cadence workflows to
generate all decision outcomes.

<!-- How have you verified this change? Tested locally? Added a unit
test? Checked in staging env? -->
**How did you test it?**

Unit Test

<!-- Assuming the worst case, what can be broken when deploying this
change to production? -->
**Potential risks**

<!-- Is it notable for release? e.g. schema updates, configuration or
data migration required? If so, please mention it, and also update
CHANGELOG.md -->
**Release notes**

<!-- Is there any documentation updates should be made for config,
https://cadenceworkflow.io/docs/operation-guide/setup/ ? If so, please
open an PR in https://github.com/cadence-workflow/cadence-docs -->
**Documentation Changes**

---------

Signed-off-by: Shijie Sheng <liouvetren@gmail.com>

Author: shijiesheng

cadence/_internal/workflow/deterministic_event_loop.py

@@ -27,6 +27,16 @@ def __init__(self):
         self._stopping = False
         self._closed = False
 
+    def run_until_yield(self):
+        """Run until stop() is called."""
+        self._run_forever_setup()
+        try:
+            while self._ready:
+                self._run_once()
+        finally:
+            self._run_forever_cleanup()
+
+    # Event Loop APIs
     def call_soon(
         self,
         callback: Callable[[Unpack[_Ts]], object],

cadence/_internal/workflow/workflow_engine.py

@@ -225,10 +225,7 @@ def _execute_workflow_once(
                     self._workflow_instance.run(workflow_input)
                 )
 
-                # signal the loop to stop after the first run
-                self._loop.stop()
-                # this starts the loop and runs once then stops with cleanup
-                self._loop.run_forever()
+                self._loop.run_until_yield()
 
         except Exception as e:
             logger.error(

tests/cadence/_internal/workflow/test_deterministic_event_loop.py

@@ -76,3 +76,17 @@ def test_create_task(self):
         size = 10000
         results = self.loop.run_until_complete(coro_await_task(size))
         assert results == list(range(size))
+
+    def test_run_once(self):
+        # run once won't clear the read queue
+        self.loop.create_task(coro_await_task(10))
+        self.loop.stop()
+        self.loop.run_forever()
+        assert len(self.loop._ready) == 10
+
+    def test_run_until_yield(self):
+        # run until yield will clear the read queue
+        task = self.loop.create_task(coro_await_task(3))
+        self.loop.run_until_yield()
+        assert len(self.loop._ready) == 0
+        assert task.done() is True