Fix bug to pass resource parameters to ray task options()#62
Fix bug to pass resource parameters to ray task options()#62eywalker merged 2 commits intowalkerlab:mainfrom
Conversation
|
I made a second commit to fix a separate issue and forgot that these changes would be reflected here! But it is another bugfix. AI summary here: Summary Root Cause: In PodNodeStream.run_async(), when processing existing cached entries, the code incorrectly drops ALL columns from target_entries: python# In run_async() - BUGGY: This removes the tag columns (e.g., subject, session_date, probe_id), which are then expected when creating TableStream(existing, tag_columns=tag_keys). The Fix: PodNodeStream.run() has the correct implementation that only drops specific system columns: |
eywalker
left a comment
There was a problem hiding this comment.
Great catch! Thanks for the fix.
Happy if this PR is just a conversation starter, and if the desired behavior should be implemented in the source code another way.
Prior to these changes in this PR, passing resource options to Ray did not work, as in the following example:
I would receive an error message saying that whatever I'm passing to Ray is not a valid argument, and it would list valid arguments like
num_cpusandnum_gpus. I think the entire dict was likely getting passed instead of as kwargs, so I used ** to pass the contents of the dict as arguments. Works like a charm now.