Keeping this issue open for now for the initial refactoring follow-ups. The current focus is just on getting this extracted from our core repo. Afterwards I'd like to start refactoring the core, maybe introducing better abstractions to allow easier modification by users of the lib.