Set listener qps and burst - #4558
Conversation
|
Hello! Thank you for your contribution. Please review our contribution guidelines to understand the project's testing and code conventions. |
There was a problem hiding this comment.
Pull request overview
This PR adjusts the ghalistener scaler’s in-cluster Kubernetes client configuration to reduce client-go throttling during high job throughput, aligning expected throughput with the listener batch size behavior discussed in #4554.
Changes:
- Set Kubernetes client-go rate limit defaults in the listener scaler to
QPS=50andBurst=100. - Reformat the “Ephemeral runner set scaled” log call (no functional change).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| conf.QPS = 50 | ||
| conf.Burst = 100 |
There was a problem hiding this comment.
I'm happy with these new values even though they're hardcoded.
If they do need to be configurable I think the easiest way to go about it would be to use environment variables against the listener container since there doesn't seem to be a good method of passing configuration to the listener at the moment.
| @@ -216,7 +219,8 @@ func (w *Scaler) HandleDesiredRunnerCount(ctx context.Context, count int) (int, | |||
| return 0, fmt.Errorf("could not patch ephemeral runner set , patch JSON: %s, error: %w", string(mergePatch), err) | |||
|
@nikola-jokic is there any way to push this PR? Listener throttling is a huge bottleneck for us. Thanks 🙏 ❤️ |
Since the size of the batch is currently 50, and the 50 should not be a huge number for requests/s, set the QPS to 50 with burst 100 in the listener client.
Fixes #4554