Skip to content

Remove the CSV build cap #902

Open
Open

Description

@bencap

Summary

The CSV build cap limits each API process to a couple of large score set CSV builds at once, because a build is CPU-bound and several at once stall the process. It's a stopgap. Once exports are cheap or served from cache it only turns downloads away, so this issue removes it.

Scope

  • Delete lib/csv/build_limiter.py and its uses in the three CSV routes in routers/score_sets.py.
  • Remove CSV_BUILD_SLOTS and CSV_BUILD_MIN_ROWS from the README and the API compose files in mavedb-deployment.
  • Remove the 503 note from the Quickstart's Rate Limits section, if nothing else still returns it for downloads.

Before removing

Acceptance criteria

  • No score set CSV route returns 503 when other builds are running.
  • A load test of 10 concurrent full downloads of the largest score set doesn't slow down other routes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    app: backendTask implementation touches the backend

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions