diff --git a/.claude/skills/audit-the-list/SKILL.md b/.claude/skills/audit-the-list/SKILL.md new file mode 100644 index 0000000000..4c2b576cd0 --- /dev/null +++ b/.claude/skills/audit-the-list/SKILL.md @@ -0,0 +1,39 @@ +--- +name: audit-the-list +description: Audit README.md sections against the shortlist rules — re-verify every entry's verdict with live data, promote or demote challengers, restructure oversized use cases, prune, and evaluate proposed additions. Fire when the maintainer asks to audit, sweep, prune, re-check, or maintain sections, or asks whether an entry (or a proposed one) deserves its slot. +argument-hint: [all or specific sections] +--- + +# Audit the list + +An Audit re-runs the admission test over live sections of README.md: every entry re-verified against current data, tiers reassessed, structure reconsidered, then the maintainer adjudicates and the result is committed. The rules live in CONTRIBUTING.md (Admission, Evidence, Entry Ordering) and CONTEXT.md (vocabulary) — this skill is the process that applies them, not a second copy of them. Entry changes land only on the maintainer's explicit go. + +## 1. Scope + +Resolve the scope from the arguments. Named sections mean exactly those, whether or not they were audited before (a re-run is how a past verdict gets rechecked). `all` is ambiguous — `AskUserQuestion` whether it means every section or only never-audited ones; prior audits are recorded in git history (`git log --oneline --grep="sweep\|audit"`). With no arguments, AskUserQuestion listing the never-audited Thematic Groups. Batch the work one Thematic Group per sitting. Done when the section list is settled. + +## 2. Evidence + +Fetch live evidence for every entry in scope before judging anything (CLAUDE.md verification rule): + +- **Downloads/month**: `cd website && UV_PYTHON=3.13 uv run python fetch_pypi_downloads_via_clickpy.py` — free keyless ClickPy sweep of the full README, sole writer of `data/pypi_downloads.tsv` (rewritten from scratch each run; header row, every row stamped with its `fetched_at` date). A cache whose `fetched_at` is within the last 7 days is current enough for verdicts — skip the sweep; older than that, re-run it (costs ~1s). Cross-checks print to stdout, take explicit names, and never touch the cache: `fetch_pypi_downloads_via_bigquery.py ...` (canonical source, maintainer's own GCP account, `--dry-run` first — the docstring carries the cost constraints; full-README sweeps exceed the free tier, keep name lists small), `fetch_pypi_downloads_via_pepy.py ...` (needs `PEPY_TECH_API_KEY` in repo-root `.env`, throttled to 5 requests/minute), or `https://pypistats.org/api/packages/{name}/recent` paced 8s or slower. pypistats excludes mirror/CI traffic; ClickPy, BigQuery, and pepy include it — never mix sources within one comparison. +- **Repo state**: archived flag, last push, created date, stars, description — `gh api repos/{owner}/{repo}`, GitLab API for GitLab-hosted projects. +- **PyPI metadata** (`https://pypi.org/pypi/{name}/json`) wherever a name might not be the canonical package — ownership collisions and wrong display names surface here. Wrong-package cache rows are common: the sweep looks up README display names, so when the display name differs from the canonical package, the row silently measures an unrelated squatter or a dead predecessor. When a famous entry shows a count that looks off (too small, or absurdly round), verify identity before citing it, and fetch the canonical package's count via pepy. + +Done when every entry in scope has downloads (or a stated no-signal reason), repo state, and a confirmed PyPI name. + +## 3. Verdicts + +Draft a verdict and reason for every entry, restructure before cap: decide Splits, mints, and re-homes first — cutting before restructuring destroys entries — then apply the cap tier by tier, including promotions and demotions between obvious choice and challenger. Ground every reason in the fetched evidence; label anything unverifiable as a judgment call. Where fresh evidence contradicts the standing verdict, say so in the reason instead of silently keeping the seed. + +## 4. Review and go + +Run the `preview-verdicts` skill: it generates the interactive review page and defines how the maintainer's feedback JSON comes back and gets processed. Their verdicts are final. Before touching README.md, surface what the feedback implies but does not decide — cap overflows, homeless entries after a Split, tier for a tierless flip — as named assumptions or questions, then get an explicit go. + +## 5. Execute + +One commit per section: body lists each removal with its reason and downloads figure; restructures, tier moves, and reorders ride the same commit. Format-only outcomes (no removals) are a single style commit. `UV_PYTHON=3.13 make test` before every commit, `UV_PYTHON=3.13 make build` after the last one. Generic commit helpers tend to split a section audit into structural and per-subcategory commits — if that happens, squash back to one commit per section. Done when the tree is clean, tests passed before each commit, and the build count reconciles with the adjudicated changes. + +## 6. Record + +A conclusion that outlives the sitting goes into the repo before the audit ends: admission or evidence rules into CONTRIBUTING.md, repo process and environment facts into CLAUDE.md and AGENTS.md (kept in sync), vocabulary into CONTEXT.md. Done when nothing the next audit needs is stranded in conversation. diff --git a/.claude/skills/preview-verdicts/SKILL.md b/.claude/skills/preview-verdicts/SKILL.md new file mode 100644 index 0000000000..938a02d51a --- /dev/null +++ b/.claude/skills/preview-verdicts/SKILL.md @@ -0,0 +1,24 @@ +--- +name: preview-verdicts +description: Generate the interactive keep/drop verdict preview (HTML page with per-row feedback controls) whenever a prune sweep, batch entry edit, or restructure needs maintainer review before touching README.md — and process the feedback JSON the maintainer pastes back. +--- + +# Verdict preview + +Maintainer review happens through an interactive HTML page: one row per entry with your seeded verdict and reason, a Keep/Drop toggle and a reason field for the maintainer, and a **Copy feedback** button that exports only changed or commented rows as JSON. Generate the page, wait for the pasted JSON, then apply it. Entry changes land in README.md only after the review — and only on an explicit go. + +## Generate the preview + +1. Build the `DATA` array. A group is `[section, subcategory, rows]`; a row is `[entry, url, downloads, verdict, reason]`. + - `subcategory` may carry a note after ` — ` (rendered muted): use it for proposed splits, re-homes, or anything the maintainer should weigh for the whole group. + - `downloads` is PyPI last-month as a comma-formatted string; use `—` when no signal exists (e.g. agent skill packs), `stdlib` for standard-library modules, `fetch failed` when the lookup failed. State the fetch date in the sub-header. + - `verdict` is `keep` or `drop`, seeded from the current adjudication or dry-run. + - `reason` is plain language the maintainer reads cold — no invented shorthand. When fresh evidence contradicts the seeded verdict (a big download count on a drop, a dead repo on a keep), say so in that row's reason instead of silently changing the seed. +2. Copy `template.html` (sibling of this file) and replace the placeholders: `__TITLE__` (page title), `__SUB__` (sub-header: scope, seed provenance, fetch date, and the standing instruction to flip/comment then Copy feedback), `__KEY__` (localStorage key), `__DATA__` (the array). `__KEY__` must be unique per review — slug plus date, e.g. `awesome-python-science-2026-09-01` — because saved state under a reused key bleeds a previous review's flips into rows with the same section and entry name. +3. Write the page to `tmp/awesome-python--preview.html` in the repo root (git-ignored; create the directory if needed), `open` it, and tell the maintainer the path and the return path: flip or comment rows (they highlight yellow), press **Copy feedback**, paste the JSON into the chat. Done when the page is open and the return path is stated. + +## Process the pasted feedback + +Each JSON row is `{section, subcategory, entry, my_verdict, your_verdict, reason}`. The maintainer's verdict is final — apply it, never re-argue it. An empty reason means the verdict stands unexplained; that is enough. + +Before executing, surface anything the flips imply that the maintainer has not decided: a use case pushed past its cap, an entry left homeless by a proposed split, a request that is already satisfied (a no-op). Ask, then execute on their go. Done when every pasted row is either applied or surfaced back — none silently dropped. diff --git a/.claude/skills/preview-verdicts/template.html b/.claude/skills/preview-verdicts/template.html new file mode 100644 index 0000000000..91324e82f4 --- /dev/null +++ b/.claude/skills/preview-verdicts/template.html @@ -0,0 +1,160 @@ + + + + + +__TITLE__ + + + +
+

__TITLE__

+

__SUB__

+
+ +
+
+ + + Copies only rows you changed or commented, as JSON. +
+ + + diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 05c3b80477..41890f4696 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,24 +1,26 @@ ## Project -[Project Name](url) +[project-name](url) ## Checklist +- [ ] I read [CONTRIBUTING.md](https://github.com/vinta/awesome-python/blob/master/CONTRIBUTING.md) - awesome-python is a shortlist, not a catalog - [ ] One project per PR - [ ] PR title format: `Add project-name` -- [ ] Entry format: `- [project-name](url) - Description ending with period.` -- [ ] Description is concise and short +- [ ] Entry format: `- [pypi-name](https://github.com/owner/repo) - Description ending with period.` +- [ ] Display name is the PyPI package name +- [ ] Placed in an existing use case (new sections and subcategories are maintainer-only) +- [ ] Meets all Quality Requirements: active, stable, documented, at least 1 month old -## Why This Project Is Awesome +## Which Tier -Which criterion does it meet? (pick one) +Pick one: -- [ ] **Industry Standard** - The go-to tool for a specific use case -- [ ] **Rising Star** - 5000+ stars in < 2 years, significant adoption -- [ ] **Hidden Gem** - Exceptional quality, solves niche problems elegantly +- [ ] **Obvious choice** - a tool an experienced Python developer would name when asked "what do I use for this?" +- [ ] **Challenger** - not yet the obvious choice, but a credible successor to one. Give adoption-trajectory evidence, not popularity alone. Explain: -## How It Differs +## Displacement -If similar entries exist, what makes this one unique? +If the use case is at its cap, name the entry your project replaces and argue that yours does that entry's job better. One in, one out. diff --git a/.github/workflows/deploy-website.yml b/.github/workflows/deploy-website.yml index dd748bb563..0c23761c62 100644 --- a/.github/workflows/deploy-website.yml +++ b/.github/workflows/deploy-website.yml @@ -71,6 +71,25 @@ jobs: fi python -m json.tool website/data/github_stars.json > /dev/null + - name: Restore PyPI download data cache + uses: actions/cache/restore@v4 + with: + path: website/data/pypi_downloads.tsv + key: pypi-downloads-${{ steps.date.outputs.today }} + restore-keys: pypi-downloads- + + - name: Fetch PyPI downloads + id: fetch-downloads + continue-on-error: true + run: make fetch_pypi_downloads + + - name: Save PyPI download data cache + if: steps.fetch-downloads.outcome == 'success' + uses: actions/cache/save@v4 + with: + path: website/data/pypi_downloads.tsv + key: pypi-downloads-${{ steps.date.outputs.today }} + - name: Build website run: make build diff --git a/.gitignore b/.gitignore index f081c83359..ba30760958 100644 --- a/.gitignore +++ b/.gitignore @@ -11,14 +11,9 @@ __pycache__/ # website website/output/ -website/data/ - -# docs -docs/ +website/data/* +!website/data/pypi_name_overrides.json # agents -.agents/ -.claude/skills/ -.superpowers/ .playwright-cli/ -skills-lock.json +tmp/ diff --git a/AGENTS.md b/AGENTS.md index 29c863bcd4..31f45e60e1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,31 +1,13 @@ # AGENTS.md -## Repository Overview +An opinionated guide to the best Python frameworks, libraries, tools, and resources. -An opinionated list of Python frameworks, libraries, tools, and resources. Published at [awesome-python.com](https://awesome-python.com/). +[README.md](README.md) is the single source of truth for catalog entries and README sponsor placements; `website/` renders it into the static site: [awesome-python.com](https://awesome-python.com/). -## Entry Guidelines +## Entry Rules -**Refer to [CONTRIBUTING.md](CONTRIBUTING.md)** for acceptance criteria, quality requirements, rejection rules, and entry format. Apply these rules whenever adding or removing an entry, whether reviewing a PR or committing directly. +[CONTRIBUTING.md](CONTRIBUTING.md) holds the admission rules, quality requirements, rejection rules, entry format, and ordering. Apply it whenever adding or removing an entry — direct commits included, not only PR reviews. -## Structure - -- **README.md**: Source of truth for catalog entries and README sponsor placements. Hierarchical categories with alphabetically ordered entries. -- **CONTRIBUTING.md**: Submission guidelines and review criteria. -- **SPONSORSHIP.md**: Sponsor tiers, placement rules, and the editorial-independence policy. `website/templates/sponsorship.html` separately defines which sponsorship content appears on the published website page. -- **website/**: Static site generator that builds awesome-python.com from README.md. - - `build.py`: Parses README.md and renders HTML via Jinja2 templates. - - `fetch_github_stars.py`: Fetches star counts into `website/data/`. - - `readme_parser.py`: Markdown-to-structured-data parser. - - `templates/`, `static/`: Jinja2 templates and CSS/JS assets. - - `tests/`: Pytest tests for the build pipeline. -- **Makefile**: `make install`, `make build`, `make preview`, `make test`, `make lint`, `make format`, `make typecheck`, `make fetch_github_stars`. -- **pyproject.toml**: Uses `uv` for dependency management. Python >=3.13. - -## Key Rules - -- Alphabetical ordering within categories is mandatory. -- Quality over quantity. Only "awesome" projects. -- One project per PR. -- One entry per commit when adding or deleting entries. Format, wording, or categorization changes across multiple entries may be bundled in a single commit. -- README.md is the source of truth for catalog entries and README sponsor placements; treat `SPONSORSHIP.md` and `website/templates/sponsorship.html` as separate sponsorship content surfaces. +- Every keep/drop reason must be verified against current online data at decision time — download counts, repo activity and archived status, PyPI metadata, project docs. Judging tiers — obvious choice vs challenger — also requires WebSearch evidence (adoption trajectory, community sentiment), not download counts alone. Training-data recollections are not evidence; label anything unverifiable as a judgment call. +- One entry per commit when adding or deleting entries. Exceptions: a prune sweep is one commit per section, its body listing each removal with its reason; format, wording, or categorization changes may be bundled. +- Sponsor placement never influences which projects get listed — see [SPONSORSHIP.md](SPONSORSHIP.md). `website/templates/sponsorship.html` separately defines the sponsorship content on the published website page. diff --git a/CLAUDE.md b/CLAUDE.md index a6a15c1665..5da7ad709b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,31 +1,14 @@ # CLAUDE.md -## Repository Overview +An opinionated guide to the best Python frameworks, libraries, tools, and resources. -An opinionated list of Python frameworks, libraries, tools, and resources. Published at [awesome-python.com](https://awesome-python.com/). +[README.md](README.md) is the single source of content truth; `website/` renders it into the static site: [awesome-python.com](https://awesome-python.com/). -## Entry Guidelines +## Entry Rules -**Refer to [CONTRIBUTING.md](CONTRIBUTING.md)** for acceptance criteria, quality requirements, rejection rules, and entry format. Apply these rules whenever adding or removing an entry, whether reviewing a PR or committing directly. +[CONTRIBUTING.md](CONTRIBUTING.md) holds the admission rules, quality requirements, rejection rules, entry format, and ordering. Apply it whenever adding or removing an entry — direct commits included, not only PR reviews. -## Structure - -- **README.md**: Source of truth. Hierarchical categories with alphabetically ordered entries. -- **CONTRIBUTING.md**: Submission guidelines and review criteria. -- **SPONSORSHIP.md**: Sponsor tiers, placement rules, and the editorial-independence policy. Sponsor content sits in the README header and must never influence which projects get listed. -- **website/**: Static site generator that builds awesome-python.com from README.md. - - `build.py`: Parses README.md and renders HTML via Jinja2 templates. - - `fetch_github_stars.py`: Fetches star counts into `website/data/`. - - `readme_parser.py`: Markdown-to-structured-data parser. - - `templates/`, `static/`: Jinja2 templates and CSS/JS assets. - - `tests/`: Pytest tests for the build pipeline. -- **Makefile**: `make install`, `make build`, `make preview`, `make test`, `make fetch_github_stars`. -- **pyproject.toml**: Uses `uv` for dependency management. Python >=3.13. - -## Key Rules - -- Alphabetical ordering within categories is mandatory. -- Quality over quantity. Only "awesome" projects. -- One project per PR. -- One entry per commit when adding or deleting entries. Format, wording, or categorization changes across multiple entries may be bundled in a single commit. -- README.md is the single source of content truth. +- Every keep/drop reason must be verified against current online data at decision time — download counts, repo activity and archived status, PyPI metadata, project docs. Judging tiers — obvious choice vs challenger — also requires WebSearch evidence (adoption trajectory, community sentiment), not download counts alone. Training-data recollections are not evidence; label anything unverifiable as a judgment call. +- One entry per commit when adding or deleting entries. Exceptions: a prune sweep is one commit per section, its body listing each removal with its reason; format, wording, or categorization changes may be bundled. Cross-section re-homes ride the originating audit's commit (both sides of the move in one diff). +- Resources sections are not project entries: out of audit scope, and the website never parses them. +- Sponsor placement never influences which projects get listed — see [SPONSORSHIP.md](SPONSORSHIP.md). diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000000..434afbc77b --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,63 @@ +# Awesome Python Curation + +The editorial domain of awesome-python: which projects the list carries, how they are admitted, and how they are displaced. The list is a curated shortlist of obvious choices, not a catalog of everything good. + +## Language + +### List structure + +**Entry**: +A single listed project: `- [name](url) - Description.` The unit that is admitted, displaced, or pruned. Named by PyPI package name when one exists, else by repository name. + +**Sub-item**: +An indented awesome-* also-see link under an Entry (awesome-django under django, awesome-pytest under pytest). Not an Entry: it holds no slot, does not count against the Cap, and rides its parent — re-homed or removed with it. Maintainer-only placement, and only for awesome-* lists: a companion project either earns a full Entry in its proper Use Case or is not listed. + +**Thematic Group**: +A bold group line (e.g. "AI & ML", "Web Development") that clusters Sections in the TOC and the Projects body. + +**Section**: +A `###` heading in README.md (e.g. "Testing", "AI and Agents"). Sections group entries and live under a Thematic Group. +_Avoid_: Category (overloaded — the TOC calls them categories, but rules bind to Use Cases, not Sections) + +**Subcategory**: +A named bullet inside a Section with indented entries under it (e.g. "Mock" inside "Testing"). Where present, each Subcategory is one Use Case. + +**Use Case**: +The unit of "it" in "one obvious way to do it": one distinct job a reader needs done. Defined by the list's structure — each Subcategory is a Use Case; a flat Section is a single Use Case. Submitters cannot define Use Cases; only the maintainer changes the structure (see Split). +_Avoid_: Niche, micro-category + +### Admission + +**Serves Python Developers**: +The scope test for what belongs on the list at all: Python developers use it in their Python work. Implementation language and packaging are irrelevant — uv and ty are Rust and belong; a pure-Python library nobody uses in Python work does not. +_Avoid_: Python-first, written-in-Python (old requirement — removed) + +**Obvious Choice**: +An entry an experienced Python developer would name unprompted when asked "what do I use for [Use Case]?". Certified by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars; judgment overrides the signal's known failure modes (CI-inflated counts, model releases consumed as weights rather than pip installs — extended to any project consumed outside pip: SDK downloads like renpy, deployed services like thumbor — and large-but-specific audiences misread as "niche"). A standard-library module holds a slot only when it is itself the Obvious Choice for the Use Case, not merely relevant to it. +_Avoid_: Industry Standard (old lane name), awesome (unfalsifiable) + +**Cap**: +The per-Use-Case entry limit: up to 3 Obvious Choices plus up to 2 Challengers, hard maximum 5. A qualitative bar first, a numeric backstop second — and a ceiling, not a floor: a minted Use Case may hold a single entry. + +**Displacement**: +The only admission path into a full Use Case: the PR names the entry it replaces and argues the newcomer does that entry's job better. One in, one out. +_Avoid_: One-in-one-out (informal alias) + +**Challenger**: +An entry (at most two per Use Case) that is not yet the Obvious Choice but is a credible successor to one. Marked by ordering, not description: within a Use Case, Obvious Choices are listed first, then Challengers, each tier by PyPI downloads/month descending (standard-library modules first in the Use Case outright — always Obvious Choices by the stdlib rule — other no-signal entries last in tier, alphabetically). Admission requires adoption-trajectory evidence, not popularity alone. +_Avoid_: Rising Star (old lane name), Hidden Gem (old lane name — concept removed entirely) + +**Second Tier**: +A Challenger slot holding a demoted incumbent instead of a rising successor — a former Obvious Choice or fading community standard an Audit kept below the current Obvious Choices (clickhouse-driver behind the official client, dogpile.cache and django-cacheops behind cachetools and diskcache, django-haystack in Search). It counts against the two Challenger slots and is marked by position like any Challenger; the adoption-trajectory bar gates only new admissions, not demotions. + +**Override**: +A maintainer decision, made in an Audit and explicitly recorded, that keeps an entry or use-case shape past a written limit — the activity line, the stability bar, a cap, or a structure rule. Case-by-case: an Override never loosens the written rule for any other entry, and submitters cannot cite one. + +**Split**: +Maintainer-only restructuring of an oversized Use Case into finer Use Cases (new Subcategories), considered before any trimming when the size reflects genuinely distinct jobs. A submitter's entry PR can never perform a Split. + +### Maintenance + +**Audit**: +The recurring maintenance pass over one or more Sections: every Entry's verdict re-verified against current evidence, Challengers promoted or demoted, oversized Use Cases Split, drops and admissions re-decided. The maintainer adjudicates through an interactive preview, and entry changes land only on their explicit go. The 2026-08 shortlist-reform sweeps were the first Audits. +_Avoid_: Sweep (reform-era name for a Section's first Audit) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index c3dd6c78f2..e90db05a23 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,41 +1,39 @@ # Contributing +awesome-python is a shortlist, not a catalog. Each use case lists only its obvious choices, and most rejections mean "the use case is full", not "your project is bad". Read this whole page before opening a PR. + ## Quality Requirements All submissions must satisfy **ALL** of these: -1. **Python-first**: Primarily written in Python (>50% of codebase) +1. **Serves Python Developers**: Python developers use it in their Python work. Implementation language and packaging are irrelevant — uv and ty are written in Rust, and agent skill packs are markdown, yet all belong; a pure-Python project nobody uses in Python work does not. 2. **Active**: Commits within the last 12 months 3. **Stable**: Production-ready, not alpha/beta/experimental 4. **Documented**: Clear README with examples and use cases -5. **Unique**: Adds distinct value, not "yet another X" -6. **Established**: Repository at least 1 month old +5. **Established**: Repository at least 1 month old + +## Admission + +A **use case** is one distinct job a reader needs done. Use cases are defined by the list's structure: each subcategory is a use case, and a section without subcategories is a single use case. Structure changes — new sections, new subcategories, splitting an oversized use case into finer ones — are made only by the maintainer; an entry PR can never create the subcategory it needs. + +Each use case lists at most: -## Acceptance Criteria +- **Up to 3 obvious choices** — tools an experienced Python developer would name when asked "what do I use for this?" +- **Up to 2 challengers** — tools that are not yet the obvious choice but are credible successors to one. Admission as a challenger requires adoption-trajectory evidence, not popularity alone. -Your submission must meet **ONE** of the following criteria: +Hard maximum: 5 entries per use case. This is a qualitative bar first and a numeric backstop second — most use cases should carry fewer. -### 1. Industry Standard +**Overrides**: the maintainer may exceed any limit on this page — the caps, the activity requirement, the stability requirement — for a specific entry or use case by explicit decision. An override is case-by-case; it does not loosen these rules for submissions, and citing one in a PR carries no weight. -- The go-to tool that almost everyone uses for a specific use case -- Examples: requests, flask, pandas, numpy -- Limit: 1-3 tools per category +**Displacement**: once a use case is at its cap, the only way in is to name the entry your project replaces and argue that yours does that entry's job better. One in, one out. -### 2. Rising Star +**Dual-listing**: a tool may hold entries in multiple use cases, but only when it earns its slot in each independently. List the full entry in each home with identical lines; never a "see X above" note, since the website only renders list items. Description edits update every copy in the same commit. Each slot is audited on its own: dropping one home keeps the other, and dropping the tool entirely removes all copies in one commit. Dual-listing is a maintainer decision; a PR adding a second home for an existing entry is treated as a duplicate. -- Rapid growth: 5,000+ GitHub stars in less than 1 year -- Significant community buzz and adoption -- Solving problems in new or better ways -- Examples: fastapi, ruff, uv +**Standard library**: a standard-library module is listed only where the stdlib is itself the obvious choice for the use case (tomllib yes, unittest no). -### 3. Hidden Gem +**Evidence**: admission is decided by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars. Judgment overrides the signal's known failure modes (CI-inflated counts, model releases consumed as weights rather than pip installs, large-but-specific audiences misread as "niche"). The maintainer's decision is final. -- Exceptional quality despite fewer stars (100-500 stars preferred; < 100 requires strong justification) -- Solves niche problems elegantly -- Strong recommendation from experienced developers -- **Must demonstrate real-world usage** (not a project published last week) -- Repository must be at least 3 months old with consistent activity -- Must include compelling justification in PR description +Looking for an exhaustive catalog instead? Follow the awesome-\* lists linked under individual entries (for example awesome-python-testing) — they exist precisely so this list doesn't have to be one. ## Entry Format Reference @@ -77,22 +75,28 @@ Use the **PyPI package name** as the display name so developers can copy it dire - [project](url) - Description. ``` -## Adding a New Section +### Entry Ordering + +Within a use case, the obvious choices are listed first, ordered by PyPI downloads per month from high to low; challengers follow, in the same order. Standard-library modules sort first in the use case — built-ins lead, before everything else — alphabetically when there are several. (This never conflicts with tier order: a standard-library module is only listed where it is itself the obvious choice, so it always belongs to the first tier.) Other entries without a download signal (agent skill packs, projects distributed outside PyPI) sort last within their tier, alphabetically. There is no marker in the entry text — position is the marker, so the last entries of a use case may be its challengers. + +## Changing the Structure + +Adding sections or subcategories is maintainer-only (see Admission). For maintainer reference: -1. Add section description in italics: `*Libraries for doing X.*` +1. Add the section description in italics: `*Libraries for doing X.*` 2. Add the section under the appropriate thematic group (e.g., **AI & ML**, **Web**, **Data & Science**) 3. Add the section title to the Table of Contents under its group -4. Keep entries in alphabetical order within each category +4. Order entries per Entry Ordering above ## Review Process PRs are reviewed by automated tools and maintainers: 1. **Format Check**: Entry follows the correct format -2. **Category Check**: Placed in the appropriate category/subcategory +2. **Category Check**: Placed in the appropriate use case 3. **Duplicate Check**: Not already listed or previously rejected 4. **Activity Check**: Project shows recent activity -5. **Quality Check**: Meets acceptance criteria +5. **Admission Check**: Meets the Admission rules above, including Displacement when the use case is at its cap Search previous Pull Requests and Issues before submitting, as yours may be a duplicate. @@ -101,10 +105,12 @@ Search previous Pull Requests and Issues before submitting, as yours may be a du PRs will be **closed** if: - Adding multiple projects in one PR +- The use case is at its cap and the PR makes no Displacement argument +- The PR creates a new section or subcategory and fills it (structure changes are maintainer-only) +- Coordinated multi-entry self-promotion: multiple related projects from the same organization or author, across one or several PRs - Duplicate of existing entry or recently-closed PR - Empty or placeholder PR descriptions - Placed under an inappropriate category - Project is archived or abandoned (no commits in 12+ months) - No documentation or unclear use case -- Less than 100 GitHub stars without Hidden Gem justification -- Repository less than 1 months old +- Repository less than 1 month old diff --git a/Makefile b/Makefile index 6430945e8d..5a07ad9119 100644 --- a/Makefile +++ b/Makefile @@ -7,6 +7,9 @@ install: fetch_github_stars: uv run python website/fetch_github_stars.py +fetch_pypi_downloads: + uv run python website/fetch_pypi_downloads_via_clickpy.py + test: uv run pytest website/tests/ -v @@ -23,10 +26,7 @@ build: uv run python website/build.py preview: build - uv run watchmedo shell-command \ - --patterns='*.md;*.html;*.css;*.js;*.py' \ - --recursive \ - --wait --drop \ - --command='uv run python website/build.py' \ + uv run watchfiles \ + 'uv run python website/build.py' \ README.md website/templates website/static website/data & \ python -m http.server -b 127.0.0.1 -d website/output/ 8000 diff --git a/README.md b/README.md index 19662a181f..f510f538e1 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # [Awesome Python](https://awesome-python.com/) -An opinionated guide to the best Python frameworks, libraries, tools, and resources. +The definitive list that answers "I want to do X in Python, which tool should I use?" - An opinionated guide to the best Python frameworks, libraries, tools, and projects. **Visit the [website](https://awesome-python.com/) to search and filter projects more easily.** @@ -30,6 +30,7 @@ An opinionated guide to the best Python frameworks, libraries, tools, and resour - [Authentication](#authentication) - [Admin Panels](#admin-panels) - [CMS](#cms) +- [ERP](#erp) - [Static Site Generators](#static-site-generators) **HTTP & Scraping** @@ -116,6 +117,7 @@ An opinionated guide to the best Python frameworks, libraries, tools, and resour - [Cryptography](#cryptography) - [Penetration Testing](#penetration-testing) +- [Supply Chain Security](#supply-chain-security) - [Web Security](#web-security) **Other** @@ -134,102 +136,98 @@ _Libraries for building AI applications, LLM integrations, and autonomous agents - Agent Skills - [django-ai-plugins](https://github.com/vintasoftware/django-ai-plugins) - Django backend agent skills for Django, DRF, Celery, and Django-specific code review. - - [graphify](https://github.com/safishamsi/graphify) - Turn any folder of code, SQL schemas, docs, papers, images, or videos into a queryable knowledge graph. - - [nuwa-skill](https://github.com/alchaincyf/nuwa-skill/blob/main/README_EN.md) - Nuwa distills the thinking of anyone — let Musk, Naval, Munger, and Feynman work for you. - [sentry-skills](https://github.com/getsentry/skills) - Python-focused engineering skills for code review, debugging, and backend workflows. - [trailofbits-skills](https://github.com/trailofbits/skills) - Python-friendly security skills for auditing, testing, and safer backend development. - Orchestration - - [ag2](https://github.com/ag2ai/ag2) - An open-source AgentOS for multi-agent orchestration and building agentic AI systems. - - [autogen](https://github.com/microsoft/autogen) - A programming framework for building agentic AI applications. - - [bernstein](https://github.com/sipyourdrink-ltd/bernstein) - A deterministic Python orchestrator for CLI coding agents (Claude Code, Codex, Gemini CLI, and 40+ more) with parallel git worktrees and an HMAC-signed audit chain. - - [bindu](https://github.com/getbindu/Bindu) - A framework that wraps any agent handler with DID-based cryptographic identity, A2A JSON-RPC over HTTP, OAuth2 auth, x402 (USDC) payments, and a built-in operator inbox. - - [bub](https://github.com/bubbuild/bub) - A lightweight, hook-first Python framework for channel-native agents that live alongside people. - - [crewai](https://github.com/crewAIInc/crewAI) - A framework for orchestrating role-playing autonomous AI agents for collaborative task solving. - - [dspy](https://github.com/stanfordnlp/dspy) - A framework for programming, not prompting, language models. - - [hermes-agent](https://github.com/nousresearch/hermes-agent) - An adaptive AI agent framework that grows with you. - [langchain](https://github.com/langchain-ai/langchain) - Building applications with LLMs through composability. - [langgraph](https://github.com/langchain-ai/langgraph) - Low-level orchestration framework for building stateful, long-running LLM agents. - - [livetalking](https://github.com/lipku/livetalking) - A framework for real time interactive streaming digital human. - - [openai-agents](https://github.com/openai/openai-agents-python) - OpenAI's framework for building and managing AI agents. - - [OpenChronicle](https://github.com/Einsia/OpenChronicle) - Open-source, local-first memory for any tool-capable LLM agent. - - [promptise](https://github.com/promptise-com/foundry) - A framework for building end-to-end production-ready agentic systems, scalable & secure MCP's and autonomous agents. + - [crewai](https://github.com/crewAIInc/crewAI) - A framework for orchestrating role-playing autonomous AI agents for collaborative task solving. - [pydantic-ai](https://github.com/pydantic/pydantic-ai) - A Python agent framework for building generative AI applications with structured schemas. - - [smolagents](https://github.com/huggingface/smolagents) - A lightweight library to build agents and multi-agent workflows with minimal code. - - [TradingAgents](https://github.com/TauricResearch/TradingAgents) - A multi-agents LLM financial trading framework. +- Vendor Agent SDKs + - [openai-agents](https://github.com/openai/openai-agents-python) - OpenAI's framework for building and managing AI agents. + - [claude-agent-sdk](https://github.com/anthropics/claude-agent-sdk-python) - Anthropic's Python SDK for building AI agents on Claude Code's harness — custom tools, in-process MCP servers, hooks. +- Personal Assistants + - [hermes-agent](https://github.com/nousresearch/hermes-agent) - An adaptive personal AI assistant that grows with you. +- Prompt Optimization + - [dspy](https://github.com/stanfordnlp/dspy) - A framework for programming, not prompting, language models. - Data Layer - - [entroly](https://github.com/juyterman1000/entroly) - An auditable context control plane that optimizes prompt context, stabilizes cache prefixes, and verifies answers locally with WITNESS. - [instructor](https://github.com/567-labs/instructor) - A library for extracting structured data from LLMs, powered by Pydantic. - [llama-index](https://github.com/run-llama/llama_index) - A data framework for your LLM application. - - [lumen](https://github.com/holoviz/lumen) - An agent framework for conversational analysis of SQL, dataframe and multidimensional array data. - [mem0](https://github.com/mem0ai/mem0) - An intelligent memory layer for AI agents enabling personalized interactions. - - [outlines](https://github.com/dottxt-ai/outlines) - Structured text generation for LLMs with JSON schema, regex, and grammar-constrained decoding. -- Pre-trained Models and Inference - - [diffusers](https://github.com/huggingface/diffusers) - A library that provides pre-trained diffusion models for generating and editing images, audio, and video. - - [LiteLLM](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format. - - [liter-llm](https://github.com/xberg-io/liter-llm) - A universal LLM API client with a Rust core, calling 142+ providers through a single OpenAI-compatible interface, with a built-in proxy and MCP server. - - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. - - [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models. +- Pre-trained Models - [transformers](https://github.com/huggingface/transformers) - A framework that lets you easily use pre-trained transformer models for NLP, vision, and audio tasks. - - [unsloth](https://github.com/unslothai/unsloth) - A library for faster LLM fine-tuning and training with reduced memory usage. +- LLM Inference and Serving + - [sglang](https://github.com/sgl-project/sglang) - A high-performance serving framework for large language models and multimodal models. - [vllm](https://github.com/vllm-project/vllm) - A high-throughput and memory-efficient inference and serving engine for LLMs. + - [mlx-lm](https://github.com/ml-explore/mlx-lm) - Run and fine-tune large language models on Apple Silicon with MLX. +- LLM Gateways + - [LiteLLM](https://github.com/BerriAI/litellm) - Call 100+ LLMs using OpenAI format. +- Image and Video Generation + - [diffusers](https://github.com/huggingface/diffusers) - A library that provides pre-trained diffusion models for generating and editing images, audio, and video. +- Fine-tuning + - [unsloth](https://github.com/unslothai/unsloth) - A library for faster LLM fine-tuning and training with reduced memory usage. - Speech - - [funasr](https://github.com/modelscope/FunASR) - Industrial-grade speech recognition toolkit with 170x realtime speed, 50+ languages, speaker diarization, and emotion detection. - [openai-whisper](https://github.com/openai/whisper) - A general-purpose automatic speech recognition model trained on 680k hours of multilingual and multitask supervised data. - - [SenseVoice](https://github.com/FunAudioLLM/SenseVoice) - A multilingual speech understanding model for speech recognition, language identification, emotion recognition, and audio event detection. + - [funasr](https://github.com/modelscope/FunASR) - Industrial-grade speech recognition toolkit with 170x realtime speed, 50+ languages, speaker diarization, and emotion detection. - [vibevoice](https://github.com/microsoft/VibeVoice) - A family of open-source voice AI models from Microsoft for text-to-speech and long-form speech recognition. - - [voxcpm](https://github.com/OpenBMB/VoxCPM) - A tokenizer-free text-to-speech foundation model for multilingual speech generation and voice cloning. + - [gTTS](https://github.com/pndurette/gTTS) - Python library and CLI tool for converting text to speech using Google Translate TTS. + - [kittentts](https://github.com/KittenML/KittenTTS) - Lightweight ONNX text-to-speech library with small CPU-friendly models. ### Deep Learning _Frameworks for Neural Networks and Deep Learning. Also see [awesome-deep-learning](https://github.com/ChristosChristofidis/awesome-deep-learning)._ -- [jax](https://github.com/jax-ml/jax) - A library for high-performance numerical computing with automatic differentiation and JIT compilation. -- [keras](https://github.com/keras-team/keras) - A high-level deep learning library with support for JAX, TensorFlow, and PyTorch backends. -- [pytorch](https://github.com/pytorch/pytorch) - Tensors and Dynamic neural networks in Python with strong GPU acceleration. -- [pytorch-lightning](https://github.com/Lightning-AI/pytorch-lightning) - Deep learning framework to train, deploy, and ship AI products Lightning fast. -- [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) - PyTorch implementations of Stable Baselines (deep) reinforcement learning algorithms. -- [tensorflow](https://github.com/tensorflow/tensorflow) - The most popular Deep Learning framework created by Google. +- Frameworks + - [pytorch](https://github.com/pytorch/pytorch) - Tensors and Dynamic neural networks in Python with strong GPU acceleration. + - [tensorflow](https://github.com/tensorflow/tensorflow) - The most popular Deep Learning framework created by Google. + - [keras](https://github.com/keras-team/keras) - A high-level deep learning library with support for JAX, TensorFlow, and PyTorch backends. + - [jax](https://github.com/jax-ml/jax) - A library for high-performance numerical computing with automatic differentiation and JIT compilation. + - [pytorch-lightning](https://github.com/Lightning-AI/pytorch-lightning) - Deep learning framework to train, deploy, and ship AI products Lightning fast. +- Reinforcement Learning + - [gymnasium](https://github.com/Farama-Foundation/Gymnasium) - A standard API for reinforcement learning environments with popular reference environments ([gym](https://github.com/openai/gym) successor). + - [stable-baselines3](https://github.com/DLR-RM/stable-baselines3) - PyTorch implementations of Stable Baselines (deep) reinforcement learning algorithms. ### Machine Learning _Libraries for Machine Learning. Also see [awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning#python)._ -- [catboost](https://github.com/catboost/catboost) - A fast, scalable, high performance gradient boosting on decision trees library. -- [feature_engine](https://github.com/feature-engine/feature_engine) - sklearn compatible API with the widest toolset for feature engineering and selection. -- [h2o](https://github.com/h2oai/h2o-3) - Open Source Fast Scalable Machine Learning Platform. -- [lightgbm](https://github.com/lightgbm-org/LightGBM) - A fast, distributed, high performance gradient boosting framework. -- [mindsdb](https://github.com/mindsdb/minds) - MindsDB is an open source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning models using standard queries. -- [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for probabilistic graphical models and Bayesian networks. -- [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning with extensive documentation and community support. -- [scikit-lego](https://github.com/koaning/scikit-lego) - A collection of lego bricks for scikit-learn pipelines. -- [spark.ml](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/)'s scalable [Machine Learning library](https://spark.apache.org/docs/latest/ml-guide.html) for distributed computing. -- [TabGAN](https://github.com/Diyago/Tabular-data-generation) - Synthetic tabular data generation using GANs, Diffusion Models, and LLMs. -- [timesfm](https://github.com/google-research/timesfm) - A pretrained foundation model from Google Research for time-series forecasting. -- [xgboost](https://github.com/dmlc/xgboost) - A scalable, portable, and distributed gradient boosting library. +- General + - [scikit-learn](https://github.com/scikit-learn/scikit-learn) - The most popular Python library for Machine Learning with extensive documentation and community support. + - [pgmpy](https://github.com/pgmpy/pgmpy) - A Python library for probabilistic graphical models and Bayesian networks. + - [feature-engine](https://github.com/feature-engine/feature_engine) - sklearn compatible API with the widest toolset for feature engineering and selection. +- Gradient Boosting + - [xgboost](https://github.com/dmlc/xgboost) - A scalable, portable, and distributed gradient boosting library. + - [lightgbm](https://github.com/lightgbm-org/LightGBM) - A fast, distributed, high performance gradient boosting framework. + - [catboost](https://github.com/catboost/catboost) - A fast, scalable, high performance gradient boosting on decision trees library. +- Time Series Forecasting + - [timesfm](https://github.com/google-research/timesfm) - A pretrained foundation model from Google Research for time-series forecasting. ### Natural Language Processing _Libraries for working with human languages._ - General - - [gensim](https://github.com/piskvorky/gensim) - Topic Modeling for Humans. - [nltk](https://github.com/nltk/nltk) - A leading platform for building Python programs to work with human language data. - [spacy](https://github.com/explosion/spaCy) - A library for industrial-strength natural language processing in Python and Cython. + - [gensim](https://github.com/piskvorky/gensim) - Topic Modeling for Humans. - [stanza](https://github.com/stanfordnlp/stanza) - The Stanford NLP Group's official Python library, supporting 60+ languages. - Chinese - - [funnlp](https://github.com/fighting41love/funNLP) - A collection of tools and datasets for Chinese NLP. - [jieba](https://github.com/fxsjy/jieba) - The most popular Chinese text segmentation library. + - [pypinyin](https://github.com/mozillazg/python-pinyin) - Convert Chinese hanzi (漢字) to pinyin (拼音). + - [pangu.py](https://github.com/vinta/pangu.py) - Paranoid text spacing. ### Computer Vision _Libraries for Computer Vision._ -- [easyocr](https://github.com/JaidedAI/EasyOCR) - Ready-to-use OCR with 40+ languages supported. -- [fiftyone](https://github.com/voxel51/fiftyone) - The open-source tool for building high-quality datasets and computer vision models. -- [kornia](https://github.com/kornia/kornia/) - Open Source Differentiable Computer Vision Library for PyTorch. -- [opencv](https://github.com/opencv/opencv-python) - Open Source Computer Vision Library. -- [pytesseract](https://github.com/madmaze/pytesseract) - A wrapper for [Google Tesseract OCR](https://github.com/tesseract-ocr). -- [ultralytics](https://github.com/ultralytics/ultralytics) - Ultralytics YOLO for object detection, segmentation, pose estimation, and classification with state-of-the-art accuracy and speed. +- General + - [opencv-python](https://github.com/opencv/opencv-python) - Open Source Computer Vision Library. + - [ultralytics](https://github.com/ultralytics/ultralytics) - Ultralytics YOLO for object detection, segmentation, pose estimation, and classification with state-of-the-art accuracy and speed. + - [kornia](https://github.com/kornia/kornia/) - Open Source Differentiable Computer Vision Library for PyTorch. + - [fiftyone](https://github.com/voxel51/fiftyone) - The open-source tool for building high-quality datasets and computer vision models. +- OCR + - [pytesseract](https://github.com/madmaze/pytesseract) - A wrapper for [Google Tesseract OCR](https://github.com/tesseract-ocr). + - [easyocr](https://github.com/JaidedAI/EasyOCR) - Ready-to-use OCR with 40+ languages supported. ### Recommender Systems @@ -246,69 +244,59 @@ _Libraries for building recommender systems._ _Traditional full stack web frameworks. Also see [Web APIs](#web-apis)._ - Synchronous - - [bottle](https://github.com/bottlepy/bottle) - A fast and simple micro-framework distributed as a single file with no dependencies. - - [django](https://github.com/django/django) - The most popular web framework in Python. - - [awesome-django](https://github.com/wsvincent/awesome-django) - [flask](https://github.com/pallets/flask) - A microframework for Python. - [awesome-flask](https://github.com/humiaozuzu/awesome-flask) + - [django](https://github.com/django/django) - The most popular web framework in Python. + - [awesome-django](https://github.com/wsvincent/awesome-django) + - [bottle](https://github.com/bottlepy/bottle) - A fast and simple micro-framework distributed as a single file with no dependencies. - [pyramid](https://github.com/Pylons/pyramid) - A small, fast, down-to-earth, open source Python web framework. - [awesome-pyramid](https://github.com/uralbash/awesome-pyramid) - [fasthtml](https://github.com/AnswerDotAI/fasthtml) - The fastest way to create an HTML app. - [awesome-fasthtml](https://github.com/amosgyamfi/awesome-fasthtml) - - [masonite](https://github.com/MasoniteFramework/masonite) - The modern and developer centric Python web framework. - Asynchronous - - [litestar](https://github.com/litestar-org/litestar) - Production-ready, capable and extensible ASGI Web framework. - - [microdot](https://github.com/miguelgrinberg/microdot) - The impossibly small web framework for Python and MicroPython. - - [reflex](https://github.com/reflex-dev/reflex) - A framework for building reactive, full-stack web applications entirely with Python. - - [robyn](https://github.com/sparckles/Robyn) - A high-performance async Python web framework with a Rust runtime. - [starlette](https://github.com/Kludex/starlette) - A lightweight ASGI framework and toolkit for building high-performance async services. - [tornado](https://github.com/tornadoweb/tornado) - A web framework and asynchronous networking library. + - [litestar](https://github.com/litestar-org/litestar) - Production-ready, capable and extensible ASGI Web framework. + - [reflex](https://github.com/reflex-dev/reflex) - A framework for building reactive, full-stack web applications entirely with Python. ### Web APIs -_Libraries for building RESTful and GraphQL APIs._ +_Libraries for building RESTful, GraphQL, and RPC APIs._ - Django - - [django-modern-rest](https://github.com/wemake-services/django-modern-rest) - Modern REST with speed, types, async, `msgspec`, `pydantic` and other goodies! - - [django-ninja](https://github.com/vitalik/django-ninja) - Fast, Django REST framework based on type hints and Pydantic. - [django-rest-framework](https://github.com/encode/django-rest-framework) - A powerful and flexible toolkit to build web APIs. + - [django-ninja](https://github.com/vitalik/django-ninja) - Fast, Django REST framework based on type hints and Pydantic. - [strawberry-django](https://github.com/strawberry-graphql/strawberry-django) - Strawberry GraphQL integration with Django. + - [django-modern-rest](https://github.com/wemake-services/django-modern-rest) - Modern REST with speed, types, async, `msgspec`, `pydantic` and other goodies! - Flask - [apiflask](https://github.com/apiflask/apiflask) - A lightweight Python web API framework based on Flask and Marshmallow. - Framework Agnostic - - [connexion](https://github.com/spec-first/connexion) - A spec-first framework that automatically handles requests based on your OpenAPI specification. - - [falcon](https://github.com/falconry/falcon) - A high-performance framework for building cloud APIs and web app backends. - [fastapi](https://github.com/fastapi/fastapi) - A modern, fast, web framework for building APIs with standard Python type hints. - - [sanic](https://github.com/sanic-org/sanic) - A Python web server and web framework that's written to go fast. + - [connexion](https://github.com/spec-first/connexion) - A spec-first framework that automatically handles requests based on your OpenAPI specification. - [strawberry](https://github.com/strawberry-graphql/strawberry) - A GraphQL library that leverages Python type annotations for schema definition. - - [webargs](https://github.com/marshmallow-code/webargs) - A friendly library for parsing HTTP request arguments with built-in support for popular web frameworks. +- RPC + - [grpcio](https://github.com/grpc/grpc) - HTTP/2-based RPC framework with Python bindings, built by Google. ### Web Servers _ASGI and WSGI compatible web servers._ - ASGI - - [daphne](https://github.com/django/daphne) - An HTTP, HTTP/2 and WebSocket protocol server for ASGI and ASGI-HTTP. + - [uvicorn](https://github.com/Kludex/uvicorn) - A lightning-fast ASGI server implementation, using uvloop and httptools. - [granian](https://github.com/emmett-framework/granian) - A Rust HTTP server for Python applications built on top of Hyper and Tokio, supporting WSGI/ASGI/RSGI. - [hypercorn](https://github.com/pgjones/hypercorn) - An ASGI and WSGI Server based on Hyper libraries and inspired by Gunicorn. - - [uvicorn](https://github.com/Kludex/uvicorn) - A lightning-fast ASGI server implementation, using uvloop and httptools. - WSGI - [gunicorn](https://github.com/benoitc/gunicorn) - Pre-forked, ported from Ruby's Unicorn project. - - [uwsgi](https://github.com/unbit/uwsgi) - A project aims at developing a full stack for building hosting services, written in C. - [waitress](https://github.com/Pylons/waitress) - Multi-threaded, powers Pyramid. -- RPC - - [grpcio](https://github.com/grpc/grpc) - HTTP/2-based RPC framework with Python bindings, built by Google. - - [rpyc](https://github.com/tomerfiliba-org/rpyc) (Remote Python Call) - A transparent and symmetric RPC library for Python. ### WebSocket _Libraries for working with WebSocket._ -- [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and [asyncio](https://docs.python.org/3/library/asyncio.html). +- [websockets](https://github.com/python-websockets/websockets) - A library for building WebSocket servers and clients with a focus on correctness and simplicity. - [channels](https://github.com/django/channels) - Developer-friendly asynchrony for Django. - [flask-socketio](https://github.com/miguelgrinberg/Flask-SocketIO) - Socket.IO integration for Flask applications. -- [picows](https://github.com/tarasko/picows) - Fastest WebSocket clients and servers with a frame level interface for the most demanding use-cases. -- [websockets](https://github.com/python-websockets/websockets) - A library for building WebSocket servers and clients with a focus on correctness and simplicity. +- [autobahn-python](https://github.com/crossbario/autobahn-python) - WebSocket & WAMP for Python on Twisted and [asyncio](https://docs.python.org/3/library/asyncio.html). ### Template Engines @@ -319,20 +307,20 @@ _Libraries and tools for templating and lexing._ ### Web Asset Management -_Tools for managing, compressing and minifying website assets._ +_Tools for managing, storing, compressing and minifying website assets._ -- [django-compressor](https://github.com/django-compressor/django-compressor) - Compresses linked and inline JavaScript or CSS into a single cached file. - [django-storages](https://github.com/jschneier/django-storages) - A collection of custom storage back ends for Django. +- [django-compressor](https://github.com/django-compressor/django-compressor) - Compresses linked and inline JavaScript or CSS into a single cached file. ### Authentication _Libraries for implementing authentication schemes._ - OAuth + - [oauthlib](https://github.com/oauthlib/oauthlib) - A generic and thorough implementation of the OAuth request-signing logic. - [authlib](https://github.com/authlib/authlib) - A comprehensive library for building OAuth, OpenID Connect, and JWT/JWS/JWE/JWK/JWA. - [django-allauth](https://github.com/pennersr/django-allauth) - Authentication app for Django that "just works." - [django-oauth-toolkit](https://github.com/django-oauth/django-oauth-toolkit) - OAuth 2 goodies for Django. - - [oauthlib](https://github.com/oauthlib/oauthlib) - A generic and thorough implementation of the OAuth request-signing logic. - JWT - [pyjwt](https://github.com/jpadilla/pyjwt) - JSON Web Token implementation in Python. - Permissions @@ -343,29 +331,29 @@ _Libraries for implementing authentication schemes._ _Libraries for administrative interfaces._ -- [ajenti](https://github.com/ajenti/ajenti) - The admin panel your servers deserve. -- [django-grappelli](https://github.com/sehmaschine/django-grappelli) - A jazzy skin for the Django Admin-Interface. -- [django-unfold](https://github.com/unfoldadmin/django-unfold) - Elevate your Django admin with a stunning modern interface, powerful features, and seamless user experience. - [flask-admin](https://github.com/pallets-eco/flask-admin) - Simple and extensible administrative interface framework for Flask. -- [flower](https://github.com/mher/flower) - Real-time monitor and web admin for Celery. -- [func-to-web](https://github.com/offerrall/FuncToWeb) - Instantly create web UIs from Python functions using type hints. Zero frontend code required. -- [jet-bridge](https://github.com/jet-admin/jet-bridge) - Admin panel framework for any application with nice UI (ex Jet Django). +- [django-unfold](https://github.com/unfoldadmin/django-unfold) - Elevate your Django admin with a stunning modern interface, powerful features, and seamless user experience. +- [django-grappelli](https://github.com/sehmaschine/django-grappelli) - A jazzy skin for the Django Admin-Interface. ### CMS _Content Management Systems._ -- [django-cms](https://github.com/django-cms/django-cms) - The easy-to-use and developer-friendly enterprise CMS powered by Django. -- [indico](https://github.com/indico/indico) - A feature-rich event management system, made @ [CERN](https://en.wikipedia.org/wiki/CERN). - [wagtail](https://github.com/wagtail/wagtail) - A Django content management system. +- [django-cms](https://github.com/django-cms/django-cms) - The easy-to-use and developer-friendly enterprise CMS powered by Django. + +### ERP + +_Enterprise resource planning frameworks._ + +- [odoo](https://github.com/odoo/odoo) - A suite of open source business apps: CRM, e-commerce, accounting, inventory, and thousands of community modules. ### Static Site Generators _Static site generator is a software that takes some text + templates as input and produces HTML files on the output._ -- [lektor](https://github.com/lektor/lektor) - An easy to use static CMS and blog engine. -- [nikola](https://github.com/getnikola/nikola) - A static website and blog generator. - [pelican](https://github.com/getpelican/pelican) - Static site generator that supports Markdown and reST syntax. +- [nikola](https://github.com/getnikola/nikola) - A static website and blog generator. **HTTP & Scraping** @@ -373,12 +361,14 @@ _Static site generator is a software that takes some text + templates as input a _Libraries for working with HTTP._ -- [aiohttp](https://github.com/aio-libs/aiohttp) - Asynchronous HTTP client/server framework for asyncio and Python. -- [furl](https://github.com/gruns/furl) - A small Python library that makes parsing and manipulating URLs easy. -- [httptap](https://github.com/ozeranskii/httptap) - Dissects an HTTP request into DNS, TCP, TLS, wait, and transfer phases and renders the timings as a waterfall. -- [httpx](https://github.com/encode/httpx) - A next generation HTTP client for Python. -- [requests](https://github.com/psf/requests) - HTTP Requests for Humans. -- [urllib3](https://github.com/urllib3/urllib3) - A HTTP library with thread-safe connection pooling, file post support, sanity friendly. +- Clients + - [requests](https://github.com/psf/requests) - HTTP Requests for Humans. + - [httpx](https://github.com/encode/httpx) - A next generation HTTP client for Python. + - [aiohttp](https://github.com/aio-libs/aiohttp) - Asynchronous HTTP client/server framework for asyncio and Python. + - [urllib3](https://github.com/urllib3/urllib3) - A HTTP library with thread-safe connection pooling, file post support, sanity friendly. +- URL Manipulation + - [yarl](https://github.com/aio-libs/yarl) - Yet another URL library. + - [httpx.URL](https://www.python-httpx.org/api/) - The immutable URL class bundled with HTTPX. ### Web Scraping @@ -386,23 +376,17 @@ _Libraries to automate web scraping and extract web content._ - Frameworks - [browser-use](https://github.com/browser-use/browser-use) - Make websites accessible for AI agents with easy browser automation. - - [crawl4ai](https://github.com/unclecode/crawl4ai) - An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents. - - [crawlberg](https://github.com/xberg-io/crawlberg) - A high-performance web crawling engine with a Rust core, headless-browser fallback, and built-in robots.txt and sitemap parsing. - - [mechanicalsoup](https://github.com/MechanicalSoup/MechanicalSoup) - A Python library for automating interaction with websites. - [scrapy](https://github.com/scrapy/scrapy) - A fast high-level screen scraping and web crawling framework. - - [website-downloader](https://github.com/PKHarsimran/website-downloader) - A modern wget --mirror / HTTrack alternative that turns whole websites into browsable offline copies. + - [crawl4ai](https://github.com/unclecode/crawl4ai) - An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents. - Content Extraction - [feedparser](https://github.com/kurtmckee/feedparser) - Universal feed parser. - [html2text](https://github.com/Alir3z4/html2text) - Convert HTML to Markdown-formatted text. - - [micawber](https://github.com/coleifer/micawber) - A small library for extracting rich content from URLs. - - [sumy](https://github.com/miso-belica/sumy) - A module for automatic summarization of text documents and HTML pages. - [trafilatura](https://github.com/adbar/trafilatura) - A tool for gathering text and metadata from the web, with built-in content filtering. ### Email _Libraries for sending and parsing email, and mail server management._ -- [modoboa](https://github.com/modoboa/modoboa) - A mail hosting and management platform including a modern Web UI. - [yagmail](https://github.com/kootenpv/yagmail) - Yet another Gmail/SMTP client. **Database & Storage** @@ -412,77 +396,82 @@ _Libraries for sending and parsing email, and mail server management._ _Libraries that implement Object-Relational Mapping or data mapping techniques._ - Relational Databases - - [django.db.models](https://github.com/django/django) - The Django [ORM](https://docs.djangoproject.com/en/dev/topics/db/models/). - [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) - The Python SQL Toolkit and Object Relational Mapper. - [awesome-sqlalchemy](https://github.com/dahlia/awesome-sqlalchemy) - - [dataset](https://github.com/pudo/dataset) - Store Python dicts in a database - works with SQLite, MySQL, and PostgreSQL. + - [django.db.models](https://github.com/django/django) - The Django [ORM](https://docs.djangoproject.com/en/dev/topics/db/models/). - [peewee](https://github.com/coleifer/peewee) - A small, expressive ORM. - - [pony](https://github.com/ponyorm/pony/) - ORM that provides a generator-oriented interface to SQL. - [sqlmodel](https://github.com/fastapi/sqlmodel) - SQLModel is based on Python type annotations, and powered by Pydantic and SQLAlchemy. - - [tortoise-orm](https://github.com/tortoise/tortoise-orm) - An easy-to-use asyncio ORM inspired by Django, with relations support. - NoSQL Databases - - [beanie](https://github.com/BeanieODM/beanie) - An asynchronous Python object-document mapper (ODM) for MongoDB. - - [mongoengine](https://github.com/MongoEngine/mongoengine) - A Python Object-Document-Mapper for working with MongoDB. - [pynamodb](https://github.com/pynamodb/PynamoDB) - A Pythonic interface for [Amazon DynamoDB](https://aws.amazon.com/dynamodb/). + - [mongoengine](https://github.com/MongoEngine/mongoengine) - A Python Object-Document-Mapper for working with MongoDB. + - [beanie](https://github.com/BeanieODM/beanie) - An asynchronous Python object-document mapper (ODM) for MongoDB. ### Database Drivers _Libraries for connecting and operating databases._ - MySQL - [awesome-mysql](https://github.com/shlomi-noach/awesome-mysql) - - [mysqlclient](https://github.com/PyMySQL/mysqlclient) - MySQL connector with Python 3 support ([mysql-python](https://sourceforge.net/projects/mysql-python/) fork). - [pymysql](https://github.com/PyMySQL/PyMySQL) - A pure Python MySQL driver compatible to mysql-python. + - [mysqlclient](https://github.com/PyMySQL/mysqlclient) - MySQL connector with Python 3 support ([mysql-python](https://sourceforge.net/projects/mysql-python/) fork). - PostgreSQL - [awesome-postgres](https://github.com/dhamaniasad/awesome-postgres) - [psycopg](https://github.com/psycopg/psycopg) - The most popular PostgreSQL adapter for Python. + - [asyncpg](https://github.com/MagicStack/asyncpg) - A fast PostgreSQL Database Client Library for Python/asyncio. - SQLite - [awesome-sqlite](https://github.com/planetopendata/awesome-sqlite) - - [sqlite-utils](https://github.com/simonw/sqlite-utils) - Python CLI utility and library for manipulating SQLite databases. - [sqlite3](https://docs.python.org/3/library/sqlite3.html) - (Python standard library) SQLite interface compliant with DB-API 2.0. -- Other Relational Databases + - [sqlite-utils](https://github.com/simonw/sqlite-utils) - Python CLI utility and library for manipulating SQLite databases. +- ClickHouse + - [clickhouse-connect](https://github.com/ClickHouse/clickhouse-connect) - The official ClickHouse client, with SQLAlchemy and Superset connectors. - [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) - Python driver with native interface for ClickHouse. +- Other Relational Databases + - [pyodbc](https://github.com/mkleehammer/pyodbc) - An ODBC bridge for connecting to SQL Server and any other ODBC-accessible database. + - [oracledb](https://github.com/oracle/python-oracledb) - The official Python driver for Oracle Database, successor to cx_Oracle. - [mssql-python](https://github.com/microsoft/mssql-python) - Official Microsoft driver for SQL Server and Azure SQL, built on ODBC for high performance and low memory usage. - NoSQL Databases + - [redis](https://github.com/redis/redis-py) - The Python client for Redis. + - [pymongo](https://github.com/mongodb/mongo-python-driver) - The official Python client for MongoDB. - [cassandra-driver](https://github.com/apache/cassandra-python-driver) - The Python Driver for Apache Cassandra. - [django-mongodb-backend](https://github.com/mongodb/django-mongodb-backend) - Official MongoDB database backend for Django. - - [pymongo](https://github.com/mongodb/mongo-python-driver) - The official Python client for MongoDB. - - [redis-py](https://github.com/redis/redis-py) - The Python client for Redis. ### Database -_Databases implemented in Python._ +_In-process databases usable directly from Python._ -- [chdb](https://github.com/chdb-io/chdb) - In-process OLAP SQL engine with the full ClickHouse dialect, zero-copy pandas/Arrow interop, and federation to remote ClickHouse clusters via `remoteSecure()`. -- [chromadb](https://github.com/chroma-core/chroma) - An open-source embedding database for building AI applications with embeddings and semantic search. -- [duckdb](https://github.com/duckdb/duckdb) - An in-process SQL OLAP database management system; optimized for analytics and fast queries, similar to SQLite but for analytical workloads. -- [pickledb](https://github.com/patx/pickledb) - A simple and lightweight key-value store for Python. -- [tinydb](https://github.com/msiemens/tinydb) - A tiny, document-oriented database. -- [ZODB](https://github.com/zopefoundation/ZODB) - A native object database for Python. A key-value and object graph database. -- [zvec](https://github.com/alibaba/zvec) - An embedded vector database for on-device RAG and edge AI, the SQLite of vector databases. +- Analytical + - [duckdb](https://github.com/duckdb/duckdb) - An in-process SQL OLAP database management system; optimized for analytics and fast queries, similar to SQLite but for analytical workloads. + - [chdb](https://github.com/chdb-io/chdb) - In-process OLAP SQL engine with the full ClickHouse dialect, zero-copy pandas/Arrow interop, and federation to remote ClickHouse clusters via `remoteSecure()`. +- Vector + - [chromadb](https://github.com/chroma-core/chroma) - An open-source embedding database for building AI applications with embeddings and semantic search. + - [lancedb](https://github.com/lancedb/lancedb) - A developer-friendly embedded retrieval database for multimodal AI. + - [zvec](https://github.com/alibaba/zvec) - An embedded vector database for on-device RAG and edge AI, the SQLite of vector databases. +- Key-Value & Document + - [tinydb](https://github.com/msiemens/tinydb) - A tiny, document-oriented database. ### Caching _Libraries for caching data._ - [cachetools](https://github.com/tkem/cachetools) - Extensible memoizing collections and decorators. -- [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. +- [diskcache](https://github.com/grantjenks/python-diskcache) - SQLite and file backed cache backend with faster lookups than memcached and redis. - [dogpile.cache](https://github.com/sqlalchemy/dogpile.cache) - dogpile.cache is a next generation replacement for Beaker made by the same authors. -- [python-diskcache](https://github.com/grantjenks/python-diskcache) - SQLite and file backed cache backend with faster lookups than memcached and redis. +- [django-cacheops](https://github.com/Suor/django-cacheops) - A slick ORM cache with automatic granular event-driven invalidation. ### Search _Libraries and software for indexing and performing search queries on data._ +- [elasticsearch](https://github.com/elastic/elasticsearch-py) - The official low-level Python client for [Elasticsearch](https://www.elastic.co/products/elasticsearch). +- [opensearch-py](https://github.com/opensearch-project/opensearch-py) - The official low-level Python client for [OpenSearch](https://opensearch.org/). +- [meilisearch](https://github.com/meilisearch/meilisearch-python) - The official Python client for the [Meilisearch](https://www.meilisearch.com/) search engine. - [django-haystack](https://github.com/django-haystack/django-haystack) - Modular search for Django. -- [elasticsearch-py](https://github.com/elastic/elasticsearch-py) - The official low-level Python client for [Elasticsearch](https://www.elastic.co/products/elasticsearch). -- [pysolr](https://github.com/django-haystack/pysolr) - A lightweight Python wrapper for [Apache Solr](https://lucene.apache.org/solr/). ### Serialization _Libraries for serializing complex data types._ -- [marshmallow](https://github.com/marshmallow-code/marshmallow) - A lightweight library for converting complex objects to and from simple Python datatypes. - [msgpack](https://github.com/msgpack/msgpack-python) - MessagePack serializer implementation for Python. -- [msgspec](https://github.com/msgspec/msgspec) - A fast serialization and validation library with built-in support for JSON, MessagePack, YAML, and TOML. - [orjson](https://github.com/ijl/orjson) - Fast, correct JSON library. +- [marshmallow](https://github.com/marshmallow-code/marshmallow) - A lightweight library for converting complex objects to and from simple Python datatypes. +- [msgspec](https://github.com/msgspec/msgspec) - A fast serialization and validation library with built-in support for JSON, MessagePack, YAML, and TOML. **Data & Science** @@ -490,113 +479,97 @@ _Libraries for serializing complex data types._ _Libraries for data analysis._ -- [aws-sdk-pandas](https://github.com/aws/aws-sdk-pandas) - Pandas on AWS. -- [data-profiling](https://github.com/Data-Centric-AI-Community/fg-data-profiling) - Generate detailed data profiling reports for pandas DataFrames. -- [datasette](https://github.com/simonw/datasette) - An open source multi-tool for exploring and publishing data. -- [desbordante](https://github.com/desbordante/desbordante-core/) - An open source data profiler for complex pattern discovery. -- [ibis](https://github.com/ibis-project/ibis) - A portable Python dataframe library with a single API for 20+ backends. -- [modin](https://github.com/modin-project/modin) - A drop-in pandas replacement that scales workflows by changing a single line of code. - [pandas](https://github.com/pandas-dev/pandas) - A library providing high-performance, easy-to-use data structures and data analysis tools. -- [pathway](https://github.com/pathwaycom/pathway) - Real-time data processing framework for Python with reactive dataflows. - -- [percentify](https://github.com/data-centt/percentify) - Exploratory stats and data-quality diagnostics/profiler, that supports both pandas and Polars DataFrames inputs. - - [polars](https://github.com/pola-rs/polars) - A fast DataFrame library implemented in Rust with a Python API. +- [ibis-framework](https://github.com/ibis-project/ibis) - A portable Python dataframe library with a single API for 20+ backends. ### Data Ingestion / ETL _Libraries for data extraction, transformation, and loading pipelines across multiple sources and destinations._ - General + - [awswrangler](https://github.com/aws/aws-sdk-pandas) - Pandas integration with AWS services like Athena, Glue, Redshift, S3, and DynamoDB. - [dlt](https://github.com/dlt-hub/dlt) - A Python library for building data pipelines with automatic schema inference, incremental loading, and support for multiple sources and destinations. + - [pathway](https://github.com/pathwaycom/pathway) - Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG. - Financial Data + - [yfinance](https://github.com/ranaroussi/yfinance) - Easy Pythonic way to download market and financial data from Yahoo Finance. - [akshare](https://github.com/akfamily/akshare) - A financial data interface library, built for human beings! - [edgartools](https://github.com/dgunning/edgartools) - Library for downloading structured data from SEC EDGAR filings and XBRL financial statements. - - [lumibot](https://github.com/Lumiwealth/lumibot) - Algorithmic trading framework for backtesting and live deployment across stocks, options, crypto, futures, and forex. - [openbb](https://github.com/OpenBB-finance/OpenBB) - A financial data platform for analysts, quants and AI agents. - - [yfinance](https://github.com/ranaroussi/yfinance) - Easy Pythonic way to download market and financial data from Yahoo Finance. ### Data Validation _Libraries for validating data. Used for forms in many cases._ -- [cerberus](https://github.com/pyeve/cerberus) - A lightweight and extensible data validation library. +- [pydantic](https://github.com/pydantic/pydantic) - Data validation using Python type hints. - [jsonschema](https://github.com/python-jsonschema/jsonschema) - An implementation of [JSON Schema](https://json-schema.org/) for Python. - [pandera](https://github.com/unionai-oss/pandera) - A data validation library for dataframes, with support for pandas, polars, and Spark. -- [pydantic](https://github.com/pydantic/pydantic) - Data validation using Python type hints. -- [voluptuous](https://github.com/alecthomas/voluptuous) - A Python data validation library primarily intended for validating data from untrusted sources. ### Data Visualization _Libraries for visualizing data. Also see [awesome-javascript](https://github.com/sorrycc/awesome-javascript#data-visualization)._ - Plotting - - [altair](https://github.com/vega/altair) - Declarative statistical visualization library for Python. - - [bokeh](https://github.com/bokeh/bokeh) - Interactive Web Plotting for Python. - - [bqplot](https://github.com/bqplot/bqplot) - Interactive Plotting Library for the Jupyter Notebook. - [matplotlib](https://github.com/matplotlib/matplotlib) - A Python 2D plotting library. - [plotly](https://github.com/plotly/plotly.py) - Interactive graphing library for Python. - - [plotnine](https://github.com/has2k1/plotnine) - A grammar of graphics for Python based on ggplot2. - - [pygal](https://github.com/Kozea/pygal) - A Python SVG Charts Creator. - - [pyqtgraph](https://github.com/pyqtgraph/pyqtgraph) - Interactive and realtime 2D/3D/Image plotting and science/engineering widgets. - [seaborn](https://github.com/mwaskom/seaborn) - Statistical data visualization using Matplotlib. - - [ultraplot](https://github.com/ultraplot/UltraPlot) - Matplotlib wrapper for publication-ready scientific figures with minimal code. Includes advanced subplot management, panel layouts, and batteries-included geoscience plotting. - - [vispy](https://github.com/vispy/vispy) - High-performance scientific visualization based on OpenGL. + - [altair](https://github.com/vega/altair) - Declarative statistical visualization library for Python. + - [bokeh](https://github.com/bokeh/bokeh) - Interactive Web Plotting for Python. - Specialized - [cartopy](https://github.com/SciTools/cartopy) - A cartographic python library with matplotlib support. - [pygraphviz](https://github.com/pygraphviz/pygraphviz/) - Python interface to [Graphviz](https://www.graphviz.org/). + - [graphify](https://github.com/Graphify-Labs/graphify) - Turn any folder of code, SQL schemas, docs, papers, images, or videos into a queryable knowledge graph. - Dashboards and Apps - - [gradio](https://github.com/gradio-app/gradio) - Build and share machine learning apps, all in Python. - [streamlit](https://github.com/streamlit/streamlit) - A framework which lets you build dashboards, generate reports, or create chat apps in minutes. + - [gradio](https://github.com/gradio-app/gradio) - Build and share machine learning apps, all in Python. + ### Geolocation _Libraries for geocoding addresses and working with latitudes and longitudes._ -- [django-countries](https://github.com/SmileyChris/django-countries) - A Django app that provides a country field for models and forms. -- [geodjango](https://github.com/django/django) - A world-class geographic web framework that is part of [Django](https://docs.djangoproject.com/en/dev/ref/contrib/gis/). -- [geojson](https://github.com/jazzband/geojson) - Python bindings and utilities for GeoJSON. - [geopandas](https://github.com/geopandas/geopandas) - Python tools for geographic data (GeoSeries/GeoDataFrame) built on pandas. - [geopy](https://github.com/geopy/geopy) - Python Geocoding Toolbox. +- [geojson](https://github.com/jazzband/geojson) - Python bindings and utilities for GeoJSON. +- [geodjango](https://github.com/django/django) - A world-class geographic web framework that is part of [Django](https://docs.djangoproject.com/en/dev/ref/contrib/gis/). ### Science _Libraries for scientific computing. Also see [Python-for-Scientists](https://github.com/TomNicholas/Python-for-Scientists)._ - Core - - [numba](https://github.com/numba/numba) - Python JIT compiler to LLVM aimed at scientific Python. - [numpy](https://github.com/numpy/numpy) - A fundamental package for scientific computing with Python. - [scipy](https://github.com/scipy/scipy) - A Python-based ecosystem of open-source software for mathematics, science, and engineering. - - [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. + - [numba](https://github.com/numba/numba) - Python JIT compiler to LLVM aimed at scientific Python. +- Symbolic Mathematics - [sympy](https://github.com/sympy/sympy) - A Python library for symbolic mathematics. +- Statistics + - [statsmodels](https://github.com/statsmodels/statsmodels) - Statistical modeling and econometrics in Python. - Biology and Chemistry - [biopython](https://github.com/biopython/biopython) - Biopython is a set of freely available tools for biological computation. - - [cclib](https://github.com/cclib/cclib) - A library for parsing and interpreting the results of computational chemistry packages. - - [openbabel](https://github.com/openbabel/openbabel) - A chemical toolbox designed to speak the many languages of chemical data. - [rdkit](https://github.com/rdkit/rdkit) - Cheminformatics and Machine Learning Software. - Physics and Engineering - [astropy](https://github.com/astropy/astropy) - A community Python library for Astronomy. - [obspy](https://github.com/obspy/obspy) - A Python toolbox for seismology. - - [pydy](https://github.com/pydy/pydy) - Short for Python Dynamics, used to assist with workflow in the modeling of dynamic motion. - - [PythonRobotics](https://github.com/AtsushiSakai/PythonRobotics) - This is a compilation of various robotics algorithms with visualizations. - Simulation and Modeling - - [mesa](https://github.com/mesa/mesa) - An agent-based modeling framework for building, analyzing, and visualizing complex system simulations. - - [pathsim](https://github.com/pathsim/pathsim) - A block-based system modeling and simulation framework with a browser-based visual editor. - [pymc](https://github.com/pymc-devs/pymc) - Probabilistic programming and Bayesian modeling in Python. - [simpy](https://gitlab.com/team-simpy/simpy) - A process-based discrete-event simulation framework. -- Other - - [colour](https://github.com/colour-science/colour) - Implementing a comprehensive number of colour theory transformations and algorithms. - - [manim](https://github.com/ManimCommunity/manim) - An animation engine for explanatory math videos. + - [mesa](https://github.com/mesa/mesa) - An agent-based modeling framework for building, analyzing, and visualizing complex system simulations. +- Graphs and Networks - [networkx](https://github.com/networkx/networkx) - A high-productivity software for complex networks. +- Computational Geometry - [shapely](https://github.com/shapely/shapely) - Manipulation and analysis of geometric objects in the Cartesian plane. +- Other + - [colour-science](https://github.com/colour-science/colour) - Implementing a comprehensive number of colour theory transformations and algorithms. + - [manim](https://github.com/ManimCommunity/manim) - An animation engine for explanatory math videos. ### Quantum Computing _Libraries for quantum computing._ -- [Cirq](https://github.com/quantumlib/Cirq) — A Google-developed framework focused on hardware-aware quantum circuit design for NISQ devices. -- [pennylane](https://github.com/PennyLaneAI/pennylane) — A hybrid quantum-classical machine learning library with automatic differentiation support. -- [qiskit](https://github.com/Qiskit/qiskit) — An IBM-backed quantum SDK for building, simulating, and running circuits on real quantum hardware. +- [qiskit](https://github.com/Qiskit/qiskit) - An IBM-backed quantum SDK for building, simulating, and running circuits on real quantum hardware. - [qutip](https://github.com/qutip/qutip) - Quantum Toolbox in Python. +- [pennylane](https://github.com/PennyLaneAI/pennylane) - A hybrid quantum-classical machine learning library with automatic differentiation support. +- [cirq](https://github.com/quantumlib/Cirq) - A Google-developed framework focused on hardware-aware quantum circuit design for NISQ devices. **Developer Tools** @@ -605,17 +578,18 @@ _Libraries for quantum computing._ _Python implementation of data structures, algorithms and design patterns. Also see [awesome-algorithms](https://github.com/tayllan/awesome-algorithms)._ - Algorithms - - [algorithms](https://github.com/keon/algorithms) - Minimal examples of data structures and algorithms. - [sortedcontainers](https://github.com/grantjenks/python-sortedcontainers) - Fast and pure-Python implementation of sorted collections. + - [algorithms](https://github.com/keon/algorithms) - Minimal examples of data structures and algorithms. - [thealgorithms](https://github.com/TheAlgorithms/Python) - All Algorithms implemented in Python. - Design Patterns - - [python-patterns](https://github.com/faif/python-patterns) - A collection of design patterns in Python. - [transitions](https://github.com/pytransitions/transitions) - A lightweight, object-oriented finite state machine implementation. + - [python-patterns](https://github.com/faif/python-patterns) - A collection of design patterns in Python. ### Interactive Interpreter _Interactive Python interpreters (REPL)._ +- [ipython](https://github.com/ipython/ipython) - A powerful interactive Python shell, and the kernel behind Jupyter notebooks. - [jupyter](https://github.com/jupyter/notebook) - A rich toolkit to help you make the most out of using Python interactively. - [awesome-jupyter](https://github.com/markusschanta/awesome-jupyter) - [marimo](https://github.com/marimo-team/marimo) - Transform data and train models, feels like a next-gen notebook, stored as Git-friendly Python. @@ -626,57 +600,54 @@ _Interactive Python interpreters (REPL)._ _Tools of static analysis, linters and code quality checkers. Also see [awesome-static-analysis](https://github.com/analysis-tools-dev/static-analysis)._ - Code Analysis - - [code2flow](https://github.com/scottrogowski/code2flow) - Turn your Python and JavaScript code into DOT flowcharts. + - [vulture](https://github.com/jendrikseipp/vulture) - A tool for finding and analyzing dead Python code. - [prospector](https://github.com/prospector-dev/prospector) - A tool to analyze Python code. - [repowise](https://github.com/repowise-dev/repowise) - Codebase intelligence that indexes repos into dependency graphs, git history, and auto-generated docs with dead code detection. - - [vulture](https://github.com/jendrikseipp/vulture) - A tool for finding and analyzing dead Python code. -- Code Linters - - [bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code. - - [flake8](https://github.com/PyCQA/flake8) - A wrapper around `pycodestyle`, `pyflakes` and McCabe. - - [awesome-flake8-extensions](https://github.com/DmytroLitvinov/awesome-flake8-extensions) - - [pylint](https://github.com/pylint-dev/pylint) - A fully customizable source code analyzer. -- Code Formatters +- Git Hooks + - [pre-commit](https://github.com/pre-commit/pre-commit) - A framework for managing and maintaining multi-language pre-commit hooks. +- Linters and Formatters + - [ruff](https://github.com/astral-sh/ruff) - An extremely fast Python linter and code formatter. - [black](https://github.com/psf/black) - The uncompromising Python code formatter. - [isort](https://github.com/PyCQA/isort) - A Python utility / library to sort imports. - - [ruff](https://github.com/astral-sh/ruff) - An extremely fast Python linter and code formatter. + - [pylint](https://github.com/pylint-dev/pylint) - A fully customizable source code analyzer. + - [flake8](https://github.com/PyCQA/flake8) - A wrapper around `pycodestyle`, `pyflakes` and McCabe. + - [awesome-flake8-extensions](https://github.com/DmytroLitvinov/awesome-flake8-extensions) + - [bandit](https://github.com/PyCQA/bandit) - A tool designed to find common security issues in Python code. - Refactoring - [rope](https://github.com/python-rope/rope) - Rope is a python refactoring library. - Type Checkers - [awesome-python-typing](https://github.com/typeddjango/awesome-python-typing) - [mypy](https://github.com/python/mypy) - Check variable types during compile time. - - [pyrefly](https://github.com/facebook/pyrefly) - A fast type checker and language server for Python. - [ty](https://github.com/astral-sh/ty) - An extremely fast Python type checker and language server. - - [typeshed](https://github.com/python/typeshed) - Collection of library stubs for Python, with static types. + - [pyright](https://github.com/microsoft/pyright) - Full-featured static type checker for Python from Microsoft, the engine behind Pylance. + - [pyrefly](https://github.com/facebook/pyrefly) - A fast type checker and language server for Python. - Type Annotations Generators - [monkeytype](https://github.com/Instagram/MonkeyType) - A system for Python that generates static type annotations by collecting runtime types. - - [pytype](https://github.com/google/pytype) - Pytype checks and infers types for Python code - without requiring type annotations. ### Testing _Libraries for testing codebases and generating test data. Also see [awesome-python-testing](https://github.com/cleder/awesome-python-testing)._ - Frameworks - - [hypothesis](https://github.com/HypothesisWorks/hypothesis) - Hypothesis is an advanced Quickcheck style property based testing library. - [pytest](https://github.com/pytest-dev/pytest) - A mature full-featured Python testing tool. - [awesome-pytest](https://github.com/augustogoulart/awesome-pytest) + - [hypothesis](https://github.com/HypothesisWorks/hypothesis) - Hypothesis is an advanced Quickcheck style property based testing library. - [robotframework](https://github.com/robotframework/robotframework) - A generic test automation framework. - - [scanapi](https://github.com/scanapi/scanapi) - Automated Testing and Documentation for your REST API. - - [unittest](https://docs.python.org/3/library/unittest.html) - (Python standard library) Unit testing framework. - Test Runners - - [nox](https://github.com/wntrblm/nox) - Flexible test automation for Python. - [tox](https://github.com/tox-dev/tox) - Auto builds and tests distributions in multiple Python versions -- GUI / Web Testing - - [locust](https://github.com/locustio/locust) - Scalable user load testing tool written in Python. + - [nox](https://github.com/wntrblm/nox) - Flexible test automation for Python. +- Browser Automation - [playwright-python](https://github.com/microsoft/playwright-python) - Python version of the Playwright testing and automation library. - - [pyautogui](https://github.com/asweigart/pyautogui) - PyAutoGUI is a cross-platform GUI automation Python module for human beings. - - [schemathesis](https://github.com/schemathesis/schemathesis) - A tool for automatic property-based testing of web applications built with Open API / Swagger specifications. - [selenium](https://github.com/SeleniumHQ/selenium) - Python bindings for [Selenium](https://selenium.dev/) [WebDriver](https://selenium.dev/documentation/webdriver/). +- Load Testing + - [locust](https://github.com/locustio/locust) - Scalable user load testing tool written in Python. +- API Testing + - [schemathesis](https://github.com/schemathesis/schemathesis) - A tool for automatic property-based testing of web applications built with Open API / Swagger specifications. - Mock - - [freezegun](https://github.com/spulec/freezegun) - Travel through time by mocking the datetime module. - [mock](https://docs.python.org/3/library/unittest.mock.html) - (Python standard library) A mocking and patching library. - - [mocket](https://github.com/mindflayer/python-mocket) - A socket mock framework with gevent/asyncio/SSL support. - [responses](https://github.com/getsentry/responses) - A utility library for mocking out the requests Python library. - - [respx](https://github.com/lundberg/respx) - Mock HTTPX with awesome request patterns and response side effects. + - [freezegun](https://github.com/spulec/freezegun) - Travel through time by mocking the datetime module. - [vcrpy](https://github.com/kevin1024/vcrpy) - Record and replay HTTP interactions on your tests. + - [respx](https://github.com/lundberg/respx) - Mock HTTPX with awesome request patterns and response side effects. - Object Factories - [factory_boy](https://github.com/FactoryBoy/factory_boy) - A test fixtures replacement for Python. - [polyfactory](https://github.com/litestar-org/polyfactory) - mock data generation library with support to classes (continuation of `pydantic-factories`) @@ -694,37 +665,35 @@ _Libraries for debugging code._ - [ipdb](https://github.com/gotcha/ipdb) - IPython-enabled [pdb](https://docs.python.org/3/library/pdb.html). - [pudb](https://github.com/inducer/pudb) - A full-screen, console-based Python debugger. - Tracing - - [manhole](https://github.com/ionelmc/python-manhole) - Debugging UNIX socket connections and present the stacktraces for all threads and an interactive prompt. - - [python-hunter](https://github.com/ionelmc/python-hunter) - A flexible code tracing toolkit. + - [hunter](https://github.com/ionelmc/python-hunter) - A flexible code tracing toolkit. - Profiler - [py-spy](https://github.com/benfred/py-spy) - A sampling profiler for Python programs. Written in Rust. + - [memray](https://github.com/bloomberg/memray) - A memory profiler that tracks allocations in Python code, native extensions, and the interpreter itself. + - [pyinstrument](https://github.com/joerick/pyinstrument) - A statistical wall-clock profiler with low overhead and readable call-tree output. - [scalene](https://github.com/plasma-umass/scalene) - A high-performance, high-precision CPU, GPU, and memory profiler for Python. - Others - [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) - Display various debug information for Django. - - [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to flask. - [icecream](https://github.com/gruns/icecream) - Inspect variables, expressions, and program execution with a single, simple function call. - - [memory_graph](https://github.com/bterwijn/memory_graph) - Visualize Python data at runtime to debug references, mutability, and aliasing. + - [flask-debugtoolbar](https://github.com/pallets-eco/flask-debugtoolbar) - A port of the django-debug-toolbar to flask. ### Build Tools -_Compile software from source code._ +_Compile software from source code. If you're looking for Python packaging/build tools, see [Package Management](#package-management)._ -- [bitbake](https://github.com/openembedded/bitbake) - A make-like build tool for embedded Linux. -- [doit](https://github.com/pydoit/doit) - A task runner and build tool. - [invoke](https://github.com/pyinvoke/invoke) - A tool for managing shell-oriented subprocesses and organizing executable Python code into CLI-invokable tasks. -- [platformio](https://github.com/platformio/platformio-core) - A console tool to build code with different development platforms. -- [pybuilder](https://github.com/pybuilder/pybuilder) - A continuous build tool written in pure Python. - [scons](https://github.com/SCons/scons) - A software construction tool. +- [doit](https://github.com/pydoit/doit) - A task runner and build tool. ### Documentation _Libraries for generating project documentation._ -- [diagrams](https://github.com/mingrammer/diagrams) - Diagram as Code. -- [mkdocs](https://github.com/mkdocs/mkdocs/) - Markdown friendly documentation generator. -- [pdoc](https://github.com/mitmproxy/pdoc) - Epydoc replacement to auto generate API documentation for Python libraries. - [sphinx](https://github.com/sphinx-doc/sphinx/) - Python Documentation generator. - [awesome-sphinxdoc](https://github.com/ygzgxyz/awesome-sphinxdoc) +- [mkdocs-material](https://github.com/squidfunk/mkdocs-material) - A documentation framework and Material Design theme built on MkDocs. +- [mkdocs](https://github.com/mkdocs/mkdocs/) - Markdown friendly documentation generator. +- [diagrams](https://github.com/mingrammer/diagrams) - Diagram as Code. +- [pdoc](https://github.com/mitmproxy/pdoc) - Epydoc replacement to auto generate API documentation for Python libraries. **DevOps** @@ -733,83 +702,83 @@ _Libraries for generating project documentation._ _Software and libraries for DevOps._ - Cloud Providers - - [awscli](https://github.com/aws/aws-cli) - Universal Command Line Interface for Amazon Web Services. - [boto3](https://github.com/boto/boto3) - Python interface to Amazon Web Services. + - [awscli](https://github.com/aws/aws-cli) - Universal Command Line Interface for Amazon Web Services. + - [azure-sdk-for-python](https://github.com/Azure/azure-sdk-for-python) - Microsoft Azure SDK for Python, published as per-service packages. + - [google-cloud-python](https://github.com/googleapis/google-cloud-python) - Google Cloud client libraries for Python, published as per-service packages. - Configuration Management - [ansible](https://github.com/ansible/ansible) - A radically simple IT automation platform. - - [cloudinit](https://github.com/canonical/cloud-init) - A multi-distribution package that handles early initialization of a cloud instance. - - [openstack](https://github.com/openstack/openstack) - Open source software for building private and public clouds. + - [cloud-init](https://github.com/canonical/cloud-init) - A multi-distribution package that handles early initialization of a cloud instance. - [pyinfra](https://github.com/pyinfra-dev/pyinfra) - A versatile CLI tools and python libraries to automate infrastructure. - - [saltstack](https://github.com/saltstack/salt) - Infrastructure automation and management system. + - [salt](https://github.com/saltstack/salt) - Infrastructure automation and management system. - Deployment - - [chalice](https://github.com/aws/chalice) - A Python serverless microframework for AWS. - [fabric](https://github.com/fabric/fabric) - A simple, Pythonic tool for remote execution and deployment. + - [chalice](https://github.com/aws/chalice) - A Python serverless microframework for AWS. - Monitoring and Processes - [psutil](https://github.com/giampaolo/psutil) - A cross-platform process and system utilities module. - - [sentry-python](https://github.com/getsentry/sentry-python) - Sentry SDK for Python. - - [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python. + - [sentry-sdk](https://github.com/getsentry/sentry-python) - Sentry SDK for Python. - [supervisor](https://github.com/Supervisor/supervisor) - Supervisor process control system for UNIX. + - [flower](https://github.com/mher/flower) - A real-time monitor and web admin for Celery task queues. + - [sh](https://github.com/amoffat/sh) - A full-fledged subprocess replacement for Python. - Other - - [borg](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption. + - [borgbackup](https://github.com/borgbackup/borg) - A deduplicating archiver with compression and encryption. - [chaostoolkit](https://github.com/chaostoolkit/chaostoolkit) - A Chaos Engineering toolkit & Orchestration for Developers. - - [kubesdk](https://github.com/puzl-cloud/kubesdk) - A modern async-first Kubernetes client and API model generator. - - [pre-commit](https://github.com/pre-commit/pre-commit) - A framework for managing and maintaining multi-language pre-commit hooks. ### Distributed Computing _Frameworks and libraries for Distributed Computing._ -- Batch Processing - - [dask](https://github.com/dask/dask) - A flexible parallel computing library for analytic computing. - - [luigi](https://github.com/spotify/luigi) - A module that helps you build complex pipelines of batch jobs. - - [mpi4py](https://github.com/mpi4py/mpi4py) - Python bindings for MPI. - - [pyspark](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/) Python API. - - [joblib](https://github.com/joblib/joblib) - A set of tools to provide lightweight pipelining in Python. - - [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the machine learning ecosystem. +- [ray](https://github.com/ray-project/ray/) - A system for parallel and distributed Python that unifies the machine learning ecosystem. +- [pyspark](https://github.com/apache/spark) - [Apache Spark](https://spark.apache.org/) Python API. +- [dask](https://github.com/dask/dask) - A flexible parallel computing library for analytic computing. +- [joblib](https://github.com/joblib/joblib) - A set of tools to provide lightweight pipelining in Python. +- [mpi4py](https://github.com/mpi4py/mpi4py) - Python bindings for MPI. ### Task Queues _Libraries for working with task queues._ - [celery](https://github.com/celery/celery) - An asynchronous task queue/job queue based on distributed message passing. +- [rq](https://github.com/rq/rq) - Simple job queues for Python. - [dramatiq](https://github.com/Bogdanp/dramatiq) - A fast and reliable background task processing library for Python 3. - [huey](https://github.com/coleifer/huey) - Little multi-threaded task queue. -- [rq](https://github.com/rq/rq) - Simple job queues for Python. - [taskiq](https://github.com/taskiq-python/taskiq) - Distributed task queue with native asyncio support and pluggable brokers. ### Messaging _Libraries for working with message brokers and event streaming._ +- [confluent-kafka](https://github.com/confluentinc/confluent-kafka-python) - Confluent's Python client for Apache Kafka, built on librdkafka. +- [pika](https://github.com/pika/pika) - Pure-Python RabbitMQ/AMQP 0-9-1 client library. +- [paho-mqtt](https://github.com/eclipse-paho/paho.mqtt.python) - The Eclipse Paho MQTT client for Python. - [faststream](https://github.com/ag2ai/faststream) - A framework for building asynchronous services over Apache Kafka, RabbitMQ, NATS, MQTT and Redis. ### Job Schedulers _Libraries for scheduling jobs._ -- [airflow](https://github.com/apache/airflow) - Airflow is a platform to programmatically author, schedule and monitor workflows. -- [apscheduler](https://github.com/agronholm/apscheduler) - A light but powerful in-process task scheduler that lets you schedule functions. -- [dagster](https://github.com/dagster-io/dagster) - An orchestration platform for the development, production, and observation of data assets. -- [prefect](https://github.com/PrefectHQ/prefect) - A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines. -- [schedule](https://github.com/dbader/schedule) - Python job scheduling for humans. -- [SpiffWorkflow](https://github.com/sartography/SpiffWorkflow) - A powerful workflow engine implemented in pure Python. +- Task Scheduling + - [apscheduler](https://github.com/agronholm/apscheduler) - A light but powerful in-process task scheduler that lets you schedule functions. + - [schedule](https://github.com/dbader/schedule) - Python job scheduling for humans. +- Workflow Orchestration + - [apache-airflow](https://github.com/apache/airflow) - Airflow is a platform to programmatically author, schedule and monitor workflows. + - [prefect](https://github.com/PrefectHQ/prefect) - A modern workflow orchestration framework that makes it easy to build, schedule and monitor robust data pipelines. + - [dagster](https://github.com/dagster-io/dagster) - An orchestration platform for the development, production, and observation of data assets. ### Logging _Libraries for generating and working with logs._ -- [logfmter](https://github.com/josheppinette/python-logfmter) - A standard library compatible logfmt formatter. - [logging](https://docs.python.org/3/library/logging.html) - (Python standard library) Logging facility for Python. -- [loguru](https://github.com/Delgan/loguru) - Library which aims to bring enjoyable logging in Python. - [structlog](https://github.com/hynek/structlog) - Structured logging made easy. +- [loguru](https://github.com/Delgan/loguru) - Library which aims to bring enjoyable logging in Python. ### Network Virtualization _Tools and libraries for Virtual Networking and SDN (Software Defined Networking)._ -- [mininet](https://github.com/mininet/mininet) - A popular network emulator and API written in Python. -- [napalm](https://github.com/napalm-automation/napalm) - Cross-vendor API to manipulate network devices. - [scapy](https://github.com/secdev/scapy) - A brilliant packet manipulation library. +- [napalm](https://github.com/napalm-automation/napalm) - Cross-vendor API to manipulate network devices. **CLI & GUI** @@ -819,64 +788,62 @@ _Libraries for building command-line applications._ - CLI Development - [argparse](https://docs.python.org/3/library/argparse.html) - (Python standard library) Command-line option and argument parsing. - - [cement](https://github.com/datafolklabs/cement) - CLI Application Framework for Python. - [click](https://github.com/pallets/click/) - A package for creating beautiful command line interfaces in a composable way. - - [python-fire](https://github.com/google/python-fire) - A library for creating command line interfaces from absolutely any Python object. - - [python-prompt-toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. - [typer](https://github.com/fastapi/typer) - Modern CLI framework that uses Python type hints. Built on Click and Pydantic. + - [prompt_toolkit](https://github.com/prompt-toolkit/python-prompt-toolkit) - A library for building powerful interactive command lines. + - [fire](https://github.com/google/python-fire) - A library for creating command line interfaces from absolutely any Python object. - Terminal Rendering - - [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of Progress Bar, with real-time throughput, eta and very cool animations. - - [asciimatics](https://github.com/peterbrittain/asciimatics) - A package to create full-screen text UIs (from interactive forms to ASCII animations). - - [colorama](https://github.com/tartley/colorama) - Cross-platform colored terminal text. + - [tqdm](https://github.com/tqdm/tqdm) - Fast, extensible progress bar for loops and CLI. - [rich](https://github.com/Textualize/rich) - Python library for rich text and beautiful formatting in the terminal. Also provides a great `RichHandler` log handler. + - [colorama](https://github.com/tartley/colorama) - Cross-platform colored terminal text. + - [alive-progress](https://github.com/rsalmei/alive-progress) - A new kind of Progress Bar, with real-time throughput, eta and very cool animations. +- TUI Frameworks - [textual](https://github.com/Textualize/textual) - A framework for building interactive user interfaces that run in the terminal and the browser. - - [tqdm](https://github.com/tqdm/tqdm) - Fast, extensible progress bar for loops and CLI. + - [urwid](https://github.com/urwid/urwid) - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc. + - [asciimatics](https://github.com/peterbrittain/asciimatics) - A package to create full-screen text UIs (from interactive forms to ASCII animations). ### CLI Tools -_Useful CLI-based tools for productivity._ +_Useful CLI-based tools._ -- Productivity Tools +- Database CLIs + - [pgcli](https://github.com/dbcli/pgcli) - PostgreSQL CLI with autocompletion and syntax highlighting. + - [mycli](https://github.com/dbcli/mycli) - MySQL CLI with autocompletion and syntax highlighting. + - [litecli](https://github.com/dbcli/litecli) - SQLite CLI with autocompletion and syntax highlighting. + - [iredis](https://github.com/laixintao/iredis) - Redis CLI with autocompletion and syntax highlighting. +- Downloaders + - [yt-dlp](https://github.com/yt-dlp/yt-dlp) - A command-line program to download videos from YouTube and other video sites, a fork of youtube-dl. +- HTTP Clients + - [httpie](https://github.com/httpie/cli) - A command line HTTP client, a user-friendly cURL replacement. +- Project Scaffolding - [cookiecutter](https://github.com/cookiecutter/cookiecutter) - A command-line utility that creates projects from cookiecutters (project templates). - [copier](https://github.com/copier-org/copier) - A library and command-line utility for rendering projects templates. - - [doitlive](https://github.com/sloria/doitlive) - A tool for live presentations in the terminal. - - [kaydet](https://github.com/miratcan/kaydet) - Queryable personal database and terminal diary with SQLite search, tags, metadata, and AI integration via MCP. - - [thefuck](https://github.com/nvbn/thefuck) - Correcting your previous console command. - - [tmuxp](https://github.com/tmux-python/tmuxp) - A [tmux](https://github.com/tmux/tmux) session manager. +- Shells - [xonsh](https://github.com/xonsh/xonsh/) - A Python-powered shell. Full-featured and cross-platform. - - [yt-dlp](https://github.com/yt-dlp/yt-dlp) - A command-line program to download videos from YouTube and other video sites, a fork of youtube-dl. -- CLI Enhancements - - [httpie](https://github.com/httpie/cli) - A command line HTTP client, a user-friendly cURL replacement. - - [iredis](https://github.com/laixintao/iredis) - Redis CLI with autocompletion and syntax highlighting. - - [litecli](https://github.com/dbcli/litecli) - SQLite CLI with autocompletion and syntax highlighting. - - [mycli](https://github.com/dbcli/mycli) - MySQL CLI with autocompletion and syntax highlighting. - - [pgcli](https://github.com/dbcli/pgcli) - PostgreSQL CLI with autocompletion and syntax highlighting. +- Terminal Workflow + - [tmuxp](https://github.com/tmux-python/tmuxp) - A [tmux](https://github.com/tmux/tmux) session manager. ### GUI Development _Libraries for working with graphical user interface applications._ - Desktop - - [customtkinter](https://github.com/tomschimansky/customtkinter) - A modern and customizable python UI-library based on Tkinter. - - [dearpygui](https://github.com/hoffstadt/DearPyGui) - A Simple GPU accelerated Python GUI framework - - [enaml](https://github.com/nucleic/enaml) - Creating beautiful user-interfaces with Declarative Syntax like QML. - - [kivy](https://github.com/kivy/kivy) - A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS. - - [pyglet](https://github.com/pyglet/pyglet) - A cross-platform windowing and multimedia library for Python. - [pygobject](https://github.com/GNOME/pygobject) - Python Bindings for GLib/GObject/GIO/GTK+ (GTK+3). - - [PyQt](https://www.riverbankcomputing.com/static/Docs/PyQt6/) - Python bindings for the [Qt](https://www.qt.io/) cross-platform application and UI framework. - - [pyside](https://github.com/pyside/pyside-setup) - Qt for Python offers the official Python bindings for [Qt](https://www.qt.io/), this is same as PyQt but it's the official binding with different licensing. - - [tkdesigner](https://github.com/ParthJadhav/Tkinter-Designer) - Generates Tkinter interfaces from Figma designs using the Figma API. - - [tkinter](https://docs.python.org/3/library/tkinter.html) - (Python standard library) The standard Python interface to the Tcl/Tk GUI toolkit. - - [toga](https://github.com/beeware/toga) - A Python native, OS native GUI toolkit. - [wxPython](https://github.com/wxWidgets/Phoenix) - A blending of the wxWidgets C++ class library with the Python. + - [kivy](https://github.com/kivy/kivy) - A library for creating NUI applications, running on Windows, Linux, Mac OS X, Android and iOS. + - [dearpygui](https://github.com/hoffstadt/DearPyGui) - A Simple GPU accelerated Python GUI framework + - [toga](https://github.com/beeware/toga) - A Python native, OS native GUI toolkit. +- Qt + - [PySide6](https://github.com/pyside/pyside-setup) - Qt for Python offers the official Python bindings for [Qt](https://www.qt.io/), same as PyQt6 but it's the official binding with different licensing. + - [PyQt6](https://www.riverbankcomputing.com/static/Docs/PyQt6/) - Python bindings for the [Qt](https://www.qt.io/) cross-platform application and UI framework. +- Tkinter + - [tkinter](https://docs.python.org/3/library/tkinter.html) - (Python standard library) The standard Python interface to the Tcl/Tk GUI toolkit. + - [customtkinter](https://github.com/tomschimansky/customtkinter) - A modern and customizable python UI-library based on Tkinter. + - [tkdesigner](https://github.com/ParthJadhav/Tkinter-Designer) - Generates Tkinter interfaces from Figma designs using the Figma API. - Web-based - - [buridan/ui](https://github.com/LineIndent/ui) - Composable, themeable components designed for Reflex. Extend, override, and ship without fighting the framework. Open source. - - [flet](https://github.com/flet-dev/flet) - Cross-platform GUI framework for building modern apps in pure Python. - - [nicegui](https://github.com/zauberzeug/nicegui) - An easy-to-use, Python-based UI framework, which shows up in your web browser. - [pywebview](https://github.com/r0x0r/pywebview/) - A lightweight cross-platform native wrapper around a webview component. -- Terminal - - [curses](https://docs.python.org/3/library/curses.html) - (Python standard library) The built-in wrapper for [ncurses](https://www.gnu.org/software/ncurses/) used to create terminal GUI applications. - - [urwid](https://github.com/urwid/urwid) - A library for creating terminal GUI applications with strong support for widgets, events, rich colors, etc. + - [nicegui](https://github.com/zauberzeug/nicegui) - An easy-to-use, Python-based UI framework, which shows up in your web browser. + - [flet](https://github.com/flet-dev/flet) - Cross-platform GUI framework for building modern apps in pure Python. - Wrappers - [gooey](https://github.com/chriskiehl/Gooey) - Turn command line programs into a full GUI application with one line. @@ -886,76 +853,70 @@ _Libraries for working with graphical user interface applications._ _Libraries for parsing and manipulating plain texts._ -- General - - [babel](https://github.com/python-babel/babel) - An internationalization library for Python. +- Encoding and Unicode + - [charset-normalizer](https://github.com/jawah/charset_normalizer) - Universal character encoding detector, the default of the requests ecosystem. - [chardet](https://github.com/chardet/chardet) - Python character encoding detector. - - [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas. - [ftfy](https://github.com/rspeer/python-ftfy) - Makes Unicode text less broken and more consistent automagically. - - [pangu.py](https://github.com/vinta/pangu.py) - Paranoid text spacing. +- Fuzzy Matching + - [rapidfuzz](https://github.com/rapidfuzz/RapidFuzz) - Rapid fuzzy string matching using various string metrics, with a C++ core. +- General + - [difflib](https://docs.python.org/3/library/difflib.html) - (Python standard library) Helpers for computing deltas. - [pyfiglet](https://github.com/pwaller/pyfiglet) - An implementation of figlet written in Python. - - [pypinyin](https://github.com/mozillazg/python-pinyin) - Convert Chinese hanzi (漢字) to pinyin (拼音). - - [python-slugify](https://github.com/un33k/python-slugify) - A Python slugify library that translates unicode to ASCII. - - [textdistance](https://github.com/life4/textdistance) - Compute distance between sequences with 30+ algorithms. - - [unidecode](https://github.com/avian2/unidecode) - ASCII transliterations of Unicode text. -- Unique identifiers - - [sqids](https://github.com/sqids/sqids-python) - A library for generating short unique IDs from numbers. - - [shortuuid](https://github.com/skorokithakis/shortuuid) - A generator library for concise, unambiguous and URL-safe UUIDs. +- Internationalization + - [babel](https://github.com/python-babel/babel) - An internationalization library for Python. - Parser - - [parsy](https://github.com/python-parsy/parsy) - Easy, generic parser combinator library for creating parsers. - [pygments](https://github.com/pygments/pygments) - A generic syntax highlighter. - [pyparsing](https://github.com/pyparsing/pyparsing) - A general purpose framework for generating parsers. - - [python-nameparser](https://github.com/derek73/python-nameparser) - Parsing human names into their individual components. - - [python-phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) - Parsing, formatting, storing and validating international phone numbers. - - [python-user-agents](https://github.com/selwin/python-user-agents) - Browser user agent parser. - [sqlparse](https://github.com/andialbrecht/sqlparse) - A non-validating SQL parser. - - [tree-sitter-language-pack](https://github.com/xberg-io/tree-sitter-language-pack) - A comprehensive collection of tree-sitter parsers for 300+ languages, distributed as prebuilt wheels. + - [phonenumbers](https://github.com/daviddrysdale/python-phonenumbers) - Parsing, formatting, storing and validating international phone numbers. + - [parsy](https://github.com/python-parsy/parsy) - Easy, generic parser combinator library for creating parsers. +- Transliteration and Slugs + - [python-slugify](https://github.com/un33k/python-slugify) - A Python slugify library that translates unicode to ASCII. + - [unidecode](https://github.com/avian2/unidecode) - ASCII transliterations of Unicode text. +- Unique identifiers + - [shortuuid](https://github.com/skorokithakis/shortuuid) - A generator library for concise, unambiguous and URL-safe UUIDs. + - [sqids](https://github.com/sqids/sqids-python) - A library for generating short unique IDs from numbers. ### HTML Manipulation _Libraries for working with HTML and XML._ -- [beautifulsoup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML. -- [html-to-markdown](https://github.com/xberg-io/html-to-markdown) - A fast, CommonMark-compliant HTML to Markdown converter with a Rust core, tolerant of malformed HTML. -- [justhtml](https://github.com/EmilStenstrom/justhtml/) - A pure Python HTML5 parser that just works. +- [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) - Providing Pythonic idioms for iterating, searching, and modifying HTML or XML. - [lxml](https://github.com/lxml/lxml) - A very fast, easy-to-use and versatile library for handling HTML and XML. -- [markupsafe](https://github.com/pallets/markupsafe) - Implements a XML/HTML/XHTML Markup safe string for Python. -- [pyquery](https://github.com/gawel/pyquery) - A jQuery-like library for parsing HTML. -- [tinycss2](https://github.com/Kozea/tinycss2) - A low-level CSS parser and generator written in Python. - [xmltodict](https://github.com/martinblech/xmltodict) - Working with XML feel like you are working with JSON. +- [markupsafe](https://github.com/pallets/markupsafe) - Implements a XML/HTML/XHTML Markup safe string for Python. +- [justhtml](https://github.com/EmilStenstrom/justhtml/) - A pure Python HTML5 parser that just works. ### File Format Processing _Libraries for parsing and manipulating specific text formats._ - General - - [docling](https://github.com/docling-project/docling) - Library for converting documents into structured data. - - [xberg](https://github.com/xberg-io/xberg) - High-performance document intelligence library with a Rust core, extracting text, tables, and metadata from 97+ formats including PDF, Office, images (with OCR), HTML, email, and archives. - [pyelftools](https://github.com/eliben/pyelftools) - Parsing and analyzing ELF files and DWARF debugging information. - [tablib](https://github.com/jazzband/tablib) - A module for Tabular Datasets in XLS, CSV, JSON, YAML. -- MS Office - - [docxtpl](https://github.com/elapouya/python-docx-template) - Editing a docx document by jinja2 template +- File Conversion + - [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown. + - [docling](https://github.com/docling-project/docling) - Library for converting documents into structured data. +- Excel - [openpyxl](https://openpyxl.readthedocs.io/en/stable/) - A library for reading and writing Excel 2010 xlsx/xlsm/xltx/xltm files. - - [pyexcel](https://github.com/pyexcel/pyexcel) - Providing one API for reading, manipulating and writing csv, ods, xls, xlsx and xlsm files. + - [xlsxwriter](https://github.com/jmcnamara/XlsxWriter) - A Python module for creating Excel .xlsx files. +- Word - [python-docx](https://github.com/python-openxml/python-docx) - Reads, queries and modifies Microsoft Word 2007/2008 docx files. +- PowerPoint - [python-pptx](https://github.com/scanny/python-pptx) - Python library for creating and updating PowerPoint (.pptx) files. - - [xlsxwriter](https://github.com/jmcnamara/XlsxWriter) - A Python module for creating Excel .xlsx files. - - [xlwings](https://github.com/xlwings/xlwings) - A BSD-licensed library that makes it easy to call Python from Excel and vice versa. - PDF - - [pdf_oxide](https://github.com/yfedoseev/pdf_oxide) - A fast PDF library for text extraction, image extraction, and markdown conversion, powered by Rust. - - [pdfminer.six](https://github.com/pdfminer/pdfminer.six) - Pdfminer.six is a community maintained fork of the original PDFMiner. - - [pikepdf](https://github.com/pikepdf/pikepdf) - A powerful library for reading and editing PDF files, based on qpdf. - [pypdf](https://github.com/py-pdf/pypdf) - A library capable of splitting, merging, cropping, and transforming PDF pages. - [reportlab](https://www.reportlab.com/opensource/) - Allowing Rapid creation of rich PDF documents. + - [pdfminer.six](https://github.com/pdfminer/pdfminer.six) - Pdfminer.six is a community maintained fork of the original PDFMiner. +- HTML-to-PDF - [weasyprint](https://github.com/Kozea/WeasyPrint) - A visual rendering engine for HTML and CSS that can export to PDF. - Markdown - [markdown-it-py](https://github.com/executablebooks/markdown-it-py) - Markdown parser with 100% CommonMark support, extensions, and syntax plugins. - [markdown](https://github.com/Python-Markdown/markdown) - A Python implementation of John Gruber’s Markdown. - - [markitdown](https://github.com/microsoft/markitdown) - Python tool for converting files and office documents to Markdown. - [mistune](https://github.com/lepture/mistune) - Fastest and full featured pure Python parsers of Markdown. - Data Formats - - [csvkit](https://github.com/wireservice/csvkit) - Utilities for converting to and working with CSV. - - [pyyaml](https://github.com/yaml/pyyaml) - YAML implementations for Python. - [tomllib](https://docs.python.org/3/library/tomllib.html) - (Python standard library) Parse TOML files. + - [pyyaml](https://github.com/yaml/pyyaml) - YAML implementations for Python. ### File Manipulation @@ -963,9 +924,9 @@ _Libraries for file manipulation._ - [mimetypes](https://docs.python.org/3/library/mimetypes.html) - (Python standard library) Map filenames to MIME types. - [pathlib](https://docs.python.org/3/library/pathlib.html) - (Python standard library) A cross-platform, object-oriented path library. -- [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. -- [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. - [watchfiles](https://github.com/samuelcolvin/watchfiles) - Simple, modern and fast file watching and code reload in python. +- [watchdog](https://github.com/gorakhargosh/watchdog) - API and shell utilities to monitor file system events. +- [python-magic](https://github.com/ahupp/python-magic) - A Python interface to the libmagic file type identification library. **Media** @@ -973,44 +934,46 @@ _Libraries for file manipulation._ _Libraries for manipulating images._ -- [pillow](https://github.com/python-pillow/Pillow) - Pillow is the friendly [PIL](https://www.pythonware.com/products/pil/) fork. -- [pymatting](https://github.com/pymatting/pymatting) - A library for alpha matting. -- [python-barcode](https://github.com/WhyNotHugo/python-barcode) - Create barcodes in Python with no extra dependencies. -- [python-qrcode](https://github.com/lincolnloop/python-qrcode) - A pure Python QR Code generator. -- [pyvips](https://github.com/libvips/pyvips) - A fast image processing library with low memory needs. -- [scikit-image](https://github.com/scikit-image/scikit-image) - A Python library for (scientific) image processing. -- [thumbor](https://github.com/thumbor/thumbor) - A smart imaging service. It enables on-demand crop, re-sizing and flipping of images. -- [wand](https://github.com/emcconville/wand) - Python bindings for [MagickWand](https://www.imagemagick.org/script/magick-wand.php), C API for ImageMagick. +- Barcodes and QR Codes + - [qrcode](https://github.com/lincolnloop/python-qrcode) - A pure Python QR Code generator. + - [python-barcode](https://github.com/WhyNotHugo/python-barcode) - Create barcodes in Python with no extra dependencies. +- General + - [pillow](https://github.com/python-pillow/Pillow) - Pillow is the friendly [PIL](https://www.pythonware.com/products/pil/) fork. + - [scikit-image](https://github.com/scikit-image/scikit-image) - A Python library for (scientific) image processing. + - [rembg](https://github.com/danielgatis/rembg) - A tool to remove image backgrounds. + - [wand](https://github.com/emcconville/wand) - Python bindings for [MagickWand](https://www.imagemagick.org/script/magick-wand.php), C API for ImageMagick. + - [pyvips](https://github.com/libvips/pyvips) - A fast image processing library with low memory needs. +- Image Serving + - [thumbor](https://github.com/thumbor/thumbor) - A smart imaging service. It enables on-demand crop, re-sizing and flipping of images. ### Audio & Video Processing _Libraries for manipulating audio, video, and their metadata._ - Audio - - [birdnet](https://github.com/birdnet-team/BirdNET-Analyzer) - Deep learning framework for acoustic species detection; identifies bird species from audio recordings using TensorFlow. - - [gtts](https://github.com/pndurette/gTTS) - Python library and CLI tool for converting text to speech using Google Translate TTS. - - [KittenTTS](https://github.com/KittenML/KittenTTS) - Lightweight ONNX text-to-speech library with small CPU-friendly models. - - [librosa](https://github.com/librosa/librosa) - Python library for audio and music analysis. - - [matchering](https://github.com/sergree/matchering) - A library for automated reference audio mastering. - [pydub](https://github.com/jiaaro/pydub) - Manipulate audio with a simple and easy high level interface. + - [librosa](https://github.com/librosa/librosa) - Python library for audio and music analysis. - Video - [moviepy](https://github.com/Zulko/moviepy) - A module for script-based movie editing with many formats, including animated GIFs. - [vidgear](https://github.com/abhiTronix/vidgear) - Most Powerful multi-threaded Video Processing framework. - Metadata - - [beets](https://github.com/beetbox/beets) - A music library manager and [MusicBrainz](https://musicbrainz.org/) tagger. - [mutagen](https://github.com/quodlibet/mutagen) - A Python module to handle audio metadata. - [tinytag](https://github.com/tinytag/tinytag) - A library for reading music meta data of MP3, OGG, FLAC and Wave files. + - [beets](https://github.com/beetbox/beets) - A music library manager and [MusicBrainz](https://musicbrainz.org/) tagger. ### Game Development _Awesome game development libraries._ -- [arcade](https://github.com/pythonarcade/arcade) - Arcade is a modern Python framework for crafting games with compelling graphics and sound. -- [panda3d](https://github.com/panda3d/panda3d) - 3D game engine developed by Disney. -- [py-sdl2](https://github.com/py-sdl/py-sdl2) - A ctypes based wrapper for the SDL2 library. -- [pygame](https://github.com/pygame/pygame) - Pygame is a set of Python modules designed for writing games. -- [pyopengl](https://github.com/mcfletch/pyopengl) - Python ctypes bindings for OpenGL and it's related APIs. -- [renpy](https://github.com/renpy/renpy) - A Visual Novel engine. +- 3D Engines + - [panda3d](https://github.com/panda3d/panda3d) - 3D game engine developed by Disney. +- Game Frameworks + - [pygame](https://github.com/pygame/pygame) - Pygame is a set of Python modules designed for writing games. + - [pyglet](https://github.com/pyglet/pyglet) - A cross-platform windowing and multimedia library for Python. + - [pygame-ce](https://github.com/pygame-community/pygame-ce) - An actively developed drop-in replacement with new features and performance improvements ([pygame](https://github.com/pygame/pygame) fork). + - [arcade](https://github.com/pythonarcade/arcade) - Arcade is a modern Python framework for crafting games with compelling graphics and sound. +- Visual Novels + - [renpy](https://github.com/renpy/renpy) - A Visual Novel engine. **Python Language** @@ -1019,11 +982,10 @@ _Awesome game development libraries._ _Implementations of Python._ - [cpython](https://github.com/python/cpython) - Default, most widely used implementation of the Python programming language written in C. -- [cython](https://github.com/cython/cython) - Optimizing Static Compiler for Python. -- [ironpython](https://github.com/IronLanguages/ironpython3) - Implementation of the Python programming language written in C#. - [micropython](https://github.com/micropython/micropython) - A lean and efficient Python programming language implementation. -- [pyodide](https://github.com/pyodide/pyodide) - Python distribution for the browser and Node.js based on WebAssembly. - [pypy](https://github.com/pypy/pypy) - A very fast and compliant implementation of the Python language. +- [Cython](https://github.com/cython/cython) - Optimizing Static Compiler for Python. +- [pyodide](https://github.com/pyodide/pyodide) - Python distribution for the browser and Node.js based on WebAssembly. ### Built-in Classes Enhancement @@ -1031,42 +993,44 @@ _Libraries for enhancing Python built-in classes._ - [attrs](https://github.com/python-attrs/attrs) - Replacement for `__init__`, `__eq__`, `__repr__`, etc. boilerplate in class definitions. - [bidict](https://github.com/jab/bidict) - Efficient, Pythonic bidirectional map data structures and related functionality. -- [box](https://github.com/cdgriffith/Box) - Python dictionaries with advanced dot notation access. - [uuid-utils](https://github.com/aminalaee/uuid-utils) - A fast, Rust-backed drop-in replacement for Python's built-in `uuid` module, supporting RFC 9562 (UUIDv6, UUIDv7, and UUIDv8). +- [python-box](https://github.com/cdgriffith/Box) - Python dictionaries with advanced dot notation access. ### Functional Programming _Functional Programming with Python._ -- [coconut](https://github.com/evhub/coconut) - A variant of Python built for simple, elegant, Pythonic functional programming. - [functools](https://docs.python.org/3/library/functools.html) - (Python standard library) Higher-order functions and operations on callable objects. -- [funcy](https://github.com/Suor/funcy) - A fancy and practical functional tools. - [more-itertools](https://github.com/more-itertools/more-itertools) - More routines for operating on iterables, beyond `itertools`. -- [returns](https://github.com/dry-python/returns) - A set of type-safe monads, transformers, and composition utilities. - [toolz](https://github.com/pytoolz/toolz) - A collection of functional utilities for iterators, functions, and dictionaries. Also available as [cytoolz](https://github.com/pytoolz/cytoolz/) for Cython-accelerated performance. +- [funcy](https://github.com/Suor/funcy) - A fancy and practical functional tools. +- [returns](https://github.com/dry-python/returns) - A set of type-safe monads, transformers, and composition utilities. ### Asynchronous Programming _Libraries for asynchronous, concurrent and parallel execution. Also see [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio)._ -- [anyio](https://github.com/agronholm/anyio) - A high-level async concurrency and networking framework that works on top of asyncio or trio. -- [asyncio](https://docs.python.org/3/library/asyncio.html) - (Python standard library) Asynchronous I/O, event loop, coroutines and tasks. - - [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio) -- [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) - (Python standard library) A high-level interface for asynchronously executing callables. -- [gevent](https://github.com/gevent/gevent) - A coroutine-based Python networking library that uses [greenlet](https://github.com/python-greenlet/greenlet). -- [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) - (Python standard library) Process-based parallelism. -- [trio](https://github.com/python-trio/trio) - A friendly library for async concurrency and I/O. -- [twisted](https://github.com/twisted/twisted) - An event-driven networking engine. -- [uvloop](https://github.com/MagicStack/uvloop) - Ultra fast asyncio event loop. +- Async I/O + - [asyncio](https://docs.python.org/3/library/asyncio.html) - (Python standard library) Asynchronous I/O, event loop, coroutines and tasks. + - [awesome-asyncio](https://github.com/timofurrer/awesome-asyncio) + - [anyio](https://github.com/agronholm/anyio) - A high-level async concurrency and networking framework that works on top of asyncio or trio. + - [uvloop](https://github.com/MagicStack/uvloop) - Ultra fast asyncio event loop. + - [trio](https://github.com/python-trio/trio) - A friendly library for async concurrency and I/O. + - [gevent](https://github.com/gevent/gevent) - A coroutine-based Python networking library that uses [greenlet](https://github.com/python-greenlet/greenlet). + - [Twisted](https://github.com/twisted/twisted) - An event-driven networking engine. +- Parallelism + - [concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html) - (Python standard library) A high-level interface for asynchronously executing callables. + - [multiprocessing](https://docs.python.org/3/library/multiprocessing.html) - (Python standard library) Process-based parallelism. ### Date and Time _Libraries for working with dates and times._ +- [zoneinfo](https://docs.python.org/3/library/zoneinfo.html) - (Python standard library) IANA time zone support. Brings the [tz database](https://en.wikipedia.org/wiki/Tz_database) into Python. +- [python-dateutil](https://github.com/dateutil/dateutil) - Extensions to the standard Python [datetime](https://docs.python.org/3/library/datetime.html) module. - [dateparser](https://github.com/scrapinghub/dateparser) - A Python parser for human-readable dates in dozens of languages. -- [dateutil](https://github.com/dateutil/dateutil) - Extensions to the standard Python [datetime](https://docs.python.org/3/library/datetime.html) module. - [pendulum](https://github.com/python-pendulum/pendulum) - Python datetimes made easy. -- [zoneinfo](https://docs.python.org/3/library/zoneinfo.html) - (Python standard library) IANA time zone support. Brings the [tz database](https://en.wikipedia.org/wiki/Tz_database) into Python. +- [whenever](https://github.com/ariebovenberg/whenever) - A modern datetime library, type-safe and DST-safe, backed by Rust. **Python Toolchain** @@ -1074,22 +1038,25 @@ _Libraries for working with dates and times._ _Libraries for Python version and virtual environment management._ -- [KillPy](https://github.com/Tlaloc-Es/killpy) - Analyze, detect, and clean unused Python environments and pipx packages. -- [pyenv](https://github.com/pyenv/pyenv) - Simple Python version management. -- [pyenv-win](https://github.com/pyenv-win/pyenv-win) - Pyenv for Windows. -- [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. - [virtualenv](https://github.com/pypa/virtualenv) - A tool to create isolated Python environments. +- [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. +- [pyenv](https://github.com/pyenv/pyenv) - Simple Python version management. ### Package Management _Libraries for package and dependency management._ -- [conda](https://github.com/conda/conda/) - Cross-platform, Python-agnostic binary package manager. -- [hatch](https://github.com/pypa/hatch) - Modern, extensible Python project manager for environments, builds, and publishing. -- [pip](https://github.com/pypa/pip) - The package installer for Python. -- [pipx](https://github.com/pypa/pipx) - Install and Run Python Applications in Isolated Environments. Like `npx` in Node.js. -- [poetry](https://github.com/python-poetry/poetry) - Python dependency management and packaging made easy. -- [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. +- Package Managers + - [pip](https://github.com/pypa/pip) - The package installer for Python. + - [uv](https://github.com/astral-sh/uv) - An extremely fast Python version, package and project manager, written in Rust. + - [poetry](https://github.com/python-poetry/poetry) - Python dependency management and packaging made easy. + - [hatch](https://github.com/pypa/hatch) - Modern, extensible Python project manager for environments, builds, and publishing. + - [pipx](https://github.com/pypa/pipx) - Install and Run Python Applications in Isolated Environments. Like `npx` in Node.js. + - [conda](https://github.com/conda/conda/) - Cross-platform, Python-agnostic binary package manager. +- Build Backends + - [setuptools](https://github.com/pypa/setuptools) - The historical and still most widely used pyproject build backend. + - [hatchling](https://github.com/pypa/hatch) - Modern, extensible build backend from the hatch project. + - [uv-build](https://github.com/astral-sh/uv) - uv's fast, minimal build backend for pure-Python projects. ### Package Repositories @@ -1103,38 +1070,50 @@ _Local PyPI repository server and proxies._ _Libraries to create packaged executables for release distribution._ -- [cx-Freeze](https://github.com/marcelotduarte/cx_Freeze) - It is a Python tool that converts Python scripts into standalone executables and installers for Windows, macOS, and Linux. -- [Nuitka](https://github.com/Nuitka/Nuitka) - Compiles Python programs into high-performance standalone executables (cross-platform, supports all Python versions). -- [pyarmor](https://github.com/dashingsoft/pyarmor) - A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts. -- [pyinstaller](https://github.com/pyinstaller/pyinstaller) - Converts Python programs into stand-alone executables (cross-platform). -- [shiv](https://github.com/linkedin/shiv) - A command line utility for building fully self-contained zipapps (PEP 441), but with all their dependencies included. +- Executables + - [pyinstaller](https://github.com/pyinstaller/pyinstaller) - Converts Python programs into stand-alone executables (cross-platform). + - [Nuitka](https://github.com/Nuitka/Nuitka) - Compiles Python programs into high-performance standalone executables (cross-platform, supports all Python versions). + - [shiv](https://github.com/linkedin/shiv) - A command line utility for building fully self-contained zipapps (PEP 441), but with all their dependencies included. + - [cx-Freeze](https://github.com/marcelotduarte/cx_Freeze) - It is a Python tool that converts Python scripts into standalone executables and installers for Windows, macOS, and Linux. +- Obfuscation + - [pyarmor](https://github.com/dashingsoft/pyarmor) - A tool used to obfuscate python scripts, bind obfuscated scripts to fixed machine or expire obfuscated scripts. ### Configuration Files _Libraries for storing and parsing configuration options._ - [configparser](https://docs.python.org/3/library/configparser.html) - (Python standard library) INI file parser. -- [dynaconf](https://github.com/dynaconf/dynaconf) - Dynaconf is a configuration manager with plugins for Django, Flask and FastAPI. -- [hydra](https://github.com/facebookresearch/hydra) - Hydra is a framework for elegantly configuring complex applications. -- [python-decouple](https://github.com/HBNetwork/python-decouple) - Strict separation of settings from code. - [python-dotenv](https://github.com/theskumar/python-dotenv) - Reads key-value pairs from a `.env` file and sets them as environment variables. +- [pydantic-settings](https://github.com/pydantic/pydantic-settings) - Settings management using Pydantic models with validation, loading from environment variables and secrets files. +- [hydra-core](https://github.com/facebookresearch/hydra) - Hydra is a framework for elegantly configuring complex applications. +- [dynaconf](https://github.com/dynaconf/dynaconf) - Dynaconf is a configuration manager with plugins for Django, Flask and FastAPI. **Security** ### Cryptography +_Libraries for cryptographic primitives and secure protocols._ + - [cryptography](https://github.com/pyca/cryptography) - A package designed to expose cryptographic primitives and recipes to Python developers. -- [paramiko](https://github.com/paramiko/paramiko) - The leading native Python SSHv2 protocol library. - [pynacl](https://github.com/pyca/pynacl) - Python binding to the Networking and Cryptography (NaCl) library. +- [paramiko](https://github.com/paramiko/paramiko) - The leading native Python SSHv2 protocol library. +- [itsdangerous](https://github.com/pallets/itsdangerous) - Various helpers to pass trusted data to untrusted environments. ### Penetration Testing _Frameworks and tools for penetration testing._ - [mitmproxy](https://github.com/mitmproxy/mitmproxy) - An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers. -- [setoolkit](https://github.com/trustedsec/social-engineer-toolkit) - A toolkit for social engineering. -- [sherlock](https://github.com/sherlock-project/sherlock) - Hunt down social media accounts by username across social networks. - [sqlmap](https://github.com/sqlmapproject/sqlmap) - Automatic SQL injection and database takeover tool. +- [sherlock-project](https://github.com/sherlock-project/sherlock) - Hunt down social media accounts by username across social networks. +- [social-engineer-toolkit](https://github.com/trustedsec/social-engineer-toolkit) - A toolkit for social engineering. + +### Supply Chain Security + +_Tools for auditing dependencies against known vulnerabilities._ + +- [pip-audit](https://github.com/pypa/pip-audit) - Audits Python environments and dependency trees for known vulnerabilities, using the PyPI Advisory Database and OSV. +- [uv audit](https://docs.astral.sh/uv/reference/cli/#uv-audit) - uv's built-in dependency vulnerability and malware scanning backed by OSV. ### Web Security @@ -1149,9 +1128,8 @@ _Libraries for application-layer web security._ _Libraries for programming with hardware._ - [bleak](https://github.com/hbldh/bleak) - A cross platform Bluetooth Low Energy Client for Python using asyncio. -- [jumpstarter](https://github.com/jumpstarter-dev/jumpstarter) - A hardware-in-the-loop testing framework with a Python client library for automated testing on real and virtual hardware. - [pynput](https://github.com/moses-palmer/pynput) - A library to control and monitor input devices. -- [synology-api](https://github.com/N4S4/synology-api) - Python wrapper for Synology NAS APIs: Surveillance Station, File Station, Download Station, Docker, and 50+ other endpoints. +- [jumpstarter](https://github.com/jumpstarter-dev/jumpstarter) - A hardware-in-the-loop testing framework with a Python client library for automated testing on real and virtual hardware. ### Microsoft Windows @@ -1159,6 +1137,7 @@ _Python programming on Microsoft Windows._ - [pythonnet](https://github.com/pythonnet/pythonnet) - Python Integration with the .NET Common Language Runtime (CLR). - [pywin32](https://github.com/mhammond/pywin32) - Python Extensions for Windows. +- [pyenv-win](https://github.com/pyenv-win/pyenv-win) - A Python version manager for Windows ([pyenv](https://github.com/pyenv/pyenv) fork). - [winpython](https://github.com/winpython/winpython) - Portable development environment for Windows 10/11. ### Miscellaneous @@ -1167,8 +1146,6 @@ _Useful libraries or tools that don't fit in the categories above._ - [blinker](https://github.com/pallets-eco/blinker) - A fast Python in-process signal/event dispatching system. - [boltons](https://github.com/mahmoud/boltons) - A set of pure-Python utilities. -- [itsdangerous](https://github.com/pallets/itsdangerous) - Various helpers to pass trusted data to untrusted environments. -- [tryton](https://github.com/tryton/tryton) - A general-purpose business framework. ## Resources diff --git a/docs/adr/0001-shortlist-not-catalog.md b/docs/adr/0001-shortlist-not-catalog.md new file mode 100644 index 0000000000..095fad3866 --- /dev/null +++ b/docs/adr/0001-shortlist-not-catalog.md @@ -0,0 +1,26 @@ +--- +status: accepted +--- + +# awesome-python is a shortlist of obvious choices, not a catalog + +By mid-2026 the list held 576 entries across 75 sections, with entry inflow up 24x year over year (96 additions in the trailing 12 months vs 4 the year before), concentrated in sections like AI and Agents (41 entries). The old acceptance model — three lanes (Industry Standard, Rising Star, Hidden Gem), only the first capped — admitted any project good enough in isolation, so categories grew without bound and stopped answering the reader's actual question: "what do I use for X?" We decided to reposition the list as a curated shortlist, per the Zen of Python: there should be one — and preferably only one — obvious way to do it. + +## The decision + +The list's scope test is "serves Python developers", replacing the old "primarily written in Python (>50%)" requirement: implementation language and packaging are irrelevant when Python developers use the thing in their Python work (uv and ty are Rust; agent skill packs are markdown), while a pure-Python project nobody uses in Python work does not belong. Each Use Case (a subcategory, or a flat section) lists at most its Obvious Choices — up to 3, plus up to 2 marked Challengers, hard maximum 5 (numbers provisional, to be reviewed after the prune). Admission is by maintainer editorial judgment, informed primarily by PyPI download counts rather than GitHub stars, and stated as final; judgment overrides the signal's known failure modes (CI/dependency-inflated counts, model releases consumed as weights rather than pip installs, large-but-specific audiences misread as "niche"). Once a Use Case is at cap, the only way in is Displacement: the PR names the entry it replaces and argues the newcomer does that job better. Use Cases are defined by the list's existing structure; an entry PR can never create the subcategory it needs. Standard-library entries hold a slot only where the stdlib module is itself the obvious choice. The existing stock gets the same test retroactively: a staged, worst-first prune (per-section sweep commits), with removed entries deleted outright — git history is the archive. Resources sections (Newsletters, Podcasts, Websites) are out of scope for now. + +## Considered options + +- Keep the three-lane model with caps on every lane: rejected because the lanes answer the wrong question ("is this good enough to enter?") once admission is comparative; rising-star momentum becomes evidence for a Challenger slot or a Displacement rather than an admission ticket of its own, and Hidden Gem is definitionally incompatible with "obvious". +- Rules-only, no retroactive prune: rejected because every rejection would face "but X is listed" precedent arguments, and readers would see no change. +- Archive removed entries in a separate file: rejected because it recreates the catalog one click away and dilutes the identity the change exists to restore. + +## Consequences + +- The list shrinks substantially: the maintainer-reviewed preview of the three largest sections kept 45 of 80 entries. Most future PRs will be rejected for fullness, not badness. +- awesome-python.com loses long-tail search traffic for the hundreds of niche tool names it will no longer carry. Accepted deliberately: reader trust over search surface. +- Fast-moving domains (e.g. AI and Agents) list current leaders by usage and absorb churn through Displacement; an oversized Use Case is either trimmed or Split into finer Use Cases by the maintainer. +- Linked awesome-* lists (e.g. awesome-python-testing) remain as the escape valve for readers who want exhaustive catalogs. + +See CONTEXT.md for the vocabulary (Use Case, Obvious Choice, Challenger, Displacement, Split). diff --git a/docs/audit-logs.md b/docs/audit-logs.md new file mode 100644 index 0000000000..80407a2ff7 --- /dev/null +++ b/docs/audit-logs.md @@ -0,0 +1,33 @@ +# Audit Log + +awesome-python is audited section by section. Every entry gets re-verified against live data, and every removal lands in a commit whose body carries the reason. Git history is the archive. This file is the at-a-glance register of maintainer decisions that a single commit can't show. + +## Overrides + +[CONTRIBUTING.md](../CONTRIBUTING.md) allows the maintainer to exceed any limit for a specific entry or use case. Each override is recorded here. + +### Naming Exceptions + +Display names follow the canonical PyPI package name. These entries keep a different name by maintainer decision: + +- autobahn-python -- `autobahn`. +- django-rest-framework -- `djangorestframework`. +- django-rules -- `rules`. +- fasthtml -- `python-fasthtml`. +- jinja -- `Jinja2`. +- mem0 -- `mem0ai`. +- pangu.py -- `pangu`. +- playwright-python -- `playwright`. +- pytorch -- `torch`. +- strawberry -- `strawberry-graphql`. +- strawberry-django -- `strawberry-graphql-django`. + +### Mature-stable Keeps + +These entries sit past the 12-month activity requirement without an override. Each one is kept by editorial judgment: mature, stable, and no successor exists. + +- ftfy +- itsdangerous +- jieba +- jinja +- sortedcontainers diff --git a/pyproject.toml b/pyproject.toml index a1f09178f5..30ce8fc591 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ description = "An opinionated list of awesome Python frameworks, libraries, soft authors = [{ name = "Vinta Chen", email = "vinta.chen@gmail.com" }] readme = "README.md" license = "MIT" -requires-python = ">=3.13" +requires-python = ">=3.14" dependencies = [] [project.urls] @@ -13,14 +13,15 @@ Homepage = "https://awesome-python.com/" Repository = "https://github.com/vinta/awesome-python" [dependency-groups] -build = ["httpx==0.28.1", "jinja2==3.1.6", "markdown-it-py==4.0.0"] -lint = ["ruff==0.15.6", "ty==0.0.33"] -test = ["pytest==9.0.3"] +build = ["httpx>=0.28.1", "jinja2>=3.1.6", "markdown-it-py>=4.0.0"] +lint = ["ruff>=0.15.6", "ty>=0.0.33"] +test = ["pytest>=9.0.3"] +preview = ["watchfiles>=1.2.0"] dev = [ { include-group = "build" }, { include-group = "lint" }, { include-group = "test" }, - "watchdog==6.0.0", + { include-group = "preview" }, ] [tool.uv] @@ -34,7 +35,7 @@ only-binary = [":all:"] line-length = 200 [tool.ty.environment] -python-version = "3.13" +python-version = "3.14" root = ["website"] [tool.ty.terminal] diff --git a/uv.lock b/uv.lock index 6ac5047c3f..a3d6793cb8 100644 --- a/uv.lock +++ b/uv.lock @@ -1,6 +1,6 @@ version = 1 revision = 3 -requires-python = ">=3.13" +requires-python = ">=3.14" [options] exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. @@ -36,12 +36,15 @@ dev = [ { name = "pytest" }, { name = "ruff" }, { name = "ty" }, - { name = "watchdog" }, + { name = "watchfiles" }, ] lint = [ { name = "ruff" }, { name = "ty" }, ] +preview = [ + { name = "watchfiles" }, +] test = [ { name = "pytest" }, ] @@ -50,24 +53,25 @@ test = [ [package.metadata.requires-dev] build = [ - { name = "httpx", specifier = "==0.28.1" }, - { name = "jinja2", specifier = "==3.1.6" }, - { name = "markdown-it-py", specifier = "==4.0.0" }, + { name = "httpx", specifier = ">=0.28.1" }, + { name = "jinja2", specifier = ">=3.1.6" }, + { name = "markdown-it-py", specifier = ">=4.0.0" }, ] dev = [ - { name = "httpx", specifier = "==0.28.1" }, - { name = "jinja2", specifier = "==3.1.6" }, - { name = "markdown-it-py", specifier = "==4.0.0" }, - { name = "pytest", specifier = "==9.0.3" }, - { name = "ruff", specifier = "==0.15.6" }, - { name = "ty", specifier = "==0.0.33" }, - { name = "watchdog", specifier = "==6.0.0" }, + { name = "httpx", specifier = ">=0.28.1" }, + { name = "jinja2", specifier = ">=3.1.6" }, + { name = "markdown-it-py", specifier = ">=4.0.0" }, + { name = "pytest", specifier = ">=9.0.3" }, + { name = "ruff", specifier = ">=0.15.6" }, + { name = "ty", specifier = ">=0.0.33" }, + { name = "watchfiles", specifier = ">=1.2.0" }, ] lint = [ - { name = "ruff", specifier = "==0.15.6" }, - { name = "ty", specifier = "==0.0.33" }, + { name = "ruff", specifier = ">=0.15.6" }, + { name = "ty", specifier = ">=0.0.33" }, ] -test = [{ name = "pytest", specifier = "==9.0.3" }] +preview = [{ name = "watchfiles", specifier = ">=1.2.0" }] +test = [{ name = "pytest", specifier = ">=9.0.3" }] [[package]] name = "certifi" @@ -172,28 +176,6 @@ version = "3.0.3" source = { registry = "https://pypi.org/simple" } sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, - { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, - { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, - { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, - { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, - { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, - { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, - { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, - { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, - { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, - { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, - { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, - { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, - { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, - { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, - { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, - { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, - { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, - { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, - { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, - { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, - { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, @@ -320,22 +302,48 @@ wheels = [ ] [[package]] -name = "watchdog" -version = "6.0.0" +name = "watchfiles" +version = "1.2.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/db/7d/7f3d619e951c88ed75c6037b246ddcf2d322812ee8ea189be89511721d54/watchdog-6.0.0.tar.gz", hash = "sha256:9ddf7c82fda3ae8e24decda1338ede66e1c99883db93711d8fb941eaa2d8c282", size = 131220, upload-time = "2024-11-01T14:07:13.037Z" } +dependencies = [ + { name = "anyio" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cd/41/5e1a4bb12aac5f1493fa1bdc11154eca3b258ca4eba65d39c473fe19d8e9/watchfiles-1.2.0.tar.gz", hash = "sha256:c995fba777f1ea992f090f9236e9284cf7a5d1a0130dd5a3d82c598cacd76838", size = 108252, upload-time = "2026-05-18T04:32:04.251Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/68/98/b0345cabdce2041a01293ba483333582891a3bd5769b08eceb0d406056ef/watchdog-6.0.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:490ab2ef84f11129844c23fb14ecf30ef3d8a6abafd3754a6f75ca1e6654136c", size = 96480, upload-time = "2024-11-01T14:06:42.952Z" }, - { url = "https://files.pythonhosted.org/packages/85/83/cdf13902c626b28eedef7ec4f10745c52aad8a8fe7eb04ed7b1f111ca20e/watchdog-6.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:76aae96b00ae814b181bb25b1b98076d5fc84e8a53cd8885a318b42b6d3a5134", size = 88451, upload-time = "2024-11-01T14:06:45.084Z" }, - { url = "https://files.pythonhosted.org/packages/fe/c4/225c87bae08c8b9ec99030cd48ae9c4eca050a59bf5c2255853e18c87b50/watchdog-6.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a175f755fc2279e0b7312c0035d52e27211a5bc39719dd529625b1930917345b", size = 89057, upload-time = "2024-11-01T14:06:47.324Z" }, - { url = "https://files.pythonhosted.org/packages/a9/c7/ca4bf3e518cb57a686b2feb4f55a1892fd9a3dd13f470fca14e00f80ea36/watchdog-6.0.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:7607498efa04a3542ae3e05e64da8202e58159aa1fa4acddf7678d34a35d4f13", size = 79079, upload-time = "2024-11-01T14:06:59.472Z" }, - { url = "https://files.pythonhosted.org/packages/5c/51/d46dc9332f9a647593c947b4b88e2381c8dfc0942d15b8edc0310fa4abb1/watchdog-6.0.0-py3-none-manylinux2014_armv7l.whl", hash = "sha256:9041567ee8953024c83343288ccc458fd0a2d811d6a0fd68c4c22609e3490379", size = 79078, upload-time = "2024-11-01T14:07:01.431Z" }, - { url = "https://files.pythonhosted.org/packages/d4/57/04edbf5e169cd318d5f07b4766fee38e825d64b6913ca157ca32d1a42267/watchdog-6.0.0-py3-none-manylinux2014_i686.whl", hash = "sha256:82dc3e3143c7e38ec49d61af98d6558288c415eac98486a5c581726e0737c00e", size = 79076, upload-time = "2024-11-01T14:07:02.568Z" }, - { url = "https://files.pythonhosted.org/packages/ab/cc/da8422b300e13cb187d2203f20b9253e91058aaf7db65b74142013478e66/watchdog-6.0.0-py3-none-manylinux2014_ppc64.whl", hash = "sha256:212ac9b8bf1161dc91bd09c048048a95ca3a4c4f5e5d4a7d1b1a7d5752a7f96f", size = 79077, upload-time = "2024-11-01T14:07:03.893Z" }, - { url = "https://files.pythonhosted.org/packages/2c/3b/b8964e04ae1a025c44ba8e4291f86e97fac443bca31de8bd98d3263d2fcf/watchdog-6.0.0-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:e3df4cbb9a450c6d49318f6d14f4bbc80d763fa587ba46ec86f99f9e6876bb26", size = 79078, upload-time = "2024-11-01T14:07:05.189Z" }, - { url = "https://files.pythonhosted.org/packages/62/ae/a696eb424bedff7407801c257d4b1afda455fe40821a2be430e173660e81/watchdog-6.0.0-py3-none-manylinux2014_s390x.whl", hash = "sha256:2cce7cfc2008eb51feb6aab51251fd79b85d9894e98ba847408f662b3395ca3c", size = 79077, upload-time = "2024-11-01T14:07:06.376Z" }, - { url = "https://files.pythonhosted.org/packages/b5/e8/dbf020b4d98251a9860752a094d09a65e1b436ad181faf929983f697048f/watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:20ffe5b202af80ab4266dcd3e91aae72bf2da48c0d33bdb15c66658e685e94e2", size = 79078, upload-time = "2024-11-01T14:07:07.547Z" }, - { url = "https://files.pythonhosted.org/packages/07/f6/d0e5b343768e8bcb4cda79f0f2f55051bf26177ecd5651f84c07567461cf/watchdog-6.0.0-py3-none-win32.whl", hash = "sha256:07df1fdd701c5d4c8e55ef6cf55b8f0120fe1aef7ef39a1c6fc6bc2e606d517a", size = 79065, upload-time = "2024-11-01T14:07:09.525Z" }, - { url = "https://files.pythonhosted.org/packages/db/d9/c495884c6e548fce18a8f40568ff120bc3a4b7b99813081c8ac0c936fa64/watchdog-6.0.0-py3-none-win_amd64.whl", hash = "sha256:cbafb470cf848d93b5d013e2ecb245d4aa1c8fd0504e863ccefa32445359d680", size = 79070, upload-time = "2024-11-01T14:07:10.686Z" }, - { url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" }, + { url = "https://files.pythonhosted.org/packages/e7/54/a9c7ea9a82a4ac65e7004c0a03920b5cdd2f9c3b678757d9cd425aa51d53/watchfiles-1.2.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b8c8358484d5fa12ef34f05b7f4168eaf1932f408725ff6d023c33ec17bd79d4", size = 400205, upload-time = "2026-05-18T04:32:05.153Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5d/c9ab3534374a4a67450696905d6ef16a04405448b8dc52bd752ae50423d4/watchfiles-1.2.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:9f04b092229ad2c50126dd3c922c8822e51e605993764a33058d4a791ab42281", size = 392508, upload-time = "2026-05-18T04:30:54.849Z" }, + { url = "https://files.pythonhosted.org/packages/26/ca/1ad30103535cf0cecd7b993e8d50edc5351b1820e38f2d22e3df58962feb/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7a7ce236284f002a156f70add88efe5c70879cccbb658be0822c54b1306fc09d", size = 452448, upload-time = "2026-05-18T04:30:53.727Z" }, + { url = "https://files.pythonhosted.org/packages/37/a1/ceee2cdf2afbd715fa07758d39c9859513eae411b23196f7fd039e5feedd/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:b9909cc2b48468b575eefa944919e1fe8a36c5849d5c7c168f80a8c1db69398e", size = 459605, upload-time = "2026-05-18T04:30:23.312Z" }, + { url = "https://files.pythonhosted.org/packages/e8/f6/421e30fd1cb3907a84ed92ab3f1983e37ba2dca015e9a894a048418417a2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0a37faaed405c67e28e6be45a1fa4f206ef5a2860f27c237db9fa30704c38242", size = 490757, upload-time = "2026-05-18T04:30:47.358Z" }, + { url = "https://files.pythonhosted.org/packages/41/b0/55ed1b97ed08be7bba6f9a541cac15f2a858e1d74d2b07b6da70a82aab00/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9649193aa27bd9ff2e80ff29bfaa93085496c7a3a377592823cc58b77ee88add", size = 568672, upload-time = "2026-05-18T04:30:38.915Z" }, + { url = "https://files.pythonhosted.org/packages/d1/cf/d8ae8a80dd7bafab395ea7681c10237311bbf34d37704a8c744e7cf31fc7/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:4e4ff8e37f99cf1da89e255e07c9c4b37c214038c4283707bdec308cb1b0ea1f", size = 464197, upload-time = "2026-05-18T04:30:09.914Z" }, + { url = "https://files.pythonhosted.org/packages/7c/8a/3076c496ca8dafe0e8cd03fcebdfc47be4b1174b4e5b24ff6e396e6b3af2/watchfiles-1.2.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:054dc20fd2e3132b4c3883b4a00d72fd6e1f56fdaf89fccd12e8057d74cd74d7", size = 453181, upload-time = "2026-05-18T04:30:14.829Z" }, + { url = "https://files.pythonhosted.org/packages/e5/10/9745e17c98e7b8a86454df0a3c7b5686bd650383f1e9f26e4ebcbd6cc0c0/watchfiles-1.2.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:e140ed30ebde76796b686e67c182cff10ea2fbab186fafd1560f74bb5a473a6e", size = 465109, upload-time = "2026-05-18T04:30:28.123Z" }, + { url = "https://files.pythonhosted.org/packages/8f/95/8ef4a95481d3e0cb52d62a06fa6e972e81424be2d9698b91a2fecca9904c/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:bb7e52ecf68ba46d22df23467b87cffeb2146908aa523ebfe803019618cfda06", size = 630653, upload-time = "2026-05-18T04:31:49.304Z" }, + { url = "https://files.pythonhosted.org/packages/fd/e4/3b3bf36b0f829b50c6ebcb8d031583863c59f923d6a6af3d485e470d0fac/watchfiles-1.2.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:23282a321c8baf9b3a3c4afff673f9fe65eb7fdc2338d765ccad9d3d1916a5ba", size = 657838, upload-time = "2026-05-18T04:31:06.497Z" }, + { url = "https://files.pythonhosted.org/packages/21/b1/6cbbb50c1f3002ab568777d44aa21206dfb8807a840990c4037523b51812/watchfiles-1.2.0-cp314-cp314-win32.whl", hash = "sha256:c0db965c5f79aa49fe672d297cf1febc5ad149b658594944f49a54a2b96270a7", size = 275108, upload-time = "2026-05-18T04:30:06.891Z" }, + { url = "https://files.pythonhosted.org/packages/92/45/190ce6db8dcb4536682cf75d3889ff1a27182a58cb519d343cb6d9ea63d8/watchfiles-1.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:71283b39fd17e5408eb123bd37aeecfd9d54c81fc184421943208aadb879d103", size = 288441, upload-time = "2026-05-18T04:32:12.901Z" }, + { url = "https://files.pythonhosted.org/packages/74/0d/3eae1c2313ab08378431d907c3f8095ecca00f3eda33111cf4f0f2591799/watchfiles-1.2.0-cp314-cp314-win_arm64.whl", hash = "sha256:c5c19526f4e54a00f2666a6c0e9e40d582c09e865055ea7378bf0009aab857b3", size = 280684, upload-time = "2026-05-18T04:31:26.902Z" }, + { url = "https://files.pythonhosted.org/packages/b1/75/fb64e6c25d6b5ca636d03df34ffb1c6e9873303e76d27967e045f8df088f/watchfiles-1.2.0-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:d73a585accffa5ae39c17264c36ec3166d2fad7000c780f5ef83b2722afb9dd2", size = 398857, upload-time = "2026-05-18T04:32:17.108Z" }, + { url = "https://files.pythonhosted.org/packages/73/4e/9f7adf01754cbf81843722ccfec169d8f26c69778281a302855cecd2ee08/watchfiles-1.2.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ae99b14c5f21e026e0e9d96f40e07d8570ebee6cafd9d8fc318354606daa7a28", size = 392413, upload-time = "2026-05-18T04:31:07.911Z" }, + { url = "https://files.pythonhosted.org/packages/47/c8/bec626bcc2d69f44b9acb24ce7d60ed7b16b73628eea747fcbd169d8edda/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4429f3b105524a10b72c3a819b091c495d2811d419c1e1e8df773a5a5974f831", size = 452409, upload-time = "2026-05-18T04:31:20.142Z" }, + { url = "https://files.pythonhosted.org/packages/00/b7/b6362068e81e7c556d155a34c35d40ac3ef42d747b06d7f6e5bf58e359c2/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:43d818978d06062d9b22c4fab2ebe44cf5213d42dc8e62bda8c2760cfa2eeb33", size = 458827, upload-time = "2026-05-18T04:32:06.219Z" }, + { url = "https://files.pythonhosted.org/packages/67/f8/9a813fa42afb1e0b4625e75f0479826644d3ee8dc287e093799bc01f390c/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b9f732dc58b2dbe69e464ccf8fff7a03b0dd0be439da4c0720d3558527d3d6b4", size = 490104, upload-time = "2026-05-18T04:31:56.034Z" }, + { url = "https://files.pythonhosted.org/packages/2f/bf/27dfb6094ca4c9aad21298b5525b6c53cb36121ee454331d05161e58d130/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8f200104103feb097de4cab8fe4f5dd18a2026934c7dea98c55a2f5fd6d5a33b", size = 571360, upload-time = "2026-05-18T04:31:57.133Z" }, + { url = "https://files.pythonhosted.org/packages/fb/39/44a096d67270ea93df91d33877dbe91fbda3aa4f8ec2edf799d93eda8736/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:63ac26eefbf4af1741247d6fb68b11c49a25b2f7413fbd318a83a12aaa9cf666", size = 464644, upload-time = "2026-05-18T04:30:57.33Z" }, + { url = "https://files.pythonhosted.org/packages/0e/80/c7472203bad6268e3ef1ad260739704847898938ad7ea8b63a5131f46b50/watchfiles-1.2.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0c4997d4e4a55f0d02b6cde327322daf3a0400e5df6c6b15948994bf72497925", size = 454771, upload-time = "2026-05-18T04:30:48.736Z" }, + { url = "https://files.pythonhosted.org/packages/51/cf/3b10b268b4b7f0fc26e9debb5eef1998b515887840f444cd3ec80c688755/watchfiles-1.2.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:4c887eba18b7945ac73067a8b4a66f21cd46c2539b2bc68588f7be6c7eb6d26b", size = 463494, upload-time = "2026-05-18T04:31:33.826Z" }, + { url = "https://files.pythonhosted.org/packages/3d/3e/a4302545cd589262a0dc7d140e86f7688eba3f9c72776c27f7e23b8864c4/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:3416ff151bb6b5a8d8d11664974fbef4d9305b9b2957839ab5a270468fd8df30", size = 629383, upload-time = "2026-05-18T04:31:15.596Z" }, + { url = "https://files.pythonhosted.org/packages/db/99/d5649df0a9a410d45b7c882304d0b790903ac9b6e8f2cfd12114e0c6b9f2/watchfiles-1.2.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:0e831a271c035d89789cffc386b6aa1375f39f1cd25eb7ca0997e4970d152fc5", size = 656093, upload-time = "2026-05-18T04:31:58.707Z" }, + { url = "https://files.pythonhosted.org/packages/92/b9/362702539275019a54dd2e94511b31a9b89c5f9e6a21966de7eb692549fc/watchfiles-1.2.0-cp315-cp315-macosx_10_12_x86_64.whl", hash = "sha256:37a6721cdf3f65dbb13aa9503510ccb4451603ac837e44d265d7992a597e1374", size = 400109, upload-time = "2026-05-18T04:31:16.879Z" }, + { url = "https://files.pythonhosted.org/packages/8f/75/71d5ba62db781e5587bded1d944c675374bc4aa37ff33d5018d98e8b6538/watchfiles-1.2.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:2b37d10b5a63bd4d87e18472d80fa525bd670586fae62e5dd580452764879b65", size = 392167, upload-time = "2026-05-18T04:31:28.058Z" }, + { url = "https://files.pythonhosted.org/packages/3c/01/c66dd95d0423fe30d31820e2d1d5bda773764131bbb6ac0cb1cf303ac328/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0a105bc2283f67e8fbec74253ec2d94925de92ed72c0393f1206bf326b7b7b69", size = 452372, upload-time = "2026-05-18T04:31:00.836Z" }, + { url = "https://files.pythonhosted.org/packages/91/15/2fe99557e72f85627c6a8eed50d889e8d101623e060a22ad75b875cb932d/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:5327989a465505f05cfe06f04fa9d0c2fd5432bb243e10e6f012b1bdca3c8579", size = 459596, upload-time = "2026-05-18T04:31:34.96Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/d4acfa0023367428ed48351b3b9b267893037b6cadae55620c61c24bcfd4/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ecb47f183a8025b2aa18b546725c3657e542112ae9c0613a2af79b4fa8d04ad7", size = 490869, upload-time = "2026-05-18T04:31:59.923Z" }, + { url = "https://files.pythonhosted.org/packages/a4/5f/3164cbdce06c9fb95c4f7b9e2f9760b5e2797af43a9ecc317ef42a23a278/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8520a4ab0e37f770afc34459c4f8f7019e153f9124dc101c15538365875d1ab2", size = 571641, upload-time = "2026-05-18T04:32:00.948Z" }, + { url = "https://files.pythonhosted.org/packages/41/e6/85d3731c55e65cd7690f3f803d24c139588aaf863e4bf2148fe7a7fa1a19/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:71cd71740ed2c15211ebb237ced4e39a1cdf6f80566e5fe95428da1626f4fde6", size = 464444, upload-time = "2026-05-18T04:30:34.298Z" }, + { url = "https://files.pythonhosted.org/packages/f4/7d/562641012b8b09872742c3b8adf9629ec479fd78f8d68ae4a0c13da8add6/watchfiles-1.2.0-cp315-cp315-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f88af53d6ddaf72179ef613ddc905e6f4785f712b49b80b3bef9f3525e6194b4", size = 453593, upload-time = "2026-05-18T04:31:23.464Z" }, + { url = "https://files.pythonhosted.org/packages/56/fe/cb8ef3d6f929d14158fdaaad9925985b7310abc9384dcd4d82dd0016fb59/watchfiles-1.2.0-cp315-cp315-manylinux_2_31_riscv64.whl", hash = "sha256:cee9d5efd929efdac5f7e58f72b3376f676b64050a91c5b99a7094c5b2317488", size = 465096, upload-time = "2026-05-18T04:31:30.384Z" }, + { url = "https://files.pythonhosted.org/packages/25/91/80908e835e100527a9267147b08c0eee1fa6ab0ffec15edc04d1d44885f7/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_aarch64.whl", hash = "sha256:b718bf356bbc15e559bd8ef41782b573b8ae0e3f177ab244b440568d7ea02cfb", size = 630638, upload-time = "2026-05-18T04:30:49.89Z" }, + { url = "https://files.pythonhosted.org/packages/46/4b/95ab2f256bb4af3cb2eb23b9317bda984ee6e0f11733a5c004a6c95b06e3/watchfiles-1.2.0-cp315-cp315-musllinux_1_1_x86_64.whl", hash = "sha256:922c0e019fe68b3ae392965a766b02a71ba1168c932cebc3733cd52c5fe5b377", size = 657684, upload-time = "2026-05-18T04:31:32.027Z" }, ] diff --git a/website/build.py b/website/build.py index 347c4fb022..6ca467abef 100644 --- a/website/build.py +++ b/website/build.py @@ -11,6 +11,7 @@ from pathlib import Path from typing import TypedDict +from fetch_pypi_downloads_via_clickpy import normalize from jinja2 import Environment, FileSystemLoader from readme_parser import AlsoSee, ParsedGroup, ParsedSection, parse_readme, parse_sponsors, slugify @@ -52,6 +53,7 @@ class TemplateEntry(TypedDict): groups: list[str] subcategories: list[TemplateSubcategory] stars: int | None + downloads: int | None owner: str | None last_commit_at: str | None source_type: str | None @@ -96,21 +98,41 @@ def load_stars(path: Path) -> dict[str, dict]: return {} +def load_downloads(path: Path) -> dict[str, int]: + """Load last-30-day download counts from the TSV cache, keyed by normalized README name. + + Columns: name, package, downloads, fetched_at. Skips the header and + NOT_FOUND rows. Returns empty dict if the file doesn't exist. + """ + if not path.exists(): + return {} + downloads: dict[str, int] = {} + for line in path.read_text(encoding="utf-8").splitlines()[1:]: + parts = line.split("\t") + if len(parts) >= 3 and parts[2].isdigit(): + downloads[parts[0]] = int(parts[2]) + return downloads + + def sort_entries(entries: Sequence[TemplateEntry]) -> list[TemplateEntry]: - """Sort entries by stars descending, then name ascending. + """Sort entries by downloads descending, then name ascending. - Three tiers: starred entries first, stdlib second, other non-starred last. + Four tiers: entries with download counts first, then starred entries by + stars descending, then stdlib, then the rest. """ def sort_key(entry: TemplateEntry) -> tuple[int, int, int, str]: + downloads = entry["downloads"] stars = entry["stars"] name = entry["name"].lower() + if downloads is not None: + return (0, -downloads, 0, name) if stars is not None: builtin = 1 if entry.get("source_type") == "Built-in" else 0 - return (0, -stars, builtin, name) + return (1, -stars, builtin, name) if entry.get("source_type") == "Built-in": - return (1, 0, 0, name) - return (2, 0, 0, name) + return (2, 0, 0, name) + return (3, 0, 0, name) return sorted(entries, key=sort_key) @@ -481,6 +503,7 @@ def extract_entries( groups=[], subcategories=[], stars=None, + downloads=None, owner=None, last_commit_at=None, source_type=detect_source_type(entry["url"]), @@ -535,6 +558,7 @@ def build(repo_root: Path) -> None: build_date = datetime.now(UTC) stars_data = load_stars(website / "data" / "github_stars.json") + downloads_data = load_downloads(website / "data" / "pypi_downloads.tsv") repo_self = stars_data.get("vinta/awesome-python", {}) repo_stars = None @@ -551,6 +575,9 @@ def build(repo_root: Path) -> None: entry["stars"] = sd["stars"] entry["owner"] = sd["owner"] entry["last_commit_at"] = sd.get("last_commit_at", "") + # Built-in entries would hit same-named PyPI backports (e.g. asyncio), not the stdlib. + if entry.get("source_type") != "Built-in": + entry["downloads"] = downloads_data.get(normalize(entry["name"])) entries = sort_entries(entries) category_urls = {cat["name"]: category_path(cat) for cat in categories} diff --git a/website/data/pypi_name_overrides.json b/website/data/pypi_name_overrides.json new file mode 100644 index 0000000000..8c8a5f95f6 --- /dev/null +++ b/website/data/pypi_name_overrides.json @@ -0,0 +1,42 @@ +{ + "autobahn-python": { "package": "autobahn", "reason": null }, + "azure-sdk-for-python": { "package": null, "reason": "monorepo umbrella; ships as many azure-* packages, no single package represents it" }, + "cloud-init": { "package": null, "reason": "installed via distro images, not pip" }, + "concurrent-futures": { "package": null, "reason": "stdlib module" }, + "cpython": { "package": null, "reason": "not pip-installable; the cpython PyPI name was squatted" }, + "devpi": { "package": "devpi-server", "reason": "devpi is a deprecated metapackage; the server package carries real usage" }, + "difflib": { "package": null, "reason": "stdlib module" }, + "django-ai-plugins": { "package": null, "reason": "GitHub-only so far; young project, remap if it publishes" }, + "django-db-models": { "package": null, "reason": "part of Django, not a package" }, + "django-rest-framework": { "package": "djangorestframework", "reason": "django-rest-framework is a dead alias package" }, + "django-rules": { "package": "rules", "reason": "PyPI django-rules is an abandoned unrelated fork; dfunckt's project publishes as rules" }, + "fasthtml": { "package": "python-fasthtml", "reason": null }, + "geodjango": { "package": null, "reason": "part of Django (django.contrib.gis), not a package" }, + "google-cloud-python": { "package": null, "reason": "monorepo umbrella; ships as many google-cloud-* packages" }, + "graphify": { "package": null, "reason": "GitHub-only so far; young project, remap if it publishes" }, + "httpx-url": { "package": null, "reason": "class in the httpx package, not a package" }, + "jinja": { "package": "jinja2", "reason": "PyPI jinja is Jinja1; current releases publish as jinja2" }, + "mem0": { "package": "mem0ai", "reason": null }, + "micropython": { "package": null, "reason": "interpreter distributed as firmware builds, not pip" }, + "mimetypes": { "package": null, "reason": "stdlib module" }, + "odoo": { "package": null, "reason": "self-hosted suite installed from odoo.com, not pip" }, + "pangu-py": { "package": "pangu", "reason": null }, + "playwright-python": { "package": "playwright", "reason": null }, + "pyenv": { "package": null, "reason": "installed via git/homebrew; PyPI pyenv is a defunct shim" }, + "pypy": { "package": null, "reason": "interpreter distributed via pypy.org, not pip" }, + "python-patterns": { "package": null, "reason": "reference repo, not a package; PyPI name held by unrelated 2012 project" }, + "pytorch": { "package": "torch", "reason": "PyPI pytorch is a squatter; real package is torch" }, + "renpy": { "package": null, "reason": "distributed via renpy.org; PyPI name was squatted, now 404" }, + "sentry-skills": { "package": null, "reason": "agent skills repo, not a package" }, + "social-engineer-toolkit": { "package": null, "reason": "GitHub-only, installed from source" }, + "sqlite3": { "package": null, "reason": "stdlib module" }, + "strawberry": { "package": "strawberry-graphql", "reason": "PyPI strawberry is an unrelated bookmarking service" }, + "strawberry-django": { "package": "strawberry-graphql-django", "reason": null }, + "thealgorithms": { "package": null, "reason": "educational monorepo, not a package" }, + "tkinter": { "package": null, "reason": "stdlib module" }, + "tomllib": { "package": null, "reason": "stdlib module" }, + "trailofbits-skills": { "package": null, "reason": "agent skills repo, not a package" }, + "warehouse": { "package": null, "reason": "the PyPI server itself; deployed, not pip-installed" }, + "winpython": { "package": null, "reason": "distribution with its own installer; PyPI name is a 2012 relic" }, + "zoneinfo": { "package": null, "reason": "stdlib module" } +} diff --git a/website/fetch_github_stars.py b/website/fetch_github_stars.py index 48aaacf79c..3f4518854e 100644 --- a/website/fetch_github_stars.py +++ b/website/fetch_github_stars.py @@ -1,5 +1,10 @@ #!/usr/bin/env python3 -"""Fetch GitHub star counts and owner info for all GitHub repos in README.md.""" +"""Fetch GitHub star counts and owner info for all GitHub repos in README.md. + +The output file (data/github_stars.json) is gitignored; CI fetches it at +deploy time, so local runs are for previews only — never commit the data. +Entries removed from README.md leave harmless orphan keys behind. +""" import json import os diff --git a/website/fetch_pypi_downloads_via_bigquery.py b/website/fetch_pypi_downloads_via_bigquery.py new file mode 100644 index 0000000000..e8b5bbfc57 --- /dev/null +++ b/website/fetch_pypi_downloads_via_bigquery.py @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +"""Cross-check last-30-day PyPI download counts for given names via BigQuery. + +Queries the canonical source ClickPy mirrors — +`bigquery-public-data.pypi.file_downloads` via the `bq` CLI — and prints +namecount TSV to stdout. Names resolve through +data/pypi_name_overrides.json, matching the cache sweep. Maintainer-local (needs a personal GCP +account) and print-only: data/pypi_downloads.tsv is written solely by +fetch_pypi_downloads_via_clickpy.py. + +Targeted cross-checks only — full-README sweeps belong to ClickPy. The +dry-run scan estimate scales with the IN list (measured 2026-08-16: +~38 GB for 2 names, ~275 GB for 53, ~1.2 TB for the full README — past +both MAX_BYTES_BILLED and the 1 TiB/month free tier), and BigQuery +enforces MAX_BYTES_BILLED against that pre-run estimate, so oversized +name lists fail before billing. Actual billed bytes come in far lower +via cluster pruning (33.7 GB for a single name). The table is +partitioned on timestamp and clustered on the top-level `project` +column — filter on `project`, never `file.project`: both hold identical +values (verified 2026-08-16, 408M rows on one day, zero mismatches), but +only a `project` filter gets cluster pruning, and the `file` record +costs ~4x more to scan. Always --dry-run first to check the estimate. + +Usage: python fetch_pypi_downloads_via_bigquery.py [--dry-run] NAME [NAME ...] +""" + +import subprocess +import sys +from json import loads + +from fetch_pypi_downloads_via_clickpy import resolve + +MAX_BYTES_BILLED = 400_000_000_000 + + +def fetch_bigquery(names: list[str], dry_run: bool) -> dict[str, int]: + in_list = ", ".join(f"'{name}'" for name in names) + query = ( + "SELECT project, COUNT(*) AS downloads " + "FROM `bigquery-public-data.pypi.file_downloads` " + "WHERE DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) " + "AND CURRENT_DATE() " + f"AND project IN ({in_list}) " + "GROUP BY project" + ) + cmd = ["bq", "query", "--use_legacy_sql=false", "--format=json", f"--maximum_bytes_billed={MAX_BYTES_BILLED}"] + if dry_run: + cmd.append("--dry_run") + cmd.append(query) + result = subprocess.run(cmd, capture_output=True, text=True) + if result.returncode != 0: + print(result.stderr, file=sys.stderr) + sys.exit(1) + if dry_run: + print(result.stdout.strip() or result.stderr.strip()) + sys.exit(0) + rows = loads(result.stdout) + return {row["project"]: int(row["downloads"]) for row in rows} + + +def main() -> None: + dry_run = "--dry-run" in sys.argv + names = set() + for arg in sys.argv[1:]: + if arg == "--dry-run": + continue + pkg = resolve(arg) + if pkg is None: + print(f"{arg}: not pip-installable per pypi_name_overrides.json, skipping", file=sys.stderr) + else: + names.add(pkg) + names = sorted(names) + if not names: + print("Usage: python fetch_pypi_downloads_via_bigquery.py [--dry-run] NAME [NAME ...]", file=sys.stderr) + sys.exit(1) + counts = fetch_bigquery(names, dry_run) + for name in names: + print(f"{name}\t{counts.get(name, 'NOT_FOUND')}") + + +if __name__ == "__main__": + main() diff --git a/website/fetch_pypi_downloads_via_clickpy.py b/website/fetch_pypi_downloads_via_clickpy.py new file mode 100644 index 0000000000..c7c979f2cf --- /dev/null +++ b/website/fetch_pypi_downloads_via_clickpy.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +"""Fetch last-30-day PyPI download counts for all README entries via ClickPy. + +ClickPy is ClickHouse's public mirror of the PyPI download dataset +(sql-clickhouse.clickhouse.com, user `demo`, no key, no personal +account): one batched query covers the full README in under a second. +Playground limits — 60 queries/hour/IP, 1000 result rows — sit far above +a full-README sweep. The per-day table is a SummingMergeTree, so the +query must sum(count) GROUP BY, never count raw rows. Counts include +mirror/CI traffic; never mix these figures with pypistats.org, which +excludes mirrors by default. + +This script is the sole writer of data/pypi_downloads.tsv and rewrites it +from scratch each run, so entries removed from README.md drop out +naturally; names with no PyPI rows are written as NOT_FOUND. Counts are +looked up by README display name; when the display name differs from the +canonical PyPI package (a squatter or a dead predecessor would be +measured otherwise), add the mapping to the curated +data/pypi_name_overrides.json — normalized README name to +{"package": ..., "reason": ...}, where package is the real PyPI name +or null for projects that are not pip-installable so their row is +never queried, and reason records why. The file starts with a header row (name, package, +downloads, fetched_at) — package is the PyPI package the row actually +measured ("-" for null overrides) — and every row carries the sweep +date: a cache fetched within the last 7 days is current enough for +audit verdicts, so only re-run when older. Cross-checks against other +sources (fetch_pypi_downloads_via_bigquery.py, +fetch_pypi_downloads_via_pepy.py) print to stdout and never touch the +cache. + +Usage: python fetch_pypi_downloads_via_clickpy.py +""" + +import json +import re +from datetime import date +from pathlib import Path + +import httpx +from readme_parser import parse_readme + +DATA_DIR = Path(__file__).parent / "data" +OUT_FILE = DATA_DIR / "pypi_downloads.tsv" +OVERRIDES_FILE = DATA_DIR / "pypi_name_overrides.json" +README_PATH = Path(__file__).parent.parent / "README.md" +CLICKPY_URL = "https://sql-clickhouse.clickhouse.com/?user=demo" + +# PyPI normalizes names to lowercase with runs of -, _, . collapsed to -. +PYPI_NAME_RE = re.compile(r"^[a-z0-9]([a-z0-9-]*[a-z0-9])?$") + + +def normalize(name: str) -> str: + return re.sub(r"[-_.]+", "-", name.lower()) + + +def load_overrides() -> dict[str, str | None]: + """Load curated overrides: normalized README name -> PyPI package to measure (None = never query). + + Each file entry is {"package": str | null, "reason": str | null}; the reason is + documentation only — required for null packages, optional for remaps. + """ + raw = json.loads(OVERRIDES_FILE.read_text()) + return {name: entry["package"] for name, entry in raw.items()} + + +def resolve(name: str) -> str | None: + """Map a README display name to the PyPI package to measure. None = not pip-installable.""" + normalized = normalize(name) + return load_overrides().get(normalized, normalized) + + +def collect_names(readme_text: str) -> list[str]: + names = set() + for group in parse_readme(readme_text): + for section in group["categories"]: + for entry in section["entries"]: + normalized = normalize(entry["name"]) + if PYPI_NAME_RE.match(normalized): + names.add(normalized) + return sorted(names) + + +def fetch_clickpy(names: list[str]) -> dict[str, int]: + in_list = ", ".join(f"'{name}'" for name in names) + query = ( + "SELECT project, sum(count) AS downloads " + "FROM pypi.pypi_downloads_per_day " + f"WHERE project IN ({in_list}) AND date >= today() - 30 " + "GROUP BY project FORMAT JSON" + ) + resp = httpx.post(CLICKPY_URL, content=query, timeout=60) + resp.raise_for_status() + return {row["project"]: int(row["downloads"]) for row in resp.json()["data"]} + + +def main() -> None: + names = collect_names(README_PATH.read_text()) + overrides = load_overrides() + packages = {name: overrides.get(name, name) for name in names} + query_names = sorted({pkg for pkg in packages.values() if pkg}) + print(f"Querying {len(query_names)} package names...") + counts = fetch_clickpy(query_names) + fetched_at = date.today().isoformat() + rows = "\n".join(f"{name}\t{pkg or '-'}\t{counts.get(pkg, 'NOT_FOUND') if pkg else 'NOT_FOUND'}\t{fetched_at}" for name, pkg in packages.items()) + OUT_FILE.write_text(f"name\tpackage\tdownloads\tfetched_at\n{rows}\n") + print(f"Done. {len(counts)}/{len(query_names)} names found on PyPI. Cached to {OUT_FILE}") + + +if __name__ == "__main__": + main() diff --git a/website/fetch_pypi_downloads_via_pepy.py b/website/fetch_pypi_downloads_via_pepy.py new file mode 100644 index 0000000000..55eb75938e --- /dev/null +++ b/website/fetch_pypi_downloads_via_pepy.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python3 +"""Spot-check last-30-day PyPI download counts via the pepy.tech API. + +Cross-check for a handful of packages during audits — not for full-README +sweeps: the free API key is throttled to 5 requests/minute (10 burst), so +this script sleeps 12s between requests and 530 names would take ~2 hours +(use fetch_pypi_downloads_via_clickpy.py for bulk). Reads PEPY_TECH_API_KEY from the +environment, falling back to the repo-root .env. The v2 endpoint returns +~90 days of per-day per-version counts; this script sums the most recent +30 days present in the response across all versions. Names resolve +through data/pypi_name_overrides.json, matching the cache sweep. pepy counts include +mirror/CI traffic (CI filtering is a paid pepy feature), matching the +ClickPy/BigQuery figures; pypistats.org excludes mirrors, so never mix +the two in one comparison. Results print to stdout as TSV and are not +cached — data/pypi_downloads.tsv stays single-source. + +Usage: python fetch_pypi_downloads_via_pepy.py NAME [NAME ...] +""" + +import os +import sys +import time +from pathlib import Path + +import httpx +from fetch_pypi_downloads_via_clickpy import resolve + +ENV_FILE = Path(__file__).parent.parent / ".env" +PEPY_URL = "https://api.pepy.tech/api/v2/projects/{name}" +SECONDS_BETWEEN_REQUESTS = 12 + + +def load_api_key() -> str: + key = os.environ.get("PEPY_TECH_API_KEY", "") + if not key and ENV_FILE.exists(): + for line in ENV_FILE.read_text().splitlines(): + name, sep, value = line.partition("=") + if sep and name.strip() == "PEPY_TECH_API_KEY": + key = value.strip() + if not key: + print("Error: PEPY_TECH_API_KEY not set (environment or repo-root .env).", file=sys.stderr) + sys.exit(1) + return key + + +def last_30_day_total(downloads_per_day: dict[str, dict[str, int]]) -> int: + recent_days = sorted(downloads_per_day)[-30:] + return sum(sum(downloads_per_day[day].values()) for day in recent_days) + + +def main() -> None: + if len(sys.argv) < 2: + print("Usage: python fetch_pypi_downloads_via_pepy.py NAME [NAME ...]", file=sys.stderr) + sys.exit(1) + names = [] + for arg in sys.argv[1:]: + pkg = resolve(arg) + if pkg is None: + print(f"{arg}: not pip-installable per pypi_name_overrides.json, skipping", file=sys.stderr) + else: + names.append(pkg) + with httpx.Client(headers={"X-API-Key": load_api_key()}, timeout=30) as client: + for i, name in enumerate(names): + if i: + time.sleep(SECONDS_BETWEEN_REQUESTS) + resp = client.get(PEPY_URL.format(name=name)) + if resp.status_code == 404: + print(f"{name}\tNOT_FOUND") + continue + resp.raise_for_status() + print(f"{name}\t{last_30_day_total(resp.json()['downloads'])}") + + +if __name__ == "__main__": + main() diff --git a/website/readme_parser.py b/website/readme_parser.py index 61bf1e982e..84887d0f77 100644 --- a/website/readme_parser.py +++ b/website/readme_parser.py @@ -1,4 +1,13 @@ -"""Parse README.md into structured section data using markdown-it-py AST.""" +"""Parse README.md into structured section data using markdown-it-py AST. + +Empirically verified behavior worth knowing before editing README.md: +everything above `## Projects` is ignored; new subcategories need no +parser change (a bullet without a leading link plus indented entries); +a standalone all-bold paragraph inside the Projects region becomes a +Thematic Group marker; prose placed right after `## Categories` leaks +into llms.txt; the build's "Total entries" figure counts sub-items +(indented links under an entry), not just entries. +""" from __future__ import annotations diff --git a/website/static/main.js b/website/static/main.js index d5b337b341..11d438613e 100644 --- a/website/static/main.js +++ b/website/static/main.js @@ -5,7 +5,7 @@ function getScrollBehavior() { } let activeFilter = null; -let activeSort = { col: "stars", order: "desc" }; +let activeSort = { col: "downloads", order: "desc" }; const searchInput = document.querySelector(".search"); const filterBar = document.querySelector(".filter-bar"); const filterValue = document.querySelector(".filter-value"); @@ -211,7 +211,7 @@ function buildQueryString() { const params = new URLSearchParams(); const query = searchInput ? searchInput.value.trim() : ""; if (query) params.set("q", query); - if (activeSort.col !== "stars" || activeSort.order !== "desc") { + if (activeSort.col !== "downloads" || activeSort.order !== "desc") { params.set("sort", activeSort.col); params.set("order", activeSort.order); } @@ -238,6 +238,14 @@ function getSortValue(row, col) { const num = parseInt(text, 10); return isNaN(num) ? -1 : num; } + if (col === "downloads") { + const text = row + .querySelector(".col-downloads") + .textContent.trim() + .replace(/,/g, ""); + const num = parseInt(text, 10); + return isNaN(num) ? -1 : num; + } if (col === "commit-time") { const attr = row.querySelector(".col-commit").getAttribute("data-commit"); return attr ? new Date(attr).getTime() : 0; @@ -384,7 +392,7 @@ sortHeaders.forEach(function (th) { if (activeSort.col === col) { if (activeSort.order === defaultOrder) activeSort = { col: col, order: altOrder }; - else activeSort = { col: "stars", order: "desc" }; + else activeSort = { col: "downloads", order: "desc" }; } else { activeSort = { col: col, order: defaultOrder }; } @@ -467,7 +475,10 @@ if (backToTop) { const order = params.get("order"); if (q && searchInput) searchInput.value = q; if ( - (sort === "name" || sort === "stars" || sort === "commit-time") && + (sort === "name" || + sort === "stars" || + sort === "downloads" || + sort === "commit-time") && (order === "desc" || order === "asc") ) { activeSort = { col: sort, order: order }; diff --git a/website/static/style.css b/website/static/style.css index 93056570aa..1cd3a9f9f6 100644 --- a/website/static/style.css +++ b/website/static/style.css @@ -893,6 +893,14 @@ th[data-sort].sort-asc::after { letter-spacing: 0.02em; } +.col-downloads { + width: 9.5rem; + text-align: right; + white-space: nowrap; + font-variant-numeric: tabular-nums; + color: var(--ink-soft); +} + .col-commit { width: 9rem; white-space: nowrap; @@ -1041,7 +1049,8 @@ th[data-sort].sort-asc::after { color: var(--line-strong); } -.expand-commit { +.expand-commit, +.expand-downloads { display: none; } @@ -1606,11 +1615,13 @@ th[data-sort].sort-asc::after { justify-self: start; } - .col-commit { + .col-commit, + .col-downloads { display: none; } - .expand-commit { + .expand-commit, + .expand-downloads { display: inline; } diff --git a/website/templates/category.html b/website/templates/category.html index 6e4b03e792..e29e2f1780 100644 --- a/website/templates/category.html +++ b/website/templates/category.html @@ -117,6 +117,9 @@

Results

+ + + @@ -150,6 +153,13 @@

Results

else %}{{ category.name }}{% endif %} + + {% if entry.downloads is not none %}{{ + "{:,}".format(entry.downloads) }}{% else %}Not on PyPI{% endif %} + {% if entry.stars is not none %}{{ "{:,}".format(entry.stars) }}{% elif entry.source_type %}Results {% if entry.description %} - +
{{ entry.description | safe }}
{% endif %} - +
{% if entry.also_see %}
@@ -249,6 +259,10 @@

Results

>{% endif %} {% if entry.downloads is not none %}/{{ + "{:,}".format(entry.downloads) }} downloads/month{% endif %}
diff --git a/website/templates/index.html b/website/templates/index.html index e7f1c9ea56..0bac14469a 100644 --- a/website/templates/index.html +++ b/website/templates/index.html @@ -168,6 +168,9 @@

Results

+ + + @@ -201,6 +204,13 @@

Results

else %}{{ entry.categories[0] }}{% endif %} + + {% if entry.downloads is not none %}{{ + "{:,}".format(entry.downloads) }}{% else %}Not on PyPI{% endif %} + {% if entry.stars is not none %}{{ "{:,}".format(entry.stars) }}{% elif entry.source_type %}Results {% if entry.description %} - +
{{ entry.description | safe }}
{% endif %} - +
{% if entry.description %}
{{ entry.description | safe }}
@@ -298,6 +308,10 @@

Results

>{% endif %} {% if entry.downloads is not none %}/{{ + "{:,}".format(entry.downloads) }} downloads/month{% endif %}
diff --git a/website/tests/test_build.py b/website/tests/test_build.py index 0af6c0de2c..50627e27f7 100644 --- a/website/tests/test_build.py +++ b/website/tests/test_build.py @@ -17,6 +17,7 @@ detect_source_type, extract_entries, extract_github_repo, + load_downloads, load_stars, sort_entries, subcategory_path, @@ -429,6 +430,42 @@ def test_build_with_stars_sorts_by_stars(self, tmp_path): # Expand content present assert "expand-content" in html + def test_build_with_downloads_renders_column(self, tmp_path): + readme = textwrap.dedent("""\ + # T + + ## Projects + + ## Stuff + + - [My-Lib](https://github.com/org/mylib) - On PyPI. + - [no-pypi](https://example.com/none) - Not on PyPI. + - [asyncio](https://docs.python.org/3/library/asyncio.html) - Built-in. + + # Contributing + + Done. + """) + (tmp_path / "README.md").write_text(readme, encoding="utf-8") + self._copy_real_templates(tmp_path) + + data_dir = tmp_path / "website" / "data" + data_dir.mkdir(parents=True) + # Keyed by normalized README display name, like fetch_pypi_downloads_via_clickpy.py writes it + (data_dir / "pypi_downloads.tsv").write_text( + "name\tpackage\tdownloads\tfetched_at\nasyncio\tasyncio\t26305454\t2026-08-16\nmy-lib\tmy-lib\t1234567\t2026-08-16\n", + encoding="utf-8", + ) + + build(tmp_path) + + html = (tmp_path / "website" / "output" / "index.html").read_text(encoding="utf-8") + assert "1,234,567" in html + # Built-in entries never show PyPI counts: the asyncio row is the backport package + assert "26,305,454" not in html + # Default sort: entries with download counts come first + assert html.index("My-Lib") < html.index("no-pypi") + def test_build_fails_when_group_and_category_slug_collide(self, tmp_path): readme = textwrap.dedent("""\ # T @@ -974,6 +1011,7 @@ def _template_entry(name: str, stars: int | None, source_type: str | None = None groups=[], subcategories=[], stars=stars, + downloads=None, owner=None, last_commit_at=None, source_type=source_type, @@ -1199,3 +1237,16 @@ def test_preserves_lines_without_trailing_newline(self): markdown = "- [foo](https://github.com/owner/foo) - A foo." stars = {"owner/foo": {"stars": 5, "owner": "owner"}} assert annotate_entries_with_stars(markdown, stars) == ("- [foo](https://github.com/owner/foo) - A foo. (5 GitHub stars)") + + +class TestLoadDownloads: + def test_parses_tsv_and_skips_not_found(self, tmp_path): + tsv = tmp_path / "pypi_downloads.tsv" + tsv.write_text( + "name\tpackage\tdownloads\tfetched_at\naiohttp\taiohttp\t649105404\t2026-08-16\npytorch\ttorch\t50000000\t2026-08-16\ndead-pkg\t-\tNOT_FOUND\t2026-08-16\n", + encoding="utf-8", + ) + assert load_downloads(tsv) == {"aiohttp": 649105404, "pytorch": 50000000} + + def test_missing_file_returns_empty(self, tmp_path): + assert load_downloads(tmp_path / "nope.tsv") == {}