Growing library size

[number492]

According to dive, this library is now 79 MB in my Docker image:

This seems excessive for a Python library, and if multiplied by the number of users, it's arguably a serious waste of resources. Based on just the sizes, client and aio/client are mostly duplicates. Under that both api and models are huge, their __pycache__ are similarly huge. Inside models, functions like _get_openapi_to_dict, _to_legacy_item, _to_legacy_value, _to_openapi_value are basically the same, along with some member methods. Could you please do something to reduce this duplication, and the library size along with it?

[dayvenhppg2]

The duplication in client and aio/client could be resolved by refactoring shared functionality into a common module to avoid redundancy. It might also be worth investigating the models directory further; for instance, if _get_openapi_to_dict, _to_legacy_item, _to_legacy_value, and _to_openapi_value are similar, consider consolidating them into a single function with parameters to handle the variations. Additionally, do we have a specific reason for retaining the __pycache__ files in the Docker image, or can we exclude them during the build process to reduce size?

[yliaog]

/help

i think we can remove pycache files to reduce library size

[kubernetes-prow]

@yliaog:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

Details

In response to this:

/help

i think we can remove pycache files to reduce library size

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[yurnov]

Hi @number492,

You can decrease the size of your container with pip install --no-compile:

docker run -it --rm python:3.14-slim bash
root@7acabead7d27:/# pip install --no-compile kubernetes > /dev/null 2>&1
root@7acabead7d27:/# du -h --max-depth=1 /usr/local/lib/python3.14/site-packages/kubernetes
60K     /usr/local/lib/python3.14/site-packages/kubernetes/stream
156K    /usr/local/lib/python3.14/site-packages/kubernetes/config
21M     /usr/local/lib/python3.14/site-packages/kubernetes/aio
88K     /usr/local/lib/python3.14/site-packages/kubernetes/dynamic
56K     /usr/local/lib/python3.14/site-packages/kubernetes/leaderelection
21M     /usr/local/lib/python3.14/site-packages/kubernetes/client
44K     /usr/local/lib/python3.14/site-packages/kubernetes/utils
28K     /usr/local/lib/python3.14/site-packages/kubernetes/informer
48K     /usr/local/lib/python3.14/site-packages/kubernetes/watch
41M     /usr/local/lib/python3.14/site-packages/kubernetes
root@7acabead7d27:/#

As you can see, here is 41MB vs. the original 82MB:

docker run -it --rm python:3.14-slim bash
root@e7e6c6e9bf30:/# pip install kubernetes > /dev/null 2>&1
root@e7e6c6e9bf30:/# du -h --max-depth=1 /usr/local/lib/python3.14/site-packages/kubernetes
8.0K    /usr/local/lib/python3.14/site-packages/kubernetes/__pycache__
132K    /usr/local/lib/python3.14/site-packages/kubernetes/stream
336K    /usr/local/lib/python3.14/site-packages/kubernetes/config
41M     /usr/local/lib/python3.14/site-packages/kubernetes/aio
204K    /usr/local/lib/python3.14/site-packages/kubernetes/dynamic
120K    /usr/local/lib/python3.14/site-packages/kubernetes/leaderelection
40M     /usr/local/lib/python3.14/site-packages/kubernetes/client
92K     /usr/local/lib/python3.14/site-packages/kubernetes/utils
60K     /usr/local/lib/python3.14/site-packages/kubernetes/informer
96K     /usr/local/lib/python3.14/site-packages/kubernetes/watch
82M     /usr/local/lib/python3.14/site-packages/kubernetes
root@e7e6c6e9bf30:/#

Still, you will have two separate clients: classic client and asynchronous client kubernetes.aio

[lauraroth754]

Reducing duplication in the models module, particularly with functions like _get_openapi_to_dict and _to_openapi_value, could significantly trim down the library size. Have you considered refactoring these functions into a single utility function that handles variations in parameters to eliminate redundancy?

[yurnov]

The duplication isn't hand-written here — it's generator output.

_get_openapi_to_dict, _to_legacy_item, _to_legacy_value and _to_openapi_value come from the upstream OpenAPI Generator templates python/_legacy_model_dict_helpers.mustache and python/_legacy_model_identity.mustache. model_generic.mustache inlines that partial into every model file, gated on the compatibleWithPythonLegacy option, which kubernetes-client/gen enables in openapi/python.xml and openapi/python-aio.xml to keep to_dict(serialize=...), openapi_types/attribute_map and the legacy container-conversion semantics.

This client pins generator 7.25.0-SNAPSHOT at commit 830e9d156960bb7f51a5337f31636a7e73226474. Every model is listed in kubernetes/.openapi-generator/FILES and none are excluded by .openapi-generator-ignore, so refactoring them here by hand would be reverted on the next scripts/update-client.sh run.

For scale: ~2.2 KB of helpers × 1538 model files (769 sync + 769 asyncio) ≈ 3.4 MB, about 22% of the models source. So it's worth doing, but the change belongs upstream in OpenAPITools/openapi-generator — emit the helpers once into a shared module and import them, which kubernetes/client/api_client.py already effectively does for its own copy.

[Abhinav0905]

I can take an upstream-first implementation if maintainers want to pursue the generated-code reduction. I would not hand-edit the generated model files, since the next client regeneration would overwrite them.

The plan would be to update the OpenAPI Generator Python legacy-model templates so the repeated conversion helpers are emitted once in a shared module for both sync and asyncio clients, add generator snapshot tests for compatibleWithPythonLegacy, then update the pinned generator commit here and regenerate both clients. I would compare source, wheel, and installed bytecode sizes before and after so the improvement is measurable and separate it from the pip --no-compile container workaround.

Would maintainers accept an upstream OpenAPI Generator change as the prerequisite, followed by a regeneration PR in this repository?

[yliaog]

Yes, sounds good to update the OpenAPI generator to reduce the duplication. Thanks.

[Abhinav0905]

Thanks for confirming the upstream-first direction. I opened OpenAPITools/openapi-generator#24729 and implemented it in draft PR OpenAPITools/openapi-generator#24730.

The PR emits the legacy conversion helpers once for full Python client generation, updates generic/anyOf/oneOf models and ApiClient to import them, and preserves self-contained model-only generation by retaining the inline fallback when supporting files are not generated. The focused generator tests pass, all 11 runtime tests in the regenerated legacy-compatible Python sample pass, and that sample's generated package drops from 151,102 to 140,230 bytes (7.20%) while reducing seven helper copies to one.

Once the upstream change is reviewed and merged, I can follow up here by updating the pinned generator and regenerating both the synchronous and asynchronous Kubernetes clients, with source, wheel, and installed-bytecode size comparisons.

[penguinolog]

__pycache__ is generated on the client side during package installation or on the first run by interpreter.
It already in .gitignore as should be.

[jadayaanaliah26]

Refactoring _get_openapi_to_dict and its siblings into a shared mixin or base class would allow us to move the logic out of individual model files, drastically reducing the redundant bytecode generated during installation.