[helm] Restructure helm chart of Fluss

[nhuantho]

Purpose

Linked issue: close #2865

Brief change log

• Restructured values.yaml
  • coordinator -> coordinatorServer (https://fluss.apache.org/docs/concepts/architecture/).
  • tablet -> tabletServer (https://fluss.apache.org/docs/concepts/architecture/).
  • Move resource of coordinatorServer and tabletServer to inside themselves.
  • listeners -> ports.
  • storage -> persistence.
• Created values.schema.json: schema of values.yaml and standard to valid value.yaml.
• Created values.schema.schema.json: schema of values.schema.json and standard to valid values.schema.json.
• Created tools/ci/helm_ci/validate_helm.py to validate helm.
• Add ci to validate helm.
• Add ignore python in .gitignore

Tests

API and Format

Documentation

[nhuantho]

@affo Can you review for me?

[nhuantho]

@swuferhong can you review for me?

[affo]

I left some punctual comments 🤝

I think this PR tries to do 2 tasks:

1. refactor values
2. enforce schema on values.

I suggest to now let #2870 land before refactoring the values, and later address any remaining change that you want to address.

I think this PR should be only dedicated to task 2 🤝

About that, I don't exactly get why we have the Python script, as I think helm should directly pickup the values.schema.json for validation.

Maintaining the schema itself will be very hard, hence I suggest to lean for an helm plugin such as https://github.com/dadav/helm-schema for auto-generating the schema.
We can add a CI step such as:

- name: Check schema is up to date
  run: |
    helm schema --dry-run > /tmp/generated-schema.json
    diff helm/values.schema.json /tmp/generated-schema.json

to verify that the schema is up-to-date.

At this point, values should be turned into something like this for helm-schema to work:

# @schema
# type: integer
# minimum: 1
# @schema
# Number of coordinator replicas
coordinatorServer:
  replicas: 1

  # @schema
  # type: object
  # additionalProperties: false
  # @schema
  persistence:
    # @schema
    # type: boolean
    # @schema
    enabled: false

    # @schema
    # type: string
    # pattern: ^[0-9]+(Gi|Ti)$
    # @schema
    size: 8Gi

    # @schema
    # type: string
    # @schema
    storageClass: ""

For the x-docsSection property and to enforce all properties are active, we can just create a plain python script to validate they exist:

# Instead of a meta-schema, just validate the rules explicitly
for prop, schema in values_schema["properties"].items():
    assert "description" in schema, f"{prop} missing description"
    assert "default" in schema, f"{prop} missing default"
    assert "x-docsSection" in schema, f"{prop} missing x-docsSection"

No dependency, just validating the schema and easier to reason about than a schema validating a schema 🤝

In any case, I would bootstrap this without x-docsSection.

I would separate that into another PR to address automatic documentation generation 🤝

[affo]

Suggested change

	python-version: "3.11"
	python-version: "3.12"

Just better to point to that one if we don't break anything

[nhuantho]

I want to unify the Python version with .github/workflows/unstable-test-reporter.yaml

[affo]

I would favor a requirements.txt under tools/ci/helm_ci.
It would make it easier to validate locally.

The command would become pip install -r tools/ci/helm_ci/requirements.txt

[affo]

Suggested change

	- name: Run unstable test reporter
	- name: Validate values

[affo]

Why do we need this script ?

Isn't helm lint already picking values.schema.json for validation?
Is this only for the meta schema?

[nhuantho]

@affo, from my side, when everyone updated helm. Could they please explain why it needs to be updated and describe the changes in the schema?
For example:

[affo]

@nhuantho by the way, thanks for the contribution and to light the spotlight on this defect 🤝

[affo]

@nhuantho as a side note, one can already structure schema annotations by adding comments prefixed with -- to pair with helm-docs which is a tool we may add for generating the docs https://github.com/norwoodj/helm-docs

[nhuantho]

@affo, I will answer some confusion:

• I think it will confuse users who use Helm

# @schema
  # type: object
  # additionalProperties: false
  # @schema

• https://github.com/dadav/helm-schema is not fully open source, and it is easy for us to build unit tests for Helm.
• https://github.com/norwoodj/helm-docs is also not fully open source, and I think we can easily build and generate docs.

Focus pr:

• We have the first verified value change to avoid affecting the helm -> [Helm] Chart Component Configuration Isolation #2472
• Suggest defining value with a standard (airflow).
• Build a schema and describe all values of helm, and the first step for auto-generating helm docs.