[Improvement] Separate S3 storage configuration for MLRun and Kubeflow Pipeline

[GiladShapira94]

📝 Description

Introduces explicit, independent S3 storage configuration for Kubeflow Pipelines under pipelines.storage.s3.*, decoupling it from the storage.s3.* block used by MLRun and Jupyter.
Previously, all three components shared a single S3 config, making it impossible to grant scoped IAM credentials or use separate buckets without silently misconfiguring them.
Both blocks default to the bundled SeaweedFS, so default installs are unaffected - users change only the block for the component they want to redirect.
Also fixes a latent inconsistency where MLRun artifact paths were resolved from global.infrastructure.aws.bucketName (defaulting to the hardcoded string "mlrun") rather than storage.s3.bucket, meaning a user changing
storage.s3.bucket would not see that reflected in MLRUN_ARTIFACT_PATH, MLRUN_FEATURE_STORE__DATA_PREFIXES__DEFAULT, or model-monitoring paths.

🛠 Changes Made

• values.yaml — added pipelines.storage.s3.{bucket,accessKey,secretKey} under pipelines:, all defaulting to SeaweedFS values (mlrun / seaweed / seaweed123); updated storage.* comment block to clarify it is scoped
  to MLRun and Jupyter only
• templates/_helpers.tpl — added three new helpers mlrun-ce.pipelines.s3.{bucket,accessKey,secretKey} that read from pipelines.storage.s3.* directly (no fallback coupling); fixed mlrun-ce.artifactPath,
  mlrun-ce.featureStore.dataPrefix, and all three mlrun-ce.model-endpoint.monitoring.* helpers to resolve bucket via coalesce global.infrastructure.aws.bucketName storage.s3.bucket "mlrun" so storage.s3.bucket is the effective source for MLRun paths
• templates/pipelines/configmaps/pipeline-install-config.yaml — bucketName now uses mlrun-ce.pipelines.s3.bucket instead of mlrun-ce.s3.bucket
• templates/pipelines/secrets/mlpipeline-seaweedfs-artifact.yaml — accesskey/secretkey now use mlrun-ce.pipelines.s3.{accessKey,secretKey} instead of the shared mlrun-ce.s3.* helpers
• Chart.yaml — version bumped 0.11.0-rc.36 → 0.11.0-rc.37
• README.md — version matrix updated to 0.11.0-rc.37

---

✅ Checklist t

• I have tested the changes in this PR
• I confirmed whether my changes require a change in documentation and if so, I created another PR in MLRun for the relevant documentation.
• I confirmed whether my changes require changes in QA tests, for example: credentials changes, resources naming change and if so, I updated the relevant Jira ticket for QA.
• I increased the Chart version in charts/mlrun-ce/Chart.yaml.
• I confirmed that the installation works both on a local Docker Desktop environment and on a real cluster when using the required
  prerequisites.
• If installation issues were found, I updated the relevant Jira ticket with the issue and steps to reproduce, or updated the prerequisites documentation if the issue is related to missing or outdated prerequisites.
• If needed, update https://github.com/mlrun/ce/blob/development/charts/mlrun-ce/README.md with the relevant installation instructions and version Matrix.
• If needed, update the following values files for multi namespace support:
  • Admin values
  • User values Node Port
  • User values ClusterIP

---

🧪 Testing

• make helm-lint passes (✔︎ mlrun-ce => version: "0.11.0-rc.37")
• helm template verified for all three use cases:
• Use case A (default SeaweedFS for both): bucketName: "mlrun", accesskey = seaweed, secretkey = seaweed123 for Pipelines; MLRUN_ARTIFACT_PATH: s3://mlrun/... for MLRun
• Use case B (MLRun → AWS, Pipelines → SeaweedFS): setting storage.s3.* to AWS values leaves pipelines.storage.s3.* unchanged on SeaweedFS
• Use case C (both → AWS): setting both blocks independently routes each component correctly

---

🔗 References

• Ticket link: CEML-707

---

🚨 Breaking Changes?

• Yes (explain below)

• No

• Non-default storage.s3.* users with Pipelines enabled: any user who previously customizedstorage.s3.accessKey, storage.s3.secretKey, or storage.s3.bucket expecting Pipelines to inherit those values must now also set the corresponding pipelines.storage.s3.* keys — Pipelines no longer read from storage.s3.*.

• global.infrastructure.aws.bucketName users: MLRun artifact paths previously fell back to the hardcoded string "mlrun" when global.infrastructure.aws.bucketName was null; they now fall back to storage.s3.bucket
  instead. Users who had global.infrastructure.aws.bucketName: ~ and storage.s3.bucket set to a non-"mlrun" value will see their MLRun artifact paths updated automatically — which is the correct behavior but is a change from
  before.

---

🔍 Additional Notes

• The global.infrastructure.aws.bucketName field is still honored via coalesce for backwards compatibility with users who set it explicitly, but storage.s3.bucket is now the recommended single source of truth for the MLRun
  artifact bucket.