Scorecard is a backstage plugin and supports multiple metrics, grouped as a new module in the folder workspaces/scorecard/plugins.
There are already modules that support GitHub, Jira, Filecheck, OpenSSF, SonarQube. Add a new module called catalog.
This module should focus on checks on the catalog entity itself and it should contain just one CatalogMetricProvider. This metric provider should create metrics based on the configuration. This way, platform admins can ensure that all their imported catalog entities follow the same pattern.
Use cases
- Every catalog entity should have a title (
metadata.title), tags (metadata.tags), or links (metadata.links). All these fields are optional in Backstage but maybe "required" in a company.
- Every component should use one of a specific set of values for the lifecycle (
spec.lifecycle). It's a required field for Components, but the value itself isn't enforced.
- Every catalog entity of Kind: Component should use one of a specific set of values for the type (
spec.type).
- Every catalog entity should have a specific annotation (
metadata.annotations)- that is manually added or maybe set by a Catalog Processor - and here checked if the catalog processor works correctly.
- Every scaffolder template should have an owner (
spec.owner). This field is required for Components but not for Templates.
For this, admins needs to configure the provider, aligned with the Files modules
requiredAttributes configuration example
# app-config.yaml
scorecard:
metricProviders:
catalog:
requiredAttributes:
options:
checks:
- metric:
id: title
title: Title is required
description: The metadata.title should be defined for all Components.
filter:
"spec.kind": Component
field: "metadata.title"
statusMapping:
exists: found
empty: missed
emptyString: missed
emptyArray: missed
missed: missed
values: {}
statusMapping:
exists: found
empty: missed
emptyString: missed
emptyArray: missed
missed: missedThe configuration needs to read the requiredAttributes options with a list of checks. The status mapping is picked up and merged from the check (highest prio, overrides other status mappings), then from the requiredAttributes options and then fallbacked to the example above, which means, any not found, or empty string or empty array is mapped to missed. And a real value or an array with length >= 1 will be mapped to found.
The metric metadata, filter and field are required for each check. The statusMapping is optional on a check and also on the options. The filter should allow an empty object if it should really target ANY catalog entity, but the readme should recommend to have a check at least on a Kind level.
Here is another example that maps specific accepted lifecycle strings and refuses all other values:
# app-config.yaml
scorecard:
metricProviders:
catalog:
requiredAttributes:
options:
checks:
- metric:
id: lifecycle
title: lifecycle should be prod, stage, test or dev
description: The spec.lifecycle field should be one of our four accepted lifecycle values: prod, stage, test or dev.
filter:
"spec.kind": Component
field: "spec.lifecycle"
statusMapping:
exists: invalid
values:
prod: ok
stage: ok
test: ok
dev: okThe result should be that the status is "ok" for the four known lifecycle values, "invalid" when there is any other value, and missed when the lifecycle is not defined.
Scorecard is a backstage plugin and supports multiple metrics, grouped as a new module in the folder
workspaces/scorecard/plugins.There are already modules that support GitHub, Jira, Filecheck, OpenSSF, SonarQube. Add a new module called
catalog.This module should focus on checks on the catalog entity itself and it should contain just one
CatalogMetricProvider. This metric provider should create metrics based on the configuration. This way, platform admins can ensure that all their imported catalog entities follow the same pattern.Use cases
metadata.title), tags (metadata.tags), or links (metadata.links). All these fields are optional in Backstage but maybe "required" in a company.spec.lifecycle). It's a required field for Components, but the value itself isn't enforced.spec.type).metadata.annotations)- that is manually added or maybe set by a Catalog Processor - and here checked if the catalog processor works correctly.spec.owner). This field is required for Components but not for Templates.For this, admins needs to configure the provider, aligned with the Files modules
requiredAttributes configuration example
The configuration needs to read the
requiredAttributesoptions with a list of checks. The status mapping is picked up and merged from the check (highest prio, overrides other status mappings), then from therequiredAttributesoptions and then fallbacked to the example above, which means, any not found, or empty string or empty array is mapped to missed. And a real value or an array with length >= 1 will be mapped to found.The metric metadata, filter and field are required for each check. The statusMapping is optional on a check and also on the options. The filter should allow an empty object if it should really target ANY catalog entity, but the readme should recommend to have a check at least on a Kind level.
Here is another example that maps specific accepted lifecycle strings and refuses all other values:
The result should be that the status is "ok" for the four known lifecycle values, "invalid" when there is any other value, and missed when the lifecycle is not defined.