You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+20-11Lines changed: 20 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -37,6 +37,8 @@ The fixed structure must be technology agnostic. The first fields of teh fixed s
37
37
*`DataProductOwner: [String]` Data Product owner, the unique identifier of the actual user that owns, manages, and receives notifications about the Data Product. To make it technology independent it is usually the email address of the owner.
38
38
*`DataProductOwnerDisplayName [String]`: the human readable version of `DataProductOwner`.
39
39
*`Email: [Option[String]]` point of contact between consumers and maintainers of the Data Product. It could be the owner or a distribution list, but must be reliable and responsive.
40
+
*`OwnerGroup [String]`: LDAP user/group that is owning the data product
41
+
*`DevGroup [String]`: LDAP user/group that is in charge to develop and maintain the data product
40
42
*`InformationSLA: [Option[String]]` describes what SLA the Data Product team is providing to answer additional information requests about the Data Product itself.
41
43
*`Status: [Option[String]]` this is an enum representing the status of this version of the Data Product. Allowed values are: `[Draft|Published|Retired]`. This is a metadata that communicates the overall status of the Data Product but is not reflected to the actual deployment status.
42
44
*`Maturity: [Option[String]]` this is an enum to let the consumer understand if it is a tactical solution or not. It is really useful during migration from Data Warehouse or Data Lake. Allowed values are: `[Tactical|Strategic]`.
@@ -62,8 +64,6 @@ Constraints:
62
64
* Major version of the Data Product is always the same as the major version of all of its components and it is the same version that is shown in both Data Product ID and component ID.
63
65
*`InfrastructureTemplateId: [String]` the id of the microservice responsible for provisioning the component. A microservice may be capable of provisioning several components generated from different use case templates.
64
66
*`UseCaseTemplateId: [Option[String]]` the id of the template used in the builder to create the component. Could be empty in case the component was not created from a builder template.
65
-
*`Allows: [Array[String]]` It is an array of user/role/group related to LDAP/AD user. This field is defining who has access in read-only to this specific output port.
66
-
*`Owners: [Array[String]]` It is an array of user/role/group related to LDAP/AD user. This field defines who has all permissions on this specific output port.
67
67
*`DependsOn: [Array[String]]` An output port could depend on other output ports or storage areas, for example a SQL Output port could be dependent on a Raw Output Port because it is just an external table.
68
68
Constraints:
69
69
* This array will only contain IDs of other components.
@@ -73,16 +73,25 @@ Constraints:
73
73
*`CreationDate: [Optional[String]]` when this output port has been created.
74
74
*`StartDate: [Optional[String]]` the first business date present in the dataset, leave it empty for events or we can use some standard semantic like: "-7D, -1Y".
75
75
*`ProcessDescription: [Option[String]]` what is the underlying process that contributes to generate the data exposed by this output port.
76
-
*`BillingPolicy: [Option[String]]` how a consumer will be charged back when it consumes this output port.
77
-
*`SecurityPolicy: [Option[String]]` additional information related to security aspects, like restrictions, maskings, sensibile information.
78
-
*`ConsumptionPolicy: [Option[String]]` any other information needed by the consumer in order to effectively consume the data, it could be related to technical stuff, regulation, security, etc.
79
-
*`SLO:[Yaml]`
80
-
*`IntervalOfChange: [Option[String]]` how often changes in the data are reflected.
81
-
*`Timeliness: [Option[String]]` the skew between the time that a business fact occuts and when it becomes visibile in the data.
82
-
*`Endpoint: [Option[URL]]` this is the API endpoint that self-describe the output port and provide insightful information at runtime about the physical location of the data, the protocol must be used, etc.
83
-
*`Tags: [Array[Yaml]]` free tags at OutputPort level (please refer to OpenMetadata https://docs.open-metadata.org/openmetadata/schemas/entities/tagcategory)
76
+
*`DataContract: [Yaml]`: In case something is going to change in this section, it represents a breaking change because the producer is breaking the contract, this will require to create a new version of the data product to keep backward compatibility
77
+
*`Schema: [Array[Yaml]]` when it comes to describe a schema we propose to leverage OpenMetadata specification: Ref https://docs.open-metadata.org/metadata-standard/schemas/entities/table#column. Each column can have a tag array and you can choose between simples LabelTags, ClassificationTags or DescriptiveTags. Here an example of classification Tag https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/resources/json/data/tags/piiTags.json.
78
+
*`SLA: [Yaml]` Service Level Agreement, describe the quality of data delivery and the outpu tport in general. It represents the producer's overall promise to the consumers
79
+
*`IntervalOfChange: [Option[String]]` how often changes in the data are reflected.
80
+
*`Timeliness: [Option[String]]` the skew between the time that a business fact occuts and when it becomes visibile in the data.
81
+
*`UpTime: [Option[String]]` the percentage of port availability.
82
+
*`TermsAndConditions: [Option[String]]` If the data is usable only in specific environments
83
+
*`Endpoint: [Option[URL]]` this is the API endpoint that self-describe the output port and provide insightful information at runtime about the physical location of the data, the protocol must be used, etc.
84
+
*`DataSharingAgreement: [Yaml]` This part is coveringusage, privacy, purpose, limitations and is indipendent by the data contract
85
+
*`Purpose: [Option[String]]` what is the goal of this data set
86
+
*`Billing: [Option[String]]` how a consumer will be charged back when it consumes this output port.
87
+
*`Security: [Option[String]]` additional information related to security aspects, like restrictions, maskings, sensibile information and privacy.
88
+
*`IntendedUsage: [Option[String]]` any other information needed by the consumer in order to effectively consume the data, it could be related to technical stuff ( ex. Extract no more than one year of data for good performances ) or to business domains ( Ex. this data is only useful in the marketing domains )
89
+
*`Limitations: [Option[String]]` If any limitation is present it must be made super clear to the consumers.
90
+
*`LifeCycle: [Option[String]]` Describe how the data will be historicized and how and when it will be deleted
91
+
*`Confidentiality: [Option[String]]` Describe what a consumer should do to keep the information confidential, how to process and store it. Permission to share or report it.
92
+
*`Tags: [Array[Yaml]]` free tags at OutputPort level, here we can have security classification for example (please refer to OpenMetadata https://docs.open-metadata.org/openmetadata/schemas/entities/tagcategory)
84
93
*`SampleData: [Option[Yaml]]` provides a sample data of your Output Port. See OpenMetadata specification: https://docs.open-metadata.org/openmetadata/schemas/entities/table#tabledata
85
-
*`Schema: [Array[Yaml]]` when it comes to describe a schema we propose to leverage OpenMetadata specification: Ref https://docs.open-metadata.org/metadata-standard/schemas/entities/table#column. Each column can have a tag array and you can choose between simples LabelTags, ClassificationTags or DescriptiveTags. Here an example of classification Tag https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/resources/json/data/tags/piiTags.json.
94
+
86
95
*`SemanticLinking: [Option[Yaml]]` here we can express semantic relationships between this output port and other outputports (also coming from other domains and data products). For example we could say that column "customerId" of our SQL Output Port references the column "id" of the SQL Output Port of the "Customer" Data Product.
87
96
*`Specific: [Yaml]` this is a custom section where we must put all the information strictly related to a specific technology or dependent from a standard/policy defined in the federated governance.
0 commit comments