diff --git a/docs/develop/.pages b/docs/develop/.pages index a11a2dfb9..fa1d34c03 100644 --- a/docs/develop/.pages +++ b/docs/develop/.pages @@ -2,6 +2,7 @@ nav: - Develop: index.md - Accessing Graphs with Java Applications: accessing-graphs-with-java-applications - Python Plugins: python-plugins + - Marketplace Packages: marketplace-packages - cmempy - Python API: cmempy-python-api - cmemc - Python Scripts: cmemc-scripts - Build (DataIntegration) APIs: dataintegration-apis diff --git a/docs/develop/marketplace-packages/.pages b/docs/develop/marketplace-packages/.pages new file mode 100644 index 000000000..1f4c61800 --- /dev/null +++ b/docs/develop/marketplace-packages/.pages @@ -0,0 +1,4 @@ +nav: + - Marketplace Packages: index.md + - Installation and Management: installation + - Development and Publication: development diff --git a/docs/develop/marketplace-packages/development/index.md b/docs/develop/marketplace-packages/development/index.md new file mode 100644 index 000000000..4f5669f3c --- /dev/null +++ b/docs/develop/marketplace-packages/development/index.md @@ -0,0 +1,197 @@ +--- +title: "Marketplace Packages: Development and Publication" +icon: material/code-json +tags: + - Package +--- +# Development and Publication of Marketplace Packages + +## Introduction + +Marketplace Packages are archives that bundle content, functionality, and configuration from Corporate Memory for sharing and reuse. + +Each package has its own release cycle. +Packages can be installed and uninstalled during runtime. + +In order to support the development and publication of Marketplace Packages, we published a [package-template](https://github.com/eccenca/cmem-package-template). +Please have a look at this template to get started. + +This page gives an overview of the concepts you need to understand in order to develop packages. + +## Package Structure + +Use the [package-template](https://github.com/eccenca/cmem-package-template) to create the boilerplate for a package repository. + +### License + +!!! info "No publication without license" + + Packages without a license declaration cannot be published to a Corporate Memory Marketplace Server. + +Our template will bootstrap with an _Apache License 2.0 ([`Apache-2.0`](https://spdx.org/licenses/Apache-2.0.html))_. See if you need a different license. +You can remove a license entirely; however, a package that does not declare a license cannot be published. + +### Manifest + +The `manifest.json` is the central package definition. +It contains all relevant package metadata and describes the package contents. +It is used to present package details and contents to the `inspect` command, to install, configure and uninstall all parts of a package. + +#### Metadata + +`package_type` +: `project` + : A package that may ship any content, mainly intended to contain Build projects, (instance/data) graphs, SHACL shapes, workspace configuration, query catalogs, etc. + + `vocabulary` + : A package that is supposed to contribute vocabulary / ontology contents, such as `rdf:`, `org:`, `sso:`, etc. Such a package may contain multiple vocabularies / ontologies. Packaging related SHACL shapes is reasonable, too. + +`package_id` +: Unique package identifier + +`package_version` +: Semantic version identifier string of the package, but limited to proper releases. + +`metadata.name` +: The package name in English. + +`metadata.description` +: The package description in English. + +`metadata_comment` +: A maintainer or publisher comment. + +#### Files + +A package can contain graphs or Build projects. These contents are referenced in the `manifest.json`. + +##### Graphs + +Use the following structure to include a graph. +`register_as_vocabulary` and `import_into` are optional instructions. +We suggest to organize graphs in a respective sub-folder (here `graphs/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "graph", + "file_path": "graphs/file.ttl", + "graph_iri": "http://www.example.org/file/", + "register_as_vocabulary": true, + "import_into": [ + "http://www.example.org/integration_graph/" + ] + }, + … +] +``` + +##### Projects + +Use the following structure to include a project. +We suggest to organize projects in a respective sub-folder (here `projects/`), but this is up to you: + +```json +"files": [ + … + { + "file_type": "project", + "file_path": "projects/my-build-project.zip", + "project_id": "my-build-project" + }, + … +] +``` + +#### Dependencies + +Dependencies to other (vocabulary) packages or to Python plugins can be declared in the `copier copy` answers. +The dependencies are added to the `manifest.json` as described in the next sections. + +##### Python Plugin Packages + +Use the following to declare a dependency to a Python plugin: + +```json +"dependencies": [ + … + { + "dependency_type": "python-package", + "pypi_id": "cmem-plugin-pyshacl" + }, + … +] +``` + +##### (Vocabulary) Packages + +Use the following to declare a dependency to a (vocabulary) package: + +```json +"dependencies": [ + … + { + "dependency_type": "vocabulary", + "package_id": "w3c-rdfs-vocab" + } + … +] +``` + +## Package Development Cycle + +!!! info "`cmemc package` reference" + + Refer to [TODO: link](./) for the complete reference of the `package` command group. + +Some packages are simply wrapping existing artifacts into a managed structure (e.g. existing vocabulary/ontology). + +Most (solution) package development and evolution will be a back and forth between a package repository (making changes to `manifest.json` in terms of adding/removing dependencies, graph files, or Build project files) and a Corporate Memory (package development) instance. + +The following pages give an overview about this feature: + +![Corporate Memory Marketplace Package Lifecycle](../mpp-lifecycle.svg){ width="50%" } + +### Install (local) Packages + +Use the following command to install a local package folder content (or built `.cpa` file) to a Corporate Memory (package development) instance. + +```sh +cmemc package install --input PATH +``` + +Make changes to graphs, configuration, or Build projects as needed. +Newly created or imported graphs or Build projects need to be registered in `manifest.json` so they will be fetched by `export`. + +### Export Contents into a Package + +Use the following command to export the file artifacts declared in `manifest.json` from a Corporate Memory (package development) instance to a local package folder. + +```sh +cmemc package export PACKAGE_ID +``` + +Run this to initially populate package contents from a solution configuration. You can also use it to update contents after making changes on your Corporate Memory (package development) instance, capturing them for building and releasing as a Marketplace Package. + +### Inspect Packages + +Review and verify the contents of a package with the following command: + +```sh +cmemc package inspect PACKAGE_PATH +``` + +### Build Packages + +During development you can install a package from a local path (plain folder or a `.cpa` package) using the `cmemc package install --input PATH` command. + +Use the `cmemc package build` command. +This will build a package archive from a package directory. + +This command processes a package directory, validates its content including the manifest, and creates a versioned Corporate Memory package archive (`.cpa`) with the following naming convention: `{package_id}-v{version}.cpa`. + +### Publish Packages + +Package archives can be published to the Marketplace Server using the `cmemc package publish` command. +After being published packages can be found and installed directly from the Marketplace Server (potential users do not need to have the local package folder or `.cpa` file available). diff --git a/docs/develop/marketplace-packages/index.md b/docs/develop/marketplace-packages/index.md new file mode 100644 index 000000000..e00af74b5 --- /dev/null +++ b/docs/develop/marketplace-packages/index.md @@ -0,0 +1,49 @@ +--- +status: new +title: "Marketplace Packages: Overview" +icon: material/shopping +tags: + - Package +hide: + - toc +--- + +# Marketplace Packages + +Starting with version 26.1, we support the creation and use of Marketplace Packages. + +Marketplace Packages bundle everything for a specific Corporate Memory–based solution or project into a single shareable, managed artifact: + +- Vocabularies / Ontologies +- (SKOS) Taxonomies +- (Instance / Data) Graphs +- Build Projects +- Dependencies on + - [python-plugins](../python-plugins/index.md) + - (other) Marketplace Packages + +The lifecycle of a Corporate Memory Marketplace Package is shown in the following flowchart. + +![Corporate Memory Marketplace Package Lifecycle](mpp-lifecycle.svg){ width="50%" } + +The following pages give an overview of this feature: + +
+ +- :material-download-circle-outline: [Installation and Management](installation/index.md) + + --- + + Intended for Linked Data Experts and Corporate Memory Admins, this page outlines how to (un)install and manage Marketplace Packages. + + This section discusses the lifecycle commands and stages `install`, `list` and `uninstall`. + +- :material-code-json: [Development and Publication](development/index.md) + + --- + + Intended for Linked Data Experts, Consultants, and Partners, this page gives an overview of how to start developing and publish Marketplace Packages. + + This section discusses the lifecycle commands and stages `copier copy`, _Package Definition and Release_, `inspect`, `install --input PATH` (from local), _Solution Development and Configuration_, `export`, `build`, and `publish`. + +
diff --git a/docs/develop/marketplace-packages/installation/index.md b/docs/develop/marketplace-packages/installation/index.md new file mode 100644 index 000000000..cdd086849 --- /dev/null +++ b/docs/develop/marketplace-packages/installation/index.md @@ -0,0 +1,49 @@ +--- +title: "Marketplace Packages: Installation and Management" +icon: material/download-circle-outline +tags: + - Package +--- +# Installation and Management of Marketplace Packages + +## Introduction + +Marketplace Packages can be installed directly from a Corporate Memory Marketplace Server or from local package archives (`.cpa` files). + +This page describes how to install, list, and uninstall packages using `cmemc`. + +!!! info "`cmemc package` reference" + + Refer to [TODO: link](./) for the complete reference of the `package` command group. + +## Install Packages + +Use the following command to install a package from a Marketplace Server: + +```sh +cmemc package install PACKAGE_ID +``` + +For installing local package archives (`.cpa` files) or package directories, use the `--input` option: + +```sh +cmemc package install --input PATH +``` + +## List Packages + +Use the following command to list all installed packages: + +```sh +cmemc package list +``` + +## Uninstall Packages + +Use the following command to uninstall a package: + +```sh +cmemc package uninstall PACKAGE_ID +``` + +This removes all package contents from the Corporate Memory instance, including graphs and Build projects that were installed as part of the package. diff --git a/docs/develop/marketplace-packages/mmd.txt b/docs/develop/marketplace-packages/mmd.txt new file mode 100644 index 000000000..fc2ee5b30 --- /dev/null +++ b/docs/develop/marketplace-packages/mmd.txt @@ -0,0 +1,13 @@ +# mermaid + +## live editor + +https://mermaid.live/edit#pako:eNqFU12L2zAQ_CuLIHVb7DSxL3YiykFJHnpwgaMfL636oNgbR0SWjCIdlwv575WcuA6FUvxg7WhmZ3clnUipKySUJEnClBVWIoWlNq023CKssdHmCGtu9mhbyUuEJ17ueY3wKLZYHkuJTHXa0egklLAUThBJXT_iM8qIQlThxtVRDJHdYYMBUeis4TKCM5xHI6YO1lutBK8Nb5LnlKnwdSAw0tutcBvSC60-bsw9VxV8QYn8gCF8K3XJ5TtGgB-g3dc3creR4rDDqqPd9NGT3WYgj8uW9_0NaeENLB8-LFdXhecMiq9aulCTL-8ZQlFLrbai7sR_T7HXN9gMCR7CSsprff-QCHUI84Ve9F2JXnYhOCXCzH6-_wVJch8mABRK3Qo04XdkKkDDlpe3WFqmQsMBDkVd8JAWQjEeF6p1Fp4-ffs8JPg_s2MMXvjiW7I3CbwlhY0Tshr8_TF4sL0cVmglAGHD-wxuLPR9A3uyHbDQPQ2j6MkkJrURFaHWOIxJg6bhISQnpsCPsbuQjFC_vF5JRpg6e1nL1Q-tm15ptKt3hG65PPjItdVwX_-gBlWFZqmdsoSmWZp1WQg9kRdCs2k-nswWxSy7mxaLfF7MYnIkdDpPx5NJUeR5kWZ5ls7PMXntfCfjYpIWd9lsvkizrJjli5hgJaw268tz7V7t-TfNwDqo + +## cli + +https://github.com/mermaid-js/mermaid-cli + +```sh +mmdc -i mpp-lifecycle.mmd -o mpp-lifecycle.svg -b transparent +``` diff --git a/docs/develop/marketplace-packages/mpp-lifecycle.mmd b/docs/develop/marketplace-packages/mpp-lifecycle.mmd new file mode 100644 index 000000000..22960e9d9 --- /dev/null +++ b/docs/develop/marketplace-packages/mpp-lifecycle.mmd @@ -0,0 +1,25 @@ +--- +#title: Corporate Memory Marketplace Package Lifecycle +--- +%%{init: { 'logLevel': 'debug', 'theme': 'neutral' } }%% +stateDiagram-v2 + + +state "Package Definition
and Release
(local)" as pkg +state "Published
(Marketplace)" as pub +state ".cpa Package
(local & CI/CD)" as cpa +state "Solution Dev and Config
(Corporate Memory)" as cmem +state "Installed
(Corporate Memory)" as ins +%% state "Uninstalled" as uni + +[*] --> pkg : copier copy +pkg --> pkg : inspect +cpa --> cmem : install
--input PATH +pkg --> cmem : install
--input PATH +cmem --> pkg : export +pkg --> cpa : build +cpa --> pub : publish + +pub --> ins : install +ins --> ins : list +ins --> [*] : uninstall diff --git a/docs/develop/marketplace-packages/mpp-lifecycle.svg b/docs/develop/marketplace-packages/mpp-lifecycle.svg new file mode 100644 index 000000000..a6835805a --- /dev/null +++ b/docs/develop/marketplace-packages/mpp-lifecycle.svg @@ -0,0 +1 @@ +

copier copy

inspect

install
--input PATH

install
--input PATH

export

build

publish

install

list

uninstall

Package Definition
and Release
(local)

Published
(Marketplace)

.cpa Package
(local & CI/CD)

Solution Dev and Config
(Corporate Memory)

Installed
(Corporate Memory)

 \ No newline at end of file diff --git a/docs/develop/python-plugins/installation/index.md b/docs/develop/python-plugins/installation/index.md index 538538263..770665282 100644 --- a/docs/develop/python-plugins/installation/index.md +++ b/docs/develop/python-plugins/installation/index.md @@ -8,7 +8,7 @@ tags: # Installation and Usage of Python Plugins Plugins are a released as parts of Python packages. -They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain of multiple plugins. +They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain multiple plugins. ## Installation