Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .copier-answers.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Changes here will be overwritten by Copier
_commit: v7.0.0
_commit: v7.3.0-24-g169e035
_src_path: gh:eccenca/cmem-plugin-template
author_mail: cmempy-developer@eccenca.com
author_name: eccenca GmbH
Expand Down
10 changes: 7 additions & 3 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,16 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v5

- name: Install Task
uses: arduino/setup-task@v2

- name: Set up python
id: setup-python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: '3.11'
python-version: '3.13'

- name: Install and configure poetry
uses: snok/install-poetry@v1
Expand Down Expand Up @@ -57,6 +57,10 @@ jobs:
run: |
task check:pytest

- name: deptry
run: |
task check:deptry

- name: safety
run: |
task check:safety
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,16 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v5

- name: Install Task
uses: arduino/setup-task@v2

- name: Set up python
id: setup-python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: '3.11'
python-version: '3.13'

- name: Install and configure poetry
uses: snok/install-poetry@v1
Expand Down
13 changes: 8 additions & 5 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
default:
image: docker-registry.eccenca.com/eccenca-python:v3.11.4
image: docker-registry.eccenca.com/eccenca-python:v3.13.8
# all jobs can be interrupted in case a new commit is pushed
interruptible: true
before_script:
Expand Down Expand Up @@ -53,10 +53,12 @@ pytest:
junit:
- dist/junit-pytest.xml
paths:
- dist/badge-coverage.svg
- dist/badge-tests.svg
- dist/coverage
- dist/coverage.xml
- dist/*

deptry:
stage: test
script:
- task check:deptry

safety:
stage: test
Expand All @@ -69,6 +71,7 @@ build:
- mypy
- pytest
- safety
- deptry
script:
- task build
artifacts:
Expand Down
2 changes: 1 addition & 1 deletion .python-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
3.11
3.13
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,26 @@
<!-- markdownlint-disable MD012 MD013 MD024 MD033 -->
# Change Log

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](https://semver.org/)

## [1.2.0] 2025-10-15

## Changed

- Update of dependencies and template
- validation of python 3.13 compatibility
- python 3.13 required


## [1.1.2] 2024-12-24

### Changed

- update `Validate Entities` plugin documentation


## [1.1.1] 2024-11-28

### Changed
Expand Down
2 changes: 1 addition & 1 deletion README-public.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Validate graph and data structures.

[![eccenca Corporate Memory][cmem-shield]][cmem-link]

This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com). You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line clients like this:
This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com). You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line client like this:

```
cmemc admin workspace python install cmem-plugin-validation
Expand Down
14 changes: 10 additions & 4 deletions Taskfile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@ includes:
custom:
taskfile: ./TaskfileCustom.yaml
optional: true
flatten: true
plugin:
taskfile: .tasks-plugin.yml
optional: true
flatten: true

tasks:

Expand Down Expand Up @@ -68,7 +70,6 @@ tasks:
| head -1 | cut -d " " -f 2 | cut -d "." -f 1-2

poetry:install:
internal: true
desc: Install dependencies managed by Poetry
run: once
deps:
Expand Down Expand Up @@ -110,6 +111,7 @@ tasks:
cmds:
- task: check:ruff
- task: check:mypy
- task: check:deptry
- task: check:safety

check:pytest:
Expand Down Expand Up @@ -157,9 +159,13 @@ tasks:
<<: *preparation
cmds:
# ignore 51358 safety - dev dependency only
# ignore 67599 pip - dev dependency only
# ignore 70612 jinja2 - dev dependency only
- poetry run safety check -i 51358 -i 67599 -i 70612
- poetry run safety check -i 51358

check:deptry:
desc: Complain about unused or missing dependencies
<<: *preparation
cmds:
- poetry run deptry .

check:ruff:
desc: Complain about everything else
Expand Down
2,138 changes: 1,322 additions & 816 deletions poetry.lock

Large diffs are not rendered by default.

27 changes: 14 additions & 13 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,29 @@ keywords = [
homepage = "https://github.com/eccenca/cmem-plugin-validation"

[tool.poetry.dependencies]# if you need to change python version here, change it also in .python-version
python = "^3.11"
jsonschema = "^4.23.0"
python = "^3.13"
jsonschema = "^4.25.1"
cmem-cmempy = ">=25.3.0"
requests = ">=2.0.1"

[tool.poetry.dependencies.cmem-plugin-base]
version = "^4.7.0"
version = "^4.15.0"
allow-prereleases = false

[tool.poetry.group.dev.dependencies.cmem-cmemc]
version = "^24.2.0"
version = ">=24.2.0"

[tool.poetry.group.dev.dependencies]
genbadge = {extras = ["coverage"], version = "^1.1.1"}
mypy = "^1.11.1"
pip = "^24"
pytest = "^8.3.2"
pytest-cov = "^5.0.0"
deptry = "^0.23.1"
genbadge = {extras = ["coverage"], version = "^1.1.2"}
mypy = "^1.18.2"
pip = "^25.2"
pytest = "^8.4.2"
pytest-cov = "^7.0.0"
pytest-dotenv = "^0.5.2"
pytest-html = "^4.1.1"
pytest-memray = { version = "^1.7.0", markers = "platform_system != 'Windows'" }
ruff = "^0.6.1"
pytest-memray = { version = "^1.8.0", markers = "platform_system != 'Windows'" }
ruff = "^0.13.3"
safety = "^1.10.3"
types-requests = "^2.31.0.20240406"

Expand Down Expand Up @@ -76,7 +79,6 @@ line-ending = "lf" # Use `\n` line endings for all files
[tool.ruff.lint]
select = ["ALL"]
ignore = [
"ANN101", # Missing type annotation for self in method
"ANN204", # Missing return type annotation for special method `__init__`
"COM812", # missing-trailing-comma
"D107", # Missing docstring in __init__
Expand All @@ -94,4 +96,3 @@ ignore = [
"S101", # use of assert detected
"TRY003", # Avoid specifying long messages outside the exception class
]

12 changes: 9 additions & 3 deletions tests/test_validate_entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,17 @@
import json
from collections.abc import Generator
from dataclasses import dataclass
from os import environ
from pathlib import Path

import pytest
from cmem.cmempy.workspace.projects.datasets.dataset import make_new_dataset
from cmem.cmempy.workspace.projects.project import delete_project, make_new_project
from cmem.cmempy.workspace.projects.resources.resource import create_resource, get_resource_response
from cmem_plugin_base.testing import TestExecutionContext

from cmem_plugin_validation.validate_entities.task import SOURCE, TARGET, ValidateEntity
from tests.fixtures import FIXTURE_DIR
from tests.utils import TestExecutionContext, needs_cmem


@dataclass
Expand Down Expand Up @@ -68,13 +69,18 @@ def project() -> Generator[TestSetup, None, None]:
delete_project(_.project_name)


needs_cmem = pytest.mark.skipif(
environ.get("CMEM_BASE_URI", "") == "", reason="Needs CMEM configuration"
)


@needs_cmem
def test_configuration(project: TestSetup) -> None:
"""Test configuration setup"""
_ = project
with pytest.raises(
ValueError,
match="When using the source mode 'dataset', you need to select a Source JSON Dataset.",
match=r"When using the source mode 'dataset', you need to select a Source JSON Dataset.",
):
ValidateEntity(
source_mode=SOURCE.dataset,
Expand All @@ -84,7 +90,7 @@ def test_configuration(project: TestSetup) -> None:
)
with pytest.raises(
ValueError,
match="When using the target mode 'dataset', you need to select a Target JSON dataset.",
match=r"When using the target mode 'dataset', you need to select a Target JSON dataset.",
):
ValidateEntity(
source_mode=SOURCE.entities,
Expand Down
42 changes: 25 additions & 17 deletions tests/test_validate_graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,11 @@
import cmem.cmempy.dp.proxy.graph as graph_api
import pytest
from cmem_plugin_base.dataintegration.entity import Entities
from cmem_plugin_base.testing import TestExecutionContext
from requests import HTTPError, codes

from cmem_plugin_validation.validate_graph.task import ValidateGraph
from tests.fixtures import FIXTURE_DIR
from tests.utils import TestExecutionContext


def _get_triple_count(graph: str) -> int:
Expand Down Expand Up @@ -42,12 +43,19 @@ def test_setup() -> Generator[TestSetup, Any, None]:
_ = TestSetup()
graph_api.post_streamed(replace=True, file=_.persons_file, graph=_.persons_graph)
graph_api.post_streamed(replace=True, file=_.shapes_file, graph=_.shapes_graph)
graph_api.delete(graph=_.result_graph)
try:
graph_api.delete(graph=_.result_graph)
except HTTPError as error:
if error.response.status_code != codes.not_found:
raise HTTPError from error
yield _
# purge setup
graph_api.delete(graph=_.persons_graph)
graph_api.delete(graph=_.shapes_graph)
graph_api.delete(graph=_.result_graph)
for graph in [_.persons_graph, _.shapes_graph, _.result_graph]:
try:
graph_api.delete(graph=graph)
except HTTPError as error:
if error.response.status_code != codes.not_found:
raise HTTPError from error


def test_fails(test_setup: TestSetup) -> None:
Expand Down Expand Up @@ -84,9 +92,9 @@ def test_output_results(test_setup: TestSetup) -> None:
assert isinstance(result, Entities)
entities = list(result.entities)
assert len(entities) == 1, "There should be a single violation entity"
assert entities[0].values[1] == [
"http://example.org/persons/2"
], "focus node of the only violation should be person 2"
assert entities[0].values[1] == ["http://example.org/persons/2"], (
"focus node of the only violation should be person 2"
)


def test_safe_as_graph(test_setup: TestSetup) -> None:
Expand All @@ -104,14 +112,14 @@ def test_safe_as_graph(test_setup: TestSetup) -> None:
result_graph_triples = _get_triple_count(_.result_graph)
assert result_graph_triples > 0, "result graph should be empty"
task.execute(context=TestExecutionContext(), inputs=[])
assert (
_get_triple_count(_.result_graph) == result_graph_triples * 2
), "result graph should have two equal result sets"
assert _get_triple_count(_.result_graph) == result_graph_triples * 2, (
"result graph should have two equal result sets"
)
task.clear_result_graph = True
task.execute(context=TestExecutionContext(), inputs=[])
assert (
_get_triple_count(_.result_graph) == result_graph_triples
), "result graph should have as single result sets again"
assert _get_triple_count(_.result_graph) == result_graph_triples, (
"result graph should have as single result sets again"
)


def test_different_query(test_setup: TestSetup) -> None:
Expand All @@ -135,6 +143,6 @@ def test_different_query(test_setup: TestSetup) -> None:
}
"""
task.sparql_query = query
assert (
task.execute(context=TestExecutionContext(), inputs=[]) is None
), "Should no violations, since no person was validated"
assert task.execute(context=TestExecutionContext(), inputs=[]) is None, (
"Should no violations, since no person was validated"
)
Loading