From b33971d692c6c92ddd274cad9afa8d3c9ee32549 Mon Sep 17 00:00:00 2001 From: "Donald F. Coffin" Date: Tue, 17 Nov 2026 18:37:16 -0500 Subject: [PATCH 1/2] feat(common): ESPI resources inside atom:content, and XSD validation of ESPI XML (#197) Atom entries now carry their ESPI resource as the single child of , as REQ.21's examples and CMD-certified Data Custodian feeds do, for both usage (UsageAtomEntryDto) and customer (CustomerAtomEntryDto) entries. The constructors and getContent() keep taking and returning the resource; getResource() now returns it after unmarshalling (it read a base-class field that was never set, so it was always null). - EspiXmlSchemaValidator (openespi-common xml/) validates a feed or entry against atom.xsd + espi.xsd + customer.xsd (the GBA DMD validator's files) from the classpath, with a local xml.xsd, reporting every issue with its line and column. It never fetches external DTDs or schemas. - The validator found two DC export defects, now fixed: feeds wrote a feed-level atom:published (RFC 4287 has none; atom.xsd rejects it), and an empty feed's self link had no type (atom.xsd requires one; it is now taken from the feed href). - AtomFeedExportTest checks that UsagePoint, empty and Customer feeds from the DC export validate; AtomContentAndSchemaValidationTest covers the content wrapper, a certified-style feed, round trips, error reporting with lines, and that external DTDs are not followed. - Third Party: UsagePointRESTRepositoryImpl handles an empty feed and skips entries that are not UsagePoints. The two @Disabled tests of UsagePointRESTRepositoryIntegrationTest are replaced by UsagePointRESTRepositoryImplTests (real WebClient, stubbed exchange). - docs/ESPI_ATOM_PROFILE.md records how ESPI deliberately differs from RFC 4287 (no atom:author for privacy, bare atom:content, link rel/type required) so they are not "fixed"; the DTOs, BaseExportService, the validator and CLAUDE.md point to it. ServiceCategory and other enums are still written as Java constant names (#278). Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 1 + docs/ESPI_ATOM_PROFILE.md | 59 ++++ .../espi/common/dto/atom/AtomEntryDto.java | 46 ++- .../espi/common/dto/atom/AtomFeedDto.java | 16 +- .../common/dto/atom/CustomerAtomEntryDto.java | 73 +++-- .../common/dto/atom/UsageAtomEntryDto.java | 108 ++++--- .../common/service/BaseExportService.java | 20 +- .../common/xml/EspiXmlSchemaValidator.java | 234 ++++++++++++++ .../main/resources/schema/ESPI_4.1/xml.xsd | 287 ++++++++++++++++++ .../common/service/AtomFeedExportTest.java | 21 +- .../AtomContentAndSchemaValidationTest.java | 209 +++++++++++++ .../impl/UsagePointRESTRepositoryImpl.java | 20 +- ...agePointRESTRepositoryIntegrationTest.java | 138 --------- .../UsagePointRESTRepositoryImplTests.java | 164 ++++++++++ 14 files changed, 1139 insertions(+), 257 deletions(-) create mode 100644 docs/ESPI_ATOM_PROFILE.md create mode 100644 openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java create mode 100644 openespi-common/src/main/resources/schema/ESPI_4.1/xml.xsd create mode 100644 openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java delete mode 100644 openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/integration/repository/UsagePointRESTRepositoryIntegrationTest.java create mode 100644 openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java diff --git a/CLAUDE.md b/CLAUDE.md index 31d8063f..0a065dde 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -235,6 +235,7 @@ ESPI uses Atom XML feeds for data exchange. Key patterns: - Entry links include `rel="self"` (resource URL), `rel="up"` (parent collection), `rel="related"` (related resources) - All IDs are Type 5 UUIDs in format: `urn:uuid:xxxxxxxx-xxxx-5xxx-xxxx-xxxxxxxxxxxx` - Link type attributes use `espi-entry/{resource}` or `espi-feed/{resource}` format +- Each entry carries its resource inside a bare `` (no `type`); feeds and entries have no `atom:author` (privacy). This is an ESPI profile of RFC 4287: do not "fix" it to match the RFC. See `docs/ESPI_ATOM_PROFILE.md`; `EspiXmlSchemaValidator` (openespi-common `xml/`) validates against atom.xsd + espi.xsd + customer.xsd ### Core ESPI Resources diff --git a/docs/ESPI_ATOM_PROFILE.md b/docs/ESPI_ATOM_PROFILE.md new file mode 100644 index 00000000..4175721a --- /dev/null +++ b/docs/ESPI_ATOM_PROFILE.md @@ -0,0 +1,59 @@ +# ESPI Atom profile + +ESPI resources are exchanged as Atom documents, "as described by Atom, IETF RFC 4287" (NAESB REQ.21 v4.1, REQ.21.6.2, p. 167). The HTTP method conventions come from the Atom Publishing Protocol, RFC 5023 (REQ.21.6.2.1, p. 169). RFC 5023 adds only its own `app:` namespace elements; Atom feeds and entries are as RFC 4287 defines them. + +ESPI does not use RFC 4287 to the letter. It uses an **ESPI profile** of it, defined in practice by: + +- the GBA `atom.xsd` used by the GBA DMD validator, copied unchanged to `openespi-common/src/main/resources/schema/ESPI_4.1/atom.xsd`; +- REQ.21's examples; +- the feeds of CMD-certified Data Custodians. + +This page lists the differences so that they are not "fixed" by mistake. **Do not change the code to follow RFC 4287 where this page says ESPI differs.** Doing so would either reveal which utility the data comes from, or make our feeds differ from every certified feed. + +## Intentional differences + +| # | ESPI (what we write and accept) | RFC 4287 | Why ESPI differs | +|---|---|---|---| +| 1 | No `atom:author` on feeds or entries | 4.1.1 / 4.1.2: a feed or each entry MUST have one or more `atom:author` | **Privacy.** An author would tie the data to a specific utility. espi.xsd, REQ.21's examples and certified feeds have none. | +| 2 | `` has no `type` attribute, and holds the ESPI resource (`espi:UsagePoint`, `cust:Customer`, ...) as its single child | 4.1.3.1: without `type`, content is treated as `type="text"`; 4.1.3.3 rule 1: text content MUST NOT contain child elements. An XML media type (`type="application/xml"`) would be needed. | REQ.21's examples, certified feeds and the DMD validator all use a bare ``. Adding a `type` would make our feeds differ from all of them for no gain. | +| 3 | Every `atom:link` has `rel` and `type`; every entry has at least a `self` and an `up` link | 4.2.7.2 / 4.2.7.3: `rel` and `type` are optional; any number of links | Required by the GBA `atom.xsd`. ESPI navigation depends on `self`, `up` and `related` links, with types such as `espi-entry/UsagePoint` and `espi-feed/UsagePoint`. | + +ESPI follows RFC 4287 where `atom.xsd` is looser. The code does not rely on that looseness: + +- **No feed-level `atom:published`.** RFC 4287 4.1.1 defines none, and `atom.xsd` rejects it. Only entries have `published` (REQ.21.6.2: `published` = created, `updated` = last changed). +- **Dates carry a time zone** (RFC 4287 3.3). `OffsetDateTimeAdapter` always writes the offset or `Z`. + +## What `atom.xsd` does not check + +The schema uses `xs:choice maxOccurs="unbounded"` for feeds, entries and persons, so it **cannot** enforce these RFC 4287 rules: + +- exactly one `id`, `title` and `updated` per feed and entry, and at most one `published` per entry; +- a time zone on dates (`xs:dateTime` allows a local time); +- entries after the feed's metadata. + +`EspiXmlSchemaValidator` reports only what the schemas report, the same as the DMD validator. Checks for the rules above could be added as warnings. Never add checks for differences 1–3. + +## Known issues in `atom.xsd` + +These are GBA schema matters, not to be changed in our copy, which must stay identical to the DMD validator's: + +- **Email pattern:** `atom:email` uses the pattern `\w+@(\w+\.)+\w+`, which rejects valid addresses such as `first.last@example.com` (RFC 4287 3.2.3). ESPI feeds carry no `author`, so this does not affect them. +- **`atom:source`:** an entry's `atom:source` is typed `textType`, while the `sourceType` the schema defines is unused, so a `source` holding feed metadata (RFC 4287 4.2.11) is rejected. ESPI does not use `source`. + +## Open question: link `type` for related resources + +REQ.21 p. 168 says related links "shall" use `type="application/atom+xml"`, for example `UsagePoint -> MeterReading`. Certified feeds, the DMD validator's samples and our Data Custodian use the ESPI form instead, `type="espi-entry/MeterReading"` or `espi-feed/...`. + +`atom.xsd` accepts either, because `type` is any string, and our Third Party does not depend on the attribute. We follow certified practice until the GBA decides whether REQ.21 needs an erratum. + +## Where this is implemented + +| Concern | Code | +|---|---| +| Feed (no `author`, no feed `published`) | `openespi-common/.../dto/atom/AtomFeedDto.java` | +| Entry (resource inside a bare `atom:content`) | `UsageAtomEntryDto.java`, `CustomerAtomEntryDto.java` | +| Links (`type` on every link, also on an empty feed) | `openespi-common/.../service/BaseExportService.java` (`buildFeed`) | +| Validation against atom.xsd + espi.xsd + customer.xsd | `openespi-common/.../xml/EspiXmlSchemaValidator.java` | +| Tests | `AtomContentAndSchemaValidationTest`, `AtomFeedExportTest` | + +History: #197 (the `atom:content` wrapper, the schema validator, the feed `published` and empty-feed link fixes), #247 (feed date adapters, entry types). diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomEntryDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomEntryDto.java index 69095a74..a6318b55 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomEntryDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomEntryDto.java @@ -68,39 +68,36 @@ public abstract class AtomEntryDto { private List links; /** - * ESPI resource content - subclasses define specific @XmlElements - * for their domain (usage or customer) to ensure proper namespace declarations. + * Constructor for the Atom metadata; the subclass sets the ESPI resource ({@link #setContent}). */ - @XmlTransient - protected Object content; - - /** - * All-args constructor. - */ - public AtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated, - List links, Object content) { + protected AtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated, + List links) { this.id = id; this.title = title; this.published = published; this.updated = updated; this.links = links; - this.content = content; } /** - * Convenience constructor for testing with auto-generated timestamps. - * Creates an AtomEntryDto with current timestamp and no links. - * - * @param id the entry identifier (urn:uuid:xxx format) - * @param title the entry title - * @param content the resource content + * Constructor with the current time (to the second) as published and updated, and no links. */ - public AtomEntryDto(String id, String title, Object content) { - LocalDateTime localDateTime = LocalDateTime.now().truncatedTo(java.time.temporal.ChronoUnit.SECONDS); - OffsetDateTime now = localDateTime.atOffset(ZoneOffset.UTC).toZonedDateTime().toOffsetDateTime(); - this(id, title, now, now, null, content); + protected AtomEntryDto(String id, String title) { + OffsetDateTime now = LocalDateTime.now().truncatedTo(java.time.temporal.ChronoUnit.SECONDS) + .atOffset(ZoneOffset.UTC); + this(id, title, now, now, null); } + /** + * The ESPI resource this entry carries ({@code espi:UsagePoint}, {@code cust:Customer}, ...). In the XML + * it is the single child of the entry's {@code atom:content} element, as REQ.21 and certified feeds use + * it (#197). Each subclass maps it for its own schema (usage or customer), so that only that schema's + * namespace is declared. + */ + public abstract Object getContent(); + + public abstract void setContent(Object content); + /** * Gets the self link from the entry links. @@ -127,11 +124,12 @@ public LinkDto getUpLink() { } /** - * Gets the resource content from the entry. + * Gets the ESPI resource carried by the entry: the same as {@link #getContent()}. It used to return a + * base-class field that unmarshalling never set, so it was always null for a parsed entry (#197). * - * @return resource content or null if not available + * @return the resource, or null if the entry has none */ public Object getResource() { - return content; // != null ? content.resource() : null; + return getContent(); } } \ No newline at end of file diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java index 0be283cf..6ac75a06 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java @@ -39,11 +39,16 @@ * NOTE: @XmlSeeAlso was removed to prevent namespace pollution. Domain-specific * export services (UsageExportService, CustomerExportService) explicitly register * needed classes in JAXBContext to ensure only 2 namespaces are declared. + *

+ * ESPI profile of RFC 4287 (see {@code docs/ESPI_ATOM_PROFILE.md}): a feed has no + * {@code atom:author}, on purpose. RFC 4287 4.1.1 requires one, but it would tie the data to a + * specific utility (privacy), and espi.xsd and certified feeds have none. Do not add it. A feed also has + * no {@code atom:published} (RFC 4287 defines none for a feed; atom.xsd rejects it). */ @XmlRootElement(name = "feed", namespace = "http://www.w3.org/2005/Atom") @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "AtomFeed", namespace = "http://www.w3.org/2005/Atom", propOrder = { - "id", "title", "published", "updated", "links", "entries" + "id", "title", "updated", "links", "entries" }) @Getter @Setter @@ -57,11 +62,14 @@ public class AtomFeedDto { @XmlElement(name = "title") private String title; - // Without the adapter JAXB cannot map OffsetDateTime and writes an empty element (#247). - @XmlElement(name = "published") - @XmlJavaTypeAdapter(OffsetDateTimeAdapter.class) + /** + * Not written: an Atom feed has no {@code published} element (RFC 4287 4.1.1; atom.xsd's feedType + * rejects it, and certified ESPI feeds have none). Only entries carry {@code published} (#197). + */ + @XmlTransient private OffsetDateTime published; + // Without the adapter JAXB cannot map OffsetDateTime and writes an empty element (#247). @XmlElement(name = "updated") @XmlJavaTypeAdapter(OffsetDateTimeAdapter.class) private OffsetDateTime updated; diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/CustomerAtomEntryDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/CustomerAtomEntryDto.java index 34951990..4aa9c806 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/CustomerAtomEntryDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/CustomerAtomEntryDto.java @@ -32,41 +32,31 @@ * Specialized entry for customer.xsd resources (Customer, CustomerAccount, etc.) * that ensures only the cust namespace (http://naesb.org/espi/customer) is declared. *

+ * The resource is the single child of the entry's {@code atom:content} element (#197), as in + * {@link UsageAtomEntryDto}: a bare {@code atom:content} with no {@code type} and no {@code atom:author}, + * the ESPI profile of RFC 4287 ({@code docs/ESPI_ATOM_PROFILE.md}). + *

* Per NAESB ESPI standard, usage and customer domains are mutually exclusive. */ @XmlRootElement(name = "entry", namespace = "http://www.w3.org/2005/Atom") @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "CustomerAtomEntry", namespace = "http://www.w3.org/2005/Atom", propOrder = { - "content" + "atomContent" }) @NoArgsConstructor public class CustomerAtomEntryDto extends AtomEntryDto { - /** - * Customer domain resource content - only customer.xsd types. - * JAXB will only declare xmlns:cust namespace since all types use http://naesb.org/espi/customer. - */ - @XmlElements({ - @XmlElement(name = "Customer", type = CustomerDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "CustomerAccount", type = CustomerAccountDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "CustomerAgreement", type = CustomerAgreementDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "EndDevice", type = EndDeviceDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "Meter", type = MeterDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "ProgramDateIdMappings", type = ProgramDateIdMappingsDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "ServiceLocation", type = ServiceLocationDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "Statement", type = StatementDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "StatementRef", type = StatementRefDto.class, namespace = "http://naesb.org/espi/customer"), - @XmlElement(name = "ServiceSupplier", type = ServiceSupplierDto.class, namespace = "http://naesb.org/espi/customer") - }) - protected Object content; + /** The {@code atom:content} element holding one customer.xsd resource. */ + @XmlElement(name = "content", namespace = "http://www.w3.org/2005/Atom") + private Content atomContent; /** * All-args constructor. */ public CustomerAtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated, List links, Object content) { - super(id, title, published, updated, links, content); - this.content = content; + super(id, title, published, updated, links); + setContent(content); } /** @@ -77,24 +67,45 @@ public CustomerAtomEntryDto(String id, String title, OffsetDateTime published, O * @param content the customer resource content */ public CustomerAtomEntryDto(String id, String title, Object content) { - super(id, title, content); - this.content = content; + super(id, title); + setContent(content); } - /** - * Override getContent to return this class's content field (with @XmlElements) - * instead of parent's @XmlTransient content field. - */ @Override public Object getContent() { - return this.content; + return atomContent == null ? null : atomContent.resource; } - /** - * Override setContent to set this class's content field. - */ @Override public void setContent(Object content) { - this.content = content; + this.atomContent = content == null ? null : new Content(content); + } + + /** + * The {@code atom:content} element of a customer entry. Only customer.xsd types are listed, so JAXB + * declares only the {@code cust} namespace (http://naesb.org/espi/customer). + */ + @XmlAccessorType(XmlAccessType.FIELD) + @XmlType(name = "CustomerAtomContent", namespace = "http://www.w3.org/2005/Atom") + @NoArgsConstructor + public static class Content { + + @XmlElements({ + @XmlElement(name = "Customer", type = CustomerDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "CustomerAccount", type = CustomerAccountDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "CustomerAgreement", type = CustomerAgreementDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "EndDevice", type = EndDeviceDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "Meter", type = MeterDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "ProgramDateIdMappings", type = ProgramDateIdMappingsDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "ServiceLocation", type = ServiceLocationDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "Statement", type = StatementDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "StatementRef", type = StatementRefDto.class, namespace = "http://naesb.org/espi/customer"), + @XmlElement(name = "ServiceSupplier", type = ServiceSupplierDto.class, namespace = "http://naesb.org/espi/customer") + }) + private Object resource; + + Content(Object resource) { + this.resource = resource; + } } } diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/UsageAtomEntryDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/UsageAtomEntryDto.java index af8e7e66..89fcb2f9 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/UsageAtomEntryDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/UsageAtomEntryDto.java @@ -33,53 +33,42 @@ * Specialized entry for usage.xsd resources (UsagePoint, MeterReading, etc.) * that ensures only the espi namespace (http://naesb.org/espi) is declared. *

+ * The resource is the single child of the entry's {@code atom:content} element (#197): + *

{@code
+ * 
+ *     ... ... 
+ *     
+ *         ...
+ *     
+ * 
+ * }
+ *

+ * ESPI profile of RFC 4287 (see {@code docs/ESPI_ATOM_PROFILE.md}): {@code atom:content} has no + * {@code type} attribute, as in REQ.21's examples, certified feeds and the DMD validator. RFC 4287 + * 4.1.3.1 would treat it as text and require an XML media type for element content; do not add one. The + * entry has no {@code atom:author} either (privacy: it would identify the utility). + *

* Per NAESB ESPI standard, usage and customer domains are mutually exclusive. */ @XmlRootElement(name = "entry", namespace = "http://www.w3.org/2005/Atom") @XmlAccessorType(XmlAccessType.FIELD) @XmlType(name = "UsageAtomEntry", namespace = "http://www.w3.org/2005/Atom", propOrder = { - "content" + "atomContent" }) @NoArgsConstructor public class UsageAtomEntryDto extends AtomEntryDto { - /** - * Usage domain resource content - only usage.xsd types. - * JAXB will only declare xmlns:espi namespace since all types use http://naesb.org/espi. - */ - @XmlElements({ - @XmlElement(name = "UsagePoint", type = UsagePointDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "MeterReading", type = MeterReadingDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "IntervalBlock", type = IntervalBlockDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "ReadingType", type = ReadingTypeDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "ElectricPowerQualitySummary", type = ElectricPowerQualitySummaryDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "UsageSummary", type = UsageSummaryDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "TimeConfiguration", type = TimeConfigurationDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "ApplicationInformation", type = ApplicationInformationDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "Authorization", type = AuthorizationDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "Subscription", type = SubscriptionDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "BatchList", type = BatchListDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "LineItem", type = LineItemDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "ServiceDeliveryPoint", type = ServiceDeliveryPointDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "ReadingQuality", type = ReadingQualityDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "IntervalReading", type = IntervalReadingDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "DateTimeInterval", type = DateTimeIntervalDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "TariffRiderRef", type = TariffRiderRefDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "TariffRiderRefs", type = TariffRiderRefsDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "PnodeRef", type = PnodeRefDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "PnodeRefs", type = PnodeRefsDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "AggregatedNodeRef", type = AggregatedNodeRefDto.class, namespace = "http://naesb.org/espi"), - @XmlElement(name = "AggregatedNodeRefs", type = AggregatedNodeRefsDto.class, namespace = "http://naesb.org/espi") - }) - protected Object content; + /** The {@code atom:content} element holding one usage.xsd resource. */ + @XmlElement(name = "content", namespace = "http://www.w3.org/2005/Atom") + private Content atomContent; /** * All-args constructor. */ public UsageAtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated, List links, Object content) { - super(id, title, published, updated, links, content); - this.content = content; + super(id, title, published, updated, links); + setContent(content); } /** @@ -90,24 +79,57 @@ public UsageAtomEntryDto(String id, String title, OffsetDateTime published, Offs * @param content the usage resource content */ public UsageAtomEntryDto(String id, String title, Object content) { - super(id, title, content); - this.content = content; + super(id, title); + setContent(content); } - /** - * Override getContent to return this class's content field (with @XmlElements) - * instead of parent's @XmlTransient content field. - */ @Override public Object getContent() { - return this.content; + return atomContent == null ? null : atomContent.resource; } - /** - * Override setContent to set this class's content field. - */ @Override public void setContent(Object content) { - this.content = content; + this.atomContent = content == null ? null : new Content(content); + } + + /** + * The {@code atom:content} element of a usage entry. Only usage.xsd types are listed, so JAXB declares + * only the {@code espi} namespace (http://naesb.org/espi). + */ + @XmlAccessorType(XmlAccessType.FIELD) + @XmlType(name = "UsageAtomContent", namespace = "http://www.w3.org/2005/Atom") + @NoArgsConstructor + public static class Content { + + @XmlElements({ + @XmlElement(name = "UsagePoint", type = UsagePointDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "MeterReading", type = MeterReadingDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "IntervalBlock", type = IntervalBlockDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "ReadingType", type = ReadingTypeDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "ElectricPowerQualitySummary", type = ElectricPowerQualitySummaryDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "UsageSummary", type = UsageSummaryDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "TimeConfiguration", type = TimeConfigurationDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "ApplicationInformation", type = ApplicationInformationDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "Authorization", type = AuthorizationDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "Subscription", type = SubscriptionDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "BatchList", type = BatchListDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "LineItem", type = LineItemDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "ServiceDeliveryPoint", type = ServiceDeliveryPointDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "ReadingQuality", type = ReadingQualityDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "IntervalReading", type = IntervalReadingDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "DateTimeInterval", type = DateTimeIntervalDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "TariffRiderRef", type = TariffRiderRefDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "TariffRiderRefs", type = TariffRiderRefsDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "PnodeRef", type = PnodeRefDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "PnodeRefs", type = PnodeRefsDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "AggregatedNodeRef", type = AggregatedNodeRefDto.class, namespace = "http://naesb.org/espi"), + @XmlElement(name = "AggregatedNodeRefs", type = AggregatedNodeRefsDto.class, namespace = "http://naesb.org/espi") + }) + private Object resource; + + Content(Object resource) { + this.resource = resource; + } } } diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java index 41af22ca..f2a2dc70 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java @@ -238,14 +238,19 @@ public void exportFeed(String feedHref, String titl /** * Builds an Atom feed whose entries carry the ESPI resources, per REQ.21: *

    - *
  • feed: {@code urn:uuid} id (UUID v5 of {@code feedHref}), title, published/updated (now, UTC), - * {@code self} link of type {@code espi-feed/};
  • + *
  • feed: {@code urn:uuid} id (UUID v5 of {@code feedHref}), title, updated (now, UTC; an Atom + * feed has no published), {@code self} link of type {@code espi-feed/}, also when the + * feed is empty;
  • *
  • entry: {@code urn:uuid:}, title (the entity description, else the resource name), * the entity's published/updated, {@code self} ({@code espi-entry/}) and {@code up} - * ({@code espi-feed/}) links, and the mapped DTO as {@code }.
  • + * ({@code espi-feed/}) links, and the mapped DTO inside {@code }. *
* Stored self/up links are used when they are absolute URLs; otherwise they are derived from * {@code feedHref} ({@code /} and {@code feedHref}). + *

+ * Every link carries {@code rel} and {@code type}, and every entry has {@code self} and {@code up}: the GBA + * atom.xsd requires them, although RFC 4287 makes them optional. No {@code atom:author} is written + * (privacy). See {@code docs/ESPI_ATOM_PROFILE.md}. */ public AtomFeedDto buildFeed(String feedHref, String title, List entities, Function toContent) { @@ -277,8 +282,9 @@ public AtomFeedDto buildFeed(String feedHref, Strin feed.setTitle(title); feed.setPublished(now); feed.setUpdated(now); + // atom.xsd requires a type on every link: an empty feed takes the resource from its href (#197). feed.setLinks(List.of(new LinkDto("self", feedHref, - resource != null ? "espi-feed/" + resource : null))); + "espi-feed/" + (resource != null ? resource : lastPathSegment(feedHref))))); feed.setEntries(entries); return feed; } @@ -295,6 +301,12 @@ protected AtomEntryDto newEntry(String id, String title, OffsetDateTime publishe private static final EspiIdGeneratorService ID_GENERATOR = new EspiIdGeneratorService(); + /** The last path segment of a feed href ({@code .../resource/UsagePoint?x=1} gives {@code UsagePoint}). */ + static String lastPathSegment(String href) { + String path = href.replaceFirst("[?#].*$", "").replaceFirst("/+$", ""); + return path.substring(path.lastIndexOf('/') + 1); + } + /** The ESPI resource name: the DTO's {@code @XmlRootElement} name, else its class name without "Dto". */ static String resourceName(Object content) { XmlRootElement root = content.getClass().getAnnotation(XmlRootElement.class); diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java new file mode 100644 index 00000000..8cf281c9 --- /dev/null +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java @@ -0,0 +1,234 @@ +/* + * + * Copyright (c) 2025-2026 Green Button Alliance, Inc. + * + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.greenbuttonalliance.espi.common.xml; + +import org.w3c.dom.ls.LSInput; +import org.w3c.dom.ls.LSResourceResolver; +import org.xml.sax.ErrorHandler; +import org.xml.sax.SAXException; +import org.xml.sax.SAXParseException; + +import javax.xml.XMLConstants; +import javax.xml.transform.Source; +import javax.xml.transform.stream.StreamSource; +import javax.xml.validation.Schema; +import javax.xml.validation.SchemaFactory; +import javax.xml.validation.Validator; +import java.io.ByteArrayInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.Reader; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** + * Validates ESPI XML (an Atom {@code feed} or {@code entry}) against the normative schemas: atom.xsd, + * espi.xsd and customer.xsd (REQ.21 v4.1 6.5 and 6.6), the same files the GBA DMD validator uses (#197). + * + *

Atom's {@code content} and {@code entry} accept foreign elements with strict processing, so an ESPI + * resource inside them ({@code espi:UsagePoint}, {@code cust:Customer}, ...) is validated against its own + * schema. Every problem is reported with its line and column, not only the first. The schemas are read from + * the classpath; nothing is fetched from the network, and the document's DTDs and schema locations are + * never followed.

+ * + *

atom.xsd is the GBA's ESPI profile of RFC 4287, not RFC 4287 itself (see + * {@code docs/ESPI_ATOM_PROFILE.md}). It is stricter than the RFC (every link needs {@code rel} and + * {@code type}; an entry needs at least two links) and looser (it cannot enforce exactly one {@code id}, + * {@code title} or {@code updated}, or a time zone on dates). This class reports exactly what the schemas + * report, like the DMD validator. Never add checks that ESPI deliberately differs from, such as a missing + * {@code atom:author} (privacy) or {@code atom:content} without a {@code type}. Keep the schema files + * identical to the DMD validator's.

+ * + *

Thread-safe: the compiled {@link Schema} is shared, and each call uses its own {@link Validator}.

+ */ +public final class EspiXmlSchemaValidator { + + private static final String SCHEMA_ROOT = "/schema/"; + private static final String ATOM_XSD = SCHEMA_ROOT + "ESPI_4.1/atom.xsd"; + private static final String XML_XSD = SCHEMA_ROOT + "ESPI_4.1/xml.xsd"; + private static final String ESPI_XSD = SCHEMA_ROOT + "ESPI_4.0/espi.xsd"; + private static final String CUSTOMER_XSD = SCHEMA_ROOT + "ESPI_4.0/customer.xsd"; + + /** One problem found in the document. */ + public record Issue(Severity severity, int line, int column, String message) { + + @Override + public String toString() { + return severity + " at line " + line + ", column " + column + ": " + message; + } + } + + public enum Severity { WARNING, ERROR, FATAL } + + /** The outcome of one validation; valid when there is no ERROR or FATAL issue. */ + public record Result(List issues) { + + public Result { + issues = List.copyOf(issues); + } + + public boolean valid() { + return errorCount() == 0; + } + + public long errorCount() { + return issues.stream().filter(issue -> issue.severity() != Severity.WARNING).count(); + } + } + + private static final class Holder { + private static final EspiXmlSchemaValidator INSTANCE = new EspiXmlSchemaValidator(); + } + + private final Schema schema; + + private EspiXmlSchemaValidator() { + this.schema = compile(); + } + + /** The shared validator; the schemas are compiled once, on first use. */ + public static EspiXmlSchemaValidator instance() { + return Holder.INSTANCE; + } + + public Result validate(String xml) { + return validate(xml.getBytes(StandardCharsets.UTF_8)); + } + + /** Validates the raw bytes as received, so the XML declaration's encoding is honoured. */ + public Result validate(byte[] xml) { + List issues = new ArrayList<>(); + Validator validator = schema.newValidator(); + try { + validator.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, ""); + validator.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, ""); + validator.setErrorHandler(new CollectingErrorHandler(issues)); + validator.validate(new StreamSource(new ByteArrayInputStream(xml))); + } catch (SAXParseException ex) { + // Already recorded by the error handler (a fatal error stops the parse). + } catch (SAXException | IOException ex) { + issues.add(new Issue(Severity.FATAL, -1, -1, ex.getMessage())); + } + return new Result(issues); + } + + private static Schema compile() { + SchemaFactory factory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI); + try { + factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true); + // The imports are answered from the classpath by the resolver below; only those may load. + factory.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, ""); + factory.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "file,jar,jrt"); + factory.setResourceResolver(new ClasspathSchemaResolver()); + return factory.newSchema(new Source[] {source(ATOM_XSD), source(ESPI_XSD), source(CUSTOMER_XSD)}); + } catch (SAXException ex) { + throw new IllegalStateException("The ESPI schemas (atom.xsd, espi.xsd, customer.xsd) could not be compiled", ex); + } + } + + private static Source source(String resource) { + URL url = resourceUrl(resource); + return new StreamSource(url.toExternalForm()); + } + + private static URL resourceUrl(String resource) { + URL url = EspiXmlSchemaValidator.class.getResource(resource); + if (url == null) { + throw new IllegalStateException("Schema not on the classpath: " + resource); + } + return url; + } + + /** + * espi.xsd and customer.xsd import {@code atom.xsd} by a relative location (it lives in another folder + * here), and atom.xsd imports {@code xml.xsd} from w3.org. Both are answered from the classpath. + */ + private static final class ClasspathSchemaResolver implements LSResourceResolver { + + @Override + public LSInput resolveResource(String type, String namespaceUri, String publicId, String systemId, + String baseUri) { + if (XMLConstants.XML_NS_URI.equals(namespaceUri)) { + return input(XML_XSD, publicId); + } + if ("http://www.w3.org/2005/Atom".equals(namespaceUri)) { + return input(ATOM_XSD, publicId); + } + return null; + } + + private static LSInput input(String resource, String publicId) { + return new ClasspathInput(resourceUrl(resource).toExternalForm(), publicId); + } + } + + /** An {@link LSInput} whose system id is a classpath URL; the parser opens it itself. */ + private record ClasspathInput(String systemId, String publicId) implements LSInput { + + @Override public String getSystemId() { return systemId; } + @Override public String getPublicId() { return publicId; } + @Override public Reader getCharacterStream() { return null; } + @Override public InputStream getByteStream() { return null; } + @Override public String getStringData() { return null; } + @Override public String getBaseURI() { return null; } + @Override public String getEncoding() { return null; } + @Override public boolean getCertifiedText() { return false; } + @Override public void setCharacterStream(Reader characterStream) { } + @Override public void setByteStream(InputStream byteStream) { } + @Override public void setStringData(String stringData) { } + @Override public void setSystemId(String systemId) { } + @Override public void setPublicId(String publicId) { } + @Override public void setBaseURI(String baseUri) { } + @Override public void setEncoding(String encoding) { } + @Override public void setCertifiedText(boolean certifiedText) { } + } + + private record CollectingErrorHandler(List issues) implements ErrorHandler { + + @Override + public void warning(SAXParseException ex) { + add(Severity.WARNING, ex); + } + + @Override + public void error(SAXParseException ex) { + add(Severity.ERROR, ex); + } + + @Override + public void fatalError(SAXParseException ex) throws SAXParseException { + add(Severity.FATAL, ex); + throw ex; + } + + private void add(Severity severity, SAXParseException ex) { + issues.add(new Issue(severity, ex.getLineNumber(), ex.getColumnNumber(), ex.getMessage())); + } + } + + /** For tests and diagnostics: the issues as text, one per line. */ + public static List describe(Result result) { + return result == null ? Collections.emptyList() + : result.issues().stream().map(Issue::toString).toList(); + } +} diff --git a/openespi-common/src/main/resources/schema/ESPI_4.1/xml.xsd b/openespi-common/src/main/resources/schema/ESPI_4.1/xml.xsd new file mode 100644 index 00000000..41f5f2de --- /dev/null +++ b/openespi-common/src/main/resources/schema/ESPI_4.1/xml.xsd @@ -0,0 +1,287 @@ + + + + + + +
+

About the XML namespace

+ +
+

+ This schema document describes the XML namespace, in a form + suitable for import by other schema documents. +

+

+ See + http://www.w3.org/XML/1998/namespace.html and + + http://www.w3.org/TR/REC-xml for information + about this namespace. +

+

+ Note that local names in this namespace are intended to be + defined only by the World Wide Web Consortium or its subgroups. + The names currently defined in this namespace are listed below. + They should not be used with conflicting semantics by any Working + Group, specification, or document instance. +

+

+ See further below in this document for more information about how to refer to this schema document from your own + XSD schema documents and about the + namespace-versioning policy governing this schema document. +

+
+
+
+
+ + + + +
+ +

lang (as an attribute name)

+

+ denotes an attribute whose value + is a language code for the natural language of the content of + any element; its value is inherited. This name is reserved + by virtue of its definition in the XML specification.

+ +
+
+

Notes

+

+ Attempting to install the relevant ISO 2- and 3-letter + codes as the enumerated possible values is probably never + going to be a realistic possibility. +

+

+ See BCP 47 at + http://www.rfc-editor.org/rfc/bcp/bcp47.txt + and the IANA language subtag registry at + + http://www.iana.org/assignments/language-subtag-registry + for further information. +

+

+ The union allows for the 'un-declaration' of xml:lang with + the empty string. +

+
+
+
+ + + + + + + + + +
+ + + + +
+ +

space (as an attribute name)

+

+ denotes an attribute whose + value is a keyword indicating what whitespace processing + discipline is intended for the content of the element; its + value is inherited. This name is reserved by virtue of its + definition in the XML specification.

+ +
+
+
+ + + + + + +
+ + + +
+ +

base (as an attribute name)

+

+ denotes an attribute whose value + provides a URI to be used as the base for interpreting any + relative URIs in the scope of the element on which it + appears; its value is inherited. This name is reserved + by virtue of its definition in the XML Base specification.

+ +

+ See http://www.w3.org/TR/xmlbase/ + for information about this attribute. +

+
+
+
+
+ + + + +
+ +

id (as an attribute name)

+

+ denotes an attribute whose value + should be interpreted as if declared to be of type ID. + This name is reserved by virtue of its definition in the + xml:id specification.

+ +

+ See http://www.w3.org/TR/xml-id/ + for information about this attribute. +

+
+
+
+
+ + + + + + + + + + +
+ +

Father (in any context at all)

+ +
+

+ denotes Jon Bosak, the chair of + the original XML Working Group. This name is reserved by + the following decision of the W3C XML Plenary and + XML Coordination groups: +

+
+

+ In appreciation for his vision, leadership and + dedication the W3C XML Plenary on this 10th day of + February, 2000, reserves for Jon Bosak in perpetuity + the XML name "xml:Father". +

+
+
+
+
+
+ + + +
+

About this schema document

+ +
+

+ This schema defines attributes and an attribute group suitable + for use by schemas wishing to allow xml:base, + xml:lang, xml:space or + xml:id attributes on elements they define. +

+

+ To enable this, such a schema must import this schema for + the XML namespace, e.g. as follows: +

+
+          <schema . . .>
+           . . .
+           <import namespace="http://www.w3.org/XML/1998/namespace"
+                      schemaLocation="http://www.w3.org/2001/xml.xsd"/>
+     
+

+ or +

+
+           <import namespace="http://www.w3.org/XML/1998/namespace"
+                      schemaLocation="http://www.w3.org/2009/01/xml.xsd"/>
+     
+

+ Subsequently, qualified reference to any of the attributes or the + group defined below will have the desired effect, e.g. +

+
+          <type . . .>
+           . . .
+           <attributeGroup ref="xml:specialAttrs"/>
+     
+

+ will define a type which will schema-validate an instance element + with any of those attributes. +

+
+
+
+
+ + + +
+

Versioning policy for this schema document

+
+

+ In keeping with the XML Schema WG's standard versioning + policy, this schema document will persist at + + http://www.w3.org/2009/01/xml.xsd. +

+

+ At the date of issue it can also be found at + + http://www.w3.org/2001/xml.xsd. +

+

+ The schema document at that URI may however change in the future, + in order to remain compatible with the latest version of XML + Schema itself, or with the XML namespace itself. In other words, + if the XML Schema or XML namespaces change, the version of this + document at + http://www.w3.org/2001/xml.xsd + + will change accordingly; the version at + + http://www.w3.org/2009/01/xml.xsd + + will not change. +

+

+ Previous dated (and unchanging) versions of this schema + document are at: +

+ +
+
+
+
+ +
+ diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java index f044f791..8d10b052 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java @@ -23,6 +23,7 @@ import org.greenbuttonalliance.espi.common.dto.usage.UsagePointDto; import org.greenbuttonalliance.espi.common.service.impl.CustomerExportService; import org.greenbuttonalliance.espi.common.service.impl.UsageExportService; +import org.greenbuttonalliance.espi.common.xml.EspiXmlSchemaValidator; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; @@ -84,6 +85,12 @@ void usagePointsBecomeAFeed() { .doesNotContain("xsi:type", "", ""); assertThat(feedId).matches("urn:uuid:[0-9a-f]{8}-[0-9a-f]{4}-5[0-9a-f]{3}-[0-9a-f]{4}-[0-9a-f]{12}"); assertThat(count(xml, "")).isEqualTo(2); + assertThat(xml).containsPattern("\\s*]*>\\s*[^<]*\\s*[^<]*\\s*"); + EspiXmlSchemaValidator.Result validation = EspiXmlSchemaValidator.instance().validate(xml); + assertThat(validation.valid()).as(String.join("\n", EspiXmlSchemaValidator.describe(validation))).isTrue(); } @Test @@ -92,9 +99,11 @@ void emptyListIsAnEmptyFeed() { String xml = export(usage, List.of()); assertThat(xml) - .contains("\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}Z") .doesNotContain("\\s* links = java.util.regex.Pattern.compile("]*/>").matcher(xml) .results().map(java.util.regex.MatchResult::group).toList(); - // 2 feeds' self links + 1 entry's self and up links + // 2 feeds' self links + 1 entry's self and up links; atom.xsd requires type on every link (#197) assertThat(links).hasSize(4) .allSatisfy(link -> assertThat(link) - .matches("")); - assertThat(links).filteredOn(link -> link.contains("type=")).hasSize(3); + .matches("")); } @Test diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java new file mode 100644 index 00000000..a204cd15 --- /dev/null +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java @@ -0,0 +1,209 @@ +/* + * + * Copyright (c) 2025-2026 Green Button Alliance, Inc. + * + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.greenbuttonalliance.espi.common.xml; + +import jakarta.xml.bind.JAXBContext; +import jakarta.xml.bind.Marshaller; +import org.greenbuttonalliance.espi.common.dto.atom.AtomFeedDto; +import org.greenbuttonalliance.espi.common.dto.atom.CustomerAtomEntryDto; +import org.greenbuttonalliance.espi.common.dto.atom.LinkDto; +import org.greenbuttonalliance.espi.common.dto.atom.UsageAtomEntryDto; +import org.greenbuttonalliance.espi.common.dto.customer.CustomerDto; +import org.greenbuttonalliance.espi.common.dto.usage.UsagePointDto; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; + +import java.io.StringReader; +import java.io.StringWriter; +import java.time.OffsetDateTime; +import java.time.ZoneOffset; +import java.util.List; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Atom entries carry their ESPI resource inside {@code atom:content}, as REQ.21 and certified Data + * Custodian feeds do, and {@link EspiXmlSchemaValidator} checks a document against atom.xsd, espi.xsd and + * customer.xsd (#197). + */ +class AtomContentAndSchemaValidationTest { + + private static final OffsetDateTime TIME = OffsetDateTime.of(2026, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC); + private static final String UP_ID = "urn:uuid:07171718-b165-522a-b413-11d9eeb1ca43"; + private static final String BASE = "https://dc.example/DataCustodian/espi/1_1/resource"; + + /** + * The shape of a certified Data Custodian's feed (from a CMD certification ScriptLog), with synthetic + * values: each resource inside atom:content, ServiceCategory as a numeric kind. + */ + private static final String CERTIFIED_STYLE_FEED = """ + + + urn:uuid:3f1c2b2e-9d3a-5b1e-8f0e-000000000001 + Green Button Usage Feed + 2026-01-01T00:00:00Z + + + urn:uuid:07171718-b165-522a-b413-11d9eeb1ca43 + + + UsagePoint + + + + 0 + + 1 + + + 2026-01-01T00:00:00Z + 2026-01-01T00:00:00Z + + + """; + + private static String marshal(Object root, Class... types) throws Exception { + Marshaller marshaller = JAXBContext.newInstance(types).createMarshaller(); + marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); + StringWriter writer = new StringWriter(); + marshaller.marshal(root, writer); + return writer.toString(); + } + + private static UsageAtomEntryDto usagePointEntry() { + UsagePointDto usagePoint = new UsagePointDto(); + usagePoint.setStatus((short) 1); + return new UsageAtomEntryDto(UP_ID, "Sample residential electric service", TIME, TIME, + List.of(new LinkDto("self", BASE + "/UsagePoint/1", "espi-entry/UsagePoint"), + new LinkDto("up", BASE + "/UsagePoint", "espi-feed/UsagePoint")), + usagePoint); + } + + @Test + @DisplayName("a usage entry writes its resource inside atom:content, and the feed validates against the schemas") + void usage_entry_wraps_the_resource_in_content() throws Exception { + AtomFeedDto feed = new AtomFeedDto("urn:uuid:f69c8759-2ef5-5f18-876c-c19f67a26d67", "UsagePoint Feed", + TIME, TIME, List.of(new LinkDto("self", BASE + "/UsagePoint", "espi-feed/UsagePoint")), + List.of(usagePointEntry())); + + String xml = marshal(feed, AtomFeedDto.class, UsageAtomEntryDto.class); + + assertThat(xml).containsPattern("(?s)<(\\w+:)?content>\\s*<(\\w+:)?UsagePoint>.*\\s*"); + EspiXmlSchemaValidator.Result result = EspiXmlSchemaValidator.instance().validate(xml); + assertThat(result.valid()).as(String.join("\n", EspiXmlSchemaValidator.describe(result)) + "\n" + xml).isTrue(); + } + + @Test + @DisplayName("a customer entry writes its resource inside atom:content too") + void customer_entry_wraps_the_resource_in_content() throws Exception { + CustomerAtomEntryDto entry = new CustomerAtomEntryDto("urn:uuid:a3a0d3c4-7c8e-5a8b-9a39-000000000002", + "Customer", TIME, TIME, + List.of(new LinkDto("self", BASE + "/RetailCustomer/1/Customer/2", "espi-entry/Customer"), + new LinkDto("up", BASE + "/RetailCustomer/1/Customer", "espi-feed/Customer")), + new CustomerDto()); + + assertThat(marshal(entry, CustomerAtomEntryDto.class)) + .containsPattern("(?s)<(\\w+:)?content>\\s*<(\\w+:)?Customer\\s*/?>"); + } + + @Test + @DisplayName("a certified-style feed validates, and its entry's resource is read from atom:content") + void certified_style_feed_is_read() throws Exception { + assertThat(EspiXmlSchemaValidator.instance().validate(CERTIFIED_STYLE_FEED).valid()).isTrue(); + + AtomFeedDto feed = (AtomFeedDto) JAXBContext.newInstance(AtomFeedDto.class, UsageAtomEntryDto.class) + .createUnmarshaller().unmarshal(new StringReader(CERTIFIED_STYLE_FEED)); + + assertThat(feed.getEntries()).singleElement() + .satisfies(entry -> assertThat(entry.getResource()) + .isSameAs(entry.getContent()) + .isInstanceOfSatisfying(UsagePointDto.class, + usagePoint -> assertThat(usagePoint.getStatus()).isEqualTo((short) 1))); + } + + @Test + @DisplayName("a round trip keeps the resource: getResource() is no longer null after unmarshalling") + void round_trip_keeps_the_resource() throws Exception { + JAXBContext context = JAXBContext.newInstance(UsageAtomEntryDto.class); + String xml = marshal(usagePointEntry(), UsageAtomEntryDto.class); + + UsageAtomEntryDto parsed = (UsageAtomEntryDto) context.createUnmarshaller().unmarshal(new StringReader(xml)); + + assertThat(parsed.getResource()).isInstanceOfSatisfying(UsagePointDto.class, + usagePoint -> assertThat(usagePoint.getStatus()).isEqualTo((short) 1)); + } + + @Test + @DisplayName("a resource directly in the entry (no atom:content) is not read as the entry's resource") + void bare_resource_is_not_the_content() throws Exception { + String bare = CERTIFIED_STYLE_FEED.replace("", "").replace("", ""); + + AtomFeedDto feed = (AtomFeedDto) JAXBContext.newInstance(AtomFeedDto.class, UsageAtomEntryDto.class) + .createUnmarshaller().unmarshal(new StringReader(bare)); + + assertThat(feed.getEntries()).singleElement().satisfies(entry -> assertThat(entry.getResource()).isNull()); + } + + @Test + @DisplayName("schema errors are all reported with their line: an enum constant name for ServiceCategory (#278)") + void schema_errors_are_reported_with_lines() { + String invalid = CERTIFIED_STYLE_FEED.replace(""" + + 0 + """, "ELECTRICITY"); + + EspiXmlSchemaValidator.Result result = EspiXmlSchemaValidator.instance().validate(invalid); + + assertThat(result.valid()).isFalse(); + assertThat(result.issues()) + .isNotEmpty() + .allSatisfy(issue -> assertThat(issue.line()).isEqualTo(14)) + .anySatisfy(issue -> assertThat(issue.message()).contains("ServiceCategory")); + } + + @Test + @DisplayName("an empty feed is valid ESPI") + void empty_feed_is_valid() { + String empty = CERTIFIED_STYLE_FEED.replaceAll("(?s).*", ""); + + assertThat(EspiXmlSchemaValidator.instance().validate(empty).valid()).isTrue(); + } + + @Test + @DisplayName("not well-formed XML is a fatal issue") + void malformed_xml_is_fatal() { + assertThat(EspiXmlSchemaValidator.instance().validate("").issues()) + .singleElement() + .satisfies(issue -> assertThat(issue.severity()).isEqualTo(EspiXmlSchemaValidator.Severity.FATAL)); + } + + @Test + @DisplayName("a document's external DTD is never fetched") + void external_dtd_is_not_followed() { + String withDtd = """ + + + + """; + + assertThat(EspiXmlSchemaValidator.instance().validate(withDtd).issues()) + .anySatisfy(issue -> assertThat(issue.message()).containsIgnoringCase("accessExternalDTD")); + } +} diff --git a/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java b/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java index e1649351..417f90e5 100755 --- a/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java +++ b/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java @@ -32,6 +32,7 @@ import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Repository; import org.springframework.web.reactive.function.client.WebClient; +import org.greenbuttonalliance.espi.common.dto.atom.AtomEntryDto; import org.greenbuttonalliance.espi.common.dto.atom.AtomFeedDto; import org.greenbuttonalliance.espi.common.dto.atom.UsageAtomEntryDto; import org.greenbuttonalliance.espi.common.dto.usage.UsagePointDto; @@ -42,6 +43,7 @@ import jakarta.xml.bind.Unmarshaller; import java.io.ByteArrayInputStream; import java.io.StringReader; +import java.util.ArrayList; import java.util.List; import java.util.UUID; import java.util.stream.Collectors; @@ -120,15 +122,17 @@ public List findAllByRetailCustomerId(Long retailCustomerId) Unmarshaller unmarshaller = jaxbContext.createUnmarshaller(); AtomFeedDto feedDto = (AtomFeedDto) unmarshaller.unmarshal(new StringReader(xmlResponse)); - // Use openespi-common mappers for transformation - List usagePoints = feedDto.getEntries().stream() - .map(entry -> { - UsagePointDto dto = (UsagePointDto) entry.getResource(); - return usagePointMapper.toEntity(dto); - }) + // An empty feed has no entries at all (#197). Each entry's resource is the child of its + // atom:content; a feed may also carry other resources (MeterReading, ReadingType, ...), which are + // not UsagePoints. + if (feedDto == null || feedDto.getEntries() == null) { + return new ArrayList<>(); + } + return feedDto.getEntries().stream() + .map(AtomEntryDto::getResource) + .filter(UsagePointDto.class::isInstance) + .map(dto -> usagePointMapper.toEntity((UsagePointDto) dto)) .collect(Collectors.toList()); - - return usagePoints; } @Override diff --git a/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/integration/repository/UsagePointRESTRepositoryIntegrationTest.java b/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/integration/repository/UsagePointRESTRepositoryIntegrationTest.java deleted file mode 100644 index 48b5e0dd..00000000 --- a/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/integration/repository/UsagePointRESTRepositoryIntegrationTest.java +++ /dev/null @@ -1,138 +0,0 @@ -/* - * - * Copyright (c) 2025-2026 Green Button Alliance, Inc. - * - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -package org.greenbuttonalliance.espi.thirdparty.integration.repository; - - -import org.greenbuttonalliance.espi.common.domain.usage.AuthorizationEntity; -import org.greenbuttonalliance.espi.common.domain.usage.UsagePointEntity; -import org.greenbuttonalliance.espi.common.service.AuthorizationService; -import org.greenbuttonalliance.espi.thirdparty.repository.impl.UsagePointRESTRepositoryImpl; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Disabled; -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.test.context.bean.override.mockito.MockitoBean; -import org.springframework.web.reactive.function.client.WebClient; -import org.testcontainers.containers.MySQLContainer; -import org.testcontainers.junit.jupiter.Container; -import reactor.core.publisher.Mono; - -import java.util.Collections; -import java.util.List; -import java.util.UUID; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.Mockito.when; - -//@SpringBootTest -//@Testcontainers -//@TestPropertySource(properties = { -// "spring.profiles.active=testcontainers-mysql", -// "spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver" -//}) -public class UsagePointRESTRepositoryIntegrationTest { - - @Container - static MySQLContainer mysql = new MySQLContainer<>("mysql:9.5.0") - .withDatabaseName("testdb") - .withUsername("test") - .withPassword("test"); - - @Autowired - private UsagePointRESTRepositoryImpl usagePointRESTRepository; - - @MockitoBean - private AuthorizationService authorizationService; - - @MockitoBean - private WebClient webClient; - - @MockitoBean - private WebClient.RequestHeadersUriSpec requestHeadersUriSpec; - - @MockitoBean - private WebClient.RequestHeadersSpec requestHeadersSpec; - - @MockitoBean - private WebClient.ResponseSpec responseSpec; - - private AuthorizationEntity mockAuthorization; - - @BeforeEach - public void setUp() { - mockAuthorization = new AuthorizationEntity(); - mockAuthorization.setResourceURI("http://localhost:8080/DataCustodian/espi/1_1/resource/Batch/RetailCustomer/1/UsagePoint"); - mockAuthorization.setAccessToken("test-access-token"); - - when(authorizationService.findAllByRetailCustomerId(any(Long.class))) - .thenReturn(Collections.singletonList(mockAuthorization)); - } - - @Disabled - @Test - public void testFindAllByRetailCustomerId_WithValidResponse() throws Exception { - // Mock XML response from data custodian - String xmlResponse = """ - - - urn:uuid:test-feed - Green Button Usage Feed - 2025-01-01T00:00:00Z - - urn:uuid:test-entry - Usage Point - - - - 0 - - - - - - """; - - // Mock WebClient behavior - when(webClient.get()).thenReturn(requestHeadersUriSpec); - when(requestHeadersUriSpec.uri(mockAuthorization.getResourceURI())).thenReturn(requestHeadersSpec); - when(requestHeadersSpec.headers(org.mockito.ArgumentMatchers.any())).thenReturn(requestHeadersSpec); - when(requestHeadersSpec.retrieve()).thenReturn(responseSpec); - when(responseSpec.bodyToMono(String.class)).thenReturn(Mono.just(xmlResponse)); - - // Execute test - List result = usagePointRESTRepository.findAllByRetailCustomerId(1000001L); - - // Verify results - assertNotNull(result); - assertTrue(result.isEmpty()); // Empty because our simple XML doesn't map to proper UsagePoint structure - } - - @Disabled - @Test - public void testFindByHashedId_WithExistingUsagePoint() throws Exception { - // This test would require a more complete XML structure to work properly - // For now, we'll test the method exists and handles null gracefully - UsagePointEntity result = usagePointRESTRepository.findByHashedId(1000004L, "test-hashed-id"); - - // Should return null when no matching usage points found - assertNull(result); - } -} \ No newline at end of file diff --git a/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java b/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java new file mode 100644 index 00000000..259c1552 --- /dev/null +++ b/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java @@ -0,0 +1,164 @@ +/* + * + * Copyright (c) 2025-2026 Green Button Alliance, Inc. + * + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ + +package org.greenbuttonalliance.espi.thirdparty.repository.impl; + +import org.greenbuttonalliance.espi.common.domain.usage.AuthorizationEntity; +import org.greenbuttonalliance.espi.common.domain.usage.UsagePointEntity; +import org.greenbuttonalliance.espi.common.dto.usage.UsagePointDto; +import org.greenbuttonalliance.espi.common.mapper.usage.UsagePointMapper; +import org.greenbuttonalliance.espi.common.service.AuthorizationService; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.springframework.http.HttpHeaders; +import org.springframework.http.HttpStatus; +import org.springframework.http.MediaType; +import org.springframework.web.reactive.function.client.ClientResponse; +import org.springframework.web.reactive.function.client.WebClient; +import reactor.core.publisher.Mono; + +import java.util.List; +import java.util.UUID; +import java.util.concurrent.atomic.AtomicReference; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +/** + * The Third Party reads a Data Custodian's UsagePoint feed (#197): each resource inside atom:content, an + * empty feed, and entries that are not UsagePoints. Replaces the two {@code @Disabled} tests of the former + * {@code UsagePointRESTRepositoryIntegrationTest}; the WebClient is real, with a stubbed exchange. + */ +class UsagePointRESTRepositoryImplTests { + + private static final String RESOURCE_URI = "https://dc.example/DataCustodian/espi/1_1/resource/Batch/Subscription/1"; + private static final UUID USAGE_POINT_ID = UUID.fromString("07171718-b165-522a-b413-11d9eeb1ca43"); + + private static final String FEED = """ + + + urn:uuid:3f1c2b2e-9d3a-5b1e-8f0e-000000000001 + Green Button Usage Feed + 2026-01-01T00:00:00Z + + + urn:uuid:07171718-b165-522a-b413-11d9eeb1ca43 + + + UsagePoint + + + 1 + + + 2026-01-01T00:00:00Z + 2026-01-01T00:00:00Z + + + urn:uuid:41a6bd77-3d03-59c4-9ab2-65f2cb09b1ed + + + MeterReading + + + + 2026-01-01T00:00:00Z + 2026-01-01T00:00:00Z + + + """; + + private static final String EMPTY_FEED = """ + + + urn:uuid:3f1c2b2e-9d3a-5b1e-8f0e-000000000001 + Green Button Usage Feed + 2026-01-01T00:00:00Z + + """; + + private final AuthorizationService authorizationService = mock(AuthorizationService.class); + private final UsagePointMapper usagePointMapper = mock(UsagePointMapper.class); + private final UsagePointRESTRepositoryImpl repository = new UsagePointRESTRepositoryImpl(); + private final AtomicReference requested = new AtomicReference<>(); + private final AtomicReference authorizationHeader = new AtomicReference<>(); + + @BeforeEach + void setUp() { + AuthorizationEntity authorization = new AuthorizationEntity(); + authorization.setResourceURI(RESOURCE_URI); + authorization.setAccessToken("customer-token"); + when(authorizationService.findAllByRetailCustomerId(any(Long.class))).thenReturn(List.of(authorization)); + when(usagePointMapper.toEntity(any(UsagePointDto.class))).thenAnswer(invocation -> { + UsagePointEntity entity = new UsagePointEntity(); + entity.setId(USAGE_POINT_ID); + entity.setStatus(((UsagePointDto) invocation.getArgument(0)).getStatus()); + return entity; + }); + repository.setAuthorizationService(authorizationService); + repository.setUsagePointMapper(usagePointMapper); + } + + private void dataCustodianAnswers(String body) { + repository.setWebClient(WebClient.builder().exchangeFunction(request -> { + requested.set(request.url().toString()); + authorizationHeader.set(request.headers().getFirst(HttpHeaders.AUTHORIZATION)); + return Mono.just(ClientResponse.create(HttpStatus.OK) + .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_ATOM_XML_VALUE) + .body(body) + .build()); + }).build()); + } + + @Test + @DisplayName("reads the UsagePoint from atom:content, with the customer's token, and skips other resources") + void reads_usage_points_from_content() throws Exception { + dataCustodianAnswers(FEED); + + List usagePoints = repository.findAllByRetailCustomerId(1L); + + assertThat(requested.get()).isEqualTo(RESOURCE_URI); + assertThat(authorizationHeader.get()).isEqualTo("Bearer customer-token"); + assertThat(usagePoints).singleElement() + .satisfies(usagePoint -> assertThat(usagePoint.getStatus()).isEqualTo((short) 1)); + } + + @Test + @DisplayName("an empty feed gives no UsagePoints, not a NullPointerException") + void empty_feed() throws Exception { + dataCustodianAnswers(EMPTY_FEED); + + assertThat(repository.findAllByRetailCustomerId(1L)).isEmpty(); + } + + @Test + @DisplayName("finds a UsagePoint of the feed by its id, and answers null for an unknown id") + void find_by_hashed_id() throws Exception { + dataCustodianAnswers(FEED); + + assertThat(repository.findByHashedId(1L, USAGE_POINT_ID.toString())) + .isNotNull() + .extracting(UsagePointEntity::getId) + .isEqualTo(USAGE_POINT_ID); + assertThat(repository.findByHashedId(1L, UUID.randomUUID().toString())).isNull(); + } +} From 96cd36c37a83afe249d8b55e8898c3692317bd6e Mon Sep 17 00:00:00 2001 From: "Donald F. Coffin" Date: Wed, 18 Nov 2026 00:05:40 -0500 Subject: [PATCH 2/2] fix: address the #280 review of the atom:content and validator changes (#197) - Third Party: each UsagePoint takes its id from the entry's atom:id (the mapper copies only espi.xsd fields), so findByHashedId no longer fails on a null id; entries that are not UsagePoints are counted in a warning (never logged as content). The test's mapper no longer sets the id. - EspiXmlSchemaValidator: validate(String) reads characters, so a declared encoding cannot garble the text; a SAXParseException thrown without the error handler is recorded as FATAL, never reported as valid. - AtomFeedDto: the unused, never-written feed published field is removed (RFC 4287 4.1.1, atom.xsd), with its setters and constructor arguments. - Tests: AssertJ assertions on one object chained (CLAUDE.md); new tests for the entry id, a declared ISO-8859-1 encoding, and a non-uuid id. Co-Authored-By: Claude Opus 5.5 --- .../espi/common/dto/atom/AtomFeedDto.java | 17 +++--- .../common/dto/usage/SubscriptionDto.java | 3 +- .../common/service/BaseExportService.java | 1 - .../service/impl/DtoExportServiceFacade.java | 3 +- .../service/impl/DtoExportServiceImpl.java | 3 +- .../common/xml/EspiXmlSchemaValidator.java | 20 +++++-- .../dto/customer/CustomerAccountDtoTest.java | 6 +- .../customer/CustomerAgreementDtoTest.java | 8 +-- .../common/dto/customer/CustomerDtoTest.java | 10 ++-- .../common/dto/customer/EndDeviceDtoTest.java | 10 ++-- .../common/dto/customer/MeterDtoTest.java | 12 ++-- .../dto/customer/ServiceLocationDtoTest.java | 12 ++-- .../dto/customer/ServiceSupplierDtoTest.java | 16 +++--- .../common/dto/usage/SubscriptionDtoTest.java | 13 +++-- .../common/service/AtomFeedExportTest.java | 8 +-- .../AtomContentAndSchemaValidationTest.java | 22 +++++++- .../impl/UsagePointRESTRepositoryImpl.java | 55 ++++++++++++++++--- .../UsagePointRESTRepositoryImplTests.java | 24 +++++++- 18 files changed, 167 insertions(+), 76 deletions(-) diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java index 6ac75a06..272548d9 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/atom/AtomFeedDto.java @@ -19,7 +19,12 @@ package org.greenbuttonalliance.espi.common.dto.atom; -import jakarta.xml.bind.annotation.*; +import jakarta.xml.bind.annotation.XmlAccessType; +import jakarta.xml.bind.annotation.XmlAccessorType; +import jakarta.xml.bind.annotation.XmlElement; +import jakarta.xml.bind.annotation.XmlElementRef; +import jakarta.xml.bind.annotation.XmlRootElement; +import jakarta.xml.bind.annotation.XmlType; import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter; import lombok.AllArgsConstructor; import org.greenbuttonalliance.espi.common.utils.OffsetDateTimeAdapter; @@ -62,12 +67,8 @@ public class AtomFeedDto { @XmlElement(name = "title") private String title; - /** - * Not written: an Atom feed has no {@code published} element (RFC 4287 4.1.1; atom.xsd's feedType - * rejects it, and certified ESPI feeds have none). Only entries carry {@code published} (#197). - */ - @XmlTransient - private OffsetDateTime published; + // No published field: an Atom feed has no published element (RFC 4287 4.1.1; atom.xsd's feedType rejects + // it, and certified ESPI feeds have none). Only entries carry published (#197). // Without the adapter JAXB cannot map OffsetDateTime and writes an empty element (#247). @XmlElement(name = "updated") @@ -89,7 +90,7 @@ public class AtomFeedDto { * Constructor for basic feed data. */ public AtomFeedDto(String id, String title) { - this(id, title, OffsetDateTime.now(), OffsetDateTime.now(), null, null); + this(id, title, OffsetDateTime.now(), null, null); } /** diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDto.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDto.java index 82f80985..67a24434 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDto.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDto.java @@ -235,7 +235,8 @@ public AtomFeedDto toAtomFeed() { String id = "urn:uuid:" + subscriptionId; String title = schemaType == SchemaType.CUSTOMER ? CUSTOMER_FEED_TITLE : ENERGY_FEED_TITLE; - return new AtomFeedDto(id, title, published, updated, links, entries); + // A feed has no published of its own (RFC 4287 4.1.1, atom.xsd; #197). + return new AtomFeedDto(id, title, updated, links, entries); } // Getters for testing diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java index f2a2dc70..c0d4d684 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/BaseExportService.java @@ -280,7 +280,6 @@ public AtomFeedDto buildFeed(String feedHref, Strin AtomFeedDto feed = new AtomFeedDto(); feed.setId("urn:uuid:" + ID_GENERATOR.generateEspiId(feedHref)); feed.setTitle(title); - feed.setPublished(now); feed.setUpdated(now); // atom.xsd requires a type on every link: an empty feed takes the resource from its href (#197). feed.setLinks(List.of(new LinkDto("self", feedHref, diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceFacade.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceFacade.java index 74ec2aec..c0cd0b62 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceFacade.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceFacade.java @@ -176,8 +176,7 @@ public AtomFeedDto createAtomFeed(String title, List entries) { return new AtomFeedDto( UUID.randomUUID().toString(), // id title, // title - now, // published - now, // updated + now, // updated (a feed has no published) null, // links entries // entries ); diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceImpl.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceImpl.java index 5a08e369..04c57dbd 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceImpl.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/service/impl/DtoExportServiceImpl.java @@ -310,8 +310,7 @@ public AtomFeedDto createAtomFeed(String title, List entries) { return new AtomFeedDto( UUID.randomUUID().toString(), // id title, // title - now, // published - now, // updated + now, // updated (a feed has no published) null, // links entries // entries ); diff --git a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java index 8cf281c9..07fbd8e4 100644 --- a/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java +++ b/openespi-common/src/main/java/org/greenbuttonalliance/espi/common/xml/EspiXmlSchemaValidator.java @@ -35,8 +35,8 @@ import java.io.IOException; import java.io.InputStream; import java.io.Reader; +import java.io.StringReader; import java.net.URL; -import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.Collections; import java.util.List; @@ -111,21 +111,33 @@ public static EspiXmlSchemaValidator instance() { return Holder.INSTANCE; } + /** + * Validates already-decoded text. It is read as characters, so an {@code encoding} in the XML + * declaration is ignored (the text has no bytes to decode). + */ public Result validate(String xml) { - return validate(xml.getBytes(StandardCharsets.UTF_8)); + return validate(new StreamSource(new StringReader(xml))); } /** Validates the raw bytes as received, so the XML declaration's encoding is honoured. */ public Result validate(byte[] xml) { + return validate(new StreamSource(new ByteArrayInputStream(xml))); + } + + private Result validate(StreamSource source) { List issues = new ArrayList<>(); Validator validator = schema.newValidator(); try { validator.setProperty(XMLConstants.ACCESS_EXTERNAL_DTD, ""); validator.setProperty(XMLConstants.ACCESS_EXTERNAL_SCHEMA, ""); validator.setErrorHandler(new CollectingErrorHandler(issues)); - validator.validate(new StreamSource(new ByteArrayInputStream(xml))); + validator.validate(source); } catch (SAXParseException ex) { - // Already recorded by the error handler (a fatal error stops the parse). + // Normally already recorded by fatalError; record it if the parser threw it without calling the + // handler, so a document that never finished parsing is never reported as valid. + if (issues.stream().noneMatch(issue -> issue.severity() == Severity.FATAL)) { + issues.add(new Issue(Severity.FATAL, ex.getLineNumber(), ex.getColumnNumber(), ex.getMessage())); + } } catch (SAXException | IOException ex) { issues.add(new Issue(Severity.FATAL, -1, -1, ex.getMessage())); } diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAccountDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAccountDtoTest.java index 5723f6dd..b537480f 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAccountDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAccountDtoTest.java @@ -66,7 +66,7 @@ void shouldExportCustomerAccountWithCompleteDocumentFields() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Customer Account Feed", now, now, null, + "urn:uuid:feed-id", "Customer Account Feed", now, null, List.of(entry) ); @@ -131,7 +131,7 @@ void shouldVerifyCustomerAccountFieldOrder() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -186,7 +186,7 @@ void shouldUseCorrectCustomerNamespace() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAgreementDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAgreementDtoTest.java index 0fd87de6..3275893e 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAgreementDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerAgreementDtoTest.java @@ -67,7 +67,7 @@ void shouldExportCustomerAgreementWithCompleteDocumentAndAgreementFields() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Customer Agreement Feed", now, now, null, + "urn:uuid:feed-id", "Customer Agreement Feed", now, null, List.of(entry) ); @@ -143,7 +143,7 @@ void shouldVerifyCustomerAgreementFieldOrder() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -206,7 +206,7 @@ void shouldUseCorrectCustomerNamespace() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -264,7 +264,7 @@ void shouldExportElectronicAddressWithAllFields() { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerDtoTest.java index 0839d870..f2f1d339 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/CustomerDtoTest.java @@ -72,7 +72,7 @@ void shouldExportCustomerWithRealisticData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Customer Feed", now, now, null, + "urn:uuid:feed-id", "Customer Feed", now, null, List.of(entry) ); @@ -124,7 +124,7 @@ void shouldVerifyCustomerFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -169,7 +169,7 @@ void shouldVerifyOrganisationFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -218,7 +218,7 @@ void shouldExportCustomerWithMinimalData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -246,7 +246,7 @@ void shouldUseCorrectCustomerNamespace() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/EndDeviceDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/EndDeviceDtoTest.java index 1f667939..92864d97 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/EndDeviceDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/EndDeviceDtoTest.java @@ -72,7 +72,7 @@ void shouldExportEndDeviceWithRealisticData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "EndDevice Feed", now, now, null, + "urn:uuid:feed-id", "EndDevice Feed", now, null, List.of(entry) ); @@ -128,7 +128,7 @@ void shouldVerifyEndDeviceFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -196,7 +196,7 @@ void shouldExportMinimalEndDevice() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Minimal Feed", now, now, null, + "urn:uuid:feed-id", "Minimal Feed", now, null, List.of(entry) ); @@ -235,7 +235,7 @@ void shouldExportEndDeviceWithLifecycleDates() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Lifecycle Feed", now, now, null, + "urn:uuid:feed-id", "Lifecycle Feed", now, null, List.of(entry) ); @@ -277,7 +277,7 @@ void shouldExportEndDeviceWithAcceptanceTest() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "AcceptanceTest Feed", now, now, null, + "urn:uuid:feed-id", "AcceptanceTest Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/MeterDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/MeterDtoTest.java index 41960b50..354b3493 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/MeterDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/MeterDtoTest.java @@ -72,7 +72,7 @@ void shouldExportMeterWithRealisticData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Meter Feed", now, now, null, + "urn:uuid:feed-id", "Meter Feed", now, null, List.of(entry) ); @@ -133,7 +133,7 @@ void shouldVerifyMeterFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -208,7 +208,7 @@ void shouldExportMinimalMeter() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Minimal Feed", now, now, null, + "urn:uuid:feed-id", "Minimal Feed", now, null, List.of(entry) ); @@ -248,7 +248,7 @@ void shouldExportMeterWithLifecycleDates() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Lifecycle Feed", now, now, null, + "urn:uuid:feed-id", "Lifecycle Feed", now, null, List.of(entry) ); @@ -290,7 +290,7 @@ void shouldExportMeterWithAcceptanceTest() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "AcceptanceTest Feed", now, now, null, + "urn:uuid:feed-id", "AcceptanceTest Feed", now, null, List.of(entry) ); @@ -331,7 +331,7 @@ void shouldExportMeterWithMeterMultipliers() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "MeterMultipliers Feed", now, now, null, + "urn:uuid:feed-id", "MeterMultipliers Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceLocationDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceLocationDtoTest.java index 6f86eacb..4b854934 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceLocationDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceLocationDtoTest.java @@ -71,7 +71,7 @@ void shouldExportServiceLocationWithRealisticData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "ServiceLocation Feed", now, now, null, + "urn:uuid:feed-id", "ServiceLocation Feed", now, null, List.of(entry) ); @@ -115,7 +115,7 @@ void shouldVerifyServiceLocationFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -177,7 +177,7 @@ void shouldVerifyTelephoneNumber8FieldCompliance() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -213,7 +213,7 @@ void shouldVerifyStatus4FieldCompliance() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -246,7 +246,7 @@ void shouldExportServiceLocationWithMinimalData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -274,7 +274,7 @@ void shouldVerifyUsagePointsCrossStreamReference() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceSupplierDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceSupplierDtoTest.java index 9ff1c78c..898814ee 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceSupplierDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/customer/ServiceSupplierDtoTest.java @@ -71,7 +71,7 @@ void shouldExportServiceSupplierWithRealisticData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "ServiceSupplier Feed", now, now, null, + "urn:uuid:feed-id", "ServiceSupplier Feed", now, null, List.of(entry) ); @@ -116,7 +116,7 @@ void shouldVerifyServiceSupplierFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -152,7 +152,7 @@ void shouldVerifyOrganisationFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -205,7 +205,7 @@ void shouldExportServiceSupplierWithMinimalData() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -234,7 +234,7 @@ void shouldUseCorrectCustomerNamespace() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -266,7 +266,7 @@ void shouldExportServiceSupplierWithTelephoneNumbers() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -300,7 +300,7 @@ void shouldVerifyTelephoneNumberFieldOrder() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); @@ -353,7 +353,7 @@ void shouldExportServiceSupplierWithAllSupplierKindValues() throws IOException { ); AtomFeedDto feed = new AtomFeedDto( - "urn:uuid:feed-id", "Test Feed", now, now, null, + "urn:uuid:feed-id", "Test Feed", now, null, List.of(entry) ); diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDtoTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDtoTest.java index 7b0e8113..177803f4 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDtoTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/dto/usage/SubscriptionDtoTest.java @@ -22,6 +22,7 @@ import org.greenbuttonalliance.espi.common.dto.atom.UsageAtomEntryDto; import org.greenbuttonalliance.espi.common.dto.atom.AtomFeedDto; import org.greenbuttonalliance.espi.common.dto.atom.LinkDto; +import org.assertj.core.api.InstanceOfAssertFactories; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; @@ -185,11 +186,13 @@ void shouldConvertToAtomFeedDtoWithEnergyTitle() { AtomFeedDto feed = dto.toAtomFeed(); - assertThat(feed.getId()).isEqualTo("urn:uuid:" + TEST_SUBSCRIPTION_ID); - assertThat(feed.getTitle()).isEqualTo("Green Button Energy Feed"); - assertThat(feed.getPublished()).isEqualTo(now); - assertThat(feed.getUpdated()).isEqualTo(now); - assertThat(feed.getLinks()).hasSize(1); + // A feed has no published (#197): only id, title, updated and links. + assertThat(feed) + .returns("urn:uuid:" + TEST_SUBSCRIPTION_ID, AtomFeedDto::getId) + .returns("Green Button Energy Feed", AtomFeedDto::getTitle) + .returns(now, AtomFeedDto::getUpdated) + .extracting(AtomFeedDto::getLinks, InstanceOfAssertFactories.LIST) + .hasSize(1); } @Test diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java index 8d10b052..449ac746 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/service/AtomFeedExportTest.java @@ -82,13 +82,13 @@ void usagePointsBecomeAFeed() { .contains("rel=\"up\" href=\"" + FEED + "\" type=\"espi-feed/UsagePoint\"") .contains("2026-01-02T03:04:05Z") .contains("", ""); + .doesNotContain("xsi:type", "", "") + // #197: each resource inside atom:content; a feed has no published of its own (RFC 4287). + .containsPattern("\\s*]*>\\s*[^<]*\\s*[^<]*\\s*"); assertThat(feedId).matches("urn:uuid:[0-9a-f]{8}-[0-9a-f]{4}-5[0-9a-f]{3}-[0-9a-f]{4}-[0-9a-f]{12}"); assertThat(count(xml, "")).isEqualTo(2); - assertThat(xml).containsPattern("\\s*]*>\\s*[^<]*\\s*[^<]*\\s*"); EspiXmlSchemaValidator.Result validation = EspiXmlSchemaValidator.instance().validate(xml); assertThat(validation.valid()).as(String.join("\n", EspiXmlSchemaValidator.describe(validation))).isTrue(); } diff --git a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java index a204cd15..b5d35728 100644 --- a/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java +++ b/openespi-common/src/test/java/org/greenbuttonalliance/espi/common/xml/AtomContentAndSchemaValidationTest.java @@ -27,11 +27,13 @@ import org.greenbuttonalliance.espi.common.dto.atom.UsageAtomEntryDto; import org.greenbuttonalliance.espi.common.dto.customer.CustomerDto; import org.greenbuttonalliance.espi.common.dto.usage.UsagePointDto; +import org.assertj.core.api.InstanceOfAssertFactories; import org.junit.jupiter.api.DisplayName; import org.junit.jupiter.api.Test; import java.io.StringReader; import java.io.StringWriter; +import java.nio.charset.StandardCharsets; import java.time.OffsetDateTime; import java.time.ZoneOffset; import java.util.List; @@ -100,7 +102,7 @@ private static UsageAtomEntryDto usagePointEntry() { @DisplayName("a usage entry writes its resource inside atom:content, and the feed validates against the schemas") void usage_entry_wraps_the_resource_in_content() throws Exception { AtomFeedDto feed = new AtomFeedDto("urn:uuid:f69c8759-2ef5-5f18-876c-c19f67a26d67", "UsagePoint Feed", - TIME, TIME, List.of(new LinkDto("self", BASE + "/UsagePoint", "espi-feed/UsagePoint")), + TIME, List.of(new LinkDto("self", BASE + "/UsagePoint", "espi-feed/UsagePoint")), List.of(usagePointEntry())); String xml = marshal(feed, AtomFeedDto.class, UsageAtomEntryDto.class); @@ -171,13 +173,27 @@ void schema_errors_are_reported_with_lines() { EspiXmlSchemaValidator.Result result = EspiXmlSchemaValidator.instance().validate(invalid); - assertThat(result.valid()).isFalse(); - assertThat(result.issues()) + assertThat(result) + .returns(false, EspiXmlSchemaValidator.Result::valid) + .extracting(EspiXmlSchemaValidator.Result::issues, InstanceOfAssertFactories.list(EspiXmlSchemaValidator.Issue.class)) .isNotEmpty() .allSatisfy(issue -> assertThat(issue.line()).isEqualTo(14)) .anySatisfy(issue -> assertThat(issue.message()).contains("ServiceCategory")); } + @Test + @DisplayName("text is validated as characters, whatever encoding its declaration names; bytes use that encoding") + void declared_encoding() { + String latin1 = CERTIFIED_STYLE_FEED + .replace("encoding=\"UTF-8\"", "encoding=\"ISO-8859-1\"") + .replace("Green Button Usage Feed", "Données d'énergie"); + + assertThat(List.of( + EspiXmlSchemaValidator.instance().validate(latin1), + EspiXmlSchemaValidator.instance().validate(latin1.getBytes(StandardCharsets.ISO_8859_1)))) + .allSatisfy(result -> assertThat(result.issues()).isEmpty()); + } + @Test @DisplayName("an empty feed is valid ESPI") void empty_feed_is_valid() { diff --git a/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java b/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java index 417f90e5..9624c702 100755 --- a/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java +++ b/openespi-thirdparty/src/main/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImpl.java @@ -28,6 +28,8 @@ import org.greenbuttonalliance.espi.common.service.RetailCustomerService; // UsagePointService removed in migration import org.greenbuttonalliance.espi.thirdparty.repository.UsagePointRESTRepository; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Repository; @@ -46,11 +48,13 @@ import java.util.ArrayList; import java.util.List; import java.util.UUID; -import java.util.stream.Collectors; @Repository public class UsagePointRESTRepositoryImpl implements UsagePointRESTRepository { + private static final Logger log = LoggerFactory.getLogger(UsagePointRESTRepositoryImpl.class); + private static final String URN_UUID = "urn:uuid:"; + @Autowired private AuthorizationService authorizationService; @@ -128,11 +132,47 @@ public List findAllByRetailCustomerId(Long retailCustomerId) if (feedDto == null || feedDto.getEntries() == null) { return new ArrayList<>(); } - return feedDto.getEntries().stream() - .map(AtomEntryDto::getResource) - .filter(UsagePointDto.class::isInstance) - .map(dto -> usagePointMapper.toEntity((UsagePointDto) dto)) - .collect(Collectors.toList()); + List usagePoints = new ArrayList<>(); + int skipped = 0; + for (AtomEntryDto entry : feedDto.getEntries()) { + if (entry.getResource() instanceof UsagePointDto dto) { + usagePoints.add(toEntity(entry, dto)); + } else { + skipped++; + } + } + if (skipped > 0) { + // Counts only: a feed may carry customer PII, so its content is never logged. + log.warn("{} of {} feed entries from {} are not UsagePoints (another resource, or no atom:content)", + skipped, feedDto.getEntries().size(), authorization.getResourceURI()); + } + return usagePoints; + } + + /** + * The mapper copies only the espi.xsd UsagePoint fields; the resource's id is the entry's + * {@code atom:id} ({@code urn:uuid:...}), so it is set here. + */ + private UsagePointEntity toEntity(AtomEntryDto entry, UsagePointDto dto) { + UsagePointEntity entity = usagePointMapper.toEntity(dto); + entity.setId(entryUuid(entry.getId())); + return entity; + } + + static UUID entryUuid(String atomId) { + if (atomId == null) { + return null; + } + String value = atomId.strip(); + if (value.regionMatches(true, 0, URN_UUID, 0, URN_UUID.length())) { + value = value.substring(URN_UUID.length()); + } + try { + return UUID.fromString(value); + } catch (IllegalArgumentException ex) { + log.warn("Feed entry id is not a urn:uuid; the UsagePoint has no id"); + return null; + } } @Override @@ -141,7 +181,8 @@ public UsagePointEntity findByHashedId(Long retailCustomerId, List usagePoints = findAllByRetailCustomerId(retailCustomerId); for (UsagePointEntity usagePoint : usagePoints) { - if (usagePoint.getHashedId().equalsIgnoreCase(usagePointHashedId)) { + // An entry whose atom:id is not a urn:uuid gives a UsagePoint with no id. + if (usagePoint.getHashedId() != null && usagePoint.getHashedId().equalsIgnoreCase(usagePointHashedId)) { return usagePoint; } } diff --git a/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java b/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java index 259c1552..0387ecd2 100644 --- a/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java +++ b/openespi-thirdparty/src/test/java/org/greenbuttonalliance/espi/thirdparty/repository/impl/UsagePointRESTRepositoryImplTests.java @@ -108,9 +108,9 @@ void setUp() { authorization.setResourceURI(RESOURCE_URI); authorization.setAccessToken("customer-token"); when(authorizationService.findAllByRetailCustomerId(any(Long.class))).thenReturn(List.of(authorization)); + // Like the real UsagePointMapper: only espi.xsd fields, no id (the id comes from the entry's atom:id). when(usagePointMapper.toEntity(any(UsagePointDto.class))).thenAnswer(invocation -> { UsagePointEntity entity = new UsagePointEntity(); - entity.setId(USAGE_POINT_ID); entity.setStatus(((UsagePointDto) invocation.getArgument(0)).getStatus()); return entity; }); @@ -139,7 +139,27 @@ void reads_usage_points_from_content() throws Exception { assertThat(requested.get()).isEqualTo(RESOURCE_URI); assertThat(authorizationHeader.get()).isEqualTo("Bearer customer-token"); assertThat(usagePoints).singleElement() - .satisfies(usagePoint -> assertThat(usagePoint.getStatus()).isEqualTo((short) 1)); + .extracting(UsagePointEntity::getId, UsagePointEntity::getStatus) + .containsExactly(USAGE_POINT_ID, (short) 1); + } + + @Test + @DisplayName("the entry id gives the UsagePoint id: urn:uuid prefix in any case, or none; anything else is no id") + void entry_uuid() { + assertThat(List.of("urn:uuid:" + USAGE_POINT_ID, " URN:UUID:" + USAGE_POINT_ID + " ", USAGE_POINT_ID.toString())) + .map(UsagePointRESTRepositoryImpl::entryUuid) + .containsOnly(USAGE_POINT_ID); + assertThat(java.util.Arrays.asList(null, "urn:uuid:not-a-uuid", "tag:example,2026:1")) + .map(UsagePointRESTRepositoryImpl::entryUuid) + .containsOnlyNulls(); + } + + @Test + @DisplayName("an entry id that is not a urn:uuid gives a UsagePoint with no id, and findByHashedId answers null") + void entry_without_uuid() throws Exception { + dataCustodianAnswers(FEED.replace("urn:uuid:" + USAGE_POINT_ID, "tag:example,2026:1")); + + assertThat(repository.findByHashedId(1L, USAGE_POINT_ID.toString())).isNull(); } @Test