Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,7 @@ ESPI uses Atom XML feeds for data exchange. Key patterns:
- Entry links include `rel="self"` (resource URL), `rel="up"` (parent collection), `rel="related"` (related resources)
- All IDs are Type 5 UUIDs in format: `urn:uuid:xxxxxxxx-xxxx-5xxx-xxxx-xxxxxxxxxxxx`
- Link type attributes use `espi-entry/{resource}` or `espi-feed/{resource}` format
- Each entry carries its resource inside a bare `<atom:content>` (no `type`); feeds and entries have no `atom:author` (privacy). This is an ESPI profile of RFC 4287: do not "fix" it to match the RFC. See `docs/ESPI_ATOM_PROFILE.md`; `EspiXmlSchemaValidator` (openespi-common `xml/`) validates against atom.xsd + espi.xsd + customer.xsd

### Core ESPI Resources

Expand Down
59 changes: 59 additions & 0 deletions docs/ESPI_ATOM_PROFILE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# ESPI Atom profile

ESPI resources are exchanged as Atom documents, "as described by Atom, IETF RFC 4287" (NAESB REQ.21 v4.1, REQ.21.6.2, p. 167). The HTTP method conventions come from the Atom Publishing Protocol, RFC 5023 (REQ.21.6.2.1, p. 169). RFC 5023 adds only its own `app:` namespace elements; Atom feeds and entries are as RFC 4287 defines them.

ESPI does not use RFC 4287 to the letter. It uses an **ESPI profile** of it, defined in practice by:

- the GBA `atom.xsd` used by the GBA DMD validator, copied unchanged to `openespi-common/src/main/resources/schema/ESPI_4.1/atom.xsd`;
- REQ.21's examples;
- the feeds of CMD-certified Data Custodians.

This page lists the differences so that they are not "fixed" by mistake. **Do not change the code to follow RFC 4287 where this page says ESPI differs.** Doing so would either reveal which utility the data comes from, or make our feeds differ from every certified feed.

## Intentional differences

| # | ESPI (what we write and accept) | RFC 4287 | Why ESPI differs |
|---|---|---|---|
| 1 | No `atom:author` on feeds or entries | 4.1.1 / 4.1.2: a feed or each entry MUST have one or more `atom:author` | **Privacy.** An author would tie the data to a specific utility. espi.xsd, REQ.21's examples and certified feeds have none. |
| 2 | `<atom:content>` has no `type` attribute, and holds the ESPI resource (`espi:UsagePoint`, `cust:Customer`, ...) as its single child | 4.1.3.1: without `type`, content is treated as `type="text"`; 4.1.3.3 rule 1: text content MUST NOT contain child elements. An XML media type (`type="application/xml"`) would be needed. | REQ.21's examples, certified feeds and the DMD validator all use a bare `<content>`. Adding a `type` would make our feeds differ from all of them for no gain. |
| 3 | Every `atom:link` has `rel` and `type`; every entry has at least a `self` and an `up` link | 4.2.7.2 / 4.2.7.3: `rel` and `type` are optional; any number of links | Required by the GBA `atom.xsd`. ESPI navigation depends on `self`, `up` and `related` links, with types such as `espi-entry/UsagePoint` and `espi-feed/UsagePoint`. |

ESPI follows RFC 4287 where `atom.xsd` is looser. The code does not rely on that looseness:

- **No feed-level `atom:published`.** RFC 4287 4.1.1 defines none, and `atom.xsd` rejects it. Only entries have `published` (REQ.21.6.2: `published` = created, `updated` = last changed).
- **Dates carry a time zone** (RFC 4287 3.3). `OffsetDateTimeAdapter` always writes the offset or `Z`.

## What `atom.xsd` does not check

The schema uses `xs:choice maxOccurs="unbounded"` for feeds, entries and persons, so it **cannot** enforce these RFC 4287 rules:

- exactly one `id`, `title` and `updated` per feed and entry, and at most one `published` per entry;
- a time zone on dates (`xs:dateTime` allows a local time);
- entries after the feed's metadata.

`EspiXmlSchemaValidator` reports only what the schemas report, the same as the DMD validator. Checks for the rules above could be added as warnings. Never add checks for differences 1–3.

## Known issues in `atom.xsd`

These are GBA schema matters, not to be changed in our copy, which must stay identical to the DMD validator's:

- **Email pattern:** `atom:email` uses the pattern `\w+@(\w+\.)+\w+`, which rejects valid addresses such as `first.last@example.com` (RFC 4287 3.2.3). ESPI feeds carry no `author`, so this does not affect them.
- **`atom:source`:** an entry's `atom:source` is typed `textType`, while the `sourceType` the schema defines is unused, so a `source` holding feed metadata (RFC 4287 4.2.11) is rejected. ESPI does not use `source`.

## Open question: link `type` for related resources

REQ.21 p. 168 says related links "shall" use `type="application/atom+xml"`, for example `UsagePoint -> MeterReading`. Certified feeds, the DMD validator's samples and our Data Custodian use the ESPI form instead, `type="espi-entry/MeterReading"` or `espi-feed/...`.

`atom.xsd` accepts either, because `type` is any string, and our Third Party does not depend on the attribute. We follow certified practice until the GBA decides whether REQ.21 needs an erratum.

## Where this is implemented

| Concern | Code |
|---|---|
| Feed (no `author`, no feed `published`) | `openespi-common/.../dto/atom/AtomFeedDto.java` |
| Entry (resource inside a bare `atom:content`) | `UsageAtomEntryDto.java`, `CustomerAtomEntryDto.java` |
| Links (`type` on every link, also on an empty feed) | `openespi-common/.../service/BaseExportService.java` (`buildFeed`) |
| Validation against atom.xsd + espi.xsd + customer.xsd | `openespi-common/.../xml/EspiXmlSchemaValidator.java` |
| Tests | `AtomContentAndSchemaValidationTest`, `AtomFeedExportTest` |

History: #197 (the `atom:content` wrapper, the schema validator, the feed `published` and empty-feed link fixes), #247 (feed date adapters, entry types).
Original file line number Diff line number Diff line change
Expand Up @@ -68,39 +68,36 @@ public abstract class AtomEntryDto {
private List<LinkDto> links;

/**
* ESPI resource content - subclasses define specific @XmlElements
* for their domain (usage or customer) to ensure proper namespace declarations.
* Constructor for the Atom metadata; the subclass sets the ESPI resource ({@link #setContent}).
*/
@XmlTransient
protected Object content;

/**
* All-args constructor.
*/
public AtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated,
List<LinkDto> links, Object content) {
protected AtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated,
List<LinkDto> links) {
this.id = id;
this.title = title;
this.published = published;
this.updated = updated;
this.links = links;
this.content = content;
}

/**
* Convenience constructor for testing with auto-generated timestamps.
* Creates an AtomEntryDto with current timestamp and no links.
*
* @param id the entry identifier (urn:uuid:xxx format)
* @param title the entry title
* @param content the resource content
* Constructor with the current time (to the second) as published and updated, and no links.
*/
public AtomEntryDto(String id, String title, Object content) {
LocalDateTime localDateTime = LocalDateTime.now().truncatedTo(java.time.temporal.ChronoUnit.SECONDS);
OffsetDateTime now = localDateTime.atOffset(ZoneOffset.UTC).toZonedDateTime().toOffsetDateTime();
this(id, title, now, now, null, content);
protected AtomEntryDto(String id, String title) {
OffsetDateTime now = LocalDateTime.now().truncatedTo(java.time.temporal.ChronoUnit.SECONDS)
.atOffset(ZoneOffset.UTC);
this(id, title, now, now, null);
}

/**
* The ESPI resource this entry carries ({@code espi:UsagePoint}, {@code cust:Customer}, ...). In the XML
* it is the single child of the entry's {@code atom:content} element, as REQ.21 and certified feeds use
* it (#197). Each subclass maps it for its own schema (usage or customer), so that only that schema's
* namespace is declared.
*/
public abstract Object getContent();

public abstract void setContent(Object content);


/**
* Gets the self link from the entry links.
Expand All @@ -127,11 +124,12 @@ public LinkDto getUpLink() {
}

/**
* Gets the resource content from the entry.
* Gets the ESPI resource carried by the entry: the same as {@link #getContent()}. It used to return a
* base-class field that unmarshalling never set, so it was always null for a parsed entry (#197).
*
* @return resource content or null if not available
* @return the resource, or null if the entry has none
*/
public Object getResource() {
return content; // != null ? content.resource() : null;
return getContent();
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@

package org.greenbuttonalliance.espi.common.dto.atom;

import jakarta.xml.bind.annotation.*;
import jakarta.xml.bind.annotation.XmlAccessType;
import jakarta.xml.bind.annotation.XmlAccessorType;
import jakarta.xml.bind.annotation.XmlElement;
import jakarta.xml.bind.annotation.XmlElementRef;
import jakarta.xml.bind.annotation.XmlRootElement;
import jakarta.xml.bind.annotation.XmlType;
import jakarta.xml.bind.annotation.adapters.XmlJavaTypeAdapter;
import lombok.AllArgsConstructor;
import org.greenbuttonalliance.espi.common.utils.OffsetDateTimeAdapter;
Expand All @@ -39,11 +44,16 @@
* NOTE: @XmlSeeAlso was removed to prevent namespace pollution. Domain-specific
* export services (UsageExportService, CustomerExportService) explicitly register
* needed classes in JAXBContext to ensure only 2 namespaces are declared.
* <p>
* ESPI profile of RFC 4287 (see {@code docs/ESPI_ATOM_PROFILE.md}): a feed has <strong>no
* {@code atom:author}</strong>, on purpose. RFC 4287 4.1.1 requires one, but it would tie the data to a
* specific utility (privacy), and espi.xsd and certified feeds have none. Do not add it. A feed also has
* no {@code atom:published} (RFC 4287 defines none for a feed; atom.xsd rejects it).
*/
@XmlRootElement(name = "feed", namespace = "http://www.w3.org/2005/Atom")
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "AtomFeed", namespace = "http://www.w3.org/2005/Atom", propOrder = {
"id", "title", "published", "updated", "links", "entries"
"id", "title", "updated", "links", "entries"
})
@Getter
@Setter
Expand All @@ -57,11 +67,10 @@ public class AtomFeedDto {
@XmlElement(name = "title")
private String title;

// Without the adapter JAXB cannot map OffsetDateTime and writes an empty element (#247).
@XmlElement(name = "published")
@XmlJavaTypeAdapter(OffsetDateTimeAdapter.class)
private OffsetDateTime published;
// No published field: an Atom feed has no published element (RFC 4287 4.1.1; atom.xsd's feedType rejects
// it, and certified ESPI feeds have none). Only entries carry published (#197).

// Without the adapter JAXB cannot map OffsetDateTime and writes an empty element (#247).
@XmlElement(name = "updated")
@XmlJavaTypeAdapter(OffsetDateTimeAdapter.class)
private OffsetDateTime updated;
Expand All @@ -81,7 +90,7 @@ public class AtomFeedDto {
* Constructor for basic feed data.
*/
public AtomFeedDto(String id, String title) {
this(id, title, OffsetDateTime.now(), OffsetDateTime.now(), null, null);
this(id, title, OffsetDateTime.now(), null, null);
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,41 +32,31 @@
* Specialized entry for customer.xsd resources (Customer, CustomerAccount, etc.)
* that ensures only the cust namespace (http://naesb.org/espi/customer) is declared.
* <p>
* The resource is the single child of the entry's {@code atom:content} element (#197), as in
* {@link UsageAtomEntryDto}: a bare {@code atom:content} with no {@code type} and no {@code atom:author},
* the ESPI profile of RFC 4287 ({@code docs/ESPI_ATOM_PROFILE.md}).
* <p>
* Per NAESB ESPI standard, usage and customer domains are mutually exclusive.
*/
@XmlRootElement(name = "entry", namespace = "http://www.w3.org/2005/Atom")
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "CustomerAtomEntry", namespace = "http://www.w3.org/2005/Atom", propOrder = {
"content"
"atomContent"
})
@NoArgsConstructor
public class CustomerAtomEntryDto extends AtomEntryDto {

/**
* Customer domain resource content - only customer.xsd types.
* JAXB will only declare xmlns:cust namespace since all types use http://naesb.org/espi/customer.
*/
@XmlElements({
@XmlElement(name = "Customer", type = CustomerDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "CustomerAccount", type = CustomerAccountDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "CustomerAgreement", type = CustomerAgreementDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "EndDevice", type = EndDeviceDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "Meter", type = MeterDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ProgramDateIdMappings", type = ProgramDateIdMappingsDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ServiceLocation", type = ServiceLocationDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "Statement", type = StatementDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "StatementRef", type = StatementRefDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ServiceSupplier", type = ServiceSupplierDto.class, namespace = "http://naesb.org/espi/customer")
})
protected Object content;
/** The {@code atom:content} element holding one customer.xsd resource. */
@XmlElement(name = "content", namespace = "http://www.w3.org/2005/Atom")
private Content atomContent;

/**
* All-args constructor.
*/
public CustomerAtomEntryDto(String id, String title, OffsetDateTime published, OffsetDateTime updated,
List<LinkDto> links, Object content) {
super(id, title, published, updated, links, content);
this.content = content;
super(id, title, published, updated, links);
setContent(content);
}

/**
Expand All @@ -77,24 +67,45 @@ public CustomerAtomEntryDto(String id, String title, OffsetDateTime published, O
* @param content the customer resource content
*/
public CustomerAtomEntryDto(String id, String title, Object content) {
super(id, title, content);
this.content = content;
super(id, title);
setContent(content);
}

/**
* Override getContent to return this class's content field (with @XmlElements)
* instead of parent's @XmlTransient content field.
*/
@Override
public Object getContent() {
return this.content;
return atomContent == null ? null : atomContent.resource;
}

/**
* Override setContent to set this class's content field.
*/
@Override
public void setContent(Object content) {
this.content = content;
this.atomContent = content == null ? null : new Content(content);
}

/**
* The {@code atom:content} element of a customer entry. Only customer.xsd types are listed, so JAXB
* declares only the {@code cust} namespace (http://naesb.org/espi/customer).
*/
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "CustomerAtomContent", namespace = "http://www.w3.org/2005/Atom")
@NoArgsConstructor
public static class Content {

@XmlElements({
@XmlElement(name = "Customer", type = CustomerDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "CustomerAccount", type = CustomerAccountDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "CustomerAgreement", type = CustomerAgreementDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "EndDevice", type = EndDeviceDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "Meter", type = MeterDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ProgramDateIdMappings", type = ProgramDateIdMappingsDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ServiceLocation", type = ServiceLocationDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "Statement", type = StatementDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "StatementRef", type = StatementRefDto.class, namespace = "http://naesb.org/espi/customer"),
@XmlElement(name = "ServiceSupplier", type = ServiceSupplierDto.class, namespace = "http://naesb.org/espi/customer")
})
private Object resource;

Content(Object resource) {
this.resource = resource;
}
}
}
Loading
Loading