Skip to content

Implement lossless RVT write and save-as foundation - #188

Open
AndreyStartsev wants to merge 3 commits into
IOExtend:mainfrom
AndreyStartsev:feat/rvt-write-foundation
Open

AndreyStartsev wants to merge 3 commits into
IOExtend:mainfrom
AndreyStartsev:feat/rvt-write-foundation

Conversation

@AndreyStartsev

Copy link
Copy Markdown
Contributor

What changes and why

Implements the lossless .rvt / .rfa write and save-as foundation (RvtWriter, RvtDocument.SaveAs, PagedStreamCodec, PartitionStreamEditor, and
vtio save-as).

Closes #20

Summary

  • Write-capability matrix (RvtWriter.InspectCapabilities): Classifies local project (.rvt, .rte) and family (.rfa, .rft) containers across releases 2020 to 2027 (4-byte element IDs for 2020 to 2023, 8-byte element IDs for 2024 to 2027) and operations (SaveAs, PagedStreamRewrite, PartitionRecordRewrite). Explicitly refuses unsupported releases, saved/schema release mismatches, workshared models, cloud models, damaged page parity, and invalid partition framing or record checksums with stable RvtWriteRefusedException.Kind codes while leaving the source file untouched.
  • Paged stream re-encoding (PagedStreamCodec): Re-encodes single-member paged gzip streams (Formats/* without prefix and Global/* preserving the original 8-byte header prefix), seals pages with cyclic Hamming parity (PageCodec.EncodePadded), and verifies parity and decompression round-trip before returning.
  • Targeted partition record rewriting (PartitionStreamEditor): Rewrites targeted records in Partitions/ (101..103) while copying untouched blocks (including loaded-family keyed blocks and their original SignatureMarker payloads), untouched segment chains, and untouched or unknown-class records byte-for-byte. Updates [size][body][size] framing and channel 102/103 Adler-32 checksums, respects the 131,072-byte (128 KB) segment body bound (mode = 0 batches and 2 -> 3 -> 1 multi-segment chains) with preserved upper flags and opaque dwords, writes an unsigned SignatureMarker on edited blocks, and self-verifies page parity and full PartitionFraming before returning.
  • Public API, CLI & documentation:
    • Adds RvtWriter (InspectCapabilities, SaveAs, Rewrite), RvtDocument.SourcePath, and RvtDocument.SaveAs.
    • Adds
      vtio save-as <source.rvt|source.rfa> [<output.rvt|output.rfa>] [--check] [--report <report.json>] (exit codes , 1, 2, and 5 when refused by the matrix).
    • Adds docs/writing.md and updates docs/public-api.md and docs/toc.yml.

How it was checked

  • dotnet build RvtIO.slnx -c Release has no warnings in src/RvtIO or src/RvtIO.Cli across
    et8.0 and
    et10.0
  • dotnet test passes (10,399 passed, 0 failed on both
    et8.0 and
    et10.0, including synthetic 2020-2027 .rvt/.rfa round-trips, multi-chain/multi-segment >128 KB rewrites, unknown class preservation, refusal codes, CLI exit codes, and RvtWriterSampleTests when RVTIO_TEST_FILES is configured)
  • dotnet pack RvtIO.slnx -c Release --no-build and dotnet docfx docfx.json --warningsAsErrors succeed with warnings and errors
  • Unchanged SaveAs copies and length-changing Partitions/ record rewrites read back cleanly in RvtIO (preserving element identities, relationships, geometry, and external references) and open with Audit = True in corresponding Revit releases without recovery warnings
  • Exports of the same files before and after are identical (read and IFC/JSON export paths are unchanged)
  • No Revit models, customer data or vendor files are added

@mahach666 mahach666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for three independently reproduced defects in the new writer: unavailable sharing/hosting metadata bypasses the capability gate; unchanged SaveAs loses opaque storage metadata; and an incompatible replacement schema can be published successfully even though the writer then rejects its own output.

The merge preview against current main builds and passes the default suites (10,402 tests on .NET 8 and 10,406 on .NET 10; 439 skipped on each). These passing tests do not cover the failures described inline. Unchanged SaveAs stream/identity checks on two real release samples also passed; native Revit Audit validation was not independently performed.

Comment thread src/RvtIO/Writing/RvtWriter.cs Resolved
Comment thread src/RvtIO/Writing/RvtWriter.cs Resolved
Comment thread src/RvtIO/Writing/RvtWriter.cs Resolved
@AndreyStartsev

Copy link
Copy Markdown
Contributor Author

Addressed all three review items in f0e4050 with regression tests:

  1. Unavailable BasicFileInfo metadata (RvtWriter): RvtWriteCapabilities.IsWorkshared and IsCloudModel are now bool? (null when unknown). Missing or unreadable BasicFileInfo, or missing/unrecognised Worksharing, IsSingleUserCloudModel, or saving-release fields, now refuse all write operations with missing-metadata (Worksharing: Enabled is also classified explicitly as true).
  2. Opaque storage metadata preservation (ContainerRewriter / SaveAs): ContainerRewriter now copies and verifies child storage CLSID, StateBits, CreationFileTime, and ModifiedFileTime as well as root HeaderCLSID, CLSID, StateBits, and ModifiedFileTime on read-back before moving the temporary file into place.
  3. Post-replacement Formats/Latest and output eligibility (RvtWriter.Rewrite): Replacement Formats/Latest payloads are checked against SavedRelease, supported release bounds, ElementIdWidth, and existing Partitions/* framing marker class IDs (release-mismatch, missing-framing-markers, incompatible-schema-payload), and edited .tmp containers are re-inspected via beforeCheck prior to publishing so incompatible combinations never produce output.

@mahach666 mahach666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed f0e4050 and its merge preview against current main (cd243be). All three previously reported defects are fixed and independently verified; the corresponding threads are resolved. No additional blocking findings.

Validation:

  • Release solution builds pass on the PR head and merge preview.
  • Default merge-preview suites: 10,406 passed on .NET 8 and 10,410 passed on .NET 10; zero failures, 439 skipped on each.
  • Independent checks confirm refusal of missing/incomplete metadata, preservation of root/child storage metadata, and rejection of incompatible replacement schemas without publishing output.
  • SaveAs copies of two real samples (releases 2022 and 2027) preserve all stream bytes, storage metadata and element identities; source hashes are unchanged.
  • Both private writer sample tests pass on these samples on .NET 10.
  • Package creation and DocFX with warnings-as-errors pass.

Limits: the hosted CI and Docs runs remain action_required and are not counted as passing. Native Revit Audit and the full private fixture corpus were not independently run. No merge performed.

@mahach666 mahach666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes after native acceptance of f0e4050. This review supersedes my earlier approval: SaveAs and Rewrite can report success while breaking a valid relative RVT link when the destination directory changes. This fails the external-reference preservation criterion in #20.

The issue was independently reproduced in Revit 2022, with a same-directory SaveAs as a passing control. Native checks also exercised Audit opening and length-changing record writes in Revit 2020, 2022, 2025 and 2027. Source hashes remained unchanged. The three previously reported defects remain fixed; the new destination-dependent reference failure still blocks acceptance.

Preserve the resolved reference targets, or explicitly refuse the unsupported relocation before publishing output, and add a regression covering both destination cases.

Comment on lines +223 to +227
var containerResult = ContainerRewriter.Rewrite(
sourcePath,
fullOutput,
replacements ?? NoStreamReplacements,
verifyEditedOutput);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve relative reference targets when changing directories

This copies the container to a new location without checking whether relative external references still resolve to the same files. For example, a host in input/host.rvt referencing ../links/linked.rvt resolves that reference differently after saving to output/nested/host.rvt. In a controlled Revit 2022 project, both SaveAs and Rewrite return success, but the previously valid RVT link changes from Loaded to NotFound, and RevitLinkInstance.GetLinkDocument() returns null. The original linked file still exists. Saving into the source directory keeps the link loaded, confirming that relocation causes the failure.

Preserve the resolved targets of supported relative references, or refuse destination changes that cannot preserve them before publishing the file. Add a regression with a valid relative link and both same-directory and different-directory destinations; checking stored reference bytes alone misses this failure.

@mahach666

Copy link
Copy Markdown
Collaborator

@AndreyStartsev could you fix this?

- Add RvtWriter, RvtWriteCapabilities, RvtWriteRequest, RvtWriteResult, RvtWriteRefusedException, and RvtDocument.SaveAs for local 2020-2027 .rvt and .rfa files
- Add PagedStreamCodec for Formats/* and Global/* paged gzip streams and PartitionStreamEditor for targeted Partitions/<n> record rewrites with Adler-32 checksums, 128 KB segment bounds, and untouched block, chain, and opaque payload preservation
- Add rvtio save-as CLI command (--check and --report) and DocFX documentation (docs/writing.md)
- Add synthetic and optional fixture tests across net8.0 and net10.0

Closes IOExtend#20
- Refuse writes with missing-metadata when BasicFileInfo is missing,
  unreadable, or lacks recognised Worksharing, IsSingleUserCloudModel,
  or saving-release fields; expose IsWorkshared and IsCloudModel as
  nullable bool? and classify Worksharing: Enabled as workshared.
- Preserve and verify root and child storage directory-entry metadata
  (CLSID, StateBits, CreationFileTime, ModifiedFileTime, HeaderCLSID)
  in ContainerRewriter across SaveAs and Rewrite.
- Validate replacement Formats/Latest payloads against SavedRelease,
  supported release bounds, ElementIdWidth, and existing partition
  framing marker class IDs, and re-inspect edited temporary containers
  before publishing.
- Add regression tests covering missing/incomplete BasicFileInfo,
  opaque storage metadata preservation and tampering detection, and
  valid-but-incompatible replacement schemas.
@AndreyStartsev
AndreyStartsev force-pushed the feat/rvt-write-foundation branch from f0e4050 to e6c624f Compare November 13, 2026 00:38
@AndreyStartsev

Copy link
Copy Markdown
Contributor Author

Rebased onto main (c0d6157) and addressed the relative external reference check (e6c624f):

  • Why explicit refusal is used over rewriting TransmissionData alone: Revit ignores TransmissionData when isTransmitted="false" and resolves links from ExternalFileReferenceCell / ExternalResourceReferenceCell records inside Partitions/<n>, while setting isTransmitted="true" marks the file as an unfinalized transmitted package and detaches workshared models. Until internal reference cells are decoded for writing, cross-directory writes that would move a relative target now fail cleanly before publishing any output file.
  • RvtWriter validation: ValidateExternalReferenceRelocation parses TransmissionData (LastSavedPath / DesiredPath, excluding Imported CAD and Absolute / Server / Content / Library paths), normalizes path separators across platforms, resolves each relative target against both the source and destination directories, and throws RvtWriteRefusedException("relative-reference-relocation", ...) whenever the resolved target path changes. Cross-directory writes also refuse unreadable TransmissionData (unreadable-transmission-data), PointCloudType partition records (which TransmissionData omits), and RvtLinkSymbol partition records when TransmissionData has been stripped (missing-metadata).
  • Tests and docs: Added regression tests in RvtWriterTests and SaveAsCommandTests with a real relative link target on disk covering same-directory preservation (verifying resolved target paths) and cross-directory refusal across RvtWriter.SaveAs, RvtDocument.SaveAs, RvtWriter.Rewrite, and rvtio save-as (exit code 5), plus updated docs/writing.md and docs/test-fixtures.md.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a lossless RVT write and save-as foundation

2 participants