diff --git a/CHANGELOG.md b/CHANGELOG.md index 87b09a2..f7791ab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ API between minor releases. ### Added +- CLI and diagnostics: `--strict` exits with code 6 (`RvtExitCodes.PartialRead`) instead of 0 when a read finishes partial without exceeding a reader limit (a stream member failed to decode, was rebuilt by structural recovery, or decoded without a matching checksum, or `Global/ElemTable` decoded no row). Nothing is exported when exit code 6 is returned; `limits.strict` records the option and `issues[]` lists the affected streams and reasons (`failed-members`, `recovered-members`, `unverified-members`, `no-rows`) alongside `streams[]`. Default behavior without `--strict` is unchanged. (Closes #54) - Default output, current IFC profile: every beam with a straight drive curve gets `Pset_BeamCommon.Slope`, the angle in degrees of its drive curve to the horizontal (0 for a horizontal beam, never negative), whether or not BaseQuantities are exported and whatever its body; its quantities are unchanged. A beam without a straight drive curve gets no Slope. - Default output, current IFC profile: `Pset_BeamCommon.Span` of a beam without a stored length is the length of its straight drive curve; a beam without one has no Span. - Default output, current IFC profile: a round column gets the volume and surface of the measured node that carries its two exact end faces in place of the open tessellated side. A round column is a vertical cylinder: exactly two exact planar end faces, horizontal, without holes, of at least eight corners on one circle (the same for both faces), each with an exact area within 0.5% of the area of that circle (a regular polygon of 32 corners or fewer is refused), and every other face with its corners on the cylinder between the end faces. The node is taken only when it is the one node that holds nothing but end faces. It is used for `NetVolume` after the sources that come first for a column (the Medium solids, the nominal area of a tube, the computed host volume, the measures of all nodes) and for `CrossSectionArea` (half the surface) of a column without extrusion data. When the two end faces also have the same exact area (a vertical prism) the column gets the extrusion data `Length`, `CrossSectionArea` (the area of its circle), `OuterSurfaceArea` and `GrossVolume`. Other columns are unchanged. Measured on the round columns of one sample (corner distances to a millionth of the radius, or eight units in the last place of a 32-bit coordinate where that is more). diff --git a/README.md b/README.md index 87064df..21ddad7 100644 --- a/README.md +++ b/README.md @@ -91,8 +91,9 @@ Use `--ifc-revit-parameter-groups` for one property set per parameter group. A f | 3 | A stream exceeded a limit of the reader and the read was cut. Nothing is exported unless `--allow-partial` is given. | | 4 | The file is readable but a decoder failed on its content. | | 5 | `downgrade --check` refused the request, for example because the source is newer than release 2026, the profile is not older than the source, the saving release of a file does not match its class table, or a class name occurs twice in a table. One line on standard error. | +| 6 | `--strict` was given and the read was partial without hitting a limit (a member failed to decode, was rebuilt by structural recovery, or decoded without a matching checksum, or the element table decoded no row). Nothing is exported; the report lists the streams and the reason. | -`--report project.report.json` writes a machine-readable report: the fraction of each partition stream and of the element table that decoded, the streams cut by a limit, the elements the export left out with their class and reason, and the exported fraction. `--raise-limits` raises the limits for a very large model. A complete export without these options behaves as before. The report format and the limits are described in [the export report](docs/export-report.md). +`--report project.report.json` writes a machine-readable report: the fraction of each partition stream and of the element table that decoded, the streams cut by a limit, the elements the export left out with their class and reason, and the exported fraction. `--strict` makes partial reads that did not hit a limit fail with exit code 6 instead of exporting with exit code 0. `--raise-limits` raises the limits for a very large model. A complete export without these options behaves as before. The report format and the limits are described in [the export report](docs/export-report.md). ### Plan a downgrade to an older release (experimental) diff --git a/docs/export-report.md b/docs/export-report.md index 4a924ca..b5038c1 100644 --- a/docs/export-report.md +++ b/docs/export-report.md @@ -8,19 +8,21 @@ Without the new options the output (IFC, JSON, standard error text) and the exit | Option | Effect | | --- | --- | | `--report ` | Write the JSON report of the run to this path. Written on success and on failure. | -| `--allow-partial` | Export what was read when a reader limit cut a stream. The exit code is then 0 and the report says `"status": "partial"`. | +| `--allow-partial` | Export what was read when a reader limit cut a stream. The exit code is then 0 and the report says `"status": "partial"`. Cannot be combined with `--strict`. | +| `--strict` | Fail with exit code 6 instead of 0 when the read is partial even without hitting a limit (a member failed to decode, was rebuilt by structural recovery, or decoded without a matching checksum, or the element table decoded no row). Nothing is exported; the report lists the streams and the reason. Cannot be combined with `--allow-partial`. | | `--raise-limits` | Raise the limits listed below for a very large model. | ## Exit codes | Code | Name | Meaning | | --- | --- | --- | -| 0 | success | The export was written. With `--allow-partial` or after members of a stream failed to decode, the report can still say `"status": "partial"`: read it when you need to know. | +| 0 | success | The export was written. With `--allow-partial` or, without `--strict`, after members of a stream failed to decode or were recovered, the report can still say `"status": "partial"`: read it when you need to know, or pass `--strict` to make partial reads fail. | | 1 | usage | The command line is wrong (unchanged from earlier versions). | | 2 | input error | The input cannot be read: the file does not exist or cannot be opened, is empty, is not a compound file, or is cut or damaged so that a stream ends before its declared length. One line on standard error, no stack trace, nothing exported. | | 3 | limit exceeded | A stream exceeded a limit of the reader and the read was cut. Nothing is exported unless `--allow-partial` is given. A stream longer than the platform allows in one array cannot be read at all: that case exits with 3 even with `--allow-partial`. | | 4 | decode error | The compound file is readable but a decoder failed on its content (damaged or unsupported data). One line on standard error; the report holds the exception with its stack trace. | | 5 | refused | `downgrade --check` refused the request (for example a source newer than release 2026, a profile not older than the source, a saving release that does not match the class table, or an ambiguous class name). One line on standard error. | +| 6 | partial read | `--strict` was given and the read finished partial without exceeding a limit (members of a stream failed to decode, were rebuilt by structural recovery, or decoded without a matching checksum, or the element table decoded no row). One line on standard error naming the streams and the reason, nothing exported; the report holds the stream counts and `issues[]`. | A crash that is none of these (a defect in the exporter after the read) still ends the process with the runtime's unhandled exception code. @@ -44,7 +46,7 @@ The report is one JSON object, UTF-8, property names in camelCase. `schema` name "status": "complete", "exitCode": 0, "input": { "file": "project.rvt", "bytes": 9293824 }, - "limits": { "raised": false, "partialAllowed": false, "hit": [] }, + "limits": { "raised": false, "partialAllowed": false, "strict": false, "hit": [] }, "streams": [ { "name": "Partitions/0", "rawBytes": 8187086, "members": 695, "decoded": 695, "failed": 0, "recovered": 0, "unverified": 0, "decodedFraction": 1 } @@ -62,15 +64,15 @@ The report is one JSON object, UTF-8, property names in camelCase. `schema` name | Property | Meaning | | --- | --- | -| `status` | `complete`: no limit cut a stream and every member of every stream decoded and verified. `partial`: a limit cut a stream, a member did not decode, a member decoded without its checksum matching, a member was rebuilt by structural recovery, or the element table decoded no row. `failed`: the file could not be read at all (exit codes 2 and 4, and 3 for a stream over the platform array length). A read cut by a limit that is not accepted (exit code 3) is `partial` and has no `export` property, because nothing was exported. | +| `status` | `complete`: no limit cut a stream and every member of every stream decoded and verified. `partial`: a limit cut a stream, a member did not decode, a member decoded without its checksum matching, a member was rebuilt by structural recovery, or the element table decoded no row. `failed`: the file could not be read at all (exit codes 2 and 4, and 3 for a stream over the platform array length). A read cut by a limit that is not accepted (exit code 3) or a partial read refused by `--strict` (exit code 6) is `partial` and has no `export` property, because nothing was exported. | | `exitCode` | The exit code of the process. | | `input.file`, `input.bytes` | File name (no directory) and size of the input. | | `error` | Only when `status` is `failed`: `kind` (`cannot-open`, `empty-file`, `not-a-compound-file`, `truncated-or-damaged`, `limit-exceeded`, `decode-failure`), `message`, and for `decode-failure` the `detail` with the stack trace. | -| `limits.raised`, `limits.partialAllowed` | Whether `--raise-limits` and `--allow-partial` were given. | +| `limits.raised`, `limits.partialAllowed`, `limits.strict` | Whether `--raise-limits`, `--allow-partial` and `--strict` were given. | | `limits.hit[]` | Every stream cut by a limit: `stream`, `limit` (code from the table above), `value` (the limit in force), `raisable`, `detail`. | | `streams[]` | The partition, Global, Formats and element table streams. `members` are the gzip members the stream is cut into; `decoded` produced output; `failed` produced none and their data is missing (a stream that holds bytes but not one gzip member is one failed member); `recovered` were rescued by a structural relock; `unverified` were kept although their checksum did not match or was never checked, which includes every rescued member. `decodedFraction` is `decoded / members`. | | `elements.inTable`, `elements.decoded` | Rows of the element table, and those with a parsed record. `decodedFraction` is their quotient (1 for an empty table). | -| `issues[]` | Other findings that make a read `partial`, as `stream`, `code`, `detail`. The one code is `no-rows`: the element table decoded no row (a project always has elements, so the table did not decode). | +| `issues[]` | Findings that make a read `partial`, as `stream`, `code`, `detail`: `failed-members` (one or more gzip members of the stream produced no output), `recovered-members` (one or more members were rebuilt by structural recovery), `unverified-members` (one or more members decoded without a matching checksum), and `no-rows` (`Global/ElemTable` decoded no row; a project always has elements, so the table did not decode). | | `elements.undecoded[]` | Rows of the element table without a parsed record, as `class`, `reason`, `count`, `examples`. The class of an element that never decoded is `unknown`. | | `export.format` | `ifc`, `json` (the compact export) or `json-full`. | | `export.considered`, `exported`, `skipped` | Parsed elements the export looked at, wrote, and left out, counted as the exporter counts them (`exported` counts elements, not product entities: for example, a roof element that writes two product entities, an `IfcRoof` and an `IfcSlab` with the same tag, counts once in `exported`, so the exported total can be below the number of product entities). `exportedFraction` is `exported / considered`. | @@ -91,11 +93,12 @@ The compact export writes the rows of the element table: `block-only-record` and ## Scripting ``` -rvtio project.rvt project.ifc --report project.report.json +rvtio project.rvt project.ifc --strict --report project.report.json case $? in - 0) ;; # done; check "status" in the report if a partial read matters + 0) ;; # complete read and export written 2) echo "the file is cut or damaged" ;; 3) echo "limit exceeded: retry with --raise-limits" ;; 4) echo "the content could not be decoded" ;; + 6) echo "the read was partial: see streams and issues in project.report.json" ;; esac ``` diff --git a/docs/public-api.md b/docs/public-api.md index d2e1319..0dfcd0c 100644 --- a/docs/public-api.md +++ b/docs/public-api.md @@ -13,7 +13,7 @@ projects through `InternalsVisibleTo`. | `RvtReaderOptions` | Options of a read, including `DiagnosticOutput`, `ResolveCurrentCopies` (keeps only the current copy of an element that several partition streams hold; default true, set it to false to decode every copy), `RaiseLimits` and `Diagnostics`. `RevitLocalizationDirectory` is `[Obsolete]` and ignored: the texts are built in. | | `RvtReadDiagnostics`, `StreamRecord`, `ReaderLimitHit` | What a read collects when `RvtReaderOptions.Diagnostics` is set: the partition streams read and the streams cut by a limit. | | `ExportReport`, `SkipLog` | The machine-readable report of a run (`ToJson`) and the elements an export left out; see [the export report](export-report.md). | -| `RvtExitCodes`, `RvtInputException`, `RvtLimitException`, `RvtContainerCheck` | The exit codes 0 to 4 of the command line tool (code 5 is returned only by `downgrade --check`), the exceptions `RvtFileReader.Read` throws for a file it cannot read, and the check that raises them. | +| `RvtExitCodes`, `RvtInputException`, `RvtLimitException`, `RvtContainerCheck` | The exit codes 0 to 4 and 6 of the command line tool (code 5 is returned only by `downgrade --check`), the exceptions `RvtFileReader.Read` throws for a file it cannot read, and the check that raises them. | | `FilteredElementCollector` | Query over the elements of a document (`OfCategory`, `WhereElementIsNotElementType`, `Where`, ...). | ## Model diff --git a/src/RvtIO.Cli/ExportCommand.cs b/src/RvtIO.Cli/ExportCommand.cs index a780626..8ff0b9a 100644 --- a/src/RvtIO.Cli/ExportCommand.cs +++ b/src/RvtIO.Cli/ExportCommand.cs @@ -31,9 +31,16 @@ private static int Export(ExportArguments arguments) }; // --report : the machine-readable report of the run; --allow-partial: export a read that a reader limit cut (exit 0 instead of 3); + // --strict: fail when a read is partial even without hitting a limit (exit 6 instead of 0); // --raise-limits: raise the limits that cut a damaged stream while it is recovered (very large models) bool allowPartial = args.Contains("--allow-partial"); + bool strict = args.Contains("--strict"); bool raiseLimits = args.Contains("--raise-limits"); + if (allowPartial && strict) + { + Console.Error.WriteLine("--allow-partial and --strict exclude each other."); + return RvtExitCodes.Usage; + } string? reportPath = ValueOf(args, "--report"); if (args.Contains("--report") && (reportPath is null or "" || reportPath.StartsWith("--", StringComparison.Ordinal))) { @@ -57,32 +64,40 @@ private static int Export(ExportArguments arguments) try { rvtInfo = reader.Read(rvtPath); } catch (RvtInputException ex) { - return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, ex.Kind, ex.Message, RvtExitCodes.InputError, readDiagnostics, raiseLimits, allowPartial), + return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, ex.Kind, ex.Message, RvtExitCodes.InputError, readDiagnostics, raiseLimits, allowPartial, null, strict), $"error: {ex.Message} Nothing was exported (exit code {RvtExitCodes.InputError})."); } catch (RvtLimitException ex) { readDiagnostics.LimitHits.Add(ex.Hit); - return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, "limit-exceeded", ex.Message, RvtExitCodes.LimitExceeded, readDiagnostics, raiseLimits, allowPartial), + return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, "limit-exceeded", ex.Message, RvtExitCodes.LimitExceeded, readDiagnostics, raiseLimits, allowPartial, null, strict), $"error: {ex.Message}. The limit cannot be raised. Nothing was exported (exit code {RvtExitCodes.LimitExceeded})."); } // The container is readable, so this is a decoder failure on damaged or unsupported content: one line on the console, the stack trace in the report. catch (Exception ex) when (ex is not OutOfMemoryException) { string message = $"The content of '{inputName}' could not be decoded: {ex.GetType().Name}: {ex.Message}"; - return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, "decode-failure", message, RvtExitCodes.DecodeError, readDiagnostics, raiseLimits, allowPartial, ex.ToString()), + return EndRun(reportPath, ExportReport.Failed(inputName, inputBytes, "decode-failure", message, RvtExitCodes.DecodeError, readDiagnostics, raiseLimits, allowPartial, ex.ToString(), strict), $"error: {message}. Nothing was exported (exit code {RvtExitCodes.DecodeError}). The report (--report) holds the stack trace."); } Console.Error.WriteLine($"Timing: read {swRead.Elapsed.TotalSeconds:F1}s"); var report = ExportReport.ForRead(inputName, inputBytes, readDiagnostics, raiseLimits, allowPartial, - reader.LastParsingState?.ElemTable.Keys.ToList() ?? [], rvtInfo); + reader.LastParsingState?.ElemTable.Keys.ToList() ?? [], rvtInfo, strict); foreach (var hit in readDiagnostics.LimitHits) Console.Error.WriteLine($"warning: stream {hit.Stream} was cut by the reader limit {hit.Limit} ({hit.Value.ToString("N0", System.Globalization.CultureInfo.InvariantCulture)}): {hit.Detail}"); - if (report.ExitCode != RvtExitCodes.Success) + if (report.ExitCode == RvtExitCodes.LimitExceeded) return EndRun(reportPath, report, $"error: a stream exceeded a reader limit and the read was cut. Nothing was exported (exit code {report.ExitCode}). " + (raiseLimits ? "The limits are already raised; use --allow-partial to export what was read." : "Use --raise-limits to raise the limits, or --allow-partial to export what was read.")); + if (report.ExitCode == RvtExitCodes.PartialRead) + { + string reasons = report.Issues.Count > 0 + ? string.Join("; ", report.Issues.Select(issue => $"{issue.Stream}: {issue.Detail}")) + : "partial stream decode"; + return EndRun(reportPath, report, + $"error: the read of '{inputName}' was partial ({reasons}). Nothing was exported (exit code {report.ExitCode}). Omit --strict to export what was read."); + } if (outputPath.EndsWith(".ifc", StringComparison.OrdinalIgnoreCase)) { diff --git a/src/RvtIO.Cli/Program.cs b/src/RvtIO.Cli/Program.cs index 34cbe26..5db9d58 100644 --- a/src/RvtIO.Cli/Program.cs +++ b/src/RvtIO.Cli/Program.cs @@ -37,10 +37,12 @@ internal static partial class Program --resolve-current-copies the earlier opt-in for the default, accepted without effect --report write the JSON report of the run (read streams, limits, skipped elements, exported fraction) --allow-partial export what was read when a reader limit cut a stream (exit code 0 instead of 3) + --strict fail when a read is partial even without hitting a limit (exit code 6 instead of 0; nothing is exported) --raise-limits raise the limits that cut a damaged stream while it is recovered (very large models) Exit codes: 0 done; 1 usage error; 2 the input cannot be read (missing, not a Revit file, cut or damaged); - 3 a stream exceeded a reader limit and the read was cut; 4 a decoder failed on the content of a readable file. + 3 a stream exceeded a reader limit and the read was cut; 4 a decoder failed on the content of a readable file; + 6 the read was partial and --strict was given. The report format is described in docs/export-report.md. Report options (read the file and print or write a report instead of exporting): @@ -101,7 +103,7 @@ Developer options (measurement and comparison dumps, not a stable interface): private static readonly HashSet SwitchOptions = [ "--json-full", "--compact-json", "--ifc-revit-parameters", "--ifc-revit-parameter-groups", "--ifc-no-base-quantities", ..RvtIO.Cli.OpeningAxesOption.Names, "--r22-layout", - "--no-r22-layout", ..RvtIO.Cli.CurrentCopiesOption.Names, "--allow-partial", "--raise-limits", "--api-demo", "--dev-solids-only", "--elemtable-diag", "--tf-calibrate", "--name-scan", "--typemap", "--block-map", + "--no-r22-layout", ..RvtIO.Cli.CurrentCopiesOption.Names, "--allow-partial", "--strict", "--raise-limits", "--api-demo", "--dev-solids-only", "--elemtable-diag", "--tf-calibrate", "--name-scan", "--typemap", "--block-map", "--decode-blocks", "--decode-catc", "--wide", "--blocks", "--no-recover", "--no-d64", "--trim", "--dump-partial", ]; diff --git a/src/RvtIO/Diagnostics/ExportReport.cs b/src/RvtIO/Diagnostics/ExportReport.cs index bf5b66d..f12a2ae 100644 --- a/src/RvtIO/Diagnostics/ExportReport.cs +++ b/src/RvtIO/Diagnostics/ExportReport.cs @@ -7,7 +7,8 @@ namespace RvtIO; /// Process exit codes of the command line exporter (documented in the README and in docs/export-report.md). public static class RvtExitCodes { - /// The export finished. With --allow-partial it may have read less than the whole file: the report says so. + /// The export finished. With --allow-partial, or without --strict when a stream lost or recovered a member, + /// it may have read less than the whole file: the report says so. public const int Success = 0; /// The command line is wrong. Unchanged from earlier versions. public const int Usage = 1; @@ -18,11 +19,24 @@ public static class RvtExitCodes public const int LimitExceeded = 3; /// The container is readable but a decoder failed on its content (damaged or unsupported data). The report holds the exception. public const int DecodeError = 4; + /// With --strict, the read finished without exceeding a limit, or --allow-partial accepted a limit, + /// but the read is partial (a member failed to decode, was rebuilt by structural recovery, or decoded without a matching checksum, + /// or the element table decoded no row). Nothing is exported; the report lists the streams and the reason. + public const int PartialRead = 6; /// The exit code of a finished read: when a limit cut a stream and partial output /// was not allowed, else . public static int ForRead(RvtReadDiagnostics diagnostics, bool allowPartial) => - diagnostics.LimitHits.Count > 0 && !allowPartial ? LimitExceeded : Success; + ForRead(diagnostics, allowPartial, strict: false, partialRead: false); + + /// The exit code of a finished read: when a limit cut a stream and partial output + /// was not allowed, else when is true and the read is partial, + /// else . + public static int ForRead(RvtReadDiagnostics diagnostics, bool allowPartial, bool strict, bool partialRead = false) => + diagnostics.LimitHits.Count > 0 && !allowPartial ? LimitExceeded + : strict && (partialRead || diagnostics.LimitHits.Count > 0 + || diagnostics.Streams.Any(s => s.Failed > 0 || s.Recovered > 0 || s.Unverified > 0)) ? PartialRead + : Success; } /// The input cannot be read at all (missing, empty, not a compound file, or a stream ends before its declared length). @@ -128,6 +142,8 @@ public sealed class ReportLimits public bool Raised { get; set; } /// True when the run accepted a read cut by a limit (--allow-partial). public bool PartialAllowed { get; set; } + /// True when the run refused a partial read (--strict). + public bool Strict { get; set; } /// Every stream a limit cut. public List Hit { get; set; } = new(); } @@ -165,7 +181,7 @@ public sealed class ReportIssue /// Storage and name of the stream, for example Global/ElemTable. public string Stream { get; set; } = ""; - /// Stable code: no-rows (the element table decoded no row). + /// Stable code: no-rows, failed-members, recovered-members or unverified-members. public string Code { get; set; } = ""; /// One line for a person. @@ -278,7 +294,12 @@ internal static string StripSourcePaths(string trace) => /// The report of a file that could not be read. public static ExportReport Failed(string file, long bytes, string kind, string message, int exitCode, RvtReadDiagnostics? diagnostics, - bool raised, bool partialAllowed, string? detail = null) + bool raised, bool partialAllowed, string? detail = null) => + Failed(file, bytes, kind, message, exitCode, diagnostics, raised, partialAllowed, detail, strict: false); + + /// The report of a file that could not be read. + public static ExportReport Failed(string file, long bytes, string kind, string message, int exitCode, RvtReadDiagnostics? diagnostics, + bool raised, bool partialAllowed, string? detail, bool strict) { var report = new ExportReport { Status = "failed", ExitCode = exitCode }; report.Input.File = file; @@ -286,19 +307,26 @@ public static ExportReport Failed(string file, long bytes, string kind, string m report.Error = new ReportError { Kind = kind, Message = message, Detail = detail == null ? null : StripSourcePaths(detail) }; report.Limits.Raised = raised; report.Limits.PartialAllowed = partialAllowed; + report.Limits.Strict = strict; if (diagnostics != null) report.Fill(diagnostics); return report; } /// The report of a finished read. are the rows of the element table. public static ExportReport ForRead(string file, long bytes, RvtReadDiagnostics diagnostics, bool raised, bool partialAllowed, - IReadOnlyCollection elementIds, RevitFileInfo info) + IReadOnlyCollection elementIds, RevitFileInfo info) => + ForRead(file, bytes, diagnostics, raised, partialAllowed, elementIds, info, strict: false); + + /// The report of a finished read. are the rows of the element table. + public static ExportReport ForRead(string file, long bytes, RvtReadDiagnostics diagnostics, bool raised, bool partialAllowed, + IReadOnlyCollection elementIds, RevitFileInfo info, bool strict) { - var report = new ExportReport { ExitCode = RvtExitCodes.ForRead(diagnostics, partialAllowed) }; + var report = new ExportReport(); report.Input.File = file; report.Input.Bytes = bytes; report.Limits.Raised = raised; report.Limits.PartialAllowed = partialAllowed; + report.Limits.Strict = strict; report.Fill(diagnostics); // A row counts as decoded when a parsed record has its id; the discovered and block-only records have none in the table. var parsed = new HashSet(info.Elements.Where(element => !element.IsBlockOnly).Select(element => (long)(uint)element.Id)); @@ -313,11 +341,23 @@ public static ExportReport ForRead(string file, long bytes, RvtReadDiagnostics d DecodedFraction = Fraction(decoded, elementIds.Count), Undecoded = undecoded.Groups(), }; + foreach (var stream in report.Streams) + { + if (stream.Failed > 0) + report.Issues.Add(new ReportIssue { Stream = stream.Name, Code = "failed-members", Detail = $"{stream.Failed} of {stream.Members} member(s) failed to decode" }); + if (stream.Recovered > 0) + report.Issues.Add(new ReportIssue { Stream = stream.Name, Code = "recovered-members", Detail = $"{stream.Recovered} of {stream.Members} member(s) rebuilt by structural recovery" }); + int unverifiedWithoutRecovery = stream.Unverified - stream.Recovered; + if (unverifiedWithoutRecovery > 0) + report.Issues.Add(new ReportIssue { Stream = stream.Name, Code = "unverified-members", Detail = $"{unverifiedWithoutRecovery} of {stream.Members} member(s) decoded without a matching checksum" }); + } // A project always has elements: a table without a row did not decode, whatever the streams say. if (elementIds.Count == 0) report.Issues.Add(new ReportIssue { Stream = "Global/ElemTable", Code = "no-rows", Detail = "the element table decoded no row" }); bool lostMembers = report.Streams.Any(stream => stream.Failed > 0 || stream.Unverified > 0 || stream.Recovered > 0); - report.Status = diagnostics.LimitHits.Count > 0 || lostMembers || report.Issues.Count > 0 ? "partial" : "complete"; + bool isPartial = diagnostics.LimitHits.Count > 0 || lostMembers || report.Issues.Count > 0; + report.Status = isPartial ? "partial" : "complete"; + report.ExitCode = RvtExitCodes.ForRead(diagnostics, partialAllowed, strict, isPartial); return report; } diff --git a/tests/RvtIO.Tests/CliExitCodeTests.cs b/tests/RvtIO.Tests/CliExitCodeTests.cs index f91cd85..48133a3 100644 --- a/tests/RvtIO.Tests/CliExitCodeTests.cs +++ b/tests/RvtIO.Tests/CliExitCodeTests.cs @@ -60,6 +60,122 @@ private static byte[] CompoundFile(int streamBytes) finally { if (File.Exists(path)) File.Delete(path); } } + private static (int ExitCode, string StdErr) RunWithOutput(params string[] arguments) + { + var start = new ProcessStartInfo("dotnet") { RedirectStandardOutput = true, RedirectStandardError = true, UseShellExecute = false }; + start.ArgumentList.Add(Path.Combine(AppContext.BaseDirectory, "RvtIO.Cli.dll")); + foreach (string argument in arguments) start.ArgumentList.Add(argument); + using var process = Process.Start(start)!; + var output = process.StandardOutput.ReadToEndAsync(); + var error = process.StandardError.ReadToEndAsync(); + Assert.True(process.WaitForExit(120_000), "The tool did not end within two minutes."); + Task.WaitAll(output, error); + return (process.ExitCode, error.Result); + } + + private static void WriteSyntheticRvt(string path, byte[]? extraPartition = null) + { + using var elemTable = new MemoryStream(); + using (var writer = new BinaryWriter(elemTable)) + { + writer.Write((ushort)1312); + writer.Write(1); + writer.Write(12345); writer.Write(100); writer.Write(64); + writer.Write(523); writer.Write(999); writer.Write(7); writer.Write(42); + } + byte[] intactPartition = CoverageCensusTests.Partition( + CoverageCensusTests.Block(false, CoverageCensusTests.R(12345, CoverageCensusTests.Body("Plain", 5, 0, 0, 0)))); + var streams = new List<(string Path, byte[] Bytes)> + { + ("Global/ElemTable", RvtIO.Writing.PageCodec.EncodePadded(RvtIO.Writing.GzipMember.Write(elemTable.ToArray()))), + ("Global/History", RvtIO.Writing.PageCodec.EncodePadded(RvtIO.Writing.GzipMember.Write(new byte[64]))), + ("Formats/Latest", RvtIO.Writing.PageCodec.EncodePadded(RvtIO.Writing.GzipMember.Write(CoverageCensusTests.TableBytes))), + ("Partitions/1", intactPartition), + }; + if (extraPartition != null) + streams.Add(("Partitions/2", extraPartition)); + CoverageCensusTests.WriteFile(path, streams.ToArray()); + } + + [Fact] + public void AnIntactFileWithStrictOption_ExitsZeroAndWritesOutputAndCompleteReport() + { + string directory = Path.Combine(Path.GetTempPath(), "rvtio-cli-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(directory); + try + { + string rvt = Path.Combine(directory, "intact.rvt"); + string outJson = Path.Combine(directory, "output.json"); + string report = Path.Combine(directory, "report.json"); + WriteSyntheticRvt(rvt); + + var (exit, _) = RunWithOutput(rvt, outJson, "--strict", "--report", report); + Assert.Equal(RvtExitCodes.Success, exit); + Assert.True(File.Exists(outJson)); + using var json = JsonDocument.Parse(File.ReadAllText(report)); + Assert.Equal("complete", json.RootElement.GetProperty("status").GetString()); + Assert.Equal(RvtExitCodes.Success, json.RootElement.GetProperty("exitCode").GetInt32()); + Assert.True(json.RootElement.GetProperty("limits").GetProperty("strict").GetBoolean()); + Assert.Equal(0, json.RootElement.GetProperty("issues").GetArrayLength()); + } + finally { Directory.Delete(directory, recursive: true); } + } + + [Fact] + public void AFileWithADamagedMember_ExitsZeroWithoutStrictAndPartialReadCodeWithStrict() + { + string directory = Path.Combine(Path.GetTempPath(), "rvtio-cli-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(directory); + try + { + string rvt = Path.Combine(directory, "damaged.rvt"); + byte[] badMember = new byte[512]; + new Random(11).NextBytes(badMember); + badMember[0] = 0; badMember[1] = 1; + WriteSyntheticRvt(rvt, extraPartition: badMember); + + string lenientOut = Path.Combine(directory, "lenient.json"); + string lenientReport = Path.Combine(directory, "lenient-report.json"); + Assert.Equal(RvtExitCodes.Success, Run(rvt, lenientOut, "--report", lenientReport)); + Assert.True(File.Exists(lenientOut)); + using (var json = JsonDocument.Parse(File.ReadAllText(lenientReport))) + { + var root = json.RootElement; + Assert.Equal("partial", root.GetProperty("status").GetString()); + Assert.Equal(RvtExitCodes.Success, root.GetProperty("exitCode").GetInt32()); + Assert.False(root.GetProperty("limits").GetProperty("strict").GetBoolean()); + var p2 = root.GetProperty("streams").EnumerateArray().Single(s => s.GetProperty("name").GetString() == "Partitions/2"); + Assert.True(p2.GetProperty("failed").GetInt32() > 0); + var issue = Assert.Single(root.GetProperty("issues").EnumerateArray()); + Assert.Equal("Partitions/2", issue.GetProperty("stream").GetString()); + Assert.Equal("failed-members", issue.GetProperty("code").GetString()); + } + + string strictOut = Path.Combine(directory, "strict.json"); + string strictReport = Path.Combine(directory, "strict-report.json"); + var (strictExit, stderr) = RunWithOutput(rvt, strictOut, "--strict", "--report", strictReport); + Assert.Equal(RvtExitCodes.PartialRead, strictExit); + Assert.False(File.Exists(strictOut)); + Assert.Contains("Partitions/2", stderr); + Assert.Contains("exit code 6", stderr); + using (var json = JsonDocument.Parse(File.ReadAllText(strictReport))) + { + var root = json.RootElement; + Assert.Equal("partial", root.GetProperty("status").GetString()); + Assert.Equal(RvtExitCodes.PartialRead, root.GetProperty("exitCode").GetInt32()); + Assert.True(root.GetProperty("limits").GetProperty("strict").GetBoolean()); + Assert.False(root.TryGetProperty("export", out _)); + var issue = Assert.Single(root.GetProperty("issues").EnumerateArray()); + Assert.Equal("Partitions/2", issue.GetProperty("stream").GetString()); + Assert.Equal("failed-members", issue.GetProperty("code").GetString()); + } + + // --allow-partial and --strict exclude each other + Assert.Equal(RvtExitCodes.Usage, Run(rvt, strictOut, "--allow-partial", "--strict")); + } + finally { Directory.Delete(directory, recursive: true); } + } + [Fact] public void AnInputThatIsNoCompoundFile_EndsWithTheInputErrorCodeAndAFailedReport() => RunOn("this is not a Revit file"u8.ToArray(), "not-a-compound-file"); diff --git a/tests/RvtIO.Tests/ExportReportTests.cs b/tests/RvtIO.Tests/ExportReportTests.cs index 6735518..4387834 100644 --- a/tests/RvtIO.Tests/ExportReportTests.cs +++ b/tests/RvtIO.Tests/ExportReportTests.cs @@ -513,6 +513,50 @@ public void RebuiltMembersTreatStreamAsUnverifiedInReportStatus() Assert.Equal("partial", report.Status); } + [Theory] + [InlineData(1, 0, 0, true, "Partitions/0", "failed-members")] + [InlineData(0, 1, 1, true, "Partitions/0", "recovered-members")] + [InlineData(0, 0, 1, true, "Partitions/0", "unverified-members")] + [InlineData(0, 0, 0, false, "Global/ElemTable", "no-rows")] + public void StrictModeExitsWithPartialReadCodeAndListsTheStreamAndReason( + int failed, int recovered, int unverified, bool hasRows, string expectedStream, string expectedCode) + { + var diagnostics = new RvtReadDiagnostics(); + diagnostics.Streams.Add(new StreamRecord("Partitions/0", 1000, 4, 4 - failed, failed, recovered, unverified)); + uint[] rows = hasRows ? [1u] : []; + + var lenient = ExportReport.ForRead("test.rvt", 1000, diagnostics, false, false, rows, new RevitFileInfo(), strict: false); + Assert.Equal("partial", lenient.Status); + Assert.Equal(RvtExitCodes.Success, lenient.ExitCode); + Assert.False(lenient.Limits.Strict); + var lenientIssue = Assert.Single(lenient.Issues); + Assert.Equal((expectedStream, expectedCode), (lenientIssue.Stream, lenientIssue.Code)); + + var strictReport = ExportReport.ForRead("test.rvt", 1000, diagnostics, false, false, rows, new RevitFileInfo(), strict: true); + Assert.Equal("partial", strictReport.Status); + Assert.Equal(RvtExitCodes.PartialRead, strictReport.ExitCode); + Assert.True(strictReport.Limits.Strict); + var strictIssue = Assert.Single(strictReport.Issues); + Assert.Equal((expectedStream, expectedCode), (strictIssue.Stream, strictIssue.Code)); + } + + [Fact] + public void StrictModeKeepsExitCodeZeroOnCompleteReadAndLimitCodeOnUnacceptedLimitHit() + { + var complete = new RvtReadDiagnostics(); + complete.Streams.Add(new StreamRecord("Partitions/0", 1000, 4, 4, 0, 0, 0)); + var completeReport = ExportReport.ForRead("test.rvt", 1000, complete, false, false, [1u], new RevitFileInfo(), strict: true); + Assert.Equal("complete", completeReport.Status); + Assert.Equal(RvtExitCodes.Success, completeReport.ExitCode); + Assert.Empty(completeReport.Issues); + + var limitCut = new RvtReadDiagnostics(); + limitCut.LimitHits.Add(new ReaderLimitHit("Global/ElemTable", "elem-table-recovery-bytes", 1000, true, "cut")); + var limitReport = ExportReport.ForRead("test.rvt", 1000, limitCut, false, partialAllowed: false, [1u], new RevitFileInfo(), strict: true); + Assert.Equal("partial", limitReport.Status); + Assert.Equal(RvtExitCodes.LimitExceeded, limitReport.ExitCode); + } + private sealed class TempFile : IDisposable { public string Path { get; } = System.IO.Path.Combine(System.IO.Path.GetTempPath(), "rr-report-" + Guid.NewGuid() + ".rvt");