From 52bb22cf4f3087337199583dd35832903d17cdc6 Mon Sep 17 00:00:00 2001 From: mahach666 <79989386+mahach666@users.noreply.github.com> Date: Thu, 12 Nov 2026 18:02:16 +0300 Subject: [PATCH] Read the block records of the project block by its framing In a file without its own class schema whose partition streams frame, the block parser reads the records of the project's own block as the framing lists them instead of scanning the chunks of those streams. Blocks of loaded family documents are no longer read as project elements. The geometry of a record that runs into the next segment or is over the size limit of the scan is not decoded, as the scan never reached it. Refs #48. --- CHANGELOG.md | 1 + docs/test-fixtures.md | 2 +- src/RvtIO/Internal/ParsingState.cs | 7 + src/RvtIO/Parsing/BlockParser.cs | 467 +++++++++++----------- src/RvtIO/Parsing/PartitionFraming.cs | 30 +- src/RvtIO/RvtFileReader.cs | 68 ++-- tests/RvtIO.Tests/ApiTests.cs | 8 +- tests/RvtIO.Tests/OwnElementClassTests.cs | 48 +++ 8 files changed, 370 insertions(+), 261 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a644ac..f9e9fe7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,7 @@ API between minor releases. ### Changed - Reading: in a file without its own class schema (Revit 2020 and 2022) whose partition streams are read by their framing, the root object of each element, from which the reader takes the class of the element and the stair and railing facts it keeps per root, is the channel-102 record of the element in the project's own block, in its current copy. Before, the stream walk counted every block it met for the id, including copies of the id in the documents of loaded families, and missed roots it did not reach; an element with two counted roots or none had no class. On 23 such files 59 elements in 7 files now get their class (a schedule, a view, a wall, a rebar, a sketch plane and others): in the compact JSON they carry the name of their class and their usual properties instead of an empty name, and the export report names their class instead of `unknown`; the IFC of all 23 files is unchanged. Refs #47. +- Reading: in the same files the block reader, which collects the classes, geometry and other per-element facts of every element, reads the records of the project's own block as the framing lists them, in their current copy, instead of scanning the stream for anything that looks like a block. A record that runs into the next segment of its stream is read too; the blocks of the documents of loaded families, whose element ids repeat project ids, are no longer read as project elements. The geometry of a record that runs into the next segment or holds more than 1,000,000 bytes is not decoded, as the scan never reached it (the decoder takes up to minutes on such a record). Measured on 26 project files against the branch this starts from: the IFC and the compact JSON of 25 are identical; the export report of 22 differs in the elements it counts (ids that only a family document held are no longer counted) and in the category of ids that the Revit API does not return. In one architectural file 8 sketch lines change their length and 14 walls the name of their type: these elements keep two copies from different increments, which the increment table does not tell apart, and the reader now takes the other copy. Against Revit 2022, 5 of the 8 lengths are now right and 3 were right before (6465 of the 6482 sketch lines of the file are right, 6463 before), and the type name of the 14 walls is wrong both before and now. In the element list of `RvtDocument` of one test file (59,201 entries before), the 8,135 elements Revit returns keep their category, type, name, parameters and references; of the entries Revit does not return as elements, 49 without a category are gone and 50,738 change (fewer parameters on 50,564 of them, 3,737,513 parameters in all before and 3,161,540 now, and a name on 16,373). Not changed: which of two such copies is current. Refs #48. - Reading: the element index of the files with their own class schema (the header category, design option and outline of each element, the class of its root object and the field walk of that object) and the reinforcement reader read a partition stream whose framing holds by its records: the channel-101 records of the project's own block are the element headers and its channel-102 records the root objects, one of each per element. Before, a scan of the stream took any byte run that looked like an item, including the records of the documents of loaded families, whose element ids repeat project ids; a value on which such a copy differed was dropped. On 29 files the scan found 5,987,630 header-like and 6,640,008 block-like items, the framing gives 848,325 of each. A stream whose framing does not hold is scanned as before. On 52 project files of the releases 2020 to 2027 the IFC and the export report are identical to the output of the main branch; in the compact JSON of one Revit 2021 file one element (the pen width table) now gets its name and properties, as in the other files. - Reading: a partition stream whose framing holds (content blocks, segments, gzip members with their CRC and length, record lengths, the owners of each block) is cut into its segments by that framing; the search for gzip members and its recovery of damaged members are kept for a stream whose framing does not hold, and the diagnostics say which way a stream was read. On 52 project files of the releases 2020 to 2027 the segments are the chunks the search found, byte for byte, in all 434 partition streams; the IFC, the compact JSON and the export report of 14 of them, one or more of every release, are identical to the output of the main branch. - Field walk by the class schema of the file: a text is read as long as its count says, where the walk stopped at a text longer than 100,000 characters; a text longer than the rest of the record stops it as any read past the end does. `rvtio coverage` also reads an extensible storage entity in the block of a loaded family by the schemas of that family's own document: up to release 2024 a family keeps them in its `ADocument` in `Global/ContentDocuments` (one document per content block, under the key of the block), from release 2025 `Global/Latest` holds them. On 52 project files of the releases 2020 to 2027 every record of every partition stream is now walked to its last byte (29,794,710 records; 117 were not: 45 at a long text, 72 at an entity of a family document); no class row has fewer exact records. Output: of ten files that hold such a text, eight family types of lighting fixtures in two MEP files get their parameter list in the compact JSON, and in three files of one MEP project one view type and one sun study change the reason they are left out of the IFC for; the IFC of the ten files does not change. diff --git a/docs/test-fixtures.md b/docs/test-fixtures.md index 7dc2511..0b9e3bd 100644 --- a/docs/test-fixtures.md +++ b/docs/test-fixtures.md @@ -45,7 +45,7 @@ folder (`TestFiles.Scratch`); no test uses a fixed absolute path. ## What runs without the private files -With no private file at all, 9686 of the 10105 test cases run and pass and 419 are skipped: 413 need private files (listed +With no private file at all, 9688 of the 10107 test cases run and pass and 419 are skipped: 413 need private files (listed below; 17 of them only `bim_dump.txt` or `formats_latest_decompressed.bin`) and 6 need other inputs (two need symbolic links, three need `RVTIO_FREEFORM_FULL`, `RVTIO_CYL_FULL` or `RVTIO_MESH_FULL`, one is a known gap). The tests that run use synthetic data built in the tests and the small fixtures committed under `tests/RvtIO.Tests/Fixtures`. With the full private fixture folder the diff --git a/src/RvtIO/Internal/ParsingState.cs b/src/RvtIO/Internal/ParsingState.cs index ad0300b..3ba8d1e 100644 --- a/src/RvtIO/Internal/ParsingState.cs +++ b/src/RvtIO/Internal/ParsingState.cs @@ -117,6 +117,13 @@ public Ifc.IfcNumericUnitPlan? OwnDocumentIfcNumericUnitPlan /// its segments and the records of the project's own block, without the copies record routing marks as not current. public IReadOnlyList? FramedPartitions { get; set; } + /// How many of the first are the chunks of the partition streams (the chunks of Formats and Global follow). + public int PartitionChunkCount { get; set; } + + /// True for a file without its own class schema whose every partition stream is read by its framing: the readers take the + /// records of the project's own block () instead of scanning the partition chunks. + internal bool ReadsFramedRecords => WideSchema == null && FramedPartitions is { Count: > 0 } framed && framed.All(f => f is { IdWidth: 4 }); + /// The own name of an element of a file, read from its root block by the schema-driven walk. public Dictionary WideNames { get; } = new(); diff --git a/src/RvtIO/Parsing/BlockParser.cs b/src/RvtIO/Parsing/BlockParser.cs index f9a272b..1b2eef1 100644 --- a/src/RvtIO/Parsing/BlockParser.cs +++ b/src/RvtIO/Parsing/BlockParser.cs @@ -35,6 +35,10 @@ internal static void ApplyPendingGReps(ParsingState state, List pen } } + /// The largest block data the scan of a chunk reads (a larger block is not parsed); the geometry of a framed record over it is not + /// decoded. + internal const uint MaxBlockSize = 1_000_000; + public static bool ParseChunk(byte[] chunk, ParsingState state, bool collectBlockOnlyIds = false, List? deferred = null) { @@ -78,7 +82,7 @@ public static bool ParseChunk(byte[] chunk, ParsingState state, bool collectBloc uint dataSize = BitConverter.ToUInt32(chunk, pos + 8); // Skip blocks with implausible dataSize (check before bounds to avoid int overflow) - if (dataSize > 1_000_000) { pos++; continue; } + if (dataSize > MaxBlockSize) { pos++; continue; } // Skip zero-elemId entries (padding/terminators) in 4-byte strides if (eid == 0) { pos += 4; continue; } if (dataSize == 0) { pos++; continue; } @@ -98,265 +102,274 @@ public static bool ParseChunk(byte[] chunk, ParsingState state, bool collectBloc if (!knownOwner && !hasSizeFooter) { pos++; continue; } blockCount++; - int dataStart = pos + 16; - int dataEnd = dataStart + (int)dataSize; - uint classIdx = BitConverter.ToUInt32(chunk, pos + 12); + if (!ParseBlock(chunk, pos, eid, dataSize, knownOwner, state, collectBlockOnlyIds, pendingAll)) { pos++; continue; } + pos += 16 + (int)dataSize; + } + if (deferred == null) ApplyPendingGReps(state, pendingAll); + return blockCount > 0; + } + + /// Reads one block [elemId:4][hash:4][dataSize:4][classIdx:4][data] whose header the caller accepted: the block inventory, the + /// category of its owner and every fact the block parser takes from a block. False when the class is 0 (no block). + internal static bool ParseBlock(byte[] chunk, int pos, uint eid, uint dataSize, bool knownOwner, ParsingState state, + bool collectBlockOnlyIds, List pendingAll) + { + int dataStart = pos + 16; + int dataEnd = dataStart + (int)dataSize; + uint classIdx = BitConverter.ToUInt32(chunk, pos + 12); + + if (!knownOwner) + { + if (collectBlockOnlyIds) + state.BlockOnlyIds.Add(eid); + } - if (!knownOwner) + // Collect block classIdx inventory (low 16 bits = class type, high 16 varies per block) + uint classKey = classIdx & 0xFFFF; + // classKey=0 is never a valid class - these are false positives from scanner misalignment + if (classKey == 0) return false; + if (ElementIdEncoding.IsSupported(eid)) + { + if (!state.BlockClassInventory.TryGetValue(classKey, out var bcs)) + state.BlockClassInventory[classKey] = bcs = new BlockClassStats(); + bcs.Count++; + bcs.TotalBytes += dataSize; + if (dataSize < bcs.MinSize) bcs.MinSize = (int)dataSize; + if (dataSize > bcs.MaxSize) bcs.MaxSize = (int)dataSize; + if (bcs.SampleElemIds.Count < 5) bcs.SampleElemIds.Add(eid); + if (bcs.SampleData == null) { - if (collectBlockOnlyIds) - state.BlockOnlyIds.Add(eid); + int sampleLen = Math.Min(32, (int)dataSize); + bcs.SampleData = new byte[sampleLen]; + Array.Copy(chunk, dataStart, bcs.SampleData, 0, sampleLen); } - - // Collect block classIdx inventory (low 16 bits = class type, high 16 varies per block) - uint classKey = classIdx & 0xFFFF; - // classKey=0 is never a valid class - these are false positives from scanner misalignment - if (classKey == 0) { pos++; continue; } - if (ElementIdEncoding.IsSupported(eid)) + if (state.ElemTable.TryGetValue(eid, out uint typeId) && typeId != 0) { - if (!state.BlockClassInventory.TryGetValue(classKey, out var bcs)) - state.BlockClassInventory[classKey] = bcs = new BlockClassStats(); - bcs.Count++; - bcs.TotalBytes += dataSize; - if (dataSize < bcs.MinSize) bcs.MinSize = (int)dataSize; - if (dataSize > bcs.MaxSize) bcs.MaxSize = (int)dataSize; - if (bcs.SampleElemIds.Count < 5) bcs.SampleElemIds.Add(eid); - if (bcs.SampleData == null) - { - int sampleLen = Math.Min(32, (int)dataSize); - bcs.SampleData = new byte[sampleLen]; - Array.Copy(chunk, dataStart, bcs.SampleData, 0, sampleLen); - } - if (state.ElemTable.TryGetValue(eid, out uint typeId) && typeId != 0) - { - bcs.TypeIdCounts.TryGetValue(typeId, out int tc); - bcs.TypeIdCounts[typeId] = tc + 1; - } + bcs.TypeIdCounts.TryGetValue(typeId, out int tc); + bcs.TypeIdCounts[typeId] = tc + 1; } + } + + // Assign the category from the Tf classIdx mapping (if the element has none yet): + // first the id registry, then the R22 def table (id -> class name) + // joined with the name -> category map. + ushort tfClassId = (ushort)(classIdx & 0xFFFF); + if (tfClassId > 0 && state.Records.TryGetValue(eid, out var recForCat) && !recForCat.CategoryId.HasValue) + { + if (TfClassRegistry.R22TfClassIdToCategory.TryGetValue(tfClassId, out int catId)) + recForCat.CategoryId = catId; + // NOTE: an automatic fallback via R22ClassDefs.Names -> NameToCategory is not used: + // early categorization changes downstream pass behavior and needs per-row validation first. + } + + if (tfClassId == 480 && !state.BasePointCoordinates.ContainsKey(eid)) + { + var coordinates = TfBlockDeserializer.ExtractBasePointCoordinates( + chunk, pos, dataSize, tfClassId); + if (coordinates is { } bp) state.BasePointCoordinates[eid] = bp; + } - // Assign the category from the Tf classIdx mapping (if the element has none yet): - // first the id registry, then the R22 def table (id -> class name) - // joined with the name -> category map. - ushort tfClassId = (ushort)(classIdx & 0xFFFF); - if (tfClassId > 0 && state.Records.TryGetValue(eid, out var recForCat) && !recForCat.CategoryId.HasValue) + // Project Level blocks are small and may live in structurally recovered chunks + // that the continuous anchored walk cannot enter. Decode them directly here so + // DatumPlane.m_text and m_isBuildingStory are not lost. + if (tfClassId == 2299) + { + var np = TfBlockDeserializer.ExtractParams(chunk, pos, dataSize, tfClassId) + ?? new TfBlockDeserializer.NativeParams(); + np.Fields["__isLevel"] = 1; + var levelName = TfBlockDeserializer.ExtractLevelName( + chunk, pos, dataSize, tfClassId); + if (!string.IsNullOrWhiteSpace(levelName)) + np.StringFields["m_text"] = levelName; + if (TfBlockDeserializer.ExtractLevelElevation( + chunk, pos, dataSize, tfClassId) is double levelElevation) + np.DoubleFields["__levelElevation"] = levelElevation; + state.NativeParams[eid] = np; + } + + if (state.AllKnownIds.Contains(eid)) + { + ExtractBlockStrings(chunk, dataStart, dataEnd, eid, state.ElemStrings); + + ExtractBlockDoubles(chunk, dataStart, dataEnd, eid, state.ElemDoubles); + + // Blocks with high classIdx word = 0xFFFF contain BIP key->double pairs at stride 12 + if ((classIdx >> 16) == 0xFFFF) { - if (TfClassRegistry.R22TfClassIdToCategory.TryGetValue(tfClassId, out int catId)) - recForCat.CategoryId = catId; - // NOTE: an automatic fallback via R22ClassDefs.Names -> NameToCategory is not used: - // early categorization changes downstream pass behavior and needs per-row validation first. + ExtractBipDoublesFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipDoubles, state.AllKnownIds); + ExtractBaseFieldsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBaseFields); + ExtractWallIntBipsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipInts, state.Records); + ExtractHostObjBipsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipInts); + ExtractCurveDriverFromBlock(chunk, dataStart, dataEnd, eid, state.CurveDriverCandidates, state.Records); + // Only extract names from proper Tf-serialized SymbolInfo blocks (not render material blocks = 0xFFFFFFFF) + if (classIdx != 0xFFFFFFFF) + ExtractNameFromBlock(chunk, dataStart, dataEnd, eid, state.NativeElementNames); } - if (tfClassId == 480 && !state.BasePointCoordinates.ContainsKey(eid)) + // classIdx=0x702 blocks contain user param values (strings + doubles) and BIP ElementId values + if ((classIdx & 0xFFFF) == 0x0702) { - var coordinates = TfBlockDeserializer.ExtractBasePointCoordinates( - chunk, pos, dataSize, tfClassId); - if (coordinates is { } bp) state.BasePointCoordinates[eid] = bp; + ExtractParamValuesFrom0702Block(chunk, dataStart, dataEnd, eid, state.AllKnownIds, state.ElemParamStrings, state.ElemBipDoubles); + ExtractBipIntsFrom0702Block(chunk, dataStart, dataEnd, eid, state.AllKnownIds, state.ElemBipInts, state.Records); + ExtractFamilyInstanceTransform(chunk, dataStart, dataEnd, eid, state.FamilyInstanceTransforms, state.Records); + ExtractInsertSillFrom0702Block(chunk, dataStart, dataEnd, eid, state.ElemBipDoubles); } - // Project Level blocks are small and may live in structurally recovered chunks - // that the continuous anchored walk cannot enter. Decode them directly here so - // DatumPlane.m_text and m_isBuildingStory are not lost. - if (tfClassId == 2299) + // Dimension / constraint reference geometry. Two block families share the + // same payload layout ([self elemId @0x24][null refs][FCFFFFFF marker] + // [u32 point count @marker+7][reference-point (x,y,z) f64 triples @marker+11]): + // 0x091F - dimensions (OST_Dimensions / OST_WeakDims / OST_Constraints) + // 0x009D - constraints (OST_Constraints; coincident points = alignment/lock) + if (((classIdx & 0xFFFF) == 0x091F || (classIdx & 0xFFFF) == 0x009D) + && !state.Dimensions.ContainsKey(eid)) + ExtractDimensionData(chunk, dataStart, dataEnd, eid, state.Dimensions); + + // classIdx=0x0346 blocks: OST_SketchLines - the 2D profile curves of sketches. + // The curve is stored in the same GInfoBlob GLine/GArc form as wall location + // curves (Flags 0x01080004), decoded into a LocationCurve. + if ((classIdx & 0xFFFF) == 0x0346) { - var np = TfBlockDeserializer.ExtractParams(chunk, pos, dataSize, tfClassId) - ?? new TfBlockDeserializer.NativeParams(); - np.Fields["__isLevel"] = 1; - var levelName = TfBlockDeserializer.ExtractLevelName( - chunk, pos, dataSize, tfClassId); - if (!string.IsNullOrWhiteSpace(levelName)) - np.StringFields["m_text"] = levelName; - if (TfBlockDeserializer.ExtractLevelElevation( - chunk, pos, dataSize, tfClassId) is double levelElevation) - np.DoubleFields["__levelElevation"] = levelElevation; - state.NativeParams[eid] = np; + state.CurveBlockOwners.Add(eid); + if (!state.SketchCurves.ContainsKey(eid)) + ExtractSketchCurve(chunk, dataStart, dataEnd, eid, state.SketchCurves, state.DetailCurveStyles); } - if (state.AllKnownIds.Contains(eid)) + // Native parameter stores are extracted by the continuous-stream walker + // (TfStreamWalker in RvtFileReader) - blocks cross chunk boundaries, so the + // per-chunk walk here sees only ~8% of them. + + // classIdx=0x09B6 blocks: OST_Materials - the material's shading appearance + // (name + simple-graphics color, shininess, transparency, smoothness). + if ((classIdx & 0xFFFF) == 0x09B6 && !state.MaterialAppearances.ContainsKey(eid)) + ExtractMaterialAppearance(chunk, dataStart, dataEnd, eid, state.MaterialAppearances, state.MaterialFinishes); + + // HVAC load energy defaults (heat gains, lighting/power loads, air changes, set + // points) at fixed offsets. 0x085A = HVACLoadSpaceType, 0x0857 = HVACLoadBuildingType + // (same field layout). + if (((classIdx & 0xFFFF) == 0x085A || (classIdx & 0xFFFF) == 0x0857) + && !state.HvacSpaceTypeParams.ContainsKey(eid)) + ExtractHvacSpaceType(chunk, dataStart, dataEnd, eid, + state.HvacSpaceTypeParams, state.HvacScheduleRefs); + + // classIdx=0x0884 blocks: analytical model Hub nodes - own-block signature + // used to categorize the (otherwise uncategorized) node as OST_AnalyticalNodes. + if ((classIdx & 0xFFFF) == 0x0884) + state.AnalyticalNodeOwners.Add(eid); + + // classIdx=0x03E3 blocks: view/sheet info. For ViewSheet elements the block + // carries [8 zero bytes][len-prefixed sheet number][len-prefixed issue date]. + if ((classIdx & 0xFFFF) == 0x03E3 && !state.SheetInfo.ContainsKey(eid)) + ExtractSheetInfo(chunk, dataStart, dataEnd, eid, state.SheetInfo); + + // View display name: name-field signature in the element's own block. + if (!state.ViewNames.ContainsKey(eid)) + ExtractViewName(chunk, dataStart, dataEnd, eid, state.ViewNames); + + // classIdx=0x0E17 blocks (R22): Level class-specific data. + // Offset 0xF4 contains the level elevation as a double (in feet). + if ((classIdx & 0xFFFF) == 0x0E17 && dataSize >= 0xFC) { - ExtractBlockStrings(chunk, dataStart, dataEnd, eid, state.ElemStrings); - - ExtractBlockDoubles(chunk, dataStart, dataEnd, eid, state.ElemDoubles); - - // Blocks with high classIdx word = 0xFFFF contain BIP key->double pairs at stride 12 - if ((classIdx >> 16) == 0xFFFF) - { - ExtractBipDoublesFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipDoubles, state.AllKnownIds); - ExtractBaseFieldsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBaseFields); - ExtractWallIntBipsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipInts, state.Records); - ExtractHostObjBipsFromBlock(chunk, dataStart, dataEnd, eid, state.ElemBipInts); - ExtractCurveDriverFromBlock(chunk, dataStart, dataEnd, eid, state.CurveDriverCandidates, state.Records); - // Only extract names from proper Tf-serialized SymbolInfo blocks (not render material blocks = 0xFFFFFFFF) - if (classIdx != 0xFFFFFFFF) - ExtractNameFromBlock(chunk, dataStart, dataEnd, eid, state.NativeElementNames); - } - - // classIdx=0x702 blocks contain user param values (strings + doubles) and BIP ElementId values - if ((classIdx & 0xFFFF) == 0x0702) + double elev = BitConverter.ToDouble(chunk, dataStart + 0xF4); + if (!double.IsNaN(elev) && !double.IsInfinity(elev) && Math.Abs(elev) < 100_000) { - ExtractParamValuesFrom0702Block(chunk, dataStart, dataEnd, eid, state.AllKnownIds, state.ElemParamStrings, state.ElemBipDoubles); - ExtractBipIntsFrom0702Block(chunk, dataStart, dataEnd, eid, state.AllKnownIds, state.ElemBipInts, state.Records); - ExtractFamilyInstanceTransform(chunk, dataStart, dataEnd, eid, state.FamilyInstanceTransforms, state.Records); - ExtractInsertSillFrom0702Block(chunk, dataStart, dataEnd, eid, state.ElemBipDoubles); - } - - // Dimension / constraint reference geometry. Two block families share the - // same payload layout ([self elemId @0x24][null refs][FCFFFFFF marker] - // [u32 point count @marker+7][reference-point (x,y,z) f64 triples @marker+11]): - // 0x091F - dimensions (OST_Dimensions / OST_WeakDims / OST_Constraints) - // 0x009D - constraints (OST_Constraints; coincident points = alignment/lock) - if (((classIdx & 0xFFFF) == 0x091F || (classIdx & 0xFFFF) == 0x009D) - && !state.Dimensions.ContainsKey(eid)) - ExtractDimensionData(chunk, dataStart, dataEnd, eid, state.Dimensions); - - // classIdx=0x0346 blocks: OST_SketchLines - the 2D profile curves of sketches. - // The curve is stored in the same GInfoBlob GLine/GArc form as wall location - // curves (Flags 0x01080004), decoded into a LocationCurve. - if ((classIdx & 0xFFFF) == 0x0346) - { - state.CurveBlockOwners.Add(eid); - if (!state.SketchCurves.ContainsKey(eid)) - ExtractSketchCurve(chunk, dataStart, dataEnd, eid, state.SketchCurves, state.DetailCurveStyles); - } - - // Native parameter stores are extracted by the continuous-stream walker - // (TfStreamWalker in RvtFileReader) - blocks cross chunk boundaries, so the - // per-chunk walk here sees only ~8% of them. - - // classIdx=0x09B6 blocks: OST_Materials - the material's shading appearance - // (name + simple-graphics color, shininess, transparency, smoothness). - if ((classIdx & 0xFFFF) == 0x09B6 && !state.MaterialAppearances.ContainsKey(eid)) - ExtractMaterialAppearance(chunk, dataStart, dataEnd, eid, state.MaterialAppearances, state.MaterialFinishes); - - // HVAC load energy defaults (heat gains, lighting/power loads, air changes, set - // points) at fixed offsets. 0x085A = HVACLoadSpaceType, 0x0857 = HVACLoadBuildingType - // (same field layout). - if (((classIdx & 0xFFFF) == 0x085A || (classIdx & 0xFFFF) == 0x0857) - && !state.HvacSpaceTypeParams.ContainsKey(eid)) - ExtractHvacSpaceType(chunk, dataStart, dataEnd, eid, - state.HvacSpaceTypeParams, state.HvacScheduleRefs); - - // classIdx=0x0884 blocks: analytical model Hub nodes - own-block signature - // used to categorize the (otherwise uncategorized) node as OST_AnalyticalNodes. - if ((classIdx & 0xFFFF) == 0x0884) - state.AnalyticalNodeOwners.Add(eid); - - // classIdx=0x03E3 blocks: view/sheet info. For ViewSheet elements the block - // carries [8 zero bytes][len-prefixed sheet number][len-prefixed issue date]. - if ((classIdx & 0xFFFF) == 0x03E3 && !state.SheetInfo.ContainsKey(eid)) - ExtractSheetInfo(chunk, dataStart, dataEnd, eid, state.SheetInfo); - - // View display name: name-field signature in the element's own block. - if (!state.ViewNames.ContainsKey(eid)) - ExtractViewName(chunk, dataStart, dataEnd, eid, state.ViewNames); - - // classIdx=0x0E17 blocks (R22): Level class-specific data. - // Offset 0xF4 contains the level elevation as a double (in feet). - if ((classIdx & 0xFFFF) == 0x0E17 && dataSize >= 0xFC) - { - double elev = BitConverter.ToDouble(chunk, dataStart + 0xF4); - if (!double.IsNaN(elev) && !double.IsInfinity(elev) && Math.Abs(elev) < 100_000) - { - if (!state.ElemBipDoubles.TryGetValue(eid, out var bipMap)) - state.ElemBipDoubles[eid] = bipMap = new Dictionary(); - bipMap[-1007102] = elev; // LEVEL_ELEV - } + if (!state.ElemBipDoubles.TryGetValue(eid, out var bipMap)) + state.ElemBipDoubles[eid] = bipMap = new Dictionary(); + bipMap[-1007102] = elev; // LEVEL_ELEV } + } - // classIdx=0x00012100 blocks: CompoundStructure data for wall types. - // Offset 0x18 contains the wall function enum (0=Interior..5=CoreShaft). - if (classIdx == 0x00012100 && dataSize >= 0x1C) - { - int funcVal = BitConverter.ToInt32(chunk, dataStart + 0x18); - if (funcVal >= 0 && funcVal <= 5) - state.WallTypeFunctions[eid] = funcVal; - } + // classIdx=0x00012100 blocks: CompoundStructure data for wall types. + // Offset 0x18 contains the wall function enum (0=Interior..5=CoreShaft). + if (classIdx == 0x00012100 && dataSize >= 0x1C) + { + int funcVal = BitConverter.ToInt32(chunk, dataStart + 0x18); + if (funcVal >= 0 && funcVal <= 5) + state.WallTypeFunctions[eid] = funcVal; + } - // 0xFFFF0200 / 0xFFFF1026 blocks: CompoundStructure with layer data. - // Scan for marker 0x1003 followed by layer count + 29-byte layer entries. - if ((classIdx >> 16) == 0xFFFF && dataSize > 100 && !state.CompoundLayers.ContainsKey(eid)) - { - ExtractCompoundLayers(chunk, dataStart, dataEnd, eid, state.CompoundLayers); - } + // 0xFFFF0200 / 0xFFFF1026 blocks: CompoundStructure with layer data. + // Scan for marker 0x1003 followed by layer count + 29-byte layer entries. + if ((classIdx >> 16) == 0xFFFF && dataSize > 100 && !state.CompoundLayers.ContainsKey(eid)) + { + ExtractCompoundLayers(chunk, dataStart, dataEnd, eid, state.CompoundLayers); + } - // classIdx=0x0FBD blocks: ForgeTypeId Spec Registry - unit format definitions - if ((classIdx & 0xFFFF) == 0x0FBD && state.ForgeSpecRegistry.Count == 0) - { - ExtractForgeSpecRegistry(chunk, dataStart, dataEnd, state.ForgeSpecRegistry); - } + // classIdx=0x0FBD blocks: ForgeTypeId Spec Registry - unit format definitions + if ((classIdx & 0xFFFF) == 0x0FBD && state.ForgeSpecRegistry.Count == 0) + { + ExtractForgeSpecRegistry(chunk, dataStart, dataEnd, state.ForgeSpecRegistry); + } - // classIdx=0x0B3A blocks: Family Parameter Definitions - ForgeId -> spec type mapping - if ((classIdx & 0xFFFF) == 0x0B3A) - { - ExtractForgeParamSpec(chunk, dataStart, dataEnd, state.ForgeParamSpecs); - } + // classIdx=0x0B3A blocks: Family Parameter Definitions - ForgeId -> spec type mapping + if ((classIdx & 0xFFFF) == 0x0B3A) + { + ExtractForgeParamSpec(chunk, dataStart, dataEnd, state.ForgeParamSpecs); + } - // 0x0725 blocks: FamilyType/Symbol definitions - extract element name - // The first valid string after the Tf header (~offset 400+) is the type name - if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400 && !state.NativeElementNames.ContainsKey(eid)) - { - ExtractNameFrom0725Block(chunk, dataStart, dataEnd, eid, state.NativeElementNames); - } + // 0x0725 blocks: FamilyType/Symbol definitions - extract element name + // The first valid string after the Tf header (~offset 400+) is the type name + if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400 && !state.NativeElementNames.ContainsKey(eid)) + { + ExtractNameFrom0725Block(chunk, dataStart, dataEnd, eid, state.NativeElementNames); + } - // 0x0725 blocks of family symbols ALSO contain type-level param value entries - // (after the chain table) shaped as `[type:4][marker:u16=1][zeros:8][double:8] - // [FFFFFFFF][paramKey:4]`. These are family-doc defaults that apply to all - // instances unless overridden by the instance's 0x0702 block. - if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400) - { - ExtractFamilySymbolDoubles(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockDoubles); - ExtractFamilySymbolStrings(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockStrings); - ExtractFamilySymbolInts(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockInts); - ExtractFamilySymbolElemIdPairs(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockElemIds); + // 0x0725 blocks of family symbols ALSO contain type-level param value entries + // (after the chain table) shaped as `[type:4][marker:u16=1][zeros:8][double:8] + // [FFFFFFFF][paramKey:4]`. These are family-doc defaults that apply to all + // instances unless overridden by the instance's 0x0702 block. + if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400) + { + ExtractFamilySymbolDoubles(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockDoubles); + ExtractFamilySymbolStrings(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockStrings); + ExtractFamilySymbolInts(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockInts); + ExtractFamilySymbolElemIdPairs(chunk, dataStart, dataEnd, eid, state.FamilySymbolBlockElemIds); - if (!state.FamilyCutGreps.ContainsKey(eid)) - pendingAll.Add(new PendingGRep(chunk, eid, dataStart - 16, dataSize, true)); - } + if (!state.FamilyCutGreps.ContainsKey(eid)) + pendingAll.Add(new PendingGRep(chunk, eid, dataStart - 16, dataSize, true)); + } - // 0x0725 blocks also contain the family param mapping table. - // Each entry is 30 bytes: [FFFFFFFF:4][binKey:4][zeros:4][marker:u32 at offset 12][14 bytes]. - // Entries with marker == 1 form a linked list: bin paramKey of entry N maps to - // bin paramKey of entry N+1 (the actual project param def ID). - if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400 && !state.FamilyParamMaps.ContainsKey(eid)) - { - var (map, typeValues) = ExtractFamilyParamMapAndValues(chunk, dataStart, dataEnd); - if (map.Count > 0) - state.FamilyParamMaps[eid] = map; - if (typeValues.Count > 0) - state.FamilyTypeValues[eid] = typeValues; - } + // 0x0725 blocks also contain the family param mapping table. + // Each entry is 30 bytes: [FFFFFFFF:4][binKey:4][zeros:4][marker:u32 at offset 12][14 bytes]. + // Entries with marker == 1 form a linked list: bin paramKey of entry N maps to + // bin paramKey of entry N+1 (the actual project param def ID). + if ((classIdx & 0xFFFF) == 0x0725 && dataSize > 400 && !state.FamilyParamMaps.ContainsKey(eid)) + { + var (map, typeValues) = ExtractFamilyParamMapAndValues(chunk, dataStart, dataEnd); + if (map.Count > 0) + state.FamilyParamMaps[eid] = map; + if (typeValues.Count > 0) + state.FamilyTypeValues[eid] = typeValues; + } - // GRep blocks (classIdx & 0xFFFF == 0x07D7 R22, 0x07D6 R20): geometry representation - // Some elements have multiple GRep blocks - prefer the one with world-coordinate - // face directions (non-axis-aligned XVectors indicate world coordinates). - if ((classIdx & 0xFFFF) is 0x07D7 or 0x07D6) + // GRep blocks (classIdx & 0xFFFF == 0x07D7 R22, 0x07D6 R20): geometry representation + // Some elements have multiple GRep blocks - prefer the one with world-coordinate + // face directions (non-axis-aligned XVectors indicate world coordinates). + if ((classIdx & 0xFFFF) is 0x07D7 or 0x07D6) + { + // Diagnostic: RVTIO_DUMP_GREP= dumps the raw GRep block. + if (DumpGrepEnvironment is { } dge + && dge.Split(',').Any(x => uint.TryParse(x, out uint dgeId) && dgeId == eid)) { - // Diagnostic: RVTIO_DUMP_GREP= dumps the raw GRep block. - if (DumpGrepEnvironment is { } dge - && dge.Split(',').Any(x => uint.TryParse(x, out uint dgeId) && dgeId == eid)) + try { - try - { - int hdr = dataStart - 16; - int dlen = Math.Min(dataEnd - hdr, chunk.Length - hdr); - var dbuf = new byte[dlen]; - Array.Copy(chunk, hdr, dbuf, 0, dlen); - File.WriteAllBytes(Path.Combine(Path.GetTempPath(), $"grep7d7_{eid}_{classIdx:X8}_{hdr}.bin"), dbuf); - } - catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) - { - DiagnosticSink.WriteLine($"Block dump failed: {ex.Message}"); - } + int hdr = dataStart - 16; + int dlen = Math.Min(dataEnd - hdr, chunk.Length - hdr); + var dbuf = new byte[dlen]; + Array.Copy(chunk, hdr, dbuf, 0, dlen); + File.WriteAllBytes(Path.Combine(Path.GetTempPath(), $"grep7d7_{eid}_{classIdx:X8}_{hdr}.bin"), dbuf); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) + { + DiagnosticSink.WriteLine($"Block dump failed: {ex.Message}"); } - // Production geometry is decoded only through the declared Tf schema: - // a pattern scan cannot confirm field ownership. - pendingAll.Add(new PendingGRep(chunk, eid, dataStart - 16, dataSize, false)); } + // Production geometry is decoded only through the declared Tf schema: + // a pattern scan cannot confirm field ownership. + pendingAll.Add(new PendingGRep(chunk, eid, dataStart - 16, dataSize, false)); } - - pos += 16 + (int)dataSize; } - if (deferred == null) ApplyPendingGReps(state, pendingAll); - return blockCount > 0; + + return true; } /// Merges one decoded GRep block into the element's geometry (arrival-order rules). diff --git a/src/RvtIO/Parsing/PartitionFraming.cs b/src/RvtIO/Parsing/PartitionFraming.cs index 1eb57d8..f7da39c 100644 --- a/src/RvtIO/Parsing/PartitionFraming.cs +++ b/src/RvtIO/Parsing/PartitionFraming.cs @@ -38,7 +38,35 @@ internal static class PartitionFraming /// A partition stream read by its framing: the output of every segment in file order, and the records of the project's own block /// (the null-key map) in file order, without the chain terminators. The records of the keyed blocks (loaded family documents) are not listed. /// is the byte width of the owner the walk read. - internal sealed record Framed(IReadOnlyList Segments, List Items, int IdWidth); + internal sealed record Framed(IReadOnlyList Segments, List Items, int IdWidth) + { + private long[]? starts; + + /// Where the bytes of the item are: in its segment when it lies in one, else in a copy (an item can run into the next segment + /// of its chain); Copied tells which. + internal (byte[] Data, int Start, bool Copied) Bytes(ProjectItem item) + { + if (starts == null) + { + var at = new long[Segments.Count]; + for (int i = 1; i < at.Length; i++) at[i] = at[i - 1] + Segments[i - 1].Length; + starts = at; + } + int s = Array.BinarySearch(starts, item.Offset); + if (s < 0) s = ~s - 1; + while (s < starts.Length - 1 && starts[s + 1] <= item.Offset && Segments[s].Length == 0) s++; + int from = (int)(item.Offset - starts[s]); + if (from + (long)item.Length <= Segments[s].Length) return (Segments[s], from, false); + var copy = new byte[item.Length]; + for (int done = 0; done < item.Length; s++, from = 0) + { + int n = Math.Min(item.Length - done, Segments[s].Length - from); + Buffer.BlockCopy(Segments[s], from, copy, done, n); + done += n; + } + return (copy, 0, true); + } + } internal static Tags? TagsOf(FormatsSchema schema) { diff --git a/src/RvtIO/RvtFileReader.cs b/src/RvtIO/RvtFileReader.cs index 098ae47..c8f853f 100644 --- a/src/RvtIO/RvtFileReader.cs +++ b/src/RvtIO/RvtFileReader.cs @@ -104,6 +104,7 @@ public RevitFileInfo Read(string rvtPath) R22View = r22View, StaleCopiesRemoved = staleCopiesRemoved, FramedPartitions = framedStreams, + PartitionChunkCount = partitionStreams.Sum(s => s.Count), }; ReadWideElementIndex(root, state, partitionStreams, framedStreams, wideSchema, diag); ReadRebarBars(root, state, partitionStreams, framedStreams, wideSchema); @@ -872,6 +873,7 @@ internal static void MergeElemScanStringsInChunkOrder( R22View = r22View, StaleCopiesRemoved = staleCopiesRemoved, FramedPartitions = framedStreams, + PartitionChunkCount = partitionStreams.Sum(s => s.Count), }; ReadWideElementIndex(root, state, partitionStreams, framedStreams, wideSchema, diag); ReadRebarBars(root, state, partitionStreams, framedStreams, wideSchema); @@ -2137,39 +2139,41 @@ internal static void AddDriveCurveFromRewrittenBlock(TfBlockDeserializer.NativeP /// wall () is read from its root record once every root is counted. internal static void RecordFramedRoots(ParsingState state) { - var walls = new List<(byte[] Item, ushort Class, uint Owner)>(); + var walls = new List<(byte[] Data, int Start, uint Size, ushort Class, uint Owner)>(); foreach (var frame in state.FramedPartitions!) - { - var starts = new long[frame!.Segments.Count]; - for (int i = 1; i < starts.Length; i++) starts[i] = starts[i - 1] + frame.Segments[i - 1].Length; - foreach (var record in frame.Items) + foreach (var record in frame!.Items) { if (record.Channel != 102 || record.Length < 18 || record.Owner > uint.MaxValue) continue; // the item [owner:4][checksum:4][size:4][class:2 ...][size:4] is the block layout the walk hands over - var item = ItemBytes(frame.Segments, starts, record.Offset, record.Length); - ushort classId = BitConverter.ToUInt16(item, 12); - StairComponentSource.Record(state, item, 0, (uint)(item.Length - 16), classId, (uint)record.Owner); - if (classId is 3524 or 361) walls.Add((item, classId, (uint)record.Owner)); + var (data, start, _) = frame.Bytes(record); + ushort classId = BitConverter.ToUInt16(data, start + 12); + StairComponentSource.Record(state, data, start, (uint)(record.Length - 16), classId, (uint)record.Owner); + if (classId is 3524 or 361) walls.Add((data, start, (uint)(record.Length - 16), classId, (uint)record.Owner)); } - } - foreach (var (item, classId, owner) in walls) RecordWallLocationLine(state, item, 0, (uint)(item.Length - 16), classId, owner); + foreach (var (data, start, size, classId, owner) in walls) RecordWallLocationLine(state, data, start, size, classId, owner); } - /// The bytes at of the concatenated segments (an item can run into the next segment of its chain). - private static byte[] ItemBytes(IReadOnlyList segments, long[] starts, long offset, int length) + /// Hands the channel-102 and 103 records of the project's own block (the root objects and the + /// geometry of the elements, in their current copy) instead of the blocks a scan of the partition chunks finds: the scan also takes the blocks + /// of the documents of loaded families, whose ids repeat project ids, and it stops or jumps at a header-like byte run and at a block that runs + /// into the next chunk. + internal static void ParseFramedBlocks(ParsingState state, bool collectBlockOnlyIds, List pending) { - var item = new byte[length]; - int s = Array.BinarySearch(starts, offset); - if (s < 0) s = ~s - 1; - for (int done = 0; done < length; s++) - { - int from = (int)(offset + done - starts[s]), n = Math.Min(length - done, segments[s].Length - from); - Buffer.BlockCopy(segments[s], from, item, done, n); - done += n; - } - return item; + foreach (var frame in state.FramedPartitions!) + foreach (var record in frame!.Items) + { + if (record.Channel == 101 || record.Length < 18 || record.Owner is 0 or > uint.MaxValue) continue; + var (data, start, copied) = frame.Bytes(record); + uint eid = (uint)record.Owner; + int queued = pending.Count; + BlockParser.ParseBlock(data, start, eid, (uint)(record.Length - 16), state.AllKnownIds.Contains(eid), state, collectBlockOnlyIds, pending); + // shortcut: the geometry of a record that runs into the next segment or is over the size limit of the scan is not decoded (the + // scan never reached such a block, and the decoder takes minutes on some of them: 28 to 180 s each for ten 0.7 to 0.9 MB geometry + // records of one model, none of which changed the output); decode it once the geometry decoder is linear in the block size. + if ((copied || record.Length - 16 > BlockParser.MaxBlockSize) && pending.Count > queued) + pending.RemoveRange(queued, pending.Count - queued); + } } - /// Continuous-stream anchored block walk (see TfStreamWalker): native param /// stores + full-coverage re-run of block-driven signals (curve owners, node owners, /// registry categories, sketch/detail curves). @@ -2185,7 +2189,7 @@ private static void RunTfStreamWalk(ParsingState state) if (uint.TryParse(part, out uint dge)) dumpGrepEids.Add(dge); var walkGReps = new List<(byte[] Segment, int Start, uint Size, uint Eid)>(); // the roots the class and stair sources count: the records of the project's block when every stream is read by its framing - bool framedRoots = state.WideSchema == null && state.FramedPartitions is { Count: > 0 } framed && framed.All(f => f is { IdWidth: 4 }); + bool framedRoots = state.ReadsFramedRecords; if (framedRoots) RecordFramedRoots(state); TfStreamWalker.Walk(state.Chunks, state.AllKnownIds, (seg, bs, ds, cls, eid) => { @@ -2497,11 +2501,16 @@ private static void ParseBlocks(ParsingState state, TextWriter? diag) BlockParser.CollectBipStrings(chunk, state); Lap("bip strings"); + // a file read by its framing: the blocks of the partitions are its records, the chunks of Formats and Global are scanned + var partitionChunks = state.ReadsFramedRecords + ? state.Chunks.Take(state.PartitionChunkCount).ToHashSet(ReferenceEqualityComparer.Instance) : null; + bool Scanned(byte[] chunk) => partitionChunks?.Contains(chunk) != true; // First pass: parse blocks, collect block-only IDs var firstPassGReps = new List(); - foreach (var chunk in state.NonRecordChunks) + if (partitionChunks != null) ParseFramedBlocks(state, collectBlockOnlyIds: true, firstPassGReps); + foreach (var chunk in state.NonRecordChunks.Where(Scanned)) BlockParser.ParseChunk(chunk, state, collectBlockOnlyIds: true, deferred: firstPassGReps); - foreach (var chunk in state.RecordChunks) + foreach (var chunk in state.RecordChunks.Where(Scanned)) BlockParser.ParseChunk(chunk, state, collectBlockOnlyIds: true, deferred: firstPassGReps); BlockParser.ApplyPendingGReps(state, firstPassGReps); diag?.WriteLine($"Block-only elements: {state.BlockOnlyIds.Count}"); @@ -2513,9 +2522,10 @@ private static void ParseBlocks(ParsingState state, TextWriter? diag) foreach (var bid in state.BlockOnlyIds) state.AllKnownIds.Add(bid); var secondPassGReps = new List(); - foreach (var chunk in state.NonRecordChunks) + if (partitionChunks != null) ParseFramedBlocks(state, collectBlockOnlyIds: false, secondPassGReps); + foreach (var chunk in state.NonRecordChunks.Where(Scanned)) BlockParser.ParseChunk(chunk, state, deferred: secondPassGReps); - foreach (var chunk in state.RecordChunks) + foreach (var chunk in state.RecordChunks.Where(Scanned)) BlockParser.ParseChunk(chunk, state, deferred: secondPassGReps); BlockParser.ApplyPendingGReps(state, secondPassGReps); } diff --git a/tests/RvtIO.Tests/ApiTests.cs b/tests/RvtIO.Tests/ApiTests.cs index c10a5af..b8c6139 100644 --- a/tests/RvtIO.Tests/ApiTests.cs +++ b/tests/RvtIO.Tests/ApiTests.cs @@ -1069,8 +1069,9 @@ public void InvestigateNameStoragePattern() [SkippableFact] public void SearchBetonString() { - Assert.Equal(45, _doc.Elements.Count(e => e.Name != null && e.Name.Contains("Бетон"))); - Assert.Equal(54, _doc.Elements.Count(e => e.Name != null && e.Name.Contains("300"))); + // both counts include id 172544, a load case by its header that Revit does not return as an element + Assert.Equal(46, _doc.Elements.Count(e => e.Name != null && e.Name.Contains("Бетон"))); + Assert.Equal(55,_doc.Elements.Count(e => e.Name != null && e.Name.Contains("300"))); var material = _doc.GetElement(100935)!; Assert.Equal("OST_Materials", material.Category); Assert.Equal("Бетон", material.Name); @@ -1899,7 +1900,8 @@ public void CheckWall290112() public void DeepInvestigate100907() { Assert.Equal(21, Enumerable.Range(100900, 21).Count(id => _doc.GetElement(id) != null)); - Assert.Equal([100900, 100901, 100902, 100903, 100904, 100905, 100906, 100907, 159004, 159005, 159006, 159007, 290112], + // 265364, a revision cloud by its header, is not an element Revit returns + Assert.Equal([100900, 100901, 100902, 100903, 100904, 100905, 100906, 100907, 159004, 159005, 159006, 159007, 265364, 290112], _doc.Elements.Where(e => e.Name != null && e.Name.Contains("Трехслойные")).Select(e => e.Id).OrderBy(id => id)); Assert.Equal(100907, _doc.GetElement(290112)!.TypeId); Assert.Null(_doc.GetElement(527)); diff --git a/tests/RvtIO.Tests/OwnElementClassTests.cs b/tests/RvtIO.Tests/OwnElementClassTests.cs index 4245016..0d26eb5 100644 --- a/tests/RvtIO.Tests/OwnElementClassTests.cs +++ b/tests/RvtIO.Tests/OwnElementClassTests.cs @@ -119,6 +119,54 @@ public void TheRootOfAnElement_IsItsRecordInTheProjectBlock_NotACopyOfItsIdInAFa Assert.Equal(new StairComponentSource.ElementClassRoot(2, null), walked.OwnedElementClassRoots[123]); } + [Fact] + public void TheBlockParser_ReadsTheRecordsOfTheProjectBlock_EvenAcrossSegments_AndNotTheBlocksOfAFamilyDocument() + { + // the project's record of element 123 runs into the next segment; behind it a block of element 456 in the document of a loaded + // family, which the framing does not list as a record of the project + byte[] record = Frame(3708), family = Frame(157); + BitConverter.GetBytes(456u).CopyTo(family, 0); + byte[] stream = [.. record, .. family]; + var state = new ParsingState + { + FramedPartitions = [new PartitionFraming.Framed([stream[..10], stream[10..]], [new(102, 123, 0, record.Length)], 4)], + }; + + RvtFileReader.ParseFramedBlocks(state, collectBlockOnlyIds: true, []); + + Assert.Equal(1, state.BlockClassInventory[3708].Count); + Assert.False(state.BlockClassInventory.ContainsKey(157)); + Assert.Equal([123u], state.BlockOnlyIds); + } + + [Fact] + public void TheGeometryOfARecordThatRunsIntoTheNextSegmentOrIsOverTheSizeLimit_IsNotDecoded() + { + byte[] crossing = Frame(0x07D7), whole = Frame(0x07D7); + BitConverter.GetBytes(456u).CopyTo(whole, 0); + // [owner][checksum][size][class][data][size] with data over the limit of the scan + uint size = BlockParser.MaxBlockSize + 4; + byte[] large = new byte[16 + size]; + BitConverter.GetBytes(789u).CopyTo(large, 0); + BitConverter.GetBytes(size).CopyTo(large, 8); + BitConverter.GetBytes((ushort)0x07D7).CopyTo(large, 12); + BitConverter.GetBytes(size).CopyTo(large, 16 + (int)size - 4); + byte[] stream = [.. crossing, .. whole, .. large]; + var state = new ParsingState + { + AllKnownIds = [123, 456, 789], + FramedPartitions = [new PartitionFraming.Framed([stream[..10], stream[10..]], + [new(103, 123, 0, crossing.Length), new(103, 456, crossing.Length, whole.Length), + new(103, 789, crossing.Length + whole.Length, large.Length)], 4)], + }; + var pending = new List(); + + RvtFileReader.ParseFramedBlocks(state, collectBlockOnlyIds: true, pending); + + Assert.Equal(3, state.BlockClassInventory[0x07D7].Count); + Assert.Equal([456u], pending.Select(p => p.Eid)); + } + // TESTFILE.rvt is saved in five increments; reading it with and without the removal of the superseded copies changes the records around // the elements that one partition holds, but not their roots. [SkippableFact]