Write the bars of a rebar system that holds an extensible storage entity

CHANGELOG.md

@@ -56,7 +56,7 @@ API between minor releases.
 
 ### Changed
 
-- Default output, IFC and JSON: the readers behind the output read the values of an extensible storage entity by the schema of that entity, as `rvtio coverage` does, so the objects behind those values are read in their place instead of out of step, and the record of an element that keeps an entity is read to its end. An entity whose schema the file does not hold, or with a field type that is not read, is passed as before. What lies behind an entity is still left out where it was before (the parameters a store behind it would list as unset, the bars of a rebar system that holds one): those rules were made when the values could not be read, and what they would give now is not measured. Measured on 28 project files of the releases 2020 to 2026 against the output of the main branch, and for the IFC also against the reference: no IFC value became worse. On eight MEP files the in/out direction and the name of 887 distribution ports are now those of the reference (331, 331, 83, 83, 25, 25, 8 and 1), and on ten MEP files 3,221 assignments of a port or an element to a system are added that the reference has; 48 added assignments are not in the reference at that system (in four files; not checked in Revit). 6 systems the reference has are written that were missing; 26 pipe fitting types that the reference does not have are no longer written, and 9 others it does not have either are (types of a single fitting each, whose `GlobalId` changes with the parameters now read). In two files of one architectural project 6 reinforcing bars each are written that were left out, with the `GlobalId` of the reference. In the compact JSON 1,214 family instances of that project (three files) and 3,086 of another (one file) get the parameter list that the other instances of their categories already had, and 53 family types of one MEP file get theirs. Against Revit, on 172 of the 1,214 in the file of release 2021, every value the compact JSON writes as Revit shows it (type, mark, comments, phases, heights, levels, offsets and others, 1,561 values) is equal, and the fields the compact JSON writes in its own form (family, type and host as names, the design option as `None`, justifications as numbers, an empty level as `None`) are written as for 170 instances of the same categories that had their list before. The IFC of 16 of the 28 files and the JSON of 23 do not change.
+- Default output, IFC and JSON: the readers behind the output read the values of an extensible storage entity by the schema of that entity, as `rvtio coverage` does, so the objects behind those values are read in their place instead of out of step, and the record of an element that keeps an entity is read to its end. An entity whose schema the file does not hold, or with a field type that is not read, is passed as before. The parameters that a store behind an entity would list as unset are still left out, as before; what that rule would give now is not measured. The bars of a rebar system that holds an entity are read now as well. A RebarInSystem that is the one RebarInSystem of its system takes the `GlobalId` of the system, as in the reference; the bar of a system with more keeps its own sub-element `GlobalId` (the number of its bars plus 3). Measured on 28 project files of the releases 2020 to 2026 against the output of the main branch, and for the IFC also against the reference: no IFC value became worse. On eight MEP files the in/out direction and the name of 887 distribution ports are now those of the reference (331, 331, 83, 83, 25, 25, 8 and 1), and on ten MEP files 3,221 assignments of a port or an element to a system are added that the reference has; 48 added assignments are not in the reference at that system (in four files; not checked in Revit). 6 systems the reference has are written that were missing; 26 pipe fitting types that the reference does not have are no longer written, and 9 others it does not have either are (types of a single fitting each, whose `GlobalId` changes with the parameters now read). In three files of one architectural project (releases 2021, 2023 and 2024) 42, 48 and 48 reinforcing bars that were left out are written, all with the `GlobalId` of the reference (in the file of release 2024, 26 of the 42 bars of its systems are the one bar of their system); the three files now write 159, 168 and 168 of the 169 bars of the reference (117, 120 and 120 before). The one bar of a system is not put into a group of its system, which it stands for. In the compact JSON 1,214 family instances of that project (three files) and 3,086 of another (one file) get the parameter list that the other instances of their categories already had, and 53 family types of one MEP file get theirs. Against Revit, on 172 of the 1,214 in the file of release 2021, every value the compact JSON writes as Revit shows it (type, mark, comments, phases, heights, levels, offsets and others, 1,561 values) is equal, and the fields the compact JSON writes in its own form (family, type and host as names, the design option as `None`, justifications as numbers, an empty level as `None`) are written as for 170 instances of the same categories that had their list before. The IFC of 16 of the 28 files and the JSON of 23 do not change.
 - The repository moved to the IOExtend organization. The project and repository URLs of the packages, the links in the documentation and the address of the documentation site (<https://ioextend.github.io/RvtIO/>) name the new place. The old repository address redirects to the new one; the old address of the documentation site does not.
 - `rvtio coverage` reads the values of an extensible storage entity by the schema of that entity. `ESEntity.m_blob` is an owned pointer whose class tag is not written: a handle of 0 is no entity and nothing follows; behind 0xFFFFFFFF stands the 16-byte GUID of the entity's schema where the class tag would be, and the values are queued like any owned object, so they lie behind every object queued earlier and in front of the objects queued later. The values are the fields of that schema in the order of `ESField.m_entryIndex` (the schema lists them by name), each one value, a count and the values of an array, or a count and the pairs of a map, of the type `ESField.m_fieldTypeName` names: a text, a scalar, or a class of the class schema read inline, among them an entity, which queues its own values. The entity schemas are read from `Global/Latest`, an `ADocument` that the same walk reads: `ESSchemaStorage.m_storedSchemas` in the class schemas up to release 2024, `ESSchemaStorage.m_schemaUsageMap` from release 2025. Before, the walk passed an entity as one without values, read the objects behind the values out of step and named the stop after whatever class came next. On 52 project files of the releases 2020 to 2027 the records that are not exact go from 27,847 to 117 of 29,794,710, and every `FamilyInstance` record of the project blocks is exact (24,044 of 131,261 were not); 154 class rows have more exact records, in files of every release but 2026, whose one file gains nothing, and no class row of any channel of any file has fewer. Left are 72 records in the documents of loaded families whose entity names a schema that `Global/Latest` of the project does not hold (the new reason `entity schema not stored`) and 45 records that stop at the 100,000 character limit of a text (4 of them inside the values of an entity). When the entity schemas of a file cannot be read, every entity with values stops its record with `entity schema not stored` instead of being passed; a handle other than 0 and 0xFFFFFFFF stops with `entity handle`, and a schema with a field the walk does not read with `entity field type` (neither occurs in the 52 files). Some kinds of values are read by the same grammar although no record of those files holds one (`short`, `int64_t`, a single `char`, arrays of `bool`, `float` and `GUIDvalue`, and maps that occur only empty). The readers of the export are not given the entity schemas and read an entity as before; their output was not compared. `rvtio coverage <file> --trace <class|owner,...>` is new: a developer aid that prints the walk of single records (the queued objects with their offsets, the fields of the class the walk stopped in, the bytes around the stop). See [the coverage report](docs/coverage-report.md).
 

docs/test-fixtures.md

@@ -45,7 +45,7 @@ folder (`TestFiles.Scratch`); no test uses a fixed absolute path.
 
 ## What runs without the private files
 
-With no private file at all, 9672 of the 10090 test cases run and pass and 418 are skipped: 412 need private files (listed
+With no private file at all, 9673 of the 10091 test cases run and pass and 418 are skipped: 412 need private files (listed
 below; 17 of them only `bim_dump.txt` or `formats_latest_decompressed.bin`) and 6 need other inputs (two need symbolic links, three
 need `RVTIO_FREEFORM_FULL`, `RVTIO_CYL_FULL` or `RVTIO_MESH_FULL`, one is a known gap). The tests that run use synthetic data built
 in the tests and the small fixtures committed under `tests/RvtIO.Tests/Fixtures`. With the full private fixture folder the

src/RvtIO/Ifc/IfcExporter.Rebar.cs

@@ -42,12 +42,15 @@ private bool WriteRebarProduct(StepWriter w, Element elem, IReadOnlyDictionary<i
         string typeLabel = shapeInName && data.SystemId == 0 && data.ShapeName != null ? $"{barTypeName} : Shape {data.ShapeName}" : barTypeName;
         string elemName = $"{RebarFamilyLabel}:{typeLabel}:{elem.Id.ToString(CultureInfo.InvariantCulture)}";
         string objectType = data.SystemId > 0 ? barTypeName : $"{RebarFamilyLabel}:{barTypeName}";
-        // RebarInSystem: the sub-element index is the number of bars plus 3; a Rebar has none
-        int entityId = WriteRebarBar(w, elem, data, ownerHistId, axisId, parentPlacementId, bodySubCtxId, elemName, objectType, data.SystemId > 0 ? data.Bars.Count + 3 : 0);
+        // RebarInSystem: the sub-element index is the number of bars plus 3; a Rebar has none. The one RebarInSystem of a system has the
+        // GlobalId of the system.
+        Element? system = data.OnlyBarOfSystem ? elemById.GetValueOrDefault(data.SystemId) : null;
+        int entityId = WriteRebarBar(w, elem, data, ownerHistId, axisId, parentPlacementId, bodySubCtxId, elemName, objectType,
+            system != null ? 0 : data.SystemId > 0 ? data.Bars.Count + 3 : 0, globalIdOf: system);
         ExportQuantityTakeOffPropertySet(w, elem, entityId, ownerHistId, barTypeName, propertySingleValueIds);
         PlaceRebarInStorey(elem, data, elemById, entityId, storeyIds, defaultStoreyId, storeyContents);
-        // No group of their system is written for the bars of an assembly
-        if (data.SystemId > 0 && !data.InAssembly)
+        // No group of their system is written for the bars of an assembly, nor for the one bar of a system, which carries its GlobalId
+        if (data.SystemId > 0 && !data.InAssembly && system == null)
         {
             if (!_rebarSystemMembers.TryGetValue(data.SystemId, out var members)) _rebarSystemMembers[data.SystemId] = members = [];
             members.Add(entityId);
@@ -92,9 +95,10 @@ private bool WriteRebarContainer(StepWriter w, Element elem, RebarBars data, IRe
     }
 
     /// <summary>One IfcReinforcingBar of <paramref name="data"/> (its swept disks, placement and quantities), without property set and storey.
-    /// <paramref name="fallbackGlobalId"/> is its GlobalId when the element has no usable export id (the items of a container share their element).</summary>
+    /// <paramref name="fallbackGlobalId"/> is its GlobalId when the element has no usable export id (the items of a container share their element);
+    /// <paramref name="globalIdOf"/> is the element whose GlobalId the bar takes instead of its own (the system of its one RebarInSystem).</summary>
     private int WriteRebarBar(StepWriter w, Element elem, RebarBars data, int ownerHistId, int axisId, int parentPlacementId, int bodySubCtxId,
-        string elemName, string objectType, int subElementIndex, string? fallbackGlobalId = null)
+        string elemName, string objectType, int subElementIndex, string? fallbackGlobalId = null, Element? globalIdOf = null)
     {
         var items = new List<string>();
         double firstBarLengthFt = 0;
@@ -117,7 +121,7 @@ private int WriteRebarBar(StepWriter w, Element elem, RebarBars data, int ownerH
         double nominalMm = data.NominalDiameter * FeetToMillimetres;
         double area = Math.PI * Math.Pow(nominalMm / 2000, 2);
 
-        string globalId = ResolveExactElementGlobalId(elem, _usedExportIds, subElementIndex, CurrentProfile) ?? fallbackGlobalId ?? IfcGuid.FromElementId(elem.Id);
+        string globalId = ResolveExactElementGlobalId(globalIdOf ?? elem, _usedExportIds, subElementIndex, CurrentProfile) ?? fallbackGlobalId ?? IfcGuid.FromElementId(elem.Id);
         return w.Add("IFCREINFORCINGBAR",
             $"{S(globalId)},{R(ownerHistId)},{S(elemName)},{S("")},{S(objectType)},{R(placementId)},{R(shapeId)},{S(elem.Id.ToString(CultureInfo.InvariantCulture))},"
             + $"{S("")},{D(nominalMm)},{D(area)},{D(barLengthMm)},.NOTDEFINED.,$");

src/RvtIO/Parsing/RebarReader.cs

@@ -45,10 +45,11 @@ private static double[] Point(double[] t, double[] p)
 /// <see cref="LengthRoundingMillimetres"/> is the total length rounding of the document (ReinforcementSettings, nearest), 0 for none;
 /// <see cref="InAssembly"/> is true for a bar that is a member of an assembly (AssemblyGroupMembership cell).
 /// A RebarContainer holds no bars of its own: <see cref="Items"/> are its items (one bar each, <see cref="ItemIndex"/> is the index of the item
-/// in the container) and <see cref="BarTypeName"/> is the name of the container type.</summary>
+/// in the container) and <see cref="BarTypeName"/> is the name of the container type.
+/// <see cref="OnlyBarOfSystem"/> is true for a RebarInSystem that is the one RebarInSystem of its system.</summary>
 internal sealed record RebarBars(int BarTypeId, int SystemId, double? BarLength, IReadOnlyList<IReadOnlyList<RebarSegment>> Bars,
     double NominalDiameter = 0, double ModelDiameter = 0, string? BarTypeName = null, double LengthRoundingMillimetres = 0, string? ShapeName = null,
-    bool InAssembly = false, IReadOnlyList<RebarBars>? Items = null, int ItemIndex = -1);
+    bool InAssembly = false, IReadOnlyList<RebarBars>? Items = null, int ItemIndex = -1, bool OnlyBarOfSystem = false);
 
 /// <summary>A beam system (JoistSystem): its type (JoistSystemAttr, <see cref="TypeName"/> from SymbolInfo.m_name) and the beams it holds
 /// (HeldBeam.m_existingBeamId, in stored order, deleted ones left out).</summary>
@@ -215,8 +216,7 @@ internal static ReinforcementData Read(
             else if (cls == system)
             {
                 if (ReadLayers(walker, walk, byQueue) is { } found) layers.TryAdd(id, found);
-                // a system behind an empty extensible storage entity stays unread, as in ReadLayers
-                if (heldLayout && walk.EmptyEntityAt < 0)
+                if (heldLayout)
                     foreach (var ((layer, index), bar) in ReadHeld(walk.Captures)) held.TryAdd((id, layer, index), bar);
             }
             else if (ReadMember(walker, walk, byQueue) is { } member) members.TryAdd(id, member);
@@ -244,11 +244,14 @@ internal static ReinforcementData Read(
             if (solvedItems.Count > 0 && containerNames.TryGetValue(typeId, out var typeName))
                 result.TryAdd(id, new RebarBars(-1, 0, null, [], BarTypeName: typeName, Items: solvedItems, InAssembly: inAssembly));
         }
+        var barsOfSystem = members.Values.GroupBy(member => member.System).ToDictionary(group => group.Key, group => group.Count());
         foreach (var (id, (systemId, layerIndex, index, inAssembly)) in members)
         {
+            bool only = barsOfSystem[systemId] == 1;
             if (held.TryGetValue(((uint)systemId, layerIndex, index), out var heldBar))
             {
-                if (Solve(heldBar) is { } solvedHeld) result.TryAdd(id, solvedHeld with { SystemId = systemId, ShapeName = null, InAssembly = inAssembly });
+                if (Solve(heldBar) is { } solvedHeld)
+                    result.TryAdd(id, solvedHeld with { SystemId = systemId, ShapeName = null, InAssembly = inAssembly, OnlyBarOfSystem = only });
                 continue;
             }
             if (!layers.TryGetValue((uint)systemId, out var found) || layerIndex < 0 || layerIndex >= found.Count) continue;
@@ -257,7 +260,7 @@ internal static ReinforcementData Read(
             var cut = layer.Cuts != null && index >= 0 && index < layer.Cuts.Count && diameters.TryGetValue(layer.BarTypeId, out var barDiameters)
                 ? CutBars(layer.Lines, layer.Cuts[index], barDiameters.Model / 2) : null;
             if (cut == null && (index < 0 || index >= layer.Instances.Count)) continue;
-            result.TryAdd(id, new RebarBars(layer.BarTypeId, systemId, null, cut ?? layer.Instances[index], InAssembly: inAssembly));
+            result.TryAdd(id, new RebarBars(layer.BarTypeId, systemId, null, cut ?? layer.Instances[index], InAssembly: inAssembly, OnlyBarOfSystem: only));
         }
         // the diameters of the bar types come with the pass, so they are attached once every element was read
         foreach (var (id, bars) in result.ToList())
@@ -433,9 +436,6 @@ private static bool InAssembly(WideBlockWalker walker, WideBlockWalker.Walk walk
 
     internal static List<Layer>? ReadLayers(WideBlockWalker walker, WideBlockWalker.Walk walk, Dictionary<int, List<WideBlockWalker.Capture>> byQueue)
     {
-        // The bars of a system behind an empty extensible storage entity are not written yet: their GlobalId and bar count are not
-        // understood (the walk used to stop at the entity, so these systems were never read).
-        if (walk.EmptyEntityAt >= 0) return null;
         if (!byQueue.TryGetValue(-1, out var root) || walk.QueueCountAtStart.Count == 0) return null;
         // the layers are consecutive fields of the root; per layer the capture groups open with m_barType
         var groups = new List<List<WideBlockWalker.Capture>>();

tests/RvtIO.Tests/RebarInSystemMemberTests.cs

@@ -5,7 +5,7 @@
 
 namespace RvtIO.Tests;
 
-/// <summary>The layers of a RebarSystem behind an empty extensible storage entity are not read: the bars of its RebarInSystem members are not written yet.</summary>
+/// <summary>The layers of a RebarSystem are read whether or not an extensible storage entity lies in front of them.</summary>
 public class RebarInSystemMemberTests
 {
     private const int IdentifierId = 1, ElementIdId = 2, GuidId = 3, PairId = 4, EntityId = 5, CellId = 6, CurveId = 7, LineId = 8, LayerId = 9, SystemId = 10;
@@ -61,5 +61,5 @@ private static SchemaField Field(string name, byte type, byte mods = 0, int? ref
     public void ReadLayers_ReadsASystemWithoutAnEntity() => Assert.Single(ReadLayers(emptyEntity: false)!);
 
     [Fact]
-    public void ReadLayers_SkipsASystemBehindAnEmptyEntity() => Assert.Null(ReadLayers(emptyEntity: true));
+    public void ReadLayers_ReadsASystemBehindAnEmptyEntity() => Assert.Single(ReadLayers(emptyEntity: true)!);
 }