Describe split of rule definitions between osi and osi-validation

[jdsika]

• OSI Validator checks the compliance of OSI messages with predefined rules
• OSI defines a set of rule predicates here
• OSI uses the rule predicates to define a rule set embedded into the .proto definition:

    // The interface version used by the sender (i.e. the simulation
    // environment).
    //
    // \rules
    // is_set
    // \endrules
    //
    optional InterfaceVersion version = 1;
  

• subject (InterfaceVersion) predicate (is_set) object ( implicitly true)
• OSI Validator may define a superset of rule predicates
• The mapping of rules must result in unique rule ids as seen in the qc-trace e.g. by concatination of subject_predicate_object
• Rule checking is implemented in osi-validation
• The rules are currently generated from the OSI .proto files with rules2yml.py and this may be part of the solution and shall be moved to osi
• From this the documentation may be automatically generated
• The mapping from the unique rule ids to an implementation in osi-validation requires to use the same keys to link it to an implementation
• Rules are generated can be customized by the user

Scope of osi-validation

The checking of rules which are centered on individual fields and individual messages which may include conditional logic regarding other fileds inside overall top level message.

Out of scope

• complex metrics
• calculations across time stamp (inter frame checks)
• thresholding and statistical evaluations
• rules that require additional parameter

Related issues:

• A description of the rules is missing from the documentation open-simulation-interface#786
• qc-osi: rule definitions open-simulation-interface#829

[ClemensLinnhoff]

General considerations and proposals:

• Composed rule IDs
  • Assign every rule a unique rule ID according to the schema: message_incl_name_space.field.rule_idx
  • example 1: SensorView.version.0
  • example 2: LidarSensorView.Reflection.signal_strength.0
  • example 3: RadarSensorView.Reflection.signal_strength.0
  • Idea behind using the full name space
    • Some messages are not unique, see Reflection example above.
    • The namespace (or parent message) is needed to identify the actual message.
  • Idea behind giving each rule of one field an index
    • Give consecutive Integer index to the rules of one field.
    • Removes complexity from identifying conditional rules.
    • Explicitly add indices to the rules in the proto files / the documentation.
• Generate one rule file per proto file (similar to rules2yml)
  • Only include messages and fields that actually have rules.
  • Map unique rule ID to rule definition.
  • Example: SensorView.version.0: is_set
• Challenges
  • Currently the rule predicate refers_to has a message as a value. The problem is, that the rule actually should not refer to a message but to a specific field. Just from the message name it is not clear, which field is meant, because a message can be used in multiple fields.
  • Suggestion: Give full path to the field beginning from the lowest top-level message, e.g. instead of just referring toLogicalLane, refer to GroundTruth.logical_lane (top-level_message.field_string).

What do you think @jdsika, @pmai