Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions _approaches/inconsistency-detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,20 @@ repositories:
url: https://github.com/ardoco/DetectingInconsistenciesInSoftwareArchitectureDocumentationUsingTraceabilityLinkRecovery
---

> **Note:** As of ARDoCo V2, the terminology for inconsistency types has been updated for clarity:
>
> - **Text Entity Absent from Model (TEAM)** - formerly "Missing Model Elements (MME)"
> - **Model Entity Absent from Text (MEAT)** - formerly "Unmentioned/Undocumented Model Elements (UME)"

![Approach Overview](/assets/img/approaches/icsa23-inconsistency.svg){:width="100%" style="background-color: white; border-radius: 8px; padding: 10px; display: block; margin: 0 auto;"}

The ArDoCo inconsistency detection approach uses trace link recovery to detect inconsistencies between natural-language architecture documentation and formal models.
It identifies two kinds of issues:

(a) Unmentioned Model Elements (UMEs): components or interfaces that appear in the model but are never described in the documentation;
(b) Missing Model Elements (MMEs): elements mentioned in the text that do not exist in the model.
(a) **Model Entity Absent from Text (MEAT)**: components or interfaces that appear in the model but are never described in the documentation;
(b) **Text Entity Absent from Model (TEAM)**: elements mentioned in the text that do not exist in the model.

The method runs a TLR procedure (namely SWATTR) and then flags any model element with no corresponding text link (a UME) or any sentence that refers to a non-modeled item (an MME).
The method runs a TLR procedure (namely SWATTR) and then flags any model element with no corresponding text link (a MEAT) or any sentence that refers to a non-modeled item (a TEAM).

- Detection strategy: Use the TLR results as a bridge. After linking as many sentences to model elements as possible, any "orphan" model nodes or text mentions indicate a consistency gap. For example, if the model has a "Cache" component with no sentence linked, that is a UME; if the doc talks about "Common" but the model lacks it, that is an MME.
- Results: The approach achieved an excellent F1 (0.81) for the underlying trace recovery. For inconsistency detection, it attained ~93% accuracy in identifying UMEs and ~75% for MMEs, significantly better than naive baselines. These results suggest that using trace links is a promising way to find documentation-model mismatches.
- Detection strategy: Use the TLR results as a bridge. After linking as many sentences to model elements as possible, any "orphan" model nodes or text mentions indicate a consistency gap. For example, if the model has a "Cache" component with no sentence linked, that is a MEAT; if the doc talks about "Common" but the model lacks it, that is a TEAM.
- Results: The approach achieved an excellent F1 (0.81) for the underlying trace recovery. For inconsistency detection, it attained ~93% accuracy in identifying MEAT and ~75% for TEAM, significantly better than naive baselines. These results suggest that using trace links is a promising way to find documentation-model mismatches.
2 changes: 2 additions & 0 deletions _conferences/icsa23.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ additional_presentations:

![Inconsistency Detection Overview](/assets/img/approaches/icsa23-inconsistency.svg){:width="100%" style="background-color: white; border-radius: 8px; padding: 10px; display: block; margin: 0 auto;"}

> **Terminology Update (V2):** This paper uses the original terminology "Missing Model Elements (MME)" and "Unmentioned Model Elements (UME)". As of ARDoCo V2, these have been renamed to **Text Entity Absent from Model (TEAM)** and **Model Entity Absent from Text (MEAT)** respectively for improved clarity.
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The terminology note states "Unmentioned Model Elements (UME)" but the PR description and the _approaches/inconsistency-detection.md file consistently use "Undocumented Model Elements (UME)". This should be updated to "Undocumented Model Elements (UME)" for consistency.

Suggested change
> **Terminology Update (V2):** This paper uses the original terminology "Missing Model Elements (MME)" and "Unmentioned Model Elements (UME)". As of ARDoCo V2, these have been renamed to **Text Entity Absent from Model (TEAM)** and **Model Entity Absent from Text (MEAT)** respectively for improved clarity.
> **Terminology Update (V2):** This paper uses the original terminology "Missing Model Elements (MME)" and "Undocumented Model Elements (UME)". As of ARDoCo V2, these have been renamed to **Text Entity Absent from Model (TEAM)** and **Model Entity Absent from Text (MEAT)** respectively for improved clarity.

Copilot uses AI. Check for mistakes.

## Abstract

Documenting software architecture is important for a system’s success. Software architecture documentation (SAD) makes information about the system available and eases comprehensibility. There are different forms of SADs like natural language texts and formal models with different benefits and different purposes. However, there can be inconsistent information in different SADs for the same system. Inconsistent documentation then can cause flaws in development and maintenance. To tackle this, we present an approach for inconsistency detection in natural language SAD and formal architecture models. We make use of traceability link recovery (TLR) and extend an existing approach. We utilize the results from TLR to detect unmentioned (i.e., model elements without natural language documentation) and missing model elements (i.e., described but not modeled elements). In our evaluation, we measure how the adaptations on TLR affected its performance. Moreover, we evaluate the inconsistency detection. We use a benchmark with multiple open source projects and compare the results with existing and baseline approaches. For TLR, we achieve an excellent F1-score of 0.81, significantly outperforming the other approaches by at least 0.24. Our approach also achieves excellent results (accuracy: 0.93) for detecting unmentioned model elements and good results for detecting missing model elements (accuracy: 0.75). These results also significantly outperform competing baselines. Although we see room for improvements, the results show that detecting inconsistencies using TLR is promising.
Loading