Versioned links for tracking suspicious links

[danwos]

Versioned links concept

Goal

Find out linked objects of changed Sphinx-Needs objects between two timestamps.
Feature is often called suspicious links in some other tools.

Use case

Some existing requirements have changed between SW release A and B.
The test team needs to know which test cases needs to be rechecked, as the underlying requirement has changed.
Therefore for all changed requirements, their linked objects must be reported.

Requirements

• Two git commits (ids, tags, latest on branch) define the "time window".
• Found "suspicious objects" can be cleared/checked. Use case: Requirement got already anaylzed and no change on linked objects is needed.
• The output shall be a pdf/md/html report
• Solution must work on CI and locally.
• A rerun of the script with the same input data shall produce the same result.

Challenges

Two kind of suspicious links

There are two kind of suspicious links:

1. Parent object got changed with knowledge

In the first scenario, the user knows what he/she is doing. And can already decide if this change is changing something in the context or the statement of the parent object. So rephrasing the content is not a problem. In this scenario not all changed parent object lead to suspicious links.

Technical outlook:
This means, a change must be marked by the user, so that technically the object is flagged as "changed" and therefore checks for suspicious links must be performed.

2. Parent object got changed without knowledge

A change by accident, without knowing that something important had changed.
Important is, that not all changes on all options need to be checked.
For instance, a status change is ok.

Technical outlook:
This can be checked by comparing the same object in Version A and B for specified attributes.

Conclusion

Both scenarios need to be treated differently, as

1. Their result/outcome is different, and the one from 2) can be much bigger as the one from 1).

Clearing findings

When a suspicious object is found and corrected, this clearing of this finding must be documented somewhere. Otherwise, a rerun of the script will report it again.

Also the clearing should not be specific to the report, but to delta timestamps of its detection.

Example: Given release A, B, C, all created in a row.

Checking for suspicious links between B and C brings up one finding, which got resolved and cleared.

Checking now for A and C would bring up this link again, even as it got solved and checked for C.

Technical outlook:
To know if wanted changes (see scenario 1) from above) are resolved, this information must be part of the document / Sphinx-Needs object itself.
This allows us to detect suspicious links with the current release (C) only. No matter if we want to have the delta from A or B, it doesn't matter (for scenario 1!) ).

Technical outcome

We separate the solution between wanted and unwanted changes.

Wanted changes -> Versioned links

The user is responsible for marking a Sphinx-Needs object as changed.
The script does no checks on content or options.

Changes are marked as Version in the Sphinx-Needs object itself.
A link contains the version and gets "suspicious" if it doesn't match

.. req: A requirement
   :id: REQ_001
   :version: 2

   Some content

.. spec: A spec
   :id: SPEC_001
   :version: 1
   :solves: REQ_001:2        <-- Version added

   Version for link can be set by "LINK:VERSION"

.. spec: Another spec
   :id: SPEC_002
   :version: 3
   :solves: REQ_001:1        <-- Links to old version 

   Some content

.. spec: Another spec
   :id: SPEC_004
   :version: 1
   :related: REQ_001         <-- No version

   Links without version are still allowed

As alternative, the object version can be already part of the need-id:

.. req: A requirement
   :id: REQ_001:2   # instead :version: 2

Benefits

• Detection happens on current documentation only (One build)
• Clearing happens on current documentation only (No diff reports/clearing between different versions)
• Feedback to developer can be given during coding (IDE extension)
• Suspicious reporting can be part of the current Sphinx build
• Backwards compatible, as no version in links is still allowed
• Constraints can be defined, that links of type X must always have a specified version.

Drawback

• Maintaining the version and link-version is extra work

Unwanted changes

Out of scope for Sphinx-Needs, as this needs to happen outside a Sphinx build because access to data from different git commits/tags is needed.

[chrisjsewell]

Thanks @danwos I will have a look more in time

As alternative, the object version can be already part of the need-id

I just wanted to comment first, since I saw this presented yesterday:
this is an absolute no for me, I would not support this; having a single string encapsulate multiple pieces of information is always problematic to parse