Printing mdBook SRS to different files / Generating multi-file software artifacts

[BilalM04]

Tickets to revisit after this discussion is resolved:

• Generate .bib file name from SRS TeX file #388

---

Currently, the way printing SRS documents are setup, it is only possible to print to a single file. Each example makes a call to the gen function in Language.Drasil.Generate, passing in the SRS, which is a Document:

Drasil/code/drasil-example/projectile/app/Main.hs

Line 19 in bfcf92f

gen (DocSpec (docChoices SRS [HTML, TeX, JSON, Markdown]) "Projectile_SRS") srs printSetting

Then, through a series of functions in Language.Drasil.Generate, particularly prnt, prntDoc, printDoc', and writeDoc, a call to a format specific gen function is made:

Drasil/code/drasil-gen/lib/Language/Drasil/Generate.hs

Lines 93 to 101 in bfcf92f

	-- | Renders the documents.
	writeDoc :: PrintingInformation -> DocType -> Format -> Filename -> Document -> Doc
	writeDoc s _ TeX _ doc = genTeX doc s
	writeDoc s _ HTML fn doc = genHTML s fn doc
	writeDoc s dt JSON _ doc = genJSON s dt doc
	writeDoc s _ Markdown _ doc = genMD s doc
	writeDoc _ _ _ _ _ = error "we can only write TeX/HTML/JSON (for now)"

These format specific gen functions (genHTML, genTeX, genJSON, genMD) return a single Doc, which contains the entire SRS. This Doc is then printed to its respective SRS file.

The format specific gen functions receive a single Document, which contains all the SRS details. The Document it receives is in the format of the data type defined in Language.Drasil.Document, where the SRS contents is represented through [Section]. The format specific gen functions, using the makeDocument function, then convert it to the Document data type defined in Language.Drasil.Printing.LayoutObj, where the SRS content is represented through [LayoutObj]. The SRS printers then render these LayoutObj's into a single Doc. LayoutObj is defined as follows:

Drasil/code/drasil-printers/lib/Language/Drasil/Printing/LayoutObj.hs

Lines 30 to 43 in bfcf92f

	data LayoutObj =
	Table Tags [[Spec]] Label Bool Caption -- ^ Holds all information needed for a table.
	| Header Depth Title Label -- ^ Holds all information needed for a header.
	| Paragraph Contents -- ^ Paragraph.
	| EqnBlock Contents -- ^ Equation block.
	| Definition DType [(String,[LayoutObj])] Label -- ^ Definition. Holds the type, contents, and a label.
	| List ListType -- ^ List.
	| Figure Label Caption Filepath MaxWidthPercent -- ^ Holds all information needed for a figure.
	| Graph [(Spec, Spec)] (Maybe Width) (Maybe Height) Caption Label -- ^ Holds all information needed for a graph.
	| CodeBlock Contents -- ^ Code block.
	| HDiv Tags [LayoutObj] Label -- ^ Holds tags, more contents, and a label.
	| Cell [LayoutObj]
	-- this shouldn't be here, it should have been expanded.
	| Bib BibRef -- ^ Bibliography section.

---

The description above is how Drasil currently generates SRS documents (rather my understanding of it, please correct me if I'm wrong). The problem here is that the SRS is treated as a single document all throughout its journey from conception to being printed.

After playing around with the code for printing, I found the general structure of the SRS contents (represented as [LayoutObj]) follows a nested structure. The top-level LayoutObjs in the list are HDivs for each of the primary sections (closed dot in the TOC). The HDivs contain [LayoutObj], which is used to nest the sub-sections (open dot in the TOC). Similarly. the sections denoted by the square in the TOC are also nested.

Here are the table of contents for reference

• Table of Contents
• Reference Material
  • Table of Units
  • Table of Symbol
  • Abbreviations and Acronyms
• Introduction
  • Purpose of Document
  • Scope of Requirements
  • Characteristics of Intended Reader
  • Organization of Document
• General System Description
  • System Context
  • User Characteristics
  • System Constraints
• Specific System Description
  • Problem Description
    • Terminology and Definitions
    • Physical System Description
    • Goal Statements
  • Solution Characteristics Specification
    • Assumptions
    • Theoretical Models
    • General Definitions
    • Data Definitions
    • Instance Models
    • Data Constraints
    • Properties of a Correct Solution
• Requirements
  • Functional Requirements
  • Non-Functional Requirements
• Traceability Matrices and Graphs
• Values of Auxiliary Constants
• References

From my understanding, some HDivs, regardless of whether it is nested or not, should be printed into separate files. Whereas there are some HDivs that do not need to be printed to a separate file, i.e. the ones containing block equations. However, this nested structure makes it difficult to separate the SRS into different sections. I can think of several "hacky" ways to go about it; however being new to Drasil and Haskell, I am unsure of a good design.

---

What is the best way to print the mdBook SRS into separate Documents?

[JacquesCarette]

You're probably going to need to generalize this to (at least) return a 'set' of documents. It might be clever to put in a bit of abstraction now. I'm guessing that a Map from some reasonable set of 'key's to Docs might be a good first implementation.

You might want to take a peek at drasil-code and drasil-GOOL (with some help from @B-rando1 ) as it deals with many files already, so its architecture may be something to learn from.

In any case: this is a sub-optimal design of the gen functions -- we've just never needed to generate multiple files for documents!

[balacij]

Note: in my prototype, I did not yet build the Doc 'hole' feature out yet.

[balacij]

I think that document 'holes' are a reasonable feature to desire. It reminds me of LaTeX builds -- the reason why we run the LaTeX compiler multiple times is so that we can compile all of our floating lists and whatnot in early builds, so that later builds can make use of said information. However, that information is dependent on the fact that the latex has not changed.

[balacij]

In regard to #1025, would the stateful renderer (and Docs with 'hole' support) discussed above be helpful? I think so... but I'm not confident.

Our current approach involves searching our rendered chunks to figure out what things references other things, which is fair, but it is disconnected from how we render chunks, which is ultimately what we care about. Additionally, this feature of figuring out what things reference other things in a rendered document is probably a useful feature for all kinds of documents, not just our SRS. So it might be something that we prefer at the document-renderer level, so that if others decide to use the document language to render chunks, the feature is already built. This also extends to us: if we choose to generate a VnV, this feature comes for cheap.

[balacij]

This all being said: I need to upgrade my prototype if I want to convince anyone, with at least the following:

1. Document holes.
2. A system for building chunk renderers that automagically build (and record) chunk anchors.

[smiths]

Cross-file references are an important problem to solve. This won't be the last time they come up. When we eventually generate design and testing documents, we will need plenty of traceability between the documents. In particular, we will (in some sense) want everything to trace back to requirements. For LaTeX documents I've used the extern package for this in the past, but we are using hyperref for our internal links. A little googling suggests that hyperref can do external links to other files, although we would need to do more research to find if this is the best approach. Whatever solution we find for mdbooks will hopefully be consistent with cross-referencing between documents in other contexts in the future.

[smiths]

@BilalM04, I like your description above of how the SRS is generated. I'm guessing that you deduced this from reading the code, or maybe it is documented somewhere. If it isn't documented, we should talk about capturing the overview somewhere for the benefit of future Drasil developers. I'll add this discussion to our Monday meeting (#3791).