Questions about the keyword index

[Cicorione]

Hi @pgundlach I am trying to understand how to create a keyword index but I am confused about your method explained in the documentation.

You stated in the documentation:

The index is compiled differently in practice, of course. Since only the sorting is to be shown here, the keyword and the page number are given[...]

This means that I should pass the catalog two times, the first to create a record and a second time to "parse" the alphabetically? Or do I have to create the record before and the run the index (you wrote command makeindex for instance) separately from the main "catalog".

Is it possible saving the page and sorting in one pass since the index is usually at the end any publication?

Thanks in advance! 🙏

[pgundlach]

Not sure if I understand all aspects of the question.

I usually have a list of index items in a variable. For example:

<SetVariable variable="indexentries">
    <Copy-of select="$indexentries"/>
    <Element name="indexentry">
      <Attribute name="name" select="@word"/> ①
      <Attribute name="page" select="@page"/>
    </Element>
  </SetVariable>

This stores the equivalent of <indexentry name="..." page=".." /> in a variable $indexentries (and appends to the current value of the variable).

Depending on your needs, you have a few possibilities:

• if you go through your data, perhaps there is an important word or an article name which you need to appear in the index. Then use the method above.
• Perhaps you have already prepared a list of index entries in the data somewhere, but you don't know where these items appear, you can use <Mark> to insert invisible markers into the text and then go through this list index entries, ask for the page number and create a data structure as above. (similar to the first item in this list)

This surely depends on the data you have.

Once you have this variable with the data, call the command <Makeindex> and you have a sorted list. Then you can write this list to the hard drive or place this list in the PDF document.

There is no sorting and collecting the page in one pass.

[Cicorione]

Hi Patrick, the auot-index is a key piece in my strategy...

To recap:

1. I got that "There is no sorting and collecting the page in one pass" ; I'll do a double pass then.
2. I want extract the keywords for the index, so I am going to review the code you suggested.
3. I did not understand how call the command makeindex it is like: sp makeindex ?

Thanks... 🙏

[pgundlach]

1. I got that "There is no sorting and collecting the page in one pass" ; I'll do a double pass then.

unless you collect the data first and want the index at the end of the document, then you can use it in a single pass.

3. I did not understand how call the command makeindex it is like: sp makeindex ?

No, it is a command like PlaceObject.

So the “algorithm” would be something like this:

1. go through each chapter / article group / section and collect the data and page you want in the index (using the <SetVariable> code above).
2. The contents of the variable is now unsorted. and not grouped. So you need some kind of sorting and grouping command, and <MakeIndex> does exactly this. After calling <MakeIndex> you get a data structure similar to this (from the doc):

<index>
  <section name="E">
    <indexentry name="Elevator" page="4"/>
  </section>
  <section name="G">
    <indexentry name="Garage" page="2"/>
    <indexentry name="Giraffe" page="1"/>
    <indexentry name="Greeting" page="3"/>
  </section>
</index>

Now you can save the result in a variable and process the data. See https://github.com/speedata/examples/blob/2c24d4bd20277cb5afe68b183daa9416849eaad2/technical/index/layout.xml#L11 as an example.

[Cicorione]

Hi Patrick I am still studying this, I'd like to avoid to repeat the same stupidity I had with the single page TOC... 🤦‍♂️

Here are my thoughts, it would be a waste of energy pass two time the layout, it would be better to create the index in a semi-auto way...

I wonder if I can:

1. run the layout one time generate the PDF and recording the markers;
2. and then using the recorded markers to another layout, which basically draws exclusively the index, to do the second pass and create the index with <MakeIndex>
3. Merge the index with the main PDF (outside Publisher)

The third point is there only to explain the process.

To recap, the first time I would run:
sp

The second time:
sp --layout=index.xml

Is that possible? 🤷‍♂️

[pgundlach]

This should be possible. Although I don't understand why you don't do this in one layout.xml file.

Where (in the document) do you want your index to appear? At the beginning or at the end? (Or at some other place?)

[Cicorione]

Hi Patrick the main is reason is the laptop I use to learn and practicing Publisher is very low power, also the index is something that you need only when is time to go to print, or when your product is ready to be finalized, I do believe that makes sense have it as separated PDF to merge later.

On my layout the index is going to the end of the catalog.

Thanks! 🙏

[pgundlach]

The following comment does not apply when you learn and practice the speedata Publisher.

I still believe you are over complicating things, but surely I don't know the exact requirements.

I would put the indexing in the main layout file, and decide on the command line if this is a final or intermediate PDF. You can do this with modes (see https://doc.speedata.de/publisher/en/advancedtopics/advanced-cotrollayout/#_control_via_mode)

Have you seen the indexing example in the examples repository? I believe the indexing is so quick that it is neglectable compared to the rest of the PDF generation. With a separate run, you get a different PDF which you need to merge into the main document in a separate step.

But of course I don't want to write how anyone should use the speedata Publisher, just writing up my 2 cents...

[Cicorione]

Thank you Patrick, over complicating stuff is one of my best skills...

I studied the code but I haven't start to work on it, I was still thinking what method would had been better...

But I definitely go for having into the main layout, I'll implement the control mode later, but it is actually pretty neat as feature!

[Cicorione]

Hi @pgundlach

I brutally copied and pasted your code and tried to adapt it to my node structures, but it fails disrupting completely the layout... 😩

And this is a little bit weird because it should operate it on the index section, which is at the very bottom of the layout, after section that populate the body of the PDF.

In my knowledge the index section shouldn't conflict with the previous section because the previous is operating a ForAll routine... 🤔

Perhaps there is some overlapping in the nodes and variables for instance:

<SetVariable variable="index">
			<Element name="index">
				<Makeindex select="$indexentries" sortkey="name" section="idxsection" pagenumber="page" />
			</Element>
		</SetVariable>

I have already a node section so I renamed it idxsection

	<Record element="keyword">
		<SetVariable variable="indexentries">
			<Copy-of select="$indexentries" />
			<Element name="indexentry">
				<Attribute name="name" select="@title" />
				<Attribute name="page" select="@page" />
			</Element>
		</SetVariable>
	</Record>

@title: I am recording the node section and @title is an attribute of a subnode: section/subsection/product/@title not sure if I have to declare something like: ../@title.

The problem is that is breaking apart a lot of stuff, for instance:

<*> ...{left} .. cycle withcolor spcolor( accent )
                                                  ; 
I'm afraid I don't know how to apply that operation to that
combination of types. Continue, and I'll return the second
argument (see above) as the result of the operation.

! An expression can't begin with `;'.
<inserted text> 
                0
<to be read again> 
                   ;
<*> ...left} .. cycle withcolor spcolor( accent );
                                                   
I'm afraid I need some sort of value in order to continue,
so I've tentatively inserted `0'. You may want to
delete this zero and insert something else;
see Chapter 27 of The METAFONTbook for an example.


 
E:  [page 8 line 47, data line (unknown)] Executing endfig;: [1]
 
E:  Could not create metapost image (graphic=accentroundedcorners)
E:  [page 8 line 47, data line (unknown)] Something is wrong with <PlaceObject>, content is missing 
·   Call command (name=Table,line=698)
·   Call command (name=ForAll,line=699)
W:  [page 8] table without contents 

The metapost code to create the rounded boxes gets broken the moment I invoke the index block otherwise it renders the PDF properly.

Sorry for the pebcak news... 😩

[pgundlach]

I can't see what is going wrong. I would need to see the whole code. You know where to find me :-)

[Cicorione]

Yes I will, thanks! 🙏