Add initial manpage, generated from README.md by go-md2man

emtee.1

@@ -0,0 +1,356 @@
+.TH EMTEE 1 "Version 1.0.0: Jan 2019"
+.SH NAME
+.PP
+emtee \- a faster\-startup emerge \-DuU \-\-with\-bdeps=y @world (et al.)
+
+.SH SYNOPSIS
+.PP
+\fB\fCemtee\fR [\fB\fC\-a\fR] [\fB\fC\-A\fR] [\fB\fC\-b\fR] [\fB\fC\-c\fR] [\fB\fC\-C\fR] [\fB\fC\-d\fR] [\fB\fC\-e\fR args] [\fB\fC\-E\fR args]
+[\fB\fC\-h\fR] [\fB\fC\-p\fR] [\fB\fC\-N\fR] [\fB\fC\-s\fR set] [\fB\fC\-S\fR] [\fB\fC\-v\fR] [\fB\fC\-V\fR]
+
+.SH DESCRIPTION
+.PP
+\fB\fCemtee\fR is a simple script to speed up \fI@world\fP updates on Gentoo
+Linux systems, by significantly decreasing the time taken for the
+\fB\fCemerge\fR(1)'s initial dependency resolution stage (often the dominant
+component for SBCs and other resource\-challenged systems, if a binhost
+is deployed).
+
+.PP
+It may be used (with appropriate options) in place of:
+
+.RS
+.IP \(bu 2
+\fB\fCemerge \-\-with\-bdeps=y \-\-deep \-\-update \-\-changed\-use\fR \fI@world\fP,
+.IP \(bu 2
+\fB\fCemerge \-\-with\-bdeps=y \-\-deep \-\-update \-\-newuse \-\-ask \-\-verbose\fR \fI@world\fP;
+
+.RE
+
+.PP
+and similar invocations.
+
+.PP
+For example, you could achieve the same effect as the above commands
+with:
+
+.RS
+.IP \(bu 2
+\fB\fCemtee\fR, and
+.IP \(bu 2
+\fB\fCemtee \-\-newuse \-\-ask \-\-verbose\fR (or, equivalently, \fB\fCemtee \-\-Nav\fR)
+
+.RE
+
+.PP
+respectively.
+
+.PP
+Generally speaking, \fB\fCemtee\fR will take materially less time
+to get the first actual build underway than its counterpart \fB\fCemerge
+\-\-update\fR, while still targeting an identical build list, permitting
+job parallelism, and triggering any required slot\-driven rebuilds, and
+block\-driven unmerges.
+
+.SH OPTIONS
+.PP
+\fB\fC\-a\fR, \fB\fC\-\-ask\fR
+  Turns on interactive mode for the real \fB\fCemerge\fR step (i.e.
+  during phase 4, as described in \fIALGORITHM\fP, below).
+
+.PP
+\fB\fC\-A\fR, \fB\fC\-\-alert\fR
+  Uses terminal bell notification for all interactive prompts in
+  the real \fB\fCemerge\fR step. Selecting this turns on \fB\fC\-a\fR by default.
+
+.PP
+\fB\fC\-b\fR, \fB\fC\-\-with\-bdeps=y|n\fR
+  Specifies whether or not to pull\-in build time dependencies. NB,
+  defaults to \fB\fCy\fR if unspecified, in contrast to \fB\fCemerge\fR\&'s default
+  behaviour.
+
+.PP
+\fB\fC\-c\fR, \fB\fC\-\-crosscheck\fR
+  Checks the final build list against the result obtained by running a
+  conventional \fB\fC\-\-deep\fR \fB\fCemerge\fR, and prints a \fB\fCdiff\fR(1) of the package
+  list; also provides a comparitive timing, for benchmarking purposes.
+  NB: does not check the \fIordering\fP of the packages in the two lists.
+
+.PP
+\fB\fC\-C\fR, \fB\fC\-\-strict\-crosscheck\fR
+  As for \fB\fC\-c\fR, but exits with an error if the \fB\fCemtee\fR and conventional
+  \fB\fC\-\-deep \-\-update\fR \fB\fCemerge\fR lists do not match.
+
+.PP
+\fB\fC\-d\fR, \fB\fC\-\-debug\fR
+  Prints some additional debugging information during the process.
+
+.PP
+\fB\fC\-e\fR, \fB\fC\-\-emptytree\-emerge\-args=\fR\fIADDITIONAL\_ARGS\fP
+  Passes the specified arguments to the initial (\fB\fC\-\-pretend
+  \-\-emptytree\fR) \fB\fCemerge\fR step used to caclulate the (unfiltered) build
+  list. Note that these arguments are \fInot\fP passed to the real
+  \fB\fCemerge\fR step; you need to use \fB\fC\-E\fR for that.
+
+.PP
+\fB\fC\-E\fR, \fB\fC==emerge\-args=\fR\fIADDITIONAL\_ARGS\fP
+  Passes the specified arguments to the real \fB\fCemerge\fR step. Note that
+  these arguments are \fInot\fP passed to the preliminary \fB\fCemerge\fR step; you
+  need to use \fB\fC\-e\fR for that.
+.br
+
+.br
+  Note also that you can achieve the effect of the \fB\fC\-a\fR \fB\fC\-A\fR, \fB\fC\-p\fR and \fB\fC\-v\fR
+  options by setting them directly via \fB\fC\-E\fR, if you prefer. They are
+  provided as syntactic sugar, for convenience.
+
+.PP
+\fB\fC\-h\fR, \fB\fC\-\-help\fR
+  Prints a short help message, and exits.
+
+.PP
+\fB\fC\-p\fR, \fB\fC\-\-pretend\fR
+  Passes the \fB\fC\-\-pretend\fR option to the real \fB\fCemerge\fR step, resulting in
+  a 'dry run' (nothing will actually be updated).
+
+.PP
+\fB\fC\-N\fR, \fB\fC\-\-newuse\fR
+  Rebuild packages which have had any USE flag changes, even if those
+  changes don't affect flags the user has enabled (the more
+  conservative \fB\fC\-\-changed\-use\fR behaviour is the default).
+
+.PP
+\fB\fC\-s\fR, \fB\fC\-\-set\fR=\fISET\fP
+  Uses the specified set (e.g. \fI@system\fP) in preference to the default
+  \fI@world\fP\&. You can pass regular package names here as well, but note
+  that using a regular \fB\fCemerge\fR is likely to be faster for such targets.
+
+.PP
+\fB\fC\-S\fR, \fB\fC\-\-force\-slot\-rebuilds\fR
+  Checks for any slot\-change\-driven rebuilds, and adds these (reverse)
+  dependencies to the build list; this option should not generally be
+  required, as the real \fB\fCemerge\fR step will trigger such rebuilds
+  automatically anyway.
+
+.PP
+\fB\fC\-v\fR, \fB\fC\-\-verbose\fR
+  Passes the \fB\fC\-\-verbose\fR option to the real \fB\fCemerge\fR step
+
+.PP
+\fB\fC\-V\-\fR, \fB\fC\-\-version\fR
+  Prints \fB\fCemtee\fR\&'s version number, and exits.
+
+.SH ALGORITHM
+.PP
+The \fB\fCemtee\fR process runs as follows:
+
+.RS
+.IP "  1." 5
+Derive a full, versioned build list for the \fI@world\fP set and its
+entire deep dependency tree, via \fB\fCemerge \-\-with\-bdeps=y \-\-pretend
+\-\-emptytree \-\-verbose [opts]\fR \fI@world\fP, which Portage can do
+relatively quickly. The resulting list, as it is derived as if \fIno\fP
+packages were installed to begin with, will automatically contain
+all necessary packages at their 'best' versions (which may entail
+upgrades, downgrades, new slots etc.  wrt the currently installed
+set).
+.IP "  2." 5
+Filter this list, by marking each fully\-qualified atom
+(\fIFQA\fP=\fI$CATEGORY/$PF\fP) within it for building (or not). Begin
+with all \fIFQAs\fP unmarked.
+.br
+
+.br
+
+.RS
+.IP \(bu 2
+Then (pass 1), mark anything which isn't a block, uninstall or reinstall for build;
+.IP \(bu 2
+Then (pass 2), check each reinstall, to see if its \fIactive\fP
+USE flag set is changing (default behaviour), or if \fIany\fP of
+its USE flags are changing (\fB\fC\-N\fR/\fB\fC\-\-newuse\fR behaviour), and if
+so, mark that package for build (fortunately, the \fB\fC\-\-verbose\fR
+output from step 1 contains the necessary USE flag delta
+information to allow us to easily work this out).
+.IP \(bu 2
+Then (pass 3), if \fB\fC\-S\fR/\fB\fC\-\-force\-slot\-rebuilds\fR is in use, for
+each marked package on the list whose slot or subslot is
+changing (also inferable from the phase 1 output), search
+\fI/var/db/pkg/<FQA>/RDEPENDS\fP (and \fIDEPENDS\fP, if
+\fB\fC\-\-with\-bdeps=y\fR, the default, is active) for any matching slot
+dependencies.  Mark each such located (reverse) dependency that
+is \fIalso\fP on the original \fB\fC\-\-emptytree\fR list (and not a block
+or uninstall) for build.
+.br
+
+.br
+Note that pass 3 is skipped by default, since the phase 4 emerge
+(aka the real \fB\fCemerge\fR) will automatically trigger any
+necessary slot rebuilds anyway, so it is redundant except for in a
+few esoteric situations.
+
+.RE
+.IP "  3." 5
+Iff \fB\fC\-c\fR/\fB\fC\-\-crosscheck\fR (or \fB\fC\-C\fR/\fB\fC\-\-strict\-crosscheck\fR) passed,
+compare the \fIFQA\fP build list produced by invoking \fB\fCemerge \-\-bdeps=y
+\-\-pretend \-\-deep \-\-update [\-\-changed\-use|\-\-newuse] [opts]\fR \fI@world\fP
+(adapted for specified options appropriately), with that produced
+by invoking \fB\fCemerge \-\-oneshot \-\-pretend [opts]\fR
+\fI<filtered-FQA-build-list-from-phase-2>\fP\&. If any differences are
+found, report them (and, additionally, stop the build in such a
+case, if \fB\fC\-S\fR/\fB\fC\-\-strict\-crosscheck\fR specified). Also report
+a series of comparative (total elapsed wall\-clock) timings for both
+alternatives, for benchmarking purposes.
+.br
+
+.br
+Note: crosschecking should \fIonly\fP be used for reassurance or
+benchmarking, as it will, of necessity, be slower than the baseline
+in total time cost (since the check involves running both that
+\fIand\fP the newer, \fB\fC\-\-emptytree\fR\-based approach)! So, if your goal is
+to improve emerge times, do \fInot\fP pass \fB\fC\-s\fR/\fB\fC\-S\fR\&.
+.IP "  4." 5
+Invoke the real \fB\fCemerge\fR, as: \fB\fCemerge \-\-oneshot [opts]\fR
+\fI<filtered-FQA-build-list-from-phase-2>\fP\&.
+.br
+
+.br
+Note that additional arguments may be passed to this invocation, both
+explicitly (via \fB\fC\-E\fR/\fB\fC\-\-emerge\-args\fR) and implicitly, via one of
+the impacting options (\fB\fC\-v\fR/\fB\fC\-\-verbose\fR, \fB\fC\-a\fR/\fB\fC\-\-ask\fR,
+\fB\fC\-A\fR/\fB\fC\-\-alert\fR, or \fB\fC\-p\fR/\fB\fC\-\-pretend\fR).
+
+.RE
+
+.SH BASIS
+.PP
+Why is this approach faster? Well, the main claims behind \fB\fCemtee\fR are:
+
+.RS
+.IP "  1." 5
+An \fB\fC\-\-emptytree\fR \fB\fCemerge\fR of \fI@world\fP yields the same versioned package list
+that a \fB\fC\-\-deep \-\-update\fR \fB\fCemerge\fR would arrive at.
+.br
+
+.br
+That is, for \fB\fCemtee\fR to work, it must be true that for a consistent,
+depcleaned Gentoo system with a recently updated set of ebuild
+repositories, if \fB\fCemerge \-\-with\-bdeps=y \-\-emptytree\fR \fI@world\fP is
+invoked and runs successfully to conclusion, then an immediately
+following \fB\fCemerge \-\-with\-bdeps=y \-\-deep \-\-changed\-use \-\-update\fR
+\fI@world\fP will always be a no\-op.
+.br
+
+.br
+Or, to put it another way, we claim that the list of
+fully\-qualified atoms (\fIFQAs\fP, where an \fIFQA\fP is \fI$CATEGORY/$PF\fP)
+produced by running \fB\fCemerge \-\-with\-bdeps=y \-\-pretend \-\-emptytree
+\-\-verbose\fR \fI@world\fP will always describe the same end state reached
+by running \fB\fCemerge \-\-with\-bdeps=y \-\-deep \-\-update
+[\-\-changed\-use|\-\-newuse]\fR \fI@world\fP from same starting conditions,
+as regards packages and versions, anyhow.
+.IP "  2." 5
+It also contains sufficient information to simulate \fB\fC\-\-changed\-use\fR
+and \fB\fC\-\-newuse\fR\&.
+.br
+
+.br
+Of course, the issue is that in addition to new versions (\fI[N]\fP),
+package upgrades (\fI[U]\fP), downgrades (\fI[UD]\fP), new slots (\fI[NS]\fP)
+blocks and uninstalls, such a list will generally also contain a
+huge number of reinstalls (\fI[R]\fP). Some of these will genuinely
+need doing (in light of changed USE flags etc.), but many,
+usually the vast majority, will be redundant.
+.br
+
+.br
+Fortunately, for common rebuild selections (such as \fB\fC\-\-changed\-use\fR
+and \fB\fC\-\-newuse\fR), we can easily identify which is which, using only
+the information provided by the \fB\fC\-\-pretend \-\-emptytree\fR \fB\fCemerge\fR
+itself \- since in its output, changes to the USE flag active set
+for a given package are shown with an \fI*\fP suffix, and changes to
+the remaining set with a \fI%\fP suffix, when \fB\fC\-\-verbose\fR is used.
+.IP "  3." 5
+Producing such a list, and then shallow emerging it, reduces the net
+dependency calculation time.
+.br
+
+.br
+Finally, we also claim that for a Gentoo system with many installed
+packages, the time taken to 1) generate an \fB\fC\-\-emptytree\fR \fI@world\fP
+\fIFQA\fP list for all packages, 2) filter this to leave only those
+elements that actually \fIneed\fP an install or reinstall (given the
+current package set and \fB\fC\-\-changed\-use\fR/\fB\fC\-\-newuse\fR
+etc. preference); and 3) invoke a \fB\fC\-\-oneshot\fR \fB\fCemerge\fR on the
+resulting list (of \fI=$CATEGORY/$PF\fP \fIFQAs\fP), to the point the first
+build actually starts, can be up to an \fIorder of magnitude\fP less
+than the equivalent time to first build commencement for a \fB\fC\-\-deep
+\-\-update\fR based \fI@world\fP \fB\fCemerge\fR (for a system with many installed
+packages and where the number of required updates is (relatively)
+small).  Yet, if the other claims above are correct, the resulting
+merge lists for both approaches will be identical. Furthermore,
+this real \fB\fC\-\-oneshot\fR \fB\fCemerge\fR will still deal with triggered slot
+change rebuilds and soft block uninstalls for us, and (subject to
+\fIEMERGE\_DEFAULT\_OPTS\fP) allow the scheduled builds to be fully
+parallelized.
+
+.RE
+
+.SH ADVANTAGES
+.PP
+The speedup for the dependency phase just mentioned, can
+translate to hours saved on slow SBCs with binhost backing (where the
+build phase itself is relatively low cost). The efficiency gains fall
+if a large number of packages require updating, however.
+
+.PP
+Another advantage of this approach is that for some complex updates,
+with many blockers, \fB\fCemerge \-\-with\-bdeps=y \-\-pretend \-\-emptytree
+\-\-verbose\fR \fI@world\fP can sometimes derive a valid list of \fIFQAs\fP, in
+cases where \fB\fCemerge \-\-with\-bdeps=y \-\-pretend \-\-deep \-\-update\fR \fI@world\fP
+fails so to do, even with heavy backtracking (although this is a
+comparatively rare situation).
+
+.PP
+Note: in the context of this script, an \fIFQA\fP, or fully qualified
+atom, is taken to be \fI$CATEGORY/$PF\fP, so for example:
+\fIsys\-apps/package\-a\-1.0.4\_rc4\_p3\-r2\fP\&.
+
+.SH BUGS
+.PP
+A number of nice \fB\fCemerge\fR features don't work with \fB\fCemtee\fR, such as
+\fB\fC\-\-changed\-deps\fR etc. The focus has been on \fB\fC\-\-changed\-use\fR and
+\fB\fC\-\-newuse\fR, which are the most common.
+
+.PP
+To operate correctly, \fB\fCemtee\fR needs to be able to parse the output
+from \fB\fCemerge\fR\&. So, if the latter's format changes in the future,
+expect breakage ><
+
+.PP
+The script's efficiency gains degrade rapidly as the number of
+packages requiring upgrade increases.
+
+.SH COPYRIGHT
+.PP
+Copyright © 2018\-2019 sakaki
+
+.PP
+License GPLv3+ (GNU GPL version 3 or later)
+.br
+
+\[la]http://gnu.org/licenses/gpl.html\[ra]
+
+.PP
+This is free software, you are free to change and redistribute it.
+.br
+There is NO WARRANTY, to the extent permitted by law.
+
+.SH AUTHOR
+.PP
+sakaki — send bug reports or comments to 
+\[la]sakaki@deciban.com\[ra]
+
+.SH SEE ALSO
+.PP
+\fB\fCdiff\fR(1), \fB\fCemerge\fR(1), \fB\fCportage\fR(5)