This manual is for Eva version 0.5-pre.
Copyright (C) 2020-2024 Martin Edström <meedstrom91@gmail.com>
You can redistribute this document and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This document is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
- GNU/Linux or other Unix-like systems. Should work on MacOS to my knowledge.
- GNU Emacs 27+
- GNU R
- gnuplot
- On X:
xprintidleorx11idle, or to run under Mutter (GNOME’s window manager). - On Wayland: to run under Mutter (GNOME’s window manager).
- For other WMs than Mutter,
kidletimeorswayidlemight work, please open issue.
- For other WMs than Mutter,
On Debian, you can run:
apt-get install r-recommended gnuplot xprintidle
You should let Emacs always run: add it to xinitrc, Startup Applications or so. In principle, we could spin out some functionality as a separate program in Python or Scheme that always runs like your hundred other background processes, but such separation would be overengineering because the package is targeting those who always run Emacs anyway. Every time the program would be scheduled to do something, it’d have to emit a notification, possibly an OS dialog, and start up Emacs anyway. We’d be relying on a functional notification daemon and dbus (brittle assumption), gtk/qt toolkits (brittle assumption), access to executables (brittle assumption, NixOS anyone?), POSIX scripts (unportable without a lot of effort), service declarations (which service manager?), synced config files (brittle) and on and on. One of the raisons d’etre for Emacs is to make app development both fast and solid by just ignoring the OS.
We use run-ess-r from ESS because it has sanity checks and handles encoding issues well, but the flip side is that user configuration can cause it to return errors on startup (e.g. ess-history-file at an unreadable location) and stop eva-mode from turning on. It’s your responsibility to have a functional ESS.
Many of the prompts you will get probably benefit from a completion system, where you can see the possible options. I’ve only tested Selectrum so far. In addition, if your system does not call abort-recursive-edit to exit a prompt (by way of C-g), you might see some misbehavior.
If you have straight.el, you can install the package like so:
(use-package eva
:straight (eva :type git :host github :repo "meedstrom/eva"
:files (:defaults "assets" "renv" "*.R" "*.gnuplot"))Alternatively with Doom Emacs, this goes in packages.el:
(package! eva
:recipe (:host github :repo "meedstrom/eva"
:files (:defaults "assets" "renv" "*.R" "*.gnuplot")))The package syncs a lot of data to disk, because its data is of interest across sessions, so if you customize eva-cache-dir-path or eva-mem-history-path, try to keep it correct. If you have multiple inits via chemacs, have them set one absolute path for these (chemacs can actually help with this so it needn’t be repeated inside each of the inits).
You should activate eva-mode upon init or soon after. The reasons are:
- To bring the buffer logger online, which is necessary for the automatic org-clocker to guess what is going on. Even if it’s not important, it’s useful to have data on what “not important” looks like.
- To calculate idleness correctly. We count idleness from the last time Emacs was running and this mode was on, so any stretch of time you use Emacs without enabling the mode, we assume you just weren’t at the computer.
You’ll want to set eva-items in your init. Please see eva-config.el (M-x find-library eva-config) for a full example, but here’s a subset:
(setq eva-items
(list (eva-item-create :fn #'eva-query-sleep
:dataset "~/self-data/sleep.tsv"
:min-hours-wait 5
:lookup-posted-time t)
(eva-item-create :fn #'eva-query-weight
:dataset "~/self-data/weight.tsv"
:max-entries-per-day 1)
(eva-item-create :fn #'eva-query-mood
:dataset "~/self-data/mood.tsv")))Here we expose one of the main customization targets. As I explain in the next section, I expect you to eventually write your own defuns to replace #'eva-query-weight et al here, so you’d change the :fn value to some #'my-query-weight. I also expect you’ll create new items for whatever bizarre stuff you want to track. When you do, see the source of the existing functions for how it’s done.
You shouldn’t need to read up on the built-in queries, try them out and hopefully you find them intuitive.
The arguments to eva-item-create are as follows:
:fn- the function to call that will query user for this info:dataset- where to save the info:min-hours-wait- the minimum amount of hours to wait before it is ok to query you for this info again:max-entries-per-day- max amount of entries to make in a given day; this makes sense for some kinds of info
It also has :last-called and :dismissals for internal use—do not set these.
(ASIDE: There is also the boolean
:lookup-posted-time, mainly existing for the special case ofeva-query-sleepbecause that prompts about things that happened in the past instead of the now, recording sleep-timestamps that are in the past. I.e. it could be 3pm at the time it asks, but you woke up at 6am so 6am is the timestamp saved in the log. This affects the calculation of whether it’s a good time to ask again.)
The order in which the items come in this list reflects the order in which you will be asked. To disable one of them, it is not necessary to remove it from this list, just cancel the query a few times with C-g and the VA will ask you if (s)he should disable it, which is recorded separately in eva-mem. To reenable, try M-x eva-reenable-fn and enter the name of that function, or simply edit eva-disabled-fns.
To prevent latency during a session, Eva initializes ESS at emacs init. Currently the builtin functions that use the R process are:
eva-plot-weight
If you do not use these in your eva-items, you can prevent the ESS initialization by setting eva-init-r to nil.
(use-package eva
;; Things that should be set before load
:custom
(eva-va-name "Alfred")
(eva-user-name "Bruce")
(eva-user-short-title "sir") ;; don't like titles? put in your name again
(eva-user-birthday "1963-02-19")
(eva-idle-log-path "~/self-data/idle.tsv")
(eva-buffer-focus-log-path "~/self-data/buffer-focus.tsv")
(eva-buffer-info-path "~/self-data/Self_data/buffer-info.tsv")
(eva-main-ledger-path "~/finances.ledger")
(eva-main-datetree-path "~/org/diary.org")
(ess-ask-for-ess-directory nil) ;; Prevent annoying ESS startup prompt.
:config
(require 'eva-builtin)
;; These are looked up by `eva-present-diary', but org-journal is not needed.
(setq org-journal-dir "~/org/journal/")
(setq org-journal-file-format "%F.org")
(add-hook 'eva-after-load-vars-hook #'eva-check-dangling-clock)
(add-hook 'eva-after-load-vars-hook #'eva-check-org-vars)
;; HINT: Though not likely you'll want to, you can use the same object
;; multiple times in the queue, you'll just have to assign the output of
;; an (eva-item-create) to an external variable and refer to it.
(setq eva-items
(list
(eva-item-create :fn #'eva-greet
:min-hours-wait 1)
(eva-item-create :fn #'eva-query-mood
:dataset "~/self-data/mood.tsv"
:min-hours-wait 1)
(eva-item-create :fn #'eva-present-diary
:max-successes-per-day 1)
(eva-item-create :fn #'eva-query-weight
:dataset "~/self-data/weight.tsv"
:max-entries-per-day 1)
(eva-item-create :fn #'eva-plot-weight
:max-entries-per-day 1)
(eva-item-create :fn #'eva-query-sleep
:dataset "~/self-data/sleep.tsv"
:min-hours-wait 5
:lookup-posted-time t)
(eva-item-create :fn #'eva-present-ledger-report)
;; May be slow
;; (eva-item-create :fn #'eva-present-org-agenda-log-archive)
(eva-item-create :fn #'eva-present-org-agenda-log)
(eva-item-create :fn #'eva-query-ingredients
:dataset "~/self-data/ingredients.tsv"
:min-hours-wait 5)
(eva-item-create :fn #'eva-query-cold-shower
:dataset "~/self-data/cold.tsv"
:max-entries-per-day 1)
(eva-item-create :fn #'eva-query-activity
:dataset "~/self-data/activities.tsv"
:min-hours-wait 1)
;; you can inline define the functions too
(eva-item-create
:fn (eva-defun my-bye ()
(message (eva-emit "All done for now."))
(bury-buffer (eva-buffer-chat)))
:min-hours-wait 0)))
;; Hotkeys in the chat buffer
(transient-replace-suffix 'eva-dispatch '(0)
'["General actions"
("q" "Quit the chat" bury-buffer)
("l" "View Ledger report" eva-present-ledger-report)
("f" "View Ledger file" eva-present-ledger-file)
("a" "View Org agenda" org-agenda-list)])
(define-key eva-chat-mode-map (kbd "l") #'eva-present-ledger-report)
(define-key eva-chat-mode-map (kbd "f") #'eva-present-ledger-file)
(define-key eva-chat-mode-map (kbd "a") #'org-agenda-list)
;; Activities (for `eva-query-activity'). These are cl objects for forward
;; compatibility; right now only :name is used, to fill out completion
;; candidates.
(setq eva-activity-list
(list (eva-activity-create :name "sleep")
(eva-activity-create :name "studying")
(eva-activity-create :name "coding")
(eva-activity-create :name "unknown")))
(eva-mode))Note that when you re-eval (setq eva-items ...) seen in the previous section, it will reset the items’ keys :last-called and :dismissals. This doesn’t matter much while you are experimenting, but if you care about it, you can immediately eval (eva--mem-restore-items-values) before the reset values are written to disk (by a timer that runs every other minute).
Bit of a tangent, but you could use the functions from both org-journal and the org-roam dailies if your org-journal-file-format is the same file name format as used by your org-roam-dailies-capture-templates. So your org-journal-dir can refer to the same location as (concat org-roam-directory org-roam-dailies-directory). Just a tip. That’s pretty much what you have to do now if you want eva-present-diary to scan your org-roam dailies, since it as yet doesn’t scan them specifically. Although it’s likely you don’t use org-journal, in which case you can simply set org-journal-dir to your ~/org-roam/daily/ equivalent.
For as long as eva-mode is enabled, it will start a questioning session when you return to the computer after being away for more than 90 minutes (configurable at eva-idle-threshold-secs-long). This works even if the computer is off during that time, because it writes eva--last-online to disk at eva-mem-history-path.
In the chat buffer, there is a Transient menu available by typing h. Most hotkeys within match the hotkeys in the chat buffer.
To resume a broken session, type r in the chat buffer, or M-x eva-resume from anywhere.
You can type - or + in the chat buffer to increment or decrement the date the questions will apply to. That’s useful if you forgot something yesterday. Pay attention to the date in each prompt. Type 0 to reset it to today.
If you wish to force a session right now, instead of waiting for the VA to butt in, M-x eva-session-new should do it.
You’ll customize this package primarily by creating defuns. It’s so personal what people want a VA for, that simple user options (variables) would not scale. I would do you no service by making variable references all over the place. Better you get started with the defuns and we both save energy. As a plus, it gives our code more clarity.
Some premade functions are listed as follows. Read their source to see how to write your own.
- Queries
eva-query-weighteva-query-moodeva-query-sleepeva-query-activity
- Excursions
eva-plot-weighteva-present-ledger-reporteva-present-diary
- Misc
eva-greet
Now, there are two main kinds of functions: queries and excursions, and a function can be both at the same time. The distinction is:
- Pure queries are simple: they prompt for user input, do something with it (usually write something to disk), and finish, letting the next item in the queue take over.
- Excursions send you away from the chat buffer and quit the interactive session with the VA. For example, it may send you to a ledger report (
eva-present-ledger-report). The VA has, so to speak, lost control of the conversation. To proceed to the next item, it waits for the user to either kill every buffer spawned by the excursion, or manually resume the session witheva-resume.- To be an excursion, the function must push each spawned buffer onto
eva-excursion-buffersand then calleva-start-excursion.
- To be an excursion, the function must push each spawned buffer onto
If the current set of greetings e.g. “Nice to see you again” aren’t to your tastes, you may want to override one or more of these.
eva-greetings(variable)eva-daytime-appropriate-greetings(function)
You can also put an alternative to eva-greet in eva-items, but there’ll still be a couple of places where the above get used.
By default, your entry point is eva-run-queue (called automatically throughout the day via eva-session-butt-in-gently). It tries to go through every currently relevant item in eva-items. To force a new session, you can also call M-x eva-session-new.
To understand better how the package works, you can make a different entry point. The sky’s the limit. This snippet contains a fairly “dumb” approach:
(defun my-custom-session (&optional just-idled)
(setq eva-date (ts-now))
(and just-idled
(eva-ynp "Have you slept?")
(eva-query-sleep))
(unless (eva-logged-today-p "~/self-data/weight.tsv")
(eva-query-weight))
(eva-query-mood)
(and (eva-ynp "Up to reflect?")
(eva-ynp "Have you learned something?")
(org-capture nil "s")) ;; let's say you have a capture template on "s"
(if (eva-ynp "How about some flashcards?")
(org-drill))
(unless (eva-logged-today "~/self-data/meditation.csv")
(eva-query-meditation eva-date))
(unless (eva-logged-today "~/self-data/cold.csv")
(when (eva-ynp "Have you had a cold shower yet?")
(eva-query-cold-shower)))
(if (eva-ynp "Have you paid for anything since yesterday?")
(eva-present-ledger-file))
(if (eva-ynp "Shall I remind you of your life goals? Don't be shy.")
(view-file "/home/kept/Journal/gtd.org"))
(and (>= 1 (eva-query-mood))
(doctor))
(eva-plot-weight)
(eva-plot-mood)
(eva-present-diary)
(and (-all-p #'null (-map #'eva-logged-today-p
(-map #'eva-item-dataset eva-items)))
(eva-ynp "Shall I come back in an hour?")
(run-with-timer 3600 nil #'my-custom-session)))If you want a more intelligent session, populate eva--queue and then call eva-run-queue – it’s what the session- functions do.
You can enrich your agenda log (l hotkey in the agenda view) with Git commit history so they show up like this:
Week-agenda (W33): Monday 16 August 2021 W33 eva: 9:04...... fix :Memacs:git:: Tuesday 17 August 2021 eva: 10:40...... Comply with checkdoc, flycheck, package-lint :Memacs:git:: eva: 11:06...... Hopefully fixed bug nulling mood-alist :Memacs:git:: eva: 11:37...... Post design goals :Memacs:git:: eva: 12:20...... minor :Memacs:git:: eva: 20:38...... Add makem.sh as submodule :Memacs:git:: eva: 21:55...... Move code around and rename :Memacs:git:: eva: 23:21...... Fix bug that ran mode turn-off code on init :Memacs:git:: Wednesday 18 August 2021 eva: 0:41...... simplify :Memacs:git:: eva: 0:42...... had corrupt dataset, add a check for next time :Memacs:git:: eva: 0:47...... fix :Memacs:git:: eva: 2:19...... Make R take user-supplied dataset path :Memacs:git:: eva: 3:09...... fixes :Memacs:git:: eva: 3:11...... Better guard clauses :Memacs:git:: eva: 3:56...... Remove test obsoleted by emacs-lisp-macroexpand :Memacs:git:: eva: 4:33...... Rename test.el :Memacs:git:: eva: 5:01...... alignment :Memacs:git:: eva: 5:33...... Fix wrap :Memacs:git:: eva: 5:35...... Always track query successes :Memacs:git:: eva: 5:50...... minor :Memacs:git:: eva: 16:27...... Settle on a single boilerplate macro :Memacs:git:: eva: 16:30...... Add debug messages :Memacs:git:: eva: 16:31...... fix :Memacs:git:: eva: 17:40...... style :Memacs:git:: eva: 18:03...... Clearer init :Memacs:git:: eva: 20:21...... cleanup and document :Memacs:git:: eva: 20:38...... Update licensing :Memacs:git:: Thursday 19 August 2021 Friday 20 August 2021 Saturday 21 August 2021 Sunday 22 August 2021
That’s especially nice when you are regularly reviewing the past.
When I first read about Memacs, I thought it would be a beast to set up and get working, but it’s just a collection of independent Python scripts. So let’s use one of them to achieve the above.
First, download all its scripts with something like pip3 install --user memacs, which will put the executable memacs_git, among other memacs_* executables, into ~/.local/bin/. Path may vary depending on your OS.
Then set up a regular job that collects your Git histories. Here’s a way to do that from your initfiles. Edit paths as necessary. I apologize for mixing shell commands with lisp.
(defun my-file-size (file)
"Returns the size of FILE in bytes."
(unless (file-readable-p file)
(error "File %S is unreadable; can't acquire its filesize"
file))
(nth 7 (file-attributes file)))
(setq my-all-git-repos
(seq-filter (lambda (x)
(and (file-directory-p x)
(member ".git" (directory-files x))))
;; Paths to specific git repos
(append '("/home/kept/Knowledge_base"
"/home/kept/Journal/Finances"
"/home/kept/Guix channel"
"/home/kept/Fiction"
"/home/kept/Dotfiles")
;; Paths to parent dirs of many git repos
(directory-files "/home/kept/Code" t)
(directory-files "/home/kept/Coursework" t))))
(defun my-memacs-scan-git ()
(require 'f)
(require 'cl-lib)
(let ((my-archive-dir (shell-quote-argument "/home/kept/Archive/memacs/git/")))
(make-directory "/tmp/rev-lists" t)
(and
(executable-find "git")
(executable-find "memacs_git")
(bound-and-true-p my-all-git-repos)
(dolist (dir my-all-git-repos t)
(let ((default-directory dir))
(start-process-shell-command
"Memacs_Job_Git_1"
nil
(concat "git rev-list --all --pretty=raw > /tmp/rev-lists/"
(shell-quote-argument (file-name-nondirectory dir))))))
(file-exists-p my-archive-dir)
(run-with-timer
5 nil (lambda ()
(dolist (repo-history (directory-files "/tmp/rev-lists" t
(rx bol (not (any "." "..")))))
(unless (= 0 (my-file-size repo-history))
(let ((basename (shell-quote-argument (file-name-nondirectory repo-history))))
(start-process
"Memacs_Job_Git_2" nil
"memacs_git" "-f" repo-history "-o"
(concat my-archive-dir basename ".org_archive"))
(f-touch (concat my-archive-dir basename ".org"))
(cl-pushnew my-archive-dir org-agenda-files))))))))
;; Re-run myself in an hour.
(run-with-timer (* 60 60) nil #'my-memacs-scan-git))
(my-memacs-scan-git)You may have to restart Emacs for the agenda to properly update. Anyway, now when you type v A in the agenda, these Git commits will show up. You will also be shown that view by the builtin eva-present-org-agenda.
Do you want to be reminded every 10 minutes of the currently clocked task? A timer can do it:
(require 'named-timer) ;; an indispensable 70-line library
(with-eval-after-load 'org
(named-timer-run :my-clock-reminder nil 600
(defun my-clock-remind ()
(require 'notifications) ;; built-in
(when (org-clock-is-active)
;; NOTE: will error if you don't have dbus
(notifications-notify :title eva-ai-name
:body (concat "Currently working on: "
org-clock-current-task))))))To remind you of dangling clocks after a restart, Org’s builtin way is as follows.
(setq org-clock-persist t)
(setq org-clock-auto-clock-resolution 'always)
(org-clock-persistence-insinuate)
;; (org-clock-auto-clockout-insinuate) ;; unrelated but nice?
(add-hook 'org-clock-in-hook #'org-clock-save) ;; in case of a crash
(eval-after-load 'org #'org-resolve-clocks)With those settings, whenever opening an Org buffer it will scan agenda files for dangling clocks.
However, if you (require 'eva-builtin), the VA will remember the active Org clock, so if you don’t want to load Org on every Emacs startup, that’s where it can help you. The following hook will ask about turning on Org if and only if the VA remembers an unfinished clock from last session. After thus loading Org, your Org config will react to the dangling clock and can take it from there.
(require 'eva-builtin)
(add-hook 'eva-after-load-vars-hook #'eva-check-dangling-clock)eva-query-sleep is made to be flexible. It can log either or both of two variables: wake-up time, and sleep quantity. Thus, its log file doesn’t have the usual full date/time stamp, instead tracking date and wake-up time separately. The wake-up time can be left blank if you don’t know when you woke up (or don’t consider it important), you can still enter the approximate sleep quantity. What’s important is to realize that it’s best for each row in this dataset to represent one sleep block. Thus, if you later recall when it was you woke up, you shouldn’t just add a new line, but edit the dataset.
This does diary backlook; shows you past entries on this date last month, the month before that, and so on. I’ve found it good in so many ways.
Currently this works best with daily diary files in the org-journal style (no need for org-journal installed), i.e. when you have a folder somewhere that contains files named 2020-01-01.org, 2020-01-02.org, 2020-01-03.org and so on. If you do have org-journal, it also checks org-journal-file-format in case of a custom file-name format, but org-journal-file-type must be 'daily (the default).
The presenter also looks for a datetree file set at eva-main-datetree-path. This is for those of you who capture writings to a datetree, by way of a member of org-capture-templates targeting something like (file+olp+datetree "~/diary.org").
(This is fully automated and not something you’d add to eva-items.)
The buffer logger writes two files, buffer-focus.tsv and buffer-info.tsv (or whatever you rename them to). If you are familiar with the notion of normalized databases, it’s self-explanatory: buffer-focus.tsv associates timestamps with buffers, identified by an UUID, and buffer-info.tsv associates those UUIDs with various facts about those buffers, such as title, file (if any) and major mode.
It’s possible you’ll want the buffer logger to record something more. A good example is to record eww-current-url of an eww buffer. Your options are to redefine eva--log-buffer (in which case please open an issue), or simply tell eww to change its buffer title to match the URL. However, this relies on the TBD feature that considers each new title a new buffer with new UUID.
(This is fully automated and not something you’d add to eva-items.)
The idleness logger is a byproduct of essential functionality. It’s not much of an assistant without idle awareness, so you get this log for free.
Since we have append-only datasets, you don’t need to worry if you inadvertently create two files – you can cut and paste the contents from the old file at any time.
Want to add a lot of rows at once? You can edit the .tsv file directly. When you do, you’ll see it has Unix timestamps for each row. They represent the posted time (time-this-row-was-added) and are a typical feature of append-only databases. Just eval-print (float-time) to get a new timestamp and reuse that for every row you’re adding, even though they may be observations from the past. If relevant, date/time goes in a separate field, you don’t abuse the posted-time for this.
Take an example. Suppose you currently have only three datapoints in the log, looking like this:
1612431985.7806770 2021-02-04 10:46:22 +0100 87 1612521756.8125120 2021-02-05 11:42:32 +0100 86.8 1613462960.1966035 2021-02-16 09:09:14 +0100 85
and you have a bunch of older observations on the same topic you wrote on a piece of paper, from January. Expand it to this:
1612431985.7806770 2021-02-04 10:46:22 +0100 87 1612521756.8125120 2021-02-05 11:42:32 +0100 86.8 1613462960.1966035 2021-02-16 09:09:14 +0100 85 1613963000.0000000 2021-01-11 00:00:00 +0100 85.1 1613963000.0000000 2021-01-12 00:00:00 +0100 84 1613963000.0000000 2021-01-14 00:00:00 +0100 85
As long as the Unix timestamps are always greater or equal to the ones above, it’s valid. Nothing bad will happen if they’re not in order, as it stands, but that data could come into use at some point (for sanity checks if not a matter for analysis in its own right) so it’s a matter of hygiene.
See https://github.com/meedstrom/eva/issues
- [2021-09-18]:
eva-present-org-agendarenamed toeva-present-org-agenda-log-archive. The old name will work for a while, but will eventually be redefined. - [2021-08-29]
eva-check-org-variablesrenamed toeva-check-org-vars - [2021-08-24]
eva-mem-pushnewrenamed toeva-mem-push - [2021-08-24]
eva-mem-pushnew-altrenamed toeva-mem-push-alt - [2021-08-23]
eva-ai-namerenamed toeva-va-name - [2021-08-23]
eva-dbg-fnrenamed toeva-debug-fn
Work in progress: map of concepts.
digraph concepts {
subgraph directed {
Logs -> Auto-logged
Logs -> "User response"
Auto-logged -> "Buffer focus"
Auto-logged -> "Buffer existence"
"User response" -> Weight
"User response" -> Meditation
"User response" -> "Cold showers"
"User response" -> "Mood"
Excursions -> "Diary re-reading"
Excursions -> "Ledger"
}
subgraph undirected {
edge [dir=none]
"Disk writes" -- Logs
"Disk writes" -- "Disk access"
"Disk access" -- Ledger
"Disk access" -- "On startup"
}
}