-
Notifications
You must be signed in to change notification settings - Fork 1
/
README.txt
152 lines (124 loc) · 8.63 KB
/
README.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
Every time a Pi-Star system is booted, the dashboard comes up with an empty LH (Last Heard)
list. Internally, Pi-Star creates a number of running log files in temporary storage (tmpf)
which are lost when the system is shut down and restarted. Retention of these files for post
analysis or to restore current activity after a sudden or planned outage is desirable.
This project creates the necessary scripts and SYSTEMD units (.service and .timer) to backup
(save) and restore the MMDVM files automatically. Backups are taken on a periodic basis and
during shutdowns; restores are initiated during regular system startup. Aging (rotation) of
the backups occurs during Pi-Star's nightly update processing.
To install, download the two Build-* files; execute BUILD-Log-Backup-Restore-Tasks.sh first
(install the required scripts), then execute the BUILD-Log-Systemd-Tasks.sh file to actually
set the system to run the necessary scripts on a periodic basis and during shutdowns/restarts.
wget 'https://github.com/kn2tod/pistar-mmdvm-log-backup-restores/raw/main/Build-MMDVM-Log-Backup-Restore-Tasks.sh'
wget 'https://github.com/kn2tod/pistar-mmdvm-log-backup-restores/raw/main/Build-MMDVM-Log-Systemd-Tasks.sh'
or
wget 'https://raw.githubusercontent.com/kn2tod/pistar-mmdvm-log-backup-restores/main/Build-MMDVM-Log-Backup-Restore-Tasks.sh'
wget 'https://raw.githubusercontent.com/kn2tod/pistar-mmdvm-log-backup-restores/main/Build-MMDVM-Log-Systemd-Tasks.sh'
then
sudo bash Build-MMDVM-Log-Backup-Restore-Tasks.sh
sudo bash Build-MMDVM-Log-Systemd-Tasks.sh
Alternately, you can simply clone this project:
rpi-rw
git clone https://github.com/kn2tod/pistar-mmdvm-log-backup-restores.git
cd pistar-mmdvm-log-backup-restores
sudo bash Build-MMDVM-Log-Backup-Restore-Tasks.sh
sudo bash Build-MMDVM-Log-Systemd-Tasks
(These two build scripts can be rerun as needed to incorporate any changes to the process.)
The remaining files (mmdvm-log-* and pistar-mmdvm-log-*) in this project are included for a
look-see for what's going on, as these files are created by the Build-* scripts and do not
need to be downloaded.
----
The MMDVM log files are created by the MMDVM "hat" firmware in the /var/log/pi-star directory;
only two or three files should be present: yesterday's file, today's file, and tomorrow's file;
as the firmware runs on UTC, a file postfixed with tomorrow's date (but with today's create
date) will be created depending on the time zone you are operating under (all the files in
/var/log/pi-star are time-stamped according to the current time zone). The files in this
directory are aged (rotated) so no more than two or three files should be present at any given
time.
Log files are copied to the /home/pi-star/.mlogs directory; backups are scheduled every 10
minutes and usually cover most operating situtations. Backups are also taken as part of a
regulary initiated shutdown/reboot. If the system shuts down unexpectedly (power outage),
some log data may be lost.
14 days of backups are currently retained; only the last 2-3 files are restored however.
The retention is set to allow post-operative analysis of the logs for various activities.
Aging (rotation) of the backups is performed via a script added to /etc/cron.daily/ which runs
when the normal nightly Pi-Star update processing kicks off.
In addition to the MMDVM log files, additional files are created to log shutdowns and restores:
zero-length files (reboot-*/restore-*); because RPi's do not have an RTC, the restore event log
file will not have the correct time stamps, as the restore usally occurs before the system is
resynced with the Internet so that last known (faked) time is used for the time-stamp.
---
For monitoring, use this command (add to .bash_aliases):
alias mlogq='ls -lAtr /home/pi-star/.mlogs; ls -lA /var/log/pi-star'
Sample result:
pi-star@pi-star-1(ro):~$ mlogq
total 5856
-rw-r--r-- 1 root root 227848 Oct 26 19:59 MMDVM-2021-10-26.log
-rw-r--r-- 1 root root 445545 Oct 27 19:59 MMDVM-2021-10-27.log
-rw-r--r-- 1 root root 546074 Oct 28 19:57 MMDVM-2021-10-28.log
-rw-r--r-- 1 root root 315078 Oct 29 19:58 MMDVM-2021-10-29.log
-rw-r--r-- 1 root root 794596 Oct 30 19:50 MMDVM-2021-10-30.log
-rw-r--r-- 1 root root 339394 Oct 31 19:36 MMDVM-2021-10-31.log
-rw-r--r-- 1 root root 0 Nov 1 13:58 reboot-2021-11-01-13:58:35
-rw-r--r-- 1 root root 0 Nov 1 13:58 restore-2021-11-01-13:58:57
-rw-r--r-- 1 root root 391424 Nov 1 19:55 MMDVM-2021-11-01.log
-rw-r--r-- 1 root root 359001 Nov 2 19:56 MMDVM-2021-11-02.log
-rw-r--r-- 1 root root 277095 Nov 3 19:59 MMDVM-2021-11-03.log
-rw-r--r-- 1 root root 324603 Nov 4 19:55 MMDVM-2021-11-04.log
-rw-r--r-- 1 root root 0 Nov 5 10:58 reboot-2021-11-05-10:58:52
-rw-r--r-- 1 root root 0 Nov 5 10:59 restore-2021-11-05-10:59:18
-rw-r--r-- 1 root root 650884 Nov 5 19:51 MMDVM-2021-11-05.log
-rw-r--r-- 1 root root 351975 Nov 6 19:58 MMDVM-2021-11-06.log
-rw-r--r-- 1 root root 660605 Nov 7 18:49 MMDVM-2021-11-07.log
-rw-r--r-- 1 root root 260342 Nov 8 18:56 MMDVM-2021-11-08.log
-rw-r--r-- 1 root root 29597 Nov 8 19:39 MMDVM-2021-11-09.log
total 936
-rw-r--r-- 1 root root 660605 Nov 7 18:49 MMDVM-2021-11-07.log
-rw-r--r-- 1 root root 260342 Nov 8 18:56 MMDVM-2021-11-08.log
-rw-r--r-- 1 root root 31047 Nov 8 19:42 MMDVM-2021-11-09.log
Sample (extract) from system logs:
pi-star@pi-star-1(ro):~$ grep -shie "MMDVM" {/var/log/syslog.1,/var/log/syslog}
Nov 9 13:20:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 13:20:02 pi-star-9 pistar-mmdvm-log-backups[5138]: MMDVM-2021-11-09.log backed up
Nov 9 13:20:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 13:30:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 13:30:02 pi-star-9 pistar-mmdvm-log-backups[14346]: MMDVM-2021-11-09.log backed up
Nov 9 13:30:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 13:40:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 13:40:01 pi-star-9 pistar-mmdvm-log-backups[23556]: MMDVM-2021-11-09.log backed up
Nov 9 13:40:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 13:50:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 13:50:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:00:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:00:01 pi-star-9 pistar-mmdvm-log-backups[9641]: MMDVM-2021-11-09.log backed up
Nov 9 14:00:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:10:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:10:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:20:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:20:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:30:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:30:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:40:02 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:40:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 14:50:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 14:50:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 15:00:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 15:00:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 15:10:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 15:10:01 pi-star-9 pistar-mmdvm-log-backups[9479]: MMDVM-2021-11-09.log backed up
Nov 9 15:10:01 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 15:20:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 15:20:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 15:30:01 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 15:30:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
Nov 9 15:40:02 pi-star-9 systemd[1]: Started Backup MMDVM log files.
Nov 9 15:40:02 pi-star-9 systemd[1]: mmdvm-log-backup.service: Succeeded.
------
Note: here is another approach to the problem of log retention:
<https://vk3erw.com/pi-star/pi-star-how-to-preserve-log-files-between-reboots>
This involves changing the location of the log files from temp (virtual) memory to
regular disk storage but does so by sacrificing some of the advantages of using
virtual memory. This works fine for faster Pi's with faster storage but can be
problematic for slower combinations of processor and storage. The rentention
periods for backups in this approach are also different than those provided in
the scripts listed above.