-
Notifications
You must be signed in to change notification settings - Fork 3
Home
Welcome to the emf wiki!
Backup Notes:
Backup DB (minus emissions schema) on a daily basis
Backup changed datasets/qa steps from that day during that nightly backup
Weekly process will tar backups to ASM
From Brad Viviano (need to format this...)
I will, I just didn't want to interfere with what you where doing. At some point we'll have to schedule the initial full backup of all the Emissions data (set the start date to be 1900-01-01 or something and let it run). I've gotten the non emissions backup running. The script also now creates a file /usr/pgsql/emf-backup_lastrun.txt which holds the time stamp of the last time the backup ran successfully, if you don't specify a date on the command line with "-s", it will read that file and use that time/date for the emissions SQL query. This way we can schedule it out of cron and it'll backup every change since the last time it ran successfully. The time stamp has resolution to the second, so would let us schedule backups during the day if you needed/wanted (backup at noon let's say, so it would then update /backups/EMF with anything that had changed in the morning).
I set up the non emissions backup file so the file name has the date attached to it:
-bash-4.1$ ll /backups/EMF/nonemissions_schema total 728584 -rw------- 1 postgres postgres 746062340 Jun 11 15:03 emf_20150611.bck
There is an entry in the /etc/epa/backups/emf-backup.cfg configuration file that lists how many of those to keep, currently it is set at 10. On the 11th day, the script will delete the oldest non-emissions backup, if the previous backup completed without error.
There is a similar setting for the log files, the script will keep 60 days worth of backup log files and then clean up so we don't inadvertently fill up /var.
Last thing I need to do, other then doing some checking to make sure all error conditions are accounted for, is to get the script, once a week (every Sunday or something) to tar up all the EMF data and copy it to ASM. That will give you your point in time snapshot we talked about. I plan on working on that today.
In regards to the backup report. You can always review the log on sage in /var/log/emf_backup
root@sage emf_backup]# ls -l /var/log/emf_backup/ total 4 -rw------- 1 postgres postgres 3445 Jun 11 15:03 emf_20150611.log
But the script also has a few options for mailing that log automatically. I can send you a copy via email of the backup log every time it runs, only when it errors out, or not at all. Let me know if you and/or Alison would like to see a copy of the log and under what conditions (again, it's just a setting in the configuration file, so no extra work on my part).