A python script to shift the timestamp on syslog and httpd log data. Useful for forensicators combating time skew, time zones, and other such foolery.
$ timeshift.py --help
usage: timeshift.py [-h] [-m {syslog,httpdlog,rfc3339,cobaltstrike}]
[-o OFFSET] [-i {second,minute,hour,day}] [-y YEAR]
[-r INFILE] [-w OUTFILE]
Shift the date for all entries in an input data set by a specified interval of
time. Offset and interval options are required when using syslog mode.
optional arguments:
-h, --help show this help message and exit
-m {syslog,httpdlog,rfc3339,cobaltstrike}, --mode {syslog,httpdlog,rfc3339,cobaltstrike}
Type of timestamp to seek and adjust (default =
syslog)
-o OFFSET, --offset OFFSET
Amount of time to shift (pos/neg integer, only
required for "syslog" mode
-i {second,minute,hour,day}, --interval {second,minute,hour,day}
Interval of time to shift (only required for "syslog"
and "cobaltstrike" modes
-y YEAR, --year YEAR Year to assume (default 2018)
-r INFILE, --infile INFILE
Input file to process (default STDIN)
-w OUTFILE, --outfile OUTFILE
Output file to create - will be overwritten if exists
(default STDOUT)
Original contents of syslog file:
$ cat maillog
Jun 8 15:20:02 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root
Jun 8 15:20:02 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total
Jun 8 15:20:02 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00
Jun 8 15:20:02 proxy sm-msp-queue[2308]: starting daemon (8.13.8): queueing@01:00:00
Assuming source file is reflected in EDT (UTC-0400), change to UTC (as it should be!):
$ timeshift.py -m syslog -o 4 -i hour -r maillog
Jun 8 19:20:02 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root
Jun 8 19:20:02 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total
Jun 8 19:20:02 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00
Jun 8 19:20:02 proxy sm-msp-queue[2308]: starting daemon (8.13.8): queueing@01:00:00
Correct sendmail entries in source file to account for the system's clock being 23 seconds fast
$ grep sendmail maillog | ./timeshift.py -m syslog -o -23 -i second
Jun 8 15:19:39 proxy sendmail[2295]: alias database /etc/aliases rebuilt by root
Jun 8 15:19:39 proxy sendmail[2295]: /etc/aliases: 76 aliases, longest 10 bytes, 765 bytes total
Jun 8 15:19:39 proxy sendmail[2300]: starting daemon (8.13.8): SMTP+queueing@01:00:00
Original contents of HTTPD access log file:
$ cat lewestech.com-access
82.220.38.35 - - [11/Oct/2015:10:42:02 +0400] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0"
208.115.113.85 - - [11/Oct/2015:11:27:15 +0400] "GET /clients/clients/waggies-by-maggie HTTP/1.1" 301 128 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
65.254.225.173 - - [11/Oct/2015:11:29:49 +0400] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0"
82.239.166.225 - - [11/Oct/2015:11:58:49 +0400] "GET /tag/for572/ HTTP/1.1" 200 24951 "https://www.google.fr" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
82.239.166.225 - - [11/Oct/2015:11:58:50 +0400] "GET /wp-content/themes/lewestech/style.css HTTP/1.1" 200 10512 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
82.239.166.225 - - [11/Oct/2015:11:58:51 +0400] "GET /wp-content/themes/lewestech/scripts/utils.js HTTP/1.1" 200 123 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
Convert HTTPD access log with UTC offset to UTC native
$ cat lewestech.com-access | ./timeshift.py -m httpd
82.220.38.35 - - [11/Oct/2015:06:42:02 +0000] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0"
208.115.113.85 - - [11/Oct/2015:07:27:15 +0000] "GET /clients/clients/waggies-by-maggie HTTP/1.1" 301 128 "-" "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
65.254.225.173 - - [11/Oct/2015:07:29:49 +0000] "POST /wp-login.php HTTP/1.1" 200 4697 "http://lewestech.com/wp-login.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0"
82.239.166.225 - - [11/Oct/2015:07:58:49 +0000] "GET /tag/for572/ HTTP/1.1" 200 24951 "https://www.google.fr" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
82.239.166.225 - - [11/Oct/2015:07:58:50 +0000] "GET /wp-content/themes/lewestech/style.css HTTP/1.1" 200 10512 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
82.239.166.225 - - [11/Oct/2015:07:58:51 +0000] "GET /wp-content/themes/lewestech/scripts/utils.js HTTP/1.1" 200 123 "http://lewestech.com/tag/for572/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0"
Original contents of RFC3339 timestamp file:
$ cat messages
<5>2016-05-05T23:12:09.649085-05:00 quaff kernel:[27198521.247185] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21110 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-05T23:12:09.649157-05:00 quaff kernel:[27198521.247213] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21111 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-05T23:12:09.649161-05:00 quaff kernel:[27198521.247228] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21112 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-05T23:12:09.649163-05:00 quaff kernel:[27198521.247252] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21113 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-05T23:12:09.649165-05:00 quaff kernel:[27198521.247273] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21114 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
Convert RFC3339 timestamps with UTC offset to UTC native
$ ./timeshift.py -m rfc3339 -r messages
<5>2016-05-06T04:12:09.649085+00:00 quaff kernel:[27198521.247185] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21110 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-06T04:12:09.649157+00:00 quaff kernel:[27198521.247213] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21111 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-06T04:12:09.649161+00:00 quaff kernel:[27198521.247228] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21112 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-06T04:12:09.649163+00:00 quaff kernel:[27198521.247252] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21113 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
<5>2016-05-06T04:12:09.649165+00:00 quaff kernel:[27198521.247273] Firewall-DENY_INPUT: IN=venet0 OUT= MAC= SRC=188.143.a.b DST=205.186.x.y LEN=40 TOS=0x00 PREC=0x00 TTL=54 ID=21114 DF PROTO=TCP SPT=43052 DPT=80 WINDOW=0 RES=0x00 RST URGP=0
Original contenst of Cobalt Strike log file:
$ cat cobaltstrike.txt
08/23 21:38:35 [input] <user> download file.zip
08/23 21:38:35 [task] Tasked beacon to download file.zip
08/23 21:38:42 [checkin] host called home, sent: 37 bytes
08/23 21:38:42 [output]
started download of C:\Users\victim\Documents\file.zip (14892 bytes)
08/23 21:38:42 [output]
download of file.zip is complete
08/23 21:39:40 [input] <user> ls 20180823
08/23 21:39:40 [task] Tasked beacon to list files in 20180823
08/23 21:39:43 [checkin] host called home, sent: 26 bytes
Assuming source file is reflected in EDT (UTC-0400), change to UTC (as it should be!):
$ timeshift.py -m cobaltstrike -o 4 -i hour -r cobaltstrike.txt
2018-08-24T01:38:35 [input] <user> download file.zip
2018-08-24T01:38:35 [task] Tasked beacon to download file.zip
2018-08-24T01:38:42 [checkin] host called home, sent: 37 bytes
2018-08-24T01:38:42 [output]
started download of C:\Users\victim\Documents\file.zip (14892 bytes)
2018-08-24T01:38:42 [output]
download of file.zip is complete
2018-08-24T01:39:40 [input] <user> ls 20180823
2018-08-24T01:39:40 [task] Tasked beacon to list files in 20180823
2018-08-24T01:39:43 [checkin] host called home, sent: 26 bytes