Skip to content

Commit

Permalink
v1.8.0
Browse files Browse the repository at this point in the history
  • Loading branch information
circulosmeos committed Nov 21, 2024
1 parent 4cb03cf commit 51d51e3
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 8 deletions.
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,14 +98,14 @@ Copy gztool.c to the directory where you compiled zlib, and do:
Usage
=====

gztool (v1.7.0)
gztool (v1.8.0)
GZIP files indexer, compressor and data retriever.
Create small indexes for gzipped files and use them
for quick and random-positioned data extraction.
No more waiting when the end of a 10 GiB gzip is needed!
//github.com/circulosmeos/gztool (by Roberto S. Galende)

$ gztool [-[abLnsv] #] [-[1..9]AcCdDeEfFhilpPrRStTwWxXzZ|u[cCdD]] [-I <INDEX>] <FILE>...
$ gztool [-[abLnsv] #] [-[1..9]AcCdDeEfFhilpPqrRStTwWxXzZ|u[cCdD]] [-I <INDEX>] <FILE>...

Note that actions `-bcStT` proceed to an index file creation (if
none exists) INTERLEAVED with data flow. As data flow and
Expand Down Expand Up @@ -149,6 +149,8 @@ Usage
-P: like `-p`, but when used with `-[ST]` implies that checking
for errors in stream is made as quick as possible as the gzip file
grows. Warning: this may lead to some errors not being patched.
-q: extract data as indicated with `-[bL]` of a still-growing gzip file
and continue Supervising & extracting to STDOUT even after eof.
-r #: (range): Number of bytes to extract when using `-[bL]`.
Accepts '0', '0x', and suffixes 'kmgtpe' (^10) 'KMGTPE' (^2).
-R #: (Range): Number of lines to extract when using `-[bL]`.
Expand Down Expand Up @@ -249,6 +251,10 @@ Examples of use

$ gztool -L 10m -WI None compressed_text_file.gz

* Extract data from line 10 million, to STDOUT, on a still-growing file (`-q`), so if that line number still has not been reached, `gztool` will patiently wait until the gzip file grows enough to store that line, and show just it (`-R 1`). `-q` to wait for content can be used with both `-L` (line numbers) and `-b` (byte positions):

$ gztool -q -L 10m -R 1 compressed_text_file.gz

* Extract all data from a [rsyslog's veryRobustZip](https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html#veryrobustzip) that contains dirty data. This *corrupted-gzip-files* can arise when using [rsyslog's veryRobustZip omfile option](https://www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html#veryrobustzip) and the process that is logging is abruptly terminated and then restarted - this produces an incorrectly-terminated-gzip stream that is followed by another gzip stream **in the same file**. `gzip` (nor `zlib`) cannot read this files beyond the point of error. But `gztool` can correctly extract all data (and only good data) using `-p` (*patch*) parameter:

$ gztool -p -b0 compressed_text_file.gz
Expand Down Expand Up @@ -447,7 +453,7 @@ Other interesting links
Version
=======

This version is **v1.7.0**.
This version is **v1.8.0**.

Please, read the *Disclaimer*. In case of any errors, please open an [issue](https://github.com/circulosmeos/gztool/issues).

Expand Down
2 changes: 1 addition & 1 deletion configure.ac
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
AC_INIT([gztool], [1.7.0], [roberto.s.galende@gmail.com])
AC_INIT([gztool], [1.8.0], [roberto.s.galende@gmail.com])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_PROG_CC
AC_PROG_CC_C99
Expand Down
12 changes: 10 additions & 2 deletions gztool.1
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
.\" First parameter, NAME, should be all caps
.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection
.\" other parameters are allowed: see man(7), man(1)
.TH gztool 1 "Nov 17 2024" "gztool v1.7.0"
.TH gztool 1 "Nov 21 2024" "gztool v1.8.0"
.\" Please adjust this date whenever revising the manpage.
.\"
.\" Some roff macros, for reference:
Expand All @@ -21,7 +21,7 @@
gztool \- extract random-positioned data from gzip files, even like `tail -f`
.SH SYNOPSIS
.B gztool
.RI \ [\ [-[abLnsv]\ #]\ [-[1..9]AcCdDeEfFhilpPrRStTwWxXzZ|u[cCdD]]\ [-I\ <INDEX>]\ ]\ "files"\ ...
.RI \ [\ [-[abLnsv]\ #]\ [-[1..9]AcCdDeEfFhilpPqrRStTwWxXzZ|u[cCdD]]\ [-I\ <INDEX>]\ ]\ "files"\ ...
.br

Note that actions `-bcStT` proceed to an index file creation (if
Expand Down Expand Up @@ -130,6 +130,10 @@ like `-p`, but when used with `-[ST]` implies that checking
for errors in stream is made as quick as possible as the gzip file
grows. Warning: this may lead to some errors not being patched.
.TP
.BR \-q
extract data as indicated with `-[bL]` of a still-growing gzip file
and continue Supervising & extracting to STDOUT even after eof.
.TP
.BR \-r\ #
(range): Number of bytes to extract when using `-[bL]`.
Accepts '0', '0x', and suffixes 'kmgtpe' (^10) 'KMGTPE' (^2).
Expand Down Expand Up @@ -265,6 +269,10 @@ Creating and index for all "*gz" files in a directory:
.BR \ \ \ \ $\ gztool\ -L\ 10m\ -WI\ None\ compressed_text_file.gz
.br

* Extract data from line 10 million, to STDOUT, on a still-growing file (`-q`), so if that line number still has not been reached, `gztool` will patiently wait until the gzip file grows enough to store that line, and show just it (`-R 1`). `-q` to wait for content can be used with both `-L` (line numbers) and `-b` (byte positions):

.BR \ \ \ \ $\ gztool\ -q\ -L\ 10m\ -R\ 1\ compressed_text_file.gz
.br

* Extract all data from a \fBrsyslog's veryRobustZip\fP (//www.rsyslog.com/doc/v8-stable/configuration/modules/omfile.html#veryrobustzip) that contains dirty data. This *corrupted-gzip-files* can arise when using \fBrsyslog's veryRobustZip omfile option\fP and the process that is logging is abruptly terminated and then restarted - this produces an incorrectly-terminated-gzip stream that is followed by another gzip stream \fBin the same file\fP. `gzip` (nor `zlib`) cannot read this files beyond the point of error. But `gztool` can correctly extract all data (and only good data) using `-p` (*patch*) parameter:

Expand Down
4 changes: 2 additions & 2 deletions gztool.c
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
//
// LICENSE:
//
// v0.1 to v1.7* by Roberto S. Galende, 2019, 2020, 2021, 2022, 2023, 2024
// v0.1 to v1.8* by Roberto S. Galende, 2019, 2020, 2021, 2022, 2023, 2024
// //github.com/circulosmeos/gztool
// A work by Roberto S. Galende
// distributed under the same License terms covering
Expand Down Expand Up @@ -123,7 +123,7 @@
#include <config.h>
#else
#define PACKAGE_NAME "gztool"
#define PACKAGE_VERSION "1.7.0"
#define PACKAGE_VERSION "1.8.0"
#endif

#define _XOPEN_SOURCE 500 // expose <unistd.h>'s pread()
Expand Down

0 comments on commit 51d51e3

Please sign in to comment.