Update to libmseed v3 (#3)

ChangeLog

@@ -1,3 +1,11 @@
+2023.XXX: 4.0
+	- Update to libmseed 3, with support for miniSEED 3, URLs, etc.
+	- Filenames can now contain read range with end: file@start-end
+	- Added -H option to add HTTP headers
+	- Added -u option to take user:pass
+	- Remove -e (encoding) and -r (reclen) options
+	- Changed -Q to -P, -Q still supported but undocumented
+
 2017.283: 3.8
 	- Update libmseed to 2.19.5, with fix for leap second calculation.
 
@@ -24,7 +32,7 @@
 	truncation for SYNC output.
 
 2011.144: 3.4.1
-	- Update to libmseed 2.6.1 with fixes for more graceful handling of bad 
+	- Update to libmseed 2.6.1 with fixes for more graceful handling of bad
 	data and corner-case IRIS archive files.
 
 2011.057: 3.4
@@ -39,7 +47,7 @@
 
 2010.048: 3.3
 	- Update to libmseed 2.5, fix for sorting MSTraceSeg entries.
-	
+
 2010.026: 3.2
 	- Update libmseed to version 2.4.
 

Makefile

@@ -1,13 +1,23 @@
+# Automatically configure URL support if libcurl is present
+# Test for curl-config command and add build options if so
+ifneq (,$(shell command -v curl-config))
+        export LM_CURL_VERSION=$(shell curl-config --version)
+        export CFLAGS:=$(CFLAGS) -DLIBMSEED_URL
+        export LDFLAGS:=$(LDFLAGS) $(shell curl-config --libs)
+        $(info Configured with $(LM_CURL_VERSION))
+endif
 
-DIRS = libmseed src
+.PHONY: all clean
+all clean: libmseed
+	$(MAKE) -C src $@
 
-all clean static install gcc gcc32 gcc64 debug gccdebug gcc32debug gcc64debug ::
-	@for d in $(DIRS) ; do \
-	    echo "Running $(MAKE) $@ in $$d" ; \
-	    if [ -f $$d/Makefile -o -f $$d/makefile ] ; \
-	        then ( cd $$d && $(MAKE) $@ ) ; \
-	    elif [ -d $$d ] ; \
-	        then ( echo "ERROR: no Makefile/makefile in $$d for $(CC)" ) ; \
-	    fi ; \
-	done
+.PHONY: libmseed
+libmseed:
+	$(MAKE) -C $@ $(MAKECMDGOALS)
 
+.PHONY: install
+install:
+	@echo
+	@echo "No install method"
+	@echo "Copy the binary and documentation to desired location"
+	@echo

README.md

@@ -9,14 +9,26 @@ For usage infromation see the [msi manual](doc/msi.md) in the
 
 ## Downloading and building
 
-The [releases](https://github.com/iris-edu/msi/releases) area
+The [releases](https://github.com/earthscope/msi/releases) area
 contains release versions.
 
 In most environments a simple 'make' will build the program.
 
 The CC and CFLAGS environment variables can be used to configure
 the build parameters.
 
-## Licensing
+## License
 
-GNU GPL version 3.  See included LICENSE file for details.
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Copyright (C) 2023 Chad Trabant, EarthScope Data Service

doc/msi.1

@@ -1,16 +1,16 @@
-.TH MSI 1 2015/03/03
+.TH MSI 1 2023/02/17
 .SH NAME
-Mini-SEED Inspector
+miniSEED Inspector
 
 .SH SYNOPSIS
 .nf
 msi [options] file1 [file2 file3 ...]
 
 .fi
 .SH DESCRIPTION
-\fBmsi\fP is a general purpose Mini-SEED parser.  In addition to
-parsing and displaying the contents of Mini-SEED records, \fBmsi\fP
-will derive continuous trace information for Mini-SEED data.
+\fBmsi\fP is a general purpose miniSEED parser.  In addition to
+parsing and displaying the contents of miniSEED records, \fBmsi\fP
+will derive continuous trace information for miniSEED data.
 
 By default, \fBmsi\fP will output a single line of information to
 summarize each record parsed.  More verbose or different output can be
@@ -39,53 +39,54 @@ Print program usage and exit.
 Be more verbose.  This flag can be used multiple times ("-v -v" or
 "-vv") for more verbosity.
 
-.IP "-r \fIreclen\fP"
-Specify the input record length in bytes.  By default the
-record length of each input Mini-SEED record is automatically
-detected, this option forces the record length.
+.IP "-H \fIHeader\fP"
+Add custom header to URL-based reads.  Only available when compiled
+with URL support.
 
-.IP "-e \fIencoding\fP"
-Specify the data encoding format.  These encoding values are the same
-as those specified in the SEED 1000 Blockette.
+.IP "-u \fIuser:pass\fP"
+Set \fIuser\fP and \fIpass\fP credentials for URL-based reads.  Only
+available when compiled with URL support.
 
 .IP "-ts \fItime\fP"
-Limit processing to Mini-SEED records that contain or start after
+Limit processing to miniSEED records that contain or start after
 \fItime\fP.  The format of the \fItime\fP arguement
-is: 'YYYY[,DDD,HH,MM,SS,FFFFFF]' where valid delimiters are either
-commas (,), colons (:) or periods (.).
+is: 'YYYY[-MM-DDThh:mm:ss.ffff], or 'YYYY[,DDD,HH,MM,SS,FFFFFF]',
+or Unix/POSIX epoch seconds.
 
 .IP "-te \fItime\fP"
-Limit processing to Mini-SEED records that contain or end before
+Limit processing to miniSEED records that contain or end before
 \fItime\fP.  The format of the \fItime\fP arguement
-is: 'YYYY[,DDD,HH,MM,SS,FFFFFF]' where valid delimiters are either
-commas (,), colons (:) or periods (.).
+is: 'YYYY[-MM-DDThh:mm:ss.ffff], or 'YYYY[,DDD,HH,MM,SS,FFFFFF]',
+or Unix/POSIX epoch seconds.
 
 .IP "-M \fImatch\fP"
-Limit processing to Mini-SEED records that match the \fImatch\fP
-regular expression.  For each input record a source name string
-composed of 'NET_STA_LOC_CHAN_QUAL' is created and compared to the
-regular expression.  If the match expression begins with an '@'
-character it is assumed to indicate a file containing a list of
-expressions to match, see the \fBMATCH OR REJECT LIST FILE\fR section
-below.
+Limit processing to miniSEED records that match the \fImatch\fP
+regular expression.  This pattern is matched against the
+Source Identifier for each record, often following this pattern:
+'FDSN:<network>_<station>_<location>_<band>_<source>_<subsource>'
+
+If the match expression begins with an '@' character it is assumed
+to indicate a file containing a list of expressions to match, see
+the \fBMATCH OR REJECT LIST FILE\fR section below.
 
 .IP "-R \fIreject\fP"
-Limit processing to Mini-SEED records that do not match the
-\fIreject\fP regular expression.  For each input record a source name
-string composed of 'NET_STA_LOC_CHAN_QUAL' is created and compared to
-the regular expression.  If the reject expression begins with an '@'
-character it is assumed to indicate a file containing a list of
-expressions to reject, see the \fBMATCH OR REJECT LIST FILE\fR section
-below.
+Limit processing to miniSEED records that do not match the
+\fIreject\fP regular expression.  This pattern is matched against
+the Source Identifier for each record, often following this pattern:
+'FDSN:<network>_<station>_<location>_<band>_<source>_<subsource>'
+
+If the reject expression begins with an '@' character it is assumed
+to indicate a file containing a list of expressions to reject, see
+the \fBMATCH OR REJECT LIST FILE\fR section below.
 
 .IP "-n \fIcount\fP"
 Only process \fIcount\fP input records.
 
-.IP "-y         "
-Do not skip non-SEED data records.  By default the program will
-quietly skip any data which cannot be identified as a SEED data
-record, allowing the program to work with full SEED volumes and skip
-bad data.  This option is useful to help identify bad data records.
+.IP "-snd       "
+Skip non-miniSEED records.  By default the program will stop when
+it encounters data that cannot be identified as a miniSEED record.
+This option can be useful with full SEED volumes or files with bad
+data.
 
 .IP "-p         "
 Print details of each record header.  This flag can be used multiple
@@ -98,7 +99,7 @@ details.
 
 .IP "-L         "
 Include data latency when printing header details.  The latency is
-calcualed as the difference between the time of the last sample and
+calculated as the difference between the time of the last sample and
 the current time from the host computer.
 
 .IP "-s         "
@@ -121,7 +122,8 @@ Print a sorted trace list after processing all input records and
 suppress record-by-record output, i.e. trace list only.
 
 .IP "-tg        "
-Include gap estimates when approriate in trace listings.
+Include gap estimates when approriate in trace listings and
+suppress record-by-record output, i.e. trace list only.
 
 .IP "-tt \fIsecs\fP"
 Specify a time tolerance for constructing continous trace
@@ -153,18 +155,19 @@ of seconds.
 Print a sorted SYNC format trace list after processing all input
 records and suppress record-by-record output.
 
-.IP "-Q         "
-Additionally group input data based on SEED data qualitiy flags.  By
-default data are grouped by network, station, location, channel and
-adjacent time windows, this option adds data quality to the grouping
-parameters.
+.IP "-P         "
+Additionally group input data by publication.  Note: for miniSEED
+version 2 records, SEED data qualitiy values are translated to
+publication versions. By default data are grouped by network, station,
+location, channel and adjacent time windows, this option adds
+publication version to the grouping parameters.
 
 .IP "-tf \fIformat\fP"
 Specify the time stamp format for trace and gap/overlap lists.  The
 \fIformat\fP can be one of the following (default = 0):
 .nf
-  0 = SEED time, e.g. "2005,068,00:00:01.000000"
-  1 = ISO time, e.g. "2005-03-09T00:00:01.000000"
+  0 = SEED day-of-year time, e.g. "2005,068,00:00:01.000000"
+  1 = ISO year-month-day time, e.g. "2005-03-09T00:00:01.000000"
   2 = Epoch seconds, e.g. "1110326401.000000"
 .fi
 
@@ -173,16 +176,17 @@ Decompress/unpack data in input records and write the binary samples
 to \fIbinfile\fP.
 
 .IP "-o \fIoutfile\fP"
-Write all processed Mini-SEED records to \fIoutfile\fP.
+Write all processed miniSEED records to \fIoutfile\fP.
 
 .SH "INPUT FILES"
 
 An input file name may be followed by an \fB@\fP charater followed by
-a byte offset.  In this case the program will interpret the byte
-offset as the starting offset into the file.  Useful for diagnosing
-specific records.  As an example an input file specified as
+a byte range in the pattern \fBSTART[-END]\fP, where the END offset is
+optional.  As an example an input file specified as
 \fBANMO.mseed@8192\fP would result in the file \fBANMO.mseed\fP being
-read starting at byte 8192.
+read starting at byte 8192.  An optional end offset can be specified,
+e.g. \fBANMO.mseed@8192-12288\fP would start reading at offset 8192
+and stop after offset 12288.
 
 .SH "INPUT LIST FILE"
 A list file can be used to specify input files, one file per line.
@@ -201,7 +205,7 @@ data/day1.mseed
 data/day2.mseed
 data/day3.mseed
 .fi
-	
+
 .SH "MATCH OR REJECT LIST FILE"
 A list file used with either the \fB-M\fP or \fB-R\fP options contains
 a list of regular expressions (one on each line) that will be combined
@@ -216,16 +220,19 @@ example, if the following command line option was used:
 The 'match.list' file might look like this:
 
 .nf
-IU_ANMO_.*
-IU_ADK_00_BHZ.*
-II_BFO_00_BHZ_Q
+FDSN:IU_ANMO_.*
+FDSN:IU_ADK_00_B_H_Z
+FDSN:II_BFO_00_B_H_Z
 .fi
 
 .SH LEAP SECOND LIST FILE
+NOTE: A list of leap seconds is included in the program and no external
+list should be needed unless a leap second is added after year 2023.
+
 If the environment variable LIBMSEED_LEAPSECOND_FILE is set it is
 expected to indicate a file containing a list of leap seconds as
 published by NIST and IETF, usually available here:
-http://www.ietf.org/timezones/data/leap-seconds.list
+https://www.ietf.org/timezones/data/leap-seconds.list
 
 If present, the leap seconds listed in this file will be used to
 adjust the time coverage for records that contain a leap second.
@@ -234,5 +241,5 @@ Also, leap second indicators in the miniSEED headers will be ignored.
 .SH AUTHOR
 .nf
 Chad Trabant
-IRIS Data Management Center
+EarthScope Data Services
 .fi