Cardano DB Sync Sancho-4.1.0

NOTE: THIS RELEASE IS ONLY RECOMMENDED FOR USE ON THE SANCHONET AND PUBLIC TEST ENVIRONMENTS (E.G. PREVIEW) AT THIS TIME. IT IS NOT RECOMMENDED FOR USE ON MAINNET UNTIL FURTHER TESTING

This release adds support for node-8.8.0-pre and probably node-8.9.0. It may be compatible with later node versions, but it hasn't been tested
It is advised to resync from genesis when upgrading to this release

Schema
Docker image

Changelog

• Moved modularity flags to the db-sync config, documented here
• Made committee proposals more readable, with the addition of a new table new_committee_member for new members 1633. Also new_committee is renamed to new_committee_info
• Added a new table committee_hash which is now referenced by other tables instead of using the hash directly. This in particular solves 1571
• Added support for offchain vote metadata and partial support for CIP-100 and CIP-108
• Fixed script hash false negative check 1646

Vote offchain data

This release partially follows CIP-100 and proposed CIP-108. It fetches the off chain data, pointed by on chain voting anchors and populates the off_chain_vote_data table or off_chain_vote_fetch_error if fetching or parsing fails. The off_chain_vote_data is extended with all optional or necessary fields as they are described at CIP-100 and CIP-108. Also three new tables are added off_chain_vote_author, off_chain_vote_reference, off_chain_vote_external_update.

The hash and witness validation is not fully implemented, so db-sync will store a warning in almost all cases at off_chain_vote_data.warning and off_chain_vote_author.warning. Users that decide that some metadata are in fact invalid can edit the off_chain_vote_data.is_valid

Users can also directly access the original bytes at off_chain_vote_data.bytes or the original json at off_chain_vote_data.json. In particular the latter is stored in a a Postgres jsonb type, which allow a big number of builtin queries. For example

SELECT * FROM off_chain_vote_data where json ? 'authors'
returns all the metadata which have a build author or

SELECT * FROM off_chain_vote_data where json @> '{"authors":"xyz"}';
returns all metadata with a specific authors. Postgres makes all these queries indexable.

Fetching or parsing errors are inserted into off_chain_vote_fetch_error. These can be 404 "Not Found", expected JSON, but got : "text/html, InvalidUrlException, JSON decode error to name a few.