v0.13.0

Changelog

Features

• M3Query: Federated queries (e.g. for cross region/zone queries) configurable to return a warning and partial results than a hard error to route around unhealthy regions (#1938)
• M3Query: Add ability to turn on GRPC reflection for testing query endpoints using utilities like grpcurl (#1856)
• M3Query: Add ability to use multiple config files for config overrides (#1934)
• M3DB: Add ability to replicate writes best effort directly to multiple clusters by specifying multiple etcd clusters for M3DB client (#1859)
• M3Coordinator: Add ability to replicate Prometheus remote write requests by forwarding the compressed body to downstream remote write endpoints (#1922, #1940)

Bug Fixes

• M3Coordinator: Fix ability to delete coordinator placements (#1918)
• M3Aggregator: Ensure flush manager worker pool is at least size 1 irregardless of num CPUs (#1881)

Documentation

• M3: Add FAQ and seed with some frequently asked questions asking (#1927)
• M3Coordinator: Fix the cluster placement endpoint documentation in the manual cluster setup instructions (#1936)

v0.12.0

Changelog

Features

• M3DB: Add support for limiting the amount of outstanding repaired data that can be held in memory at once (#1885)
• M3Coordinator: Add namespace and placement information to the dump returned by the debug endpoint (#1896)
• M3Coordinator: Add DELETE api for the /topic endpoint (#1926)

Performance

• M3Query: Improve performance of temporal functions (#1917)
• M3DB: Only trigger bootstrap during topology change if a node receives new shards. This should significantly reduce the amount of time certain topology changes (such as node removes) took when performed on larger clusters (#1868)

Features

• m3aggregator: Make YAML serialization roundtrip for config related types (#1864)

v0.11.0

Changelog

Migration Disclaimer

Version 0.11.0 of M3 includes further work on supporting writing data at arbitrary times (within the retention period). While most of these changes are transparent to the user in terms of functionality and performance, we had to make a change to the naming format of the files that get persisted to disk (#1720). This change was required to handle multiple fileset volumes per block, which is necessary after introducing "cold flushes" (#1624).

The redesign is backwards compatible but not forwards compatible. This means that you should be able upgrade your < 0.11.0 clusters to 0.11.0 with no issues, but you will not be able to downgrade without taking some additional steps.

Troubleshooting and Rolling Back

If you run into any issues with the upgrade or need to downgrade to a previous version for any reason, follow these steps:

1. Stop the node that is having trouble with the upgrade or that you're trying to downgrade.
2. Remove the filesets that include a volume index in them, e.g. this is a filename with the new volumed format: fileset-1257890400000000000-0-data.db, and this is the corresponding filename in the original format: fileset-1257890400000000000-data.db.
3. Modify the bootstrappers config in the M3DB YAML file from filesystem, commitlog, peers, uninitialized_topology to filesystem, peers, commitlog, uninitialized_topology. This will force the node to bootstrap from its peers instead of the local snapshot and commitlog files it has on disk, which is important otherwise when the node restarts, it will think that it has already been bootstrapped.
4. Turn the node back on.

Features

• M3DB: Add cluster repair capability for non-indexed data (#1831, #1849, #1862, #1874)
• M3DB: Enable arbitrary out of order writes for non-indexed data (#1780, #1790, #1829)
• M3DB: Expose debug dump on /debug/dump for zip file of profiles and data about the environment for debugging (#1811)
• M3Query: Allow lookback duration to be set in query parameters for configurable lookback windowing (#1793)

Performance

• M3DB: Improve base line write performance using pre-encoded tags instead of encoding just before commit log write (#1898, #1904)
• M3DB: Avoid allocating expensive digest readers in seek.Open() (#1835)

Bug Fixes

• M3Query: Fix PromQL variadic functions with no params specified (#1846)
• M3Query: Fix PromQL absent function (#1871)
• M3Query: Fix PromQL binary comparisons that use the time function (#1888)
• M3Query: Fix multi-deployment queries (e.g. cross region) using temporal functions (#1901)
• M3DB: Index queries to local node now returns only IDs owned by the node at time of read (#1822)
• M3DB: Automatically create folder for KV cache, avoiding common error logs (#1757)
• M3Query: Batch remote server responses to avoid large individual RPC messages (#1784)
• M3DB: Use rlimit and setcap together to raise file descriptor limits to desired setting if possible (#1792, #1745, #1850)

Documentation

• M3DB: Add shard and recommended block sizes (#1890)
• M3Coordinator: Improve namespace specific read/write header documentation (#1884)

v0.10.2

Changelog

Performance

• M3DB: First of a set of incremental bootstrap improvements, improve commit log reader speed (#1724)

Bug Fixes

• M3Query: Add verbose step size parsing errors to identify problematic queries (#1734)
• M3Coordinator: Add verbose errors to identify metrics arriving too late for aggregation and make lateness policies able to be set by config (#1731)

v0.10.1

Changelog

Bug Fixes

• M3Coordinator: Use table based approach for aggregation tile buffer past calculation with good defaults for Prometheus remote write latency (#1717)

v0.10.0

Changelog

Features

• M3Query: Add multi-zone and multi-region configuration for coordinator (#1687)
• M3Query: Add debug param to GET /api/v1/namespace endpoint for better readability (#1698)
• M3Coordinator: Add "ingest_latency" histogram metric and return datapoint too old/new errors with offending timestamps (#1716)

Performance

• M3DB: Add forward index write capability which eases index pressure at block boundaries (#1613)
• M3Query: Drop empty series from appearing in output when keepNaNs option is set to disabled (#1682, #1684)
• M3DB: Build and release M3DB with Go 1.12 (#1674)

Bug Fixes

• M3DB: Fix a bug where peer bootstrapping would sometimes get stuck for a short period of time due to other M3DB nodes (temporarily) returning more than one block of data for a given time period (#1707)
• M3DB: Fix a bug that would cause multiple orders of magnitude slow-down in peer bootstrapping / node replacements when one of the nodes in the cluster was hard down (#1677)
• M3Query: Fix a bug with parsing Graphite find queries by adding MatchField and MatchNotField match types, and explicitly making use of them in graphite queries (#1676)
• M3Query: Propagate limit settings when using Prometheus Remote Read (#1685)
• M3Coordinator: Return a 404 rather than a 500 if we attempt to delete a nonexistent placement (#1701)
• M3Coordinator: Return HTTP 400 if all sent samples encounter too old/new or other bad request error (#1692)

v0.9.6

0.9.6

Bug Fixes

• M3DB: Revert Docker setcap change to allow image to run with no special flags (#1672)

v0.9.5

Performance

• M3DB: Use ReadAt() pread syscall to reduce syscalls per time series data read and half the number of open FDs with default configuration (#1664)

Bug Fixes

• M3Query: Add missing support for PromQL unary expressions (#1647)
• M3DB: Call setrlimit to set open FDs hard limit for docker image and in Dockerfile use setcap to enable capability (#1666)
• M3DB: Updated Tally to 3.3.10 to issue correct error message when listen address conflict occurs with Prometheus reporter (#1660)

v0.9.4

Changelog

Bug Fixes

• M3Query: Adding complete tags query fanout support with integration test (#1656)

v0.9.3

Changelog

Bug Fixes

• M3Query: Add docker integration test for query fanout and fix query aggregating results (#1652)
• M3DB: Restore process stats disabled by the open FD counting performance fix (#1648)