Skip to content

Latest commit

 

History

History
550 lines (386 loc) · 15 KB

README.adoc

File metadata and controls

550 lines (386 loc) · 15 KB
Raw

Redis Smart Cache

Redis Smart Cache is a JDBC query cache for Redis Stack, Redis Cloud, and Redis Enterprise.

Build Status Download Coverage

Redis Smart Cache lets you add caching to your application without changing the code.

Implemented as a wrapper around your backend database’s JDBC driver, Redis Smart Cache can cache slow, repeated SQL queries in Redis, bypassing expensive database calls and greatly improving response times.

Table of Contents

Background

Redis is an in-memory data store designed to serve data with the fastest possible response times. For this reason, Redis is frequently used for caching.

It’s not always easy to modify your code to utilize Redis as a cache. You need to think about a number of issues, including:

  • Result set serialization and deserialization

  • Fault-tolerance

  • Cache invalidation

This is why we built Redis Smart Cache. After adding a single dependency and setting some basic configuration, your JDBC application can take advantage of Redis as a cache without any changes to your code.

redis smart cache flow

Quick start

To understand how Redis Smart Cache works, it’s best to try it for yourself.

This example showcases an application that continuously performs queries against a MySQL database and uses Redis Smart Cache to cache query results.

First, clone this git repository:

git clone https://github.com/redis-field-engineering/redis-smart-cache.git
cd redis-smart-cache

Next, use Docker Compose to launch containers for MySQL, Grafana, Redis Stack, and a Redis Smart Cache example app instance:

docker compose up

Point your web browser to the Redis Smart Cache Grafana dashboard. This is available through the Grafana instance running at https://localhost:3000.

Initially, no caching rules are active. You can see this in the Query Mean Latency metric (~2 s).

Dashboard without caching

To add a caching rule, start the Smart Cache CLI by opening a new terminal window and running demo-cli.

./demo-cli

Select List application queries:

View application queries

Then press [Enter] to select a query to cache:

Select query

Once the query is displayed, enter a time-to-live (TTL) of 300s:

Set TTL

To commit the new caching rule, press 'c'. Then follow the prompt to confirm the configuration change.

The application will immediately begin caching queries. You can observe this from the Smart Cache Grafana dashboard:

Dashboard with Caching

Important: If the Dashboard is not being populated, check your version of Grafana, earlier versions may have issues. When in doubt, peform a fresh pull of grafana by issuing "docker pull grafana/grafana" prior to bring the application up. Version 9.5.2, while 7.x showed the behavior this note was added for (lack of data on the dashboard).

Installation

To use Redis Smart Cache with an existing application, you’ll need to add the Redis Smart Cache JDBC driver as an application dependency.

Maven
<dependency>
    <groupId>com.redis</groupId>
    <artifactId>redis-smart-cache-jdbc</artifactId>
    <version>0.3.3</version>
</dependency>
Gradle
dependencies {
    implementation 'com.redis:redis-smart-cache-jdbc:0.3.3'
}

The next step is to configure Redis Smart Cache, as described below.

Usage

First, ensure that your application is using Redis Smart Cache as its JDBC driver:

com.redis.smartcache.Driver

Next, set your JDBC URI to the URI of your Redis instance prefixed by jdbc: for example:

jdbc:redis://cache.redis.cloud:6379

See Lettuce’s URI syntax for all of the possible URI parameters you can use here.

Next step is providing bootstrap configuration which is done through JDBC properties.

Configuration

Refer to the Property Types section below for details on different property types.

Application

smartcache.name

  • Type: string

  • Default value: smartcache

Name used to uniquely identify an application. This is the prefix for all Redis keys used by Redis Smart Cache, such as cache entries, ruleset configuration, metrics, etc.

Note
 This property should be the same across all instances of an application.
smartcache.query-cache-capacity

  • Type: integer

  • Default value: 10000

Capacity of the parsed query cache.

Driver

smartcache.driver.class-name

Class name of the backend database JDBC driver, for example oracle.jdbc.OracleDriver.

smartcache.driver.url

JDBC URL for the backend database, for example jdbc:oracle:thin:@myhost:1521:orcl.

Additional properties

You can also include any property your backend JDBC driver requires, like username or password. These will be passed to the backend JDBC driver as is.

Redis

To further configure how Redis Smart Cache interacts with Redis, set the following properties:

smartcache.redis.key-separator

  • Type: string

  • Default value: :

Delimiter to use between key elements.

smartcache.redis.cluster

  • Type: boolean

  • Default value: false

Connect to a Redis Cluster.

smartcache.redis.tls

  • Type: boolean

  • Default value: false

Establish a secure TLS connection.

smartcache.redis.tls-verify

  • Type: string

  • Allowed values: NONE, CA, FULL

  • Default value: NONE

TLS verification mode. When set to NONE, no verification is performed. In CA mode the Certificate Authority and certificate are verified but not that the hostname matches. Use FULL mode for full certificate verification.

smartcache.redis.username

Authenticate using the provided username. Overrides username in Redis URI. Requires password.

smartcache.redis.password

Authenticate using the provided password. Overrides password in Redis URI.

smartcache.redis.codec-buffer-capacity

Maximum capacity of the buffer used to encode a result set.

Metrics

smartcache.metrics.enabled

  • Type: boolean

  • Default value: true

Enable publishing of metrics.

smartcache.metrics.registry

  • Type: string

  • Allowed values: REDIS, SIMPLE, JMX

  • Default value: REDIS

Meter registry type. Use REDIS for TimeSeries + RediSearch, SIMPLE for in-memory, JMX for JMX registry.

smartcache.metrics.step

Metrics publishing interval.

Property Types

Redis Smart Cache configuration properties support different value types.

boolean

The properties of type boolean support two values, true or false.

data size

The properties of type data size support values that describe an amount of data, measured in byte-based units. These units are incremented in multiples of 1024, so one megabyte is 1024 kilobytes, one kilobyte is 1024 bytes, and so on. For example, the value 6MB describes six megabytes.

The data size type supports the following units:

  • B: Bytes

  • kB: Kilobytes

  • MB: Megabytes

  • GB: Gigabytes

duration

The properties of type duration support values describing an amount of time, using the syntax of a non-negative number followed by a time unit. For example, the value 7m describes seven minutes.

The duration type supports the following units:

  • ns: Nanoseconds

  • us: Microseconds

  • ms: Milliseconds

  • s: Seconds

  • m: Minutes

  • h: Hours

  • d: Days

A duration of 0 is treated as zero regardless of the unit that follows. For example, 0s and 0m both mean the same thing.

Properties of type duration also support decimal values, such as 2.25d. These are handled as a fractional value of the specified unit. For example, the value 1.5m equals one and a half minutes, or 90 seconds.

integer

The properties of type integer support whole numeric values, such as 5 and 1000. Negative values are supported as well, for example -7. Integer type values must be whole numbers, decimal values such as 2.5 are not supported.

Some integer type properties enforce their own minimum and maximum values.

string

The properties of type string support a set of values that consist of a sequence of characters. Allowed values are defined on a property-by-property basis, refer to the specific property for its supported and default values.

list

The properties of type list support a set of values consistent with how Jackson properties are serialized in an array with field.position e.g. if the first rule in the configuration was a tables-any "customers,orders" it would be serialized as something equivalent to:

1) 1) "1682625171488-0"
   2)  1) "rules.1.tables-any.1"
       2) "customers"
       3) "rules.1.tables-any.2"
       4) "orders"

Rules

Redis Smart Cache uses rules to determine how SQL queries are cached. Rule configuration is stored in a Redis stream located at the key smartcache:config and can be modified at runtime. Redis Smart Cache will dynamically update to reflect configuration messages added to the stream.

Here is the default rule configuration:

1) 1) "1682622362951-0"
   2)  1) "rules.1.ttl"
       2) "0s"

This default configuration contains a single passthrough rule (i.e. it applies to all SQL queries) with a TTL of 0 seconds, which means that no SQL results will be cached.

Rules are processed in order and consist of criteria (conditions) and actions (results). Only the first rule with matching criteria will be considered, and its action applied.

Criteria

tables

  • Type: list

  • Example:

    1) 1) "1682625092479-0"
   2)  1) "rules.1.ttl"
       2) "50m"
       3) "rules.1.tables.1"
       4) "products"
       5) "rules.1.tables.2"
       6) "customers"
       7) "rules.1.tables.3"
       8) "orders"

Triggers if the given tables are exactly the same as the list in the SQL query (order does not matter).

tablesAny

  • Type: list

  • Example:

    1) 1) "1682625171488-0"
   2)  1) "rules.1.ttl"
       2) "50m"
       3) "rules.1.tables-any.1"
       4) "products"
       5) "rules.1.tables-any.2"
       6) "customers"
       7) "rules.1.tables-any.3"
       8) "orders"

Triggers if any of the given tables shows up in the SQL query.

tablesAll

  • Type: list

  • Example:

    1) 1) "1682625239844-0"
   2)  1) "rules.1.ttl"
       2) "50m"
       3) "rules.1.tables-all.1"
       4) "products"
       5) "rules.1.tables-all.2"
       6) "customers"
       7) "rules.1.tables-all.3"
       8) "orders"

Triggers if all the given tables show up in the SQL query.

queryIds

  • Type: list

  • Example:

    1) 1) "1682626067029-0"
   2) 1) "rules.1.ttl"
      2) "50m"
      3) "rules.1.query-ids.1"
      4) "5a934c95"
      5) "rules.1.query-ids.2"
      6) "beab0f6f"

Triggers if the SQL query ID matches any of the given IDs.

Tip
 An ID is the CRC32 hash of the SQL query. You can use an online CRC32 calculator like this one to compute the ID.
regex

  • Type: string

  • Example:

    1) 1) "1682626136637-0"
   2) 1) "rules.1.ttl"
      2) "50m"
      3) "rules.1.regex"
      4) "SELECT \\* FROM test\\.w*"

Triggers if given regular expression matches the SQL query.

Tip
 It is a good idea to test regexes at regex101.com.

Action

ttl

  • Type: duration

  • Default value: 0s

  • Example:

    1) 1) "1682626067029-0"
   2) 1) "rules.1.ttl"
      2) "50m"

Sets the time-to-live for the corresponding cache entry.

Use 0s to disable caching.

Examples

SELECT * FROM customers c, products p, orders o
Criteria Value Match

tables

orders, products

Check

tables

orders, products, customers

Check

tablesAny

transactions

Check

tablesAny

transactions, orders

Check

tablesAll

transactions, orders.products

Check

tablesAll

orders, products

Check

queryIds

aaff2b5c, 56789

Check

queryIds

a3bb9911, abcd1233

Check

regex

SELECT .+ FROM trans.*

Check

regex

SELECT .+ FROM cust.*

Check

Support

Redis Smart Cache is supported by Redis, Inc. on a good faith effort basis. To report bugs, request features, or receive assistance, please file an issue.

License

Redis Smart Cache is licensed under the MIT License. Copyright © 2023 Redis, Inc.