proxy_cache_use_stale like directive

[lloydzhou]

the proxy_cache_use_stale directive can determines in which cases a stale cached response can be used when an error occurs during communication with the proxied server.

but when i using srcache+redis, the reids always delete keys...

[agentzh]

@lloydzhou This is relying on the cache backend storage (like redis) to implement expiration. If you have ideas to implement this "use stale" feature in ngx_srcache, then feel free to submit a pull request. Thanks!

[lloydzhou]

@agentzh
there's a demo: "caching-with-redis", may be we can just set the long expire time. and return one header named "X-Expire" from srcache_fetch request, then we can calculate the real expire time in srcache-module.

srcache_fetch GET /redis $key;
srcache_store PUT /redis2 key=$escaped_key&exptime=3720;
# we just want to set expire 120 seconds
# but we also want to server stale data in 1 hours
# so set exptime to 3600+120=3720;

we need update redis-module to add header "X-Expire".

 location = /redis {
   internal;
   set_md5 $redis_key $args;
   redis_pass 127.0.0.1:6379;
 }

[agentzh]

@lloydzhou Maybe you can just use a little bit of Lua for such extra logic in your srcache_fetch subrequest. No need to change existing nginx C modules as far as I can see :)

[lloydzhou]

Cached in reids, and set expire time into "X-Expire" header.

location /api {
    default_type text/css;

    set $key $uri;
    set_escape_uri $escaped_key $key;

    srcache_fetch GET /redis key=$escaped_key&db=1;
    srcache_store PUT /redis2 key=$escaped_key&exptime=3720&db=1;
    echo hello world;
    add_header X-Cache $srcache_fetch_status;
    add_header X-Store $srcache_store_status;

    # fastcgi_pass/proxy_pass/drizzle_pass/postgres_pass/echo/etc
}

get cached content from redis by using redis2-nginx-module:

location = /redis_internal {
    internal;

    set_unescape_uri $key $arg_key;
    set_md5 $key;

    redis2_query select $arg_db;
    redis2_query ttl $key;
    redis2_query get $key;
    redis2_pass 127.0.0.1:6379;
}

parse the result, and set the expire time into "X-Expire" header

location = /redis_internal {
    internal;

    set_unescape_uri $key $arg_key;
    set_md5 $key;

    redis2_query select $arg_db;
    redis2_query ttl $key;
    redis2_query get $key;
    redis2_pass 127.0.0.1:6379;
}

parse the result, and set the expire time into "X-Expire" header

location = /redis {
    internal;

    content_by_lua '
        local res = ngx.location.capture("/redis_internal", {args=ngx.var.args})
        local c = res.body
        if not (res.status == 200 and string.sub(c, 1, 3) == "+OK") then
            ngx.exit(500)
        end
        local i, j = string.find(c, ":%d+", 6)
        if not i then
            ngx.exit(404)
        end
        ngx.header["X-Expire"] = string.sub(c, i+1, j)
        local m, n = string.find(c, "$%d+", j+2)
        if not m then
            ngx.exit(404)
        end
        ngx.header["Content-Length"] = string.sub(c, m+1, n)
        ngx.print(string.sub(c, n+3, -2))
        ngx.exit(ngx.HTTP_OK)
    ';
}

Store content into redis by using redis2-nginx-module

location = /redis2 {
    internal;

    set_unescape_uri $exptime $arg_exptime;
    set_unescape_uri $key $arg_key;
    set_md5 $key;

    redis2_query select $arg_db;
    redis2_query set $key $echo_request_body;
    redis2_query expire $key $exptime;
    redis2_pass 127.0.0.1:6379;
}

TODO
at last need check the expire time in srcache-module, by parse the header. so we can server stale data in some case.

[lloydzhou]

@agentzh
i see there's one directive "srcache_response_cache_control" to control the cached content.
so, maybe we can using this directive to server stale data. the real expire time is min of "response cache control" and "$expire_time".

[agentzh]

@lloydzhou Nope, not really. That directive runs in the header filter phase. It's already too late to serve anything else in that phase. The ngx_proxy module does implement the use stale cache thing directly in the underlying upstream mechanism. The best bet for us to redo it in Lua is to rely on the upcoming balancer_by_lua* directive in ngx_lua, which also runs directly inside the stock upstream facility.

[lloydzhou]

@agentzh
I have write one little Lua library to server stale in redis.
https://github.com/lloydzhou/lua-resty-cache
always set the redis expires to (real expires time + stale time)
using lua-resty-lock to make sure only one request should be using to populate new cache

[agentzh]

@lloydzhou cool :)

We have something similar with memcached at CloudFlare. In addition, we have a secondary caching layer based on lua_shared_dict ;)

[lloydzhou]

@agentzh
I found another way to make srcache-module work with stale data.
using nginx-eval-module to set $skip, and then use it to config the "srcache_fetch" and "error_page" directive.
in eval subrequest, check the ttl, can skip fetch cache and auto update cache.
using error_page to server stale data, if there's stale cache in redis and catch error in backend server.

upstream www {
    server 127.0.0.1:9999;
}
upstream redis {
    server 127.0.0.1:6379;
    keepalive 1024;
}
lua_shared_dict srcache_locks 100k;

server {
    listen 80;

    location @fetch {
        default_type text/css;
        set $key $uri;
        srcache_fetch GET /redis $key;
        add_header X-Cache $srcache_fetch_status;
        echo not found $key;
    }

    location / {
        default_type text/css;

        set $key $uri;
        set_escape_uri $escaped_key $key;

        eval $fetch {
            rewrite /eval_([0-9]+)/(.*) /$2 break;
            content_by_lua '
                local parser = require "redis.parser"
                local lock = require "resty.lock"
                local key = ngx.var.uri
                local stale = 10
                local res = ngx.location.capture("/redisttl", { args={key=key}})
                local ttl = parser.parse_reply(res.body)
                -- stale time, need update the cache
                if ttl < stale then
                    -- cache missing, no need to using srcache_fetch, go to backend server, and store new cache
                    if ttl == -2 then
                        ngx.print("MISS")
                    else
                        -- remove expire time for key
                        if ttl > 0 then
                            ngx.location.capture("/redispersist", { args={ key=key } })
                        end
                       -- get a lock, if success, do not fetch cache, create new one, the lock will release in "exptime".
                        -- if can not get the lock, just using the stale data from redis.
                        local l = lock:new("srcache_locks", {exptime=5, timeout=0.01})
                        if l and l:lock(key) then
                            ngx.print("STALE")
                        else
                            ngx.print("FETCH")
                        end
                    end
                else
                    ngx.print("FETCH")
                end
            ';
        }

        if ($fetch = FETCH){ srcache_fetch GET /redis $key;}
        srcache_store PUT /redis2 key=$escaped_key&exptime=15;
        add_header X-Fetch $fetch;
        add_header X-Cache $srcache_fetch_status;
        add_header X-Store $srcache_store_status;
        proxy_pass http://www;
        if ($fetch = STALE){ error_page 502 =200 @fetch;}
    }

    location = /redisttl {
        internal;
        set_unescape_uri $key $arg_key;
        set_md5 $key;
        redis2_query ttl $key;
        redis2_pass redis;
    }
    location = /redispersist {
        internal;
        set_unescape_uri $key $arg_key;
        set_md5 $key;
        redis2_query persist $key;
        redis2_pass redis;
    }
    location = /redis {
        internal;

        set_md5 $redis_key $args;
        redis_pass redis;
    }
    location = /redis2 {
        internal;
        set_unescape_uri $exptime $arg_exptime;
        set_unescape_uri $key $arg_key;
        set_md5 $key;
        redis2_query set $key $echo_request_body;
        redis2_query expire $key $exptime;
        redis2_pass redis;
    }
}

[lloydzhou]

@agentzh
i have update the config, can update cache in background by using "lua-resty-http" + "ngx.timer.at".
https://gist.github.com/lloydzhou/d1dfc41f56866c4b82a6

[rahul286]

@agentzh I came here looking for a similar solution. Any chances this will get merged into srcache-nginx-module.

fastcgi_cache_use_stale is really helpful when people entire cache gets flushed on a large site.

[rahul286]

@lloydzhou May I know what is the latest update on this? Are you still using https://github.com/lloydzhou/lua-resty-cache or https://gist.github.com/lloydzhou/d1dfc41f56866c4b82a6 ?

[lloydzhou]

@rahul286 i using https://github.com/lloydzhou/lua-resty-cache in production

[gaoyuear]

here is my solution of this problem, combine the srcache with nginx's proxy_cache to setup a two level caching.

here is the snippet

# standard srcache config
  upstream srcache_backend {
     server redis-server:6379;
     keeplive 5;
 }
 
  # standard proxy_cache config
 proxy_cache_path cache_tmp keys_zone=tagcache:500m;
 
 server {
    # standard srcache config
   location = /redis {
       ....
       redis_pass srcache_backend;
   }
   location = /redis2 {
       ....
       redis2_pass srcache_backend;
   }
   location / {
       # standard srcache config
       set $cache_skip 0;
       srcache_store_skip $cache_skip;
       srcache_fetch GET /redis db=$cache_database&key=$cache_key;
       srcache_store PUT /redis2 db=$cache_database&exptime=3600&key=$cache_key;
       srcache_store_statuses 200 204;
 
       #  standard proxy_cache config
       proxy_cache tagcache;
       proxy_cache_key "$cache_key";
       proxy_cache_lock on;
       proxy_cache_lock_age 5s;
       proxy_cache_lock_timeout 5s;
       proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
       proxy_cache_background_update on;
       proxy_cache_valid 200 204 1h;     # better to match the srcache expire
       
       proxy_set_header Host $upstream_host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header X-Forwarded-Scheme $scheme;
       proxy_pass $target;
 
       ### the trick is here  ###
       # do not store in srcache when proxy_cache is stale
       header_filter_by_lua_block {
            local cache_status = ngx.var.upstream_cache_status
            if cache_status == 'STALE' or cache_status == 'UPDATING' then
                ngx.var.cache_skip = 1
            end
       }
    }
}

if add '$srcache_fetch_status $srcache_store_status $upstream_cache_status' in the access log, we could see the following:

1. first batch of requests when cache is cold
   one request is 'MISS STORE MISS', that one request is passing to upstream and updating both proxy_cache on local disk and srcache on redis.
   other requests are 'MISS STORE HIT', proxy_cache locks them (proxy_cache_lock directive), then fetch the cache result of the first proxied request. here each request also updates the redis once.

2. following batch of requests when cache is hot
   status is 'HIT BYPASS -', srcache works

3. following simultaneous requests when cache is expired (srcache expires, local disk has stale content)
   all requests get 'MISS BYPASS STALE' or 'MISS BYPASS UPDATING', proxy cache serves stale content to clients immediately, meanwhile one independent request is issued to upstream server and then update the disk cache. During this period, srcache doesn't store the stale cache.

4. following one request when local disk cache is updated.
   status is 'MISS STORE HIT', populates the local disk cache to srcache.

The benefits of above config:

1. maximize the leverage of the mature plugins of nginx, especially the complex functionality of proxy_cache.
2. two levels of cache, srcache is better for nginx cluster that share the one cache content, local disk is the second level to be the fallback when redis is down.

[kapouer]

Now we just need a way to send stale link preload/preconnect headers as "103 Early Hints", instead of sending the whole staled response, while waiting for the fresh response to be available.