ElixirRetry

Simple Elixir macros for linear retry, exponential backoff and wait with composable delays.

Installation

Add retry to your list of dependencies in mix.exs:

  def deps do
    [{:retry, "~> 0.18"}]
  end

Ensure retry is started before your application:

  def application do
    [applications: [:retry]]
  end

Documentation

Check out the API reference for the latest documentation.

Features

Retrying

The retry([with: _,] do: _, after: _, else: _) macro provides a way to retry a block of code on failure with a variety of delay and give up behaviors. By default, the execution of a block is considered a failure if it returns :error, {:error, _} or raises a runtime error.

Both the values and exceptions that will be retried can be customized. To control which values will be retried, provide the atoms option. To control which exceptions are retried, provide the rescue_only option. For example:

retry with: ..., atoms: [:not_ok], rescue_only: [CustomError] do
  ...
end

Both atoms and rescue_only can accept a number of different types:

• An atom (for example: :not_okay, SomeStruct, or CustomError). In this case, the do block will be retried in any of the following cases:
  • The atom itself is returned
  • The atom is returned in the first position of a two-tuple (for example, {:not_okay, _})
  • A struct of that type is returned/raised
• The special atom :all. In this case, all values/exceptions will be retried.
• A function (for example: fn val -> String.starts_with?(val, "ok") end) or partial function (for example: fn {:error, %SomeStruct{reason: "busy"}} -> true). The function will be called with the return value and the do block will be retried if the function returns a truthy value. If the function returns a falsy value or if no function clause matches, the do block will not be retried.
• A list of any of the above. The do block will be retried if any of the items in the list matches.

The after block evaluates only when the do block returns a valid value before timeout. On the other hand, the else block evaluates only when the do block remains erroneous after timeout. Both are optional. By default, the else clause will return the last erroneous value or re-raise the last exception. The default after clause will simply return the last successful value.

Example -- constant backoff

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

This example retries every 100 milliseconds and gives up after 10 attempts.

Example -- linear backoff

result = retry with: linear_backoff(10, 2) |> cap(1_000) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

This example increases the delay linearly with each retry, starting with 10 milliseconds, caps the delay at 1 second and gives up after 10 attempts.

Example -- exponential backoff

result = retry with: exponential_backoff() |> randomize |> expiry(10_000), rescue_only: [TimeoutError] do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error
end

Example -- optional clauses

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
end

This example is equivalent to:

result = retry with: constant_backoff(100) |> Stream.take(10) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  e when is_exception(e) -> raise e
  e -> e
end

Example -- retry annotation

use Retry.Annotation

@retry with: constant_backoff(100) |> Stream.take(10)
def some_func(arg) do
  ExternalApi.do_something # fails if other system is down
end

This example shows how you can annotate a function to retry every 100 milliseconds and gives up after 10 attempts.

Delay streams

The with: option of retry accepts any Stream that yields integers. These integers will be interpreted as the amount of time to delay before retrying a failed operation. When the stream is exhausted retry will give up, returning the last value of the block.

Example

result = retry with: Stream.cycle([500]) do
  ExternalApi.do_something # fails if other system is down
after
  result -> result
else
  error -> error  
end

This will retry failures forever, waiting 0.5 seconds between attempts.

Retry.DelayStreams provides a set of fully composable helper functions for building useful delay behaviors such as the ones in previous examples. See the Retry.DelayStreams module docs for full details and addition behavior not covered here. For convenience these functions are imported by use Retry so you can, usually, use them without prefixing them with the module name.

Waiting

Similar to retry(with: _, do: _), the wait(delay_stream, do: _, after: _, else: _) macro provides a way to wait for a block of code to be truthy with a variety of delay and give up behaviors. The execution of a block is considered a failure if it returns false or nil.

wait constant_backoff(100) |> expiry(1_000) do
  we_there_yet?
after
  _ ->
    {:ok, "We have arrived!"}
else
  _ ->
    {:error, "We're still on our way :("}
end

This example retries every 100 milliseconds and expires after 1 second.

The after block evaluates only when the do block returns a truthy value. On the other hand, the else block evaluates only when the do block remains falsy after timeout. Both are optional. By default, a success value will be returned as {:ok, value} and an erroneous value will be returned as {:error, value}.

Pretty nifty for those pesky asynchronous tests and building more reliable systems in general!