Configuring Retry Policies in Choreo Connect

The backend endpoint can be unavailable for a short period of time due to various reasons, such as due to the load or due to a transient error. Choreo Connect handles such scenarios by configuring a Retry Policy which would make the router retry the HTTP request on behalf of the client. You can configure a Retry Policy at the endpoint level. In addition, you can set Global Level Retry Policy Configurations, which you can use to enforce limitations on the defined Endpoint Level Retry Policies.

Endpoint Level Retry Policy

You can define a Retry Policy for an endpoint in the x-wso2-production-endpoint and/or x-wso2-sandbox-endpoint OpenAPI extension of the API's OpenAPI definition, based on the endpoint type (i.e., production or sandbox), when deploying an API via the apictl (CLI Tool).

Example:

For example, if the Retry Policy is defined as follows, it will result in the router retrying three times if the backend responds with a 504 status code.

x-wso2-production-endpoints:
    urls:
      - https://localhost:2380/v3
    advanceEndpointConfig:
      retryConfig:
        count: 3
        statusCodes:
          - 504
x-wso2-sandbox-endpoints:
    urls:
      - https://localhost:2380/v3
    advanceEndpointConfig:
      retryConfig:
        count: 3
        statusCodes:
          - 504
Field Description
count The number of additional retry attempts before giving up.
statusCodes The HTTP status codes to retry. The accepted HTTP status code range is as follows: 401 to 598

When a Retry Policy is executed, the interval between two retries is a random number between 0 and (2^N-1)B.

  • N is the retry number.
  • B is the default base interval of 25ms (milliseconds).
  • The interval between the retries being random prevents a backend from getting a spike of retry requests (when triggered by many clients) and mitigates the potential unavailability of the backend that could occur due to the retries.

For more information, see x-envoy-max-retries in the official Envoy documentation.

Note

  • At the moment, MWARE ESB does not support adding a Retry Policy via the MWARE ESB UI.
  • If required, when configuring the production and sandbox endpoints, you can also specify the retry count in the Endpoints page via the MWARE ESB UI. For more information, see Endpoint timeouts.
  • The retry functionality in Choreo Connect will work when the value for Retries Before Suspension is set via the MWARE ESB UI. However, when run with Choreo Connect, this does not lead to a suspension of the endpoint, unlike when run with an API Gateway.
  • If the retry functionality is set via the Retries Before Suspension field in the MWARE ESB UI, the global level status codes in Choreo Connect will be considered when the retrying process takes place.

Global Level Retry Policy Configurations

Note

The Global Level Retry Policy Configurations are applicable only when Endpoint Level Retry Policies are defined.

  1. Open the Choreo Connect configuration file according to the deployment type you are using.

    Click here to see the configuration file location for your Choreo Connect deployment.

    Navigate to the correct folder path and open the config.toml or config-toml-configmap.yaml file based on your Choreo Connect deployment.

    DeploymentModeFile nameDirectory
    Docker ComposeChoreo Connect as a Standalone Gatewayconfig.toml<CHOREO-CONNECT_HOME>/docker-compose/choreo-connect/conf/
    Docker ComposeChoreo Connect with MWARE ESB as a Control Planeconfig.toml<CHOREO-CONNECT_HOME>/docker-compose/choreo-connect-with-apim/conf/
    KubernetesChoreo Connect as a Standalone Gatewayconfig-toml-configmap.yaml<CHOREO-CONNECT_HOME>/k8s-artifacts/choreo-connect/
    KubernetesChoreo Connect with MWARE ESB as a Control Planeconfig-toml-configmap.yaml<CHOREO-CONNECT_HOME>/k8s-artifacts/choreo-connect-with-apim/

  2. Use the following configurations to define the Global Level Retry Policy.

    Note

    If the Endpoint Level Retry Policies and the Global Level Retry Policies are not defined, the default values available for the Global Level Retry Policies are applied.

    [router.upstream.retry]
  maxRetryCount = 5
  baseIntervalInMillis = 25
  statusCodes = [ 504 ]
    Field Description
    maxRetryCount Maximum value that can be set as the count within the retry configurations in an OpenAPI definition or when configured via the MWARE ESB UI.
    baseIntervalInMillis Base interval for the Envoy's exponential retry back off algorithm.
    statusCodes The retry mechanism will be enabled for the HTTP status codes that are specified in this list. This is used when the retry configuration is set via the MWARE ESB UI by setting the Retries Before Suspension field or when all the status codes specified in the Endpoint Level Retry Policy are out of range.

    For more information with regard to the Global Level Retry Policy Configurations, see Upstream Retry.

See Also

Top