Fastly-generated errors

When errors happen during the processing of a request, Fastly may generate a synthetic HTTP response and these may be delivered to the end user, often with a 503 (service unavailable) status code.

This page describes the possible conditions that produce this outcome and how to handle such errors at the edge.

WARNING: Purging cache in response to unexplained elevated error rates may make things worse. Purges typically increase traffic to origin, which may exacerbate a problem rather than fixing it. Try to determine the cause of errors and be certain purging is the right solution.

Sources of errors

Often you'll become aware of errors in your Fastly service because undesirable HTTP responses (with HTTP status codes of 400 or higher) are being delivered to end users and reported to you.

But this doesn't necessarily mean something is wrong with your Fastly service. HTTP error responses may have a variety of possible causes:

Your origin server served it. A well formed, valid HTTP response from your origin server is not considered an error by Fastly (and will not invoke vcl_error in VCL services), though it may itself represent an error that has occurred in your origin server. Unless you configure your service to handle such responses, they will be delivered to end users, and potentially cached.
An explicit error statement in your VCL was executed. The error statement can be used in most VCL subroutines to immediately stop the normal VCL workflow, construct a synthetic response object, and transfer control to the vcl_error subroutine.
You constructed an error response in a Compute@Edge application. Every Compute@Edge SDK provides the ability to construct responses from scratch, and you may have a code path that generates an error response.
Something happened that caused Fastly to throw an error. These errors are created by Fastly and surfaced in various ways - both in VCL and Compute@Edge services. This page describes the possible errors of this type, how they are triggered and how to resolve them.

Fastly-generated errors

These are errors that are generated by Fastly which will halt your program and produce output to the client. Often they will emit output to STDERR too.

End user viewable error policy

Fastly believes that you own the relationship with your customers, and content that end users receive from your website should not be obviously delivered by Fastly. In general, users are not familiar with the role of edge networks, and should not need to be. Therefore, where errors occur and a response must be sent to the client, but your service is not able to produce one itself, Fastly will send a response that is either empty or unbranded.

Routing errors

Fastly may respond to some requests directly from our routing systems, without invoking your service. This often happens when the request is presented on a port that we do not support, or is syntactically invalid. In such cases our response is either to drop the connection or to issue a standards-compliant HTTP error response. The following list itemizes notable responses that may be generated at this layer.

HTTP Status	Reason text	Description
`414`	`URI too long`	The length of the URL exceeds our platform limits. On Compute@Edge services, this may create a `400` instead.
`421`	`Misdirected Request`	Indicates a mismatch between the hostname provided in the TLS handshake (e.g., in the SAN entries in the TLS certificate) and the requested hostname in the HTTP `Host` header. This may happen if you are attempting to do domain fronting which is not supported by Fastly by default. Consider setting a host header override on the backend.
`503`	`Loop detected`	The request appears to originate from the same Fastly service that it is trying to invoke, or the request has transited too many Fastly servers. See loop detection.

Fastly-generated routing errors may occur when an end user request is presented to a Fastly POP, but also when requests are forwarded from one Fastly service to another (see service chaining) or from one Fastly POP to another (see shielding).

Error responses from our routing systems will either have an empty response body or will cite error 54113, a reference to Fastly's Autonomous System number.

NOTE: Error 54113 supersedes the obscure guru mediation error which we used for many years.

If you see error responses from your Fastly hosted domain and believe these errors are being generated before your code is invoked, contact Fastly support for assistance.

Errors in VCL

When an error occurs during execution of a VCL service, Fastly generates a synthetic HTTP response with a 503 status code, and invokes vcl_error. The value of obj.status is always 503 and obj.response is set to a descriptive name for the error. You can therefore intercept Fastly-generated errors by writing custom VCL in vcl_error:

sub vcl_error { ... }

Fastly VCL

if (obj.status == 503 && obj.response == "backend read error") { ... }

The following table itemizes the possible errors that may be generated by Fastly during VCL execution.

WARNING: obj.response is case sensitive and capitalization is somewhat inconsistent between different errors. Starting from HTTP/2 the HTTP response "reason" text (obj.response) is no longer sent along with the response, so if delivered to the end user unmodified, it is generally not possible to tell the difference between these errors.

`obj.response`	Description
`All backends failed or unhealthy`	A director used for balancing requests among a group of backends fails because all the backends are unhealthy or multiple backends from which the director tried to fetch failed with the same error. If the director uses a `chash` policy, the error is suffixed with the policy name, i.e. `"All backends failed or unhealthy (chash)"`.
`Backend is unhealthy`	Your configured health check reports the selected backend as down. Fastly will not send traffic to backends that are failing health checks, but traffic can be automatically routed away from these backends using load balancing or redundancy and failover.
`backend read error`	A timeout occurred when Fastly attempted to fetch content from your origin server over an established connection.
`backend write error`	A timeout occurred when Fastly attempted to write a request body to your origin server over an established connection (typically on `POST` requests).
`Backend unavailable, connection timeout`	Same as `connection timed out`.
`Backend unavailable, connection failed`	Same as `connection timed out`.
`Backend unavailable, socket(2) failed`	An internal error on a Fastly cache node prevented a socket being allocated for the fetch.
`Backend.max_conn reached`	Fastly attempted to make a request to a backend that has reached its maximum number of connections.
`between bytes timeout`	No data was received on an origin response for more than the allowed time out configured on the backend's `between_bytes_timeout` property.
`certificate has expired`	A certificate installed at the origin has expired. To resolve this, renew your certificate or download a new one.
`Certificate verification failed`	Rare edge case triggered by a narrow range of expired certificate or domain mismatch errors.
`client read error`	A problem occurred between the end user and Fastly. This is often because the user aborts the request by navigating away from the page.
`Connection refused`	Fastly attempted to make a connection to your origin and the server refused the connection. It typically appears when the wrong port is specified for the backend.
`connection timed out`	A timeout occurred while waiting to establish a TCP connection from Fastly to your origin or waiting for your origin to respond to the request.
`first byte timeout`	Fastly established a connection to your origin, but the origin did not start sending the response within the time configured for the backend's first byte timeout.
`Generic SSL handshake error`	Same as `SSL handshake error`
`hostname doesn't match against certificate`	The certificate hostname specified in your service's origin TLS settings does not match either the Common Name (CN) or available Subject Alternate Names (SANs). To resolve this error, enter a certificate hostname value that matches the CN or SAN entries on your origin's certificate.
`Illegal Vary header from backend`	A backend returned a malformed `Vary` header with its response. A well-formed `Vary` header specifies that the response can only be used with certain qualifying future requests. If the `Vary` header is malformed Fastly cannot use the response.
`Maximum threads for service reached`	This error occurs when Fastly detects that a service has exceeded a safety limit on the number of concurrent requests. Typically this indicates that a service is experiencing an unusually high load, that an origin is slow, or that features like request collapsing are being intentionally avoided.
`Missing ssl_hostname/ssl_cert_hostname to check against`	Caused by misconfigurations in the TLS certificate, but rarely seen.
`No healthy backends`	A director used for balancing requests among a group of backends is unable to make a backend request because there are no healthy backends available in its group.
`No healthy IP available for the backend`	Health checks or DNS resolution on the backend are failing
`No stale object available`	This error occurs when you configure Fastly to serve stale objects in the event of a backend failure but the stale object has expired and your backend is still failing for some reason (thus, no stale object is available). To resolve this error, you will need either to fix your origin or check your network.
`Quorum weight not reached`	A director used for balancing requests among a group of backends can't serve traffic based on its configuration because it does not have enough available backends in its group. To resolve any of these errors, you should either check for and resolve any issues with your origin or make sure the quorum setting is correct. Also, make sure you are setting the quorum setting correctly. If the director uses a `chash` policy, the error is suffixed with the policy name, i.e. `"All backends failed or unhealthy (chash)"`.
`Response object too large`	The object being fetched from origin exceeds the resource size limit of your Fastly service. You can use the Segmented Caching feature to eliminate these errors.
`Server did not provide certificate`	Origin TLS certificate is not configured correctly. In most cases this will trigger `SSL handshake error`, and not this one.
`SSL handshake error`	TLS negotiation between Fastly and your origin failed. To fix this error, review and correct your host's TLS configurations.
`unable to get local issuer certificate`	A certificate in the certificate chain is missing or invalid. To better determine which of these issues is the cause of the error, we suggest running an SSL test on your origin to highlight any issues with the certificate installed there.

Since VCL errors trigger vcl_error and run your VCL code, it is possible to intercept these and serve a custom error page.

Memory overflows

During VCL execution, exceeding some limits will cause the VCL program to terminate immediately, without invoking vcl_error. In these cases a 503 error is always generated and emitted to the client, and cannot be modified by your service configuration. The most common types of overflows are:

Workspace exhaustion: VCL programs allocate memory greedily and free it when the request ends. Use workspace.bytes_free to understand how much workspace is available.
Header overflows: VCL programs may only define around 90 HTTP headers in total. The exact number available to your program depends on the number of headers used by Fastly during your request.

Errors in Compute@Edge

Unhandled errors, panics or exceptions in Compute@Edge programs usually produce descriptive output on stderr, and if the program terminates before generating a response, will cause a blank HTTP 500 (internal server error) to be served to the client.

stderr output can be captured using log tailing.

The kinds of output produced when programs terminate prematurely is affected by which compiler is used to construct the Compute@Edge program (which in practical terms means which language the program is written in). For example, Rust based Compute@Edge apps generate panics when they halt abruptly, while JavaScript applications will report unhandled promises, errors, and exceptions.

Platform errors triggered by Compute@Edge

In addition to runtime panics and errors in the WebAssembly program, which report their own error states, it's possible for Compute@Edge apps to trigger platform errors by invoking Fastly's system interfaces and APIs in unexpected or illegal ways. In such cases the app will produce output on stderr such as "Fastly Error {number}".

If you encounter persistent errors of this kind, please contact Fastly support.

Troubleshooting

Many of the errors that Fastly generates are related to requests arriving at Fastly from end users ("client requests") or going from Fastly to your origin servers ("backend requests"). Here are some ideas to help you debug these operations.

Fastly-to-origin requests

Timeouts apply to multiple stages of the request when Fastly makes requests to a backend, and these can be configured as properties of the backend in the API, UI or CLI. In the case of errors like backend read error, backend write error, connection timed out, or first byte timeout, first check the performance of the origin server. Our defaults are generous, and hitting these timeouts generally indicates a very poorly performing origin server. However, if you need to, adjust the timeouts in the backend configuration, notably the first_byte_timeout property (also available in the web interface).

Enabling shielding can also help to alleviate backend connection problems by limiting connections from Fastly to your origin to just one Fastly POP, and reducing the overall number of requests.

Except for requests flagged for PASS, Fastly enforces a 60 second timeout on requests between one Fastly node and another, which cannot be changed. Therefore, if your request transits more than one Fastly server (usually as a result of shielding or clustering), the maximum connection and first byte timeouts are effectively 60 seconds on any requests that may produce a cacheable response.

If you want a timeout higher than 60s and do not need the response to be cached by Fastly, flag the request as PASS in vcl_recv:

return(pass);

If you do want to use the cache, but must have a maximum timeout higher than 60s, it's possible to disable clustering, but this will result in a significant degradation in cache performance because the response will be cached separately on each Fastly server.

By default, Fastly limits you to 200 connections per backend from a single edge node to protect origins from overload. For the majority of sites, this should be enough, but it can be increased by specifying a value for max_conn in the backend definition. However, reaching this limit can be an indication of other backend problems, and care should be taken before simply increasing the limit.

Client-to-Fastly requests

Requests from end users to Fastly rarely produce errors in your service by themselves, but when they do the cause is usually misconfigured TLS or domain settings. If you see a significant volume of these errors, contact Fastly support for help identifying the network issue.

TLS

Badly configured TLS can trigger a number of errors within Fastly. Connections between end users and Fastly are secured using certificates hosted by Fastly, and managed by a TLS configuration. See routing traffic to Fastly to learn more.

Loop detection

We automatically detect situations where your Fastly service appears to be forwarding requests endlessly to itself. When a loop is detected, Fastly blocks the request at the routing stage and generates a 503 error with a Loop detected reason text. Loops often occur when the same hostname is configured as both the domain and the backend for the same service, and the DNS for the domain resolves to Fastly.

To avoid creating a loop, ensure that traffic from Fastly to your origin server does not inadvertently go back to Fastly. Commonly this is done in one of these ways:

Use a separate hostname (e.g. origin.example.com) pointing to your origin server, and configure that as the address of the backend in your Fastly service configuration.
Use an IP address instead of a hostname for your backend's address within your Fastly service's configuration. This allows your origin server to be configured with the same domain as your Fastly service.

A request will be rejected as a loop if it exceeds one of these limits:

at most 3 prior visits to this POP on behalf of this Fastly service
at most 3 prior unique Fastly services executed by the request
at most 11 total prior hops, where a hop is defined as a handoff from one Fastly server to another

Although loop detected errors are triggered before your service code is invoked, the response will be returned to the parent instance of your service code, where it can be handled in VCL services by reading bresp.status and beresp.response in vcl_fetch or in Compute@Edge by examining the status and reason text exposed by the Response object.

On this page: