Serve robots.txt from the edge

Include full text of robots.txt in VCL, serve as a synthetic response to avoid robots.txt requests hitting your origin.

VCL

Use this solution in your VCL service (click RUN below to test this solution or clone it to make changes):

User contributed notes

We welcome comments that add use cases, ideas, tips, and caveats. All comments will be moderated before publication. To post support questions, visit our support center and we'll find you the help you need.