Serve robots.txt from the edge
Include full text of robots.txt in VCL, serve as a synthetic response to avoid robots.txt requests hitting your origin.
User contributed notes
We welcome comments that add use cases, ideas, tips, and caveats. All comments will be moderated before publication. To post support questions, visit our support center and we'll find you the help you need.