Robots.txt
The robots.txt file tells search engine crawlers which parts of a site they can or can't crawl.
Placed at the domain root, this simple text file gives instructions per crawler and per folder. It doesn't guarantee a page stays out of search results, it only prevents it from being crawled.
An error in this file can block access to an entire site without any visible warning. Checking it after every deployment avoids this rare but costly risk.