Cloudflare Crawler Hints

  • Thread starter Thread starter Sushubh
  • Start date Start date
  • Replies Replies 0
  • Views Views 1,311

Sushubh

Admin
Staff member
Messages
408,791
Location
Gurgaon
ISP
Excitel
Airtel
Today, we’re excited to announce Crawler Hints. Crawler Hints provide high quality data to search engine crawlers on when content has been changed on sites using Cloudflare, allowing them to precisely time their crawling, avoid wasteful crawls, and generally reduce resource consumption of customer origins, crawler infrastructure, and Cloudflare infrastructure in the process. The cherry on top: because search engine crawlers now receive signals on when content is fresh, the search experiences powered by these “good bots” will improve, delighting Internet users at large with more relevant and useful content. Crawler Hints is a win for the Internet and a win for the Internet’s energy footprint.
 

Back