High-Quality Content Is Crawled More Often [Video]

SEO professionals have long discussed the concept of a “crawl budget,” which refers to the limited number of pages search engines can crawl daily.

The assumption is that sites must stay within this allotted budget to get pages indexed.

However, in a recent podcast, Google’s Search Relations team debunks misconceptions about crawl budgets and explains how Google prioritizes crawling.

How Googlebot Prioritizes Crawling

Dave Smart, an SEO consultant and Google Product Expert, acknowledges the confusion surrounding crawl budget:

“I think there’s a lot of myths out there about crawling, about what it is and what it isn’t. And things like crawl budgets and phrases you hear thrown around that may be quite confusing to people.”

So, how does Google decide what to crawl?

Smart explains the process:

“You need to do it by looking at what’s known, finding somewhere to start, a starting point. And from that, you get the links and stuff, and then you would try and determine what’s important to go and fetch …