How Google Search Works

Almost every piece of SEO advice you will read assumes you already know what Google is doing behind the scenes. It is worth ten minutes to actually learn it, because most beginner mistakes come from thinking Google works in one step when it really works in three.

Three separate steps, not one

Google describes its own process as three stages: crawling, indexing, and serving. A page can pass one and fail the next, which is why "my page isn't on Google" almost never has a single answer.

Crawling is Google downloading your page. An automated program called Googlebot visits web addresses and fetches whatever is there, the way your browser does when you open a link.

Indexing is Google reading what it downloaded and deciding what the page is about — the text, the images, the headings — then filing it away in an enormous database called the index. Google also decides at this stage which version of a page is the main one, if several addresses show similar content.

Serving is what happens when somebody searches. Google looks in its index — never at the live web — and picks results it thinks answer the question.

Picture a bakery with a website. Googlebot fetching the opening-hours page is crawling. Google working out that the page is about a bakery in a particular city and storing it is indexing. Someone typing "bakery near me" and seeing the shop is serving.

Where it actually breaks

Each stage fails in its own way, and the fix is different every time.

Google never finds the page. Googlebot discovers new addresses mostly by following links from pages it already knows. A page that nothing links to, and that appears in no sitemap, may simply never be discovered. This is the most common cause of a brand-new site being invisible.

Google is blocked from fetching it. A file called robots.txt can tell Googlebot not to visit parts of a site. It is easy to block far more than you intended, and the site will look completely normal to you while doing it.

Google fetched it but chose not to index it. This surprises people, and it is worth being blunt about: Google does not promise to index every page it crawls. Thin, duplicated, or low-value pages routinely get crawled and then left out.

It is indexed but never shown. Being in the index only makes a page eligible. Whether it appears for a given search is a separate question, and that is what the rest of SEO is about.

What to do about it

• Make sure every page you care about is reachable by clicking links from your homepage. If a human cannot get there by clicking, Googlebot probably cannot either.
• Publish an XML sitemap — a machine-readable list of your pages — so Google has a second way to discover them.
• Verify your site in Google Search Console, the free tool that reports which of your pages are indexed and why the others are not. It is the only place this information exists.
• Use Search Console's URL Inspection tool on a page that is missing. It tells you which of the three stages the page actually reached.
• Give Google time. Crawling and indexing are not instant, and a new page may take days.

Common mistakes

Assuming you have to submit pages to Google. You do not. Google finds pages by following links. Submitting a sitemap helps discovery, but there is no queue to join and no fee to pay.

Reading "crawled" as "will rank". These are three stages apart. A page can be crawled today, indexed next week, and still never appear for a competitive search.

Believing paid tools speed it up. Nothing you buy makes Google index a page faster. Services selling "instant indexing" are selling something they do not control.

Sources: In-depth guide to how Google Search works and Google Search Essentials.