Search engines like Google cannot search the millions of pages on the Internet once you hit the “Search” button – it would take far too much time. Instead, web spiders (also called web crawlers) are constantly scanning web pages, following links between them, making a note of the keywords they contain, and producing a search index on Google’s own servers. When you hit “Search”, it is this index, not the web pages, which are searched. This video from Google explains the process well:
Scanning every page on the Internet takes time, so it can take several days or even weeks for a new page to be indexed. For a similar reason, you can sometimes click on a search result only to find the content is no longer there – it has been removed but the index has yet to be updated.Search results are important for many stakeholders: users expect accurate and up-to-date results; advertisers expect their ads to be shown to interested users, and businesses want a high search ranking to drive traffic to their site. Tomorrow we will look at some of the ways businesses can improve their search rank.