Google's Loss of Parts of the Search Index

The search engine Google serves over five billion search queries daily. The top positions in the SERPs are a central size for webmasters and new pages are always indexed. For minor issues with the Search Index, most site owners and users will not get much. 

But this year, Google encountered indexing issues that were more complex. In addition, Google expresses itself explicitly via blog post - and indicates what has been learned.

The problems with the search index

Google Search Outreach's Vincent Courson has written a long blog post on how indexing issues can arise and how Google plans to communicate them.

Already in April there was a problem. Google temporarily lost parts of the search index. Also in May there had been an incident after which new content could not be indexed on Google. And in the past week, similar problems had become visible again. This was then fixed very quickly.

But how come such problems? After a search query, it will be transferred to Google's nearest data center where the SERPs will be generated. Therefore changes to pages in all data centers worldwide must be mapped.

Now updates are often rolled out successively via various data centers. When changes were made to the Search Index in April, parts of the provisioning system had failed; and on a Friday. So some documents were lost in the update and Google "lost parts of the search index". As soon as there were reports in the social media, Google was already returning the search index to its previous, stable status.

Also the Search Console was affected

While there were problems with the Search Index, Search Console was also affected. Because some data comes from the search index. As soon as Google had to return to a previous version of the Search Index, it also stopped updating the Search Console data foundation. That was the reason for the plateaus in the reports of some users.

Thus, some users were initially confused; The reason was that Google had to postpone the Search Console update by a few days.

Other bugs on Google have sometimes been independent of Search Index issues. For example, problems with the indexing of new News Content. In addition, some URLs began to direct Googlebot to pages that were not directly related. But even these inconveniences could be resolved quickly.

How Google wants to help webmasters quickly in case of problems

An immediate help for the pages can not afford Google in Indexing problem and the like. Most importantly, webmasters are notified as soon as they detect inconsistencies in traffic, Search Console, and so on. To do this, Google relies mainly on communication via social media, especially Twitter. 

In addition, those affected or interested can find information in Search Console, such as the Data Anomalies support page in Search Console, which documents current issues. In addition, Google will now provide direct notifications in Search Console if data from bugs could no longer be accurate.

In addition to the direct communication with the users, Google also relies on internal optimization. In terms of communication itself, this resulted in an approach to more quickly posting information in the Search Console, directly reporting to the Search Console Anomalies page, and tweeting in high frequency. In the future, problems should be made equally transparent. This allowed the webmasters to calm down faster.

Regarding the May issue mentioned earlier, Google claims it was discovered on May 22 at 5:30 am California time and tweeted at 6:40 am.

Since there was no long-term effect, nothing was published on the anomalies page. Nevertheless, there is always confusion with such problems. So Google is working on a more comprehensive solution for notification in the Search Console. However, implementing this will take longer, Courson stresses.

Not all problems come from Google

The blog post emphasizes once again that the majority of indexing problems are caused by individual pages and errors or problems with them. However, using the Search Console and, in particular, the Search Console Help Center, webmasters can work on debugging after the errors have been detected. For error detection itself can be found various tools in the Google universe.

Anyone who still finds a problem that can not be solved or even be part of a problem with Google, can contact the search engine directly. Either through the webmaster community , through personal contact, via product feedback tools, or on social media, YouTube, or Twitter, For webmasters, that means tracking their own data as well as Google's information channels. 

Because often the problems are solved very quickly; and that is also communicated. Anyone who is experiencing disadvantages in traffic, etc. due to a problem occurring, may initially expect little restraint from the transparency, but at least knows the reason and knows that other websites are also struggling with the problem.