Remove "robots=noindex" meta tag to allow translated pages in search results by smikitky · Pull Request #6260 · reactjs/react.dev
Currently, the translated content of each language version of the docs is set to explicitly opt-out from web crawlers until almost all of the site's translation are completed. The relevant code is as follows:
| const deployedTranslations = [ | |
| 'en', | |
| 'zh-hans', | |
| 'es', | |
| // We'll add more languages when they have enough content. | |
| // Please DO NOT edit this list without a discussion in the reactjs/react.dev repo. | |
| // It must be the same between all translations. | |
| ]; | |
| let shouldPreventIndexing = false; | |
| if ( | |
| siteConfig.languageCode !== 'en' && | |
| !deployedTranslations.includes(siteConfig.languageCode) | |
| ) { | |
| shouldPreventIndexing = true; | |
| } |
| {shouldPreventIndexing && <meta name="robots" content="noindex" />} |
Due to this, except for very few languages whose translation has been completed (currently only 2), most language versions are still not displayed in Google search results. They are still not linked from react.dev, either, so it's extremely hard to find the new translated site.
We have been translating the docs into Japanese for several months, but the results are still almost "invisible" on the web. Sadly, from what I can see on Twitter, many Japanese React users still believe that the effort to translate react.dev into Japanese hasn't even started!
Even if fewer than 10 articles have been translated, those articles are translated, and there is no need to hide them. If someone searches for "React useEffect" and there is a translated reference for it, they should be able to view it. After all, it's Google's job to evaluate the value of each page; there's no need for us to intentionally specify "robot=noindex".
What's worse, because of this "noindex", when you search for "React" in Japan (either with Google or Bing), the legacy Japanese version still happily comes at the top:
According to @gaearon's comment here, the React team would rather have the latest content rank higher than the legacy Japanese content. However, Google's algorithm doesn't think so; it believes that the legacy Japanese site is more useful for average Japanese developers (which I must agree). Removing the meta tag will help the latest content appear near the top even if it's not translated yet.
For example, if you search for "React useEffect" in Japan, the official legacy page still comes at the top, followed by many unofficial articles. The latest official reference (in English) is only ranked at the 11th position. The Japanese translated version has been published for more than 3 months, but it's nowhere to be found in the search results. Likewise, if you search for "React tutorial" in Japan, the class-based legacy official tutorial is still at the top, and the latest tutorial is at the 37th position because it's only available in English.