GoogleOther is a generic crawler used by Google for various non-search purposes, including research and development by internal teams. Here’s a detailed overview based on recent insights from Google’s Gary Illyes:
What is GoogleOther?
GoogleOther is a versatile crawler created by Google to fetch publicly accessible content from websites for purposes beyond search. It is designed to handle different types of content through its three variations:
GoogleOther: A general-purpose crawler for public URLs.
GoogleOther-Image: Specialized in crawling public image URLs.
GoogleOther-Video: Focused on crawling public video URLs.
These crawlers are primarily used for internal research and development, supporting various Google products and features.
What Non-Search Features Does GoogleOther Support?
Gary Illyes highlighted that GoogleOther does not directly support specific non-search features but is employed for a broad range of internal research and development purposes. The generic nature of GoogleOther means it could be used for testing new products or enhancing existing ones across Google’s ecosystem. However, Google does not provide detailed information on all possible applications of GoogleOther.
Pros and Cons of Blocking GoogleOther
Pros
Resource Management: Although GoogleOther is not known to have a significant impact on server resources, blocking it may help conserve resources if its activity becomes noticeable.
Control Over Research Participation: Blocking GoogleOther ensures that your site is not included in any internal testing or research that Google might conduct. This could be beneficial if you prefer not to contribute to unannounced projects or tests.
Cons
Potential Benefits Lost: Blocking GoogleOther might prevent your site from participating in valuable research that could influence future Google products or features, potentially offering benefits if those features relate to search or other Google services.
Unknown Purposes: Since GoogleOther is used for various internal purposes, blocking it means you might miss out on opportunities related to market research or other innovations. It is not associated with training AI models for public-facing applications, but it could be involved in other internal projects.
Impact on Google Services: Although Gary Illyes confirmed that blocking GoogleOther does not affect Google Search (which is handled by Googlebot), there could be other unforeseen impacts on Google’s ecosystem that are not explicitly detailed.
Conclusion
Deciding whether to block GoogleOther depends on your specific priorities and concerns. While it might seem like a straightforward decision, the potential benefits and drawbacks are not always clear-cut. If you value control over how your site is used in Google’s internal research, you might choose to block it. However, if you are open to contributing to Google’s development of new features or enhancements, allowing GoogleOther to crawl your site might be beneficial.
You Might Be Interested In
