Forum not indexed on Google?
Posted: December 19th, 2014, 8:26 pm
Hi all,
Is it just me, or is this forum not indexed by Google? It was trying to find an old post of mine (couldn't remember where I had posted it) and I couldn't find it at all, but when I tried Bing out of desperation it was the first (and only) result, here on this forum.
I notice that the XML sitemaps don't include direct links to the forum, so I am wondering whether Google's crawler takes that to mean that the forum should not appear in search results?
Just wondering whether this is intentional or not? If not, perhaps it's worth investigating as it might mean people aren't finding this site when there's a lot of good content in the forum posts!
I'm not sure whether the solution is to somehow merge the forum into the sitemaps (maybe the forum software can produce sitemaps automatically to reduce load from web crawlers) or whether you can just add it as a specific inclusion in robots.txt and have it crawled normally.
Is it just me, or is this forum not indexed by Google? It was trying to find an old post of mine (couldn't remember where I had posted it) and I couldn't find it at all, but when I tried Bing out of desperation it was the first (and only) result, here on this forum.
I notice that the XML sitemaps don't include direct links to the forum, so I am wondering whether Google's crawler takes that to mean that the forum should not appear in search results?
Just wondering whether this is intentional or not? If not, perhaps it's worth investigating as it might mean people aren't finding this site when there's a lot of good content in the forum posts!
I'm not sure whether the solution is to somehow merge the forum into the sitemaps (maybe the forum software can produce sitemaps automatically to reduce load from web crawlers) or whether you can just add it as a specific inclusion in robots.txt and have it crawled normally.