Page 1 of 1

Forum not indexed on Google?

Posted: December 19th, 2014, 8:26 pm
by Malvineous
Hi all,

Is it just me, or is this forum not indexed by Google? It was trying to find an old post of mine (couldn't remember where I had posted it) and I couldn't find it at all, but when I tried Bing out of desperation it was the first (and only) result, here on this forum.

I notice that the XML sitemaps don't include direct links to the forum, so I am wondering whether Google's crawler takes that to mean that the forum should not appear in search results?

Just wondering whether this is intentional or not? If not, perhaps it's worth investigating as it might mean people aren't finding this site when there's a lot of good content in the forum posts!

I'm not sure whether the solution is to somehow merge the forum into the sitemaps (maybe the forum software can produce sitemaps automatically to reduce load from web crawlers) or whether you can just add it as a specific inclusion in robots.txt and have it crawled normally.

Re: Forum not indexed on Google?

Posted: December 19th, 2014, 10:36 pm
by DOSGuy
It shouldn't need to be in the sitemaps; Google should crawl the forum unless it's excluded in the robots.txt, which it isn't. It's not intentional. I'll see what I can do.

Re: Forum not indexed on Google?

Posted: December 20th, 2014, 5:07 pm
by MrFlibble
Interesting how I asked DOSGuy exactly the same question just a while ago in a PM conversation.

I've always thought that it was intentional though, to keep spammers and the like out.

Re: Forum not indexed on Google?

Posted: April 28th, 2015, 11:57 pm
by DOSGuy
This is finally fixed. Apparently I disabled about half of the bots when I first installed phpBB3... potentially 8 years ago. Oops.

My attempts to fix it earlier today failed, but upgrading to 3.1.3 has fixed all of the bot accounts, so Google has been instructed to crawl and index the forum.

Re: Forum not indexed on Google?

Posted: April 29th, 2015, 12:40 am
by developertn
Just a positive feedback on your new upgrade. The colors and the font styles look cleaner in my opinion.

Re: Forum not indexed on Google?

Posted: April 29th, 2015, 1:14 am
by Malvineous
Nice! Glad to hear the problem has been solved. Fingers crossed it won't increase the server load too much.

Re: Forum not indexed on Google?

Posted: April 29th, 2015, 7:13 am
by DOSGuy
If we can handle 5000 visitors a day, I don't think Google's spider will affect us too much.

Re: Forum not indexed on Google?

Posted: April 29th, 2015, 1:28 pm
by MrFlibble
Glad you fixed this issue!

Is there a way to set the old style forum theme as a selectable skin?

Re: Forum not indexed on Google?

Posted: April 30th, 2015, 8:35 pm
by DOSGuy
Subsilver2 was dropped from phpBB 3.1 and is no longer being maintained.

Re: Forum not indexed on Google?

Posted: May 1st, 2015, 2:39 pm
by MrFlibble
And they don't have anything in stock that at least looks like the old skin?

Don't get me wrong, this is no big deal really, but I'm a bit conservative about these things. Of course when choosing between a certain forum appearance and the forum being accessible to search engines there's no question what should have priority.