Following on from the previous post, thought I’d update interested readers on the results of my investigation into the mysterious case of the disappearing blog.
To briefly recap, for many months, when “Boomtown Rap” is typed into Google, the home page has appeared as top listing. A few days ago, it disappeared – not only from top listing, but from any listing at all! Maybe it’s down among the dead men, but I’ve checked multiple pages deep, and there is no sign of it.
So, for all practical purposes, my blog appears to have been de-indexed. External links to The Boomtown Rap are still appearing in the listings, as are seemingly random links to various earlier BR posts of mine, but the home page? NADA!
My research indicates that this case of the disappearing blog is not unique. Another Perth-based blogger, Simone, suffered the same fate with her popular blog EnjoyPerth – but in her case the Google ex-communication was total. That is, all trace of her blog simply disappeared from Google’s listings overnight – even external links!
It turned out that her blog had been infiltrated by a hacker, who had planted a ‘hidden’ SPAM harvester at the bottom of her home page. Google had apparently detected the multiple inbound SPAM links reaped by the harvester and – as is fair enough – implemented their policy of penalising sites that illegitimately optimise themselves for search engines using dummy inbound links.
Simone, however, was innocent of utlising an illegitimate SEO strategy and Google didn’t bother contacting her to explain their drastic retaliatory action until Google’s Matt Cutts was made aware of the situation through the TechCrunch site’s expose of her dilemma. See When Google Strikes: The Story Of EnjoyPerth.net
All’s well that ends well; Simone’s blog was re-instated and before long was back in the listings gathering traffic. Without the assistance of the influential TechCrunch, though, and some tech-savvy friends, EnjoyPerth might have been obliterated and many months of effort on Simone’s part sabotaged by the dirty work of a hacker. And it seems to me that Google’s customer relations could do with some refinement.
Abrupt, unexplained de-indexing is a pretty savage measure, and in the case of some sites, could potentially destroy businesses and incomes and lead to real hardship. When people are innocent of transgressing Google’s rules – as Simone was – imposing a blog death sentence without trial or even notification that a capital offence has been committed seems nothing short of fascistic.
In fact, I do not believe that Google is the bully on the search engine block. Rather, it has grown too large for its own good. I’m guessing that it lacks the resources to action sound customer relations every time a serious SEO transgression comes on to their radar screens.
In effect, however, as Simone’s case demonstrates, Google’s punitive actions can be heavy-handed, unfair and damaging, not to mention personally traumatic to the victim. A company that was once seen as a maverick – a refreshing antidote to a stuffy, inhumane corporate system – is now in danger, by virtue of its staggering growth and size alone, of falling victim to its own success and being perceived as just another monster in an Establishment full of them.
To get back to my own case, I have received some good advice from the AussieBloggers Forum and my friend Christine, of Semfire Search Engine Marketing, which I am about to implement. For the benefit of interested onlookers and maybe other bloggers who wake up one morning to find their baby gone, I will detail my remedial attempts and provide updates as they happen.
Firstly, I should communicate Christine’s view that my disappearing blog listing may not be a result of any wilful action on Google’s part. She says my recent post,Boomtown Lament, was indexed by Google (cached on 21 January) and that she suspects the current crisis is just a glitch.
She has examined the coding on my home page and can find no trace of SPAM harvesters. And as previously mentioned, there are still external links and individual post links to my blog appearing in Google’s listings. So my situation appears to be different from Simone’s.
But what to do? This is what I have been advised:
1. Register for Google’s Webmaster Tools. I have done so, “verified” my blog (this is explained by Google after your WT registration is accepted) and sent a “Reconsideration Request” to Google, explaining the current situation and pleading innocent to any flouting of their rules – at least that I am aware of. I will post the gist of any response I receive from them.
2. Upgrade to the latest version of WordPress. This is a task I have been avoiding for months. The time has come – I can put it off no longer (sucking in deep breath as I write).
3. Install the WordPress “All-in-one-SEO-pack” plugin. In advising me thus, Christine stated: I’m putting my bet on the fact that your page titles are very similar and you don’t have a description metatag for your pages so Google sees your posts as possible duplicate content.
4. I’m also wondering whether self-referentiality in some of my recent blog titles (ie: referring to The Boomtown Rap by name in the titles) may have been interpreted by Google as some form of duplication intended to boost my listing…which it certainly was not. I was already being listed at number 1 – I had no reason to resort to such tactics. Besides, I work hard on my titles and would not compromise dramatic effect for some bloody SEO consideration. My self-referentiality was appropriate, since the posts concerned were about the blog itself.
Will keep you posted…
Rolan,
I had to upgrade my Movable Type publishing system over Xams to deal with the spam. It’s beginning to become a yearly occurence.
I remember now that I had a similar experience about a year ago—the traffic died by over 50%. I had no idea why. At the time I thought that it might have had something to do with the redesign and upgrade of thought-factory-net that had just taken place.
It now looks something akin to what you are experiencing.
Thanks for your ongoing interest in this annoying chapter, Gary – appreciate your contribution.
I have finally gotten around to upgrading to the newest version of WordPress, so watching on with hope that things will soon return to normal with the Google listings.
After your sudden drop off, long did your traffic take to get back to ‘normal’?
I had a strange experience. I wrote a blog article yesterday about Kavam.net, and it was ranked on the first page. Today it’s completely unindexed: http://aldebaranwebdesign.com/blog/kavam-limelight-network-searchme-robot-website-traffic/
Any comments? I’ve never had this happen before, just that one article has been knocked out.
Hi Jill. Hmm, that’s weird. Seems like a lot of strange stuff going on lately.
This blog’s home page is back at the top of the Google listings at the moment, but for the last week it’s been playing peek-a-boo. There one day, AWOL the next.
Sorry, I have no suggestions to explain your strange experience. Guess we just have to trust that time and the Google gods will put things right again…eventually.
Rolan, well I did find your blog after searching on “my blog has disappeared from google” unfortunately (for me anyway) my wordpress blog has also gone missing from the SERPS. I’m already signed up with Google webmaster central and my blog has been verified. So WTF? A real annoying feeling, I don’t even know if I’m doing anything wrong, I checked the supplemental index and I’m not there either.
Steve.
3 weeks ago the same thing here. Google has moved me from page 1 and 2 to page 150, 350 ect…. for search terms that still come up on page 1 or 2 on Yahoo and MSN .
Some of our competitors that are using blogger and wordpress have suffered similar fates. In fact some search terms yield no blog results for pages 1,2, or 3!
I am waiting it out in case there is a issue at Google that is causing this. I have had this happen in the past, only to show back up 30 days later.
Steve, Re: “A real annoying feeling, I don’t even know if I’m doing anything wrong…” Precisely! My hunch is that you’re not and that I wasn’t. My home page is back at #1 on the Google listings. It popped back intermittently a couple of weeks ago, then disappeared again, then re-appeared etc. It has remained there the last weeks, so hopefully whatever was wrong is now resolved. I suspect the same will happen with you. Still, annoying – as you say – and disconcerting.
Hi Joe. Well, it seems that whatever the issue is is affecting people from all over. Thanks for your post. As mentioned above, my experience appears to be replicating yours, in that my home page is back again.
I now can’t imagine any of this is Google being punitive…seems they’re having some glitches. Alarming and anxiety-inducing for the victims, though…
Hi Rolan, Did you fill out the resubmission form as advised or did it just come back up without any major action on your part? one of my sites has also dropped from serps for phrases that were number 1 for a long time. It happened last night. The weird thing is that mine also is not found using the domain name without the .co.uk. This should always be number 1. Doing a site:www.domain.com command on my site shows everything normal. All links are normal and webmaster tools is not showing any error. I really hope this will rectify itself soon. Best Regards, Shai.
Hi Shai.
I did fill out the resubmission form, but have heard nothing back from Google. My home page just reappeared and seems to now be back for good. I suspect your problem will resolve itself similarly…but you do wonder what’s going on with Google. And you can’t help but feel anxious when this sort of thing happens.
Anyway, hope it all returns to normal for you soon. Pls drop back and let us know.
Very interesting! This has happened twice with my blog already the first time was not an alarming issue because my blog was 3 weeks old when it happened. It resolved itself two days later. Again I have all but vanished from rankings in google. Not really sure what happened but I will wait it out again. Only this time I get about 60 to 70 page views a day with numerous adsense clicks. Again the blog is still only 2 months old.
I had read an article in which google has two indexing spiders that go out. One is deep indexing and the other is meant to be quick index so that items can be indexed quickly. What was said is with new blogs since the deep indexing is not in place or just getting in place the spider that does quick indexing sometimes will miss the site or certain files and ends up not indexing the site, which reverts to the state it would be at the last deep indexing – hence newer bloggs vanishing. Google said they were aware of the issue and were trying to improve it. I am guessing this to be the issue with mine. We shall see.
I know see the importance also of getting my own domain and stepping out of the google blogger into something else. I am still researching what that something else will be.
Thanks for your comment, Jeremy, and for the info about Google’s two indexing spiders. Hadn’t come across that before.
Good luck with your move to your own domain and a possibly a change of blogging app.
Just happened to me. My site, which is a couple of years old, has been listed near the top of blog search for a long time, now it’s disappeared. Webmaster tools reports it all OK and was last indexed July 3.
Weird.
It’s concerning and baffling, ibofightback – but it seems that most of the time things return to ‘normal’ within a few weeks all by themselves. Frustrating waiting for it to happen, though…
I found my way to this article after my own site experienced a black-listing (or unindexing) of a recent post. It was also, unfortunately, my most popular posting before disappearing. No clue why it was completely taken out of Google’s listing.
After monitoring my page’s popularity over the last few days, I was shocked to find it gone this morning. It’s amazing, the power of a single search engine has on my site traffic. The rest of my site is (for now) intact in Google’s indexed listings.
I guess I’m just commenting to say it happened to me too, but seemingly for none of the potential reasons mentioned in this article (or any of the websites I’ve hit thus far). It’s a complete mystery to me right now. I’ve been using Google’s webmaster tools for some time now and have a valid and functional site map that Google seems to read about once a day. These webmaster tools are how I started getting better noticed/indexed on Google to begin with.
Oh well. More investigating to do. I suppose I’ll be one of the growing number of people to do a write-up about my own unindexing experience once I’ve looked in to it further. I only wonder if my Google unindexing write-up will also end up unindexed? LOL Hey – got to laugh or I’ll just continue to get angry and frustrated.
Well i got the same problem article was on top one and then disappear but then it gets back within three days to top two and reach top one on same day, then i think and realize that if Google don’t do then this to any site which are on top ONE then how user know about other Good Sites?
I am having the same problem, I had a WP blog article at number 4 first page for three days, then poof, the post is gone from everywhere, this happened to another post I wrote and then about 2 days later it returned from space to be number one on first page again, I would love to hear more about this from anyone, thanks for any thoughts on the matter.
I had a post that was 1st page #4 58 minutes after I published it using just the first two words of the title. I checked again today and the post has completely disappeared from Google. Even if I put in all the words in the title, I will get pages that listed my post but not my post.
I recounted a customer service issue I had with a local vendor. Do wordpress.com/google censor this? If so, my faith in anything on the internet has just gone way down.
TheReelTodd, Zulqarnain, Lawyer, JohnOpincar – thanks for your posts.
I sympathise! I have to say, though, I don’t think anything is being censored.
Although my blog is back to usual position in the Google listings after its VERY disconcerting disappearance for a few weeks, I am still experiencing exactly the same issues you describe: that is, particular posts just go AWOL for days or weeks at a time – then appear one day as if they’d never left!
It has to be a Google glitch. There appear to be no thematic commonalities, ruling out censorship. And in time, the problem always seems to resolve itself. Annoying, and potentially worse than annoying if you have anything significant invested in having your site/post appear unfailingly in the listings.
I contacted Google, but received no response. There doesn’t seem to be much we can do to rectify the situation.
If any of you guys hear any more that might go further to explaining the situation, please post back here. A lot of people having these sorts of problems land on this page, so it would be a help to the blogging/web community if any useful feedback could be left here.
Cheers, and hope the mysterious and mischievous Google fairy spirits your missing posts back on to the listings very soon!
Hey to you guys, and thanks for the input about this, I have read some really goog articles about ranking with duplicate content, just a theory, I read that Google takes the highest ranking duplicate content, and boots the other one to the neverland, I found that you can use the index/nofollow to disallow your hits to be passed on to your website, I use WP so this was happening a lot, when I index/nofollow it stopped blog page from losing the rank, and allowed me to stay on first page again, just a thought, good luck to you guys.
Hi every1. I have the same problem like you guys. My blog is around 2 months old, and now it’s lost in search engine. Before, I could use site:e-product.blogspot.com example, and I could see around 140 pages listed. Now nothing. In webmaster tool it’s saying all is ok. Blog is verified etc…I just hope that they did not flag me some how for what ever reason. There is nothing left but to wait, and see what happens.
Nadir, hang in there. I’m sure Google have no intention of “flagging” you, and that your blog will re-emerge on their listings(hopefully sooner rather than later).
Lawyer,
I don’t understand your ‘index/no follow’ advice (what is ‘index/no follow’?). Could you please elaborate?
I just moved my blog a week ago from lovehatemusicblog.blogspot.com to my own domain, katuwapitiya.com, I don’t know if moving a blog counts as duplicate posting but for the past few days, my site had been at the top of the results (especially for the rare search of “katuwapitiya” lol) but now has completely disappeared. I’m reading the thing about the spiders and that makes sense, but I’ve submitted a resubmission request and I hope things work out.
On the bright side, I found this site from google for a “blog disappeared from google” search.
Thank you for your suggestion, my few blog posts have been removed from google they were ranking fine previously, it may be due to my frequent title change.
Thanks again, I appreciate the effort you put in, I couldn’t have understood this problem without you. Now I will focus on writing a new post.