SEQanswers

Go Back   SEQanswers > Site News > Site Announcements



Similar Threads
Thread Thread Starter Forum Replies Last Post
HaloPlex Updates loretta.manning Sample Prep / Library Generation 0 06-22-2012 10:28 AM
dbSNP updates and the effect on variant annotation results warrenemmett Bioinformatics 0 10-18-2011 12:31 PM
Dont get the updates from FORUM apratap General 2 07-31-2009 09:38 AM
SEQanswers Registers 1000th member! Traffic/posts continue to grow! ECO Site Announcements 4 09-07-2008 05:03 PM
deCODEme service updates, Nov. 23 ECO Personalized Genomics 0 11-23-2007 08:50 AM

Reply
 
Thread Tools
Old 01-25-2013, 11:00 AM   #1
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,358
Default Updates on traffic/errors...

So if you've been here for the last three days you've probably noticed some sporadic performance...apologies for that.

I'm continuing to monitor it (and please continue emailing me if you run into problems), but it seems like the cause was that Baidu (search engine) has been indexing the wiki and forums with a crazy amount of robotic power. Meaning 6-8 requests per second, from ~30 different IPs, of different complex wiki pages that bring the database server to it's knees.

I am now blocking their IPs, we'll see if it holds.

Grrrrrrrrrr.
ECO is offline   Reply With Quote
Old 02-04-2013, 08:13 PM   #2
seb567
Senior Member
 
Location: Québec, Canada

Join Date: Jul 2008
Posts: 260
Default

I understand that Baiduspider does not respect the robots.txt !

http://seqanswers.com/robots.txt

User-agent: *
Disallow: /w/
Disallow: /wiki/Special:


User-agent: Baiduspider
Disallow: /
seb567 is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 10:29 PM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Single Sign On provided by vBSSO