SEQanswers

Go Back   SEQanswers > Site News > Site Feedback/Suggestions

Similar Threads
Thread Thread Starter Forum Replies Last Post
New to SEQanswers Jdbrune Introductions 0 02-28-2012 05:33 AM
Hi SEQanswers! lukas1848 Introductions 0 10-25-2011 09:29 AM
Hello SEQanswers! hmmngs Introductions 0 09-21-2011 07:26 AM
Thank you SEQanswers jiaco Introductions 0 05-17-2010 07:01 AM
Welcome to SEQanswers.com! ECO Site Announcements 0 10-28-2007 04:59 PM

Reply
 
Thread Tools
Old 04-19-2012, 03:33 AM   #1
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default SeqAnswers connectivity problems

For a few weeks now we've been experiencing consistent connectivity problems to SeqAnswers from our institute. I've been chasing this up with our networking people who say that they can't see any reason for the problem at our end so I'd like to try to track this from the SeqAnswers side as well. All page loads are currently taking around 30 seconds, but generate no errors and the site works normally (if very slowly).

As far as we can tell it's only SeqAnswers which is affected - we've not seen similar behaviour on other sites, and the problem seems to be related to our site, in that if I go to the site from my home connection then it's back to normal levels of responsiveness. Unfortunately the problem is bad enough that several people here have basically stopped using the site from work, which is a shame as they were making heavy use of it before.

I've tried to collect some useful diagnostic information to try to localise where the problem is happening, and have stuck some data up on our website.

We've tried running a traceroute to seqanswers but it looks like a server somewhere close to your site swallows the UDP packets so we can't get a full trace - but it looks quick enough up to that point.

I've also done a wireshark packet capture of a simple GET request for your index page. Looking at the results (which you'll need to view in wireshark) it looks like it takes 29 seconds to get an ACK response to our initial GET request which really does make it look like it's the seqanswers end which is taking the extra time - though I'm guessing our setup might be related.

If I were to take a wild guess I'd note that this slowdown happened at about the time we make some big DNS changes on our site. Is it possible that seqanswers is doing a reverse lookup on our IPs when we access the site, and that this is timing out each time we make a request?

I'm happy to spend some time debugging this as I'd like to get our users back on the site - any help we can get from the seqanswers end would be much appreciated.

Thanks

Simon.
simonandrews is offline   Reply With Quote
Old 04-19-2012, 05:22 AM   #2
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

Another side effect, a lot of the time now, when I write a new post I get an error after about 30 seconds saying:

This forum requires that you wait 30 seconds between posts. Please try again in 15 seconds.

So it looks like something is trying to process my request twice?
simonandrews is offline   Reply With Quote
Old 04-19-2012, 05:39 AM   #3
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 3,161
Default

Simon: I wonder if the hosting provider for SeqAnswers.com is using some list of domains that are flagged/blacklisted for spam/network attacks. Your institute's domain may have got on that list for some reason, affecting connectivity.

We have seen this locally when an email provider like Yahoo periodically blacklists the entire university domain because of a perceived spam threat. It takes some time to convince Yahoo that it is not the case. This generally affects email delivery to yahoo.com addresses but I wonder if same could apply to connectivity.
GenoMax is offline   Reply With Quote
Old 04-19-2012, 05:56 AM   #4
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

I guess we could be being greylisted (the site does work eventually so we're not being blocked) by whoever is hosting the server, but normally those sorts of blocks are temporary and expire after a while, and this has been going on for weeks. In any case we'd need the cooperation from someone on the seqanswers side to find this out for us.
simonandrews is offline   Reply With Quote
Old 04-24-2012, 08:03 AM   #5
james hadfield
Moderator
Cambridge, UK
Community Forum
 
Location: Cambridge, UK

Join Date: Feb 2008
Posts: 217
Default

I had been seeing problems where the site would take ages to connect, making it frustrating to use. I'm glad to say it seems to work far better now and I am back online with everyone at SEQanswers.
I've even posted a couple of responses today.
james hadfield is offline   Reply With Quote
Old 04-24-2012, 08:10 AM   #6
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

Hey guys, sorry about the issues. Simon/James as far as I've heard you're the only reports of slowness...but I'll look at your tracert and see if my host has anything to offer as far as a solution.
ECO is offline   Reply With Quote
Old 04-24-2012, 08:11 AM   #7
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

Can you ping ds1715.dreamservers.com and see if it's just as slow?
ECO is offline   Reply With Quote
Old 04-24-2012, 08:31 AM   #8
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

I can't ping ds1715.dreamservers.com from here, but if I try to ssh to it it takes about 30 seconds to get a login prompt, so I guess whatever issue we're seeing affects that too.
simonandrews is offline   Reply With Quote
Old 04-25-2012, 06:37 AM   #9
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

We've now managed to get onto a machine which doesn't block our ping packets, and the ping times to ds1715.dreamservers.com is about 1.5ms so the basic connectivity is fine, and there's something about the interaction with the web server itself which is slow.
simonandrews is offline   Reply With Quote
Old 04-25-2012, 07:54 AM   #10
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

Thanks Simon. Can you email or PM me the IP address you're having problems accessing the site from?
ECO is offline   Reply With Quote
Old 04-25-2012, 07:55 AM   #11
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

Oh and your packet capture doesn't appear to be a packet capture to me...looks like a text capture of some HTML of the front page.
ECO is offline   Reply With Quote
Old 04-25-2012, 08:19 AM   #12
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

The capture file is a tcpdump capture file which you'd view in wireshark. The only bits which are human readable are the text parts which are sent by your sever which is why it looks like that in a text editor. I'll see if I can export a completely text version for you if you'd prefer that.

The problem seems to be affecting the whole of our IP range which is 149.155.144-151.* If you want to investigate a single IP then you can try 149.155.147.36 which is the linux development box where I did the packet capture and on which I've just reloaded the home page.
simonandrews is offline   Reply With Quote
Old 04-25-2012, 08:23 AM   #13
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

I've put up a full text version of the capture here. You should be able to view this just in a browser.
simonandrews is offline   Reply With Quote
Old 04-25-2012, 12:15 PM   #14
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

Thanks again Simon. Support tickets have been filed, I'll let you know.
ECO is offline   Reply With Quote
Old 04-25-2012, 02:53 PM   #15
Veleno
Member
 
Location: UK

Join Date: Aug 2011
Posts: 16
Default

I'm in between James and Simon, closer to James actually but it was working fine today and for the past few weeks I don't remember any problems.

Must be Babraham rd.....
Veleno is offline   Reply With Quote
Old 04-25-2012, 04:29 PM   #16
ECO
--Site Admin--
 
Location: SF Bay Area, CA, USA

Join Date: Oct 2007
Posts: 1,297
Default

That's what my host is telling me...no blacklists or blocking are in place. I'm hoping they'll look at the tracert but so far there is no active blocking on my end.
ECO is offline   Reply With Quote
Old 04-26-2012, 12:16 AM   #17
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

It doesn't look like a block or a routing problem since ping times are OK. Even when I ssh to that server the first part of the key exchange seems OK, it's the second phase which is slow. My bet is still on this being a DNS issue since it happened about the time we had a big DNS restructuring. Our internal IPs (which is what your server will see) don't have an entry in our external DNS, but maybe something on the seqanswers server is taking a long time to figure that out.

Could you try doing a DNS lookup on the IP I listed above on the server and see if it returns in a reasonable time? If it's not that I'm stumped.

[Writing this from home - where using the site is lovely and quick!]
simonandrews is offline   Reply With Quote
Old 04-26-2012, 05:20 AM   #18
GenoMax
Senior Member
 
Location: East Coast USA

Join Date: Feb 2008
Posts: 3,161
Default

DNS lookup on 149.155.147.36 is failing from my location (no matter what name servers I use, local or even with google's public name servers).

Unless SeqAnswers hosting provider is doing reverse name lookup (which would be unusual for just http connections)...
GenoMax is offline   Reply With Quote
Old 04-26-2012, 05:37 AM   #19
simonandrews
Simon Andrews
 
Location: Babraham Inst, Cambridge, UK

Join Date: May 2009
Posts: 839
Default

The lookup should fail (our internal IPs aren't in the external DNS), but it should fail quickly. My worry is that somehow it's taking a long time for the server to determine that there's no hostname to find.

Doing a reverse lookup might happen from a webserver if you're using name based logging or filtering.
simonandrews is offline   Reply With Quote
Old 04-26-2012, 05:45 AM   #20
sklages
Senior Member
 
Location: Berlin, DE

Join Date: May 2008
Posts: 531
Default

Quote:
Originally Posted by GenoMax View Post
DNS lookup on 149.155.147.36 is failing from my location
same here, Berlin/Germany.

Last edited by sklages; 04-26-2012 at 05:48 AM. Reason: OK, needless answer, I read Simon's post too late ;-)
sklages is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off




All times are GMT -8. The time now is 03:38 AM.


Powered by vBulletin® Version 3.8.6
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.