Jump to content
Arthurdent

Recent Outages Of Net-7 And Teamspeak

Recommended Posts

Hi everyone.

 

The last week or so, we have had some server issues which ultimately caused a couple of long outages for Net-7.

(yes, that's why the websites were down yesterday, as well as TS)

 

The issue was traced to a faulty NIC and has hopefully been resolved. Yay !!!

Share this post


Link to post
Share on other sites

Yay! I was beginning to worry that I broke it after my extended hiatus from here. It figures it goes down just as I have time to play again. :)

Share this post


Link to post
Share on other sites

Hi everyone.

 

The last week or so, we have had some server issues which ultimately caused a couple of long outages for Net-7.

(yes, that's why the websites were down yesterday, as well as TS)

 

The issue was traced to a faulty NIC and has hopefully been resolved. Yay !!!

 

Umm... Might have created some unintended knock-on effects...

 

Since precisely the point at which you resolved the faulty NIC issue, my docking and gating hangs have gone from 1 out of 100 docks/gates, to around 5 out of every 6.

 

Everything else is peachy... Can farm all day long in a sector, or build Agrippa items until I run out of Duplium Ore... But as soon as I start gating or docking... Things get very frustrating.

 

May not be the only person experiencing this:

 

https://forum.enb-emulator.com/index.php?/topic/11354-game-freezing-at-gates/

Share this post


Link to post
Share on other sites

Umm... Might have created some unintended knock-on effects...

 

 web and game servers are two different physical server

Share this post


Link to post
Share on other sites

 web and game servers are two different physical server

That's what I thought actually...

 

I've been running some Wireshark traces to try and track down the problem, and there is some http traffic that seems - I think - to be looking after logon authentication... Or something..? Usually pops up after a (successful) gate. I guess there must be some link between the Web and Game servers.

 

Reads like Game traffic runs to 74.208.192.215... And Web traffic to 74.208.132.71.

 

Client certainly seems to be checking in with the web server during normal Gameplay though.

 

Of course, if the web server is plugged into the same switch as the game server, could also be some knock-on effect there with the new NIC.

 

Still haven't given up on it being something at my end; but the issues I've got that are making the Game near-unplayable did kick in on exactly the same day as the new NIC in the Web Server was announced.

Share this post


Link to post
Share on other sites

I've been running some Wireshark traces to try and track down the problem, and there is some http traffic that seems - I think - to be looking after logon authentication... Or something..? Usually pops up after a (successful) gate. I guess there must be some link between the Web and Game servers.


if you have portal open, then it does periodic pings.
 

Client certainly seems to be checking in with the web server during normal Gameplay though.


Only time game uses https connection, is during login process and even then its against game server ip.

Share this post


Link to post
Share on other sites

Thanks for the info Karu.

 

Should make the Wireshark logs a bit easier to read, now that I can ignore that.

 

Have to put it down to an unfortunate coincidence that my gate/dock hangs began around the time the NIC was changed in the Web server...

 

Back to the investigations!

Share this post


Link to post
Share on other sites

Hi Holyman,

 

Try using the option on our launcher to delete all the .th6 files if you haven't. Do some more testing and see if you're still dropping. If this is happening anywhere other than Zweihander space (which uses a port that is known for certain types of attacks and blocked by some ISPs by default), it's usually the fault of these buggers. They're kind of like dictionaries or tables of contents for the sound and graphics asset files and are generated when the game is launched. They may have been corrupted somehow.

 

Outside of that, wireshark would be some good to have. If you can't isolate it but can send me some PCAPs we can look them over and see if anything jumps out. Karu is correct though there are only a select few instances where the web server is contacted by the game. Outside of what he mentioned, which if I'm not mistaken is the patching during logon, the intergalactic news net terminal in some stations will reach out to net-7.org and open it in the terminal's hyper-basic browser component.

 

I don't know of any other reasons offhand that it would talk with the web server for anything for a game session.

Share this post


Link to post
Share on other sites

Thanks Kyp.

 

Pretty sure I have the "Delete .th6 files" checked constantly... But will double check.

 

Have been meeting the insistent demands of Mrs. Holyman and the Holykids the last couple of evenings (though "Ant-Man" was not entirely unenjoyable...); but anticipate having some time and space for a good diagnostics session this evening.

 

Have picked up some chatter on the ISP's forum that a recent firmware upgrade on certain models of cable-hubs has had an impact on multi-casting... Which may or may not directly impact EnB, but could be having knock-on effects elsewhere on my LAN.

 

Have been diarising my diagnostic attempts so far in the other thread in "Tech. Support", so won't repeat them here..! Suffice to say that my packet drops/resets are consistent and replicable, which makes life easier for me diagnostically, than ghastly "intermittent" ones..!

 

I'll PM you a few rounds of PCAPs over the weekend... You may be able to spot something my tired and distracted eyes cannot..!

 

Cheers,

 

H.

Share this post


Link to post
Share on other sites

Have just come back from being away from the game for a couple of month and everytime i gate, dock or wh the client wont load the next screen. The client icon in the task bar when you hover the mouse over it just says "stage 1 completed" and goes no further. I have to terminate the client and re launch to get back into the game. All 4 boxes are ticked, have also tried updating drivers ect.

 

Any ideas?

 

Tolk

Share this post


Link to post
Share on other sites

Sounds like maybe some kind of protective software is preventing communication back to your system. It's waiting for a handoff from our server.

Share this post


Link to post
Share on other sites

Have just come back from being away from the game for a couple of month and everytime i gate, dock or wh the client wont load the next screen. The client icon in the task bar when you hover the mouse over it just says "stage 1 completed" and goes no further. I have to terminate the client and re launch to get back into the game. All 4 boxes are ticked, have also tried updating drivers ect.

 

Any ideas?

 

Tolk

 

^^^This is *EXACTLY* what I have been getting since 14th July.

 

On all 3 computers that I play/have played the Game on. Have been playing without any problems at all for a solid six months. Since 14th July: exactly as described above.

 

I have put my ISP through their paces, and they can't find anything wrong. Have stripped off all AV, Anti-Malware and Firewall protection from my machines: still exactly the same result.

 

Only inconsistencies I have had to the above is when I try and gate certain characters into and out of Primus from Endriago. Then, it lets me gate as often as I like. But as soon as I try to gate into Lagarto, Tarsis or Altair III - back to being stuck at "Stage 1 Completed".

 

Really have put everything I've got at my end through the ringer. The gate hangs are 100% repeatable, on all 3 machines I can use.

 

Conversely: sticking in one sector and farming (once I can get into the sector), is totally free of any problems. No lag, no jitter, nothing weird, everything as it should be.

 

Since this obviously isn't affecting a large number of players, but is affecting a few, I'm tending to think of a Windows Update patch... But I've rolled them all back on one machine, and no difference.

 

Soooo....Frustrating! Haven't been able to play properly for a couple of weeks, and I'm nearly through the Cold Turkey shakes... Even fired up Assassin's Creed: Unity today...

 

Do *NOT* want to stop playing EnB!!

Share this post


Link to post
Share on other sites

Nothing has changed on my pc, other than windows updates, i have not installed or uninstalled any other software and my AV is the same as i have always used. Same as Holyman when in a system everything is fine no lag or blips ect. Its very strange, i had been playing the game for around 18 months or so with no issues befor i took my little break from the game. Will try roll back ect to see if it changes anything.

Share this post


Link to post
Share on other sites

Holyman,

 

Nothing in those PCAPs of note then? We'll have to figure out how to replicate the situation to understand what may be going on. We may have to work with you. Please send myself, Zackman, and Redoc a PM regarding this and give me the results/list of testing efforts. We'll do our best to find out what's going on there.

 

Tolkenmoon,

 

Will get back with everyone on this in a post once we do some testing.

Share this post


Link to post
Share on other sites

Holyman,

 

Nothing in those PCAPs of note then? We'll have to figure out how to replicate the situation to understand what may be going on. We may have to work with you. Please send myself, Zackman, and Redoc a PM regarding this and give me the results/list of testing efforts. We'll do our best to find out what's going on there.

 

 

Nothing screamingly obvious.

 

Next bout of testing I'm going to do will be focusing on intermediate certificate revocation lists...

 

Going to have to broaden the IP address filter on the P-Cap, and see who else my PCs are trying to check in with during authentication.

 

I sent you a P-Cap yesterday which is filtered to just traffic between my PC and the two IP addresses I have on your side. Highlights of that capture set are the four RSTs I get just after the encryption is set-up, and just before the "Rules of Service" dialogue box appears.

 

The two curve-balls I get that may or may not be useful threads to follow, are:

 

I have one character (out of lots..!) that isn't always guaranteed to get stuck at "Stage 1 Completed": that's my PP, "Holybusiness".

 

And the other is... Primus Sector. Which once I get into (with any character), I seem to be able to keep gating in and out of without any problems.

 

Maybe irrelevant, but best to highlight.

 

Will PM you, Zackman and Redoc when I've got some more P-Caps relating to Intermediate Certificates.

 

Cheers,

 

H.

Share this post


Link to post
Share on other sites

dont forget to capture localhost interface for proxy-client comms

Share this post


Link to post
Share on other sites

Duly noted Karu!

Share this post


Link to post
Share on other sites

Also, just so we are aware, do you multi-box when you are experiencing issues?

Share this post


Link to post
Share on other sites

Also, just so we are aware, do you multi-box when you are experiencing issues?

I have multi-boxed (a lot!) in the past... Nearly always, in fact.

 

But since this issue cropped up, all attempts and all diagnostic sessions have been carried out with just a single client running on a single machine on my network.

 

Was wading through some less-filtered Wireshark Performance Captures last night... And the one thing that really stood out... Is just how interested Google is in my interactions with the EnB back-end!

 

Had previously filtered down to just interactions between my client machine  and the 2 EnB IP addresses. But when I lifted that filter last night, I saw a ton of activity between my machine and around half-a-dozen Google servers.

 

I didn't have any browsers open. Only other application running that could plausibly be interacting with the Internet was my AV software (McAfee)... But it wasn't.

 

But soon as I launched the Net-7 Launcher, and then when I hit the button to launch the game itself, there was heavy traffic between my machine and those Google servers.

 

Would like to ask Tolkenmoon if Chrome is his primary browser actually... It is my primary browser on all three of the machines I use for EnB.

 

I know that Google recently killed off SSL 3.0 support in Chrome... But I thought that was earlier in the year (Chrome 40 was released at the end of January, I think...).

 

There were a couple of Chrome (Stable-Channel) updates on or about the 14th July:

 

https://chromium.googlesource.com/chromium/src/+log/43.0.2357.132..43.0.2357.134?pretty=fuller&n=10000

 

https://chromium.googlesource.com/chromium/src/+log/43.0.2357.130..43.0.2357.132?pretty=fuller&n=10000

 

...They would coincide with when my problems started. And if it is related to Chrome being the default browser on all my machines, that might explain why the problem isn't much more wide-spread among the EnB User-Base.

 

When I next get the chance, I'm going to first try changing my default browser to IE (11); and if I don't get anywhere with that, I'm going to rebuild one of my Windows machines from scratch, and not let anything Google anywhere near it: see if I still get the Google server traffic when I launch the Net-7 Launcher and Game.

 

Will keep you posted!

Share this post


Link to post
Share on other sites

Hi.

 

So last night i decided to check my broadband router, the firewall was set to medium ( i am with virgin media) i disabled the firewall and was able to connect to the game log in and gate, wh and dock withought any issues, i re enabled the router fire wall and set it to low and can now still gate ect without issue. Have just been in game again and everything seems ok. Not sure what the low setting allowes and the medium blocks. |Will see if i can delve into its settings more and let you know if i find anything.

 

Tolk

Share this post


Link to post
Share on other sites

Ah... Virgin Media...

 

Same as me..!

 

And there *WAS* a firmware upgrade pushed out early-to-mid July... Which is apparently proving problematic.

 

Right! Going to follow your steps then... Let's see how I get on!

Share this post


Link to post
Share on other sites

Goddsdammitt..!

 

I *SWEAR* one of the first things I did was to disable the firewall on the Virgin hub... I swear I did..!

 

Wonder what else I did or didn't do at the same time..?

 

Anyhoo... Ummm... Well... That's sorted it! Set the hub firewall to "Low" and am now docking and gating like a good 'un..! Even multi-boxing and multi-clienting again!!

 

I'd be feeling a lot more ashamed and embarrassed about all the fuss, packet-captures, and over-technical suppositions I produced around this issue... But I'm just so happy to be back in the Game... I'm not!

 

Thanks very much to everyone who helped out and tolerated me on this: and especially thanks to you TolkenMoon - you are officially now My Hero!

 

:blush:

Share this post


Link to post
Share on other sites

My pleasure, glad you are back in game.

 

I didn't know they had pushed out a new firmware update for the routers, i had one of those what if i turn it off moments and see what happens, glad i did now :-)

 

Fly safe

 

Tolk

Share this post


Link to post
Share on other sites

Awesome. Good to hear.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...