Kyp [LDEV] Posted January 21, 2013 Report Share Posted January 21, 2013 I've made another change, let me know how it goes. Quote Link to comment Share on other sites More sharing options...
DigitalHytop Posted January 22, 2013 Author Report Share Posted January 22, 2013 I've made another change, let me know how it goes. Last night seemed somewhat better but I still had several stuck at load screens and a lot of lag. Spent a lot of time in combat and not so much roaming around, so I'm not sure if this was the reason for less issues or not. But I was noticing the lag during combat as well (while targeting next mob to kill, taking several seconds to actually engage the thrusters to autofollow. Hitting fire button and also taking several seconds to engage). Also, on a couple occasions I was dropped to desktop attempting to select jobs at the terminal in Aragoth Station, prior to grinding combat. All of this started at the usual 12pm EST and lasted on through to 1:30am. Quote Link to comment Share on other sites More sharing options...
VincentTH Posted January 22, 2013 Report Share Posted January 22, 2013 (edited) Since the fix Kyp put in, and especially Build 371, gating has improved a lot. THere are still occasional hangs in the Blue Galactic Map Screen (BGMS), but those happen rarely outside of the 9PM-10PM PST window, when the ISP does its nightly backup/maintenance (I guess). I think we could improve that with a longer timeout or more retries from the server, because the client is obviously waiting for a packet from the server which never comes. Last night, I did a single simple trade run from Pluto to Earth, and it took 4 BGMS's to reach Earth stations (2 dock./undocks and 4 gates). All happened starting at 9PM PST, and ended around 10PM PST. I was fighting in Rag, so when the blackout time was about to start (8:30-45PM PST), I thought, let's find something less dangerous to do instead of risking XP debt !!!!! :-) [EDIT] It is definitely not a DNS problem, since the Net-7 portal web pages load fine. Edited January 22, 2013 by VincentTH Quote Link to comment Share on other sites More sharing options...
Kyp [LDEV] Posted January 23, 2013 Report Share Posted January 23, 2013 I have disabled or moved everything that was running around this time of night, I'm running out of options. I'm thinking we might have to try to duplicate it on a weekend so I can trace what's going on at the game server and see if anythings getting dropped. Quote Link to comment Share on other sites More sharing options...
Spa[IS] Posted January 23, 2013 Report Share Posted January 23, 2013 I noticed around that time whilst gating to desktop and freezing on loading screens, when I was warping between gates, all of the mobs that I clicked on were at 99% and "bleeding" Just another observation Quote Link to comment Share on other sites More sharing options...
SiSL Posted January 23, 2013 Report Share Posted January 23, 2013 For me, it is happening mostly 7:00 gmt + 2, around 5:00 gmt and takes about an hour or more... Then all fine... Quote Link to comment Share on other sites More sharing options...
DigitalHytop Posted February 1, 2013 Author Report Share Posted February 1, 2013 Have to resurrect this post. This problem is still happening. Although it seemed to lighten up some, it is still going on. Tonight is worse than ever. I've already had to end task two times exiting Arogoth Station after taking job runs. Again, I'm on east coast and have a guild member on the west coast who is also experiencing serious lag while doing regular combat.Kyp, as a side note, I saw your post about disabling the ipv6. I did this and it hasn't done anything to alleviate the issue. Something is still interfering with the server between 12m and 1 to 1:30am ET. Also, lots of serious lag moments during other times for the past week. Quote Link to comment Share on other sites More sharing options...
SiSL Posted February 1, 2013 Report Share Posted February 1, 2013 In Germany time between 06:00am & 07:00am Quote Link to comment Share on other sites More sharing options...
Zackman [LDEV] Posted February 1, 2013 Report Share Posted February 1, 2013 Then it must be somewhere in the network of the hoster since i have a script running on a server here in Europe doing traces to the Net-7 servers, checking ping/response times in that time window. The logs from tonight don't show anything strange. What i can think of tho: Server is hosted in a segment at the ISP where 40-50 other machines are also located. 90% of all people/customers tend to make backups (scheduled jobs) all at midnight (funny but true - that why i have mine spooled to 4am). Anyway, if a dozen machines in that segment start to transfer gigabytes(!) of backupdata to a central backup server the hoster offers, the switches in that network segment start to get hit a lot, so everything starts to lag. This is normally no problem at all, but in a realtime game environment, a few 100ms start to become really recognizable. BUT...then i would see a ping-lag in my logs as well, which i dont - so i think of something else which seems more likely: Maybe it is not even the ISP network farm but some other hoster jamming the backbone by transfering gigabytes of data nightly. Eg. these new "Windmill power generators" have a software that is producing tons of data per hour and they get transfered nightly. At least i know this here from Europe because a friend of mine works for a hoster and told me once that needed to put a customer (windmill farm) on a 10 GIG fibre line... This would explain why MY(!) route to the server is fine while some other users in the U.S. are maybe using a route touching a backbone that might be jammed for you at midnight. Just a guess but seeing the logs indicates such scenario - esp. due to the fact knowing we are not running backups at that time. Finally....there isn't really much (nun) we can do about it because it's outside of our "world". Quote Link to comment Share on other sites More sharing options...
Spa[IS] Posted February 1, 2013 Report Share Posted February 1, 2013 A pity, the time I get the problems is gmt +9.5 at 4pm till 6pm I cannot gate or use terminals for the 2 hours or suffer the gate to desktop or terminal freeze, so I usually jump on my JE, and go mining in a sector for that time Quote Link to comment Share on other sites More sharing options...
SiSL Posted February 1, 2013 Report Share Posted February 1, 2013 Zack, I'm in Europe if you can say Europe as you can see <-- However, I feel it at same time with US people feels as well... During this hours, not only lag (it is not constant, but like peeks all sudden and gone), but I think master server handoffs greatly increase during gating... That's how I noticed the hours above... What I feel like packet losses rather than ping times... After 7:00am Germany time, it all ends and goes back to normal... But during 6am and 7am basically best th Three nights ago, during this hour, me & a few others (what I got from chat) locked on analyze, it never responded back about analyze or fail (Lost a good Enforcer's Retort) Two nights ago, my character is so lagged that I ended up "out of map" Yes, I was literally out of Glenn map, closest nav was 250k (west of Sol gate I was), I could not warp anywhere, can talk, chat even. A few JD's tried to pull me from that hole. may be they read this and confirm anomaly... Best thing during this hours, is to stay in the zone rather than moving anywhere... If you are in a single zone, lag comes and goes momenteraily, but if you travel or gate a lot master handoffs increase... You might just like to check logins/logouts during these hours. Quote Link to comment Share on other sites More sharing options...
Zackman [LDEV] Posted February 1, 2013 Report Share Posted February 1, 2013 Yep, my log is exactly at these hours. I start the log from 05am my time (which is 11pm server time) and have it running for 3 hours (8am my time, 02am server time). I was just pointing out that it is obv(!) not originating from the Net-7 servers or the server farm the machine is hosted at since the log would show lag for me as well then - so it must be related to certain lines/backbones going TO our server (which im not using on my route). Quote Link to comment Share on other sites More sharing options...
SiSL Posted February 1, 2013 Report Share Posted February 1, 2013 Yep, my log is exactly at these hours. I start the log from 05am my time (which is 11pm server time) and have it running for 3 hours (8am my time, 02am server time). I was just pointing out that it is obv(!) not originating from the Net-7 servers or the server farm the machine is hosted at since the log would show lag for me as well then - so it must be related to certain lines/backbones going TO our server (which im not using on my route). Yes, it looks like connection rather than server itself. Somewhere in between it stucks, most likely after Germany nodes but which one :) 3 8 ms 7 ms 7 ms 212.156.123.69.static.turktelekom.com.tr [212.156.123.69] 4 7 ms 7 ms 7 ms gyrttpe-2-3-gyrttpe-3-1.turktelekom.com.tr [212.156.118.30] 5 48 ms 49 ms 49 ms fr-col-3-gyrttpe-2-3.turktelekom.com.tr [212.156.101.85] 6 56 ms 55 ms 56 ms xe-8-2-2.edge5.Frankfurt1.Level3.net [212.162.4.157] 7 186 ms 186 ms 186 ms vlan60.csw1.Frankfurt1.Level3.net [4.69.154.62] 8 189 ms 188 ms 188 ms ae-61-61.ebr1.Frankfurt1.Level3.net [4.69.140.1] 9 186 ms 187 ms 187 ms ae-47-47.ebr2.Paris1.Level3.net [4.69.143.142] 10 188 ms 188 ms 188 ms ae-43-43.ebr2.Washington1.Level3.net [4.69.137.58] 11 195 ms 200 ms 198 ms ae-72-72.csw2.Washington1.Level3.net [4.69.134.150] 12 187 ms 188 ms 190 ms ae-71-71.ebr1.Washington1.Level3.net [4.69.134.133] 13 189 ms 189 ms 188 ms ae-2-2.ebr3.Atlanta2.Level3.net [4.69.132.85] 14 189 ms 190 ms 188 ms ae-7-7.ebr3.Dallas1.Level3.net [4.69.134.21] 15 190 ms 199 ms 200 ms ae-73-73.csw2.Dallas1.Level3.net [4.69.151.145] 16 193 ms 188 ms 188 ms ae-72-72.ebr2.Dallas1.Level3.net [4.69.151.142] 17 196 ms 189 ms 204 ms ae-5-5.car1.KansasCity1.Level3.net [4.69.135.229] 18 236 ms 237 ms 237 ms 11-INTERNET.car1.KansasCity1.Level3.net [4.53.32.10] 19 235 ms 235 ms 236 ms ae-11.bb-c.ms.mkc.us.oneandone.net [74.208.1.82] 20 250 ms 250 ms 250 ms ae-10.bb-c.slr.lxa.us.oneandone.net [74.208.6.106] 21 179 ms 179 ms 179 ms ae-10.gw-distp-a.slr.lxa.oneandone.net [74.208.1.101] 22 179 ms 179 ms 179 ms ae-1.gw-prtr-r4-1a.slr.lxa.oneandone.net [74.208.1.176] 23 247 ms 252 ms 247 ms sunrise [74.208.192.215] Quote Link to comment Share on other sites More sharing options...
Zackman [LDEV] Posted February 1, 2013 Report Share Posted February 1, 2013 I suspect the node in LA: ae-10.bb-c.slr.lxa.us.oneandone.net This one showed some pretty jumping values for me the other day when i did some tests! Quote Link to comment Share on other sites More sharing options...
Stanig[IS] Posted February 1, 2013 Report Share Posted February 1, 2013 Its Skynet. Quote Link to comment Share on other sites More sharing options...
Kyp [LDEV] Posted February 1, 2013 Report Share Posted February 1, 2013 Ill ask them about that node and see if they can check it out. Quote Link to comment Share on other sites More sharing options...
VincentTH Posted February 2, 2013 Report Share Posted February 2, 2013 (edited) One thing we can do is to increase the retry count in both the proxy on the client side and the server side. It appears now, that the server sends the master handoff only once with no retries (or too short/too few a retry). (Note that the Blue Galactic Map hang during gating also happens outside of the 9PM-10PM PST window too, but less frequently.) @Zackman: IMHO the ping test is not a good test. During the 9-10P PST window, if I gate 4 toons at the same time on 2 comps, 3 would go through, one would get the blue screen. So it is not the ping latency that causes that, more like routers fragment the data packets, and some of them got dropped. A more accurate ping test would be using the -f -l flag during that window to see how often the fragmentation is, compared to regular time outside that window). My 2 cents Edited February 2, 2013 by VincentTH Quote Link to comment Share on other sites More sharing options...
DigitalHytop Posted February 11, 2013 Author Report Share Posted February 11, 2013 Kyp, any further find for a resolve on this problem? Last night was another really, really bad night. This time around though, when I'd hang on gating, I'd force quit the game and log back in. When I came back in, I only had 1/8th of my reactor and hadn't done anything prior to hang other than hit the enter gate button. Also, prior to the midnight lagfest, while selecting 150 jobs at Prasad terminal, I was dropped completely out of game to desktop. This happened about 5 or 6 times in the course of about an hour. Odd thing is, I had an alt logged in at Jove's Fury and it showed that I immediately logged out on each occasion. Usually during a hang event, it won't show me logging out until I've actually logged back in after end tasking. I had been grinding jobs for the past week and never had this issue. I had just started doing the 150 jobs at Prasad, so I'm not sure if it has something to do with this station specifically or not, but figured I'd mention it. Quote Link to comment Share on other sites More sharing options...
Kyp [LDEV] Posted February 11, 2013 Report Share Posted February 11, 2013 Not sure, nothing shows in our logs around these times and no jobs appear to be running. Despite it being a coincidence with it happening around the time i was previously running a plesk backup im not sure what to say. Can you provide an MTR trace to us when youre experiencing the issue? Quote Link to comment Share on other sites More sharing options...
Zackman [LDEV] Posted February 11, 2013 Report Share Posted February 11, 2013 The MTR i had running the last night shows no packet drops - so the complete route including the game server was ok for the route from germany. If there is something happening, it might be some network parts / backbones within the USA affecting certain (but not all) routes to the sunrise server. Quote Link to comment Share on other sites More sharing options...
Spa[IS] Posted February 12, 2013 Report Share Posted February 12, 2013 I still get ctd at around 3:30 pm till 6pm (gmt +9.5) gating or in terminals . Very common to get terminal freeze and/or loading screen freeze. worse place for me is Earth Station, in job terminal, drops to desktop very regulary followed secondly by loading screen freeze when in a terminal,I can tell when I'm about to be booted when clicking on a explore job it freezes and takes about 1 to 2 minutes to display where the job is going, or ctd instantly 9 times out of 10 when in loading screen I know its frozen as the red circle around destination is not present, but now and then the red circle is present but I'm frozen, weird thing tho when first logging on, after selecting avatar, sometimes the loading screen has no red circle but I load into sector/station after a longer than normal delay, about 1 minute before I am in sector/docking bay Quote Link to comment Share on other sites More sharing options...
Kyp [LDEV] Posted February 13, 2013 Report Share Posted February 13, 2013 I pulled an MTR from Melbourne, AU to sunrise via a service called 'websitepulse' and have forwarded it to the host to analyze their routes. Hopefully they will have good news for us. Quote Link to comment Share on other sites More sharing options...
Spa[IS] Posted February 13, 2013 Report Share Posted February 13, 2013 Thanks Kyp, If I see any other tell tales during the "Real Life Timer" I'll report them here Quote Link to comment Share on other sites More sharing options...
Kyp [LDEV] Posted February 13, 2013 Report Share Posted February 13, 2013 They're asking for someone from the location experiencing trouble to use this tool: http://winmtr.net/ and record: "Our Network team requires a minimum of 1000 packets sent (which should only take a few minutes with WinMTR), as this gives us a good snapshot of network behavior." Preferably when you are noticing the issues and also when you are not. Quote Link to comment Share on other sites More sharing options...
DigitalHytop Posted February 13, 2013 Author Report Share Posted February 13, 2013 (edited) They're asking for someone from the location experiencing trouble to use this tool: http://winmtr.net/ and record: "Our Network team requires a minimum of 1000packets sent (which should only take a few minutes with WinMTR), as this givesus a good snapshot of network behavior." Preferably when you are noticing the issues and also when you are not.Kyp, I couldn't figure out how to attach the file so I'm just pasting it below. I ran this during normal game time between 9 and 9:30pm EST. I'll try to do another run around midnight tonight. Not sure what the onlinehome-server.com is, I configured the test to run on sunrise.net-7.org. | WinMTR statistics | | Host - % | Sent | Recv | Best | Avrg | Wrst | Last | | u15438806.onlinehome-server.com - 1 | 1382 | 1377 | 62 | 133 | 1157 | 926 | |___________________________________________|______|______|______|______|______|______| WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider Edited February 13, 2013 by DigitalHytop Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.