Movement stutters/jitters when a map runs for days

Obsidian General.
User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Movement stutters/jitters when a map runs for days

Post by Adambean » Thu Mar 13, 2014 4:54 pm

Hi there,

I'm wondering if it's a known issue where player movement appears very stuttery/jittery after a map has been running for days on end. Couldn't see it posted already so I'll ask. :)

What has happened is that I have a private server at home for a couple of friends. We were playing harvest (obviously) and as my home server is a 24/7 computer I just left SRCDS running as a service. I've also got the time limit on infinite so we could resume harvesting later and not lose equipment.

When the map started a few days ago there were no lag issues at all. I connect to my home server over Gigabit ethernet (with 9k jumbo frames) so I get no latency, and my friends had between 20-40 ms latency most of the time. They also said there was no noticeable lag other than a tiny bit with the gravity gun as expected. However after 4 days I joined the server and noticed that moving around feels very stuttery, clearly noticeable visually.

I've uploaded a video of it happening. It doesn't happen when looking around, just when moving. It's somewhat harder to see in the video than when playing, possibly as the video recording is only taking 30 frames/second instead of the 120 produced.
[BBvideo 640,480]http://www.youtube.com/watch?v=3VTWP-hKfy0[/BBvideo]

The server frame rate and CPU usage appears to be fine according to the SRCDS stats (took 10 samples over 30 seconds and they are all roughly the same).

Code: Select all

CPU   In      Out        Uptime   Users   FPS    Players
11.02 5011.89 7495.35    5798     3       99.88  1
(uptime is 4 days 38 minutes)

It's configured to run at 100 FPS, with an unlimited maximum network rate (sv_maxrate 0) and maximum update rate of 50. My client runs at 120 FPS (double-VSync'd) too. I don't really have an expert knowledge of the Source engine under the hood as I'm more of a GoldSrc engine user. Please do let me know if you would like any further information about the game server or its' underlying system/hardware.

As for not losing equipment: Yep I could easily just put our money in the ATM and restart the map, or even just use the "givepoints" cheat so we could buy exactly the same equipment we had again. We do have quite a lot of produce backstock though, so ideally we'd like to be able to have the map continuously running as there isn't an entity state save yet. Alternatively would you be aware of a plug-in (for SourceMod) that can manipulate entities and/or provide a command to directly trigger entities by their target name? I could easily restore our farm to this state with something like that.

Finally thanks for giving us Obisidan Conflict for free. We really enjoy the Harvest map especially, and have good times on Paysan too. I would put money on this being an issue with the underlying Source engine anyway, and not Obsidian Conflict. No blame/complaint is coming your way from me at all. (as one of the leading developers for Sven Co-op I fully appreciate the amount of work that goes into games like these :))
Thanks,

Adam Reece.

User avatar
TESLA-X4
Level 10 Bug Hunter
Level 10 Bug Hunter
Posts: 572
Joined: Wed Jul 01, 2009 3:26 pm
Location: $Recycle.Bin

Re: Movement stutters/jitters when a map runs for days

Post by TESLA-X4 » Thu Mar 13, 2014 9:01 pm

Hi Adam, thanks for dropping by.

Interesting issue, I imagine there's something going on with the simulation that eventually gets bogged down over time (seeing as the significant majority of the maps out there are designed to be completed and trigger a map change as opposed to running indefinitely). I wonder if fast-forwarding the game time through manipulating host_timescale will allow recreating the issue, and whether it's an engine issue or something we broke; I'll need to try that to find out.

Sadly, I'm not aware of any publicly released plugin that can save the map's state, nor of one that allows targeting entities by name (though it's been a while since I last checked the SM plugins list on their site, so things may have changed since then in the latter case).

Likewise, thanks for making Sven Co-op awesome. ;)
Apparently, Valve can't make games beyond the number 2.
I think the only time we'll get a Source SDK code update is when it starts having purchasable hats integrated, i.e. hatconomy.

Please do not make assumptions you are not knowledgeable enough to make - that just backfires on yourself.
Tick me off, and I'll be sure to give you the golden treatment. Haven't you heard? Silence is golden.

User avatar
Neico
Lead Coder
Lead Coder
Posts: 1811
Joined: Tue Aug 15, 2006 3:39 pm
Location: Germany
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Neico » Fri Mar 14, 2014 4:57 am

This looks like a lag compensation / prediction issue,
which may be caused be the amount of objects colliding.
(which can take up quite a lot of calculation time per tick and might result in the server not having enough time for player calculations)
oc_harvest always had a thing for too many things inside the storage spaces in the pantry.


you might want to try out if it's less stuttering if you put all those objects on the floor, so that almost nothing collides (the day/night cycle save trigger contains the entire pantry, not just the boxes)
Image

User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Adambean » Fri Mar 14, 2014 6:23 am

One of the big baskets in the pantry does have enough watermelons inside so they stack above the rim of the basket, so that could be it. Going to work shortly but later today I'll move half of them to the other basket to see if that helps.
Thanks,

Adam Reece.

User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Adambean » Sun Mar 16, 2014 4:05 pm

My home server had to reboot before I got a chance to try anything. Next time I play (probably soon this week) I'll fill up a basket of watermelons again. I also thought about possibly lowering the server tickrate from 100 to 33 so it wouldn't have to process the physics options so often. Not sure if that's relevant, but worth finding out, and if it does improve conditions I can just add a tickrate line to the map's configuration file.
TESLA-X4 wrote:Sadly, I'm not aware of any publicly released plugin that can save the map's state, nor of one that allows targeting entities by name (though it's been a while since I last checked the SM plugins list on their site, so things may have changed since then in the latter case).
Would there happen to be a cheat command built into the OC mod that can fire a target name? Our mappers have found it useful to be able to use command "trigger entityname" to fire all entities with that name using the USE_TOGGLE fire type. Might be something OC mappers would like too. (requires sv_cheats enabled)
TESLA-X4 wrote:Likewise, thanks for making Sven Co-op awesome. ;)
I've even copied the menu music to my music library... Really like the rock version of Path of Borallis. Been listening to it at work today. :D Sounds sad but I delay joining a server just so I can hear more of the song in the main menu haha.
Thanks,

Adam Reece.

User avatar
TESLA-X4
Level 10 Bug Hunter
Level 10 Bug Hunter
Posts: 572
Joined: Wed Jul 01, 2009 3:26 pm
Location: $Recycle.Bin

Re: Movement stutters/jitters when a map runs for days

Post by TESLA-X4 » Sun Mar 16, 2014 5:26 pm

Adambean wrote:My home server had to reboot before I got a chance to try anything. Next time I play (probably soon this week) I'll fill up a basket of watermelons again. I also thought about possibly lowering the server tickrate from 100 to 33 so it wouldn't have to process the physics options so often. Not sure if that's relevant, but worth finding out, and if it does improve conditions I can just add a tickrate line to the map's configuration file.
Possibly, Source servers are usually run at 66 tick. I remember reading somewhere that 100 tick puts extra load on the server and can cause odd physics behavior.
Adambean wrote:Would there happen to be a cheat command built into the OC mod that can fire a target name? Our mappers have found it useful to be able to use command "trigger entityname" to fire all entities with that name using the USE_TOGGLE fire type. Might be something OC mappers would like too. (requires sv_cheats enabled)
There certainly is, and it's also available in most Source-based mods/games by default: ent_fire. Naturally, sv_cheats should be set to 1.

Code: Select all

Basic usage:

ent_fire !picker <input>
 ^ Fires the specified input on the entity under your crosshair.

ent_fire <target_name> <input>
 ^ Fires the specified input on the entity by target name (or classname to affect every instance of the entity's class).

Advanced usage:

ent_fire <!picker/target_name> AddOutput "<key> <value>"
 ^ Allows changing an entity's keyvalue pair on the fly. However, not all changes will be visible. Quotes are mandatory.

ent_fire <!picker/target_name> AddOutput "<output> <output_target>:<target_input>:<input_argument>:<delay>"
 ^ Adds a new output to the entity. Be cautious when using this command because outputs cannot be removed once added.
  A saddistic example (used on a prop_physics): ent_fire !picker addoutput "OnPhysGunPickup !activator:ignite::0"
Adambean wrote:I've even copied the menu music to my music library... Really like the rock version of Path of Borallis. Been listening to it at work today. :D Sounds sad but I delay joining a server just so I can hear more of the song in the main menu haha.
In that case, I think you would like this song as well - it's a fairly interesting combination. Not part of OC, but I figured you'd like it. :)
Apparently, Valve can't make games beyond the number 2.
I think the only time we'll get a Source SDK code update is when it starts having purchasable hats integrated, i.e. hatconomy.

Please do not make assumptions you are not knowledgeable enough to make - that just backfires on yourself.
Tick me off, and I'll be sure to give you the golden treatment. Haven't you heard? Silence is golden.

User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Adambean » Sun Mar 16, 2014 8:51 pm

Ah, nice command reference there. :)

I've started again and racked up a load of produce [ 1 | 2 | 3 ], but this time running at tickrate 50 instead of 100. I'll leave it running for a few days to see how it pans out on my return.

I'll check out that tune later. I also stole Classical Gas used on Harvest for my music library too. :D
Thanks,

Adam Reece.

User avatar
Neico
Lead Coder
Lead Coder
Posts: 1811
Joined: Tue Aug 15, 2006 3:39 pm
Location: Germany
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Neico » Mon Mar 17, 2014 3:50 am

it's recommended to never run a oc server higher then 25 tick (didn't I put a limit in there anyway, since some server hosters refused to listen? So it might be very well useless to specify anything higher then that)

the reason behind that is that if you set it higher to like 66+ tick you'll die from using a simple elevator
(tough when I'm around this issue might happen anyway :D)
Image

User avatar
MaestraFénix
Team Member
Team Member
Posts: 2460
Joined: Mon May 10, 2010 11:50 am
Location: Spain
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by MaestraFénix » Mon Mar 17, 2014 2:42 pm

Neico wrote:it's recommended to never run a oc server higher then 25 tick (didn't I put a limit in there anyway, since some server hosters refused to listen? So it might be very well useless to specify anything higher then that)

the reason behind that is that if you set it higher to like 66+ tick you'll die from using a simple elevator
(tough when I'm around this issue might happen anyway :D)
Noted. I'll add that to the server guide when I can.

Also hello Adam :D
Click on my photo to add me into your friends in Steam
Image
Steam photos | Steam videos


Image

User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Adambean » Mon Mar 17, 2014 5:55 pm

21 hours into the map. Rejoined, no lag at all at 50 ticks/second. I should leave it longer. Plan to play with a friend on Thursday and from her house too so hopefully SRCDS doesn't crash by then.

Also hi. :)
Thanks,

Adam Reece.

User avatar
Shana
Lead Developer
Lead Developer
Posts: 3000
Joined: Tue Aug 29, 2006 8:12 pm
Location: Germany
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Shana » Tue Mar 18, 2014 6:01 am

Neico wrote:it's recommended to never run a oc server higher then 25 tick (didn't I put a limit in there anyway, since some server hosters refused to listen? So it might be very well useless to specify anything higher then that)

the reason behind that is that if you set it higher to like 66+ tick you'll die from using a simple elevator
(tough when I'm around this issue might happen anyway :D)
I think it was 66, at least the official server runs on 66.

And hi, nice to see the SC team enjoying OC. :D
Image

User avatar
TESLA-X4
Level 10 Bug Hunter
Level 10 Bug Hunter
Posts: 572
Joined: Wed Jul 01, 2009 3:26 pm
Location: $Recycle.Bin

Re: Movement stutters/jitters when a map runs for days

Post by TESLA-X4 » Tue Mar 18, 2014 7:59 am

Also, afaik servers should usually be run at more than 100 FPS - try adding +fps_max <value> to the server's command line. I use 700, but Source servers typically run at 250-500 FPS. Setting this too high can cause simulation spikes as the (server's) framerate fluctuates too greatly, so don't go crazy with that limit. :P
Apparently, Valve can't make games beyond the number 2.
I think the only time we'll get a Source SDK code update is when it starts having purchasable hats integrated, i.e. hatconomy.

Please do not make assumptions you are not knowledgeable enough to make - that just backfires on yourself.
Tick me off, and I'll be sure to give you the golden treatment. Haven't you heard? Silence is golden.

User avatar
MaestraFénix
Team Member
Team Member
Posts: 2460
Joined: Mon May 10, 2010 11:50 am
Location: Spain
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by MaestraFénix » Tue Mar 18, 2014 2:25 pm

TESLA-X4 wrote:Also, afaik servers should usually be run at more than 100 FPS - try adding +fps_max <value> to the server's command line. I use 700, but Source servers typically run at 250-500 FPS. Setting this too high can cause simulation spikes as the (server's) framerate fluctuates too greatly, so don't go crazy with that limit. :P
*Since higher FPS values means more CPU load.

Also, to complement the response of Shana, Valve usually sets at 33 the tick of their servers (or at least that was the default in CS:S until the OrangeBox Update, that made it impossible to raise the value, creating a major uproar in the community, among other changes).
Click on my photo to add me into your friends in Steam
Image
Steam photos | Steam videos


Image

User avatar
Adambean
Vertex
Vertex
Posts: 7
Joined: Thu Mar 13, 2014 4:24 pm
Location: Cardiff, South Wales [UK]
Contact:

Re: Movement stutters/jitters when a map runs for days

Post by Adambean » Tue Mar 18, 2014 4:20 pm

Isn't fps_max the variable in source that controls the tickrate in the Source engine? (replaces sys_ticrate from GoldSrc) Or have I missed something... When I say I've lowered the tickrate from 100 to 50 I did it with the fps_max CVAR.
Thanks,

Adam Reece.

User avatar
TESLA-X4
Level 10 Bug Hunter
Level 10 Bug Hunter
Posts: 572
Joined: Wed Jul 01, 2009 3:26 pm
Location: $Recycle.Bin

Re: Movement stutters/jitters when a map runs for days

Post by TESLA-X4 » Tue Mar 18, 2014 6:22 pm

Oh, nope - what you're looking for is in fact the -tickrate <value> switch.

Actually, I'll just post my entire command line here for reference:

Code: Select all

srcds.exe -console -noassert -nobreakpad -nogamestats -nohltv -preload -tickrate 66 -game obsidian -insecure +map oc_lobby +fps_max 700 +maxplayers 10
However, this was before Valve switched over to Steampipe (I haven't got an updated server due to disk space and time constraints).

In addition, you will most likely want to add -nomessagebox to prevent the notorious Engine Error message box from stalling the process and preventing an automatic restart after a crash.
Apparently, Valve can't make games beyond the number 2.
I think the only time we'll get a Source SDK code update is when it starts having purchasable hats integrated, i.e. hatconomy.

Please do not make assumptions you are not knowledgeable enough to make - that just backfires on yourself.
Tick me off, and I'll be sure to give you the golden treatment. Haven't you heard? Silence is golden.

Post Reply