Fixed Timers breaking randomly

Not sure, I know a couple other people complained about this on the Airdrop Settings support page, and I also run Legacy servers on the same machine, and their timers perform perfectly fine.

Timers with your test plugin just failed for me, made it to timer #5696.

But, using Windows Server 2012 R2 with everything updated, there should be no problems with the servers time. Server also runs a Minecraft Server, which does run off timed events based on the Servers Time, and they perform perfectly fine. On your test machine are the timers still functioning?

Guess the next step is to set up a localhost on my own computer and see if it does fine overnight.

 

I got stopped working timer after:

First run: [3/17/2015 6:10:25 PM] [Oxide] 6:10 PM [Info] TEST TIMER - #1

Last run: [3/18/2015 4:00:23 AM] [Oxide] 4:00 AM [Info] TEST TIMER - #60

So it seems like with timers still something wrong. Also why it after 60 times call have 2 seconds shift?

 

Well, did function correctly on my computer, made it past timer 16k. So, certainly something with the Dedicated Machine that is causing them to get killed quickly, as I've never had them make it past the 8 hour mark.

 

Yeah, I can try that. Also set the OS Update Interval for time to be a bit higher so it gets a more smooth return.

Already have Affinity set to core 3 and 4 on the machine.

Will however change it to different cores. However, the test server was on Core 0.

 

As for duplicate - i'll try, launched another test server with only timer test plugin. So now we should wait.

As for AFFINITY - you probably forgot, but i'm using linux it works bit different. Anyway i have set "taskset -c 5" so it uses 6 core (core #2 of processor #2).
Normally, my servers use from 2 to 14 core numbers at same time, in total i have 16 cores (4 processors).

 

Try change the server process affinity while the server is running. Apparently bad hardware or BIOS firmware can cause different CPUs to return completely different clock values so this may help reproduce the issue faster if that is the cause of the problem.

 

Ok, tried this, set timer to repeat every 10 seconds and changed many times for different cores/processors - timer still work fine. no shift or break. So i'll continue waiting.

ps i don't think that issue with my hardware itself. Other people also reports about this problem. But this can be rust or unity bug or something else, don't know. I never had problems with my hardware, it running two years without any glitch and almost without reboots (only when linux update or electricity lost for long time).

 

Since we cannot reproduce the issue ourselves, the more information you can provide, the faster we will be able to pinpoint the cause of the issue.

The current timer library implementation is extremely basic and completely unoptimized, so there is very little that could possibly go wrong. Once you can reproduce the issue again, and test loading a new plugin, we will have a better idea of what is happening and will be able to decide how to proceed further.

I would like to get to the bottom of this so that it can be solved before I release the new timer library which has been rewritten to be far more efficient and scale up allowing a near infinite number of timers with very little overhead.

 

Alright, if you have some things in particular that you need to know, don't hesitate to ask.

On a VPS, using HyperVM. Operating System is Windows Server 2012 R2, using Tcadmin to handle the Rust server.
Changed core affinity to 7 and 8, but just did some testing and changed it between cores a lot with the server running, could see with resource monitor that it was switching, but the timers continued to function just fine. Also moved the server over to an SSD this morning, but I highly doubt that'll effect anything concerning this.

The VPS has 10 cores assigned to it, 14gb of Ram, and all cores are underloaded when the servers are running. Ram typically stays at about 3-8gb free.

Got Wulfs timer going again, so far its doing fine, its at #3,000. So, still have a ways to go before we know if changing the OStimeinterval fixed it. If and when it dies, I'll do as you say and get a duplicate of the plugin loaded onto it and give you the results.

 

The latest build snapshot which was created a few minutes ago includes the new timer library. I also included detection for reverse clock skewing on the off chance that is the issue which is occurring on your machine. If the timer clock suddenly jumps into the past, it will print out a warning and attempt to correct the clock so that timers can continue running.

Keep in mind that the new timer library is almost entirely rewritten with far more efficient timer dispatch and there are now more moving parts compared to the library you are currently using.

 

Havent gotten the chance to put the latest snapshot build on there, but the timer just died. Made a duplicate of the plugin, and put it on the server, and that oens timers are going off. The original ones timers aren't functioning though.

 

Got it running, will get back with the results in the morning .
[DOUBLEPOST=1426790680,1426751767][/DOUBLEPOST]Alright, so woke up to a log that is 410mb, and this was spammed.
4:32 AM [Warning] Time travelling detected! Timers were updated 15230.78 seconds in the future? We will attempt to recover but this should really never happen!

It started happening right after timer #3402, no timers occurred after that in the flood of time traveling.

Code:

2:43 AM [Info] 5 SECOND REPEAT TIMER - #3402
2:43 AM [Warning] Time travelling detected! Timers were updated 21783.07 seconds in the future? We will attempt to recover but this should really never happen!

See oxide updated, will try it on that.

Just saw this;
"If you are experiencing clock jumps, please try limit your servers affinity to a single core and let us know if that helps."
My test server that this occurred on was limited to a single core when this happened, however my main server, which is still not updated to the oxide with the timer changes, is on multiple cores.


Is there any other means of running a timed event that isn't through a timer? Trying to think of a way that I can reload the plugins that are majorly effected by this on my server that isn't through a timer/won't break along with the timers it needs to fix. Only thing I can think of is through an external rcon system, but Rust Admin does not seem to want to function with my experimental server (Always disconnects right away).

Here is the processor it runs on if thats of any help;
http://ark.intel.com/products/81059/Intel-Xeon-Processor-E5-2697-v3-35M-Cache-2_60-GHz

So does appear to be a problem just with people on Emulated OSes, strange though that Legacy Timers work perfectly fine, which gives me hope this can be fixed .

 

For now i didn't get any timer problems after update, but it usually happens after 8-10 hours so i'll wait and watch for logs. Because i'm using linux and wine not sure if this problem affect me. Also my cpus is 4xAMD Opteron 8389 2.90GH, so it definitely different platform (just in case this can sort of hardware problems).

So i'll report tomorrow if something happens or not.

 

So far timers seem to be more stable for me too, made it past the 4 hour mark at least. Will keep you guys updated.

Want to also thank you guys for the hard work you've put into trying to figure this out for us, it does not go unappreciated!

 

I moved the timer library to the Unity extension and changed it to use the Unity clock which is what was used for Legacy Oxide. This clock should be more reliable on a wide range of hardware.

 

So far its working charms, has made it 3x further than I've seen it get in the past!

 

Didn't get any problems yet after 12+ hours. Also never have any time traveling warnings. Thank you for fixes)