Server freezing

Discussion in 'Rust Discussion' started by Togoshige, May 20, 2017.

  1. Since Thursday/Friday May 11-12th, my server has randomly frozen 7 times.
    RustDedicated.exe console window still shows, but is frozen/locked up,
    I have to manually close and restart it.

    I do use 90+ oxide plugins, LOL,
    I have since shortened the list down to 60+ plugins.

    Is anyone else experiencing freezing problems?
    I cannot pinpoint the issue. I cant find anything speaking out to me in the logs.

    Ive had 99% uptime for 3 months,
    I use NFO Servers virtual server and start server with a batch script with an infinite loop, so if theres a crash, server will auto restart,
    I also use Smooth Restarter to restart exe once a day.
    When I update the server, I also do the update again to do validation.

    Oxide 2.0.3281 for 593 (1987.144.1)

    I checked Bug Reporting thread on Reddit and couldnt find anyone with same issue,
    There are some oxide threads of similar issue in the past for others.

    Ideas:
    - fresh download/install/re-setup of all server files
    - remove plugin by plugin
    - start with core small amount of plugins and slowly add more
    - write script to check if server frozen and kill it and start it back up
     
  2. No crashes? What are the last few lines or last 10 lines in the log?
     
  3. Yeah no crashes, just stall/freeze, its frozen 2 more times since I posted this

    Ive noticed a trend in the log of "Calling 'X' on 'Y took Zms" messages appearing and appearing more and taking longer and longer

    The last thing in the logs has been random, someone dying, or server saving, or RCON asking for status, etc

    Ive cut down to 49 plugins now. Also re-validated steam files again and installed latest oxide version.
    [DOUBLEPOST=1495308026][/DOUBLEPOST]I see the Oxide logs, where are the Rust logs stored?
     
  4. Wulf

    Wulf Community Admin

    When you see the hook call warnings, that could mean that the server performance in general is poor, which in turn is causing plugin hooks to not be as responsive. It's not a general indicator that it's the plugin itself causing it.
     
  5. Good point Wulf, Ive seen the hook warnings in the past when one plugin was clogging up/ slowing everything else up, it was making and waiting on a bunch of database calls

    Hmmm, server is usually around 30-40% CPU and 50-60% RAM

    I did just find "output_log.txt" inside RustDedicated_Data, but I havent checked on it after a freeze,
    Are there any other logs I should be looking at?
     
  6. Wulf

    Wulf Community Admin

    Yeah, one plugin could be performing some resource intensive tasks that are causing the server to be sluggish.
     
  7. Reporting back, Ive gone from 90 plugins to 45 plugins, Im still getting freezes,
    its very random, sometimes it takes 2-3 days to happen, sometimes just 2 hours

    I dont see any more lag in the oxide logs, all those plugins showing symptoms of lag were removed.
    The logs in general dont tell me anything now :(

    Options:
    - Keep cutting plugins
    - Start with 0 plugins and work up
    - fresh/clean install of Rust files

    Has anyone else experienced freezing?
    I will gladly post the list of plugins I use.
     
  8. Wulf

    Wulf Community Admin

    Every thread that mentions freezing is not related to your thread. ;)

    Notice some of those threads are vanilla with no plugins, and many if not most are because of selecting text on their console window, not plugins.
     
  9. Agreed, I read through all the posts for all of those threads, maybe ~2 are only semi-related, the others are definitely related :) Just trying to do my due diligence, I dont know for sure if its a plugin causing the problem :(
     
  10. Put +nav_disabled true in your startup and give that a trial.
     
  11. Hey Slut, I setup server to use nav_disabled, no luck, server froze again, damn, ty though! was worth a shot.

    Im now down to 38 plugins.

    Question, if a plugin is in an infinite loop or something when server freezes,
    Would I be able to unload the plugins one by one and see if server unfreezes?
    Ill try next time it freezes and report back
     
  12. DM me the names of your plugin and I'll help you drop that list down to plugins I know don't break.
     
  13. Currently just removed to test if freeze will appear again:

    AutoDoors.cs
    CCUpdateCheck.cs (Chaos Code)
    ChatHead.cs
    Clans.cs (Rust IO)
    InfoPanel.cs
    NoDecay.cs
    NoEscape.cs
    RulesGUI.cs
    ServerInfo.cs
    TreasureBox.cs (Chaos Code)

    =====

    Still installed on server:

    AdvertMessages.cs
    AliasSystem.cs
    AutoPurge.cs
    Clans.cs
    ComponentBlocker.cs
    CorpseDuration.cs
    CraftingController.cs
    EasyChatCommands.cs
    EasyVote.cs
    FancyDrop.cs
    FireArrows.cs
    GatherManager.cs
    HitIcon.cs
    Ignore.cs
    Kits.cs
    NightLantern.cs
    NoGiveNotices.cs
    NoHeli.cs
    PrivateMessage.cs
    ProtocolKickInfo.cs
    QuickSmelt.cs
    RandomSpawns.cs (Chaos Code)
    RemoveAAA.cs
    RemoverTool.cs
    RespawnMessages.cs
    SignArtist.cs
    SkinBox.cs (Chaos Code)
    SleeperAnimalProtection.cs
    SmoothRestart.cs
    Spawns.cs
    StackSizeController.cs
    StorageCleaner.cs
    StructureGrades.cs
    Updater.cs
    WarpSystem.cs
    ZLevelsRemastered.cs
    ZoneManager.cs
     
  14. The list looks fine, what specs is it running under?
     
  15. It reminds me of a problem I had with gameservers. I found that the problem was that it lacked RAM and crashed the server ... Look at the logs if you see the ram limit warning


    Edit :


    If it is not for RAM problem check deleting the smooth restarter. By the way, how many entities do you have?
     
  16. Hmmmm, one of my players mentioned RAM too, in case of a spike, Ill definitley look into that

    Ill also test removing Smooth Restarter as next test

    Right now there are 72,000 entities, its been pretty low for all the previous freezes
     
  17. You can put a url of your host? I would bet money that you have limited RAM insurance ...
     
  18. Wulf

    Wulf Community Admin

    Many hosts force servers to restart if they hit a memory limit, it's not actually crashing.