1. Probably you had installed some of the intensive work plugins.
    Also, the GC when it collects garbage, stuck your server for a second, so you can see that low fps right in that moment.
    Also, you may have batching.colliders true and if your players build a gigant house, it hurts the server a lot.
    There is a bunch of reasons, actually.
    But you provided not enough an information for a more deep diagnostic.