How To Improve Minecraft Server Performance

Discussion in 'Bukkit Help' started by TnT, Jan 15, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    TnT Retired Staff

    See my post on how to install Craftbukkit on Ubuntu 10.10

    I started my quest to improve performance from being annoyed with the "[Warning] Can't keep up! Did the system time change, or is the server overloaded?" messages. The system time did not change, and the server is not overloaded. I run my server on an ESXi VM (the free ESXi - you can try this at home) but it is just as possible to do the same thing with a physical machine. My Specs: 4600+, 6GB RAM, 40 GB HD. Full backups daily, incremental backups every hour. Nothing fancy, but not shit either.

    My quest is not over, I have not resolved those messages by mere software changes alone, but I will, eventually. I will update these instructions as things change, or I discover more/better things to do.

    I will focus my attention on GNU/Linux (specifically Ubuntu 10.10 x64) as that is what I've found to best allow me to do everything I need. Any x64 GNU/Linux would do. As would any x64 version of BSD, Unix, Mac, or Windows. Everything I will mention is possible in all OS's, however, my instructions will be focused on a single one. Also, links I provide I found to be the most helpful for me. Your Google foo may differ.

    First, [2] install the [3] x64 Java version - you will need this to allocate enough RAM. (Enough is subjective. Change value as desired, and allowed by the limits of your hardware.) I found using the /opt directory quite handy, as its unused in Ubuntu, so I could put Minecraft, c10t, Java, etc, all as subdirs of /opt.

    The most common tip, and the easiest to implement - change the amount of RAM your java process gets allocated. How much hardware will limit how high you can go (obviously).

    These are easiest to put into a batch file or shell script (start.sh for bukkit) Ex:
    Code:
    java -Xms512M -Xmx2G -jar craftbukkit.jar
    java -Xms512M -Xmx3072M -jar craftbukkit.jar
    java -Xms512M -Xmx16G -jar craftbukkit.jar
    Next trick is to run the world inside a RAM disk. Now, because the world is held within RAM, things become more volatile than using a HD, but it becomes much faster. BEFORE you create this ramdisk, you need to move your world to a different folder. Once this ramdisk is mounted, the world folder will be empty (or so it appears - you can unmount the ramdisk to see the files if you haven't moved them out).
    Below assumes Minecraft is "installed" in the /opt/craftbukkit folder. Add this line to your /etc/fstab
    Code:
    ramdisk /opt/craftbukkit/world tmpfs mode=1777,size=512m
    On reboot, this mounts the ramdisk onto your world folder. If you do not want to wait for a reboot (who does?) type this:
    Code:
    sudo mount /opt/craftbukkit/world
    Now, copy your world back to the /opt/craftbukkit/world folder, and you should be able to start up your server. I gave the ramdisk 512 MB because, shit, why not? I only use 137 MB right now, but I like to know I have plenty of room to grow. If I get close to exceeding that, I'll increase it.
    Here are the scripts I use to start and stop my server. Remember, I to also run backups every hour off the entire /opt/craftbukkit directory, which includes this ramdisk world, so at worst you will lose an hour. This script, performs the following tasks:
    1. Set ownership of ramdisk folder
    2. Copy the world from the HD world area (named backup in my case) to ramdisk.
    3. Run the java server process.
    4. Once the server finishes (server stops, crashes, etc), remove the stale HD staging area, recreate a fresh HD staging area and moves the files from ramdisk to the HD staging area.
    I found when I remounted the ramdisk, or rebooted, it had root privs. So I need to be there to launch my server (no big deal for me) as my first script requires sudo privs. I ~think~ I can get away without it, but I didn't want to risk it. Script assume it is located in the /opt/craftbukkit directory and you run them from there as well.
    Code:
    #!/bin/sh
    sudo chown -R <user for minecraft>:<group for user> /opt/craftbukkit/world
    cp -r backup/world ./
    java -Xms512M -Xmx3072M -jar craftbukkit.jar nogui
    rm -r backup/
    mkdir backup/
    cp -r world/ backup/
     
  2. Offline

    feverdream

    Because you know, everybody here has 16 gigs of ram in their server ... Oh wait, we dont.

    Notchs used Java. That, and he used a method of I/O that is slow, and he does a lot of it as chunks are small and many of them are loaded and unloaded from disk as people move around becouse he loads stuff you cant even use or see. That is why its slow.
     
    mattataki, fredghostkyle1 and Lamp like this.
  3. Offline

    OriginalSyn

    6gb, not 16. Ram is cheap, if you can get 4 for less than $60
     
    fredghostkyle1 likes this.
  4. Offline

    feverdream

    He clearly posted:

    java -Xms12G -Xmx16G -jar craftbukkit.jar nogui
    I see a "12G" there..
    Still, my thoughts still stand. The save work was designed wrong, that is why its slow.
     
  5. Offline

    OriginalSyn

    That was 3 examples of how to execute the same command with different amounts of ram allocated.
     
  6. Offline

    feverdream

    Yes, I know.
     
  7. Offline

    Clinton

    We've gotten rid of those messages simply by better CPU and IO performance on our VPS'. Now that we use hardware RAID 10 we never see those messages pop up, not even when the server first starts up. We'll see it occasionally on start up if a user is running a lot of mods/plugins.

    You might consider trying it on some faster disks.
     
  8. Offline

    OriginalSyn

    Hmmm ok but then why did you say

    Seems like you assumed that you need 16 gb of ram to do this but he provides example of much less.

    That's certainly another solution to the problem, but one that requires significantly more money in hardware.
     
  9. Offline

    TnT Retired Staff

    So far, mine is running without those warnings as well, but I won't lie - I still see them once in a while.

    Code:
    tnt@Dozer:/opt/minecraft/craftbukkit$ ./start.sh
    [sudo] password for tnt:
    World Copied from HD into ramdisk
    --2011-01-15 20:38:00--  http://artifacts.lukegb.com/artifactory/plugins-snapshot/org/bukkit/craftbukkit/0.0.1-SNAPSHOT/craftbukkit-0.0.1-SNAPSHOT.jar
    Resolving artifacts.lukegb.com... 67.228.153.21
    Connecting to artifacts.lukegb.com|67.228.153.21|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 981896 (959K) [application/java-archive]
    Saving to: `craftbukkit-0.0.1-SNAPSHOT.jar'
    
    100%[======================================>] 981,896      694K/s   in 1.4s
    
    2011-01-15 20:38:02 (694 KB/s) - `craftbukkit-0.0.1-SNAPSHOT.jar' saved [981896/981896]
    
    Downloaded Latest Version of Craftbukkit!
    2011-01-15 20:38:02 [INFO] Starting minecraft server version Beta 1.2_01
    2011-01-15 20:38:02 [INFO] Loading properties
    2011-01-15 20:38:02 [INFO] Starting Minecraft server on *:25565
    2011-01-15 20:38:03 [INFO] Preparing level "world"
    2011-01-15 20:38:03 [INFO] Preparing start region
    Preparing spawn area: 0%
    Preparing spawn area: 0%
    Preparing spawn area: 3%
    Preparing spawn area: 8%
    Preparing spawn area: 28%
    Preparing spawn area: 61%
    Preparing spawn area: 97%
    SimpleWarp version 1.3 is enabled
    2011-01-15 20:38:11 [INFO] Minecart Mania Core version 0.54 is enabled!
    2011-01-15 20:38:11 [INFO] Done! For help, type "help" or "?"
    
    You'll have to had damn fast disks to beat a ramdisk for I/O. Sure, I'd love to get faster hardware, and you can definitely build a server powerful enough to never see those messages. For me, I went the route of optimizing what I currently had in order to run the server. Of course, if you can drop the money on better hardware, go nuts.
     
  10. Offline

    Clinton

    Well when you're hosting VPS' RAMdisks just isn't viable. You can't trump true hard drive space ever if you have it setup correctly. I'm in no way saying RAMdisks aren't fast.

    Of course our setups aren't cheap for our current nodes either, 6 x 500GB hardware RAID with about a $300 RAID controller.
     
  11. Offline

    TnT Retired Staff

    Very true, on a large map you can easily grow to multiple hundreds of megs, or even gigs in size. Plugins like borderlands could help with that if that's a concern. As I said, it's all dependent on the limits of your hardware.
     
  12. Offline

    SeeD419

    Jeez I've never even heard of a ramdisk. I just allocated 2gb to Java. What exactly is a ramdisk? Is this separate hardware? Or changes made with how the OS uses the RAM?
     
    fredghostkyle1 likes this.
  13. Offline

    mindless728

    essentially you take a portion of you ram and make it look like a hard drive for when you need extremely fast io, though i wouldn't recommend it unless you have a large amount of memory (8GB+)
     
  14. Offline

    Clinton

    I'd also like to say that it's not a hard drive because your RAM will get cleared and everything will be wiped.
     
    Plo124 and Major like this.
  15. Offline

    SeeD419

    Maybe I should just buy a SSD and call it a day.

    4gb is probably not enough to do this with? Not even allocate just 2gb?
     
  16. Offline

    TnT Retired Staff

    How big is your world? If you had a 2GB java process, you could run a 1GB ramdisk, if your world is really that big.


    That's why you backup, and have it copy to a hard disk location when you shut down the server.
     
  17. Offline

    SeeD419

    I think it's a bit over 160mb right now.
     
  18. Offline

    TnT Retired Staff

    Then you could easily get away with a 256 MB ramdisk. Just keep an eye on your world size.
     
  19. Offline

    mindless728

    actually an SSD would be a bad idea even though it would perform well, you would kill the SSD pretty quickly due to the number of small writes you would be performing
     
  20. Offline

    Clinton

    That exactly.
     
  21. Offline

    Treetop

    nothing you said in this work ( 1st post)
     
  22. Offline

    TnT Retired Staff

    Can you please be more specific. I do everything I've mentioned above on my server, and it does work.
     
  23. Offline

    Treetop

    ramdisk /opt/craftbukkit/world tmpfs mode 1777,size=512m
    i add the to the area you specified and i get errors when i try and mount.
    ( im not trying to be an asshole/troll, I'm grateful for you post.)
    spent 2 hours trying to get it to work
     
  24. Offline

    TnT Retired Staff

    Ubuntu Server 10.10? Does the directory: /opt/craftbukkit/world exist? What error do you get?
     
  25. Offline

    Treetop

    Code:
    proc            /proc           proc    nodev,noexec,nosuid 0       0
    /dev/mapper/oldubuntu-root /               ext4    errors=remount-ro 0       1
    # /boot was on /dev/sda1 during installation
    UUID=34779cc4-2ba5-4af8-9099-5b98b18bd1cf /boot           ext2    defaults        0       2
    /dev/mapper/oldubuntu-swap_1 none            swap    sw              0       0
    ramdisk /opt/world tmpfs mode 1777,size=512m
    ^ fstab

    Code:
    @oldubuntu:~# sudo mount /opt/world
    [mntent]: warning: no final newline at the end of /etc/fstab
    [mntent]: line 13 in /etc/fstab is bad
    mount: can't find /opt/world in /etc/fstab or /etc/mtab
    
    ^ try to mount

    I restarted after putting the info in the fstab and it wouldnt even boot. i was able to recover my stuff though thank god
     
  26. Offline

    TnT Retired Staff

    Interesting. Try following this post. Says to simply add a new line to the end of your fstab. Just posting mine so you can see I'm not talking out my ass.

    My fstab:

    Code:
    proc            /proc           proc    nodev,noexec,nosuid 0       0
    /dev/mapper/Dozer-root /               ext4    errors=remount-ro 0       1
    # /boot was on /dev/sda1 during installation
    UUID=0dc69f1b-7669-4137-92d4-6336ade34883 /boot           ext2    defaults        0       2
    /dev/mapper/Dozer-swap_1 none            swap    sw              0       0
    /dev/fd0        /media/floppy0  auto    rw,user,noauto,exec,utf8 0       0
    ramdisk /opt/minecraft/craftbukkit/world tmpfs mode=1777,size=512m
    //192.168.0.250/pictures /opt/pictures cifs credentials=/opt/credentials,rw,iocharset=utf8,uid=1000,gid=1000 0 0
    
     
  27. Offline

    RchGrav

    This isn't EXACTLY true... while I agree with you that a standard Consumer SSD w/ a Sandforce controller & MLC Flash would be a bad idea ... You can still consider using SSD but you need to use a drive based upon SLC NAND chips. There is a new MLC spec being peddled called eMLC which means "Enterprise MLC", but still only approaches 30% of the write cycles of a typical SLC based SSD.

    In my case I configured my SLC SSD w/ 4K inodes (a logical optimization specific to minecraft world saves) and disabled the updates of last file accessed time via the "noatime" setting in the fstab to further reduce writes.. it would also probably be a good idea to use an OS or Kernel optimized for TRIM... (Ubuntu 10.10 has the right kernel, 10.4 does not.) Also, the drive I chose is able to do writes as small as 512 bytes, so that a minimal region of flash is able to be reprogrammed for chunk updates.

    I banked upon the concept of SLC vs MLC being lost with most people, and it paid off. Here is the 64GB Samsung SSD drive I won in an auction for $72...

    http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=250733819661

    It was a pleasant surprise when it arrived and I immediately I checked the SMART data to find it only had 9 power cycles and about 8 hours of powered on time. Lucky for me the differences between MLC and SLC are not highly publicized since SLC typically carries a big price tag. (Froogle shows a street price of this SSD as $500) As of this point in time it has well over 1000 power on hours hosting my minecraft server... It still has ZERO ECC errors, correctable or uncorrectable.. every SMART stat available shows absolutely no indiciation of even the slightest hiccup during that period. The rest of the storage on the server is RAID 10 w/ a Hotspare, and the system backs up the SSD to the RAID array on a regular basis.

    http://minecraftjungle.com/slcsmartdata.png

    I guess time will tell... but so far it looks VERY promising..

    By comparison.. I had initially installed an A-DATA Sandforce 64GB MLC SSD.. and it only ran my MC server for 5 days and when I checked the SMART data it already had a few regions of the NAND disabled, and a SMART warning present. My curiosity regarding the speed of the Sandforce controller won out over my better judgement of using MLC storage... but then just as quickly my better judgement prevailed. :)

    Here is some data detailing the differences between SLC & MLC solid state drives.
    http://www.psism.com/SLC vs MLC.pdf
    http://www.storagesearch.com/ssd-slc-mlc-notes.html
     
    THG3 likes this.
  28. Offline

    SeeD419

    Holy fuck I want the guy above me to build me server.

    Is there a way to create a ram disk in Windows? Or is that just a total joke? I'm not running a very large server, so I'm just running Vista x64. My HDD is the 10k Raptor, so hopefully that helps my speed a bit. I'm only running 4gb DDR3 though. Still need to grab a new mobo down the line as I'm maxed out.
     
  29. Offline

    TnT Retired Staff

    He's definitely done his research when it comes to SSD's, so I'd listen to him. The thing with SSD's is you need an OS that supports all the features they have built in to ensure they don't die too quickly (aka, TRIM, etc), which is basically what he pointed out.

    You can run a ramdisk on windows too. A quick Google search yielded this result.

    Things to note: When notch changes the way world data is held, a lot of I/O issues should disappear. Spending money on hardware to fix these issues now may not be the way you want to go, but hey, your budget, your server, your choice. If you are looking to spend money though, I'd look at costs of RAM vs costs of a SSD and determine which way you want to go before you blindly rush down any one path. Also, you really have to do your own "cost/benefit analysis" aka: is it worth spending the money to ease some of these problems now? If you have people pitching in for server costs, sure. If you're like me and only host for upwards of 20 people, none of which are paying anything, then maybe not. For me, a ramdisk cost nothing, and gives me better I/O than any hard drive storage could possibly give, but I had enough RAM to handle the task already.
     
  30. Offline

    SeeD419

    Awesome thanks! Will try this.
     
Thread Status:
Not open for further replies.

Share This Page