Optimize JAVA for MineCraft in a EXTREME way :P -- NEED FEEDBACK, THX !! [- I AM BACK -!]

Discussion in 'Bukkit Discussion' started by Therawroy, Jul 12, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    Kane

    @Therawroy I careless for ram and more about CPU. Should I still be looking into these?
     
  2. Offline

    blutherz

    This java settings looking good im running them now for already 3 days on my ubuntu 10 lts x64 server with 8GB RAM and giving the server 4GB.
    At moment i have only 6 player but this with 10 worlds and 54 plugins
     
  3. Offline

    odielag

    Glad to hear it! BTW, I took off the last thing on the line, the "cp" because I don't know how to do the whole argument.
     
  4. Offline

    Therawroy

    Guys, thx first for all your feedback =)

    And to show you that i still working on this thing, you should jump to Page 1. I changed the whole command line and tweaked it a lot more ;)

    With the command "-native" i could even more get the memory down with a working game .... BUT this command doesnt work on many server configurations .... give it a try ;)
     
  5. Offline

    cyberdude

    I can't wait to see someone test this with the new build 1000, which should "fix for a huge memory leak". Wondering whether the fix means that we don't need these optimized run switches, or if these optimization's will improve it even more.
     
  6. Offline

    Therawroy

    Maybe they used some of the datas i provided :p

    PS: I also tested it with 1000 and 1001 Version of bukkit ;)
     
  7. Offline

    odielag

    Just updated my wrapper.properties for my 24gb ram centos java1.7 machine that uses rtoolkit
    (cpu usage may have seemed to go down)

    Code:
    server-arguments=server,Xincgc,Xmx12G,Xms4G,Xmn1G,XX:NewRatio=2,XX:CMSFullGCsBeforeCompaction=1,XX:SoftRefLRUPolicyMSPerMB=2048,XX:+CMSParallelRemarkEnabled,XX:+UseParNewGC,XX:+UseAdaptiveSizePolicy,XX:+DisableExplicitGC,Xnoclassgc,oss4M,ss4M,XX:+UseFastAccessorMethods,XX:CMSInitiatingOccupancyFraction=90,XX:+UseConcMarkSweepGC,XX:UseSSE=4,XX:+UseCMSCompactAtFullCollection,XX:ParallelGCThreads=8,XX:+AggressiveOpts,nogui
     
  8. Offline

    Therawroy

    try to set "-native" after java !! Helps a lot !

    Ok Guys :p

    First Benchmark with CB #1000:

    - 32 Bit Java 1.7.0 + Ubuntu 32 Bit
    - Plugins: 75
    - Maps: 3 (World BIG / Nether / Dungeon)
    - Used also DynMap on each Server and rendered all 3 Cards with DynMap v 0.20 HD Mode
    - Java Native Mode DISABLED (Helps even more)
    - All Maps and Plugins were in 2 different RAM-Drives with a size of 1024MB


    Standart Config:
    After 50% --> 1.1GB Ram used !
    After 75% --> 1.3GB Ram used !
    After 100% and 2 Hours (no players) --> 1.4GB Ram used !


    Optimized Config:
    After 50% --> 629MB Ram used !
    After 75% --> 653MB Ram used !
    After 100% and 2 Hours (no players) --> 691MB Ram used !

    Optimized Config with "-native":
    After 50% --> 529MB Ram used !
    After 75% --> 561MB Ram used !
    After 100% and 2 Hours (no players) --> 612MB Ram used !

    Speed Difference --> 10 - 25% faster [Depends on the Stuff you do]

    You can see it in the time difference how the Tiles were rendered...


    [​IMG]

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  9. Offline

    Kane

    Looks really good. So are your configs also reducing the load and cpu usage? I worry when dealing with memory tweaks that really its just going cause more cpu usage.

    My goal is to reduce cpu then memory I mean never had issues with 16 gigs of ram yet. I guess the guys with 1-2 maybe even 4 do but never had issues with the bigger guy.

    Always CPU and CPU lol
     
  10. Offline

    Mhalkyer

    @Therawroy EXCELLENT work man, really great stuff. I had a few questions/comments:

    1. Does the player have to edit their Java any? Will they get any performance boost using Java 1.7?
    2. What does the '-cp' param do?
    3. Could you add an explanation to the OP of the params?
    4. If you edit your post, you should be able to replace the :p with =P
     
  11. Offline

    Therawroy

    I will write a little guide soon which shows you some tricks, what you can also optimize, beside the command line. Also i will tell more what most of the comnmands mean.
    For me its important to hear from your experience without trying to optimize it by yourself ;)

    The Guide will include things like:
    - Java Priority change
    - Priority change
    - Command line explanation
    - some small Java Tweaks ;)
    ......

    Answer:
    1. Normally the Users should update to Java 1.7.0. Its released since 2 Days and will also be the base for other things. But you will get a benchmark of the difference soon between 1.6.0 VS 1.7.0
    2. Think of the "-cp" parameter as something like that i can use parameters, which arent allowed in this way or need to be coded. Its essential for most paramters or you get Problems with many PlugIns. You can try it without and should see the difference ;) PLS BACKUP YOUR WORLD FIRST !
    3. soon in a seperate guide
    4. didnt work =( i tried

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  12. Offline

    Mhalkyer

    Thanks for the response, very helpful =)
     
  13. Offline

    Kane

    Code:
    [plain]:) [b]test[/b][/plain]
     
  14. Offline

    Rabus

    XX:parallelGCThreads=4 - Set the number to the same amount of Cores in your Server

    Why cores? Not threads ... ?
     
  15. Offline

    odielag

    One suggestions for all linux users out there, use a program like "screen" because what screen does is it makes a virtual console that you can go into and out of.... here is my start command... (mainly copied the OP)

    Code:
    #!/bin/sh
    screen -AS odminecraft java -native -server -Xincgc -Xmx12G -Xms4G -Xmn1G -XX:NewRatio=3 -XX:+UseThreadPriorities -XX:CMSFullGCsBeforeCompaction=1 -XX:SoftRefLRUPolicyMSPerMB=2048 -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy -XX:+DisableExplicitGC -Xnoclassgc -oss4M -ss4M -XX:+UseFastAccessorMethods -XX:CMSInitiatingOccupancyFraction=90 -XX:+UseConcMarkSweepGC -XX:UseSSE=4 -XX:+UseCMSCompactAtFullCollection -XX:ParallelGCThreads=8 -XX:+AggressiveOpts -cp "craftbukkit-0.0.1-SNAPSHOT.jar" -jar "craftbukkit-0.0.1-SNAPSHOT.jar" nogui
     
  16. Offline

    Kane

    I was told by a few IT people to set to cores not threads also.
     
  17. Offline

    will d

    I feel like this startup line greatly complicates what is otherwise a simple task of starting a java instance, while also introduces many tweaks that work counter-productively to the way Java garbage collection works. In other words, not only does this (I've only listed one instead of four for space concerns) complicate start-up, it fails to accomplish what you claim it does, as well as it is not fitting for all the people who might have [32 bit servers]--in the general sense.

    First off, you should be separating them (further) by number of cores in a system. You fail to mention that these tweaks do not make sense to a single core machine. -XX:+UseParNewGC and -XX:+UseConcMarkSweepGC are useless to single core machines. To be more helpful, list "for 32 bit servers with one core" and "for 32 bit servers with >1 core"
    The reason that one would give 2GB from the start is: if your server will eventually end up using it (expected world traffic reaches ~2gb), then in the meantime, the JVM will not have to waste overhead on increasing the size by 20% each time the young generation memory space maxes out; each max-out of the young generation space incurs a young-generation garbage collection and then the reallocation (and resulting fragmentation). If you insist on keeping the two values separate (xmx and xms), at least tweak -XX:YoungGenerationSizeIncrement otherwise a server that is on its way from 512MB to 2048MB will incur 8 or more garbage collections you didn't need to have in order to get to the max heap size you already are willing to give to the app.
    It doesn't mean that the young collector gets 256MB, it means the young generations' space = 256MB. Based on your example, of a maximum heap size of 2048MB: (if you omit the Xmn argument, based on SurvivorRatio=32 default)
    1. the young+survivor size = 682MB
    2. young size = 642MB
    3. survivor size (2 each) = 20MB x 2
    Because you adjusted it, you forced the following:

    1. the young+survivor size = 682MB
    2. young size = 256MB
    3. survivor size (2 each) = 213MB x 2
    Having more survivor size than a young generation is a BAD layout.

    A NewRatio=3 would be even less useful setup. Given the maximum heap size of 2048MB (and omitting yoru Xmn argument)
    1. the young+survivor size = 512MB
    2. young size = 482MB
    3. survivor size (2 each) = 15MB x2
    by using your Xmn argument:
    1. the young+survivor size = 512MB
    2. young size = 256MB
    3. survivor size (2 each) = 128MB x2
    Ideally, a larger heap for the young generation (which can be collected in parallel) would be the best configuration. It would prevent short lived objects from prematurely being moved into the tenured generation, where GC is more expensive. A young generation, too, should never exceed half of the heap, too, so there is an ideal number, but not one that is reached through your NewRatio argument.


    NewRatio should stay low, SurvivalRatio can stay high, but maybe be lowered to something like 16 instead of default 32.

    This is a strictly useless tweak; it does not accomplish anything practical at all. You've increased the default value from 1000 to 2048. The value is measured in milliseconds, so I'm not sure why you changed the 10^x value to a 2^x one, which isnt really appropriate or meaningful. With regards to its function, after the last strong link to an object is garbage collected, you're telling the GC to wait an addition 1048 milliseconds to consider that link stale. This provides no performance benefit...at all. Might even hurt on a microscopic level.

    There are a few other things, but for the most part I see those as the glaring issues demanding attention.
     
  18. Offline

    Monstertke


    And thread.
     
  19. Offline

    Therawroy

    Monstertke: lol, i really understand you and some others ... but! there is some difference if you change some stuff ... but its not that big in CPU or RAM advantage, but i search for a good balance between them.

    So in the end 10-20% more speed or less ram is really not that important ... but is it really sooo bad that i try to help other people ? I mean there is no one here who had an problem or lost his world or whatever .... its for the small servers on this world, so that they can also play minecraft .....

    PS: Got my new Server Ready :p

    Core i7 950 // 8GB RAM // 1500GB HDD RAID0 and i try Ubuntu 11.10 with Kernel 3.0 RC5 :p

    PSS: I also measure the difference between the Java Command Lines with built in tools and some extern Tools, so i can give you in the end a much shorter command line ;)

    And i see you got some technical skill in this things....but java and the use of the commando lines (and the monitoring) showed me which commandos worked .... and i mean look at the RAM and Performance ... what is there else to say ?
    I got also my technical skill, and this would not be the first time that i had an better answer to a question then most people who are involved with it. And besides .... its only for help, but i am bored after reading that.

    Copy, Google and understand the command is one thing ... monitoring and testing and searching for the reason is another.

    Enough of the harsh words :p Its monday .... thats all

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  20. Offline

    will d

    If you care more about CPU usage than memory usage, you should not be using NewRatio=3 or -Xmn256M at such a low value (in fact, you'd be better off not setting this argument at all). Both of these changed settings (as opposed to leaving them omitted or default), cause more CPU usage.
     
  21. Offline

    Kane

    Thanks. I mean to me I don't give a shit about memory all about cpu haha.
     
  22. Offline

    Therawroy

    Want a little bit more horsepower [EXPERIMENTAL] .... then put -Xrs in your command line ;)

    Also....search with the command "top" for the PID of "java" and use the command:

    renice 18 "PID NR" example: renice 18 3765

    The command gives "java" a much higher priority

    PLS PLS PLS dont set it higher ... even 19 can crash the total system !!!!
     
  23. Offline

    will d

    I'm actually kind of baffled at this post here. I don't mean to sound mean, but this particular post is only illustrating you're not properly comprehending the java arguments you're talking about.

    As you say, "Copy, Google and understand the command is one thing ... monitoring and testing and searching for the reason is another."

    So the question I raise is this: did you test your server without -Xrs and then WITH -Xrs and then found you got a 'little bit more horsepower'?

    If so, I am astounded, since the function of -Xrs has no performance ramifications: "Note that -Xrs (reduce signal usage) is a JVM option that can be used to prevent the JVM from using most signals. "
    (reference: https://www.ibm.com/developerworks/ibm/library/i-signalhandling/)
    In other words, this argument makes numerous events, such as "Bus error (attempt to address nonexistent memory location)" or "File size limit exceeded" or ....... "Termination request. JVM will exit normally." basically do nothing. These are functions...interior functions you do not want to disable and provide no performance impact. To think that by ignoring requests for a JVM to close normally you are squeezing out extra CPU power is ... downright folly. And if you are saving CPU power, is it...what, one dozen out of the 3 million you get per second?

    This is just downright terrible advice; it's hard to be diplomatic about it. And that doesn't even tackle the other glaring issue of renicing java.... You know that the renice scale goes from negative to positive, right...where -20 means "high priority" and +20 means "lowest possible priority?" So you're recommending to all these users to put java priority lower than every other possible service on the machine. How is that maximizing performance?
     
    fysics likes this.
  24. Offline

    Simanova

    20 = low priority
    -20 = high priority

    its easy
     
  25. Offline

    Therawroy

    nothing :p its late here

    So i can ALSO make a mistake =( .... PLS forgive me .... but its nice that you all have so much effort in this case =)

    He is right ... change it to (-)

    And Xrs- is helping a little bit if you are using a Virtual Machine .... its not over 5-8% .... but you know , i like it long .... like my command lines xD

    Sorry .... :p Its late here
     
  26. Offline

    Monstertke

    This thread is a day late and a dollar short anyway. This has been mostly thrashed to death on other threads. And its obvious at this point that will d knows FAR more about this subject than you do. All you have to do is read the whitepaper with a little comprehension and you will get farther than what you are suggesting.

    http://pastebin.com/WyhT4wZ1
     
  27. Offline

    Therawroy

    Ladys, i got my new server ... no vServer and Power like a Beast ... and enough RAM :p

    And to get some heat away from the thread, i can understand now why people with Big Servers dont see that much difference ... you can notice some things and also measure it, but its a huge difference from my vServer.

    In the end ... if you got the Power and Ram, you probably dont need all the stuff in the command line ..... but if you have a real poor server and low memory ..... my command line is the way to go =)
    [Until you buy a new Server]

    Thx for all your Feedback ... even if you tried to be mean to me :p And Sorry for my english ... i am german ;)

    And for now, my last command line .... specially optimized for ya and almost BETA Status :p Thx to VMWare and the abilities it gave me to test the stuff directly.

    -- And if you dont know what comes now .... here again ..... BACKUP YOUR WORLD !!!


    1. - Orange commands can be changed or removed (if needed)
    2. - Pink commands can make Problems (try or die :p)


    [64 Bit Optimized for pure Satisfaction]

    java -native -server -Xincgc -Xmx6144M -Xms2048M -Xmn512M -XX:NewRatio=3 -Xrs -XX:+UseThreadPriorities -XX:CMSFullGCsBeforeCompaction=1 -XX:SoftRefLRUPolicyMSPerMB=2048 -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy -XX:+DisableExplicitGC -Xnoclassgc -oss4M -ss4M -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseConcMarkSweepGC -XX:UseSSE=4 -XX:+UseCMSCompactAtFullCollection -XX:ParallelGCThreads=4 -Xcheck:jni -Xfuture -XX:+AggressiveOpts -cp "craftbukkit-0.0.1-SNAPSHOT.jar" -jar "craftbukkit-0.0.1-SNAPSHOT.jar" nogui


    [32 Bit Optimized for longtime Crafting !!!]

    java -native -server -Xincgc -Xmx2048M -Xms512M -Xmn256M -XX:NewRatio=2 -Xrs -XX:+UseThreadPriorities -XX:CMSFullGCsBeforeCompaction=1 -XX:SoftRefLRUPolicyMSPerMB=1024 -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy -XX:+DisableExplicitGC -Xnoclassgc -oss4M -ss4M -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=90 -XX:+UseConcMarkSweepGC -XX:UseSSE=4 -XX:+UseCMSCompactAtFullCollection -XX:ParallelGCThreads=2 -Xcheck:jni -Xfuture -XX:+AggressiveOpts -cp "craftbukkit-0.0.1-SNAPSHOT.jar" -jar "craftbukkit-0.0.1-SNAPSHOT.jar" nogui

    - And did you see ??? I got a fix for the smiley problem :p
    -->> With this Settings, i tried to gain something like 5-10% better Java Utilisation for 32 / 64 Bit Systems ..... Have fun =)​

    -- And if you dont know what comes now .... here again ..... BACKUP YOUR WORLD !!!

    I like the Internet ... you can see all dates ;) And this was today .... nice try ... next one ;) .... nah read my text above and cool down :p

    And for the Rest ... try my new command line above and have fun :p



    [​IMG]
    [​IMG]
    By: a guest | Jul 20th, 2011 | Syntax: None | Size: 0.48 KB | Views: 11 | Expires: Never

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  28. Offline

    Monstertke

    Lol the above paste is copied out of my McMyAdmin .conf about 20 seconds before I replied to your post. So yea Im busted. ;)

    http://forums.bukkit.org/threads/ho...-24gb-dedicated-server.3967/page-2#post-59105

    This ^^^^ is what I was talking about when I said that this was a rehash. But whatever, Java is forgiving it will either create the VM or it wont. Also throw 75 people on then run your benchmark, Cause dynmap fullrender isnt very taxing and I doubt that any of its chunkloads arent picked up By Gc in the young generation when it is most efficient.

    With Bukkit #1000 I can go 36 hours with an average of 40 people (high 60 low 10) on, before hitting the FullGC thrash of death. Core i7 860, 16Gb DDR3 with 12 set to largepages and dedicated to java.

    Ps. Xcheck:jni is pointless, its a debugging tool and ads overhead.
     
  29. Offline

    pomo4ka

    Hi, can you tell us how to distribute the server on every core?
     
  30. Offline

    Zalastri

    No, he could give you a vague guess on how to wrongly do it though.
     
Thread Status:
Not open for further replies.

Share This Page