Map Height

Discussion in 'Bukkit Discussion' started by Scyfi, Sep 14, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    JesterAzazel

    Ok, I'll google how to do that. Thanks for the advice. :)
     
  2. Offline

    xianthax

    Made some progress on this...looks really crappy....

    So the world height is technically defined in one location in rv.class (which is the World class).

    However, its defined as final and initialized on definition, as a result its been inlined to a constant every where that its accessed. My thought about 'notch leaving us a trail' wasn't accurate but close, that appears to be an artifact of the inlining from the compiler but no in every instance, there are cases where the value has been inlined but isn't used as a short.

    So I have a list of everywhere which these values were inlined to constants, but its hits almost 30 source files and over 100 locations, additionally there are some other constants defined in World that appear to have an impact on the process (the 7 and 11 are used as bit shifts, not clear the real effect at the time).

    What that means is that the only way to make a mod that would allow setting the height appear to require modifying ~30 source files which will probably create a compatibility nightmare.
     
  3. Offline

    Alzurana

    Well notch said there would be only one value within the level class but it's an old post from twitter.
    http://twitter.com/#!/notch/status/93295073374113792
    Maybe it's not in 1.8.1, already and that's why we get that much locations.
    Even if you would have to modify so much base classes there would be an answer to the compatibility nightmare.
    The minecraftForge by spacetoad.
    http://www.minecraftforum.net/topic/514000-api-minecraft-forge/
    It's a pretty young open source project to form one API for all base class modifications. At least, that would be a solution for the client side.

    I still don't feel comftable with the idea of modding the client. The server should just transmit the height and the client should follow... : /
    A prayer to notch.
    Sad, that I can't try it on my own, my left hand is broken. xD Even writing here is a pain in the..
     
  4. Offline

    ghost0001

    JesterAzazel:

    i made a server that runs from my home PC in Texas. I used the DMZ method of placement on my network. I posted a TUT here.
     
  5. Offline

    xianthax

    it would be much easier if the variables were not declared final then you could just change them to have an effect, maybe we can get him to do so in a 1.8.2 or something.

    I have gone through and modded the height to 512 but i'm getting buffer overflows. There are 2 other variables in the World class, a 7 and an 11 that are used in awkward ways to index arrays or data, i need to play with these values more before i have something that may work.

    I'm not disagreeing with what he said, there is one value in the World class for the height.

    However, it was declared final and initialized immediately, so the java compiler optimized away the variable accesses. So while there was 1 value in the original source, it no long has any effect in decompiled source.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 19, 2016
  6. Offline

    Alzurana

    Ah I see. Inline stuff, totally forgot about that.
    Would be awesome if MC could read that value out of a config.
    Oviously a dynamic height range increases the performance load, also due to the fact, that all the constant values would become dynamic.
    Sometimes I wish MC would be open source c++... xD
     
  7. Offline

    xianthax

    They are defined as public without a getter method involved, so i doubt the performance impact would be noticeable.
     
  8. Offline

    JesterAzazel

    Part of that was helpful, but part of it wasn't. Not everyone's router settings look the same.
    At any rate, I saw a similar tutorial and there was one part of it that wasn't clear, but yours really helped me understand that particular part. Thanks.
     
  9. Offline

    xianthax

    So with the release of MCP life is much easier, bytecode editing sucks....

    So i haven't figured out the relationship of all the variables yet BUT, i do have a taller world:

    http://i.imgur.com/CHteq.jpg

    more progress.....

    pretty straight forward to get a 256 block high world.

    newb tower example: http://i.imgur.com/RW6wA.jpg

    The easily available options allow:

    Changing world height (average terrain height will always be half this using the built in variables, so the 256 block world from the shot above has the average terrain height at 128 ish)

    Changing the water level (oceans only, ponds still form at surface)

    Changing the bio generators height (i think) basically if this is 1 less than the world height, its like a normal world and it can never be = or greater than the world height or a index overflows. Setting this very low results in an almost entirely stone world, i think this primarily effects the biome generation code, there are only a few hooks to it in the code.

    The other two values, which default to 7 and 11 have to be adjusted in accordance with the world height chosen in the following manor:

    128 - 7 - 11
    256 - 8 - 12
    512 - 9 - 13
    etc.

    If you don't adjust them in this manor it causes the chunks to be handled in very odd ways, see my screenshot in the last post.

    What sucks:

    1)
    Going to 512 pretty much bombs, i mean it 'works' but generating the initial world takes a LONG time, generating chunks takes a LONG time, i'm on a core i7 920 clocked at 4.2 ghz and it look about 20 minutes to get into a 512 tall world in a playable manor and whenever a new chunk had to be generated it would just lock up for a few minutes. It may be plausible to have worlds this high but i think that all chunks would have to be pre-genned by the server. Its not clear what the cause of the cpu usage explosion is, going from 128 to 256 is really noticeable on my system, i'll try to profile the code and see whats up, it may be fixable, but appears to be rooted in the chunk generation, once it loads up the graphics performance is ok (noticeably slower but ok).

    2)
    There isn't a simple way to determine the average terrain height. I'd love to have the 'ground' stay at 63 on average but be able to build up to 1024, more work needed to see if this is possible in a reasonable manor. This may be most easily solved by using a larger height in conjunction phoenixterraingen to tweak the surface height.

    As far as using this for multiplayer, as long as the bukkit side has the changes to the generation code (which are much the same as the client) i believe it should work, i'll try to hack up a test against the default server.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 19, 2016
  10. Offline

    Cere4l

    you say youre on a core I7 so im guessing the problem isnt the cpu. i mean technically if you puy yhe height to 512 itd only be the same stress for the server as 4 people running off in different directions and an i7 should handle that with ease.

    Do you have an SSD or a normal disk or a ramdisk though. Im hoping that might make a huge difference still and make it normally playable again
     
  11. Offline

    bluegobln

    What about increasing the available build height without any affect on the generation code whatsoever? Say if I want to just build a slightly taller building...

    Or attempt to import an alpha map that is 0ver 128 in height?

    Doable? Changes nothing?

    If that is doable (preferably with a server only plugin) I would greatly desire to test / make use of it.
     
  12. Offline

    Sugarbit

    Just a guess, but... it's probably minecraft's lighting computations that make things really slow. Sunlight goes down as a cone, not a straight line, so the taller the world, the more area each block of sunlight at maxheight will eventually cover when it hits the ground.
     
  13. Offline

    xianthax

    I have 2 128GB OCZ Vertex 2 SSD's in RAID 0 in this system.

    Its not the same load as 4 players if there are algorithms in there that blow up at O(x^n) or something.

    Not possibly with an easy change, may be possible but eventually but will require alot more testing. Also may be possible by using the height change in combination with phoenixTerrainGen to adjust the generators height output.

    That would be a pain, actually just switching between maps of different heights without corrupting them will take some work as the height change impacts the chunk data directly.

    Not possible without a client mod as the world height is hard coded all over the place. I'm targeting spout/spoutcraft to get a multiplayer version going after I work a bit more on the slowness, i'd really like to get a 1024 blocks for some real 'to scale' construction.

    That could be part of it, however it appears to be multiple issues. I built a new version with optimization on and its definitively better at 512, but still not really smoothly playable.

    1) Chunk generation is dreadfully slow. This isn't all that bad really, in theory you can just pre gen an 8000 block radius and put a world border up.

    2) World update in the server tick is terribly slow. This is the bigger issue for usability of tall maps, I need to profile this more to get a better handle on whats happening. Crossing my fingers that its as simple as some poorly chosen data structures.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 19, 2016
  14. Offline

    ghost0001

    so, if my server defines the values of max height, any client can build to it. thats what i'm getting out of this. so, how easy would it be to add something that is not a power of 2?
     
  15. Offline

    xianthax

    Sorta, not really to the stage of having the details of that process worked out yet.

    Basically what needed to happen:

    1) Server defines height which effects how generation happens (this is easy enough)
    2) Server tells the client what height the world is
    3) Client updates its internal values for world height

    Steps 2 and 3 are the kicker since with default bukkit / the default client, this is not supported. The workable solution will be using Spout on the bukkit side and SpoutCraft on the client side which will allow 2 and 3 to take place but there is work to be done there and this obviously means anyone playing on the server must have SpoutCraft installed.

    Not possible today (or ever without changes by notch), the height has to be a multiple of 2 from 128 as a base so 128, 256,512,1024 or the minecraft code will explode. There is no easy fix for this, its pretty well ingrained in the code and i doubt it will ever be deemed worth the effort to change.
     
  16. Offline

    Gerbil

    This is awesome! In practice how do I change this myself? How many places would I have to change?
     
  17. Offline

    Cere4l


    Im not very familiar with spout, would it be possible to have one normal world. with a normal map height and everything. and a portal transporting you to another world where spout IS active with extra map height, with some sort of warning for people who dont have spout.

    and with that i mean, would it be possible without me learning programming of course ;) because obviously it would be possible in the end. but im speaking excisting plugins and the future spoutheight(?)

    ah that was the only guess i could offer in terms of speed :(
     
  18. Offline

    Celeixen

    Canyouseeme.org <-- use that to test just type in the port your running the server on.
     
  19. Offline

    JamesS

    If somebody could explain in plain english how we can change the height of maps, if at all possible, that would be great! I'm having a hard time wading through the jargon of Java codes.

    I am an Artist and Architect. I am not supposed to think about these things... :<
     
  20. Offline

    xianthax

    I am still working on this however the challenges will come with integrating this with both spoutcraft, spout and internal craftbukkit changes that are required.

    Really changing the hardcoded height and getting that working was just the first step, additional things will need to be done to store or extract the height used when loading the saved world. Changing the hard coded values then loading a world of a different height just making it blow up and corrupt the world today.

    I don't think its my hardware but it does appear to be at least related to chunk saving, it happens once a second which is exactly the trigger of a massive lag issue with large world height...

    In a twitter post a couple days ago notch said he was working on moving chunk saving to a separate thread to mitigate this issue (which does appear at normal height its just much less pronounced).

    I'll try this out with 1.9 to see if its in there as soon as MCP is out for 1.9 and I have some time (assuming the change is in 1.9).

    If you want to play with this yourself here are the steps, i'm not releasing a mod yet as there is work to be done to ensure that you can still load standard height worlds without wrecking your saves. In other words only run this on clean .minecraft folder or make sure you backup your saves before trying to load them with the modified height client.

    This assumes you know java and can install / use MCP and an IDE (intelliJ or eclipse recommended)

    1) In World.java there are 5 member variables defined, the default names are like rvfield_a or something like that, whatever the decompiler spit out as the original names are just a,b,c,d,e.

    Anyway they are the only 'public final int' variables defined in World.java and they are grouped together, easy to find.

    2) (Optional) Rename these variables to something you can remember more easily. What I use ends up looking like this:

    public final int shift7 = 7;
    public final int shift11 = 11;
    public final int worldHeight = 128;
    public final int biomeHeight = 127;
    public final int waterHeight = 63;

    3) The fun part. The compiler has optimized away access to these variables, thus we need to go through all the code and remove this optimization. Luckily the compiler left behind clues for us to use for find the correct locations. There are over 100 spots and like 25-30 files this is where an IDE will make life much easier.

    A simple example of what your looking for:

    worldObj.getClass();
    int i = 128 - 1;

    The bit in bold is what your looking for as the remnants of the compilers optimization, this tells you that in the next line of code the compiler has inserted a static value (the 128) in place of a variable. We need to undo this, thus we need to turn this code into:

    worldObj.getClass();
    int i = worldObj.worldHeight - 1;

    We just replaced the 128 with an access to the variable we renamed in step 2. Its easy to figure out which variable to use based on the static values you find, that is 128 is always worldHeight, so for instance:

    worldObj.getClass();
    worldObj.getClass();
    for(int i1 = j << 11 | k << 7; l > 0 && Block.lightOpacity[blocks[(i1 + l) - 1] & 0xff] == 0; l--) { }

    needs to be replaced with

    worldObj.getClass();
    worldObj.getClass();
    for(int i1 = j << worldObj.shift11 | k << worldObj.shift7; l > 0 && Block.lightOpacity[blocks[(i1 + l) - 1] & 0xff] == 0; l--) { }

    Minecraft does very little class reflection so the easiest way to find all the locations you need to change is to do a full project search for 'getClass()' 85% of lines found are locations that need to be changed. Note that the variable is not always named 'worldObj' it varies depending on the location but its easy to figure out.

    4) Other stuff / gotchas

    The compiler inlining in world.java is not indicated with getClass() accesses, however it appears that every use of 128, 127, 63, 7, 11 as literals in this file should be accesses to the newly renamed variables so you can just search for instances of these numbers in this file.

    There are a couple spots where the 127 value (biomeHeight) is access and stored as a byte, in these locations the variable type should be changed to int.

    5) At this point you can now just change the values in the World.java file and build/run and your good to go assuming you changed all the locations correctly.

    Picking values

    worldHeight (default 128)
    must be changed in multiples of 2 so 128, 256,512,1024, etc

    biomeHeight (default 127)
    Changes how high the biome generation code goes (i think). Setting this 1 less than the worldHeight will give you a 'normal' looking world. It can not be greater than or equal to the worldHeight or it'll crash. Setting this very low will result in non-generated biome surface, that is, it'll be a normalish looking world, just all made out of stone.

    waterHeight(default 63)
    appears to only set the ocean / non-lake water height and doesn't seem to effect anything else. Setting it low gives you little to no oceans just deep valleys and huge mountains. Setting it high gives you water world. For a normal world change it according to the world height, so 128 -> 63, 256 -> 127, 512 -> 255, etc

    shift7(default 7) and shift 11(default 11)
    these need to be changed based on the world height chosen in following manor:

    worldHeight - shift7 - shift11
    128 - 7 - 11
    256 - 8 - 12
    512 - 9 - 13
    etc.

    Setting these less than the recommended value results in chunks not fully generating / load and other crazy stuff, see the first screenshot i posted in this thread for an example. Setting higher will crash the client.


    That's it, step 3 is the time consuming and tedious part. If you have any issues with the process let me know, but I'm unlikely to spend much time answering "I don't know java" or "Whats an IDE" type of questions.

    Found a bug in my changes that was the cause of most of the crazy chunk generation / loading times, however the crazy long chunk updates once per second are still there.

    This did allow me to generate a 1024 block high world though so there is hope!
    Screenshots are with FOV at Quake Pro, otherwise you can't see high enough :p
    [​IMG]

    and this is the performance problem:

    [​IMG]


    The frame rate between those massive world update times (the white) is actually pretty good, 30-60fps on my machine.

    Not sure how many are excited about this but i'll keep updating here even though its not working with bukkit yet. That is the ultimate goal here.

    With the 1.9 pre-release it looks like 2 awesome things happened.

    1) The controlling variables are no long final and hence aren't inlined, thus 1 file can be changed in 1 place to change the world height. Additionally the secondary values are automatically calculated so its down to just 1 value.

    2) The chunk saving moving to a separate thread fixed most of the performance issues for tall worlds, there are still occasional pauses but its playable.

    I'm getting 40-60fps moving around a 1024 block high world and only getting major pauses if i force a lot of chunks to gen (flying into to areas quickly):

    [​IMG]

    If you are feeling adventurous and want to try this out:

    Backup all your worlds, get a fresh 1.9 pre release installed this will not work with 1.8.x.

    Don't blame me if you corrupt a saved file trying to open it with this installed, work with new worlds / worlds you created with this installed only.

    Add the following file to minecraft.jar in .minecraft/bin download

    Delete the META-INF folder from minecraft.jar

    Thats it, start as normal, create a new world and it will be 1024 blocks high. This may perform terribly on your system, no clue yet.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 19, 2016
  21. Offline

    Nico1991

    Just tried to modify it by myself:

    Code:
    src\minecraft\net\minecraft\src\Chunk.java:108: error: cannot find symbol
    for(k1 = j << worldObj.shift11 | k << worldObj.shift7; j1 > 0 && Block.lightOpac
    ity[blocks[(k1 + j1) - 1] & 0xff] == 0; j1--) { }
    ^
    
    symbol:   variable k
    location: class Chunk
    1 error
    1 warning
    You've got any help for me?
     
  22. Offline

    xianthax

    Variable in that function is l not k if your working with MCP44's decompile. You really shouldn't change the source other than changing the literal '128' to a variable access.
     
  23. Offline

    Takel

    xianthax, spectacular work digging through the code and trying to get the higher world heights (and posting your results).

    Seeing the 1024 height maps in action has kept alive that little flame in me that wants to build a massive tower to the heavens that is so tall you cannot see the top of it from the ground.
     
    Geethebluesky likes this.
  24. Offline

    xianthax

    Lets hope it can be a reality! I'm really looking to get this going to start a "true to life" scale creative server :)

    I have tried modding the stock 1.9 pre-release server for a higher world height as it includes the nice new single variable just like the client does now. However no dice so far, I can log in fine but the world generates very strangely. I have a feeling theres a bug somewhere in the server code, looks like something in chunk loading/generation, lets hope for a fix with 1.9 final. Failing that it can probably be sorted with the MCP / CraftBukkit releases for 1.9.

    I also tried a SP 2048 high world but the ravine generator explodes with worlds over 1024 blocks, meh, 1024 is enough to build Burj Khalifa so thats good for now.
     
  25. Offline

    Mugz

    Not gonna lie, Xianthax. You're probably my favourite person ever right now. I have this thread as a constant tab so I can keep up on your work - it's amazing. Keep doing great things.
     
    Pim1234 likes this.
  26. Offline

    oliverw92

    Impressive work! Can't wait to see if you get any further :)
     
  27. Offline

    Sugarbit

    Would a hypothetical plugin/server be able to take existing worlds and extend their heights, or would this require writing a script to fill in the gaps in new, taller chunks?
     
  28. Offline

    Cere4l

    Incentive time? the moment this is released im putting down 10 euros for bukkit and 10 for this project :p (it was possible to have per project donations right?)
     
  29. Offline

    QChronoD

    I'm so there man!
    Funny you mention the Burj, I've been waiting and waiting for something to come out that would let me build it. (Already been messing around with floor plans on a local server.)
     
  30. Offline

    xianthax

    Theoretically possible but will be difficult. If you already have the world generated as much as you want it would be far easier, having to generate new chunks and get them to line up with the old chunks would take a lot of tweaking of the generator code to pull off.
     
Thread Status:
Not open for further replies.

Share This Page