Solved Reverting from Anvil world format to support WGenUtils.

Hello everyone,

My work in developing a world generation plugin would be made extraordinarily easier if I were able to use WGenUtils, detailed here. However, it has never been updated to work properly with the post-Anvil map format generation of block sections. Does anyone know of an alternative to WGenUtils (fire and forget functions for generation of caves and ravines), or a way to rework the block section generation to still work with WGenUtils?

Thanks, any help would be tremendously appreciated.

 

Since I have no experience of this whatsoever, I cannot help you, but I hope someone responds to this with an answer.

 

Well, even if you don't have any experience with world generation specifically, perhaps you could still help me--I think I've found the changes needed to use this library. The new method of generating world uses a 2D array of bytes--the old method used a 1D array. If this 2D byte array could be repacked into a 1D byte array, it could be run through the methods of WGenUtils. Reversing the process and converting back to a 2D array would allow us to generate the world with the modifications needed for it.

I'm going to begin experimenting.

 

Derthmonuter

Code:

y = index % 128
index -= y
index /= 128
z = index % 16
index -= z
x = index / 16

from array index to chunk local coords

 

fireblast709
Are those chunk local coordinates in the newer system or the older one? Because I've managed to take apart the 2D array and stitch it into a 1D, and undo the process, but I'm clearly not constructing my 1D array the way it should be. I'm currently just reattaching the current system of 16 16x16x16 chunks into one large 16x16x256 chunk as if they were all stacked on top of one another.

Do you know how the old chunks positioned their block coordinates into the 1D array?

 

Derthmonuter the world height was 128 back then

 

fireblast709
Ah, I see. So this is a handy way to get the old-world coordinates from the array index. Is there an equivalent you know of to get the new local coordinates of one of the 16x16x16 world pieces?

Basically, I need to know how to undo this operation:
((y & 0xF) << 8) | (z << 4) | x

 

Derthmonuter in bits, yyyyzzzzxxxx. So 001001000001 is 1,2+(section*16),4

 

fireblast709
I know absolutely nothing about bitwise operators. I'm reading about them right now in an attempt to properly break that byte apart into the y, z, and x components, but it's definitely a struggle. If you happen to have a grasp of this far better than I do, I'd appreciate you simply throwing the proper unpacking code at me.

 

Derthmonuter the first 4 bits can be retrieved with a simple bitwise AND operation on the bitmask 0x0F. If you want the middle set (z), shift 4 to the right (>>) and use the bitwise AND to extract. Likewise for the leftmost group (y), aside that you shift twice (or once 8, obviously). Lastly you need to fix the y, since it is section-local. The first array dimension should be the slice index, so adding 16 * index should give the correct y (16 slices of 16 height, like you already mentioned before)

 

fireblast709
What do I seem to be doing wrong? I want to write some utility classes to assist with this packing and unpacking before I try delving back into the library work.

Code:java

public class BitPacking {
 
    /**
    * @param args the command line arguments
    */
    public static void main(String[] args) {
        byte[][] chunk = new byte[16][];
        Scanner scanner = new Scanner(System.in);
        System.out.println("Please enter the x, y, then z coordinates.");
        int x = scanner.nextInt();
        int y = scanner.nextInt();
        int z = scanner.nextInt();
 
        if (chunk[y >> 4] == null) {
            chunk[y >> 4] = new byte[16 * 16 * 16];
        }
//        chunk[y >> 4][((y & 0xF) << 8) | (z << 4) | x] = 1;
//        System.out.println("Packing a y of " + ((y & 0xF) << 8));
        int index = (((y & 0xF) << 8) | (z << 4) | x);
        System.out.println("Inner index of " + index);
        int unpackedY = index & 0x0f;
        int unpackedZ = index >> 4 & 0x0f;
        int unpackedX = index >> 8 & 0x0f;
 
        System.out.println("Unpacking the information entered...");
        System.out.println("Unpacked: " + unpackedX + " " + unpackedY + " " + unpackedZ);
    }
 
}

However, this is giving an unexpected result:
Please enter the x, y, then z coordinates.
2
3
6
Inner index of 866
Unpacking the information entered...
Unpacked: 3 2 6
BUILD SUCCESSFUL (total time: 5 seconds)

 

Derthmonuter your unpackedY is actually unpackedX and vice versa. Also try () around the bitshift

 

Modified it slightly and now I'm getting the expected results. I know I still haven't handled fixing the y's, but that's not too difficult to do.

Code:java

public class BitPacking {
 
    /**
    * @param args the command line arguments
    */
    public static void main(String[] args) {
        byte[][] chunk = new byte[16][];
        Scanner scanner = new Scanner(System.in);
        System.out.println("Please enter the x, y, then z coordinates.");
        int x = scanner.nextInt();
        int y = scanner.nextInt();
        int z = scanner.nextInt();
        System.out.println("Working in the " + (y >> 4) + " chunk");
        if (chunk[y >> 4] == null) {
            chunk[y >> 4] = new byte[16 * 16 * 16];
        }
//        chunk[y >> 4][((y & 0xF) << 8) | (z << 4) | x] = 1;
//        System.out.println("Packing a y of " + ((y & 0xF) << 8));
        int index = (((y & 0xF) << 8) | (z << 4) | x);
        System.out.println("Inner index of " + index);
        int unpackedY = index >> 8 & 0x0f;
        int unpackedZ = index >> 4 & 0x0f;
        int unpackedX = index & 0x0f;
 
        System.out.println("Unpacking the information entered...");
        System.out.println("Unpacked: " + unpackedX + " " + unpackedY + " " + unpackedZ);
    }
 
}

So at this point I think I have everything I'll need, just gonna try thinking aloud here. Any input from anyone is welcome to correct me--
1) I have a 2D array packed in the new way.
2) I chop this in half to turn it into another 2D array with a 128 maximum height. That's all the old style supported so that's all the library will probably support as well.
3) I will loop through the entire 2D array, and for each element unpack it to get its 0-15, 0-127, 0-15 chunk coordinates.
4) I will encode these coordinates in the old style to find their appropriate indices for the 1D array, and then copy the material data from their points in the 2D array into those new indices.
5) The 1D array will be properly converted to and is ready for use with WGenUtils.
6) I will decode the 1D array as fireblast709 gave the equations to above.
7) These will give the chunk coordinates as needed to be re-encoded (with WGenUtil changes) into the new style once more--a 2D array.
8) This is only the bottom half of the 2D array. It will be reattached to the top 128 blocks it was broken off of in step 2.
9) Once reassembled, the 2D array will be ready to be passed to further world generation.
10) I heave a sigh of relief if this all works.

Alright. I've done everything on my list above and it's not working. I've run into a snag--the new method which generates block sections does just that--it generates sections. My approach needs the entire chunk to be completed before it gets processed by the cave generator.

Does anyone know of a good approach which would allow me to know for certain that the entire chunk has finished being generated?

Update: I finally got it all working.
It required a bit of tweaking of WGenUtils code itself.
The limitation remains that it can't seem to change any blocks above 128.
Once I polish it up a bit more I'll see about a release.

I've bundled what I've created into a little package here for anyone who would like to continue trying to use WGenUtils. What's included in that zip folder is a modified WGenUtils .jar which is hardcoded to use 128 height, rather than break when trying to use 256 height. There is also a copy of the Example.java file shown below.

Code:java

import java.util.Random;
import org.bukkit.Material;
import org.bukkit.World;
import org.bukkit.generator.ChunkGenerator;
import world.generation.utilities.MapGenBase;
import world.generation.utilities.MapGenCaves;
import world.generation.utilities.MapGenOre;
import world.generation.utilities.MapGenRavine;
 
/**
* A class to demonstrate some features of the recreated WGenUtils.
*
* @author Tim Clancy
* @version 8.5.2014
*/
public class Example extends ChunkGenerator {
 
@Override
public byte[][] generateBlockSections(World world, Random random, int chunkX, int chunkZ, BiomeGrid biomeGrid) {
byte[][] chunk = new byte[world.getMaxHeight() / 16][]; //Create the 2D byte array for terrain.
//
//Typically here is where one would do the bulk of terrain generation--such as the creation of the Perlin noise heightmap.
//
//Convert the 2D array to a 1D array so WGenUtils can work.
byte[] singleByte = toSingleByte(chunk);
 
//Generate caves with WGenUtils
MapGenBase cavesGen = new MapGenCaves();
cavesGen.generate(world, chunkX, chunkZ, singleByte);
 
//Generate ravines with WGenUtils
MapGenBase ravineGen = new MapGenRavine();
ravineGen.generate(world, chunkX, chunkZ, singleByte);
 
//Generate ore last
MapGenOre oreGen = new MapGenOre();
oreGen.generate(world, chunkX, chunkZ, singleByte);
 
//Reattach the pieces of array.
byte[][] newBottomHalf = toDoubleByte(singleByte);
for (int i = 0; i < 8; ++i) {
for (int j = 0; j < 4096; ++j) {
if (chunk[I] != null) {[/I]
[I] chunk[I][j] = newBottomHalf[I][j];[/I][/I][/I]
[I] }[/I]
[I] }[/I]
[I] }[/I]
 
[I] //Return the entire 2D array[/I]
[I] return chunk;[/I]
[I] }[/I]
 
[I] //The conversion methods.[/I]
[I] private byte[] toSingleByte(byte[][] chunk) {[/I]
[I] byte[] single = new byte[32768];[/I]
[I] //Only grab up to 128 y to respect old height limit.[/I]
[I] for (int sector = 0; sector < 8; ++sector) {[/I]
 
[I] if (chunk[sector] != null) {[/I]
[I] for (int index = 0; index < 4096; ++index) {[/I]
[I] int y = (index >> 8 & 0x0f) + 16 * sector;[/I]
[I] int z = index >> 4 & 0x0f;[/I]
[I] int x = index & 0x0f;[/I]
 
[I] int oldIndex = (x * 16 + z) * 128 + y;[/I]
[I] single[oldIndex] = chunk[sector][index];[/I]
[I] }[/I]
[I] } else {[/I]
[I] for (int index = 0; index < 4096; ++index) {[/I]
[I] int y = (index >> 8 & 0x0f) + 16 * sector;[/I]
[I] int z = index >> 4 & 0x0f;[/I]
[I] int x = index & 0x0f;[/I]
 
[I] int oldIndex = (x * 16 + z) * 128 + y;[/I]
[I] single[oldIndex] = (byte) Material.AIR.getId();[/I]
[I] }[/I]
[I] }[/I]
[I] }[/I]
[I] return single;[/I]
[I] }[/I]
 
[I] private byte[][] toDoubleByte(byte[] singleByte) {[/I]
[I] byte[][] doubleArray = new byte[8][4096];[/I]
[I] for (int index = 0; index < 32768; ++index) {[/I]
[I] int x;[/I]
[I] int y;[/I]
[I] int z;[/I]
 
[I] int calculationIndex = index;[/I]
[I] y = calculationIndex % 128;[/I]
[I] calculationIndex -= y;[/I]
[I] calculationIndex /= 128;[/I]
[I] z = calculationIndex % 16;[/I]
[I] calculationIndex -= z;[/I]
[I] x = calculationIndex / 16;[/I]
 
[I] if (doubleArray[y >> 4] == null) {[/I]
[I] doubleArray[y >> 4] = new byte[16 * 16 * 16];[/I]
[I] }[/I]
[I] doubleArray[y >> 4][((y & 0xF) << 8) | (z << 4) | x] = singleByte[index];[/I]
[I] }[/I]
[I] return doubleArray;[/I]
[I] }[/I]
[I]}[/I]

Currently I've only fixed cave, ore, and ravine generation. The caves look pretty good, the ravines are stellar, but the ore generation is lackluster for me. However, it might still work for you because I'm testing all of this on MCPC+ 1.6.4.

The limitation remains that WGenUtils can only consider blocks below 128 for modification. For most worlds, this is acceptable. A special thanks to fireblast709 for help wrapping my head around the byte-packing nonsense, and a notification to pookeythekid that it's done. At some point I might consider modifying all of the library to revive its other world generation features, but if someone wants to do that before me, one simply needs to go to the Github page, download the source, and replace any instances of "worldObj.getMaxHeight()" with 128 before using with my converters.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Derthmonuter Yay you solved it! xD