PSA: Minecraft 1.7 update; possibly losing structures

Discussion in 'Community News and Announcements' started by EvilSeph, Oct 23, 2013.

Thread Status:
Not open for further replies.
  1. Offline

    EvilSeph Retired Staff

    Minecraft 1.7 introduces changes to the world generation code that could very likely result in structures (like strongholds and nether fortresses) generated in 1.6 no longer being recognised as valid. As this would mean that special behaviour for structures would stop working as expected (like wither skeletons spawning in nether fortresses, for example), we felt that it was important to give our Server Admins a heads up.

    Minecraft 1.6.4 introduces a new structure saving feature that should ensure that structures generated before 1.7 continue to be recognised as valid structures after updating. If you don't want to lose your structures from 1.6, you need to update to 1.6.4 first and make sure the structures are saved.

    What needs to be done
    If you're planning to continue running your 1.6 world come 1.7 and want to ensure that structures retain their special behaviour as expected, there is an extra step you need to take as part of the update process. Unfortunately, we've received reports that taking this extra step results in a noticeable performance hit so you'll have to decide for yourself what the best action is to take on your server. We've provided some extra useful information to help you make this decision in the "Things to note" section below.

    Before updating your server from 1.6 to 1.7, you should ensure you update to 1.6.4-R2.0 first and then visit all the structures in the worlds you wish to save. You can do this manually, but it is recommended you use our utility plugin StructureSaver (see below) to streamline this process and make things easier.

    Manually saving:
    Recommended - automated saving:
    • Download and update to CraftBukkit 1.6.4-R2.0
    • Download and install our StructureSaver utility plugin
    • Run the /save-structures [world] command (see below for detailed information)
    • Remove StructureSaver and restart your server
    First you'll need to download and update to CraftBukkit 1.6.4-R2.0

    Then you'll need to download and install StructureSaver from our GitHub here:
    https://github.com/Bukkit/StructureSaver/releases/download/v1.2/structuresaver-1.2.jar

    Once it is enabled, it supports the following commands:
    /save-structures <worldname> - saves structures in a specific world only
    /save-structures <worldname> force - saves structures in a specific world only, after removing existing structure data
    /save-structures - saves all structures in every world
    /save-structures force - saves all structures in every world after removing existing structure data

    And has the following permission:
    structuresaver.save

    Neglecting to address this ahead of time will result in structures from old worlds no longer being recognised as valid structures come 1.7.

    Things to note:
    Performance issues
    Since releasing the first version of StructureSaver, we've been receiving consistent reports that there is a noticeable decrease in server performance after StructureSaver has saved the structures on the server. To the best of our knowledge, we're certain that this issue isn't caused by StructureSaver itself as the issue continues after the plugin has been removed.

    Note that this decrease in performance is different from the immediate hit you see right after running StructureSaver (which is why we recommend you remove StructureSaver after you're done using it and restart your server).

    Run it once, keep a backup
    The nice thing about StructureSaver is that as long as you have a back up of your world, you're able to run a 1.6.4-R2.0 server with StructureSaver on it and save the structures whenever you want.

    You also have the option of running it now and backing up the resulting structure files in the 'data' folder inside your world's folder instead. Should you then decide later that you need the structures you saved from 1.6.4-R2.0, you can simply replace the already existing structure data as you please.

    Pick and choose
    Once you've run StructureSaver, it is completely up to you what you do with the saved structures. If you feel like saving Mineshafts is not necessary for your server, you can choose to delete the Mineshaft.dat file. Note, however, that it will be recreated by Minecraft whenever a Mineshaft is rediscovered (this applies to all structure types).

    Here's a list of the files created and what they provide:
    Mineshaft.dat - Stores the data for Mineshafts, not sure what the special feature is for Mineshaft.dat
    Fortress.dat - Stores the data for Nether Fortresses, determines where Wither Skeletons and Blazes can spawn
    Stronghold.dat - Stores the data for Strongholds, determines the location of Strongholds for Ender Eye tracking
    Temple.dat - Stores the data for Jungle Temples, Desert Temples and Witch Huts, determines where Witches can spawn
    Village.dat - Stores the data for Villages, not sure what the special feature is for Village.dat

    FAQs
    What is the actual problem with structures?
    As new features are added to Minecraft, sometimes the world generation logic needs to change along with it. Since Minecraft contains several 'structure specific' features, when the world generation logic changes, it could result in older structures no longer being valid.

    Minecraft 1.7 is one of those updates where the world generation logic has changed, meaning that worlds generated in 1.6 might have structures that are no longer recognised after updating to 1.7. In order to address this problem, Minecraft 1.6.4 introduced the ability to save structures to disk so that Minecraft is able to check for stored structures before trying to dynamically determine where they are.

    What do you mean by structures?
    When we say structures we mean areas in the world that Minecraft recognises as being a part of a special structure. This includes nether fortresses, strongholds, mineshafts, temples, witch huts and villages.

    Does this affect player built buildings?
    No, this only affects special structures generated and recognised by Minecraft.

    What does StructureSaver actually do?
    Simply put:
    StructureSaver is a utility we've provided that makes saving structures easier by providing a streamlined process to trigger the built in structure saving in Minecraft 1.6.4 and above.

    In detail:
    StructureSaver goes through every region file in your world and runs the appropriate structure generation code (without loading any chunks). Once all structures have been generated in your world, the plugin fires off a save to prompt Minecraft to write all the structures to disk.

    Where are structures saved?
    Structures saved by Minecraft 1.6.4 and above are stored in the 'data' folder within your world's folder. The possible names for these new files are:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is my server using a lot of resources after running StructureSaver?
    We believe this might be caused by how structures are handled in Minecraft 1.6.4 and higher. Essentially, once a structure has been saved, it needs to be loaded into memory at all times in order to be accessible and saveable.

    How do I revert the changes StructureSaver made?
    As StructureSaver merely triggers the structure saving feature built into Minecraft 1.6.4 and above, you simply need to remove the following files from the 'data' folder within your world's folder:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is this plugin not on BukkitDev?
    StructureSaver is a utility that is meant to only need to be used once in the transition from Minecraft 1.6.4 to Minecraft 1.7. As such, we felt that it didn't really belong on BukkitDev since it is purely built for one purpose and a specific Minecraft version.

    Delay in getting 1.6.4-R1.0 out
    Lastly, I'd like to apologise for how long it took us to release a 1.6.4 Recommended Build. We were really hoping to promote a Recommended Build immediately after we released our 1.6.4 update in order to address this, but we've been experiencing technical difficulties as our infrastructure broke down.

    Thankfully, as you're all now aware, we've managed to get everything back up and running and everything is now back to normal.
     
    EDWIN808, AJR200, mg180c and 20 others like this.
  2. Offline

    deep15mine

    Having done the StructureSaver process successfully, my question is what happens with large schematics that have been saved through Worldedit when they are loaded and pasted into the world / map again. Do we just re-run StructureSaver on that world again or would there be some other process? Thanks...
     
  3. Offline

    Black Hole

    I tested the fix and found only one issue: The file idcounts.dat is read and written from the custom world data folder, so map id start over at 0 on custom worlds. This leads to overwriting of existing maps.

    I removed all the structure info files, and started structure saver. All files went into the right folder. At the next start, all structure info are read from the right folders. I also tested this by removing the region files of the test world. I noticed that the server is also starting faster (about 19 seconds versus 28 seconds before, still slower than the 8 seconds without structure files) and less RAM usage.
     
  4. Offline

    Mark_Pino

    Does this mean that updating to 1.7 also removes buildings that I made myself?
     
  5. Offline

    tim740

    No it Doesn't remove anything
     
  6. Offline

    OMEGA48

    Only the structures that are made in game. Not what players make.
     
  7. Offline

    lmaomissile

    So if I don't run this on my server, generated structures will literally disappear? Will they be replaced? That would actually be nice since our server started at beta 1.7 and strongholds have always been messed up...
     
  8. Offline

    OMEGA48

    They won't disappear. They will be unable to function correctly and the mobs won't spawn. IE Witches won't spawn at Witch Huts, Villagers won't spawn at villages,... etc.
     
  9. Offline

    EvilSeph Retired Staff


    We've just pushed a fix for this issue as well and would appreciate it if you could test it out and report back, thanks again for the report!
     
  10. Offline

    Black Hole

    The fix is working fine, new maps gets the maximum ID, maximum ID is increased. New maps are saved properly. Existing and new maps are updated with world changes and properly loaded after server restart. :)
     
  11. Offline

    sssunny

    Is this the 1.2 version? If true i would like to test it with Multiverse plugin an some different worlds. One of them is really big - it exists since 1.0 release.
    BTW: always appreciating your great and generous work!
     
  12. Offline

    EvilSeph Retired Staff

    We've just released StructureSaver v1.2. The first post has been updated with the new link.

    https://github.com/Bukkit/StructureSaver/releases/download/v1.2/structuresaver-1.2.jar

    Changes include:
    • Added ability to force a save which deletes existing structure data first (/save-structures force or /save-structures <world> force)
    • Added tab complete support.
    • Check if structure generation is enabled before attempting to save structures.
    It is highly recommended you update to the recently released CraftBukkit 1.6.4-R2.0 first.
     
  13. Offline

    Bobcat00

    If we only have the three normal worlds (overworld/nether/end), use 1.6.4-R1.0, and have not run StructureSaver, will the bugs BlackHole found affect us?

    EDIT: I guess I'm asking under what circumstances would these bugs affect us.
     
  14. Offline

    spydercanopus

    After using structure-saver plugin and restarting, the server runs out of memory. I have a 25GB world and 12GB RAM allocated to it, but it's not enough.

    So will 1.7 have to load ALL the structures into memory on each load?

    All I can think is that I'll need to create a huge page file unless this is a temporary issue.

    Deleting the structure-saver files fixes the memory error. Do I even need to keep the files it creates? I mean, once it runs, it should be ok right to delete the files it creates?
     
  15. Offline

    Zeta416

    I might of missed this so sorry if it's been solved. After running the structure saver, my server runs with very high RAM usage BUT when I get rid of just the mineshaft.dat files, it's back to normal, running smoothly. I have narrowed this down that on my server, the mineshaft.dat files are causing the high RAM. Anyone else having this issue? I know others are having high RAM usage.
     
    spydercanopus likes this.
  16. Offline

    TnT Retired Staff

    That's how Minecraft 1.6.4 and above works.

    At very least, backup the files it creates. Some will be more important than others.
     
  17. Offline

    spydercanopus


    If I don't have the .dat files it creates inside the world's data folder, will there be issues? I'll hang onto them but when the 5 .dat files are in that directory, my server uses SOOO much RAM.
     
  18. Offline

    LHammonds

    I have updated my server to CraftBukkit 1.6.4-R2.0 and installed StructureSaver-1.2

    I ran the command individually for each world and it worked for all of them except for one.

    Code:
    2013-10-31 09:37:17 [INFO] Saving structures for 'pvp'. See your console for details.
    2013-10-31 09:37:17 [INFO] [StructureSaver] Generating structures for 'pvp'...
    2013-10-31 09:37:17 [WARNING] Unexpected exception while parsing console command "save-structures pvp"
    org.bukkit.command.CommandException: Unhandled exception executing command 'save-structures' in plugin StructureSaver v1.2
        at org.bukkit.command.PluginCommand.execute(PluginCommand.java:46)
        at org.bukkit.command.SimpleCommandMap.dispatch(SimpleCommandMap.java:192)
        at org.bukkit.craftbukkit.v1_6_R3.CraftServer.dispatchCommand(CraftServer.java:528)
        at org.bukkit.craftbukkit.v1_6_R3.CraftServer.dispatchServerCommand(CraftServer.java:515)
        at net.minecraft.server.v1_6_R3.DedicatedServer.as(DedicatedServer.java:263)
        at net.minecraft.server.v1_6_R3.DedicatedServer.t(DedicatedServer.java:228)
        at net.minecraft.server.v1_6_R3.MinecraftServer.s(MinecraftServer.java:488)
        at net.minecraft.server.v1_6_R3.MinecraftServer.run(MinecraftServer.java:421)
        at net.minecraft.server.v1_6_R3.ThreadServerApplication.run(SourceFile:583)
    Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
        at java.util.ArrayList.rangeCheck(ArrayList.java:635)
        at java.util.ArrayList.set(ArrayList.java:426)
        at net.minecraft.server.v1_6_R3.RegionFile.<init>(RegionFile.java:66)
        at com.evilmidget38.structuresaver.StructureSaver.saveStructures(StructureSaver.java:77)
        at com.evilmidget38.structuresaver.StructureSaver.onCommand(StructureSaver.java:146)
        at org.bukkit.command.PluginCommand.execute(PluginCommand.java:44)
        ... 8 more
    
    I looked in the "pvp/data" folder and it does not show any files. So the delete worked but not the recreation of any structure files. It is similar to my "nopvp" world in size and use so I expected to see Mineshaft.dat, Stronghold.dat, Temple.dat, Village.date and Villages.dat.

    Not sure what might be the problem. I do not have any users connected and the server has 6GB available to it. The "pvp" world size is 215MB so I don't think it is a size or lack or RAM issue.

    I will jump into the game and visit some structure locations to see if the automated save works.

    EDIT: I found a village but no files were generated under "pvp/data"

    Also, I tried the "force" option as well as letting it try to do all worlds at once, nothing has worked for the "pvp" world. The Nether and End dimensions for "pvp" world did work though.

    Environment
    • Server: Ubuntu Server 12.04.3 LTS, 64-bit
    • Java: 1.7.0_45, 64-bit
    • RAM: 16 GB (6 GB allocated to this particular server)
    • Minecraft: CraftBukkit 1.6.4-R2.0
    LHammonds
     
  19. Offline

    TnT Retired Staff

    Read the first post again, this is all covered by the FAQ.
     
  20. Offline

    spydercanopus


    Yes, it's the mineshafts.dat file that's eating up my ram. The file was over 129MB when the others are in the KB range.
     
  21. Offline

    bigbend

    I just updated to 1.6.4-R2.0 and ran StructureSaver 1.2 with /save-structures force. After the process is completed, I only have villages.dat in my main world folder, nothing else. The data folders for world_nether and world_the_end are completely empty! I also have a Multiverse world, with only villages.dat shows in that world's folder too. What went wrong?

    (By the way, Mineshafts special feature is cave spider spawning, and villages spawn villagers ;))

    Thanks for your help as always!
     
  22. Offline

    SnipsRevival

    Don't the mob spawners spawn the cave spiders rather than the mineshaft itself?
     
  23. Offline

    Dementedz

    EvilSeph
    So there is no need for a complete world/nether/end reset due to this update if we follow these guidelines correctly?
     
  24. Offline

    Puggles

    I want to make a new world for my server once bukkit for 1.7 comes out. Will this problem be fixed by then, if i make a new world???
     
  25. Offline

    Bobcat00

    But just because Minecraft does it, Bukkit doesn't have to. I mean if Minecraft jumped off a bridge, would Bukkit follow it?
     
  26. Offline

    feildmaster Retired Staff

    If you only have the normal worlds, the map bug will not effect you.
     
  27. Offline

    Vern8k

    Bukkit would simply create a "jumping off a bridge" API so that plugin authors could hook to it.
     
  28. Offline

    Bubbbaaa

    I updated from 1.6.4, backed up files. I ran world_nether and world_the_end with no problems but "world" stops as soon as it starts and server crashes with java error and nothing to show in console. I restart and still the same. Wish it showed an error in console...
     
  29. Offline

    evilmidget38 Retired Staff

    Bubbbaaa What is indicating a crash if there's nothing in the console?
     
    lukegb likes this.
  30. Offline

    Aquana

    Will craftbukkit for 1.7 contain a fix for that issue by replacing the way vanilla handles these files?
     
  31. Offline

    Bubbbaaa

    A little background:
    Something in WOLD is corrupt somewhere. I have been having this issue for while, about 4 weeks. The server would crash, everyone would get knocked off with Java error on the in-game desktop I guess you call it. Some Java errors would show in console and sometimes would not. I would have to restart the server if it did not restart on it's own. I would send in support ticket to my hosting company. One time errors showed exact coords where the problem was. I regenerated that area and all seemed well for a week straight with out any crashes. Thought it was fixed.
    Now:
    When I run the /save-structures force or /save-structures world you see console message: deleting the data files for world (0) and wont go any further. then seconds after that, the server crashes with Java error on in game desktop again. Everyone gets booted off, have to force the server to stop, then restart it again. I was able to /save-structures world_nether and world_the_end with no problems or errors. After getting the server to restart, console might show a line or 2 with java errors but not show where.

    I think I'm forced to start from scratch when bukkit comes out with 1.7.2... It's strange too because my server is only 4 months old and experiencing problems like this (java crashes and corrupt spots in world). It's also kind of sad because all that was built we have to start all over again.
     
Thread Status:
Not open for further replies.

Share This Page