PSA: Minecraft 1.7 update; possibly losing structures

Discussion in 'Community News and Announcements' started by EvilSeph, Oct 23, 2013.

Thread Status:
Not open for further replies.
  1. Offline

    EvilSeph Retired Staff

    Minecraft 1.7 introduces changes to the world generation code that could very likely result in structures (like strongholds and nether fortresses) generated in 1.6 no longer being recognised as valid. As this would mean that special behaviour for structures would stop working as expected (like wither skeletons spawning in nether fortresses, for example), we felt that it was important to give our Server Admins a heads up.

    Minecraft 1.6.4 introduces a new structure saving feature that should ensure that structures generated before 1.7 continue to be recognised as valid structures after updating. If you don't want to lose your structures from 1.6, you need to update to 1.6.4 first and make sure the structures are saved.

    What needs to be done
    If you're planning to continue running your 1.6 world come 1.7 and want to ensure that structures retain their special behaviour as expected, there is an extra step you need to take as part of the update process. Unfortunately, we've received reports that taking this extra step results in a noticeable performance hit so you'll have to decide for yourself what the best action is to take on your server. We've provided some extra useful information to help you make this decision in the "Things to note" section below.

    Before updating your server from 1.6 to 1.7, you should ensure you update to 1.6.4-R2.0 first and then visit all the structures in the worlds you wish to save. You can do this manually, but it is recommended you use our utility plugin StructureSaver (see below) to streamline this process and make things easier.

    Manually saving:
    Recommended - automated saving:
    • Download and update to CraftBukkit 1.6.4-R2.0
    • Download and install our StructureSaver utility plugin
    • Run the /save-structures [world] command (see below for detailed information)
    • Remove StructureSaver and restart your server
    First you'll need to download and update to CraftBukkit 1.6.4-R2.0

    Then you'll need to download and install StructureSaver from our GitHub here:
    https://github.com/Bukkit/StructureSaver/releases/download/v1.2/structuresaver-1.2.jar

    Once it is enabled, it supports the following commands:
    /save-structures <worldname> - saves structures in a specific world only
    /save-structures <worldname> force - saves structures in a specific world only, after removing existing structure data
    /save-structures - saves all structures in every world
    /save-structures force - saves all structures in every world after removing existing structure data

    And has the following permission:
    structuresaver.save

    Neglecting to address this ahead of time will result in structures from old worlds no longer being recognised as valid structures come 1.7.

    Things to note:
    Performance issues
    Since releasing the first version of StructureSaver, we've been receiving consistent reports that there is a noticeable decrease in server performance after StructureSaver has saved the structures on the server. To the best of our knowledge, we're certain that this issue isn't caused by StructureSaver itself as the issue continues after the plugin has been removed.

    Note that this decrease in performance is different from the immediate hit you see right after running StructureSaver (which is why we recommend you remove StructureSaver after you're done using it and restart your server).

    Run it once, keep a backup
    The nice thing about StructureSaver is that as long as you have a back up of your world, you're able to run a 1.6.4-R2.0 server with StructureSaver on it and save the structures whenever you want.

    You also have the option of running it now and backing up the resulting structure files in the 'data' folder inside your world's folder instead. Should you then decide later that you need the structures you saved from 1.6.4-R2.0, you can simply replace the already existing structure data as you please.

    Pick and choose
    Once you've run StructureSaver, it is completely up to you what you do with the saved structures. If you feel like saving Mineshafts is not necessary for your server, you can choose to delete the Mineshaft.dat file. Note, however, that it will be recreated by Minecraft whenever a Mineshaft is rediscovered (this applies to all structure types).

    Here's a list of the files created and what they provide:
    Mineshaft.dat - Stores the data for Mineshafts, not sure what the special feature is for Mineshaft.dat
    Fortress.dat - Stores the data for Nether Fortresses, determines where Wither Skeletons and Blazes can spawn
    Stronghold.dat - Stores the data for Strongholds, determines the location of Strongholds for Ender Eye tracking
    Temple.dat - Stores the data for Jungle Temples, Desert Temples and Witch Huts, determines where Witches can spawn
    Village.dat - Stores the data for Villages, not sure what the special feature is for Village.dat

    FAQs
    What is the actual problem with structures?
    As new features are added to Minecraft, sometimes the world generation logic needs to change along with it. Since Minecraft contains several 'structure specific' features, when the world generation logic changes, it could result in older structures no longer being valid.

    Minecraft 1.7 is one of those updates where the world generation logic has changed, meaning that worlds generated in 1.6 might have structures that are no longer recognised after updating to 1.7. In order to address this problem, Minecraft 1.6.4 introduced the ability to save structures to disk so that Minecraft is able to check for stored structures before trying to dynamically determine where they are.

    What do you mean by structures?
    When we say structures we mean areas in the world that Minecraft recognises as being a part of a special structure. This includes nether fortresses, strongholds, mineshafts, temples, witch huts and villages.

    Does this affect player built buildings?
    No, this only affects special structures generated and recognised by Minecraft.

    What does StructureSaver actually do?
    Simply put:
    StructureSaver is a utility we've provided that makes saving structures easier by providing a streamlined process to trigger the built in structure saving in Minecraft 1.6.4 and above.

    In detail:
    StructureSaver goes through every region file in your world and runs the appropriate structure generation code (without loading any chunks). Once all structures have been generated in your world, the plugin fires off a save to prompt Minecraft to write all the structures to disk.

    Where are structures saved?
    Structures saved by Minecraft 1.6.4 and above are stored in the 'data' folder within your world's folder. The possible names for these new files are:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is my server using a lot of resources after running StructureSaver?
    We believe this might be caused by how structures are handled in Minecraft 1.6.4 and higher. Essentially, once a structure has been saved, it needs to be loaded into memory at all times in order to be accessible and saveable.

    How do I revert the changes StructureSaver made?
    As StructureSaver merely triggers the structure saving feature built into Minecraft 1.6.4 and above, you simply need to remove the following files from the 'data' folder within your world's folder:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is this plugin not on BukkitDev?
    StructureSaver is a utility that is meant to only need to be used once in the transition from Minecraft 1.6.4 to Minecraft 1.7. As such, we felt that it didn't really belong on BukkitDev since it is purely built for one purpose and a specific Minecraft version.

    Delay in getting 1.6.4-R1.0 out
    Lastly, I'd like to apologise for how long it took us to release a 1.6.4 Recommended Build. We were really hoping to promote a Recommended Build immediately after we released our 1.6.4 update in order to address this, but we've been experiencing technical difficulties as our infrastructure broke down.

    Thankfully, as you're all now aware, we've managed to get everything back up and running and everything is now back to normal.
     
    EDWIN808, AJR200, mg180c and 20 others like this.
  2. Offline

    MistaHardkill


    Yes, must be the performance, even though I did it locally, allocated 8 of my 32GB of RAM to it and ran it on a 6 of my 12 threads on my CPU. It was fine, using less than 500mb on idle. After structure saver? 3GB on idle, if it even starts at all, with the same (updated) plugins and craftbukkit jar that was being used before the structuresaver plugin was run. It is structuresaver. I've covered my bases, and by process of elimination, it is, without a doubt, structuresaver.

    Sorry to burst your bubble, man.
     
  3. Offline

    Sketaful

    Sounds like it keep all the shunks loaded even after it's done. Having a big map and every chunk on the map loaded at the same time would break most servers...
     
  4. Offline

    TnT Retired Staff

    I addressed this situation below. It is the performance in 1.6.4, not the plugin. This plugin just brings the performance issues to light.
     
  5. Offline

    caelum19

    I was wonderrrring what StructureSaver was when I visited the github
     
  6. Offline

    MistaHardkill

    If it were the performance of 1.6.4, than the server would run just as bad from a map generated BY 1.6.4. It doesn't. So, that makes no sense, at least to me.
     
  7. Offline

    Minerva

    Someone correct me if I'm wrong.
    From my understanding, 1.6.4+ stores structure data in separate files. The SS plugin causes the server to load all generated chunks and thus to write these files. The problem seems to come into play when 1.6.4 tries to read these files while loading the world.
    I assume that SS doesn't actually do the writing?
     
  8. Offline

    robotnikthingy

    Exactly. And the biggest one of those files is mineshaft.dat

    I deleted that one in my main world and it worked fine
     
  9. Offline

    Ultimate_n00b

    Not at all. You shouldn't need to even run it, as you don't have any structures to save. Your generator (at least how most plugins should do skyblock worlds) doesn't generate any of these structures.
     
  10. Offline

    FuRiouSOne

    This worked fine on my small prison server but when i tried on my bigger PVE server (world is 8000x8000) it just crashes with http://pastebin.com/4J44PJ5V
     
  11. Offline

    Dacheetz

    I find it hilarious how users can diagnose code and tell the developers that they are incorrect, when the developers are familiar with 100% of the code, and users 0%.
    I feel that is one good way to seriously embarrass yourself.
    If TnT says its not a StructureSaver issue but rather a 1.6.4, well that's how it is!
    Myself and all my users agree that it will be more fun to start fresh with 1.7.2, so I'm unaffected, but enjoy reading these comments. Well done, Bukkit!
    Looking forward to craftbukkit_1_7_2.jar :D
     
    1mpre55 and markdf like this.
  12. Offline

    crazyeye443

    I just want to say great job Bukkit. You do a great job of updating and keeping up with Mojang's updates. Your server software makes for a really enjoyable time.
     
  13. Offline

    Bobcat00

    I think we mainly want to understand what the (possible) issues are, so each of us can make an educated decision on how to proceed with our servers. Knowledge is power!
     
  14. Offline

    TnT Retired Staff

    You will notice the same problems on a 1 GB world generated with 1.6.4 or a 1 GB world with the same amount of structures generated on a version prior to 1.6.4 and you run StructureSaver on that world to save the structures. The only thing StructureSaver does is save the structures.
     
  15. Offline

    spookyDHD

    i got it 2
     
  16. Offline

    MrSuperWerer

    Anyone got this issue as well??? :confused:
     
  17. Offline

    evilmidget38 Retired Staff

    MrSuperWerer That "issue" is because StructureSaver holds up the entire server for several seconds, so Minecraft freaks out when it's done.
     
    lukegb likes this.
  18. Offline

    adamwbb

    yeah i notice.
    it seems that 1.6.4 has a memory leak
     
  19. Offline

    MrSuperWerer

    Ok thank you!

    Again, keep up the good job bukkit, lots of thanks.
     
  20. Offline

    EvilSeph Retired Staff

    I've updated the first post with more information. Hopefully this clarifies everything and covers what we know so far. It was a bit tricky to format it in a way that makes it easy to read though...
     
    swimmer1929 likes this.
  21. Offline

    theearlypc423

    looks good :D
     
  22. Offline

    burnzz

    thanks for including the FAQ. i found it easy to read, and follow along.
    i've also found deleting mineshaft.dat (previously posted) really decreased the usage hit.
     
  23. Offline

    ShootToMaim

    <3 evilseff :3
     
  24. Offline

    Black Hole

    Bukkit doesn't handle secondary world servers properly. This means that structure data for custom worlds besides the primary overworld, nether or end are not saved separately. Instead all the data goes into one directory, messing up the structures of the primary worlds and leaving the secondary worlds unprepared for the 1.7 update.
     
  25. Offline

    evilmidget38 Retired Staff

    This would actually be incorrect. If you see https://github.com/Bukkit/CraftBukk.../org/bukkit/craftbukkit/CraftServer.java#L727 all custom worlds are a WorldServer, and not a SecondaryWorldServer. The only SecondaryWorldServers are the Nether and the End, which have no conflicting structures with the main world, and saving their structure data with the main world's structure data actually mirrors Vanilla behavior.

    EDIT: Testing this, and getting weird results.... You may be on to something.

    EDIT2: Now I look silly... https://github.com/Bukkit/CraftBukk.../org/bukkit/craftbukkit/CraftServer.java#L733
     
    lukegb, markdf and Goblom like this.
  26. Offline

    BearFather

    I just did this process and now it seems I have a horrible lag on my server. Chat, console commands, everything seems to take a second to do. So here's my question. If I remove the files it made, will this help, or will I loose structures till 1.7?

    EDIT: Hmm was noticing none of my query stuff wasn't working on my server, so went and look and my server.properties file was reset to default. I copied over a backup of it, restarted my server and all the lag went away. Start up time was back to normal again. Not sure if this is the actual issue, or if it was coincidence.
     
  27. Offline

    McGoerk

    BearFather: You should be able to safely remove the Mineshaft.dat and Village.dat without loosing important information since no special mobs spawn in those areas.
    You could make a backup of those, if they ever introduce mobs for thos areas.
    If you remove the other files too, none of the area specific mobs will spawn in those places.

    I have decided that for my server ths isn't an issue, since we will make an extra world with 1.7 and if necessary we can set up spawners in the old worlds structures. "Converting" the old worlds takes too big of a toll for our server and only a little benefit.

    But anyway thanks again Bukkit-Team for the clarification and the opportunity to keep our world intact, it's too bad that the 1.6 and 1.7 world handling is so crappy. Why saving all structures in one big file and keep that in memory all the time, that is just silly.

    I can only hope our server will be able to handle the new map when our players explore it.
     
  28. Offline

    Bobcat00

    Ummm.... Villagers and iron golems spawn in villages. Without villages.dat I would assume they won't spawn and/or won't know the limits of the village and will wander away.
     
  29. Offline

    Black Hole

    evilmidget38: Sorry, I mixed up the terms for world servers. I'll try to explain the issue with more details:

    Bukkit doesn't handle structure info for custom worlds properly. This means that structure data for custom worlds besides the main overworld, nether or end are not saved separately. Instead all the data goes into the data directory of the main overworld, messing up the structures of the main worlds and leaving the custom worlds unprepared for the 1.7 update.

    While it it is right to save the player generated maps there in one place, it's not right to save the structure info for all worlds in the same files. The structure info is saved to the files without a world name or uuid.

    The problem gets even worse after running structure saver on all worlds and then creating a new world. This new world will be created with all the structures of the existing worlds mixed together.

    I created a bug report at leaky with some pictures: https://bukkit.atlassian.net/browse/BUKKIT-4871
     
  30. Offline

    robotnikthingy

    Oh, well thats a big problem. I would rather delay 1.7.2 bukkit to get that problem fixed first
     
  31. Offline

    EvilSeph Retired Staff

    Black Hole

    Thanks for the bug report, we've fixed this bug and will be promoting a 1.6.4-R2.0 soon. We'd appreciate it if you could give the fix a quick test and report back :)
     
Thread Status:
Not open for further replies.

Share This Page