PSA: Minecraft 1.7 update; possibly losing structures

Discussion in 'Community News and Announcements' started by EvilSeph, Oct 23, 2013.

Thread Status:
Not open for further replies.
  1. Offline

    EvilSeph

    Minecraft 1.7 introduces changes to the world generation code that could very likely result in structures (like strongholds and nether fortresses) generated in 1.6 no longer being recognised as valid. As this would mean that special behaviour for structures would stop working as expected (like wither skeletons spawning in nether fortresses, for example), we felt that it was important to give our Server Admins a heads up.

    Minecraft 1.6.4 introduces a new structure saving feature that should ensure that structures generated before 1.7 continue to be recognised as valid structures after updating. If you don't want to lose your structures from 1.6, you need to update to 1.6.4 first and make sure the structures are saved.

    What needs to be done
    If you're planning to continue running your 1.6 world come 1.7 and want to ensure that structures retain their special behaviour as expected, there is an extra step you need to take as part of the update process. Unfortunately, we've received reports that taking this extra step results in a noticeable performance hit so you'll have to decide for yourself what the best action is to take on your server. We've provided some extra useful information to help you make this decision in the "Things to note" section below.

    Before updating your server from 1.6 to 1.7, you should ensure you update to 1.6.4-R2.0 first and then visit all the structures in the worlds you wish to save. You can do this manually, but it is recommended you use our utility plugin StructureSaver (see below) to streamline this process and make things easier.

    Manually saving:
    Recommended - automated saving:
    • Download and update to CraftBukkit 1.6.4-R2.0
    • Download and install our StructureSaver utility plugin
    • Run the /save-structures [world] command (see below for detailed information)
    • Remove StructureSaver and restart your server
    First you'll need to download and update to CraftBukkit 1.6.4-R2.0

    Then you'll need to download and install StructureSaver from our GitHub here:
    https://github.com/Bukkit/StructureSaver/releases/download/v1.2/structuresaver-1.2.jar

    Once it is enabled, it supports the following commands:
    /save-structures <worldname> - saves structures in a specific world only
    /save-structures <worldname> force - saves structures in a specific world only, after removing existing structure data
    /save-structures - saves all structures in every world
    /save-structures force - saves all structures in every world after removing existing structure data

    And has the following permission:
    structuresaver.save

    Neglecting to address this ahead of time will result in structures from old worlds no longer being recognised as valid structures come 1.7.

    Things to note:
    Performance issues
    Since releasing the first version of StructureSaver, we've been receiving consistent reports that there is a noticeable decrease in server performance after StructureSaver has saved the structures on the server. To the best of our knowledge, we're certain that this issue isn't caused by StructureSaver itself as the issue continues after the plugin has been removed.

    Note that this decrease in performance is different from the immediate hit you see right after running StructureSaver (which is why we recommend you remove StructureSaver after you're done using it and restart your server).

    Run it once, keep a backup
    The nice thing about StructureSaver is that as long as you have a back up of your world, you're able to run a 1.6.4-R2.0 server with StructureSaver on it and save the structures whenever you want.

    You also have the option of running it now and backing up the resulting structure files in the 'data' folder inside your world's folder instead. Should you then decide later that you need the structures you saved from 1.6.4-R2.0, you can simply replace the already existing structure data as you please.

    Pick and choose
    Once you've run StructureSaver, it is completely up to you what you do with the saved structures. If you feel like saving Mineshafts is not necessary for your server, you can choose to delete the Mineshaft.dat file. Note, however, that it will be recreated by Minecraft whenever a Mineshaft is rediscovered (this applies to all structure types).

    Here's a list of the files created and what they provide:
    Mineshaft.dat - Stores the data for Mineshafts, not sure what the special feature is for Mineshaft.dat
    Fortress.dat - Stores the data for Nether Fortresses, determines where Wither Skeletons and Blazes can spawn
    Stronghold.dat - Stores the data for Strongholds, determines the location of Strongholds for Ender Eye tracking
    Temple.dat - Stores the data for Jungle Temples, Desert Temples and Witch Huts, determines where Witches can spawn
    Village.dat - Stores the data for Villages, not sure what the special feature is for Village.dat

    FAQs
    What is the actual problem with structures?
    As new features are added to Minecraft, sometimes the world generation logic needs to change along with it. Since Minecraft contains several 'structure specific' features, when the world generation logic changes, it could result in older structures no longer being valid.

    Minecraft 1.7 is one of those updates where the world generation logic has changed, meaning that worlds generated in 1.6 might have structures that are no longer recognised after updating to 1.7. In order to address this problem, Minecraft 1.6.4 introduced the ability to save structures to disk so that Minecraft is able to check for stored structures before trying to dynamically determine where they are.

    What do you mean by structures?
    When we say structures we mean areas in the world that Minecraft recognises as being a part of a special structure. This includes nether fortresses, strongholds, mineshafts, temples, witch huts and villages.

    Does this affect player built buildings?
    No, this only affects special structures generated and recognised by Minecraft.

    What does StructureSaver actually do?
    Simply put:
    StructureSaver is a utility we've provided that makes saving structures easier by providing a streamlined process to trigger the built in structure saving in Minecraft 1.6.4 and above.

    In detail:
    StructureSaver goes through every region file in your world and runs the appropriate structure generation code (without loading any chunks). Once all structures have been generated in your world, the plugin fires off a save to prompt Minecraft to write all the structures to disk.

    Where are structures saved?
    Structures saved by Minecraft 1.6.4 and above are stored in the 'data' folder within your world's folder. The possible names for these new files are:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is my server using a lot of resources after running StructureSaver?
    We believe this might be caused by how structures are handled in Minecraft 1.6.4 and higher. Essentially, once a structure has been saved, it needs to be loaded into memory at all times in order to be accessible and saveable.

    How do I revert the changes StructureSaver made?
    As StructureSaver merely triggers the structure saving feature built into Minecraft 1.6.4 and above, you simply need to remove the following files from the 'data' folder within your world's folder:
    Mineshaft.dat
    Fortress.dat
    Stronghold.dat
    Temple.dat
    Village.dat

    Why is this plugin not on BukkitDev?
    StructureSaver is a utility that is meant to only need to be used once in the transition from Minecraft 1.6.4 to Minecraft 1.7. As such, we felt that it didn't really belong on BukkitDev since it is purely built for one purpose and a specific Minecraft version.

    Delay in getting 1.6.4-R1.0 out
    Lastly, I'd like to apologise for how long it took us to release a 1.6.4 Recommended Build. We were really hoping to promote a Recommended Build immediately after we released our 1.6.4 update in order to address this, but we've been experiencing technical difficulties as our infrastructure broke down.

    Thankfully, as you're all now aware, we've managed to get everything back up and running and everything is now back to normal.
     
    EDWIN808, AJR200, mg180c and 20 others like this.
  2. Offline

    drtshock

  3. Offline

    ThunderGemios10

    Thanks for the Info! i will start doing these for my server
     
  4. Offline

    SnipsRevival

    Is anyone else getting extremely high RAM and/or CPU usage after updating to 1.6.4? I installed both the latest craftbukkit plus structuresaver. Then I ran the command /save-structures. During this time both the CPU and RAM usage was maxed. I assumed this was normal, so I waited until I was pretty sure it was done. Then I stopped the server, uninstalled structuresaver and restarted the server. However, the server is now consistently taking almost 3 minutes to fully start and RAM usage is now constantly at 100%. CPU usage also spikes to 100% somewhat frequently.
     
    BlueJayofEvil and tngamer14 like this.
  5. Offline

    andymob

    this is freaking awesome!!! thanks for that!
     
  6. Offline

    Dr_Bunsen

    That is great!

    But my only question is, are you guys already looking into 1.7? Since the server has already been released by mojang.
    Since they say they changed the networking, is that going to be an issue?

    But Whatever you guys do, you guys rock!
     
  7. Offline

    qPureEvilp

    Oh an great work on the save structures (assuming it works, have not tested that part yet) but it went way fast.... I was thinking it was going to be something that took awhile. It took about 10 seconds lol.
     
  8. Offline

    thapengwin

    Good job guys!
     
  9. Offline

    Mavale

    Hi, this is a great work.

    I tested the plugin on my server console but I got many messages like that :
    14:40:40 [INFOS] [StructureSaver] Saving all structures...
    14:40:40 [INFOS] [StructureSaver] Generating structures for Sdragorn...
    14:40:40 [GRAVE] [StructureSaver] Unable to handle region: r.-1.-1.mcr
    ...

    Do you know why ?
     
  10. Offline

    TnT

    Great work team.
     
    joehot2000 and XD 3VIL M0NKEY like this.
  11. I'm getting loads and loads of "Unable to handle region" errors as well. Should I worry? (I'm running on dev versions of 1.6.x for a long time now).
     
  12. And tell me results by replying to this comment, thank you very much!
     
  13. Offline

    Doctacosa

    All .mcr files are outdated and unused as of Minecraft 1.2, and were replaced by the .mca files. You can go into your server files and safely delete all those ending in .mcr for all worlds.
     
    spydercanopus likes this.
  14. All my errors are *.MCR related. So if what Doctacosa said is true, it's not really an issue.
    Are you definitely sure, that I can delete all mcr files? I'm running my server for ages now, and if anything will get lost, my users will never, ever, EVER forgive me.

    Code:
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.-1.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.-2.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.-3.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.-4.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.0.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.1.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.2.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-1.3.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-2.-1.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-2.-2.mcr
    2013-10-24 16:02:32 [SEVERE] [StructureSaver] Unable to handle region: r.-2.-3.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-2.0.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-2.1.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-2.2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-2.3.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.-1.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.-2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.0.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.1.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-3.3.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-4.-1.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-4.-2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-4.0.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-4.2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-4.3.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-5.-1.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.-5.-2.mcr
    2013-10-24 16:02:33 [SEVERE] [StructureSaver] Unable to handle region: r.0.-1.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.-2.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.-3.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.-4.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.0.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.1.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.2.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.0.3.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.-1.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.-2.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.-3.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.-4.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.0.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.1.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.2.mcr
    2013-10-24 16:02:34 [SEVERE] [StructureSaver] Unable to handle region: r.1.3.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.-1.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.-2.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.-3.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.-4.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.0.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.1.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.2.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.2.3.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.-1.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.-2.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.0.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.1.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.2.mcr
    2013-10-24 16:02:35 [SEVERE] [StructureSaver] Unable to handle region: r.3.3.mcr
     
  15. Offline

    TnT


    Backup your world. Delete the .mcr files. Try again.

    You should have no problem deleting them, but backups can always be a very handy thing to have.
     

  16. I did that, and the StructureSaver worked flawlessly (or at least I hope so). I'll tell my users to check all their buildings for any strange changes.
     
  17. Offline

    Mavale

    Very well : after deleting all mcr files there is no error yet.

    Thanks a lot :) . A quick visit in the world show no change (actually ^^).
     
  18. Offline

    Maximvdw

    I am having trouble using the StructureSaver plugin,
    I have the latest build of Craftbukkit.
    But I keep getting "Too many open files" on my 90000x90000 map.
    My max file handler limit it 40k...

    Any help :S

    EDIT: I do not have any mcr files (1.6.2 generated map)

    - Maximvdw
     
  19. Offline

    LHammonds

    Thanks for the info and the structure-saver plugin. I was able to update all my old worlds (and deleted the unused .mcr files).

    Maximvdw, are you trying to do this on a Windows, Linux or Mac machine? If using Windows, see if it works on a Linux machine. You could even create a virtual Linux server on your PC to test this.
     
  20. Offline

    Maximvdw

    Thanks for replying LHammonds,

    EDIT: changed my ulimit -n to 100k
    Still the same problem (just takes longer before the error starts)
    I am using this on an Ubuntu Linux 10.04.3

    Could this be a problem in the plugin that the files are not gettings closed or something :S?
    just thinking of solutions xD...

    The javadocs: http://didz.co.uk/forgedocs/net/minecraft/src/RegionFile.html
    say there is a .close() for regionfile.. However I don't see it being used?


    EDIT 2: SOLUTION:
    I have added a close to the RandomAccessFile
    of the RegionFile.

    I have also added a little debug line that prints the region file name. So I can kinda see
    how long it will take

    This caused my error to stop (recompiled)
    Code:
        public void saveStructures(World world) {
            getLogger().info("Generating structures for "+world.getName()+"...");
            long start = System.currentTimeMillis();
            File regionDir = getRegionsLocation(world);
            if (regionDir == null) {
                getLogger().severe("Unable to locate the region files for: "+world.getName());
                return;
            }
            // Extract the x and z coordinates from the region files.
            Pattern coordinatePattern = Pattern.compile("r\\.(.+)\\.(.+)\\.mca");
            for (File file : regionDir.listFiles()) {
          getLogger().info(file.getName());
                Matcher matcher = coordinatePattern.matcher(file.getName());
                int regionX = 0;
                int regionZ = 0;
                if (matcher.find()) {
                    regionX = Integer.parseInt(matcher.group(1));
                    regionZ = Integer.parseInt(matcher.group(2));
                } else {
                    getLogger().severe("Unable to handle region: "+file.getName());
                    continue;
                }
                regionX = regionX << 5;
                regionZ = regionZ << 5;
                RegionFile region = new RegionFile(file);
                IChunkProvider chunkProvider = ((ChunkProviderServer) ((CraftWorld) world).getHandle().chunkProvider).chunkProvider;
                // Iterate over all potential chunks in the region.
                for (int chunkX = 0; chunkX < 32; chunkX++) {
                    for (int chunkZ = 0; chunkZ < 32; chunkZ++) {
                        // Check if the chunk exists.
                        if (region.c(chunkX, chunkZ)) {
                            // Create structures for that chunk!
                            chunkProvider.recreateStructures(regionX+chunkX, regionZ+chunkZ);
                        }
                    }
                }
                try {
    region.c();
    } catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
    }
         
            }
    

    Same version as 1.0 just with a 'region.c()' added so it closes the region file after reading it
    This will help servers with RAM trouble and the Too many files error

    Gr,
    Maxim
     
    evilmidget38 likes this.
  21. Offline

    SnipsRevival


    I feel like I should add that I just downgraded my server back to 1.6.2, and everything is back to normal. However, if I try and then install 1.6.4 again, the same problems come back. So for now I am forced to run my server on 1.6.2.
     
  22. Offline

    Maximvdw

    SnipsRevival are you sure that it is not a plugin causing the lagg?
    Check if all of them are updated to 1.6.4 and compatible
     
  23. Offline

    ThaSourceGaming

    Thank you for the update on the update :p
     
  24. Offline

    sling

    I just coverted a copy of my 1.6.2 with no errors in console but now ram usage has increased and chunk loading in-game takes ages and chunks will not be unloaded correctly. Moving back and forth in a biome causes high ram usage and very poor chunk loading/unloading.
     
  25. Offline

    Niknea

    Sorry I'm a bit confused. If you we update will the entire map not save? Or just the generated builds like strongholds or monster spawners?
     
  26. Offline

    SnipsRevival

    Well copied over the exact same plugins that I use on my public server and placed them on the server I use for plugin development, and it worked fine, so I don't think this is a plugin problem.

    When I watch the console while the server is starting, I can see that the world generation is what is taking a large portion of the startup time. When I am running 1.6.2 it takes about 12 seconds to fully start the server, but when I am running 1.6.4, it takes almost 3 minutes.
     
  27. Offline

    Doctacosa

    The update is about saving internally the location of structures. For example, this will allow Wither Skeletons in Nether fortresses to spawn in the right locations, even after moving to 1.7. You will NOT LOSE any buildings or map elements if you skip this step, but you'll get strange side effects related to mob spawning and similar.
     
  28. Offline

    sling

    ok, my server is running fine now. poor chunk loading seems to be client related and optifine solved the problem. ram usage around 500mb with 1 player connected.
     
  29. Offline

    xize

    Awesome thanks for letting us know:)
    just a small question what would happen with the data values in 1.7 regarding the api?
    I had seen some tweets speculating that data values may get removed and gets replaced by enum kind of names but does that also include in the api?

    Thanks
     
  30. Offline

    evilmidget38

    Maximvdw Thanks for the PR you made to StructureSaver. I've pulled it, and we'll be updating the available build shortly.
     
    lukegb, Maximvdw and drtshock like this.
Thread Status:
Not open for further replies.

Share This Page