[Library] Edit or create NBT tags with a compact class - no OBC/NMS

As we all know, Minecraft is under continuous and extensive development, and it can be difficult for a project like Bukkit to keep up while still maintaining a high quality API. Admittedly, some progress has been made since the release of 1.6.2 in July, including the horse API, but it's still lacking fundamental features such as the ability to manipulate custom attributes.

I did publish a provisional attribute API with the intent of bridging the expected gap between the release and the official API, but I didn't anticipate quite how long it would actually take. So instead, I'm making it more permanent by introducing a new version-independent NBT library:
https://gist.github.com/aadnk/6753244

For 1.7.2, use this version.

To use it, just download the source-file and copy it into your project. Then you'll be able to create (or edit) a compound as simply as so.

Code:java

// Use NbtFactory.fromCompound(obj); to load from a NBTCompound class.
NbtCompound compound = NbtFactory.createCompound();
NbtCompound author = NbtFactory.createCompound();
 
compound.put("released", 2009);
compound.put("game", "Minecraft");
compound.put("author", author);
compound.put("bytes", new byte[] { 1, 2, 3 });
compound.put("integers", new int[] { 1, 2, 3});
compound.put("list", NbtFactory.createList(1, 2, 3));
 
// This is one way to populate inner compounds
author.put("name", "Markus Persson");
 
// Or you can use the "path" syntax
compound.putPath("fan.name", "Kristian Stangeland");
 
System.out.println(compound.getHandle().getClass());
System.out.println(compound.getInteger("released", 0));
System.out.println(compound.getString("game", ""));
System.out.println(compound.getList("list", false));
System.out.println(compound.getPath("author.name"));
System.out.println(compound.getPath("fan.name"));
 
// Output:
// class net.minecraft.server.v1_6_R2.NBTTagCompound
// 2009
// Minecraft
// [1, 2, 3]
// Markus Persson
// Kristian Stangeland

The internal net.minecraft.server.NBTCompound is represented as a Map<String, Object> (or NbtCompound). You can create inner compounds using createCompound(), or by letting the library do it for you automatically with putPath(). It's also possible to retrieve values with a relative path, similar to YamlConfiguration. Once you're done reading or modifying the compound (or list), you'll be able to get its underlying NBT tag by calling getHandle().

The point here is that you can modify existing NBTCompounds, like the tag compound in ItemStack. There's even a convenience method for retrieving or setting this tag in NbtFactory:

Code:java

ItemStack stack = NbtFactory.getCraftItemStack(new ItemStack(Material.GOLD_AXE));
NbtCompound other = NbtFactory.fromItemTag(stack);
 
// Do whatever
other.putPath("display.Name", "New display");
other.putPath("display.Lore", NbtFactory.createList("Line 1", "Line 2"));

People familiar with my work may have noticed some similarities with the NBT library in ProtocolLib - and yes, I borrowed heavily from ProtocolLib, especially the class names. But I believe this newer version is far easier to use (no more generics), and much more compact. The only downside is the lack of support for MCPC+, in which case I'd still recommend using ProtocolLib.

 

So I am a bit confused by what this does.. but is it basically storing data? Or can it modify all item data?

 

It's mainly for editing NBT data in memory, like the tag field on an ItemStack. That's where everything accessible with ItemMeta is stored (display name/lore, enchantments, armor color, book content), as well as attributes. You could also use it to "save" or "load" entities, like PortableHorses, but you'll have to use reflection to get its NBTTagCompound first.

Of course, I could add support for loading and saving NBT files, if people were interested. But it's always possible to use an external library in those cases, unlike when the NBT is stored in memory. So I would only consider that a bonus.

 

Comphenix this would work for entity attributes as well?

 

Comphenix Are we able to create persistent data with this?As example store data to an itemstack that will not get filtered out by craftbukkit and will be saved on reload/startup or is it all just in memory?

 

Sort off - you can use Entity.load(NBTTagCompound source) and Entity.save(NBTTagCompound dest) (called a and b respectively) to get the NBT data of an entity, modify it, and put it back in memory. But it might be easier to just modify its AttributeBaseMap.
Alternatively, just give the entity an item stack with the attributes you want to add.

You can, in fact. You just have to use the attribute system instead of custom NBT tags. Take a look at this thread.

 

Comphenix Well I was wondering because for my plugin, backpacks++, i need to set "invisible" data, at the moment Im just using the lore to set the owner, backpack id and backpack type but it looks ugly so Im looking for a way to hide the data...

 

Just try AttributeStorage for your self. It's nearly invisible (it will just add a small empty line to the item), and it doesn't eat up any bandwidth to the client. It also allows multiple plugins to store data on the same item, without causing any conflict.

The only problem is that it will only work for 1.6.2. But other than that, it's perfect.

 

Comphenix Thanks, I'll try it tomorrow! Also wouldnt another option be to use packets? If I capture the right packet and set the lore to null or wouldn't that work? Anyways this isn't a support forum so I will stop spamming it. Nice library and good job !

 

Sure, that will work too. That's what PortableHorses does, among other things (and ItemRenamer). I mention that possibility here. But it is more complicated, especially if you want to do it yourself without the help of ItemRenamer.

Thanks.

 

Comphenix Thanks! I take a look at all options and see which one fits my needs the best.

 

I've updated the library a little. It's now possible to save and load compounds from files and streams (with an optional compression flag). This is fairly easy to do (with Guava):

Code:java

compound.saveTo(Files.newOutputStreamSupplier(file), StreamOptions.GZIP_COMPRESSION);
 
// Load the compound
NbtCompound loaded = NbtFactory.fromStream(Files.newInputStreamSupplier(file), StreamOptions.GZIP_COMPRESSION);
System.out.println(loaded.getHandle());

 

Alright, so I want to be able to tag an item and detect it anywhere. Is it possible to do using this class?
I thought this would do it:
Tagging:

Code:java

AttributeStorage att = AttributeStorage.newTarget(this.item, UUID.randomUUID());
att.setData("Weapon:");

And then to get it:

Code:java

public static Weapon getWeapon(ItemStack item) {
   Attributes att = new Attributes(item);
   for (Attribute a : att.values()) {
      if (a.getName().contains("Weapon:")) {
         return w;
      }
   }
   return null;
}

Am I doing something wrong?

 

Why not simply copy the example code on this page? Unless otherwise stated, it's always tested to make sure it's working. No need to create your own version.

It's also a bit of a waste to generate a new UUID every time. Instead, just use it as a key to identify data belonging to your plugin, allowing other plugins to use attribute system on the same item. Generate a UUID here, and store it as a constant.

 

Huh, cool. Thanks for the help.
EDIT:

Actually, is there a way to do this WITHOUT ProtocolLib/NMS? While I do agree that ProtocolLib is the awesomest thing ever, I can't use it for this situation.

 

Yes ... that's the whole point of this thread. It's a NBT library that works without referencing CraftBukkit or ProtocolLib ...

Here's a version of the Attribute API that uses the library on this page.

 

Ah, that's what I was missing. Thank you.

 

Comphenix trying to use this with 1.7 player.dat files. Getting the following error when trying load it.
http://pastebin.com/ch0aH5gX

Code:java

NbtFactory.fromStream(Files.newInputStreamSupplier(file), StreamOptions.NO_COMPRESSION);

Any ideas?

 

Comphenix Y U Make me rewrite portablehorses now?!

attribute storage looks awesome, I'd love to figure out how to make portable horses use that instead, but doing it without breaking everyones saddles is going to be tricky :\

 

And here is a version that is compatible with 1.7.2, if anyone is interested.

 

I'm experimenting with NbtFactory and noticed that NbtCompound's getPath will throw an IllegalArgumentException if the root of the path does not exist.

For example, if "test" in "test.key" exists but "key" does not, then getPath will return null. However, if "test" does not exist then it will throw an IllegalArgumentException.

I'm curious as to the reasoning for this. Why not simply return null if the root does not exist? Or accept a second parameter"default" which returns an object in-case the path does not exist or resolves null?

As a side note, the error doesn't make a whole lot of sense either. It will say:

Code:

Caused by: java.lang.IllegalArgumentException: Cannot find test in [test]

Custom nbt paths are also lost when an item goes through an anvil or a hopper. Is there a way to preserve this information?

I can also store information in the "AttributeModifiers" nbt tag, which is preserved through hoppers, but is still deleted when going through an anvil.

 

Fair enough, it's a simple enough fix in any case. You can find the updated version here, as usual.

But really - you just try modifying the NBT class yourself, as this class will always be c0pied into your project, instead of accessed through a shared library.

CraftBukkit will always recreate the NBT tag compound after a call to setItemMeta(), without any custom NBT tag compounds. This is by design, unfortunately, so there's probably no point in submitting any bug reports.

However, I was not aware that anvils would clear attribute modifiers. That sounds like a bug - perhaps you should submit that?

 

Yes, I did modify the code just as I described. However, I'm very rusty with my Java and was curious as to whether there was an important design decision I might have overlooked as per your original implementation.

I was writing a listener class which copies the nbt data I want from slot 0 to slot 2 when an item goes through an anvil which fixed THAT issue. But now I'm finding that custom atrributes get stripped simply by hovering over it in the inventory.

It may have something to do with bukkit filtering out unknown attributes. Still investigating.

I'm considering putting the information I was to store in lore instead and using ProtocolLib top hide it from the client. Do you have a code example on how to do this?

So from what I now understand, bukkit does some kind of nbt cleansing when the user hovers over an item in their inventory. This affects custom nbt entries as well as unknown attributes.

It would appear as though the safest place to store custom item information is in the lore section and then hide it client-side with ProtocolLib. I would have preferred to avoid such a dependency.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

I do, but it depends on ItemRenamer in addition to ProtocolLib. Doing it with ProtocolLib alone requires a good deal of code - I suggest you start reading the source code of ItemRenamer.

But, I had some spare time just now, and I tried to reproduce your problem, without luck (using this AttributeStorage):

Code:java

private final UUID uuid = UUID.fromString("d9bc56b0-6a9f-11e3-981f-0800200c9a66");
 
@Override
public void onEnable() {
    getServer().getPluginManager().registerEvents(this, this);
}
 
@Override
public boolean onCommand(CommandSender sender, Command command, String label, String[] args) {
    if (sender instanceof Player) {
        ItemStack stack = new ItemStack(Material.BOOK);
        AttributeStorage storage = AttributeStorage.newTarget(stack, uuid);
 
        storage.setData("Test");
        ((Player) sender).getInventory().addItem(storage.getTarget());
    }
    return true;
}
 
@EventHandler
public void onEvent(InventoryClickEvent e) {
    checkItem(e.getCurrentItem());
    checkItem(e.getCursor());
}
 
private void checkItem(ItemStack stack) {
    if (stack == null || stack.getType() == Material.AIR)
        return;
    AttributeStorage storage = AttributeStorage.newTarget(stack, uuid);
 
    if (storage.getData(null) != null) {
        System.out.println("Data: " + storage.getData(null));
    }
}

I just can't get the book to lose its data, whether I rename it with an anvil, store it in a chest or throw it into a hopper. Are you sure you're using AttributeStorage correctly?

 

Thank you for the example code. It does exactly what I was trying to do and helped me solve the problem I was having with attributes: namely how to get a valid attribute to store information but not to display its text (well, almost).

To clarify how to reproduce my problem, which is more along the lines of an incorrect use of AttributeModifiers as Bukkit has decided to implement them. For example, I can store information in the "name" field of an attribute if its type is "generic.maxHealth", because this is a valid type of AttributeName. However, if I try to store information in a custom attribute type such as "generic.myCustomType" then the data will store.. until the user hovers over it with his mouse in the inventory. Then Bukkit will erase any unknown attributes, most likely during some call that evaluates how to display the information in the little popup window for the hovered item.

Where and when Bukkit decides to cleanse an item seems to vary. Sometimes it's when the user hovers over it; sometimes when it's moved around in the inventory; or when it goes through an anvil. But anything which Bukkit does not consider to be valid is as unsafe as any other custom NBT which does not align to vanilla.

What your example code has shown me is that it is still possible to utilize valid attributes discretely by setting the "Amount" to 0, which causes the field to display as empty, which works just as well. It's unfortunate that the field, while blank, still displays as a line when hovering over the item. But I can still use ProtocolLib to solve this and in the event that a required update is not immediately available, I won't have something unpleasant being displayed client-side in the interim.

I think instead of implementing a way to hide these blank lines within my plugin, I could decouple this into a new plugin whose sole purpose is to hide blank client-side attributes. This would provide both an adequate solution and completely remove ProtocolLib as a dependency from any plugin I write using Attributes as a storage method.

Hhhmm. Food for thought.

 

Comphenix I did some playing around with AttributeStorage chunking up a large file (300KB) and storing in across several attributes. the client was disconnected with a netty error along the lines of 'unexpected packet size' . I didn't have this issue with a 27KB file. so this leads me to believe attributes ARE transmitted to the client and do consume bandwidth, and they're limited by whatever the maximum packet size is.

 

I'm so sorry; I don't know how I could have missed this. I store data using a dummy attribute modifier's name, and this information is stripped from the PacketPlayOutUpdateAttributes, with only the dummy modifier itself remaining. This is only 25 bytes for each modifier (2 longs + 1 double + 1 byte), a negligible amount. This prevents the data from being broadcasted to every nearby player.

But the item stack itself, along with attributes in NBT form, is also transmitted through PacketPlayOutSetSlot and PacketPlayOutWindowItems. The data does end up on the client-side, unless you manually strip it out with ItemRenamer or ProtocolLib in the aforementioned packets.

Still, if you do need to store a large piece of data, I suggest keeping it in a MySQL database (blob) or file and refer to it through a UUID or file path. Otherwise, it will always be loaded in memory, and it will inflate the ".dat" player files tremendously.

Incidentally, I do believe I found an upper limit to the packet size. If you look in PacketSplitter, you'll see that the packet length is a prefixed varint of 3 bytes, which can encode at most 2^(7^3) = 2^21 = 2 097 152 bytes (2 MB). But there are some limits to the decoded size of the NBT data that is way below this, as can be demonstrated in this example:

Code:java

@Override
public boolean onCommand(CommandSender sender, Command command, String label, String[] args) {
    if (sender instanceof Player)  {
        ItemStack stack = new ItemStack(Material.GOLD_AXE);
        Random rnd = new Random();
 
        for (int i = 0; i < 5; i++) {
            AttributeStorage storage = AttributeStorage.newTarget(stack, UUID.randomUUID());
            byte[] garbage = new byte[35535];
            rnd.nextBytes(garbage);
 
            storage.setData(new String(garbage, Charsets.ISO_8859_1));
            stack = storage.getTarget();
        }
 
        ((Player) sender).getInventory().addItem(stack);
    }
    return true;
}

This will only crash the client, not the server.

 

Comphenix No need to apologize, you're blazing new trail here, there are bound to be things missed. I was more or less doing this as a proof-of-concept just to see what was possible. I've been considering rewriting my build-in-a-box plugin to use this technique and store schematics in the stack, or at least a delta between a schematic and the 'current state' of a build. it currently does store it's data in a database but since we can't reliably track items in game, and have no way to know if an item still exists in the world, the plugin can't safely delete anything, and the database grows indefinitely. This has been the main reason I let that plugin stagnate, I want to rewrite it with a more reliable data storage system, but embedding data IN the itemstack seems to be the only way to make sure it doesn't get 'lost'.

 

Comphenix So if I wanted to edit the nbt tags of a minecart how would I go around doing it

 

You can't do that directly - entities are only converted to NBT when they're written to disk, they don't store NBT data in memory like item stacks.

Instead, try modifying their fields directly. Here's how you would add or remove attributes, for instance.