AutoUpdate - Update your plugins!

This is up for grabs. See here why.

I want to present you a class I wrote.

Overview
This class handles plugin updates. It does this by broadcasting new updates to admins and giving them a command to update a specific or all plugins.

What makes this special?

• Easy setup.
• Automatic update check at BukkitDev via bukget.
• Uses bukkits build-in update API.
• One permission set for all plugins.
• Standard config node for all plugins.
• One command for all plugins.
• Multi-threading for best performance.

Permissions
There are two permission nodes you have to care (and tell your users) about:

• autoupdate.announce - Users with this node will be notified about new updates.
• autoupdate.update.<yourPluginName> - Users with this node will be able to /update your plugin.

Both nodes default to op.

Known caveeats
None.

Command
One simple command:
/update <PluginName> - To update <PluginName> or all plugins if none given.

Setup
The lazy way
If you're a lucky winner all you have to do is to put this into your plugin and add

Code:java

new AutoUpdate(this);

into your onEnable();
Yes, that's it. But I won't guarant you that everything works like expected this way.

The right way
First go the lazy way, but this time put

Code:java

new AutoUpdate(this);

between the codeblock that loads the config and the one that saves it, example:

Code:java

Configuration config = getConfig();
this.foo = config.getString("foo");
new AutoUpdate(this);
config.set("foo", foo);
saveConfig();

If you reload the config at any time make sure you are reseting the config, too. Example:

Code:java

Configuration config;
AutoUpdate autoUpdate;
public void onEnable()
{
config = getConfig();
this.foo = config.getString("foo");
autoUpdate = new AutoUpdate(this);
config.set("foo", foo);
saveConfig();
}
 
public void foo()
{
...
reloadConfig()
config = getConfig();
autoUpdate.resetConfig();
...
}

If you use a custom config, give it to the calls. Example:

Code:java

myConfig.load(something);
AutoUpdate autoUpdate = new AutoUpdate(this, myConfig);
...
myConfig = somethingOther;
myConfig.load(foo);
autoUpdate.setConfig(myConfig);

Important: If you cancel all your tasks restart the updater task! Example:

Code:java

scheduler.cancelTasks(this);
autoUpdate.restartMainTask();

Configuration
The configuration is inside of the class, look:

Code:java

 /*
* Configuration:
*
* delay = The delay this class checks for new updates. This time is in ticks (1 tick = 1/20 second).
* ymlPrefix = A prefix added to the version string from your plugin.yml.
* ymlSuffix = A suffix added to the version string from your plugin.yml.
* bukkitdevPrefix = A prefix added to the version string fetched from bukkitDev.
* bukkitdevSuffix = A suffix added to the version string fetched from bukkitDev.
* bukitdevSlug = The bukkitDev Slug. Leave empty for autodetection (uses plugin.getName().toLowerCase()).
* COLOR_INFO = The default text color.
* COLOR_OK = The text color for positive messages.
* COLOR_ERROR = The text color for error messages.
*/
private final long delay = 72000L;
private final String ymlPrefix = "v";
private final String ymlSuffix = "";
private final String bukkitdevPrefix = "";
private final String bukkitdevSuffix = "";
private String bukkitdevSlug = "";
private final ChatColor COLOR_INFO = ChatColor.PURPLE;
private final ChatColor COLOR_OK = ChatColor.GREEN;
private final ChatColor COLOR_ERROR = ChatColor.RED;
/*
* End of configuration.

Moar docs!
JavaDocs!

Videos


Plugins using this

• BananaRegion
• BukkitHTTPD
• EndReset
• KillStreaks
• RideThaDragon
• V10lift

Want your plugin added to the list? Just leave a reply.

Changelog

• v1.4: Reflecting latest changes in the bukget API.
• v1.3: Finally made the command usable from the console.
• v1.2: Silenced HTTP errors. - Better lock handling. - Removed some (now that bukget reads from the jar file directly) unneeded settings. - Removed the caching server. - Switched to bukgets API v2 this breaks older versions! Updating is highly recommended!
• v1.1: Switched to JSON.simple (suggested by Comphenix). - Added debug mode (request by iKeirNez).
• v1.0: Fixed a bug in the bug catcher. - Better JSON handling. - Splitted into a version using JSON and one using StrippedJSON. - Fixed corrupted downloads. - Added bukget caching via a server owned by Mikeambrose3 - Added automated fallback to bukget if the caching server isn't available. - Rised min. update check time to 1h. - Fixed a locking issue that can cause a endless loop in minecrafts main thread. Updating is highly recommended!
• v0.9: Fixed a couple of bugs.
• v0.8: Even better permission handling.
• v0.7: Better permission handling.
• v0.6: Made command case insensitive. - Better file handling.
• v0.5: Better messages. - Reduced amount of messages. - Raised default update delay from 1h to 3h. - Added minimum update delay (30 minutes).
• v0.4: Fixed typos. - Better config management. - Added enable/disable messages.
• v0.3: Fixed JavaDoc. - Added more config options. - Fixed error handling.
• v0.2: Better error handling.

Where's the class now?
v1.4: http://pastie.org/5460452
v1.3: http://pastie.org/4785865
v1.2: http://pastie.org/4698473
v1.1: http://pastie.org/4369717
v1.0: Build-in JSON - External JSON
v0.9: http://pastie.org/4200566
v0.8: http://pastie.org/4200239
v0.7: http://pastie.org/4200160
v0.6: http://pastie.org/4199997
v0.5: http://pastie.org/4198258
v0.4: http://pastie.org/4197314
v0.3: http://pastie.org/4196770
v0.2: http://pastie.org/4194586
v0.1: http://pastie.org/4194472

Want to live on the bleeding etch? Check out . Pull requests welcome!

 

Unfortunately This, and bukget violate the ToU for Curse.com.

 

Sleaker I don't think this breaks the ToU, quotes please?
For legal question about bukget ask their support as I'm not related to them in any way.

 

curse specifically prohibits use of auto-downloaders unless expressly permitted by them or released by them, that's all.

Other than that I think there was also a precedent for the bukkit team to deny any plugin approvals that used auto-downloaders because they could be too easily manipulated for malicious purposes.

 

Sleaker Again: Quotes please. The only part of the Curse ToS that may be against the use of this class would be this:

But this class uses bukget to get the information, so this does not "send more request messages to the Curse servers ..." and again: I' not responsible for bukget.
If anything this class advertises the curse network as it broadcasts the bukkitdev link of the plugin to Admins (having the autoupdate.announce permission node).

Really?
http://forums.bukkit.org/threads/auto-updater-say-wat.19827/
http://forums.bukkit.org/threads/au...asons-update-urgency-and-force-updates.21209/
http://forums.bukkit.org/threads/admn-dev-pluginupdater-v1-4-plugin-management-1-1-r7.61858/
http://dev.bukkit.org/server-mods/lazyupdate/
http://dev.bukkit.org/server-mods/pluginupdater/
http://dev.bukkit.org/server-mods/voxelupdate/

 

might be okay then. I dunno. I jsut stay as far away from this kind of stuff as possible. To each his own i guess.

 

The use of auto updaters using anything other than dev.bukkit.org as the source of the download will cause the file to not be approved.

We are working with Curse to make this aspect easier on server administrators, and plugin writers (aka, static link to latest file and an intelligent way of determining whether the latest file is different than the one existing one the server).

 

But as far as I see none of the things I linked here uses dev.bukkit.org as source. But this one here uses dev.bukkit.org to download the jar file (to get the file information and the bukkitDev link it uses bukget).

BTW: The first files using this class where approved at bukkitDev.

 

V10lator
did you ever get onCommand working? or does it still use commandPreprocess?

 

Seems like a lot of code.

This is what I do:

Code:JAVA

boolean is_latest_version()
{
DocumentBuilder dbf;
try
{
dbf = DocumentBuilderFactory.newInstance().newDocumentBuilder();
Document doc = dbf.parse("[url]http://dev.bukkit.org/server-mods/gui-creator/files.rss[/url]");
XPath xpath = XPathFactory.newInstance().newXPath();
String s = ((Element) xpath.evaluate("//item[1]/title", doc,
XPathConstants.NODE)).getTextContent();
return (s.equalsIgnoreCase(getDescription().getVersion()));
}
catch (Exception e) {return true;}
}

I'm sure there's something horribly wrong with it but it works fine for me

God I hate this editor! Keeps adding url tags

 

Still uses the PlayerCommandPreprocessEvent. Any help is welcome.

coldandtired Well, have a look at "What makes this special?" (and at the video) in the first post to see why there's that much code.

 

Sorry, I was referring to just the checking for updates section

 

coldandtired Well, but your source hasn't all the information AutoUpdate needs (compare http://dev.bukkit.org/server-mods/bukkithttpd/files.rss and http://bukget.org/api/plugin/bukkithttpd/latest ) and even if the code is bigger it works in a different thread and just parses a bit JSON data...

 

Thanks for the links, we'll clean those up. Getting the file info from dev.bukkit.org and downloading the files from dev.bukkit.org is allowed in an autoupdater.

 

You're confusing me. I put a lot of work into this project and it starts spreading into plugins and even being loved by stuff, but I don't know if it's allowed or not.

Again: it uses bukget to get informations about the latest file (it fetches http://bukget.org/api/plugin/<bukkitDevSlug>/latest ) and informs the user if there's a new update. If the user decides to update it downloads the file directly from dev.bukkit.org. I think I could change the class to get the file information directly, but then:
1. The class would bring a lot of traffic to dev.bukkit.org, possibly breaking the ToS:

cause
2. I can't find a page at dev.bukkit.org containing all the information the class needs. The rss feed coldandtired notified about would be a good start, but the class could only take two (bukkitdev link + new version) of the 4 needed informations from it, while it had to throw a lot of useless data into the trash.

tyzoid I had a deeper look into dynamic command registration now: It seems like you basically need to register your new command at the CraftServer.
Creating the new command is doable with some reflection:

Code:java

 Constructor<PluginCommand> c = PluginCommand.class.getConstructor(String.class, Plugin.class);
c.setAccessible(true);
PluginCommand cmd = c.newInstance("update", plugin);
c.setAccessible(false);

But to register it we need to hook into CB code and I don't want to force users of this class to figure around with CB:

Code:java

 CraftServer cs = (CraftServer)plugin.getServer();
SimpleCommandMap scm = cs.getCommandMap();
scm.register("update", cmd);

 

V10lator
The method you use is the only way we allow it. Bukget is the API provided for dev.bukkit.org, but it is not meant for extensive public consumption that an auto updater would cause. This is why you may break the ToS by creating an auto updater , and why we have not encouraged their use before.

Even with Bukget, we understand its not a fully realized API. Bukget was never meant to be utilized as extensively as it is after the community found it, and the load on that API is quite extensive. We are working with Curse to provide a better method to allow auto update plugins, with a more robust API. You cannot find a page on dev.bukkit.org containing all the information because one does not exist at this time.

To adequately do an auto updater, the program must check for a new file, understand if the file is newer than the one on the server already, get the link and stage it in the plugin/update folder for the next restart/reload. We need a better API to provide those abilities, and we will continue to work with Curse to get one.

 

I have an important question to everybody:
Should this class depend on the JSON library or use a inlined, stripped-down version?


I ask this cause first I tried to provide a stipped down JSON lib for use with this project: http://forums.bukkit.org/threads/dynamic-in-line-dependencie-handling.84753/ but it didn't seem to work that easy.
But I could easily in-line this stripped down version. That's how it looks: http://pastie.org/4204455
As you see it's still a lot of code, but way less than the original lib. It allows us to do what we need to do, but nothing more. Also some internals where tuned for faster execution speed (still a WIP).

Pros for in-lining:

• Fast, (relative) lightweigt JSON handling.
• No dependencies.
• Its easy to cut out the inlined version and use the default lib (just remove the in-lined code and import the then missing classes from org.json).

Cons:

• - Duplicated JSON code (needed ram = JSON code * plugins using this class).

What do you think?

 

How big would each JSON string be?

if it's nothing more than 5000 characters, I'd say go for it.

 

TnT So bukget == bukkitdev. I (and I think others, too) Didn't know that.
If that means that the Curse ToS is valid for bukget then you're right: This class may break it. But at bukget there's no information that it's associated with Curse, nor do they tell me I have to accept the Curse ToS at any point. Also this API is not designed to be watched by a web browser, so the Curse ToS doesn't make much sense here (may even be invalid).

As you seem to be involved in bukget/bukkitdevAPI: Could you ask if it's okay to use bukget for this class till the better API is ready? As you see it tries to fetch as little data as possible and if needed I could rise the default (and the min.) update check time more future (currently: Default: every 3 h (changeable by the dev using this class), min: 30 mins).
This question is important for me as if I'm not allowed to use bukget for this I will stop doing so, but till now nobody tried to stop me...

 

V10lator
A possible solution to your extra traffic delemma, why not just cache the results on a seperate domain (run by you, or someone who would donate space).

If you cache them for ~4 hours, then high-volume traffic created by this plugin would be diverted to the other server, and hence, not put much additional load on Curse's servers.

 

Around that size: http://bukget.org/api/plugin/bukkithttpd/latest

But was has the data to be parsed to do with the size/efficiency/... of the code doing it, especially if the stripped version is directly based on top of the org.json one? ^^

Great idea! Will have a talk to my hoster if he would allow that.

 

Why not cache for 24 hours? We're talking about plugin updates, not heart transplants.

V10lator Auto Updates are becoming a large concern lately, both from an API perspective, and a community safety one, hence not hearing anything about it until now. We are forced to take more interest in the process now, as more people discover this API, or attempt to create an auto updater by other means.

 

Well, the idea is that it would give reasonably recent results without causing lag.

With 24 hours, server administrators would have to wait an entire day (or plugin authors to see if it worked) to get the update.

 

I was writing a plugin just like this. Ran into some issues with locked files. I could overrite the plugin in the plugins folder but (FULLY) unlocking and unloading it was the true problem. I kept pulling an IOException and could get bukkit to let go of the jar.
But a little help I made up some methods of checking for an update.

Show Spoiler Hide Spoiler

Code:

 
public void updateCheck() throws MalformedURLException, IOException, ParseException, UnknownDependencyException, InvalidPluginException, InvalidDescriptionException{
Plugin[] plugins = this.getServer().getPluginManager().getPlugins();
for(Plugin plugin: plugins){
final String name = plugin.getName();
boolean update = updateExists(plugin);
if(update && !plugin.equals(this) && !this.plugins.contains(plugin.getName())){
final File actual = new File("plugins/", name+".jar");
this.getServer().getPluginManager().disablePlugin(plugin);
this.getServer().getServicesManager().unregisterAll(plugin);
//Changed object
//actual.delete();
 
wget(name);
if(actual != null){
this.getServer().getScheduler().scheduleAsyncDelayedTask(this,new Runnable(){
 
@Override
public void run() {
PluginManager man = Bukkit.getServer().getPluginManager();
try {
man.loadPlugin(actual);
} catch (UnknownDependencyException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvalidPluginException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvalidDescriptionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//if(!man.isPluginEnabled(name)){
man.enablePlugin(man.getPlugin(name));
//}
 
}
 
}, this.getServer().getConnectionThrottle());
}
}
}
 
}
private boolean updateExists(Plugin plugin) throws IOException, ParseException, MalformedURLException{
URL url = new URL("http://api.bukget.org/api/plugin/"+plugin.getName().toLowerCase()+"s");
URLConnection connection = url.openConnection();
connection.addRequestProperty("Referer","http://" + Bukkit.getServer().getIp());
String line;
StringBuilder builder = new StringBuilder();
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
while((line = reader.readLine()) != null) {
builder.append(line);
}
JSONParser parser = new JSONParser();
if(builder.toString().equals("")){
return false;
}
Object obj = parser.parse(builder.toString());
JSONObject jsonObject = (JSONObject) obj;
JSONArray version = (JSONArray) jsonObject.get("versions");
this.getLogger().log(Level.INFO, version.toString());
if(version.size() < 1) return false;
JSONObject latest = (JSONObject) version.get(1);
long date = Long.parseLong((latest.get("date").toString()));
File actual = new File("plugins/", plugin.getName());
if(Math.abs(date - actual.lastModified()) >  172800000){
return true;
}
return false;
}
public void wget(String name) throws MalformedURLException, IOException{
BufferedInputStream in = new BufferedInputStream(new URL("http://api.bukget.org/api/plugin/"+name.toLowerCase()+"/latest/download").openStream());
FileOutputStream fos = new FileOutputStream("plugins/"+name+".jar", false);
BufferedOutputStream bout = new BufferedOutputStream(fos,1024);
byte[] data = new byte[1024];
int x=0;
while((x=in.read(data,0,1024))>=0)
{
bout.write(data,0,x);
}
bout.close();
in.close();
}

 

You could try this code, it updates the plugin, and does not require a restart.

Show Spoiler Hide Spoiler

http://forums.bukkit.org/posts/1196984/

Code:

public String checkUpdate(boolean forceUpdate) {
  String link = "";
  try {
    URL rss = new URL("http://dev.bukkit.org/server-mods/" + this.getDescription().getName() + "/files.rss");
    String search = "version-";
    ReadableByteChannel rbc = Channels.newChannel(rss.openStream());
    this.getDataFolder().mkdir();
    File outputFile = new File(this.getDataFolder(), this.getDescription().getName() + ".tmp");
    outputFile.createNewFile();
    FileOutputStream fos = new FileOutputStream(outputFile);
    fos.getChannel().transferFrom(rbc, 0, 1 << 24);
    fos.close();
    Scanner s = new Scanner(outputFile);
    int line = 0;
    while(s.hasNextLine()) {
      link = s.nextLine();
      if(link.contains("<link>")) {
        line++;
      }
      if(line == 2) {
        link = link.substring(link.indexOf(">") + 1);
        link = link.substring(0, link.indexOf("<"));
        break;
      }
    }
    s.close();
    outputFile.delete();
    String newVersion = link.substring(link.lastIndexOf(search) + find.search(), link.lastIndexOf("/")).replace("-", ".");
    String currentVersion = this.getDescription().getVersion();
    if(!newVersion.equals(currentVersion)) {
      if(forceUpdate) {
        downloadJar(downloadSite(link));
      }
    }
  }
  catch (IOException e) {
    e.printStackTrace();
  }
  return link;
}
public String downloadSite(String url) {
  String link = "";
  try {
    URL website = new URL(url);
    ReadableByteChannel rbc = Channels.newChannel(website.openStream());
    File outputFile = new File(this.getDataFolder(), this.getDescription().getName() + ".tmp");
    this.getDataFolder().mkdir();
    outputFile.createNewFile();
    FileOutputStream fos = new FileOutputStream(outputFile);
    fos.getChannel().transferFrom(rbc, 0, 1 << 24);
    fos.close();
    Scanner s = new Scanner(outputFile);
    while(s.hasNextLine()) {
      link = s.nextLine();
      if(link.contains("user-action-download")) {
        link = link.substring(link.indexOf("href"));
        link = link.substring(link.indexOf("\"") + 1, link.lastIndexOf("\""));
        break;
      }
    }
    s.close();
    outputFile.delete();
  }
  catch (IOException e) {
    e.printStackTrace();
  }
  return link;
}
public void downloadJar(String url) {
  try {
    URL website = new URL(url);
    ReadableByteChannel rbc = Channels.newChannel(website.openStream());
    File outputFile = new File("plugins", this.getDescription().getName() + ".jar");
    FileOutputStream fos = new FileOutputStream(outputFile);
    fos.getChannel().transferFrom(rbc, 0, 1 << 24);
    fos.close();
    log.info("Reloading" + this.getDescription().getName() + " v" + this.getDescription().getVersion());
    unloadPlugin(this.getDescription().getName());
    loadPlugin(this.getDescription().getName());
  }
  catch(Exception e) {
    e.printStackTrace();
  }
}
@SuppressWarnings("unchecked")
private void unloadPlugin(final String pluginName) throws NoSuchFieldException, IllegalAccessException {
  PluginManager manager = getServer().getPluginManager();
  SimplePluginManager spm = (SimplePluginManager) manager;
  SimpleCommandMap commandMap = null;
  List<Plugin> plugins = null;
  Map<String, Plugin> lookupNames = null;
  Map<String, Command> knownCommands = null;
  Map<Event, SortedSet<RegisteredListener>> listeners = null;
  boolean reloadlisteners = true;
  if(spm != null) {
    Field pluginsField = spm.getClass().getDeclaredField("plugins");
    pluginsField.setAccessible(true);
    plugins = (List<Plugin>) pluginsField.get(spm);
    Field lookupNamesField = spm.getClass().getDeclaredField("lookupNames");
    lookupNamesField.setAccessible(true);
    lookupNames = (Map<String, Plugin>) lookupNamesField.get(spm);
    try {
      Field listenersField = spm.getClass().getDeclaredField("listeners");
      listenersField.setAccessible(true);
      listeners = (Map<Event, SortedSet<RegisteredListener>>) listenersField.get(spm);
    }
    catch (Exception e) {
      reloadlisteners = false;
    }
    Field commandMapField = spm.getClass().getDeclaredField("commandMap");
    commandMapField.setAccessible(true);
    commandMap = (SimpleCommandMap) commandMapField.get(spm);
    Field knownCommandsField = commandMap.getClass().getDeclaredField("knownCommands");
    knownCommandsField.setAccessible(true);
    knownCommands = (Map<String, Command>) knownCommandsField.get(commandMap);
  }
  for(Plugin pl : getServer().getPluginManager().getPlugins()) {
    if(pl.getDescription().getName().equalsIgnoreCase(pluginName)) {
      manager.disablePlugin(pl);
      if(plugins != null && plugins.contains(pl)) {
        plugins.remove(pl);
      }
      if(lookupNames != null && lookupNames.containsKey(pluginName)) {
        lookupNames.remove(pluginName);
      }
      if(listeners != null && reloadlisteners) {
        for(SortedSet<RegisteredListener> set : listeners.values()) {
          for(Iterator<RegisteredListener> it = set.iterator(); it.hasNext();) {
            RegisteredListener value = it.next();
            if(value.getPlugin() == pl) {
              it.remove();
            }
          }
        }
      }
      if(commandMap != null) {
        for(Iterator<Map.Entry<String, Command>> it = knownCommands.entrySet().iterator(); it.hasNext();) {
          Map.Entry<String, Command> entry = it.next();
          if(entry.getValue() instanceof PluginCommand) {
            PluginCommand c = (PluginCommand) entry.getValue();
            if(c.getPlugin() == pl) {
              c.unregister(commandMap);
              it.remove();
            }
          }
        }
      }
    }
  }
}
public void loadPlugin(final String pluginName) throws InvalidPluginException, InvalidDescriptionException {
  PluginManager manager = getServer().getPluginManager();
  Plugin plugin = manager.loadPlugin(new File("plugins", pluginName + ".jar"));
  if(plugin == null) {
    return;
  }
  manager.enablePlugin(plugin);
}

The only thing is, like V10lator, if we can't use something like Bukget, then you'll end up with a lot of useless data being downloaded.

 

TnT Well, 24 hours seems a bit high, especially as bukget caches, too...
It's planned to set up a caching server at http://bukget.v10lator.de/ (URL may change later) which caches for 6 hours at the beginning. I want to fine-tune the caching time based at bandwith (to bukget) when everything is running.

The caching is done via htaccess, php and mysql. As the page at the URL linked above isn't ready yet you can have a preview look here:
http://www.v10lator.de/test/
just put the bukkitdev slug at the end, example:
http://www.v10lator.de/test/bpermissions

Important notice for everyone: This caching mechanism is meant to be used by AutoUpdate only! Do not use it for anything other!

Credits for hosting the caching site goes to Mikeambrose3

 

v1.0 is finally released and the caching server is up and running. Thanks again to Mikeambrose3 for the hosting.

It is highly recommended to update to v1.0 as there are important bug fixes! All other versions are no longer supported!

 

Okay I am using this in my plugin but the problem is that we upload our plugin in a ZIP file, do you think there would be anyway for this to unzip the zip and use the jar file in there? The name and directory of the jar file would have to be specified though because there are multiple files in the zip.

Thanks

 

iKeirNez Zip files aren't supported atm and I don't think I'll implement them as using zip files is bad anyway. Why don't you include the content of your zip in your jar file and extract it if needed?

 

Ok I will try and figure something out.