Tutorial - Use external library(s) with your plugin

Discussion in 'Resources' started by fletch_to_99, Oct 3, 2012.

Thread Status:
Not open for further replies.
  1. Offline

    fletch_to_99

    This tutorial aims to show you how to use external libraries in your plugin without any user interaction. I ran into the issue the other day when making my plugin monsteremergency (http://dev.bukkit.org/server-mods/monsteremergency/) because it required use of the JavaMail(http://www.oracle.com/technetwork/java/javamail/index.html) API which is not built into JavaSE. The way I achieve using external jars requires your plugin to be packed as a jar containing the external libs.

    1.0 - Setting up the project

    The first step to using an external library is to add it to your project in your preferred IDE. It also helps to have the jars in the same project folder that you are building from. See fig 1 below.
    [​IMG]
    fig 1 - Jars in local project folder

    1.1 - Verify we are running from Jar

    Now that you have the libs added to your project you're going to need to verify that the instance of your plugin is running from Jar. To do this I would reccomend creating a class called "JarUtils" or something similar. In that class add:
    Code:
        private static boolean RUNNING_FROM_JAR = false;
     
        static {
            final URL resource = JarUtils.class.getClassLoader()
                    .getResource("plugin.yml");
            if (resource != null) {
                RUNNING_FROM_JAR = true;
            }
        }
    1.3 - Getting ready to extract the libs

    Now you're going to need a way of fetching the running Jar. To do this add the following to the jar utils:
    Code:
        public static JarFile getRunningJar() throws IOException {
            if (!RUNNING_FROM_JAR) {
                return null; // null if not running from jar
            }
            String path = new File(JavaUtils.class.getProtectionDomain()
                    .getCodeSource().getLocation().getPath()).getAbsolutePath();
            path = URLDecoder.decode(path, "UTF-8");
            return new JarFile(path);
        }
    The code above will get your plugins Jar file. You now need a way to extract the libs from your plugins jar file. Add the following to your JarUtils class.

    Code:
        public static boolean extractFromJar(final String fileName,
                final String dest) throws IOException {
            if (getRunningJar() == null) {
                return false;
            }
            final File file = new File(dest);
            if (file.isDirectory()) {
                file.mkdir();
                return false;
            }
            if (!file.exists()) {
                file.getParentFile().mkdirs();
            }
     
            final JarFile jar = getRunningJar();
            final Enumeration<JarEntry> e = jar.entries();
            while (e.hasMoreElements()) {
                final JarEntry je = e.nextElement();
                if (!je.getName().contains(fileName)) {
                    continue;
                }
                final InputStream in = new BufferedInputStream(
                        jar.getInputStream(je));
                final OutputStream out = new BufferedOutputStream(
                        new FileOutputStream(file));
                copyInputStream(in, out);
                jar.close();
                return true;
            }
            jar.close();
            return false;
        }
     
        private final static void copyInputStream(final InputStream in,
                final OutputStream out) throws IOException {
            try {
                final byte[] buff = new byte[4096];
                int n;
                while ((n = in.read(buff)) > 0) {
                    out.write(buff, 0, n);
                }
            } finally {
                out.flush();
                out.close();
                in.close();
            }
        }
    Heres a breakdown of the above code:

    Code:
            if (getRunningJar() == null) {
                return false;
            }
            final File file = new File(dest);
            if (file.isDirectory()) {
                file.mkdir();
                return false;
            }
            if (!file.exists()) {
                file.getParentFile().mkdirs();
            }
     
            final JarFile jar = getRunningJar();
    This is used to check that the location we will be extracting to is 1. A file and not a directory and 2. That its parent directories exist. If they do we can then fetch an instance of the running jar which we will use to find the libs to extract.

    Code:
    final Enumeration<JarEntry> e = jar.entries();
            while (e.hasMoreElements()) {
                final JarEntry je = e.nextElement();
                if (!je.getName().contains(fileName)) {
                    continue;
                }
                final InputStream in = new BufferedInputStream(
                        jar.getInputStream(je));
                final OutputStream out = new BufferedOutputStream(
                        new FileOutputStream(file));
                copyInputStream(in, out);
                jar.close();
                return true;
            }
            jar.close();
            return false;
    We then loop through all of the files in our plugins Jar looking for the filename defined by parameter 1. If we find a match we copy over the file to the location defined by parameter 2 using copyInputStream, and close the streams then return true saying the file was successfully extracted.

    Code:
      private final static void copyInputStream(final InputStream in,
                final OutputStream out) throws IOException {
            try {
                final byte[] buff = new byte[4096];
                int n;
                while ((n = in.read(buff)) > 0) {
                    out.write(buff, 0, n);
                }
            } finally {
                out.flush();
                out.close();
                in.close();
            }
        }
    copyInputStream copys the bytes from an input stream and writes them to an output stream. in.read will return 0 when no bytes are read, at this point we know that it was successfully copied we can now proceed to closing the streams

    1.4 - Extracting the libs

    Now we need to extract the libs so we can later use them. To do this add the following somewhere to your onEnable() in your plugins main class.

    Code:
            try {
                final File[] libs = new File[] {
                        new File(getDataFolder(), "activation.jar"),
                        new File(getDataFolder(), "mail.jar") };
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        JarUtils.extractFromJar(lib.getName(),
                                lib.getAbsolutePath());
                    }
                }
            } catch (final Exception e) {
                e.printStackTrace();
            }
    What this does is extracts the libs. First I created an array containing the locations I will extract the libs to. I then loop through this array extracting the jars 1 by 1 I check to make sure that they haven't already been extracted from previous runs of the plugin. JarUtils#extractFromJar(String filename [the name of the library packed in your plugins Jar], String fileLocation [The final location of the library after being extracted]) is used from 1.3 to extract them.

    1.5 - Loading the libs

    Now that the libs are extracted the plugin needs to load them so they can be used during runtime, otherwise you will get ClassNotFound exceptions. The proper way of doing this would be to make a classloader, however with a simple reflection hack it can be done in under 10 lines. Somewhere in your plugins mainclass you are going to want to add:
    Code:
        private void addClassPath(final URL url) throws IOException {
            final URLClassLoader sysloader = (URLClassLoader) ClassLoader
                    .getSystemClassLoader();
            final Class<URLClassLoader> sysclass = URLClassLoader.class;
            try {
                final Method method = sysclass.getDeclaredMethod("addURL",
                        new Class[] { URL.class });
                method.setAccessible(true);
                method.invoke(sysloader, new Object[] { url });
            } catch (final Throwable t) {
                t.printStackTrace();
                throw new IOException("Error adding " + url
                        + " to system classloader");
            }
        }
    What this does is uses Reflection to invoke the addURL method in the system classloader. Basically faking the -classpath variable in essence you are adding the external libs to the classpath during runtime. See http://en.wikipedia.org/wiki/Classpath_(Java) for more information.

    Your now going to add the following code to you JarUtils class:
    Code:
        public static URL getJarUrl(final File file) throws IOException {
            return new URL("jar:" + file.toURI().toURL().toExternalForm() + "!/");
        }
    This properly fetches the URL of the of the Lib because Jar files do some wonky stuff when trying to get their URL.

    Now somewhere in your onEnable() your going to want to add:

    Code:
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        getLogger().warning(
                                "There was a critical error loading My plugin! Could not find lib: "
                                        + lib.getName());
                        Bukkit.getServer().getPluginManager().disablePlugin(this);
                        return;
                    }
                    addClassPath(JarUtils.getJarUrl(lib));
                }
    What the above does is loops through all of your libs again this time adding them to the classpath. If a lib was not found then the plugin gets shut down to prevent further errors.

    Your final onEnable should look similar too:
    Code:
            try {
                final File[] libs = new File[] {
                        new File(getDataFolder(), "activation.jar"),
                        new File(getDataFolder(), "mail.jar") };
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        JarUtils.extractFromJar(lib.getName(),
                                lib.getAbsolutePath());
                    }
                }
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        getLogger().warning(
                                "There was a critical error loading My plugin! Could not find lib: "
                                        + lib.getName());
                        Bukkit.getServer().getPluginManager().disablePlugin(this);
                        return;
                    }
                    addClassPath(JarUtils.getJarUrl(lib));
                }
            } catch (final Exception e) {
                e.printStackTrace();
            }
    1.6 - Creating your plugin

    When creating your plugin be sure to pack the libs in the jar. If using eclipse when going to create the jar be sure to check the libs. I have mine tucked away in a folder called lib, which is in the project's directory. See fig 2 below.

    [​IMG]
    Fig 2 - Packing your plugin with your libs included

    1.7 - TL;DR

    JarUtils class:
    Code:
    package org.monstercraft.info.plugin.utils;
     
    import java.io.BufferedInputStream;
    import java.io.BufferedOutputStream;
    import java.io.File;
    import java.io.FileOutputStream;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.OutputStream;
    import java.net.URL;
    import java.net.URLDecoder;
    import java.util.Enumeration;
    import java.util.jar.JarEntry;
    import java.util.jar.JarFile;
     
    import org.monstercraft.info.MonsterEmergency;
     
    public class JarUtils {
     
        public static boolean extractFromJar(final String fileName,
                final String dest) throws IOException {
            if (getRunningJar() == null) {
                return false;
            }
            final File file = new File(dest);
            if (file.isDirectory()) {
                file.mkdir();
                return false;
            }
            if (!file.exists()) {
                file.getParentFile().mkdirs();
            }
     
            final JarFile jar = getRunningJar();
            final Enumeration<JarEntry> e = jar.entries();
            while (e.hasMoreElements()) {
                final JarEntry je = e.nextElement();
                if (!je.getName().contains(fileName)) {
                    continue;
                }
                final InputStream in = new BufferedInputStream(
                        jar.getInputStream(je));
                final OutputStream out = new BufferedOutputStream(
                        new FileOutputStream(file));
                copyInputStream(in, out);
                jar.close();
                return true;
            }
            jar.close();
            return false;
        }
     
        private final static void copyInputStream(final InputStream in,
                final OutputStream out) throws IOException {
            try {
                final byte[] buff = new byte[4096];
                int n;
                while ((n = in.read(buff)) > 0) {
                    out.write(buff, 0, n);
                }
            } finally {
                out.flush();
                out.close();
                in.close();
            }
        }
     
        public static URL getJarUrl(final File file) throws IOException {
            return new URL("jar:" + file.toURI().toURL().toExternalForm() + "!/");
        }
     
        public static JarFile getRunningJar() throws IOException {
            if (!RUNNING_FROM_JAR) {
                return null; // null if not running from jar
            }
            String path = new File(JarUtils.class.getProtectionDomain()
                    .getCodeSource().getLocation().getPath()).getAbsolutePath();
            path = URLDecoder.decode(path, "UTF-8");
            return new JarFile(path);
        }
     
        private static boolean RUNNING_FROM_JAR = false;
     
        static {
            final URL resource = JarUtils.class.getClassLoader()
                    .getResource("plugin.yml");
            if (resource != null) {
                RUNNING_FROM_JAR = true;
            }
        }
     
    }
    
    Add to your onEnable:
    Code:
            try {
                final File[] libs = new File[] {
                        new File(getDataFolder(), "activation.jar"),
                        new File(getDataFolder(), "mail.jar") };
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        JarUtils.extractFromJar(lib.getName(),
                                lib.getAbsolutePath());
                    }
                }
                for (final File lib : libs) {
                    if (!lib.exists()) {
                        getLogger().warning(
                                "There was a critical error loading My plugin! Could not find lib: "
                                        + lib.getName());
                        Bukkit.getServer().getPluginManager().disablePlugin(this);
                        return;
                    }
                    addClassPath(JarUtils.getJarUrl(lib));
                }
            } catch (final Exception e) {
                e.printStackTrace();
            }
    Add below your onEnable:
    Code:
        private void addClassPath(final URL url) throws IOException {
            final URLClassLoader sysloader = (URLClassLoader) ClassLoader
                    .getSystemClassLoader();
            final Class<URLClassLoader> sysclass = URLClassLoader.class;
            try {
                final Method method = sysclass.getDeclaredMethod("addURL",
                        new Class[] { URL.class });
                method.setAccessible(true);
                method.invoke(sysloader, new Object[] { url });
            } catch (final Throwable t) {
                t.printStackTrace();
                throw new IOException("Error adding " + url
                        + " to system classloader");
            }
        }
    2.0 - Comments, Questions or Concerns?

    Just leave them below, also if you have a suggestion/improvement for my code just let me know! I'm not perfect :p
     
  2. Offline

    fletch_to_99

    120 views and no replies :(
     
    CeramicTitan likes this.
  3. Offline

    gamemakertim

    Nice done,
    no reply needed
     
    fletch_to_99 likes this.
  4. Offline

    Gildan27

    This made my day. My only complaint is that I did not find this post two hours earlier.
     
  5. Offline

    Wackenov

    This is amazing, using this on my current plugin that requires several libraries. Will give proper credit on release. Thank you so much.
     
  6. Offline

    md_5

    Why do this when you can:
    a) Use maven shading
    b) Add classpath to manifest

    Both are far less complicated, reliable and don't require any code.
     
    aman207 likes this.
  7. Offline

    fletch_to_99

    a) Never heard of maven shading (time to google)
    b) I didn't know you could do that :O I'll make that the tl;dr version ;) Also would this work for unknown paths? For example if on my computer I have it in c:\fletch\server\lib\lib.jar and someone else had a different path, would that work?

    Thanks for your feedback md_5 :D
     
  8. Offline

    md_5

    Basically in your jar add a new file:
    META-INF/MANIFEST.MF
    and add a line like:
    Class-Path: lib/mail.jar;lib/etc.jar;lib/kittens.jar

    The only issue with this option is that once the jars are downloaded the plugin must be reloaded.
    Maven shading on the other hand (maven being a project build system), includes the classes inside the jar itself, so no downloading or reloading needed. 90% of plugins use this option.
     
  9. Offline

    Mr. X

    Thanks for this!
     
  10. Offline

    Sagacious_Zed Bukkit Docs

    I was having a rather long talk with feildmaster about this, I think I came to the conclusion that bukkit does not respect the Class-Path entry in plugins.
     
  11. Offline

    md_5

    I could almost swear it does, if not the plugins I have seen doing that are misinformed.
     
  12. Offline

    greatman

    Another way I found out to load .jars into bukkit:
    Code:
    ((PluginClassLoader) myplugin.getClassLoader()).addURL(new File(path).toURI().toURL());
    path being the path to the .jar file (Example: lib/h2.sql).
     
  13. Offline

    vemacs

    Unfortunately, even after doing that, I get the exact same ClassDefNotFoundError when starting the plugin.
     
  14. Offline

    javaguy78

    You are a god! thank you for this.
     
  15. Offline

    fletch_to_99

    Glad I could help :)
     
  16. Offline

    BLuFeNiX

    This is great! You just saved me a big headache. Thanks!
     
  17. Offline

    CaptainBern Retired Staff

    It's nice but I'm going to keep on using maven, but try to put your code between
    Code:java
    1. code here
    because it reads easier.
     
  18. Offline

    BLuFeNiX

    Hi,
    I implemented your code into my project, but it only works if I specify "-Xverify:none" on the command line when I start my server:
    Code:
    java -Xverify:none -jar ./craftbukkit.jar nogui
    Otherwise, I get the following error:
    Code:
    org.bukkit.plugin.InvalidPluginException: java.lang.NoClassDefFoundError
    None of my plugin code actually runs if I don't specify "-Xverify:none", not even if I put a static{} initializer in my class. Have you run into this problem? Do you have any suggestions?
     
  19. Offline

    wreed12345

    @fletch_to_99 BLuFeNiX I am having the same problem! This would would awesome if you could help us both out
     
  20. Offline

    Ultimate_n00b

    Just stick with Maven shading.
     
  21. Offline

    BLuFeNiX

    Yes, I did find a solution! It turns out, I was catching an exception that was thrown from within the library I was loading. I changed it from catch (SpecificExternalLibraryException e) {} to the standard catch (Exception e) {} and it worked! Let me know if you need any clarification about this.
     
  22. Offline

    NLGamingBross


    You from youtube! I am dutch to :) yaaay!






    P.s Nice Tut
    !
     
  23. Offline

    Lactem

    Thanks for a working tutorial. Maven shading tutorials are impossible to find. My complaints: There's so much code for this one task. Also, you said this:
    public static JarFile getRunningJar() throws IOException {
    if (!RUNNING_FROM_JAR) {
    return null; // null if not running from jar
    }
    String path = new File(JavaUtils.class.getProtectionDomain()
    .getCodeSource().getLocation().getPath()).getAbsolutePath();
    path = URLDecoder.decode(path, "UTF-8");
    return new JarFile(path);
    }
    That doesn't work. I think you meant to say String path = new File(JarUtils.class.etc...... instead of String path = new File(JavaUtils.class.etc..... Other than that, everything is great. Nice work!

    EDIT: I still think this is a great tutorial and works wonderfully for private plugins, but it can never be used for a plugin submitted to Bukkit Dev. (I'm speaking from experience here.) They will reject it because this way of using an external JAR is not acceptable. Instead, you have to use Maven shading. If you know how to do this and would like to make a tutorial for it, I think it would benefit people more.
     
  24. Offline

    _Hybrid

Thread Status:
Not open for further replies.

Share This Page