[WIP] Sync - Providing all your data storage needs [COMPLETED]

Discussion in 'WIP and Development Status' started by Adamki11s, Dec 21, 2011.

Thread Status:
Not open for further replies.
  1. [​IMG]
    Sync



    This plugin was inspired by the problems I see lots of developers having with saving and loading data and aims to centralise most forms of data storage to increase server efficiency whilst providing a variety of robust features.

    Sync will provide a variety of features related to the saving and loading of data in various forms. It will mimic and improve on Bukkit's configuration class, provide a solid SQL manager for both MySQL and SQLite alongside a Web Manager for related queries, grabbing source code etc. Sync will also provide caches and support for saving some unserializable Bukkit objects such as Locations.

    I also have plans to make Sync replace some basic, yet important process such as world saving, backing up plugin data etc.

    In Short this aims to be the data alternative to vault.

    Follow my Progress on GitHub
    Follow me on Twitter
    Email Me : [email protected]
     
    alfa5 and r3Fuze like this.
  2. Do want! This looks like a cool project.
     
    Adamki11s likes this.
  3. I had this idea at 2am this morning so I apologise for any incoherency :p.
     
    tips48 likes this.
  4. Just want to gauge the interest on this so - bump.
     
  5. Offline

    Don Redhorse

    LIKE... a lot... now we just need some other "HUB" plugins like REGIONS(TM)(notThePluginThough) and all region plugins would hook up to that so that every other plugin could hook up to and retrieve canBuild etc from one place..
     
  6. Yes I did have other similar ideas for centralisation of various functions but I don't want to deviate from what this plugin is meant to provide. I may instead make a separate plugin.
     
  7. Offline

    Don Redhorse

    no probs... I was just saying what I'm saying since several months... there are some missing key concepts in bukkit..

    and I like to see that at least the plugin developers pluck them..
     
  8. Could you make a prerelease? :p
     
  9. Ill put up a link for the latest development build soon but I haven't even done documentation or java comments ;)
     
  10. Offline

    bergerkiller

    Adamki11s what could this plugin be used for? I can imagine storing things what block loggers log, but will it be able to store and load custom classes? Will it work like a hashmap-way (give it a plugin + string and it returns you the data object(s)), or differently?

    I probably won't use it, as the only thing I store raw right now are things like the minecart groups of TrainCarts...but still interested in what it will add :)

    Ps. Will it be able to store chunks or blocks? This would be great for rollback plugins or even a custom database-based world chunk loader.
     
  11. Well I have quite a lot planned which I haven't divulged ;).

    It will provide an alternative class to Bukkits Configuration which a lot are confused by and for some reason is marked as deprecated. Plus I feel more comfortable using my own and have added support for comments, forced new lines and all of which are stored in a linked hash map. It always frustrated me when using Bukkit's Configuration the order in which I added data into the configuration class would not necessarily mean it would be written in that order.

    I will also be adding in complete MySQL support which will also support SQLite and I plan for it to be easily accessible and easy to pickup and learn for inexperienced and advanced developers alike, supporting prepared statements, connection pooling and probably a 'noob' friendly way of creating tables and executing queries.

    I'm not sure how I plan to implement it but I definitely plan to add support for saving chunks, and blocks of data such as how regios and world guard backup regions. As I know from experience that it's complicated to do and would like to provide a bridge to make this easier for devs.

    I have already worked on a cache features, as for custom classes that sounds like a good idea which I'll have to look into :)

    Also included will be basic plugin statistics such as time installed, times run, etc. I may add commands executed but that's not really top of my priorities. I also plan to bundle a lot of commonly run plugins such as message broadcasters and automatic world saves into Sync (I will code these myself, not rip from other devs).
     
  12. Offline

    bergerkiller

    Adamki11s the point is that I've tried this before, but failed horribly:

    Saving isn't too hard, you can do it the way Minecraft does it: give it a save function. But loading these classes remains hard to do.

    You need to have some sort of identifier in the saved mess to find out what is actually stored. Using unique numbers or even names is not infallible. I tried using a static 'register' method, which calls newInstance() on the registered classes, but the point is that other plugins have to register their class before the plugin even loads the data...

    The above was 'planned' as 'Rail meta data', to add a meta data to rails blocks dynamically and call functions inside when a minecart rides over it, etc.

    How will you handle such things which provide a single shared cache?
     
  13. Well as you said it is quite complicated and I have yet to test this but I plan to do it like this.

    Every object manager such as SyncIO and SyncWeb, SyncCache etc can be accessed thorugh the SyncControl class of which can be obtained by just getting the plugin, casting and calling the method which return a new instance of SyncControl.

    The idea is that a plugin will only utilise one SyncCache, any more would be un-necessary. They can then reference this. Each instance will also be added to a static list for the schedulers to loop through.

    If I'm missing the point could you please clarify.
     
  14. Offline

    bergerkiller

    Adamki11s how will you store custom classes? People obviously don't want to perform annoying casting or parse the bytes stored to get their class.

    Say I want to store this class:
    Code:
    public class MyClass {
        private String name;
        private OtherClass data;
        public void load(DataInputStream stream) throws IOException {
            this.name = stream.readUTF();
            this.data = new OtherClass();
            this.data.load(stream);
        }
        public void save(DataOutputStream stream) throws IOException {
            stream.writeUTF(this.name);
            this.data.save(stream);
        }
    }
    How will you go about returning this class from your cache? Use a static class loader plugins can register to? Use Class.newInstance() and load that? Interfaces?

    It's one of those awesome things that can make the developers' life a lot easier.

    Or will it simply use basic Bukkit/value types stored behind keys? Like get("myclass.name")
     
  15. Hmm Either I will require them to cast their own class or I have some ideas for automatic detection which I shall play around with :)
     
  16. bergerkiller
    Just in case you were still interested I thought I'd let you know that I have found an efficient solution to provide an interface for easily serialising custom classes and any objects which implement the Serializable interface. There is a holder which contains a list of wrappers and each wrapper has its own tag which can be used to grab the object inside the wrapper whcih can then be easily cast assuming the plugin knows what class it should be, which it should.

    I am also going to write various algorithms to save lists of blocks which suit different needs and have implemented version control which allows plugin developers to provide support to auto update their plugins by posting a few lines of text on any webpage. I am also implementing easy ways to save non serializable Bukkit Objects such as location, block and itemstack.
     
  17. Offline

    iPhysX

    Like. Adam has been working really hard on this recently! :p
     
    Adamki11s likes this.
  18. Offline

    Don Redhorse

Thread Status:
Not open for further replies.

Share This Page