Flatfile

Discussion in 'Bukkit Discussion' started by Penorzilla, Jan 2, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    croxis

    A dream situation would be bukkit handling all database communication, making it platform agnostic for plugin authors. That way all the plugin needs to do is say "Hey Mr. Bukkit, store this please!" and then bukkit would deal with talking to the database. It would free plugin devs from learning sql, opens up the possibility for other database types, and protects the database from a plugin dev dropping the whole thing.
     
  2. Offline

    AnonymousJohn

    That would be ideal, but not everyone would use it, making it bulk. It would be better if a plugin developer could make a library for that to distribute to other plugin developers, that way it'd only be there if it were necessary. In fact, that'd be pretty fun to do. I think I'll take you up on that.
     
  3. Offline

    legendblade

    In the same vein, I'd considered adding Hibernate to my plugins, but, not sure if the extra overhead is worth it. Having an ORM backend for the plugins would be nice, though.
     
  4. Offline

    8e8

    Rule of thumb: If you don't understand it, learn about it.

    aka google
     
  5. Offline

    croxis

    I'd love to do it, if I can ever get the sample plugin to even compile. Now if this was in python....
     
  6. Offline

    8e8

  7. Offline

    rayman86

    Even if Bukkit is designed to be configurable (itself) for MySQL, that will not be able to prevent plugin developers from manually opening files for their plugins. Its part of Java to be able to do this, and you don't have to use a method the Bukkit team develops to be able to read flatfiles.
     
  8. Offline

    AnonymousJohn

    Exactly. See my flatfile tutorial back on the hMod forums. No hMod/Bukkit methods/classes needed. Just pure Java.
     
  9. Offline

    Sky.NET

    I donĀ“t really understand why people use flatfile anyway... (except of one file that just handles the global mysql-specific data and should use a stadard like XML)
    F.e. with MySQL you are able to code a very nice webinterface where you control the whole server including all plugins. No more open 42562387489 files for reconfiguring your plugins. The second point here is, some plugins i used on my MCS are using the configfiles the whole time, the result is that i have to stop my server to change the config, thats even bullshit. Additionally reading files the whole time (or many times) is extremly performance incriminating. With MySQL you could implement some command like '/rpcfg [Pluginname]' what reloads config on the fly if you changed some setting.
    It also brings backward-compatibility to Plugins, if some major change is done, you could leave the old tables as they are and do the new things with data from a new table, or just write a little converter if the old information needs to be handled in a new way (new tables), or just with adding a new column whitch will not affect any other/older versions.

    Everything here is easy to do, no need to learn much MySQL-Commands, just the MySQL basics whitch can be learned in 1-3 days (if you arent a blockhead).

    In short:

    MySQL:
    • Faster
    • More Flexible
    • More backwardcompatibility
    • Easier to handle
    • Opens new ways for administrating your server over webinterfaces
    • More modern
    • Uses datatypes
    • Very clear interface to change things manually
    Flatfile:
    • Extremly Slow/Unperformant (if you have many users on your server, it will be screwed up by flatfilereading!)
    • Old/Noobish
    • Source of exceptions, if you F.E. put in a string and your plugin expect a int, it will spam the console with ugly throws, and maybe crashing your server!
    • Typing strange chains of characters into textfiles and open/edit/save them one by one is extremly annoying, timewasting and senseless.
    • Fills up your filesystem with a huge amount of txt-files, what makes your folders unclearly
    • Sometimes you have to STOP your server to change the configuration of a plugin
    The only users benefit from flatfile are dumb users or such that are too lazy to learn MySQL in a few days, and how to use it in Java, and such people should not code any plugin.

    The last thing i say about this topic is, using flatfile is (except SQLite/XML etc.) a very noobish way of keeping data, i was doing this the first months of my programming experience too (im programming in Java, AS2.0, C#, C++, PHP and some webrelated things like JS/Ajax etc. since 4 years now), but early changed to MySQL/MSSQL/SQLite due to professionallity and the above reasons.

    Word! 100% agree!
     
  10. Offline

    Killie01

    SQLite is a database on the computer and not on the web right?
     
  11. Offline

    croxis

    For one, if something is on the web it is on a computer somewhere.
    SQLite is similar to a flatfile in that the database is a easy to identify file, and, unless you are using Network File System, reside on the same computer as your minecraft server.

    MYSQL is a server program just like apache is for serving web pages. It can be run on the same machine as the minecraft server or on a different machine.
     
  12. Offline

    Killie01

    So if i would have a dedicated server its not on my pc ( sqlite)
     
  13. Offline

    Sky.NET

  14. Offline

    Killie01

    that is not what i ment actually,
    for my BSM if someone has dedicated will the thing be on his pc or on server or is both possible
     
  15. Offline

    AnonymousJohn

    I understand (and agree with) most of your reasoning, but I feel very insulted and offended that you would call me a "dumb user too lazy to learn MySQL and how to use it in Java," and that I "should not code any plugin." I know MySQL, and I know how to use it in Java, but for many things MySQL is unecessarily complicated (ResultSets are still hard for me to navigate). In many cases, simplicity is best, and Flatfiles are more suited for the situation than MySQL.

    When you need to reconfigure which plugins you use, you only need to open ONE file and modify ONE line, and that's really easy to do. Each plugin should be designed to have only ONE config file, otherwise keep it in the config file of Bukkit. Many people use flatfile because they can open the folder up and SEE what files they have, OPEN the files and SEE exactly what data is in it, rather than trying to come up with a line of SQL syntax just to view each piece of data, then another to modify just one line.
    I agree that MySQL is faster, more flexible, and more modern, but I disagree with your statement that it's easier to handle, unless it's through a web-interface (in which case the web-interface does all the work and it's easy to navigate). With the constant-width command-line interface you're given to work with, you're limited on what you can see easily (if you try to look at info longer than the width of the interface, it overflows onto the next line, making the database extremely hard to read).

    With your argument that Flatfiles are a source of exceptions, look at MySQL: won't it throw an exception and "spam your console with ugly throws" if you misform your SQL statement? Flatfiles only throw exceptions when the file doesn't exist, or exists in an area where hMod/Bukkit doesn't have access to it, or if you try to continue reading past the end of a file, and these are VERY easy to avoid. Much more so than the SQL exceptions. What if you put in a string instead of an int when writing the file? You don't, by controlling what info you write to the file, and if the server manager modifies it (kinda stupid on his part), that's what the conformity checks in your read method are for. When you're reading the file, you can easily check for conformity to your format for each line just through the conversions and then just skip over the line if it doesn't fit.

    Flatfiles may fill your folders with lots of txt files, but MySQL fills your database with lots of tables, and you may have to stop your server to reconfigure plugins whether you're using Flatfile or MySQL, just depending on what the plugin does. This is not a problem limited to Flatfile.

    You can easily write up a converter from an old flatfile format to a new one, probably easier than you can a MySQL converter.

    I agree that reading flatfiles all the time is bad for performance. That's why you don't. You only read/write when things change, and when you have something that changes frequently enough, you DO use MySQL.

    Finally, you speak of "professionality" when you can't even form proper sentences and use correct grammar. Maybe you should learn to spell before you talk about being "professional."
     
  16. Offline

    rayman86

    Read most of your post John, would be a bit easier on the eyes if it weren't one massive paragraph.

    Basically what I gathered I agree with. There are pros and cons for each method. What it boils down to is which one is 'better' or 'more efficient'. To decide this, I look at 2 factors:
    1. The frequency of read/write operations on the data
    2. The size of the data and possible growth of the data
    If the frequency is low and size is low (as well as growth rate), then flatfile is the more efficient method as it does not require the overhead that a DBMS (MySQL, Oracle, etc) software requires and will not cause performance hits as it will be, in most cases, an hit once or rarely operation. Mainly to me this means simple configuration information. If I simply need to see how the configuration for my plugin has been setup by the end-user, I will generally only need to read this information once and keep it saved while I am running. Only time I will need to write this is if it has been changed.

    Inversely, I would never use flatfiles for data that has the potential to grow large, rapidly, or will require I/O operations frequently. The overhead required for the DBMS would outweigh the load that performing these operations on a large flatfile (or often) would cost.

    And of course, if there are a lot of plugins running then the overhead tradeoff will decrease a bit. Obviously the best choice for which to use would be up to the server administrator, based of his/her setup and amount of plugins. Allow the option for both and let them choose.
     
  17. Offline

    croxis

    It sounds like we are talking about the two different definitions . A server is a computer program. That is it. A web server is a computer programs that serves web pages. A MySQL server is a program that serves and stores database information. We also happen to use the word server to talk about the physical machine that runs these programs.

    So a bukkit minecraft server can exist on the same machine as a mysql server, or be on separate machines on the opposite side of the world.

    How your BSM fits in relation to all of this is really up to you. If it is a wrapper it will have to be on the same physical machine as a bukkit server. If it communicates with bukkit through a plugin it does not.
     
  18. Offline

    rayman86

    Either or really. Java (i.e. the plugin, heck anything that connects to the database really) only really cares if it knows where to find the server. That could be localhost (on the same computer) or on another computer in the same network, or even on another network. You could even connect to multiple MySQL databases on different computers if you were so inclined. As long as you have the syntax in properly for the connection, it will work just fine.
     
  19. Offline

    AnonymousJohn

    Yeah, I'm kinda bad about that massive paragraph thing. Sorry about that. But in any case, I agree whole-heartedly with what you're saying here. Both have pros and cons, and the server administrator should be able to choose whether their plugins should run flatfile or SQL in most cases.

    EDIT:: went back and split up the massive paragraph to make it a little easier to read.
     
  20. Offline

    Afforess

    That's a pretty offensive statement. Your list is totally bogus, and false. Flatfiles or SQL databases are only as good as the programmer who creates them. You can have efficient, non-exception throwing flatfiles, and equally terrible, slow, and confusing SQL databases. I choose flatfiles because they are far more descriptive than SQL databases. Could you do this with a SQL database?

    Code:
    ---------------
    Setting: Pressure Plate Rails
    Value: true
    Description: Minecarts will treat wooden and stone pressure plate like they were regular
                 minecart rails, allowing for the creation of neutral intersections
                 and redstone triggers.
    ---------------
    
    That's what my settings look like. If you put garbage in, like 593jdjf as the value, it just ignores it. No exceptions. Only read once, and reloadplugin reload's the config just fine.

    And it is the business of the user to determine whether to use a programmer's plugin. People who write shoddy, broken code are easy to spot, and are avoided fairly quickly.



    I disagree. Freedom to screw up is an essential freedom to facilitate learning. If you baby a person every step of the way, how can they learn?​


    I understand your concern - basically you want programmer's to ensure, or be ensured that they aren't slowing down your server, wrecking havoc with your machine, and causing other issues. Instead of lambasting flatfiles, which aren't the root cause of this (merely a symptom), why not attempt to fix it? Create a high quality plugin standard, where plugins authors can post that their plugin meets XYZ standards, and their code get's reviewed to ensure that it is met; so that regular users know that this particular plugin won't be the source of problems. Perhaps that's not the right solution - then do something else - but flatfiles aren't the root cause here.


    I do. I believe it says "Failed to load settings for XYZ".

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 5, 2016
  21. Offline

    Sky.NET

    Nevermind, i am German, and as long as you understand me, its okay xD

    Haha are you kidding me? Tell me one (private) person whos administrating his whole MySQL database by commandline! Theres no one, every admin who is using MySQL uses PHPMyAdmin or sth. like that.

    And in that fact, its very easy to use MySQL too, just one click, and you have an overview about every table, if the plugintables are wellnamed, you dont get in problems with much tables, just name the tables 'plugin_pluginName_config', 'plugin_pluginName_logdata' and so on.
    If you click 2 times, you can see every value of a plugins settings.
    With a 3rd click you can change every setting.
    The 4th click is saving it, and its okay. Nothing to learn, nothing to do. Just how to login to PHPMyAdmin and use your mouse and keyboard.

    Many people use flatfile because they are scared from such an unknown and mighty thing like 'MySQL' they heard much of it, but still are scared and thinking 'oh that would be too diffucult to learn in 2 minutes', but there is nothing to learn. Just self-explanatory click and fill out.
    No one likes to have 3249856 textfiles, buried in 2394865 folders if they can have a simple interface that allows them to change everything.

    Lololol sorry for the laughting,but if u use a webinterface like PHPMyAdmin it will tell you 'hey you are using a wrong datatype here, please fill a [datatpye] xxx in'.
    And if the programmer does fails, yes of course it would spam my console, but thats not the buisness of the user using the plugin.
    I dont talk about controlling plugindata you write to file using java, i was talking about administrators knowing nothing about that buisness and using strings instead of ints or sth like that.
    My sight is most times the sight of the user, he is the one who is at the end of the programmingprocess and he would face all your bugs and codingfails you overlooked.

    My statement is a little bit to agressive, i know that, dont wanna hurt anyone with that statement.
    Afforess? I can add descriptions to MySQL tables/columns too, thats no argument you should use (or did you mean self-explanatory?).

    And you should remember its everytime the buisness of the coder what and how hes coding, but if hes dumb, he will screw up everything, YOU only load you file one time, thats not meaning that there arent dumb idiots out there who misscode anything to reload their config every fucking second.
    Addionally there are standarts, and you should even take a look at them if you arent a professional developer.
    You have to think about the whole thing in a gerneral way, not only from your point.

    Lets get back to the point 'source of exceptions':
    With flatfile most administratos have too much freedom, that means they can do what they want with the data in the files, and they can do too much wrong things, not every MC-Server admin is an IT-Specialist, you have to remind that.
    And much people coding plugins are newbies at programming, knowing nothing about OOP or clean coding strategies, using try-catch-finally or other things like that.
    Addtionaly if someone is coding a webinterface to administrate the MCS (maybe me?) there are serveral ways to get 100% sure the input is the right datatype AND is saved in the right place, the right way, with flatfile only, you are left alone with your 32487562345 textfiles.

    I would agree with 'if you have only 2 lines of config you can use a flatfile' but there are still much aruments against that. I dont like to use MySQL and flatfile @ one time together, mixing up everything is mixing up administrators heads and causes stress.

    Another question is, why do you use ur own ugly form of storing data in flatfiles, only readable for those who reverse your way of align data in files? You should use a standard like CSV or XML so other programmers who maybe wanna read your plugins configuration can do that in a wellformed and everywhere supported way otherwise its a brassiness.
    If someone wanna code a 'configurationhelper' and every plugincoder has his own standard to save his data, the programmer will get a headache with using regex soon.

    If you are IT-Professional or Softwareengineer you immediately should reconsider about clean coding. If not, you should do it too. Creating own standards is not our buisness.

    Aaaand why dont throws your plugin a message on the console like

    '[Pluginname] : [WARNING] Value XXX for setting YYY is incorrect -> shutting down plugin'
    or
    '[Pluginname] : [WARNING] Configfile is corrupted @ line XXX -> shutting down plugin'
    or
    '[Pluginname] : [WARNING] Value XXX for setting YYY is incorrect -> using standard-value of XXX'

    You leave the Serveradmin with nothing, just ur unusable plugin because of missconfiguration?
    Or with standartsetting (i would wonder why my setting dont work ingame, and am left alone with that question too)
    Thats not what i meant, i dont like pure javaexceptions spamming much times and screwing up my logfiles, but if i missconfigurated something, i want to get a clean short message what ive done wrong.
    Thats what i mean with clean coding, that is very important to be a good programmer!
    And if you dont wanna be a good programmer, stop that shit, go out and change your hobby to skateboarding or something else (or stop giving plugins to the community).

    Sometimes my text sounds agressive, dont wanna hurt anyone.
     
  22. Offline

    Apostledanub

    All I'm interested in is keeping my server running for a few friends and myself (which totals about 5 people). Programming plugins is far beyond me. Will I still have to learn MySQL if I want to use Bukkit?
     
  23. Offline

    Sky.NET

    People are coming and going, you cant see the future. You can spot out ONE person who is coding NOW ugly things, but not if someone will do that in future...

    Two questions:
    Is a Minecraftserver the right place learn how things are working?I dont think so.
    Do they WANT to learn anything or just keep up their servers and use nice funny plugins?I think so (most times).

    I agree, but thats not my buisness, the coders of bukkit have to take care about this (IMHO its not too late now, but if they dont do that, and release the mod, everything is lost)
    I dont lambast flatfiles themself, SQLite is a flatfile too, even CSV or XML, but ugly plaintextfiles, formatted by the coder himself... thats not what i at least call clean coding.
    Your are right, and somebody in this thread wrote, that data-storing has to be handled by bukkit, that would cut out every problem that could appear! Its the only 100% solution whitch will satisfy allmost everyone.

    More details please, if i change 20 of 50 settings and dont exactly remember what i changed, it will be a headache to change every possible value that could be wrong (in my case i just copy the settingsfile to have a working backup but not everyone is thinking so far).

    You dont have to learn anything but installing XAMPP (its just klicking a few times and pressing enter a few times) loggin in to PHPMyAdmin and using PHPMyAdmin (very simple webinterface where you dont have to know ANY mysql-command).
     
  24. Offline

    Apostledanub

    That sounds reassuring, thank you.
     
  25. Offline

    Sky.NET

    If u use Windows, you have to go to Control Panel, click on Services, and then search for the service 'windows process activation', doubleclick it, stop it there, and set it not to start with windows.
    Thats because its fucking with apache (it tries to launch the windows build in webserver everytime).
    Just a little tip to avoid you from googling that shit =D.
     
  26. SQLite is an excellent comprimise for all those saying DB is better or FF is better.

    for those who do not know about SQLite here is the short an sweet of it.

    - It is a database.
    - It uses SQL (some slight differences with certain niche aspects regarding dynamic types and such, nothign you need to worry about really).
    - It DOES NOT require a seperate PROCESS/SERVER, it is run from a library (jar in our case) that interfaces with a DB file for us.

    I understand the arguments against flat files, so I ask the bukkit developers this.
    Can you make a utility class for those just wanting simple configuration storage? Something that acts in a manner similar to this.

    Code:
    ConfigFile pluginConfig = ConfigFile("plugin.config.db");
    String allowSubset = pluginConfig.getString("configVariable");
    pluginConfig.setString("anotherConfigVariable","value");

    This would make it easy for people starting off, and preventing a massive amount of homebrew'd versions which increase server bloat and have varying performance/compatibility issues.

    I must stress though that such a utility class would NOT BE FEASIBLE for advanced plugins, such as a cuboid or chest protecter, these would need to use SQLite directly, thus also for the utility config class I would like to ask that it exposes the underlying SQLite related objects, letting advanced plugin developers to levy both simpleconfig (enable a section of functionality) as well as complex storage (storing permissions for blocks etc).
     
  27. Offline

    Ryan00793

    Just get navicat its very easy also get the lite version since its free
     
  28. Offline

    Nijikokun

    Forcing something on people is unfair, sure some mgmt is needed here.

    Don't start forcing your ideal database, ideal setup, or ideal features on someone just because it benefits you. Not everyone has a system like you, nor do they operate the same as you.

    Flatfile is there for a reason, SQLite exists for other reasons, MySQL exists for some. There are literally thousands of databases that I could list here. The point is, all cater to some idealistic reason one way or another and all you are doing is arguing over spilled milk.

    Instead of saying "This should be used" or "That" how about use an abstraction layer that allows you to choose which one and the plugin simply stores data in that method. Of course the abstraction layer should be in Bukkit and not in the plugins, this allows for the greatest control, with flatfile support being the alternative if you do not want SQL databases.

    Because I could come in here and go "MongoDB is the best." and you would all start on another tangent. Lets stop with this penis contest aye?
     
  29. Offline

    matejdro

    What about old data? Would you provide some method to convert old data in flatfile to mysql? Or everything like Stargates, homes, cuboids etc. will be lost?
     
  30. Offline

    Sky.NET

    I never sayed anything is the best, you fully missunderstood what i am talking about!

    I not take care about what a plugincoder has to learn to build good plugins, thats coding, you have to be good or to learn more bevor you release anything, dont put any user into your learningprocess, thats not fair.

    But thats what many plugincoders do, they are learning how to code, and push out their plugins on a state whitch is totally unacceptable, but it is acceptated by the most MCS running people because they need the functionality of those plugins.

    What it has to do with flatfile? Nothing directly, but allow to use bad formated 'i-do-what-i-want-and-dont-take-care-about-coding-standards' plaintextfiles is exactly the opposite of supporting coders to learn how to produce good code.
    Btw. SQLite IS flatfile-database, XML IS flatfile-database, CSV IS flatfile-database!!!

    That:

    --------
    blah=blah
    blubbblubb=123
    blabli=sadjfh
    #comment
    #comment
    dsfgdsfg=234
    sfgsdf=dfg
    ---------

    is NOT flatfile-database, thats useless nonsense, just trying to store data readable by humans anywhere and not caring about any standard. Using plaintext, not flatfile-database standarts is the reason i never will call that files a 'flatfile-database'.

    so what?
    learning how to use XML? too complicated in my opinion.
    learning how to use CSV? a good choice of mysql-haters (and its far not complicated)
    learning a few simple mysql-commands? a good choice of people who want to give out nice coded, performant and durable plugins.

    Look at the basic config files from hmod, they are all using the CSV standard (http://tools.ietf.org/html/rfc4180).

    Look at that drawing:
    [​IMG]

    That would be the ultimate and best way to handle ANY data used and touched by ANY plugins.
    The Plugincoders have not to learn anything new, the admins still can choose their favourite storing-system.
    Only the bukkit team has to implement some MySQL, SQLite and flatfile (XML/CSV) support in bukkit.
    Thats all, i have nothin more to say than this is just the perfect solution everyone should be happy with!
     
Thread Status:
Not open for further replies.

Share This Page