Large config file memory usage

When using the standard getConfig methods from the Bukkit FileConfiguration API, my understanding is that the entire file is read at server start up, and all values are cached in memory. How does this affect memory usage if a plugin has a very large config file?

My reason for asking is that I am considering storing UUID to Name mappings for a pre-1.7.5 plugin that I am converting to use UUID's, using SagaciousZed's ConfigAccessor class which I believe uses the same FileConfiguration and YamlConfiguration classes as the built-in version, but allows one to pass a filename instead of only accessing config.yml.

If this plugin were to be used on a very large server with tens of thousands or even millions of users, would it try to cache all of the entries from the file in memory? It seems to me that this could potentially cause a server to run out of memory at some point.

Again, the purpose is to create a flat file database with UUID as key and player name as value, so I can do name look ups of offline players from UUID, using bukkit versions prior to 1.7.5 that lack the player.GetName(UniqueID) method, without requiring an SQL data store.

As I write this post, I'm beginning to believe that implementing this using Sqlite is probably the best course of action, so that whether the server owner has an SQL database server available or not is irrelevant. Still, I'd like to hear comments on how big is too big when it comes to using yaml files with the FileConfiguration API. Comments on better ways of doing UUID's with older server versions are also welcome.

 

If you're using flatfiles for millions of users, you're going to have more problems than caching them in memory. Like read/write times...

If you have that many players, use a database.

 

I suggest writing your own UUID/name getter that sends a request to the mojang servers, if you want to support Bukkit versions that lack Bukkit#getOfflinePlayer(String/UUID). The exact information you need for sending these requests can be found here.

 

@Garris0n:​

No, I don't have millions of users, more like dozens. But I'd like this plugin to be usable by tiny servers and huge servers alike, with the least possible fuss on the part of the owner. Although I suppose it wouldn't be unreasonable to expect the very large servers to already have a database, and the technical chops to configure the plugin to use it. And those giant servers probably aren't running on older versions of bukkit, either. Still, I was kind of contemplating a one size fits all solution for the sake of simplicity.

Traks:

Ok, that looks simpler than I expected it to be. But I already have all the usernames that I am interested in, they are currently used as the keys in the data store. When the plugin initially converts these to uuids, I will need to query mojang's server, but that's a different problem, that I'm sure has been asked and answered. (But thanks for making me look at the mojang api, it is much simpler than I anticipated.)

I would still like to store the uuid->name mappings locally, as an offline cache basically. In this instance, I'm not overly concerned about the names becoming stale. My plugin will be using only uuids as keys internally (once the conversion process is completed), I would just like to be able to display a name rather than uuid to other players when necessary. I'll update the names whenever a player logs in again, if their name has changed.

Thanks for your feedback, I'm really just trying to flesh out some ideas on the best approach to using uuid's with older versions of the bukkit api.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.