Persistent Chunks

Discussion in 'Bukkit Discussion' started by legendblade, Jan 4, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    legendblade

    This is mostly just a random thought that was floating through my head today, but...

    Something that I was discussing in the hMod forums within the last week would be a way to keep certain chunks always loaded (either through hMod's loadChunk or by faking the server into thinking that there's a player in the area to send data to). With Bukkit, if we could set up some way to achieve this goal, it would be really very useful; chunks that contain areas that always need to be active (such as minecart stations) or heavily trafficked areas could be specified to always be loaded in the server memory.
     
  2. Offline

    AnonymousJohn

    Just based on what I've seen so far, this would actually be pretty easy to do now with Bukkit (they have an onChunkUnloaded hook!! :D). I'll get on that. As soon as I finish my Chem homework (curse you school for intruding on my winter break!)
     
  3. Offline

    Raphfrk

    It would be interesting to find out what the actual load/unload algorithm is for the game.

    Presumably, when the server needs more memory, it unloads the chunks that haven't been accessed in a while.

    If a chunk is "touched", it then gets moved to the front of the queue. You would then need to figure out what counts as touching the chunk.

    Checking with isChunkLoaded means that the chunk will be unloaded every so often.
     
  4. Offline

    AnonymousJohn

    That's why there's an onChunkUnloaded hook. :p
     
  5. Offline

    legendblade

    Glad to hear about the onChunkUnloaded hook; that would make something like this a breeze, I'd think (especially if we're allowed to tell the server "no" :)).

    Oh, and John, if you've got other things you need to do, don't worry; once I get a bit more time myself, I can probably handle this type of plugin no problem. It was just a pain back in hMod when I was thinking I'd have to deal with the obfuscated code of the core server to accomplish what I wanted. (still appreciate the help)
     
  6. Offline

    AnonymousJohn

    No no, it's fine. I find time. In fact, I have a prototype going right now:
    http://dl.dropbox.com/u/17716587/ChunkReload.jar
    Mind you, I haven't tested it yet. I just know it runs. Current commands:
    "/setchunkreload [radius]" - sets the chunk the user is in and chunks within the given radius to automatically stay in memory. If you don't give it a radius value, it defaults to 5. Note that the radius is in a rough circle, and not a square.
    "/unsetchunkreload" - takes the chunk the user is in and chunks within the previously given radius out of the reload loop.
    "/listreloadchunks" - lists the chunks you've set to reload in the format "chunkx chunkz radius^2"
    "/gotochunk <x> <z>" - puts you at block (0,*highest*,0) in the chunk with the give x and z coordinates.
    Note that all chunk coordinates are relative to chunks, and not the same as block coordinates. Also note that it doesn't load the chunks listed in the list of reload chunks on startup, due to the fact that there is currently no way to do that. That's the point of the gotochunk command. I need to make the listreloadchunks command use pages, but that's for later.
     
  7. Offline

    Valrix

    I like this idea and it would work great for minecart tracks too. You could set it up where when you hit a sign or something the whole track is loaded so your trip down the track is at a constant speed. Unless I'm not understand this correctly, haha.
     
  8. Offline

    legendblade

    Thank you. I'll have to get a Bukkit server set up sometime soon and test it out. :)

    At the moment, I'm not really playing around with Bukkit and trying to get a more general... login server, if you will, set up so that my players can connect to one IP/port and get handed off to the proper server as defined by a database. So far, I've gotten Minecraft to send the login server the account name, and it connects and starts sending data back and forth to the actual game server, but after a bit the client goes boom. :(
     
  9. Offline

    AnonymousJohn

    Uh ohs! I've always wanted to do something like that. If you got it working you could get a ton of rep on the Minecraft forums, as it would make multi-world so much easier to implement (even though technically it's multi-server). If you did it right, you could make a plugin or something that sent the login server a message to switch the player's server. Then just link inventories between the servers and voila! Inter-world travel from within the game is now possible! :D so cool!
     
  10. Offline

    legendblade

    I'm thinking, without making the login server do -too- much, at the very least, all you'd have to do is get booted and relog and you'd be in the other server.

    On the other hand, if I did go into more detail, I could probably convince the login server to cache the player's original login information to hand off to the second server, then let the second server pass back the initial terrain load... *ponder*

    But... that's assuming I get the client->login server->actual server to play nicely just in my basic tests. I need to look more into Mineserver / the obfuscated code to see if there's something I'm just missing in passing the data back and forth (possibly the login server is mangling one of the bytes, or just not flushing in the right place...)
     
  11. Offline

    AnonymousJohn

    I wish you luck with your login server, and hope to see it working soon. I don't have the time to search through the obfuscated/Mineserver code to figure out how the packets are sent, nor the time to write a server like that. Too much school, not enough time.
     
  12. Offline

    Valrix

    That's a really cool idea. Though, wouldn't it need a client hack in order to get it to work? Well, I guess unless you had a login server with portals or something that when it gets activated would log the user out, but not actually boot them to the client's log out screen and move them to the server they want to go to. Sounds pretty complicated, I wish you luck.
     
  13. Offline

    legendblade

    Thanks. I hope I can get it to work (preferably by this weekend, lol).

    I'm not sure if it'll require a client hack or not. Basically, the way I'm doing it is the client opens a connection to the login server as if it were the real server. The login server then opens a secondary connection to the target server. Now, unless I'm mistaken, the login server can probably close and open a second connection, and, if it passes the right info to the server, and silences a few of the server's responses, it may be possible to change servers without the client being any the wiser.
     
  14. Offline

    Valrix

    That should work as long as the client never gets a response telling it that the connection has been closed. I think that will only work though if the client allows for multiple simultaneous server connections. All it would need is to add the end server's connection to the client just before the login server closes it's connection and all should work out just fine.
     
  15. Offline

    AnonymousJohn

    I think actually it works like this: (S is a minecraft server, L is the login server, C is the client)
    Code:
    S S  S
    \ | /
      L
      |
      C
    
    Basically the client connects to the login server, which sends stuff it receives from the client to whatever server it's supposed to send it to, and then sends stuff it receives from the servers to the client. To switch worlds, tell the server the player's currently logged onto to log off the player, but don't send the log-off packet to the client (so the client still thinks it's connected). Instead, send a new "log on" packet to the server you want to reconnect to and start sending the packets from that server back to the client.
     
  16. Offline

    Valrix

    That could work, unless the client doesn't allow connection to a new server while currently connected to one which is where the problem would be. So the only way to know for sure is to test it out and if it works, then that would be really freakin' cool, haha.
     
  17. Offline

    legendblade

    Yup, that's the thought. :)

    I think I finally found the right set of input / output streams to use after scrapping all the old code and starting over (properly, instead of just a proof of concept). Now I just have to finish writing the base classes before I can test the basic functionality to see if that crashes the client again. :eek:

    ... but first, sleep.
    --- merged: Jan 6, 2011 6:01 AM ---
    The client would never know. :)

    After reading the protocol documentation, as long as the login server silences like... two or three of the real server's responses to client disconnect / client connect events, it'll seem to the client as if nothing has changed (except every block around the client).

    Of course, the other main issue, once all is said and done: cross-server chat / etc (and inventory).
     
  18. Offline

    Valrix

    Nice. Cross-server chat shouldn't be that bad, all you need to do is have a chat server running and push both servers to the same chat server. I made one in PHP that's easy to use, though you'd have to be running OS X or possibly Linux in order to use it. Inventory can also be done pretty easily on OS X by using rsync, I did it as a test and it worked pretty well, the only problem being that I believe the player's location on the map is also saved to their .dat file which causes problems when sending that between servers, so unless you can find a way to edit out the players location and only bring in inventory you're going to have problems. My friend tried to figure a way to do this in Windows but didn't have much luck.
     
  19. Offline

    legendblade

    Just for my own purposes, most of the problems with inventory / player location is solved quite easily - I run all my servers off of Linux, and thus I can move the player storage off to one server and use NFS to link to it from all of the other servers.

    Now, the reason why this works for me is that all of my worlds will be variations on the main world (the nether and - eventually - aether, will look exactly like the main world, but with certain block types switched out for others). Thus, when a player goes between servers, they're supposed to be in the same spot they left from. :)

    Of course, eventually, if I get this all working, I'd make sure to include other ways of doing this so that people who don't have my exact circumstances can benefit.
     
  20. Offline

    AnonymousJohn

    I find it kinda funny how off-topic we've become. Started with a simple idea for a plugin, then went to the idea of a login-server that acts as a go-between to allow the client to switch between servers smoothly, effectively creating multi-world environments.
     
  21. Offline

    legendblade

    Well... it's okay since I was the one who both created the thread and derailed it... right? <.<

    I managed to get a very very limited version to work - only problem is that right now it's slowish (playable, but slow) due to having to send / receive one byte at a time (variable packet lengths without a real delimiter... fml). Just reading up on the protocol now to try and figure out how to get around that problem.
     
  22. Offline

    Raphfrk

    I was actually looking into that too (well since yesterday :)), as part of Server Port development. The current version just implements a pure passthrough.

    Have you looked at this site?.

    I was reading about the authentication system.

    The name verification process is roughly

    the server sends the player a random number (server hash)
    the player sends that number + username + password to the main minecraft site
    the server sends the number to the main minecraft server site and the site says that the real player has confirmed that number

    I think you would have to disable this process on all your servers as your proxy isn't going to be able to get the username/password from your players.

    You could then have your proxy perform the name verification process and maybe fall back to password when the MC servers are down.

    My thoughts on the reconnecting are that you would have to track all entity creation packets and chunk load packets.

    When the player disconnects, order the client to unload all chunks and destroy all entities. You connect to the 2nd server and the first thing it sends is new chunk + entity data.

    I think what it would look like to the player would be a white out and then new chunks arriving. I was thinking that what I would actually do is place the player in solid rock in some far off chunk and then add an additional unload packet at the end of login sequence.

    One issue would be a normal world to Nether jump. I am not sure if you can change server type on the fly. What would be worth testing would be what happens if you send the client a 2nd login packet. Does it disregard it, crash or update the world variable.

    Want to create a new thread, as this is interesting in its own right :) ?

    There is a way to check how much data has been received by the socket. You can then read just that number of bytes and it is guaranteed not to block.

    My bridge class works roughly as

    while( EOF not reached ) {
    -- while( data is available ) {
    ---- check how much data is available
    ---- read that much data
    ---- write that to the output
    ---- flush output
    -- }
    -- read one byte (this blocks)
    -- write that byte
    -- flush output
    }

    This means that it reads data from the socket as it arrives, but can read in blocks. Also, there is no delay between reading and writing.
     
  23. Offline

    legendblade

    @Raphfrk If I could split the topic from that post, I probably would. Might just ask a mod to do that.

    As for the rest;

    Great site; I've been using it pretty much since I started all of this.

    That sounds like a good plan for that part.

    I'm not entirely certain if that part's necessary, mostly because of this line from packet 0x33:

    That makes me think that if I just allow the second server to send the normal startup events, it will (hopefully) just replace everything in those chunks.

    On the other hand you are correct, I will probably have to destroy all entities manually. :(

    Ah hell.. that is part of the login packet. XD I guess I will have to send that to the client.


    I managed to come to the same conclusion last night right before I went to bed; works (fairly) flawlessly right now. Still have to test it under more server load, though. Then this weekend will be spent putting together my server cabinet (yay heavy 3 and 4U servers... >.<)
     
  24. Offline

    Raphfrk

    Also, you would have to make sure that your servers only accept connections from your proxy. Hmm, does the default server allow an IP whitelist?

    My concern is that data will remain in the clients memory of chunks that are far away from its current location.

    I couldn't find a way for the client to request chunks form the server. It seems like the server handles memory management for the clients, which seems backwards.

    I was thinking about that. The main difference is the colour of the sky. Since in Nether, you can't see the sky (normally), then it might make no difference.

    I am not sure if sounds effects are local. If the client thinks that it is in the normal world (but with nether blocks + entities), you wouldn't get Nether specific sound effects etc.

    However, with Aether, it is probably all sky?
     
  25. Offline

    legendblade

    I was actually already thinking about that. I was considering having the servers set up to require basic password authentication (which only the proxy would know the password), and be behind a firewall (only allow incoming connections from the proxy's IP).


    I think the client only retains chunks that are within it's view distance.


    True; for my world, I'd need to do the sky color change, though; it's all open air in my "nether".
     
  26. Offline

    Raphfrk

    I have created an initial version of this.

    All it does is act as a passthrough.

    However, it decodes every packet and then resends it, rather than just acting as a byte passthrough.

    If a player is kicked it will display the reason to the console.

    Source
    Jar Releases
     
  27. Offline

    legendblade

    ... of course you would, right when I'm getting everything set up to test mine. ;)

    Looks like you have a bit more done on yours, though - I still have to get the packet decoding done.
     
  28. Offline

    Raphfrk

    Sorry :).

    In fairness, I think you have larger plans for yours.

    All I want mine to do is allow people to reconnect between servers.

    One of the issues with using the proxy is that it requires all servers to be local.

    I am not sure I want to encourage people to use the plugin only for linking servers that they own. However, I think that is what it is mostly used for.

    The Packet Class has 2 arrays, one Class[] and one Object[].

    The class list gives the types for the fields and the object list is the actual objects.

    This means that I could just list what was in each packet.

    I had to add a few custom types to handle the variable length fields.

    String (has to be converted to/from UTF-8 and the first short gives the length)

    IntSizedByteArray
    IntSizedTripleByteArray
    ItemArray
    ItemElement
    MultiBlockArray

    This is the packet configuration list (with the middle snipped from Packet.java), which may have errors. I connected 2 clients to the proxy and I didn't get any unknown packets.

    Code:
            packetTypes.put((byte)0x00, new Class[] {});
             packetTypes.put((byte)0x01, new Class[] {Integer.class, String.class, String.class, Long.class, Byte.class});
             packetTypes.put((byte)0x02, new Class[] {String.class} );
             packetTypes.put((byte)0x03, new Class[] {String.class} );
    
    
    <snip>
    
    
            packetTypes.put((byte)0x6A, new Class[] {Byte.class, Short.class, Boolean.class});
             packetTypes.put((byte)0x82, new Class[] {Integer.class,  Short.class, Integer.class, String.class, String.class, String.class,  String.class});
             packetTypes.put((byte)0xFF, new Class[] {String.class});
    
     
  29. Offline

    legendblade

    Mmm... maybe; depends on how adventurous I feel. :)

    Yeah; I develop stuff for a corporate data security and asset recycling company, so, I've picked up 8 servers, a cabinet, and some misc other equipment.

    Right now, I'm setting up a server each for: the login server, the main world server, the nether, the aether (eventually), the web server, the database server, and another server that's just dedicated to rendering map updates (Brownan's overviewer ftw).



    Thanks for the help. Even just getting your opinion on all this has been a huge help. :)
     
  30. Offline

    Raphfrk

    Actually, has the standard client/server actually implemented password protection?
     
Thread Status:
Not open for further replies.

Share This Page