[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    codename_B

    One comment I would make is thought the bandwidth dropped my CPU usage quadrupled with this plugin enabled - something to take into consideration.
     
  3. Yes , Do I haveto portforward to 25564 where do I change that ?
     
  4. Offline

    Soryo

    Sorry if I repeat information, I've been bouncing back and forth on my post and yours and searching online for info so it's a little scattered.
    Code:
    2011-05-03 16:45:20 [INFO] WARNING: reconnectfile parameter not set
    2011-05-03 16:45:20 [INFO] WARNING: players will be connected to the default server regardless of last server connected to
    this doesn't matter, right? I left them blank in the config file. And when it says default server, it means for multiverse, and similar plugins, right?

    I started to try to do some testing with this... but I'm not sure if it's working correctly or not. I can log in just fine, everything seems normal. I ran around for a little bit then logged out and saw there were no cache files in the folder I put the one that I use as an executable, or the one in my server folder.
    I'm not sure if it's actually connecting to it. What I did was...
    -put the .jar into my plugins folder.
    -copied it and pasted it into a folder on my desktop (CPL)
    -ran the one on my desktop which created a CPL_cache file
    -set the parameters as such...
    ServerName [my IP address] Port [20000]
    Local Server Name [localhost] Port [25564]


    So this means that the proxy (still not exactly sure what it is, but I have an idea) doesn't go to minecraft.net at all? Or is it that the Bukkit server becomes the proxy and they connect to minecraft.net for you? I executed the plugin and started it up and connected normally. Then I took out one of my password letters and it didn't connect so that theory seems out the window.
    I'm wondering if it's when the minecraft.net severs that are down, you can still connect through bukkit and play multiplayer... and it's just that it wouldn't connect because I did my pw wrong.
    I'm a little confused about this... (wishes I can draw diagrams of what's going on in my tiny head)

    And the backend server is the craftbukkit, correct? That was somewhere in a post earlier I think...
    These servers are the ones that we host? Not the authentication ones?

    I've tried to run the server and used the exectuble on my computer as well, but I haven't seen this value change... you said if it's blank it's just the default value, so it probably does the monitoring by itself? Is the executable supposed to have a config file as well?

    Ok, what I gather from this....

    Player --> CraftProxy --> MYCraftProxy --> My Server

    Does this mean I also have to run the CraftProxy for it to work? or is it just the one in the plugin file that opens it up? That's the one I'm assuming....
     
  5. Would anyone come to install it with Teamviewer lol
     
  6. Offline

    Cid

    Ok, again, I am getting this Unable to connect to backend server...

    I was told Joy had fixed it, but a phrase like... "I changed that and it worked" really doesn't tell a whole lot about WHAT was changed.

    Can someone please help me or is support for this error a lost cause? I have never setup a MC server but am familiar with coding, is there some property in the server settings that I have to change to get it to allow a connect from 127.0.0.1 and if so what is it?
     
  7. Offline

    Kane

    @Raphfrk Is there any way possible to ever add this to the client? Might break other mods though. Lets just say my players will never use such a sweet feature... But 90% of them do download something that should not be downloaded that's pre modded and if this was included and they did nothing but click their normal launcher...

    Lets just say then they would use it.

    Though it is Bandwidth though it might not be that needed. We got a 1gig line and really its the unloading and loading on the server side that causes lag hehe.
     
  8. Offline

    ShadeShadows

    I've tried everything to get this working on my machine. I want to be fully sure its working before I open my server to my group of friends (who regularly complain about lag)

    I got the 'cannot connect to backend server' error, fixed that, but now whenever i try to connect to localhost:20000 it says outdated server! and if i mess around with the settings in the gui I get a 'initial handshake fail' and then every error is 'only one connect per 5 seconds is allowed' changing everything back to default fixes this but gives me the outdated server error again

    Am I even doing this right? I set servername to my computer's IP (192.168.0.xxxxx) and the port to 25565

    then i set the port below it to 20000?

    and how will I explain what to do to my players? This really needs a proper tutorial with screenshots to get working without problems :/

    right now i went through and read this entire thread and still don't know how i can fix this properly. Would really like to reduce lag on my server.
     
  9. Offline

    Twisted

    In the gui you need to set your server's ip and port as 20000, then in minecraft you connect to localhost.

    If you're running the server and client on the same computer you need to change the local server port to something other than 25565 such as 25564. Assuming you changed it to 25564 you then need to connect in minecraft to localhost:25564
     
  10. Offline

    Cid

    [quote="ShadeShadows, post: 267357"
    I got the 'cannot connect to backend server' error, fixed that, /quote]

    Can you elaborate on how you fixed this? I am stuck at this error.

    Thanks
     
  11. Offline

    KrazyBonez

    I had it working, after much failure i finally had it working...for all of a half hour then everyone gets kicked. I restart the servers, no luck. I restart the proxy server, just to find out that when i try to connect to the proxy, it doesnt get anything, i time out in my client and there is no record of me even trying on the proxy command prompt. I didnt change anything when they all got kicked, i was actually eating. What can i do to fix it?

    Edit: Disabling and enabling my internet connection seems to have fixed it, odd, but fixed. Thanks anyways :D

    But i do got a question, can you give me an example of allowing someone to use a gate? what are the gate types? Obsidian? Sign?
     
  12. Offline

    thepackett

    just as a quick verification:
    the only thing players need to do after they download it is type the ip of my server under server name?
    they don't need to mess with the port number or anything?
    EDIT: and besides port forwarding i don't need to do anything either?
     
  13. Offline

    NotYetRated

    Testing on my server currently. In the morning will have some users testing. Currently showing:

    Data Saved: -432 kB (-9% reduction)

    Have been up about 5 minutes and not moved since log in.
     
  14. Offline

    Raphfrk

    Was this the first connect to the server with the cache?

    On the first connect, it doesn't have the data cached and it has to send hash code information, so it actually uses more bandwidth than normal.

    The next connect normally gives around 70%+ reduction.

    Also, it compresses chunk updates. If a player just stands in one place for ages, then no chunk updates are being sent, so it won't cause a reduction.

    It will give the best compression for players who move around a lot. These are also the players that would have caused the most bandwidth, so it reduces the bandwidth for high bandwidth players more than low bandwidth players (which is what you want).

    Right, that is inherent in the process.

    The problem is that I need to decompress the data, check for matches and then recompress it.

    No, you should just forward port 20000 to the computer the server is on. If you want players to also connect normally, then you should also forward 25565 to the computer the server is on.

    No port forwarding is required at the players' side.

    It doesn't matter, the proxy was originally made for teleporting between servers, so default server is the one new players connect to. If you set a reconnect file, it remembers the last server each player is on.

    See the first question on the FAQ :).

    The issue is that if you are running the plugin and GUI on the same computer, you have to change the connect port.

    - start the GUI
    - change the listen port from 25565 to 25564
    - press start
    - open minecraft client and connect to localhost:25564

    The GUI should say that you have connect.

    Note: this is not required for players who aren't on the same computer as the server

    They just need to:

    - start the GUI
    - enter the hostname/IP for your server
    - press start
    - open minecraft client and connect to localhost

    The proxy just acts as a simple pass-through.

    You connect to the Bukkit server and once login is completed, the proxies start doing the caching.

    The client checks your password when you start it. It never sends your password to the actual server.

    The login process is:

    - server sends you a random number, say 1234567
    - your client connects to minecraft.net, using your password/username and says "1234567 confirmed"
    - the server connects to minecraft.net and says "has <username> confirmed 1234567?"
    - If YES, then you are allowed to connect

    If the minecraft.net server is down, you can't connect to any server, unless it is in offline mode.

    Yes.

    Yes.
    The GUI and plugin are craft proxy.

    Also, I updated the OP, if you are on the same computer as the server, you need to connect in a slightly different way.

    Right, setting everything to default should just work.

    Also, I updated the OP, the connection method is slightly different for players on the same computer as the server.

    @ShadeShadows

    Are you on the same computer as the server, if so, can you try with the updated instructions in the OP. It should work with just default settings.

    I would probably have it so that it starts the minecraft client, once you press start.

    It would help players who have a low bandwidth connection, but MC doesn't really have that high a download speed. I would see this as useful for people who are hosting servers on non-dedicated lines.

    I am not sure what your hosting plan is. How much bandwidth does your server use. If you are charged by bandwidth, then this would represent a saving.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  15. Someone should post some screenshots or a tutorial because I give up . I can't do anything , this isnt clear for me
     
  16. Offline

    Raphfrk

    It should just work.

    I think the main problem was that people were testing it with a client on the same computer as the server.

    I would suggest you delete the config file in the /plugins/craftproxylite Directory and start again.

    It should work with the defaults. Just make sure you follow the instructions for


    Players (on the same computer as the server)
    * Download .jar file and double-click
    * Change the listen port from 25565 to 25564
    * Press start
    * Connect to localhost:25564 with their client


    If you don't do the above then it causes a port number collision.
     
  17. Offline

    codename_B

    Would this also be able to run in a full bandwidth transparent mode to speed up chunk loading only?

    I have LOADS of bandwidth so its not an issue for my but some of my players are on very slow connections.
    If I could send them copies of my map file (another request - use the current minecraft map format to save locally) then they could load those and have the other chunks load in the background and override the saved files as they load.

    This would eliminate the need for a hash check and probably even a server-side plugin, hence eliminating the extra CPU usage. (and though it wouldn't change the bandwidth usage at all, it would speed up chunk loading ZILLIONS client side.) All that would be needed would be the caching proxy running on the port.

    Also this would be extremely useful with that function - and I can see 100% of my players using this if that option was added - I don't know if that's outside the scope of this project but that would effectively eliminate client-side chunk errors and make the whole map loading process infinitely more seamless.
     
  18. Offline

    Raphfrk

    They would still need to know what the updates were since their last version.

    What I could do is calculate the hash of the compressed chunk. If the player already has that, then there is no need to compress/decompress the file.

    However, even 1 block or light level change in the chunk would make that invalid. I don't think there would be many hits.

    Chunk loading is throttled by the server. It would never be instant.

    I don't really want to be re-ordering the data stream. The cache isn't supposed to change anything, just reduce bandwidth usage.
     
  19. Offline

    codename_B

    What if the proxy was running in a layer where it loaded the chunks from disk before the new chunks were recieved by the server, then overrides the chunks on disk, if you see what I'm saying. You could distribute complete copies of your map to your low bandwidth users and increase their experience.

    Or does that count as re-ordering the data stream? Isn't that basically what your proxy does now minus the checks?
     
  20. Offline

    Cid

    I am not hosting the server locally. I have a server through a host.
     
  21. Offline

    Raphfrk

     
    Last edited by a moderator: May 14, 2016
  22. Offline

    Cid

    Code:
    2011-05-03 18:14:54 [INFO] Connection from 174.xxx.xxx.165/52130
    2011-05-03 18:14:55 [INFO] [6:14:55 PM] 174.xxx.xxx.165/52130 (cscheck): Connecting to: 25565
    2011-05-03 18:14:56 [INFO] [6:14:56 PM] 174.xxx.xxx.165/52130 (cscheck): Unable to open socket to localhost:25565
    2011-05-03 18:14:56 [INFO] [6:14:56 PM] 174.xx.xxx.165/52130 (cscheck): Unable to open connection to backend server
    This is the only thing that shows up in the log.

    I have entered x's in the IP for security.

    I also see this during server start up if it helps at all...

    Code:
    2011-05-03 18:33:34 [INFO] CraftProxy-Lite started in plugin mode
    2011-05-03 18:33:34 [INFO] Command line args: [25565, 23456, staticlocalhost, bridge_connection, quiet]
    2011-05-03 18:33:34 [INFO] Starting Craftproxy-lite version 40
    2011-05-03 18:33:34 [INFO] Name authentication enabled
     
  23. Offline

    HadrienM

    It says : Serveur Outdated.

    console log :
    Code:
    15:44:19 [INFO] Disconnecting [Hidden] [/87.66.92.225:61422]: Outdated ser
    ver!
    15:44:21 [INFO] Connection reset
    
    Plugin CraftProxy settings :
    Code:
    clientversion=
    bridge_connection=true
    reconnectfile=
    monitor=
    seed=
    cache_limit=
    default_server=25565
    quiet=true
    debug=false
    listen_hostname_port=20000
    password=
    auth_off=false
    log=
    banned=
    info=false
    dimension=
    staticlocalhost=true
    disable_flood=false
    
    Server settings :
    Code:
    #Minecraft server properties
    #Thu Apr 21 16:33:28 CEST 2011
    level-name=LaForteresse
    hellworld=false
    verify-names=false
    spawn-monsters=true
    plugins=
    online-mode=false
    spawn-animals=true
    pvp=true
    server-ip=192.168.2.2
    max-players=20
    level-seed=
    server-port=25565
    allow-flight=false
    white-list=false
    spawn-protection=5
    
    GUI CP settings :
    Code:
    connect_hostname=XXX
    listen_port=25564
    connect_port=25565
    
     
  24. Offline

    Raphfrk

    You are connecting to the wrong port with the GUI, see below.

    Ok, these are defaults. The plugin will listen on port 20000.

    You need to update the default_server, since you specify an IP in server.properties.

    The server-ip is non-default, it is normally blank.

    You have changed connect_port, it should stay as 20000.
     
  25. Offline

    Cid

    This might also help you out...
    Server.properties

    Code:
    #Minecraft server properties
    #Sat Apr 30 07:22:32 PDT 2011
    level-name=world
    hellworld=false
    spawn-monsters=true
    online-mode=true
    spawn-animals=true
    max-players=8
    server-ip=207.210.252.12
    pvp=true
    level-seed=
    server-port=23456
    allow-flight=true
    white-list=true
    spawn-protection=16
    auto-heal=default

    Code:
    proxylite.txt:
    
    clientversion=
    reconnectfile=
    monitor=
    seed=
    cache_limit=
    default_server=23456
    quiet=true
    debug=false
    listen_hostname_port=25565
    password=
    auth_off=false
    log=
    banned=
    info=false
    dimension=
    staticlocalhost=true
    disable_flood=false
    
     
  26. Offline

    HadrienM

    Thank you, i will try with these settings :)

    Hum, in your post, can you hide the connect_hostname= in GUI CP settings ? (quote). Noticed i haven't hide it :p

    EDIT :

    Settings changed like you said, restart the server, and i connect on
    localhost:20000

    console log :
    Code:
    16:10:35 [INFO] Connection from 87.XX.XX.225/62061
    16:10:35 [INFO] Disconnecting due to connect flood protection
    16:10:36 [INFO] [16:10:36] 127.0.0.1/62057 (hidden): Client sent bad log
    in packet - (Id = null)
    16:10:36 [INFO] [16:10:36] 127.0.0.1/62057 (hidden): Closing connection
    to server
    16:10:36 [INFO] [16:10:36] 127.0.0.1/62057 (hidden): Closing connection
    to client
    16:10:36 [INFO] [16:10:36] 87.XX.XX.225/62059 (hidden): Client sent bad
    login packet - (Id = null)
    16:10:36 [INFO] [16:10:36] 87.XX.XX.225/62059 (hidden): Closing connecti
    on to server
    16:10:36 [INFO] [16:10:36] 87.XX.XX.225/62059 (hidden): Closing connecti
    on to client
    
    Clients says :

    Only one connection blabla 5 seconds (flood)

    Must i disable flood in plugin settings ?

    Sorry for inconveniance, i'm trying from yesterday :$
     
  27. Offline

    Raphfrk

    Have you updated server.properties:

    server-port=23456

    ?

    No, you connect on localhost:25464.

    In fact, better to update the GUI config to:

    connect_hostname=minekingdom.no-ip.org
    listen_port=25565
    connect_port=20000

    and then you can connect with the minecraft client to just localhost

    the GUI connects to the plugin on port 20000

    Unless you are spamming to connect every 5 seconds, it shouldn't matter.

    Probably something else is wrong.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  28. Offline

    Cid

    Yes, I edited my previous post to show both server.props and proxylite.txt. I have the server port set to 23456 in the server.props file.
     
  29. Offline

    Raphfrk

    In the plugin config file change

    Change
    default_server=23456
    to
    default_server=207.210.252.12:23456
     
  30. Offline

    Joy

    Been running all day now, several players using it. Seems to work, but not sure how effective it is, I can't really feel chunks appearing faster. Also, the measurement must be off, I often have it showing minus reduction on the client, same for others :p
     
  31. Offline

    Raphfrk

    There is an increase in size when you first load a chunk, since hashes have to be transferred.

    It probably happens when you move to a place you haven't been since you started caching.

    How much data is stored in your CPL_cache directory?

    Also, it doesn't actually speed up chunk loads. The minecraft server throttles them.

    I might see if there is a way to throttle that effect.
     

Share This Page