[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    Simanova

    iam running craftproxy via screen - so i get not output - perhaps you should develope a logfile for v120

    Error while opening a doublechest filled with misc stuff ( running with debug man parameter )

    [​IMG]

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  3. Offline

    cind3r69

    I totally posted but it disappeared, go figure. I also run via screen but I captured the info while in the screen session (screen -x SCREENNAME). Additionally here is a large sample of the "respawn" issue when collected via screen :

    Item stack array to large
    ... <- alot of item stack messges.
    Item stack array to large
    [7:42:51 AM] my.ip.addr/49193 (cind3r69): EOF reached
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    [7:42:52 AM] my.ip.addr/49193 (cind3r69): Closed connection to server
    [7:42:52 AM] my.ip.addr/49193 (cind3r69): Closed connection to client
    Connection from my.ip.addr/49198
    [7:43:03 AM] my.ip.addr/49198: Forwarding ping
    Connection from my.ip.addr/49200
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connecting to : remote.server.ip 255
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Attempting to connect to: remote.server.ip
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connection successful
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connecting using proxy to server co
    [7:43:06 AM] my.ip.addr/49200 (cind3r69): Server login successful
    Unknown packet Id 63
    Unknown packet Id 63
    b 40 4e d9 99 99 80 0 3 40 51 80 0 0 0 0 0 40 51 e7 ae 14 80 0 0 bf e6 66 66 60
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Upstream link
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Unable to read packet
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Packets: [11, 11, 11, 11, 11, 13, 1
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Timeout
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Closed connection to server
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Closed connection to client
    Item stack array to large
    Item stack array to large
    ..... <------------- just here to indicate alot of this message.
    Item stack array to large
    Item stack array to large
    [7:42:51 AM] my.ip.addr/49193 (cind3r69): EOF reached
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    Item stack array to large
    [7:42:52 AM] my.ip.addr/49193 (cind3r69): Closed connection to server
    [7:42:52 AM] my.ip.addr/49193 (cind3r69): Closed connection to client
    Connection from my.ip.addr/49198
    [7:43:03 AM] my.ip.addr/49198: Forwarding ping
    Connection from my.ip.addr/49200
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connecting to : remote.server.ip 25565
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Attempting to connect to: remote.server.ip:25565
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connection successful
    [7:43:05 AM] my.ip.addr/49200 (cind3r69): Connecting using proxy to server connection format
    [7:43:06 AM] my.ip.addr/49200 (cind3r69): Server login successful
    Unknown packet Id 63
    Unknown packet Id 63
    b 40 4e d9 99 99 80 0 3 40 51 80 0 0 0 0 0 40 51 e7 ae 14 80 0 0 bf e6 66 66 60 0 0 c 1 a 1 a 1 0 90 a7 38 78 0 a2 c9 be 2 0 9 e 8d 1a 0 fb d3 f5 d7 0 24 6f e8 fb 9 0 1 0 0 80 c4 5a b7 47 1a f4 1a 68 0 9a 73 *63* 1f
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Upstream link
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Unable to read packet
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Packets: [11, 11, 11, 11, 11, 13, 11, 11, 11, 0, 11, 10, 10, 0, 0, 0, 0, 0, 9, 104]
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Timeout
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Closed connection to server
    [7:43:14 AM] my.ip.addr/49200 (cind3r69): Closed connection to client
     
  4. Offline

    Raphfrk

    Not sure, are you running the proxy on a different computer to the one you are logging in from?

    I guess I should add two flood protection counters.

    Anyway, I updated the respawn packet, they had the length wrong on the wiki.
     
    cind3r69 likes this.
  5. Offline

    cvxx7q

    Is this akin to the tcpbufferv4.jar thats been floating around since minecraft's early days?
     
  6. Offline

    Raphfrk

    No. I think what that does is buffer up one or more packets into larger TCP packets?

    This actually modifies the data going to the client. Your players run a mini proxy on their computers that reconstructs the proper packets, using info from the cache.

    The effect is that your server needs to send them less info. Instead of sending all the chunk data, it can sends references to data stored in the cache.
     
    cvxx7q likes this.
  7. Offline

    Simanova

    Testing CP v.120 atm
     
  8. Offline

    AVirusC

    Ok, with the newest version of the Proxy I am getting this error on the client:
    Code:
    Internal exception: java.io.IOException: Received string length longer than maximum allowed (278 > 16)
    
    It happens when I enter the portal to server2. Both servers are on the same host. I have 2 instances of the proxy running, 1 for each MC server. Is this necessary for portals to work? Or do I only need the proxy on the primary MC server? No error appears in the proxy log so I'm not sure what is happening.

    Clients are able to join the both server via their respective proxies on 25565 and 25567. Its just I get the error above when attempting to utilize the portal.

    My configs are below:

    Server 1:

    Server Port Core:

    Code:
    normal_worlds=world,world2,skylands
    block_inv_transfer=creative,smp
    nether_worlds=world_nether
    global_hostname=localhost:25565
    
    CraftProxyLiter:

    Code:
    clientversion=11
    bridge_connection=false
    reconnectfile=
    monitor=
    seed=
    cache_limit=
    default_server=localhost:20000
    quiet=false
    debug=true
    listen_hostname_port=25565
    password=
    auth_off=true
    log=craftproxy.log
    banned=
    info=true
    dimension=
    staticlocalhost=true
    log_time_off=true
    disable_flood=true
    whitelist=
    
    Server 2:

    Server Port Core:

    Code:
    normal_worlds=smp
    block_inv_transfer=creative,smp
    nether_worlds=smp_nether
    global_hostname=localhost:25567
    
    CraftProxyLiter:

    Code:
    clientversion=11
    bridge_connection=false
    reconnectfile=
    monitor=
    seed=
    cache_limit=
    default_server=localhost:20001
    quiet=false
    debug=true
    listen_hostname_port=25567
    password=
    auth_off=true
    log=craftproxy.log
    banned=
    info=true
    dimension=
    staticlocalhost=true
    log_time_off=true
    disable_flood=true
    whitelist=
    
     
  9. Offline

    Simanova

    Ive found a trick:

    i start Craftproxy 5 times, every time a different port for every user.

    So data cant collide within a port.
     
  10. Offline

    Raphfrk

    This should be fixed now.
     
  11. Offline

    Simanova

    Testing v 121 atm - :)

    PS: My players are getting disconnected, if the server connection ( upload ) is more than 90% used.
     
  12. Offline

    Raphfrk

    I don't think I changed anything, unless Notch added some extra timeouts ... is there any info in the logs?
     
  13. Offline

    AgentKid

    @Raphfrk Awesome job getting it updated to 1.8, but from what I can tell the anti-flood protection is taking pings into account too, causing everyone who's client has pinged within the last 5 seconds or so not to be able to log in. Any way you can take a look at that?
     
  14. Offline

    Raphfrk

    I can't test this locally, but assuming I haven't coded it wrong, it should work now.
     
  15. Offline

    The PC Tech Guy

    How does this work compared to Spout? I thought it had some type of chunk caching itself.
     
  16. Offline

    Raphfrk

    Spout does pretty much everything this does.

    The advantage of this is that you don't need to modify the client at all.

    However, assuming your players are willing to use Spout, then that seems an easier solution. Otherwise, they need to have use the local proxy.

    I wrong the caching system for Spout and it is almost identical to this one. It is potentially slightly more efficient.

    Also, the 2 systems are compatible. If you connect with a Spout client to a Spout enabled server, then the spout compression is used instead.
     
    The PC Tech Guy likes this.
  17. Offline

    AVirusC

    Ok, that error is gone. But I'm getting:

    Code:
    [CraftProxyLiter] Protocol stream closed
    
    And now I'm seeing this in the log:

    Code:
    local.ip/52647 (AVirusC): Connecting to : localhost 20001
    local.ip/52647 (AVirusC): Attempting to connect to: localhost:20001
    local.ip/52647 (AVirusC): Connection successful
    local.ip/52647 (AVirusC): Connecting using proxy to server connection format
    local.ip/52647 (AVirusC): Server login successful
    local.ip/52647 (AVirusC): Kicked with: [Serverport] You have teleported, please connect to : localhost:25565
    local.ip/52647 (AVirusC): Redirect detected: localhost:25565
    local.ip/52647 (AVirusC): Closed connection to server
    local.ip/52647 (AVirusC): Connecting to : localhost 25565
    local.ip/52647 (AVirusC): Attempting to connect to: localhost:25565
    local.ip/52647 (AVirusC): Connection successful
    local.ip/52647 (AVirusC): Connecting using proxy to server connection format
    local.ip/52647 (AVirusC): Server login successful
    local.ip/52647 (AVirusC): EOF reached
    local.ip/52647 (AVirusC): Unable to flush output stream
    local.ip/52647 (AVirusC): Closed connection to server
    local.ip/52647 (AVirusC): Closed connection to client
    
     
  18. Offline

    Simanova

    This problem exists since CP version 60 or lower.

    I think its minecraft related. If the client cant get enough data the connection will close.

    I dont know if it is possible to pretend a stable connection via CP.
     
  19. Offline

    Raphfrk

    Ahh, it could be a timeout. That is a little tricky since I want it to timeout when necessary.

    Does it happen every time you reconnect?

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  20. Offline

    Simanova

    Could you increase to timeout to 60 seconds or more?
    Or make it adjustable via mandatory parameter. :)
     
  21. Offline

    AVirusC

    No, this only happens when entering a portal from one server to the other. If you enter the portal to server 2 and you have the reconnectfile in place and reconnect to server1 after the error, you login to server2 via the proxy. No problem. Enter the portal on 2 to go to 1 and get before mentioned error. Reconnect and you are on server 1 as intended.

    I also noticed this error on the server log on server 2 after attempting to portal there from 1:

    Code:
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Failed to verify username!
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    10:11:02 [INFO] Disconnecting AVirusC [/127.0.0.1:61664]: Protocol error
    
     
  22. Offline

    Raphfrk

    By reconnect, I meant teleport between servers. It fails every time ?

    You need to put both servers into offline mode. Make sure you have authentication switched on for the proxy, i.e., don't use the auth_off parameter.

    The proxy will then handle authentication.
     
  23. Offline

    AVirusC

    Yes it did fail every time.

    Ok so I put both in offline mode and turned the proxy on server 1 to use auth and it works.
     
  24. Offline

    dorkstitch

    Greetings,

    I seem to be having the same issue as AVirusC. Auth fails on the second server with bridging in place with the server handling auth. It seems to be sending a wrong serverid on teleport, which actually fails as NO on the auth server.

    I need both of my servers online mode due to mcbans. How would you link two servers together with bridging and have it auth properly?
     
  25. Offline

    AVirusC

    Yes, I have multi-server teleporting enabled now but mcbans doesn't function. =/ I was about to ask the same thing. I need the server to be able to auth. =)
     
  26. Offline

    dorkstitch

    It seems bridging does not work properly when teleporting. It works fine when connecting directly to the proxy of each repspective servers.

    I put the second server as proxy auth, but then once i am on my second server, I cannot get back to the main server. (Failed to verify username) because the main server uses bridging to handle auth and is getting a wrong serverid in the auth string.
     
  27. Offline

    Simanova

    Is this the part where the timeout is managed?

     
  28. Offline

    Raphfrk

    You should have 1 proxy for both servers (assuming they are on the same LAN).

    Yeah, I updated the code. However, I am not sure if it will help much, the server and client have hard coded 30 second timeouts.

    If you want to teleport between servers, you need to set both servers to offline mode and have the proxy handle the authentication.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  29. Offline

    AVirusC


    Or force Spoutcraft on your players. =) (which I am honestly considering at this point)
     
  30. Offline

    Simanova

    ok ill test it

    Is it possible to add an update function for the client gui?

    [​IMG]

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  31. Offline

    omnija

    if i could make 1 request, would it be possible to make it so it can minimize into system tray?

    also my current size {MB} is always 0 and buffer latency is always 0 when using it on the client side.
     

Share This Page