[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    rtcabooservb

    :confused: So I can remove craftproxy now to just use spout?
     
  3. Offline

    Raphfrk

    Both actually, it caches chunks and also allows auto-redirects.

    I wasn't planning to add more features to this plugin anyway, but may have time to keep it from breaking.

    Spout is also a more efficient solution. CraftProxy has to decompress and then recompress all packets and also parse the protocol stream. With Sprout packets can be processed before they are compressed by the server and after they are decompressed by the client.

    Also, you don't have to explain to your users how to setup the local proxy server, they just connect with the Spout client.

    Well, once the they actually release the recommended build.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    thomasbomb likes this.
  4. Offline

    Takel

    Hmmm, one of the things I like with my current configuration is that I only need to expose the CraftProxy Liter port and all the connections are then routed locally. I might need to test to see if Spout can pass off the server connections using Transporter.
     
  5. Offline

    Raphfrk

    Actually, thinking about it, another benefit is that you don't have to re-authenticate every time you change servers. If you were connected to a server and then minecraft.net went down, you could still reconnect between servers.
     
  6. Offline

    Simanova

    Using CP 107 and CB 1046

    Code:
    # ====================================================================================
    #     Proxy Parameter
    # ====================================================================================
        CP_NAME="CraftProxy"
        CP="-Xmx256M -server -jar craftproxy.jar "
        CP_PORT="20000 25565"
        CP_OPTIONS="auth_off quiet staticlocalhost compression_level 9"
    
    Ram Usage is too high, maybe a memory leak?
    CP 105 used only 100mb Ram.
     
  7. Offline

    Raphfrk

    Not sure, does this happen with 1 person on the server (i.e. is it easy to reproduce)? Also, does it happen with CB 1045?

    If you could track it down to a particular CB version change, it would make it much easier for me to find.
     
  8. Offline

    Simanova

    CP takes all memory it gets from -Xmx
    It happens every day, they are 0-5 players online

    I have to investigate the problem to give more detail - may take up to 2 days

    And another issue:

    If a player is connected by local network and other players by internet, the lan player is causing network troubles for the internet players - this means:

    > higher latency for some internet players
    > disconnects for some internet players

    This happens actualy only with 1 player on my server.
    Are network users prefered by craftproxy? ( i know its senseless, because data compression isn nessesary local )

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  9. Offline

    Raphfrk

    Is that new? All players are handled by separate threads, but I guess a local player would naturally have a better connection.

    That was an issue way back with the fairness manager, but I bypassed that.

    I could add a bandwidth limiter, so that no single player goes over say 50kbps.
     
  10. Offline

    Killerrabbit

    This needs to be made into Spout so I can enforce my players to automatically get it....
     
  11. Offline

    thomasbomb

    It is.
     
  12. Offline

    Killerrabbit

    It is? If they have to start minecraft with a craftproxy client, they cant start the spout client to connect to minecraft servers?
     
  13. Offline

    Raphfrk

    No, I have pretty much implemented the caching directly into spout. It will be included in the next RB of Spout.

    You just run Spout plugin + Spout client and it should work. Press f3 in game to see cache info.

    Anyway, I added bandwidth limiting to the proxy, add

    bandwidth_limit 100

    and the bandwidth will be limited to 100kbps per user for download.

    I think 2*<upload speed>/<number of users> might be reasonable here.
     
  14. Offline

    Simanova

    New information about the memory leak:

    Its solved by removing the "Xmx" tag for startup
    It seems CraftProxy acts weird with this parameter

    -- > I also removed the parameter "-server"​

    Some additional info ( complete server restart, without mem leak ):

    • after startup
      [​IMG]
    • after 30 sec - first player joined
      [​IMG]
    • after 1 min
      [​IMG]
    • second player joined[​IMG]
    • third player joined[​IMG]
    • fourth player joined[​IMG]
    • after ~ 6 hours - 5 player joined[​IMG]

    [​IMG]

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  15. Offline

    Raphfrk

    When the memory drops again, was that you restarting the program? 14GB seems high, to say the least.

    You can't eliminate a memory leak by changing parameters. Memory leaks are caused by data that isn't properly released. If I didn't release it, then command line changes can't suddenly make it be released.

    Parameters can change how java handles garbage collection though.

    Hmm, I could force gc to happen every 10 mins.

    Xmx is supposed to set the max memory allowed. What had you it set to ? Maybe you had set it to a really high number.

    I think what might be happening is that the garbage collector doesn't actually anything unless you are running out of memory. Since there was spare memory, it just didn't do anything. Did Craft Proxy crash with an out of memory exception?
     
  16. Offline

    Jdbye

    @Raphfrk Would it be possible for you to make a client mod version of this that doesn't require a separate proxy app? I have a custom client for the players on my server and if I could just bundle a mod into it it would be a lot easier for people than if they have to start a separate app each time they want to play.
     
  17. Offline

    Simanova


    Its not crashing , the result is a high latency and a stressed cpu.
    I figured out that the memory leak doesent depend on Xmx - perhaps CP is acting weird when available ram is very low ( < 100 mb ) and swap is deactivated.

    my server has 8gb of ram - 4 gb for minecraft - 2 gb for a ramdisk - 2gb for system


    picture of today: ( hope it helps )

    [​IMG]
    [​IMG]
     
  18. Offline

    Raphfrk

    Hmm, also, have you tried with just the -server parameter removed?

    What is that supposed to do?

    The problem could be something like increased disconnects are causing more threads to be created, and it snowballs.
     
  19. Offline

    Simanova

    -server parameter was removed

     
  20. Offline

    Lemonstein

    Gonna try this out when I'm not on vacation.

    Is there any way for the server to discern whether or not the client connected is running CraftProxy? It would be nice to be able to provide my players with some sort of iConomy incentive to run the thing.
     
  21. Offline

    Raphfrk

    Btw, were you using Spout, there is a bug in Spout that keeps connections open?
     
  22. Offline

    Simanova

    No iam not using spout
     
  23. Offline

    Raphfrk

    Can you start the proxy with

    -XX:+HeapDumpOnOutOfMemoryError

    This should cause it to do a heap dump when it runs out of memory. Actually, does it actually throw an out of memory exception?
     
  24. Offline

    Simanova

    Ill try it, thanks!
     
  25. Offline

    AgentKid

    @Raphfrk Was messing around earlier and found out about ModloaderMP for craftbukkit and wanted to try using it with the proxy. I didn't expect it to work, and it didn't and I don't expect support for this, but here's the log if you want to take a look at it. The modloaderMP server is running ModloaderMP and Planes mod for bukkit.
    Code:
    Connection from 192.168.5.4/64420
    [9:35:24 PM] 192.168.5.4/64420 (AgentKid): Connecting to : fusioncraft.org 26565
    [9:35:24 PM] 192.168.5.4/64420 (AgentKid): Attempting to connect to: fusioncraft.org:26565
    [9:35:24 PM] 192.168.5.4/64420 (AgentKid): Connection successful
    [9:35:24 PM] 192.168.5.4/64420 (AgentKid): Connecting using proxy to server connection format
    [9:35:25 PM] 192.168.5.4/64420 (AgentKid): Server login successful
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Kicked with: [Redirect] please reconnect to: fusioncraft.org:26571
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Redirect detected: fusioncraft.org:26571
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Closed connection to server
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Connecting to : fusioncraft.org 26571
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Attempting to connect to: fusioncraft.org:26571
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Connection successful
    [9:37:19 PM] 192.168.5.4/64420 (AgentKid): Connecting using proxy to server connection format
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid): Server login successful
     Unknown packet Id e6
     Unknown packet Id e6
    16 1 0 34 67 0 0 28 0 32 4 0 0 67 0 0 29 0 59 30 0 0 67 0 0 2b 1 9 40 0 0 67 0 0 2c 0 4 3e 0 0 *e6* 11 41 11 f8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1f 0 0 0 9 0 1 0 1c 0 0 0 a 0 60 0 43 0 c1
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid): Downstream link
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid):  Unable to read packet
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid): Packets: [103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103, 103]
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid): Closed connection to server
    [9:37:20 PM] 192.168.5.4/64420 (AgentKid): Closed connection to client
    
     
  26. Offline

    Raphfrk

    Yeah, has an unknown packet. If you can get the packet format I can (possibly) add support.

    CraftProxy supports the Spout custom packet system for example.
     
  27. Offline

    LEOcab

    Mr. Raphfrk: hey it's me again! I'm still using your awesome plugin. :D

    Could you explain how to use the new bandwidth_limit parameter? No matter what I set it to, I (or anybody else) can't play. It just loads a few chunks around me and that's it, it doesn't show my chats or anything else. In fact, it does this even without specifying the option... version 107 works just fine.
     
  28. Offline

    AgentKid

    Hmmm, doesn't look like the Modloader MP system is open source and I'm not sure how to find it without the source :/ I'll keep searching, though.
     
  29. Offline

    Raphfrk

    Hmm, maybe I uploaded the wrong version. Does it work with just 1 player online (since that is how I tested it).
     
  30. Offline

    Simanova

    Ive got some news:

    with "-XX:+HeapDumpOnOutOfMemoryError" same effect

    should i reroll to v105?

    [​IMG]
     
  31. Offline

    Raphfrk

    It is supposed to generated a HeapDump file if java runs out of memory.

    The file would be something.hprof and in the directory that you ran the server in.

    However, it only happens if you have an out of memory exception.

    Worth a try, it is like trying to find a needle in a haystack. If there was 1 build that had it and one that didn't I could focus on the differences.

    Another option would be to try and find out which version of CB causes the issue.
     

Share This Page