[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    letmedl

    Hi i have been looking for something like this for a while. Anyway what i found before this was a program called: tcpbuffer, i think the latest is v6. I don't know if its still active however it supposedly compressed the server and clients packets, so that they were less frequent. I don't think it has much merit, however if it does could you incorporate features like this. I can only host like 4 people on my server, terrible 50 mbps upload.
     
  3. Offline

    ledhead900

    looks like u have exact same issue as me I had that same issue before even updated it with prev dev build.


    Yep it was grate it was a non official attempt to fix the 100% shotty network code notch refuses to lift finger on to fix.

    What the Tcpbuffer did was fix the "Disconnect - Overflow" issue since Minecraft 1.3 Notch has made changes to the network code and turned it to shit.

    he sends packets far to much lots of small packets too fast this plays havoc on routers that detect potential DDOS attacks and other wise flood the user making latency very high in the prosses.

    This is why every version since 1.3 has been utter crap for latency.
    What this and tcpbuffer does is take over the packet sending avoided notchs crap mostly.

    Result is a server that would norm support only 8 players online to support 20 players online if they all use it.

    Its pretty obviouse and notch has some explaining to do that will never actually happen of course I dont think anything he could say to make up for it would excuse the fact that the game ran smoother and less latency in alpha then it does nearing Release. Thats fact that everyone seems to be completely ignorant to.
     
  4. Offline

    letmedl

    Thanks for the great response ledhead900 i take it from that their is no further optimizations that could be made too beta onwards packets?
     
  5. Offline

    rtcabooservb

    :( Sounds like it was a great help ledhead900. Do you know if there is any other type of plugin or modification that works just like you described?

    Also I ran into a bug while using craftproxy v65. First off, it says it is version 56 in the console. I also get spammed to death with the error "Lock Failed'. I also can't run it as a plugin, has to be standalone.
    Here is my batch file to run it and no I am not using serverport at the moment.
    Code:
    @echo off
    java -jar CraftProxyLiter.jar 25565 1337 auth quiet reconnectfile reconnect.txt staticlocalhost bridge_connection
    pause
    Using CB 785
    Using Craftproxy v65 even though it says v56 in the console... It is under the name CraftProxyLiter.
     
  6. Offline

    Raphfrk

    @ledhead900

    @Simanova

    Sorry, was late when I finally uploaded it. I figured it was better to upload something than wait until this morning.

    You need to add auth_off to command line. This switches off *proxy* based authentication.

    The proxy then acts as a pure bridge during authentication. If you leave your Bukkit server in online mode, then the Bukkit server will do authentication.

    In fact, if you leave your server in online mode, then the proxy can't do anything that would allow the player to bypass authentication.

    If you don't turn authentication off on the proxy, then the proxy will accept the connection (and do authentication) and then try to connect to the Bukkit server. However, the proxy doesn't have the password.

    This one should do it now. It uses a buffered output stream and flushes the stream at least once every 100ms (I should probably add a switch for that).

    Need to update the transfer program.

    I need to add plugin support to the new version and the lock failed message is debug code (and doesn't indicate a problem).

    That is the new name, since it is a re-write of the Lite version, I am not sure why it isn't incrementing properly.

    Did you get compression?

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  7. Offline

    Simanova

    My first impression of v65: It's working great! But further testing is needed

    cb786
     
  8. Offline

    -soL-

    so do i get it right? Its standalone-only at the moment? cause if i try to run it as server plugin the "could not load plugin" message comes.
     
  9. Offline

    Raphfrk

    That's right. I had it in the OP, but have moved the message to beside the download link.
     
  10. Offline

    -soL-

    hum ok...so how i start it standalone? i mean i see the
    java -jar CraftProxyLite.jar 25565 <target-server-hostname>:20000 local_cache auth_off

    command....but how to use it? EDIT: (on windows)
     
  11. Offline

    rtcabooservb

    Even though it is a debug code, it likes to repeat itself 400 times a minute. This slows down the proxy and causes a minor chat lag.

    I also had a question. If you turned off authentication through the proxy and kept the main server online to authenticate, and had people connect with the default ip while having the proxy auto redirect them, will bandwidth be higher if the main server is kept online and if people connected through both the proxy and the main server with the assigned port?

    Also had another question. Is it important to add "staticlocalhost" and "bridge_connection" to the startup batch? Since the proxy runs without it and people can connect without it, I was just curious.

    Would be great if I could also get an example of what I should add to the startup file too and what parameters I should add. For example, auth_off from what you stated should be added. This is what I have now, even though nothing ever gets written in the reconnectfile. I plan on letting bukkit do the authenticating if it is faster and less demanding so I might have to add auto_off to the proxy and turn the server online in the settings.
    Code:
    @echo off
    java -jar CraftProxyLiter.jar 25565 1337 auth quiet reconnectfile reconnect.txt
    pause
    Much help is obliged. :)
     
  12. Offline

    Raphfrk

    Hmm, ok. It didn't show up much for me, but I was only connecting with 1 account.

    I am not sure exactly what you are asking.

    Do you want players to be able to connect directly to the server (and so bypass the proxy)?

    A player who directly connects to the proxy's port (without the GUI) will just be redirected as a passthrough. I think there might be a slight drop in bandwidth due to the fact that it combined multiple packets into a single packet, but it shouldn't be that large a change.

    Btw, what kind of CPU load are people seeing, hopefully, it is reduced.

    Also, if people are seeing low CPU, I could in theory increase the gzip compression effort. I has 10 settings

    0: No compression
    1: Best speed (Notch uses this)
    .
    .
    9: Best size

    I am not sure how much of a difference it would make though and what the CPU/bandwidth tradeoffs are.

    They aren't required any more. Auth_off is the same as bridge connection now.

    I updated the OP with the new recommended command lines.
     
  13. Offline

    -soL-

    well i made it run now....had to set path for the java bin folder. Standalone starts but as soon as i try to connect to localhost:22233 (thats the port of the server) it says unable to open socket to minecraft server. Does that means the servers 20000 port isn't forwarded?
     
  14. Offline

    rtcabooservb

    I do run a server with 40+ on peak times so, yeah. The debug code can get crazy.

    For the compressions, can you give me an example on what they are? I'm a bit new to it and haven't heard about it yet. :p I always ran craftproxylite/liter as a standalone plugin to reduce the bandwidth upkeep it takes to login. From what you are telling me, is it useless to run it if no one uses the clientside version?
     
  15. Offline

    Simanova

    Its useless, because traffic is caused by clients.
    CraftproxyLiter as serverplugin / serverstandalone is only for decompressing data ( translate datastreams )
     
  16. Offline

    Raphfrk

    You will still get the "fairness manager" which should reduce latency when people login/teleport as it gives normal packets priority. It also combines small packets into larger ones which should reduce bandwidth a little (not sure how much).

    In fact, I would be interested in knowing if you are seeing much of an improvement, even if nobody is using the GUI.

    However, if the player uses the GUI, then it can reduce bandwidth by 70%+. I was getting 75% when I logged in the second time into the server. It is really intended for use with the GUI.

    The GUI shows how much bandwidth has been saved since you logged into the server.

    When a player logs in the first time, it will actually show a negative saving since it has to exchange the hash codes. However, that is normally only about 10% or so extra.

    Also, are you noticing that the proxy is generating much CPU load?

    Can you list all the ports you are using and what computer everything is running on (local/remote).

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  17. Offline

    Simanova

    2 issues: (v65, cb786)


    1) My console get spammed by "Lock failed" What does this mean?

    2) Otherwise i detected a problem - only 1 person can connect at one time to the server - is this a known bug?
     
  18. Offline

    Raphfrk

    That is debug code. It means that the output stream timer tried to flush the output buffer while it was being written to.

    It should be able to handle it. I will need to test it.

    When I tested with 2, both would have connected to the same GUI, so I need to run 2 GUIs and 2 clients.
     
  19. Offline

    Kizzard

    Hi, I'd like to say thanks for this! I've tested it so far with just one player (me) and it seems to be working great. I've got the cache cranked up to 1GB and seeing some great b/w savings especially when travelling by our rail system.

    I especially love the fact you also allow non caching clients to connect. Compatiblity is a bonus!
     
  20. Offline

    Raphfrk

    Are you seeing the issue where only 1 player can connect?

    Btw, are you having all players connect via the proxy?

    I can have 2 players connect to the same server.

    I have noticed that it doesn't seem to compress the 2nd time a player connects unless the GUI is restarted.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  21. Offline

    eriorkless

    Hi, I'm really interested in your plugin but I was wondering two things:
    Do we HAVE to change the port?
    Also, I was curious if you could make a video tutorial. I could probably figure it out, but I sure wouldn't mind one :)

    Thanks,
    eriorkless
     
  22. Offline

    ledhead900

    well that very much explains it thanks you.

    Edit

    Ok auth_off worked like charm but this new version has a sucifient amount of delay between actually doing anything other then moving around.

    Open chest takes 5s
    hit a mob takes 3s
    use levers doors about 2s

    Those are respone times from me connectin locally and clicking the mouse and waiting.
    prev versions did not have this issue.
     
  23. Offline

    Faust

    I have also noticed the latency which is most likely a result from the packet buffering, It might need some tweaking.
     
  24. Offline

    Raphfrk

    I have updated.

    The locked file spam should be eliminated and I have fixed a bug where the same user wouldn't get compression without re-starting the GUI.
     
  25. Offline

    Simanova

    latency is increased by 200-500 millisec if connecting via lan




    Good news, ill try it!
     
  26. Offline

    RandomMiner

    I think I can safely report that too, e.g. connecting to example.no-ip.org:25564 yields no lag while connecting to localhost:25564 or by using the IP results in the above mentioned lag. Using no port (and therefore not the craftproxy) results in no lag at all.

    I actually was going crazy here checking security settings and wondering where that lag comes from when connecting with craftproxy (and in nature I suspect my System first before bothering other people).

    I would still be searching for a mistake, if the above post would not exist.

    Edit: FYI still happens in Version 70
     
  27. Offline

    Phaedrus

    I'm eagerly watching the development of this plugin.

    @Raphfrk I have a wickedly fast CPU in my server, let me know if you want to do some compression testing. 4.8ghz core i7
     
  28. Offline

    Raphfrk

    I have updated again. It was waiting to long for buffering.

    Also, you can now set the latency parameter using the command option

    bufferlatency 0

    The number is in ms.

    One option would be to use a higher compression ratio for the deflate algorithm. Normally, the proxy uses a deflate effort of 1 (max speed).

    A test of 1 chunk showed that level 5 (of 9) can compress by another 22%. and level 9 can compress by 33% more than default. Level 9 takes about 3 times longer. However, that would only apply to uncompressed chunks.

    In theory, this would give improved compression relative to the standard level, even for players without the GUI proxy.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  29. Offline

    Phaedrus

    It would be cool if that were a configurable variable. 22% would be worth it in a lot of cases, especially if you have cores sitting idle.
     
  30. Offline

    Raphfrk

    I added a command line switch for it. It looks like it took the compression from around 75% to 85%.
     
  31. Offline

    Kizzard

    Aha, yes I did notice some amount of latency. Glad to know this can be fixed. If you set this to 0 does it effectively disable the packet concatenation feature? Can you explain exactly how beneficial this feature is? EG. if I wanted to reduce latency as much as possible by setting this to 0, what would I lose?

    Thanks!

    PS I think you need to update the top of the OP and the title to say the new version number.

    PPS Maybe you could you put the verion number in the GUI (I see it in CLI mode)

    PPPS The CLI examples have 'craftproxylite' instead of 'craftproxyliter'

    :D
     

Share This Page