[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    Pjstaab

    Why?
     
  3. Offline

    Raphfrk

    I have to agree with this question :). The main benefits of the proxy are to reduce bandwidth for the server admin, since download bandwidth is normally a big concern.

    However, I guess I can add it.
     
  4. Offline

    spunkiie

    Yeap, but will reduce overall bandwidth as well for the user.. I'm right ?

    So this is the answer :D

    Thank you in advance.

    btw: We have a 200+ server, so we have to give some 'extras' for premium users ;)
     
  5. Offline

    Raphfrk

    Right, I am just not sure how useful that is :), since most users will have enough download bandwidth.

    Cool, have you tried that proxy with that many people, I would be interested to know how well it performs?
     
  6. Offline

    spunkiie

    Well, we already have about 70 VIP users.. :D

    When you make possible to let only them use the proxy, I will for sure give you feedback
     
  7. Offline

    Raphfrk

    From here:

    Also, I like the idea suggested by one of the commentators to have the client do chunk generation.

    You could XOR the actual chunk data with the default chunk and compress that.

    A newly generated chunk would be all zeros (meaning no differences), which would compress to almost nothing.
     
    mgmchenry likes this.
  8. Offline

    ledhead900

    This is some good stuff your learning I'm learning Notch is learning beer anyone:cool:
     
  9. Offline

    Simanova

    Great news!
     
  10. Offline

    mgmchenry

    That is brilliant! I *Love* that idea.
     
    Pim1234 likes this.
  11. Offline

    Takel

    Considering how the client already has the seed value to calculate the correct biomes and weather effects... that would be a very elegant solution. Worst case scenario (where 100% of the blocks have changed) would simply be the status norm.
     
    Pim1234 likes this.
  12. Offline

    Pim1234

    is there or will there be a way to use this with mineshafter?
     
  13. Offline

    thepackett

    looking forward to seeing the newest version as a plugin :D
     
  14. Offline

    Raphfrk

    It is a plugin already.

    You just add the .jar file into the plugin directory.
     
  15. Offline

    Kyrin

    First off, thanks for the plugin, works great, both standalone and in plugin mode, easy setup and my users love it (so does my little DSL connection), biggest problem I'm having now is apparently it's having issues with the login auth, I thought it was working fine 'cause standalone the default is on, but apparently in the config.txt it's off by default... So I didn't notice it wasn't "working" 'till I was trying to get the standalone working on another connection... Anyway, I have only legit users on my server, so this isn't an issue with them not having accounts, but this is what I'm getting when people are trying to login:

    ==Proxy as a Plugin==
    Server:
    Code:
    17:34:33 [INFO] Connection from 192.168.0.15/50384
    17:34:33 [INFO] 192.168.0.15/50384 (Kyrin): Attempting to connect to: localhost:25565
    17:34:33 [INFO] 192.168.0.15/50384 (Kyrin): Connection successful
    17:34:34 [INFO] Disconnecting Kyrin [/127.0.0.1:44291]: Failed to verify username!
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Authing with http://www.minecraft.net/game/checkserver.jsp?user=Kyrin&serverId=xxxxx
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Auth successful
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Server login successful
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Client requested caching mode
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): EOF reached
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Timeout
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Closed connection to server
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Closed connection to client
    
    ==Proxy Standalone==
    Proxy:
    Code:
    Connection from 205.206.13x.xxx/16610
    [7:18:49 PM] 205.206.13x.xxx/16610 (Kyrin): Attempting to connect to: 192.168.25.1:25565
    [7:18:49 PM] 205.206.13x.xxx/16610 (Kyrin): Connection successful
    [7:18:52 PM] 205.206.13x.xxx/16610 (Kyrin): Authing with http://www.minecraft.net/game/checkserver.jsp?user=Kyrin&serverId=xxxxx
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Auth successful
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Server login successful
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Client requested caching mode
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): EOF reached
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Timeout
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Closed connection to server
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Closed connection to client
    
    Server:
    Code:
    18:18:53 [INFO] Disconnecting Kyrin [/192.168.25.2:45784]: Failed to verify username!
    
     
  16. Offline

    Meiun

    i get booted from my test server with a message saying authentication failed with the proxy. i cant even log in with proxy side authentication on. even though the proxy says i passed authentication the server says i fail. without the proxy it works perfectly.

    woops looks like someone beat me to it. ive been testing and running into this all day.
    i would like to add that that its a problem with plugin and standalone with me, auth on AND off
     
  17. Offline

    TruffleDucks

    Can you describe this plugin more? I tried looking up the word cached, but it doesn't make sense :eek:
     
  18. Offline

    ledhead900

    ==Proxy as a Plugin==
    Server:
    Code:
    17:34:33 [INFO] Connection from 192.168.0.15/50384
    17:34:33 [INFO] 192.168.0.15/50384 (Kyrin): Attempting to connect to: localhost:25565
    17:34:33 [INFO] 192.168.0.15/50384 (Kyrin): Connection successful
    17:34:34 [INFO] Disconnecting Kyrin [/127.0.0.1:44291]: Failed to verify username!
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Authing with http://www.minecraft.net/game/checkserver.jsp?user=Kyrin&serverId=xxxxx
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Auth successful
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Server login successful
    17:34:34 [INFO] 192.168.0.15/50384 (Kyrin): Client requested caching mode
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): EOF reached
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Timeout
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Closed connection to server
    17:34:35 [INFO] 192.168.0.15/50384 (Kyrin): Closed connection to client
    
    ^ Turn authentication off in proxy config.




    ==Proxy Standalone==
    Proxy:
    Code:
    Connection from 205.206.13x.xxx/16610
    [7:18:49 PM] 205.206.13x.xxx/16610 (Kyrin): Attempting to connect to: 192.168.25.1:25565
    [7:18:49 PM] 205.206.13x.xxx/16610 (Kyrin): Connection successful
    [7:18:52 PM] 205.206.13x.xxx/16610 (Kyrin): Authing with http://www.minecraft.net/game/checkserver.jsp?user=Kyrin&serverId=xxxxx
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Auth successful
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Server login successful
    [7:18:53 PM] 205.206.13x.xxx/16610 (Kyrin): Client requested caching mode
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): EOF reached
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Timeout
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Closed connection to server
    [7:18:54 PM] 205.206.13x.xxx/16610 (Kyrin): Closed connection to client
    
    Proxy Authentication is enabled so it allows to connnect fine to proxy! below code shows that proxy did not pass your login to the server so server sees you as a non paid user.

    Server:
    Code:
    18:18:53 [INFO] Disconnecting Kyrin [/192.168.25.2:45784]: Failed to verify username!
    
    Server user authentication is enabled turn off proxy authentication when connecting with proxy to a local server using bridge mode and keep auth on in server config.

    Optionally Turn both authentications off for unprotcted mode and allow cracked clients to join.


    In short if using a proxy as a plugin or standalone running for a single server and its connecting locally ie: on the same machine as the server is enable proxy bridge mode and disable auth via prxy.

    Hope this helps.
     
  19. Offline

    Raphfrk

    @Meiun
    @Kyrin

    If the proxy is handling authentication, then the server should be in offline mode.

    There is no need for both the to do.

    If you are only running one server (so not using serverport), then I would recommend that you just leave auth_off for the proxy and have the server handle authentication.

    [edit]
    @ledhead900

    Thanks for jumping in there with the info :).
    [/edit]

    If you click on the "how it works" button in the OP, there is more info.

    The basic idea is that every time you go to a new place in the world, the server sends you all the chunk data for that place. The plugin remembers what was sent the last time you were in a place and so the server doesn't need to send the info every time.
     
  20. Offline

    Kyrin

    I didn't realize that with the proxy having authentication turned off that it would rely on the server to do it, I assumed that the proxy was somehow bypassing that authentication requirement on the server, however I realize that's kinda beyond the scope of its ability, especially in standalone mode... All I cared about was that the users are definitely getting their ID checked at some door, as it were, either the proxy or server... Thanks for clearing it up!
     
  21. Offline

    Raphfrk

    Yeah, I was thinking that I should change the parameters to auth_by_proxy auth_by_server rather than auth_off and auth.
     
    Kyrin likes this.
  22. Offline

    Kyrin

    I just love how it works, and that I can use it as a proper proxy to allow my one server to be accessed from my other connection as well, even though the hosting server isn't multi-homed... One last thing, how does it handle multiple users on the client side? I'd like to be able to have people at a LAN connect to a proxy client, whilst having the server located somewhere across the internet, in order to reduce traffic and such...
     
  23. Offline

    Raphfrk

    There would be just one cache in that case that would be shared between all the users who connect.
     
  24. Offline

    Kyrin

    Cool, just wanted to make sure that would work authenticating multiple users and such... Love the plugin, thanks so much for making this!
     
  25. Offline

    thepackett

    you said the newest version (96?) doesn't support plugin mode, i'm sorry i should have phrased that better.

    on that note, if i have the version that does support the plugin mode can i connect to that using the newest version?
     
  26. Offline

    Kyrin

    This doesn't seem to support multi-worlds... or am I missing something? 'cause my people are dying when they travel between worlds...
     
  27. Offline

    Raphfrk

    Sorry, I deleted that line from the OP. The latest version supports plugin mode.

    It should, what plugin are you using for multi-world?

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  28. Offline

    mgmchenry

    Does Craftproxy support vanilla servers? If not, why not?

    I run two servers, bukkit and Vanilla. Today I'm also running a third because 1.6 is sooooooo not ready, but I want to let people try it out. I'm in the US and I have users in China that experience extreme lag that might (I hope) be resolved by this. I'd like to try it on my 1.5_02 vanilla server. Will it work on my 1.4 Bukkit server?

    What sort of work would need to be put in to make that happen? (I'm a 10-year .Net veteran with moderate Java skills, but plenty of network programming experience.)
     
  29. Offline

    Takel

    Personally with my experiences with the CraftProxy, I don't see why it can't work on vanilla servers since it can be executed stand-alone.

    Just remember that CraftProxy cannot assist with latency as a result of pure hard physics. It won't reduce the RTT of your users to your server but it can reduce the bandwidth requirements and assist with faster rendering of chunks since the client no longer has to wait on as much data from the server provided the cache is warm. For all intents and purposes, its a standard internal network cache for web resources, but it deals with Minecraft terrain chunks instead of HTML/whatever.

    I am assuming of course that CraftProxy isn't doing anything but packaging/repackaging terrain data and there's nothing in the code which is version specific to Minecraft.

    Oh and don't take my word as gospel... this is only from my observations since I haven't dug into the code myself. Got my own pressing issues to deal with...
     
  30. Offline

    Raphfrk

    It should. The protocol is identical (since they same client is used).

    In fact, it is only a plugin to make it easier for server admins to install.

    If you want to use it with vanilla, you would need to run it in standalone mode.

    No, Notch broke backward compatibility with the version 1.5 protocol.

    The changes were pretty extensive. All strings were changed to a different format and a field was dropped from the login packet.

    1.5 to 1.6 appears to have very little changes.

    The respawn packet now allows the dimension to be changed (though seems not the world seed).

    Well, it has to understand the protocol in order to extract the chunk data packets, and that is inherently Minecraft specific :).

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  31. Offline

    Takel

    very true... I was wondering more of data structure differences such as the ones you've highlighted between 1.4 and 1.5.
     

Share This Page