[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    Simanova

    In deed Craftproxy is not really a plugin, but it is the best software on bukkit.org.
     
  3. Offline

    Raphfrk

    I modified the client using mcp so that it sends packet CD instead the encryption packet and it stays connected, you probably have some kind of packet setup error.
     
  4. Offline

    Major

    There's no need to encrypt when you're sending it from the server proxy to the client proxy?
     
  5. Offline

    TyrOvC

    I'm having a very busy week, so won't have time to sit down and work on this until the weekend.

    Good to know, thanks! I had tried to update the packet unit array to match all the protocol changes, but I must have missed something somewhere. It's just proving to be a pain to figure out what.

    Not technically necessary, but the proxy has to be able to handle encryption for clients not using craftproxy. So might as well encrypt it between proxies too.
     
    lol768 likes this.
  6. Offline

    Raphfrk

    Btw, you are aware that some packets now have up and downstream versions? That would cause an issue if you only matched the packet in one direction.

    You would need to add a flag of some kind to indicate which direction the link is in.

    Right, and the encryption isn't that massive a load.

    It occurs to me since we have both proxies under control, it might be possible to circumvent the authing problem. The server proxy could sent the local proxy the info needed to decode the stream.

    Something like

    Proxy acts as passthrough for these
    - Client sends handshake (might have to modify it to have the server's hostname/port)
    - Server sends encryption public key
    - Client creates secret and auths with minecraft servers
    - Client sends encrypted secret to the server
    - Server uses that info to confirm auth
    - Server sends empty encryption packet
    -- this is blocked

    <Server and proxy exchange info about encryption used>

    An aux password would be required here. This could be implemented when the player connects for the first time and be generated and stored by the local proxy.

    The issue becomes that once you have it setup, another server emulate a local proxy, and thus session steal for players who would directly connect, so there needs to be some way to verify that the local proxy is actually "authorized" by the client.

    Another option would be to have encryption on the uplink only, since that doesn't have to be cached.
     
  7. Offline

    Simanova

    I wish i could give you some hours or go to work for you :)
     
    lol768 likes this.
  8. Offline

    Raphfrk

    Not much use for people looking for caching, but md_5 has released his proxy (limited to 20 connections at a time). This allows server to server teleporting.

    The post is here
     
  9. Offline

    Simanova

    I cant post on spout forums, because iam not a member there.
    But ive read all these posts - so it should be possible to split up craftproxy funcionality in server-teleporting and caching?
    Rubberband will handle teleporting functionality while craftproxy remains with caching.

    Please correct me, if iam wrong!
     
  10. Offline

    Raphfrk

    Well, having one piece of software do both is easier because you don't have to parse the stream twice. However, teleporting doesn't require compress/decompressing the chunk data packets. Also, obviously, CraftProxy isn't compatible with the encryption system.
     
  11. Offline

    TyrOvC

    I've got everything on the server part finished, minus handling the new map chunk bulk packet for caching. Will be putting on github if anyone wants to help with the rest.

    Edit:
    In addition to caching, there's also the fact that md_5's proxy is nerfed to 20 connections at a time. If you want to keep your stuff proprietary, shouldn't release a gutted version at all in my opinion. :/
     
    lol768 likes this.
  12. Offline

    lol768

    Have you got server->server redirection working?

    Agreed.
     
  13. Offline

    TyrOvC

    Client -> Serverside Proxy -> Minecraft Server works, with chunk compression manager and such disabled.

    What needs to be finished:
    Client -> Proxy -> Proxy -> Minecraft Server

    For chunk compression and caching, something needs to be done with the new MapChunkBulk packet.

    https://github.com/tyrovc/CraftProxyLiter

    The build order on referenced libraries to need to be:
    1. Bouncy Castle Java Crypto Library
    2. Bukkit
     
  14. Offline

    lol768

    I wish I could help but I'm really not skilled enough to contribute anything of use. I wish you the best of luck with coding the rest of it though and I can always help test it.

    I'll try and build it at some point tomorrow.
     
  15. Offline

    nacs

    Does "chunk compression manager and such disabled" simply mean that the backend server needs to be in offline mode or does the Craftbukkit source need to be patched to do this?

    There is a .Net based Minecraft proxy software here: https://github.com/pdelvo/Pdelvo.Minecraft.Proxy that I have tested that only requires the backend server to be in offline mode for server to server teleports to work (no Craftbukkit patching required). Would your patched craftproxyliter work the same way (I have no need for the chunk caching feature of Craftproxy)?

    Thanks.
     
  16. Offline

    Raphfrk

    Cool, great to see you are making progress.

    I don't really agree, better that they are releasing something than nothing. Also, your release of this might prompt them to open it more (and/or add caching too).

    Have you managed to get the encryption stuff working, or are you keeping everything in offline mode?

    So teleporting is working?

    I would suggest handling it using the standard map packet system.

    I am not really convinced putting 100 chunks together would actually make much difference.

    So, effectively, the server side proxy would convert the bulk chunk packet into many standard chunk update packets. These could then be processed using the standard caching mechanism.

    I see that you have bouncy castle, so I guess encryption is working. Is the plan to have the player enter their password into their client proxy?

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  17. Offline

    lol768

    It might be that I'm doing something wrong or the proxy isn't developed yet, however when I built it and tried it out with this command:
    Code:
    java -jar test.jar 25565 25566 debug info
    And then attempted to connect to the offline server I get this error:
    Show Spoiler



    Disabling name authentication with auth_off causes this to occur:
    Show Spoiler


    Turning online mode on on the server and trying both auth and auth_off with the proxy gives this kick message:
    Show Spoiler

    [​IMG]
    Somehow I don't think this image will work, so here's a link: http://i.imgur.com/SsxW9.png

    Console:
    Server listening on port: 25565
    Connection from 127.0.0.1/43118
    [13:02:13] 127.0.0.1/43118: Forwarding ping
    Connection from 127.0.0.1/43121
    [13:02:14] 127.0.0.1/43121 (lol768): Connecting to : localhost 25566
    [13:02:14] 127.0.0.1/43121 (lol768): Attempting to connect to: localhost:25566
    [13:02:14] 127.0.0.1/43121 (lol768): Connecting to: localhost:25566 from 127.0.10.10
    [13:02:14] 127.0.0.1/43121 (lol768): Connection successful
    [13:02:14] 127.0.0.1/43121 (lol768): Connecting using proxy to server connection format
    Handshake forwarded to server
    Authing with session.minecraft.net
    Switching to encrypted stream on client socket
    [13:02:16] 127.0.0.1/43121 (lol768): Server login successful
    [13:02:16] 127.0.0.1/43121 (lol768): EOF reached
    [13:02:17] 127.0.0.1/43121 (lol768): Timeout
    [13:02:17] 127.0.0.1/43121 (lol768): Closed connection to server
    [13:02:17] 127.0.0.1/43121 (lol768): Closed connection to client


    I might be doing something really stupid (or I might be trying to do something that can't yet be done), can you give me some advice on where I'm going wrong or how the server needs to be setup for this to work?
     
  18. Offline

    Raphfrk

    It doesn't work with 1.3.x, since I haven't updated it.

    I don't have time to work on it, but TyrOvC , is looking into it.

    [Edit] looks like you were using his build already ... so never-mind, you will have to ask him :p.
     
  19. Offline

    lol768

    Forgot to clarify, I'm using TyrOvC's modification of your code from GitHub.

    Thanks anyway. It'll probably turn out to be my fault anyway :)
     
  20. Offline

    Raphfrk

    The only requirement is that the backend server is in offline mode. The proxy handles authing players.

    Cool, that is new. How does it support teleporting, is there a CraftBukkit plugin somewhere?

    The way CraftProxy works is that it detects a special kick packet that has a special format starting with "[Redirect]". The proxy then breaks the connection to the server and reconnects to the other server.
     
  21. Offline

    TyrOvC

    By chunk compression manager and such disabled, I meant that chunk caching is not actually functional yet.

    nacs
    That .Net based proxy looks cool, and it's probably another option if all you need it for is server to server teleports.

    Raphfrk
    Server to server teleporting works, just all the fun chunk caching doesn't :p
    Only way I can work out in the protocol to authenticate the player is either installing an actual client mod, or entering their password in the Proxy. Entering password into the proxy would be easier to update between Minecraft versions, so that's probably it.

    lol768
    At the moment, it only works right in remote proxy mode, i.e. running as a plugin on the server that players connect to. Clientside proxy still needs some way of handling the new authentication scheme.
    I think you were running it clientside, right?

    I guess I need to add more specific instructions too, the minecraft server itself must be running in offline mode, so that only the proxy does authentication and encryption.
     
  22. Offline

    lol768

    Nope, I was running it server-side (but not as a plugin). You can supply the proxy arguments such as where to listen and where to locally redirect the connecting players to and just run it as a standalone program. If you can get this functionality to work, it would be awesome.

    Thanks. I wasn't sure on this (although I did try this combination).
     
  23. Offline

    Raphfrk

    Another option is to have the server proxy send the encryption info to the client proxy.

    However, doing that in a safe way isn't entirely clear.

    Effectively, the client would login into the server proxy and the client proxy would just be a passthrough. This is all plaintext packets. Once connected, the server proxy would send a packet containing the keys to decrypt the connection.

    However, that eliminates the benefit of the encryption entirely, since a MITM could read the special packets.

    One option would be to add a password specifically for the proxy. When a user connects to a server the first time, it would have to create a password somehow.

    I assume you are going to have offline mode for client to proxy and proxy to server then?
     
  24. Offline

    TyrOvC

    That's a clever idea. Might be hard to get the position in the stream aligned right when the proxy sends to keys to decrypt the connection to the client proxy though.

    Not quite sure what you mean. If you're talking about using offline mode to disable encryption, it's going to look like
    Client with no clientside proxy:
    <Client> encrypted stream <Proxy> unencrypted stream <Server>
    Client with clientside proxy:
    <Client> unencrypted stream <Proxy> encrypted stream <Proxy> unencrypted stream <Server>

    I think the best way to do it would be for the player to give username/password information to the proxy, which the proxy then uses to authenticate with minecraft.net. Clientside proxy would send the client the packets to appear to be in offline mode, so the client skips encryption. Only way I can think of that actually keeps the security of the new protocol, and doesn't require a client mod.


    Edit:
    Sending the keys between the proxies could actually be secure, but it'd require a third party like session.minecraft.net I think. I'm not a cryptographer though.
     
  25. Offline

    Raphfrk

    That's what I meant.

    Btw, the client proxy would only need the password, since the client would send the username.

    Right.

    Yeah, that's kind of what I meant with requiring a special password. The player would enter their "CraftProxy" password and then the proxy would check with some 3rd party. However, easier to just have the players enter their password than have a second auth server that could go offline.
     
  26. Offline

    nacs

    It works very similarly to Craftproxyliter where you can send a simple kick message to the client with the IP of the server to connect to and the proxy intercepts and redirects.

    I'm also getting the same socketwrite exception when running CPL and connecting to it (the backend server is running in offline mode). Your modified CPL actually connects, does the MC auth properly and I even see the MOTD and a flash of the map loading in the Minecraft client before I get disconnected with a "Internal exception: java.io.IOException: Bad packet id 50" (and the CPL console shows the socketexception that lol768 pasted above.
     
  27. Offline

    Raphfrk

    I might ask him to add support for the way Craft Proxy does it. Does he have a forum for it somewhere?
     
  28. Offline

    user_90720735

    Hi Raphfrk, I've been trying to use your plugin on a tekkit server, I have all my ports forwarded correctly, and can connect without the proxy, but when I try using the proxy, I get a packet id error (mainly 114), and the client proxy says craftproxy kicked me. Is there anyway you can help with this? Thanks in advance.
     
  29. Offline

    Raphfrk

    It doesn't work with 1.3, TyrOvC is looking into updating it.
     
  30. Offline

    lol768

    Tekkit is based on 1.2.5, if I recall correctly (may be totally wrong).
     
  31. Offline

    Raphfrk

    Hmm, Joekom0725 , is the server in offline mode?

    Actually, if it is a complex mod, then it might use special packets?
     

Share This Page