[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    Raphfrk

    Ok, I found a problem with how packet 52 was defined in the code.

    The reason finding the bug was hard was mostly down to me misreading the error reports and trying to find a bug with packet 51 (map packet). Packet 52 is referenced in nearly all the logs posted in the thread.
     
  3. Offline

    ShadowDrakken

    I'm using Bukkit's default world gen, seed is 779427292549665184 and I'm right near the spawn
     
  4. Offline

    Raphfrk

    I think the latest update should fix the problem. I needed to move around a bit to be kicked.
     
  5. Offline

    ShadowDrakken

    So far so good! :D No more jitters crossing chunks and no more disconnects (v152)

    Still has the fake init packet messages on the server, but I assume that has to do with debugging?

    I'll keep you posted on my end if any more problems surface.
     
  6. Offline

    Simanova

    152 is working really good, no disconnect so far.

    With a bit more testing 152 can be called stable version.



    some debug log
     
  7. Offline

    Raphfrk

    Partly, it means that the server is sending chunk data without telling the client to initialize the chunk first. This would cause previous versions of the client to ignore the chunk data.

    The proxy tracks which chunks are active and an uninitialized chunk would indicate that I messed up the tracking (or the server made a mistake). This is needed for server to server teleporting.

    However, the new protocol allows these packets to auto-initialize. The tracking now (version 153) just silently adds the chunk to the list, while previously it would give a warning.
     
    Simanova likes this.
  8. Offline

    ultimateowner

    I'm not sure when the plugin will support inter-server teleporting again, but can someone please tell me how to get it working now? I have serverport plugin set up, just can't get it to work with craftproxy.
     
  9. Offline

    Raphfrk

    I added support for the new hostname feature in the latest client. This means that you can connect to different servers based on the hostname provided.

    What stage are you at? I assume you have it at the point where ServerPort will give a kick message that tells you to connect to the other server?

    What does it say when you kick? You need it to make sense to the proxy, not the client.
     
  10. Offline

    ultimateowner

    The kick message is "[Serverport] You have been teleported, please connect to : Wireless_Broadband_Router"
     
  11. Offline

    Raphfrk

    Ok, that is wrong :p. It needs to be a hostname : port.

    You need to open /plugins/Server Port/serverport.txt

    There should be a line

    globalhostname=<something>

    You should change that to whatever the proxy would need to use to connect to the server.

    For example

    globalhostname=localhost:25565

    Then test again without the proxy. The kick message should say to connect to localhost:25565 when moving to that server.

    You need to do that for all your servers.

    If you then connect via the proxy, it should detect that kick message and reconnect automatically.
     
  12. Offline

    FRLanc

    Does spout do the same?
     
  13. Offline

    rickky

    craftproxy
    Code:
    127.0.0.1/58153 (): Connecting to : localhost 25566
    127.0.0.1/58153 (): Attempting to connect to: localhost:25566
    127.0.0.1/58153 (): Connection successful
    127.0.0.1/58153 (): Connecting using proxy to server connection format
    127.0.0.1/58153 (): Server login successful
    127.0.0.1/58153 (): Redirect detected: localhost:25567
    127.0.0.1/58153 (): Unable to flush output stream
    127.0.0.1/58153 (): EOF reached
    127.0.0.1/58153 (): Closed connection to server
    127.0.0.1/58153 (): Connecting to : localhost 25567
    127.0.0.1/58153 (): Attempting to connect to: localhost:25567
    127.0.0.1/58153 (): Connection successful
    127.0.0.1/58153 (): Connecting using proxy to server connection format
    127.0.0.1/58153 (): Server login successful
    127.0.0.1/58153 (): Unable to send dimension setup packet
    127.0.0.1/58153 (): EOF reached
    127.0.0.1/58153 (): Unable to flush output stream
    127.0.0.1/58153 (): Closed connection to server
    127.0.0.1/58153 (): Closed connection to client
    
    localhost:25566
    Code:
    [INFO]  [/127.0.10.10:58154] logged in with entity id 625 at ([world] 519.9730311524057, 64.0, 449.34883421513433)
    [INFO] [ServerPort] Checking if  is on 2
    [INFO] [ServerPort] Forwarding  to localhost in order to reach 2
    [INFO] [ServerPort] Local IPs detected using localhost for teleport (localhost)
    localhost:25567
    Code:
    [INFO]  [/127.0.10.11:58158] logged in with entity id 720 at ([world] 519.5, 64.0, 449.5)
    [INFO] Connection reset
    [INFO]  lost connection: disconnect.endOfStream
    cb version
    Code:
    This server is running CraftBukkit version git-Bukkit-jenkins-CraftBukkit++-134 (MC: 1.2.3) (Implementing API version 1.2.3-R0.2-SNAPSHOT)
    and
    Code:
    This server is running CraftBukkit version git-Bukkit-1.2.3-R0.2
    -56-gd686727-b2094jnks (MC: 1.2.3) (Implementing API version 1.2.3-R0.3-SNAPSHOT
    minecraft
    Black screen
     
  14. Offline

    Raphfrk

    Well, yeah, but I haven't updated the plugin yet.
     
  15. Offline

    Simanova

    EvilSeph said 9 minutes ago:
    Does this mean: craftproxy needs an update, too?
     
  16. Offline

    Raphfrk

    Yeah, needs an update. However, it is just a fixed length packet, so easy. I have updated it already.

    There was a problem with the respawn packet, it should work now.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  17. Offline

    rickky

    Thank you!:)
     
  18. Offline

    Simanova

    Our server will upgrade from 1.1 to 1.2.4 today.
    A lot of users are going to test the new (156) craftproxy.

    Cross your fingers Raphfrk :)
     
  19. Offline

    djchefsev

    HOLY SHIT!!!!
     
  20. Offline

    Raphfrk

    What could possibly go wrong :p.
     
  21. Offline

    ultimateowner

    Now when I enter the portal it tells me to connect to the current server I'm on's IP, not the one I'm teleporting to. With and without the proxy on. I have the proxy on both servers running as plugin.

    Ok. Now I got it but it keeps auto reconnecting me back to the same server not the one I'm trying to teleport to.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: Jul 17, 2016
  22. Offline

    Raphfrk

    The globalhostname in /plugins/server port/serverport.txt sets where the name hostname/port for each server.
     
  23. Offline

    ultimateowner

    Pretty sure I got it working now. But whenever I log in it kinda just messes up and spam teleports me then spams the console with errors.
     
  24. Offline

    Raphfrk

    I think I fixed that. However, can you keep server port comments to the server port thread (plus this issue was discussed there).
     
  25. Offline

    FunCraft

    does this only reduce bandwidth or it helps by improving the performance of the server?
    and what's the difference if I have only the server side of this plugin?
     
  26. Offline

    ShadowDrakken

    It doesn't specifically aim to improve performance, but since the server has to send less chunks, it's doing less work, so it will perform better as a result. Albeit not as much improvement as plugins designed specifically for performance gain.
     
  27. Offline

    FunCraft

    So if I only use the server-side of this plugin,will I gain an increase in performance?
     
  28. Offline

    ShadowDrakken

    Edit: my bad, I misread... just running the server side without the client side won't improve performance because the server continues to do the same work since it's handling unmodified connections.

    coulda sworn I just answered that... yes, it will improve performance, however, not by much since it's not designed for that purpose. If you're on a high capacity server (like 20+ people online at all time) you'll see more benefit than on a low capacity server.

    one of the best ways to improve performance is to run CraftBukkit off a RAMdisk on a machine with plenty of high speed RAM and a reasonable CPU (multi-core helps a lot). There's several free options for RAMdisks on all three major OSes. Also look around the various projects here, there's a few that help tweak the server so that it does less work at various expenses (there's always a trade off).
     
  29. Offline

    Raphfrk

    If you only have this plugin on the server side, then it won't do much. It will allow players to automatically reconnect between servers.

    The process for the proxy is

    Server:
    - decompress chunk data
    - determine if the client already has copies of parts of the chunk
    - replace those parts by references
    - recompress the chunk data
    - flag the packet as cached

    << hopefully reduced network data sent >>

    Client:
    - decompress chunk data for flagged packets
    - replace references with the local data
    - recompress the chunk data

    All of the steps on the server use up CPU. However, the advantage is that amount of data you need to send to the client is reduced.

    So, the proxy isn't that useful if the server have a very good connection. It will help for players who have a poor connection.

    What it is good for is servers that are run on good machines, but have a poor upload speed. This normally means servers that are run on home broadband connections.

    To give an example, this site uses approx the following suggestions. (I have nothing to do with that site).

    Upload: (upload bandwidth in Mbits/sec) * 3
    Download: (download bandwidth in Mbits/sec) * 6
    RAM: (RAM in GB) * 7.5

    Anyway, so say you had a machine

    Upload: 1Mbit/sec => 1 * 3 = 3 players
    Download: 5Mbit/sec => 5 * 6 = 30 players
    RAM 8GB => 8 * 7.5 = 60 players

    The lowest value is 3 players, so your machine could handle 3 players and it is limited by upload speed.

    If you use the proxy, then you might get (upload bandwidth in Mbits/sec) * 6 instead of * 3. That would mean with the proxy, you can have 6 players. NOTE: that is just an example, I don't have the actual numbers.

    The reduction in network bandwidth also helps with teleport/login lag spikes, since that is when the proxy does most of its work.

    On the other hand, say you had a hosted machine with really good bandwidth.

    Upload: 100Mbit/sec => 100 * 3 = 300 players
    Download: 100Mbit/sec => 100 * 6 = 600 players
    RAM 4GB => 4 * 7.5 = 30 players

    In that case, you can only handle 30 players, even though you have loads of bandwidth. However, in this case, increasing the upload limit from 300 to 600 with the proxy doesn't help, since it is your server's RAM that is the problem. Also, it may in fact reduce performance, since you are using CPU to run the proxy without getting any benefit.
     
  30. Offline

    FunCraft

    Thanks for the information.
    I now deactivated the plugin because I used only the server side plugin and my hosting service has 100 mb/s upload/download and 4GB of RAM so it doesn't need the proxy to compress chunks.I hope that by removing this the CPU will not be as stressed as before.Thank you very much for your answer and informing me about what the plugin actually does.I recommend this plugin for persons who host their server at home,I used this plugin when I hosted my server on my PC and players said that there was less lag then without the plugin.If you have a great hosting company with great bandwidth then this will not give you many benefits.
     
  31. Offline

    Raphfrk

    That is pretty much the situation. The main point is to reduce bandwidth and if you have loads of bandwidth, then that doesn't help much. However, it uses CPU to get do the compression.

    It might help with players who have very slow connections, but even then, most players have good enough connections to handle the download anyway.
     

Share This Page