[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

Discussion in 'Archived: Plugin Releases' started by Raphfrk, May 1, 2011.

  1. Offline

    Raphfrk

    CraftProxy - Reduce bandwidth use by caching chunk data
    Version: 0.2.0

    Description

    This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

    I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

    Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

    The system can reduce bandwidth by 70-90% (after the 2nd login).

    Setup

    Server

    Add the plugin file to the plugins folder

    Client

    Start minecraft client and login
    Double click on the client jar file
    Enter login details
    Enter the server location/port in the GUI
    Press start on the GUI
    Connect to localhost on the minecraft client

    Stable Builds

    None yet

    Dev Builds

    Warning: These may not be stable

    Client
    Plugin

    Had it get the compress/decompression gain backwards.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
    Jushy, Martin1704, Nathan C and 24 others like this.
  2. Offline

    sfxworks

    So then...what do I use...?
    Is there not a way to cahce the files without having to use a proxy?
     
  3. Offline

    Raphfrk

    You need the proxy. I could update the banned username system, so you can also add IP addresses. However, you wouldn't be able to change it from within the game.
     
  4. Offline

    sfxworks

    Banned username system with an IP address would be real nice. I don't mind editing it out of game.
     
  5. Offline

    Raphfrk

    Update 101 adds this. If you add an IP (one per line) to the banned list file, then that IP will be blocked.

    Your reconnect issue should be fixed by update 100.
     
  6. Offline

    Simanova

    Does white-listing work correctly?
     
  7. Offline

    Raphfrk

    I think so. Basically, only players on the banned list should be able to connect.

    However, I may have broken it with the IP ban list change. What issue are you seeing?
     
  8. Offline

    Simanova

    Only a theorethical question - no testings made^^
     
  9. Offline

    sfxworks

    Unable to open socket to minecraft. My users keep getting that when trying to connect with craftproxylite. Using your 101 Release.

    Here is the server log when someone tries to connect.
    Code:
    2011-06-03 20:07:23 [INFO] 68.210.98.222/52702 (Shenanigans_): Connecting to : localhost 25565
    2011-06-03 20:07:23 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: localhost:25565
    2011-06-03 20:07:24 [INFO] 68.210.98.222/52702 (Shenanigans_): Trying alternative IPs on localhost, this is slow
    2011-06-03 20:07:24 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: 0:0:0:0:0:0:0:1:25565
    2011-06-03 20:07:25 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: 127.0.0.1:25565
    2011-06-03 20:07:26 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: fe80:0:0:0:28bd:ad58:33e7:420b%11:25565
    2011-06-03 20:07:27 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: 192.168.0.2:25565
    2011-06-03 20:07:28 [INFO] 68.210.98.222/52702 (Shenanigans_): Attempting to connect to: fe80:0:0:0:0:100:7f:fffe%12:25565
    2011-06-03 20:07:29 [INFO] 68.210.98.222/52702 (Shenanigans_): Unable to open socket to localhost:25565
    2011-06-03 20:07:29 [INFO] 68.210.98.222/52702 (Shenanigans_): Unable to open server socket
     
  10. Offline

    MX Tombo

    Ok so i've gone over this a bit and have some questions

    1. Where it says name for remote server. Do my players put in my server ip there or in the port section?

    2. I use a wireless connection and i have read that it causes problems, does it? And if so is there a way to fix it

    3. I use hamachi for my server as well, Will this affect the plugin?

    4. Whats local server used for?

    5. Can this work with online mode false?

    Im pretty sure that most of my questions are herp derp :p
     
  11. Offline

    ssechaud

    Am not sure if this is a problem, was using v99 and seen this in the servers instance of the proxy:

    Code:
    [14:16:31] 84.9.247.209/62847 (CurlyJ): Server login successful
    [14:16:31] 84.9.247.209/62847 (CurlyJ): Chunk update packet sent for unallocated chunk -15, 7 adding fake init packet
    [14:16:31] 84.9.247.209/62847 (CurlyJ): Client requested caching mode
    Exception in thread "CompressionThread" java.util.concurrent.RejectedExecutionException
    	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1956)
    	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
    	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
    	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:120)
    	at com.raphfrk.compression.Compressor.compress(Compressor.java:177)
    	at com.raphfrk.craftproxyliter.CompressionManager$CompressionThread.run(CompressionManager.java:74)
    [14:17:06] 84.9.247.209/62847 (CurlyJ): EOF reached
    [14:17:06] 84.9.247.209/62847 (CurlyJ): Unable to flush output stream
    [14:17:06] 84.9.247.209/62847 (CurlyJ): Closed connection to server
    [14:17:06] 84.9.247.209/62847 (CurlyJ): Closed connection to client
    
     
  12. Offline

    Raphfrk

    Where?

    On the GUI, the remote server is the IP of your server.

    On the minecraft client, you connect to localhost:<local port you set using the GUI>

    I think that that using hamachi will cancel the effects of using wireless.

    The local server (on the GUI) is a local proxy server (on the user's machine). They connect to that server, and then that server connects to your server's proxy and then that proxy connects to your actual server.

    There is a diagram in the OP under one of the spoiler tags.

    If you set auth_off, then the server handles authentication and not the proxy.

    What would players type in to directly connect to the server?

    Is the proxy on the same computer as the server?

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 14, 2016
  13. Offline

    sfxworks

    They follow these exact instructions. So everything should be typed in properly (unless its different in 101). Along with that, I have 101 in my plugins folder and so do they. So when they try to connect on localhost:25565, that error occurs. The port specified (1440) is open on my end.
     
  14. Offline

    Raphfrk

    That log was from your proxy server right? If so, the issue is that your (proxy) server isn't able to connect to your (minecraft) server.

    How have you set up your proxy? If your minecraft server is not on port 25565, then you need to update the default_server setting in

    /plugins/CraftProxyLiter/proxylite.txt

    default_server=25565

    The proxy is scanning all local IP addresses to try to find the minecraft server (and failing to find it).

    Also, I am not sure setting buffer latency to 1000 is a good idea. That seems a little high.
     
  15. Offline

    rtcabooservb

    I'm using CB 823 and the latest version of performancetweaks and craftproxy. I notice a slight delay in chat while using these plugins, but when I remove them, the chat is back to no delay. The delay is about 50-100ms. Anyone else experiencing this issue?
     
  16. Offline

    Raphfrk

    Have you set you set the buffer latency to 0?
     
  17. Offline

    rtcabooservb

    Yes I have.
     
  18. Offline

    Raphfrk

    It might be worth trying each of the plugins in turn. CraftProxy may add latency though due to processing time. However, chat is supposed to be considered high priority.
     
  19. Offline

    dompie

    Hi, I have a user that can not connect to my server when using craftproxyliter + serverport. He is able to connect without (serverport + craftproxyliter) wo the server.
    The Message I see on my Gameserver is
    Code:
    13:13:00 [INFO] fex11 [/127.0.0.1:42127] logged in with entity id 238543 at (-3378.3967818499345, 79.0, -377.1263218597)
    13:13:13 [INFO] Connection reset
    13:13:13 [INFO] fex11 lost connection: disconnect.endOfStream
    13:13:21 [INFO] fex11 [/127.0.0.1:42130] logged in with entity id 239645 at (-5.79353731789563, 93.0, 31.48280775825455)
    13:13:26 [INFO] Connection reset
    13:13:26 [INFO] fex11 lost connection: disconnect.endOfStream
    13:13:57 [INFO] fex11 [/127.0.0.1:42133] logged in with entity id 239838 at (-0.967584942198495, 95.0, 24.66295946467743)
    13:14:05 [INFO] Connection reset
    13:14:06 [INFO] fex11 lost connection: disconnect.endOfStream
    13:15:02 [INFO] [Permissions] All worlds reloaded.
    13:16:22 [INFO] fex11 [/127.0.0.1:43246] logged in with entity id 239851 at (-0.967584942198495, 95.0, 24.66295946467743)
    13:16:32 [INFO] Connection reset
    13:16:32 [INFO] fex11 lost connection: disconnect.endOfStream
    13:16:40 [INFO] fex11 [/127.0.0.1:43249] logged in with entity id 239920 at (12.28766057834644, 96.0, 27.749336938479583)
    13:16:48 [INFO] Connection reset
    13:16:48 [INFO] fex11 lost connection: disconnect.endOfStream
    13:16:55 [INFO] fex11 [/127.0.0.1:43252] logged in with entity id 240087 at (6.5048205463821684, 96.0, 17.54805390213158)
    13:16:57 [WARNING] Can't keep up! Did the system time change, or is the server overloaded?
    13:17:03 [INFO] Connection reset
    13:17:03 [INFO] fex11 lost connection: disconnect.endOfStream
    13:17:10 [INFO] fex11 [/127.0.0.1:43255] logged in with entity id 240532 at (1.7561753125471848, 88.0, 2.8267624889638268)
    13:17:17 [INFO] Connection reset
    13:17:17 [INFO] fex11 lost connection: disconnect.endOfStream
    13:17:26 [INFO] fex11 [/127.0.0.1:43258] logged in with entity id 240631 at (10.983239589644658, 83.0, -5.419843567821022)
    13:17:34 [INFO] Connection reset
    13:17:34 [INFO] fex11 lost connection: disconnect.endOfStream
    
    While craftproxyliter tells me:
    Code:
    Buffer mis-calculation for length??
    [1:16:21 PM] 79.235.127.68/49277 (fex11): Attempting to connect to: localhost:25570
    [1:16:21 PM] 79.235.127.68/49277 (fex11): Connection successful
    [1:16:21 PM] 79.235.127.68/49277 (fex11): Connecting using proxy to server connection format
    [1:16:23 PM] 79.235.127.68/49277 (fex11): Authing with EDITED:xxx
    [1:16:24 PM] 79.235.127.68/49277 (fex11): Auth successful
    [1:16:24 PM] 79.235.127.68/49277 (fex11): Server login successful
    Buffer mis-calculation for length??
    [1:16:31 PM] 79.235.127.68/49277 (fex11): Timeout
    [1:16:32 PM] 79.235.127.68/49277 (fex11): Closed connection to server
    [1:16:32 PM] 79.235.127.68/49277 (fex11): Closed connection to client
    Connection from 79.235.127.68/49290
    [1:16:38 PM] 79.235.127.68/49290 (fex11): Attempting to connect to: localhost:25570
    [1:16:38 PM] 79.235.127.68/49290 (fex11): Connection successful
    [1:16:38 PM] 79.235.127.68/49290 (fex11): Connecting using proxy to server connection format
    [1:16:40 PM] 79.235.127.68/49290 (fex11): Authing with EDITED:xxx
    [1:16:41 PM] 79.235.127.68/49290 (fex11): Auth successful
    [1:16:41 PM] 79.235.127.68/49290 (fex11): Server login successful
    Buffer mis-calculation for length??
    [1:16:47 PM] 79.235.127.68/49290 (fex11): Timeout
    [1:16:48 PM] 79.235.127.68/49290 (fex11): Closed connection to server
    [1:16:48 PM] 79.235.127.68/49290 (fex11): Closed connection to client
    Connection from 79.235.127.68/49294
    [1:16:53 PM] 79.235.127.68/49294 (fex11): Attempting to connect to: localhost:25570
    [1:16:53 PM] 79.235.127.68/49294 (fex11): Connection successful
    [1:16:53 PM] 79.235.127.68/49294 (fex11): Connecting using proxy to server connection format
    [1:16:55 PM] 79.235.127.68/49294 (fex11): Authing with EDITED:xxx
    [1:16:55 PM] 79.235.127.68/49294 (fex11): Auth successful
    [1:16:55 PM] 79.235.127.68/49294 (fex11): Server login successful
    Buffer mis-calculation for length??
    [1:17:03 PM] 79.235.127.68/49294 (fex11): Timeout
    [1:17:03 PM] 79.235.127.68/49294 (fex11): Closed connection to server
    [1:17:03 PM] 79.235.127.68/49294 (fex11): Closed connection to client
    Connection from 79.235.127.68/49297
    [1:17:09 PM] 79.235.127.68/49297 (fex11): Attempting to connect to: localhost:25570
    [1:17:09 PM] 79.235.127.68/49297 (fex11): Connection successful
    [1:17:09 PM] 79.235.127.68/49297 (fex11): Connecting using proxy to server connection format
    [1:17:10 PM] 79.235.127.68/49297 (fex11): Authing with EDITED:xxx
    [1:17:10 PM] 77.179.89.218/49410 (mstriker): Chunk update packet sent for unallocated chunk 42, 30 adding fake init packet
    [1:17:10 PM] 79.235.127.68/49297 (fex11): Auth successful
    [1:17:10 PM] 79.235.127.68/49297 (fex11): Server login successful
    [1:17:12 PM] 77.179.89.218/49410 (mstriker): Chunk update packet sent for unallocated chunk 63, 10 adding fake init packet
    Buffer mis-calculation for length??
    [1:17:16 PM] 79.235.127.68/49297 (fex11): Timeout
    [1:17:17 PM] 79.235.127.68/49297 (fex11): Closed connection to server
    
    I'm using CB 823, craftproxy 99 with serverport from yesterday. I have already tried to log in with his account from my PC. That worked. What could be the reason for this behaviour on the client side? I'm afraid this might be something more users might come across in future. I'd like to be prepared.

    EDIT:
    The user sees "kind of ...protocol stream closed..." error.
     
  20. Offline

    Raphfrk

    I wonder if it is some miscalculation due to the length of his name. Have you any other players with 5 letter names?

    I added better error messages.
     
  21. Offline

    dompie

    Only one active. He was not online since I updated to 1.6.

    Seen a minute ago also this one on bash, but not in logfile:
    Code:
    Exception in thread "main" java.lang.NullPointerException
            at com.raphfrk.craftproxyliter.Main.main(Main.java:147)
    
     
  22. Offline

    ssechaud

  23. Offline

    dompie

    Hey, here is the output from craftproxy version 102 regarding my issue.
    Code:
    [5:37:59 PM][5:38:11 PM] 79.235.127.68/49273 (fex11): Connecting to : localhost 25570
    [5:38:11 PM] 79.235.127.68/49273 (fex11): Attempting to connect to: localhost:25570
    [5:38:11 PM] 79.235.127.68/49273 (fex11): Connection successful
    [5:38:11 PM] 79.235.127.68/49273 (fex11): Connecting using proxy to server connection format
    [5:38:12 PM] 79.235.127.68/49273 (fex11): Authing with ...
    [5:38:12 PM] 79.235.127.68/49273 (fex11): Auth successful
    [5:38:12 PM] 79.235.127.68/49273 (fex11): Server login successful
    Buffer mis-calculation for length??
    IO ERROR
    [5:38:18 PM] 79.235.127.68/49273 (fex11): Closed connection to server
    [5:38:18 PM] 79.235.127.68/49273 (fex11): Closed connection to client
    
    I hope this helps.

    EDIT:
    He says he is getting the following error:
    Code:
    java.net.SocketException:Network is unreachable:connect
    
     
  24. Offline

    Chojin

    Hey !
    Nice to see proxy evolution :D

    I have some questions.

    I have 3 proxy. They have 'reconnect connects password blabla' as some options.

    My platform is as this:
    - Main proxy
    - Proxy Server1 (which contains several minecraft servers)
    - Proxy Server2 (which also contains several minecraft servers)

    First question, when I use multiple proxy to reach a server, I see connects file does not work. I see lots of line 'playersomething=' but nothing after =. So it does not store last server (or last proxy) reached by the player.
    Bug ? Or did I misconfigure something ?

    Second question, how handle proxy Server2 in my platform ?
    My Hub is on Server1, so when people connects, they connects to Main proxy -> Proxy Server1 -> Hub minecraft server.
    But when they want to reach a server2 minecraft server, how can I configure things ? I can't just invite hub and server2 minecraft servers because they would send directly player from Hub to Server2 minecraft server without going through Proxy Server2.
    Or I have to hack serverport Hub configuration to tell Server2 minecraft server is reachable at Proxy Server2 or something ?

    Thx !
     
  25. Offline

    Florian K.

    Thanks for making the effort of making this plugin!

    Here is how you could make it save even more bandwith (on the cost of memory and disk space):

    The basic idea

    Store on the server a exact duplicate of what gets sent to each user. When a chunk needs to be send
    again to the user send only the difference between the current state of the chunk and the state of the chunk the client has. Note that the server knows what the client has stored since he has stored that too.

    Advantage & Disadvantage

    Of course this will make the server need much more space to in order to store the world as the clients know it.
    But the gain is very huge: Instead of having to transfer a hole chunk only the few blocks that have changed needs be be sent.
    That can be a house builded by the player or a cactus which has grown one block.

    How to reduce the bandwith used even further

    Going even further the server could store a copy of the chunk as it was when it was first generated. Then the client can simply generate the chunk for himself and the server has only to send what has happend since then. That can be for example a waterflow which started flowing from a source block once the world simulation started.

    This way clients would have to download only the difference between the original generated world and the current state of the world. That can be very very few bytes.

    Possible Problems and what to do about it

    In order to work against for example data corruption the server should generate a checksum of what he thinks the client has and send that too.
     
  26. Offline

    MX Tombo

    Umm but could you please explain "the server handles authentication and not the proxy" a bit further? Or are you basically saying that your plugin doesnt even worry about auth?
     
  27. Offline

    dompie

    If you have "auth" then the proxy will connect to minecraft.net and handle the authentication stuff. No matter what your server.properties says. Turning "auth_off" the proxy skips authentication towards minecraft.net and leaves the authentiaction stuff to the server.

    Hope this makes things clear.
     
  28. Offline

    MX Tombo

    Yep thanks :D
     
  29. Offline

    tomgsx

    I used version 98 of this and a couple of my users randomly got kicked with a long java exception error message and were unable to log back on. The console sent up an error about the chunk and when I teleported to the same chunk they were in prior to getting booted, I got the same message, d/c'ed and could not reconnect until server reset. Has anyone else had this issue? Or do I need to try and repeat it with newest version and post logs?
     
  30. Offline

    dompie

    I have seen today also Buffer miscalculations for other users with longer usernames (than 5 chars). They usually don't have any problems with connections. Are you interested in a full server.log and craftproxy.log ?

    I'm with ServerPort 375 and CraftProxy 102 on the following System:
    Code:
    :~/$ uname -a
    Linux play.matasia.de 2.6.38.2-jb1 #2 SMP Fri Apr 1 09:20:14 CEST 2011 x86_64 GNU/Linux
    :~/$ java -version
    java version "1.6.0_24"
    Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
    Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)
    :~/$ cat /etc/issue
    Ubuntu 10.04.2 LTS \n \l
    
    Sorry for so many postings and reports but CraftProxy handles low level OSI stuff. This is why I'm paying so much attention to it.
    Code:
    [10:42:02 AM] 89.182.181.178/49365 (TanoMano1): Connecting to : localhost 25570
    [10:42:02 AM] 89.182.181.178/49365 (TanoMano1): Attempting to connect to: localhost:25570
    [10:42:02 AM] 89.182.181.178/49365 (TanoMano1): Connection successful
    [10:42:02 AM] 89.182.181.178/49365 (TanoMano1): Connecting using proxy to server connection format
    [10:42:03 AM] 89.182.181.178/49365 (TanoMano1): Authing with ...
    [10:42:04 AM] 89.182.181.178/49365 (TanoMano1): Auth successful
    [10:42:04 AM] 89.182.181.178/49365 (TanoMano1): Server login successful
    IO ERROR
    [10:42:14 AM] 89.182.181.178/49365 (TanoMano1): Timeout
    [10:42:14 AM] 89.182.181.178/49365 (TanoMano1): Unable to flush output stream
    [10:42:14 AM] 89.182.181.178/49365 (TanoMano1): Closed connection to server
    [10:42:14 AM] 89.182.181.178/49365 (TanoMano1): Closed connection to client
    
     
  31. Offline

    Gangsta_hotdog

    I have a question. Does this still allow people not using the client proxy to join? I would like to be able to join my own server over LAN without using the proxy. Also, how much difference would I see on a connection with only 0.50 up speed? I can host around 5 players on that connection at the moment, how many people can I expect to see if I use this mod?
     

Share This Page