[MISC] CraftProxy 0.2.0 - Reduce bandwidth use by caching chunk data [1.6]

CraftProxy - Reduce bandwidth use by caching chunk data
Version: 0.2.0

Description

This plugin and local client allows chunk data to be cached locally by players. This reduces the bandwidth required for hosting the server. It also helps users who are on slower connections.

I am not sure what the status of this system is. It doesn't count as a plugin, since it has 2 parts.

Even when running the plugin, players who don't use the client proxy can still connect. However, they will use the full bandwidth.

The system can reduce bandwidth by 70-90% (after the 2nd login).

Setup

Server

Add the plugin file to the plugins folder

Client

Start minecraft client and login
Double click on the client jar file
Enter login details
Enter the server location/port in the GUI
Press start on the GUI
Connect to localhost on the minecraft client

Stable Builds

None yet

Dev Builds

Warning: These may not be stable

Client
Plugin

Had it get the compress/decompression gain backwards.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

You say that Spout can do also all of this here.
How can I use Spout to save the infos, like your reconnect file?
Who Player is on which Server.

 

Iam running Minecraft and CP into a ramdisk - the generated .hprof file was not saved on a harddisk.

This error log was created, as the server crashed --> http://paste2.org/p/1586228

Ill try a previous version of CP ( v106 )

 

If you add -XX:HeapDumpPath=/path/to/generate/heapdump.hprof you can set the location of the dump.

 

Raphfrk, can u give some advices about client-side?

My users play with our own launcher (Anjo Caido's rebuild) and own client.
So what best way to launch CPL hidden and automatically? I think, best way is client integration and run proxy like thread, in client initialization. But this way = 2-3 days of work for me)

I compile launcher with Intellij IDEA. Can i make jar, that will start 2 classes - LaucherFrame and main class of CPL?

 

You would probably need to use another main class that calls them.

 

How to make connection to my server via proxy as the one way to connect?

 

Thx.

I spent the day trying to compile your sources. I have the Bukkit api and I downloaded ebean api, tracked depending on, started to download json. Then he looked at his watch. The day passed.

Can u share a set of API? Or say a string of code that can run jar inside the jar?
I pray a lot, but compiling something for so long (

Thanks a lot.

 

CraftBukkit is what you want to compile. It uses Maven, but has a lot of dependencies.

You could take a look at how the plugin does it. See CraftProxyLiter.java.

It is the main file for the plugin, but right at the end, it calls the main class (in a separate thread) for the proxy server.

Code:

		mainThread = new Thread(new Runnable() {
			public void run() {
				Main.main(argsArray, false);
			}
		});
		
		mainThread.start();

Also, to stop the main thread you can interrupt it.

Code:

Main.killServer();

If you make it so that only the proxy's port is forwarded by your firewall, then players will have to use the proxy.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Thanks for this answer and another big one for this plugin.

 

Testing now CP v105 - if it fails - ill try "-XX:HeapDumpPath=/path/to/generate/heapdump.hprof"

Thanks for help!

 

Btw, as I said before, it might be better to find out which CB version is causing the problem. I can then see what they changed. Even if you narrow it down to 5-10 builds.

Didn't you say 1000 works fine?

 

Ill try now recommended build 1060 - hope it helps

 

Thx. I remove this class from client side and add StartMainWithArgs - is works.

What should i write here, if my client and server have 17 protocol?

Code:

clientversion=

 

17 I guess. I don't think it uses it anymore anyway.

 

Ok, some news:

The CP memory leak does not depends on which CP version you use.
Its defenatly CraftBukkit.

The leak appeared around 1039 - 1043

The leak is still present with CB 1060 RC

 

So, there is no leak with 1039 and a leak with 1043?

The chunk compression changes are the most suspicious .. but dunno. Can you narrow it to 1 change . Anyway, I will have a look at those commits tomarrow and see how I get on.

Also, the heap dump would be cool, I have a program that will scan a heap dump and tell you exactly what the memory leak is made of. I used it to fix 2 Spout memory leaks.

Btw, thanks for all the effort you are putting into this.

Also, what plugins are you running when testing?

 

Is Spoutcraft's built-in caching working fully yet? I want to know if I can let certain users stop using the standalone version.

 

Ill try to generate a heap dump, how large will this file be?

 

Apologies for intruding on a discussion regarding a memory leak, but I've got a little 'feature request' if it can be considered one. With the gradual migration towards using Spout, I was wondering if there are any plans to add a 'Spout pass through' into the standalone CraftProxyLiter which simply passes through Spout packets (which have all the nice stuff done to it already) while doing what it normally does to non-Spout users and still maintain the connection handling as it is at the moment. In other words, CPL does the current authenticating and handling the client connections to the server. If the client is a Spout client, then don't attempt to do any packet modifications and just pass it through to the Spout enabled server. If the client is a CPL client, do its awesome magic. If the client is a stock standard vanilla, pass it through as per normal. One port, three different client needs.

My main rationale behind wanting such a thing is that communicating two different addresses to connect to that are reliant on a conditional (if you're using Spout, connect to this, otherwise connect to that) is trouble waiting to happen, or at the very least a bunch of people are going to get things wrong/sub-optimal. I'd much prefer to do all the nasty configuration server side, by someone with a clue rather than relying on the users to get their configurations done right.

Am I asking too much? Or should I start preparing the documentation to inform my users of the upcoming changes?

 

Hmm, looks like hundreds of MB. You would need to host it somewhere.

One option would be to set the mx parameter to 256MB, so that it dumps quicker.

Something like (but including any other options you normally use).

java -Xms256M -Xmx256M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump2/dump.hprof -jar CraftProxyLiter.jar 20000 25565

HeapDumpPath is a physical disk path, so that it gets saved.

Spout should be compatible with this already. A spout client can connect via the proxy.

Also, I think CPL handles reconnects better than Spout, since it doesn't have to re-authenticate each time you teleport between worlds.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Run client side CPL with this params:

Code:

                params[0] = "25565";
                params[1] = "my.server:20000";
                params[2] = "local_cache";
                params[3] = "auth_off";
                Main.main(params, false);

Client's console:

Code:

Connecting to localhost, 25565
Connection from 127.0.0.1/54250
[12:30:25] 127.0.0.1/54250 (test): Connecting to : my.server 20000
[12:30:25] 127.0.0.1/54250 (test): Attempting to connect to: my.server:20000
[12:30:25] 127.0.0.1/54250 (test): Connection successful
[12:30:25] 127.0.0.1/54250 (test): Connecting using proxy to server connection
format
[12:30:26] 127.0.0.1/54250 (test): Server login successful
Loading file: 64
Loading file: 65
Loading file: 67
Loading file: 69
Loading file: 71

But CPL_cache is empty. Is working cache? Why not save files?

 

That I understand completely. There's currently no other way to connect to my server aside from going through CPL and it's working without a hitch when I'm testing it with the Spout client. I was just curious if CPL avoided double handling the Spout Server/Client connections. I agree completely with the CPL arrangement when it comes to reconnects. Avoiding the re-authentication when hopping between servers and having the reconnect be done effectively internally on the server side is great.

 

Iam able to upload at maximum 20 mb, because my internet connection has 40kb/s upload available.
And every second the upload is crowded the people scream on my server
Should i analyse the dump with your tool?

Your client will save the files when you stop craftproxy by pressing the "stop" button

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

You mean that Spout has caching and so has CPL?

If you directly connect the Spout client to the proxy (so don't run the local proxy at all), it will treat it like a vanilla client, so will set the connect to non-caching.

The Spout based caching will then happen, as normal. CPL will mostly act as a passthrough. It still needs to parse entity data, so that it can destroy all entities.

If you are worried about reconnects happening twice, CPL doesn't actually pass the kick packet to the client, so the Spout client won't actually know about the kick.

Spout has a way to update world seed data, so I might add support for that. This makes biomes (so rain/snow) work properly when teleporting between 2 worlds.

The cache is stored client side. The only reason it creates the directory at all is because it is connecting to the server (so is a client), but the server doesn't actually send cached packets.

This is the tool.

You want to download:

Stand-alone Eclipse RCP Applications

The 64-bit version didn't work for me, so I switched to 32-bit and it worked.

I will get more info when I get home. However, IIRC, it is pretty much

- unzip the file to <directory>
- cd <directory>
- cd mat
- MemoryAnalyzer (*)
- File -> acquire dump file
- Select memory leaks

* This is for linux. On windows, if it doesn't work, try eclipsec.exe

You can then post a screen shot of the result. They normally give 1-2 suspects.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Ok, that answers everything that I wanted to know. Many thanks for your time.

 

Thx.

Is it possible to save files, if CPL is a thread and closing minecraft closes CPL?

Maybe i should to bind the listener of exit button and the function for saving files?

By the way what name have this function?

 

Ill use the eclipsec.exe on windows, because linux has no desktop environment.
Now its time to wait for a crash

 

Yeah, it's annoying. I did a web search and there doesn't seem to be a command to dump memory. Did you set the max memory to 256MB to speed it up?

 

Yes "-Xms256M -Xmx256M"
But CP will not crash until aprox. 2 gig

 

bump