Constant server crashing

Hey guys this has been going on ever since 1.2 and we would greatly appreciate ANY help.
Ever since we updated to 1.2 (a couple days after release) our server has been crashing at least once a day, sometimes it will crash 5 times in a row within 2 minutes or eachother.

We are currently using the 1.2.5 bukkit jar. (However this happened prior with 1.2.4 and below)

We get this (2012-04-12 11:02:05 [INFO] <[DB]AgainstMe182>[A]: Ping timeout! Looks like a crash! Check server console (it gives me error in console)) error EVERYTIME the server crashes, as you can see its pretty undescriptive (also this error is IN the server console it tells us to check.)

We are using a host who provides us with 2gb ram for 48 players yet these crashes can happen with 6 people online. Our host has told us that a plugin is mismanaging memory but we've tried removing the most intensive ones, and left the popular ones (since we haven't read any complaints about them.)

We are running Borderguard, Orebfuscator, lagmeter, Worldedit, Anticheat, Permissionsbukkit, Vault, Worldguard, Essentials, Herobounty, essentials protect, simplecarts, essentials spawn, vanishnopacket, essentials chat, precious stones, and simpleclans.

FInally our host has recommended removing a c3po file which I can not find!

 

Give us the console output of the " gc " command please.

 

12.04 11:25:49 [Server] INFO Nether "world_nether": 33chunks,5entities
12.04 11:25:49 [Server] INFO World "world": 2843chunks,881entities
12.04 11:25:49 [Server] INFO Free memory: 1,078 MB
12.04 11:25:49 [Server] INFO Allocated memory: 2,020 MB
12.04 11:25:49 [Server] INFO Maximum memory: 2,020 MB

Thanks for quick reply

That free memory fluctuates drastically all the time.

 

Are all your plugins up to date? Please go to dev.bukkit.org and check.
Do errors appear when the sever crashes? [e.g. outOfMemory errors relating to a certain plugin]

Those 881 entities seem quite odd - Since you have worldedit, I recommend you running these commands in your main world " /remove items 20000" and " /remove boat 20000" and " /remove minecart 20000 "

Then re-post the output of the gc command.

 

12.04 11:36:23 [Server] INFO Nether "world_nether": 33chunks,5entities
12.04 11:36:23 [Server] INFO World "world": 2314chunks,673entities
12.04 11:36:23 [Server] INFO Free memory: 615 MB
12.04 11:36:23 [Server] INFO Allocated memory: 2,015 MB
12.04 11:36:23 [Server] INFO Maximum memory: 2,015 MB

It removed 8 minecart entities and 55 items.

I am pretty sure they are all up to date with the exception of vault. It does not have a folder in my main plugin directory, should it? Or should it be hidden somewhere?

As for errors on the crash, we only get the PING TIMEOUT! check server console error... I can't copy and paste it as log has been cleared since last night. The only correlation that we've noticed is a lot of the time one of our mods or admins will use a /tp /ban command and it seems to follow soon after.

Again, all your help is very much appreciated! This has been a long ongoing problem.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

I don't think Vault will have a folder as it is an API for plugins.

If you can get one of the errors (don't chop it short or anything, full error is needed), that would help greatly.

 

Full server.log please.

 

^ If this guy can't help you, nobody can.

 

Where is a proper place to paste the server log? It is too large for here, and I cannot seem to attach the .log file.

Two things that stick out, 2012-04-11 19:55:25 [INFO] Initializing c3p0-0.9.1.2 [built 21-May-2007 15:04:56; debug? true; trace: 10 (line 1607) This is the file I believe the host pointed out.
A crash happens around line 1100-1200 but I can not find the error.
Please inform me if there is a better place to paste this stuff, thanks again

ok i've found a site...
firsthalf- http://pastebin.com/aux7mSLu
secondhalf- http://pastebin.com/N2NjYYNP

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Code:

2012-04-11 19:39:50 [INFO] Scoopthestuff lost connection: disconnect.quitting
2012-04-11 19:39:50 [SEVERE] java.net.SocketException: Socket closed
2012-04-11 19:39:50 [SEVERE]    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
2012-04-11 19:39:50 [SEVERE]    at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
2012-04-11 19:39:50 [SEVERE]    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
2012-04-11 19:39:50 [SEVERE]    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
2012-04-11 19:39:50 [SEVERE]    at java.io.DataOutputStream.flush(DataOutputStream.java:123)
2012-04-11 19:39:50 [SEVERE]    at net.minecraft.server.NetworkWriterThread.run(SourceFile:103)

Code:

2012-04-12 10:18:34 [SEVERE] Error at SQL INSERT Query: java.sql.SQLException: columns x, y, z, world are not unique
2012-04-12 10:18:34 [SEVERE] Query: INSERT INTO `pstone_fields` (  `x`,  `y`, `z`, `world`, `radius`, `height`, `velocity`, `type_id`, `data`, `owner`, `name`, `packed_allowed`, `last_used`, `flags`) VALUES ( 392,46,543,'world',10,21,0.0,0,0,'_Set','','c:adm','1334251114400','{}');

These are the only severe errors I can find in the log (they occur a few times) but they don't seem like the type that would cause your server to crash.

 

Right, yet this whole time with 7 players on the available ram will go from 80% available down to 30%.

I can also confirm plugins to be up to date except for vault, im hunting for it.

 

Vault update for 1.2.5: http://dev.bukkit.org/server-mods/vault/files/28-vault-1-2-14/

 

Thanks, im just unsure of where to place it in my ftp.
edit:
I see, i'm a fool, the .jar is in plugin folder, it just doesn't create a folder thanks!

 

Download the .jar file and place it in your plugins/ folder.

 

Just an update, we're at 12 players and its available memory is bouncing around from 5% available to 85%

 

It's not an OpenVZ VPS with burstable memory is it?

 

I'm not exactly sure where to look but we are supplied with 2gb dedicated ram

Also I should have mentioned, these crashes always occur with a minimum 5 minute roll back

So I restarted to update vault and after restart we STILL have 675 entities for world. Could this be causing the problem?

Any help? feels like we were getting somewhere.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Is the error listing "ping time out" actually followed by a crash? I see two starts of the server in both those log files - try stopping your server, renaming your server.log to server.bak, starting your server and post the crash again.

Also, if you look at this snippit of your log, there is something going on with your C3PO plugin. Might be worth a shot removing it. While you're at it, make sure the rest of your plugins are up to date.
http://pastebin.com/KkeHcmki

 

I will post the error at my next crash, its been running pretty smooth today, but that won't last forever. All my plugins have been checked and are current version, (this crash has been happening since 1.2 anyway.) Where might I find this c3po file? I can't track it down. This is also the file the host recommended moving.

And of course thanks for any help!

Anyone with word on the c3po file? Still no crash btw, feels odd HOPING for one to happen after it usually happens so consistently.

OK so for whatever reason my server crashed three times last night without giving me a single error, just seems to restart itself. Here is a paste of the three restarts and what was happening right before it.

http://pastebin.com/nCHgKFB7

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Code:

2012-04-13 00:00:09 [INFO] [EssentialsProtect] Enabling EssentialsProtect vPre2.9.1.2
2012-04-13 00:00:09 [INFO] Failed to get local InetAddress for VMID. This is unlikely to matter. At all. We'll add some extra randomness

I get this too! It's really bugging me.

I've asked twice on the essentials topic with no help given whatsoever.

Contact your host - make sure your server hasn't been set to auto-kill/restart itself at X minute/hour intervals.

The crash at line 408 is oddly pretty much exactly at midnight, quite a 'round figure' in terms of time. Check there's definitely no scheduled server stop/kill/whatever.

 

Although it is unfortunate to hear you are having problems as well, I'm glad i'm not alone!
My host however, seems to know of the error and says it is not the cause of any issues.

 

[In case you haven't noticed, I updated my last post]

 

Ah, I see now. Problem is, it crashed once before that at 11:57 and then again after the midnight crash. I think its just a coincidence because any other night we would just crash at random times throughout the day. I just find it extremely odd that it does seem to just be a restart this time around as opposed to a crash, but still nowhere there did it say "Minecraft Stopped" or "Minecraft Restarting."

 

This doesn't look like a bukkit/java issue as there are no errors in your log.
Your best bet is to ask your host or if they're adamant that after carrying out extensive checks there is no issue, make an issue on the bukkit JIRA instance detailing as much information as you possibly can (However I don't recommend going down the JIRA route at all unless you have errors to provide).

 

Try removing essentialprotect, it seems to me that that is one thing causing the error on reload/startup

 

It is causing the error, but essentialsprotect still works.
I can't get rid of it on my server without forcing people to change all their chest protection signs (I'd love to switch to worldguard's chest protect functionality)

 

I'll give it a shot! Luckily its sort of a redundant plugin anyway. Just need to double check that nothing important is protected with it, and i'll let you know. Could take up to a day for another crash to happen as well.

I like preciousstones very much as it allows people to create their own protection, Worldguard for the spawn ; ).

So removing essentials protect has removed the initial row of errors. Good start!

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

They're not errors, they're information. Note the [INFO] tag rather than [SEVERE].

 

Understood, I had to put it back in anyway because I couldn't disable placing lava quickly in Worldguard. As long as its not an error i'll just leave essprotect in.

 

Stop lava placing in WG by editing the blacklist.txt file for the worlds you want to disable it in.

more info: http://wiki.sk89q.com/wiki/WorldGuard/Blacklist