Solved Long loading time related to CommandBlocks

For some time now our server has been suffering from long loading times. Usually it takes up to 5 minutes (or 9/10 on most multicraft hosts we have used so far) to boot up. We only have around 23 plugins, and when I disable them it makes only about 20 seconds max of difference. It happens on all systems: Mac, Windows and Linux, also the amount of RAM or CPU doesn't really matter. Also there are not really large differences between SSD and HDD.

Recently I discovered that if I remove all TileEntities (around 7k), that the server boots in just 5-10 seconds, so I would say it is related to the TileEntities. This is a part of the list of the TileEntities in the server: http://gyazo.com/c5d649ab23962de9595b7f4d0054db32 . After I discovered this I started to research which TileEntity types exactly caused this long boot time.

I did these tests on my PC, on which the server boots in around 269-271 seconds (270 average)
Each test I resetted a backup and removed another type of TileEntities, these were the results:

• Removing 2755signs caused the server to boot only 1 second faster
• Removing 636 skulls caused the server to boot only 2 seconds faster
• Removing 553 CommandBlocks, the server booted in 5 seconds(265 seconds faster)

Then I to set "enable-command-block=false", but the startup time was (kind of obviously) the same.

The "real" question
I was now wondering why loading CommandBlocks takes that large amount of time, while for example the signs and Skulls load A LOT faster. I know that 553 is actually a crazy amount for commandblocks, so using an automated script (by using a custom made Bukkit plugin) I managed to remove 26 empty commandblocks, already shortening the startuptime with 15 seconds. Though, on the other hand it is also weird that 3391 Signs+Skulls take only a combined 3 seconds to load. Is this long loading time normal for CommandBlocks? And does anybody have tips to shorten the boot time other then removing them all?

Edit: I retried removing the full plugin folder and now the server boots in onl 1 second. It seems to be related to Permissions and CommandBlocks and I assume that every plugin that gets loaded, the permissions get recalculated and somehow it affects the CommandBlocks causing the extreme bootup time. I don't really know if I should either report it as an issue or just keep it here.

I found that the CommandBlocks we're actually indirectly the cause. As I said before it is related to permissions. If, for example, I remove all described permissions in the NoCheatPlus plugin.yml (as that plugin contained the most described permissions in the plugin.yml), the server boots around 200 secs faster. As we're using PermissionsEX 2.19, the plugin.yml doesn't seem to be usefull for us so I can clean them up as a temporary fix for our server boot problems. As I assume this is a "bug" in Bukkit I'll mark the topic as solved.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

I've reported this issue ~5 months ago.
We're still running a modified build of CraftBukkit since back then and haven't encountered any problems yet.
I still haven't received an answer from Wolvereness.

That's the ticket I'm talking about: BUKKIT-4823