What happened to CI / Jenkins?

Three days ago we were forced to lock down our build server, ci.bukkit.org, due to high load from all of you eagerly smashing F5 to see if a new Recommended Build had come out. While it is awesome that we have such a following that this is even possible, unfortunately CI runs on the same box as the rest of our sites and took down our site and forums whenever CI was under heavy load.

As a result, I had to make the difficult decision of stopping all public access to our build service in order to restore stability and reliability to our website and forums. Once we had done this and the site was mostly reachable and usable, we immediately went on a hunt to determine what the cause of the poor performance was on our servers. Although the site is relatively stable, we're still seeing problems crop up now and again and are doing whatever we can to get down to the bottom of things. As part of this effort, we'll be experiencing minimal downtime on Monday as we'll be updating various aspects of our main server and it will require a reboot. After doing so, we'll continue to monitor the situation and see what we can do to continue to improve things.

So what are we doing to solve these issues? We're actively working on developing a better way for the community to find and acquire our builds and releases and hope to have a suitable solution ready within the next few days.

While we fully intend to continue to utilise Jenkins to provide us with continuous integration and reactive build compiling, we feel that Jenkins isn't a suitable solution for providing a user-friendly download experience for our community. As a result, we've been working on an easy-to use solution better designed to meet the needs of both our developers and our server admins. By combining the invaluable information usually provided by Jenkins (build numbers, changes and associated commits between builds, and so on) with a better user experience and intuitive design, we'll be able to provide an optimised, well cached site that can withstand the level of traffic we see day to day while still providing the community with up-to-date information about the latest builds and more.

Thanks for your patience and understanding while we investigate the cause of the issues we've been having and make changes to address them, your continued unwavering support means the world to us.

 

Thanks for keeping us up-to-date

 

Thank you!

 

Um... You can look at the Bukkit Repository, and there's either a recommended or an in-dev build for 1.1 there...
And yeah, thanks EvilSeph for keeping us up to date and for trying to keep us happy. xD

 

Thanks for the update. Looking forward to whatever you guys decide and how to do it.

 

Yeah, cheers

 

This SYN ACK flood DOS wouldn't happen if people were on Twitter

 

Thanks for keeping us informed!

 

Thanks for letting us know
Maybe you could put the Jenkins side stuff of another server or limit the resources its aloud to use.

 

Thanks for informing us

 

When will bukkit update to 1.1?

 

disable the auto refresh feature.

 

When will bukkit update? to 1.1

 

Would it be possible for you to have a newsletter system? That would email us every time a new update comes out and we would be able to choose if we wanted RBs and/or dev builds.

This would be an awesome feature and I can see alot of people using this.

Keir

 

And here I thought I was the only one frantically using my face to repeatedly mash the refresh button in anticipation for the next dev build.

 

eh ?!?! the server was overloaded because many used it ... it was not attacked

 

I like iKeirNez 's idea. That would also reduce the amount of people spamming F5 on Jenkins, and now will probably spam F5 on their email. Also, any estimate on when bukkit will update to 1.1?

 

I pretty much like the jenkins download - its much nicer than some weird download pages that do nothing but being useless.
Also you might want to nice jenkins to 5 or 10 - as i have learned that jenkins WILL take all your ressources.
So just make jenkins not be that priorised when taking up all the ressources
at least that solved all my problems with a ci server running a webserver.

 

A dDoS attack is a load of computers sending loads of requests to server, with the intend to overload it.
A load of Bukkit fans who spam refresh requests (f5) to a server can have the same effect (it is actually the same...).

Hibris meant that if more people used Twitter (to read, not to puke their lives on the web) and would follow Craftbukkit, they would know the instand a RB is released, so they dont have to f5 spam Jenkins.

So one could state that the Bukkit community dDoS'd their own server .

 

Thank you so much for the update, keep up the great work guys it means a lot.

 

Thats it pretty much.

 

People need to learn to press F5, wait 3-4 hours, then press F5 again. A RB doesn't come out every couple of minutes.

P.S. Will we still be able to find the changelog from build to build somewhere after these changes?

 

It's kinda funny. Every time there's a new thread, regardless of the topic, a whole crowd of people cry out to the same tune: When will version x.y be out. We all want the next RB but it won't be here any sooner because we ask

Dear bukkit-team. Would there be any way to communicate your progress to the community? I mean, after the initial 3 steps of your update (which you do communicate nicely), you move on into bugfixing (and building support for new features?) and things just gets quiet except for a barrage of @CraftBukkit tweets. An indication like "fixed n of y issues" would be freat. I know the 'y' will increase as you go on and more issues emerge, but eventually the 'x' and 'y' will meet and we have a RB. Just the numbers would be fine (on a page that can handle refreshing). Or maybe even a simple graph?

Anyway, good work and thanks for spending your spare time on bukkit!

 

Hm and everybody know think this is good and make all happy? Transparency like Mojang do is good, transparency like bukkit team do...hm make your own view of it....

 

The Jenkins UI is actually good... we get access to any revision, we see a build status icon, we get comments on the build to determine if we want to try it and we can use back and next to step through builds. Perhaps you need a separate web server or file server, but I wouldn't put the blame on Jenkins.

If you do create a new UI, please provide access to all successful builds with comments for each build.

Thanks for publishing the link to this thread on twitter.

Good call. You could use a communications volunteer that is aware of what you are doing, but not actually doing coding, and let that person focus on external communications. Reasonable people don't mind a 25-day delay if they know how the progress is going.

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

Thanks EvilSeph! Good luck .

 

Upload latest build to S3 temporarily?

 

I vote for a simple post-build step added to the Jenkins jobs. In that step, just append the current build tag and git revision, dependent build tag and git revision, and a link to the jar in the public maven repository to a static HTML file, and make that file publicly accessible. That's all anybody who isn't actually making commits or altering build parameters needs, and you avoid the load of having everybody go through Winstone, which doesn't seem to be the most efficient server ever written.

 

I love the ideas of both the emails and the x of y issues resolved indicator. People are just more satisfied if they think they have a progress estimate, myself included. Either way, thanks for all you do, Bukkit team.