[WEB/ADMIN/API] BukGet: The Unofficial BukkitDev JSON API

Discussion in 'Bukkit Tools' started by SteveMcGrath, Dec 4, 2012.

Thread Status:
Not open for further replies.
  1. Offline

    Puseidr

    hmm i'm reading the plugin.yml inside of every .jar-file (plugins) and the information inside is only like:


    but how do i get the slug of plugin? i thought i can use the name/main to search it with your api to get informations about a new version and downloadlink
     
  2. Offline

    SteveMcGrath

    The slug is a BukkitDev context. basically the URL slug that BukkitDev assigned to that plugin. Most people use plugin managers to handle this for them, however as mentioned, parsing the main field seems to be a good idea to add.
     
  3. Offline

    Puseidr

    :) nice so read, so i can use my script to checkup plugins and update them
     
  4. Offline

    SteveMcGrath

  5. Offline

    uvbeenzaned

    SteveMcGrath

    Someone reminded me of this project here on the forums again after I released my own web project that I plan to add a similar API to. For the first note, Bukget is looking awesome! I just looked on your Github page also. Will you be doing anything with the Bukget plugin? I have been dying to have a serious repo plugin. I've even thought about making my own. Let me know if it will be worked on any time soon.

    Also, I've been working on a BukkitDev parsing project called BPM or Bukkit Package Manager. It is a C# console app that can be scripted that can download plugins from BukkitDev through parsing and finds the latest version. It also makes massive indexes of BukkitDev too and puts them out in CSV format. The whole project is here: https://github.com/uvbeenzaned/BPM. The CSV is puts out is always going to be here: http://dw.networkery.co/bukkitdev/dev.bukkit.org-dumps/latest.csv.

    There is also a PHP project I completed recently that parses this CSV and dynamically creates an operator controlled searchable table of plugins through one script here: http://scripts.networkery.co/nbdis/

    Please let me know what you think of these projects and if you think that they are worth anything because I would like to get another professional developers opinion on this. I haven't really gotten a good opinion on this yet. Thanks!
     
  6. Offline

    Antariano

    I started working on the plugin store for SpaceCP today, I'll keep you posted SteveMcGrath :)
     
  7. Offline

    uvbeenzaned

    I'll be looking forward to that!
     
  8. Offline

    Puseidr

    any news about, when we can search the "main"?
     
  9. Offline

    Bertware

    Puseidr likes this.
  10. Offline

    Puseidr

    ah it works!
    but not all plugins are listed with the main-field. can you force an update thru all plugins?
     
  11. Offline

    Bertware

    I think it just takes a few days before all plugins are updated in the database. And of course, the plugin.yml file should be filled out correctly for it to work.
     
  12. Offline

    SteveMcGrath


    Sorry about the delay, apparently Bukkit's forums isnt emailing me when new things happen. No clue why. I'll be the first to say that I know nothing of C#. I'm just a python ninja ;) Now that main is supported, I have actually been thinking of adding in plugin update support into baskit as well now.

    Also, main should have been fully populated within a week of it being added. I did a full generation with all the data, so let me know if anything isn't coming through.
     
  13. Offline

    uvbeenzaned

    Ok I wasn't sure when I posted if you knew anything of c#. Did you check out the PHP search page I posted about? I would appreciate an opinion on that. I also have one question for you, how long does it take your plugin database to populate from the parsing?
     
  14. Offline

    SteveMcGrath

    I did take a look, its an interesting idea. I have been working with a Javascript developer to get a full frontend for the API working for inclusion into the main website, however it's slow-going as he's doing it in his free time.

    As for the plugin database updates, look at the schedule: http://bukget.org/pages/docs/index.html
     
  15. Offline

    uvbeenzaned

    Lol, it's funny your generation schedule is exactly the same as mine. I have windows task scheduler running a BPM script every 6 hrs and then uploading the plugin database to an apache webserver over ftp.
     
  16. Offline

    Jon

  17. Offline

    SteveMcGrath

    Bump.

    There is currently an outage of the bukget API. the server seems to be having massive I/O contention, which is creating massive issues with the API. I have shut down the API for the time being and have start building a ground-up rebuild of the service on a new host. We will be migrating to that host as soon as the fresh DB has been populated. Please note that things like popularity will be imported as soon as the database has been migrated, however it may be entirely possible that some popularity data may be lost.
     
  18. Offline

    uvbeenzaned

    SteveMcGrath
    For a temporary back up, I have this service of mine that anyone may use while Bukget is being worked on. This parseable raw plugin database/API is updated every six hours: http://scripts.networkery.co/nbdis/raw/?all=true This database is sorted by plugin number, name, project link, and last update time.
     
  19. Offline

    SteveMcGrath


    I'm actually just changing over the DNS records to the new host now. The new host has the first 84 pages or so of plugins loaded. I am importing the webstats as we speak. So most people should be able to get the information for the more recent plugins without issue.
     
    FoxDie likes this.
  20. Offline

    FoxDie

    well that explains why i couldnt get my BukGet working thanks for the info looks like ill have to wait
     
  21. Offline

    uvbeenzaned

    Ok cool. I figured I'd try to pitch in and help you through your troubles. :)
     
  22. Offline

    SteveMcGrath


    The API is back up with the 1700 or so most recently updated. Historical data is still being replicated in. I'm already seeing a touch of I/O contention on the newer host (no more than 5% I/O wait thankfully) and I think I reall yneed to get some help to start scaling this out to more servers to distribute the load. If anyone is willing to donate a VPS or physical hardware, we need it!

    Also if anyone has experience in Linux Admin, I need you people as well. I can't manage the system all the time, and I need help. I have a day-job that has me traveling, so I can't always watch over everything.
     
    FoxDie likes this.
  23. Offline

    FoxDie

    thanks for getting the api back up its very nice to use. i would help you but i dont have the money.

    at first i thought my bukget bugged out on me or something so i did a little research and found this
     
  24. Offline

    SteveMcGrath

    FoxDie Well keep in mind we still need more bodies on the bukget team...right now its just myself and I do travel a bit for work...so if I could get some other people to help manage the systems then we would be quicker to respond to problems.
     
  25. Offline

    Bertware

  26. Offline

    SteveMcGrath


    Competent Linux Admins. Understanding of MongoDB, Nginx, and Python is a plus as thats what we are using.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 30, 2016
  27. Offline

    Puseidr

    hmm it looks like your service isn't working correctly, some plugin-updates are not in your api
     
  28. Offline

    SteveMcGrath

    Actually everything is working as expected. The plugin updates cronjob was not reactivated because as of thursday afternoon, the database was still repopulating. Between Flying home from Canada, and assisting with Security & Networking at Thotcon and SecurityBSides Chicago, I havent had the time to login and re-activate the cronjob. This action howerver has been taken and the first update post-repopulation is occuring as I'm typing this up. Sorry for the delay on getting everything running, however please keep in mind that I'm just 1 guy developing and running this, and this is just a hobby project of mine. If you do see issues in the future, please file a ticket at bukget.org.

    Also, as more people are using the API, I do really need help from 2 or 3 people (mostly for coverage) to make sure that any issues can be quickly responded to. I am looking for people with Linux experience who wouldn't mind getting their hands dirty with some python, mongodb, and nginx from time to time (this is what bukget runs on). We are also looking at expanding to support multiple servers to prevent another issue from taking us out like this last one did.
     
  29. Offline

    Bertware

    I do know some stuff about nginx and vps/dedicated servers on linux, don't know python but I do know .NET, php, SQL, ...
    I don't have that much spare time, but I would be able to look into issues instantly if I can get some sort of mail notification for issues.
     
  30. Offline

    SteveMcGrath

    That is something I actually need to get going. Currently I'm just manually checking everything every day or so. I need a better system.
     
Thread Status:
Not open for further replies.

Share This Page