Plugins Site Update: The Old Is New Again

Posted on by

We’ve gotten a lot of feedback since last week’s announcement about the plugins site’s unfortunate tumble into oblivion, and I’d like to address a few of the most important concerns that have surfaced since.

“Could you make the old backup available for posterity?”

Yes. We can — and have. Over the weekend, we restored the most recent backup we had, and the original site is now living at archive.plugins.jquery.com; you should be able to browse through everything that’s there to your heart’s content. We also applied the most recent user information we had, so if you had an account on the old site at any point in the last year, it should still work. However, the site is closed to new user registrations. If you really need a new account, please get in touch with me personally and I can get that straightened out for you. We’ve also set up a redirect, so that if you should encounter any links to plugins.jquery.com in your browsing, you’ll (hopefully) end up at the corresponding page in the archive.

Just get a backup from the Wayback Machine!

While the Internet Archive has cached versions of content that was updated more recently than last October, we just don’t have the people-power to re-create the lost posts manually in the new archive site. If you have an account, you can feel free to add “new” or old plugins, or update existing ones, should you desire to. However, this archive will not be indexed by search engines.

If you hate CMS-es so much, what’s with WordPress?

We’re in the middle of a network-wide redesign, and WordPress offers us a valuable set of tools when it comes to theming, searching, and serving a group of sites. Our new motto, however, is pull requests, not passwords; we’re implementing theming, documentation, plugins, and more in such a way that contribution will not actually require an account on our CMS at all. As I outlined in the initial post, the plugin submission process will only involve adding a post-receive hook to your repository. In the event of a similar catastrophe, we’re made sure we’ll be able to replay the entire plugin contribution history and get the site back up to speed right away. Our goal is to leverage the WordPress features we find useful without it serving as a barrier to entry or as the canonical warehouse of content. If you are of the mind that WordPress is always a bad idea, no matter what, no matter how, you’re certainly entitled to that opinion, but at this point, it’s not particularly beneficial to the conversation.

Git(Hub) is hard

The new plugins site will serve as an index of plugins, with a simple “download” button right on each plugin’s page. You will not have to just browse around GitHub looking for jQuery plugins. If you don’t know git and only ever want to download jQuery plugins, you don’t have to learn it. However, if you want to submit plugins, you’ll have to be using some sort of source control that you can at least mirror in git. This is by design: it can be really easy to build a jQuery plugin, but that doesn’t mean it’s necessarily fit for public consumption. Requiring the use of source control and package.json are passive mechanisms that will help ensure that plugins which proliferate are authored by developers who have met a reasonable baseline (and aren’t selling batteries). We’re only targeting GitHub support for launch, but we’d like to add support for other services as well. We are actively avoiding the use of GitHub-specific features that would force us to limit the site to GithHub users permanently.

It’s A Conspiracy!

Some have called into question the veracity of my account, and that’s understandable, given the timing and circumstances. But believe me, the last thing I wanted to do after spending a day manually pruning spam from the directory was turn around and cause a gigantic headache for thousands of people, including myself and my colleagues. I hope the re-launching of the last backup at least partially allays these concerns. Additionally, we’re starting off with GitHub simply because it has a very broad user base already, and it has been incredibly positive for us since we shifted to it for development of jQuery Core, UI, and Mobile.

Thanks again for bearing with us during this transition.

38 thoughts on “Plugins Site Update: The Old Is New Again

  1. Thanks for the backup restore, and for the hard work.

    One of my suggestions/question is unanswered: Why did the hoster not have daily backups with at least a week of daily backup archives ? That’s unbelievable from a serious hoster.

    And of course: Has lesson been taken and are now *all* jQuery sites backed up daily at hoster and weekly off-site at to a jQuery foundation disk ?

    This assurance is needed to prevent the next failure from having consequences. Too much people rely on jQuery to not have something serious in place for the hosting.

  2. Dan Heberden on said:

    @Beat –

    No, that’s been our responsibility and some of the backups are hit and miss; we’ve been looking to improve this (among many other things) for our own sanity, let alone responding to emergencies. We’re actually pretty close to getting things setup properly including the added benefit of being easier to manage.

    Learning a lesson? Sure – but it’s not like we’re full-time employees getting paid to do this, so jumping around obligations of work, family and personal lives – particularly around the holidays – is a difficult thing to manage; not to say it doesn’t need to be done, and that you aren’t right, but things like getting the backup up and running, Adam writing blog posts to the community to communicate the situation, it all takes time ya know?

    As for assurance, I can’t promise things will be perfect; but those part of the jQuery team are incredibly passionate about this project and work really hard to sustain and maintain it. Like, we’ve been spending so much time – our own personal time – to update our deployment, storage, workflow, etc of our websites. We’ve worked hard to make it something that can be public and more easily contributed to.

    Thanks for sharing your concerns and perspective about all of this – it’s nice to hear/see what the community feels is important :) Trust me, we’re working hard to make sure everything is awesome :D

  3. JustinKramp on said:

    Keep up the good work. I wholeheartedly applaud your plans to move forward with WordPress and Github. I use/develop WordPress sites daily and keep wanting to use Git effectively so it makes sense to me. As a relative newcomer to jQuery I am excited to see what happens next and can only hope this community can continue to support the efforts of voluteers such as yourself/selves.

  4. Requiring Github or any other form of source-control just to prevent spammers is a design flaw.

    The problem was that the submit system didn’t do anything to prevent spam. Fixing that doesn’t require a GitHub dependancy at all. Source-control could be a seperate thing altogether to add versioning and bugtracking for those who choose to use it.

  5. Jay Evans on said:

    Just wanted to say thanks for all the hard work and am looking forward to seeing what you do next!

    Jay

  6. jQuery Rocks on said:

    @River: I do agree that requiring GitHub/source control is not a good technique for reducing spam. I’m also confident that a useful, eloquent plugin for jQuery could be authored without the use of source control.

    Since the designers of the new plugins section are using WordPress, there are excellent plugins available for that CMS for just about everything; I know Akismet is a great plugin for comment spam, but as for uploads of code, I’m not sure what the specific solution would be.

    At the same time, if everyone had their source code on their own GitHub account, it would not have been possible for the “plugins database” to be permanently deleted, along with the source code – repairing the plugins site would have been much easier, as the actual code would not have been concentrated on the plugins site’s database.

    Perhaps they’re trying to avoid this sort of debacle in the future? Also, for the more complex plugins, source control is probably needed to retain some sort of organization anyway.

  7. I agree that using something like Github to establish a baseline for developers submitting plugins will raise and maintain the quality of the plugins. There was far too much junk out there, and the good stuff was making its way to Github anyway, so jQuery’s move to Github is fitting in a way. You have my applause.

  8. @jQuery Rocks: They shouldn’t need Github to guard against moderation screwups, simply not offering the hosting of scripts and just having an index they backup regularly would take care of that.

    A plugin index shouldn’t be dictating how you write or where you host your code.

  9. If you guys can possibly indicate an Expected Date for the new version would be much appriciated.

    Had the jQuery plugin fully available on github for quite a while even before the site crashed, and the callback is already set up, all I need to do is push the files once available. So an expected time-frame would be helpful rather then having to check the site on a daily basis. :)

  10. I’m looking forward to the new plugin listing site. I never really liked the old one. Plus I finally made my first plugin and I’m anxious to throw it up for everyone.

  11. Sunil M on said:

    should have started the announcement with ‘Good news everybody I deleted the entire database!’ :) One time I worked on a college final paper and finally finished in triumph at 3am, but before I saved it windows 98 crashed…It’s not a pretty sight to see a grown man cry.

  12. Hi guys :) A bit of a random question sorry but I can’t find any other way to contact you.

    Is there an RSS feed for this blog, or ideally a feed or email list to be on to be notified when new stable versions of jQuery have been released?

  13. I’m actually quite happy that the plugin archive got murdered that way! The whole thing was half-filled with old and buggy plugins. And going the “github” and “npm” way is awesome news… these tools are what the future of web development is about.

    Anyone knows if anything else need “murduring” on the web ? All those thousands of CMS ? Maybe Adam could destroy half of those too ?

  14. Matthew on said:

    I’m with Mathieu. I think it’s an extremely happy accident that the plugin archive was nuked, as it had become next to useless, and searching for plugins relevant to what you actually wanted was damn near impossible. Kudos, my friend, for deleting everything and forcing a change. I’m sure it was an innocent “mistake”. (Nudge nudge wink wink say no more.)

  15. JQuery has turn Javascript in Jesusqurist. Has saved the internet, and myself from getting crazy with Internet Explorer bugs.

    The old plugin archive was a useless mess. Finding stuff there was horrible ineficient. Part of it is thie 800px wide design. Hope the next site is as good as jquery deserve.

  16. hmmmWhere on said:

    Now if we could only get the Jquery UI domain re-registered I might be able to push forward.

  17. I kinda like the idea of the move to github, not because its source controlled, but like said, it will get rid of some of those sunday coders ..tho not all.

    To have a solid plugin architecture, you need on one side things like voting, rating, listing by popular, etc. This is really, really important to wade through tons of content, and push the crud to the bottom of the listings.

    On the other side you need to make sure that not every single john doe can contribute to you repository, this can be achieved in many ways, peer review, forcing ppl to use more professional coding tools/approaches.

    Maybe one thing that could benefit the repository, is to have some automatic jslint parsing or the like of the code, and establish a note on that ..underneath a certain note, you get admission to the listing rejected. you could even refuse plugins that don’t adhere to a certain level of code documentation, which i think is very important too.

  18. Without even considering github, this could all also happened through a simple user-interace in the spirit of theme-roller (well kinda far fetched) where the user uploads a compliant zip package, the site unpacks the extension, does some code analysis, and returns info to the submitter to say that the plugin was accepted or simply list what kind of errors where found in his code (bad coding style, missing documentation …).

  19. I like the idea of the repo being based on git, with github being one of the places it exists. Everyone remember, Git doesn’t equal GitHub (necessarily).

    I also really want to believe in conspiracy… Something inside me says (and wants to believe) that somebody was ordered to push the big red button.

  20. Bill on said:

    While I appreciate that its extremely difficult to give a definitive delivery date for the new plugins site, not having a date or even a progress indication is really hurting the credibility of jQuery.

    Is it possible to have some indication as regards how far you have progressed and, in broad terms, the target date to have something (maybe Alpha or Beta) for users to see and comment on.

    Having the Backup site on-line is a major step forward but it makes it look as though jQuery (plugins) is frozen in time…

  21. Guys, the new site is nowhere to be seen months after the down-time. And the old archive site… its down as well. Back to square one…

  22. I’m with Mathieu. I think it’s an extremely happy accident that the plugin archive was nuked, as it had become next to useless, and searching for plugins relevant to what you actually wanted was damn near impossible. Kudos, my friend, for deleting everything and forcing a change. I’m sure it was an innocent “mistake”. (Nudge nudge wink wink say no more.)

  23. I like the idea of the repo being based on git, with github being one of the places it exists. Everyone remember, Git doesn’t equal GitHub (necessarily).

    I also really want to believe in conspiracy… Something inside me says (and wants to believe) that somebody was ordered to push the big red button.

  24. Requiring Github or any other form of source-control just to prevent spammers is a design flaw.

    The problem was that the submit system didn’t do anything to prevent spam. Fixing that doesn’t require a GitHub dependancy at all. Source-control could be a seperate thing altogether to add versioning and bugtracking for those who choose to use it.

  25. I agree that using something like Github to establish a baseline for developers submitting plugins will raise and maintain the quality of the plugins. There was far too much junk out there, and the good stuff was making its way to Github anyway, so jQuery’s move to Github is fitting in a way. You have my applause.