What Is Happening To The jQuery Plugins Site?
For about a week, instead of hosting several hundred jQuery plugins and several thousand advertisements for laptop batteries, our plugins repository has been serving up a pretty pathetic message about spam and an allusion to some “new submission process.” This happened very suddenly, and we’re sorry to everyone who’s been inconvenienced. Please allow me a few minutes to explain what happened, where we’re headed, and how it impacts you. If you’re in a rush, here’s the short version.
Though the plugins site you’ve known
and loved was a valuable tool when it was first set up, it gradually became something of a white elephant for the project. While powerful distribution tools like GitHub and npm have come to the fore, we’ve been stuck in an aging, CMS-oriented paradigm that frustrated developers and consumers of plugins alike. Many people moved onto alternative sources for finding and vetting plugins. Furthermore, the sites’s original implementors and maintainers had since moved on from active involvement within the jQuery project. While the team faced a steady stream of complaints about usability and a general lack of features, the site itself faced a veritable barrage of (several flavors of) spam. There was your standard, keyword-rich SEO garbage, but there was also something slightly more insidious: the batch posting of under-documented, demo-free plugins with links to paid download sites by third parties trawling for affiliate cash. While this wasn’t explcitly disallowed, it led to a terrible, confusing experience for users and gave the site the distinct sense that all was not on the up and up.
The Best Laid Plans…
Sensitive to all these problems, the team began drawing up requirements for a complete overhaul. We knew we wanted to get out of the business of accepting uploads, serving downloads, and generally legislating the plugin “release” process on our own servers. With our official projects already on GitHub, we knew we wanted to leverage the many wheels they’d already invented for distribution, versioning, and facilitating open, collaborative development. Another requirement was to create a standard schema for authors to clearly delineate dependencies, like which version(s) of jQuery a plugin supports, as well as other plugins and CSS assets it requires to be functional. We also wanted to introduce some quality control, with both passive mechanisms like requiring GitHub and a manifest file for distribution, as well as active ones like user ratings. Finally, we wanted a clean slate; with stricter requirements for submission, there could be no mass import of all the old plugins. We made a lot of progress preparing specifications, but hampered by a lack of resources and a number of other projects, we never were able to get too deep into the implementation phase. After all, the old site “worked!”
…Often Go Awry
As the glut of spam grew worse and multiple reports started showing up on the jQuery Core bug tracker, I wanted to at least take some steps to wipe a bit of the spam and egg off our face. With newly provisioned access to the administrative tools on the site, I teamed up with the Drupal Views Bulk Operations module and set out to identify and delete spammers and their posts. Within a day or two, nearly 90% of the alleged “content” was gone from the site. I continued to monitor the situation over the ensuing days, and deleted spam as it came in. Unfortunately, I likely cast too wide of a net, and threw out several perfectly good babies with the bathwater. Even more unfortunately, I didn’t back up the database before I began this process. At this point, you can probably see where this story is headed. Later in the week, while I was attempting to delete four spam items, I was left completely horrified when the results of the operation reported that the remaining 10% — every single plugin remaining in the database — had been purged. All that remained was a year-old backup. Of course, I realize that this is a dreadful outcome, and I take full responsibility for it. If it helps, I am very receptive to hate mail and tweets reprimanding me for being irresponsible, unprofessional, or just stupid.
At this juncture, we were left with two choices.
- Keep on kicking the can down the road: Restore from the old backup, losing a lot of data anyway, and have authors go through the painstaking process of re-uploading their plugins, even though the site was slated to be deprecated entirely anyway.
- Use this glaring mistake as the impetus to hunker down, cleave from the past, and finally implement the site we’ve talked about for so long.
We’ve chosen the latter.
Nuts And Bolts
Over the past few days, we’ve started converting our plans into action, building out an infrastructure that’s backed by GitHub. There are two requirements for listing a plugin on the new site:
- A valid package.json file
We’ve followed the lead of CommonJS and npm and created a schema for specifying dependencies, delivery, and other metadata of jQuery plugins. While the format is largely similar to those other projects, we’ve had to make some minor tweaks to account for some plugin-specific details.
- At least one versioned release
This means having tagged your release point(s) with a valid semantic version number (semver) string.
We’ve pared down the submission and maintenance process to a single, one-time step: adding a post-receive hook to your plugin’s GitHub repository. Assuming your plugin meets the guidelines, a page will be created on the plugins site to present your usage and download information. We’ll keep track of new releases as you push them.
In The Interim
We recognize that the old site was still serving as an regular resource for a lot of people, especially newer jQuery users, who simply valued the existence of a central browsing location, despite its flaws. Though many experienced users had moved on to other sites, or relied on relationships with trusted authors and word of mouth, these can take time to develop. Until we’re able to launch the new system, we’re happy to direct you to several other directories and people who can help pick up the slack:
If, in the process of searching these or any other directories, you are directed back to the current site at plugins.jquery.com, typically a quick search for the author’s name and the plugin name will yield an alternative site where it was hosted. In addition, DailyJS just did a useful roundup of alternatives that goes into more detail.
Next Steps For Plugin Authors
If you’re a plugin developer who wants to make sure your plugins will be ready to go on day one (or even beforehand), you’ll want to make sure your plugins are up on GitHub, and then you’ll want to get started on creating your package.json files and making sure your versions are appropriately tagged. If you aren’t already familiar with Git and GitHub, then this is probably a very good time for you to take the plunge and get started. If you prefer another SCM system, you can look into setting up a mirror to git from svn or hg, or other project hosting sites, like bitbucket or gitorious. If you don’t develop your plugins as open source, or you don’t use any source control at all, we will not be able to accomodate you at this time.
How Can I Help?
Though the site is still a work in progress, you can track progress and even set up your own local development right now at github.com/jquery/plugins.jquery.com/. We’re still working on getting a public staging environment together.
Though the site is essentially “powered by” GitHub, there are obviously a number of moving parts in play. The site will actually be served from inside of WordPress, which is populated with the contents of the indexed plugins using a node.js tool. There are still many kinks to be ironed out, so if that sounds like a stack you want to jump on, please join us over at the repo. We’ll be using GitHub Issues for all feature requests, bugs, and discussion.
While we’ve put a great deal of thought into putting together our
package.json schema, it’s still a living document, and if you have questions or comments on why it is the way that it is, head on over to this issue.
In an ideal world, this certainly wouldn’t have happened exactly as it did. Sadly, it did. We hope you’ll accept our apologies for the many ways this transition might make your job harder, give you some extra work you weren’t expecting, or just plain leave a bad taste in your mouth. If you are able to forgive us, bear with us, and maybe even lend a hand, we’re confident we’ll be able to deliver the modern, useful plugins site the community deserves.
So that was a lot of words, here’s the takeaway.
- We’d been planning on replacing the original, spam-ridden plugin site for quite some time
- In the process of deleting on the spam, all the plugins were deleted and we didn’t have a recent backup
- Instead of burning cycles keeping the old site on life support, we decided to make a clean break and kick development on the new site into high gear.
- The new site is powered by GitHub and a package manifest for plugins
- Plugins from the old site were never going to be automatically imported into the new one.
- The repo is at github.com/jquery/plugins.jquery.com
- We are very sorry, but also very excited!
Reply to jQuery Rocks:
I certainly didn’t mean to come across in anyway critical to the jquery teams! We are lucky to have a dedicated and talented teaming working for the user community. It is just that it is frustrating to have no way (for a considerable period) for the users to contribute their applications, based on jQuery, and for the rest of us to be made aware of the options out there. Otherwise we are re-inventing wheels the whole time.
I think if we know what was happening and when the plans were to deliver an alternative (or just reinstate the old system for a while) then we would all feel better. As you say a date for UI 1.9 would be of interest too!
I wasn’t really referring to your posts, Bill – I was mostly talking about others’ comments that the jQuery team should be fired, and where people were swearing, calling the author of the article various things, etc.
Hey! We really miss the Plugin site. Can you mail us about the progress?
Reply to jQuery Rocks
That’s a relief – Though I’d caused offence – I am a jQuery fan too.
I do rely a lot on jQuery plugins (and some of my own coding) to make my websites memorable. Without the PlugIns site I’m struggling to reInvent the wheel and my productivity has fallen off a cliff.
I do agree that abuse and calling people names is never the right solution, and also counter-productive as I’m sure it will demotivate the talented group who provide this excellent resource.
I am using Jquery Idle timeout plugin it’s working fine for aspx pages but it’s not working for fancybox modalpopup. when i clickon OK to continue from fancybox modalpopup then my session is expiring and going to logoff page. Does anybody has idea about this?
In the past I have found JQuery plugin site extremely useful. The only time the excuse regarding ‘several thousand advertisements for laptop batteries’ came up was following the discovery that no one had actually backed up irreplaceable data. All concerned should have a career rethink and leave this kind of stuff to experts.
P.S. Just how long is this going to take to clear up? First post appeared on 8th December 2011.
Oh I miss the old plugin site!
Through this experience you’ll never forgot to make a backup, I think.
Good luck for the new site! Could you crate an email reminder for people who want to use this site as fast as one can?
I think that’s the kind of thing you only learn the hard way. Most engineers out there will probably relate to this :-)
Anyways, the important thing is when will the new site be up ? The old site probably had its problems, but having nothing to work with is worse.
I belive you’ve missed one important point while moving to github…Simplicity… Old repository was really simple, not perfect but more usable for most users…
No activity on this project and no updates? jQuery core is starting to look like a ******* unprofessional mess.
Love jquery, starting to REALLY resent the website, more specifically the plugins pages. Can’t ever seem to download anything, the damn “we’re fixing this” box at the top has been there seemingly forever and github as an alternative is a mess. Bad judgement call there.
Please guys, it’s going on three months since you took down the plugins. Even as messy as you thougtht the old plugin pages were, it was far superior to no plugin pages at all. You could have left things “as is” until you launched the new plugins section. Three months is just unacceptable any way you spin this.
Should have used this as well http://drupal.org/project/backup_migrate
Excelent Ideia. Go on! If you need some help I can help to put this on.