Author Archives: evan

Git, WordPress plugins, and a bit of sanity: Scatter

A few months ago I made a tool called Scatter to help automate deploys and keep them separated from code. This weekend I completely rewrote it in Ruby (from Python) and added some new features (install it with gem install scatter_deploy). One of those new features is shared deploy scripts, which turns out to be a really handy way to deploy WordPress plugins from Git to WordPress.org. I made a 5 minute screencast that shows the process of deploying and updating a WordPress.org plugin without ever touching SVN.

First, a bit of background. WordPress.org’s plugin system uses Subversion. I use Git for all of my projects. While there are a bunch of ways to communicate between the two systems, they’ve all been varying degrees of annoying for me. For a while I was using this workflow from Boone Gorges based on git-svn. It had some big problems though — it’s slow, and it causes huge problems when you try to do things in Git that SVN doesn’t understand (like use submodules). As a result, even Boone doesn’t actually use that Workflow anymore. I also considered this workflow from Daniel Bachhuber, which basically handles the problem by creating two repos in the same “place”. That felt very manual and error-prone though.

When I added support for shared deploy scripts in Scatter, I decided to use that to automate this whole plugin process. I started with Ben Balter’s plugin deploy script and made a few small changes to (1) add some safety checks and (2) build in assumptions that fit Scatter’s process. The result is that I can now run scatter -s wp from anywhere inside a WordPress plugin Git repository, and have everything just work. You can see some more details in the Scatter Deploy Examples readme.

This is not the only thing that Scatter does — it’s a more general deploy tool — but it’s one of the use cases that I think is pretty cool.

Finding the origin of specific code in Git repos

One of my common uses for version control is looking for the origin of a particular piece of code, like a function definition. Both SVN and Git have commands called blame, which lets you see who is responsible for a particular line of code at a particular time. For large, long-term projects, that might change a lot over time and make it hard to find the actual source of a particular bug/feature/oddity/etc.

Git has an amazing command called bisect, which is like blame on steroids. This post has an okay article on using bisect. The very short explanation is that bisect intelligently searches through a repository to find the exact commit that a particular thing happened — for example, when a bug was introduced. As you progress through each check, you tell Git whether the current version is good or bad, and Git searches until it finds the right commit.

You can also use git bisect run to automate the whole thing by giving git a script or command to run. Yesterday I made a very small helper which accepts a regular expression and returns 1 when it’s found and 0 when it’s not. In git bisect terminology, that means that the script “succeeds” when the regex doesn’t match and fails when it does. This might be counterintuitive, but the logic makes sense if you’re trying to find the first occurrence of something.

First, it relies on this script, which I call regex-does-not-exist.sh and saved along with my dotfiles.

#!/usr/bin/env bash

if [[ 0 -eq $(ack "$1" | wc -l) ]]; then
  exit 0
else
  exit 1
fi

Next, I added this function to my .zshrc file (the same would work in your .profile, .bash_profile, etc). Note the magic $LOCAL_DOTFILES, which is just a path to the root where this script lives. You’re on your own for producing that path and locating the file, but in my case it’s this: export LOCAL_DOTFILES="$(readlink ~/.zshrc | xargs dirname )"

git_find_source() {
  git bisect start
  git bisect bad
  git bisect good $(git log --pretty=format:%H | tail -1) # Start from the first commit
  git bisect run $LOCAL_DOTFILES/regex-does-not-exist.sh $1
  git bisect reset
}

Now, I can just the git_find_source command in my shell to find the first time a regex finds a match in a repository’s history. Let’s say I wanted to find out when the wp_remote_get() function was introduced in WordPress. I’ll truncate the output a little bit here using ack.

$ git_find_source "function wp_remote_get\(" | ack "first bad commit"
3c99cfc4f50a6b7819fb1c31e7408cf34c4c4e9f is the first bad commit

Then I can check out that commit and see that it is indeed when the function was first defined.

Git manifest

Moving lots of repositories between computers is a tedious job, so I decided to automate it. Meet Git manifest.

I generally rely on the version control I’m already using to help move code between computers, but I have a bunch of repositories I keep pretty active and moving those around seemed more annoying than it should be. gitmanifest automates it by storing the data in a simple JSON file, then parsing the JSON on the other end and using it to clone all your repositories in the same directory structure.

To create an export file, run gitmanifest from the directory you want to “copy”. It will recurse through all of the child git directories and create a file called gitmanifest.json. Any repositories that don’t have remotes will be skipped.

git-manifest-export

To import your repositories, run gitmanifest /path/to/gitmanifest.json and it will recreate the directory structure for you. Any repositories that conflict with pre-existing directories will be skipped.

git-manifest-import

The Challenge for WordPress Plugin Businesses

A few days ago Pippin Williamson wrote a post about the WordPress plugins business. It’s worth reading, but I’ll paraphrase its main ideas:

  • WordPress plugins are getting more powerful and sophisticated
  • Plugin-based businesses are growing
  • Requirements that used to fall to custom themes are now being solved more granularly with plugins

I agree with these ideas in general. Plugins are certainly getting more powerful and mature. The best examples of this are the plugins-as-platforms that are becoming significantly more popular, like BuddyPress and Pippin’s own Easy Digital Downloads.

At the end of his post, Pippin makes a prediction:

If you are on the edge about entering the world of commercial plugin development, now is the time.

Here’s where I disagree. We got to talking about it on Twitter earlier and I decided that I’d rather take a few extra characters to explain my opinion.


@ I don’t think WP consumers are sophisticated enough yet to support plugins businesses on the scale of theme businesses
@evansolomon
Evan Solomon

First, I’ll start with a premise. The WordPress theme market is a success. It has high end agencies like 10up and Range, consumer-facing marketplaces like ThemeForest, developer ecosystems like Genesis, and great tools like _s. The commercial WordPress theme market is large and growing.

Conversely, the commercial WordPress plugin market is significantly smaller and I think will face a much more difficult path to success.

Themes have a natural source of demand because the easiest thing to understand about a website is the way it looks, which is the primary thing a theme controls. Most technical products have consumers that are much less knowledgable about the products than the creators, and often the creators that win are the ones that market the products in a way that matches the consumers’ understanding of it — even when that understanding is drastically flawed. Themes fit this model perfectly because of the natural tendency to express what you want in your website primarily by its interface.

Plugins tend to be much more abstract. I think understanding a plugin often requires understanding how it works in addition to what it does. The reason is that plugins are a lot less standardized and are naturally suited to do a lot more things, so basic descriptions of what they do are likely to cause more confusion than they solve. In addition to doing a wider variety of things, plugins also tend to do them in a relatively granular way. While you can only have one theme, you can have countless plugins and plugins can interact with (and even require) each other. Again, to make sense of that you need a deeper understanding of the system than just a list of features — you need to be able to make sense of how that interaction works.

In short, plugins are complicated, and complicated things have a smaller market for natural demand, even if they’re ultimately a better solution for customers.

That brings us to customers. WordPress’ greatest strength and source of growth is its usefulness to non-technical people. It can be hard to remember, but in the not-too-distant past it was nearly impossible for a non-technical person to make their own website. While I think it’s still probably not an awesome idea for a lot of people to do it, it’s now incredibly easy and a lot of that is due to the WordPress ecosystem. Despite this being one of WordPress’ greatest strengths, it’s also one of its biggest problems because it creates a market full of people who often don’t really understand what they’re doing. This lack of understanding has actually been a real value to the commercial theme market because its resulted in themes being the hammer to an “every WordPress problem is a nail” syndrome. Themes are often chosen irresponsibly and forced to solve ill-suited problems — see every theme that bundles 15 image sliders and 25 shortcodes. When presented a WordPress requirement, it’s incredibly common for the solution to start with a trip to a “premium theme” website, almost regardless of whether a theme is a logical way to solve the problem.

There are obvious reasons this benefits theme businesses. Similarly, there are obvious reasons this hinders plugin businesses. Here are the assumptions I’m making:

  1. WordPress theme businesses are, in aggregate, a success
  2. Theme business success has been significantly driven by customers’ overfitting WordPress demands to theme solutions
  3. Plugins require sophisticated customers that can think about problems granularly and understand multi-part solutions
  4. The conditions that drive #2 also hinder #3

As I mentioned to Pippin on Twitter, I’d like to be proven wrong. I think a growing plugin market is important to WordPress. There are some outliers, for sure. Ecommerce plugins, an example Pippin chose, are an early success story, as are some strong brands. This might even be a good thing for customers because it’s harder to build a business with bad plugins than bad themes. And it’s certainly a good thing for those people and companies that have found a way to build successful plugin businesses. As a result, I think they’re a lot more defensible than theme businesses. But it’s bad for the market as a whole.

The plugin market needs more demand and more supply to grow to a scale where it can support more businesses. WordPress’ economic moat is driven significantly by the number of people that earn their income from it. That market creates a lot more people that are incentivized to improve WordPress than would otherwise exist. Right now that moat is mostly composed of the businesses that make themes successful — agencies, marketplaces, etc. WordPress would benefit greatly by expanding the pool to plugins, which is why I hope that I am wrong and Pippin is right.

More JavaScript keystroke plugins, Underscore style

Yesterday I wrote a jQuery plugin to bind callbacks to keystrokes. Today I decided to write a second version as an Underscore.js mixin. It’s in the same repository on Github, as _.keystroke, available in CofeeScript, JavaScript, and minified JS.

Functionally it does the exact same thing and works in the exact same way. The only difference is that it’s exposed as _.keyStroke() instead of $.keyStroke(). In theory, if you don’t want to load the whole jQuery library this is a more efficient way to handle keystroke bindings. In reality, I just thought it was fun to think about a new approach to the problem. It also made Aaron Jorbin slightly less disappointed in me.


@ It’s my new mission in life to rid you of your jQuery dependency.
@aaronjorbin
Aaron Jorbin

jQuery keystroke plugin

I am a big fan of keyboard shortcuts. I have been experimenting with customizing some browser behavior using keyboard shortcuts to trigger events and found it kind of annoying. Here is an example of binding handlers to keystrokes in JavaScript (source is in CoffeeScript here). It’s a bit of tedious work.

I looked around for a jQuery plugin to do this but didn’t find one. I’d never written one myself, and last night I decided to try. The result was jQuery.keyStroke. I’m still playing with it a bit, but so far it’s proving useful, at least for me.

Using the plugin looks something like this:

$.keyStroke( 83, function() { 'You pressed ALT + S!'; }, { modKeys: [ 'altKey' ] } );

$.keyStroke() takes two required arguments (requriedKeys and callback) and a third optional (options) argument.

Required

  • requiredKeys: Array of JavaScript keyCodes for your keystroke. Can be an integer (rather than an array) if you only want to use one keyCode, not including modifier keys passed in the options argument. Order is not important, and ordered keystrokes are not supported. This page is helpful for finding keyCodes.
  • callback: A function to call when your keystroke is executed.

Options

Options should be passed via an object, e.g. {arguments: ['foo', 'bar'], context: someValueForThis, modKeys: ['altKey']}

  • arguments: An array of arguments to be passed to your callback when your keystroke is exectuted. The last keydown‘s event object is always passed as the first argument to callback.
  • context: The value of this for your callback.
  • preventDefault: Whether or not to preventDefault() on the keydown event that triggers your keystroke. Defaults to true.
  • modKeys: Array of strings that match keydown event properties and can be used to include modifier keys in your keystroke. The modKeys option is only used if requiredKeys is a single (non-array) value, otherwise it is ignored. Examples: 'altKey', 'ctrlKey', 'metaKey', 'shiftKey'

That’s about it. It works like most other jQuery plugins. Load jQuery first, then load the plugin file. There’s a regular and minified version in the Github repo. If you want to take a look at how it works, you’re probably better off reading the CoffeeScript file than the JavaScript file. The JavaScript is compiled from the CoffeeScript.

Update

I added an Underscore mixin to the same repository. It offers the same API, with $.keyStroke() replaced by _.keyStroke().

Lessons learned in CoffeeScript

A couple months ago I decided to spend a day learning CoffeeScript.

CoffeeScript is a little language that compiles into JavaScript. Underneath all those awkward braces and semicolons, JavaScript has always had a gorgeous object model at its heart. CoffeeScript is an attempt to expose the good parts of JavaScript in a simple way.

coffeescript.org

I could talk a lot about why I like CoffeeScript and how much more enjoyable it is to write than JavaScript. But earlier today I was thinking about something different — good practices I’ve learned from CoffeeScript and incorporated into work in other languages. There are three that stand out for me.

Always return something

By far my favorite CoffeeScript feature is its implicit return values. CoffeeScript functions will automatically return their final values, even without using the return keyword. If you have a function with multiple possible final values, like an if/else conditional, the CoffeeScript compiler is smart about making sure both are return’d. I consistently forgot about this feature at first, and for a while it seemed a little weird, but as I used it more I really started to love it for two reasons. First, it’s a good assumption that the last thing a function does is the best thing for it to return, and at least as good as returning undefined. Second, it means I can always assume that a function returns something.

Always be explicit

CoffeeScript automatically handles one of the most annoying parts of JavaScript, declaring local variables with the var keyword. Any variable you use in CoffeeScript is automatically declared with var at the top of its scope, which is awesome. On the other hand, that means that if you want to use non-local variables you have to explicitly declare them. This confused me a bit at first but now I really enjoy being forced to use, for example, window.foo instead of just foo. It’s much clearer than the implicit approach and reduces lots of possible confusion.

Always be defensive

I debated whether or not to include this one because I think there are some styles of defensive coding that are bad, but CoffeeScript does a great job with it. The easiest example of this is the top level closure that CoffeeScript wraps your code in by default. Most (good) JavaScript you see looks something like this: ( function() { ...some JS here... } )(). That is a closure and it contains the scope of your code so that you don’t accidentally pollute the global namespace. CoffeeScript does this for you automatically, which lets you remove a couple lines of syntactic cruft from your code. The automatic var declarations is another example of good, defensive assumptions. Another good one, which I won’t go into here, is function scope binding via the “fat arrow” syntax (=>).

So…

While all of these are things I’ve picked up or been reminded of from CoffeeScript, I’ve been applying them all to work in other languages, too. It’s reminded me how valuable it can be to explore new approaches to old ideas.

CoffeeScript is an awesome language and makes writing JavaScript a whole lot more fun. I’m trying pretty hard to not write anymore JavaScript and move everything I work on to CoffeeScript. If you want to learn more about it, the CoffeeScript homepage has a lot of good information. I also used Code School’s CoffeeScript course and can’t recommend it highly enough.

Manage lots of Git repositories faster: git pluck

Managing lots of Git repositories can be annoying, especially if you want to keep up with upstream changes but aren’t actively working on the project. Tonight I continued my fascination with making your own tools and wrote a little command line helper to automate the process; it’s called gitpluck and naturally it’s on Github.

Usage

gitpluck recursively searches for git repositories in subdirectories of your current directory and lets you pass arbitrary commands to each of them. The design is strongly inspired by Git core’s git submodule foreach pattern. For example, if you want to run git pull on each of the repositories within your current directory, you could run gitpluck pull. If you wanted to be more specific, you could run gitpluck pull origin master. To see what the state of each repository is, you could run gitpluck status. You get the idea.

Installation

Like most command line tools, gitpluck is most useful if it’s located somewhere that your $PATH has access to. I keep a directory in ~/bin for this sort of thing, and symlink whatever scripts I want to use into there. Something like this should do the trick:

ln -s /path/to/git-pluck/gitpluck ~/bin

Syntactic sugar

I also find that it’s convenient to be able to access gitpluck more like other Git commands, i.e. as git pluck. To do this, I wrote a small function that overloads the “real” git command. It checks for specific instances where I want to customize its behavior, and otherwise passes off my input to Git. This gets loaded in my shell config (e.g. .bashrc or .zshrc).

# Git wrapper
# Used to extend/overload git commands
git() {
  if [[ $1 == "pluck" ]];
    # Remove the `pluck` arg
    shift
    command gitpluck "$@";
  else
    command git "$@";
  fi
}

Non-Git commands

By deafult git pluck assumes you’re going to pass Git commands to each repository, so it prefixes your command with ‘git ‘. That’s why the short syntax like git pluck pull works. If you want to disable the prefixing, you can pass a --nogit flag as either the first or last arugment. For example, git pluck ls --nogit or git pluck --nogit ls would both list the files in each repository.

Fun stuff

The name is inspired by the variety of handy pluck-based helper functions around, like WordPress’ wp_list_pluck() and underscore.js’ _.pluck(). I’ve only been using it for a few hours, but so far it seems pretty useful. Go check it out.

Cogitate: A simple and readable WordPress theme

Over the weekend I wrote a new theme for this site, which I called “Cogitate”.

Cogitate: v. To take careful thought or think carefully about; ponder.

Cogitate is a child theme of the soon-to-be default WordPress theme, Twenty Twelve. It doesn’t change a ton, but I had a few goals with Cogitate that extended past Twenty Twelve:

  • Ridiculously readable text
  • Minimalism without sacrificing fanciness
  • Support a blog and an independent-but-consistently-styled front page

I just released the code on Github here: https://github.com/evansolomon/wp-cogitate

For now, it’s going to stay only on Github and not on WordPress.org for two reasons. First, since I wrote Cogitate primarily for my own use there are probably some parts of the WordPress Theme Review that it wouldn’t pass. Second, I used a couple of PHP features introduced after 5.2.4, which is the minimum WordPress requirement. You can see those in the dependencies section of the README.

There are still a few things that I want to clean up, but overall I’m pretty happy with the result. Twenty Twelve is a great parent theme to work from.

Scatter, a deploy helper

Update: This project has been completely rewritten. The Github readme is up to date and there’s a post about some of the new features here.

Meet Scatter, a handy deploy helper I wrote this weekend.

Deploying web projects can be a bit of a pain, especially when you don’t manage them all the same way. I maintain a couple of sites, like this one — which uses Capistrano — and IsValid — which uses a good old fashioned git pull. It’s also frustrating not to have a good place to keep deploy-related code for open source projects (like IsValid).

At Automattic/WordPress.com, we have an awesome-yet-simple deploy tool that our systems team maintains. From a developer’s point of view, it’s as easy as typing deploy wpcom into a terminal. If I wanted to deploy another project I just change out the argument(s). For example, if I want to deploy our internal “Mission Control” site, I would type deploy mc.

Inspired by our deploy command and Mark Jaquith’s WP Stack, I decided to write a generic command line deploy tool; thus, Scatter was born.

The idea behind Scatter is to support whatever deploy “system” I wanted to use for a project. Scatter does that by letting you define a custom script for each project, called deploy (just an executable, extension-less file) and making it easy to access the one you want with some simple git repository parsing (or explicit arguments). It also supports a fallback to a basic cap deploy if you provide a Capfile, which is what I do for this site.

Scatter is designed to keep your deploy and web code separate and make it easy to access one while developing the other. I wrote a detailed Readme for Scatter, so if you want to know more I recommend you check it out.