Ryan wrote a really great comment over at rc3.org in response to an interview Scott did talking about how we work over at GitHub. Since I can’t seem to link to the comment directly, I’m reproducing it here:
Having worked in this environment for a little while, I’m not sure I agree that these are really prerequisites, but I’ll try to comment on each:
Your developers must be users of the product.
This is probably the key to everything. Seriously. Take everything below with a grain of salt.
Your developers must be able to iterate without relying too much on other members of the team.
We rely on each other quite a lot, actually. When somebody starts pasting screen caps of some concept into the campfire and they’re really good, people want to jump in and be a part of it. Sometimes you’re working on something and are totally content and loving it but you see that somebody else is really close and they could use a hand and so you make a decision to drop what you’re doing and switch over to that because, hey, let’s just ship that real quick because it’s close and it’s amazing and it’s not like the thing you’re working on has a deadline.
There’s a tremendous amount of freedom and flexibility. More times than not, that results in people using that flexibility to work together in ways that may not have been foreseen or planned.
There’s also times when people don’t jump in on a new concept. Those things may move more slowly. Maybe they should move more slowly. There’s times when something is worked on for a couple of days and then people lose interest for whatever reason. Maybe it’s not interesting.
I think this is part of what Scott was referring to when he said, “open source software style of business”. In an open source project, it’s hard for bad ideas to gain a lot of support. The maintainer doesn’t really get to tell other contributors what to work on. They can set a vision and tone, but an idea sinks or swims largely on its own merits and whether the people contributing feel it’s worthwhile as a collective.
I guess conventional wisdom is that team planning meetings (in whatever methodology you subscribe to) can be used to get the same results in a more orderly fashion. You get everyone in a room and you ask them, “do you think this is interesting? Should we prioritize X over Y?” And everyone says, “Hmmmm. Yes, let’s do Y first and then X but definitely not B,” and there’s debate and eventually everyone comes to an agreement and starts working on whatever was decided. There’s a lot of thinking and talking and committing involved.
The problem with the planning approach in my experience is this: what people think/say they want to do and what they really want to do (or what they’re capable of doing) are often wildly different things. It’s kind of like: “put your money where your mouth is.” Say you show me some thing and ask me whether I think it’s cool or valuable and I say, “yeah, it’s great and I’d totally buy it, yeah.” And then you say, “give me $100 and you can have this thing,” and now I’m not so sure. The act of actually having to take money out of my pocket and trade it for this thing is a powerful device. All the bullshit is distilled down to a very simple binary event: I pay or I go do something else. There’s no hypotheticals at that point.
I think time is kind of like that. It’s hard to tell how it’s best spent until a real person is faced with a decision to actually spend it or not spend it. At GitHub, we have a very simple voting/prioritization system: whatever people decide to work on is a priority, by definition. If something is interesting, more people end up working on it. This doesn’t mean there’s no priorities. It just means we find out about them instead of deciding on them.
And I want to stress that there’s more than just this selfish I’m-intellectually-curious-about-X-and-so-X-is-all-I’m-going-to-work-on thing going on here. You want to ship. You want to make money. You want people to love the shit you put out. You want to kick ass. You want your coworkers to kick ass. You want the site to be stable and fast and reliable. All of those things are at work all the time and they play into what you decide to work on at every moment.
The business must not have customers who are promised certain features by a certain date. Customers of every software company I’ve ever worked for have requested features that no developer wants to work on, but they pay the bills, so we worked on them anyway.
I’ll agree that dates are hard with this kind of setup, but dates are always hard. We’re just honest about it. I’d say the only difference between GitHub and most other places I’ve worked at in practice is that I apologize for being late a lot less. The same work gets done in the same amount of time, there’s just fewer estimation errors because there’s less estimating. Actually, I feel like I get more done “on time” because I have more freedom to take things that really aren’t time critical and shift them around because I wasn’t forced to put an artificial deadline on them in the first place.
As for working on stuff that’s tedious or maybe not immediately gratifying, that happens all the time. Again, the goal isn’t always to just work on what’s intellectually interesting. Things can be interesting for all kinds of reasons. Money is certainly interesting. Helping somebody out in a jam on a support request is interesting, even though the work might not be all that fun. Plugging a security vulnerability 20 minutes after it was reported is interesting. Fixing a bug that’s flooding your exception notification system is interesting.
I want GitHub’s availability and reliability to be unmatched. The steps required to get there aren’t all intellectually stimulating. Some of the stuff is a real chore. But I want to get there dammit. Getting there is interesting.
Also, most things are interesting to someone. There’s a few areas right now that no one has a particularly strong interest in. When things become neglected, they’re very quickly obvious. They don’t have a chance to fester. They just pop and are ugly and everybody has to look at it all the time and it’s annoying. At GitHub, those things are taken as a sign that we should consider hiring somebody who is interested in that thing. The more everyone is forced to take time away from what they would rather be doing to deal with the thing, the more pressure is created to find the person that’s going to love that thing and fix it for good.
I didn’t realize I had so much to say about this. I guess I feel like I’ve already learned a ton from these guys and am anxious to share. I’m still trying to figure out how it works so damn well myself. I doubt anyone really understands the thing. We’re all pretty good at rolling with it.
Ryan’s right that we’ve never really sat down to discuss how our process works, short of us all agreeing that we like it. I’m curious to see in the upcoming years as we continue to hire additional people how our setup evolves.
Hiring talented developers like Scott, Ryan, and Kyle lead me to believe that as long as we’re hesitant and selective about who works for us, there’s no reason we can’t carry on this way without implementing bizarre solutions to problems that shouldn’t exist in the first place.
I jotted down some notes on hiring people a while ago, perhaps it’s time to actually write the post.
There has been a vocal minority calling out the GitHub team on our extracurricular activities recently, an opinion summed up fairly well by an anonymous commenter on the recent Engine Yard blog post about our transition to Rackspace:
PS. GitHub guys. I am no-tea-totaller, but you need to layoff the constant stream of alcohol related events, and posts on twitter, etc. Paying customers don’t react well seeing party boys whooping it up when the service continues to suffer from frequent outages, slow queues, occasional inability to push/pull, frequent breakage after code pushes, etc. You need to be seen as serious about the product you are delivering as your customers are about theirs, and behave as though committed to it 110%. Please remember that other git hosting solutions are rising up and the opportunity cost of switching git providers is low due to Git’s distributed nature.
I started to write a post in response to this, but a commenter shortly thereafter nails it so well that I’d rather just include it here:
Getting involved in developer communities, whether via social networking online or in person, is part of the GitHub guys’ jobs. Nobody can be productive at a keyboard twenty-four hours a day, seven days a week. Is it a crime for them to spend a couple evenings a month buying drinks for their customers and potential customers, and finding out how to better serve those customers?
I’m a paying GitHub customer, and I think they’re 110% committed while also being shrewd networkers. If you went out and had a beer with them, you could find out exactly why the site is slow and what they’re doing about it. They’re trying to do a lot with a very small team, and they’re doing a damn good job.
The next time you get up on your high horse to chastise someone for not being “110% committed”, put yourself in that position. Do you have a life? Do you do anything other than work? Do your activities outside of work directly benefit your company? Get real.
I think the truly unfortunate part is that we could put up a constant serious businessmen facade so that certain people can’t accuse of having too much fun, but what they’re failing to recognize is that we do what we do so we can meet GitHub users, figure out what they like, what they don’t like, form business relationships, meet new friends, and the list goes on and on. If there’s one thing I hope you can’t accuse our team of is that we’re unapproachable.
Any city we travel to, we always make an effort to either host an event or participate in events already happening, it’s all about community outreach and meeting new people. Would hosting a hack-fest make more sense for GitHub? Maybe, so just in case, we sponsor those too. But, when we host events at a bar, people actually talk to each other and socialize far beyond what they would do sitting in front of their laptop.
It’s amazing what a couple of beers will do to loosen up the crowd and allow folks to delve into candid conversations about our service. Make no mistake, we like going out and we like having fun, but we get to have fun and receive feedback at the same time. Sounds like a pretty sweet deal to me.
Furthermore, things like including a bottle of bourbon with our prizes is so simple and inexpensive, but surprisingly effective at spreading the word how could we not continue to do it? It works!
Truly, it may look like we’re constantly out drinking and goofing off, but I dare you to find anyone on our team that doesn’t work their ass off and care deeply about our service. We eat, sleep, and breathe GitHub and you’re only fooling yourself if you think otherwise.
Reading the latest in Git fanboy criticism has felt like a time warp back to when I started learning about Rails. Just for fun here’s a quick run-down of my life for the last few years:
- Four years ago I was a J2EE webapp author when I discovered Rails. It turned out writing Ruby was way more fun; I told the world about it, was promptly dismissed as a fanboy, but I stuck with it.
- My Rails skills landed me a well-paying job.
- Last year I was a Subversion user when I discovered Git. It turned out using Git was way more fun; I told the world about it, was promptly dismissed as a fanboy, but I stuck with it.
- My Git skills landed me a spot in a successful startup (for those wondering, this was ultimately the goal after college).
I understand that your mileage may certainly vary jumping on bandwagons, but I don’t know why people are immediately dismissive when there are crowds of really enthusiastic developers.
Assuming their opinions are genuine, there’s a good chance there’s something to what they’re saying even if it isn’t always done with the greatest of tact.
This isn’t fashion design, when developers get excited about things, it normally means it’s helping them become better developers.
I’m a Rails fanboy and a Git fanboy and I don’t give a fuck who knows it.
When we tell people why they should use Git over Subversion, the go-to line is, “Git does Subversion better than Subversion, but it does a lot more than that.”
The “lot more” is comprised of a bunch of stuff that makes Git really shine, but it can be pretty overwhelming for those coming from other SCM’s like Subversion.
That said, there’s nothing stopping you from using Git just like you use Subversion while you’re making the transition.
Assuming you’ve installed the necessary software and have a remote repository somewhere, this is how you would grab the code and push your changes back with Subversion:
$ svn checkout svn://foo.googlecode.com/svn/trunk foo
# make your changes
$ svn commit -m "my first commit"
And how would you do it in Git:
$ git clone git@github.com:pjhyett/foo.git
# make your changes
$ git commit -a -m "my first commit"
$ git push
One more command to make it happen in Git. That extra command has large implications, but for the purposes of this post, that’s all we’re talking about, one extra command.
See, it really isn’t that hard.
Update: I’d be remiss to not also mention that the equivalent of updating your local copy in Subversion compared to Git is `svn update` and `git pull`, respectively. Only one command in both cases.
There’s something about this picture that encapsulates everything I love about San Francisco.
The eclectic mix of people, trading my car for public transportation (drop-top trolley!), views worthy of a Gap commercial, palm trees seemingly unaware of the cold, all the while heading towards the Farmer’s Market bringing back memories of growing up in the Midwest. There’s a bunch of internet people here, too, believe it or not :-)
Do yourself a favor and visit this city, I’ll be in North Beach if you want to say hi.
I was listening to the GitHub interview on the Web 2.0 Show that I missed when I heard Chris mention something that deserves repeating.
When you show up to project page on SourceForge, you’re completely befuddled as to where to go next:
You begin to ask yourself some basic questions: how old is this project, is this even the right project, where can I just download the f’ing thing?
The first glaring problem after I resized the screenshot is that the only things still legible on the page are the advertisements.
Now, let’s take a look at a GitHub project page:
Even after a resize, a few things are clear: you can find the download button quickly (that will actually download a tarball instead of taking you through thirty-seven more pages), when the last commit happened, and a number of other easily visible things you may be interested in.
The project’s homepage also happens to be the source browser, which for any geek (myself included) tickles a certain funny bone, being able to look at the code immediately. Furthermore, the page also displays the root-level README (if provided), which normally holds a multitude of useful information; something you should be able to read before bothering to download the project.
And hey, no ads! GitHub charges for private repositories, thus relieving the burden of trying to generate revenue via pageviews.
It’s not just Visual
Sure, someone could skin SourceForge to not be totally brain-dead, but there are also fundamental differences as to how the sites work.
If you have a grievance with one of the libraries hosted on SourceForge, you’re stuck filing a bug report or emailing the owner. The project could be years old and virtually unmaintained, but nevertheless, that’s the standard protocol.
The complete reverse of that on GitHub is you can click one button to fork the project and have your own copy to do with what you will. Whether it be making your fixes and submitting them back to the original owner, keeping them to yourself, or even convince people that your fork is the repository that people should be using from now on.
Welcome to the new age of open source programming and we’ve only just begun, fork me.
Hot on the heels of the FamSpam launch only a few short months ago, Chris Wanstrath, Tom Preston-Werner, and I launched GitHub.
I know I speak for the rest of the team that we’re all really excited to see it open to the public after a rewarding beta period that garnered some awesome feedback.
For those wondering what I’m even talking about, GitHub is a hosted code repository made specifically for the Git version control system.
Hosted code isn’t anything new, but we think we’ve created an environment that fosters open source collaboration like never before. You’re also welcome to keep your code private if you’d like.
‘Course you don’t have to take my word for it, have a look at what other people are saying about GitHub
...is for companies that know neither how the internet works nor how to build an interesting product.
(gleaned from personal experience)
I know most of my audience is compromised of web developers so please pay attention:
Unless you’re building a financial application, please do not require me to have a password of more than 4 characters and/or some ridiculous combination of letters and numbers.
Why is the information that you’re storing on your site so valuable that you need to inconvenience me?
Ok, maybe we launched a couple of weeks ago, but we decided to wait until after the holidays to make the official announcement.
Chris and I decided that although Err Free was making decent money, we realized that we weren’t having enough fun, so we built our first product, FamSpam!
In the simplest sense, it lets my mom email all of her children by just having to remember one email: hyett@famspam.com. Each family also gets their own site where they can browse photos and search through conversations, but its main goal is to keep families talking.
FamSpam’s first success story is my own, because I haven’t received an angry phone call from my mom wondering why I haven’t spoken to her in two weeks since we started to use it :-)
We also just launched Dynamite, the Err Free blog, where we’ll be talking about the stuff we’ve learned running a consultancy/startup over the past year. That also includes some interesting code discussions that don’t really belong on Err the Blog.
It’s gonna be a good year.
Sometimes it’s to my chagrin to admit to my Ruby on Rails colleagues that I actually bothered to get my computer science degree. Majoring in linguistics or dropping out halfway through is much more fashionable these days. That feeling isn’t completely unfounded, though. These days, the technologies and concepts that I’m using are things I’ve learned on my own.
That said, I recently stumbled upon a coding practice that I thought was a revelation until I remembered that it was something I was taught in CS101.
I give, what is it?
Write your program on paper first, using the terminal to only regurgitate what you’ve already written.
The computer is a huge distraction, something that should be avoided when you’re brainstorming over important code.
When I’m in front of the screen, I find myself too concerned with getting the program running, getting the feature built, having something pretty to look at, than actually bothering to think if what I’m writing is actually a good idea.
Here’s what happened
The Powerbook I’m currently using has terrible battery life, two hours on a good day. On a five hour flight, that leaves me with a fair amount of downtime. Two hours I could be reading a book, but I’ve always found flights to be incredibly productive because there are no distractions in the air.
On the last flight home, instead of immediately running the laptop’s battery dry, I whipped out my notepad and started jotting bits of code for solving some problems with FamSpam.
<plug>Go sign up for the launch annoucement if you’re terrible like I am with keeping in touch with your family, it’s launching just in time for the holidays!</plug>
A couple of hours later I had rewritten chunks of code three to four times until I was really happy with the solution, one that I know I wouldn’t have come up with had I just been banging away at the keys.
Once I had finally opened up my laptop, I needed only about 30 minutes to have my solution written and fully-tested.
Final thoughts
Again, what I really think it boils down to are those moments of reflection that not being in front of the screen provides you.
Sure, you can sit back and ponder while on the computer, but I find myself constantly saying, “well, what if I just try this.” Applying band-aids to a large gash instead of getting it stitched it up.
Next time you’re heading to the cafe, instead of bringing your laptop, bring your notepad and see what happens. Walking up to the computer with the solution already in hand is an awesome feeling.
See, that CS degree wasn’t complete bullshit.
Programmers are arguably some of the smartest people in the world. We solve complex problems all day, every day, and we’re always hungry for more. We strive to find simplicity in even the most complicated situations. If Eistein were alive today, he’d probably be hacking the Linux Kernel along side Alan Cox during his lunch break.
The issue arises when you take those minds, throw in the impersonality of the internet, and mix in a whole bunch of ego.
Case in point: Vlad the Deployer. It was written as a Capistrano replacement to handle application deployment simpler than the aforementioned project. Make no mistake, the library rocks; I’m using it in production already.
Capistrano certainly became the victim of its own success. It needed to be everything to everyone in the Rails community, because it was the only thing out there that handled deployment with any sort of ease.
Futhermore, its author, Jamis Buck, recently said that its dependencies were written at a different time and place than he’s in now and has been working on rewriting them to remove unnecessary complexity. It takes a big person to admit that his hugely successful application needs a lot of work.
It doesn’t take a big person to rip on an open-source app that has been one of the most useful libraries for Rails since the framework’s inception. Vlad’s authors, the Ruby Hit Squad, have made it some bizarre mission of theirs to insult Capistrano in an attempt to promote their replacement. I have no idea what they’re accomplishing with their tactics (see image above for an example) other than alienating someone that dedicates a portion of his time contributing to a framework that allows them to make their livelihood programming in Ruby.
I’m picking on Vlad, but I’ve seen this time and time again with open-source projects and find the attitude unbelievably off-putting as it reflects poorly on the open-source community as a whole. I’m certainly not immune to bad-mouthing projects that I’m not excited about, but I’d like to think that as the years progress that I’m learning to control the emotions that ran rampant when I was 16.
As the saying goes, “Speak softly and carry a big stick, you will go far.” Let others know about your alternative, but allow the code do most of the talking for itself.
If you need to hone your Ruby chops prior to heading to Berlin for RailsConf Europe check out the Lone Star Ruby Conference in Austin, TX September 7-8. I was digging the city hard after visiting for SXSW and can’t wait to get back there.
Chris and I will be representing Err Free at the conference with talks entitled On Your Best Behavior: BDD in Ruby and I Built this Killer Rails site, Now What?, respectively.
I promise mine will be entertaining, with the large caveat that it’s the first talk on Saturday after what will surely have been a long night of drinking on Friday.
There’s an early-bird registration ending tomorrow night, so if you were at all interested in attending, I’d recommend doing so soon.
While I was at CNET, traffic was growing steadily on our sites, but we had a rude awakening one particular day.
The “Oh Shit” moment
I remember the morning when my boss said to me:
We’re having a conference call in a few minutes with CNET’s SEO dude, we lost half of our uniques starting last Thursday, why don’t you join in.
The first thing I did is wonder what the hell I did wrong, because in the back of my mind, this problem was inevitably engineering’s fault.
The realization that Google owns you
Chowhound was getting 85% of its traffic from Google. It has around 500k topics, averaging 4.5 posts per topic, each an opportunity to become indexed by Google.
This particular week, Google decided to purge its index of stale data, and in doing so, eliminated the majority of the pages Chowhound had indexed, thus slashing our uniques in half. An interesting side-note was that our pageviews didn’t drop nearly as much, because the core users that were still visiting the site had a much higher pageview per visit ratio than the Google visitors.
What we did wrong
The solution was actually quite simple. On every board, we originally used ‘Next’ and ‘Previous’ links that would paginate you to the corresponding page of topics. Straightforward, but horrible for Google’s bots. They’d click a few of the Next links and say the hell with this and move on.
To give you an idea, the Manhattan board alone has over 800 pages of topics, and when Google is only looking at the first three pages, you have a massive problem.
How we fixed it
Simply put, I wrote the will_paginate plugin. Instead of just having ‘Next’ and ‘Previous’, it also displays a window of page numbers that allows bots the ability to navigate hundreds more pages within one click. Now we’re getting somewhere.
Eureka!
As the traffic began to come back in a serious way, well beyond where it was before the dip, the notion of SEO became quite clear to me. Given a site with tons of great content, instead of building more features, you should be trying to leverage what you already have.
This also makes a lot of sense from a financial standpoint. How long would it take your team to build a new feature for 10k more uniques a day, how long would it take them to add meta tags and a sitemap for 10k more uniques a day. There’s no comparison.
The Checklist
SEO isn’t rocket science (unless you’re trying to push the envelope), you just have to be aware of a few guidelines.
Page-specific
- Unique title tag
- Unique meta description (Yahoo is the only one using meta keywords anymore)
- One h1 tag (the text used in your h2-h4 tags is still very important, though)
Site-wide
- Allow only quality outbound links (eg.
rel="nofollow" on all user-generated links)
- Block duplicate content from being indexed with robots.txt
- Structure your CSS so the content can go first
- Consistent navigation that allows bots to get to any page on your site within 2 or 3 clicks
Piece of cake. This discussion can go far deeper than these points, but if you do the stuff above, you’re 90% of the way there. The last 10% is the most time-consuming, so I leave that to you to decide if it’s worth your time.
I know a lot of people think of SEO as a dirty word, and that it conjures up negatives connotations of link-baiting and spam, but that really couldn’t be further from the truth. I’m just going to start calling it FT…Free Traffic.
Speaking of Which
Err Free does more than just Rails, we’d be happy to apply what we’ve learned about SEO to your site. Who doesn’t love Free Traffic?
The Err Free team will be accepting beers at RailsConf for anyone interested, come find us.
Chris will be talking about his acts_as_cached plugin on Saturday. He promised me it’s gonna be good and he’s not one to disappoint.
Also, if you’re going to be in the Bay area after RailsConf, there’s a three hour session on the 22nd concerning whether Rails can scale.
It’s $15, has smart people talking, and there’s an open bar afterwards…need I say more? Check it out.