Speaking at Percona MySQL Conference and Sphinx Search Day

I’m happy to see that the vacuum created by O’Reilly Media dropping the annual MySQL Conference has been filled by the good people at Percona in the form of the Percona MySQL Conference, which is in Santa Clara from April 10-12th.

I’ll be speaking at 3:00pm on the 12th. My talk is titled Living with SQL and NoSQL at craigslist: a Pragmatic Approach. The abstract for the talk is:

craigslist uses a variety of data storage systems in its backend systems: in-memory, SQL, and NoSQL. This talk is an overview of how craigslist works with a focus on the data storage and management choices that were made in each of its major subsystems. These include MySQL, memcached, Redis, MongoDB, Sphinx, and the filesystem. Special attention will be paid to the benefits and tradeoffs associated with choosing from the various popular data storage systems, including long-term viability, support, and ease of integration.

If you’re interested in learning more about our data systems at craigslist, drop by.

Then, on the following day (April 13th), the folks behind Sphinx, Drizzle, and SkySQL are putting on a one-day mini-conference that features speakers on a variety of interesting topics. I’ll be speaking as part of Sphinx Search Day 2012. I’m looking forward to meeting Ivinco there as well.

I don’t yet have an abstract for my Sphinx talk, but it’ll probably be similar to a brown bag talk I recently gave at craigslist. It was an overview of how we’re using a mix “traditional” batch-updating Sphinx and Real-Time Sphinx to power some new features.

Hope to see you there!

Posted in Uncategorized | 16 Comments

From the comments of Never Make Counter Offers on Bram Cohen’s weblog,

…Money is really like a chronic disease, my financial situation is something that intrudes on what I want to do, and prevents me from doing it unless I keep it treated. Either the work is interesting, or it isn’t what I want to do with my life. (You turn 40 at the same time regardless of what you’ve been doing, the question “was it worth it” is not a dollar value question.)

I suspect all real geeks are like this. Being paid very well to sit in a cubicle with nothing to do (been there, quit that) is low-grade torture.

Rob Landley

Posted on by Jeremy Zawodny | 4 Comments

magit makes git awesome in emacs

I recently came across the Meet Magit video which is an excellent introduction to the magit emacs mode for git. I use both Emacs and Git a lot at work and at home, so having something a bit more comprehensive and useful than just the standard version control integration that you get with Emacs by default sounded good.

But watching that video, I realized that no only it magit completely awesome, I actually learned a fair amount about git in the process. I’d almost say it’s worth watching even if you have no interest in magit but do feel like there’s more you could know about git.

Having seen the video, I find that I most often refer to the Magit Cheatsheet and the Magit User Manual. If you’re an emacs and git user, you probably owe it to yourself to spend 15-20 minutes watching the video and trying out magit. I think you’ll quickly realize how useful it is.

Posted in craigslist, programming | Leave a comment

craigslist Charitable Fund Donates to the Perl Foundation

I’m very happy to see the announcement on the Perl Foundations’s home page this morning.

According to craigslist CEO and former Perl Hacker Jim Buckmaster, “craigslist has gloried in and relied upon Perl for most of its software development for more than a decade. craigslist Charitable Fund is honored to recognize the wizardry and generosity of the Perl community, help ensure the ongoing maintenance of Perl5, and contribute to the future evolution of Perl.” He added, “It was unclear at first how best to give something back to Perl. Fortunately there was more than one way to do it.”

And I’m very happy that craigslist (my employer) recognizes the value that Perl has provided over the years in all the work we do at craigslist. I’m sure the $100,000 will go a lon way toward helping with ongoing Perl development and maintenance this year.

Here’s to more great stuff from Perl in 2012!

Discussion also on Hacker News.

Oh, by the way. We’re hiring great Perl hackers at craigslist. Let me know if you’re interested. 🙂

Posted in craigslist, perl | 1 Comment

Visiting Frozen Tenaya Lake

Yesterday Kathleen and I visited Tenaya Lake in the Tioga Pass area of Yosemite National Park. The weather has been odd this season. While it’s been quite cold at the higher elevations, there’s been virtually no precipitation yet, so the lake has frozen over but it is still easily accessible.

Tenaya Lake (frozen)

The ice is easily 8 inches or more think in most places, so lots of folks were out wandering around. She wrote up a posting with more pictures on HowToEatAndLive: Walking on Tenaya Lake.

I had a blast sliding around on the lake! At times like this it really pays to live nearby.

Given the rarity of these conditions, we may not be able to do this again for another 10 years or more. And it’s good that we went when we did. The weather is starting to change–headed for a more seasonal mix of cold and rain/snow.

Lots of pictures, including some really cool ice shots she took, are in my Tenaya Lake album on Picasa.

Posted in fun | 1 Comment

Renewed Weight Loss, Health, and a new blog…

Back in mid-2006, I documented my fairly dramatic weight loss in a series of blog postings. Some of you may even remember that. It was on boingboing and all that.

I went from a high of 224 pounds to a low of 164, which made for a total loss of 60 pounds (I’m about 6 feet tall). It’s interesting (and a little disturbing) to go back and re-read what I wrote back then:

Fast forward to January of 2011, and I had gained back about half the weight. I was hovering just below 200 pounds and had needed to purchase larger jeans a few times during the five years in between. Considering the success rate of most diets, that’s not bad but it’s also hardly ideal. Other issues had crept up along the way too.

The story of how I re-lost most of that weight and generally got a lot healthier is the subject of Learning to Eat Well and Stay Healthy, a blog post I just wrote on a new site called HowToEatAndLive that my wife Kathleen and I have started for 2012. It’s going to document a lot of what we’ve learned about health, nutrition, and living in the last year (and interesting things we continue to discover in the new year). We’ve both learned a lot and have tons to write.

If you’re curious to learn about how I lost the weight I gained back and have kept it off, that’s the place to go. It’s a little rough around the edges yet, but we’re working to add some finishing touches as well as a lot more information.

Posted in health | 53 Comments

On Good Enough Code

Reading There’s no shame in code that is simply “good enough” (and the discussion on hacker news), I find myself strongly agreeing. I have this system I built about 3 years ago at craigslist. Since then it has been extended in various ways and now runs on about four times the hardware it did back then. And it handles far, far more data.

In other words, I’m still evolving it (in small steps) and helping it to grow and change to suit our needs. But every now and then, while reading through the code, I get the urge to “improve” it in some way. Often that means adding a new abstraction “just in case” I need in the future. Or there’s that one script that runs from cron, which is written in bash. I’d love to re-write in Perl. But that’d be more work for virtually no benefit.

Why do I not just make these changes? Two reasons:

  1. the stuff works and is stable
  2. every time you change something you may also break something

The fact that I can look back on code I wrote a few years ago and identify ways that I’d do it better is good. It means I’m still learning. But the fact that I can successfully resist the urge to change the code is even better.

The quest for perfection is tricky. You can spend a lot of time chasing it and there’s no telling if you’ll every actually get there. And even if you do, what other things did you put aside in the process?

I think this mentality can apply to a of things in life. One of my hopes for 2012 is that I can better see that.

Posted in Uncategorized | 10 Comments

Christmas on the Beach

Christmas on the Beach

Shot just south of Morro Bay, California.

Posted in Uncategorized | 2 Comments

Easy and Delicious Turkey Dinner

We had a few friends over for Thanksgiving this year and were lucky enough to split the cooking with them. One of my tasks was making the turkey. We had purchased a pair of 12 pound Diestel Turkeys through our local CSA and planned to use one of them for the dinner.

Looking around for ideas, I stumbled upon Mom’s Roast Turkey on SimplyRecipes and it seemed too easy to be that good. But that couldn’t have been farther from the truth.  It ended up being some of the best turkey we’d every had. The veggies inside were fantastic, the bird was cooked perfectly, and the juices made for amazingly rich and delicious gravy.

2011 Thanksgiving Turkey

We used some lemon infused olive oil on the outside, along with some excellent French grey sea salt, fresh rosemary, and fresh oregano. Inside was some fresh lemon juice, organic carrots, celery, and onions.

I know that pictures really don’t do it justice, but let’s just say that we’ve been eating leftovers for days now and haven’t begun to get sick of them. 🙂

In addition to the turkey and gravy, we had some excellent stuffing, garlic mashed potatoes, mashed sweet potatoes with brown sugar, asparagus, and an amazing cranberry/apple/orange sauce.

Pictures are here: Thanksgiving 2011 on Picasa.

Posted in cooking, food | 1 Comment

Looking forward to Redis 2.6

In reading Short term Redis plans, I’m happy to see the “More introspection” section. For a long time some in the Redis community have asked for the ability to publish key names to a channel when they expire. And, while I sympathize with their desire for such a feature, I also realize that it’s not the greatest solution to the problem (since pub/sub is best effort–a client could be disconnected for a bit and miss messages).

But it looks like Salvatore is taking things a step or two farther…

There is a plan to use Pub/Sub in order to communicate events happening inside Redis, like a key that expired, clients connecting / disconnecting, operations performed against keys. We’ll probably allow the user to script this feature with Lua so that you can, for instance, push all the keys expired inside a list as well, or other things that can’t be reliably done with clients and Pub/Sub since the client is not guaranteed to get all the messages (it can get disconnected for some reason).

This strikes me as really good. He’s been listening to feature requests for a long time. Some appear and vanish after a short time, while others persist. This has been a persistent request for a long time now. Building it in a way that allows for robust notification ought to make everyone happy.

I’ve personally not allowed myself to design systems that would require knowing when a key expires, but seeing this on the roadmap really does open up a lot of possibilities for future work.

Posted in programming, redis, tech | 1 Comment