Category Archives: craigslist

Speaking at OpenWest 2014 Conference: real-time search infrastructure architecture at craigslist

Just a quick heads-up that I’ll be giving a 50 minute talk at OpenWest in Utah on Thursday, May 8th. The talk is titled real-time search infrastructure architecture at craigslist and is completely new. We’ve recently completed the 3 major … Continue reading

Posted in craigslist, sphinx | Leave a comment

Seeking Sucks: Spoiled by SSDs

I’m in the process of rebuilding full-text indexes for a good sized document collection that lives in a sharded MongoDB cluster. And the funny thing about this is that I don’t really use MongoDB that much. I mean we put … Continue reading

Posted in craigslist, mongodb, nosql, sphinx, tech | 9 Comments

Handling Database Failover at Craigslist

There has been some interesting discussion on-line recently about how to handle database (meaning MySQL, but really it applies to other systems too) failover. The discussion that I’ve followed so far, in order, is: GitHub’s report on their automated failover … Continue reading

Posted in craigslist, mysql, tech, yahoo | 11 Comments

magit makes git awesome in emacs

I recently came across the Meet Magit video which is an excellent introduction to the magit emacs mode for git. I use both Emacs and Git a lot at work and at home, so having something a bit more comprehensive … Continue reading

Posted in craigslist, programming | Leave a comment

craigslist Charitable Fund Donates to the Perl Foundation

I’m very happy to see the announcement on the Perl Foundations’s home page this morning. According to craigslist CEO and former Perl Hacker Jim Buckmaster, “craigslist has gloried in and relied upon Perl for most of its software development for more than … Continue reading

Posted in craigslist, perl | 1 Comment

Experimenting with Real-Time Search using Sphinx

In the last few weeks I’ve been experimenting with using real-time indexes in Sphinx to allow full-text searching of data recently added to craigslist. While this posting is not intended to be a comprehensive guide to what I’ve implemented (that … Continue reading

Posted in craigslist, sphinx, tech | 12 Comments

Video from my MongoSF 2011 Talk

To go along with the Slides from my MongoSF 2011 Talk, the video is now on-line thanks to 10gen. I had a great time at MongoSF 2011, learned some important new things about MongoDB and got a sense of just how … Continue reading

Posted in craigslist, mongodb, nosql | 1 Comment

Slides from my MongoSF 2011 Talk

Due to a surprisingly long conversion delay, the slides from my talk at MongoSF 2011 are now available: I’m sure video will be available at some point too, which will make the slides quite a bit more useful. But if … Continue reading

Posted in craigslist, mongodb | 5 Comments

Speaking at MongoSF on May 24th

I’m a little late to be posting this the night before. In any case, tomorrow afternoon I’ll be presenting at MongoSF 2011. My talk title is Lessons Learned from Migrating 2+ Billion Documents at Craigslist and it picks up where my … Continue reading

Posted in craigslist, mongodb, nosql | 2 Comments

MongoDB Pre-Splitting for Faster Data Loading and Importing

Once nice thing about playing with Webdis is that I can watch the import rate of my multi-billion document MongoDB import in nearly real-time. The downside of that is that I quickly found that once I got a few hundred … Continue reading

Posted in craigslist, mongodb, programming, tech | 17 Comments