Webdis is Full of Awesome

I’ve long wanted a lightweight system for gathering and graphing simple performance metrics without a lot of centralized processing. I could use something like StatsD from the Etsy folks but got inspired by reading about Redis at Disqus the other day.

So I decided to start charting the process of migrating data into our MongoDB archive cluster. I already have worker processes that are tracking their own performance. And I have a “manager” process that makes sure the workers have work to do and also aggregates a bit of info about what the workers are doing. The workers all self-report into a Redis instance, so I updated the manger to store a few bits of aggregate data back in Redis as well.

To graph this data in the browser, I used a combination of jquery, flot, and Webdis. Before I say more about webdis, here’s what it looks like:

mongo-watch screenshot

If you work with Redis and haven’t heard of webdis, you should really check it out. It’s a tiny event-based web server designed specifically to put a REST interface on Redis. Webdis can transform Redis output into a variety of very useful formats:

  • JSON
  • JSONP
  • XML
  • BSON
  • Text

In other words, it makes it trivial to build little applications using jQuery in the browser without having an app server involved. Just put the logic in the JavaScript and the data in Redis. Webdis gives you a handy way to move data in and out of Redis using methods that feel very natural in JavaScript.

I’m toying with the idea of generalizing this a little bit so it’s possible to throw arbitrary stats at Redis and have an interface for exploring and plotting them.

About Jeremy Zawodny

I'm a software engineer and pilot. I work at craigslist by day, hacking on various bits of back-end software and data systems. As a pilot, I fly Glastar N97BM and high performance gliders in the northern California and Nevada area. I'm also the original author of "High Performance MySQL" published by O'Reilly Media. I still speak at conferences and user groups on occasion.
This entry was posted in craigslist, programming, redis, tech. Bookmark the permalink.

2 Responses to Webdis is Full of Awesome

  1. Pingback: Tweets that mention Webdis is Full of Awesome | Jeremy Zawodny's blog -- Topsy.com

  2. Pingback: MongoDB Pre-Splitting for Faster Data Loading and Importing | Jeremy Zawodny's blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s