I’ve long wanted a lightweight system for gathering and graphing simple performance metrics without a lot of centralized processing. I could use something like StatsD from the Etsy folks but got inspired by reading about Redis at Disqus the other day.
So I decided to start charting the process of migrating data into our MongoDB archive cluster. I already have worker processes that are tracking their own performance. And I have a “manager” process that makes sure the workers have work to do and also aggregates a bit of info about what the workers are doing. The workers all self-report into a Redis instance, so I updated the manger to store a few bits of aggregate data back in Redis as well.
If you work with Redis and haven’t heard of webdis, you should really check it out. It’s a tiny event-based web server designed specifically to put a REST interface on Redis. Webdis can transform Redis output into a variety of very useful formats:
I’m toying with the idea of generalizing this a little bit so it’s possible to throw arbitrary stats at Redis and have an interface for exploring and plotting them.