| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

01_CaseyForbesFromRavelry

Page history last edited by Makoto Inoue 13 years, 11 months ago

Use Case Interview 01: Casey Forbes from Ravelry 

 

Introduction:

 

Ravelry is online knitting and crochet community with 3.6 mil view per day. It is first featured at Tim Bray's blog 

 

 

 

 

At the blog post, the use case of Tokyo Cabinet/ Tyrant was briefly mentioned.

 

"I’ve been using Tokyo Cabinet/Tyrant instead of memcached in some places for caching larger objects. We do a lot of rendering Markdown into HTML — pretty much everywhere on the site — it’s a waste of energy to render but it’s also a huge amount of data to store in memcached."

 

I was interested to know more about the TC/TT use case, and asked questions to Casey Forbes, the founder and the sole developer of Ravelry.

 

Q: Is it for page cache or fragment cache? (Detail about Ruby On Rails caching is here)

 

A: We are using Tokyo for a fragment cache (page fragments, but in particular - we have lots of markdown formatted text that is stored in Tokyo once it is rendered into HTML)  We rarely remove things from Tokyo as most things are replaced/updated when they expire and not removed.

 

Q: Was it a drop in replacement, or did you have to write lots of logic to mimic memcache usege?

 

It's a drop-in replacement for memcached as Tokyo Tyrant can speak the memcached protocol. The native protocol may have more features or speed but we haven't looked into that at all.

 

Q: How much data do you store ?

 

We've stored up to 25 GB but we are currently storing 10 GB of data. Performance is so close to memcached (even though it hits the disk) that speed is really a non-issue.

 

I think it is a very nice solution for storing large chunks of HTML, etc - MySQL is not a very good solution for this (waste of Innodb buffer pools, lots of growth in database files, less than ideal performance when dealing with large tables of blobs) and memcached can become full very fast depending on how much memory you have to devote to caches.

 

 I have had a crash that led to corruption of the database - it is definitely less resistant to failure than Innodb (for example). Because of this, I'd be nervous about using a non-replicated Tokyo for non-cache data. I have not tried any of the repair tools - when this happened I just started from an empty cache which was fine for our needs.

 

Q: Any gotchas when using TC for memcached usage

 

The main gotcha is that you need to enable the large files option (opts=l) on startup to have a database larger than 2 GB. You may also want to tune other parameters, such as xmsiz and bnum.

 

Many thanks to Casey for sharing his information.

Comments (0)

You don't have permission to comment on this page.