Tokyo Cabinet Wiki

 

This is an unofficial page to collect links of Tokyo Cabinet, Tokyo Tyrant and related projects. Current versions are: Tokyo Cabinet 1.4.47, Tokyo Tyrant 1.1.41.

 

Official Content

 

 

• 1978th.net Technical blog of Mikio Hirabayashi, the creator of TC/TT, and so on. Some of them will be translated at 1978thNetTableOfContents
• Introduction to Tokyo Products
• Introduction to Kyoto Products

 

 

Community

 

•  Tokyo Cabinet Users Google Group
•  Official Support page within Mixi = Japanese only. You need Mixi account to login.
•  MyNoSQL = MyNoSQL is a blog dedicated to all NoSQL related topics. This is link to tags related to TC
•  My SQL Performance Blog = Though it's about MySQL, lots of insightful articles about NoSQL including TC/TT 
• nosql-database.org = They also have link pages to all NoSQL related topics including TC/TT
• Twitter search result for TC and TT 
• Bitbucket search results for Tokyo
• Github search results for Tokyo 
• @tcwiki = Tokyo Cabinet Wiki twitter account. Please tweet to @tcwiki if you find new TC/TT related links 

 

Introductory Topics

 

• Versatile Storage Options With Tokyo Cabinet = Examples of using TC as Memcache++,Queue,Counter,Tag Cloud
• Tokyo Cabinet: Beyond Key-Value Store(igvita.com) = This is the blog entry which made TC famous.
• Tokyo Cabinet In One Hour =  Oreilly WebCast by a developer at Engine Yard.
• Edo Cabinet Ruby Kaigi 2009 = By the author of rufus-tokyo ruby library. With links to slide and video page as well
• Tokyo Cabinet as a Key-Value Store = By the author of oklahoma_mixer ruby library
• Tokyo Cabinet and Python
• Comparison of Tokyo Cabinet and Redis = Redis explains how it is different from Tokyo Cabinet.

 

Advanced Topics

• Patches for extending tokyocabinet etc.

 

Functionalities

 

 

• Comparing functionalities of each database types.

 

 

Extending TC/TT 

• Extending Tokyo Cabinet DB with Lua(igvita.com) = MUST READ. Lua is Javascript like super fast script embedded in TT. It explains how you can use Lua  to extend TT like Redis set's and Couch MapReduce. With links to slide and video pages as well.

 

Performance 

• MySQL-Memcached or NOSQL Tokyo Tyrant(Part 1 ~ 3) = Very detailed benchmarking report with MySQL, Memcached and TT
• MemcacheDB, Tokyo Tyrant, Redis performance test = Performance comparison with all config info. Redis is great reading/writing small data sets, but TT wins with writing large data sets.
• MongoDB and TT Benchmark = The blog is written in Chinese, but you should be able to read performance result table. The link is to the benchmarking script repo.
• ActiveRecord + MySQL vs Tokyo Cabinet / Tyrant table through rufus-tokyo = Results.
• ruby-tokyotyrant (C) vs. TokyoTyrant::RDB (Ruby), vs. Rufus::Tokyo (Ruby+FFI) vs. Memcached (C) vs. memcache-client (Ruby) vs. mongo (Ruby+C)
• Tokyo Tyrant tuning parameters (by Mike Dierken) = a well-organized reference (same as in the documentation but much more readable)
• A few key Tokyo Cabinet Notes (by Matthew Yonkovit) = a few interesting gotcha’s, facts, and settings people who use or want to use Tokyo Cabinet/Tyrant stack should know.

 

Backup and Recovery

 

• How to Recover a Tokyo Cabinet Database = By Toru Maesaka, colleague of Mikio Hirabayashi and the author of BlitzDB. 

 

 

Third Party Libraries

( There are standard libraries provided by the author at http://1978th.net/tokyocabinet/)

 

Ruby

 

• http://github.com/actsasflinn/ruby-tokyotyrant = c extension for Ruby to access TokyoTyrant databases. Includes consistent hashing for distributed key-value store use case.
• http://github.com/JEG2/oklahoma_mixer = An full featured and robust FFI interface to Tokyo Cabinet. No support to TT yet. Oklahoma mixer is a dance music every Japanese used to dance with at school and camp. Here is the video.
• http://github.com/copiousfreetime/tyrantmanager = A command line tool for managing Tokyo Tyrant instances  
• http://github.com/tsukasaoishi/miyazakiresistance = RoR Active Record like ORM
• http://activetokyocabi.rubyforge.org = A module to extend RoR Active Record 
• http://github.com/alexyoung/dystopian_index = RoR plugin to use Tokyo Dystopia, a full text search system 
• http://github.com/jmettraux/rufus-tokyo = Retired. Now at Maintenance mode

 

Python

 

Tokyo Cabinet API:

• pytc = standard bindings (BDB, HDB)
• tc = Python bindings (based on pytc, added support for TDB)
• python-tokyocabinet = Python bindings (a complete wrapper; BDB, HDB, TDB)

Tokyo Tyrant API:

• Pyrant = A pure-Python dict-like API for Tokyo Tyrant (based on PyTyrant and PyTableTyrant).

Modelling:

• Doqu (ex PyModels/Docu) = A lightweight schema/query framework for document databases (from in-memory and DBM to MongoDB). Native support for Tokyo Cabinet TDB (directly and via Tyrant).

Plugins, reusable applications:

• http://github.com/ericflo/django-tokyo-sessions = A session backend for Django
• http://labs.gree.jp/Top/OpenSource/Flare-en.html = Distributed, and persistent key-value storage compatible w memcached. it has a pluggable storage compatible with Tokyo Cabinet

 

Scheme

 

• a (somewhat incomplete) binding for Chicken Scheme: tokyocabinet.egg http://wiki.call-cc.org/eggref/4/tokyocabinet

 

http://github.com/hillegass/BNRPersistence =A library to use Tokyo Cabinet for iPhone & Cocoa. 

http://github.com/edvakf/node-tokyocabinet =  Tokyo Cabinet binding for Node.js. So far only implemented interfaces defined in tokyocabinet.idl.

 

http://github.com/actsasflinn/node-tyrant = A node.js module for the Tokyo Tyrant database 

 

Sample Projects (To explore the possible use case of TC/TT)

 

• http://github.com/oleganza/serials = Scalable serial numbers database (example of TokyoCabinet usage)

• http://github.com/kurokikaze/cassiterite = Simple blog on Node.js + Nginx + Tokyo Tyrant
• http://github.com/markoa/gradmx = Rails app which utilizes Tokyo Tyrant through a simple wrapper around rufus-tokyo 

 

Other libraries/frameworks which utilize TC/TT

 

• http://getcloudkit.com/ = RESTful JSON Storage Rack(Ruby) appliance with oAuth/OpenID support. Also works with CouchDB.
• http://github.com/wycats/moneta = A unified interface to key/value stores. Supports other data stores (eg: redis/couch/memcache/file/s3)
• http://github.com/etolabo/kumofs = Distributed Key-Value store on top of TC. 
• http://opensource.plurk.com/LightCloud = Distributed Key-Value store on top of TT or Redis. Extensible via Lua. Python Interface exists. Ruby Interface under porting.
• http://mrflip.github.com/edamame = Beanstalk + Tokyo Tyrant = Edamame, a fast persistent distributed priority job queue
• https://launchpad.net/blitzdb = Storage engine for the Drizzle that aims to be highly efficient for general purpose use. BlitzDB uses TC for storage. Detailed article is here.  
• http://kawanet.blogspot.com/2009/05/tdserver-experimental-http-interface.html = tdserver: An Experimental HTTP Interface for Tokyo Dystopia full text search system.
• http://www.sevenforge.com/meguro = Meguro, a simple Javascript Map/Reduce framework   
• http://github.com/joshbuddy/tokyo_cache_cow = Memcache server with substring deleting and aggregate functions

• http://code.google.com/p/tcdatabase/ = Rewrite of TC Table DB on top of TC BTree. Supports multiple tables per port. The  detailed slide is here 
  

• lessfs: a deduplicating file system (based on FUSE)

 

Who uses Tokyo Cabinet/Tokyo Tyrant?

• mixi.jp = Japan's biggest Social Network
• Ravelry = Online knitting and crochet community with 3.6 mil view per day
• Netvibes =  To store all users feeds. Use case slide 
• Plakatt =  Event sharing site, to store user actions, search and explore queries, ad and promotion configurations

 

Use Case Interviews

 

• Casey Forbes (Ravelry) 
• Mikio Hirabayashi (Mixi)