Skip to content

Zookeeper – Distributed cluster software

by Topper on May 27th, 2010

I read this article about Apache Zookeeper at Igvita and was intrigued.

I started looking around for ruby libraries, but nothing was as mature as I would like. I forked a branch on github and chugged along on getting the jruby and c versions to work with the same api.

Zookeeper exposes a super simple API, but with that simple stuff you can build a lot of complex cluster logic.

PLAIN TEXT
RUBY:
  1. zk = ZooKeeper.new("localhost:2181", :watcher => :default) #this will handle events using my built-in event handler
  2. zk2 = ZooKeeper.new("localhost:2181", :watcher => false) #this one won't receive watch events
  3.  
  4. zk.watcher.register("/mypath") do |event, zookeeper_client|
  5.   $stderr.puts("got an event on: #{event.path}")
  6. end
  7.  
  8. zk.exists?("/mypath", :watch => true) # returns nil, but sets up the app to watch for the existence of /mypath
  9. zk2.create("/mypath", "my data up to 1mb", :mode => :ephemeral)
  10. # create modes can be any of
  11. # :persistent_sequential, :ephemeral_sequential, :persistent, :ephemeral
  12.  
  13. # now the registered watcher will fire (at least within a few 100 miliseconds)
  14. # because we set that node to be :ephemeral - when zk2 closes its connection, the "/mypath" will go away
  15. # but watches are one-time firing only - so we need to set it up again
  16. zk.exists?("/mypath", :watch => true) #returns true
  17.  
  18. zk2.close! #or delete or whatever
  19.  
  20. # the watcher fires again and
  21. zk.exists?("/mypath") #returns false

A limited api of create, delete, get, set, watch lets you do some really advanced things around a cluster.

Examples

I added some abstractions based on the Zookeeper recipes.

Locks

PLAIN TEXT
RUBY:
  1. #these 2 clients could be on totally separate boxes, different processes, whatever
  2. zk = ZooKeeper.new("localhost:2181", :watcher => :default)
  3. zk2 = ZooKeeper.new("localhost:2181", :watcher => :default)
  4.  
  5. lock1 = zk.locker("/mypath")
  6. lock1.lock #true
  7.  
  8. lock2 = zk2.locker("/mypath")
  9. lock2.lock #false
  10.  
  11. lock1.unlock #true
  12. lock2.lock #true
  13.  
  14. # locks are also released on a client close/crash
  15. lock1.lock #false
  16. zk2.close!
  17. lock1.lock #true

Message Queues

I also implemented a simple message queue on top of zookeeper. However, because of the way the zookeeper "children" calls are made (returning all children), I wouldn't recommend using this for queues where pending messages will reach into the thousands.

PLAIN TEXT
RUBY:
  1. client1 = ZooKeeper.new("localhost:2181", :watcher => :default)
  2.   client2 = ZooKeeper.new("localhost:2181", :watcher => :default)
  3.  
  4.   publisher = client1.queue("myqueue")
  5.   receiver = client2.queue("myqueue")
  6.  
  7.   receiver.subscribe do |title, data|
  8.     # data will be whatever was published, title will be the node name
  9.     # for the message
  10.  
  11.     $stderr.puts "got a message with: #{data}"
  12.  
  13.     # having a true state from the block will mark the message as 'answered'
  14.     # sending back a false will requeue
  15.  
  16.     true
  17.   end

More...

There's a ton you can do with this thing (priority queues, meta data store, etc). I think it's a nice addition to the ruby toolset.

From → Social Web