“What’s the future of CouchDB? It’s Couchbase.” —Damien Katz

“The future of CouchDB is CouchDB.” —Noah Slater

First of all, don’t panic. The Apache CouchDB project is thriving. There are plenty of core contributors who are dedicated to the project. If you’re using CouchDB, you’re fine. If you’re considering using CouchDB, you’ll be fine. CouchDB has a bright future ahead of it. Couchbase has repeatedly said that they’re not the CouchDB company, so this shouldn’t be a surprise to anyone.

My interpretation of this announcement is that CouchDB is forking. Going forward, you’ll have two choices, either Apache CouchDB or Couchbase Server. The road map for Apache CouchDB will continue to be determined by community consensus. The road map for Couchbase Server will be determined by Couchbase, the company.

CouchDB has two futures: CouchDB and Couchbase Server. Traditionally, forking a project in this manner has been considered a Bad Thing. From The Jargon File entry for forked:

An open-source software project is said to have forked or be forked when the project group fissions into two or more parts pursuing separate lines of development (or, less commonly, when a third party unconnected to the project group begins its own line of development). Forking is considered a Bad Thing — not merely because it implies a lot of wasted effort in the future, but because forks tend to be accompanied by a great deal of strife and acrimony between the successor groups over issues of legitimacy, succession, and design direction. There is serious social pressure against forking. As a result, major forks (such as the Gnu-Emacs/XEmacs split, the fissionings of the 386BSD group into three daughter projects, and the short-lived GCC/EGCS split) are rare enough that they are remembered individually in hacker folklore.

In my humble opinion, forking should no longer be considered a Bad Thing. Distributed revision control systems, such as Git, have made it much easier to merge changes back-and-forth between forked projects. If an individual or organization wants to take a project in a different direction, great! It sounds like Couchbase Server will be designed to meet a set of needs that are not being met by CouchDB. If so, a fork is perfectly appropriate as now both sets of needs can be met.

5. CouchDB and Domain-Driven Design

This post covered two topics that are of great interest to me. Document databases like CouchDB have much potential when it comes to domain-driven design. The post outlined some techniques for serializing domain objects into CouchDB documents.

4. Addressing the NoSQL Criticism

I heard a lot of NoSQL bashing this year at OSCON. As a result, I decided to write a rebuttal to many of the criticisms that I heard. This post inspired a lively comment thread.

3. Testing PHP 5.4

This was a short post on how to help the PHP Quality Assurance Team by running the tests for the latest development version of PHP 5.4. It covered downloading, compiling, and running the tests. Most importantly, it also covered reporting your test results.

2. Exploring RabbitMQ and PHP

I was exploring the possibility of using RabbitMQ for an upcoming project. I decided to post some notes on my experience getting started with RabbitMQ and integrating it with PHP. I haven’t ended up using RabbitMQ in production yet, but I’m confident that I’ll end up using it at some point.

1. CouchDB jQuery Plugin Reference

I had been frustrated by the lack of documentation for the CouchDB jQuery plugin—so decided to write my own. After I wrote the blog post, Max Ogden pointed me to Dale Harvey‘s documentation generator for jquery.couch.js and a copy of the generated jQuery CouchDB documentation. Max and Dale are also working on a more flexible jquery.couch2.js.

Personally, I’d like to encourage submissions on the following topics:

• More talks related to free/open source software (PHP, Ruby, Python, etc.)
• Arduino and/or other hardware programming
• Software development patterns and practices
• Web analytics and metrics
• Front-end web development (JavaScript, CSS, etc.)
• Node.js and other emerging technologies
• Mobile application development
• Anything you’re passionate about!

Propose a session or two now—you know you want to!

I’ll be using RabbitMQ first from PHP, but I plan on using it to send and receive messages to and from other systems. Following are the steps I used to get RabbitMQ and PHP’s AMQP library setup on my development machine.

First, I installed RabbitMQ using MacPorts:

$ sudo port install rabbitmq-server

Then, I started RabbitMQ:

$ sudo rabbitmq-server -detached

Next, I installed the librabbitmq library using a slight variation of the instructions on PHP’s AMQP Installation page (you may need to install Mercurial first):

$ hg clone http://hg.rabbitmq.com/rabbitmq-c/rev/3c549bb09c16 rabbitmq-c
$ cd rabbitmq-c
$ hg clone http://hg.rabbitmq.com/rabbitmq-codegen/rev/f8b34141e6cb codegen
$ autoreconf -i && ./configure && make && sudo make install

Then, I installed the AMQP extension using PECL:

$ sudo pecl install amqp-beta

To test that everything works, I opened up two interactive PHP shells using php -a. I ran the following code in the first PHP shell:

$exchangeName = 'messages';
$routeKey = 'routeA';
$message = 'Hello, world.';
$connection = new AMQPConnection();
$connection->connect();
$exchange = new AMQPExchange($connection);
$exchange->declare($exchangeName);

I then ran the following code in the second PHP shell:

$exchangeName = 'messages';
$routeKey = 'routeA';
$connection = new AMQPConnection();
$connection->connect();
$queue = new AMQPQueue($connection);
$queue->declare($exchangeName);
$queue->bind($exchangeName, $routeKey);

Back in the first PHP shell:

$exchange->publish($message, $routeKey);

Back in the second PHP shell:

$message = $queue->get();
print_r($message);

Here is the output I got from the print_r statement:

Array
(
    [routing_key] => routeA
    [exchange] => messages
    [delivery_tag] => 1
    [Content-type] => text/plain
    [count] => 0
    [msg] => Hello, world.
)

There are several other options that can be set, and a lot more to learn about RabbitMQ and AMP. Check out the documentation for PHP’s AMQP extension for details about working with AMQP servers from PHP.

First, make sure that you have subversion, autoconf, automake, gcc, bison, flex, and re2c installed. I’m using a Mac, so I was able to install most of these using MacPorts (Homebrew would have worked, too). Next, do a checkout of the PHP 5.4 branch:

$ svn co https://svn.php.net/repository/php/php-src/branches/PHP_5_4 php54

Then:

$ cd php54
$ ./buildconf
$ ./configure
$ make
$ make test

Both the make and make test steps may take awhile. Assuming you get some failed tests like I did, then you will see the following message and prompt:

You may have found a problem in PHP.
This report can be automatically sent to the PHP QA team at
http://qa.php.net/reports and http://news.php.net/php.qa.reports
This gives us a better understanding of PHP's behavior.
If you don't want to send the report immediately you can choose
option "s" to save it.    You can then email it to qa-reports@lists.php.net later.
Do you want to send this report now? [Yns]:

I opted to send the report now, so entered Y. Next, I was prompted for my email address:

Please enter your email address.
(Your address will be mangled so that it will not go out on any
mailinglist in plain text):

I entered my email address and then:

Posting to http://qa.php.net/buildtest-process.php

Thank you for helping to make PHP better.

Pretty straightforward. At this point, your report should be aggregated on the PHP Test Reports Summary page. If you want to run more tests, then configure as many extensions as possible during the configure step. Take a look at the shell script in Rasmus’ instructions for details. If you’d like, you can examine the failed tests and see if you can come up with a patch to fix the problem. Again, see Rasmus’ instructions for details.

If you blog and would like to review Scaling CouchDB or my previous book, Writing and Querying MapReduce Views in CouchDB, then please let me know and I can arrange to have you sent a review copy.

Update (3/31/2011): At the request of a commenter, here is the table of contents:

1. Defining Scaling Goals
   1. What is Scalability?
   2. Capacity Planning
   3. The CAP Theorem
      1. Consistency
      2. Availability
      3. Partition Tolerance
2. Tuning and Designing for Scale
   1. Performance Tips
   2. Document Design
3. Replication
   1. Filters and Specifying Documents
   2. Conflict Resolution
      1. Picking the Same Revision as CouchDB
      2. Picking a Conflicted Revision
      3. Merging Revisions
4. Load Balancing
   1. CouchDB Nodes
   2. Replication Setup
   3. Proxy Server Configuration
   4. Testing
5. Clustering
   1. BigCouch
   2. Lounge
   3. Pillow
6. Distributed Load Testing
   1. Installing Tsung
   2. Configuring Tsung
   3. Running Tsung
   4. Monitoring
   5. Identifying Bottlenecks
   6. Test Configuration

In my experience, web developers who are new to CouchDB often encounter three main barriers to understanding and using CouchDB. First, its JSON documents. Second, its RESTful HTTP API. Third, its MapReduce views. The first two are fairly straightforward and many web developers already have experience with JSON and RESTful HTTP APIs. Once you understand the benefits of using JSON as a document format and exposing the database through a RESTful HTTP API, then understanding MapReduce views becomes the main barrier.

The goal of this book is to walk readers through both writing MapReduce views and then querying these same views. Both the Map and Reduce steps are explained separately. Several example Map functions are demonstrated and the built-in Reduce functions are covered. There are also discussions about custom Reduce functions and the limitations of MapReduce. There are examples of creating both temporary views and saving views permanently to design documents. Finally, there is a chapter on querying views which talks about range queries, limiting rows, skipping rows, reversing results, exact grouping, group levels, and including the original documents in query results.

P.S. If you are a blogger and would like to review this book, then please ping me and let me know.

Atom Syndication Format

Most of VPR’s web content is currently stored in a proprietary content management system (CMS). The first step was publishing the needed content from the CMS out into an open format. VPR’s Dan Allen published from the CMS Atom feeds of the various content we would need for the new homepage.

PHP and Zend Framework

Once we had the needed Atom feeds, we created a Zend Framework web application to import these feeds. Components such as Zend_Feed_Reader made the job easier. The weather data comes from the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service’s National Digital Forecast Database XML/REST Service. The market data comes from the Google Finance API.

CouchDB

All of the data (except for weather, which is pulled directly with a bit of caching) is stored in CouchDB. Since we’re storing documents with varying metadata (i.e. Atom entries) choosing a document-oriented database made a lot of sense. CouchDB’s map/reduce views make for very efficient queries. Its RESTful HTTP API provides a lot of options for scaling and caching. We’re caching documents within the Zend Framework application and conditionally requesting these documents from CouchDB with ETags when a cache is available. The current cache is simply file-based but we’re considering using Memcached for even better performance. On a related note, the homepage itself also supports conditional HTTP requests.

Apache and nginx

We had the pleasure of working with another great Vermont company, ClearBearing, on this project. ClearBearing handles the infrastructure layer for VPR.net. The current website was (and still is) served up by Apache. ClearBearing already had nginx in place as a reverse proxy to handle caching. Since we weren’t replacing the entire CMS we needed a way for the existing CMS and this new application to coexist. We decided to host the new application separately (still administered by ClearBearing) and reverse proxy requests for certain URIs (primarily the homepage) to the new application. This provides a clean separation between the new application and the existing CMS while still allowing for them to be hosted on the same domain from an end-user perspective.

Licensing

You’ll notice that all of the standards used are open standards and all of the technologies used are free and open source software. Additionally, all of the code we have written for VPR is licensed to them under the permissive New BSD License. This means that VPR is free to use this code however they want, study and modify the code, and redistribute the code as-is or modified. They can do all of this without paying additional licensing fees.

Design

While this post is primarily about the technology behind VPR’s new homepage, I should mention that Found Line’s Jason Pelletier designed the new homepage and implemented all of its HTML and CSS. We worked closely with Jonathan Butler and John Van Hoesen, Vice President for News & Programming, to make sure the new design meets the needs of website visitors and the organization.

Next Steps

The launch of this new homepage was just one iteration with many more to come. As Jonathan Butler mentioned in his blog post, VPR has plans to further improve online services for their listeners. Look for a redesigned VPR Classical page soon. There are also plans for better mobile access to VPR and VPR Classical programming. Fortunately much of the work that has been done can be leveraged to create additional online services for VPR’s listeners.

A special thanks for the support of VPR’s listeners—you made this new homepage possible! If you have questions, comments, complaints, or suggestions about the new homepage then please contact VPR directly with your feedback (although you’re welcome to comment here as well).

There will be four rooms with sessions on .NET, PHP, Ruby, Python, Java, and more. Two of the rooms will have .NET topics and another will have sessions on free/open source software. There will be a fourth room where developers will be introduced to various technologies that they may not use every day. For example, are you a .NET developer? Come and learn about PHP, Ruby, Python, or Java. A Ruby developer? Learn about .NET, PHP, Python, or Java. You get the idea.

Vermont Code Camp is a great way to bring together technology user groups from around Vermont. The Vermont.NET User Group is the primary organizer of the event but we hope to see others from the Burlington, VT PHP Users Group (which I organize), the Vermont Ruby On Rails User Group, the Vermont Area Group of Unix Enthusiasts (VAGUE), the Green Mountain Oracle Users Group, the New England Adobe User Group, and the VT SQL Server Users Group there. Do you know of any other Vermont technology user groups? Please let me know so that we can invite them!