Bradley Holt » REST

Defining a RESTful Framework

Bradley Holt — Fri, 19 Aug 2011 17:41:49 +0000

Web application frameworks have varying support for the concepts behind Representational State Transfer (REST). Most web application frameworks, if not all, allow you to create “fully” RESTful web applications. However, there does not seem to be a focus on explicitly applying RESTful principles. So, here are the key concepts that I’d like to see addressed:

Embrace, and don’t abstract, the Hypertext Transfer Protocol (HTTP).
Focus on entities/resources—identified by full Uniform Resource Identifiers (URIs).
Use HTTP methods (GET, POST, PUT, DELETE, OPTIONS, HEAD) to perform operations on entities/resources.
Allow for self-described messages through the use of header fields, such as Accept and Content-Type.
Make hypermedia controls a core concept (perhaps using PHACTOR as a starting point), not just a byproduct of rendering.
Think of web applications as state transition systems. Representations of entities/resources are states, and hypermedia controls define the available state transitions.

Benefits:

Interoperability: Focusing on open standards allows for easier integration with other systems.
Cacheability: Embracing HTTP gives you many caching options, almost for free.
Testability: Self-contained and self-describing messages are very testable. Decomposing a complex system into states and available state transitions greatly reduces the complexity of the system and its tests.

CouchDB jQuery Plugin Reference

Bradley Holt — Thu, 14 Jul 2011 22:18:46 +0000

I’ve had a difficult time finding documentation on the CouchDB jQuery plugin that ships with CouchDB. So, I’ve decided to create my own reference and share it with you. This should cover almost the entire CouchDB API that is available through the version of the plugin that ships with CouchDB 1.1.0.

Setup

The same-origin policy effectively requires that the HTML from which the JavaScript is loaded must be served up from CouchDB (you could instead use a reverse proxy server). This can be done by attaching an HTML document to a CouchDB document. You can do this manually, or through the use of CouchApps.

CouchDB ships with several useful JavaScript libraries, including jQuery. Here is an example of an HTML document that takes advantage of CouchDB’s included JavaScript libraries:



  
    CouchDB jQuery Examples

By default, CouchDB will use an empty string as its URL prefix. However, you can configure this before your first API call. For example:

$.couch.urlPrefix = "http://localhost:5984";

You will typically not want to change this value since the same-origin policy prevents you from accessing CouchDB databases hosted at a different origin.

Server API

Server Information

JavaScript example:

$.couch.info({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/ 200 OK

Console output:

{
    "couchdb"="Welcome",
    "version"="1.1.0"
}

All DBs

JavaScript example:

$.couch.allDbs({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/_all_dbs 200 OK

Console output:

[
    "_replicator",
    "_users"
]

User Signup

JavaScript example:

var userDoc = {
    _id: "org.couchdb.user:bob",
    name: "bob"
};
$.couch.signup(userDoc, "supersecurepassword", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

PUT http://localhost:5984/_users/org.couchdb.user%3Abob 201 Created

Console output:

{
    "ok"=true,
    "id"="org.couchdb.user:bob",
    "rev"="1-230dc0625bd3c4aac735846cc152c296"
}

Login

JavaScript example:

$.couch.login({
    name: "bob",
    password: "supersecurepassword",
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

POST http://localhost:5984/_session 200 OK

Console output:

{
    "ok":true,
    "name":"bob",
    "roles":[]
}

This will result in an AuthSession cookie being set and then sent back to the server on subsequent requests, authenticating the user on each request. The session length is 10 minutes by default, but can be set in CouchDB’s couch_httpd_auth.timeout configuration option.

Session

JavaScript example:

$.couch.session({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/_session 200 OK

Console output:

{
    "ok":true,
    "userCtx":{
        "name":"bob",
         "roles":[]
    },
    "info":{
        "authentication_db":"_users",
        "authentication_handlers":[
            "oauth",
            "cookie",
            "default"
        ],
        "authenticated":"cookie"
    }
}

Logout

JavaScript example:

$.couch.logout({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

DELETE http://_:_@localhost:5984/_session 200 OK

Console output:

{
    "ok":true
}

This will set the AuthSession cookie to an empty string, effectively logging the client out.

Active Tasks

JavaScript example:

$.couch.activeTasks({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/_active_tasks 200 OK

Console output:

[]

Server Configuration

JavaScript example:

$.couch.config({
    success: function(data) {
        console.log(data);
    }
}, "uuids", "algorithm");

Resulting HTTP request:

GET http://localhost:5984/_config/uuids/algorithm 200 OK

Console output:

"sequential"

The second and third parameters are the section and option parameters, respectively. Omit the option parameter if you’d like to retrieve the entire section. Omit both the section and option parameters if you’d like to see the entire server configuration. Add an optional fourth parameter to set the configuration option’s value, which will result in a PUT request with the value.

User DB

JavaScript example:

$.couch.userDb(function(data) {
    console.log(data);
});

Resulting HTTP request:

GET http://localhost:5984/_session 200 OK

Console output:

{
    "name"="_users",
    "uri"="../_users/"
}

The URI will be relative to the HTML executing the script.

New UUID

JavaScript example:

var uuid = $.couch.newUUID();
console.log(uuid);

Resulting HTTP request:

GET http://localhost:5984/_uuids?count=1 200 OK

Console output:

d12ee5ea1df6baa2b06451f44a0156fa

This function takes a count parameter as an optional first parameter. If the count parameter is greater than one, then it will keep a cache of UUIDs that will be returned on subsequent calls, rather than making another request to the server.

Replicate

JavaScript example:

$.couch.replicate("mydb", "otherdb", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
}, {
    create_target: true
});

Resulting HTTP request:

POST http://localhost:5984/_replicate 200 OK

Console output:

{
    "ok":true,
    "session_id":"705797bae87456c52f5b76f44fe5f245",
    "source_last_seq":27,
    "replication_id_version":2,
    "history":[
        {
            "session_id":"705797bae87456c52f5b76f44fe5f245",
            "start_time":"Wed, 13 Jul 2011 22:40:09 GMT",
            "end_time":"Wed, 13 Jul 2011 22:40:09 GMT",
            "start_last_seq":0,
            "end_last_seq":27,
            "recorded_seq":27,
            "missing_checked":0,
            "missing_found":14,
            "docs_read":14,
            "docs_written":14,
            "doc_write_failures":0
        }
    ]
}

Database API

Create Database

JavaScript example:

$.couch.db("mydb").create({
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

PUT http://localhost:5984/mydb/ 201 Created

Console output:

{
    "ok":true
}

Database Info

JavaScript example:

$.couch.db("mydb").info({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/mydb/ 200 OK

Console output:

{
    "db_name":"mydb",
    "doc_count":0,
    "doc_del_count":0,
    "update_seq":0,
    "purge_seq":0,
    "compact_running":false,
    "disk_size":79,
    "instance_start_time":"1310597000825187",
    "disk_format_version":5,
    "committed_update_seq":0
}

Save New Document

JavaScript example:

var doc = {};
$.couch.db("mydb").saveDoc(doc, {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/ 201 Created

Console output:

{
    "ok":true,
    "id":"d12ee5ea1df6baa2b06451f44a019ab9",
    "rev":"1-967a00dff5e02add41819138abb3284d"
}

Open Document

JavaScript example:

$.couch.db("mydb").openDoc("d12ee5ea1df6baa2b06451f44a019ab9", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

GET http://localhost:5984/mydb/d12ee5ea1df6baa2b06451f44a019ab9 200 OK

Console output:

{
    "_id":"d12ee5ea1df6baa2b06451f44a019ab9",
    "_rev":"1-967a00dff5e02add41819138abb3284d"
}

Note that the HTTP response includes an Etag HTTP header. If you do a subsequent request for the same document, and your client supports caching and conditional HTTP requests, then an If-None-Match header should be sent using the value from the previous response’s Etag header. If the document has not been modified, then CouchDB will send a 304 Not Modified response and no response body, saving bandwidth and speeding up the response.

Save Updated Document

JavaScript example:

var doc = {
    _id: "d12ee5ea1df6baa2b06451f44a019ab9",
    _rev: "1-967a00dff5e02add41819138abb3284d",
    foo: "bar"
};
$.couch.db("mydb").saveDoc(doc, {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

PUT http://localhost:5984/mydb/d12ee5ea1df6baa2b06451f44a019ab9 201 Created

Console output:

{
    "ok":true,
    "id":"d12ee5ea1df6baa2b06451f44a019ab9",
    "rev":"2-13839535feb250d3d8290998b8af17c3"
}

Remove Document

JavaScript example:

var doc = {
    _id: "d12ee5ea1df6baa2b06451f44a019ab9",
    _rev: "2-13839535feb250d3d8290998b8af17c3"
};
$.couch.db("mydb").removeDoc(doc, {
     success: function(data) {
         console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

DELETE http://localhost:5984/mydb/d12ee5ea1df6baa2b06451f44a019ab9?rev=2-13839535… 200 OK

Console output:

{
    "id":d12ee5ea1df6baa2b06451f44a019ab9
    "ok":true,
    "rev":"3-1f04f977685e1108b4664f70b09c6f65"
}

Bulk Save

JavaScript example:

$.couch.db("mydb").bulkSave({"docs": [{}, {}]}, {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_bulk_docs?successStatus=201 201 Created

Console output:

[
    {
        "id":"d12ee5ea1df6baa2b06451f44a01a0d8",
        "rev":"1-967a00dff5e02add41819138abb3284d"
    },
    {
        "id":"d12ee5ea1df6baa2b06451f44a01a75a",
        "rev":"1-967a00dff5e02add41819138abb3284d"
    }
]

Bulk Remove

JavaScript example:

var docs = [
    {
        _id: "d12ee5ea1df6baa2b06451f44a01a0d8",
        _rev: "1-967a00dff5e02add41819138abb3284d"
    },
    {
        _id: "d12ee5ea1df6baa2b06451f44a01a75a",
        _rev: "1-967a00dff5e02add41819138abb3284d"
    }
];
$.couch.db("mydb").bulkRemove({"docs": docs}, {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_bulk_docs?successStatus=201 201 Created

Console output:

[
    {
        "id":"d12ee5ea1df6baa2b06451f44a01a0d8",
        "rev":"2-eec205a9d413992850a6e32678485900"
    },
    {
        "id":"d12ee5ea1df6baa2b06451f44a01a75a",
        "rev":"2-eec205a9d413992850a6e32678485900"
    }
]

This operates almost exactly like the bulk save, but instead sets the deleted flag to true on the documents.

All Documents

JavaScript example:

$.couch.db("mydb").allDocs({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET http://localhost:5984/mydb/_all_docs 200 OK

Console output:

{
    "total_rows":11,
    "offset":0,
    "rows":[
        {
            "id":"_design/default",
            "key":"_design/default",
            "value":{
                "rev":"9-5212dde9da06f1933dbe29811fc380d4"
            }
        },
        {
            "id":"d12ee5ea1df6baa2b06451f44a002cef",
            "key":"d12ee5ea1df6baa2b06451f44a002cef",
            "value":{
                "rev":"1-967a00dff5e02add41819138abb3284d"
            }
        },
        …
    ]
}

All Design Documents

JavaScript example:

$.couch.db("mydb").allDesignDocs({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

GET 
http://localhost:5984/mydb/_all_docs?startkey=%22_design%22&endkey=%22_design0%22 200 OK

Console output:

{
    "total_rows":11,
    "offset":0,
    "rows":[
        {
            "id":"_design/default",
            "key":"_design/default",
            "value":{
                "rev":"9-5212dde9da06f1933dbe29811fc380d4"
            }
        }
    ]
}

All Apps

JavaScript example:

$.couch.db("tutorial").allApps({
    success: function(data) {
        console.log(data);
    },
    eachApp: function(appName, appPath, ddoc) {
        console.log(appName);
        console.log(appPath);
        console.log(ddoc);
    }
});

Resulting HTTP requests:

GET http://localhost:5984/tutorial/_all_docs?startkey=%22_design%22&endkey=%22_design0%22 200 OK

GET http://localhost:5984/tutorial/_design/tutorial 200 OK

Console output:

tutorial

/tutorial/_design/tutorial/index.html

{
    "_id":"_design/tutorial",
    "_rev":"5-276897ecff6dc272c0b9143aa23e8a2e",
    …
}

This is intended as a convenience to find metadata about CouchApps.

Query

JavaScript example:

var mapFunction = function(doc) {
    emit();
};
$.couch.db("mydb").query(mapFunction, "_count", "javascript", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    },
    reduce: false
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_temp_view?reduce=false 200 OK

Console output:

{
    "total_rows":10,
    "offset":0,
    "rows":[
        {
            "id":"d12ee5ea1df6baa2b06451f44a002cef",
            "key":null,
            "value":null
        },
        {
            "id":"d12ee5ea1df6baa2b06451f44a0037b5",
            "key":null,
            "value":null
        },
        {
            "id":"d12ee5ea1df6baa2b06451f44a003b06",
            "key":null,
            "value":null
        },
        …
    ]
}

This creates a temporary view as defined by your map and reduce functions. Temporary views are useful in development, but should be replaced with views permanently saved to design documents for production. Temporary views are slow, especially with a large number of documents.

View

JavaScript example:

$.couch.db("mydb").view("default/all", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    },
    reduce: false
});

Resulting HTTP request:

GET http://localhost:5984/mydb/_design/default/_view/all?reduce=false 200 OK

Console output:

{
    "total_rows":10,
    "offset":0,
    "rows":[
        {
            "id":"d12ee5ea1df6baa2b06451f44a002cef",
            "key":null,
            "value":null
        },
        {
            "id":"d12ee5ea1df6baa2b06451f44a0037b5",
            "key":null,
            "value":null
        },
        {
            "id":"d12ee5ea1df6baa2b06451f44a003b06",
            "key":null,
            "value":null
        },
        …
    ]
}

List

JavaScript example:

$.couch.db("mydb").list("default/all", "all", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    },
    reduce: false
});

Resulting HTTP request:

GET http://localhost:5984/mydb/_design/default/_list/all/all?reduce=false 200 OK

The console output will depend on your list’s behavior.

Compact

JavaScript example:

$.couch.db("mydb").compact({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_compact 202 Accepted

Console output:

{
    "ok":true
}

View Cleanup

JavaScript example:

$.couch.db("mydb").viewCleanup({
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_view_cleanup 202 Accepted

Console output:

{
    "ok":true
}

Compact View

JavaScript example:

$.couch.db("mydb").compactView("default", {
    success: function(data) {
        console.log(data);
    }
});

Resulting HTTP request:

POST http://localhost:5984/mydb/_compact/default 202 Accepted

Console output:

{
    "ok":true
}

Changes

JavaScript example:

$.couch.db("mydb").changes().onChange(function(data) {
    console.log(data);
});

Resulting HTTP requests:

GET http://localhost:5984/mydb/ 200 OK

GET http://localhost:5984/mydb/_changes?heartbeat=10000&feed=longpoll&since=34 200 OK

This will keep a connection open until changes are detected, and then open up another connection to listen for subsequent changes starting from the next sequence.

Console output when a new document is created:

{
    "results":[
        {
            "seq":35,
            "id":"d12ee5ea1df6baa2b06451f44a01b7b5",
            "changes":[
                {
                    "rev":"1-967a00dff5e02add41819138abb3284d"
                }
            ]
        }
    ],
    "last_seq":35
}

This is very powerful feature. It allows you to listen for any changes to documents in the database and immediately respond to those changes.

Copy Document

JavaScript example:

$.couch.db("mydb").copyDoc("d12ee5ea1df6baa2b06451f44a01b7b5", {
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
}, {
    beforeSend: function(xhr) {
        xhr.setRequestHeader("Destination", "aNewDocId");
    }
});

Resulting HTTP request:

COPY http://localhost:5984/mydb/d12ee5ea1df6baa2b06451f44a01b7b5 201 Created

Console output:

{
    "id":"aNewDocId",
    "rev":"1-967a00dff5e02add41819138abb3284d"
}

Drop Database

JavaScript example:

$.couch.db("mydb").drop({
    success: function(data) {
        console.log(data);
    },
    error: function(status) {
        console.log(status);
    }
});

Resulting HTTP request:

DELETE http://localhost:5984/mydb/ 200 OK

Console output:

{
    "ok":true
}

I’ll be giving a presentation on CouchApps at OSCON if you want to learn more about building CouchApps. This presentation will be a part of the JavaScript and HTML5 track. You can save 20% on registration with the code OS11RAD. I’ll also be presenting a tutorial on Learning CouchDB at OSCON Data.

You can also check out my books Writing and Querying MapReduce Views in CouchDB and Scaling CouchDB if you’re interested in learning more about CouchDB in general.

Update (7/15/2011): Max Ogden has pointed me to Dale Harvey‘s documentation generator for jquery.couch.js and a copy of the generated jQuery CouchDB documentation. Max and Dale are also working on a more flexible jquery.couch2.js.

CouchDB 1.1 Released

Bradley Holt — Tue, 07 Jun 2011 21:40:11 +0000

CouchDB 1.1 was released this past Friday. CouchBase has a CouchDB Release 1.1 Feature Guide that provides details on the new features available in CouchDB 1.1. Some of the key new features include:

A replicator database that stores replications which are automatically restarted when the server restarts
Native SSL support—you can now connect to CouchDB securely without maintaining a separate proxy server that handles SSL
HTTP range requests—this allows for resumable downloads and skippable audio and video streams
HTTP proxying—allows external content to be served from your CouchDB URL
Map functions now support CommonJS—this allows for reuse of JavaScript code
More granular ETags on view responses—this increases the probability of a 304 Not Modified response when doing conditional requests

See the CouchDB Release 1.1 Feature Guide for the complete list and more detail.

Scaling CouchDB

Bradley Holt — Wed, 30 Mar 2011 14:41:55 +0000

My latest book, Scaling CouchDB, is now available in ebook format. This is a short book (about 72 pages) and serves as a practical guide to scaling CouchDB and designing a distributed system to meet your capacity needs. Replication, conflict resolution, load balancing, clustering, distributed load testing, and monitoring are covered. The chapters on load balancing (using Apache) and distributed load testing (using Tsung) are broadly applicable, even if you aren’t a CouchDB user.

If you blog and would like to review Scaling CouchDB or my previous book, Writing and Querying MapReduce Views in CouchDB, then please let me know and I can arrange to have you sent a review copy.

Update (3/31/2011): At the request of a commenter, here is the table of contents:

Defining Scaling Goals
1. What is Scalability?
2. Capacity Planning
3. The CAP Theorem
  1. Consistency
  2. Availability
  3. Partition Tolerance
Tuning and Designing for Scale
1. Performance Tips
2. Document Design
Replication
1. Filters and Specifying Documents
2. Conflict Resolution
  1. Picking the Same Revision as CouchDB
  2. Picking a Conflicted Revision
  3. Merging Revisions
Load Balancing
1. CouchDB Nodes
2. Replication Setup
3. Proxy Server Configuration
4. Testing
Clustering
1. BigCouch
2. Lounge
3. Pillow
Distributed Load Testing
1. Installing Tsung
2. Configuring Tsung
3. Running Tsung
4. Monitoring
5. Identifying Bottlenecks
6. Test Configuration

One Web

Bradley Holt — Mon, 28 Mar 2011 16:41:50 +0000

Almost two years ago, Luke Wroblewski first described a trend in web development called mobile first. The basic idea was that web applications should be designed for mobile first, as opposed to designed for the desktop first. Luke provided some compelling reasons for this including the explosive growth of mobile adoption, the fact that mobile forces you to focus on key areas of your application, and that mobile extends your capabilities. Today I saw this tweet from Chris Shiflett:

The Web still trumps the Mobile Web. If you’re making a web app, don’t let “mobile first” lead you astray.

Luke’s original premise is not wrong, but I think many web developers have interpreted “mobile first” as “mobile only”—often intentionally, sometimes by accident. As Chris pointed out, it’s easy to let “mobile first” lead you astray. My response to Chris was that the goal should be One Web, even if “mobile first”. “Mobile only” should rarely, if ever, be the goal. The concept of One Web is described in the W3C’s Mobile Web Best Practices 1.0 Basic Guidelines. From the original document:

The recommendations in this document are intended to improve the experience of the Web on mobile devices. While the recommendations are not specifically addressed at the desktop browsing experience, it must be understood that they are made in the context of wishing to work towards “One Web”.

As discussed in the Scope document [Scope], One Web means making, as far as is reasonable, the same information and services available to users irrespective of the device they are using. However, it does not mean that exactly the same information is available in exactly the same representation across all devices. The context of mobile use, device capability variations, bandwidth issues and mobile network capabilities all affect the representation. Furthermore, some services and information are more suitable for and targeted at particular user contexts (see 5.1.1 Thematic Consistency of Resource Identified by a URI).

Let’s not let the web fragment into a “mobile web”, a “desktop web”, and a “whatever comes next” web—there’s no reason for this. The underlying technology is designed to allow for One Web, as Ben Ramsey added:

@BradleyHolt @shiflett This is why we have content types, accept headers, user agent strings, and content negotiation.

If you’re considering taking a mobile first approach, please consider taking the One Web approach instead. Your main focus can be on mobile to start, but at least deliver something of value that is not dependent on the client being a mobile device. As Ben suggested, use content types, accept headers, user agent strings, and content negotiation to deliver the best experience based on your user’s device or browser.

CouchDB at New York PHP

Bradley Holt — Wed, 23 Feb 2011 23:06:49 +0000

Last night I gave a presentation on CouchDB at the New York PHP User Group. I talked about the basics of CouchDB, its JSON documents, its RESTful API, writing and querying MapReduce views, using CouchDB from within PHP, and scaling. The talk was broadcast and recorded on Ustream.

A big thanks to New York PHP (especially Hans Zaunere, Daniel Krook, Alan Seiden, and Isaac Foster) for having me as a guest! If you like what you see here then I hope you’ll consider buying my book, Writing and Querying MapReduce Views in CouchDB or my upcoming book, Scaling CouchDB.

Update (2/27/2011): I’ve uploaded the slides from my talk to SlideShare and used the recorded audio to create a slidecast.

CouchDB at New York PHP

View more webinars from Bradley Holt

Writing and Querying MapReduce Views in CouchDB

Bradley Holt — Mon, 31 Jan 2011 16:43:29 +0000

My first book, Writing and Querying MapReduce Views in CouchDB, has been published by O’Reilly Media. It is a short and concise ebook with step-by-step instructions and lots of sample code. Most examples are provided both in Futon and using CouchDB’s RESTful HTTP API (using cURL).

In my experience, web developers who are new to CouchDB often encounter three main barriers to understanding and using CouchDB. First, its JSON documents. Second, its RESTful HTTP API. Third, its MapReduce views. The first two are fairly straightforward and many web developers already have experience with JSON and RESTful HTTP APIs. Once you understand the benefits of using JSON as a document format and exposing the database through a RESTful HTTP API, then understanding MapReduce views becomes the main barrier.

The goal of this book is to walk readers through both writing MapReduce views and then querying these same views. Both the Map and Reduce steps are explained separately. Several example Map functions are demonstrated and the built-in Reduce functions are covered. There are also discussions about custom Reduce functions and the limitations of MapReduce. There are examples of creating both temporary views and saving views permanently to design documents. Finally, there is a chapter on querying views which talks about range queries, limiting rows, skipping rows, reversing results, exact grouping, group levels, and including the original documents in query results.

P.S. If you are a blogger and would like to review this book, then please ping me and let me know.

Behind The New VPR Homepage

Bradley Holt — Mon, 11 Oct 2010 15:09:47 +0000

Found Line had the privilege of designing and developing Vermont Public Radio‘s (VPR) new homepage. VPR’s Online Manager, Jonathan Butler, wrote a blog post about how the redesigned homepage delivers more content to VPR.net visitors. Here I’ll talk about the technology behind this new homepage.

Atom Syndication Format

Most of VPR’s web content is currently stored in a proprietary content management system (CMS). The first step was publishing the needed content from the CMS out into an open format. VPR’s Dan Allen published from the CMS Atom feeds of the various content we would need for the new homepage.

PHP and Zend Framework

Once we had the needed Atom feeds, we created a Zend Framework web application to import these feeds. Components such as Zend_Feed_Reader made the job easier. The weather data comes from the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service’s National Digital Forecast Database XML/REST Service. The market data comes from the Google Finance API.

CouchDB

All of the data (except for weather, which is pulled directly with a bit of caching) is stored in CouchDB. Since we’re storing documents with varying metadata (i.e. Atom entries) choosing a document-oriented database made a lot of sense. CouchDB’s map/reduce views make for very efficient queries. Its RESTful HTTP API provides a lot of options for scaling and caching. We’re caching documents within the Zend Framework application and conditionally requesting these documents from CouchDB with ETags when a cache is available. The current cache is simply file-based but we’re considering using Memcached for even better performance. On a related note, the homepage itself also supports conditional HTTP requests.

Apache and nginx

We had the pleasure of working with another great Vermont company, ClearBearing, on this project. ClearBearing handles the infrastructure layer for VPR.net. The current website was (and still is) served up by Apache. ClearBearing already had nginx in place as a reverse proxy to handle caching. Since we weren’t replacing the entire CMS we needed a way for the existing CMS and this new application to coexist. We decided to host the new application separately (still administered by ClearBearing) and reverse proxy requests for certain URIs (primarily the homepage) to the new application. This provides a clean separation between the new application and the existing CMS while still allowing for them to be hosted on the same domain from an end-user perspective.

Licensing

You’ll notice that all of the standards used are open standards and all of the technologies used are free and open source software. Additionally, all of the code we have written for VPR is licensed to them under the permissive New BSD License. This means that VPR is free to use this code however they want, study and modify the code, and redistribute the code as-is or modified. They can do all of this without paying additional licensing fees.

Design

While this post is primarily about the technology behind VPR’s new homepage, I should mention that Found Line’s Jason Pelletier designed the new homepage and implemented all of its HTML and CSS. We worked closely with Jonathan Butler and John Van Hoesen, Vice President for News & Programming, to make sure the new design meets the needs of website visitors and the organization.

Next Steps

The launch of this new homepage was just one iteration with many more to come. As Jonathan Butler mentioned in his blog post, VPR has plans to further improve online services for their listeners. Look for a redesigned VPR Classical page soon. There are also plans for better mobile access to VPR and VPR Classical programming. Fortunately much of the work that has been done can be leveraged to create additional online services for VPR’s listeners.

A special thanks for the support of VPR’s listeners—you made this new homepage possible! If you have questions, comments, complaints, or suggestions about the new homepage then please contact VPR directly with your feedback (although you’re welcome to comment here as well).

CouchDB 1.0 Released

Bradley Holt — Wed, 14 Jul 2010 16:25:59 +0000

Apache CouchDB is a free/open source RESTful JSON document (NoSQL) database with map reduce views and peer-based replication. Version 1.0 was just released today and is 300% faster than the previous version and includes Microsoft Windows support, an authentication system, and flexible replicator options. The New York Times, ReadWriteEnterprise and InfoWorld covered the release. Couchio has a clever release announcement as well. I’ve been reading up on CouchDB since Matthew Weier O’Phinney‘s presentation on Document Databases at the last Burlington, VT PHP Users Group meeting and I may have some projects where CouchDB will be a good fit.

Bradley Holt » REST

Top Five Posts of 2011

5. CouchDB and Domain-Driven Design

4. Addressing the NoSQL Criticism

3. Testing PHP 5.4

2. Exploring RabbitMQ and PHP

1. CouchDB jQuery Plugin Reference

Defining a RESTful Framework

CouchDB jQuery Plugin Reference

Setup

Server API

Server Information

All DBs

User Signup

Login

Session

Logout

Active Tasks

Server Configuration

User DB

New UUID

Replicate

Database API

Create Database

Database Info

Save New Document

Open Document

Save Updated Document

Remove Document

Bulk Save

Bulk Remove

All Documents

All Design Documents

All Apps

Query

View

List

Compact

View Cleanup

Compact View

Changes

Copy Document

Drop Database

CouchDB 1.1 Released

Scaling CouchDB

One Web

CouchDB at New York PHP

Writing and Querying MapReduce Views in CouchDB

Behind The New VPR Homepage

Atom Syndication Format

PHP and Zend Framework

CouchDB

Apache and nginx

Licensing

Design

Next Steps

CouchDB 1.0 Released