Text

· · to read

Here’s a copy of the email I sent to the CouchDB users mailing list:

I made a couple of npm (node.js) modules for editing CouchDB design documents that involves fewer files than python CouchApp, but like python CouchApp supports two-way sync. The function source is left intact when converting between JavaScript source and JSON. The JavaScript source version just shows it in an embedded function expression, which makes it syntax highlightable.

http://benatkin.com/2012/09/04/js2json-and-json2js/
https://github.com/benatkin/js2json
https://github.com/benatkin/json2js

Here’s a quick example. If I stick this in books.js:

module.exports = {
  "views": {
    "author": {
      "map": function(doc) {
        if (doc.type == 'book')
          emit(doc.author, null);
      }
    },
    "title": {
      "map": function(doc) {
        if (doc.type == 'book')
          emit(doc.title, null);
      }
    }
  }
}

…and run this (after npm install json2js js2json):

var js2json = require('js2json');
var json2js = require('json2js');
var fs = require('fs');

var jsSource = fs.readFileSync('books.js', 'utf8');
var jsonValue = js2json.convert(jsSource);
fs.writeFileSync('books.json', JSON.stringify(jsonValue, null, 2) + "\n");
var jsSourceFromJson = json2js.convert(jsonValue);
fs.writeFileSync('books-from-json.js', jsSourceFromJson + "\n");

…I get the following in books.json:

{
  "views": {
    "author": {
      "map": "function(doc) {\n  if (doc.type == 'book')\n
emit(doc.author, null);\n}"
    },
    "title": {
      "map": "function(doc) {\n  if (doc.type == 'book')\n
emit(doc.title, null);\n}"
    }
  }
}

…and books-from-json.js is exactly the same as books.js.

I explain it more in my blog post (linked at the top of this message). I need to add a cli tool that syncs, a way to handle attachments, and a way to handle embedded multiline strings for it to be a full-featured design doc editor. I have much bigger plans for this, though: I want to break up CouchApps into a bunch of smaller documents! The source and tests for these two modules is programmed in the same style. I think storing functions in JSON makes CouchDB just a little bit like Smalltalk, with a much more familiar language.

Thanks for reading. Feedback welcome and appreciated.

Text

· · to read

I made a couple of small node.js modules to help me develop couchapps.

CouchApp is two things: a concept and a tool. The concept is an application that runs on CouchDB, using its API. The API has three basic components for presenting data (it has others for editing data but that isn’t covered in this post):

  • views: A view in CouchDB is a Map and optionally a Reduce function, which enables indexing JSON documents using the powerful MapReduce programming model.
  • lists: A list is a function that renders documents from a view, into an any format (commonly HTML).
  • shows: A show function renders a single document, into any format.

With these simple functions, a blog or a photo gallery can be made viewable. The URLs can get a bit awkward, but CouchDB provides a rewrites feature to solve it.

These functions go into a CouchDB design document. CouchDB is a JSON database, and a design document is a JSON document that contains functions as strings. A single design document, with attachments for CSS, which can be served directly to the web browser, can provide access to data without relying on client-side JavaScript (instead using server-side JavaScript).

The tool is a python command-line utility that maps a CouchDB design document to a directory structure. It can clone a design document directly from the database. It breaks it into a lot of little files, including a separate JavaScript file for each function.

This can be a lot of files, when there is one file for each function. Here is the CouchOne pages wiki CouchApp. It has a lot of views.

Mikeal Rogers had perhaps the first solution to this problem: node.couchapp.js. It runs on node.js and allows programmatic building of a design document, and puts functions in strings to prepare the JSON using Function.toString(). This loses the python CouchApp’s advantage of being able to go back and forth from file to CouchDB without losing anything.

I wanted something that could sync back and forth and is more like the original JSON than python’s CouchApp. To make it usable, I need functions shown across multiple lines with syntax highlighting. My hack was to convert the JavaScript to JSON. With CommonJS I can convert JSON to JavaScript easily, just by adding module.exports = to the front of the file. I can then replace the function strings with the direct value of the strings. To go from JavaScript to JSON I need to get the original function source. With JavaScript’s fn.toString(), where fn is a function, I get the source without the comments. I want the comments. It also isn’t indented properly. With Esprima I wrote code to extract the raw function source and fix up the indentation. Examples of this are in the READMEs of the two projects, json2js and js2json.

The projects below are still in early development and barely tested, but I’m putting them to use right now, by finally putting one of my fancy domain names to good use. ;) In the meantime, if this interests you, please check them out and give me feedback.

Repositoryjson2js
Ownerbenatkin
Repositoryjs2json
Ownerbenatkin
Text

· · to read

Since I love FireBug, and CouchDB has a JSON API, I attempted to combine the two. My goal was to find a way to use jQuery‘s AJAX functions to create, read, and update CouchDB documents.

Getting Started

The first thing I tried was creating a merb project with a single page referencing jQuery, and accessing CouchDB from there. The browser’s security model got in my way, however. A page on one port on localhost can’t access another port with AJAX. I searched for ways to give a page special permission, but my search turned up empty and I figured that I’d rather not go against the browser anyway, if I could help it.

The next thing I tried was getting a page that references jQuery to be served by CouchDB’s web server. That way the page and jQuery’s JSON API would be on the same port and Firefox would be happy. I opened up CouchDB’s Futon Utility Client at http://localhost:5984/_utils/ and found that it already had the latest version of jQuery included! Problem solved.

I opened the FireBug console (which was already enabled on localhost from when I was trying to get it working with my Merb project) and started trying out jQuery’s AJAX functions on CouchDB.

The GET request returned successfully! However, FireBug doesn’t retain the response, so I have to click Load Response to get it.

OK, there’s CouchDB’s welcome message! Expanding the GET request and clicking Load Response is tedious, though, so a better technique is needed. The first thing that comes to mind is doing a synchronous request. I find that get() doesn’t support this option, so I need to use ajax().

Aside from FireBug’s quoting FAIL, it works nicely. What I’d really like, though, is for FireBug to pretty-print the JSON. So I throw an eval() statement around it, and add the left side of an assignment statement because otherwise I would get an error (another way to get it working is to throw square brackets around the expression).

The output is quite a bit nicer, in my opinion.

Creating a Database

To create a database I use the HttpDatabaseApi page on the CouchDB wiki. Creating a database requires use of a PUT request, which can be made with jQuery’s ajax() function if the browser supports it (and Firefox does).

CouchDB returns a simple OK message to let me know it worked.

Creating a Document

Now that I have a database, I can create a document using the HttpDocumentApi. I opt to take the cavalier approach of having the server generate an ID rather than supplying an ID of my own. To do this, I send a POST with the database’s URL for the address and the document’s contents for the POST data. To send the POST data, I call ajax() with the document’s contents (as a JSON expression) for the data option and the string ‘json’ for the dataType option.

It didn’t work. I expand the POST and look at the data sent to find out why.

The ajax() function didn’t serialize the data! It’s not built into jQuery. It makes sense, given that one of jQuery’s features is its small footprint. I remember that when I viewed the source of the main page of CouchDB’s Futon Utility Client I saw a JSON include. It’s in http://localhost:5984/_utils/script/json2.js. It’s not a jQuery plug-in, but its interface is fairly simple. To serialize data, you call JSON.stringify() with an object as the first parameter. I add this to my statement and try again.

It worked! Or, at least, it didn’t error. It returned a globally unique ID (GUID) generated by the server. Now I use the ID to GET the document’s contents.

Only a summary of the object is shown. If I click the object it expands to a full view.

The data is there. The revision number is also present. One of CouchDB’s coolest features is built-in versioning.

After I realized that I was given only a summary of the object, I tried creating a document again and seeing if I missed any info that was supplied in the response to the POST request. It turns out that I did. The revision (_rev) was given. I’d like to have Firebug print out the whole response (and not a short summary), but I don’t see any way of doing so.

Updating a Document

Documents can be updated by sending a PUT request with the document’s URL as the address and the new contents as the data. The new contents must include the revision number upon which the update is based. This is to prevent conflicts. If a revision number other than that of the latest revision is supplied, it means that another client updated it first.

I update the document by getting the latest copy, storing it in a variable, removing the ID (since it’s already in the URL), changing the radius, and sending a PUT request with the new database contents.

The document is updated. If I expand the object I can see the new revision number.

Pretty cool, huh?

Conclusion & Next Steps

Before I tried using CouchDB with Firebug, I tried using the Futon Utility Client. I felt that I learned more and better retained what I learned when I used Firebug. Firebug as it is right now has a couple of issues that will keep me from using it as an environment in which to try doing different things with CouchDB. The first issue is that the pretty-print doesn’t show the full object in the console view, and there’s no option in the UI to make it do so. The second issue is that I find the editor to be insufficient. When the editor is in vertical mode, there’s no way to go through the history of commands entered. When the editor is in horizontal mode, it can’t expand beyond one line. When I switch between tabs, the text in the editor is often lost.

Firebug is, however, being actively worked on, and will hopefully get some features that will make it easier and more fun to use to play with JavaScript API’s and web services.

I would like to get the custom JavaScript environment mentioned at the top of this post set up and working with CouchDB. One way to get it working might be to set up a proxy of sorts to CouchDB. I could have any requests starting with /couchdb be forwarded to the CouchDB port with the /couchdb part stripped off.

This was fun. I think the next thing I’ll do with CouchDB is try writing a simple wiki using CouchDB and Merb.