Text

· · to read

Here’s a copy of the email I sent to the CouchDB users mailing list:

I made a couple of npm (node.js) modules for editing CouchDB design documents that involves fewer files than python CouchApp, but like python CouchApp supports two-way sync. The function source is left intact when converting between JavaScript source and JSON. The JavaScript source version just shows it in an embedded function expression, which makes it syntax highlightable.

http://benatkin.com/2012/09/04/js2json-and-json2js/
https://github.com/benatkin/js2json
https://github.com/benatkin/json2js

Here’s a quick example. If I stick this in books.js:

module.exports = {
  "views": {
    "author": {
      "map": function(doc) {
        if (doc.type == 'book')
          emit(doc.author, null);
      }
    },
    "title": {
      "map": function(doc) {
        if (doc.type == 'book')
          emit(doc.title, null);
      }
    }
  }
}

…and run this (after npm install json2js js2json):

var js2json = require('js2json');
var json2js = require('json2js');
var fs = require('fs');

var jsSource = fs.readFileSync('books.js', 'utf8');
var jsonValue = js2json.convert(jsSource);
fs.writeFileSync('books.json', JSON.stringify(jsonValue, null, 2) + "\n");
var jsSourceFromJson = json2js.convert(jsonValue);
fs.writeFileSync('books-from-json.js', jsSourceFromJson + "\n");

…I get the following in books.json:

{
  "views": {
    "author": {
      "map": "function(doc) {\n  if (doc.type == 'book')\n
emit(doc.author, null);\n}"
    },
    "title": {
      "map": "function(doc) {\n  if (doc.type == 'book')\n
emit(doc.title, null);\n}"
    }
  }
}

…and books-from-json.js is exactly the same as books.js.

I explain it more in my blog post (linked at the top of this message). I need to add a cli tool that syncs, a way to handle attachments, and a way to handle embedded multiline strings for it to be a full-featured design doc editor. I have much bigger plans for this, though: I want to break up CouchApps into a bunch of smaller documents! The source and tests for these two modules is programmed in the same style. I think storing functions in JSON makes CouchDB just a little bit like Smalltalk, with a much more familiar language.

Thanks for reading. Feedback welcome and appreciated.

Text

· · to read

I made a couple of small node.js modules to help me develop couchapps.

CouchApp is two things: a concept and a tool. The concept is an application that runs on CouchDB, using its API. The API has three basic components for presenting data (it has others for editing data but that isn’t covered in this post):

  • views: A view in CouchDB is a Map and optionally a Reduce function, which enables indexing JSON documents using the powerful MapReduce programming model.
  • lists: A list is a function that renders documents from a view, into an any format (commonly HTML).
  • shows: A show function renders a single document, into any format.

With these simple functions, a blog or a photo gallery can be made viewable. The URLs can get a bit awkward, but CouchDB provides a rewrites feature to solve it.

These functions go into a CouchDB design document. CouchDB is a JSON database, and a design document is a JSON document that contains functions as strings. A single design document, with attachments for CSS, which can be served directly to the web browser, can provide access to data without relying on client-side JavaScript (instead using server-side JavaScript).

The tool is a python command-line utility that maps a CouchDB design document to a directory structure. It can clone a design document directly from the database. It breaks it into a lot of little files, including a separate JavaScript file for each function.

This can be a lot of files, when there is one file for each function. Here is the CouchOne pages wiki CouchApp. It has a lot of views.

Mikeal Rogers had perhaps the first solution to this problem: node.couchapp.js. It runs on node.js and allows programmatic building of a design document, and puts functions in strings to prepare the JSON using Function.toString(). This loses the python CouchApp’s advantage of being able to go back and forth from file to CouchDB without losing anything.

I wanted something that could sync back and forth and is more like the original JSON than python’s CouchApp. To make it usable, I need functions shown across multiple lines with syntax highlighting. My hack was to convert the JavaScript to JSON. With CommonJS I can convert JSON to JavaScript easily, just by adding module.exports = to the front of the file. I can then replace the function strings with the direct value of the strings. To go from JavaScript to JSON I need to get the original function source. With JavaScript’s fn.toString(), where fn is a function, I get the source without the comments. I want the comments. It also isn’t indented properly. With Esprima I wrote code to extract the raw function source and fix up the indentation. Examples of this are in the READMEs of the two projects, json2js and js2json.

The projects below are still in early development and barely tested, but I’m putting them to use right now, by finally putting one of my fancy domain names to good use. ;) In the meantime, if this interests you, please check them out and give me feedback.

Repositoryjson2js
Ownerbenatkin
Repositoryjs2json
Ownerbenatkin
Text

· · to read

Connect has a session middleware that has a pluggable API for session storage. There is a session store for redis, written by TJ Holowaychuk, who maintains both connect and express. There are also session stores for CouchDB, MongoDB, and postgresql that look to be well-maintained and ready for production use.

These are great, but I wanted to store my session data in cookies, because the amount of session data I plan to use is tiny, and because my app is designed to handle high-latency CouchDB database connections gracefully.

I had a hard time finding a session store that stores session data in the cookies. The session middleware uses cookies, but it uses them to store the keys to access the session data, not the session data itself. I found an example but no actively maintained session store for cookies.

After some more searching, I found that the way to store sessions in cookies is to use a whole different middleware that comes with connect! It’s called cookieSession. To use it all I have to do is add this code snippet, and ensure that I have session_secret set in my app settings:

app.use(connect.cookieParser(app.set('session_secret')));
app.use(connect.cookieSession());

When using cookie sessions it’s important that the cookie data is small and that the cookie is signed using a session secret, to prevent session fixation. This is documented in the excellent Ruby On Rails Security Guide. Even if you aren’t using RoR I recommend reading it.

Text

· · to read

A function, all by itself:

module.exports = function(string) {
  return string.replace(/&(?!\w+;|#\d+;|#x[\da-f]+;)/gi, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;').replace(/"/g, '&quot;').replace(/'/g, '&#x27;').replace(/\//g,'&#x2F;');
};

This is the escapeHTML() function from Backbone.js, which is hidden behind a closure. Thankfully it’s now in the latest version of Underscore.js, which is depended upon by Backbone. If an old version of underscore was being used and escapeHTML() was the one missing utility function that a developer needed, putting it in a module by itself would be one way of supplying it.

Side note: Backbone and Underscore escape the exact six characters specified in Rule #1 of the Open Source Web Application Security Project (OWASP)’s XSS (Cross Site Scripting) Prevention Cheat Sheet.