Links: Parsing in Ruby and JavaScript

I’ve been reading quite a bit about parsing and templating in ruby as I attempt to port a templating engine from JavaScript to Ruby. Here are some scattered links:

Jison

Jison is a parser generator for JavaScript that has separate lex and bnf definitions. It’s used by Handlebars.js.

Repositoryjison
Ownerzaach

Treetop

Treetop is a parser generator for Ruby. It’s installed with Ruby On Rails, through the mail gem which is installed by ActionMailer.

Repositorytreetop

Citrus

Citrus is another promising parsing gem for Ruby. It seems to be very easy to get started with, and I like many of the design decisions.

Repositorycitrus

Temple

Temple is a templating-specific library that helps with a lot of the AST transformation. It doesn’t seem to have a CFG syntax so it seems that using treetop or citrus would make sense for complex grammars. Otherwise, strscan could be used.

Repositorytemple
Ownerjudofyr

temple-mustache

This is an implementation of a mustache renderer in . It uses strscan to generate the initial parse tree, and Temple to generate the ruby code. It supports mustache sections.

Repositorytemple-mustache
Ownerminad

Slim

Slim is a Haml-like templating library for Ruby that’s used in production by many. It uses Temple, with a line-based parser, which works well because it uses significant indentation for nesting.

Repositoryslim
Ownerstonean

Deleting from Chrome’s History

I wrote a script to delete URLs containing a string from Chrome’s history. It works for me. I don’t understand everything that it does, though, so it may have serious flaws. Use at your own risk.

# Removing history entries from Chrome that contain a search phrase
# Exit out of Chrome first
# Back up files and do this at your own risk

# gem install sqlite3 && gem install sequel

require 'sequel'

search_string = 'reddit'

# Delete history cache files
path = File.expand_path('~/Library/Application Support/Google/Chrome/Default')
cache_dirs = Dir.entries(path).select {|dir| dir != 'History' && dir.index('History') == 0}.map {|dir| File.join path, dir}
cache_dirs.each {|dir| File.delete dir}

# Delete matching history from sqlite3
DB = Sequel.sqlite File.join(path, 'History')
matching_urls = DB[:urls].filter(:url.like("%#{search_string}%"))
puts %Q[Deleting #{matching_urls.count} urls matching "#{search_string}"]
matching_urls.delete

I really like sqlite and Sequel.

Resources for Learning Rails

Here is a list of a few awesome sites offering tools for learning Rails:

PeepCode

Pricing: Starts at $12 for one screencast; Twitter: @peepcode and @topfunky

I bought my first PeepCode screencast back in 2008, before Rails and Merb merged. It’s been around for a while, but Geoffrey Grosenbach does a great job of keeping it up-to-date. There are now a number of screencasts about current topics of interest to the Ruby on Rails community, including Rails 3, Backbone.js, PostgreSQL, CoffeeScript, and John Barnette.

Railscasts

Pricing: Free weekly episodes, additional weekly episodes and revised episodes for $9/mo (RailsCasts Pro); Twitter: @railscasts

Railscasts is great for keeping up on Ruby On Rails, and because it’s free, it’s also great for recommending to people who are curious about Rails but not ready to spend money to help them learn it. Ryan Bates is great at explaining things. He covers a wide variety of topics in his screencasts and presents them in a nice format with code snippets.

Update: since I posted this Ryan Bates released Railscasts Pro, which is fantastic!

Ruby on Rails Tutorial

Pricing: $26 (book), $85 (screencasts), $95 (both); Twitter: @railstutorial

Michael Hartl is a physicist who does a number of other things (see his about page) including Web Development with Ruby On Rails. I’ve watched all of his screencasts; they’re fantastic. He builds a web application TDD-style and teaches many different concepts including MVC and how TDD can help with authorization (which is trickier than authentication IMO).

Rails Apps

Pricing: Free; Twitter: @rails_apps

I always thought that the Rails Starter App templates were cool, but they were missing something: in-depth tutorials and a comparison between the many different templates and quick-start tools. Daniel Kehoe has provided both and many more useful resources in his Rails Apps GitHub account.

Code School

Pricing: Free (one substantial free product), $45-55 (single paid products); Twitter: @codeschool

Code School is a set of tutorials designed to help developers to quickly get up to speed building web applications. Gregg Pollack and Envy Labs have been doing podcasts and screencasts for quite some time, and it’s not surprising that they still do it, because they’re very enthusiastic in front of a camera and a microphone.

the one-line sinatra app

The five-line sinatra app on http://www.sinatrarb.com/ is pretty damn impressive:

require 'rubygems'
require 'sinatra'
get '/hi' do
  "Hello World!"
end

I can do four lines better than that, though:

require 'sinatra'

So what does that get you?

batkin:cholla ben$ ruby -rubygems cholla.rb
== Sinatra/0.9.0.4 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.0.0 codename That's What She Said)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop
127.0.0.1 - - [03/May/2009 10:13:46] "GET /index.html HTTP/1.1" 200 7 0.0019

A basic http file server, perfect for working with plain html, javascript, and CSS! Just create a public/ directory alongside the one-line sinatra app, and have at it! If you create an index.html file in public/, it will be served up when you go to http://localhost:4567/. Try it!

Once you decide to add some server-side code, you can simply go in to the sinatra app and add it.

Practice

I just watched Corey Haines’ lightning talk from this year’s acts_as_conference:

He makes a lot of really good points in a short amount of time. One of the things he says is that if we have to look at documentation for a technique, we’re unlikely to use that technique when we’re in a time crunch. I think this is a very good point. Another time when I might not use the documentation is when I’m just dabbling.

To learn things, he suggests doing arbitrary tasks repetitively to get them engrained into our minds.

I spent a number of hours doing arbitrary tasks with vim, and I’ve gotten pretty good at using vim IMHO. I wish I could say I was as good at jQuery, but I haven’t got it down as well despite doing real projects in it. I think perhaps it’s best that I pick an arbitrary task, or at least do a real, but small, task repetitively, or keep trying to improve it, until I start to really get the hang of it.

On a side note, Corey Haines’ has quit working a regular job and started traveling around and pair programming with people in exchange for room and board. He calls himself a Software Journeyman. It will be interesting to hear what he learns from the experience. I hope his journey takes him to Arizona at some point!

Injecting a gem

When installing Merb on my laptop, which runs Ubuntu Linux, I ran into a compiler error with hpricot, one of Merb’s dependencies. Before bothering to see what might be causing it, I checked to see if there was a precompiled apt-get package available. There was! I installed it and tried installing Merb again.

It still tried to install hpricot, however, and when I said to skip it, it said it could not install Merb because it was missing hpricot. I could have used force, or “pretty please”, but I wanted to instead tell the packaging system that I have hpricot installed.

I seem to remember a UNIX packaging system having an inject command — which tells the packaging system a program is already installed. I may have got the terminology wrong. I searched for such a command for rubygems but couldn’t find one. I did, however, find another way to convince rubygems that I already had hpricot.

I decided to do some digging in the gems directory. To find the gems directory, I typed:

ben@magicthise:~$ gem environment
RubyGems Environment:
  - VERSION: 0.9.4 (0.9.4)
  - INSTALLATION DIRECTORY: /var/lib/gems/1.8
  - GEM PATH:
     - /var/lib/gems/1.8
  - REMOTE SOURCES:
     - http://gems.rubyforge.org
ben@magicthise:~$ cd /var/lib/gems/1.8/
ben@magicthise:/var/lib/gems/1.8$ ls
bin  cache  doc  gems  source_cache  specifications
ben@magicthise:/var/lib/gems/1.8$ cd specifications/
ben@magicthise:/var/lib/gems/1.8/specifications$ ls
abstract-1.0.0.gemspec          merb-cache-0.9.2.gemspec
actionmailer-2.0.2.gemspec      merb-core-0.9.2.gemspec
...
ben@magicthise:/var/lib/gems/1.8/specifications$

I took a look at the gemspec files and found that they contained the name and version of the library. I copied the smallest of the files to hpricot-0.6.gemspec and used the gemspec documentation to find out which parameters are required. I wound up with the following in /var/lib/gems/1.8/specifications/hpricot-1.6.rb.

Gem::Specification.new do |s|
  s.name = %q{hpricot}
  s.version = "0.6"
end

I tried running:

ben@magicthise:~$ gem install merb --include-dependencies

And it worked!

It might not be everything that’s needed to get it working — but now I at least know how to inject a package.

Ruby AST ramblings…

I just read an article calling for a way to get an AST for Ruby code from within Ruby. I think it’s a great idea, and I think static analysis, code manipulation/refactoring are great reasons to support such a thing. Another consequence would be macros, which are a more controversial feature. The only downside I can think of is that it could be misused, and to leave out a feature for that reason would go against the Ruby philosophy, in my opinion. One of the regulars at the Phoenix Ruby Users’ Group argued against using a dumbed-down “teaching language”, saying that when you suggest using a language you don’t want to use yourself, you’re talking down to the learner. I agree. I think the best way to keep people from misusing features is not to remove the features, but to educate them. People in the Ruby community have always done a good job of making sure there’s lots of good example code out there, and giving constructive criticism when someone posts a bad example.

While the reasons in the article are enough, I can think of another good reason for having AST support — having the ability to constrain code to a certain set of features. There would be two different uses for this:

  • Running code from an untrusted source – This could include web template designers, or even users. Ning is an example of a site that lets users run their own code — but there is a huge overhead to facilitate this and to sandbox everything. If it could be verified that code doesn’t do anything dangerous, I think a new type of Web app plug-ins would emerge. Instead of having to set up an app on a separate site and use REST or SOAP to communicate, people could just throw together a little script in a domain-specific language. One idea I have for this sort of technology would be a Flickr for html and image generators (like ajaxload.info).
  • Enforcing decoupling – When managing a large software project, I think it would nice to specify which classes do what, and have it enforced. Maybe there is some class that’s supposed to be all about math, but that goes in an application that gives output to users. To keep math programmers from putting presentation logic in mathematical code, you could constrain them to a DSL that doesn’t have strings. Or, you could keep template programmers from doing networking code. The check could be done at runtime, or commit time, with plug-ins to the version control system.

There is a Perl library that parses Perl, but it would be nice to have one for Ruby that’s written in C and optimized for speed, and that with certainty matches up to what the interpreter understands.

I think that once an AST implementation was built, something like RSpec could be created which does compile-time or commit-time analysis. That would be cool.