(ns com.deskchecked.utils
  (:use clojure.contrib.str-utils)
  (:import [java.security MessageDigest]))

(defn sha
  "Generates a SHA-256 hash of the given input plaintext."
  [input]
  (let [md (MessageDigest/getInstance "SHA-256")]
    (. md update (.getBytes input))
    (let [digest (.digest md)]
      (str-join "" (map #(Integer/toHexString (bit-and % 0xff)) digest)))))

There’s obviously a dependency on clojure-contrib here which you can do away with if you don’t need it. And of course, you can pick a hashing algorithm to suit your needs.

Nothing exciting, but I figure I can’t be the only person in need of this sort of stuff.

Since part 1 was published, Scala 2.7 has been released which — among other things — introduced changes to the parser combinator library. Changes that meant the source code from part 1 will not compile in Scala 2.7. Sorry about that. Updated, working code can be found at the end of this post.

So what exactly has changed? Well, let’s see …

keyword() no longer discards its result

A bit of a refresher first:

In both Scala 2.6 and 2.7, the keyword implicit is called whenever you use a string in your “grammar rules”. For example:

def sum = expr ~ "+" ~ expr

Is the equivalent of:

def sum = expr.~(keyword("+").expr)

In Scala 2.6’s parser combinator library, the result of the keyword() call was actually the UnitParser — that is, a parser that would discard its result. At the time, that meant we could use the tilde operator (”~”) to create a sequenced parser and everything was fine:

def sum = expr ~ "+" ~ expr ^^
    ((left : Expr, right : Expr) => Sum(left, right))

In 2.7, the keyword() implicit returns a Parser[String] rather than a UnitParser. This means we have to either deal with the newly introduced tokens as follows:

def sum = expr ~ "+" ~ expr ^^
    ((left : Expr, op : String, right : Expr) => Sum(left, right))

Or alternatively …

Use <~ and ~> to indicate the important parts of a parse rule

Let’s start with something simple based on the 2.6 combinators:

def bracketExpr = "(" ~ expr ~ ")" ^^ ((e : Expr) => e)

Again, in 2.7 we know that keyword() is no longer a UnitParser, so we have to deal with it like so:

def bracketExpr = "(" ~ expr ~ ")" ^^
    ((l : String, e : Expr, r : String) => e)

Alright, this compiles and does what we expect. But why should we keep those strings around if we don’t need them? They sure do clutter up the code a whole bunch.

<~ and ~> can be used to include or discard the result of a given parser. <~ builds a parser that takes the result of two parsers (the parsers to its left and right) and builds a parser that takes the result of both and discards the result of the one on the right. Conversely, ~> yields a parser that takes the result of both parsers and discards the result on the left.

So, using these two operators we can rewrite bracketExpr as follows:

def bracketExpr = "(" ~> expr <~ ")" ^^
    ((e : Expr) => e)

Ah. Much better.

The ^^^ operator

I had to go digging in the Scala source code to work out what exactly this one does.

First, let’s take a look at some 2.6 code:

def simpleExpr = term * (
    "+" ^^ ((x : Expr, y : Expr) => Add(x, y)) |
    "-" ^^ ((x : Expr, y : Expr) => Sub(x, y))
)

In 2.6, this can parse zero or more repetitions of “term” interleaved by “+” and “-” (check out part 1 if you need a refresher on how the * combinator works). In 2.7 it’s a compile-time error because of the fact keyword() is no longer a UnitParser (are you seeing a pattern here? ), and ^^ is trying to pass the resulting String on to the anonymous method in each case.

If we use the ^^^ operator here, we can effectively discard the result of the keyword parse, and build a parser that uses a simple anonymous method to parse the current pair of terms:

def simpleExpr = term * (
    "+" ^^^ ((x : Expr, y : Expr) => Add(x, y)) |
    "-" ^^^ ((x : Expr, y : Expr) => Sub(x, y))
)

Exactly what we’re after. This compiles and behaves as expected.

What else?

There may be more changes to the parser combinator library which I haven’t covered here, but I’m not going to go looking for any more changes since the updated code seems to work just fine. This should be enough to at least understand the updated code for the compiler described in part 1 without needing to deal with any cryptic compiler errors.

Finally, the new code!

Special thanks to Harshad for sending through working code for 2.7 ages ago which I never got around to posting here. This code, along with the Scala API docs, was used to figure out just what had changed since 2.6. The code below is derived from some code he sent to me a few months back.

I’m really sorry this has been so long in the making. I’ll try to get around to writing the “real” part two of this article. In the meantime, here’s the updated code. Thanks! Please post or email any comments or questions.

An Introduction to List Initialization and List.iter

Here’s the source code we’re going to be breaking down:

let main () =
    let seq = ["a"; "b"; "c"] in
    List.iter print_endline seq ;;

let _ = main ()

Looks scary, doesn’t it? It’s actually quite simple, but we’ll get to the source code in a moment. First, let’s see the program’s output. Save this file as tmhoo02.ml and run it like so:

$ ocaml tmhoo02.ml

You should see the following:

a
b
c

Now, let’s take a look at the interesting bits of the source code:

let seq = ["a"; "b"; "c"] in

As you (hopefully!) learned in part 1, the let syntax binds the variable seq to some expression. The in keyword indicates that this binding is a named local expression whose value to be discarded when the function exits (think “local variable”). But what exactly does that expression mean?

["a"; "b"; "c"]

This is the most direct way to initialize a list in Ocaml. Each element in a list is separated by a semi-colon. This particular list has three elements: the strings “a”, “b” and “c”. As our list comprises of string elements, we can say that this is a string list. Note that unlike some dynamically typed languages (e.g. Python and Ruby), every Ocaml list element must be of the same type: you have a list of string elements or a list of integer elements, but you simply can’t mix the two. Honestly, it won’t compile.

List.iter print_endline seq ;;

Here we’re calling the iter function of the List module – a module which is bundled as part of the Ocaml standard library. This function takes two parameters: the first is another function that accepts a single parameter, the second is a list.

As you might suspect, List.iter iterates over the list, passing each element in the list to the function one by one. To word it another way, inside List.iter we’re calling print_endline for each element in our list to the equivalent of: print_endline “a”, print_endline “b”, print_endline “c”. This explains the output of our example program.

Finally:

let _ = main ()

main is evaluated here and its return value (the <em>unit</em> value) is ignored by the underscore wildcard. You’ll notice in part 1 I omitted everything up to the equals operator but, at the behest of others who know better, I’ve included it here in part two. I’m not going to say anything more about it, other than the fact that the underscore wildcard is a simple but powerful construct you’ll likely become very familiar with at a later date. If you’d prefer to use the syntax from part 1, that should work too (but don’t come crawling to me when other Ocaml programmers come down on you like a ton of bricks ).

Advanced Discussion of the Type System

As a sneak preview into Ocaml’s type system, I can tell you that List.iter isn’t restricted to string lists. In fact, it can work with lists containing elements of any type. However, because each element of the list is passed to the function given to List.iter as the first parameter, that function must accept a parameter the same type as a single element in the list. Since print_endline accepts a string, we must use a string list. Or: since we’re passing in a string list as the second parameter to List.iter, the function passed as the first parameter must itself accept a string as its sole parameter.

Confused? Read that last paragraph again carefully. If you still don’t understand it, don’t worry too much: we’ll cover Ocaml’s type system more in the next few tutorials.

Until Next Time …

So that wraps up tutorial number two. I’ve already got parts three and four in the pipeline and I can tell you they introduce a fair bit of new material. Take your time trying to understand what was covered in this tutorial and post a comment or two if you have any suggestions, questions or improvements.

I’m sure there’s room for improvement, so I’ve uploaded a tarball of the bzr repository. You can get it here: Thumper v0.1. Any suggestions/patches?

Note: This is not the next part of my series of Ocaml tutorials. That’s likely to come after I’ve finished moving.