TODO list for QuasiScript Core

TODO

1. Refactor from one file into several: tokenize, parse, compile, repl, etc.  --  done 
2. Improve error reporting --  done 
3. REPL --  done 
4. Complete special forms for core language
5. Command-line
6. Some attractive syntactic sugar
7. *Macros (preferably with quasiquotation notation)
8. Internal documentation
9. Readme for github
10. Packaging
11. Include a license

*This is the challenging one.

LATER

1. Doc strings
2. Pre-conditions, post-conditions, etc.
3. [Tools for] Beautiful documentation (per CoffeeScript)
4. Basic libraries (c.f. Python's "batteries included")

Adventures in special forms

As I start to flesh out QuasiScript, I am making choices about the initial set of special forms to choose.  Special forms are the basic structures (similar to the keywords in an infix language like C, Pascal, etc.) used to define the core of the language.  Other structures can be built up as macros (once I have them!) and functions.

Initially I chose special forms close to Scheme, or Arc, or CoffeeScript, but as I progress I am leaning towards making the core much more like "JavaScript with a lispy coating".

E.g. What started as "(def x 5)" now reads as "(var x 5)", and translates to js as "var x = 5;".  This lowers the cognitive switching between js and qs. 

Similarly, at first I wanted everything to be an expression (i.e. return a value), but this does not make for particularly idiomatic javascript output, at least not without adding a smart phase to the compiler to eliminate redundant closures.  So I'm taking more of an 80/20 approach to keep things idiomatic, but eliminate common returns.  E.g. "(fun (x) (* 2 x))" -> "function (x) { return 2*x; }".  Of course, something like arc's "[* 2 _]" for the same expression would be even nicer, but that's sugar for later.

Later, with macros, it should be possible to build from the core up to a more powerful language, with plenty of good stuff out-of-the-box.

Starting from scratch

I've completed a more than good-enough tokenizer to be going on with, and in short order.

The beauty of starting from scratch -- re-inventing the wheel, if you like -- is that you learn.  Even if you subsequently pick up a more mature component, you'll be in a much better position to evaluate, and where necessary, debug.  A good analogy is to the experience of driving a car if you've already built or re-built one.  Your mental model is deeper than a regular driver [user].

The challenge is the open-endedness of the task, all the options and decisions.  This is where approaches like test-driven-design/development and (used) design-by-contract (not yet, ran into some Node-related problems) help a lot.  They do a few things: you are obliged to make explicit your assumptions, and the computer tests them for you.  Inevitably, there are mistakes.  Better to fail fast and make steady progress than have the illusion of progress, but in fact be digging yourself into a deeper and deeper tar pit.

The answer to choice paralysis is to proceed, but use TDD or DBC or some-such to rigorously test out your choices, and allow yourself to go back and change.  An over-arching objective of simplicity of design, layering a la SICP, and the discipline to keep refactoring as you go helps too.

Getting started

QuasiScript is my personal open source project to write a Lisp-to-JavaScript compiler.

It's a passion project, with an aim to learn and explore nooks and crannies along the way.

Why?
JavaScript is ubiquitous, and really should have been "Scheme in the browser", and now CoffeeScript has shown how to fix the JavaScript quirks, but stops at a Ruby/Python syntax rather than going all the way to Lisp. 

Also, with the rise of node.js, server-side and network programming in and on JavaScript just got a whole lot more interesting.

Influences
I started off by trying to contribute to Jacob Rothstein's Sibilant project, which already does an impressive job, but quickly founded that I needed to learn more of the basics, so here I am!

Warming up
I've started by porting Peter Norvig's lis.py Scheme interpreter to JavaScript, specifically node.js, with nodeunit for tests (invaluable) and npm for package management.  But an interpreter is only an interpreter.

Other influences are

• Lisps: Scheme, arc, clojure
• Role models: CoffeeScript, Sibilant
• Also: Eiffel (use of contracts to get correct code, fast)

Getting stuck in
Abdulaziz Ghuloum has written an excellent paper, An Incremental Approach to Compiler Construction [pdf], which outlines the kind of approach I will try to follow: essentially a test-driven approach.  Ghuloum goes from Scheme to x86 assembly with Scheme as the implementation language.

I'll be bootstrapping out of JavaScript, and start with a Scheme-like language, but it will change along the way (although making the core language forms "skinnable" is an option).

For incremental steps, I intend to start by trying to follow CoffeeScript's capabilities, and its readable generated JavaScript.