Code layout of Fan (a metaprogramming tool for OCaml)

Fan is a metaprogramming tool for OCaml. I believe it would be an
invaluable tool for the community when it’s production ready.

If you are attracted by the power of Camlp4 while frustrated with the
complexity, or slowness, I would be very glad that you could join and contribute.

It’s already ready for accepting external contribution, since the
development of Fan would be pinned to OCaml 4.01.0 for a while.

1 Code layout

The source is scattered in four directories
common, treeparser, src, cold, and unitest

In the common directory, all sources are written in vanilla OCaml,
it defines basic primitives which dumps Fan’s abstract syntax into
OCaml’s abstract syntax.

Note that all compiler related modules is isolated in this
directory. That said, in the future, if we would like to support
different patched compiler, for example, metaocaml or Lexifi’s mlfi
compiler, only this directory needs to be touched.

In the treeparser directory, it is also written in vanilla Ocaml,
it defines the runtime of the parsing structure.

The src directory is written in Fan’s syntax.
Keep in mind that Fan’s syntax is essentially the same as OCaml’s
syntax, except that it allows quotations, and other tiny
differences (parens around tuple is necessary, not optional).

The cold directory is a mirror of src directory in vanilla
, though it is much verbose compared with src.

So when you get the source tree, the initial build is typically

ocamlbuild cold/fan.native

The first command ocamlbuild cold/fan.native would build a binary
composed of modules from common, treeparser and cold. Since
they are all written in vanilla ocaml, no preprocessors are
needed for the initial compilation.

If you are a third party user, that’s pretty much all you need to
know. As a developer of Fan, the next command is

./re cold fan.native

This shell script would symlink _build/cold/fan.native to
_build/boot/fan, which would be used by compiling src.

Now you can compile the src directory

ocamlbuild src/fan.native

The command ocamlbuild cold/fan.native would build a binary
composed of modules from common, treeparser and cold.

Note that at this time src is a mirror of cold, after preprocessored
by fan, the produced binary src/fan.native should be the same as

Now you can

./re src fan.native

But this does not make much sense since src/fan.native is exactly
the same as ./cold/fan.native.

Okay, in most cases, your development would be in such directories:
common, treeparser, src and unitest.

If you only touch common, treeparser or unitest, commit the changes
and send me a pull request.

If you also touch the files in src directory, you should mirror
those changes back to cold directory. Here we go:


Yes, now all the changes in src will be mirrored back to cold
For a simple change, commit and done. For a complex change that you
are not sure whether it would break anything or not, try to run:

./hb fan.native

The command above would first build src/fan.native using the
current preprocessor _build/boot/fan.

When it’s done, it would first remove the directories _build/src,
and _build/common, _build/treeparser. Then it would set
_build/boot/fan to be the new build preprocessor src/fan.native.
After that it would call ocamlbuild src/fan.native to build a new
preprocessor based on the existing preprocessor.

Then it would compare the two preprocessors, if they are exactly the
same, it means we manage to have a successful bootstrap. There is a
large chance that your change is correct.


make test

If everything goes well, it’s safe to commit now.

When the bootstrap fails, generally two cases: 1. the comparison
does not tell you the two preprocessors are the same, the normal
workflow is to repeat the command ./hb fan.native again. 2. It
fails to compile, since you always have cold/fan.native compiled,
fall back to such preprocessor and see where you made wrong.