dosync Archive Pages Categories Tags

In Stillness, Movement

07 April 2015

A while back I wrote about Google Closure Modules, and talked a little bit about code motion. A few days ago I tried dog-fooding the discussed ClojureScript :modules functionality on Mori, which exports ClojureScript's standard library for JavaScript usage. To my surprise attempting to split Mori into base and extras modules resulted in a fairly large base (~36K gzipped) and surprisingly small extras (~9K gzipped). Cross module code motion did not appear to be kicking in.

I examined Closure Compiler, specifically CrossModuleCodeMotion.java looking for clues. This lead me to the intuitively named method canMoveValue. The comment for this method reads:

private static boolean canMoveValue(
    ReferenceCollectingCallback collector, Scope scope, Node n) {
  // the value is only movable if it's
  // a) nothing,
  // b) a constant literal,
  // c) a function, or
  // d) an array/object literal of movable values.
  // e) a function stub generated by CrossModuleMethodMotion.
  if (n == null || NodeUtil.isLiteralValue(n, true) || 
      n.isFunction()) {
    // ...
  }
  // ...
}

Cross module code motion is very conservative, only obviously static values will be moved and c) immediately jumped out at me as a concern. For performance reasons multi-arity and variadic functions in ClojureScript are actually implemented as functions with direct methods attached as properties (see my previous post). But that's our problem right there! Constructing multi-arity and variadic function values requires an invoke and invokes aren't on the list of things that Google Closure will move.

Simply put, large chunks of the ClojureScript standard library were considered too dynamic to move by the Closure Compiler.

So after assessing a variety of ways to make the top-level more static, I ended up deciding that top-level fns should be treated as a special case and, to avoid further complications to the actual compiler, to implement the change via customizing the defn macro. Being able to do significant work at the macro level is definitely one of the big perks of Lisp - it will probably come as no surprise to the Lisperati that the ClojureScript macros file is bigger than the compiler file!

The actual changeset is pretty technical and of interest mostly to experienced Clojure devs and ClojureScript compiler hackers, you can see it here.

After this change the end result for Mori is a much more reasonable split, ~27K gzipped for the base module and ~18K gzipped for the extras. We can likely further improve this, but already a significant amount of ClojureScript source is now movable that previously wasn't.

For those of you unlucky souls that have to deal with JavaScript build tools this was all the configuration required to get the optimized Mori split:

{:cljs-base {:entries #{cljs.core mori}
             :output-to "release/build/mori.base.js"}
 :mutable   {:entries #{mori.mutable}
             :output-to "release/build/mori.mutable.js"}
 :extra     {:entries #{clojure.data cljs.reader clojure.set mori.extra}
             :output-to "release/build/mori.extra.js"}}

You can experiment with :modules and the enhanced code motion by using the 0.0-3178 pre-release.