A Groovy Light Table client - Step 2: Evaluating Code

Background

This is the second post in my series "A Groovy Light Table client". A blog series about steps I take when trying to build a Groovy plugin for Light Table.

In this post I will take you through some of the steps I went through to get Light Table to evaluate groovy (script) code and show results inline in the editor.

How did we get here ?

Evaluate contents of editor (cmd/ctrl + shift + enter)

(behavior ::on-eval
          :desc "Groovy: Eval current editor"
          :triggers #{:eval}
          :reaction (fn [editor]
                      (object/raise groovy :eval! {:origin editor
                                                   :info (assoc (@editor :info)
                                                           :code (ed/->val editor)
                                                           :meta {:start 0, :end (ed/last-line editor)})})))

This behavior triggers on ":eval", which is triggered to any editor (on cmd/ctrl + shift + enter in default key mapping). We just get hold of the text from the editor and gather some meta info and trigger a ":eval!" behavior on the groovy "mother" object defined in the previous blog post.

Evaluate current line/selection

(behavior ::on-eval.one
          :desc "Groovy: Eval current selection"
          :triggers #{:eval.one}
          :reaction (fn [editor]
                      (let [pos (ed/->cursor editor)
                            info (conj (:info @editor)
                                  (if (ed/selection? editor)
                                    {:code (ed/selection editor) :meta {:start (-> (ed/->cursor editor "start") :line)
                                                                        :end (-> (ed/->cursor editor "end") :line)}}
                                    {:pos pos :code (ed/line editor (:line pos)) :meta {:start (:line pos) :end (:line pos)}}))]
                        (object/raise groovy :eval! {:origin editor :info info}))))

The only difference here is that we gather the code for the current line or current selection. Then we trigger the same behavior as for evaluating the whole editor.

Our groovy Eval!

(behavior ::eval!
          :triggers #{:eval!}
          :reaction (fn [this event]
                      (let [{:keys [info origin]} event
                            client (-> @origin :client :default)]
                        (notifos/working "Evaluating groovy...")
                        (clients/send (eval/get-client! {:command :editor.eval.groovy
                                                         :origin origin
                                                         :info info
                                                         :create try-connect})
                                      :editor.eval.groovy info
                                      :only origin))))

This behavior is what actually sends off a eval request to the groovy client. Quite a lot happens under the hood (by help of inbuilt LightTable behaviors):

It tries to find a client (connection) for the editor If no connection exists it will try to create a new one. On create it will invoke the try-connect function that we defined for the gui connect/connect bar behavior in the previous blog post Once connected it will jsonify our parameters and send them off to our groovy client

The JSON might look something like:

[89,
"editor.eval.groovy",
  {"line-ending":"\n",
   "name":"sample.groovy",
   "type-name":"Groovy",
   "path":"/Users/mrundberget/Library/Application Support/LightTable/plugins/Groovy/sample.groovy",
   "mime":"text/x-groovy",
   "tags":["editor.groovy"],
   "code":"println \"hello\"",
   "meta":{"start":22,"end":22}}]

Notes

The first param is the client id for the editor that triggered the behavior. This client Id doesn’t represent the same as a connection id (ref previous blog post). Many editors may share the same connection !
The second param is the command (our groovy client will of course support many different commands, this is one of them)
The third and last parameter is our info. The code is the essential bit, but some of the meta information, like line info comes in handy when handling the response later on

The actual groovy evaluation

Command dispatch

ltClient.withStreams { input, output ->
  try {
    input.eachLine { line ->
    def (currentClientId, command, data) = new JsonSlurper().parseText(line)
    switch (command) {
    case "client.close":
      stop()
      break
    case "editor.eval.groovy":
      evalGroovy(data, currentClientId)
      break
   default:
     log "Invalid command: $command"
  }
  // ...

We parse any lines received from Light Table and based on the command received invokes the appropriate handler. In this case evalGroovy.

Eval groovy

private void evalGroovy(data, currentClientId) {
  def evalResult = scriptExecutor.execute(data.code)

  def resultParams = [meta: data.meta]
  if (evalResult.out) {
    resultParams << [out: evalResult.out]
  }
  if(evalResult.exprValues) {
    resultParams << [result: convertToClientVals(evalResult.exprValues)]
  }

  if (!evalResult.err) {
    data = [currentClientId?.toInteger(), "groovy.res", resultParams]
  } else {
    data = [currentClientId?.toInteger(), "groovy.err", [ex: evalResult.err] + resultParams]
  }
  sendData data
}

The first and most significant line is where we evaluate the groovy code received. This post would be too long if we went into all the details of what it does, but here’s a high-level summary:

We basically create a GroovyShell and compile our code to a script. Normally that would just compile a Script class. However we wish to collect a lot more information than you typically would get from default groovy script execution. So we do an AST transformation on the script class and add a custom abstract script class as a base class for the compiled script class. This allows us to inject behavior and wrap statement execution (all compiled into the script for optimal performance). That way we are able to collect information about values for most types of statements. We collect line number and value (each line could end up having many values :-) ) We run the script (capturing system.out and system.err).

The function returns:

Anything written to standard out (println etc)
Errors if any and line number for error where possible
A list for of maps with line number and value(s)

Most of the AST stuff is not something I’ve written. It’s been contributed by Jim White after I posted a question on the groovy-user mailing list. I asked for advice on which way to proceed and the response from the groovy community was awesome. Jim in particular was more than eager to contribute to the plugin. OpenSource rocks ! So when I say we, I sometimes mean we literally.

Anyways, based on the results of the script execution we notify Light Table to trigger either a ":groovy.res" behavior or a "groovy.err" behavior.

The json response for sendData for a successful execution might look something like:

[89,
 "groovy.res",
 {"meta":{"start":22,"end":23},"out":"hello\nmama\n","result":[{"line":1,"values":["null"]},{"line":2,"values":["null"]}]}]

Handling the evaluation results in Light Table

(defn notify-of-results [editor res]
  (doseq [ln (:result res)]
    (let [lineNo (+ (:line ln) (-> res :meta :start) -1)]
      (object/raise editor :editor.result (clojure.string/join " " (:values ln)) {:line lineNo :start-line lineNo}))))

(behavior ::groovy-res
          :triggers #{:groovy.res}
          :reaction (fn [editor res]
                      (notifos/done-working)
                      (when-let [o (:out res)] (.log js/console o))
                      (notify-of-results editor res)))

(defn notify-of-error [editor res]
  (let [lineNo (+ (-> res :ex :line) (-> res :meta :start) -1)]
    (object/raise editor :editor.exception (:ex res) {:line lineNo :start-line lineNo'})))

(behavior ::groovy-err
          :triggers #{:groovy.err}
          :reaction (fn [editor res]
                      (object/raise editor :groovy.res res)
                      (notify-of-error editor res)))

These are the behavior definitions that handles either successful or evaluation of scripts with errors. Basically we: Print to the Light Table Console anything that was captured to system.out/system.err by our groovy evaluation Show inline results for each line, multiple results for a line are space separated. For showing inline results we are using a predefined Light Table behavior (:editor.result) If the behavior is to handle an error, we show evaluation results up until the script exception. In addition we display details (stack trace) for the exception at the line in the script it occurred

Wiring it all up

groovy.behaviors

{:+ {:app [(:lt.objs.plugins/load-js ["codemirror/groovy.js", "groovy_compiled.js"])]
     :clients []
     :editor.groovy [:lt.plugins.groovy/on-eval
                     :lt.plugins.groovy/on-eval.one
                     :lt.plugins.groovy/groovy-res
                     :lt.plugins.groovy/groovy-err
                     [:lt.object/add-tag :watchable]]
     :files [(:lt.objs.files/file-types
              [{:name "Groovy" :exts [:groovy] :mime "text/x-groovy" :tags [:editor.groovy]}])]
     :groovy.lang [:lt.plugins.groovy/eval!
                   :lt.plugins.groovy/connect]}}

The eval and results/err behaviors are defined for the editor tag. So they are only applicable for editors marked as groovy editors. Any editor open with a file name ending in .groovy will automatically be attached to a editor.groovy tag. (You can also set it manually cmd+space → "Editor: Set current editor syntax"). The ":eval!" behavior is defined for the :groovy.lang tag. Its tied to our groovy mother object just like the connect behavior. These behaviors are totally groovy client specific, whilst the other behaviors are less so (although not exactly generic as they are now…)

Wrap up

A little bit of plumbing was needed to get this set up. But the hard parts was really coming up with the groovy AST transformation stuff. I guess by now you might have started getting an inkling that Light Table is fairly composable ? It really is super flexible. You don’t like the behavior for handling inline results for the groovy plugin ? You could easily write your own and wire it up in your user.behaviors file in Light Table. It’s wicked cool, actually it really is your editor !

Yesterday I released version 0.0.2 of the Groovy LightTable plugin. Its available through the Light Table plugin manager, or if you wish to play with the code or maybe feel like contributing feel free to fork the repo at : https://github.com/rundis/LightTable-Groovy. Pull requests are welcome.

So where to next ? I’d really like to try and create an InstaRepl editor for the plugin. A groovy script editor that evaluates code as you type. There’s gotta be one or two challenges related to that. A quick win might be to provide groovy api documentation from inside Light Table. I’ll let you know what happens in the next post.

Note	Disclaimer: I might have misunderstood some nuances of Light Table, but hopefully I’m roughly on track. If you see anything glaringly wrong, do let me know.