Exploring Bucklescript's Interop with JavaScript (in Reason)

Disclaimer: this is a blog post version of the talk I gave at the ReasonML Munich Meetup.

If you want to watch the talk, it's on Youtube:

If you're just looking for the slides, you can find them here.

Introduction

Interoperability between Reason (or Bucklescript, really) and JavaScript is very important. There's over 600 thousand packages on NPM and if you're writing Reason and targeting JavaScript as output, there's a good chance you need to use a few JavaScript packages. Integrating with the JavaScript ecosystem in general is very important and a big selling point of Reason/Bucklescript.

Let's take a look at the simplest way to integrate some JavaScript code in a Reason program:

let add = [%raw {|
  function(a, b) {
    console.log("hello from raw JavaScript!");
    return a + b
  }
|}];

Js.log(add(2, 5)); /* 7 */

Looks simple, right? You can just take raw JavaScript code and insert it in the middle of your Reason code using the raw Bucklescript attributes. This might look terrible and useless in real life but it's actually surprisingly handy. You can read about an actual use case in this section of the Bucklescript documentation.

Of course, because Bucklescript cannot guess the type of add in the above snippet, we have no type safety. We could, for instance, try to do add + 8 and because this is JavaScript, it would result in something weird at run time (not even an exception).

Now, you can do a little bit better by annotating your raw JavaScript code like this:

let add: ((int, int) => int) = [%raw {|
  function(a, b) {
    console.log("hello from raw JavaScript!");
    return a + b
  }
|}];

Js.log(add(5, 6));

Of course, we can do even better. We can use various Bucklescript attributes to design and declare a proper Foreign Function Interface (FFI). Let's take a look at a very basic example:

[@bs.val]
external sqrt : float => float = "Math.sqrt";

let four = sqrt(16.0); /* => 4 */

(If you come from an OCaml background, this API and syntax be familiar to you. This is because it is very similar to the API that OCaml uses for its C Programming Language FFI.)

In this snippet, we declare a binding for the JavaScript Math.sqrt function. We use the external keyword to declare it, and annotate its type as float => float. Finally, we indicate what it maps out to in JavaScript as the value of the external: "Math.sqrt".

Then, we can simply use the newly declared sqrt function as any other fully type-safe function in Reason.

Now, notice that we used the [@bs.val] attributes to indicate to the Bucklescript compiler that we are declaring a binding for a value. There's a lot of other attributes that the compiler provides for the interop system. You can read about all of them in the Bucklescript documentation, but for the purposes of this blog post, I'll indicate here a few examples:

Example #1 (Luxon)

Let's now take a look at a real example of creating bindings for an existing JavaScript library. I chose Luxon for the purposes of this research. It's a date/time/duration handling library written in JavaScript (by the same team that wrote the very popular moment.js library). It has a chainable, immutable API and there were also no Bucklescript bindings available for it yet.

This is a Luxon example straight from their README:

const { DateTime } = require("luxon");

DateTime
  .local()
  .setZone("America/New_York")
  .minus({ weeks: 1 })
  .endOf("day");

We're now going to create bindings for all of those types and functions so that we can then recreate that very same example but using Reason.

The first function that we have to bind is the local() constructor. It takes no arguments^[1] and returns a DateTime object representing the current time. To create a binding for it, we can start with this:

type dt;
external local : () => dt => "local";

We declare an abstract type dt (short for DateTime) and add an external declaration for local as a function that takes no arguments (unit in OCaml) and returns a value of the DateTime Object type. Its JavaScript mapping is simply "local", but we could omit it as simply "" since the name of our external declaration is the same as the JavaScript mapping.

Next, we need to add the proper Bucklescript attributes to this declaration:

type dt;
[@bs.module "luxon"] [@bs.scope "DateTime"] [@bs.val]
external local : unit => dt = "";

let myDate = local();

We need [@bs.val] because we're binding a value and [bs.scope "DateTime"] together with [@bs.module "luxon"] because the local function lives inside a global object in the luxon module. After adding these attributes, we can use the local() function like any other reason function and when we call it, we get something like this when we print out the value of myDate:

DateTime {
  ts: 2018-02-03T12:28:27.564+01:00,
  zone: Europe/Berlin,
  locale: und }

Let's take a look at what the generated Bucklescript JavaScript code looks like for the previous snippet:

// Generated by BUCKLESCRIPT VERSION 2.1.0, PLEASE EDIT WITH CARE
'use strict';

var Luxon = require("luxon");

var myDate = Luxon.DateTime.local();

console.log(myDate);

exports.myDate = myDate;
/* myDate Not a pure module */

As you can see, Bucklescript doesn't generate a function for the local() function (although you can make it do that if you want). It simply replaces every call of local in Reason to the equivalent JavaScript mapping that we defined in our FFI's declaration.

The next function that we need to create a binding for is setZone. This is a method that is applied on a DateTime object type and takes as argument a string. We can declare the binding for it like this:

[@bs.send.pipe : dt]
external setZone : string => dt = "";

Notice that we use the [@bs.send.pipe] Bucklescript attribute. It is similar to [@bs.send] in that it is also used for declaring methods. However, the pipe attribute places the main argument in the last position of the function declaration. This is a more OCaml-y way of doing this because it allows us to use the |> ("pipe" or reverse function application operator) to chain method calls, like this:

let myDate =
  local()
  |> setZone("America/New_York");

Note that the type of setZone is actually (string, dt) => dt. If you are confused about the |> operator, it is well explained here. Additionally, you can also look at its definition using rtop to see what it does:

Reason # (|>);
- : ('a, ('a) => 'b) => 'b = <fun>

Clever, right? Now, we need to create a binding for the minus function. As you might recall, in JavaScript it was used this way:

dateTimeObject.minus({ weeks: 1 })

We can look at the documentation for this function to understand what arguments it can take.

Interestingly, its only argument can be of three different types. There's two ways of creating bindings for this type of functions Bucklescript. The first one is to declare 3 different functions, all with different parameters:

[@bs.send.pipe : dt]
external minusMs : int => dt = "minus";

[@bs.send.pipe : dt]
external minus : duration => dt = "minus";

[@bs.send.pipe : dt]
external minusObj : durationObj => dt = "minus";

Note that all of the functions need to have different names because there is no parameter overloading in OCaml. However, they all map to the same JavaScript function and return the same type.

The second option here is to use polymorphic variant types to have a variadic type argument function:

[@bs.send.pipe : dt] external minus :
  ([@bs.unwrap] [
    | `Duration(duration)
    | `DurationObj(durationObj)
    | `Int(int)
  ]) => dt = "";

However, for the purposes of the talk (and this accompanying blog post), I decided to stick with the first, simpler, option. We'll only implement minusObj for the purposes of simplicity. Here's one way to do it:

type durationObj = { .
  "weeks": int,
  "months": int,
  …
};

[@bs.obj]
external makeDurObj : (
  ~weeks: int=?,
  ~months: int=?,
  …,
  unit
) => durationObj = "";

[@bs.send.pipe : dt]
external minusObj : durationObj => dt = "minus";

This looks a bit ugly… We need to declare a Bucklescript Object type that matches the options argument type. Then we use the [@bs.obj] attribute to create a function that converts a set of labelled arguments into a matching object. Finally, we declare the minusObj function (once again a method applied on dt types).

We can then call it like this:

let myDate =
  local()
  |> setZone("America/New_York")
  |> minusObj(makeDurObj(~weeks=1, ()));

There's 2 important things to note here:

1. We could hide the makeDurObj function call using shadowing using [this technique].
2. If we were writing this library from scratch in Reason/OCaml, we would very probably only support 1 API using labelled arguments. Unfortunately, because we are creating bindings for a JavaScript API, we try to match their API so that it is easier for clients of our bindings to convert their code.
3. Note how we pass a unit (()) to limit the end of our labelled arguments because otherwise the compiler would assume we are just doing currying.

Finally, we need to create a binding for the endOf function. We can write it like this:

[@bs.send.pipe : dt]
external endOf :
  (
  [@bs.string]
  [ | `year | `month | `week | `day | `hour | …]
  ) =>
  dt = "";

Note how we are using a poly variant type together with the [@bs.string] attribute to constrain the set of arguments that can be passed to this function^[2]. We just improved upon the original JavaScript by adding compile-time argument checking. This would have been impossible in pure JavaScript but because we are writing OCaml, we can do this kind of thing.

We can now compare the 2 full examples in Reason and JavaScript, and notice that they are not that different:

DateTime
  .local()
  .setZone("America/New_York")
  .minus({ weeks: 1 })
  .endOf("day");

And now in Reason:

let myDate =
  local()
  |> setZone("America/New_York")
  |> minusObj(makeDurObj(~weeks=1, ()))
  |> endOf(`day);

Example #2 (mysql.js)

Alright, that was cool. Now let's take a look at another example that makes use of the Js API that ships with Bucklescript.

Let's take a look at a code snippet that uses the mysql JS library:

var mysql      = require("mysql");
var connection = mysql.createConnection({ host: "127.0.0.1", user: "root" });

connection.query("SHOW DATABASES", function (error, results, fields) {
  if (error) {
    console.log(error);
    return;
  }

  console.log(results, fields);
});

connection.end();

I created bindings for this library recently and at first, this is how a client would use my bindings:

let conn = Mysql.createConnection(
	~host="127.0.0.1",
	~port=3306,
	~user="root",
	()
);

Mysql.query(conn, "SHOW DATABASES", (error, results, fields) =>
  switch (Js.Nullable.to_opt(error)) {
  | None =>
    Js.log(results);
    Js.log(fields);
  | Some(error) => Js.log(error##message)
  }
);

Mysql.endConnection(conn);

Pretty similar to the JavaScript right? Note that the error parameter in the callback (which is a nullable type in JavaScript) is converted to an Option type in Reason. This is because in OCaml, there are no nullable types and we use the Option variant type to achieve a similar goal.

Note, too, that this API is not actually very good. The user could skip the Option variant checking and dangerously check the values of results and fields. This is a problem in the original JavaScript API but we can improve on it by making our bindings force a better API:

let conn = Mysql.createConnection(
	~host="127.0.0.1",
	~port=3306,
	~user="root",
	()
);

Mysql.query(conn, "SHOW DATABASES", result => {
  switch (result) {
  | Ok(results) => Js.log(results.results)
  | Error(err) => Js.log(err##message)
  }
});

Mysql.endConnection(conn);

As a member of the audience cleverly pointed out here, this sort of API is not new even in the JavaScript world. Promises can be used to solve this issue (and mysql2) addresses this problem by using promises for their async APIs. However, the original `mysql.js library does not support that, but we improved upon it in our Reason bindings.

The other way round (consuming Reason libraries with JavaScript)

Because Bucklescript produces sane (readable) JavaScript code, one would expect that it would be straightforward to consume your Reason code using JavaScript. And it is, but only sometimes.

Example #1

Let's, once again, take a look at an example:

let isAre = n => n == 1 ? "is" : "are";

We can write that Reason function in a simple file and Bucklescript will generate the following:

// Generated by BUCKLESCRIPT VERSION 2.1.0, PLEASE EDIT WITH CARE
'use strict';


function isAre(n) {
  var match = +(n === 1);
  if (match !== 0) {
    return "is";
  } else {
    return "are";
  }
}

exports.isAre = isAre;
/* No side effect */

Note that:

1. All functions and top-level values are automatically exported in the generated Bucklescript output.
2. The function that we created is very similar in JavaScript. Its argument is a number and the output is also a string.

We can easily consume this function using JavaScript, like so:

const { isAre } = require("./demo.bs");

const word = "meetups";
console.log(`${word} ${isAre(word, 3)} fun`);

// => meetups are fun

Example #2

I wrote this simple function using Reason for validating required fields in a form:

let requiredFieldValidator = value =>
  if (String.length(value) == 0) {
    Js.Option.some("Field is required.");
  } else {
    None;
  };

It can be used like this, in Reason:

switch (requiredFieldValidator("test")) {
| None => Js.log("all good")
| Some(err) => Js.log("invalid field: " ++ err)
}

If you try to consume it from JavaScript, weird things can happen. This is because JavaScript has no variants and as so, you would simply get the Bucklescript internal representation for variants in JavaScript.

Let's take a look at what that is:

(source)

Note that it is 0 for None option variants and [a] for Some option variants. We should never depend on the internal representation of values in JavaScript (they might be changed in the future).

We should, instead, take advantage of Bucklescript APIs that let us interop with them, from JavaScript:

let requiredFieldValidator = value =>
  if (String.length(value) == 0) {
    Js.Option.some("Field is required.");
  } else {
    None;
  };

let requiredFieldValidatorJs = value =>
    Js.Null.from_opt(requiredFieldValidator(value));

const { requiredFieldValidatorJs } = require("./demo.bs");

console.log(requiredFieldValidatorJs("test"));
// => null

console.log(requiredFieldValidatorJs(""));
// => "Field is required"

Example #3

type person = {
  age: int,
  name: string
};

let personToString = (p: person) => {
  let name = p.name;
  let age = p.age;
  {j|$(name) is $(age) years old|j};
};

let david = {
  age: 22,
  name: "David Gomes"
};

Js.log(personToString(david));

/* => "David Gomes is 22 years old" */

In the above snippet, we declare a new record type person and a simple function personToString that outputs a string representing a person record. We then create a new record called david (note that Bucklescript automatically infers the type of david as person), and use the personToString function on it.

Given that records are currently implemented as arrays in the Bucklescript generated code, one would maybe think about consuming this function like this:

const { personToString } = require("./demo.bs");

console.log(personToString([22, "david"]));

However, as I explained before, we should never do this, because we are depending on the internal representation of record types. We can instead do this:

Reason

[@bs.deriving jsConverter]
type person = {
    age: int,
    name: string
};

JavaScript

const { personFromJs, personToString } = require("./demo.bs");

console.log(personToString(personFromJs({
    name: "David",
    age: 22,
})));

The [@bs.deriving] jsConverter, documented here, adds two functions to the generated Bucklescript code — personFromJs and personToJs. These functions do what you would expect them to do, which comes in very handy!

Key Takeaways

Let's summarize 8 things we learnt during the talk (or this accompanying blog post):

1 — Think twice before you start to bind

You don't always need to create bindings. Sometimes it might be better to just Rewrite It In Reason™, or to look for existing bindings. Use Redex or the Discord chat for this.

2 – Gradually reach full type safety, don't be afraid to start slowly

You should start your path to full type safety-ness by iterating on ever-more typed code. Don't be afraid to use raw JavaScript code and patch it up later!

3 – Bindings don't need to be 1-1 mappings of the original API (and probably shouldn't be)

As we saw in our examples, bindings don't have to be perfect copies of the original JavaScript API. In fact, they shouldn't be because OCaml is different from JavaScript, and so you should try to do things in a more OCaml-y way.

4 – You can (and should) write test for bindings

Here's an example from bs-luxon that uses Jest (and bs-jest):

test("DateTime.local(2017) is the beginning of the year", () => {
  let dt = DateTime.local(~year=2017, ());
  expect(getDateObject(dt))
  |> toEqual({
       "year": 2017,
       "month": 1,
       "day": 1,
       "hour": 0,
       "minute": 0,
       "second": 0,
       "millisecond": 0
     });
});

The best way to write tests for your bindings is to simply copy the original library tests. The above example is a rewritten test from the original Luxon Jest test set.

5 — You should use interface (.rei/.mli) files

This technique is very popular in OCaml because it allows for a separation of concerns between type signatures and actual implementation details.

6 – Inspect your JS output every now and then

Every now and then you should look at the code Bucklescript is generating. It's a great learning opportunity.

7 – Writing bindings is a great way of learning Reason/OCaml

You can learn a ton about Reason and OCaml by writing bindings for a JavaScript library!

8 – Having JavaScript clients consume your Reason libraries might require some work.

This is just something you'll have to deal with if you really care about JavaScript clients consuming your library. I wouldn't worry about this until it is needed.

The End!

I'd like to thank Axel Rauschmayer and Johannes Weber for giving me the chance to speak at the ReasonML Munich Meetup. It was great meeting a lot of people who are interested in Reason and OCaml there.

Finally, here's 3 great links if you want to learn more about this topic:

1. Bucklescript Documentation
2. Bucklescript Manual
3. Bucklescript Cookbook

Feel free to comment via Twitter @davidrfgomes.