20. Mustache lambdas for parsing data

Damian Cugley

One of the loose ends of my fanciful idea of using JSON data to power sidebars on rendered pages (see my post about resource descriptions) is how to format data when Mismiy does not know the data type.

Motivation: sidebars based on data blobs

Suppose you’re writing a series biographies of famous writers, or a blog about Star Trek fan films perhaps, and have embedded data blobs describing the subject of each post. You would like this data blob to be added to the page as a nicely formatted sidebar.

The trick is that while a human would guess a field named birthday, say, represents a date, Mismiy cannot. So it will not apply Mimsiy’s special workaround where a date field is expanded in to an object (see the post about date formatting). This would mean we have to display the raw data (which looks like 1564-04-23), rather than formatting it nicely as 23 April 1564. It’s even worse for duration values (like movie lengths, or recipe times in Google’s example), since the ISO-8601 durations datatype looks like PT1H30M: we don’t want to display it like that on the page.

How can we tell Mismiy which of an arbitrary collection of fields needs special processing based on datatype? I can imagine one stupendously over-engineered solution where Mimsiy parses the data as JSON-LD, follows the @context link to the Schema.org page and processes its JSON-LD description (perhaps needing to use OWL), from which it can infer datatypes somehow. This seems like a lot of code.

But we don’t need it: the person designing the template needs to know which datatype a field has anyway, so we just need to make it possible to express the datatype in the template itself. But how can we, given Mustache’s intentionaly limited syntax?

The way this is done in Django templates (with which I am more familiar) is a system of filters that transform values. Their syntax looks like this:

{{ birthday|date:"j N Y" }}

where birthday names a value in the template context, |date applies the filter, and :"j N Y" supplies an argument to the filter (in this case specifying the format). Mustache does not have this extra syntax. What it does have as an escape hatch is lambdas.

Using Mustache lambdas

Mustache sections, marked up like {{#foo}}…{{/foo}}, have a meaning that depends on the datatype of foo in the current context. When foo is a callable object (such as a function—called a lambda in some programming languages), the Mustache manual says the function is invoked, but is vague about the details. Chevron, the Mustache formatter used in Mismiy, follows a fairly common convention, where it invokes the callable with two arguments:

  1. the raw text content of the section, and
  2. a function for rendering partial templates, optionally supplying additional data to add to the template context.

So let’s add a lambda as_date to the template context, which might be defined something like this:

def as_date(text: str, render: Callable) -> str:
    # Get the date value:
    value = render("{{.}}")
    if isinstance(value, str):
        value = date.fromisoformat(value)

    # Now render the section text with the expanded date.
    data = expand_date(value)
    return render(text, data)

In the template, one writes something like this:

{{#birthday}}
<dt>Birthday</dt>
<dd>
    {{#as_date}}
    <time datetime="{{iso_date}}">{{day}} {{month_name}} {{year}}</time>
    {{/as_date}}
</dd>
{{/#birthday}}

There is a confusing bit of jugging involved getting the right data to the right place.

  1. The section {{#birthday}} … {{/birthday}} sets the current context to be the value of the birthday field.
  2. The function calls render("{{.}}") to obtain the date value.
  3. Mismiy’s expand_date function converts this to an object with fields such as month_name.
  4. The second call to render expands the inner template fragment with that data added to the context.

This is not the only way to do this, but it leans on Mismiy’s existing conventions.

How Mismiy can help

One of the principles of Mimsiy is that we put off implementing something until there is a burning need for it. I have plenty of experience of a feature added because a hypothetical user might want it some day then consumes endless maintenance as its code is kept up to date with other changes to the application. One advantage of writing code for myself is I can choose to leave out features.

So if I have no immediate need for data-driven sidebars, perhaps I should not be adding a feature for formatting datatypes. (Of course, should my excellent reader feel a burning need to create a review blog with structured-data-driven sidebars, then please let me know!)

Having said that, the impact of adding a couple of lambdas for formatting dates and durations will have very little impact on the rest of the code, so there is little harm in keeping it. And I originally started this as an exercise in learning how Chevron uses lambdas, so the real utility is not in immediately using the feature, but the thinking about how to implement it.

The two lambdas added to the template context for each page (hence available ‘globally’) are as_date and as_duration.

as_date

For as_date, the value of the field added to the context must be one of the following:

In the last case only the year field of the object is set.

as_duration

For as_duration the field must be in ISO duration format, which is something like this:

P [ years Y ] [ months M ] [ days D ] [ T time ]

where the square brackets enclose optional elements, and time is

[ hours H ] [ minutes M ] [ seconds S ]

For example:

It gets turned in to an object with fields years, months, days, hours, minutes, and seconds. Each field is included if and only if it is in the original representation.

Trying it out

My attentive reader will already have spotted the sidebar added to this page. This is generated from a data blob which can be seen in the source of the page as a script element by a simple-minded template data_about.html.

The short film Nemo Me Impune Lacessit (2016) is a Star Trek fan film—in other words, an amateur production of a story set in the broader Star Trek universe. The series in question, Star Trek: Intrepid is made by fans based in Dundee, Scotland, and follows a Starfleet ship (captained by Daniel Hunter, played by Nick Cook) in joint operation with the Merchant Service ship Ariadne (represented in this episode by science officer Dr Richard Garren, played by Mike Cugley). This film is a great demonstrator of the strengths and weaknesses of fan films. Unlike the old TV series, where ship interiors are standing sets and external effects are expensive, building sets is beyond a hobby budget, but CG space ships and battles are in the reach of anyone with PC suitable for 3D games and evenings to spare. Actors often gather from far and wide join in the filming, which means episodes often have to be replanned based on which characters and footage could be achieved on the day. This makes long-term plot arcs difficult to achieve. The writing can be intresting too as it comes from people steeped in the lore of the show, interested in different sorts of dramatic problems than the TV writers of yesteryear.

The title of this film is the national motto of Scotland, translating roughly as ‘Wha daur meddle wi me?’. It’s presented as an episode in a serial, with opening crawl giving the story so far and a cliff-hanger at the end. The story throws together the abovementioned officers from theoretically co-operating but also rival space services and forces them to interrupt their mutual sniping long enough to survive an attack by agents of the Orion Syndicate and save 197 miners from mines. It packs a lot of one-sided space battles, derring-do, and sarcasm in to its 11 minutes.

Posts on similar topics