Tag Archives: dust

The Dust Bowl


This dust bowl post is not about yet another American football bowl game. Nor is it about what might be coming to pass for those of us in California with no rain. Rather, it is a recap of the first gathering of dust.js contributors and users. The goal of the meeting was to set directions and schedules around the evolution of the dust templating engine (http://linkedin.github.io/dustjs/).

The companies represented at this first Dust Bowl were: Benefitfocus, eBay, LinkedIn, PayPal, StubHub, and Yahoo.

The meeting kicked off with each company providing a summary of how they are using dust. Then we moved on to the main agenda for the day:

  • New build mechanisms for the dust GitHub project
  • Quarterly release plans for 2014
  • Directions for Dust.js helpers
  •  Making extension/hooking of dust easier for frameworks
  • Security-related issues
  • Internationalization and globalized object formatting (dates, money, etc.)
  • Review of major open issues to determine concensus and direction
  • Futures: promoting dust, meetups, blogs, expand awareness, future dust bowls

What the future holds

Like kids eager to open their presents, I’m guessing you want to see what new goodies are in the works so let’s go there straight away. Some of the items are exploratory and some are quite concrete. The goal is to have a release per quarter in 2014.

Q1 2014

  • Establish collaborative mechanisms for contributors in the intervals between dust bowls. Ideal features: Mobile friendly, milestones, tasks, discussion board, notifications
  • Evaluate options for making 3rd party dust helpers easier for the community to find and load
  • Size of dustjs-helpers library is a factor when adding new helpers. Explore ways to let someone build a library with just the ones they need.

Features for Q1 2014

  • A solution for retaining whitespace in templates for HTML/JS (Issue #238)
  • Lower some error messages to warnings
  • Evaluate helpers for inclusion in core set (See section below)
  • Parser syntax errors should be passed to callback (Issue #107)
  • Unspecified parameter results in null value (Issue #252)
  • Support template that contains JavaScript // comments (Issue #300) – part of whitespace work.
  • Partial parameters lower on stack than context (Issue #313). Reverse a code change that is no longer needed after the changes around on how paths work.
  • Add CommonJS support (Issue #325)

If you only ever add features to a language, it becomes bloated.  Some things turn out to be bad decisions in hindsight or of little value.  As part of a New Year slimming down, we discussed deprecation of some language features. The new warning/error logging capability added in release 2.1.0 would provide a notification mechanism for deprecation. The features we might want to put “on notice” for future removal are:

  • Manual contexts on sections and partials {>partial:context}
  • JavaScript functions in the JSON context (use helpers)

As part of the Q1 cleanup, we will review old GitHub issues with an eye to closing ones that are unlikely to ever get done and consolidating others into a single new issue that covers several related topics.

Q2 2014

Q2 will be a performance enhancement-oriented release. We will go back and compare current performance with our baseline, determine where slowdowns arose, and determine ways to make things faster. General performance profiling will further inform areas of performance improvement work.

As part of the performance work, we will add build automation so performance changes are measured at each commit. Examining compiled code patterns help us understand if compiler changes can improve runtime performance and minification of generated code.

Organizationally, we proposed Dust Bowl II for around the end of the second quarter.

Features for Q2 2014

  • Enhance the 4-panel playground for users. Make it 3 panels by hiding generated code but with option to show it. Provide a way to support user partials and helpers in the playground.
  • Add tests to better cover internal private functions.
  • Improve testing environment to make it easier to test complex templates and contexts.
  • Domain dustjs.com has been registered. Build out a pre-release user portal to “all things dust” at this URL.
  • Evaluate allowing single quotes for parameters in sections and partials (currently only double quotes are allowed)
  • Make external APIs more easily hookable for extension.
  • Extend onLoad to allow passing context.
  • Allow user-provided alternatives to the dust cache.
  • Allow alternative logger to be hooked into the dust logging.
  • Look at async vs sync behaviors that exist today and possible ways to control which is used.

Q3 2014

Official launch of http://dustjs.com to target new and advanced users. Site would include::

  • Blog
  • Upcoming releases
  • Documentation
  • Simplified Sandbox playground for trying out dust
  • Examples ranging from simple to advanced
  • Content Restructure

Features for Q3 2014 

  • More robust XSS filtering library (LinkedIn and PayPal have these)
  • Strong emphasis on XSS risks of using current @if due to it using eval. Deprecate it and provide safer alternatives (@or, @if interpretive version)
  • LinkedIn and PayPal have security scanners for dust templates. Consider open-sourcing. See also https://github.com/smfoote/Swiffer.js for a general dust lint-like scanner
  • Extend filter implementation to give users the ability to write more powerful filters. Step 1: Make context available to filters which opens up a decent amount of extra functionality for things like double evaluation of braces  Step 2: Extend the compiler to allow a filter with parameters like: {name | filter(param1, param2, …) }.
  • Explore compiler enhancement for static final blocks in template to capture complex text blocks for multiple reuse
  • Begin regional meetups to promote dust

Q4 2014

  • Look at more “way out there” directions like HtmlBars (https://github.com/tildeio/htmlbars)
  • Look at status of Web Components work and how it might integrate with dust.

So that’s the best guess currently as to how releases look for the upcoming year.  Let’s look at a few other topics that were discussed during the day.

Directions for Dust.js helpers (some candidates for general utility inclusion)

  • @provide: http://rragan.github.io/dust-motes/docs/helpers-data.html
  • @if: interpretive version – https://github.com/rragan/dust-motes/tree/master/src/helpers/control/if
  • @iterate: Issue #48 in DustHelpers repo. See also http://rragan.github.io/dust-motes/docs/helpers-control.html

Localization/Internationalization (No release identified)

  • PayPal kraken-js open source has an internationalizaton solution for messages
  • LinkedIn @format tag tackles a number of issues like formatting numbers, currency, date/times, and more
  • Look at intl.js polyfill for future language extension areas
  • Look at https://github.com/component/component/ which has some L10N things

GitHub Releases and Issues

  • Consider using new GitHub Releases mechanism. Generally favorable opinion on doing this.
  • Look at waffle.io (Trello-style for GitHub issues) to manage issues.
  • Consider https://github.com/fat/haunt to create GitHub issue bot to help manage the backlog/new submittals.


We wrapped up the day by discussing how we can spread knowledge and interest in dust. Ideas included:

  • Blogging – your reading one right now
  • Be active on StackOverflow area for dust
  • A standard introductory presentation that can be given at local meetups
  • Contact with local college and universities to promote JavaScript, Node and dust. Possible seminar presentations.
  • Meet online quarterly and in person twice a year.

At the end of the meeting, all of us were pleased at the plans made for dust. Hopefully, the community will find some useful new features in the plans. If you want to join the action as a contributor, head right over to https://github.com/linkedin/dustjs or https://github.com/linkedin/dustjs-helpers for the helpers — check out the issues and see if you spot something you are interested in contributing to. Then fork a copy, do your thing and send us a pull request.

– Richard Ragan: @rrragan, https://github.com/rragan

Dust is eloquent – the logic wars


There are classic feuds like the Hatfields and McCoys, vi versus emacs, and PC vs Mac. In the world of templating, there seems to be just one such dispute: logic-less template languages versus template languages providing logic constructs. So why the fuss?

If you consider the Model-View-Controller (MVC) paradigm, work is divided up among the three parts and compartmentalized. Each part has its own concerns. The view takes data from the model to render the presentation. The controller receives requests, marshalls data to send to the model, and selects a view to be rendered. The model usually has business rules and application logic plus provides data to the view for rendering. The goal is to maintain a “separation of concerns” among the three parts.

Separation of concerns and logic in the template

Maintainability and flexibility are key reasons for using an MVC-like architecture. If business logic is partially in the model and partially in the view, it becomes difficult to figure out where to make changes when fixing or enhancing the system. This is where the “logic-less” proponents take their stand. If you can’t do logic in the template language, then you can’t accidentally have business logic in the template implementing the view.

Logic-less template languages

On the face of it, logic-less sounds fairly extreme. How can you do anything that is conditional if you can’t have any logic? It turns out, if you restrict the logic to just if-exists and if-not-exists, you won’t be able to do any business logic but can still have the template provide different output for different values in the data model. This is the minimal logic provided by Mustache, one of the earliest logic-less template languages. Here is an example of using all the logic forms you have in Mustache. The snippet will output a list of names or output “No names provided” if there are no names in the model data.

  No names provided

Mustache’s section notation, {{#foo}}, is similar to that of Dust except Dust uses fewer “mustaches”, e.g. {#foo}. Both languages skip rendering the content if foo is non-existent or is an empty list thus providing a form of logic wrapped in the idea of iterating over data. Mustache adds one more construct, the {{^foo}}text{/foo} which will output the text body if foo does not exist, is false or is an empty list. There you have it. That is the total extent of logic in Mustache. This deliberate minimalism makes it impossible for the template writer to add any business logic to the view. It does mean that any variation in output must be catered to via the model having extra values (or omitting them when not present is used to control the output).

The slippery slope of logic

Adding logic capabilities to template languages is an easy thing to do and tempting since the asceticism of Mustache often chafes the template writer. The downside of adding logic capabilities was not well understood in the early days of template languages. If you look at this JSP hideosity, it becomes clear how bad it can get.

<script type="text/javascript">
var myConfig = {
   <% if (lastPage) { %>
     showPreviousButton  :true,  
     showNextButton : false ,
   <%}  else  {%>
   showPreviousButton  :true,  
     showNextButton : true ,
   <%} %>

This mixes three languages in just a few lines (HTML, JavaScript and JSP) and relies on the fact that JSP is compiled down to Java code. Even if you don’t use scriptlets which are generally acknowledged to be a bad thing, JSP still has full conditional logic via the Java Standard Tag Library’s tag. The EL expressions it provide gives you the full power of Java conditionals so the path is wide open to add business logic to the template.

There is an interesting paper by Terence Parr, “Enforcing Strict Model-View Separation in Template Engines” that formalizes the study of template engines using results from formal language theory. In the paper, he defines an “entanglement index between model and view”. The index ranges from 1 (low entanglement) to 5 (maximal entanglement). Each of 5 rules violated increases the index value. The rules are:

  1. Model data must be read-only
  2. No computations can be performed on model data, e.g. price*.90 to get sale price
  3. No comparisons on model data, e.g. numAccounts < 5
  4. No assumptions or dependencies on the type of the model data
  5. Model data must not contain information to control the display/layout

His survey of a number of template languages showed they are polarized. Either they have minimum entanglement of 1 or a maximal entanglement of 5 — the slippery slope seems clear. Dust with the current set of Dust helpers has an index of at least 3 and maybe 4 depending on what the application puts in the model. If you restrict yourself to the main Dust language and avoid the helpers, you can be at an index of 1

For minimum entanglement, these are the only features a template language should provide:

  • attributes (called keys in Dust)
  • conditional processing based on presence/absence of an attribute
  • recursive template references
  • template application to multi-valued attribute

What logic capabilities does Dust offer?

The Dust base language is very similar to Mustache with a few tweaks to make it more readable. If we rewrite the above Mustache example in Dust it would be:

  No names provided

This is more concise and readable than the Mustache version. If you want to write it Mustache-style Dust still has the {^foo} form. Further, Dust offers {?foo} as the inverse of {^foo} allowing a test without committing to iteration at the same time. In the spirit of Mustache and logic-less, none of these forms permit comparison.

Be aware of how model values intended for such tests work exactly in Dust. It is not exactly the same as JavaScript.

  • Empty strings, “” or ” evaluate to false, boolean false, null, and undefined all evaluate to false. This corresponds with how these values are treated in JavaScript
  • Numeric 0 evaluates to true, as does any non-zero length string value. The treatment of numeric 0 is different from JavaScript where is it is considered false.
  • Empty array -> [] is evaluated to false. All other objects evaluate to true.

This means that using 1 and 0 for model value of true and false is a bad choice for Dust as both are considered true. Your best bet for a false representation is to just omit the value from the model or make the value an empty string.

In case you are curious what Dust has added that increases the entanglement index, it’s the following helpers: @if, @select, @eq, @ne, @gt, @lt, @gte, @lte, @math. We will take a look at these in a future post on helpers.

Why do developers need to write logic in templates?

Let’s look at the other side of the coin. Most UI developers are not maliciously looking to jam business logic into their templates. Largely, they are trying to deliver a designated UI design and user experience in a productive way and leave a flexible/maintainable solution. Why might a UI
developer feel they need logic in the template?

The data model is inadequate to do my job

If the model data is not sufficient for just exist/not-exists tests then you have only two choices: a) use template logic to do compares or b) ask the owner of the model to make changes to meet your presentation needs. Often the model construction is done in a language not known by the UI developer (like Java) or it comes from some external third party and is just a “given”. This prevents a “dive in and fix it yourself” solution.

But suppose the model is built within the scope of your overall project. What might you, the UI developer, need to ask the model owner to change and why? Here are some samples:

  • In the list of state names and abbreviations, please add “selected: true” to the U.S. state this user has in the address information. Why: I cannot mark the element “selected” without logic in the template.
  • In the list of account records, add a property “even: true” to entries that are even numbered. Why: I can’t add a class to even rows without logic in the  template. Well, not quite true. I could use nth Child CSS pseudo selector but not all browsers support it.
  • If the likesCount is zero, add an extra item to the model, “noOneLikesYou: true”. Why: I can’t test the model value, that would require logic in the template.
  • If the values are 0 and 1, could you make them ” and 1 (see Dust true/false testing above). Why: I can’t test explicit 0 and 1 values without logic.

Of course, the response from the owner of the model logic might well be “Why should I add presentation-oriented data into the model? That violates the separation of concerns and bloats the model.”

So in an effort to keep all logic out of the template, we have introduced view-oriented logic into the model. If in the future, the view needs to change, the model logic will be impacted or even more tweaks to the model will be required. So we have broken the “separation of concerns”, just in a different way.

So what if we allow logic? I poked around in some of our template code and found this example where having logic is being abused.

{@if cond="parseInt('{expirationYear}') < new Date().getFullYear()"}
    stuff to output

This is using the full power of eval in @if to do a bunch of JavaScript logic to determine what should have been a “isExpired” value in the data model.

The horns of our dilemma

We can either:

  1. severely restrict our template language to be certain careless UI developers can’t inadvertently introduce some business logic, or
  2. allow some logic in our template language to avoid presentation-oriented data having to be added to the model but ensure the logic is used only for pure UI matters.

I favor option 2,  not crippling the UI developer’s just to be sure they don’t mess up. We don’t treat other developers in this way. Instead we use mechanisms like good training, peer programming, and code reviews to be sure the code is high quality and robust. Similar techniques for UI developers can ensure that business logic stays where it should, even if we occasionally have to apply the Mallet of Loving Correction.

The following simple questions could form a checklist for UI developers employing logic beyond the simple attribute tests.

  • If you are comparing a model value to something else, is the purpose purely presentational?
  • If your logic tests involves && and ||, think hard about whether there might be some business logic going on. The more complex the test, the more suspect it should be.
  • If you are doing any math operation, make certain it is only for presentation purposes

So there you have the arguments for logic-less and what some call “less logic” template languages. You’ve seen some pros and cons and I’ve stated my preference. If you feel strongly about it one way or the other, go make a comment on Twitter with the hashtag #logicless. We’ll be interested to see your thoughts.

If you want to read more on the logic-less debates, here are some other links to peruse.

The case against logic-less templates
Logicless template redux
Cult of Logic-less template