Building a Compiler for the Digital Sequencer#

Today I’ve been thinking about how to approach the compiler for the Digital Sequencer DSL.

Because the language is still fairly small and focused, the compiler itself can also remain relatively simple.

My current thinking is that it will be split into a few clear stages:

  • Lexical Analysis — Read the input text and convert it into tokens.
  • Syntax Analysis — Check that the token sequence matches the grammar of the language and build a structured representation of the program.
  • Semantic Analysis — Validate that the program makes sense, for example by checking labels, commands, and arguments.
  • Intermediate Representation — Convert the parsed structure into a simpler internal form that is easier to work with.
  • Optimisation — Perform any simple compile-time improvements, where appropriate.
  • Code Generation — Emit the final bytecode that the sequencer will execute.

At this point, I’m not aiming for a large or overly clever compiler. The goal is simply to produce something clear, reliable, and easy to extend as the language evolves.

Internally, these stages will still exist as separate steps in the code, even if some of them are effectively pass-through for now. For example, the Optimisation stage may initially do very little, but keeping it in place makes it easier to introduce improvements later without restructuring the compiler.

Let’s take the source code line below:

PROGRAM door_control
DECLARE:
MAP DOOROPEN AS OUTPUT USING 0

PROCESS:
BEGIN
# OPEN THE DOOR
    SET DOOROPEN HIGH
END

This program would move through each stage of the compiler as follows:

Lexical Analysis Stage#

At this stage, the compiler reads the raw source text and converts it into a sequence of tokens.

For example, the following line:

SET DOOROPEN HIGH

might be tokenised into something like:

  • KEYWORD(SET)
  • IDENTIFIER(DOOROPEN)
  • KEYWORD(HIGH)

Comments such as:

# OPEN THE DOOR

would be ignored by the lexer.

The output of this stage is a flat stream of tokens that can be passed to the next stage.

Syntax Analysis Stage#

The syntax analysis stage takes the token stream and ensures it follows the grammar of the language.

In this case, the compiler would recognise structures such as:

  • PROGRAM blocks
  • DECLARE sections
  • PROCESS sections
  • Individual commands like SET

The result is a structured representation of the program, such as a parse tree or a set of command objects.

At this point, the compiler understands the shape of the program, but not whether it is valid.

Semantic Analysis Stage#

This is where the compiler starts enforcing meaning.

For the example above, checks might include:

  • Ensuring DOOROPEN has been declared
  • Verifying it is defined as an OUTPUT
  • Confirming that SET is valid for that type
  • Checking that the program structure is complete (e.g. BEGIN/END blocks)

If any of these checks fail, the compiler should produce a clear and actionable error.

This stage is likely to be one of the most important parts of the compiler, as it prevents invalid programs from reaching the device.

Intermediate Representation#

Once validated, the program can be converted into a simpler internal form.

Rather than working with raw tokens or source text, the compiler might represent the program as a list of instructions such as:

  • SET_OUTPUT pin=0 value=HIGH

This makes it easier to process and eventually convert into bytecode.

Optimisation#

At this stage, simple improvements can be applied where appropriate.

Given the simplicity of the DSL, this stage may initially do very little. However, it provides a place to introduce improvements later, such as:

  • Removing redundant instructions
  • Combining repeated operations
  • Precomputing constant values

For now, it may simply pass the instructions through unchanged.

Code Generation#

Finally, the compiler converts the internal representation into bytecode that the device can execute.

For example, the command:

SET DOOROPEN HIGH

might become a compact instruction such as:

  • [OP_SET][PIN 0][VALUE 1]

This bytecode can then be stored or transmitted to the sequencer, where it is executed deterministically.

Final Thoughts#

At this stage, the focus is not on building a highly optimised compiler, but on creating something predictable and easy to reason about.

Because this system is intended to drive real hardware, correctness and determinism are far more important than clever optimisations. A simple, well-structured compiler is far more valuable than a complex one that is difficult to debug.

As the language evolves, additional features such as variables, timing controls, or more advanced sequencing may introduce new requirements. By keeping the compiler structured from the beginning, these changes can be introduced incrementally without needing to rewrite the entire system.

If you need custom control hardware, programmable timing systems, or embedded firmware designed for your automation needs, discover how we build solutions at Protoworks (NI).