Macro Bridges (`parsley.macros`, Scala 3)

The Parser Bridge pattern is a technique for decoupling semantic actions from the parser itself. The parsley.macros package contains a macro that can handle generation of many kinds of bridge automatically to help you to get started using the technique straight away if you wish.



The Scaladoc for this page can be found at parsley.macros.

Motivation

The construction of parser bridges is something that involves a certain amount of boilerplate, despite the benefits they provide. The parsley.templates package demonstrates one way in which this can be mitigated, producing templating bridge traits which can be mixed into a provider of an apply on values in exchange for one on parsers.

However, to avoid making the library too opinionated, the templates package only exposes generators for pure parser bridges, which are those that do not incorporate additional metadata. To deal with metadata, like positions, it is necessary to define your own template bridge traits. In addition, there is still a small amount of boilerplate to be had when using these template traits.

As an example:

import parsley.Parsley
import parsley.templates.PureParserBridge2
import parsley.position.{line, col}

case class Pos(line: Int, col: Int)
object Pos extends PureParserBridge2[Int, Int, Pos]

val pos: Parsley[Pos] = Pos(line, col)

Here, we can see a few areas of boilerplate that are not needed:

There is a mirroring between the types of the arguments to Pos and the type parameters for PureParserBridge2: this has to be kept in sync, which will be enforced by the compiler, but it is annoying to change something in two places. We also have to manually ensure the bridge has the correct arity, in this case 2.
The strategy for the bridge, namely to use a pure-based semantics, is hard-coded in the bridge template. If we wanted to add some metadata (more on that later), we would have to remember to also adjust the PureParserBridge2 to be something else.
We necessarily have to introduce the companion object, where perhaps we didn't need to before. The bridge must be kept in the same file as the bridged class otherwise we will need to implement the apply ourselves.

To help mitigate this, Scala 3 parsley supports a macro-driven bridge generation mechanism that addresses the above three points, and provides other functionality on top of them.

The `bridge` Macro

The bridge macro is exposed via parsley.macros.bridge. It takes two main forms: the first is bridge[T], which creates a bridge for the type T, and bridge[T, S >: T], which constructs a bridge for the type T, but upcasts the result to construct S instead (which is a supertype of T). To see how easy this is to use, here is the previous example again:

import parsley.macros.bridge
val Pos2 = bridge[Pos]

val pos2 = Pos2(line, col)

Here, Pos2 works the same way as our Pos bridge from before, but now all of the three painpoints from above have been addressed. In fact, it is now cleaner to separate the class definition and bridge definitions in different scopes! That is it!

To be a bit more specific, bridge is what is called a transparent macro in Scala, which means it can return a different type depending on what the macro internally generated. In our case, the macro will return something derived from bridges.ErrorBridge, so in this case Pos2: bridges.ParserBridge2[Int, Int, Pos]. If we add or remove arguments, this will automatically change the type generated by the macro, so type inference will sort out the boilerplate.

The bridge macro works on pretty much anything you can throw at it:

case class A(x: Int)
object B
enum C {
    case X(x: Int)
    case Y
}

val a = bridge[A]
val b = bridge[B.type]
val x = bridge[C.X, C]       // the C is optional here, but I prefer to hide enum cases.
val y = bridge[C.Y.type, C]  // the C is optional here, but I prefer to hide enum cases.

This also includes parameterless classes, which always produce ParserSingletonBridges:

class Foo()
class Bar

val foo = bridge[Foo]
val bar = bridge[Bar]

Interestingly, for these, we don't treat them the same way we would objects, which would always return the same thing. Instead, we will actually create a fresh instance (with the fresh combinator) every time they are parsed:

import parsley.character.item
import parsley.syntax.zipped.*
(foo.from(item), foo.from(item)).zipped(_ eq _).parse("aa")
// res0: Result[String, Boolean] = Success(x = false)
(bar.from(item), bar.from(item)).zipped(_ eq _).parse("aa")
// res1: Result[String, Boolean] = Success(x = false)

This is not something that the pure templated bridge traits can achieve.

Default Parameters

Some classes might have default parameters that are not relevant to the parser, but that will be relevant for wherever that data is consumed. For example, suppose we are parsing identifiers, which will later be assigned a type: during parse time, there is clearly no type to be assigned, so it would be good if we could "forget" this for the sake of clean parsing. The bridge macro can handle this:

enum Type { case NoType }

case class Ident(name: String)(val ty: Type = Type.NoType)

val ident: parsley.bridges.ParserBridge1[String, Ident] = bridge[Ident]

Here, the ident bridge only needs a parser for the String, and will place Type.NoType into the Ident as it constructs it. A limitation of this currently is that default parameters must not be in the first set of constructor parameters for the class (i.e. notice that constructor is curried). This can be lifted in future.

Metadata Parsing

One area which the template traits fall short is that you need to construct new templates by hand when you want to do anything other than pure bridging. This is because there is a vast design space now possible.

Let's suppose we wanted to add position information to our Ident class above. Do we want it to go as the first parameter? in the second set of parameters? At the end of the first set? So many choices are possible here, and so parsley doesn't provide any default template trait for it. However, the bridge macro does have the ability to handle this as long as the user annotates exactly what is metadata:

import parsley.macros.{bridge, isMeta}

case class Ident(name: String)(@isMeta val pos: Pos, val ty: Type = Type.NoType)

val ident = bridge[Ident]
// error:
// attribute pos can only use @isMeta with a `parsley.macros.ParsableMetadata[Pos]` instance in scope
// case class Ident(name: String)(@isMeta val pos: Pos, val ty: Type = Type.NoType)
//                                            ^

As you can see from the above error message, when we use @isMeta to denote something is parser-produced metadata, we need to tell parsley how we expect it to be parsed. This is done via the ParsableMetadata typeclass. Let's create one of those:

import parsley.macros.ParsableMetadata

given ParsableMetadata[Pos] {
    val meta = pos
}

By doing this, we provide a meta parser, which is then what the bridge macro will use to parse the metadata to add into the class later. By default, parsley provides a ParsableMetadata[(Int, Int)] instance for line and column pairs.

import parsley.macros.{bridge, isMeta}

case class Ident(name: String)(@isMeta val pos: Pos, val ty: Type = Type.NoType)

val ident = bridge[Ident]

This now works. The order in which this metadata is parsed, relative to the name parser, will always be first. If there is more than one @isMeta used, they will be parsed in order of appearance, ahead of all other arguments. It remains future work to allow for an annotation that allows us to denote after which bridge parameter the metadata is parsed.

Polymorpic Bridges

When using template bridge traits, one limitation is that if the class you want to bridge is polymorphic, the bridge traits can't be mixed into the companion object! Suppose you tried:

class Poly[A](val x: A)
object Poly extends parsley.templates.PureParserBridge1[A, Poly[A]]
// error:
// Not found: type A
// object Poly extends parsley.templates.PureParserBridge1[A, Poly[A]]
//                                                         ^
// error:
// Not found: type A
// object Poly extends parsley.templates.PureParserBridge1[A, Poly[A]]
//                                                                 ^

Oops! Ok, so what if we just fix A to be a specific type?

class Poly[A](val x: A)
object Poly extends parsley.templates.PureParserBridge1[Int, Poly[Int]]
// error:
// object creation impossible, since def apply(x1: T1): R in trait PureParserBridge1 in package parsley.templates is not defined 
// (Note that
//  parameter T1 in def apply(x1: T1): R in trait PureParserBridge1 in package parsley.templates does not match
//  parameter parsley.Parsley[Int] in def apply(x1: parsley.Parsley[T1]): parsley.Parsley[R] in trait PureParserBridge1 in package parsley.templates
//  )
// object Poly extends parsley.templates.PureParserBridge1[Int, Poly[Int]]
//        ^

It wants an apply in the companion object that works on Int, but obviously the one it sees works generically on A. Basically, to use the templates here, we need to make it its own object, which also needs implementing apply by hand.

Thankfully, bridge is undeterred:

class Poly[A](val x: A)

def polyA[A] = bridge[Poly[A]]
val polyInt = bridge[Poly[Int]]

No problems either way. In fact, this even works with the @isMeta annotation:

class PolyMeta[A](@isMeta val x: A)

def polyMetaA[A: ParsableMetadata] = bridge[PolyMeta[A]]
val polyMetaPos = bridge[PolyMeta[Pos]]

Error Messages

The parser bridge traits all extend an ErrorBridge, which allows for the integration of labels and reasons into bridges. The template bridge traits inherit these, so the extender of the trait can override labels and reason and get the right behaviours while keeping them out of the parser. This is nice, and the macro supports it under a slightly different mechanism.

The bridge we've seen so far is actually not a function, but an object with an apply[T] and apply[T, S]: let's call this a bridge synthesiser. It also has two other methods: label and explain, which themselves return a new bridge synthesiser that can incorporate those components. If you use label, the synthesiser it returns does not itself have a label method, but does have an explain (and vice-versa), so you can build these up "Builder Pattern"-style. As an example:

import parsley.character.digit
case class Num(n: Int)

val num = bridge.label("number").explain("numbers are made of 1 or more digits")[Num]

num(digit.foldLeft1(0)((n, d) => n * 10 + d.asDigit)).parse("a")
// res5: Result[String, Num] = Failure((line 1, column 1):
//   unexpected "a"
//   expected number
//   numbers are made of 1 or more digits
//   >a
//    ^)

Limitations

Unfortunately, the bridge macro is not perfect. There are still some areas where the template bridge traits manage to allow slightly more flexibility. The key is that the bridge macro forces a specific shape of bridge using all the provided arguments to the given types primary constructor as-is. Any modifications you might want to make to these, perhaps to adapt to another type, will not be picked up.

For example, disambiguation bridges (as described in other pages) involve processing data into potentially different sibling types. The bridge macro can't figure this out, so that's a use-case that is ruled out.

As another example, validation bridges need to inject filtering logic into the bridge, which is not currently supported. However, this is something that I believe can be made to work (unlike the disambiguation bridges above), so this may be supported by some more synthesisers at a later point.

Macro Bridges (parsley.macros, Scala 3)

Motivation

The bridge Macro

Default Parameters

Metadata Parsing

Polymorpic Bridges

Error Messages

Limitations

Macro Bridges (`parsley.macros`, Scala 3)

The `bridge` Macro