Lexical Descriptions

Describe how scientific exponent notation can be used within real literals.

A common notation would be 1.6e3 for 1.6 × 10³, which the following ExponentDesc describes:

{-# LANGUAGE OverloadedLists #-} -- Lets us write [a] to generate a singleton Set containing a.
usualNotation :: ExponentDesc
usualNotation = ExponentsSupported
  { compulsory = False
  , chars = ['e']  -- The letter 'e' separates the significand from the exponent
  , base  = 10   -- The base of the exponent is 10, so that 2.3e5 means 2.3 × 10⁵
  , expSign = PlusOptional -- A positive exponent does not need a plus sign, but can have one.
  , expLeadingZerosAllowd = True -- We allow leading zeros on exponents; so 1.2e005 is valid.
  }

Prescribes whether or not numeric literals can be broken up by a specific symbol.

For example, can one write 300.2_3?

This type describes how whitespace and comments should be handled lexically.

Describes how numeric escape sequences should work for a given base.

This type describes the aggregation of a bunch of different sub-configurations for lexing a specific language.

See the plain smart constructor to define a LexicalDesc.

This type describes how numeric literals (integers, decimals, hexadecimals, etc...), should be lexically processed.

This type describes how symbols (textual literals in a BNF) should be processed lexically, including keywords and operators.

This includes keywords and (hard) operators that are reserved by the language. For example, in Haskell, "data" is a keyword, and "->" is a hard operator.

See the plainSymbol smart constructor for how to implement a custom name description.

This type describes how name-like things are described lexically.

In particular, this defines which characters will constitute identifiers and operators.

See the plainName smart constructor for how to implement a custom name description.

An optional predicate on characters: if pred :: CharPredicate and pred x = Just True, then the lexer should accept the character x.

Examples

   isAlphaNumPred = Just . isAlphaNum
 

   isCapital = Just . isAsciiUpper
 

Whether or not a plus sign (+) can prefix a numeric literal.

This type describes how to parse string and character literals.

Defines the escape characters, and their meaning.

This includes character escapes (e.g. tabs, carriage returns), and numeric escapes, such as binary (usually "0b") and hexadecimal, "0x".

Describes how many digits a numeric escape sequence is allowed.

This lexical description contains the template plain<...> descriptions defined in this module. See plainName, plainSymbol, plainNumeric, plainText and plainSpace for how this description configures the lexer.

This is a blank name description template, which should be extended to form a custom name description.

In its default state, plainName makes no characters able to be part of an identifier or operator. To change this, one should use record field copies, for example:

myNameDesc :: NameDesc
myNameDesc = plainName
  { identifierStart = myIdentifierStartPredicate
  , identifierLetter = myIdentifierLetterPredicate
  }

myNameDesc with then lex identifiers according to the given predicates.

This is a blank symbol description template, which should be extended to form a custom symbol description.

In its default state, plainSymbol has no keywords or reserved/hard operators. To change this, one should use record field copies, for example:

{-# LANGUAGE OverloadedLists #-} -- This lets us write [a,b] to get a Set containing a and b
                                 -- If you don't want to use this, just use fromList [a,b]
mySymbolDesc :: SymbolDesc
mySymbolDesc = plainSymbol
  { hardKeywords = ["data", "where"]
  , hardOperators = ["->"]
  , caseSensitive = True
  }

mySymbolDesc with then treat data and where as keywords, and -> as a reserved operator.

This is a blank numeric description template, which should be extended to form a custom numeric description.

In its default state, plainNumeric allows for hex-, oct-, and bin-ary numeric literals, with the standard prefixes. To change this, one should use record field copies.

This is a blank text description template, which should be extended to form a custom text description.

In its default state, plainText parses characters as symbols between ' and ', and strings between " and ". To change this, one should use record field copies, for example:

{-# LANGUAGE OverloadedLists #-} -- This lets us write [a,b] to get a Set containing a and b
                                 -- If you don't want to use this, just use fromList [a,b]
myPlainText:: TextDesc
myPlainText= plainText
  { characterLiteralEnd = a
  , stringEnds = [(b, c)]
  }

myPlainText with then parse characters as a single character between a and a, and a string as characters between b and c.

This is a blank whitespace description template, which should be extended to form the desired whitespace descriptions.

In its default state, plainName makes no comments possible, and the only whitespace characters are those defined by isSpace

This is a blank escape description template, which should be extended to form a custom escape description.

In its default state, plainEscape the only escape symbol is a backslash, "\\". To change this, one should use record field copies, for example:

{-# LANGUAGE OverloadedLists #-} -- This lets us write [a,b] to get a Set containing a and b,
                                 -- and [(a,b),(c,d)] for a Map which sends a ↦ b and c ↦ d
myPlainEscape:: EscapeDesc
myPlainEscape= plainEscape
  { literals = a
  , stringEnds = [(b, c)]
  , mapping = [("t",0x0009), ("r",0x000D)]
  , hexadecimalEscape = NumericSupported TODO
  }

myPlainText with then parse characters as a single character between a and a, and a string as characters between b and c.