Contents
The lexical analysis is independent of the syntax parsing and the semantic analysis. The lexical analyzer splits the source text into tokens. The lexical grammar describes the syntax of these tokens. The grammar is designed to be suitable for high-speed scanning and to facilitate the implementation of a correct scanner. It has a minimum of special case rules and there is only one phase of translation.
Source Text
SourceFile: ByteOrderMark Moduleopt Shebang Moduleopt Moduleopt
ByteOrderMark: \uFEFF Shebang: #! Charactersopt EndOfShebang EndOfShebang: \u000A EndOfFile
Source text can be encoded as any one of the following:
- ASCII (strictly, 7-bit ASCII)
- UTF-8
- UTF-16BE
- UTF-16LE
- UTF-32BE
- UTF-32LE
One of the following UTF BOMs (Byte Order Marks) can be present at the beginning of the source text:
| Format | BOM |
|---|---|
| UTF-8 | EF BB BF |
| UTF-16BE | FE FF |
| UTF-16LE | FF FE |
| UTF-32BE | 00 00 FE FF |
| UTF-32LE | FF FE 00 00 |
| ASCII | no BOM |
If the source file does not begin with a BOM, then the first character must be less than or equal to U+0000007F.
The source text is decoded from its source representation into Unicode Characters. The Characters are further divided into: WhiteSpace, EndOfLine, Comments, SpecialTokenSequences, and Tokens, with the source terminated by an EndOfFile.
The source text is split into tokens using the maximal munch algorithm, i.e., the lexical analyzer assumes the longest possible token. For example, >> is a right-shift token rather than two greater-than tokens. There are two exceptions to this rule:
- A .. embedded between what appear to be two floating point literals, as in 1..2, is interpreted as if the .. were separated by a space from the first integer.
- A 1.a is interpreted as the three tokens 1, ., and a, whereas 1. a is interpreted as the two tokens 1. and a.
Character Set
Character: any Unicode character
End of File
EndOfFile: physical end of the file \u0000 \u001A
The source text is terminated by whichever comes first.
End of Line
EndOfLine: \u000D \u000A \u000D \u000A \u2028 \u2029 EndOfFile
White Space
WhiteSpace: Space Space WhiteSpace Space: \u0020 \u0009 \u000B \u000C
Comments
Comment: BlockComment LineComment NestingBlockComment BlockComment: /* Charactersopt */ LineComment: // Charactersopt EndOfLine NestingBlockComment: /+ NestingBlockCommentCharactersopt +/ NestingBlockCommentCharacters: NestingBlockCommentCharacter NestingBlockCommentCharacter NestingBlockCommentCharacters NestingBlockCommentCharacter: Character NestingBlockComment Characters: Character Character Characters
There are three kinds of comments:
- Block comments can span multiple lines, but do not nest.
- Line comments terminate at the end of the line.
- Nesting block comments can span multiple lines and can nest.
The contents of strings and comments are not tokenized. Consequently, comment openings occurring within a string do not begin a comment, and string delimiters within a comment do not affect the recognition of comment closings and nested /+ comment openings. With the exception of /+ occurring within a /+ comment, comment openings within a comment are ignored.
a = 1; a = " +/ 1"; a = */ 3;
Comments cannot be used as token concatenators, for example, abc/**/def is two tokens, abc and def, not one abcdef token.
Tokens
Tokens: Token Token Tokens Token: { } TokenNoBraces TokenNoBraces: Identifier StringLiteral InterpolationExpressionSequence CharacterLiteral IntegerLiteral FloatLiteral Keyword / /= . .. ... & &= && | |= || - -= -- + += ++ < <= << <<= > >= >>= >>>= >> >>> ! != ( ) [ ] ? , ; : $ = == * *= % %= ^ ^= ^^ ^^= ~ ~= @ =>
Identifiers
Identifier: IdentifierStart IdentifierStart IdentifierChars IdentifierChars: IdentifierChar IdentifierChar IdentifierChars IdentifierStart: _ Letter UniversalAlpha IdentifierChar: IdentifierStart 0 NonZeroDigit
Identifiers start with a letter, _, or universal alpha, and are followed by any number of letters, _, digits, or universal alphas. Universal alphas are as defined in ISO/IEC 9899:1999(E) Appendix D of the C99 Standard. Identifiers can be arbitrarily long, and are case sensitive.
Implementation Defined: Identifiers starting with __ (two underscores) are reserved.
String Literals
StringLiteral: WysiwygString AlternateWysiwygString DoubleQuotedString DelimitedString TokenString HexString
A string literal is either a wysiwyg quoted string, a double quoted string, a delimited string, a token string, or a hex string.
In all string literal forms, an EndOfLine is regarded as a single \n character.
Wysiwyg Strings
WysiwygString: r" WysiwygCharactersopt " StringPostfixopt AlternateWysiwygString: ` WysiwygCharactersopt ` StringPostfixopt WysiwygCharacters: WysiwygCharacter WysiwygCharacter WysiwygCharacters WysiwygCharacter: Character EndOfLine
Wysiwyg ("what you see is what you get") quoted strings can be defined using either of two syntaxes.
In the first form, they are enclosed between r" and ". All characters between the r" and " are part of the string. There are no escape sequences inside wysiwyg strings.
r"I am Oz" r"c:\games\Sudoku.exe" r"ab\n"
Alternatively, wysiwyg strings can be enclosed by backquotes, using the ` character.
`the Great and Powerful.` `c:\games\Empire.exe` `The "lazy" dog` `a"b\n`
See also InterpolatedWysiwygLiteral
Double Quoted Strings
DoubleQuotedString: " DoubleQuotedCharactersopt " StringPostfixopt DoubleQuotedCharacters: DoubleQuotedCharacter DoubleQuotedCharacter DoubleQuotedCharacters DoubleQuotedCharacter: Character EscapeSequence EndOfLine
Double quoted strings are enclosed by "". EscapeSequences can be embedded in them.
"Who are you?" "c:\\games\\Doom.exe" "ab\n" "ab "
See also InterpolatedDoubleQuotedLiteral
Delimited Strings
DelimitedString: q" Delimiter WysiwygCharactersopt MatchingDelimiter " StringPostfixopt q"( ParenDelimitedCharactersopt )" StringPostfixopt q"[ BracketDelimitedCharactersopt ]" StringPostfixopt q"{ BraceDelimitedCharactersopt }" StringPostfixopt q"< AngleDelimitedCharactersopt >" StringPostfixopt Delimiter: Identifier MatchingDelimiter: Identifier ParenDelimitedCharacters: WysiwygCharacter WysiwygCharacter ParenDelimitedCharacters ( ParenDelimitedCharactersopt ) BracketDelimitedCharacters: WysiwygCharacter WysiwygCharacter BracketDelimitedCharacters [ BracketDelimitedCharactersopt ] BraceDelimitedCharacters: WysiwygCharacter WysiwygCharacter BraceDelimitedCharacters { BraceDelimitedCharactersopt } AngleDelimitedCharacters: WysiwygCharacter WysiwygCharacter AngleDelimitedCharacters < AngleDelimitedCharactersopt >
Delimited strings use various forms of delimiters. The delimiter, whether a character or identifier, must immediately follow the " without any intervening whitespace. The terminating delimiter must immediately precede the closing " without any intervening whitespace. A nesting delimiter nests, and is one of the following characters:
| Delimiter | Matching Delimiter |
|---|---|
| [ | ] |
| ( | ) |
| < | > |
| { | } |
q"(foo(xxx))" q"[foo{]"
If the delimiter is an identifier, the identifier must be immediately followed by a newline, and the matching delimiter must be the same identifier starting at the beginning of the line:
writeln(q"EOS
This
is a multi-line
heredoc string
EOS"
);
The newline following the opening identifier is not part of the string, but the last newline before the closing identifier is part of the string. The closing identifier must be placed on its own line at the leftmost column.
Otherwise, the matching delimiter is the same as the delimiter character:
q"/foo]/"
Token Strings
TokenString: q{ TokenStringTokensopt } StringPostfixopt TokenStringTokens: TokenStringToken TokenStringToken TokenStringTokens TokenStringToken: TokenNoBraces { TokenStringTokensopt }
Token strings open with the characters q{ and close with the token }. In between must be valid D tokens. The { and } tokens nest. The string is formed of all the characters between the opening and closing of the token string, including comments.
q{this is the voice of} q{/*}*/ } q{ world(q{control}); } q{ __TIME__ }
See also InterpolatedTokenLiteral
Hex Strings
HexString: x" HexStringCharsopt " StringPostfixopt HexStringChars: HexStringChar HexStringChar HexStringChars HexStringChar: HexDigit WhiteSpace EndOfLine
Hex strings allow string literals to be created using hex data. The hex data need not form valid UTF characters.
x"0A" x"00 FBCD 32FD 0A"
Whitespace and newlines are ignored, so the hex data can be easily formatted. The number of hex characters must be a multiple of 2.
String Postfix
StringPostfix: c w d
The optional StringPostfix character gives a specific type to the string, rather than it being inferred from the context. The types corresponding to the postfix characters are:
| Postfix | Type | Alias |
|---|---|---|
| c | immutable(char)[] | string |
| w | immutable(wchar)[] | wstring |
| d | immutable(dchar)[] | dstring |
"hello"c "hello"w "hello"d
The string literals are assembled as UTF-8 char arrays, and the postfix is applied to convert to wchar or dchar as necessary as a final step.
Escape Sequences
EscapeSequence: \' \" \? \\ \0 \a \b \f \n \r \t \v \x HexDigit HexDigit \ OctalDigit \ OctalDigit OctalDigit \ OctalDigit OctalDigit OctalDigit \u HexDigit HexDigit HexDigit HexDigit \U HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit HexDigit \ NamedCharacterEntity OctalDigit: 0 1 2 3 4 5 6 7
| Sequence | Meaning |
|---|---|
| \' | Literal single-quote: ' |
| \" | Literal double-quote: " |
| \? | Literal question mark: ? |
| \\ | Literal backslash: \ |
| \0 | Binary zero (NUL, U+0000). |
| \a | BEL (alarm) character (U+0007). |
| \b | Backspace (U+0008). |
| \f | Form feed (FF) (U+000C). |
| \n | End-of-line (U+000A). |
| \r | Carriage return (U+000D). |
| \t | Horizontal tab (U+0009). |
| \v | Vertical tab (U+000B). |
| \xnn | Byte value in hexadecimal, where nn is
specified as two hexadecimal digits. For example: \xFF represents the character with the value 255. See also: std.conv.hexString. |
| \n \nn \nnn | Byte value in
octal. For example: \101 represents the character with the value 65 ('A'). Analogous to hexadecimal characters, the largest byte value is \377 (= \xFF in hexadecimal or 255 in decimal) See also: std.conv.octal. |
| \unnnn | Unicode character U+nnnn, where
nnnn are four hexadecimal digits. For example, \u03B3 represents the Unicode character γ (U+03B3 - GREEK SMALL LETTER GAMMA). |
| \Unnnnnnnn | Unicode character U+nnnnnnnn,
where nnnnnnnn are 8 hexadecimal digits. For example, \U0001F603 represents the Unicode character U+1F603 (SMILING FACE WITH OPEN MOUTH). |
| \name | Named character entity from the HTML5
specification. These names begin with & and end with ;, e.g., €. See NamedCharacterEntity. |
Character Literals
CharacterLiteral: ' SingleQuotedCharacter ' SingleQuotedCharacter: Character EscapeSequence
Character literals are a single character or escape sequence enclosed by single quotes.
'h' '\n' '\\'
A character literal resolves to one of type char, wchar, or dchar (see Basic Data Types).
- If the literal is a \u escape sequence, it resolves to type wchar.
- If the literal is a \U escape sequence, it resolves to type dchar.
Otherwise, it resolves to the type with the smallest size it will fit into.
Integer Literals
IntegerLiteral: Integer Integer IntegerSuffix Integer: DecimalInteger BinaryInteger HexadecimalInteger IntegerSuffix: L u U Lu LU uL UL
DecimalInteger: 0 Underscoresopt NonZeroDigit NonZeroDigit DecimalDigitsUS Underscores: _ Underscores _ NonZeroDigit: 1 2 3 4 5 6 7 8 9 DecimalDigits: DecimalDigit DecimalDigit DecimalDigits DecimalDigitsUS: DecimalDigitUS DecimalDigitUS DecimalDigitsUS DecimalDigitsNoSingleUS: DecimalDigitsUSopt DecimalDigit DecimalDigitsUSopt DecimalDigitsNoStartingUS: DecimalDigit DecimalDigit DecimalDigitsUS DecimalDigit: 0 NonZeroDigit DecimalDigitUS: DecimalDigit _
BinaryInteger: BinPrefix BinaryDigitsNoSingleUS BinPrefix: 0b 0B BinaryDigitsNoSingleUS: BinaryDigitsUSopt BinaryDigit BinaryDigitsUSopt BinaryDigitsUS: BinaryDigitUS BinaryDigitUS BinaryDigitsUS BinaryDigit: 0 1 BinaryDigitUS: BinaryDigit _
HexadecimalInteger: HexPrefix HexDigitsNoSingleUS HexDigits: HexDigit HexDigit HexDigits HexDigitsUS: HexDigitUS HexDigitUS HexDigitsUS HexDigitsNoSingleUS: HexDigitsUSopt HexDigit HexDigitsUSopt HexDigitsNoStartingUS: HexDigit HexDigit HexDigitsUS HexDigit: DecimalDigit HexLetter HexDigitUS: HexDigit _ HexLetter: a b c d e f A B C D E F
Integers can be specified in decimal, binary, or hexadecimal.
- Decimal integers are a sequence of decimal digits.
- Binary integers are a sequence of binary digits preceded by a ‘0b’ or ‘0B’.
- C-style octal integer notation (e.g. 0167) was deemed too easy to mix up with decimal notation; it is only fully supported in string literals. D still supports octal integer literals interpreted at compile time through the std.conv.octal template, as in octal!167.
- Hexadecimal integers are a sequence of hexadecimal digits preceded by a ‘0x’ or ‘0X’.
10 0b1010 0xA
Integers can have embedded ‘_’ characters after a digit to improve readability, which are ignored.
20_000 867_5309 1_522_000 0xBAAD_F00D
Integers can be immediately followed by one ‘L’ or one of ‘u’ or ‘U’ or both. Note that there is no ‘l’ suffix.
The type of the integer is resolved as follows:
| Literal | Type | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Usual decimal notation | |||||||||
| 0 .. 2_147_483_647 | int | ||||||||
| 2_147_483_648 .. 9_223_372_036_854_775_807 | long | ||||||||
| 9_223_372_036_854_775_808 .. 18_446_744_073_709_551_615 | ulong | ||||||||
| Explicit suffixes | |||||||||
| 0L .. 9_223_372_036_854_775_807L | long | ||||||||
| 0U .. 4_294_967_295U | uint | ||||||||
| 4_294_967_296U .. 18_446_744_073_709_551_615U | ulong | ||||||||
| 0UL .. 18_446_744_073_709_551_615UL | ulong | ||||||||
| Hexadecimal notation | |||||||||
| 0x0 .. 0x7FFF_FFFF | int | ||||||||
| 0x8000_0000 .. 0xFFFF_FFFF | uint | ||||||||
| 0x1_0000_0000 .. 0x7FFF_FFFF_FFFF_FFFF | long | ||||||||
| 0x8000_0000_0000_0000 .. 0xFFFF_FFFF_FFFF_FFFF | ulong | ||||||||
| Hexadecimal notation with explicit suffixes | |||||||||
| 0x0L .. 0x7FFF_FFFF_FFFF_FFFFL | long | ||||||||
| 0x8000_0000_0000_0000L .. 0xFFFF_FFFF_FFFF_FFFFL | ulong | ||||||||
| 0x0U .. 0xFFFF_FFFFU | uint | ||||||||
| 0x1_0000_0000U .. 0xFFFF_FFFF_FFFF_FFFFU | ulong | ||||||||
| 0x0UL .. 0xFFFF_FFFF_FFFF_FFFFUL | ulong | ||||||||
An integer literal may not exceed these values.
Best Practices: Octal integer notation is not supported for integer literals. However, octal integer literals can be interpreted at compile time through the std.conv.octal template, as in octal!167.
Floating Point Literals
FloatLiteral: Float Suffixopt Integer FloatSuffix ImaginarySuffixopt Integer RealSuffixopt ImaginarySuffix Float: DecimalFloat HexFloat DecimalFloat: LeadingDecimal . DecimalDigitsNoStartingUSopt LeadingDecimal . DecimalDigitsNoStartingUS DecimalExponent . DecimalDigitsNoStartingUS DecimalExponentopt LeadingDecimal DecimalExponent DecimalExponent: DecimalExponentStart DecimalDigitsNoSingleUS DecimalExponentStart: e E e+ E+ e- E- HexFloat: HexPrefix HexDigitsNoSingleUS . HexDigitsNoStartingUS HexExponent HexPrefix . HexDigitsNoStartingUS HexExponent HexPrefix HexDigitsNoSingleUS HexExponent HexPrefix HexExponent HexPrefix: 0x 0X HexExponent: HexExponentStart DecimalDigitsNoSingleUS HexExponentStart: p P p+ P+ p- P- Suffix: FloatSuffix ImaginarySuffixopt RealSuffix ImaginarySuffixopt ImaginarySuffix FloatSuffix: f F RealSuffix: LImaginarySuffix: i LeadingDecimal: DecimalInteger 0 DecimalDigitsNoSingleUS
Floats can be in decimal or hexadecimal format, and must have at least one digit and either a decimal point, an exponent, or a FloatSuffix.
Decimal floats can have an exponent which is e or E followed by a decimal number serving as the exponent of 10.
-1.0 1e2 1e-2 -1.175494351e-38F
Hexadecimal floats are preceded by a 0x or 0X and the exponent is a p or P followed by a decimal number serving as the exponent of 2.
0xAp0 0x1p2 0x1.FFFFFFFFFFFFFp1023 0x1p-52
Floating literals can have embedded _ characters after a digit to improve readability, which are ignored.
2.645_751 6.022140857E+23 6_022.140857E+20 6_022_.140_857E+20_
- Floating literals with no suffix are of type double.
- Floating literals followed by f or F are of type float.
- Floating literals followed by L are of type real.
0.0 0F 0.0L
The literal may not exceed the range of the type. The literal is rounded to fit into the significant digits of the type.
If a floating literal has a . and a type suffix, at least one digit must be in-between:
1f; 1.f; 1.;
Note: Floating literals followed by i to denote imaginary floating point values have been deprecated.
Keywords
Keywords are reserved identifiers.
Keyword: abstract alias align asm assert autobodybool break byte case cast catchcdoublecentcfloatchar class const continuecrealdchar debug default delegatedeletedeprecated do double else enum export extern false final finally float for foreach foreach_reverse function gotoidoubleififloatimmutable import in inout int interface invariantirealis lazy long macro mixin module new nothrow null out override package pragma private protected public pure real ref return scope shared short static struct super switch synchronized template this throw true try typeid typeof ubyteucentuint ulong union unittest ushort version void wchar while with __FILE__ __FILE_FULL_PATH__ __FUNCTION__ __LINE__ __MODULE__ __PRETTY_FUNCTION__ __gshared __parameters __rvalue __traits __vector
Special Tokens
These tokens are replaced with other tokens according to the following table:
| Special Token | Replaced with |
|---|---|
| __DATE__ | string literal of the date of compilation "mmm dd yyyy" |
| __EOF__ | tells the scanner to ignore everything after this token |
| __TIME__ | string literal of the time of compilation "hh:mm:ss" |
| __TIMESTAMP__ | string literal of the date and time of compilation "www mmm dd hh:mm:ss yyyy" |
| __VENDOR__ | Compiler vendor string |
| __VERSION__ | Compiler version as an integer |
Implementation Defined: The replacement string literal for __VENDOR__ and the replacement integer value for __VERSION__.
Special Token Sequences
SpecialTokenSequence: # line IntegerLiteral Filespecopt EndOfLine # line __LINE__ Filespecopt EndOfLine
Filespec: " DoubleQuotedCharactersopt "
Special token sequences are processed by the lexical analyzer, may appear between any other tokens, and do not affect the syntax parsing.
Special token sequences are terminated by the first newline that follows the first # token at the beginning of the sequence.
There is currently only one special token sequence, #line.
This sets the line number of the next source line to IntegerLiteral, and optionally the current source file name to Filespec, beginning with the next line of source text.
For example:
int #line 6 "pkg/mod.d"
x;
Implementation Defined: The source file and line number is typically used for printing error messages and for mapping generated code back to the source for the symbolic debugging output.
Copyright © 1999-2026 by the D Language Foundation | Page generated by Ddoc on Thu Jan 15 22:47:58 2026