The text of the program is kept in units called source files in this document.

A source file together with all the headers and source files included via the preprocessing directive #include, less any source lines skipped by any of the conditional inclusion ([cpp.cond]) preprocessing directives, as modified by the implementation-defined behavior of any conditionally-supported-directives ([cpp.pre]) and pragmas ([cpp.pragma]), if any, is called a preprocessing translation unit.

Characters in a character-literal other than a multicharacter or non-encodable character literal or in a string-literal are encoded as a sequence of one or more code units, as determined by the encoding-prefix ([lex.ccon], [lex.string]); this is termed the respective literal encoding.

The ordinary literal encoding is the encoding applied to an ordinary character or string literal.

The wide literal encoding is the encoding applied to a wide character or string literal.

A literal encoding or a locale-specific encoding of one of the execution character sets ([character.seq]) encodes each element of the basic literal character set as a single code unit with non-negative value, distinct from the code unit for any other such element.

The U+0000 null character is encoded as the value 0.

No other element of the translation character set is encoded with a code unit of value 0.

The code unit value of each decimal digit character after the digit 0 (U+0030) is one greater than the value of the previous.

The ordinary and wide literal encodings are otherwise implementation-defined.

For a UTF-8, UTF-16, or UTF-32 literal, the implementation shall encode the Unicode scalar value corresponding to each character of the translation character set as specified in the Unicode Standard for the respective Unicode encoding form.

n-char:
any member of the translation character set except the U+007d right curly bracket or new-line character

The universal-character-name construct provides a way to name any element in the translation character set using just the basic character set.

If a universal-character-name outside the c-char-sequence or s-char-sequence of a character-literal or string-literal (in either case, including within a user-defined-literal) corresponds to a control character or to a character in the basic character set, the program is ill-formed.

A universal-character-name of the form \u hex-quad, \U hex-quad hex-quad, or \u{simple-hexadecimal-digit-sequence} designates the character in the translation character set whose Unicode scalar value is the hexadecimal number represented by the sequence of hexadecimal-digits in the universal-character-name.

The program is ill-formed if that number is not a Unicode scalar value.

A universal-character-name that is a named-universal-character designates the corresponding character in the Unicode Standard (chapter 4.8 Name) if the n-char-sequence is equal to its character name or to one of its character name aliases of type “control”, “correction”, or “alternate”; otherwise, the program is ill-formed.

A preprocessing token is the minimal lexical element of the language in translation phases 3 through 5.

The categories of preprocessing token are: header names, placeholder tokens produced by preprocessing import and module directives (import-keyword, module-keyword, and export-keyword), identifiers, preprocessing numbers, character literals (including user-defined character literals), string literals (including user-defined string literals), preprocessing operators and punctuators, and single non-whitespace characters that do not lexically match the other preprocessing token categories.

If a U+0027 apostrophe, a U+0022 quotation mark, or any character not in the basic character set matches the last category, the program is ill-formed.

Preprocessing tokens can be separated by whitespace; this consists of comments ([lex.comment]), or whitespace characters (U+0020 space, U+0009 character tabulation, new-line, U+000b line tabulation, and U+000c form feed), or both.

Each preprocessing token that is converted to a token ([lex.token]) shall have the lexical form of a keyword, an identifier, a literal, or an operator or punctuator.

The import-keyword is produced by processing an import directive ([cpp.import]), the module-keyword is produced by preprocessing a module directive ([cpp.module]), and the export-keyword is produced by preprocessing either of the previous two directives.

If the input stream has been parsed into preprocessing tokens up to a given character:

• If the next character begins a sequence of characters that could be the prefix and initial double quote of a raw string literal, such as R", the next preprocessing token shall be a raw string literal.

  Between the initial and final double quote characters of the raw string, any transformations performed in phase 2 (line splicing) are reverted; this reversion is applied before any d-char, r-char, or delimiting parenthesis is identified.

  The raw string literal is defined as the shortest sequence of characters that matches the raw-string pattern

  encoding-prefix R raw-string

• Otherwise, if the next three characters are <​::​ and the subsequent character is neither : nor >, the < is treated as a preprocessing token by itself and not as the first character of the alternative token <:.

• Otherwise, if the next three characters are [​::​ and the subsequent character is not :, or if the next three characters are [:>, the [ is treated as a preprocessing token by itself and not as the first character of the preprocessing token [:.

  [Note 3:

  The tokens [: and :] cannot be composed from digraphs.

  — end note]

• Otherwise, the next preprocessing token is the longest sequence of characters that could constitute a preprocessing token, even if that would cause further lexical analysis to fail, except that

  • a string-literal token is never formed when a token can be formed, and
  • a ([lex.header]) is only formed
    • immediately after the include, embed, or import preprocessing token in a #include ([cpp.include]), #embed ([cpp.embed]), or import ([cpp.import]) directive, respectively, or
    • immediately after a preprocessing token sequence of __has_include or __has_embed immediately followed by ( in a #if, #elif, or #embed directive ([cpp.cond], [cpp.embed]).

[Example 1: #define R "x" const char* s = R"y"; — end example]

[Example 2:

The program fragment 0xe+foo is parsed as a preprocessing number token (one that is not a valid integer-literal or floating-point-literal token), even though a parse as three preprocessing tokens 0xe, +, and foo can produce a valid expression (for example, if foo is a macro defined as 1).

Similarly, the program fragment 1E1 is parsed as a preprocessing number (one that is a valid floating-point-literal token), whether or not E is a macro name.

— end example]

[Example 3:

The program fragment x+++++y is parsed as x ++ ++ + y, which, if x and y have integral types, violates a constraint on increment operators, even though the parse x ++ + ++ y can yield a correct expression.

— end example]

h-char:
any member of the translation character set except new-line and U+003e greater-than sign

q-char:
any member of the translation character set except new-line and U+0022 quotation mark

Preprocessing number tokens lexically include all integer-literal tokens ([lex.icon]) and all floating-point-literal tokens ([lex.fcon]).

A preprocessing number does not have a type or a value; it acquires both after a successful conversion to an integer-literal token or a floating-point-literal token.

Alternative token representations are provided for some operators and punctuators.10

There are five kinds of tokens: identifiers, keywords, literals, operators, and other separators.

identifier-start:
nondigit
an element of the translation character set with the Unicode property XID_Start

identifier-continue:
digit
nondigit
an element of the translation character set with the Unicode property XID_Continue

nondigit: one of
a b c d e f g h i j k l m
n o p q r s t u v w x y z
A B C D E F G H I J K L M
N O P Q R S T U V W X Y Z _

digit: one of
0 1 2 3 4 5 6 7 8 9

The identifiers in Table 4 have a special meaning when appearing in a certain context.

When referred to in the grammar, these identifiers are used explicitly rather than using the identifier grammar production.

Unless otherwise specified, any ambiguity as to whether a given identifier has a special meaning is resolved to interpret the token as a regular identifier.

In addition, some identifiers appearing as a token or preprocessing-token are reserved for use by C++ implementations and shall not be used otherwise; no diagnostic is required.

• Each identifier that contains a double underscore __ or begins with an underscore followed by an uppercase letter, other than those specified in this document (for example, __cplusplus ([cpp.predefined])), is reserved to the implementation for any use.

• Each identifier that begins with an underscore is reserved to the implementation for use as a name in the global namespace.

keyword:
any identifier listed in Table 5
import-keyword
module-keyword
export-keyword

There are several kinds of literals.

octal-digit: one of
0 1 2 3 4 5 6 7

nonzero-digit: one of
1 2 3 4 5 6 7 8 9

hexadecimal-prefix: one of
0x 0X

hexadecimal-digit: one of
0 1 2 3 4 5 6 7 8 9
a b c d e f
A B C D E F

unsigned-suffix: one of
u U

long-long-suffix: one of
ll LL

In an integer-literal, the sequence of binary-digits, octal-digits, digits, or hexadecimal-digits is interpreted as a base N integer as shown in Table 7; the lexically first digit of the sequence of digits is the most significant.

The hexadecimal-digits a through f and A through F have decimal values ten through fifteen.

Except for integer-literals containing a size-suffix, if the value of an integer-literal cannot be represented by any type in its list and an extended integer type ([basic.fundamental]) can represent its value, it may have that extended integer type.

If all of the types in the list for the integer-literal are signed, the extended integer type is signed.

If all of the types in the list for the integer-literal are unsigned, the extended integer type is unsigned.

If the list contains both signed and unsigned types, the extended integer type may be signed or unsigned.

If an integer-literal cannot be represented by any of the allowed types, the program is ill-formed.

encoding-prefix: one of
u8  u  U  L

basic-c-char:
any member of the translation character set except the U+0027 apostrophe,
   U+005c reverse solidus, or new-line character

simple-escape-sequence-char: one of
' " ? \ a b f n r t v

conditional-escape-sequence-char:
any member of the basic character set that is not an octal-digit, a simple-escape-sequence-char, or the characters N, o, u, U, or x

A multicharacter literal is a character-literal whose c-char-sequence consists of more than one c-char.

A multicharacter literal shall not have an encoding-prefix.

If a multicharacter literal contains a c-char that is not encodable as a single code unit in the ordinary literal encoding, the program is ill-formed.

Multicharacter literals are conditionally-supported.

In translation phase 4, the value of a character-literal is determined using the range of representable values of the character-literal's type in translation phase 7.

A multicharacter literal has an implementation-defined value.

floating-point-suffix: one of
f l f16 f32 f64 f128 bf16 F L F16 F32 F64 F128 BF16

The significand of a floating-point-literal is the fractional-constant or digit-sequence of a decimal-floating-point-literal or the hexadecimal-fractional-constant or hexadecimal-digit-sequence of a hexadecimal-floating-point-literal.

In the significand, the sequence of digits or hexadecimal-digits and optional period are interpreted as a base N real number s, where N is 10 for a decimal-floating-point-literal and 16 for a hexadecimal-floating-point-literal.

If an exponent-part or binary-exponent-part is present, the exponent e of the floating-point-literal is the result of interpreting the sequence of an optional sign and the digits as a base 10 integer.

Otherwise, the exponent e is 0.

The scaled value of the literal is for a decimal-floating-point-literal and for a hexadecimal-floating-point-literal.

If the scaled value is not in the range of representable values for its type, the program is ill-formed.

Otherwise, the value of a floating-point-literal is the scaled value if representable, else the larger or smaller representable value nearest the scaled value, chosen in an implementation-defined manner.

basic-s-char:
any member of the translation character set except the U+0022 quotation mark,
   U+005c reverse solidus, or new-line character

r-char:
any member of the translation character set, except a U+0029 right parenthesis followed by
   the initial d-char-sequence (which may be empty) followed by a U+0022 quotation mark

d-char:
any member of the basic character set except:
   U+0020 space, U+0028 left parenthesis, U+0029 right parenthesis, U+005c reverse solidus,
   U+0009 character tabulation, U+000b line tabulation, U+000c form feed, and new-line

A string-literal that has an R in the prefix is a raw string literal.

The d-char-sequence serves as a delimiter.

The terminating d-char-sequence of a raw-string is the same sequence of characters as the initial d-char-sequence.

A d-char-sequence shall consist of at most 16 characters.

[Note 1:

The characters '(' and ')' can appear in a raw-string.

Thus, R"delimiter((a|b))delimiter" is equivalent to "(a|b)".

— end note]

[Note 2:

A source-file new-line in a raw string literal results in a new-line in the resulting execution string literal.

Assuming no whitespace at the beginning of lines in the following example, the assert will succeed: const char* p = R"(a\ b c)"; assert(std::strcmp(p, "a\\\nb\nc") == 0);

— end note]

[Example 1:

The raw string R"a( )\ a" )a" is equivalent to "\n)\\\na\"\n".

The raw string R"(x = "\"y\"")" is equivalent to "x = \"\\\"y\\\"\"".

— end example]

Ordinary string literals and UTF-8 string literals are also referred to as narrow string literals.

The string-literals in any sequence of adjacent string-literals shall have at most one unique encoding-prefix among them.

The common encoding-prefix of the sequence is that encoding-prefix, if any.

Evaluating a string-literal results in a string literal object with static storage duration ([basic.stc]).

String literal objects are initialized with the sequence of code unit values corresponding to the string-literal's sequence of s-chars (originally from non-raw string literals) and r-chars (originally from raw string literals), plus a terminating U+0000 null character, in order as follows:

• The sequence of characters denoted by each contiguous sequence of basic-s-chars, r-chars, simple-escape-sequences ([lex.ccon]), and universal-character-names ([lex.charset]) is encoded to a code unit sequence using the string-literal's associated character encoding.

  If a character lacks representation in the associated character encoding, then the program is ill-formed.

  [Note 6:

  No character lacks representation in any Unicode encoding form.

  — end note]

  When encoding a stateful character encoding, implementations should encode the first such sequence beginning with the initial encoding state and encode subsequent sequences beginning with the final encoding state of the prior sequence.

  [Note 7:

  The encoded code unit sequence can differ from the sequence of code units that would be obtained by encoding each character independently.

  — end note]

• Each numeric-escape-sequence ([lex.ccon]) contributes a single code unit with a value as follows:

  • Let v be the integer value represented by the octal number comprising the sequence of octal-digits in an octal-escape-sequence or by the hexadecimal number comprising the sequence of hexadecimal-digits in a hexadecimal-escape-sequence.

  • If v does not exceed the range of representable values of the string-literal's array element type, then the value is v.

  • Otherwise, if the string-literal's encoding-prefix is absent or L, and v does not exceed the range of representable values of the corresponding unsigned type for the underlying type of the string-literal's array element type, then the value is the unique value of the string-literal's array element type T that is congruent to v modulo , where N is the width of T.

  • Otherwise, the program is ill-formed.

  When encoding a stateful character encoding, these sequences should have no effect on encoding state.

• Each conditional-escape-sequence ([lex.ccon]) contributes an implementation-defined code unit sequence.

  When encoding a stateful character encoding, it is implementation-defined what effect these sequences have on encoding state.

An unevaluated-string shall have no encoding-prefix.

Each universal-character-name and each simple-escape-sequence in an unevaluated-string is replaced by the member of the translation character set it denotes.

An unevaluated-string that contains a numeric-escape-sequence or a conditional-escape-sequence is ill-formed.

An unevaluated-string is never evaluated and its interpretation depends on the context in which it appears.

boolean-literal:
false
true

The Boolean literals are the keywords false and true.

Such literals have type bool.

The pointer literal is the keyword nullptr.

It has type std​::​nullptr_t.

If a token matches both user-defined-literal and another literal kind, it is treated as the latter.

The syntactic non-terminal preceding the ud-suffix in a user-defined-literal is taken to be the longest sequence of characters that could match that non-terminal.

A user-defined-literal is treated as a call to a literal operator or literal operator template ([over.literal]).

To determine the form of this call for a given user-defined-literal L with ud-suffix X, first let S be the set of declarations found by unqualified lookup for the literal-operator-id whose literal suffix identifier is X ([basic.lookup.unqual]).

S shall not be empty.

If L is a user-defined-integer-literal, let n be the literal without its ud-suffix.

If S contains a literal operator with parameter type unsigned long long, the literal L is treated as a call of the form operator ""X(nULL)

Otherwise, S shall contain a raw literal operator or a numeric literal operator template ([over.literal]) but not both.

If S contains a raw literal operator, the literal L is treated as a call of the form operator ""X("n")

Otherwise (S contains a numeric literal operator template), L is treated as a call of the form operator ""X<'', '', ... ''>() where n is the source character sequence .

If L is a user-defined-floating-point-literal, let f be the literal without its ud-suffix.

If S contains a literal operator with parameter type long double, the literal L is treated as a call of the form operator ""X(fL)

Otherwise, S shall contain a raw literal operator or a numeric literal operator template ([over.literal]) but not both.

If S contains a raw literal operator, the literal L is treated as a call of the form operator ""X("f")

Otherwise (S contains a numeric literal operator template), L is treated as a call of the form operator ""X<'', '', ... ''>() where f is the source character sequence .

If L is a user-defined-string-literal, let str be the literal without its ud-suffix and let len be the number of code units in str (i.e., its length excluding the terminating null character).

If S contains a literal operator template with a constant template parameter for which str is a well-formed template-argument, the literal L is treated as a call of the form operator ""X<str>()

Otherwise, the literal L is treated as a call of the form operator ""X(str, len)

If L is a user-defined-character-literal, let ch be the literal without its ud-suffix.

S shall contain a literal operator whose only parameter has the type of ch and the literal L is treated as a call of the form operator ""X(ch)

[Example 2: long double operator ""_w(long double); std::string operator ""_w(const char16_t*, std::size_t); unsigned operator ""_w(const char*); int main() { 1.2_w; u"one"_w; 12_w; "two"_w; } — end example]

In translation phase 5 ([lex.phases]), adjacent string-literals are concatenated and user-defined-string-literals are considered string-literals for that purpose.

During concatenation, ud-suffixes are removed and ignored and the concatenation process occurs as described in [lex.string].

At the end of phase 5, if a string-literal is the result of a concatenation involving at least one user-defined-string-literal, all the participating user-defined-string-literals shall have the same ud-suffix and that suffix is applied to the result of the concatenation.

[Example 3: int main() { L"A" "B" "C"_x; "P"_x "Q" "R"_y; } — end example]