Struct TokenStream
1.15.0 · Source
pub struct TokenStream(/* private fields */);Expand description
The main type provided by this crate, representing an abstract stream of tokens, or, more specifically, a sequence of token trees. The type provides interfaces for iterating over those token trees and, conversely, collecting a number of token trees into one stream.
This is both the input and output of #[proc_macro], #[proc_macro_attribute]
and #[proc_macro_derive] definitions.
Source§
1.29.0 · Source
Returns an empty TokenStream containing no token trees.
1.29.0 · Source
Checks if this TokenStream is empty.
Source 🔬This is a nightly-only experimental API. (proc_macro_expand #90765)
proc_macro_expand #90765)Parses this TokenStream as an expression and attempts to expand any
macros within it. Returns the expanded TokenStream.
Currently only expressions expanding to literals will succeed, although this may be relaxed in the future.
NOTE: In error conditions, expand_expr may leave macros unexpanded,
report an error, failing compilation, and/or return an Err(..). The
specific behavior for any error condition, and what conditions are
considered errors, is unspecified and may change in the future.
1.15.0 · Source§Prints the token stream as a string that is supposed to be losslessly convertible back
into the same token stream (modulo spans), except for possibly TokenTree::Groups
with Delimiter::None delimiters and negative numeric literals.
Prints the token stream as a string that is supposed to be losslessly convertible back
into the same token stream (modulo spans), except for possibly TokenTree::Groups
with Delimiter::None delimiters and negative numeric literals.
Note: the exact form of the output is subject to change, e.g. there might
be changes in the whitespace used between tokens. Therefore, you should
not do any kind of simple substring matching on the output string (as
produced by to_string) to implement a proc macro, because that matching
might stop working if such changes happen. Instead, you should work at the
TokenTree level, e.g. matching against TokenTree::Ident,
TokenTree::Punct, or TokenTree::Literal.
1.92.0 · Source§
1.92.0 · Source§
1.92.0 · Source§
1.92.0 · Source§
1.30.0 · Source§
1.30.0 · Source§
1.15.0 · Source§A “flattening” operation on token streams, collects token trees
from multiple token streams into a single stream.
A “flattening” operation on token streams, collects token trees from multiple token streams into a single stream.
1.15.0 · Source§Attempts to break the string into tokens and parse those tokens into a token stream.
May fail for a number of reasons, for example, if the string contains unbalanced delimiters
or characters not existing in the language.
All tokens in the parsed stream get Span::call_site() spans.
Attempts to break the string into tokens and parse those tokens into a token stream.
May fail for a number of reasons, for example, if the string contains unbalanced delimiters
or characters not existing in the language.
All tokens in the parsed stream get Span::call_site() spans.
NOTE: some errors may cause panics instead of returning LexError. We reserve the right to
change these errors into LexErrors later.