When developing parsers, it is often crucial to identify the exact byte offset of tokens within the input stream. This capability is particularly valuable for generating precise error messages or for implementing featurse like syntax highlighting and linting.
LALRPOP facilitates this through two dedicated tracking macros: @L and @R. The @L macro binds to the starting byte position of the token appearing immediately to its right. In contrast, @R binds to the index of the first byte following the token located to its left. Both macros yield values of type usize.
The following grammar rule demonstrates how too capture these coordinates to a regular expression match:
MyToken = {
<begin_index:> <content: r=""> <stop_index:> => {
// `begin_index` holds the starting byte offset
// `content` is the matched string slice
// `stop_index` is the offset of the byte immediately following the token
}
}</stop_index:></content:></begin_index:>
These macros are versatile and can be integrated into larger macro definitions within your grammar to handle complex parsing scenarios.
Because @R points to the byte immediately after the token, the pair of indices naturally forms a half-open interval. This allows for the direct construction of a Range<usize>, which is often the preferred format for representing spans in Rust.
TokenSpan: std::ops::Range<usize> = {
<left_boundary:> <val: r=""> <right_boundary:> => {
left_boundary..right_boundary
}
}</right_boundary:></val:></left_boundary:></usize>