fix: Allow CommonTokenStream to reset properly by jimidle · Pull Request #4245 · antlr/antlr4

@jimidle

  - There is a flag in CommonTokenStream that indicates whether EOF has
    already been fetched/seen. This was not being reset when a new TokenSource
    was given to an existing CommonTokenStream and there was no Reset() function
    which was unfortunately not exported and not part of the interface for TokenStream.
  - Reset the flag when a new source is provided
  - Export the Reset() function so that a TokenStream can be rewound

Signed-off-by: Jim.Idle <jimi@idle.ws>