1.NounThe conversion of a stream of characters to a stream of meaningful tokens; normally to simplify parsing.
"While it's often not difficult to identify tokens while parsing, having a separate stage for lexical analysis simplifies the structure of your compiler."