Lexical Analysis : Introduction - Tokens,Patterns,Lexeme

Lexical analysis is the first phase of compiler. It is a process of taking Input string of charers and producing sequence of symbols called tokens are lexeme, which may be handled more easily by parser.\r
\r
Interion of lexical analysis with parser is explained.\r
\r
Tokens\r
\r
Tokens are sequence of charer that can be treated as a single logical unit, tokens may be a) identifiers b)keywords c)operators d)special symbols e)constants.\r
\r
Lexemes\r
\r
A lexeme is a sequence of charers, in a source program that is matched by a pattern for token.\r
\r
Pattern Rule of description is a pattern.\r
\r
Attribute for token when more than one pattern, matches a lexeme, lexical analyzer must provide additional information about particular lexeme,that is matched to subsequent phase of compiler.\r
\r
Examples for Patterns,lexemes & tokens are explained.