How does a lexer return a semantic value that the parser uses?

Question

Is it always necessary to do so? What does it look like?

Answer 1

Lexer don't care about semantic the only mission in life for lexers is to convert the source code (stream of characters) into tokens each has this form <Token_type, Information_related_to_token> the information maybe the value of the token (string), the name of the operator (=) ...

Tokens then are sent to a parser that deals with syntactic analysis. as a side job a lexer can create a symbols table.

Answer 2

In yacc, your lexer gets a global variable named yylval which is a C union. Back in yacc, this becomes the value for $1, $2, etc.

Answer 3

Lexers don't deal with semantics, they only deal with turning a stream of characters into tokens (sequences of characters that have meaning to the compiler). Semantics are determined during syntactic analysis. See this answer to a previous question for further details on the stages of compilation.