Action symbols Semantic record Semantic record

(1)

SEMANTIC PROCESSING SEMANTIC PROCESSING

Chuen-Liang Chen Department of Computer Science

and Information Engineering National Taiwan University

Taipei, TAIWAN

to determine when to call semantic routines

1. <program> →→→→#startbegin <statement list> end 2. <statement list> →→→→<statement> { <statement> } 3. <statement> →→→→<ident> := <expression>#assign ; 4. <statement> →→→→read ( <id list> ) ;

5. <statement> →→→→write ( <expr list> ) ;

6. <id list> →→→→<ident>#read_id{ , <ident> #read_id} 7. <expr list> →→→→<expression>#write_id

{ , <expression>#write_id}

8. <expression> →→→→<primary> { <add op> <primary>#gen_infix} 9. <primary> →→→→( <expression> )

10. <primary> →→→→<ident>

11. <primary> →→→→INTLITERAL #process_literal 12. <add op> →→→→+#process_op

13. <add op> →→→→- #process_op 14. <ident> →→→→ID #process_id

15. <system goal> →→→→<program> SCANEOF#finish

possibly, with some modifications

Action symbols

Action symbols Semantic record Semantic record

to keep semantic information associated with grammar symbol

#define MAXIDLEN 33 typedef char string[MAXIDLEN];

/* for operators */

typedef struct operator {

enum op { PLUS, MINUS } operator;

} op_rec;

/* for <primary> and <expression> */

enum expr { IDEXPR, LITERALEXPR, TEMPEXPR };

typedef struct expression { enum expr kind;

union {

stringname;/* for IDEXPR, TEMPEXPR */

intval; /* for LITERALEXPR */

};

} expr_rec;

Parser + semantic routines Parser + semantic routines

void expression(void) {

token t;

/* <expression> ::= <primary> { <add op> <primary> } */

primary();

for (t = next_token(); t == PLUSOP || t

== MINUSOP; t = next_token()) { add_op();

primary();

} }

void expression (expr_rec *result) {

expr_rec left_operand, right_operand;

op_rec op;

/* <expression> ::= <primary> { <add op>

<primary> #gen_infix} */

primary(&left_operand) while (next_token() == PLUSOP ||

next_token() == MINUSOP) { add_op(&op);

primary(&right_operand);

left_operand = gen_infix(left_operand, op, right_operand);

}

*result = left_operand;

}

QUIZ: where is syntatic structure?

Semantics - meaning Semantics - meaning

syntax : semantics = structure : meaning

implementation of “meaning” --

attributeattached to each node of (abstract) syntax tree

operations on “meaning” --

understand

–associating semantic information (attribute) to each node –initially, on some nodes (leaves, usually)

propagation until “decorated”

check “meaningful”

–checking static semantics

–may only dependent on attribute or also dependent on structure

interpret

–generating code

(intermediate representation or final output of compiler)

Derivation tree v.s. abstract syntax tree Derivation tree v.s. abstract syntax tree

<assign stmt>

id := <exp>

<prim>

<term>

const +

<term>

id

<term>

id

*

<prim>

:=

id +

*

const id

id

<if stmt>

if <cond> then <stmts>endif

if-then-endif

<cond> <stmts>

Brief example of semantic processing Brief example of semantic processing

example --Y := 3 * X + I

abstract syntax tree: output:( 3, int ) ⇒⇒⇒⇒( 3.0, real )⁵ 3.0 * X →→→→T1⁶ ( I, int ) ⇒⇒⇒⇒ ( II, real )¹⁰ T1 + II →→→→ _T2¹¹ T2 →→→→Y¹⁴

post-order traversal

after step 7, the lowest level are useless

encountered tree is not the whole tree, usually :=

id +

* id

const id

check ¹³ check⁹

check⁴ (Y, real)¹

(3, int)² (X, real)³ (T1, real)⁷ (I, int)⁸

(T2, real) ¹²

Semantic processing techniques Semantic processing techniques

semantic record-- representation of meaning

semantic routine -- executor for semantic processing

when to call?

do what?

semantic stack

communications among semantic routines

–local variables, parameters (for non-table-driven parser) –semantic stack (for table-driven parser)

Semantic record

(1/2)

Semantic record

^(1/2)

representation for attribute

parameters among semantic routines

unify declarationis required when passing through semantic stack example --

#define MAXIDLEN 33 typedef char string[MAXIDLEN];

typedef struct operator { enum op { PLUS, MINUS } operator;

} op_rec;

enum expr

{ IDEXPR, LITERALEXPR, TEMPEXPR };

typedef struct expression { enum expr kind;

union { string name;

/* for IDEXPR and TEMPEXPR */

int val; /* for LITERALEXPR */

} expr_rec;

enum semantic_record_kind { OPREC, EXPRREC, ERROR };

typedef struct sem_rec {

enum semantic_record_kind record_kind;

union {

op_rec op_record; /* OPREC */

expr_rec expr_record; /* EXPRREC */

/* empty variant */ /* ERROR */

};

} semantic_record;

(2)

3 :

X :

+ :

3+X :

Semantic record

(2/2)

Semantic record

^(2/2)

EXPRREC

T1 TEMPEXPR EXPRREC

X IDEXPR EXPRREC

00000011 LITERALEXPR

OPREC PLUS

Semantic routines

(1/7)

Semantic routines

^(1/7)

action symbolsin grammar

the same for top-down and bottom-up parser except triggering places

for top-down parsing

may appear anywherein production rule, due to predictive nature

push onto parse stack when the production rule is predicted

execute and pop out of parse stack when it is on the top

for bottom-up parsing

be able to appear only after a product rule is fully recognized i.e., at the very end of right-hand side

– state -- all possible partially matched production rules

rewriting of some grammar rules is required

– <stmt> →→→→if <exp> #start_if then <stmts> endif #finish_if

⇒

⇒<stmt> →→→→<if_head> then <stmts> endif #finish_if

<if_head> →→→→if <exp> #start_if /* called semantic hook */

Yacc automatically does the rewriting

Semantic routines

(2/7)

Semantic routines

^(2/7)

example grammar with parameterizedaction symbols

<program> →→→→#startbegin <statement list> end

<statement tail>→→→→<statement> <statement tail> | λλλλ

<statement> →→→→<ident> := <expression>; #assign($1,$3)

<statement> →→→→read ( <id list> );

<statement> →→→→write ( <expr list> );

<id tail> →→→→, <ident> #read_id($2)<id tail> | λλλλ

<expr list> →→→→<expression> #write_expr($1)<expr tail>

<expr tail> →→→→, <expression> #write_expr($2)<expr tail> | λλλλ

<expression> →→→→<primary> #copy($1,$2)<primary tail> #copy($2,$$)

<primary tail> →→→→<add op><primary>#gen_infix($$,$1,$2,$3)<primary tail>#copy($3,$$)

<primary tail> →→→→ λλλλ

<primary> →→→→( <expression> ) #copy($2,$$)

<primary> →→→→<ident> #copy($1,$$)

<primary> →→→→INTLlTERAL#process_literal($$)

<add op> →→→→PLUSOP #process_op($$)

<add op> →→→→MINUSOP #process_op($$)

<ident> →→→→ID #process_id($$)

<system goal> →→→→<program> $ #finish

Semantic routines

(3/7)

Semantic routines

^(3/7)

example semantic routines

#include <assert.h>

/* <primary> →→→→( <expression> ) #copy($2,$$)*/

void copy(semantic_record *source, semantic_record *dest) {

/* Copy information from one part of the Semantic Stack to another */

*dest = *source;

}

/* <ident> →→→→ID #process_id($$)*/

void process_id(semantic_record *id_record) {

/* Declare ID & build corresponding semantic record */

check_id(token_buffer);

id_record->record_kind = EXPRREC;

id_record->expr_record.kind = IDEXPR;

strcpy(id_record->expr_record.name,token_buffer);

}

Semantic routines

(4/7)

Semantic routines

^(4/7)

/* <primary> →→→→INTLlTERAL#process_literal($$)*/

void process_literal(semantic_record *id_record) {

/* Convert literal to a numeric representation and build semantic record. */

id_record->record_kind = EXPRREC;

id_record->expr_record.kind = LITERALEXPR;

sscanf(token_buffer, "%d", &id_record->expr_record.val);

}

/* <add op> →→→→PLUSOP #process_op($$)*/

void process_op(semantic_record *op) {

/* Produce operator descriptor. */

op->record_kind = OPREC;

if (current_token == PLUSOP) op->op_record.operator = PLUS;

else

op->op_record.operator = MINUS;

}

Semantic routines

(5/7)

Semantic routines

^(5/7)

/*<primary tail> →→→→<add op><primary>#gen_infix($$,$1,$2,$3)<primary tail>#copy($3,$$)*/

void gen_infix(const semantic_record e1, const semantic_record op, const semantic_record e2, semantic_record *result ) {

assert(e1.record_kind == EXPRREC);

assert(op.record_kind = OPREC);

assert(e2.record_kind == EXPRREC);

/* Result is an expr_rec with temp variant set. */

result->record_kind = EXPRREC;

result->expr_record.kind = TEMPEXPR;

/* Generate code for infix operation.

* Get result temp and semantic record for result. */

strcpy(result->expr_record.name, get_temp());

generate(extract(op), extract(e1), extract(e2), result->expr_record.name);

}

Semantic routines

(6/7)

Semantic routines

^(6/7)

/* <statement> →→→→<ident> := <expression>; #assign($1,$3)*/

void assign(const semantic_record target, const semantic_record source) {

assert(target.record_kind == EXPRREC);

assert(target.expr_record.kind = IDEXPR);

assert(source.record_kind == EXPRREC);

/* Generate code for assignment. */

generate("Store", extract(source), target.expr_record.name, "");

}

/* <id list> →→→→<ident> #read_id($1)<id tail> */

void read_id(const semantic_record in_var) {

assert(in_var.record_kind == EXPRREC);

assert(in_var.expr record.kind = IDEXPR);

/* Generate code for read. */

generate("Read", in_var.expr_record.name, "Integer", " ");

}/* <expr tail> →→→→, <expression>#write_expr($2)<expr tail> | λλλλ*/

void write_expr(const semantic_record out_expr) {

assert(out_expr.record_kind == EXPRREC);

generate("Write", extract(out_expr), "Integer", " ");

}

Semantic routines

(7/7)

Semantic routines

^(7/7)

<stmt>

<id>1 := <exp>9 ; #a($1,$3)

ID #i($$) 2 #c($1,$2) <pt>3,8#c($2,$$)

INT #l($$) <ao>4 6 #g($$,$1,$2,$3) <pt>7 #c($3,$$)

PLUS #o($$)

ID

<id>5 #c($1,$$)

#i($$) λ λ λ λ

1 2,3 4 5,6 7,8,9

tracing example

EXPRREC T1 TEMPEXPR EXPRREC

X IDEXPR EXPRREC

Y IDEXPR

EXPRREC 00000011 LITERALEXPR

OPREC PLUS

Semantic stack Semantic stack

the place to interchange information among semantic routines

be not necessarily treated as an abstract stack

action-controlled : controlled by action routines

parser-controlled : controlled by theparser driver

action-controlled semantic stack

open the interface of stack to all semantic action routines

disadvantages:

1.difficult to change

2.action routines have to manipulate the stack

QUIZ: detailed implementationQUIZ: detailed implementation

(3)

Tracing example

(1/2)

Tracing example

^(1/2)

Step Remaining Input Parse Stack Action

(1) begin A:=BB-314+A; end $ <s.g.> Predict 22

(2) begin A:=BB-314+A; end $  $ Predict 1

(3) begin A:=BB-314+A; end $ begin <s.l.> end$ Match

(4) A:=BB-314+A; end $ <s.l.> end $ Predict 2

(5) A:=BB-314+A; end $ <s> <s.t.>end $ Predict 5 (6) A:=BB-314+A; end $ ID := <e> ;<s.t.> end $ Match (7) :=BB-314+A; end $ := <e> ; <s.t.> end $ Match (8) BB-314+A; end $ <e> ; <s.t.> end $ Predict 14 (9) BB-314+A; end $  <p.t.>; <s.t.> end $ Predict 18 (10) BB-314+A; end $ ID<p.t.> ; <s.t.> end $ Match (11) -314+A; end $ <p.t.> ; <s.t.> end $ Predict 15 (12) -314+A; end $ <a.o.> <p.t.>; <s.t.> end $ Predict 21 (13) -314+A; end $ - <p.t.> ; <s.t.> end $ Match

Example of shift-reduce parsing

(3/3)

Example of shift-reduce parsing

^(3/3)

grammar G₀

1. <program> →→→→begin <stmts> end $ 2. <stmts> →→→→SimpleStmt ; <stmts>

3. <stmts> →→→→begin <stmts> end ; <stmts>

4. <stmts> →→→→ λλλλ

tracing steps

Step Parse Stack Remaining Input Action

(1) 0 beginSimpleStmt ; SimpleStmt ; end $ Shift 1

(2) 0,1 SimpleStmt; SimpleStmt ; end $ Shift 5

(3) 0,1,5 ; SimpleStmt ; end $ Shift 6

(4) 0,1,5,6 SimpleStmt; end $ Shift 5

(5) 0,1,5,6,5 ; end $ Shift 6

(6) 0,1,5,6,5,6,λλλλ end $ /* goto(6,<stmts>) = 10 */ Reduce 4 (7) 0,1,5,6,5,6,10end $ /* goto(6,<stmts>) = 10 */ Reduce 2 (8) 0,1,5,6,10 end $ /* goto(1,<stmts>) = 2 */ Reduce 2

(9) 0,1,2 end $ Shift 3

(10) 0,1,2,3 $ Accept

QUIZ: compare LL and LR parse stack

Parser-controlled semantic stack Parser-controlled semantic stack

for LR parser

example --Y := 3 * X + I scanned

– grammar -- <S> →→→→id := <E> # assign

<E> →→→→<E> + <T> # add – still useful semantic information --

1.the location of Y

2.the location of temporary to store 3*X – parse stack --id := <E> +

O.K. ; can be combined with parse stack

QUIZ: how to combine?QUIZ: how to combine?

for LL parser

example (continued)

– parse stack --<T> #add #assign

need new technique

LL parser-controlled semantic stack

(1/11)

LL parser-controlled semantic stack

^(1/11)

LL driver void lldriver(void) {

int left_index = -1, right_index = -1;

int current_index, top_index;

/* Push the Start Symbol onto

* an empty parse stack.*/

push(s);

/* Initialize the semantic stack. */

current_index = 0; top_index = 1;

while (! stack_empty() ) {

/* Let a be the current input token. */

X = pop();

if (is_nonterminal(X)

&& T[X][a] = X →→→→Y1. . . Ym) { /* Expand nonterminal */

Push EOP(left, right, current, top_index) on the parse stack;

Push Y_m. . . Y_lon the parse stack;

left_index = current_index;

right_index = top_index;

top_index += m;

/* m is # of non-action symbols */

current_index = right_index;

} else if (is_terminal(X) && X == a) { Place token information from scanner

in sem_stack[current_index];

current_index++;

scanner(&a); /* Get next token */

} else if (X == EOP) {

Restore left, right, current, top_index from the EOP symbol;

/* Move to next symbol in RHS */

/* of previous production */

current_index++;

} else if (is_action_symbol(X)) Call Semantic Routine corresponding

to X;

else

/* Process syntax error */

} }

LL parser-controlled semantic stack

(2/11)

LL parser-controlled semantic stack

^(2/11)

action: pridict<system goal> →→→→<program> $ #finish

top_index

right_index, current_index left_index

$

<program>

<system goal>

<program>

$

#finish EOP(-1,-1,0,1)

parse stack semantic stack

LL parser-controlled semantic stack

(3/11)

LL parser-controlled semantic stack

^(3/11)

action: pridict<program> →→→→#start begin <statement list> end

top_index

"end"

<stmt list>

"begin"

$

<program>

<system goal>

#start

“begin”

<stmt list>

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

LL parser-controlled semantic stack

(4/11)

LL parser-controlled semantic stack

^(4/11)

action: do #start; match begin

top_index

right_index left_index current_index

"end"

<stmt list>

"begin"

$

<program>

<system goal>

<stmt list>

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

LL parser-controlled semantic stack

(5/11)

LL parser-controlled semantic stack

^(5/11)

action: predict <statement list> →→→→<statement> <statement tail>

top_index

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

<statement>

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

LL parser-controlled semantic stack

(6/11)

LL parser-controlled semantic stack

^(6/11)

action: predict <statement> →→→→<ident> := <expression>; #assign($1,$3)

top_index

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

<ident>

":="

<expression>

(4)

LL parser-controlled semantic stack

(7/11)

LL parser-controlled semantic stack

^(7/11)

action: predict <ident> →→→→ID #process_id($$)

top_index

right_index, current_index

left_index

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

ID

#proc_id($$) EOP(6,8,8,12)

":="

<expression>

ID parse stack semantic stack

LL parser-controlled semantic stack

(8/11)

LL parser-controlled semantic stack

^(8/11)

action: match ID

top_index, current_index right_index

left_index

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

#proc_id($$) EOP(6,8,8,12)

":="

<expression>

ID parse stack semantic stack

LL parser-controlled semantic stack

(9/11)

LL parser-controlled semantic stack

^(9/11)

action: do #proc_id; restore EOP(6,8,8,12)

top_index

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

":="

<expression>

LL parser-controlled semantic stack

(10/11)

LL parser-controlled semantic stack

^(10/11)

action: match :=

top_index

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

<expression>

LL parser-controlled semantic stack

(11/11)

LL parser-controlled semantic stack

^(11/11)

action: predict <expression> →→→→<primary> #copy($1,$2) <primary tail> #copy($2,$$)

<expression>

":="

<ident>

<stmt tail>

<statement>

"end"

<stmt list>

"begin"

$

<program>

<system goal>

top_index

";"

#assign($1,$3) EOP(4,6,6,8)

<stmt tail>

EOP(1,3,4,6)

"end"

EOP(0,1,1,3)

$

#finish EOP(-1,-1,0,1)

<primary>

#copy($1,$2)

<primary tail>

#copy($2,$$) EOP(6,8,10,12)

<primary tail>

<primary>

parse stack

QUIZ: space complexityQUIZ: space complexity

QUIZ: how to improve?QUIZ: how to improve?

QUIZ: comparison, QUIZ: comparison, LL v.s. LR LL v.s. LR action v.s. parser action v.s. parser-- controlled controlled semantic stack

Intermediate code Intermediate code

to separate high-level language-dependent realizations from low-level machine-dependent realizations

forms of intermediate codes

postfix notation

three-address code

–1 opcode + 2 input operands + 1 result operand

abstract syntax tree (DAG)

QUIZ: comparisonQUIZ: comparison

Example tuple language

(1/3)

Example tuple language

^(1/3)

an alternative representation of three-address code varying number of operands

ADDI, ADDF, SUBI, SUBF, MULTI, MULTF, DIVI, DIVF, MOD, REM, EXPI, EXPF, AND, OR, XOR, EQ, NE, GT, GE, LT, LE

RESULT := ARG1 OP ARG2 UMINUS ARG2 := -ARG1

NOT ARG2 := not ARG1

ASSIGN ARG3 := ARG1, size is ARG2

FLOAT ARG2 := FLOAT(ARG1 ) [ARG 1 in an integer]

ADDRESS ARG2 := the address of ARG1

RANGETEST abort execution if ARG3 < ARG1 or ARG3 > ARG2 LABEL ARG1 is used to label the next tuple

JUMP jump to tuple labeled ARG1 JUMP0 jump to ARG2 if ARG1 = 0 JUMP1 jump to ARG2 if ARG1 = 1 CASEJUMP ARG1 is case selector expression CASELABEL ARG1 is a case statement label CASERANGE ARG1 is lower bound of label range,

ARG2 is upper bound of range

Example tuple language

(2/3)

Example tuple language

^(2/3)

CASEEND no arguments, end of case statement PROCENTRY enter subprogram at nesting level ARG1 PROCEXIT exit subprogram at nesting level ARG1 STARTCALL ARG1 is temporary to reference activation record REFPARAM ARG1 is actual parameter

ARG2 is parameter offset

ARG3 is reference to activation record COPYIN ARG1 is actual parameter

ARG3 is reference to activation record COPYOUT ARG1 is actual parameter

ARG3 is reference to activation record COPYINOUT ARG1 is actual parameter

ARG3 is reference to activation record PROCJUMP ARG1 is subprogram start address (a label)

ARG2 is reference to activation record

Example tuple language

(3/3)

Example tuple language

^(3/3)

example

begin (READI, A)

read(A,B); (READI, B)

if A > B then (GT, A, B, t1)

C := A + 5; (JUMP0, t1, L1)

else (ADDI, A, 5, C)

C := B + 5; (JUMP, L2)

end if; (LABEL, L1)

write(2 * (C -1)); (ADDI, B, 5, C)

end (LABEL, L2)

(SUBI, C, 1, t2) (MULTI, 2, t2, t3) (WRITEI, t3)