parsing - ANTLR4- dynamically inject token -

so i'm writing python parser , need dynamically generate indent , dedent tokens (because python doesn't use explicit delimiters) according python grammar specification.

basically have stack of integers representing indentation levels. in embedded java action in indent token, check if current level of indentation higher level on top of stack; if is, push on; if not, call skip().

the problem is, if current indentation level matches level multiple levels down in stack, have generate multiple dedent tokens, , can't figure out how that.

my current code: (note within_indent_block , current_indent_level managed elsewhere)

fragment dent: {within_indent_block}? (space|tab)+;  indent: {within_indent_block}? dent         {if(current_indent_level > whitespace_stack.peek().intvalue()){                  whitespace_stack.push(new integer(current_indent_level));                  within_indent_block = false;          }else{                  skip();          }          }          ;      dedent: {within_indent_block}? dent         {if(current_indent_level < whitespace_stack.peek().intvalue()){             while(current_indent_level < whitespace_stack.peek().intvalue()){                       whitespace_stack.pop();                       <<injectdedenttoken()>>; //how do             }          }else{                skip();          }          }          ; 

how do , / or there better way?

there few problems code have posted.

1. the indent , dedent rules semantically identical (considering predicates , rule references, ignoring actions). since indent appears first, means can never have token produced dedent rule grammar.
2. the {within_indent_block}? predicate appears before reference dent inside dent fragment rule itself. duplication serves no purpose slow down lexer.

the actual handling of post-matching actions best placed in override of lexer.nexttoken(). example, start following.

private final deque<token> pendingtokens = new arraydeque<>();  @override public token nexttoken() {     while (pendingtokens.isempty()) {         token token = super.nexttoken();         switch (token.gettype()) {         case indent:             // handle indent here. skip token, don't add             // pendingtokens queue , super.nexttoken()             // called again.             break;          case dedent:             // handle indent here. skip token, don't add             // pendingtokens queue , super.nexttoken()             // called again.             break;          default:             pendingtokens.add(token);             break;         }     }      return pendingtokens.poll(); }