At the moment, passing "public ! public" to the Boo Lexer currently causes it to throw an exception when the tokenizer reaches the '!' which is good. However, if you catch the exception and call lexer.nextToken() again, the lexer remains on the unknown character, repeatedly throwing the exception. Unfortunately this means you are not able to use the Boo Lexer as a cheap-and-easy way to parse documents for important tokens, becaues you cannot step past any invalid tokens that stand between you and great justice. In my opinion it would be nice if the lexer were able to step "over" bad tokens once they have been first encountered, provided the exception is caught.