Could the Tokenizer's source code be made available? I'm trying to solve this in Python first and since it's not available i'm hand-writing my own.

thanks. i really like list comprehensions and generator expressions so i wanted to apply them to this case.

Run the code locally with some debugging messages to see why it times out, i.e. an infinite loop or something. to see what code is being run just do a console.log(code) or print(code) or whatever within the interpret function.

$ python2
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 26 2016, 12:10:39) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> input = 10
>>> ^D
$ python3
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> input = 10
>>> ^D

Afaik input is no longer a reserved keyword in python
it may have been when you commented though.