How do Ruby and Python implement their interactive consoles?

Question

When implementing the interpreter for my programming language I first thought of a simple console window which allows the user to enter some code which is then executed as a standalone program as a shell.

But there are severe problems: If every line of code the user enters is handled as a standalone program, it has to go through the tokenizer and parser and is then just executed by the interpreter - what about functions then?

• How can the Python/Ruby interactive consoles (IDLE, irb) "share" the code? How is the code entered handled?

Example:

>> def x:
>>  print("Blah")
>> 
>> x()

Where is the function stored so it can be called at any time again?

How can the interactive console take everything entered as obviously one program without executing everything over and over again?

Answer 1

For Python, an expression isn't complete until all parentheses, brackets, etc. match up. This is fairly easy to detect. A function/class definition isn't complete until a completely blank line is entered. The compiler then compiles the entered expression or definition, and runs it.

Much like a normal function, class, module, etc., the REPL has its own local scope. It's this scope that is used for variables and definitions entered into the REPL.

Answer 2

You can learn more about the Python interactive console by reading the documentation for the code module:

The code module provides facilities to implement read-eval-print loops in Python. Two classes and convenience functions are included which can be used to build applications which provide an interactive interpreter prompt.

http://docs.python.org/library/code.html

Answer 3

Most of these languages use a parser which has a kind of "token stream" -- that is, the parser keeps taking tokens (a string, symbol, operator, etc) from the input stream until it has a full expression, then it returns that parsed expression where it might be compiled to bytecode or otherwise executed. A REPL loop is relatively simple to handle given that structure, as the parser basically asks for more input, and you give the user a prompt and have the user enter more input. You might need a bit of communication from the parser to the reader to make it render things like continuation prompts.

Python and Ruby both execute statements immediately, in-order (a function declaration is one statement). So you can execute code statement-by-statement at the interpreter to largely the same effect as in a source file.