Originally posted to idle-dev, but thought this might be a better place. Let me know if it isn't.

Hi,

I was curious about the internals of IDLE, and noticed that IDLE uses executes user code in a "subprocess" that's separate from the Python interpreter that is running IDLE itself (which does tasks such as making the window and coloring the text).

As far as I understand, IDLE runs a modified version of code.InteractiveInterpreter by sending user code through a socket. Even the IDLE documentation says that without a subprocess, "user code is not isolated from IDLE itself." However, some minimal testing of InteractiveInterpreter leads me to believe that the Interpreter object has its own view of local/global variables and therefore shouldn't be able to affect the calling interpreter (please correct me if I'm wrong).

So my question is a combination of "Why does IDLE use a subprocess?" and "Why is InteractiveInterpreter not secure enough?" What possible security vulnerabilities exist if one uses IDLE without the subprocess? If anyone knows (or could point me to information on) why IDLE is designed this way, I'd really appreciate it. Thank you!

Jim

On 29/05/23 8:10 am, James Schaffler wrote:
> However, some minimal testing of InteractiveInterpreter leads me to believe that the Interpreter object has its own view of local/global variables and therefore shouldn't be able to affect the calling interpreter

Globals you create by executing code in the REPL have their own
namespace. But everything else is shared -- builtins, imported
Python modules, imported C extension modules, etc. etc.

There's a long-running project to make it possible to have
multiple fully-isolated Python interpreters in one process, but
the way CPython is structured makes that very difficult to achieve,
and as far as I know it's not there yet.

In the case of IDLE, there's really no reason not to use a
subprocess[1]. It's easy and guarantees 100% isolation.

[1] Well, mostly. There used to be a small hitch on Windows with
the default firewall settings not letting you connect to a local
socket (nice one, Microsoft). I don't know whether that's still
an issue.

--
Greg

On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> Globals you create by executing code in the REPL have their own
> namespace. But everything else is shared -- builtins, imported
> Python modules, imported C extension modules, etc. etc.

Thanks for the explanation. Could you elaborate on precisely how "everything else is shared"? As far as I understand, if you run the following code:

from code import InteractiveInterpreter
interp = InteractiveInterpreter()
import numpy as np
interp.runcode("np.__name__")

this will result in the error
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'np' is not defined

which seems to imply that imports in the parent shell are not shared with interpreters and vice versa (if you swap the places of the import and the __name__ call).

Thanks,
Jim

On Wed, 31 May 2023 at 12:03, James Schaffler via Python-list
<python-list@python.org> wrote:
>
> On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> > Globals you create by executing code in the REPL have their own
> > namespace. But everything else is shared -- builtins, imported
> > Python modules, imported C extension modules, etc. etc.
>
> Thanks for the explanation. Could you elaborate on precisely how "everything else is shared"? As far as I understand, if you run the following code:
>
> from code import InteractiveInterpreter
> interp = InteractiveInterpreter()
> import numpy as np
> interp.runcode("np.__name__")
>
> this will result in the error
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> NameError: name 'np' is not defined
>
> which seems to imply that imports in the parent shell are not shared with interpreters and vice versa (if you swap the places of the import and the __name__ call).
>

Yep, what you're seeing there is the namespace and nothing else. But
if you mess with an actual builtin object, it'll be changed for the
other interpreter too.

>>> import ctypes
>>> ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
>>> 41+1
43
>>> from code import InteractiveInterpreter
>>> interp = InteractiveInterpreter()
>>> interp.runcode("print(41+1)")
43

(Note that this only works in CPython and only with integers small
enough to be in the cache, meaning that there is only one such object
representing that integer.)

The same is true of C extensions, which often have their own internal
state, and that state isn't isolated to a single interpreter.

Better isolation is coming with PEP 554
https://peps.python.org/pep-0554/ which also has some great
information about what currently is NOT isolated. (Also, even then,
some things won't be fully isolated; I think that the ctypes trick
above might still affect a subinterpreter even in a post-PEP554
world.)

ChrisA

On Tuesday, May 30th, 2023 at 10:18 PM, Chris Angelico wrote:
> Yep, what you're seeing there is the namespace and nothing else. But
> if you mess with an actual builtin object, it'll be changed for the
> other interpreter too.
>
> > > > import ctypes
> > > > ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
> > > > 41+1
>
> 43
>
> > > > from code import InteractiveInterpreter
> > > > interp = InteractiveInterpreter()
> > > > interp.runcode("print(41+1)")
>
> 43
>
> (Note that this only works in CPython and only with integers small
> enough to be in the cache, meaning that there is only one such object
> representing that integer.)
>
> The same is true of C extensions, which often have their own internal
> state, and that state isn't isolated to a single interpreter.
>
> Better isolation is coming with PEP 554
> https://peps.python.org/pep-0554/ which also has some great
> information about what currently is NOT isolated. (Also, even then,
> some things won't be fully isolated; I think that the ctypes trick
> above might still affect a subinterpreter even in a post-PEP554
> world.)

Amazing example! Thank you everyone for the detailed responses - will be sure to check out the PEP as well.

Jim