«

> There's no reliable way to differentiate between names which you defined
> using something other than an import statement, names you defined with an
> import statement with the intent of publishing them as part of your external
> API, and names you defined with an import statement which you only intended
> to use internally. So Python doesn't even try to differentiate betweem them.
>
ok, that makes sense

> >"[It] imports whatever names are defined in the package [including]
> >any submodules of the package that were explicitly loaded by previous
> >import statements."
This is the part that has me confused -- why does "from package import
*" go on to import names that were explicitly loaded by previous
import statements?

Here is a contrived example (contrived by virtue of culling it out of
an existing code base I was reorganizing into packages when I stumbled
across this:

Directory structure:
../
../SDRGen
../SDRGen/__init__.py
../SDRGen/TFGenerator.py

The SDRGen/__init__.py file is empty.
The SDRGen/TFGenerator.py file contains the following 2 lines:

class TFGenerator:
pass


Now, at the python prompt I type (and get) the following:

>>> from SDRGen.TFGenerator import *
>>> TFGenerator
<class SDRGen.TFGenerator.TFGenerator at 0xb7eae8bc>
>>> from SDRGen import *
>>> TFGenerator
<module 'SDRGen.TFGenerator' from 'SDRGen/TFGenerator.py'

It looks to me like, when I executed the "from SDRGen import *"
statement, the import "[included a] submodule of the package that
[was] explicitly loaded by [a] previous import statement." Granted
this example is probably not the best example of style and grace in a
Python program, and I'm in the process of cleaning up things such as
"from ... import *" and naming files and classes with identical names,
but when I stumbled across this I found it confusing, to say the
least.

Why does Python include the submodules that were explicitly loaded by
previous imports? Does it go out of it's way to do so? If so, why?
What purpose does it serve? Or, is it a natural fallout of the manner
in which imports are processed? If so, could somebody guide my
intuition as to why this would be such a natural fallout?

Thanks for the enlightenment...

--wpd

"Patrick Doyle" <> wrote:

> Why does Python include the submodules that were explicitly loaded by
> previous imports? Does it go out of it's way to do so? If so, why?
> What purpose does it serve? Or, is it a natural fallout of the manner
> in which imports are processed? If so, could somebody guide my
> intuition as to why this would be such a natural fallout?
>

It is a natural fallout of the manner in which imports are processed.
When you explicitly load a submodule a reference to the submodule is
stored in the parent module. When you do 'from module import *' and the
imported module doesn't define __all__, you create a reference in the
current module to everything referenced by the imported module (except
for variables beginning with '_').

Python makes no distinction at all between the objects named in the
imported module: they all get imported, i.e. assigned into the current
scope no matter how they got there.

If you use your example:

>>> import SDRGen
>>> dir(SDRGen)
['__builtins__', '__doc__', '__file__', '__name__', '__path__']
>>> from SDRGen.TFGenerator import *
>>> dir(SDRGen)
['TFGenerator', '__builtins__', '__doc__', '__file__', '__name__',
'__path__']
>>>

The line 'from SDRGen.TFGenerator import *' always creates the name
TFGenerator in the SDRGen module (and it will create the SDRGen module
if it has to). A subsequent 'from SDRGen import *' is simply doing the
equivalent of:

for name in dir(SDRGen):
if not name.startswith('_'):
setattr(current_module, getattr(SDRGen, name))

except of course the variables name and current_module don't exist.

On 7 Aug 2007 13:54:21 GMT, Duncan Booth <> wrote:
> "Patrick Doyle" <> wrote:
>
> > Why does Python include the submodules that were explicitly loaded by
> > previous imports? Does it go out of it's way to do so? If so, why?
> > What purpose does it serve? Or, is it a natural fallout of the manner
> > in which imports are processed? If so, could somebody guide my
> > intuition as to why this would be such a natural fallout?
> >
>
> It is a natural fallout of the manner in which imports are processed.
> When you explicitly load a submodule a reference to the submodule is
> stored in the parent module. When you do 'from module import *' and the
> imported module doesn't define __all__, you create a reference in the
> current module to everything referenced by the imported module (except
> for variables beginning with '_').
>
Ahhh, I see it now
>>> import SDRGen
>>> dir()
['SDRGen', '__builtins__', '__doc__', '__name__']

>>> dir(SDRGen)
['__builtins__', '__doc__', '__file__', '__name__', '__path__']

>>> from SDRGen import *
>>> dir()
['SDRGen', '__builtins__', '__doc__', '__name__']
>>> dir(SDRGen)
['__builtins__', '__doc__', '__file__', '__name__', '__path__']
(notice, no changes)

>>> from SDRGen.TFGenerator import TFGenerator
>>> dir()
['SDRGen', 'TFGenerator', '__builtins__', '__doc__', '__name__']
>>> TFGenerator
<class SDRGen.TFGenerator.TFGenerator at 0xb7ee80ec>
(as expected)

>>> dir(SDRGen)
['TFGenerator', '__builtins__', '__doc__', '__file__', '__name__', '__path__']
>>> SDRGen.TFGenerator
<module 'SDRGen.TFGenerator' from 'SDRGen/TFGenerator.pyc'>

and that's what led to my confusion... when my script later did a
>>> from SDRGen import *

The 'TFGenerator' name got changed from a reference to the
'TFGenerator' class to a reference to the 'SDRGen.TFGenerator' module.

and now it makes sense.

Thanks.

--wpd

Patrick Doyle wrote:
> This is the part that has me confused -- why does "from package import
> *" go on to import names that were explicitly loaded by previous
> import statements?

Because there's no easy way for it *not* to do that.
All it does is grab whatever names are defined in the
module at the time. It can't tell whether those names
got there as a result of an import statement or some
other way.

A more pertinent question is why it *doesn't* import
submodules that haven't already been explicitly imported.
Probably this is just because it could be quite expensive
to do so -- you would potentially be triggering a lot
of disk activity to load the previously unused modules
into memory, many of which you may not be going to use.

Just in case it's not clear, "importing" can actually
mean two different things. If the module hasn't been
imported before, it has to be loaded into memory, which
is expensive. If it has been imported before, then
you're just creating another reference to the same
module object, which is very cheap. An import *
just does the cheap part, and leaves you to explicitly
ask for the expensive part if you really want it.

--
Greg