Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > PyWart: Module access syntax

Reply
Thread Tools

PyWart: Module access syntax

 
 
Rick Johnson
Guest
Posts: n/a
 
      01-11-2013

Python's module/package access uses dot notation.

mod1.mod2.mod3.modN

Like many warts of the language, this wart is not so apparent when first learning the language. The dot seems innocently sufficient, however, in truthit is woefully inadequate! Observe:

name1.name2.name3.name4.name5

Can you tell me which names are modules, which are classes, and which are methods/functions? Wait, i know the argument you'll use:

"""I can decipher from spelling! Modules use all lowercase, classes use initial uppercase, and methods use words_sep_by_underscore. I'm so smart!"""

Indeed. But that's only *_IF_* everybody used the same style guide. And as we know many people don't follow the style guide (not to mention the style guide has issues!) And since style is NOT enforced, we suffer the unintuitive syntax nightmare! The solution is obvious by using proper syntax.

import lib:gui:tkinter:dialogs.SimpleDialog as Blah

You /could/ use two colons:

import lib::gui::tkinter::dialogs.SimpleDialog as Blah

....and i think this syntax does help to separate the identifiers more clearly, but since these imports can be quite long, i prefer the single colon myself.
 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-11-2013
On Thu, 10 Jan 2013 22:01:37 -0800, Rick Johnson wrote:

> Python's module/package access uses dot notation.
>
> mod1.mod2.mod3.modN
>
> Like many warts of the language, this wart is not so apparent when first
> learning the language. The dot seems innocently sufficient, however, in
> truth it is woefully inadequate! Observe:
>
> name1.name2.name3.name4.name5
>
> Can you tell me which names are modules, which are classes, and which
> are methods/functions? Wait, i know the argument you'll use:
>
> """I can decipher from spelling! Modules use all lowercase, classes
> use initial uppercase, and methods use words_sep_by_underscore. I'm so
> smart!"""


Wrong. My answer is...

Who cares? Why do you care which names are modules, classes, etc? This is
Python, and duck-typing rules!

Today I have:

x = mylib.stuff.spam().parrot.speak()

where:

- mylib is a module
- stuff is a separate module imported into mylib
- spam is a class
- parrot a data attribute
- and speak a method.

But then tomorrow I re-factor the code and turn:

- mylib into a package
- stuff into a module of the package
- spam into a factory function
- parrot into a property
- and speak into a dynamically generated static method created using
__getattr__.

How do I write this new code? Exactly the same as the old code -- no
changes are required!

x = mylib.stuff.spam().parrot.speak()

*It doesn't matter* what kind of entity each of the dotted names
represent, so long as they have the expected interface.


> The solution is obvious by using proper syntax.


Solution to what? You can only have a solution once you have identified a
problem. You have not identified a problem. In any case, your suggestion
is *not* obvious.

> import lib:gui:tkinter:dialogs.SimpleDialog as Blah


Which names are packages, modules, classes, methods, functions, or other
objects?

Why do you have lib:gui but dialogs.SimpleDialog? Is the rule "classes
should always be preceded by a dot?"

Following this import:

import lib:gui

how do I access attributes of that module?

x = lib:gui:function() perhaps? Now I have to remember which attribute
access uses dot operator and which uses colon operator.

Does this mean there needs to four new be special methods:

__getcolonattribute__
__getcolonattr__
__setcolonattr__
__delcolonattr__

to match the regular dot operator methods

__getattribute__
__getattr__
__setattr__
__delattr__

? Or do you think that the Python compiler should special-case attribute
access from modules?


> You /could/ use two colons:
>
> import lib::gui::tkinter::dialogs.SimpleDialog as Blah


Before thinking about the syntax, you need to think about the behaviour,
identify what actual problem you hope to solve, and how you hope to solve
it. Not just throw random syntax around and hope that it's "obvious".



--
Steven
 
Reply With Quote
 
 
 
 
Rick Johnson
Guest
Posts: n/a
 
      01-12-2013

On Friday, 1-11-2013 10:02:34 AM, Steven D'Aprano wrote:
> Solution to what? You can only have a solution once you have identified a
> problem. You have not identified a problem. In any case, your suggestion
> is *not* obvious.


The problem is that by using the dot ubiquitously we are obfuscating the path to an identifier.

> > import lib:gui:tkinter:dialogs.SimpleDialog as Blah

>
> Which names are packages, modules, classes, methods, functions, or other
> objects?
>
> Why do you have lib:gui but dialogs.SimpleDialog? Is the rule "classes
> should always be preceded by a dot?"


No the rules are:
* "Colon" must be used to access a "module" (or a package).
* "Dot" must be used to access a "module member".

In the line "lib:gui:tkinter:dialogs.SimpleDialog", "lib", "gui", "tkinter", and "dialogs" are all packages (or modules: same thing as far as paths are concerned). "SimpleDialog" however is a class, who's identifier is a member of the module "lib:gui:tkinter:dialogs".

> Following this import:
>
> import lib:gui
>
> how do I access attributes of that module?
>
> x = lib:gui:function() perhaps? Now I have to remember which attribute
> access uses dot operator and which uses colon operator.


It's simple: MODULES&PACKAGES use colon, MODULE MEMBERS use dot. How many times must i explain these simple rules?

If you don't know which names are modules and which names are members then how could a programmer possibly use the API in an intelligent way Steven? This syntax does not help the programmer much. Well, it can be beneficial tothe programmer if he gets a *PathError* because he foolishly tried to instance a module named "simpledialog" when he actually meant to instance the object "simpledialog.SimpleDialog". (notice i did not use the word class!)

Traceback (most recent call last):
File "<blah>", line 1, in <module>
dlg = lib:gui:tkinter:dialogs.simpledialog()
PathError: name 'simpledialog' is a module NOT a object!

But that is not the reason for this syntax Steven, it is just a pleasant side effect.

> Does this mean there needs to four new be special methods:
>
> __getcolonattribute__
> __getcolonattr__
> __setcolonattr__
> __delcolonattr__


Gawd no. getattr, setattr, and delattr will remain unchanged. The only change is how a /path/ to an identifier is "formed".

> to match the regular dot operator methods
>
> __getattribute__
> __getattr__
> __setattr__
> __delattr__
>
> ? Or do you think that the Python compiler should special-case attribute
> access from modules?


There is no "special casing" needed. Python could happily replace all colons in a path with dots and interpret the path internally just as it does today.

> Before thinking about the syntax, you need to think about the behaviour,
> identify what actual problem you hope to solve, and how you hope to solve
> it. Not just throw random syntax around and hope that it's "obvious".


*The problem:*
.... is readability. The current dot syntax used ubiquitously in paths is not conveying the proper information to the reader, and in-fact obfuscating the code.

*Method to solve the problem:*
.... by introducing a new syntax that requires all module access to use the colon and all module members to use the dot.

*The syntax:*
.... is perfect. Of course we could argue over /which/ char to use, but no matter which char prevails, we are gaining explicitness at zero overhead to the programmer because we are replacing one single char(dot) for one singlechar(colon). The maintainer is the real winner. This is a win-win all around.
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      01-12-2013
On Sat, Jan 12, 2013 at 3:34 PM, Rick Johnson
<(E-Mail Removed)> wrote:
> *The problem:*
> ... is readability. The current dot syntax used ubiquitously in paths is not conveying the proper information to the reader, and in-fact obfuscating the code.


Please explain how this is a problem. As Steven said, there is NO
useful difference. I don't *care* whether it's a package, a module, or
whatever. Module with class with static member? Fine. Package with
module with class? Also fine. Imported special object that uses dunder
methods to simulate either of the above? What's it matter to me, as
long as I get my final result!

Syntactic salt is seldom helpful.

ChrisA
 
Reply With Quote
 
Rick Johnson
Guest
Posts: n/a
 
      01-12-2013
On Friday, January 11, 2013 10:40:36 PM UTC-6, Chris Angelico wrote:
> On Sat, Jan 12, 2013 at 3:34 PM, Rick Johnson


> > *The problem:*
> > ... is readability. The current dot syntax used ubiquitously in paths is not conveying the proper information to the reader, and in-fact obfuscating the code.

>
> Please explain how this is a problem.



What is this importing?

"import lib.gui.simpledialog"

....is that the "simpledialog module" or "SimpleDialog object"? Since namingconventions are not ENFORCED, we can NEVER be sure if an identifier is a object or module. And since object definitions (aka: classes) are often placed into a module with the same name, how are we to know? Why should we be forced to open source files to find out a small detail that proper syntax can easily provide?

This is a matter of READABILITY, Christopher. It's one or the other (or thestatus quo):

1. Enforce naming conventions.
2. Enforce path syntax.
3. Continue to obfuscate code.

The choice is yours.
 
Reply With Quote
 
Rick Johnson
Guest
Posts: n/a
 
      01-12-2013
On Friday, January 11, 2013 10:40:36 PM UTC-6, Chris Angelico wrote:
> On Sat, Jan 12, 2013 at 3:34 PM, Rick Johnson


> > *The problem:*
> > ... is readability. The current dot syntax used ubiquitously in paths is not conveying the proper information to the reader, and in-fact obfuscating the code.

>
> Please explain how this is a problem.



What is this importing?

"import lib.gui.simpledialog"

....is that the "simpledialog module" or "SimpleDialog object"? Since namingconventions are not ENFORCED, we can NEVER be sure if an identifier is a object or module. And since object definitions (aka: classes) are often placed into a module with the same name, how are we to know? Why should we be forced to open source files to find out a small detail that proper syntax can easily provide?

This is a matter of READABILITY, Christopher. It's one or the other (or thestatus quo):

1. Enforce naming conventions.
2. Enforce path syntax.
3. Continue to obfuscate code.

The choice is yours.
 
Reply With Quote
 
Chris Angelico
Guest
Posts: n/a
 
      01-12-2013
On Sat, Jan 12, 2013 at 4:46 PM, Rick Johnson
<(E-Mail Removed)> wrote:
> This is a matter of READABILITY, Christopher. It's one or the other (or the status quo):
>
> 1. Enforce naming conventions.
> 2. Enforce path syntax.
> 3. Continue to duck type, like Python is good at.
>
> The choice is yours.


FTFY.

ChrisA
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-12-2013
On Fri, 11 Jan 2013 20:34:20 -0800, Rick Johnson wrote:


>> > import lib:gui:tkinter:dialogs.SimpleDialog as Blah

>>
>> Which names are packages, modules, classes, methods, functions, or
>> other objects?
>>
>> Why do you have lib:gui but dialogs.SimpleDialog? Is the rule "classes
>> should always be preceded by a dot?"

>
> No the rules are:
> * "Colon" must be used to access a "module" (or a package).
> * "Dot" must be used to access a "module member".


So what do you do for, say, os.path? According to the first rule, you
must write it as osath because path is a module; according to the
second rule, you must write it as os.path because path is a member of os.
So which rule wins?

If I do this:

import math
import string
math.string = string


is that legal, or do I have to write "math:string = string"?



Suppose I do this:


import random
if random.random() < 0.5:
math.string = "NOBODY expects the Spanish Inquisition!"
else:
math.string = string # assuming this actually is allowed


How do I access the string member?

try:
print math.string # only works if string is a str object
except SomeException:
print math:string # only works if string is a module object


That would suck *and* blow at the same time. I don't need to know the
type of any other member object in order to look it up, why should I have
to care whether it is a module?

Now, suppose I then do this:

class Blob: pass

blob = Blob()
blob.math = math # or should that be blob:math ?

Now do I have to write this?

blob.math:string.letters

(assuming that math:string is a module, not a str object).


[...]
> It's simple: MODULES&PACKAGES use colon, MODULE MEMBERS use dot. How
> many times must i explain these simple rules?


At the time I asked the question, you hadn't explained it *at all*.


[...]
> This syntax does not help the programmer much. Well, it can be
> beneficial to the programmer if he gets a *PathError* because he
> foolishly tried to instance a module named "simpledialog" when he
> actually meant to instance the object "simpledialog.SimpleDialog".
> (notice i did not use the word class!)


"Instance" is a noun. The verb you are looking for is "instantiate".


> Traceback (most recent call last):
> File "<blah>", line 1, in <module>
> dlg = lib:gui:tkinter:dialogs.simpledialog()
> PathError: name 'simpledialog' is a module NOT a object!


Of course it is an object. *Everything* in Python is an object. Modules
are objects. Strings are objects. Types and classes are objects. None is
an object. Metaclasses are objects. Properties are objects. Exceptions
are objects. Have I made it clear yet?


[...]
> *The problem:*
> ... is readability. The current dot syntax used ubiquitously in paths is
> not conveying the proper information to the reader, and in-fact
> obfuscating the code.


So you say. I think you are caring too much about the type of members and
not enough about what interface they provide. Why do you care if I grab
module.my_singleton_object and replace my_singleton_object with a module?

Modules are singletons in Python, in the sense that every[1] time you
import a module you get the same object. Using modules as members for
their singleton properties, not for their importability, is a common
technique. With your proposal, I need to know whether a member is a
module, or any other type, before I can access it. That is a really
shitty design. Instead of having one syntax for *all* attribute access,
now you have two, and we have to care whether a member is a module or
not, which we never did before.

Worse, you couple the syntax to implementation: if I use a module as a
singleton, I need to use one syntax; if I use a custom type as a
singleton, I have to use different syntax. Whichever choice I make
*today*, if the implementation changes, the required syntax changes and
my code will break.




[1] Well, almost. There are ways to accidentally or deliberately confuse
the import machinery.


--
Steven
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      01-12-2013
On Fri, 11 Jan 2013 21:46:36 -0800, Rick Johnson wrote:

> On Friday, January 11, 2013 10:40:36 PM UTC-6, Chris Angelico wrote:
>> On Sat, Jan 12, 2013 at 3:34 PM, Rick Johnson

>
>> > *The problem:*
>> > ... is readability. The current dot syntax used ubiquitously in paths
>> > is not conveying the proper information to the reader, and in-fact
>> > obfuscating the code.

>>
>> Please explain how this is a problem.

>
>
> What is this importing?
>
> "import lib.gui.simpledialog"
>
> ...is that the "simpledialog module" or "SimpleDialog object"?


It has to be the first, because:

- you can't import members of modules directly using dot syntax

("import math.cos" will fail)

- and it can't be SimpleDialog, because Python is case-sensitive and you
wrote simpledialog.


But ignoring the import, and just looking at the dotted name:

lib.gui.simpledialog

who cares what simpledialog happens to be? So long as you know the
expected interface (say, "it's a callable object that takes three
arguments"), you can use it the same way whether it is a function, a
method, a class or something else. The implementation could change, and
you need not care.

If you actually do care what the type of simpledialog is, you can find
out easily:

type(lib.gui.simpledialog)



--
Steven
 
Reply With Quote
 
Ian Kelly
Guest
Posts: n/a
 
      01-12-2013
On Fri, Jan 11, 2013 at 9:34 PM, Rick Johnson
<(E-Mail Removed)> wrote:
> No the rules are:
> * "Colon" must be used to access a "module" (or a package).
> * "Dot" must be used to access a "module member".


What about module a that does not natively contain module b, but
imports it as a member like so?

a.py:
import b

Must this be accessed as a:b or a.b?

What about a module that is referenced by some other object that is
not a module? Should it be "my_object.pickle_module" or
"my_objectickle_module"?

> It's simple: MODULES&PACKAGES use colon, MODULE MEMBERS use dot. How manytimes must i explain these simple rules?


Since you didn't actually explain them at all in your initial post, I
would have to say that once is enough.

> If you don't know which names are modules and which names are members then how could a programmer possibly use the API in an intelligent way Steven?


You might start by reading the documentation.

> This syntax does not help the programmer much. Well, it can be beneficialto the programmer if he gets a *PathError* because he foolishly tried to instance a module named "simpledialog" when he actually meant to instance the object "simpledialog.SimpleDialog". (notice i did not use the word class!)


The problem here is bad naming on the part of the library designer,
not bad syntax. Why is SimpleDialog confusingly contained in a module
also named simpledialog? This is not Java; there is no absurd
requirement of one class per file. A clearer path to SimpleDialog
would just be "lib.gui.tkinter.dialogs.SimpleDialog".

> Traceback (most recent call last):
> File "<blah>", line 1, in <module>
> dlg = lib:gui:tkinter:dialogs.simpledialog()
> PathError: name 'simpledialog' is a module NOT a object!


See, here is the thing that seems to be eluding you. In Python,
modules *are* objects. You can bind names to them, just like any
other object. You can add them to collections. You can introspect
them. Hell, you can even subclass ModuleType and create modules that
can be called or multiplied or iterated over or any crazy thing you
can think of. Your : syntax is trying to treat modules as being
somehow special, but the fact is that they're *not* special, which is
why I think it's a bad idea.

>> Does this mean there needs to four new be special methods:
>>
>> __getcolonattribute__
>> __getcolonattr__
>> __setcolonattr__
>> __delcolonattr__

>
> Gawd no. getattr, setattr, and delattr will remain unchanged. The only change is how a /path/ to an identifier is "formed".


Then how is ModuleType.__getattr__ supposed to know whether to raise a
"PathError" or not, after it determines whether the requested object
turned out to be a module or not? It has no way of knowing whether it
was invoked by '.' or by ':' or by getattr or by direct invocation.

Or are you instead proposing a new bytecode "LOAD_COLON_ATTR" to
complement "LOAD_ATTR", and that every single occurrence of LOAD_ATTR
in every program would then have to check whether the loaded object
turned out to be a module and whether the object it was loaded from
also happened to be a module, all just to raise an exception in that
case instead of just giving the programmer what he wants?
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: F2PY: Access Fortran module data from multiple python module Gabriel Genellina Python 0 11-21-2008 04:22 AM
help - Module needs access to another module abcd Python 4 03-20-2007 10:10 PM
Re: module docstring, documentation,anything? please note is the module type/object NOT some module Maric Michaud Python 0 06-24-2006 12:42 PM
Syntax highligth with textile: Syntax+RedCloth ? gabriele renzi Ruby 2 12-31-2005 02:44 AM
Using JYTHON inside ANT : access os module -> "ImportError: no module named javaos" eric_bellard Python 1 10-07-2004 05:41 AM



Advertisments