Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Python Packages : A loomingproblem? packages might no longer work? (well not on your platform or python version anyway)

Reply
Thread Tools

Python Packages : A loomingproblem? packages might no longer work? (well not on your platform or python version anyway)

 
 
David Lyon
Guest
Posts: n/a
 
      04-22-2009

Hi all,

I'm working on a python package manager gui. Mainly because I struggle on
windows
getting python packages installed. In my mind, there seemed to be some
minor
problems, but then I did the numbers. Do the numbers I writa about here
reflect
reality?

Should we discuss stuff like this? Debate welcome...


= Introduction =

One of the big challenges for Python going forward is providing a testing
infrastructure for Python Packages.

There are now over 6,000 packages listed on PyPi - and this number can only
get bigger.

Then, there are the three major operating systems:

* Windows
* Mac
* Linix/Unix

To complicate the problem, there are now many versions of each operating
system.

Multiply those two combinations by all of the versions of Python that
already exist (not to mention the ones coming) and we start to that we are
heading into complexity. If there are 4 major windows revisions and 4 major
Linuxes, and two major Mac platforms, we end up with perphaps (6,000 x (4 +
4 + 2)) 60,000 delivery possibilities.

That number then needs to be multiplied by the number of python versions,
which possibly include 2.1, 2.2, 2.3, 2.4, 2.5, 2.6 and coming up.. the 3
series...

So that could be (60,000 x 7 (python versions)) 420,000 variations of known
python packages.

To date... the testing has been done... we have to assume... manually with
some automation.

But we can't expect package maintainers to be forever testing their own
code on platforms that they simply don't have access to.

A more reasonable and cost effective option is to have this testing done on
a server farm virtual environment building infrastructure.

In simple terms, we need to build all the packages that exist for Python on
a daily basis on all of the environments and report any issues back to the
registered maintainers.

This job is too big to be done manually. We need to use either a
Super-Computer or a Server Farm. Fortunately, Server Farms are close at
hand.

= Server Farm Virtual Environments =

Google and Amazon web services are two organisations amongst others that
offer commercial virtual server farms that could be employed to do the
above build process of all the python packages.

Python scripts would be developed utilising the python "test" frameworks to
supervise the build on each and every platform.

With this basic structure, a daily building/testing infrastructure working
across the different versions of python and operating systems, could easily
become a reality.

At present AWS offer virtual environments for both Windows and Linux. These
can be seen on these links:

*
http://developer.amazonwebservices.c...categoryID=209
*
http://developer.amazonwebservices.c...categoryID=208

A service to do building on Mac Virtual Machines needs to be located.

= Test Scripts =

A test script will be developed that will cycle through all the packages on
pypi, download the package and build it on all available platforms.

The results of the build can then be made available via some sort of web
delivery system. Describing on which platforms the builds were successful
and not.

In the past, it has been difficult for developers to test on all platforms.

These facilities are bound to improve overal code quality across the python
universe.

= Scope of Testing =

It's important to define what and can be and what cannot be tested.

The scope of the framework will be:

* to check that each package can be installed on all the relevant
platforms
using the setup.py script

* to run the built in tests within the package

* to check that the package can be de-installed on the relevant platforms



 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-23-2009
On Wed, 22 Apr 2009 19:06:18 -0400, David Lyon wrote:


> = Introduction =
>
> One of the big challenges for Python going forward is providing a
> testing infrastructure for Python Packages.


Why is it the responsibility of the Python language to provide such a
testing infrastructure?


> There are now over 6,000 packages listed on PyPi - and this number can
> only get bigger.
>
> Then, there are the three major operating systems:
>
> * Windows
> * Mac
> * Linix/Unix


Linux.

> To complicate the problem, there are now many versions of each operating
> system.
>
> Multiply those two combinations by all of the versions of Python that
> already exist (not to mention the ones coming) and we start to that we
> are heading into complexity. If there are 4 major windows revisions and
> 4 major Linuxes, and two major Mac platforms, we end up with perphaps
> (6,000 x (4 + 4 + 2)) 60,000 delivery possibilities.



But most of those packages will probably work without modification on any
platform supported by Python. We can take that as a given (subject to
correction), because the Python language itself has been tested on those
platforms.


> That number then needs to be multiplied by the number of python
> versions, which possibly include 2.1, 2.2, 2.3, 2.4, 2.5, 2.6 and coming
> up.. the 3 series...


Why? Why should every package on PyPI need to support all those Python
versions? That should be the decision of the package maintainer. If they
want to support every version of Python back to 1.0, they can, and if
they want to only support version 2.5 that's fine too.


> So that could be (60,000 x 7 (python versions)) 420,000 variations of
> known python packages.


Could be but won't be.


> To date... the testing has been done... we have to assume... manually
> with some automation.


Why do we "have to" assume this? For all we know, three quarters of the
packages on PyPI have never been tested *at all*.


> But we can't expect package maintainers to be forever testing their own
> code on platforms that they simply don't have access to.


We don't.


> A more reasonable and cost effective option is to have this testing done
> on a server farm virtual environment building infrastructure.


You haven't demonstrated that we need to have all this testing done in
the first place.


> In simple terms, we need to build all the packages that exist for Python
> on a daily basis on all of the environments and report any issues back
> to the registered maintainers.


We "need" to do this? Why? What's the dire problem you are trying to
solve? That somebody downloads a random package off PyPI and it doesn't
work? Oh dear, how will we cope!


> This job is too big to be done manually. We need to use either a
> Super-Computer


What "Super-Computers" do you know of that run Python?


> or a Server Farm. Fortunately, Server Farms are close at
> hand.
>
> = Server Farm Virtual Environments =
>
> Google and Amazon web services are two organisations amongst others that
> offer commercial virtual server farms that could be employed to do the
> above build process of all the python packages.


Who is paying for this?



> Python scripts would be developed utilising the python "test" frameworks
> to supervise the build on each and every platform.


Who do you suggest should develop these scripts?


> With this basic structure, a daily building/testing infrastructure
> working across the different versions of python and operating systems,
> could easily become a reality.


Oh yeah, "easily".




--
Steven.
 
Reply With Quote
 
 
 
 
David Lyon
Guest
Posts: n/a
 
      04-23-2009
Hi Steven,

You make some good points...

Let me try to answer them..

> Why is it the responsibility of the Python language to provide such a
> testing infrastructure?


First define "Python language".

Ok... we know it as the core interpretor. But to the developer it is also
all the packages that everybody has contributed.

The "Python language" is the interpretor and all the packages the
"community" has put together.

> Why? Why should every package on PyPI need to support all those Python
> versions? That should be the decision of the package maintainer. If they
> want to support every version of Python back to 1.0, they can, and if
> they want to only support version 2.5 that's fine too.


Why shouldn't packages support more than one python version?

Looking at it conversely....

Why should the package developer dictacte which python version the package
will run on ?

> For all we know, three quarters of the
> packages on PyPI have never been tested *at all*.


Right. Why not run some tests....

> What's the dire problem you are trying to
> solve?


Backward and forward compatability of python package resources.

> What "Super-Computers" do you know of that run Python?


Google. Amazon web services..

> Who is paying for this?


>From as little as $30 per month. Funding isn't so much the issue.


>> With this basic structure, a daily building/testing infrastructure
>> working across the different versions of python and operating systems,
>> could easily become a reality.

>
> Oh yeah, "easily".


pypi_packagelist = getallpypipackages()

for package in pypi_packagelist:
testpackageonallplatforms(package)

Best Regards

David




 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-23-2009
On Wed, 22 Apr 2009 23:27:23 -0400, David Lyon wrote:


>> Why? Why should every package on PyPI need to support all those Python
>> versions? That should be the decision of the package maintainer. If
>> they want to support every version of Python back to 1.0, they can, and
>> if they want to only support version 2.5 that's fine too.

>
> Why shouldn't packages support more than one python version?


They can. That depends on the developer of the package.


> Looking at it conversely....
>
> Why should the package developer dictacte which python version the
> package will run on ?


Because they're the developer. Who else should decide what Python
versions to support? What are you going to do, hold a gun to their head
and force them to support Python 2.1 when they've written it using 2.6
features?



>> For all we know, three quarters of the packages on PyPI have never been
>> tested *at all*.

>
> Right. Why not run some tests....


Be my guest.


>> What's the dire problem you are trying to solve?

>
> Backward and forward compatability of python package resources.


That's not a problem, that's a feature.

What's the *problem* that you are trying to solve? What bad thing will
happen if we don't build your proposed system?


>> What "Super-Computers" do you know of that run Python?

>
> Google. Amazon web services..


They're not super computers. They're distributed networks of "regular"
computers.


>> Who is paying for this?

>
>>From as little as $30 per month. Funding isn't so much the issue.


Whether it is $30 a month or $30,000 a month, who do you expect to pay it?


>>> With this basic structure, a daily building/testing infrastructure
>>> working across the different versions of python and operating systems,
>>> could easily become a reality.

>>
>> Oh yeah, "easily".

>
> pypi_packagelist = getallpypipackages()
>
> for package in pypi_packagelist:
> testpackageonallplatforms(package)



You don't have either getallpypipackages() or
testpackageonallplatforms(), so this is just Py in the Sky (pun intended)
fantasizing. But okay, for the sake of the argument, let's pretend you
have your cluster, and your test suite. You run the test, and get a list
of 18,000 errors (that's an average of three errors per package). Now
what?




--
Steven
 
Reply With Quote
 
Patrick Mullen
Guest
Posts: n/a
 
      04-23-2009
On Wed, Apr 22, 2009 at 9:33 PM, David Lyon <(E-Mail Removed)> wrote:
> Hi Steve,
>>> Why should the package developer dictacte which python version the
>>> package will run on ?

>>
>> Because they're the developer. Who else should decide what Python
>> versions to support?

> The developer shouldn't be making such decisions at all....
> What hardware or operating systems we run his/her programs on is
> up to the real world to decide.
> If you think about it logically... why are we even asking our
> developers to even "build" their packages for specific python
> versions in the first place?
> They should just:


Programmers need to know what they are programming for. Each version
of the language changes things, and the old interpreters often cannot
understand things introduced in later versions. There is no way to
magically get code written for one version to work for a different
version. Code written on any python less than 3 will work on any
higher python version up to (but not including) 3, and the same is
true for the 3 series, but going back down needs somebody to do the
conversion, unless the programmer was meticulous in making it work on
all versions.

This problem of course, isn't just looming, it's been around for a
while. Every time a new python version is introduced, there is chaos
as some library writers move over, and some stay with the old, and
others bend over backwards to support everybody. Unfortunately, it
can't be helped, and the best people to handle the situation are the
library authors themselves. The only way to prevent this would be to
not release any new versions ever. In the case of python 3, there are
probably some who would prefer to have no new versions ever, but no
progress can be made without asking the question to move along with
the moving target or stay comfortable.

Now, a system to give developers access to more test configurations
would help make it easier to support more systems. This is the
approach of snakebite. But some giant network that automatically tests
all of pypi on all configurations is infeasible, somehow automatically
forcing libraries to work is not even possible.
 
Reply With Quote
 
Daniel Fetchinson
Guest
Posts: n/a
 
      04-23-2009
>>> Why? Why should every package on PyPI need to support all those Python
>>> versions? That should be the decision of the package maintainer. If
>>> they want to support every version of Python back to 1.0, they can, and
>>> if they want to only support version 2.5 that's fine too.

>>
>> Why shouldn't packages support more than one python version?

>
> They can. That depends on the developer of the package.


And if the developer decides to support multiple versions currently
there is no infrastructural help that would be available to him. On
the other hand, there is infrastructural help available to every
developer for distributing their apps: it's PyPI itself. You could
ask, why do we need PyPI? Let the developers distribute their code in
any way they like, python.org doesn't need to support this.

The OP is just thinking out loud that it would be great if developers
could count on some help for testing various platforms and versions.
And I agree, it would indeed be great.

>> Looking at it conversely....
>>
>> Why should the package developer dictacte which python version the
>> package will run on ?

>
> Because they're the developer. Who else should decide what Python
> versions to support? What are you going to do, hold a gun to their head
> and force them to support Python 2.1 when they've written it using 2.6
> features?


Nobody wants to point a gun to anyone's head. But if a developer says
to himself (without anybody holding a gun, remember) that it would be
great if there were ways to make sure that his code works on 2.5, 2.6
and 2.7 (and not 2.4 or before for some reason) because he has good
reasons to believe that users of all these 3 python versions want to
use his code, then it would be really great if this developer had some
help from python.org. He already has help for distribution (PyPI),
community advise (comp.lang.python), job postings, etc, a good
multiplatform/multiversion testing framework would be a nice addition.

It wouldn't be mandatory for developers, just as it is not mandatory
to upload packages to PyPI, but it would be nice if something like
this is available to those developers who want to use it.

>>> For all we know, three quarters of the packages on PyPI have never been
>>> tested *at all*.

>>
>> Right. Why not run some tests....

>
> Be my guest.
>
>
>>> What's the dire problem you are trying to solve?

>>
>> Backward and forward compatability of python package resources.

>
> That's not a problem, that's a feature.
>
> What's the *problem* that you are trying to solve? What bad thing will
> happen if we don't build your proposed system?


I fail to understand what is so mysterious about this. The problem is
the following: developer X wants to support python versions a, b, c, d
on platforms U, V, W. Developer X has no access to platforms V and W
and also has no access to python versions b, c and d. Developer X can
not run tests on these platforms and python versions although he
really wants to. This situation is what is commonly referred to as a
problem.

>>> What "Super-Computers" do you know of that run Python?

>>
>> Google. Amazon web services..

>
> They're not super computers. They're distributed networks of "regular"
> computers.
>
>
>>> Who is paying for this?

>>
>>>From as little as $30 per month. Funding isn't so much the issue.

>
> Whether it is $30 a month or $30,000 a month, who do you expect to pay it?
>
>
>>>> With this basic structure, a daily building/testing infrastructure
>>>> working across the different versions of python and operating systems,
>>>> could easily become a reality.
>>>
>>> Oh yeah, "easily".

>>
>> pypi_packagelist = getallpypipackages()
>>
>> for package in pypi_packagelist:
>> testpackageonallplatforms(package)

>
>
> You don't have either getallpypipackages() or
> testpackageonallplatforms(), so this is just Py in the Sky (pun intended)
> fantasizing. But okay, for the sake of the argument, let's pretend you
> have your cluster, and your test suite. You run the test, and get a list
> of 18,000 errors (that's an average of three errors per package). Now
> what?


This testing infrastructure should be available to developers who
choose to opt in. Once they run their code and notice that there are
errors, they will go back and fix these errors that otherwise would be
undetected until a real person runs into them. Or they could clearly
state what platform and python versions the code runs on for sure and
on what platforms and python versions the code fails. This would be a
great advantage to developers.

Actually, the OP's suggestion is a really good one, the people behind
snakebite.org realized this already some time ago. To me the idea is
very clear and obviously would make the life of developers a whole lot
easier and consequently the life of users as well.

Cheers,
Daniel



--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-23-2009
On Wed, 22 Apr 2009 23:47:08 -0700, Daniel Fetchinson wrote:

>>>> Why? Why should every package on PyPI need to support all those
>>>> Python versions? That should be the decision of the package
>>>> maintainer. If they want to support every version of Python back to
>>>> 1.0, they can, and if they want to only support version 2.5 that's
>>>> fine too.
>>>
>>> Why shouldn't packages support more than one python version?

>>
>> They can. That depends on the developer of the package.

>
> And if the developer decides to support multiple versions currently
> there is no infrastructural help that would be available to him. On the
> other hand, there is infrastructural help available to every developer
> for distributing their apps: it's PyPI itself. You could ask, why do we
> need PyPI? Let the developers distribute their code in any way they
> like, python.org doesn't need to support this.


They don't *need* to support distribution. That they do is a nice-to-
have, not a must-have -- there are other solutions, like SourceForge. We
can be grateful for PyPI without believing that it's a must-have.


> The OP is just thinking out loud that it would be great if developers
> could count on some help for testing various platforms and versions. And
> I agree, it would indeed be great.


To me, the OP doesn't seem to be asking "Wouldn't it be good if
developers could get access to whatever systems they wanted?" and more
like "Somebody needs to develop this amazing technology that will just
magically make every package on PyPI work on all these different
platforms, otherwise the Sky Will Fall". Just look at the subject line:
the issue is described as "a looming problem", it's suggested that
packages "might no longer work".

He's also worried about the combinational explosion of packages,
platforms and versions, which to my mind is only a problem if you think
that every package must work on every platform with every version of
Python.

I suspect you're primed to read the OP's suggestion in the most positive
light, because you're already aware of snakebite, and so you're seeing
his posts in terms of what snakebite is offering. I had never even heard
of snakebite before now, so I was seeing it in a very different light.


[...]
>>>> What's the dire problem you are trying to solve?
>>>
>>> Backward and forward compatability of python package resources.

>>
>> That's not a problem, that's a feature.
>>
>> What's the *problem* that you are trying to solve? What bad thing will
>> happen if we don't build your proposed system?

>
> I fail to understand what is so mysterious about this. The problem is
> the following: developer X wants to support python versions a, b, c, d
> on platforms U, V, W. Developer X has no access to platforms V and W and
> also has no access to python versions b, c and d. Developer X can not
> run tests on these platforms and python versions although he really
> wants to. This situation is what is commonly referred to as a problem.


That's *a* problem. It has many solutions, snakebite being just one. Is
it the "looming" problem the OP is concerned about?


> This testing infrastructure should be available to developers who choose
> to opt in. Once they run their code and notice that there are errors,
> they will go back and fix these errors that otherwise would be
> undetected until a real person runs into them. Or they could clearly
> state what platform and python versions the code runs on for sure and on
> what platforms and python versions the code fails. This would be a great
> advantage to developers.


It certainly would.

But the opt-in arrangement you are talking about doesn't sound much like
the OP's suggestion that some cross-platform build-bot goes out and pulls
in every single package on PyPI and tests them all on every combination
of platforms and versions.


> Actually, the OP's suggestion is a really good one, the people behind
> snakebite.org realized this already some time ago. To me the idea is
> very clear and obviously would make the life of developers a whole lot
> easier and consequently the life of users as well.


Snakebite certainly sounds beneficial for development of Python. I'm not
sure that it would scale to every developer with a package on PyPI, but
maybe someday.



--
Steven
 
Reply With Quote
 
Lawrence D'Oliveiro
Guest
Posts: n/a
 
      04-23-2009
In message <(E-Mail Removed)>, David
Lyon wrote:

> There are now over 6,000 packages listed on PyPi - and this number can
> only get bigger.
>
> Then, there are the three major operating systems:
>
> * Windows
> * Mac
> * Linix/Unix


Major Linux distros provide 20,000 or more packages each, without breaking a
sweat. What's your big deal?

> To complicate the problem, there are now many versions of each operating
> system.


The Linux solution is to leave distro packaging to the distro maintainers.
Release the source, and they will make up the necessary packages for their
specific one-click installers. You don't have to worry about it.

 
Reply With Quote
 
David Stanek
Guest
Posts: n/a
 
      04-23-2009
On Thu, Apr 23, 2009 at 2:47 AM, Daniel Fetchinson
<(E-Mail Removed)> wrote:
>
> The OP is just thinking out loud that it would be great if developers
> could count on some help for testing various platforms and versions.
> And I agree, it would indeed be great.
>


I think you interpreted the OP differently. As I said before the idea
is not a bad one, but as a package developer I get to decide which
platforms and versions my code supports.

--
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
 
Reply With Quote
 
David Stanek
Guest
Posts: n/a
 
      04-23-2009
On Thu, Apr 23, 2009 at 12:33 AM, David Lyon <(E-Mail Removed)> wrote:
> Hi Steve,
>>> Why should the package developer dictacte which python version the
>>> package will run on ?

>>
>> Because they're the developer. Who else should decide what Python
>> versions to support?

> The developer shouldn't be making such decisions at all....
> What hardware or operating systems we run his/her programs on is
> up to the real world to decide.
> If you think about it logically... why are we even asking our
> developers to even "build" their packages for specific python
> versions in the first place?
> They should just:
>


You should abandon this argument. Your original idea of a huge build
infrastructure had some merit. This is just off the wall.

If I use win32com how do you expect me to support Linux? What about
the many packages on PYPI containing C? What if I decide to write only
to Python 3? Who will support the other platforms if not the
developer?

--
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
You might be waiting a tad longer for that new NEX RichA Digital Photography 16 10-23-2011 08:32 PM
Re: Where to get stand alone Dot Net Framework version 1.1, version2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? MowGreen [MVP] ASP .Net 5 02-09-2008 01:55 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? PA Bear [MS MVP] ASP .Net 0 02-05-2008 03:28 AM
Re: Where to get stand alone Dot Net Framework version 1.1, version 2.0, version 3.0, version 3.5, version 2.0 SP1, version 3.0 SP1 ? V Green ASP .Net 0 02-05-2008 02:45 AM



Advertisments