PyQT app accessible over network?
 

Hello all,

New guy here, with a kind of general question. Hopefully its not too
silly...

I've been working at learning python off and on now for a while, with a
couple programs in mind as a goal - kind of specialized stuff that I
can't seem to find a good match for already available, competitor
records, score-keeping & results for an amateur sports tournament. Many
places use some cobbled-together Excel spreadsheet, which has its
limitations. Others use an antiquated DOS-style application written in
PowerBasic that has issues of its own.

Probably 98-99% of the time the match administration would be done by a
single individual on a single PC, which seems like it would be nearly
ideal for a desktop application implemented in PyQt4 or similar. The
problem is (as usual) those edge cases where there are enough
volunteers/resources to have more than one person doing data entry
(maybe 2-3 in practice, but lets say 10-12 for arguments sake to pad
things a bit).

What I was wondering is what would be a good way of handling this with a
PyQt app? Build the desktop app first, and add some sort of
functionality to enable a lightweight web server and framework for the
additional data entry 'clients'? Or would it be better to create
dedicated PyQt client apps to connect to the PC/laptop running the
'main' application? Should I go a different direction entirely, with a
complete self-hosted webapp built on a framework like web2py?

As you can probably tell, I have only a vaguely fuzzy idea of 'how' at
this point... but I would like to be able to proceed with some
confidence that as I get further down the road I'm not going to run into
a dead-end and have to start over down a different path.

Thanks,

Monte

Re: PyQT app accessible over network?
 

> I've been working at learning python off and on now for a while, with
> a couple programs in mind as a goal - kind of specialized stuff that
> I can't seem to find a good match for already available, competitor
> records, score-keeping & results for an amateur sports tournament.

So you want to develop a database application. That's a standard case.

> Probably 98-99% of the time the match administration would be done by
> a single individual on a single PC, which seems like it would be
> nearly ideal for a desktop application implemented in PyQt4 or
> similar. The problem is (as usual) those edge cases where there are
> enough volunteers/resources to have more than one person doing data
> entry (maybe 2-3 in practice, but lets say 10-12 for arguments sake
> to pad things a bit).

PostgreSQL and the frameworks mentioned below don't care for the number
of clients. You could buy a zSeries (or whatever they are called now)
from IBM and serve thousands of clients simultaneously if you needed to.

> What I was wondering is what would be a good way of handling this
> with a PyQt app? Build the desktop app first, and add some sort of
> functionality to enable a lightweight web server and framework for
> the additional data entry 'clients'?

No, you just implement a GUI in whatever GUI framework you want (PyQt,
PyGTK, wxPython) and use a client-server RDBMS for storage.

No web-nonsense gadgetry required with bloated cursor-animation
"browsers" etc..

For the storage I recommend PostgreSQL, for the client GUI, there are
several frameworks available:

using PyQt (& Sqlalchemy):
Pypapi: www.pypapi.org
Camelot: www.python-camelot.com
Qtalchemy: www.qtalchemy.org

using PyGTK:
Sqlkit: sqlkit.argolinux.org (also uses Sqlalchemy)
Kiwi: www.async.com.br/projects/kiwi
Glom: www.glom.org

using wxPython:
Dabo: www.dabodev.com
Defis: sourceforge.net/projects/defis (Russian only)
GNUe: www.gnuenterprise.org

Pypapi, Camelot, Sqlkit and Dabo seem to be the most active and best
documented/supported ones.

Sincerely,

Wolfgang

Re: PyQT app accessible over network?
 

Yes, I am looking at a database-centric application. I know that the
'larger' databases such as PostgreSQL, MySQL, etc. would not have any
problem handling that small amount of traffic.

My concern is that using postgres or mysql for this would be akin to
using a sledgehammer to swat a fly, when sqlite could most likely handle
the load well enough (I think) since the handful of people doing data
entry would rarely (if ever) be trying to write to the same record.
That would be the whole point of having multiple people doing data entry
in this situation - each one handling a different competitors entry form
or submitted scores.

My other reason for wanting one 'central' app is that there are various
functions (setting up the tournament, closing registration, editing
scores, finalizing results) that I really *don't* want the
satellite/client apps to be able to do. My personal view is that sort
of thing needs to be handled from one point, by one person (the match
director or chief stats officer, depending on the size of the event).

That is why I was looking at things in terms of having one central app
that handles the database, whether locally via sqlite or postgres or
whatever, but have the clients access go through that main application
in order to ensure that all they have is a limited set of CRUD abilities
for competitor registration and entering scores.

Thanks for the links... some of those I was already aware of (Camelot,
Dabo) but some of the others are new (QtAlchemy, etc). Should make for
interesting reading!

Thanks,

Monte

Re: PyQT app accessible over network?
 

Monte: I noticed you mentioned web2py; that would be my recommendation.

You also mention different features being available to different
users; perfect use-case for web2py's built-in RBAC.

Scalability: Go with Postgres, MySQL; or considering how much data
you're talking about, even SQLite would be a close enough fit!

Another advantage of sticking to the web that hasn't been mentioned so
far is agnostic interoperability.

E.g.: you can CRUD on your TV (e.g.: if it runs Android); or on your
phone (e.g.: if you use twitter-bootstrap; which web2py comes with out
of the box; but is usable in any framework)

On Sat, Feb 23, 2013 at 3:50 AM, Monte Milanuk <memilanuk@gmail.com> wrote:
> Yes, I am looking at a database-centric application. I know that the
> 'larger' databases such as PostgreSQL, MySQL, etc. would not have any
> problem handling that small amount of traffic.
>
> My concern is that using postgres or mysql for this would be akin to using a
> sledgehammer to swat a fly, when sqlite could most likely handle the load
> well enough (I think) since the handful of people doing data entry would
> rarely (if ever) be trying to write to the same record. That would be the
> whole point of having multiple people doing data entry in this situation -
> each one handling a different competitors entry form or submitted scores.
>
> My other reason for wanting one 'central' app is that there are various
> functions (setting up the tournament, closing registration, editing scores,
> finalizing results) that I really *don't* want the satellite/client apps to
> be able to do. My personal view is that sort of thing needs to be handled
> from one point, by one person (the match director or chief stats officer,
> depending on the size of the event).
>
> That is why I was looking at things in terms of having one central app that
> handles the database, whether locally via sqlite or postgres or whatever,
> but have the clients access go through that main application in order to
> ensure that all they have is a limited set of CRUD abilities for competitor
> registration and entering scores.
>
> Thanks for the links... some of those I was already aware of (Camelot, Dabo)
> but some of the others are new (QtAlchemy, etc). Should make for
> interesting reading!
>
>
> Thanks,
>
> Monte
>
> --
> http://mail.python.org/mailman/listinfo/python-list

Re: PyQT app accessible over network?
 

On 02/22/2013 08:57 AM, Alec Taylor wrote:
> Monte: I noticed you mentioned web2py; that would be my recommendation.
>
> You also mention different features being available to different
> users; perfect use-case for web2py's built-in RBAC.
>
> Scalability: Go with Postgres, MySQL; or considering how much data
> you're talking about, even SQLite would be a close enough fit!
>
> Another advantage of sticking to the web that hasn't been mentioned so
> far is agnostic interoperability.
>
> E.g.: you can CRUD on your TV (e.g.: if it runs Android); or on your
> phone (e.g.: if you use twitter-bootstrap; which web2py comes with out
> of the box; but is usable in any framework)
>


Web2py does seem pretty attractive in that it seems to come with a lot
of functionality rolled in already. It seems to be pretty easy to
deploy... since this would be more of a case where the volunteer match
directors are not necessarily computer gurus, and something that can
literally run from a USB stick on nearly any computer has its benefits.
I've seen some examples (I think) of twitter-bootstrap in some other
demos of flask, and it looked reasonably attractive without being too
over the top. web2py's DAL seems fairly straight-forward too. Looks
like I may have to get more fluent in CSS & javascript, though...

Re: PyQT app accessible over network?
 

On 02/22/2013 02:49 PM, Monte Milanuk wrote:
> Web2py does seem pretty attractive in that it seems to come with a lot
> of functionality rolled in already. It seems to be pretty easy to
> deploy... since this would be more of a case where the volunteer match
> directors are not necessarily computer gurus, and something that can
> literally run from a USB stick on nearly any computer has its benefits.
> I've seen some examples (I think) of twitter-bootstrap in some other
> demos of flask, and it looked reasonably attractive without being too
> over the top. web2py's DAL seems fairly straight-forward too. Looks
> like I may have to get more fluent in CSS & javascript, though...

If you just use web2py to implement the database calls and business
logic, and to implement a simple, clean API (RPC really) for the clients
to talk to, then you can still use your non-web UI tools like PyQt. But
as an added bonus you can do a web interface as well. You'll have
flexibility either way. A client is a client, whether it's web-bases
and running on the same server, or a remote app using RPC over HTTP.

I think all web-based apps should expose a web service (an API). that
way you have flexibility to do a variety of front-ends. Normal web
browser, mobile browser, a standalone app (think android or iphone).

As far as doing client/server stuff with just a database engine, unless
you have tight control over the environment end to end, from a security
pov, it's not a good idea to expose the database engine itself to the
internet. Better to put a restricted web services API in front of it
that handles all the authorization needs (access-control) on the
detailed level that you require.

Re: PyQT app accessible over network?
 

On Fri, 22 Feb 2013 08:50:05 -0800, Monte Milanuk <memilanuk@gmail.com>
declaimed the following in gmane.comp.python.general:


> My concern is that using postgres or mysql for this would be akin to
> using a sledgehammer to swat a fly, when sqlite could most likely handle
> the load well enough (I think) since the handful of people doing data
> entry would rarely (if ever) be trying to write to the same record.
> That would be the whole point of having multiple people doing data entry
> in this situation - each one handling a different competitors entry form
> or submitted scores.
>

Problem: SQLite3 (and M$ JET/Access) are considered "file server"
databases. Each instance of the program accessing the database is
directly opening the database file(s). While SQLite3 has a fairly
complex locking system, the normal locking is NOT "per record". Instead
it allows for multiple readers to be active at once; the first
connection/cursor to attempt to write anything will block any new
attempts to read, and will be blocked until all other active readers
exit (and none of those other readers can attempt to write). When there
are no other open readers, the writer can finish and commit changes.

> That is why I was looking at things in terms of having one central app
> that handles the database, whether locally via sqlite or postgres or
> whatever, but have the clients access go through that main application
> in order to ensure that all they have is a limited set of CRUD abilities
> for competitor registration and entering scores.
>
At which point you've essentially written the conflict management
that a client/server system already provides.
--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/

Re: PyQT app accessible over network?
 

On Sat, Feb 23, 2013 at 11:20 AM, Dennis Lee Bieber
<wlfraed@ix.netcom.com> wrote:
> Problem: SQLite3 (and M$ JET/Access) are considered "file server"
> databases. Each instance of the program accessing the database is
> directly opening the database file(s). While SQLite3 has a fairly
> complex locking system, the normal locking is NOT "per record". Instead
> it allows for multiple readers to be active at once; the first
> connection/cursor to attempt to write anything will block any new
> attempts to read, and will be blocked until all other active readers
> exit (and none of those other readers can attempt to write). When there
> are no other open readers, the writer can finish and commit changes.

Also MySQL, when using the default MyISAM back-end. In contrast,
PostgreSQL uses MVCC to permit lock-free reading in most cases (you
keep reading the version you can "see", and a writer happily tinkers
with a new version; until the writer COMMITs, its version is invisible
to you). There's more overhead to the PostgreSQL system, but it scales
better with multiple writers. (MySQL is primarily designed for dynamic
web sites, where there are thousands of readers but only (relatively)
occasional writers.)

ChrisA

Re: PyQT app accessible over network?
 

On Sat, Feb 23, 2013 at 10:37 AM, Michael Torrie <torriem@gmail.com> wrote:
> On 02/22/2013 02:49 PM, Monte Milanuk wrote:
>> Web2py does seem pretty attractive in that it seems to come with a lot
>> of functionality rolled in already. It seems to be pretty easy to
>> deploy... since this would be more of a case where the volunteer match
>> directors are not necessarily computer gurus, and something that can
>> literally run from a USB stick on nearly any computer has its benefits.
>> I've seen some examples (I think) of twitter-bootstrap in some other
>> demos of flask, and it looked reasonably attractive without being too
>> over the top. web2py's DAL seems fairly straight-forward too. Looks
>> like I may have to get more fluent in CSS & javascript, though...
>
> If you just use web2py to implement the database calls and business
> logic, and to implement a simple, clean API (RPC really) for the clients
> to talk to, then you can still use your non-web UI tools like PyQt. But
> as an added bonus you can do a web interface as well. You'll have
> flexibility either way. A client is a client, whether it's web-bases
> and running on the same server, or a remote app using RPC over HTTP.
>
> I think all web-based apps should expose a web service (an API). that
> way you have flexibility to do a variety of front-ends. Normal web
> browser, mobile browser, a standalone app (think android or iphone).
>
> As far as doing client/server stuff with just a database engine, unless
> you have tight control over the environment end to end, from a security
> pov, it's not a good idea to expose the database engine itself to the
> internet. Better to put a restricted web services API in front of it
> that handles all the authorization needs (access-control) on the
> detailed level that you require.
> --
> http://mail.python.org/mailman/listinfo/python-list

Michael Torrie: Have seen a few PyWt examples in alpha if that's what
you describing…

But there would still be more implementation overhead then just using
e.g.: SQLFORM(db.table_name) to create a CRUD form.

I don't see any disadvantage of using web2py for everything; unless
we're talking decentralised infrastructure in which case a queuing
mechanism would likely be better; and have each client implement a
server as well. (thus still no use-case for Qt).

Also SQLite has a number of excellent features, namely 2 file deployments.

So it's very portable. Otherwise for postgres or mysql you'd probably
need to package in your own silent installer (which admittedly isn't
overly difficult; but is quite involved)…

On Sat, Feb 23, 2013 at 8:49 AM, Monte Milanuk <memilanuk@gmail.com> wrote:
> Looks like I may have to get more fluent in
> CSS & javascript, though...

Understanding how `style` attributes work, how to use FireBug (or
Chrome Dev Tools); and finding a good javascript widget library (e.g.:
from Twitter Bootstrap) should be more than enough for your project.

In fact; it's been enough for almost all my projects!

(though now I'm moving to AngularJS will need to get more involved on
the js front :P)

Re: PyQT app accessible over network?
 

> My concern is that using postgres or mysql for this would be akin to
> using a sledgehammer to swat a fly,

I wouldn't use MySQL for anything that requires anything else than
"select".

And PostgreSQL has extremely spartanic resource requirements in the
default configuration. It runs on Linux on hardware where (the
most recent) Windows alone wouldn't run.

> My other reason for wanting one 'central' app is that there are
> various functions (setting up the tournament, closing registration,
> editing scores, finalizing results) that I really *don't* want the
> satellite/client apps to be able to do.

Easy, you simply restrict access rights to the corresponding tables
for the individual users. Any halfway decent database application
framework will allow to configure the application correspondingly for
each user.

Sincerely,

Wolfgang