Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > IndexedCatalog and ZEO

Reply
Thread Tools

IndexedCatalog and ZEO

 
 
Almad
Guest
Posts: n/a
 
      03-06-2005
Hello,

I'm trying to use IndexedCatalog
[http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
[http://www.cherrypy.org] application.
As Zodb support access just from one Python thread, I must either use just
one CherryPy thread (slow), or use ZEO. However, if I understand it good,
when I use
from ZEO import ClientStorage

then ZEO is internally using FileStorage, so no IndexedCatalog is used.

Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
ZEO for myself (or run this one thread)?

Thanks,
--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
 
Reply With Quote
 
 
 
 
Leif K-Brooks
Guest
Posts: n/a
 
      03-06-2005
Almad wrote:
> Hello,
>
> I'm trying to use IndexedCatalog
> [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
> [http://www.cherrypy.org] application.
> As Zodb support access just from one Python thread, I must either use just
> one CherryPy thread (slow), or use ZEO. However, if I understand it good,
> when I use
> from ZEO import ClientStorage
>
> then ZEO is internally using FileStorage, so no IndexedCatalog is used.


If you're using IndexedCatalog's Shelf class, just pass it a host/port
tuple instead of a path:

from IndexedCatalog.Shelf import Shelf
shelf = Shelf(('localhost', 1234), [Class1, Class2])
 
Reply With Quote
 
 
 
 
Diez B. Roggisch
Guest
Posts: n/a
 
      03-06-2005
> I'm trying to use IndexedCatalog
> [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
> [http://www.cherrypy.org] application.
> As Zodb support access just from one Python thread, I must either use just
> one CherryPy thread (slow), or use ZEO. However, if I understand it good,
> when I use
> from ZEO import ClientStorage
>
> then ZEO is internally using FileStorage, so no IndexedCatalog is used.
>
> Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
> ZEO for myself (or run this one thread)?


I'm not sure that I fully understand your problem - the usage of
IndexedCatalog is IMHO unrelated to threading issues of zodb. And it's not
true that zodb supports only one thread. I use it in a multi-threaded
environment - as does zope.

But it is necessary to use a separate connection object per thread. I've
written myself a transaction service alike to the java transaction api that
creates a connection for every call made to my server. In conjunction with
proxy-objects that keep a unique key for each object, I can happily access
zodb concurrently.

The basic idea is like this:

# open a zodb file storage and store a global reference
db = ....

def get_root():
# the module threadlocal is from the cookbook
conn = threadlocal.get("connection", None)
if conn is None:
conn = db.open()
threadlocal["connection"] = conn
return conn.root()

class Data(PersistenObject):
""" my data-object. It has a property id that identifies it uniquely """
...

class DataProxy(object):
def __init__(self, id):
self.id = id

def _g_data(self):
return get_root()[self.id]

data = property(_g_data)

def __getattr__(self, name):
return getattr(self.data, name)

The magic is in the get_root()-function that fetches a thread-local
connection and returns the root-object in my zodb. In my real application,
the data-object is cached as long as I'm in one transaction, so multiple
attribute accesses are faster.

I have to add that I have no expirience with zeo - so maybe that can also
solve your problems, but that will result in a local loop network access
that also slows down your application.

Regarding the usage of IndexedCatalog: It seems you can use it in
conjunction with zeo as it has no idea of whatever storage is needed - all
it does is indexing zodb objects - which you get from zeo as well. Of
course that means that you have to keep a catalog for every thread/process
that accesses the objects. Alternatively, you maybe can make the
IndexedCatalog a zodb-stored object itself, but I'm not sure about that.

--
Regards,

Diez B. Roggisch
 
Reply With Quote
 
Almad
Guest
Posts: n/a
 
      03-07-2005
Leif K-Brooks wrote:
> from IndexedCatalog.Shelf import Shelf
> shelf = Shelf(('localhost', 1234), [Class1, Class2])


Thanks, that is what I was searching for.

--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
 
Reply With Quote
 
Almad
Guest
Posts: n/a
 
      03-07-2005
Leif K-Brooks wrote:

> from IndexedCatalog.Shelf import Shelf
> shelf = Shelf(('localhost', 1234), [Class1, Class2])


Is it also possible to connect via socket instead of port? I've tried to
pass '/tmp/zeosocket' instead of ('localhost', 1234), but it isn't
successfull. Is is any way to do that?

Thank You,
--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
ZEO client hangs when combined with other asyncore code Paul Boots Python 0 06-21-2005 10:00 AM
ZEO client hangs when combined with other asyncore code Paul Boots Python 0 06-21-2005 09:55 AM
handling more databases with ZEO Almad Python 1 05-07-2005 10:28 AM
More ZEO/ZODB issues drs Python 0 07-12-2003 12:41 AM
ZEO and COM drs Python 0 07-11-2003 08:41 PM



Advertisments