Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > ANNOUNCE: KirbyBase 1.7

Reply
Thread Tools

ANNOUNCE: KirbyBase 1.7

 
 
Jamey Cribbs
Guest
Posts: n/a
 
      01-30-2005
KirbyBase is a simple, plain-text, database management system written in
Python. It can be used either embedded in a python script or in a
client/server, multi-user mode. You use python code to express your
queries instead of having to use another language such as SQL.
KirbyBase is disk-based, not memory-based. Database changes are
immediately written to disk.

You can find more information on KirbyBase at:
http://www.netpromi.com/kirbybase.html

You can download KirbyBase for Python at:
http://www.netpromi.com/files/KirbyBase_Python_1.7.zip

Wow! It's been almost two years since the initial release of
KirbyBase. Time sure does fly! Version 1.7 includes most of the bug
fixes that have accumulated over the months and a few enhancements that
I hope you will enjoy.

I would like to thank everyone who has emailed me with comments, bug
reports, and enhancement requests/ideas. Hearing from people who
actually use KirbyBase is what makes working on it worthwhile. Please
keep the emails coming!

I would particularly like to thank Pierre Quentel, the author of
Karrigell (http://karrigell.sourceforge.net), for his contribution of
ideas and code for many of the enhancements in version 1.7 of KirbyBase.

For those of you who requested better documentation, the manual has been
completely re-written. I'm not saying it's any better, but at least
it's different.

Changes in Version 1.7:

***IMPORTANT - IF YOU ARE UPGRADING THIS COULD BITE YOU!!!***
* Changed the default value for the keyword argument 'useRegExp' to
be false instead of true. This means that, when doing a update,
delete, or select, records being selected on string fields will
be matched using exact matching instead of regular expression
matching. If you want to do regular expression matching, pass
'useRegExp = True' to the method.
***IMPORTANT***

* Added a keyword argument to select() called returnType. If set to
'object', the result list returned will contain Record objects
where each field name is an attribute of the object, so you could
refer to a record's field as plane.speed instead of plane[4]. If
set to 'dict', the result list returned will contain dictionaries
where each key is a field name and each value is a field value.
If set to 'list', the default, the result is a list of lists.

* Added a new method, insertBatch. It allows you to insert multiple
records at one time into a table. This greatly improves the speed
of batch inserts.

* Added a new public method called validate. Calling this method
with a table name will check each record of that table and
validate that all of the fields have values of the correct type.
This can be used to validate data you have put into the table by
means other than through KirbyBase, perhaps by opening the table
in a text editor and typing in information.

* Fixed a bug in _closeTable where if an exception occurred it was
blowing up because the variable 'name' did not exist.

* Fixed a bug in _writeRecord where if an exception occured it was
blowing up because the variable 'name' did not exist.

* Fixed a bug in _getMatches where I was referencing
self.field_names as a method instead of as a dictionary.

* Added a new private method, _strToBool, that converts string
values like 'True' to boolean values.

* Added a new private method, _convertInput, and moved to it the
code that ensures that the data on an insert is in proper list
format. I did this so that I did not have duplicate code in both
the insert and insertBatch methods.

* To accomodate the fact that users can now send a large batch of
records to be inserted, I changed _sendSocket so that it first
sends the length of the database command to the server, then it
actually sends the command itself, which can now be any length.

* Changed the code in _getMatches to precompile the regular
expression pattern instead of dynamically compiling every time the
pattern is compared to a table record. This should speed up
queries a little bit.

* Changed the code in select that converts table fields back to
their native types to be more efficient.

* Changed _sendSocket to use StringIO (actually cStringIO) to hold
the result set of a client/server-based query instead of just
capturing the result by concatenating records to one big string.
In informal testing on large result sets, it shaves a few tenths
of a second off the query time.


Jamey Cribbs
http://www.velocityreviews.com/forums/(E-Mail Removed)


 
Reply With Quote
 
 
 
 
Paul Rubin
Guest
Posts: n/a
 
      01-30-2005
Jamey Cribbs <(E-Mail Removed)> writes:
> KirbyBase is a simple, plain-text, database management system written
> in Python. It can be used either embedded in a python script or in a
> client/server, multi-user mode. You use python code to express your
> queries instead of having to use another language such as SQL.
> KirbyBase is disk-based, not memory-based. Database changes are
> immediately written to disk.


That's cute, especially the part about using Python expressions
instead of SQL to express queries. I don't see anything in the info
page about what happens when you have multiple clients updating the db
concurrently. Do you make any attempt to handle that?
 
Reply With Quote
 
 
 
 
Jamey Cribbs
Guest
Posts: n/a
 
      01-30-2005
Paul Rubin wrote:
> That's cute, especially the part about using Python expressions
> instead of SQL to express queries. I don't see anything in the info
> page about what happens when you have multiple clients updating the db
> concurrently. Do you make any attempt to handle that?


Yep. There are two server scripts included with the distribution. One
(kbsimpleserver.py) does serial, blocking requests, so there are no
concurrent-access issues. The second server script
(kbthreadedserver.py) is threaded and non-blocking. I have code in the
script that manages read and write locks for each table. I'm no rocket
scientist, but I have been using kbthreadedserver.py at work for several
months with no issues so far, so I am beginning to trust the code.

Jamey
 
Reply With Quote
 
Paul McGuire
Guest
Posts: n/a
 
      01-31-2005
"Jamey Cribbs" <(E-Mail Removed)> wrote in message
news:dMdLd.67932$(E-Mail Removed)...
> Paul Rubin wrote:
> > That's cute, especially the part about using Python expressions
> > instead of SQL to express queries. I don't see anything in the info
> > page about what happens when you have multiple clients updating the db
> > concurrently. Do you make any attempt to handle that?

>
> Yep. There are two server scripts included with the distribution. One
> (kbsimpleserver.py) does serial, blocking requests, so there are no
> concurrent-access issues. The second server script
> (kbthreadedserver.py) is threaded and non-blocking. I have code in the
> script that manages read and write locks for each table. I'm no rocket
> scientist, but I have been using kbthreadedserver.py at work for several
> months with no issues so far, so I am beginning to trust the code.
>
> Jamey


Before you get too confident, you might try your code on a multiprocessor
machine, under some heavy stress test (or ask one of your collaborators if
you don't have access to such a thing). Threaded code that runs on
uniprocessors can do very different/unexpected/unwanted things on
multiprocessors.

-- Paul


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
ANNOUNCE: KirbyBase 1.8 Jamey Cribbs Python 0 02-20-2005 10:06 PM
ANNOUNCE: KirbyBase 1.7.1 (Bugfix Release) Jamey Cribbs Python 0 01-31-2005 11:51 PM
ANNOUNCE: KirbyBase 1.5 Jamey Cribbs Python 0 09-04-2003 05:12 PM
ANNOUNCE: KirbyBase 1.4 Jamey Cribbs Python 0 08-27-2003 06:53 PM
ANNOUNCE: KirbyBase 1.3 Jamey Cribbs Python 0 08-16-2003 01:56 AM



Advertisments