Can I make sqlite3 or shelve work reliably on any Win/Linux/Mac?
 

Is there a way to have some kind of database (i.e. sqlite3, bsddb, dbm, etc.) that works out of the box on any Win/Linux/Mac machine with Python 2.6+ or 3.x? It's okay if the file format is different between machines, but I want my script to work without having to install anything.

Problems with current modules:

* Shelve used to do this. Unfortunately, since bsddb was deprecated/removed from the standard distro and Windows doesn't have dbm or gdbm, the only remaining option on Windows is dumbdbm, which is discouraged in the docs.

* Sqlite3 should fill the void now. However, in my experience, nearly every Linux Python install I encounter has a broken sqlite3 module ("ImportError: No module named _sqlite3"). It's a well-documented issue, but it the solution generally requires root access, which I don't have on these servers.

Potential solutions:

* Could I somehow bundle with my project the _sqlite3.so file and/or whatever else it needs? Or is there an alternate sqlite3 module I could use as a fallback that would just interface with the sqlite3 executable on the machine (i.e. /usr/local/bin/sqlite3)?

* Is there a way to drop bsddb into my project so it works out of the gate (no install) on either Linux, Windows, or Mac?

If you have any ideas, I'd be most appreciative. My objective here is just to find a portable and reliable solution that I can use for small projects.

Thanks,
Alex
--
http://alexquinn.org

Re: Can I make sqlite3 or shelve work reliably on anyWin/Linux/Mac?
 

On Mon, 22 Feb 2010 09:10:38 -0800, Alex Quinn wrote:

> Is there a way to have some kind of database (i.e. sqlite3, bsddb, dbm,
> etc.) that works out of the box on any Win/Linux/Mac machine with Python
> 2.6+ or 3.x? It's okay if the file format is different between machines,
> but I want my script to work without having to install anything.
>
> Problems with current modules:
>
> * Shelve used to do this. Unfortunately, since bsddb was
> deprecated/removed from the standard distro and Windows doesn't have dbm
> or gdbm, the only remaining option on Windows is dumbdbm, which is
> discouraged in the docs.
>
> * Sqlite3 should fill the void now. However, in my experience, nearly
> every Linux Python install I encounter has a broken sqlite3 module
> ("ImportError: No module named _sqlite3"). It's a well-documented issue,
> but it the solution generally requires root access, which I don't have
> on these servers.
>
> Potential solutions:
>
> * Could I somehow bundle with my project the _sqlite3.so file and/or
> whatever else it needs? Or is there an alternate sqlite3 module I could
> use as a fallback that would just interface with the sqlite3 executable
> on the machine (i.e. /usr/local/bin/sqlite3)?
>
> * Is there a way to drop bsddb into my project so it works out of the
> gate (no install) on either Linux, Windows, or Mac?
>
> If you have any ideas, I'd be most appreciative. My objective here is
> just to find a portable and reliable solution that I can use for small
> projects.
>
> Thanks,
> Alex

Hi,

I'm speaking with little experience here, but one thought I had is to try
compiling Pysqlite ( http://pypi.python.org/pypi/pysqlite/2.5.6 ) for
each of your target OS's, probably using GCC cross compilers for the
platforms you aren't compiling on, then sort of "splice" them together so
that _sqlite3 always points to the binary for the current OS.

This snippet might help you get started:

import sys

# Linux binary
if 'linux' in sys.platform.lower():
import _sqlite3_linux as _sqlite3

# Windows binary
elif 'win32' == sys.platform:
import _sqlite3_windows as _sqlite3

# Mac binary
elif 'darwin' == sys.platform:
import _sqlite3_mac as _sqlite3

sys.modules['_sqlite3'] = _sqlite3


I'm not exactly sure when you would run this code. It would have to be
sometime before you import the main sqlite3 module.

--
Brad Harms -- http://alphaios.net