«

Benjamin Jessup wrote:

> Hello all,
>
> What do people recommend for a file format for a python desktop
> application? Data is complex with 100s/1000s of class instances, which
> reference each other.
>
> Write the file with struct module? (Rebuild object pointers, safe,
> compact, portable, not expandable without reserved space)
>
> Use cPickle with a module/class whitelist? (Can't easily port, not
> entirely safe, compact enough, expandable)
>
> Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
>
> Any advice is greatly appreciated!


I would thinkyour options are pickle, json or database (either sqlite or
something like Postgres). I am unfamiliar with the struct module so I
cannot comment on its applicability.

I would guess that your data would be best saved by using a sqlite
database. Your biggest problemmight be how the different classes are
referencing each other. If you are using identifiers then any of these
options will probably work. If you are using aggregation then I know
that pickle will work (at least for built-in types). JSON will
keep the structure but duplicate elements.



>>> a = [ 1,2,3 ]
>>> b = [ 'a', 'b', 'c' ]
>>> a.append( b )
>>> e = [ a,b ]
>>> s = json.dumps( e )
>>> eret = json.loads( s )
>>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson

(329443808, 327677272)

>>> eret[0][3].append( 'o')
>>> eret[0][3], eret[1]

([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])

So pickle will be your easiest option, but I am not sure how well it
will scale with a large number items. Using sqlite/db should scale well
but it will take you longer/more effort to create a system for converting
your objects to and from the DB.



This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completenessof information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.

On Saturday, 29 September 2012 02:05:07 UTC+5:30, Prasad, Ramit wrote:
> Benjamin Jessup wrote:
>
> > Hello all,
>
> >
>
> > What do people recommend for a file format for a python desktop
>
> > application? Data is complex with 100s/1000s of class instances, which
>
> > reference each other.
>
> >
>
> > Write the file with struct module? (Rebuild object pointers, safe,
>
> > compact, portable, not expandable without reserved space)
>
> >
>
> > Use cPickle with a module/class whitelist? (Can't easily port, not
>
> > entirely safe, compact enough, expandable)
>
> >
>
> > Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
>
> >
>
> > Any advice is greatly appreciated!
>
>
>
> I would think your options are pickle, json or database (either sqlite or
>
> something like Postgres). I am unfamiliar with the struct module so I
>
> cannot comment on its applicability.
>
>
>
> I would guess that your data would be best saved by using a sqlite
>
> database. Your biggest problem might be how the different classes are
>
> referencing each other. If you are using identifiers then any of these
>
> options will probably work. If you are using aggregation then I know
>
> that pickle will work (at least for built-in types). JSON will
>
> keep the structure but duplicate elements.
>
>
>
>
>
> >>> a = [ 1,2,3 ]
>
> >>> b = [ 'a', 'b', 'c' ]
>
> >>> a.append( b )
>
> >>> e = [ a,b ]
>
> >>> s = json.dumps( e )
>
> >>> eret = json.loads( s )
>
> >>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
>
> (329443808, 327677272)
>
> >>> eret[0][3].append( 'o')
>
> >>> eret[0][3], eret[1]
>
> ([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])
>
>
>
> So pickle will be your easiest option, but I am not sure how well it
>
> will scale with a large number items. Using sqlite/db should scale well
>
> but it will take you longer/more effort to create a system for converting
>
> your objects to and from the DB.
>
>
>
>
>
>
>
> This email is confidential and subject to important disclaimers and
>
> conditions including on offers for the purchase or sale of
>
> securities, accuracy and completeness of information, viruses,
>
> confidentiality, legal privilege, and legal entity disclaimers,
>
> available at http://www.jpmorgan.com/pages/disclosures/email.

Guess I shouldn't read or rely your posts (virus'es could be in them)

On Saturday, 29 September 2012 02:05:07 UTC+5:30, Prasad, Ramit wrote:
> Benjamin Jessup wrote:
>
> > Hello all,
>
> >
>
> > What do people recommend for a file format for a python desktop
>
> > application? Data is complex with 100s/1000s of class instances, which
>
> > reference each other.
>
> >
>
> > Write the file with struct module? (Rebuild object pointers, safe,
>
> > compact, portable, not expandable without reserved space)
>
> >
>
> > Use cPickle with a module/class whitelist? (Can't easily port, not
>
> > entirely safe, compact enough, expandable)
>
> >
>
> > Use JSON or similar? (Rebuild object pointers, portable, expandable, size?)
>
> >
>
> > Any advice is greatly appreciated!
>
>
>
> I would think your options are pickle, json or database (either sqlite or
>
> something like Postgres). I am unfamiliar with the struct module so I
>
> cannot comment on its applicability.
>
>
>
> I would guess that your data would be best saved by using a sqlite
>
> database. Your biggest problem might be how the different classes are
>
> referencing each other. If you are using identifiers then any of these
>
> options will probably work. If you are using aggregation then I know
>
> that pickle will work (at least for built-in types). JSON will
>
> keep the structure but duplicate elements.
>
>
>
>
>
> >>> a = [ 1,2,3 ]
>
> >>> b = [ 'a', 'b', 'c' ]
>
> >>> a.append( b )
>
> >>> e = [ a,b ]
>
> >>> s = json.dumps( e )
>
> >>> eret = json.loads( s )
>
> >>> id(eret[0][3]), id(eret[1]) # Same result for json and simplejson
>
> (329443808, 327677272)
>
> >>> eret[0][3].append( 'o')
>
> >>> eret[0][3], eret[1]
>
> ([u'a', u'b', u'c', 'o'], [u'a', u'b', u'c'])
>
>
>
> So pickle will be your easiest option, but I am not sure how well it
>
> will scale with a large number items. Using sqlite/db should scale well
>
> but it will take you longer/more effort to create a system for converting
>
> your objects to and from the DB.
>
>
>
>
>
>
>
> This email is confidential and subject to important disclaimers and
>
> conditions including on offers for the purchase or sale of
>
> securities, accuracy and completeness of information, viruses,
>
> confidentiality, legal privilege, and legal entity disclaimers,
>
> available at http://www.jpmorgan.com/pages/disclosures/email.

Guess I shouldn't read or rely your posts (virus'es could be in them)