Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Alternatives for pickle?

Reply
Thread Tools

Alternatives for pickle?

 
 
Antoon Pardon
Guest
Posts: n/a
 
      10-11-2004
I'm writing a little game, a gridler application, where you
can turn pixmaps into puzzle's and try to solve them. I already
have the data structure for such a puzzle worked out, one of
the problems is writing it to a file and reading it back in.

I first went to the pickle module but there I read this.

| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.


I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.

So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.

--
Antoon Pardon
 
Reply With Quote
 
 
 
 
Irmen de Jong
Guest
Posts: n/a
 
      10-11-2004
Antoon Pardon wrote:
> So how do you serialize data in python, when you want
> a somewhat secure mechanisme. Preferably where a user
> can make a puzzle file by hand in a text editor.


There's a YAML module for Python, although if I recall
correctly this module also suffers from security issues.

You could try Gnosis utils xml pickling or the xml marshaler
from the pyxml package. They're slow, but safe.

--Irmen

 
Reply With Quote
 
 
 
 
Robert Kern
Guest
Posts: n/a
 
      10-11-2004
Antoon Pardon wrote:
> I'm writing a little game, a gridler application, where you
> can turn pixmaps into puzzle's and try to solve them. I already
> have the data structure for such a puzzle worked out, one of
> the problems is writing it to a file and reading it back in.
>
> I first went to the pickle module but there I read this.
>
> | Warning: The pickle module is not intended to be secure against
> | erroneous or maliciously constructed data. Never unpickle data
> | received from an untrusted or unauthenticated source.
>
> But since this is for a game and people should be able to
> exchange puzzles, it seems a heavy requirement to ask of
> the users to check a puzzle file for security hazards.
>
>
> I also thought about writing out a string that, when read
> back in and fed to eval would recreate the structure. But
> that seems to be just as insecure if not more so.


Indeed. Don't do that.

> So how do you serialize data in python, when you want
> a somewhat secure mechanisme. Preferably where a user
> can make a puzzle file by hand in a text editor.


I think this is a case where you need to come up with your own file
format and parse it yourself. Pickle and other such mechanisms have
security problems because they are so general. They will create objects
that you don't want.

You can always jump on the XML bandwagon if that is convenient for you.
Python has XML modules in the standard library. Depending on the
complexity of the structure, it might even be convenient to edit by hand
in a text editor.

--
Robert Kern
http://www.velocityreviews.com/forums/(E-Mail Removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
 
Reply With Quote
 
Jeff Shannon
Guest
Posts: n/a
 
      10-12-2004
Antoon Pardon wrote:

>I also thought about writing out a string that, when read
>back in and fed to eval would recreate the structure. But
>that seems to be just as insecure if not more so.
>
>


As I understand it, this is *exactly* what pickle already does.

>So how do you serialize data in python, when you want
>a somewhat secure mechanisme. Preferably where a user
>can make a puzzle file by hand in a text editor.
>
>


I'd agree with the earlier comment -- define your own file format, and
write code to parse that format and instantiate the necessary objects.
If it's hard to define something that's both effective for your
purposes, and hard to hand-code in a text editor, then consider writing
a puzzle-editor app that will allow GUI creation of puzzles which can be
saved in your custom file format.

Jeff Shannon
Technician/Programmer
Credit International


 
Reply With Quote
 
Michael Foord
Guest
Posts: n/a
 
      10-12-2004
Antoon Pardon <(E-Mail Removed)> wrote in message news:<(E-Mail Removed)>...
> I'm writing a little game, a gridler application, where you
> can turn pixmaps into puzzle's and try to solve them. I already
> have the data structure for such a puzzle worked out, one of
> the problems is writing it to a file and reading it back in.
>
> I first went to the pickle module but there I read this.
>
> | Warning: The pickle module is not intended to be secure against
> | erroneous or maliciously constructed data. Never unpickle data
> | received from an untrusted or unauthenticated source.
>


Hmmm..... I wonder how easy it is to craft a malicious pickle that
will automatically run code objects just because they are unpickled.
My guess is that it's quite difficult - I've never heard of it *ever*
being done. Someone would have to be *very* malicious to work out how
to do it on the off chance of planting a back door into someone's
machine through a program like yours. No offence intended, but if they
were going to go to all that effort I expect they might aim for
something with a wider audience.

I would expect it to be 'safe enough', but that might not be safe
enough for you !

Creating your own data format is probably the way forward - and
probably not that difficult either.

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html

> But since this is for a game and people should be able to
> exchange puzzles, it seems a heavy requirement to ask of
> the users to check a puzzle file for security hazards.
>
>
> I also thought about writing out a string that, when read
> back in and fed to eval would recreate the structure. But
> that seems to be just as insecure if not more so.
>
> So how do you serialize data in python, when you want
> a somewhat secure mechanisme. Preferably where a user
> can make a puzzle file by hand in a text editor.

 
Reply With Quote
 
Shalabh Chaturvedi
Guest
Posts: n/a
 
      10-12-2004
Antoon Pardon wrote:
> | Warning: The pickle module is not intended to be secure against
> | erroneous or maliciously constructed data. Never unpickle data
> | received from an untrusted or unauthenticated source.
>
> But since this is for a game and people should be able to
> exchange puzzles, it seems a heavy requirement to ask of
> the users to check a puzzle file for security hazards.
>


http://twistedmatrix.com/products/spread#jelly

I haven't used it myself, though. In fact you might be able to use
twisted in other ways as well.

Shalabh

--
http://www.qlime.org

 
Reply With Quote
 
Paul Rubin
Guest
Posts: n/a
 
      10-12-2004
Antoon Pardon <(E-Mail Removed)> writes:
> So how do you serialize data in python, when you want
> a somewhat secure mechanisme. Preferably where a user
> can make a puzzle file by hand in a text editor.


There are a lot of different serialization formats in the Python
library but the general ones are not secure and the secure ones are
not general. You may have to concoct an ad-hoc format just for your
puzzles.
 
Reply With Quote
 
Bengt Richter
Guest
Posts: n/a
 
      10-13-2004
On 11 Oct 2004 08:26:12 GMT, Antoon Pardon <(E-Mail Removed)> wrote:

>I'm writing a little game, a gridler application, where you
>can turn pixmaps into puzzle's and try to solve them. I already
>have the data structure for such a puzzle worked out, one of
>the problems is writing it to a file and reading it back in.
>
>I first went to the pickle module but there I read this.
>
>| Warning: The pickle module is not intended to be secure against
>| erroneous or maliciously constructed data. Never unpickle data
>| received from an untrusted or unauthenticated source.
>
>But since this is for a game and people should be able to
>exchange puzzles, it seems a heavy requirement to ask of
>the users to check a puzzle file for security hazards.
>
>
>I also thought about writing out a string that, when read
>back in and fed to eval would recreate the structure. But
>that seems to be just as insecure if not more so.
>
>So how do you serialize data in python, when you want
>a somewhat secure mechanisme. Preferably where a user
>can make a puzzle file by hand in a text editor.
>

I would consider saving and retrieving your puzzle info in a simple csv format.
You can invent your own very simple interpreter based on lines of the
form
cmd, whatever...

the csv module has methods and options to control delimiters and quoting etc,
but e.g., by default:

>>> import csv
>>> lines = """

... this, can, be, a, command, format
... cmd, easy to edit with editor, note what happened to spaces
... cmd, arg, "quoted arg", etc
... do, something, else
... push, something on a stack
... set, something,to,a,value
... call, afunction, arg1, arg2, etc
... etc
... """
>>> rdr = csv.reader(lines.splitlines())
>>> for row in rdr: print row

...
[]
['this', ' can', ' be', ' a', ' command', ' format']
['cmd', ' easy to edit with editor', ' note what happened to spaces']
['cmd', ' arg', ' "quoted arg"', ' etc']
['do', ' something', ' else']
['push', ' something on a stack']
['set', ' something', 'to', 'a', 'value']
['call', ' afunction', ' arg1', ' arg2', ' etc']
['etc']

Anything that will iterate by lines should be ok to pass to csv.reader
so you can pass an open file, e.g., file('mypuzzlesetup.txt')

Obviously, you can interpret row[0] as operation and do what you like, e.g.,


>>> rdr = csv.reader("""\

... abs, 123
... abs, -123
... sum, 1,2,3
... sum, 100,200,5
... xxx, what?
... """.splitlines())

>>> for row in rdr:

... cmd = getattr(__builtins__, row[0], None)
... if cmd is None: print 're-edit your info:',row
... else:
... args = map(int, row[1:])
... if len(args)==1: args=args[0]
... print row,'=>', cmd(args)
...
['abs', ' 123'] => 123
['abs', ' -123'] => 123
['sum', ' 1', '2', '3'] => 6
['sum', ' 100', '200', '5'] => 305
re-edit your info: ['xxx', ' what?']

The csv module also has stuff to control delimiters and a writer method, etc.
See help(csv) interactively, after importing csv.

Note that no one can introduce an xxx do do anything weird, and if you
validate all the command formats, and don't do int conversions etc without
try/except, etc, you should be able to reject anything not safe, and write
informative error messages. You don't necessarily have to abort on any and
all errors, but you can have options for that if you want to get fancy.

HTH

Regards,
Bengt Richter
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
BGP alternatives? SP Cisco 3 09-08-2009 02:32 AM
LEAP & ACS Alternatives N. Hall Cisco 2 05-28-2005 08:31 AM
MS Press 2003 books and alternatives Bill Bixby MCSE 7 04-29-2004 05:52 PM
alternatives to accessing PIX via Telnet Anne Robynn Cisco 3 01-03-2004 08:48 AM
Alternatives when user's browser doesn't accept cookies Robert V. Hanson ASP .Net 2 07-03-2003 03:24 AM



Advertisments