Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > parse a string of parameters and values

Reply
Thread Tools

parse a string of parameters and values

 
 
bsneddon
Guest
Posts: n/a
 
      12-13-2009
I have a problem that I can come up with a brute force solution to
solve but it occurred to me that there may be an
"one-- and preferably only one --obvious way to do it".

I am going to read a text file that is an export from a control
system.
It has lines with information like

base=1 name="first one" color=blue

I would like to put this info into a dictionary for processing.
I have looked at optparse and getopt maybe they are the answer but
there could
be and very straight forward way to do this task.

Thanks for your help
 
Reply With Quote
 
 
 
 
Steven D'Aprano
Guest
Posts: n/a
 
      12-13-2009
On Sat, 12 Dec 2009 16:16:32 -0800, bsneddon wrote:

> I have a problem that I can come up with a brute force solution to solve
> but it occurred to me that there may be an
> "one-- and preferably only one --obvious way to do it".


I'm not sure that "brute force" is the right description here. Generally,
"brute force" is used for situations where you check every single
possible value rather than calculate the answer directly. One classical
example is guessing the password that goes with an account. The brute
force attack is to guess every imaginable password -- eventually you'll
find the matching one. A non-brute force attack is to say "I know the
password is a recent date", which reduces the space of possible passwords
from many trillions to mere millions.

So I'm not sure that brute force is an appropriate description for this
problem. One way or another you have to read every line in the file.
Whether you read them or you farm the job out to some pre-existing
library function, they still have to be read.


> I am going to read a text file that is an export from a control system.
> It has lines with information like
>
> base=1 name="first one" color=blue
>
> I would like to put this info into a dictionary for processing.


Have you looked at the ConfigParser module?

Assuming that ConfigParser isn't suitable, you can do this if each
key=value pair is on its own line:


d = {}
for line in open(filename, 'r'):
if not line.strip():
# skip blank lines
continue
key, value = line.split('=', 1)
d[key.strip()] = value.strip()


If you have multiple keys per line, you need a more sophisticated way of
splitting them. Something like this should work:

d = {}
for line in open(filename, 'r'):
if not line.strip():
continue
terms = line.split('=')
keys = terms[0::2] # every second item starting from the first
values = terms[1::2] # every second item starting from the second
for key, value in zip(keys, values):
d[key.strip()] = value.strip()




--
Steven
 
Reply With Quote
 
 
 
 
John Machin
Guest
Posts: n/a
 
      12-13-2009
Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:

>
> On Sat, 12 Dec 2009 16:16:32 -0800, bsneddon wrote:
>


>
> > I am going to read a text file that is an export from a control system.
> > It has lines with information like
> >
> > base=1 name="first one" color=blue
> >
> > I would like to put this info into a dictionary for processing.

>
> Have you looked at the ConfigParser module?
>
> Assuming that ConfigParser isn't suitable, you can do this if each
> key=value pair is on its own line:
> [snip]
> If you have multiple keys per line, you need a more sophisticated way of
> splitting them. Something like this should work:
>
> d = {}
> for line in open(filename, 'r'):
> if not line.strip():
> continue
> terms = line.split('=')
> keys = terms[0::2] # every second item starting from the first
> values = terms[1::2] # every second item starting from the second
> for key, value in zip(keys, values):
> d[key.strip()] = value.strip()
>


There appears to be a problem with the above snippet, or you have a strange
interpretation of "put this info into a dictionary":

| >>> line = 'a=1 b=2 c=3 d=4'
| >>> d = {}
| >>> terms = line.split('=')
| >>> print terms
| ['a', '1 b', '2 c', '3 d', '4']
| >>> keys = terms[0::2] # every second item starting from the first
| >>> values = terms[1::2] # every second item starting from the second
| >>> for key, value in zip(keys, values):
| ... d[key.strip()] = value.strip()
| ...
| >>> print d
| {'a': '1 b', '2 c': '3 d'}
| >>>

Perhaps you meant

terms = re.split(r'[= ]', line)

which is an improvement, but this fails on cosmetic spaces e.g. a = 1 b = 2 ...

Try terms = filter(None, re.split(r'[= ]', line))

Now we get to the really hard part: handling the name="first one" in the OP's
example. The splitting approach has run out of steam.

The OP will need to divulge what is the protocol for escaping the " character if
it is present in the input. If nobody knows of a packaged solution to his
particular scheme, then he'll need to use something like pyparsing.




 
Reply With Quote
 
Steven D'Aprano
Guest
Posts: n/a
 
      12-13-2009
On Sun, 13 Dec 2009 05:52:04 +0000, John Machin wrote:

> Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:

[snip]
>> If you have multiple keys per line, you need a more sophisticated way
>> of splitting them. Something like this should work:

[...]
> There appears to be a problem with the above snippet, or you have a
> strange interpretation of "put this info into a dictionary":



D'oh!

In my defence, I said it "should" work, not that it did work!


--
Steven
 
Reply With Quote
 
Peter Otten
Guest
Posts: n/a
 
      12-13-2009
bsneddon wrote:

> I have a problem that I can come up with a brute force solution to
> solve but it occurred to me that there may be an
> "one-- and preferably only one --obvious way to do it".
>
> I am going to read a text file that is an export from a control
> system.
> It has lines with information like
>
> base=1 name="first one" color=blue
>
> I would like to put this info into a dictionary for processing.
> I have looked at optparse and getopt maybe they are the answer but
> there could
> be and very straight forward way to do this task.
>
> Thanks for your help


Have a look at shlex:

>>> import shlex
>>> s = 'base=1 name="first one" color=blue equal="alpha=beta" empty'
>>> dict(t.partition("=")[::2] for t in shlex.split(s))

{'color': 'blue', 'base': '1', 'name': 'first one', 'empty': '', 'equal':
'alpha=beta'}

Peter
 
Reply With Quote
 
bsneddon
Guest
Posts: n/a
 
      12-13-2009
On Dec 13, 5:28*am, Peter Otten <(E-Mail Removed)> wrote:
> bsneddon wrote:
> > I have a problem that I can come up with a brute force solution to
> > solve but it occurred to me that there may be an
> > *"one-- and preferably only one --obvious way to do it".

>
> > I am going to read a text file that is an export from a control
> > system.
> > It has lines with information like

>
> > base=1 name="first one" color=blue

>
> > I would like to put this info into a dictionary for processing.
> > I have looked at optparse and getopt maybe they are the answer but
> > there could
> > be and very straight forward way to do this task.

>
> > Thanks for your help

>
> Have a look at shlex:
>
> >>> import shlex
> >>> s = 'base=1 name="first one" color=blue equal="alpha=beta" empty'
> >>> dict(t.partition("=")[::2] for t in shlex.split(s))

>
> {'color': 'blue', 'base': '1', 'name': 'first one', 'empty': '', 'equal':
> 'alpha=beta'}
>
> Peter


Thanks to all for your input.

It seems I miss stated the problem. Text is always quoted so blue
above -> "blue".

Peter,

The part I was missing was t.partition("=") and slicing skipping by
two.
It looks like a normal split will work for me to get the arguments I
need.
To my way of thinking your is very clean any maybe the "--obvious way
to do it"
Although it was not obvious to me until seeing your post.

Bill
 
Reply With Quote
 
Gabriel Genellina
Guest
Posts: n/a
 
      12-15-2009
En Sun, 13 Dec 2009 07:28:24 -0300, Peter Otten <(E-Mail Removed)>
escribió:
> bsneddon wrote:


>> I am going to read a text file that is an export from a control
>> system.
>> It has lines with information like
>>
>> base=1 name="first one" color=blue
>>
>> I would like to put this info into a dictionary for processing.

>
>>>> import shlex
>>>> s = 'base=1 name="first one" color=blue equal="alpha=beta" empty'
>>>> dict(t.partition("=")[::2] for t in shlex.split(s))

> {'color': 'blue', 'base': '1', 'name': 'first one', 'empty': '', 'equal':
> 'alpha=beta'}


Brilliant!

--
Gabriel Genellina

 
Reply With Quote
 
Tim Chase
Guest
Posts: n/a
 
      12-15-2009
Gabriel Genellina wrote:
> Peter Otten escribió:
>> bsneddon wrote:
>>> I am going to read a text file that is an export from a control
>>> system.
>>> It has lines with information like
>>>
>>> base=1 name="first one" color=blue
>>>
>>> I would like to put this info into a dictionary for processing.
>>>>> import shlex
>>>>> s = 'base=1 name="first one" color=blue equal="alpha=beta" empty'
>>>>> dict(t.partition("=")[::2] for t in shlex.split(s))

>> {'color': 'blue', 'base': '1', 'name': 'first one', 'empty': '', 'equal':
>> 'alpha=beta'}

>
> Brilliant!


The thing I appreciated about Peter's solution was learning a
purpose for .partition() as I've always just used .split(), so I
would have done something like

>>> dict('=' in s and s.split('=', 1) or (s, '') for s in

shlex.split(s))
{'color': 'blue', 'base': '1', 'name': 'first one', 'empty': '',
'equal': 'alpha=beta'}

Using .partition() makes that a lot cleaner. However, it looks
like .partition() was added in 2.5, so for my code stuck in 2.4
deployments, I'll stick with the uglier .split()

-tkc


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
parse values from a string gerberdata Ruby 10 01-19-2012 01:06 AM
optparse: parse v. parse! ?? 7stud -- Ruby 3 02-20-2008 05:20 AM
Is there a module to organize and parse command line parameters? Ulrich Scholz Java 2 09-14-2007 11:35 AM
POST to standard HTML and parse parameters in Javascript Jay Javascript 5 03-23-2006 09:29 PM
How to parse a string like C program parse the command line string? linzhenhua1205@163.com C Programming 19 03-15-2005 07:41 PM



Advertisments