Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > how can i change the text delimiter

Reply
Thread Tools

how can i change the text delimiter

 
 
sonald
Guest
Posts: n/a
 
      08-30-2006
Hi,
Can anybody tell me how to change the text delimiter in FastCSV Parser
?
By default the text delimiter is double quotes(")
I want to change it to anything else... say a pipe (|)..
can anyone please tell me how do i go about it?

 
Reply With Quote
 
 
 
 
Amit Khemka
Guest
Posts: n/a
 
      08-30-2006
sonald <(E-Mail Removed)> wrote:
> Hi,
> Can anybody tell me how to change the text delimiter in FastCSV Parser
> ?
> By default the text delimiter is double quotes(")
> I want to change it to anything else... say a pipe (|)..
> can anyone please tell me how do i go about it?


You can use the parser constructor to specify the field seperator:
Python >>> parser(ms_double_quote = 1, field_sep = ',', auto_clear = 1)

cheers,
amit.

--
----
Amit Khemka -- onyomo.com
Home Page: www.cse.iitd.ernet.in/~csd00377
Endless the world's turn, endless the sun's Spinning, Endless the quest;
I turn again, back to my own beginning, And here, find rest.
 
Reply With Quote
 
 
 
 
Amit Khemka
Guest
Posts: n/a
 
      08-30-2006
On 8/30/06, Amit Khemka <(E-Mail Removed)> wrote:
> sonald <(E-Mail Removed)> wrote:
> > Hi,
> > Can anybody tell me how to change the text delimiter in FastCSV Parser
> > ?
> > By default the text delimiter is double quotes(")
> > I want to change it to anything else... say a pipe (|)..


Btw, I forgot to mention that AFAIK, in CSV '"' (double quotes) are
*not* field seperator delimiters character !

',' (comma) is the default field seperator, and my earlier suggestion
was considering that you want to change the default value for the
field seperator.

http://en.wikipedia.org/wiki/Comma-separated_values

cheers,
amit.
--
----
Amit Khemka -- onyomo.com
Home Page: www.cse.iitd.ernet.in/~csd00377
Endless the world's turn, endless the sun's Spinning, Endless the quest;
I turn again, back to my own beginning, And here, find rest.
 
Reply With Quote
 
sonald
Guest
Posts: n/a
 
      08-30-2006
Hi Amit,
Thanks for a quick response...
E.g record is: "askin"em"

This entire text is extracted as one string but since the qualifier is
double quotes("), therefore fastcsv parser is unable to parse it.

If we can change the text qualifier to pipe(|), then the string will
look like this:
|askin"em|

But for this the default text qualifier in fastcsv parser needs to be
changed to pipe(|). how to do this?

Also please note that the string cannot be modified at all. Thanks.


Amit Khemka wrote:
> sonald <(E-Mail Removed)> wrote:
> > Hi,
> > Can anybody tell me how to change the text delimiter in FastCSV Parser
> > ?
> > By default the text delimiter is double quotes(")
> > I want to change it to anything else... say a pipe (|)..
> > can anyone please tell me how do i go about it?

>
> You can use the parser constructor to specify the field seperator:
> Python >>> parser(ms_double_quote = 1, field_sep = ',', auto_clear = 1)
>
> cheers,
> amit.
>
> --
> ----
> Amit Khemka -- onyomo.com
> Home Page: www.cse.iitd.ernet.in/~csd00377
> Endless the world's turn, endless the sun's Spinning, Endless the quest;
> I turn again, back to my own beginning, And here, find rest.


 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      08-30-2006
"sonald" wrote:

> Thanks for a quick response...
> E.g record is: "askin"em"


that's usually stored as "askin""em" in a CSV file, and the csv module
has no problem handling that:

>>> import csv, StringIO
>>> source = StringIO.StringIO('"askin""em"\n')
>>> list(csv.reader(source))

[['askin"em']]

to use another quote character, use the quotechar option to the reader
function:

>>> source = StringIO.StringIO('|askin"em|\n')
>>> list(csv.reader(source, quotechar='|'))

[['askin"em']]

> Also please note that the string cannot be modified at all.


not even by the Python program that reads the data? sounds scary.

what's fastcsv, btw? the only thing google finds with that name is a
Ruby library...

</F>



 
Reply With Quote
 
skip@pobox.com
Guest
Posts: n/a
 
      08-30-2006

sonald> Can anybody tell me how to change the text delimiter in FastCSV
sonald> Parser? By default the text delimiter is double quotes(") I
sonald> want to change it to anything else... say a pipe (|).. can
sonald> anyone please tell me how do i go about it?

I'm not familiar with a FastCSV module in Python. Google suggests there is
a FastCsv module in Ruby.

Just in case what you are really referring to is the csv module in Python,
here's how you do it. Suppose your CSV file looks like an Excel-generated
CSV file save for the weird quote character:

import csv

class PipeQuote(csv.excel):
quotechar='|'

...

reader = csv.reader(open("somefile", "rb"), dialect=PipeQuote)
for row in reader:
print row

Full documentation is here:

http://docs.python.org/dev/lib/module-csv.html

Skip
 
Reply With Quote
 
sonald
Guest
Posts: n/a
 
      08-30-2006
Hi ,
thanks for the reply...

fast csv is the the csv module for Python...
and actually the string cannot be modified because
it is received from a third party and we are not supposed to modify the
data in any way..


for details on the fast CSV module please visit

www.object-craft.com.au/projects/csv/ or

import fastcsv
csv = fastcsv.parser(strict = 1,field_sep = ',') // part of
configuration

and somewhere in the code... we are using

data = csv.parse(line)

all i mean to say is, csv.reader is nowhere in the code
and somehow we got to modify the existing code.

looking forward to ur kind reply ...




Fredrik Lundh wrote:
> "sonald" wrote:
>
> > Thanks for a quick response...
> > E.g record is: "askin"em"

>
> that's usually stored as "askin""em" in a CSV file, and the csv module
> has no problem handling that:
>
> >>> import csv, StringIO
> >>> source = StringIO.StringIO('"askin""em"\n')
> >>> list(csv.reader(source))

> [['askin"em']]
>
> to use another quote character, use the quotechar option to the reader
> function:
>
> >>> source = StringIO.StringIO('|askin"em|\n')
> >>> list(csv.reader(source, quotechar='|'))

> [['askin"em']]
>
> > Also please note that the string cannot be modified at all.

>
> not even by the Python program that reads the data? sounds scary.
>
> what's fastcsv, btw? the only thing google finds with that name is a
> Ruby library...
>
> </F>


 
Reply With Quote
 
Fredrik Lundh
Guest
Posts: n/a
 
      08-30-2006
"sonald" wrote:

> fast csv is the the csv module for Python...


no, it's not. the csv module for Python is called "csv".

> and actually the string cannot be modified because
> it is received from a third party and we are not supposed to modify the
> data in any way..


that doesn't prevent you from using Python to modify it before you pass it to
the csv parser, though.

> for details on the fast CSV module please visit
>
> www.object-craft.com.au/projects/csv/ or


that module is called "csv", not "fastcsv". and as it says on that page, a much
improved version of that module was added to Python in version 2.3.

what Python version are you using?

</F>



 
Reply With Quote
 
skip@pobox.com
Guest
Posts: n/a
 
      08-30-2006

sonald> fast csv is the the csv module for Python... and actually the
sonald> string cannot be modified because it is received from a third
sonald> party and we are not supposed to modify the data in any way..

sonald> for details on the fast CSV module please visit

sonald> www.object-craft.com.au/projects/csv/ or

sonald> import fastcsv
sonald> csv = fastcsv.parser(strict = 1,field_sep = ',') // part of
sonald> configuration

sonald> and somewhere in the code... we are using

sonald> data = csv.parse(line)

sonald> all i mean to say is, csv.reader is nowhere in the code and
sonald> somehow we got to modify the existing code.

You're using Object Craft's old csv module. They are also the primary
authors of the csv module that's been shipped with Python since 2.3. I
suggest you convert your code to use the newer module if possible. If
that's not possible (it's not all that hard - I did it a couple years ago),
try posting a note to http://www.velocityreviews.com/forums/(E-Mail Removed). The Object Craft folks are on that
list and may be able to help you out. I think they're here as well, though
may not watch this group/list as closely.

Skip

 
Reply With Quote
 
sonald
Guest
Posts: n/a
 
      08-31-2006
Hi,
I am using
Python version python-2.4.1 and along with this there are other
installables
like:
1. fastcsv-1.0.1.win32-py2.4.exe
2. psyco-1.4.win32-py2.4.exe
3. scite-1.63-setup.exe

We are freshers here, joined new... and are now into handling this
module which validates the data files, which are provided in some
predefined format from the third party.
The data files are provided in the comma separated format.

The fastcsv package is imported in the code...
import fastcsv
and
csv = fastcsv.parser(strict = 1,field_sep = ',')

can u plz tell me where to find the parser function definition, (used
above)
so that if possible i can provide a parameter for
text qualifier or text separator or text delimiter..
just as {field_sep = ','} (as given above)

I want to handle string containing double quotes (")
but the problem is that the default text qualifier is double quote

Now if I can change the default text qualifier... to say pipe (|)
the double quote inside the string may be ignored...
plz refer to the example given in my previous query...

Thanks..






Fredrik Lundh wrote:
> "sonald" wrote:
>
> > fast csv is the the csv module for Python...

>
> no, it's not. the csv module for Python is called "csv".
>
> > and actually the string cannot be modified because
> > it is received from a third party and we are not supposed to modify the
> > data in any way..

>
> that doesn't prevent you from using Python to modify it before you pass it to
> the csv parser, though.
>
> > for details on the fast CSV module please visit
> >
> > www.object-craft.com.au/projects/csv/ or

>
> that module is called "csv", not "fastcsv". and as it says on that page, a much
> improved version of that module was added to Python in version 2.3.
>
> what Python version are you using?
>
> </F>


 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: How include a large array? Edward A. Falk C Programming 1 04-04-2013 08:07 PM
split problem if the delimiter is inside the text limiter rewonka Python 10 03-19-2009 03:24 PM
lining up delimiter separated text scooterm@hotmail.com Ruby 4 01-28-2008 06:29 PM
A Paradise DNS address change? What change? There was no change. Tony Neville NZ Computing 7 09-22-2006 01:02 PM
path delimiter in windows platform("/" could change to "\"?) Alont Perl Misc 19 09-28-2004 07:00 AM



Advertisments