Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: help I'm getting delimited

Reply
Thread Tools

Re: help I'm getting delimited

 
 
Paul Watson
Guest
Posts: n/a
 
      12-16-2008
On Tue, 2008-12-16 at 08:26 -0800, aka wrote:
> Hi, I'm going nuts over the csv.reader and UnicodeReader class.
> Somehow I can't get this method working which is supposed to read a
> csv file which name is inputted but here now hardcoded. What I need
> for now is that the string version of the list is put out for control.
> Later on I will only need to read the first column (id) of the csv
> file to be able to fill in a session var with a list of all ids.
> inp = c:/temp/test.csv
> roles = []
> try:
> fp = open(inp, 'rb')
> reader = csv.reader(fp)
> for r in reader:
> rollen.append(r)
> except:
> msg = "Er is iets mis met de UnicodeReader"
>
> return dict(file=in,roles=str(roles))
> Any help greatly appreciated!
> Cheers


Did you intend inside the loop to write:

roles.append(r)

 
Reply With Quote
 
 
 
 
MRAB
Guest
Posts: n/a
 
      12-16-2008
Paul Watson wrote:
> On Tue, 2008-12-16 at 08:26 -0800, aka wrote:
>> Hi, I'm going nuts over the csv.reader and UnicodeReader class.
>> Somehow I can't get this method working which is supposed to read a
>> csv file which name is inputted but here now hardcoded. What I need
>> for now is that the string version of the list is put out for control.
>> Later on I will only need to read the first column (id) of the csv
>> file to be able to fill in a session var with a list of all ids.
>> inp = c:/temp/test.csv
>> roles = []
>> try:
>> fp = open(inp, 'rb')
>> reader = csv.reader(fp)
>> for r in reader:
>> rollen.append(r)
>> except:
>> msg = "Er is iets mis met de UnicodeReader"
>>
>> return dict(file=in,roles=str(roles))
>> Any help greatly appreciated!
>> Cheers

>
> Did you intend inside the loop to write:
>
> roles.append(r)
>

Also, the bare "except" will catch _all_ exceptions. You should catch
only those you expect. In this case, it's catching your use of "rollen"
instead of "roles" (probably unintentional) and then complaining about
UnicodeReader, even though that's (probably, again!) not the problem.
 
Reply With Quote
 
 
 
 
aka
Guest
Posts: n/a
 
      12-17-2008
Due to being in a hurry I didn't paste correctly so I lost the try
clause (sorry).
The intention is to parse a csv file and (ultimately) put values of
column 1 ("id") in a list (so I need to append in the loop) that will
be used to fill a session var.
The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r) ## ultimately should be something like r.id
or r[0]
except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))

The roles list isn't populated at all
 
Reply With Quote
 
aka
Guest
Posts: n/a
 
      12-17-2008
Due to being in a hurry I didn't paste correctly (sorry).
The intention is to parse a csv file and (ultimately) put values of
column 1 ("id") in a list (so I need to append in the loop) that will
be used to fill a session var.
The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r)
## ultimately should be something like r.id
## first row of csv file should be skipped because of column
names
or r[0]
except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))


The roles list isn't populated at all
 
Reply With Quote
 
aka
Guest
Posts: n/a
 
      12-17-2008
Due to being in a hurry I didn't paste correctly (sorry).

The intention is to put values of column 1 ("id") in the roles list,
therefore appending within the loop, to fill a session var.

The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r)
## ultimately should be something like r.id or r[0]
## first row of csv file should be skipped because of column
names

except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))


The roles list isn't populated at all
 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      12-17-2008
On Dec 17, 9:39*pm, aka <(E-Mail Removed)> wrote:
> Due to being in a hurry I didn't paste correctly (sorry).
>
> The intention is to put values of column 1 ("id") in the roles list,
> therefore appending within the loop, to fill a session var.
>
> The complete code is:


It's *not* complete. It's missing "import csv".

>
> roles = []
> inp = 'C:/temp/test.csv'
> try:
> * * fp = open(inp, 'rb')
> * * reader = csv.reader(fp, dialect='excel', delimiter=';')
> * * for r in reader:
> * * * * roles.append(r)
> * * * * ## ultimately should be something like r.id or r[0]
> * * * * ## first row of csv file should be skipped because of column
> names
>
> except:
> * * msg = 'Something's wrong with the csv.reader'


But you don't print the message! In any case, using the try/except
like that *hides* any useful diagnostic information; it gives only an
indication that something is wrong, but not what is wrong and where it
is wrong.

If you throw away the try/except, you will get a more meaningful
message -- possibly that "csv" is not defined!! -- and the traceback
will tell you in which line the error occured.

> return dict(file=inp,roles=str(roles))


Why do you think that you need (a) that complicated expression (b) the
str() call? Assuming you are intending to make a function out of all
that, what's wrong with returning a (simple) tuple:
return inp, roles
?

The above 'return' statement is not inside a function/method. You
would have got this message:
SyntaxError: 'return' outside function
People will very soon lose patience with you if you persist in not
posting the actual code that you ran.

> The roles list isn't populated at all


This could mean (if the code that was posted is moderately similar to
that which was run) that the error happened before the first time that
roles.append(r) was executed

Please divulge the contents of test.csv -- but not if it's huge!
Considering trying to get your code to work first with a data file of
close-to-minimal size and complexity, like this:
8<---
id,other_info
tom,1
dick,2
harry,3
8<---

By the way, you mentioned the UnicodeReader class in your original
post, but you don't seem to use it ...
 
Reply With Quote
 
aka
Guest
Posts: n/a
 
      12-17-2008
Hi John, thanks.
You're right, I didn't past the method header because I thought it
didn't matter when the input filename is hardcoded.
The try/except isn't very helpful indeed so I commented it out.
You're right I wrongly referred to the UnicodeReader
class in my first post because that's ultimately where I want to go so
I outcommented it here for you to see.
The fact is that neither csv.reader nor the UnicodeReader will read
the file, while writing with the UnicodeWriter
works like a charm.
That's why I put str() around roles to see any content.
I simplified the csv-file by cutting off columns without result. The
file looks now like:

id;company;department
12;Cadillac;Research
11;Ford;Accounting
10;Chrysler;Sales

The dictionary on the return is because this code is part of my
TurboGears application.
The entire method is:

import csv
from utilities.urw import UnicodeWriter, UnicodeReader

@expose(allow_json=True)
def import_roles(self, input=None, *args, **kwargs):
inp = 'C:/temp/test.csv'
roles = []
msg = ''
## try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r[0])
fp.close()
## except:
## msg = "Something's wrong with the csv.reader"
return dict(filepath=inp,
roles=str(roles),
msg=msg)

csv.reader results in: for r in myreader: Error: line contains NULL
byte

Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
can't decode byte 0xff in position 0: unexpected code byte

Will post only complete code from now on thanks.
 
Reply With Quote
 
aka
Guest
Posts: n/a
 
      12-17-2008
Hi John, thanks.
You're right, I didn't past the method header because I thought it
didn't matter when the input filename is hardcoded.
The try/except isn't very helpful indeed so I commented it out.
You're right I wrongly referred to the UnicodeReader
class in my first post because that's ultimately where I want to go
so
I outcommented it here for you to see.
The fact is that neither csv.reader nor the UnicodeReader will read
the file, while writing with the UnicodeWriter
works like a charm.
That's why I put str() around roles to see any content.
I simplified the csv-file by cutting off columns without result. The
file looks now like:

id;company;department
12;Cadillac;Research
11;Ford;Accounting
10;Chrysler;Sales


The dictionary on the return is because this code is part of my
TurboGears application.
The entire method is:


import csv
from utilities.urw import UnicodeWriter, UnicodeReader


@expose(allow_json=True)
def import_roles(self, input=None, *args, **kwargs):
inp = 'C:/temp/test.csv'
roles = []
msg = ''
## try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r[0])
fp.close()
## except:
## msg = "Something's wrong with the csv.reader"
return dict(filepath=inp,
roles=str(roles),
msg=msg)


csv.reader results in: for r in reader: Error: line contains NULL
byte


Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
can't decode byte 0xff in position 0: unexpected code byte


Will post only complete code from now on thanks.

 
Reply With Quote
 
John Machin
Guest
Posts: n/a
 
      12-17-2008
On Dec 18, 1:28*am, aka <(E-Mail Removed)> wrote:
> Hi John, thanks.
> You're right, I didn't past the method header because I thought it
> didn't matter when the input filename is hardcoded.
> The try/except isn't very helpful indeed so I commented it out.
> You're right I wrongly referred to the UnicodeReader
> class in my first post because that's ultimately where I want to go
> so
> I outcommented it here for you to see.
> The fact is that neither csv.reader nor the UnicodeReader will read
> the file, while writing with the UnicodeWriter
> works like a charm.
> That's why I put str() around roles to see any content.
> I simplified the csv-file by cutting off columns without result. The
> file looks now like:
>
> id;company;department
> 12;Cadillac;Research
> 11;Ford;Accounting
> 10;Chrysler;Sales
>
> The dictionary on the return is because this code is part of my
> TurboGears application.
> The entire method is:
>
> import csv
> from utilities.urw * * * import UnicodeWriter, UnicodeReader


Pardon my ignorance, but what is "utilities.urw"??

>
> * * @expose(allow_json=True)


Means what? Does what?
Does the problem still happen without that?

Funny, the indentation changed there --- for the very last time, is
that the actual code of a standalone script that reproduces the
problem?


> * * def import_roles(self, input=None, *args, **kwargs):
> * * * * inp = 'C:/temp/test.csv'
> * * * * roles = []
> * * * * msg = ''
> * * * * ## try:
> * * * * fp = open(inp, 'rb')
> * * * * reader = csv.reader(fp, dialect='excel', delimiter=';')
> * * * * ## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
> * * * * for r in reader:
> * * * * * * roles.append(r[0])
> * * * * fp.close()
> * * * * ## except:
> * * * * * * ## msg = "Something's wrong with the csv.reader"
> * * * * return dict(filepath=inp,
> * * * * * * * * * * roles=str(roles),
> * * * * * * * * * * msg=msg)
>
> csv.reader results in: for r in reader: Error: line contains NULL
> byte


Looks like the file is stuffed. Have you tried inspecting it with a
tool that would actually show a '\x00' or a '\xff' unambiguously? If
you don't have a fancy one, use the Python interactive prompt:

open('your_file.csv', 'rb').read()
>
> Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
> can't decode byte 0xff in position 0: unexpected code byte
>
> Will post only complete code from now on thanks.


Just make sure it's runnable and it's what you actually ran thanks.
 
Reply With Quote
 
rdmurray@bitdance.com
Guest
Posts: n/a
 
      12-17-2008
Quoth John Machin <(E-Mail Removed)>:
> On Dec 18, 1:28 am, aka <(E-Mail Removed)> wrote:
> > @expose(allow_json=True)

>
> Means what? Does what?
> Does the problem still happen without that?


Means what he's posting is not a standalone script

He says it's part of his turbogears ap. @expose says that this
method is callable by name from a URL, and allow_json means
it can be called with a parameter requesting a json formatted
response instead of html.

> Funny, the indentation changed there --- for the very last time, is
> that the actual code of a standalone script that reproduces the
> problem?


Alex, I would strongly suggest that you move your code out into a
standalone script and debug it there (you'll get more help from this
group if you do, for one thing!). After you get it working
standalone you can incorporate it back into your Turbogears ap.

--RDM

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Help "joining" two files delimited with pipe character ("|") Miki Tebeka Python 0 12-05-2012 06:17 PM
Need help with my delimited stream wrapper class rep_movsd C++ 2 08-17-2011 07:16 PM
help I'm getting delimited aka Python 0 12-16-2008 04:26 PM
convert non-delimited to delimited RyanL Python 6 08-28-2007 12:06 AM



Advertisments