Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   Re: help I'm getting delimited (http://www.velocityreviews.com/forums/t648905-re-help-im-getting-delimited.html)

Paul Watson 12-16-2008 04:38 PM

Re: help I'm getting delimited
 
On Tue, 2008-12-16 at 08:26 -0800, aka wrote:
> Hi, I'm going nuts over the csv.reader and UnicodeReader class.
> Somehow I can't get this method working which is supposed to read a
> csv file which name is inputted but here now hardcoded. What I need
> for now is that the string version of the list is put out for control.
> Later on I will only need to read the first column (id) of the csv
> file to be able to fill in a session var with a list of all ids.
> inp = c:/temp/test.csv
> roles = []
> try:
> fp = open(inp, 'rb')
> reader = csv.reader(fp)
> for r in reader:
> rollen.append(r)
> except:
> msg = "Er is iets mis met de UnicodeReader"
>
> return dict(file=in,roles=str(roles))
> Any help greatly appreciated!
> Cheers


Did you intend inside the loop to write:

roles.append(r)


MRAB 12-16-2008 05:50 PM

Re: help I'm getting delimited
 
Paul Watson wrote:
> On Tue, 2008-12-16 at 08:26 -0800, aka wrote:
>> Hi, I'm going nuts over the csv.reader and UnicodeReader class.
>> Somehow I can't get this method working which is supposed to read a
>> csv file which name is inputted but here now hardcoded. What I need
>> for now is that the string version of the list is put out for control.
>> Later on I will only need to read the first column (id) of the csv
>> file to be able to fill in a session var with a list of all ids.
>> inp = c:/temp/test.csv
>> roles = []
>> try:
>> fp = open(inp, 'rb')
>> reader = csv.reader(fp)
>> for r in reader:
>> rollen.append(r)
>> except:
>> msg = "Er is iets mis met de UnicodeReader"
>>
>> return dict(file=in,roles=str(roles))
>> Any help greatly appreciated!
>> Cheers

>
> Did you intend inside the loop to write:
>
> roles.append(r)
>

Also, the bare "except" will catch _all_ exceptions. You should catch
only those you expect. In this case, it's catching your use of "rollen"
instead of "roles" (probably unintentional) and then complaining about
UnicodeReader, even though that's (probably, again!) not the problem.

aka 12-17-2008 10:30 AM

Re: help I'm getting delimited
 
Due to being in a hurry I didn't paste correctly so I lost the try
clause (sorry).
The intention is to parse a csv file and (ultimately) put values of
column 1 ("id") in a list (so I need to append in the loop) that will
be used to fill a session var.
The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r) ## ultimately should be something like r.id
or r[0]
except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))

The roles list isn't populated at all :(

aka 12-17-2008 10:32 AM

Re: help I'm getting delimited
 
Due to being in a hurry I didn't paste correctly (sorry).
The intention is to parse a csv file and (ultimately) put values of
column 1 ("id") in a list (so I need to append in the loop) that will
be used to fill a session var.
The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r)
## ultimately should be something like r.id
## first row of csv file should be skipped because of column
names
or r[0]
except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))


The roles list isn't populated at all :(

aka 12-17-2008 10:39 AM

Re: help I'm getting delimited
 
Due to being in a hurry I didn't paste correctly (sorry).

The intention is to put values of column 1 ("id") in the roles list,
therefore appending within the loop, to fill a session var.

The complete code is:

roles = []
inp = 'C:/temp/test.csv'
try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r)
## ultimately should be something like r.id or r[0]
## first row of csv file should be skipped because of column
names

except:
msg = 'Something's wrong with the csv.reader'
return dict(file=inp,roles=str(roles))


The roles list isn't populated at all :(

John Machin 12-17-2008 12:44 PM

Re: help I'm getting delimited
 
On Dec 17, 9:39*pm, aka <alexoploca...@gmail.com> wrote:
> Due to being in a hurry I didn't paste correctly (sorry).
>
> The intention is to put values of column 1 ("id") in the roles list,
> therefore appending within the loop, to fill a session var.
>
> The complete code is:


It's *not* complete. It's missing "import csv".

>
> roles = []
> inp = 'C:/temp/test.csv'
> try:
> * * fp = open(inp, 'rb')
> * * reader = csv.reader(fp, dialect='excel', delimiter=';')
> * * for r in reader:
> * * * * roles.append(r)
> * * * * ## ultimately should be something like r.id or r[0]
> * * * * ## first row of csv file should be skipped because of column
> names
>
> except:
> * * msg = 'Something's wrong with the csv.reader'


But you don't print the message! In any case, using the try/except
like that *hides* any useful diagnostic information; it gives only an
indication that something is wrong, but not what is wrong and where it
is wrong.

If you throw away the try/except, you will get a more meaningful
message -- possibly that "csv" is not defined!! -- and the traceback
will tell you in which line the error occured.

> return dict(file=inp,roles=str(roles))


Why do you think that you need (a) that complicated expression (b) the
str() call? Assuming you are intending to make a function out of all
that, what's wrong with returning a (simple) tuple:
return inp, roles
?

The above 'return' statement is not inside a function/method. You
would have got this message:
SyntaxError: 'return' outside function
People will very soon lose patience with you if you persist in not
posting the actual code that you ran.

> The roles list isn't populated at all :(


This could mean (if the code that was posted is moderately similar to
that which was run) that the error happened before the first time that
roles.append(r) was executed ;-)

Please divulge the contents of test.csv -- but not if it's huge!
Considering trying to get your code to work first with a data file of
close-to-minimal size and complexity, like this:
8<---
id,other_info
tom,1
dick,2
harry,3
8<---

By the way, you mentioned the UnicodeReader class in your original
post, but you don't seem to use it ...

aka 12-17-2008 01:26 PM

Re: help I'm getting delimited
 
Hi John, thanks.
You're right, I didn't past the method header because I thought it
didn't matter when the input filename is hardcoded.
The try/except isn't very helpful indeed so I commented it out.
You're right I wrongly referred to the UnicodeReader
class in my first post because that's ultimately where I want to go so
I outcommented it here for you to see.
The fact is that neither csv.reader nor the UnicodeReader will read
the file, while writing with the UnicodeWriter
works like a charm.
That's why I put str() around roles to see any content.
I simplified the csv-file by cutting off columns without result. The
file looks now like:

id;company;department
12;Cadillac;Research
11;Ford;Accounting
10;Chrysler;Sales

The dictionary on the return is because this code is part of my
TurboGears application.
The entire method is:

import csv
from utilities.urw import UnicodeWriter, UnicodeReader

@expose(allow_json=True)
def import_roles(self, input=None, *args, **kwargs):
inp = 'C:/temp/test.csv'
roles = []
msg = ''
## try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r[0])
fp.close()
## except:
## msg = "Something's wrong with the csv.reader"
return dict(filepath=inp,
roles=str(roles),
msg=msg)

csv.reader results in: for r in myreader: Error: line contains NULL
byte

Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
can't decode byte 0xff in position 0: unexpected code byte

Will post only complete code from now on thanks.

aka 12-17-2008 02:28 PM

Re: help I'm getting delimited
 
Hi John, thanks.
You're right, I didn't past the method header because I thought it
didn't matter when the input filename is hardcoded.
The try/except isn't very helpful indeed so I commented it out.
You're right I wrongly referred to the UnicodeReader
class in my first post because that's ultimately where I want to go
so
I outcommented it here for you to see.
The fact is that neither csv.reader nor the UnicodeReader will read
the file, while writing with the UnicodeWriter
works like a charm.
That's why I put str() around roles to see any content.
I simplified the csv-file by cutting off columns without result. The
file looks now like:

id;company;department
12;Cadillac;Research
11;Ford;Accounting
10;Chrysler;Sales


The dictionary on the return is because this code is part of my
TurboGears application.
The entire method is:


import csv
from utilities.urw import UnicodeWriter, UnicodeReader


@expose(allow_json=True)
def import_roles(self, input=None, *args, **kwargs):
inp = 'C:/temp/test.csv'
roles = []
msg = ''
## try:
fp = open(inp, 'rb')
reader = csv.reader(fp, dialect='excel', delimiter=';')
## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
for r in reader:
roles.append(r[0])
fp.close()
## except:
## msg = "Something's wrong with the csv.reader"
return dict(filepath=inp,
roles=str(roles),
msg=msg)


csv.reader results in: for r in reader: Error: line contains NULL
byte


Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
can't decode byte 0xff in position 0: unexpected code byte


Will post only complete code from now on thanks.


John Machin 12-17-2008 03:23 PM

Re: help I'm getting delimited
 
On Dec 18, 1:28*am, aka <alexoploca...@gmail.com> wrote:
> Hi John, thanks.
> You're right, I didn't past the method header because I thought it
> didn't matter when the input filename is hardcoded.
> The try/except isn't very helpful indeed so I commented it out.
> You're right I wrongly referred to the UnicodeReader
> class in my first post because that's ultimately where I want to go
> so
> I outcommented it here for you to see.
> The fact is that neither csv.reader nor the UnicodeReader will read
> the file, while writing with the UnicodeWriter
> works like a charm.
> That's why I put str() around roles to see any content.
> I simplified the csv-file by cutting off columns without result. The
> file looks now like:
>
> id;company;department
> 12;Cadillac;Research
> 11;Ford;Accounting
> 10;Chrysler;Sales
>
> The dictionary on the return is because this code is part of my
> TurboGears application.
> The entire method is:
>
> import csv
> from utilities.urw * * * import UnicodeWriter, UnicodeReader


Pardon my ignorance, but what is "utilities.urw"??

>
> * * @expose(allow_json=True)


Means what? Does what?
Does the problem still happen without that?

Funny, the indentation changed there --- for the very last time, is
that the actual code of a standalone script that reproduces the
problem?


> * * def import_roles(self, input=None, *args, **kwargs):
> * * * * inp = 'C:/temp/test.csv'
> * * * * roles = []
> * * * * msg = ''
> * * * * ## try:
> * * * * fp = open(inp, 'rb')
> * * * * reader = csv.reader(fp, dialect='excel', delimiter=';')
> * * * * ## reader = UnicodeReader(fp, dialect='excel', delimiter=';')
> * * * * for r in reader:
> * * * * * * roles.append(r[0])
> * * * * fp.close()
> * * * * ## except:
> * * * * * * ## msg = "Something's wrong with the csv.reader"
> * * * * return dict(filepath=inp,
> * * * * * * * * * * roles=str(roles),
> * * * * * * * * * * msg=msg)
>
> csv.reader results in: for r in reader: Error: line contains NULL
> byte


Looks like the file is stuffed. Have you tried inspecting it with a
tool that would actually show a '\x00' or a '\xff' unambiguously? If
you don't have a fancy one, use the Python interactive prompt:

open('your_file.csv', 'rb').read()
>
> Use of UnicodeReader results in: UnicodeDecodeError: 'utf8' codec
> can't decode byte 0xff in position 0: unexpected code byte
>
> Will post only complete code from now on thanks.


Just make sure it's runnable and it's what you actually ran thanks.

rdmurray@bitdance.com 12-17-2008 03:51 PM

Re: help I'm getting delimited
 
Quoth John Machin <sjmachin@lexicon.net>:
> On Dec 18, 1:28 am, aka <alexoploca...@gmail.com> wrote:
> > @expose(allow_json=True)

>
> Means what? Does what?
> Does the problem still happen without that?


Means what he's posting is not a standalone script :)

He says it's part of his turbogears ap. @expose says that this
method is callable by name from a URL, and allow_json means
it can be called with a parameter requesting a json formatted
response instead of html.

> Funny, the indentation changed there --- for the very last time, is
> that the actual code of a standalone script that reproduces the
> problem?


Alex, I would strongly suggest that you move your code out into a
standalone script and debug it there (you'll get more help from this
group if you do, for one thing!). After you get it working
standalone you can incorporate it back into your Turbogears ap.

--RDM



All times are GMT. The time now is 12:44 AM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.