Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Python (http://www.velocityreviews.com/forums/f43-python.html)
-   -   help with lists and writing to file in correct order (http://www.velocityreviews.com/forums/t352826-help-with-lists-and-writing-to-file-in-correct-order.html)

homepricemaps@gmail.com 12-26-2005 09:54 PM

help with lists and writing to file in correct order
 
hey folks,

have a logic question for you. appreciate the help in advance.

i am scraping 3 pieces of information from the html namely the food
name , store name and price. and i am doing this for many different
food items found ni the html including pizza, burgers, fries etc. what
i want is to write out to a text file in the following order:

pizza, pizza hut, 3.00
burgers, burger king, 4.00
noodles, panda inn, 2.00

html is below. does anyone have good recommendation for how to setup
the code in such a manner where it writes to the text file in th order
listed previously? any attempt i have made seems to write to the file
like this

noodles, panda inn, 3
noodles, panda inn, 4
noodles, panda inn, 2


HTML
<tr class="base"><td class="tall"><a name="D0L1" "href="his/food"
target="_blank">

<td class="desc"><h2 id="foodName">pizza</h2>

<div class="store"><a name="D0L3" "href="/xPopups/nojs"
target="_blank"><b>pizza hutt</b></a></div>

<td class="price">3.00</td>
<tr>


Steven D'Aprano 12-27-2005 01:27 AM

Re: help with lists and writing to file in correct order
 
On Mon, 26 Dec 2005 13:54:37 -0800, homepricemaps wrote:

> hey folks,
>
> have a logic question for you. appreciate the help in advance.
>
> i am scraping 3 pieces of information from the html namely the food
> name , store name and price. and i am doing this for many different
> food items found ni the html including pizza, burgers, fries etc. what
> i want is to write out to a text file in the following order:
>
> pizza, pizza hut, 3.00
> burgers, burger king, 4.00
> noodles, panda inn, 2.00
>
> html is below. does anyone have good recommendation for how to setup
> the code in such a manner where it writes to the text file in th order
> listed previously? any attempt i have made seems to write to the file
> like this
>
> noodles, panda inn, 3
> noodles, panda inn, 4
> noodles, panda inn, 2


Instead of posting the HTML, how about if you post your code? Unless we
see your code, how do you expect us to find the bug in it?



--
Steven.


homepricemaps@gmail.com 12-27-2005 01:44 AM

Re: help with lists and writing to file in correct order
 
sorry guys, here is the code

for incident in bs('a', {'class' : 'price'}):
price = ""
for oText in incident.fetchText( oRE):
price += oText.strip() + "','"

for incident in bs('div', {'class' : 'store'}):
store = ""
for oText in incident.fetchText( oRE):
store += oText.strip() + "','"

for incident in bs('h2', {'id' : 'food'}):
food = ""
for oText in incident.fetchText( oRE):
food += oText.strip() + "','"


Steven D'Aprano 12-27-2005 02:08 AM

Re: help with lists and writing to file in correct order
 
On Mon, 26 Dec 2005 17:44:43 -0800, homepricemaps wrote:

> sorry guys, here is the code
>
> for incident in bs('a', {'class' : 'price'}):
> price = ""
> for oText in incident.fetchText( oRE):
> price += oText.strip() + "','"
>
> for incident in bs('div', {'class' : 'store'}):
> store = ""
> for oText in incident.fetchText( oRE):
> store += oText.strip() + "','"
>
> for incident in bs('h2', {'id' : 'food'}):
> food = ""
> for oText in incident.fetchText( oRE):
> food += oText.strip() + "','"



This is hardly all your code -- where is the part where you actually
*write* something to the file? The problem is you are writing the same
store and food to the file over and over again. After you have collected
one line of store/food, you must write it to the file immediately, or at
least save it in a list so you can write the lot at the end.


--
Steven.


homepricemaps@gmail.com 12-27-2005 03:07 AM

Re: help with lists and writing to file in correct order
 
here is the write part:

out = open("test.txt", 'a')
out.write (store+ food+ price + "\n")
out.close()


Steven D'Aprano wrote:
> On Mon, 26 Dec 2005 17:44:43 -0800, homepricemaps wrote:
>
> > sorry guys, here is the code
> >
> > for incident in bs('a', {'class' : 'price'}):
> > price = ""
> > for oText in incident.fetchText( oRE):
> > price += oText.strip() + "','"
> >
> > for incident in bs('div', {'class' : 'store'}):
> > store = ""
> > for oText in incident.fetchText( oRE):
> > store += oText.strip() + "','"
> >
> > for incident in bs('h2', {'id' : 'food'}):
> > food = ""
> > for oText in incident.fetchText( oRE):
> > food += oText.strip() + "','"

>
>
> This is hardly all your code -- where is the part where you actually
> *write* something to the file? The problem is you are writing the same
> store and food to the file over and over again. After you have collected
> one line of store/food, you must write it to the file immediately, or at
> least save it in a list so you can write the lot at the end.
>
>
> --
> Steven.



homepricemaps@gmail.com 12-27-2005 04:01 AM

Re: help with lists and writing to file in correct order
 
the problem with writing to teh file immidiately is that it ends up
writing all food items together, and then all store items and then all
prices

i want

food, store, price
food, store, price


Scott David Daniels 12-27-2005 04:23 AM

Re: help with lists and writing to file in correct order
 
homepricemaps@gmail.com wrote:
> the problem with writing to teh file immidiately is that it ends up
> writing all food items together, and then all store items and then all
> prices
>
> i want
>
> food, store, price
> food, store, price
>

Well, if it all fits in memory, append each to its own list, and then
either finally if you can or periodically if you must:

for food, store, price in zip(foods, stores, prices):
<do some writing.>

--
-Scott David Daniels
scott.daniels@acm.org

homepricemaps@gmail.com 12-27-2005 04:56 AM

Re: help with lists and writing to file in correct order
 
sorry for asking such beginner questions but i tried this and nothing
wrote to my text file

for food, price, store in bs(food, price, store):
out = open("test.txt", 'a')
out.write (food + price + store)
out.close()


while if i write the following without the for i at least get
something?
out = open("test.txt", 'a')
out.write (food + price + store)
out.close()


Scott David Daniels wrote:
> homepricemaps@gmail.com wrote:
> > the problem with writing to teh file immidiately is that it ends up
> > writing all food items together, and then all store items and then all
> > prices
> >
> > i want
> >
> > food, store, price
> > food, store, price
> >

> Well, if it all fits in memory, append each to its own list, and then
> either finally if you can or periodically if you must:
>
> for food, store, price in zip(foods, stores, prices):
> <do some writing.>
>
> --
> -Scott David Daniels
> scott.daniels@acm.org



bonono@gmail.com 12-27-2005 05:05 AM

Re: help with lists and writing to file in correct order
 

homepricem...@gmail.com wrote:
> sorry for asking such beginner questions but i tried this and nothing
> wrote to my text file
>
> for food, price, store in bs(food, price, store):
> out = open("test.txt", 'a')
> out.write (food + price + store)
> out.close()
>
>
> while if i write the following without the for i at least get
> something?
> out = open("test.txt", 'a')
> out.write (food + price + store)
> out.close()
>

pull the open() and close() call out of the loop. And use some other
name for the variables as they are very confusing and could be error
prone to.


Steven D'Aprano 12-27-2005 08:09 AM

Re: help with lists and writing to file in correct order
 
On Mon, 26 Dec 2005 20:56:17 -0800, homepricemaps wrote:

> sorry for asking such beginner questions but i tried this and nothing
> wrote to my text file
>
> for food, price, store in bs(food, price, store):
> out = open("test.txt", 'a')
> out.write (food + price + store)
> out.close()


What are the contents of food, price and store? If "nothing wrote to my
text file", chances are all three of them are the empty string.


> while if i write the following without the for i at least get
> something?
> out = open("test.txt", 'a')
> out.write (food + price + store)
> out.close()


You get "something". That's not much help. But I predict that what you are
getting is the contents of food price and store, at least one of which are
not empty.

You need to encapsulate your code by separating the part of the code that
reads the html file from the part that writes the text file. I suggest
something like this:


def read_html_data(name_of_file):
# I don't know BeautifulSoup, so you will have to fix this...
datafile = BeautifulSoup(name_of_file)
# somehow read in the foods, prices and stores
# for each set of three, store them in a tuple (food, store, price)
# then store the tuples in a list
# something vaguely like this:
data = []
while 1:
food = datafile.get("food") # or whatever
store = datafile.get("store")
price = datafile.get("price")
data.append( (food,store,price) )
datafile.close()
return data

def write_data_to_text(datalist, name_of_file):
# Expects a list of tuples (food,store,price). Writes that list
# to name_of_file separated by newlines.
fp = file(name_of_file, "w")
for triplet in datalist:
fp.write("Food = %s, store = %s, price = %s\n" % triplet
fp.close()


Hope this helps.



--
Steven.



All times are GMT. The time now is 09:06 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.