«

On 03/06/2013 07:31 AM, Wong Wah Meng-R32813 wrote:
> Apologies as after I have left the group for a while I have forgotten how not to post a question on top of another question. Very sorry and appreciate your replies.
>
> I tried explicitly calling gc.collect() and didn't manage to see the memory footprint reduced. I probably haven't left the process idle long enough to see the internal garbage collection takes place but I will leave it idle for more than 8 hours and check again. Thanks!
>

You're top-posting, which makes things very confusing, since your
contribution to the message is out of accepted order. Put your remarks
after the part you're commenting on, and delete anything following your
message, as it clearly didn't need your comments.

Once you've called gc.collect(), there's no point in waiting 8 hours for
it to run again. It either triggered the C runtime's logic or it
didn't, and running it again won't help unless in the meantime you
rearranged the remaining allocated blocks.

Accept the fact that not all freeing of memory blocks can possibly make
it through to the OS. If they did, we'd have a minimum object size of
at least 4k on the Pentium, and larger on some other processors. We'd
also have performance that would crawl. So an external tool can only
give you a very approximate size for what's going on in your own code.

--
DaveA

On 03/06/2013 05:27 AM, Lele Gaifax wrote:
> �ίκος Γκ�33κ <> writes:
>
>> Its about the following line of code:
>>
>> current_fullpaths.add( os.path.join(root, files) )
>
> I'm sorry, typo on my part.
>
> That should have been "fullpath", not "file" (and neither "files" as you
> wrongly reported back!):
>
> # Compute a set of current fullpaths
> current_fullpaths = set()
> for root, dirs, files in os.walk(path):
> for fullpath in files:

'fullpath' is a rather misleading name to use, since the 'files' list
contains only the terminal node of the file name. It's only a full path
after you do the following join.

> current_fullpaths.add(os.path.join(root, fullpath))
>


--
DaveA

On Wed, Mar 6, 2013 at 10:52 PM, Mark Lawrence <> wrote:
> On 06/03/2013 07:45, Íßêïò Ãêñ33ê wrote:
>> blah blah blah
>> blah blah blah
>> blah blah blah
> You were told yesterday at least twice that os.walk returns a tuple but you
> still insist on refusing to take any notice of our replies when it suits
> you, preferring instead to waste everbody's time with these questions. Or
> are you trying to get into the Guinness Book of World Records for the
> laziest bastard on the planet?

This is the same person who posts under the name Ferrous Cranus. His
threads are open-ended time sinks; he generally expects everyone else
to do the work, while his contribution is to complain that the
provided code doesn't fit some arbitrary set of restrictions that
don't always even make sense.

Guinness might be interested in him as "Most Successful Troll",
though. He certainly absorbed a lot of my dev time before I gave up on
him, and by the look of things, he's still getting other people to do
his work for him.

ChrisA

On Wed, 06 Mar 2013 12:52:00 +0100, Mark Lawrence
<> wrote:

> On 06/03/2013 07:45, �ίκος Γκ�33κ wrote:
>> I'am using this snipper to read a current directory and insert all
>> filenames into a databse and then display them.
>>
>> But what happens when files are get removed form the directory?
>> The inserted records into databse remain.
>> How can i update the databse to only contain the existing filenames
>> without losing the previous stored data?
>>
>> Here is what i ahve so far:
>>
>> ==================================
>> path = "/home/nikos/public_html/data/files/"
>>
>> #read the containing folder and insert new filenames
>> for result in os.walk(path):
>
> You were told yesterday at least twice that os.walk returns a tuple but
> you still insist on refusing to take any notice of our replies when it
> suits you, preferring instead to waste everbody's time with these
> questions. Or are you trying to get into the Guinness Book of World
> Records for the laziest bastard on the planet?

Hold on a sec ...

He has

for result in os.walk(path):
for filename in result[2]:

So he *did* take notice of that.


Nikos:
Expectation is to iterate through a tuple like this:

for dirpath, dirnames, filenames in os.walk(path):
...

Dave Angel <> writes:

>> # Compute a set of current fullpaths
>> current_fullpaths = set()
>> for root, dirs, files in os.walk(path):
>> for fullpath in files:
>
> 'fullpath' is a rather misleading name to use, since the 'files' list
> contains only the terminal node of the file name. It's only a full
> path after you do the following join.

Yes, you're right.

Dunno what urged me to ``M-x replace-string file fullpath`` introducing
both an error and a bad variable name, most probably the unconscious
desire of not clobbering builtin names...

Thanks for pointing it out,
ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
| -- Fortunato Depero, 1929.

Τη ΤετάÏ�τη, 6 ΜαÏ�τίου 2013 4:04:26 μ.μ. UTC+2, ο χÏ�ήστης Michael Ross Î*γÏ�αψε:
> On Wed, 06 Mar 2013 12:52:00 +0100, Mark Lawrence
>
> <> wrote:
>
>
>
> > On 06/03/2013 07:45, �ίκος Γκ�33κ wrote:
>
> >> I'am using this snipper to read a current directory and insert all
>
> >> filenames into a databse and then display them.
>
> >>
>
> >> But what happens when files are get removed form the directory?
>
> >> The inserted records into databse remain.
>
> >> How can i update the databse to only contain the existing filenames
>
> >> without losing the previous stored data?
>
> >>
>
> >> Here is what i ahve so far:
>
> >>
>
> >> ==================================
>
> >> path = "/home/nikos/public_html/data/files/"
>
> >>
>
> >> #read the containing folder and insert new filenames
>
> >> for result in os.walk(path):
>
> >
>
> > You were told yesterday at least twice that os.walk returns a tuple but
>
> > you still insist on refusing to take any notice of our replies when it
>
> > suits you, preferring instead to waste everbody's time with these
>
> > questions. Or are you trying to get into the Guinness Book of World
>
> > Records for the laziest bastard on the planet?
>
>
>
> Hold on a sec ...
>
>
>
> He has
>
>
>
> for result in os.walk(path):
>
> for filename in result[2]:
>
>
>
> So he *did* take notice of that.
>
>
>
>
>
> Nikos:
>
> Expectation is to iterate through a tuple like this:
>
>
>
> for dirpath, dirnames, filenames in os.walk(path):
>
> ...

Thank you Michael, yes i ahve understood that myself yesterday after one ofthe guys here have shown the output of os.walk(path) in IDLE.
So, yes in fact i was in need for the 3rd item in the tuple, so to get holdon to the files.

Thank you for supporting me, some ppl here think i'am a troll and don't trythinks or ignore evrything but 'some_ppl's_opinion != True'

I'am just sometimes persistant on having thing my way that why i was calling my self Ferrous cRanus, which i changes it since it was annoying....

Τη ΤετάÏ�τη, 6 ΜαÏ�τίου 2013 4:04:26 μ.μ. UTC+2, ο χÏ�ήστης Michael Ross Î*γÏ�αψε:
> On Wed, 06 Mar 2013 12:52:00 +0100, Mark Lawrence
>
> <> wrote:
>
>
>
> > On 06/03/2013 07:45, �ίκος Γκ�33κ wrote:
>
> >> I'am using this snipper to read a current directory and insert all
>
> >> filenames into a databse and then display them.
>
> >>
>
> >> But what happens when files are get removed form the directory?
>
> >> The inserted records into databse remain.
>
> >> How can i update the databse to only contain the existing filenames
>
> >> without losing the previous stored data?
>
> >>
>
> >> Here is what i ahve so far:
>
> >>
>
> >> ==================================
>
> >> path = "/home/nikos/public_html/data/files/"
>
> >>
>
> >> #read the containing folder and insert new filenames
>
> >> for result in os.walk(path):
>
> >
>
> > You were told yesterday at least twice that os.walk returns a tuple but
>
> > you still insist on refusing to take any notice of our replies when it
>
> > suits you, preferring instead to waste everbody's time with these
>
> > questions. Or are you trying to get into the Guinness Book of World
>
> > Records for the laziest bastard on the planet?
>
>
>
> Hold on a sec ...
>
>
>
> He has
>
>
>
> for result in os.walk(path):
>
> for filename in result[2]:
>
>
>
> So he *did* take notice of that.
>
>
>
>
>
> Nikos:
>
> Expectation is to iterate through a tuple like this:
>
>
>
> for dirpath, dirnames, filenames in os.walk(path):
>
> ...

Thank you Michael, yes i ahve understood that myself yesterday after one ofthe guys here have shown the output of os.walk(path) in IDLE.
So, yes in fact i was in need for the 3rd item in the tuple, so to get holdon to the files.

Thank you for supporting me, some ppl here think i'am a troll and don't trythinks or ignore evrything but 'some_ppl's_opinion != True'

I'am just sometimes persistant on having thing my way that why i was calling my self Ferrous cRanus, which i changes it since it was annoying....

On Wed, 06 Mar 2013 10:11:12 +0000, Wong Wah Meng-R32813 wrote:

> Hello there,
>
> I am using python 2.7.1 built on HP-11.23 a Itanium 64 bit box.
>
> I discovered following behavior whereby the python process doesn't seem
> to release memory utilized even after a variable is set to None, and
> "deleted". I use glance tool to monitor the memory utilized by this
> process. Obviously after the for loop is executed, the memory used by
> this process has hiked to a few MB. However, after "del" is executed to
> both I and str variables, the memory of that process still stays at
> where it was.
>
> Any idea why?

Python does not guarantee to return memory to the operating system.
Whether it does or not depends on the OS, but as a general rule, you
should expect that it will not.


>>>> for i in range(100000L):
> ... str=str+"%s"%(i,)


You should never build large strings in that way. It risks being
horribly, horribly slow on some combinations of OS, Python implementation
and version.

Instead, you should do this:

items = ["%s" % i for i in range(100000)]
s = ''.join(items)


--
Steven

Python does not guarantee to return memory to the operating system.
Whether it does or not depends on the OS, but as a general rule, you shouldexpect that it will not.


>>>> for i in range(100000L):
> ... str=str+"%s"%(i,)


You should never build large strings in that way. It risks being
horribly, horribly slow on some combinations of OS, Python implementation
and version.

Instead, you should do this:

items = ["%s" % i for i in range(100000)]
s = ''.join(items)

[] The example is written for illustration purpose. Thanks for pointing outa better way of achieving the same result. Yes it seems so that the OS thinks the piece allocated to Python should not be taken back unless the process dies.

On Thu, Mar 7, 2013 at 5:33 PM, Wong Wah Meng-R32813
<> wrote:
> [] The example is written for illustration purpose. Thanks for pointing out a better way of achieving the same result. Yes it seems so that the OS thinks the piece allocated to Python should not be taken back unless the process dies.

Don't be too bothered by that. That memory will be reused by Python
for subsequent allocations.

ChrisA