«

On 10/19/2012 10:08 AM, Pradipto Banerjee wrote:
> Hi,
>
> I am trying to read a file into memory. The size of the file is around 1
> GB. I have a 3GB memory PC and the Windows Task Manager shows 2.3 GB
> available physical memory when I was trying to read the file. I tried to
> read the file as follows:
>
>>>> fdata = open(filename, ‘r’).read()
>
> I got a “MemoryError”. I was watching the Windows Task Manager while I
> run the python command, and it appears that python **perhaps** never
> even attempted to use more memory but gave me this error.
>
> Is there any reason why python can’t read a 1GB file in memory even when
> a 2.3 GB physical memory is available?

The real issue is likely that there is more than one copy of the file in
memory somewhere. I had a similar issue years back that I resolved by
using numeric (now numpy?) as it had a more efficient method of
importing content from disk.

Also realize that windows may not allow the full memory to user space.
I'm not sure what exactly the restrictions are, but a 4Gb windows box
doesn't always get you 4Gb of memory.

Emile