Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Re: OverflowError while sending large file via socket

Reply
Thread Tools

Re: OverflowError while sending large file via socket

 
 
Steven D'Aprano
Guest
Posts: n/a
 
      04-13-2009
On Mon, 13 Apr 2009 00:21:34 +0200, Ryniek90 wrote:

> When i wanted to send an .iso file of 4GB length, i had traceback:
> "OverflowError: requested number of bytes is more than a Python string
> can hold"
>
> Sockets are being used in every network app, i.e: p2p progs (like
> BitTorrent), and exchanged data is often bigger than 4GB.


But they don't transfer the entire file as ONE packet. Split your data
into smaller packets. I don't know what a good size for each packet would
be, but if I were doing this, I'd probably start with 4096 or 8192
*bytes*.

http://www.amk.ca/python/howto/sockets/


> So why i've
> had that Traceback? How many number of bytes Python string can hold?


The documentation doesn't seem to specify a maximum string length:

http://docs.python.org/library/stdtypes.html

which suggests to me that it will be implementation dependent. However,
I'd guess that the current CPython implementation will have a hard limit
of 2**32 bytes (4GB), and a soft limit on the amount of memory that you
have. You're trying to create a single, continuous block of memory 4GB in
size! Unless you've got *at least* 4GB of RAM, this is impossible even in
principle, and in practice you need more than that to allow for the
overhead of the operating system, Python, and any other applications you
have running.



--
Steven
 
Reply With Quote
 
 
 
 
Benjamin Peterson
Guest
Posts: n/a
 
      04-13-2009
Steven D'Aprano <steve <at> REMOVE-THIS-cybersource.com.au> writes:
>
> which suggests to me that it will be implementation dependent


The length of sequences is constrained by sys.maxsize
(and no, you can't change it).




 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
socket.unbind or socket.unlisten? - socket.error: (48, 'Addressalready in use') Laszlo Nagy Python 1 01-27-2009 05:05 PM
why does math.pow yields OverflowError (while python itself cancalculate that large number) Tzury Bar Yochay Python 5 10-23-2008 09:51 PM
chopping file while sending it via asp response.binary write .nLL ASP General 1 02-13-2007 10:47 PM
No error while sending via TCP Socket =?ISO-8859-15?Q?Martin_B=FCrkle?= Python 10 07-03-2006 04:50 PM
socket.sendall(), non-blocking sockets, and multi-threaded socket sending Tim Black Python 1 08-03-2004 01:11 PM



Advertisments
 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57