Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > sending a file chunk by chunk instead as a whole to a web server

Reply
Thread Tools

sending a file chunk by chunk instead as a whole to a web server

 
 
Sanjeeb
Guest
Posts: n/a
 
      08-02-2010
Hi,
I have a web client which send a file to a server as multipart form
data, the sending of data is from
http://code.activestate.com/recipes/...partform-data/.

I dont want to open the whole file to memory(at cliend end) and then
send, i just want to send part by part, say chunk of 1024 bytes to the
server and then assemble at the server end.

Could some one suggest what would be the best way to do this?

Regards
-Sanjeeb
 
Reply With Quote
 
 
 
 
Sanjeeb
Guest
Posts: n/a
 
      08-03-2010
On Aug 2, 11:52*am, Sanjeeb <(E-Mail Removed)> wrote:
> Hi,
> I have a web client which send a file to a server as multipart form
> data, the sending of data is fromhttp://code.activestate.com/recipes/146306-http-client-to-post-using-....
>
> I dont want to open the whole file to memory(at cliend end) and then
> send, i just want to send part by part, say chunk of 1024 bytes to the
> server and then assemble at the server end.
>
> Could some one suggest what would be the best way to do this?
>
> Regards
> -Sanjeeb


Hey any taker for this question???
 
Reply With Quote
 
 
 
 
Kushal Kumaran
Guest
Posts: n/a
 
      08-03-2010
On Mon, Aug 2, 2010 at 12:22 PM, Sanjeeb <(E-Mail Removed)> wrote:
> Hi,
> I have a web client which send a file to a server as multipart form
> data, the sending of data is from
> http://code.activestate.com/recipes/...partform-data/.
>
> I dont want to open the whole file to memory(at cliend end) and then
> send, i just want to send part by part, say chunk of 1024 bytes to the
> server and then assemble at the server end.
>
> Could some one suggest what would be the best way to do this?
>


There's no reason why sending the whole file implies reading the whole
file into memory at one time. You can just read your desired chunk
size and send it, then read the next chunk, and so on. You might have
to first find the total size to calculate what to set Content-Length
to.

ISTR questions of this nature have been asked in the past on this list.

--
regards,
kushal
 
Reply With Quote
 
Ryan Kelly
Guest
Posts: n/a
 
      08-03-2010
On Tue, 2010-08-03 at 10:45 +0530, Kushal Kumaran wrote:
> On Mon, Aug 2, 2010 at 12:22 PM, Sanjeeb <(E-Mail Removed)> wrote:
> > Hi,
> > I have a web client which send a file to a server as multipart form
> > data, the sending of data is from
> > http://code.activestate.com/recipes/...partform-data/.
> >
> > I dont want to open the whole file to memory(at cliend end) and then
> > send, i just want to send part by part, say chunk of 1024 bytes to the
> > server and then assemble at the server end.
> >
> > Could some one suggest what would be the best way to do this?
> >

>
> There's no reason why sending the whole file implies reading the whole
> file into memory at one time. You can just read your desired chunk
> size and send it, then read the next chunk, and so on. You might have
> to first find the total size to calculate what to set Content-Length
> to.


More concretely, you would restructure the encode_multipart_formdata()
function as a generator, yielding chunks of data to send one at a time.
Something like this:

def post_multipart(host,selector,fields,files):
...snip...
h.endheaders()
for chunk in encode_multipart_formdata_chunks(fields,files):
h.send(chunk)
errcode, errmsg, headers = h.getreply()
return h.file.read()


def encode_multipart_formdata(fields,files):
BOUNDARY = '----------ThIs_Is_tHe_bouNdaRY_$'
CRLF = '\r\n'
for (key, value) in fields:
yield '--' + BOUNDARY
yield 'Content-Disposition: form-data; name="%s"' % key
yield ''
yield value
for (key, filename, value) in files:
yield '--' + BOUNDARY
yield 'Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename)
...etc...
...etc...

There are many improvements to make, but this should get you started.
For example, you'll need to calculate the total content-length rather
than just calling len(body) to obtain it. That's left as an exercise to
the reader.


Ryan


--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
http://www.velocityreviews.com/forums/(E-Mail Removed) | http://www.rfk.id.au/ramblings/gpg/ for details


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEABECAAYFAkxXrpwACgkQfI5S64uP50q+fACgzcvVbFuG25 WI1B/9Daqk+Zvp
LVIAoMbwFNlvrPsz5vZqwjR8s8P5FHhY
=rhPT
-----END PGP SIGNATURE-----

 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Treeview click events - whole row instead of just the caption? Homer J. Simpson ASP .Net 1 08-15-2007 07:18 AM
pls help me when i sent mail, it vil sending twice instead of once ,am using java.mail,am sending my code.... shailajabtech@gmail.com Java 0 09-28-2006 06:38 AM
*WITHOUT* using: ValidateRequest="False" for the whole page (or my whole site).... \A_Michigan_User\ ASP .Net 2 08-21-2006 02:13 PM
What is an AVI Chunk Viewer? - AVI Chunk Viewer.jpg (0/1) mazdra76@yahooo.com Computer Support 1 03-17-2006 02:52 AM
ChunkedInputStream bad chunk size from IIS server - Any ideas? Ray Thomas Java 1 07-21-2004 10:36 PM



Advertisments