Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > Parsing MIME-encoded data in an HTTP request

Reply
Thread Tools

Parsing MIME-encoded data in an HTTP request

 
 
Ron Garret
Guest
Posts: n/a
 
      07-03-2008
I'm writing a little HTTP server and need to parse request content that
is mime-encoded. All the MIME routines in the Python standard library
seem to have been subsumed into the email package, which makes this
operation a little awkward. It seems I have to do the following:

1. Extract the content-length header from the HTTP request and use that
to read the payload.

2. Stick some artificial-looking headers onto the beginning of this
payload to make it look like an email message (including the
content-type and content-transfer-encoding headers)

3. Parse the resulting string into a email message

That works, but it feels way too hackish for my tastes. Surely there
must be a better/more standard way of doing this?

Thanks,
rg
 
Reply With Quote
 
 
 
 
s0suk3@gmail.com
Guest
Posts: n/a
 
      07-04-2008
On Jul 3, 3:59 pm, Ron Garret <(E-Mail Removed)> wrote:
> I'm writing a little HTTP server and need to parse request content that
> is mime-encoded. All the MIME routines in the Python standard library
> seem to have been subsumed into the email package, which makes this
> operation a little awkward.


To deal with messages of that kind, I've seen modules such as
'rfc822', and 'mimetools' (which apparently builds itself from
'rfc822', so it might be more complete). There's also 'mimetypes', in
case you need to deal with file extensions and their corresponding
MIME media type.

> It seems I have to do the following:
>
> 1. Extract the content-length header from the HTTP request and use that
> to read the payload.
>
> 2. Stick some artificial-looking headers onto the beginning of this
> payload to make it look like an email message (including the
> content-type and content-transfer-encoding headers)
>
> 3. Parse the resulting string into a email message
>


Email? Why does an HTTP server need to build an email message?

I remember doing things like that some time ago when building an HTTP
server myself (http://code.google.com/p/sws-d/). Incidentally, I
resisted the urge to use much of the Python's library facilities (most
things are done manually; am I a knucklehead or what!? . You might
wanna take a look to get some ideas.

Sebastian

 
Reply With Quote
 
 
 
 
Ron Garret
Guest
Posts: n/a
 
      07-04-2008
In article
<(E-Mail Removed)>,
http://www.velocityreviews.com/forums/(E-Mail Removed) wrote:

> On Jul 3, 3:59 pm, Ron Garret <(E-Mail Removed)> wrote:
> > I'm writing a little HTTP server and need to parse request content that
> > is mime-encoded. All the MIME routines in the Python standard library
> > seem to have been subsumed into the email package, which makes this
> > operation a little awkward.

>
> To deal with messages of that kind, I've seen modules such as
> 'rfc822', and 'mimetools' (which apparently builds itself from
> 'rfc822', so it might be more complete). There's also 'mimetypes', in
> case you need to deal with file extensions and their corresponding
> MIME media type.


From the mimetools docs:

"Deprecated since release 2.3. The email package should be used in
preference to the module. This module is present only to maintain
backward compatibility."

>
> > It seems I have to do the following:
> >
> > 1. Extract the content-length header from the HTTP request and use that
> > to read the payload.
> >
> > 2. Stick some artificial-looking headers onto the beginning of this
> > payload to make it look like an email message (including the
> > content-type and content-transfer-encoding headers)
> >
> > 3. Parse the resulting string into a email message
> >

>
> Email? Why does an HTTP server need to build an email message?


It shouldn't. That's my whole point. But see the docs excerpt above.

> I remember doing things like that some time ago when building an HTTP
> server myself (http://code.google.com/p/sws-d/). Incidentally, I
> resisted the urge to use much of the Python's library facilities (most
> things are done manually; am I a knucklehead or what!? . You might
> wanna take a look to get some ideas.


I'd much prefer not to reinvent this particular wheel.

rg
 
Reply With Quote
 
Michael Ströder
Guest
Posts: n/a
 
      07-04-2008
Ron Garret wrote:
> I'm writing a little HTTP server and need to parse request content that
> is mime-encoded. All the MIME routines in the Python standard library
> seem to have been subsumed into the email package, which makes this
> operation a little awkward.


How about using cgi.parse_multipart()?

Ciao, Michael.
 
Reply With Quote
 
Ron Garret
Guest
Posts: n/a
 
      07-04-2008
In article <(E-Mail Removed)>,
Michael Ströder <(E-Mail Removed)> wrote:

> Ron Garret wrote:
> > I'm writing a little HTTP server and need to parse request content that
> > is mime-encoded. All the MIME routines in the Python standard library
> > seem to have been subsumed into the email package, which makes this
> > operation a little awkward.

>
> How about using cgi.parse_multipart()?
>
> Ciao, Michael.


Unfortunately cgi.parse_multipart doesn't handle nested multiparts,
which the requests I'm getting have. You have to use a FieldStorage
object to do that, and that only works if you're actually in a cgi
environment, which I am not. The server responds to these requests
directly.

Anyway, thanks for the idea.

rg
 
Reply With Quote
 
Ron Garret
Guest
Posts: n/a
 
      07-04-2008
In article <(E-Mail Removed)>,
Ron Garret <(E-Mail Removed)> wrote:

> In article <(E-Mail Removed)>,
> Michael Ströder <(E-Mail Removed)> wrote:
>
> > Ron Garret wrote:
> > > I'm writing a little HTTP server and need to parse request content that
> > > is mime-encoded. All the MIME routines in the Python standard library
> > > seem to have been subsumed into the email package, which makes this
> > > operation a little awkward.

> >
> > How about using cgi.parse_multipart()?
> >
> > Ciao, Michael.

>
> Unfortunately cgi.parse_multipart doesn't handle nested multiparts,
> which the requests I'm getting have. You have to use a FieldStorage
> object to do that, and that only works if you're actually in a cgi
> environment, which I am not. The server responds to these requests
> directly.
>
> Anyway, thanks for the idea.
>
> rg


Hm, it actually seems to work if I manually pass in the outerboundary
parameter and environ={'REQUEST_METHOD':'POST'} That seems like the
Right Answer.

Woohoo!

Thanks Michael!

rg
 
Reply With Quote
 
Michael Ströder
Guest
Posts: n/a
 
      07-05-2008
Ron Garret wrote:
> In article <(E-Mail Removed)>,
> Ron Garret <(E-Mail Removed)> wrote:
>
>> In article <(E-Mail Removed)>,
>> Michael Ströder <(E-Mail Removed)> wrote:
>>
>>> Ron Garret wrote:
>>>> I'm writing a little HTTP server and need to parse request content that
>>>> is mime-encoded. All the MIME routines in the Python standard library
>>>> seem to have been subsumed into the email package, which makes this
>>>> operation a little awkward.
>>> How about using cgi.parse_multipart()?
>>>

>> Unfortunately cgi.parse_multipart doesn't handle nested multiparts,
>> which the requests I'm getting have. You have to use a FieldStorage
>> object to do that, and that only works if you're actually in a cgi
>> environment, which I am not. The server responds to these requests
>> directly.
>>
>> Anyway, thanks for the idea.

>
> Hm, it actually seems to work if I manually pass in the outerboundary
> parameter and environ={'REQUEST_METHOD':'POST'} That seems like the
> Right Answer.


I'm also using it to parse form parameters in a message body received by
POST.

CIao, Michael.
 
Reply With Quote
 
Ron Garret
Guest
Posts: n/a
 
      07-06-2008
In article <(E-Mail Removed)>,
Michael Ströder <(E-Mail Removed)> wrote:

> Ron Garret wrote:
> > In article <(E-Mail Removed)>,
> > Ron Garret <(E-Mail Removed)> wrote:
> >
> >> In article <(E-Mail Removed)>,
> >> Michael Ströder <(E-Mail Removed)> wrote:
> >>
> >>> Ron Garret wrote:
> >>>> I'm writing a little HTTP server and need to parse request content that
> >>>> is mime-encoded. All the MIME routines in the Python standard library
> >>>> seem to have been subsumed into the email package, which makes this
> >>>> operation a little awkward.
> >>> How about using cgi.parse_multipart()?
> >>>
> >> Unfortunately cgi.parse_multipart doesn't handle nested multiparts,
> >> which the requests I'm getting have. You have to use a FieldStorage
> >> object to do that, and that only works if you're actually in a cgi
> >> environment, which I am not. The server responds to these requests
> >> directly.
> >>
> >> Anyway, thanks for the idea.

> >
> > Hm, it actually seems to work if I manually pass in the outerboundary
> > parameter and environ={'REQUEST_METHOD':'POST'} That seems like the
> > Right Answer.

>
> I'm also using it to parse form parameters in a message body received by
> POST.
>
> CIao, Michael.


Just for the record, here's the incantation I ended up with:


class post_handler(BaseHTTPRequestHandler):
def do_POST(self):
form = cgi.FieldStorage(fp=self.rfile, headers=self.headers,
environ={'REQUEST_METHOD':'POST'})
...


works like a charm.

rg
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Making HTTP request and parsing the resulting xhtml? Bart Vandewoestyne C++ 1 10-23-2012 07:34 AM
urllib2.Request:: http Request sending successfully, but Responsecontains in valid data. nRk Python 1 02-12-2009 12:53 AM
Parsing http head request resposnse from server. Raaga Javascript 1 02-08-2007 05:23 PM
parsing a cvs stream from an http request sollento ASP .Net 3 09-05-2006 03:03 PM
Re: Accessing Request.InputStream / Request.BinaryRead *as the request is occuring*: How??? Brian Birtle ASP .Net 2 10-16-2003 02:11 PM



Advertisments