Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C++ > boost::asio for HTTP -> data truncated

Reply
Thread Tools

boost::asio for HTTP -> data truncated

 
 
Torsten Mueller
Guest
Posts: n/a
 
      08-12-2010
I use boost::asio classes to perform a GET-request to a HTTP-Server.
The request tells the server to return a JPEG image. This algorithm
works very well with most servers.

Now we have a proxy which is different. This proxy returns a content
length header for the images which seems to be right, perhaps 43kB,
but it does not return the entire contents, only 27kB or something
like that. I use an istream to read from an asio-streambuf and this
istream has the eof-bit set long before I expect it. So the images are
truncated.

Why???

The proxy is (don't ask why) a ccproxy instance. In the log files of
this proxy we can detect that also the proxy himself knows that he
sent only a small part of the data.

Does anybody know what to do? Request again?

T.M.
 
Reply With Quote
 
 
 
 
Francesco S. Carta
Guest
Posts: n/a
 
      08-12-2010
Torsten Mueller <(E-Mail Removed)>, on 12/08/2010 15:01:59, wrote:

> I use boost::asio classes to perform a GET-request to a HTTP-Server.
> The request tells the server to return a JPEG image. This algorithm
> works very well with most servers.
>
> Now we have a proxy which is different. This proxy returns a content
> length header for the images which seems to be right, perhaps 43kB,
> but it does not return the entire contents, only 27kB or something
> like that. I use an istream to read from an asio-streambuf and this
> istream has the eof-bit set long before I expect it. So the images are
> truncated.
>
> Why???
>
> The proxy is (don't ask why) a ccproxy instance. In the log files of
> this proxy we can detect that also the proxy himself knows that he
> sent only a small part of the data.
>
> Does anybody know what to do? Request again?


If the proxy is not sending complete data you cannot really do much with
your requests.

Have you tried requesting the same resource with, say, a regular
browser, and see if the image comes out complete or broken?

If the proxy is actually sending all the data but happens to misbehave
on the way (sending something that is interpreted as an eof by your
program) you might try resetting the stream state and ask for further
input beyond that point.

That's just a speculation, of course... before discarding an eof and
going on requesting more data check the received data against the
content length header value.

Content length headers can lie, of course, but you should be able to
check that out.

--
FSC - http://userscripts.org/scripts/show/59948
http://fscode.altervista.org - http://sardinias.com
 
Reply With Quote
 
 
 
 
Torsten Mueller
Guest
Posts: n/a
 
      08-13-2010
"Francesco S. Carta" <(E-Mail Removed)> wrote:

> > Does anybody know what to do? Request again?

>
> Have you tried requesting the same resource with, say, a regular
> browser, and see if the image comes out complete or broken?


It's a bit difficult because the problem happens only on a specific
network behind firewalls - and I, the developer, have no access!
People tell me, that IE never has a problem to get complete data over
the same proxy. But I'm not sure.

> If the proxy is actually sending all the data but happens to
> misbehave on the way (sending something that is interpreted as an
> eof by your program) you might try resetting the stream state and
> ask for further input beyond that point.


I had this idea, too. I try this today.

Another idea I had over night is as follows: I've seen that this
network is very very slow. To get the data of a 40kB picture I need 6
or 7 seconds. This opens the possibility to measure the time needed to
get the HTTP header. (The header is always complete.) If I know the
time needed for the HTTP header and I see it's slow I can calculate
the time needed for the HTTP body because I have the Content-Length.
And in this case I could do a break in reading to let the stream the
time needed to fetch it's data before I get an early EOF. Just an
idea.

Thanks for your reply.

T.M.
 
Reply With Quote
 
Francesco S. Carta
Guest
Posts: n/a
 
      08-13-2010
Torsten Mueller <(E-Mail Removed)>, on 13/08/2010 07:04:39, wrote:

> "Francesco S. Carta"<(E-Mail Removed)> wrote:
>
>>> Does anybody know what to do? Request again?

>>
>> Have you tried requesting the same resource with, say, a regular
>> browser, and see if the image comes out complete or broken?

>
> It's a bit difficult because the problem happens only on a specific
> network behind firewalls - and I, the developer, have no access!
> People tell me, that IE never has a problem to get complete data over
> the same proxy. But I'm not sure.


This is getting highly non-C++ related, and as such you have better
chances of getting sensible information by posting details about the
involved proxy in a pertinent group - I'm not sure about which group
would be adequate for such inquiries, though.

>> If the proxy is actually sending all the data but happens to
>> misbehave on the way (sending something that is interpreted as an
>> eof by your program) you might try resetting the stream state and
>> ask for further input beyond that point.

>
> I had this idea, too. I try this today.
>
> Another idea I had over night is as follows: I've seen that this
> network is very very slow. To get the data of a 40kB picture I need 6
> or 7 seconds. This opens the possibility to measure the time needed to
> get the HTTP header. (The header is always complete.) If I know the
> time needed for the HTTP header and I see it's slow I can calculate
> the time needed for the HTTP body because I have the Content-Length.
> And in this case I could do a break in reading to let the stream the
> time needed to fetch it's data before I get an early EOF. Just an
> idea.


I do hope for you that the idea you had will solve your issue, but there
are a lot of things that affect the time needed to fetch a resource over
the Internet, and those things are extremely variable - getting the
header could be a snap while completing the data could be slow as
molasses, but I've happened to see the other way around too, where the
very first response from the server took some good time but then the
rest of the data came fluently and quickly.

If both your computer and the proxy are within the same LAN, then that
could make the "time-needed" estimate more reliable, but I'm always in
the "speculation" field.

I believe there are other things you can do to test the connection, the
response time and the data-completeness but, once more, it's something
extremely OT here - and quite out of my gears too.

All that I can add is to wish you good luck for solving the issue as
soon as possible.

--
FSC - http://userscripts.org/scripts/show/59948
http://fscode.altervista.org - http://sardinias.com
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Why SAX parser reads truncated data ? Sanjeev Java 4 08-18-2008 08:56 AM
POST data truncated in Internet Explorer 6 Ian Lotinsky ASP .Net 1 05-02-2006 01:50 AM
Probelm to post XML data in a loop. First time XML is posted, second time data is getting truncated. Please help. vamsi.aluru@gmail.com Perl Misc 7 02-14-2006 12:09 PM
ASPNET_REGSQL error: String or binary data would be truncated. (81 George Jackson ASP .Net Security 1 01-11-2006 04:52 PM
SQLException: String or binary data would be truncated jrefactors@hotmail.com Java 1 07-15-2005 07:20 AM



Advertisments