Re: Socket window size
 

Roedy Green <roedy@mindprod.com> wrote:
> In my application, since I am dealing with real time data, I would
> prefer it if TCP/IP did not use huge buffers. Otherwise if a backlog
> occurs, I will ever after be several seconds behind in time. I will
> never catch up. I would prefer that TCP/UP block if I start to get
> behind.

If the other suggestions did not already solve your problems, you might
want to consider a switch to UDP. On the recieving end, you can set the
recieve buffer size (which as far as I know is passed down to the
underlying operating systems' network subsystem). When that recieve buffer
is full, any further packet is simply thrown away, so when the reciever is
no longer backlogged, it will get "actual" data immediatly again.

Of course, with UDP you lose all the convenience features of TCP. There
won't be retransmissions, there won't be any datastream at all, just
several unrelated data packets, so you might need to implement your own
sequence handling to detect missing data that can be re-requested later or
something like that (if it is important to have all data)

Most real-time applications (rt audio/video streaming) is using UDP for a
reason.

CU

Rene

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB

Re: Socket window size
 

On 28 Jun 2003 20:05:15 GMT, Rene <invalid@email.addr> wrote or quoted
:

>Of course, with UDP you lose all the convenience features of TCP. There
>won't be retransmissions, there won't be any datastream at all, just
>several unrelated data packets, so you might need to implement your own
>sequence handling to detect missing data that can be re-requested later or
>something like that (if it is important to have all data)

If you could do a 10K packet, that would work reasonably well.

Re: Socket window size
 

On 28 Jun 2003 20:05:15 GMT, Rene <invalid@email.addr> wrote or quoted
:

>Of course, with UDP you lose all the convenience features of TCP.

The other problem is this app has to negotiate a gauntlet of firewalls
at various customer sites. TCP/IP has a slightly better chance getting
through.

This app called Cyber View Plus is just a controller for a bank of
security cameras, that let you switch either manually or on a
pre-programmed schedule, viewing as many streams at once as your port
can handle. It uses a cheapie streaming JPEG format since the cameras
that produce it are much cheaper than ones that would do proper MP3
streaming video. The main problem with it is each image is compressed
separately. Thus each image has slightly different colours. You
don't notice it frame by frame, but you do in a stream.

The server side stores history, and allows you to review history.

Re: Socket window size
 

Roedy Green <roedy@mindprod.com> wrote:
> On 28 Jun 2003 20:05:15 GMT, Rene <invalid@email.addr> wrote or quoted
> :
>
> >Of course, with UDP you lose all the convenience features of TCP. There
> >won't be retransmissions, there won't be any datastream at all, just
> >several unrelated data packets, so you might need to implement your own
> >sequence handling to detect missing data that can be re-requested later
> >or something like that (if it is important to have all data)
>
> If you could do a 10K packet, that would work reasonably well.

Well the Ethernet maximum frame length is 1500 Bytes. Minus IP and UDP
headers, you're down to 1480 usable bytes per packet. There are systems
where you can push up the ethernet frame size (so called "jumbo frames") up
to and exceeding of 64kbytes, but not on the internet. You'd need your own
network with a bit specialized equipment.

CU

Rene

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB

Re: Socket window size
 

Roedy Green <roedy@mindprod.com> wrote:
> On 28 Jun 2003 20:05:15 GMT, Rene <invalid@email.addr> wrote or quoted
> :
>
> >Of course, with UDP you lose all the convenience features of TCP.
>
> The other problem is this app has to negotiate a gauntlet of firewalls
> at various customer sites. TCP/IP has a slightly better chance getting
> through.

For a pure packet filter, UDP or TCP is no different. For a stateful
firewall it can make a difference according to how it is set up. If you can
control the firewall (or at least make the right people allow your stream)
there won't be a problem. If you can't, yes, then TCP is probably easier to
get through.

Maybe you can tunnel it trough within a SSH or VPN tunnel?

> This app called Cyber View Plus is just a controller for a bank of
> security cameras, that let you switch either manually or on a
> pre-programmed schedule, viewing as many streams at once as your port
> can handle. It uses a cheapie streaming JPEG format since the cameras
> that produce it are much cheaper than ones that would do proper MP3
> streaming video. The main problem with it is each image is compressed
> separately. Thus each image has slightly different colours. You
> don't notice it frame by frame, but you do in a stream.

Ah you might want to normalize them a bit when displaying the stream, which
might "blur out" and hide the color difference between the frames a bit. If
they are mostly for archiving purposes, then you don't have any problem.

CU

Rene

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB

Re: Socket window size
 

"Dale King" <Dale[dot]King@thomson.net> wrote:
> "Rene" <invalid@email.addr> wrote in message
> > Well the Ethernet maximum frame length is 1500 Bytes. Minus IP and UDP
> > headers, you're down to 1480 usable bytes per packet. There are systems
> > where you can push up the ethernet frame size (so called "jumbo
> > frames")
> up
> > to and exceeding of 64kbytes, but not on the internet. You'd need your
> > own network with a bit specialized equipment.
>
> No, because Ethernet packet size does not have to correspond to the IP
> packet size. IP is designed to support fragmentation and reassembly.

No they don't need to but the ethernet frame size is the upper bound. What
I was saying was not that it is not possible to send larger data, what I
said was it is not possible to send 16Kbytes of payload (data) within one
single packet.

The maximum size per single UDP or TCP packet is quite limited, unless you
happen to control the full path from source to destination, in which case
you can increase it over 1500Bytes total (excluding headers). For the
internet as a whole, 1500 Bytes is the absolute maximum, though some hops
may support more.

However, most paths either start or end at an installation with standard
ethernet and this is where the limit kicks in. Weakest link in a chain, so
to speak. Oh and many many switches can't support more than 1536 bytes per
packet anyway. Expensive stuff may, but not the everyday equipment and
certainly not a cable/adsl modem.

> When a network is sending a datagram that is larger than the maximum
> transmission unit of the network it will be transmitted on it fragments
> it into smaller datagrams. The final receiving host then reassembles
> them.

This is true, however every fragment has a full header making it a new
packet.

The new packets are called fragments but that doesn't change the fact that
they contain a full header (they too must be routable, so they need the
header). Fragments are simply packets of their own kind. And you absolutely
don't want fragments when trying to do something in real time or near-real
time, because fragmentation means further processing overhead, which means
further latency.

And they have the potential to really lag your connection once one out of x
fragments is missing/delayed or dropped, since the already recieved
fragments are worthless when not complete - and they are kept in memory
buffers for some timeout until thrown away. Or the outstanding fragment may
arrive late so that the packet reassembly takes place quite late - in that
case you'd have won much if you'd prevented fragmentation in the first
place.

> See http://www.geocities.com/SiliconVall...rk/ipfrag.html
> for more info.

I know what IP fragmentation is. But once you send more than MTU bytes at
once, you end up with more than one packet. The packet is the "atomic" unit
of data you can send and recieve. Either you get that packet or you don't.
Its not possible to loose half the data of a packet.

The knowledge of that fact is irrelevant in normal everyday work, but when
it gets to realtime streaming, it becomes important. Especially when you
need to transfer a larger amount of bytes but that larger amount should be
"atomic" in itself (either get it full or not at all, but in no case late)

CU

Rene

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service New Rate! $9.95/Month 50GB