Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > C Programming > Data via serial port breaks down in multiple packets

Reply
Thread Tools

Data via serial port breaks down in multiple packets

 
 
Muffinman
Guest
Posts: n/a
 
      08-09-2013
Hello all,

I'm not sure whether this is the appropriate place for this question,
but I couldn't find a better place. So here it goes:

I have an application that needs to receive some data via a serial port.
The data is received, however, it breaks down in multiple packets. So
sending 18 bytes often results in two packets of 8 bytes and one of 2
(but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
many bytes I want. However, sometimes I need to receive 4 bytes, other
times 25. It is not consistent, so that's no option.

I need to process commands.
- Processing each packet at a time would not work because it can break
down commands.
- I could fill my own buffer, but timing is quite crucial...

1) So, can I solve this? Is there anything I can do with my
configuration (see below)?

2) Why does this happen? The only thing I can think of is that the
sending device skips a opportunity to send data every so bytes...

I hope someone can help.

Thanks in advance, Maarten


################# serial port polling #################
if(poll_fds[0].revents & POLLIN)
{
bytes_read = read(serial_fd, device_status_message,
sizeof(device_status_message));
if(bytes_read <= 0)
continue;
[...]

############# serial port configuraton #################
/* c_cflag
* setting all the variables according to the configuration file
* and some default settings */
serial_new_attr.c_cflag |= (CLOCAL | CREAD);
if(common_data.serial_parity == 0)
serial_new_attr.c_cflag &= ~PARENB;
else
{
serial_new_attr.c_cflag |= PARENB;
/* enable parity check and strip it from output */
serial_new_attr.c_iflag |= (INPCK | ISTRIP);
if(common_data.serial_even == 0)
serial_new_attr.c_cflag |= PARODD;
else
serial_new_attr.c_cflag &= ~PARODD;
}

if(common_data.serial_2stop == 0)
serial_new_attr.c_cflag &= ~CSTOPB;
else
serial_new_attr.c_cflag |= CSTOPB;
serial_new_attr.c_cflag &= ~CSIZE;
serial_new_attr.c_cflag |= common_data.serial_bits;

/* c_lflag
* enable raw input */
serial_new_attr.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG);

/* Disabling hardware flow control */
serial_new_attr.c_cflag &= ~CRTSCTS;
/* Disabling software flow control */
serial_new_attr.c_iflag &= ~(IXON | IXOFF | IXANY);

/* Disabling any postprocessing */
serial_new_attr.c_oflag &= ~OPOST;

/* Minimum characters to read (non-canonical) */
serial_new_attr.c_cc[VMIN] = 1;
/* Wait indefinitely for input (non-canonical) */
serial_new_attr.c_cc[VTIME] = 3;

cfsetospeed(&serial_new_attr, common_data.serial_speed);
cfsetispeed(&serial_new_attr, common_data.serial_speed);

 
Reply With Quote
 
 
 
 
James Kuyper
Guest
Posts: n/a
 
      08-09-2013
On 08/09/2013 11:55 AM, Muffinman wrote:
> Hello all,
>
> I'm not sure whether this is the appropriate place for this question,
> but I couldn't find a better place.


It isn't. The C standard says nothing about serial ports, read(),
cfsetispeed() or cfsetospeed(), nor about any of the macros or
structures you were using. Functions with those names are installed on
my desktop system, and the man pages for those functions describe them
as POSIX functions. If the functions you're using are the same ones,
then the best place to get answers to questions like this is
comp.unix.programmer.

 
Reply With Quote
 
 
 
 
glen herrmannsfeldt
Guest
Posts: n/a
 
      08-09-2013
James Kuyper <(E-Mail Removed)> wrote:

> On 08/09/2013 11:55 AM, Muffinman wrote:


>> I'm not sure whether this is the appropriate place for this question,
>> but I couldn't find a better place.


> It isn't. The C standard says nothing about serial ports, read(),
> cfsetispeed() or cfsetospeed(), nor about any of the macros or
> structures you were using.


But C does describe fread(), which you could use with serial ports,
or other data streams.

The Unix/C model for data is as a stream of bytes. If you write
data with a series of fwrite() calls, there is no reason to
expect the number of bytes read be each of a series of fread()
calls to return the same succession of record lengths.

There are systems that do preserve record lengths. Blocks on unix
tapes being one example. (I believe that comes through to fread(),
but right now I am not sure about that.)

If you need to preserve record marks, you might consider:

https://www.ietf.org/rfc/rfc1831.txt

-- glen
 
Reply With Quote
 
James Kuyper
Guest
Posts: n/a
 
      08-09-2013
On 08/09/2013 02:51 PM, glen herrmannsfeldt wrote:
> James Kuyper <(E-Mail Removed)> wrote:
>
>> On 08/09/2013 11:55 AM, Muffinman wrote:

>
>>> I'm not sure whether this is the appropriate place for this question,
>>> but I couldn't find a better place.

>
>> It isn't. The C standard says nothing about serial ports, read(),
>> cfsetispeed() or cfsetospeed(), nor about any of the macros or
>> structures you were using.

>
> But C does describe fread(), which you could use with serial ports,
> or other data streams.


And it says nothing useful about what happens when you do so, or at
least, about how it's different from what happens when you open an
ordinary file. There's really no useful answer to his question that can
be derived from the C standard, not even if you re-write it to use
fread() rather than read().

And since Unix open() and ioctl() can enable unix features that can
cause read() to behave quite differently from the way it normally works
when invoked by fread() (consider, for example, blocking vs.
non-blocking I/O), it's generally not a good idea to simply assume that
unix code calling read() can simply be replaced by C code calling
fread(). I'm not sure whether that is or could be relevant in this case
- but that's precisely the point. There will be people in
comp.unix.programmer who would be sure.
 
Reply With Quote
 
Joe Pfeiffer
Guest
Posts: n/a
 
      08-09-2013
Muffinman <news**REMOVETHIS**@koster.tk> writes:

> Hello all,
>
> I'm not sure whether this is the appropriate place for this question,
> but I couldn't find a better place. So here it goes:
>
> I have an application that needs to receive some data via a serial port.
> The data is received, however, it breaks down in multiple packets. So
> sending 18 bytes often results in two packets of 8 bytes and one of 2
> (but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
> many bytes I want. However, sometimes I need to receive 4 bytes, other
> times 25. It is not consistent, so that's no option.
>
> I need to process commands.
> - Processing each packet at a time would not work because it can break
> down commands.
> - I could fill my own buffer, but timing is quite crucial...
>
> 1) So, can I solve this? Is there anything I can do with my
> configuration (see below)?
>
> 2) Why does this happen? The only thing I can think of is that the
> sending device skips a opportunity to send data every so bytes...


You're pretty much out of luck. The read() man page says,

On success, the number of bytes read is returned (zero indicates end of
file), and the file position is advanced by this number. It is not an
error if this number is smaller than the number of bytes requested;
this may happen for example because fewer bytes are actually available
right now (maybe because we were close to end-of-file, or because we
are reading from a pipe, or from a terminal),

So... you tell the system call how many bytes you want, it tells you
how many bytes you get. Lots of my code has loops like this:

numread = 0;
while (numread < numwanted) {
thisread = read(fd, &buf[numread], numwanted-numread);
if (thisread < 0) {
perror("read at line xxx");
break;
}
numread += thisread;
}

Asking in a unix-specific newsgroup certainly couldn't hurt.
 
Reply With Quote
 
Michael Angelo Ravera
Guest
Posts: n/a
 
      08-10-2013
On Friday, August 9, 2013 8:55:03 AM UTC-7, Muffinman wrote:
> Hello all,
> I'm not sure whether this is the appropriate place for this question,
> but I couldn't find a better place. So here it goes:
>
> I have an application that needs to receive some data via a serial port.
> The data is received, however, it breaks down in multiple packets. So
> sending 18 bytes often results in two packets of 8 bytes and one of 2
> (but sometimes also 8,9&1). Setting VMIN allows me to push it up to how
> many bytes I want. However, sometimes I need to receive 4 bytes, other
> times 25. It is not consistent, so that's no option.
> I need to process commands.
>
> - Processing each packet at a time would not work because it can break
> down commands.
>
> - I could fill my own buffer, but timing is quite crucial...
>
>
>
> 1) So, can I solve this? Is there anything I can do with my
>
> configuration (see below)?
>
>
>
> 2) Why does this happen? The only thing I can think of is that the
>
> sending device skips a opportunity to send data every so bytes...
>
>
>
> I hope someone can help.
>
>
>
> Thanks in advance, Maarten
>
>
>
>
>
> ################# serial port polling #################
>
> if(poll_fds[0].revents & POLLIN)
>
> {
>
> bytes_read = read(serial_fd, device_status_message,
>
> sizeof(device_status_message));
>
> if(bytes_read <= 0)
>
> continue;
>
> [...]
>
>
>
> ############# serial port configuraton #################
>
> /* c_cflag
>
> * setting all the variables according to the configuration file
>
> * and some default settings */
>
> serial_new_attr.c_cflag |= (CLOCAL | CREAD);
>
> if(common_data.serial_parity == 0)
>
> serial_new_attr.c_cflag &= ~PARENB;
>
> else
>
> {
>
> serial_new_attr.c_cflag |= PARENB;
>
> /* enable parity check and strip it from output */
>
> serial_new_attr.c_iflag |= (INPCK | ISTRIP);
>
> if(common_data.serial_even == 0)
>
> serial_new_attr.c_cflag |= PARODD;
>
> else
>
> serial_new_attr.c_cflag &= ~PARODD;
>
> }
>
>
>
> if(common_data.serial_2stop == 0)
>
> serial_new_attr.c_cflag &= ~CSTOPB;
>
> else
>
> serial_new_attr.c_cflag |= CSTOPB;
>
> serial_new_attr.c_cflag &= ~CSIZE;
>
> serial_new_attr.c_cflag |= common_data.serial_bits;
>
>
>
> /* c_lflag
>
> * enable raw input */
>
> serial_new_attr.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG);
>
>
>
> /* Disabling hardware flow control */
>
> serial_new_attr.c_cflag &= ~CRTSCTS;
>
> /* Disabling software flow control */
>
> serial_new_attr.c_iflag &= ~(IXON | IXOFF | IXANY);
>
>
>
> /* Disabling any postprocessing */
>
> serial_new_attr.c_oflag &= ~OPOST;
>
>
>
> /* Minimum characters to read (non-canonical) */
>
> serial_new_attr.c_cc[VMIN] = 1;
>
> /* Wait indefinitely for input (non-canonical) */
>
> serial_new_attr.c_cc[VTIME] = 3;
>
>
>
> cfsetospeed(&serial_new_attr, common_data.serial_speed);
>
> cfsetispeed(&serial_new_attr, common_data.serial_speed);



This comes up wether you are reading a serial port, a socket, a terminal, or some other device (even text with funny delimiters from a disk file).

If you need to process data, you must fill a buffer until you have what the command requires and be prepared to save the leftovers for the next time.

If you can't tell where a command ends, you are badly stuck. But most things have either a length designator or a delimiter of some kind.

There are two main techniques:
One is to read data (perhaps a byte at a time) until you have a command and then read the amount of data (perhaps a byte at a time) required for that command (dealing with leftovers, if you somehow read too much).

The other is to read as much data as you can get comfortably and to parse through it (and either keeping the buffer filled or dealing with getting more whenever you aren't sure that you have enough).

It doesn't matter whether you use read (), read_ (), _read (), __read (), fread (), pread (), ioread (), Read (), READ (), recv (), recvfrom (), portread (), readport (), or GimmeSomeLovin ().

 
Reply With Quote
 
Muffinman
Guest
Posts: n/a
 
      08-11-2013
On 09-08-13 17:55, Muffinman wrote:
> 1) So, can I solve this? Is there anything I can do with my
> configuration (see below)?
>
> 2) Why does this happen? The only thing I can think of is that the
> sending device skips a opportunity to send data every so bytes...


I get a few explanations why this might happen from here and
comp.unix.programer. However, the end result seems to be: deal with it.
So I guess, that's the advice I'm gonna go with. Not sure how, but I'll
manage.

Thanks all, Maarten
 
Reply With Quote
 
glen herrmannsfeldt
Guest
Posts: n/a
 
      08-11-2013
Muffinman <news**REMOVETHIS**@koster.tk> wrote:
> On 09-08-13 17:55, Muffinman wrote:
>> 1) So, can I solve this? Is there anything I can do with my
>> configuration (see below)?


>> 2) Why does this happen? The only thing I can think of is that the
>> sending device skips a opportunity to send data every so bytes...


> I get a few explanations why this might happen from here and
> comp.unix.programer. However, the end result seems to be: deal with it.
> So I guess, that's the advice I'm gonna go with. Not sure how, but I'll
> manage.


In the serial case, each byte is pretty much independent, but it also
happens in the case of TCP. Datagram boundaries are preserved for UDP,
but TCP is a stream protocol.

I previously posted the way it is done when protocols switch from UDP
to TCP, such as NFS.

-- glen
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Enabling LFI breaks large packets on 1751 ADSL WIC Dieter Jansen Cisco 0 06-05-2005 05:32 AM
Test Serial to Serial Connection, Protocol Down... Scooter Cisco 5 12-16-2004 04:38 PM
Can I connect router Serial interface directly to a PC serial port? Faustino Dina Cisco 2 08-18-2004 02:30 AM
Re: Serial port and PS/2 port schematics OR Assistive Tech. suggestion naive.verizon@locality.net Computer Support 1 07-10-2003 11:46 AM
Re: Serial port and PS/2 port schematics °Mike° Computer Support 1 07-09-2003 10:30 PM



Advertisments