Velocity Reviews

Velocity Reviews (http://www.velocityreviews.com/forums/index.php)
-   Perl Misc (http://www.velocityreviews.com/forums/f67-perl-misc.html)
-   -   does it matter whether References headers are folded? (http://www.velocityreviews.com/forums/t899360-does-it-matter-whether-references-headers-are-folded.html)

Nomen Nescio 08-07-2006 03:50 PM

does it matter whether References headers are folded?
 
RFC 1036 says the message-IDs in the References header should be
separated by a space, but it's quite common to see them folded
(separated by a newline and some spaces or a newline and a tab).

Does it matter?

Is either folded or unfolded better or worse than the other?


Dr.Ruud 08-07-2006 04:23 PM

Re: does it matter whether References headers are folded?
 
Nomen Nescio schreef:

> RFC 1036 says the message-IDs in the References header should be
> separated by a space


s/space/blank/

<quote>
2. Message Format
[...]
A standard USENET message consists of several header lines, followed
by a blank line, followed by the body of the message. Each header
line consist of a keyword, a colon, a blank, and some additional
information. [...] The Internet convention of
continuation header lines (beginning with a blank or tab) is
allowed.
</quote>

--
Affijn, Ruud

"Gewoon is een tijger."



Nomen Nescio 08-07-2006 05:50 PM

Re: does it matter whether References headers are folded?
 
"Dr.Ruud" <rvtol+news@isolution.nl> wrote:

> Nomen Nescio schreef:
>
> > RFC 1036 says the message-IDs in the References header should be
> > separated by a space

>
> s/space/blank/
>
> <quote>
> 2. Message Format
> [...]
> A standard USENET message consists of several header lines, followed
> by a blank line, followed by the body of the message. Each header
> line consist of a keyword, a colon, a blank, and some additional
> information. [...] The Internet convention of
> continuation header lines (beginning with a blank or tab) is
> allowed.
> </quote>


Right, but are the two formats absolutely equal or is one
considerd a little better than the other?













Dr.Ruud 08-07-2006 06:06 PM

Re: does it matter whether References headers are folded?
 
Nomen Nescio schreef:
> Dr.Ruud:
>> Nomen Nescio:


>>> RFC 1036 says the message-IDs in the References header should be
>>> separated by a space

>>
>> s/space/blank/
>>
>> <quote>
>> 2. Message Format
>> [...]
>> A standard USENET message consists of several header lines,
>> followed by a blank line, followed by the body of the message.
>> Each header line consist of a keyword, a colon, a blank, and
>> some additional information. [...] The Internet convention of
>> continuation header lines (beginning with a blank or tab) is
>> allowed.
>> </quote>

>
> Right, but are the two formats absolutely equal or is one
> considerd a little better than the other?


Equal. Just don't put a fold directly after the "Keyword:". It is OK to
put each Message-ID on it's own line, but the first (the oldest) must be
on the same line as the "References: ".

You are even allowed to remove Message-IDs from the References header
field (to limit the total (unfolded) length), but certainly don't remove
the one-or-two oldest and newest ones (they are used for the "threaded
view" by many clients).

--
Affijn, Ruud

"Gewoon is een tijger."



Nomen Nescio 08-07-2006 07:30 PM

Re: does it matter whether References headers are folded?
 
Nomen Nescio wrote:

> RFC 1036 says the message-IDs in the References header should be
> separated by a space, but it's quite common to see them folded
> (separated by a newline and some spaces or a newline and a tab).


Technically speaking, after a header reaches a certain length it *must*
be folded to prevent risk of breaking some clients/transports. And
either the space or tab method is acceptable.

=46rom an RFC point of view it doesn't matter.

=46rom an anonymity POV it might (you cross posted to APAS).

Anonymous 08-07-2006 08:21 PM

Re: does it matter whether References headers are folded?
 
In article <2c0c58bebd78da6b816987b210060211@dizum.com>
Nomen Nescio <nobody@dizum.com> wrote:
>
> RFC 1036 says the message-IDs in the References header should be
> separated by a space, but it's quite common to see them folded
> (separated by a newline and some spaces or a newline and a tab).
>
> Does it matter?
>
> Is either folded or unfolded better or worse than the other?


It shouldn't matter, but there are times when it matters to some home-grown
applications. For example I've found that the BigApple mail2news software,
currently run by Bikikii I think, barfs on folded headers by only including
the first line. Other times we've seen non-folded headers truncated by a
limit on line length.

What to do.


Curt Welch 08-07-2006 08:37 PM

Re: does it matter whether References headers are folded?
 
Anonymous <nobody@bikikii.ath.cx.invalid> wrote:
> In article <2c0c58bebd78da6b816987b210060211@dizum.com>
> Nomen Nescio <nobody@dizum.com> wrote:
> >
> > RFC 1036 says the message-IDs in the References header should be
> > separated by a space, but it's quite common to see them folded
> > (separated by a newline and some spaces or a newline and a tab).
> >
> > Does it matter?
> >
> > Is either folded or unfolded better or worse than the other?

>
> It shouldn't matter, but there are times when it matters to some
> home-grown applications. For example I've found that the BigApple
> mail2news software, currently run by Bikikii I think, barfs on folded
> headers by only including the first line. Other times we've seen
> non-folded headers truncated by a limit on line length.
>
> What to do.


References: tend not to be folded and are intelligently truncated to 998
bytes by removing Message-IDs starting with the second on the line
(always leaving the first and as many of the last ones as possible).
Technically, you should be able to fold them but there was a time in the
past that it was felt it would break too many things if you folded the
References: line. I tend to suspect that's not an big issue anymore. The
998 limit was because some news software (INN at the time?) didn't allow
you to post lines longer than that. That was a limit after the folding
was removed so you couldn't "fix" the line length problem with folding.

I have no idea what INN and all the other news servers currently allow for
line lengths and whether the 998 limit is important anymore.

However, References: are one of the fields that are part of XOVER and
keeping References: lines more than 1000 bytes in length I think is just
wasteful at best since all the Message-IDs except the first and last are
generally redundant and of minor importance. So I think it is probably
wise to continue to use a truncation algorithm.

And, if I was writing a new reader, I wouldn't fold References: lines -
just because there's no advantage to doing it that I know of and a slight
risk it would cause an issue.

Here's some more thoughts about References: lines from the GNKSA:

http://www.xs4all.nl/~js/gnksa/gnksa.txt

7) Make sure followups contain valid References

When creating a followup, the software MUST create a "References: "
header line that contains, as its last element, the Message-ID of the
original article. An individual Message-ID MUST never be truncated.

The software MUST include at least three additional Message-IDs from
the original article's References header as well, if they are available.
Try to stay as close as possible to the spirit of "son-of-1036", which
states:

<<Followup agents SHOULD not shorten References headers. If
it is absolutely necessary to shorten the header, as a des-
perate last resort, a followup agent MAY do this by deleting
some of the message IDs. However, it MUST not delete the
first message ID, the last three message IDs (including that
of the immediate precursor), or any message ID mentioned in
the body of the followup.>>

However, it also says:

<<If it is absolutely necessary for an implementation to
impose a limit on the length of header lines, body lines, or
header logical lines, that limit shall be at least 1000
octets, including EOL representations.>>

So, bear in mind that news transports are not guaranteed to be able to
handle arbitrary long lines. Furthermore, keep in mind that some news
transports choke on continued (multi-line) "References: " headers.

Therefore, keep as many Message-IDs as will fit on a line starting with
"References: " with a maximum length of 998 characters. (An octet is a
byte of 8 bits, EOL representation takes two bytes.)

Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
Neither should `bogus' Message-IDs -- IDs that somehow got inserted (by
a misguided user, maybe) but don't belong to the thread.

Rationale: Threaded news-readers depend on References to do their magic.
This too is basic RFC compliance. Be as complete as the line length
limit allows, but do not propagate errors.

--
Curt Welch http://CurtWelch.Com/
curt@kcwc.com http://NewsReader.Com/

Dr.Ruud 08-07-2006 09:26 PM

Re: does it matter whether References headers are folded?
 
Curt Welch schreef:
> Anonymous <nobody@bikikii.ath.cx.invalid> wrote:
>> In article <2c0c58bebd78da6b816987b210060211@dizum.com>
>> Nomen Nescio <nobody@dizum.com> wrote:
>>>
>>> RFC 1036 says the message-IDs in the References header should be
>>> separated by a space, but it's quite common to see them folded
>>> (separated by a newline and some spaces or a newline and a tab).
>>>
>>> Does it matter?
>>>
>>> Is either folded or unfolded better or worse than the other?

>>
>> It shouldn't matter, but there are times when it matters to some
>> home-grown applications. For example I've found that the BigApple
>> mail2news software, currently run by Bikikii I think, barfs on folded
>> headers by only including the first line. Other times we've seen
>> non-folded headers truncated by a limit on line length.
>>
>> What to do.

>
> References: tend not to be folded and are intelligently truncated to
> 998 bytes by removing Message-IDs starting with the second on the
> line (always leaving the first and as many of the last ones as
> possible). Technically, you should be able to fold them but there
> was a time in the past that it was felt it would break too many
> things if you folded the References: line. I tend to suspect that's
> not an big issue anymore. The 998 limit was because some news
> software (INN at the time?) didn't allow you to post lines longer
> than that. That was a limit after the folding was removed so you
> couldn't "fix" the line length problem with folding.
>
> I have no idea what INN and all the other news servers currently
> allow for line lengths and whether the 998 limit is important anymore.
>
> However, References: are one of the fields that are part of XOVER and
> keeping References: lines more than 1000 bytes in length I think is
> just wasteful at best since all the Message-IDs except the first and
> last are generally redundant and of minor importance. So I think it
> is probably wise to continue to use a truncation algorithm.
>
> And, if I was writing a new reader, I wouldn't fold References: lines
> - just because there's no advantage to doing it that I know of and a
> slight risk it would cause an issue.
>
> Here's some more thoughts about References: lines from the GNKSA:
>
> http://www.xs4all.nl/~js/gnksa/gnksa.txt
>
> 7) Make sure followups contain valid References
>
> When creating a followup, the software MUST create a "References: "
> header line that contains, as its last element, the Message-ID of the
> original article. An individual Message-ID MUST never be truncated.
>
> The software MUST include at least three additional Message-IDs from
> the original article's References header as well, if they are
> available. Try to stay as close as possible to the spirit of
> "son-of-1036", which states:
>
> <<Followup agents SHOULD not shorten References headers. If
> it is absolutely necessary to shorten the header, as a des-
> perate last resort, a followup agent MAY do this by deleting
> some of the message IDs. However, it MUST not delete the
> first message ID, the last three message IDs (including that
> of the immediate precursor), or any message ID mentioned in
> the body of the followup.>>
>
> However, it also says:
>
> <<If it is absolutely necessary for an implementation to
> impose a limit on the length of header lines, body lines, or
> header logical lines, that limit shall be at least 1000
> octets, including EOL representations.>>
>
> So, bear in mind that news transports are not guaranteed to be able to
> handle arbitrary long lines. Furthermore, keep in mind that some news
> transports choke on continued (multi-line) "References: " headers.
>
> Therefore, keep as many Message-IDs as will fit on a line starting
> with "References: " with a maximum length of 998 characters. (An
> octet is a byte of 8 bits, EOL representation takes two bytes.)
>
> Exception: Damaged (truncated) Message-IDs SHOULD NOT be included.
> Neither should `bogus' Message-IDs -- IDs that somehow got inserted
> (by
> a misguided user, maybe) but don't belong to the thread.
>
> Rationale: Threaded news-readers depend on References to do their
> magic. This too is basic RFC compliance. Be as complete as the line
> length
> limit allows, but do not propagate errors.




--
Affijn, Ruud

"Gewoon is een tijger."



Dr.Ruud 08-07-2006 09:32 PM

Re: does it matter whether References headers are folded?
 
Curt Welch schreef:

> [son-of-1036]
> <<Followup agents SHOULD not shorten References headers. If
> it is absolutely necessary to shorten the header, as a des-
> perate last resort, a followup agent MAY do this by deleting
> some of the message IDs. However, it MUST not delete the
> first message ID, the last three message IDs (including that
> of the immediate precursor), or any message ID mentioned in
> the body of the followup.>>


There is one (very theoretical) catch in there: a <message-id> can be
that long that a References: header field with four of them wouldn't fit
in 998 characters. But then you would just drop one more (the oldest of
the tail).

--
Affijn, Ruud

"Gewoon is een tijger."



J Moreno 08-08-2006 04:52 AM

Re: does it matter whether References headers are folded?
 
Dr.Ruud <rvtol+news@isolution.nl> wrote:

> Curt Welch schreef:
>
> > [son-of-1036]
> > <<Followup agents SHOULD not shorten References headers. If
> > it is absolutely necessary to shorten the header, as a des-
> > perate last resort, a followup agent MAY do this by deleting
> > some of the message IDs. However, it MUST not delete the
> > first message ID, the last three message IDs (including that
> > of the immediate precursor), or any message ID mentioned in
> > the body of the followup.>>

>
> There is one (very theoretical) catch in there: a <message-id> can be
> that long that a References: header field with four of them wouldn't fit
> in 998 characters. But then you would just drop one more (the oldest of
> the tail).


Hmn, I'm not sure that's actually true -- or at least I'm not sure it
should be true if people play by the rules. There's a limit on the MID
because of a NNTP command, IMS it also should prevent any four MID from
exceeding the 998 limit.

Of course, not everyone plays by the rules, so you're right...still,
rather unlikely.

--
JM
"Everything is futile." -- Marvin of Borg


All times are GMT. The time now is 11:43 PM.

Powered by vBulletin®. Copyright ©2000 - 2014, vBulletin Solutions, Inc.
SEO by vBSEO ©2010, Crawlability, Inc.