What is the difference between SysLog Server and Trap server?.

What is the difference between them?. why those two are required?. Can
those servers can at any place (difference physical locations), if we
want to receive those messages, how to configure them. Thanks.

 

A Syslog server is an external unix linux or even windows (I guess)
machine that will receive log events, these are any kind of events your
system will log, and many systems can Syslog themselves without using
an external server, but give you the option so you can centralize logs.
This is totally indipendant of SNMP.

SNMP Traps are events the device considers important enough (changes in
config, overheating, etc), to which it sends to the Trap server a Trap
(which is basically an annoucement) on the event and it's value so the
server can act acoordingly...

One example of an SNMP Trap is a network enabled UPS sending a
power-out Trap so a server controller can monitor the batery status and
shut off machines conected to the ups when the level is critical.

 

HI,

syslog and SNMP are different network application layer protocols.

Syslog messages and SNMP notifications (traps and informs) provide
the same functionality, and can be used to encapsulate each other's
messages. However, they come from different sources. Classic
syslog provides no security of the messages. That is, the
sender is not verified, nor are the messages checked for
modification, and they are not encrypted. Thus, messages
can be spoofed, messages can be modified, and message
content can be viewed. There are several versions of the
SNMP protocol - SNMPv1, SNMPv2c, and SNMPv3/USM are the
most popular. SNMPv1 and SNMPv2c messages have little
supoprt for security and have the same problem as syslog
messages. SNMPv3/USM supports "strong security", and
thus can support identification of sender, checks for
message modification, and message content encryption.

How you configure each is "implementation specific".

Regards,
/david t. perkins

 

Does Syslog is also have standard like SNMP (I mean is Syslog is a
Protocol?), Is there special RFC for Syslog like SNMP?. If you know
what is the RFC number for syslog?.

 

oes Syslog is also have standard like SNMP (I mean is Syslog is a
rotocol?), Is there special RFC for Syslog like SNMP?. If you know
:what is the RFC number for syslog?.

syslog serves such different purposes from SNMP that it's nearly
useless to compare the two.

Eventually syslog was put into an RFC, but it was relatively late...
it was one of those protocols that there wasn't enough disagreement
about to make it worth standardizing earlier.

http://www.faqs.org/rfcs/rfc3164.html

 

On the contrary, syslog was among the several popular BSD protocols
that were sufficiently well defined by the BSD code that any text
definition would probably contain significant errors. I've not
previously looked at RFC 3164, but I have heard grumbling that it is
particularly bad about redefining an existing, well defined protocol.

I would have thought that almost 20 years is more than relatively late.
I'd guess that the reason RFC 3164 was so late that the syslog protocol
is painfully simple, and painfully insecure outside a safe network.

I just now looked at RFC 3164. There is so much verbiage that I
can't readily tell whether it describes the real protocol. The
other supposed RFC definitions of BSD protocols I've examined had
significant errors that were obvious to people genuinely familiar
with the protocol (e.g. RIPv1).


Vernon Schryver

 

HI,

Why do you say "syslog serves such different purposes from SNMP that it's
nearly useless to compare the two"?

And by the way, there is an IETF WG working on syslog. Here
is the URL:
http://www.ietf.org/html.charters/syslog-charter.html

Regards,
/david t. perkins

 

:Why do you say "syslog serves such different purposes from SNMP that it's
:nearly useless to compare the two"?

I said it because the poster I was replying to seemed intent on
comparing the two, starting with the "What is the difference" question
and contining on to questions about RFCs and standards "like SNMP".

Syslog SNMP
------ -----

-passive active
-insecure securable
-text-based binary based
-non-extensible extensible
-one-way two-way
-monitoring monitoring and control
-readable by machine parsible
experienced humans
- message meanings message meanings usually documented
often not documented

 

I have no clue what that means. What is active/passive here?

Syslog almost no specific format (except a very few values) since it
carries mostly text content. I would say this is extensively extensible.

Again, I am not sure what you mean, but it might not be correct.

For me, the biggest difference is that syslog messages are extremely
easy to generate (see syslog(3)) while generating SNMP notifications
usually requires more work. The price for being easy to generate is
that the receiver may have a harder time to parse the message and do
something with it. On the other hand, regular expressions can do real
wonders here. So perhaps putting that burdon on the receiver side is
actually a smart thing because coders like to put syslog() calls into
their code while writing a MIB module in order to air a notification
seems like a punishment for many programmers.

/js

 

[changing follow-ups, which I have changed back]


:> Syslog SNMP

:> -passive active

:I have no clue what that means. What is active/passive here?

With syslog your receiver system just sits there and waits for
messages. With SNMP, your system goes out and asks for values and
gets results returned.


:> -non-extensible extensible

:Syslog almost no specific format (except a very few values) since it
:carries mostly text content. I would say this is extensively extensible.

You cannot change the facilities or priority encodings, and you
cannot even put a year into the timestamp. The -only- thing you can
do with syslog is toss text into the undifferentiated morass of the
content field, and hope that the other end can understand it.


:> -one-way two-way

:Again, I am not sure what you mean, but it might not be correct.

This is another way of phrasing the active/passive point above.
The transport for syslog is unidirectional, with no concept at all
of a "reply".


:For me, the biggest difference is that syslog messages are extremely
:easy to generate (see syslog(3)) while generating SNMP notifications
:usually requires more work. The price for being easy to generate is
:that the receiver may have a harder time to parse the message and do
:something with it. On the other hand, regular expressions can do real
:wonders here. So perhaps putting that burdon on the receiver side is
:actually a smart thing because coders like to put syslog() calls into
:their code while writing a MIB module in order to air a notification
:seems like a punishment for many programmers.

I encountered this thread via one of the newsgroups you trimmed
out on the follow-ups, comp.dcom.sys.cisco, so I will write briefly
about how "smart" it is to have cisco equipment send out syslog.

I do a fair bit of work with the Cisco PIX line of firewalls,
and have worked on automatted analysis of PIX firewall messages.
There is only one commercial company that I know of which has
such an analyzer (Network Intelligence) and they are getting out of
that line. The program 'sawmill' often gets mentioned in this context,
but the PIX "plugin" for sawmill can handle exactly *two* of the
messages; those two messages do not exist from PIX 6.2 onwards and
I can pretty much guarantee that the processing for the replacement
messages is going to be wrong.

I'm not the dumbest coder in the world, but I spent literally *months*
working on my set of PIX syslog analyzers, trying to produce programs
that were flexible, useful -- and efficient. It wasn't all that
difficult in PIX 5.x thru 6.1 to parse more than 10 times faster
than Network Intelligence's offering... but that is still nowhere
near fast enough for real work.

I was writing in perl, and I can summarize a few lessons:

- PIX messages might have any of 3 different timestamp formats,
consistant for any one PIX (until reconfigured) but different PIX
feeding into the same syslog file might be configured differently

- There are at least 392 different messages produced by the PIX, many of
which have multiple modes that must be distinguished in order to parse
the message properly. For example, a single message code might talk about
-- tcp (connection oriented, must be matched up with other messages to
get the whole story)
-- udp (non connection oriented, semantically difficult to say which
side "originated" the connection)
-- icmp (no ports, the major and minor type codes appear in a completely
different location with a completely different syntax than for tcp and
udp ports)
-- other protocols, which appear as simply a protocol name for well
known protocols such as 'gre', but appear as the word 'protocol' followed
by the decimal number for other protocols it doesn't know the name of

- regular expressions aren't nearly fast enough. Even just a
regular expression that matches the line to determine whether the
message is a known PIX message isn't fast enough. (The same log
file may have non-PIX devices feeding into it, so there might not
be a match at all; the message set may have changed due to a software
upgrade; you may have encountered an undocumented message; the message
code might be in a different place because of the different timestamp
formats.) Even if you "precompile" the regular expression and are just
matching on the message codes, the match is much much too slow.

- With crafted perl code that uses heuristics to locate the message
code and using a few string comparisons and hard-coded array offsets,
and only doing matching to pull apart interface:address/port
constructs, over 75% of the processing time is spent just doing
elementary blank-delimited tokenization (i.e., the perl 'split'
operation.) This would not be necessary with a structured binary
format

- In PIX 5.2 thru 6.1, nearly all useful connection information could
be deduced by context; the only information that needed to be carried
around was whether a particular tcp or udp connection number was
still active. As of PIX 6.2, the messages were revised to drop
key positional semantics, making it necessary to carry around much
more state -- and making it impossible in some cases to figure out
whether a connection was in-going or outgoing except by studying
other logged events in order to deduce the security hierarchy

- The PIX does not even log permitted connections for protocols
other than tcp, udp, or icmp (and permitted icmp only gets logged
under the "intrusion detection signatures")

- debug (as in the 'debug' command) information for the PIX is nearly
structureless and doesn't get sent to syslog anyhow. This is,
though, exactly the sort of situation which would qualify as
"coders like to put syslog() calls into their code". Think of
sendmail and it's hundreds of debugging syslog() calls, all of
which are subject to radical change in structure in even dot dot dot
releases (also known as "small patches").


In short, except when used by highly disciplined programmers,
syslog is a poor mechanism for communicating important information
for any manner of automated analysis and control. It's just too
easy to syslog() out a sloppy message that might [-maybe-] make
sense to a human but which is difficult for a program to analyze
without building a full state machine with memory.

 

SNMP also supports notifications where "your receiver system just sits
there and waits for messages". (Note the subject line.)

Sometimes having a morass of content is better than having nothing at
all. Doing things via SNMP has its implementation costs.

[...]

You talk quite a bit about efficiency but you do not tell the details.
How many messages do you process per second on average and max? How
many messages does a single device produce per second on average
and max? Do you have any reasons to believe that multiple revisions
of SNMP instrumentations will be more consistent? Do you have any reason
to believe that SNMP MIB module designers will put the right thing into
the messages that make your life easy? Have you a reason to believe
that ASN1/BER decoding is really that much faster than blank-delimited
tokenization?

The problem with SNMP is that kicking out event notifications
is a rather complex process where you have to write MIB modules and
than usually do some not totally trivial coding. The end result is
that operators tell the SNMP community that SNMP notifications are
problematic because there are just too few of them and the details
are all in the ugly syslog messages.

What is really missing is something which is only more slightly
complicated to use than syslog() but still allows to ship structured
information around and which everybody would agree with. Not an easy
thing to come up with.

/js

 

:You talk quite a bit about efficiency but you do not tell the details.
:How many messages do you process per second on average and max?

Does it matter? Would any absolute number mean anything to you?

But since you ask:

I did the bulk of the work two years ago, and so have forgotten
the detailed numbers as I tweaked the code. If I recall correctly,
when using regular expressions the average processing rate was
85 messages per second, and when using plain blank tokenization
and hard-coded array offsets, I averaged 3120 messages per second.
When PIX 6.2 came out and destroyed the contextual information,
the additional state I had to carry around slowed processing to
approximately 556 messages per second.

This figures are from running on a 250 MHz SGI Challenge XL with
1.5 Gb of memory [which was quite a lot of its day].

Network Intelligence's Private I product, running over the same
data on an 833 MHz Pentium Pro averaged approximately 112 messages
per second -- considerably less than the 500 messages per second
than the product was rated for on that speed of system.


:How
:many messages does a single device produce per second on average
:and max?

I don't know about the max. Our long term average for the 60 days
starting Dec 1 2004 to Jan 29 2005 was 837174 messages per day,
which is about 9.7 per hour. This includes a gap where the logging
system was down for a half a day -- probably about 600,000 messages
lost due to that. The 60 day total was over 50 million messages.
The overnight traffic is *much* lower than the daytime traffic --
it isn't uncommon to fill up 10 megabytes within a matter of 2 minutes
during the day.

Note that the phrasing of your questions presupposes that the
processing is done as the messages come in, which is not the case.
Queries for specific information are usually done over 2 to 3 days
worth of data, and reports are usually done per week or per month.
Thus a typical query would be over ~ 2 1/2 million records; if
an answer is to be had within 5 minutes [otherwise the human gets
distracted into something else] then processing rates have to
exceed about 8000 messages per second.


o you have any reasons to believe that multiple revisions
f SNMP instrumentations will be more consistent?

Yes. Once Cisco starts using a MIB it is years before they switch.
The last MIB tarball that I pulled down from them had only 10 MIB
files out of 532 that represented superceeded MIBs [though the number
that had effectively fallen into disuse due to product line changes
would be higher of course.]

o you have any reason
:to believe that SNMP MIB module designers will put the right thing into
:the messages that make your life easy?

Yes. I have examined Cisco's QoS MIB (which happens to be the one
that has meaningful ACLs) and the parameters are usefully positional
and have direction flags.

:Have you a reason to believe
:that ASN1/BER decoding is really that much faster than blank-delimited
:tokenization?

That I don't know.

 

Hey Walter,

In the Cisco product line alone there are a tad bit over 8000 DIFFERENT
syslog message types.

Check out the Cisco product CNote... If you like Syslog turned to
traps...

FWIW, I do correlation in Syslog along with some SNMP Traps. And I too
wrote my own in Perl.

What do you consider fast? Would 20 traps a second sustained be OK?
What about 100?

My Syslog correlation engine REGEXS and processes syslog in near real
time (within a couple of seconds...) across Cisco, Juniper, Foundry and
a bevy of hosts. It reads from several Syslog logs simultaneously and
can handle file resets, inode changes and truncations without having to
bounce.

Right now, the peak sustained rate of Syslog I've processed is 101,842
lines of syslog in a minute but sustained over 80,000 lines a minute
for over an hour during the peak. My box is an old 2 way J Class HP...
My process runs in a SINGLE process WITHOUT THREADS.

I've been benchmarking my stuff as ~ 10X faster than Java + Threads on
10 percent of the memory footprint and I have yet to break 2 seconds to
forward an alert on to an operator. And its kicking the doggie doo out
of alot of commercial products that are similar...

SNMP Traps are much more structured. But along with that structure
comes a bit of overhead in the code. Both on the Agent and the
processing manager.

Syslog is pretty lightweight... The Date / Time fields at the front
of Syslog come from the syslogd process itself as does the Host in the
4th field. And most syslogds go ahead and resolve the inbound IP
address as part of processing the incoming syslog message.

I'm in the process now of getting ver 2 out into production which
empowers multiple sources to be processed across multiple machines and
multiple processes.

We live by the saying 24 By Forever at my work... And our first
requirement is THOU SHALT SCALE.

It is possible to handle this in near real time.

FWIW, I'm thinking about putting some similar code behind an Endace
Measurement Systems DAG Card and see if I can process the pcap filters
of a couple of OC192s.... Have to break out a quick Linux box ...

Dougie!!!

 

:My Syslog correlation engine REGEXS and processes syslog in near real
:time (within a couple of seconds...) across Cisco, Juniper, Foundry and
:a bevy of hosts.

:Right now, the peak sustained rate of Syslog I've processed is 101,842
:lines of syslog in a minute

Upon reading your posting, I did some timing tests last night on
the system I am processing on. I was using a file of about
220 megabytes, which was about 1.2 million lines.

wc -l (the standard wordcount program) took 43 seconds to process
the file.

perl with an empty while (<>) loop and printing $. at the end took
26 seconds to process the file (interesting that it was faster.)

perl asked with while (<>) loop that simply asked to 'split' the lines,
took 665 seconds to process the file, equivilent to about 116,000 lines
per minute. If any actual work had been involved on each line, it
clearly would have been impossible on my system to reach the 101,842
lines per minute you were able to process.

:2 year old HP J series

I'm working with a 10 year old SGI Challenge XL, 250 MHz.

 

Check out http://poe.perl.org

I am using the FollowTail wheel, a Filter::Line, and the box stock
Select loop.

My patterns are arranged in a hash using the pattern as the key and a
coderef to call to exec parsers/handlers.

Each parser splits the line up to 5 different ways to extract out the
relevant data and push it to a socket connecting to a Netcool TCP Port
probe.