% $Header: /home/jaguar/u0/vern/bro/bro-doc/RCS/analysis.tex,v 1.18 2000/11/04 00:59:14 vern Exp vern $

\labsectchapter{{Analyzers and Events}}{\label{analyzers}}
%HEVEA\cutdef[1]{section}

\index{analyzers|(}
\index{scripts!standard|(}
\index{standard scripts|(}
In this chapter we detail the different analyzers that Bro provides.
Some analyzers look at traffic in fairly generic terms, such as at
the level of TCP or UDP connections.  Others delve into the specifics
of a particular application that is carried on top of TCP or UDP.

As we use the term here, \emph{analyzer} primarily refers to Bro's event
engine.  We use the term \emph{script} to refer to a set of event handlers
(and related functions and variables) written in the Bro language;
\emph{module} to refer to a script that serves primarily to provide utility
(helper) functions and variables, rather than event handlers; and
\emph{handler} to denote an event handler written in the Bro language.
Furthermore, the \emph{standard script} is the
script that comes with the Bro distribution for handling the events
generated by a particular analyzer.

\note{ However, we also sometimes use \emph{analyzer} to refer to the event
handler that processes events generated by the event engine. }

We characterize the analyzers in terms of \emph{what} events they
generate, but don't here go into the details of \emph{how} they
generate the events (i.e., the nitty gritty C++ implementations of
the analyzers).
\privatenote{Somewhere go into these details.}

\labsectsection{{Activating an Analyzer}}{\label{analyzer-activation}}

\index{analyzers!activating}
\index{analyzers!instantiating}
\index{performance!analysis tradeoffs}
In general, Bro will only do the work associated with a particular
analyzer if your policy script defines one or more event handlers
associated with the analyzer.  For example, Bro will
instantiate an FTP analyzer only if your script defines an
{\tt ftp_request} or {\tt ftp_reply} handler.  If it doesn't, then
when a new FTP connection begins, Bro will only instantiate a
generic TCP analyzer for it.  This is an important point, because
some analyzers can require Bro to capture a large volume of
traffic \pxref{analyzer-filters} and perform a lot of computation;
therefore, you need to have a way to trade off between the type
of analysis you do and the performance requirements it entails,
so you can strike the best balance for your particular monitoring needs.

\privatenote{``instantiate''? ``activate''?}

\deficiency{While Bro attempts to instantiate an analyzer if you
define a handler for \emph{any} of the events the analyzer generates,
its method for doing so is incomplete: if you only define an analyzer's
less mainstream handlers, Bro may fail to instantiate the analyzer.}
\privatenote{Have an analyzer's event namespace well enough defined that Bro can
easily and definitively tell if you've defined any of its handlers.}

\labsectsubsection{{Loading Analyzers}}{\label{analyzer-load}}

\index{analyzers!loading}
\index{analyzers!\loadx}
The simplest way to use an analyzer is to \load the standard script
associated with the analyzer.  (See \xref{load-directive} for a discussion
of \loadx.)  However, there's nothing magic about these scripts; you can
freely modify or write your own.  The only caveat is that some scripts
\load other scripts, so the original version may wind up being loaded
even though you've also written your own version.
\deficiency{It would be useful to have a mechanism to fully override one
script with another.}

In this chapter we discuss each of the standard scripts
as we discuss their associated analyzers.

\labsectsubsection{{Filtering}}{\label{analyzer-filters}\label{filtering}}

\index{filters|(}
\index{analyzers!filtering|(}
\index{performance!filtering}
Most analyzers require Bro to capture a particular type of network traffic.
These traffic flows can vary immensely in volume, so different analyzers
can cost greatly differing amounts in terms of performance.

\label{capture-filter}
\label{capture-filter-var}
\indvarbegin{capture_filter}
\indvarbegin{restrict_filter}
Bro predefines two redefinable {\tt string} variables that have special
interpretations with regard to filtering.  (See \xref{redef} for a discussion
of redefinable variables.)
{\tt capture_filter} is a \indttbang{tcpdump}{filters}
filter that tells Bro what traffic it should capture.  {\tt restrict_filter}
\label{restrict-filter}
\emph{limits} what traffic Bro captures.  The {\tt tcpdump} filter Bro
uses is:
\begin{quote}
{\tt (\emph{capture_filter}) and (\emph{restrict_filter})}
\end{quote}
So, for example, if you specify:
\begin{verbatim}
    redef capture_filter = "port http";
    redef restrict_filter = "net 128.3";
\end{verbatim}
then the corresponding {\tt tcpdump} filter will be:
\begin{verbatim}
    (port http) and (net 128.3)
\end{verbatim}
which will capture all TCP port 80 traffic that has either a source
or destination address belonging to the {\tt 128.3} network (i.e.,
{\tt 128.3/16}).

\index{filtering!default}
\index{default!filtering}
If you do not define {\tt capture_filter}, then its value
is set to ``{\tt tcp or udp}'';
\privatenote{should include ICMP}
if you do not define {\tt restrict_filter}, then no restriction is
in effect.

\index{restricting traffic}
\index{traffic!restricting}
\index{excluding hosts}
\index{hosts!excluding}
You may have noticed that other than their default values, the
definitions of {\tt capture_filter} and {\tt restrict_filter}
are symmetric.  They differ only in the convention of how they
are used.  Usually, you either don't define a value for {\tt restrict_filter}
at all, or define it just once, using it to specify a restriction
that holds across your environment.  For example, either to confine
packet capture to a subset of the traffic (like the {\tt "net 128.3"}
example above), or to exclude a particular traffic source ({\tt
"not host syn-flood.magnet.com"}) or both of these
({\tt "net 128.3 and not host syn-flood.magnet.com"}).

For {\tt capture_filter}, on the other hand,
you usually don't define a single value, but instead \emph{refine}
\opind{+=}
it one or more times using the {\tt +=} initializer.  (See \xref{refinement}
for a discussion of refining a variable's initial value.)  The way
{\tt capture_filter}'s refinement is defined, it constructs a filter
that is the ``or'' of each of its refinements.  So, for example, if
at one point in your script you have:
\begin{verbatim}
    redef capture_filter += "port ftp";
\end{verbatim}
and at another:
\begin{verbatim}
    redef capture_filter += "udp port 53";
\end{verbatim}
and at a third:
\begin{verbatim}
    redef capture_filter += "len >= 512 and len <= 1024";
\end{verbatim}
then the resulting {\tt capture_filter} will be:
\begin{verbatim}
    (port ftp) or (udp port 53) or (len >= 512 and len <= 1024)
\end{verbatim}
(except there will be more parentheses, which don't actually
affect the interpretation of the filter; see \xref{add-func-attr}
for the details of how the refinement is done, and why it leads
to the extra parentheses).

{\tt restrict_filter} has the same refinement mechanism, the ``or''ing
together of the different refinement additions, though, as mentioned above,
it is not usually refined.

\begin{figure}
\label{fig:print-filter}
\begin{verbatim}
event bro_init()
    {
    if ( restrict_filter == "" && capture_filter == "" )
        print "tcp or not tcp";  # Capture everything.

    else if ( restrict_filter == "" )
        print capture_filter;

    else if ( capture_filter == "" )
        print restrict_filter;

    else
        print fmt("(%s) and (%s)", capture_filter, restrict_filter);

    exit();
    }
\end{verbatim}
\caption{
\indeventnolabel{bro_init}
\indanalyzernolabel{print-filter}
prints out the {\tt tcpdump} filter your Bro script
would use and then exits.
}
\end{figure}

\index{filters!displaying}
As you add analyzers, the final {\tt tcpdump} filter can become quite
complicated.  You can use the predefined \indanalyzer{print-filter} script
shown in \f{fig:print-filter} to print out the filter.
This script handles the \xrefevent{bro_init} event and exits after printing
the filter.  Its intended use is that you can add it to the Bro command
line (``{\tt bro \emph{my-own-script} print-filter}'') when you want to
see what filter the script \emph{my-own-script} winds up using.

\index{debugging!filtering problems}
\index{filters!errors}
\index{tcpdump!bugs}
\index{bugs!tcpdump@{\tt tcpdump}}
There are two particular uses for {\tt print-filter}.  The first is to debug
filtering problems.  Unfortunately, Bro sometimes
uses sufficiently complicated expressions that they tickle bugs in
{\tt tcpdump}'s optimizer.  You can take the filter printed out for
your script and try running it through {\tt tcpdump} by hand, and
then also try using
\indttbang{tcpdump}{turning off optimization}'s
{\tt -O} option to see if turning
off the optimizer fixes the problem.

\index{Bro!shadow}
\index{shadowing}
The second use is to provide a \emph{shadow} backup to Bro: that is,
a version of
\indttbang{tcpdump}{shadow}
running either on the same machine or a
separate machine that uses the same network filter as Bro.  While
{\tt tcpdump} can't perform any analysis of the traffic, the shadow
guards against the possibility of Bro crashing, because if it does,
you will still have a record of the subsequent network traffic which
you can run through Bro for post-analysis.

\indvarend{restrict_filter}
\indvarend{capture_filter}
\index{filters|)}
\index{analyzers!filtering|)}

\labsectsection{{General Processing Events}}{\label{bro-proc-events}}

\index{general Bro processing events}
\index{events!general Bro processing}
Bro provides the following events relating to its overall processing:

\begin{description}
\index{initialization event}
\index{startup!event}
\index{events!initialization}
\index{events!startup}
\indeventtype{bro_init}{}
generated when Bro first starts up.  In particular, after Bro has initialized
the network (or initialized to read from a save file) and executed any
initializations and global statements \pxref{global-stmts}, and just before
Bro begins to read packets from the network input source(s).

\index{termination event}
\index{finish event}
\index{network cleanup event}
\index{events!termination}
\index{events!finish}
\indeventtype{net_done}{t: time}
generated when Bro has finished reading from the network,
due to either having exhausted reading the save file(s), or having
received a terminating signal \pxref{signal-handling}.  \deficiency{ This
event is generated on a terminating signal even if Bro is not reading
network traffic. }  {\tt t} gives the time at which network processing
finished.

This event is generated \emph{before} {\tt bro_done}.  \note{ If Bro
terminates due to an invocation of \xreffunc{exit}, then this event is
\emph{not} generated. }

\index{termination event}
\index{finish event}
\index{cleanup event}
\index{events!termination}
\index{events!finish}
\indeventtype{bro_done}{}
generated when Bro is about to terminate, either due to having exhausted
reading the save file(s), receiving a terminating signal
\pxref{signal-handling}, or because Bro was run without the network input
source and has finished executing any global statements \pxref{global-stmts}.

This event is generated \emph{after} {\tt net_done}.  If you have cleanup
that only needs to be done when processing network traffic, it likely is
better done using {\tt net_done}.  \note{ If Bro terminates due to an
invocation of \xreffunc{exit}, then this event is \emph{not} generated.  }

\label{signal-handling}
\index{signal handling}
\index{handling signals}
\index{SIGTERM@\emph{SIGTERM}} \index{TERM signal@\emph{TERM} signal}
\index{SIGINT@\emph{SIGINT}} \index{INT signal@\emph{INT} signal}
\index{SIGHUP@\emph{SIGHUP}} \index{HUP signal@\emph{HUP} signal}
\indeventtype{bro_signal}{signal: count}
generated when Bro receives a signal.  Currently, the signals Bro
handles are \emph{SIGTERM}, \emph{SIGINT}, and \emph{SIGHUP}.

Receiving either of the first two terminates Bro, though if Bro is in the
middle of processing a set of events, it first finishes with them before
shutting down.  The shutdown leads to invocations of \xrefevent{net_done}
and \xrefevent{bro_done}, in that order.  \deficiency{ In this case,
Bro fails to invoke {\tt bro_signal}, clearly a bug. }

Upon receiving \emph{SIGHUP}, Bro invokes \xreffunc{flush_all} (in addition
to your handler, if any).

\index{network statistics}
\index{packets!drops}
\indeventtype{net_stats_update}{t: time, ns: net_stats}
This event includes two arguments, {\tt t}, the {\tt time} at which
the event was generated, and {\tt ns}, a {\tt net_stats} record,
as defined in \f{fig:net-stats}.  Regarding this second parameter,
the {\tt pkts_recvd} field gives the total number of packets accepted
by the packet filter so far during this execution of Bro; {\tt pkts_dropped}
gives the total number of packets reported \emph{dropped} by the kernel;
and {\tt interface_drops} gives the total number of packets reported
by the kernel as having been dropped by the network interface.

\note{An important consideration is that, as shown by experience, the
kernel's reporting of these statistics is not always accurate.
In particular, the {\tt \$pkts_dropped}
statistic is sometimes missing actual packet drops, and some operating
systems do not support the {\tt interface_drops} statistic at all.
See the \xrefindevent{ack_above_hole} event for an alternate
way to detect if packets are being dropped.}

\end{description}

\begin{figure}
\label{fig:net-stats}
\begin{verbatim}
type net_stats: record {
    # All counts are cumulative.
    pkts_recvd: count;       # Number of packets received so far.
    pkts_dropped: count;     # Number of packets *reported* dropped.
    interface_drops: count;  # Number of drops reported by interface(s).
};
\end{verbatim}
\caption{
Definition of the \indtt{net_stats}{record} record. }
\end{figure}

\labsectsection{{Generic Connection Analysis}}{\label{analy-conn}}

\index{analyzers!generic|(}
\index{connection!analysis}
\index{connection!generic analysis}
\index{generic connection analysis}
The \indanalyzer{conn} analyzer performs generic connection analysis:
connection start time, duration, sizes, hosts, and the like.  You don't
in general load \analyzer{conn} directly, but instead do so implicitly
by loading the \xrefanalyzer{tcp}, \xrefanalyzer{udp}, or \xrefanalyzer{icmp}
analyzers.
Consequently, \analyzer{conn} doesn't load a {\tt capture_filter} value
by itself, but instead uses whatever is set up by these more specific
analyzers.

\analyzer{conn} analyzes a number of events related to connections beginning
or ending.  We first describe the {\tt connection} record data type that
keeps track of the state associated with each connection \pxref{conn-record},
and then we detail the events in \xref{conn-events}.  The main output of its
analysis are one-line connection summaries, which we describe in
\xref{conn-summaries}, and in \xref{conn-functions} we give an overview
of the different callable functions provided by \analyzer{conn}.

\analyzer{conn} also loads three other Bro modules: the \xrefanalyzer{hot}
and \xrefanalyzer{scan} analyzers, and the \xrefmodule{port-name} utility
module.

\labsectsubsection{{The {\tt connection}} record}{\label{conn-record}\label{connection-type}}
\index{record!{\tt connection}}
\index{connection record}

\begin{figure}
\label{fig:conn-record}
\begin{verbatim}
type conn_id: record {
    orig_h: addr;  # Address of originating host.
    orig_p: port;  # Port used by originator.
    resp_h: addr;  # Address of responding host.
    resp_p: port;  # Port used by responder.
};

type endpoint: record {
    size: count;  # Bytes sent by this endpoint so far.
    state: count; # The endpoint's current state.
};

type connection: record {
    id: conn_id;        # Originator/responder addresses/ports.
    orig: endpoint;     # Endpoint info for originator.
    resp: endpoint;     # Endpoint info for responder.
    start_time: time;   # When the connection began.
    duration: interval; # How long it was active (or has been so far).
    service: string;    # The service we associate with it (e.g., "http").
    addl: string;       # Additional information associated with it.
    hot: count;         # How many times we've marked it as sensitive.
};
\end{verbatim}
\caption{
Definition of
\indtt{conn_id}{record}
and
\indtt{connection}{record}
records. }
\end{figure}

\index{connection!addresses}
\index{connection!initiator}
\index{connection!originator}
A \indtt{connection}{record} record holds the state associated with a connection,
as shown in \f{fig:conn-record}.  Its first field, \indttzero{id}, is
defined in terms of the \indtt{conn_id}{record} record, which has the
following fields:
\begin{description}
\indfield{orig_h}{conn_id}
The IP address of the host that originated (initiated) the connection.
In ``client/server'' terminology, this is the ``client.''

\index{connection!ports}
\indfield{orig_p}{conn_id}
The TCP or UDP port used by the connection originator (client).  For
ICMP ``connections'', it is set to 0 \pxref{icmp-conn}.

\index{connection!addresses}
\index{connection!initiator}
\index{connection!originator}
\indfield{resp_h}{conn_id}
The IP address of the host that responded (received) the connection.
In ``client/server'' terminology, this is the ``server.''

\index{connection!ports}
\indfield{resp_p}{conn_id}
The TCP or UDP port used by the connection responder (server).  For
ICMP ``connections'', it is set to 0 \pxref{icmp-conn}.

\end{description}

The \indttzero{orig} and \indttzero{resp} fields of a \indtt{connection}{record}
record both hold \indtt{endpoint}{record} record values, which consist
of the following fields:

\begin{description}
\index{connection!bytes}
\index{connection!size}
\indfield{size}{endpoint}
How many bytes the given endpoint has transmitted so far.  Note that
for some types of filtering, the size will be zero until the connection
terminates, because the nature of the filtering is to discard the
connection's intermediary packets and only capture its start/stop packets
\pxref{syn-fin-filtering}.

\begin{table}
\label{tbl:conn-record-states}
\begin{center}
\begin{tabular}{|l|l|} 
\hline
{\bf State} & {\bf Meaning} \\
\hline
\hline
{\tt TCP_INACTIVE} & The endpoint has not sent any traffic. \\
\hline
{\tt TCP_SYN_SENT} & It has sent a SYN to initiated a connection. \\
\hline
{\tt TCP_SYN_ACK_SENT} & It has sent a SYN ACK to respond to a connection request. \\
\hline
{\tt TCP_PARTIAL} & The endpoint has been active, but we did not see the
			beginning of the connection. \\
			\hline
{\tt TCP_ESTABLISHED} & The two endpoints have established a connection. \\
\hline
{\tt TCP_CLOSED} & The endpoint has sent a FIN in order to close its end of the
	connection. \\
	\hline
{\tt TCP_RESET} & The endpoint has sent a RST to abruptly terminate the\
connection. \\
\hline
\hline
{\tt UDP_INACTIVE} & The endpoint has not sent any traffic. \\
\hline
{\tt UDP_ACTIVE} & The endpoint has sent some traffic. \\
\hline
\end{tabular}
\end{center}
\caption{
TCP and UDP connection states, as stored in an \indtt{endpoint}{record} record.
}
\end{table}

\index{connection!state}
\indfield{state}{endpoint}
The current state the endpoint is in with respect to the connection.
\tbl{tbl:conn-record-states} defines the different possible states
for TCP and UDP connections.
\privatenote{Need states for [some] ICMP ``connections,'' too.}
\deficiency{The states are currently defined as {\tt count}, but should
instead be an enumerated type; but Bro does not yet support enumerated types.}

\note{ UDP ``connections'' do not have a well-defined structure, so
the states for them are quite simplistic.  See \xref{udp-conn} for
further discussion. }

\end{description}

The remaining fields in a \indtt{connection}{record}
record are:

\begin{description}
\index{connection!start time}
\index{beginning time of a connection}
\index{start time of a connection}
\indfield{start_time}{connection}
The time at which the first packet associated with this connection was seen.

\index{connection!duration}
\index{duration of a connection}
\indfield{duration}{connection}
How long the connection lasted, or, if it is still active, how long since
it began.

\index{connection!service}
\index{service associated with a connection}
\indfield{service}{connection}
The name of the service associated with the connection.  For example,
if \verb+$id$resp_p+ is \verb+tcp/80+, then the service will be
{\tt "http"}.  Usually, this mapping is provided by the \indvar{port_names}
global variable, perhaps via the \xreffunc{endpoint_id} function; but
the service does not always directly correspond to
\verb+$id$resp_p+, which is why it's a separate field.  In particular,
an FTP data connection can have a {\tt service} of {\tt "ftp-data"}
even though its \verb+$id$resp_p+ is something other than \verb+tcp/20+
(which is not consistently used by FTP servers).

If the name of the service has not yet been determined, then this field
is set to an empty string.

\index{connection!additional information}
\index{information associated with a connection}
\index{additional information associated with a connection}
\indfield{addl}{connection}
Additional information associated with the connection.  For example,
for a \emph{login} connection, this is the username associated with
the login.

\deficiency{
A significant \emph{deficiency} associated with the {\tt addl} field
is that it is simply a {\tt string} without any further structure.
In practice, this has proven too restrictive.  For example, we
may well want to associate an unambiguous username with a \emph{login}
session, \emph{and} also keep track of the names associated with
failed login attempts.  (See the \xrefanalyzer{login} analyzer
for an example of how this is implemented presently.)  What's
needed is a notion of \indtt{union}{types, need for} types
which can then take on a variety of values in a type-safe manner.
\privatenote{\index{needed!union types}}
}

If no additional information is yet associated with this connection,
then this field is set to an empty string.

\index{connection!hot}
\index{connection!sensitivity}
\index{sensitivity associated with a connection}
\indfield{hot}{connection}
How many times this connection has been marked as potentially sensitive
or reflecting a break-in.  The default value of 0 means that so far the
connection has not been regarded as ``hot''.

\note{Bro does not presently make fine-grained use of this field; the
standard scripts log connections with a non-zero {\tt hot} field,
and do not in general log those that do not, though there are exceptions.
In particular, the {\tt hot} field is \emph{not} rigorously maintained
as an indicator of trouble; it instead is used loosely as an indicator
of particular types of trouble (access to sensitive hosts or usernames).}

\end{description}

\subsection{Definitions of connections}
\index{connection!definitions}

\index{TCP!connections}
\index{connection!TCP}
Connections
for TCP are well-defined, because establishing and terminating a connection
plays a central part of the TCP protocol.  For UDP and ICMP, however, the
notion is much looser.

\index{UDP!connections@``connections''}
\label{udp-conn}
\index{connection!UDP}
\index{UDP!timeout}
For UDP, a connection begins when host $A$ sends
a packet to host $B$ for the first time, $B$ never having sent anything
to $A$.  This transmission is termed a \emph{request}, even if in fact
the application protocol being used is not based on requests and replies.
If $B$ sends a packet back, then that packet is termed a \emph{reply}.
Each packet $A$ or $B$ sends is another request or reply.
\deficiency{ There is presently no mechanism by which generic (non-RPC)
UDP connections
are terminated; Bro holds the state indefinitely.  There should probably
be a generic timeout for UDP connections that don't correspond to some
higher-level protocol (such as RPC), and a user-accessible function to
mark connections with particular timeouts. }

\index{ICMP!connections}
\index{connection!ICMP}
\index{ICMP!timeout}
For ICMP, Bro likewise creates a connection the first time it sees
an ICMP packet from $A$ to $B$, even if $B$ previously sent a packet
to $A$, because that earlier packet would have been for a different
\emph{transport} connection than the ICMP itself---the ICMP will likely
\emph{refer} to that connection, but it itself is not part of
the connection.  For simplicity, this holds even for ICMP ECHOs and
ECHO_REPLYs; if you want to pair them up, you need to do so explicitly
in the policy script.  \deficiency{ As with UDP, Bro does not time
out ICMP connections. }

\labsectsubsection{{Generic TCP connection events}}{\label{conn-events}}
\index{events!generic TCP connection}
\index{connection!events}

\index{TCP!events}
\index{TCP-specific connection events}
\index{connection events!TCP-specific}
There are a number of generic events associated with TCP connections,
all of which have a single {\tt connection} record as their argument:

\begin{description}
\index{connection!new}
\index{new connection}
\itemindevent{new_connection}
Generated whenever state for a new (TCP) connection is instantiated.

\note{Handling this event is potentially expensive.  For example,
during a SYN flooding attack, every spoofed SYN packet will lead to
a new {\tt new_connection} event.}

\index{connection!establishment}
\index{established connections}
\itemindevent{connection_established}
Generated when a connection has become established, i.e., both participating
endpoints have agreed to open the connection.

\index{connection!attempt}
\index{attempted connections}
\itemindevent{connection_attempt}
Generated when the originator (client) has unsuccessfully attempted to
establish a connection.  ``Unsuccessful'' is defined as at least
\indintvar{ATTEMPT_INTERVAL} seconds having elapsed since the client first
sent a connection establishment packet to the responder (server),
where {\tt ATTEMPT_INTERVAL} is an internal Bro variable which is
presently set to \updateme{300~seconds}.
\deficiency{This variable should be user-settable.}  If you want to
\emph{immediately} detect that a client is attempting to connect to
a server, regardless of whether it may soon succeed, then you want
to handle the {\tt new_connection} event instead.

\note{Handling this event is potentially expensive.  For example,
during a SYN flooding attack, every spoofed SYN packet will lead to
a new {\tt connection_attempt} event, albeit delayed by
{\tt ATTEMPT_INTERVAL.}} \privatenote{Perhaps generation of this event should
be spread across time rather than occurring strictly after
{\tt ATTEMPT_INTERVAL} seconds.}

\index{crud}
\index{scanning!stealth}
\index{stealth scans}
\index{connection!partial}
\index{partial connections}
\itemindevent{partial_connection}
Generated when both connection endpoints enter the {\tt TCP_PARTIAL} state
(\tbl{tbl:conn-record-states}).
This means that we have seen traffic generated by each endpoint, but
the activity did not begin with the usual connection establishment.
\deficiency{For completeness, Bro's event engine should generate another
form of {\tt partial_connection} event when a single endpoint becomes
active (see {\tt new_connection} below).  This hasn't been implemented
because our experience is network traffic often contains a great deal
of ``crud'', which would lead to a large number of these really-partial
events.  However, by not providing the event handler, we miss an opportunity
to detect certain forms of stealth scans until they begin to elicit some
form of reply.}

\index{connection!completion}
\index{connection!finished}
\index{completed connections}
\itemindevent{connection_finished}
Generated when a connection has gracefully closed.

\index{connection!rejected}
\index{rejected connections}
\itemindevent{connection_rejected}
Generated when a server rejects a connection attempt by a client.

\index{TCP Wrappers!reset vs. rejected connections}
\note{This event is only generated as the client attempts to establish
a connection.  If the server instead accepts the connection and then
later aborts it, a {\tt connection_reset} event is generated (see below).
This can happen, for example, due to use of TCP Wrappers.}

\note{Per the discussion above, a client attempting to connect to a server
will result in \emph{one} of {\tt connection_attempt},
{\tt connection_established}, or {\tt connection_rejected}; they are
mutually exclusive.}

\index{connection!completion}
\index{connection!half finished}
\index{half-finished connections}
\itemindevent{connection_half_finished}
Generated when Bro sees one endpoint of a connection attempt to gracefully
close the connection, but the other endpoint is in the {\tt TCP_INACTIVE}
state.  This can happen due to \emph{split routing} \pxref{split-routing},
in which Bro only sees one side of a connection.

\index{connection!reset}
\index{reset connections}
\privatenote{discuss RESET_DELAY?}
\itemindevent{connection_reset}
Generated when one endpoint of an established
connection terminates the connection
abruptly by sending a TCP RST packet.

\index{connection!partial close}
\index{partially closed connections}
\itemindevent{connection_partial_close}
Generated when a previously inactive endpoint attempts to close a connection
via a normal FIN handshake or an abort RST sequence.  When it sends one
of these packets, Bro waits \indintvar{PARTIAL_CLOSE_INTERVAL} (an
internal Bro variable set to 10~seconds) prior to generating the event,
to give the other endpoint a chance to close the connection normally.

\index{connection!pending}
\index{pending connections}
\itemindevent{connection_pending}
Generated for each still-open connection when Bro terminates.

\end{description}

\subsection{The \indanalyzer{tcp} analyzer}

\index{TCP!analysis}
\index{SYN control packet}
\index{FIN control packet}
\index{RST control packet}
\index{TCP control packets (SYN/FIN/RST)}
\index{control packets (SYN/FIN/RST)}
\index{packets!control (SYN/FIN/RST)}
The general {\tt tcp} analyzer lets you specify that you're interested in
generic connection analysis for TCP.  It
simply \loadx's \xrefanalyzer{conn} and adds the following
to \xrefvar{capture_filter}:
\begin{verbatim}
    tcp[13] & 0x7 != 0
\end{verbatim}
which instructs Bro to capture all TCP SYN, FIN and RST packets;
that is, the control packets that delineate the beginning (SYN)
and end (FIN) or abnormal termination (RST) of a connection.

\subsection{The \indanalyzer{udp} analyzer}

\index{UDP!analysis}
The general {\tt udp} analyzer lets you specify that you're interested in
generic connection analysis for UDP.  It
\loadx's both {\tt hot} and {\tt conn}, and defines two event handlers:

\begin{description}
\indeventtype{udp_request}{u: connection}
Invoked whenever a UDP packet is seen on the forward (request) direction of
a UDP connection.  See \xref{udp-conn} for a discussion of how Bro
defines UDP connections.

The analyzer invokes \xreffunc{check_hot} with a mode of {\tt CONN_ATTEMPTED}
and then \xreffunc{record_connection} to generate a connection summary
(necessary because Bro does not time out UDP connections, and hence
cannot generate a connection-attempt-failed event).

\indeventtype{udp_reply}{u: connection}
Invoked whenever a UDP packet is seen on the reverse (reply) direction of
a UDP connection.  See \xref{udp-conn} for a discussion of how Bro
defines UDP connections.

The analyzer invokes \xreffunc{check_hot} with a mode of {\tt CONN_ESTABLISHED}
and then again with a mode of {\tt CONN_FINISHED} to cover the general
case that the reply reflects that the connection was both established and
is now complete.  Finally, it invokes \xreffunc{record_connection} to
generate a connection summary.

\end{description}

\note{ The standard script does \emph{not} update \xrefvar{capture_filter}
to capture UDP traffic.  Unlike for TCP, where there is a natural generic
filter that captures only a subset of the traffic, the only natural UDP
filter would be simply to capture all UDP traffic, and that can often be
a huge load. }

\labsectsubsection{{Connection summaries}}{\label{conn-summaries}}
\index{connection!summaries}
\label{red-log}
The main output of \analyzer{conn} is a one-line ASCII summary
of each connection.  By tradition, these summaries are written to
a file with the name {\tt red.}\emph{tag}, where \emph{tag} uniquely
identifies the Bro session generating the logs.  (``{\tt red}'' is
mnemonic for ``reduced,'' from Bro's roots in performing protocol
analysis for Internet traffic studies.)

The summaries are produced by the \xreffunc{record_connection} function,
and have the following format:

\begin{quotation}
{\verb+<+\emph{start}\verb+>+} {\verb+<+\emph{duration}\verb+>+} {\verb+<+\emph{service}\verb+>+}
$B\sub{o}$ $B\sub{r}$ $A\sub{l}$ $A\sub{r}$
{ \verb+<+\emph{state}\verb+>+ } {\verb+<+\emph{flags}\verb+>+} {\verb+<+\emph{addl}\verb+>+}
\end{quotation}

\index{connection!start time}
\index{beginning time of a connection}
\index{start time of a connection}
\begin{description}
\item[\emph{start}]
corresponds to the connection's start time, as defined by
\xreffield{start_time}{connection}.

\index{connection!duration}
\index{duration of a connection}
\item[\emph{duration}]
gives the connection's duration, as defined by
\xreffield{duration}{connection}.

\index{connection!service}
\index{service associated with a connection}
\item[\emph{service}]
is the connection's service, as defined by
\xreffield{service}{connection}.

\index{connection!size}
\index{connection!bytes}
\index{size of connection}
\index{bytes in connection}
\item[$B\sub{o}$, $B\sub{r}$] give the number of bytes sent by the \emph{originator}
and \emph{responder}, respectively.  These correspond to the
\xreffield{size}{endpoint}
fields of the corresponding \indtt{endpoint}{record} records.

\index{connection!hosts}
\index{hosts!in a connection}
\index{connection!addresses}
\index{addresses!in a connection}
\item[$A\sub{l}$, $A\sub{r}$] correspond to the \emph{local} and \emph{remote} addresses
that participated in the connection, respectively.  The notion of which
addresses are local is controlled by the \xrefvar{local_nets}
global variable, if refined from its default value of empty.  If
{\tt local_nets} has \emph{not} been refined, then $A\sub{l}$ is the
connection \emph{responder} and $A\sub{r}$ is the connection \emph{originator}.

\note{The format and defaults for $A\sub{l}$ and $A\sub{r}$ are unintuitive; they
reflect the use of Bro's predecessor for analyzing Internet traffic
patterns, and have not been changed so as to maintain compatibility with
old, archived connection summaries.}

\begin{table}
\label{tbl:conn-file-states}
\begin{center}
\begin{tabular}{|c|l|p{3.5in}|} 
\hline
{\bf Symbol} & {\bf Name} & {\bf Meaning} \\
\hline
\hline
\verb+}+ & \indtt{S0}{connection state} &
	Connection attempt seen, no reply. \\
\hline
\verb+>+ & \indtt{S1}{connection state} &
	Connection established, not terminated. \\
\hline
\verb+>+ & \indtt{SF}{connection state} &
	Normal establishment and termination.
    Note that this is the same symbol as for state {\tt S1}.  You can tell
    the two apart because for {\tt S1} there will not be any byte counts
    in the summary, while for {\tt SF} there will be.
    \privatenote{What about for e.g. FTP, where there are partial counts
    before the connection is over?}
    \\
\hline
\verb+[+ & \indtt{REJ}{connection state} &   Connection attempt rejected. \\
\hline
\verb+}2+ & \indtt{S2}{connection state} &
	Connection established and close attempt by originator
			 seen (but no reply from responder). \\
\hline
\verb+}3+ & \indtt{S3}{connection state} &
	Connection established and close attempt by responder
			 seen (but no reply from originator). \\
\hline
\verb+>]+ & \indtt{RSTO}{connection state} &
	Connection established, originator aborted (sent a RST). \\
\hline
\verb+>[+ & \indtt{RSTR}{connection state} &
	Established, responder aborted. \\
\hline
\verb+}]+ & \indtt{RSTOS0}{connection state} &
	Originator sent a SYN followed by a RST,
		we never saw a SYN ACK from the responder. \\
\hline
\verb+<[+ & \indtt{RSTRH}{connection state} &
	Responder sent a SYN ACK followed by a RST,
		we never saw a SYN from the (purported) originator. \\
\hline
\verb+>h+ & \indtt{SH}{connection state} &
	Originator sent a SYN followed by a FIN,
		we never saw a SYN ACK from the responder (hence the
		connection was ``half" open). \\
\hline
\verb+<h+ & \indtt{SHR}{connection state} &
	Responder sent a SYN ACK followed by a FIN,
		we never saw a SYN from the originator. \\
\hline
\verb+?>?+ & \indtt{OTH}{connection state} &
	No SYN seen, just midstream traffic (a ``partial
				connection'' that was not later closed). \\
\privatenote{explain what happens for partial connections that \emph{are}
closed.}
\hline
\end{tabular}
\end{center}
\caption{
Summaries of connection states, as reported in \indtt{red}{connection summary file} files.
}
\end{table}

\index{connection!state}
\index{state of connection}
\item[\emph{state}] reflects the state of the connection at the time
the summary was written (which is usually either when the connection
terminated, or when Bro terminated).  The different states are summarized
in \tbl{tbl:conn-file-states}.  The ASCII {\bf Name} given in the Table is
what appears in the {\tt red} file; it is returned by the \xreffunc{conn_state}
function.  The {\bf Symbol} is used when generating human-readable versions
of the file---see \xrefscript{hot-report}.

For UDP connections, the analyzer reports connections for which both
endpoints have been active as {\tt SF}; those for which just the originator
was active as {\tt S0}; those for which just the responder was active
as {\tt SHR}; and those for which neither was active as {\tt OTH} (this
latter shouldn't happen!).

\index{connection!flags}
\index{flags of connection}
\item[\emph{flags}] reports a set of additional binary state associated
with the connection:

\begin{description}
\item[{\tt L}] indicates that the connection was initiated \emph{locally},
i.e., the host corresponding to $A\sub{l}$ initiated the connection.  If {\tt L}
is missing, then the host corresponding to $A\sub{r}$ initiated the connection.

\label{neighbor-marker}
\item[{\tt U}] indicates the connection involved one of the networks
listed in the \xrefvar{neighbor_nets} variable.  The use
of ``{\tt U}'' for this indication (rather than ``{\tt N}'', say) is
historical, as for the most part is the whole notion of ``neighbor network.''

Note that connection can have both {\tt L} and {\tt U} set (see next item).

\item[{\tt X}] is used to indicate that \emph{neither} the ``{\tt L}''
or ``{\tt U}'' flags is associated with this connection.  An explicit
negative indication is needed to disambiguate the \emph{flags} field
from the subsequent \emph{addl} field.

\end{description}

\index{connection!additional information}
\index{information associated with a connection}
\index{additional information associated with a connection}
\item[\emph{addl}] lists additional information associated with the
connection, i.e., as defined by \xreffield{addl}{connection}.

\end{description}

Putting all of this together, here is an example of a {\tt red} connection
summary:
\begin{verbatim}
931803523.006848 54.3776 http 7320 38891 206.132.179.35 128.32.162.134 RSTO X %103
\end{verbatim}
The connection began at timestamp 931803523.006848 (18:18:43 hours GMT
on July 12, 1999; see the \xrefutility{cf} utility for how to determine this)
and lasted 54.3776~seconds.  The service was HTTP (presuambly; this conclusion
is based just on the responder's use of port {\tt 80/tcp}).
The originator sent 7,320 bytes, and the responder sent 38,891 bytes.
Because the ``{\tt L}'' flag is absent, the connection was initiated by
host 128.32.162.134, and the responding host was 206.132.179.35.  When
the summary was written, the connection was in the ``{\tt RSTO}'' state,
i.e., after establishing the connection and transferring data, the originator 
had terminated it with a RST (this is unfortunately common for Web clients).  The connection had neither
the {\tt L} or {\tt U} flags associated with it, and there was additional
information, summarized by the string ``{\tt \%103}'' (see the
\xrefanalyzer{http} analyzer for an explanation of this information).

\labsectsubsection{{Connection functions}}{\label{conn-functions}}
\index{connection!functions}
\index{connection!functions}
We finish our discussion of generic connection analysis with a brief
summary of the different Bro functions provided by the {\tt conn} analyzer:

\begin{description}
\index{connection!size}
\index{connection!bytes}
\index{size of connection}
\index{bytes in connection}
\itemindfunc{conn_size}{(e: endpoint, is_tcp: bool): string}
returns a string giving either the number of bytes the endpoint sent
during the given connection, or {\tt "?"} if from the connection state
this can't be determined.  The {\tt is_tcp} parameter is needed
so that the function can inspect the endpoint's state to determine
whether the connection was closed.

\index{connection!state}
\index{state of connection}
\itemindfunc{conn_state}{(c: connection, is_tcp: bool): string}
returns the name associated with the connection's state, as
given in \tbl{tbl:conn-file-states}.

\index{connection!service}
\index{service associated with a connection}
\itemindfunc{determine_service}{(c: connection): bool}
sets the \xreffield{service}{connection} field of the given connection,
using \indvar{port_names}.
If you are using the \xrefanalyzer{ftp} analyzer, then it knows about FTP
data connections and maps them to {\tt port_names[20/tcp]}, i.e.,
{\tt "ftp-data"}.

\index{connection!ID}
\index{ID of connection}
\itemindfunc{full_id_string}{(c: connection): string}
returns a string identifying the connection in one of the two
following forms.  If the connection is in state {\tt S0}, {\tt S1},
or {\tt REJ}, then no data has been transferred,\footnote{\deficiency{
Actually, for state {\tt S1} data may have been transferred, and 
so this assumption should be corrected in that case.}} and the
format~is:
\begin{quote}
$A\sub{o}$~~{\verb+<+\emph{state}\verb+>+}~~{$A\sub{r}$\verb+/+\verb+<+\emph{service}\verb+>+~~\verb+<+\emph{addl}\verb+>+}
\end{quote}
where $A\sub{o}$ is the IP~address of the originator ({\tt \$id\$orig_h}),
\emph{state} is as
given in the {\bf Symbol} column of \tbl{tbl:conn-file-states}, $A\sub{r}$ is the
IP address of the responder ({\tt \$id\$resp_h}), \emph{service} gives
the application service ({\tt \$service}) as set by {\tt determine_service},
and \emph{addl} is the contents of the {\tt \$addl} field (which may be
an empty string).

\index{port!ephemeral}
\index{ephemeral port}
Note that the ephemeral port used
by the originator is not reported.  If you want to display it, use
{\tt id_string}.

So, for example:
\begin{verbatim}
    128.3.6.55 > 131.243.88.10/telnet "luser"
\end{verbatim}
identifies a connection originated by {\tt 128.3.6.55} to {\tt 131.243.88.10}'s
Telnet server, for which the additional associated information is {\tt "luser"},
the username successfully used during the authentication dialog as determined
by the \xrefanalyzer{login} analyzer.  From \tbl{tbl:conn-file-states} we see that
the connection must be in state {\tt S1}, as that's the only state of
{\tt S0}, {\tt S1}, or {\tt REJ} that has a \verb+>+ symbol.  (We can tell
it's \emph{not} in state {\tt SF} because the format used for that state
differs---see below.)

For connections in other states, Bro has size and duration information
available, and the format returned by {\tt full_id_string} is:
\begin{quote}
$A\sub{o}$~~$S\sub{o}$\verb+b+~~{\verb+<+\emph{state}\verb+>+}~~$A\sub{r}$\verb+/+\verb+<+\emph{service}\verb+>+~~$S\sub{r}$\verb+b+~~$D$\verb+s+~~\verb+<+\emph{addl}\verb+>+
\end{quote}
where $A\sub{o}$, $A\sub{r}$, \emph{state}, \emph{service}, and \emph{addl} are
as before, $S\sub{o}$ and $S\sub{r}$ give the number of bytes transmitted so far
by the originator to the responder and vice versa, and $D$ gives the
duration of the connection in seconds (reported with one decimal place)
so far.

An example of this second format is:
\begin{verbatim}
    128.3.6.55 63b > 131.243.88.10/telnet 391b 39.1s "luser"
\end{verbatim}
which reflects the same connection as before, but now {\tt 128.3.6.55} has
transmitted 63 bytes to {\tt 131.243.88.10}, which has transmitted 391 bytes
in response, and the connection has been active for 39.1 seconds.  The
``\verb+>+'' indicates that the connection is in state {\tt SF}.

\index{connection!ID}
\index{ID of connection}
\itemindfunc{id_string}{(id: conn_id): string}
returns a string identifying the connection by its address/port quadruple.
Regardless of the connection's state, the format is:
\begin{quote}
$A\sub{o}$\verb+/+$P\sub{o}$~~\verb+>+~~$A\sub{r}$\verb+/+$P\sub{r}$
\end{quote}
where $A\sub{o}$ and $A\sub{r}$ are the originator and responder addresses,
respectively, and $P\sub{o}$ and $P\sub{r}$ are representations of the originator
and responder ports as returned by the \xrefmodule{port-name} module,
i.e., either
``\verb+<+\emph{number}\verb+>/<+\emph{{\tt tcp} or {\tt udp}}\verb+>+''
or a string like ``{\tt http}'' for a well-known port such as {\tt 80/tcp}.

An example:
\begin{verbatim}
    128.3.6.55/2244 > 131.243.88.10/telnet
\end{verbatim}

Note, {\tt id_string} is implemented using a pair of calls to
\xreffunc{endpoint_id}.

\deficiency{It would be convenient to have a form of {\tt id_string}
that can incorporate a notion of directionality, for example
{\tt 128.3.6.55/2244 < 131.243.88.10/telnet} to indicate the same
connection as before, but referring specifically to the flow from
responder to originator in that connection (indicated by using ``{\tt <}''
instead of ``{\tt >}'').}

\index{hot connection!logging}
\index{connection!hot}
\index{connection!logging}
\index{logging!connection}
\itemindfunc{log_hot_conn}{(c: connection)}
logs a real-time alert of the form:
\begin{quote}
    hot: \verb+<+\emph{connection-id}\verb+>+
\end{quote}
where \emph{connection-id} is the format returned by {\tt full_id_string}.
{\tt log_hot_conn} keeps track of which connections it has logged and
will not log the same connection more than once.

\index{log file!connection summary ({\tt red})}
\index{connection!recording}
\index{recording connections}
\itemindfunc{record_connection}{(c: connection, disposition: string)}
Generates a connection summary to the \xreflog{red} file
in the format described in \xref{conn-summaries}.
If the connection's {\tt \$hot} field is positive, then also logs
the connection using {\tt log_hot_conn}.  The {\tt disposition} is a text
description of the connection's state, such as {\tt "attempt"} or
{\tt "half_finished"}; it is not presently used.

\index{connection!service}
\index{service associated with a connection}
\itemindfunc{service_name}{(c: connection): string}
returns a string describing the service associated with the connection,
computed as follows.  If the responder port ({\tt \$id\$resp_p}), $p$, is
well-known, that is, in the \xrefvar{port_names} table,
then $p$'s entry in the table is returned (such as {\tt "http"} for TCP
port~80).  Otherwise, for TCP connections, if the responder port
is less than 1024, then {\tt priv-$p$} is returned, otherwise
{\tt other-$p$}.  For UDP connections, the corresponding service
names are {\tt upriv-$p$} and {\tt uother-$p$}.

\index{connection!terminating with extreme prejudice}
\index{terminating connections forcibly}
\itemindfunc{terminate_connection}{(c: connection)}
Attempts to terminate the given connection using the {\tt rst} utility
in the current directory.  It does not check to see whether the utility
is actually present, so an unaesthetic shell error will appear if the utility
is not available.

{\tt rst} terminates connections by forging RST packets.  It is not
presently distributed with Bro, due to its potential for disruptive use.

\index{analysis!on-line}
\index{on-line analysis}
\index{analysis!off-line}
\index{off-line analysis}
If Bro is reading a trace file rather than live network traffic,
then {\tt terminate_connection} logs the {\tt rst} invocation
but does not actually invoke the utility.  In either case, it finishes
by logging that the connection is being terminated.

\end{description}

\index{analyzers!generic|)}

\labsectsection{{Site-specific information}}{\label{site-info}}

\index{analyzers!site-specific information|(}
\index{site-specific!information}
The \indanalyzer{site} analyzer is not actually an analyzer but
simply a set of global variables (and \updateme{one function}) used
to define a site's basic topological information.

\subsection{Site variables}
\index{site-specific!variables|(}

The {\tt site} module defines the following variables, all redefinable:

\index{addresses!local}
\index{local addresses}
\index{subnets}
\index{prefixes!network}
\index{network prefixes}
\begin{description}
\indvartype{local_nets}{set[net]}
Defines which {\tt net}'s Bro should consider as reflecting a local address.

Default: empty.

\index{CIDR}
\index{addresses!local}
\index{local addresses}
\index{subnets}
\index{prefixes!network}
\index{network prefixes}
\indvartype{local_16_nets}{set[net]}
Defines which /16 prefixes Bro should consider as reflecting a local address.
\deficiency{Bro currently is inconsistent regarding when it consults
{\tt local_nets} versus {\tt local_16_nets}, so you should ensure that
this variable and the previous one are always consistent.}

Default: empty.

\label{local-addresses}
\index{addresses!local}
\index{local addresses}
\indvartype{local_24_nets}{set[net]}
The same, but for /24 addresses.

Default: empty.

\label{neighbor-addresses}
\index{addresses!neighbor}
\index{neighbor addresses}
\indvartype{neighbor_nets}{set[net]}
Defines which {\tt net}'s Bro should consider as reflecting a ``neighbor.''
Neighbors networks can be treated specially in some policies, distinct
from other non-local addresses.  In particular, \xreffunc{drop_address}
will not drop connectivity to an address belonging to a neighbor.

The notion is somewhat historical, as
is the use of ``{\tt U}'' to mark neighbors in connection summaries
\pxref{neighbor-marker}.

Default: empty.

\index{addresses!neighbor}
\index{neighbor addresses}
\indvartype{neighbor_16_nets}{set[addr]}
Defines which /16 addresses Bro should consider as reflecting a neighbor;
the only use of this variable in the standard scripts is that a scan
originating from an address with one of these prefixes will not be dropped
\pxref{scan-dropping}.  \deficiency{The name is poorly chosen and should
be changed to better reflect this use.}  \deficiency{In addition, this
variable should be kept consistent with {\tt neighbor_nets}, until the
fine day when the processing is rectified to only use one variable.}

Default: empty.

\index{addresses!neighbor}
\index{neighbor addresses}
\indvartype{neighbor_24_nets}{set[net]}
The same, but for /24 addresses.

Default: empty.

\end{description}
\index{site-specific!variables|)}

\subsection{Site-specific functions}
\index{functions!site-specific}
\index{site-specific!functions|(}

Currently, the {\tt site} module only defines one function:

\begin{description}
\index{addresses!local}
\index{local addresses}
\index{site addresses}
\itemindfunc{is_local_addr}{(a: addr): bool}
returns true if the given address belongs to one of the ``local'' networks,
false otherwise.  \updateme{Currently, the test is made by masking the
address to /16 and /24 and comparing it to {\tt local_16_nets}
and {\tt local_24_nets}.}

\end{description}

\index{site-specific!functions|)}

\index{analyzers!site-specific information|)}

\section{The \analyzer{hot} Analyzer}
\index{connection!analysis}
\index{connection!hot analysis}
\index{hot connection!analysis}
\label{analyzer-hot}
The standard \indanalyzer{hot} script defines policy relating to fairly
generic notions of allowed and prohibited connections.  It defines
a number of variables that you will need to refine to customize your
site's policies.  It also provides \updateme{two} functions for checking
connections against the policies, which can be used by other of the standard
scripts.

\subsection{{\tt hot} variables}
\index{analyzers!hot!variables|(}

The standard {\tt hot} script defines the following variables, all redefinable:

\begin{description}
\index{spoofing!detection}
\index{local addresses!spoofing}
\indvartype{same_local_net_is_spoof}{bool}
If true, then a connection with a local originator address and a local
responder address is considered by \xreffunc{check_spoof}
to have been spoofed.  \deficiency{The name is poorly chosen (and may
be changed in the future) to something more accurate like
{\tt both_local_nets_is_spoof}.}

\index{DMZ!spoof detection}
\index{internal networks!spoof detection}
In general, you want to use true for a Bro that is monitoring Internet access
links (DMZs) and false for internal monitors.

Default: {\tt F}.

\index{spoofing!allowable services}
\index{local addresses!spoofing}
\indvartype{allow_spoof_services}{set[port]}
Defines a set of services (responder ports) for which Bro should not
generate alerts if it sees apparent spoofed traffic.

Default: {\tt 110/tcp} (POP version 3; \cite{rfc-pop3}).
This default was chosen because
in our experience one common form of benign spoof is an off-site laptop
attempting to read mail while still configured to use its on-site address.

\index{access!allowable address pairs}
\index{allowable address pairs}
\indvartype{allow_pairs}{set[addr, addr]}
Defines pairs of source and destination addresses for which the
source is allowed to connect to the destination.  The intent with
this variable is that the source or destination address will be a sensitive
host (such as defined with \xrefvar{hot_srcs} or
\xrefvar{hot_dsts}), for which this particular access should
be allowed.

Default: empty.

\index{access!allowable /16 network pairs}
\index{allowable /16 network pairs}
\indvartype{allow_16_net_pairs}{set[addr, addr]}
Defines pairs of source and destination /16 networks for which the
source is allowed to connect to the destination, similar to
\xrefvar{allow_pairs}.  \note{The set is
defined in terms of {\tt addr}'s and not {\tt net}'s.
So, for example, rather than specifying {\tt 128.32.}, which is a {\tt net}
constant, you'd use {\tt 128.32.0.0} (an {\tt addr} constant).  }

Default: empty.

\index{access!sensitive source addresses}
\index{sensitive source addresses}
\index{hot source addresses}
\index{addresses!hot sources}
\indvartype{hot_srcs}{table[addr] of string}
Defines source addresses that should be considered ``hot''.
A successfully established connection from such a source address
is logged, unless one of the access exception variables such as
{\tt allow_pairs} also matches the connection.  The value of the
table gives an explanatory message as to why the source is
hot; for example, {\tt "known attacker site"}.  \note{This value
is not currently used, though it aids in documenting the policy script.}

Default: empty.

Example: redefining {\tt hot_srcs} using
\begin{verbatim}
redef hot_srcs: table[addr] of string = {
    [ph33r.the.eleet.com] = "script kideez",
};
\end{verbatim}
would result in Bro alerting on any traffic coming {\tt ph33r.the.eleet.com}.
\index{kiddies!script}
\index{script kiddies}

\index{access!sensitive destination addresses}
\index{sensitive destination addresses}
\index{hot destination addresses}
\index{addresses!hot destinations}
\indvartype{hot_dsts}{table[addr] of string}
Same as {\tt hot_srcs}, except for destination addresses.

Default: empty.

\index{access!sensitive /24 source networks}
\index{sensitive /24 source networks}
\index{hot /24 source networks}
\index{networks!hot sources}
\indvartype{hot_src_24nets}{table[addr] of string}
Defines /24 source networks should be considered ``hot,''
similar to {\tt hot_srcs}.  \deficiency{Other network masks,
particularly /16, should be provided.}

Default: empty.

\index{CIA detection}
\index{Central Intelligence Agency!detection}
\index{spook detection}
Example: redefining {\tt hot_src_24nets} using
\begin{verbatim}
redef hot_src_24nets: table[addr] of string = {
    [198.81.129.0] = "CIA incoming!",
};
\end{verbatim}
would result in Bro alerting on any traffic coming from the {\tt 198.81.129/24}
network.

\index{access!sensitive /24 destination networks}
\index{sensitive /24 destination networks}
\index{hot /24 destination networks}
\index{networks!hot destinations}
\indvartype{hot_dst_24nets}{table[addr] of string}
same as {\tt hot_src_24nets}, except for destination networks.

Default: empty.

\index{access!allowable services}
\index{services!allowable}
\indvartype{allow_services}{set[port]}
Defines a set of services that are always allowed, regardless of
whether the source or destination address is ``hot.''

Default: {\tt ssh}, {\tt http}, {\tt gopher} {\tt ident}, {\tt smtp},
{\tt 20/tcp} (FTP data).

\note{The defaults are a bit unusual.  They are intended for a quite open
site with many services. }

\index{access!service allowed to a particular host}
\index{services!allowed to a particular host}
\indvartype{allow_services_to}{set[addr, port]}
Defines a set of services that are always allowed if the server is the
given host, regardless of whether the source or destination address is
``hot.''

Default: empty.

Example: redefining {\tt allow_services_to} using
\begin{verbatim}
redef allow_services_to: set[addr, port] += {
    [ns.mydomain.com, [domain, 123/tcp]],
} &redef;
\end{verbatim}
would result in Bro not alerting on any TCP DNS or NTP traffic heading
to {\tt ns.mydomain.com}.  You might add this if {\tt ns.mydomain.com}
is also in \xrefvar{hot_dsts}, because in general you want to consider
any access (other than DNS or NTP) as sensitive.

\index{access!service allowed to particular host pairs}
\index{services!allowed to particular host pairs}
\indvartype{allow_services_pairs}{set[addr, addr, port]}
Defines a set of services that are always allowed if the connection
originator is the first address and the responder (server) the second
address.

Default: empty.

Example: redefining {\tt allow_services_pairs} using
\begin{verbatim}
redef allow_services_pairs: set[addr, addr, port] += {
    [ns2.mydomain.com, ns.mydomain.com, [domain, 123/tcp]],
} &redef;
\end{verbatim}
would result in Bro not alerting on any TCP DNS or NTP traffic initiated
from {\tt ns2.mydomain.com} to {\tt ns.mydomain.com}.

\index{access!forbidden services}
\index{services!forbidden}
\indvartype{flag_successful_service}{table[port] of string}
The opposite of \xrefvar{allow_services}.
Defines a set of services that should always be flagged as sensitive,
even if neither the source nor the destination address is ``hot.''
The {\tt string} value in the table gives the reason for why
the service is considered hot.
\note{Bro currently does not use these explanatory messages.}

Default: {\tt 31337/tcp} (a popular backdoor because in stylized lettering
it spells {\tt ELEET}) and {\tt 2766/tcp} (the Solaris {\tt listen} service,
in our experience rarely used legitimately in wide-area traffic).

\index{ephemeral ports!confused with sensitive services}
\index{sensitive services!confused with ephemeral ports}
\index{FTP!ephemeral ports confused with sensitive services}
\note{Bro can flag these services erroneously when a server happens to
run a different service on the same port.  For example, if you're not
running the FTP analyzer, then Bro won't know that FTP data connections
using ephemeral ports in fact belong to legitimate FTP traffic, and will
flag any that coincide with these services.  A related problem arises 
when a user has configured their SSH access to tunnel FTP control channels
through the FTP connection, but not the corresponding data connections (so
they don't pay the expense of encrypting the data transfers), so again
Bro can't recognize that the ephemeral ports used for the data connections
does not reflect the presumed sensitive service.}

Example: redefining {\tt flag_successful_service} using
\begin{verbatim}
redef flag_successful_service: table[port] of string += {
        [1524/tcp] = "popular backdoor",
};
\end{verbatim}
would result in Bro also alerting on any successful connection to
a server running on TCP port 1524.

\index{access!forbidden inbound services}
\index{services!forbidden if inbound}
\index{inbound services!forbidden}
\indvartype{flag_successful_inbound_service}{table[port] of string}
The same as {\tt flag_\linebreak[0]successful_\linebreak[0]service}, except only applies to
connections with a remote initiator and a local responder (determined
by finding the responder address in {\tt local_nets}).

\index{etc/inetd.conf@\emph{/etc/inetd.conf}}
\index{inetd.conf@\emph{inetd.conf}}
Default: {\tt 1524/tcp} ({\tt ingreslock}, a popular backdoor because an
attacker can place an entry for the backdoor in \emph{/etc/inetd.conf} using
a service name rather than a raw port number, and hence more likely to
appear legitimate to casual inspection).  \note{There's no compelling 
reason why {\tt ingreslock} is in this table rather than the more
general {\tt flag_successful_service}, though it does tend to result
in a few more false hits than the others, presumably because it's a lower
port number, and hence more likely on some systems to be chosen for
an ephemeral port.}
\indtt{ingreslock}{popular backdoor}

\note{Symmetry would call for {\tt flag_successful_outbound_service}.
This hasn't been implemented in Bro yet simply because the
Bro development site has a threat model structured primarily around
external threats.}

\index{access!fatal inbound services}
\index{services!fatal if inbound}
\index{inbound services!fatal}
\indvartype{terminate_successful_inbound_service}{table[port] of string}
The same as {\tt flag_\linebreak[0]successful_\linebreak[0]inbound_\linebreak[0]service}, except invokes
\xreffunc{terminate_connection} in an attempt
to terminate the connection.

Default: empty.

\note{As for {\tt flag_\linebreak[0]successful_\linebreak[0]inbound_\linebreak[0]service}, it would be symmetric
to have {\tt terminate_\linebreak[0]successful_\linebreak[0]outbound_\linebreak[0]service}, and also to have
a more general {\tt terminate_\linebreak[0]successful_\linebreak[0]service}.}

\index{access!forbidden attempted services}
\index{services!forbidden if attempted}
\index{attempted services!forbidden}
\indvartype{flag_rejected_service}{table[port] of string}
Similar to {\tt flag_successful_service}, except applies to connections
that a server rejects.  For example, you could detect a particular, failed
Linux \emph{mountd} attack by adding {\tt 10752/tcp} to this table, since
that happens to be the port used by the commonly
available version of the exploit
for its backdoor if the attack succeeds.  \note{You would of course
likely also want to put {\tt 10752/tcp} in {\tt flag_successful_service};
or put the entire {\tt flag_rejected_service} table into {\tt
flag_successful_service}, as discussed in
\xref{inserting-tables-into-tables}.  }

Default: none.

\deficiency{It might make sense to have {\tt flag_attempted_service}, which
doesn't require that a server actively reject the connection, but Bro
doesn't currently have this.}

\end{description}
\index{analyzers!hot!variables|)}

\subsection{{\tt hot} functions}
\index{analyzers!hot!functions|(}

The {\tt hot} module defines two functions for external use:

\begin{description}
\index{spoofing!detection}
\index{local addresses!spoofing}
\itemindfunc{check_spoof}{(c: connection): bool}
checks the originator and responder addresses of the given connection to
determine if they are both local (and the connection is not explicitly
allowed in {\tt allow_spoof_services}).  If so, and if {\tt
same_local_net_is_spoof} is true, then marks the connection as ``hot''.

\index{Land attack}
\index{attack!Land}
The function also checks for a specific denial of service attack, the
``Land'' attack, in which the addresses are the same and so are the ports.
If so, then it generates a \xrefevent{conn_weird} event with a name of {\tt
"Land_attack"}.  It makes this check even if \xrefvar{same_local_net_is_spoof}
is false.

Returns: true if the connection is now hot (or was upon entry), false
otherwise.

\index{hot detection}
\index{detecting sensitive connections}
\index{connection!detecting sensitive}
\itemindfunc{check_hot}{(c: connection, state: count): bool}
checks the given connection against the various policy variables discussed
above, and bumps the connection's \xreffield{hot}{connection} field if it matches
the policies for being sensitive, and does not match the various exceptions.
It also uses {\tt check_spoof} to see if the connection reflects a possible
spoofing attack; and terminates the connection if
{\tt terminate_successful_service} indicates~so.

\begin{table}
\label{tbl:check-hot-states}
\begin{center}
\begin{tabular}{|l|p{2.2in}|p{2.2in}|} 
\hline
{\bf State} & {\bf Meaning} & {\bf Tests} \\
\hline
\hline
\verb+CONN_ATTEMPTED+
		& Connection attempted, no reply seen.  Note that
			you should also use this value for scans with
			indeterminant state, such as possible stealth scans.
			For example,
			\xrefevent{connection_half_finished}
			does this.
			\index{scanning!stealth}
			\index{stealth scans}
		& {\tt check_spoof}. \\
\hline
\verb+CONN_ESTABLISHED+
		& Connection established.  Also used for connections
			apparently established, per \xrefevent{partial_connection}.
		& {\tt check_spoof}, {\tt flag_successful_service},
		{\tt flag_successful_inbound_\linebreak[0]service},
		{\tt allow_services_to},
		{\tt terminate_successful_\linebreak[0]inbound_\linebreak[0]service}.
		\\
\hline
\verb+APPL_ESTABLISHED+
		& The connection has reached application-layer
			establishment.  For example, for Telnet or Rlogin,
			this is after the user has authenticated.
		& {\tt allow_services_to},
		  {\tt allow_service_pairs},
		  {\tt allow_pairs},
		  {\tt allow_16_net_pairs},
		  {\tt hot_srcs},
		  {\tt hot_dsts},
		  {\tt hot_src_24nets},
		  {\tt hot_\linebreak[0]dst_\linebreak[0]24nets}.
		\\
\hline
\verb+CONN_FINISHED+
		& The connection has finished, either cleanly or
		  abnormally (for example,
		  \xrefevent{connection_reset}).
		& Same as {\tt APPL_ESTABLISHED}, if the connection
		  exchanged non-zero amounts of data in both directions,
		  and if the service wasn't one of the ones that generates
		  {\tt APPL_ESTABLISHED}. \\
\hline
\verb+CONN_REJECTED+
		& The connection attempt was rejected by the server.
		& {\tt check_spoof}, {\tt flag_rejected_service}. \\
\hline
\end{tabular}
\end{center}
\caption{
Different connection states to use when calling {\tt check_hot}.
}
\end{table}

The caller indicates the connection's state in the second parameter to the
function, using one of the values given in Table~\ref{tbl:check-hot-states}.
As noted in the Table, the processing differs depending on the state.

In general, the pattern is to make one call when the connection is first
seen, either {\tt CONN_ATTEMPTED}, {\tt CONN_ESTABLISHED},
or {\tt CONN_REJECTED}.  If the application is one for which connections
should only be considered ``established'' after a successful pre-exchange
between originator and responder, then a subsequent call is made
with a state of {\tt APPL_ESTABLISHED}.   The idea here is to provide a
way to filter out what are in fact not really successful connections
so that they are not analyzed in terms of successful service.
Finally, for services that don't use {\tt APPL_ESTABLISHED}, a
call is made instead when the connection finishes for some reason,
using state {\tt CONN_FINISHED}.  \note{ This approach delays
alerting until the connection is over, which might be later than
you want, in which case you may need to edit {\tt check_hot} to 
provide the desired functionality.}

Returns: true if the connection is now hot (or was upon entry), false
otherwise.

\end{description}

\index{analyzers!hot!functions|)}

\labsectsection{{The \analyzer{scan} Analyzer}}{\label{analyzer-scan}}
\index{scan detection|(}
\index{detecting scans}
\index{scanning!address}
\index{scanning!port}
\index{address scanning}
\index{port scanning}
\index{passwords!guessing}
The \indanalyzer{scan} analyzer detects connection attempts to numerous machines
(address scanning), connection attempts to many different services
on the same machine (port scanning), and attempts to access many different
accounts (password guessing).  The basic methodology is to use tables to
keep track of the distinct addresses and ports to which a given host
attempts to connect, and to trigger alerts when either of these reaches
a specified size.  \deficiency{As currently written, the analyzer will not
detect distributed scans, i.e., when many sites are used to probe individually
just a few, but together a large number, of ports or addresses.}

A powerful technique that Bro potentially provides is dropping
border connectivity with remote scanning sites, though you must
supply the magic script to talk with your router and effect the
block.  See \xreffunc{drop_address} below for a discussion of the
interface provided.  \note{Naturally, providing this capability means
you might become vulnerable to denial-of-service attacks in which spoofed
packets are used in an attempt to trigger a block of a site to which
you want to have access.}

\subsection{{\tt scan} variables}
\index{analyzers!scan!variables|(}

In addition to internal variables for its bookkeeping, the analyzer
provides the following redefinable variables:

\begin{description}
\indvartype{report_peer_scan}{set[count]}
Generate a log message whenever a remote host (as determined by
\xreffunc{is_local_addr}) has attempted to connect to the given
number of distinct hosts.

Default: {\tt \{ 100, 1000, 10000, \}}.  So, for example, if
a remote host attempts to connect to 3,500 different local hosts,
a report will be generated when it makes the 100th attempt, and
another when it makes the 1,000th attempt.

\indvartype{report_outbound_peer_scan}{set[count]}
The same as {\tt report_peer_scan}, except for connections
initiated locally.

Default: {\tt \{ 1000, 10000, \}}.

\indvartype{possible_port_scan_thresh}{count}
Initially, port scan detection is done based on how many different
ports a given host connects to, regardless of on which hosts.  Once
this threshold is reached, however, then the analyzer begins tracking
ports accessed per-server, which is important for reducing false
positives.  \note{The reason this variable exists  is because it
is very expensive to track per-server ports accessed for every
active host; this variable limits such tracking to only active hosts
contacting a significant number of different ports.}

Default: {\tt 25}.

\indvartype{report_accounts_tried}{set[count]}
Whenever a remote host has attempted to access a number of local
accounts present in this set, generate a log message.  Each distinct
username/password pair is considered a different access.

Default: {\tt \{ 25, 100, 500, \}}.

\indvartype{report_remote_accounts_tried}{set[count]}
The same, except for access to remote accounts rather than local ones.

Default: {\tt \{ 100, 500, \}}.

\indvartype{skip_accounts_tried}{set[addr]}
Do not do bookkeeping for account attempts for the given hosts.

Default: empty.

\indvartype{skip_outbound_services}{set[port]}
Do not do outbound-scanning bookkeeping for connections involving
the given services.

Default: \xrefvar{allow_services}, {\tt ftp}, {\tt addl_web} (see next item).

\indvartype{addl_web}{set[port]}
Additional ports that should be considered as Web traffic (and hence skipped
for outbound-scan bookkeeping).

Default: {\tt \{ 81/tcp, 443/tcp, 8000/tcp, 8001/tcp, 8080/tcp, \}}.

\indvartype{skip_scan_sources}{set[addr]}
Hosts that are allowed to address-scan without complaint.

Default: {\tt scooter.pa-x.dec.com}, {\tt scooter2.av.pa-x.dec.com}
(AltaVista crawlers; you get the idea.)

\indvartype{skip_scan_nets_24}{set[addr, port]}
/24 networks that are allowed to address scan for the given port
without complaint.

Default: empty.

\index{connectivity!dropping}
\index{dropping connectivity}
\index{firewall!reactive}
\index{reactive firewall}
\indvartype{can_drop_connectivity}{bool}
True if the Bro has the capability of dropping connectivity,
per \xreffunc{drop_address}.

Default: false.

\index{scanning!shutting down}
\index{shutting down scans}
\indvartype{shut_down_scans}{set[port]}
Scans of these ports trigger connectivity-dropping (if the Bro
is capable of dropping connectivity), unless {\tt shut_down_all_scans}
is defined (next item).

Default: empty.

\indvartype{shut_down_all_scans}{bool}
Ignore {\tt shut_down_scans} and simply drop all scans regardless of
service.

Default: false.

\indvartype{shut_down_thresh}{count}
Shut down connectivity after a host has scanned this many addresses.`

Default: {\tt 100}.

\indvartype{never_shut_down}{set[addr]}
Purported scans from these addresses are never shut down.

Default: the root name servers ({\tt a.root-servers.net} through
{\tt m.root-servers.net}).

\end{description}

\index{analyzers!scan!variables|)}

\subsection{{\tt scan} functions}
\index{analyzers!scan!functions|(}
The standard {\tt scan} script provides the following functions:

\begin{description}
\index{connectivity!dropping}
\index{dropping connectivity}
\index{firewall!reactive}
\index{reactive firewall}
\index{scanning!shutting down}
\index{shutting down scans}
\itemindfunc{drop_address}{(a: addr, msg: string)}
Drops external connectivity to the given address and logs a notification
using the given message.

\index{drop-connectivity shell script@\emph{drop-connectivity} shell script}
\index{shell scripts!drop-connectivity@\emph{drop-connectivity}}
Dropping connectivity requires all of the following to be true:
\begin{itemize}
	\item \xrefvar{can_drop_connectivity} is true.
	\item The address is neither local
	      nor a neighbor \pxref{neighbor-addresses}.
	\item The address is not in \xrefvar{never_shut_down}.
\end{itemize}
If these checks succeed, then the script simply attempts to invoke
a shell script \emph{drop-connectivity} with a single argument, the IP
address to block.  It is up to you to provide the script, using whatever
interface to your router/firewall you have available.

The function does not return a value.

\index{scanning!stealth}
\index{stealth scans}
\itemindfunc{check_scan}{(c: connection, established: bool, reverse: bool): bool}
Updates the analyzer's internal bookkeeping on the basis of the
new connection {\tt c}.  If {\tt established} is true, then the connection
was successfully established, otherwise not.  If {\tt reverse} is true,
then the function should consider the originator/responder fields in
the connection's record as reversed.  \note{This last is needed
for some unusual new connections that may reflect stealth scanning.
For example, when the event engine sees a SYN-ack without a corresponding
SYN, it instantiates a new connection with an assumption that the SYN-ack
came from the responder (and it missed the initial SYN either due to
split routing \pxref{split-routing}, a packet drop \pxref{packet-drops},
or Bro having started running after the initial SYN was sent).}

If the originating host's activity matches the policy defined by the
variables above, then the analyzer logs this fact, and possibly
attempts to drop connectivity to the originating host.  The function
also schedules an event for 24~hours in the future (or when Bro terminates)
to generate a summary of the scanning activity (so if the host continues
scanning, you get a report on how many hosts it wound up scanning).
\deficiency{This time interval should be selectable.}

\note{Purported scans of the FTP data port ({\tt 20/tcp}) or the {\tt ident}
service ({\tt 113/tcp}) are never reported or dropped, as experience
has shown they yield too many false hits.}

The function does not return a value.

\end{description}

\index{analyzers!scan!functions|)}

\subsection{{\tt scan} event handlers}
\index{analyzers!scan!event handlers|(}

The standard {\tt scan} script defines one event handler:

\begin{description}
\indeventtype{account_tried}{c: connection, user: string, passwd: string}
The given connection made an attempt to access the given username and
password.  Each distinct username/password pair is considered a new access.
The event handler generates a log message if the access matches the logging
policy outlined above.
\label{account-tried}

\note{{\tt account_tried} events are generated by \xrefanalyzer{login}
and \xrefanalyzer{ftp} analyzers.}

\end{description}

\index{analyzers!scan!event handlers|)}

\index{scan detection|)}

\labsectsection{{The \module{port-name} Module}}{\label{module-port-name}}

The \indmodule{port-name} utility module provides one redefinable variable
and one callable function:

\begin{description}
\indvartype{port_names}{table[port] of string}
Maps TCP/UDP ports to names for the services associated with those ports.
For example, {\tt 80/tcp} maps to {\tt "http"}.  These names are used by
the \xrefanalyzer{conn} analyzer when generating connection logs
\pxref{analy-conn}.

\itemindfunc{endpoint_id}{(h: addr, p: port): string }
Returns a printable form of the given address/port connection
endpoint.  The format is either 
    \verb+<+\emph{address}\verb+>/<+\emph{service-name}\verb+>+
or
    \verb+<+\emph{address}\verb+>/<+\emph{port-number}\verb+>+
depending on whether the port appears in {\tt port_names}.
\end{description}

\labsectsection{{The \module{mt} Module}}{\label{module-mt}}
The \indmodule{mt} module is intended to provide a convenient way
to run (almost) all of the analyzers.  It \loadx's the \updateme{following}
other modules and analyzers:
\xrefmodule{log},
\xrefmodule{dns},
\xrefmodule{hot},
\xrefmodule{port-name},
\xrefmodule{frag},
\xrefmodule{tcp},
\xrefmodule{scan},
\xrefmodule{weird},
\xrefmodule{finger},
\xrefmodule{ident},
\xrefmodule{ftp},
\xrefmodule{login} and
\xrefmodule{portmapper}.
So you can run Bro using \emph{bro -i in0 mt} to have it analyze
traffic on interface \emph{in0} using the above analyzers
\pxref{network-interfaces}; or you can \load {\tt mt} to load in the above
analyzers.

\note{ The {\tt mt} analyzer doesn't load \xrefmodule{http} (because
it can prove a very high load for many sites)
nor experimental analyzers such as \xrefmodule{stepping}
or \xrefmodule{backdoor}. }

\labsectsection{{The \module{log} Module}}{\label{module-log}}

The \indmodule{log} utility module redefines a single variable:

\begin{description}
\index{log file}
\indvartype{bro_log_file}{file}
A special Bro variable used internally to specify a file where Bro should
record messages logged by \xrefstmt{log} statements (as well
as generating real-time alerts via \emph{syslog}).

Default: if the \indenv{BRO_ID} environment variable is defined,
then {\tt log.\verb+<+\emph{\$BRO_ID}\verb+>+}, otherwise {\tt bro.log}.

\note{ This value is slightly different than that returned by
\xreffunc{open_log_file}, because the latter would return
{\tt log} if {\tt \$BRO_ID} wasn't defined, and that name seems
too easy to confuse with other uses. }

See \xref{bro-log-file-global} for further discussion.

\end{description}

If you do not include this module, then Bro records log messages
to \emph{stderr}.
\index{stderr@\emph{stderr}}

\begin{figure}
\label{fig:log-hook}
\begin{verbatim}
global msg_count: table[string] of count &default = 0;

event log_summary(msg: string)
    {
    log fmt("(%s) %d times", msg, msg_count[msg]);
    }

function log_hook(msg: string): bool
    {
    if ( ++msg_count[msg] == 1 )
        # First time we've seen this message - log it.
        return T;

    if ( msg_count[msg] == 5 )
        # We've seen it five times, enough to be worth
        # summarizing.  Do so five minutes from now,
        # for whatever total we've seen by then.
        schedule +5 min { log_summary(msg) };

    return F;
    }

\end{verbatim}
\caption{
Sample definition of \indfuncnolabel{log_hook}.  
}
\end{figure}

You can also control Bro's log processing by defining the
special function \indfunc{log_hook}.  It takes a single
argument, {\tt msg: string}, the message in a just-executed
{\tt log} statement, and returns a boolean value: true if Bro
should indeed log the message, false if not.  For example,
\f{fig:log-hook} shows a definition of {\tt log_hook} that
checks each log message to see whether the same text has
been logged before.  It only logs the first instance of a message.
If a message appears at least five times, then it schedules a
future {\tt log_summary} event for 5~minutes in the future;
the purpose of this event is to summarize the total number of
times the message has appeared at that point in time.

\labsectsection{{The \module{active} Module}}{\label{module-active}}

The \indmodule{active} utility module provides a single, non-redefinable
variable that holds information about active connections: 

\begin{description}
\indvartype{active_conn}{table[conn_id] of connection}
Indexed by a {\tt conn_id} (\f{fig:conn-record}) giving the
originator/responder addresses/ports, returns the connection's
{\tt connection} record.  As usual, accessing
the table with a non-existing index results in a run-time error,
so you should first test for the presence of the index using
the {\tt in} operator.

Default: empty.
\end{description}

This functionality is quite similar to that of the \xreffunc{active_connection}
function, and \deficiency{arguably this module should be removed in favor
of the function}.  It does, however, provide a useful example of maintaining
bookkeeping by defining additional handlers for events that already have
handlers elsewhere.

\labsectsection{{The \module{demux} Module}}{\label{module-demux}}

The \indmodule{demux} utility module provides a single function:

\begin{description}
\itemindfunc{demux_conn}{(id: conn_id, tag: string, otag: string, rtag: string): bool }
Instructs Bro to write (``demultiplex'') the contents of the connection
with the given {\tt id} to a pair of files whose names are constructed
out of {\tt tag}, {\tt otag}, and {\tt rtag}, as follows.

The
originator-to-responder direction of the connection goes into a file named:
\begin{quotation}
\verb+<+\emph{otag}\verb+>.<+
\emph{tag}\verb+>.<+
\emph{orig-addr}\verb+>.<+
\emph{orig-port}\verb+>-<+
\emph{resp-addr}\verb+>.<+
\emph{resp-port}\verb+>+
\end{quotation}
and the other direction in:
\begin{quotation}
\verb+<+\emph{rtag}\verb+>.<+
\emph{tag}\verb+>.<+
\emph{resp-addr}\verb+>.<+
\emph{resp-port}\verb+>-<+
\emph{orig-addr}\verb+>.<+
\emph{orig-port}\verb+>+
\end{quotation}
Accordingly, \emph{tag} can be used to associate a unique label with
the pair of files, while \emph{otag} and \emph{rtag} provide distinct
labels for the two directions.

If Bro is already demuxing the connection, or if the connection is
not active, then nothing happens, and the function returns false.
Otherwise, it returns true.

\end{description}

Bro places demuxed streams in a directory defined by the redefinable
global {\tt demux_dir}, which defaults in the usual fashion to
{\tt open_log_file("xscript")}.

\deficiency{Experience has shown that it would be highly convenient if
Bro would demultiplex the \emph{entire} connection contents into the
files, instead of just the part of the connection seen subsequently
after the call
to {\tt demux_conn}.  One way to do this would be for {\tt demux_conn} to
offset the contents in the file by the current stream position, and then
to invoke a utility tool that goes through the Bro output trace file
\pxref{write-file} and copies the contents up to the current stream position
to the front of the file.  This utility tool might even be another instance
of Bro running with suitable arguments.}

\labsectsection{{The \module{dns} Module}}{\label{module-dns}}
The \indmodule{dns} module deals with Bro's internal mapping of hostnames
to/from IP addresses.
\deficiency{ There is no DNS protocol analyzer available at present. }
Furthermore, \deficiency{ the lookup mechanisms discussed here are not
available to the Bro script writer, other than implicitly by using hostnames
in lieu of addresses in variable initializations
\pxref{hostnames-vs-addresses}.}

\index{bro-dns-cache@{\tt .bro-dns-cache}}
\label{dns-cache}
\index{DNS!Bro's private cache}
The module's function is to handle different events that can
occur when Bro resolves hostnames upon startup.  Bro maintains its
own cache of DNS information which persists across invocations of Bro
on the same machine and by the same user.
The role of the cache is to allow Bro to resolve
hostnames even in the face of DNS outages; the philosophy is that it's
better to use old addresses than none at all, and this helps harden Bro
against attacks in which the attacker causes DNS outages in order to
prevent Bro from resolving particular sensitive hostnames (e.g.,
\xrefvar{hot_srcs}).  The cache is stored in the file ``{\tt .bro-dns-cache}''
in the user's home directory.  You can delete this file whenever you want,
for example to purge out old entries no longer needed, and Bro will recreate
it next time it's invoked using \indflagnoitem{P}.

Currently, all of the event handlers are invoked upon \emph{comparing}
the results of a new attempt to look up a name or an address versus the
results obtained the \emph{last time} Bro did the lookup.  When Bro looks
up a name for the first time, no events are generated.

Also, Bro currently only looks up hostnames to map them to addresses.
It does not perform inverse lookups.

\subsection{The \indtt{dns_mapping}{record} record}
\index{DNS!mappings}

\begin{figure}
\label{fig:dns-mapping}
\begin{verbatim}
type dns_mapping: record {
    creation_time: time;  # When the mapping was created.

    req_host: string;     # The hostname in the request, if any.
    req_addr: addr;       # The address in the request, if any.

    valid: bool;          # Whether we received an answer.
    hostname: string;     # The hostname in the answer, or "<none>".
    addrs: set[addr];     # The addresses in the answer, if any.
};
\end{verbatim}
\caption{
Definition of the \indtt{dns_mapping}{record} record. }
\end{figure}

All of the events handled by the module include at least one
record of DNS mapping information, defined by the {\tt dns_mapping}
type shown in \f{fig:dns-mapping}.  The corresponding fields are:

\begin{description}
\indfield{creation_time}{dns_mapping}
When the mapping was created.

\indfield{req_host}{dns_mapping}
The hostname looked up, or an empty string if this was not a hostname
lookup.

\indfield{req_addr}{dns_mapping}
The address looked up (reverse lookup), or {\tt 0.0.0.0} if this was not
an address lookup.

\indfield{valid}{dns_mapping}
True if an answer was received for a lookup (even if the answer was that
the request name or address does not exist in the DNS).

\indfield{hostname}{dns_mapping}
The hostname answer in response to an address lookup, or the
string {\tt "\verb+<+none\verb+>"+} if an answer was received but
it indicated there was no PTR record for the given address.

\indfield{addrs}{dns_mapping}
A set of addresses in response to a hostname lookup.  Empty
if an answer was received but it indicated that there was no A record
for the given hostname.

\end{description}

\subsection{{\tt dns} variables}
\index{modules!dns!variables|(}

The modules provides one redefinable variable:

\begin{description}
\indvartype{dns_interesting_changes}{set[string]}
The different DNS events have names associated with them.  If the
name is present in this set, then the event will be logged, otherwise
not.

One exception to this list is that DNS changes involving the
loopback address {\tt 127.0.0.1} are always considered log-worthy,
since they may reflect DNS corruption.

Default: {\tt \{ "unverified", "old name", "new name", "mapping", \}}.

\end{description}

\index{modules!dns!variables|)}

\subsection{{\tt dns} event handlers}
\index{modules!dns!event handlers|(}

The DNS module supplies the following event handlers:

\begin{description}
\indeventtype{dns_mapping_valid}{dm: dns_mapping}
The given request was looked up and it was identical to its previous
mapping.

\indeventtype{dns_mapping_unverified}{dm: dns_mapping}
The given request was looked up but no answer came back.

\indeventtype{dns_mapping_new_name}{dm: dns_mapping}
In the past, the given address did not resolve to a hostname;
this time, it did.

\indeventtype{dns_mapping_lost_name}{dm: dns_mapping}
In the past, the given address resolved to a hostname; now,
that name has gone away.  (An answer was received, but it stated
that there is no hostname corresponding to the given address.)

\indeventtype{dns_mapping_name_changed}{old_dm: dns_mapping, new_dm: dns_mapping}
The name returned this time for the given address differs from the
name returned in the past.

\indeventtype{dns_mapping_altered}{dm: dns_mapping, old_addrs: set[addr], new_addrs: set[addr]}
The addresses associated with the given hostname have changed.  Those
in {\tt old_addrs} used to be part of the set returned for the name, but
aren't any more; while those in {\tt new_addrs} didn't used to be, but
now are.  There may also be some unchanged addresses, which are those in
{\tt dm\$addrs} but not in {\tt new_addrs}.

\end{description}

\index{modules!dns!event handlers|)}


\labsectsection{{The \analyzer{finger} Analyzer}}{\label{analyzer-finger}}
\index{analyzers!application-specific|(}
\index{Finger!analysis}
The \indanalyzer{finger} analyzer processes traffic associated with
the Finger service \cite{rfc-finger}.  Bro instantiates a {\tt finger}
analyzer for any connection with service port {\tt 79/tcp} (if you
\load the finger analyzer in your script, or define your own
{\tt finger_request} or {\tt finger_reply} handlers, of course).

The analyzer uses a capture filter of ``{\tt port finger}''
\pxref{capture-filter}.

In the past, attackers often used Finger requests to obtain information
about a site's users, and sometimes to launch attacks of various forms
(buffer overflows, in particular).  In our experience, exploitation
of the service has
greatly diminished over the past years (no doubt in part to the service
being increasingly turned off, or prohibited by firewalls).  Now it is only
rarely associated with an attack.

\subsection{{\tt finger} variables}
\index{analyzers!finger!variables|(}

The standard script defines two redefinable variables:

\begin{description}
\indvartype{hot_names}{set[string]}
A list of usernames that should be considered sensitive (log-worthy)
if included in a Finger request.

Default: {\tt \{ "root", "lp", "uucp", "nuucp", "demos", "operator", "sync",
"guest", "visitor", \}}.

\indvartype{max_request_length}{count}
The largest reasonable request size (used to flag possible buffer
overflow attacks).  Bro marks a connection as ``hot'' if its request
exceeds this length, and truncates its logging of the request to
this many bytes, followed by {\tt "..."}.

Default: {\tt 80}.

\end{description}

\index{analyzers!finger!variables|)}

\subsection{{\tt finger} event handlers}
\index{analyzers!finger!event handlers|(}

The standard script defines one event handler:

\begin{description}
\indeventtype{finger_request}{c: connection, request: string, full: bool}
Invoked upon connection {\tt c} having made the request {\tt request}.
The {\tt full} flag is true if the request included the ``long format''
option (which the event engine will have removed from the request).

The standard script flags long requests and truncates them as noted above,
and then checks whether the request is for a name in {\tt hot_names}.
It then formats the request either by placing double quotation marks
around it, or, if the request was empty---indicating a request for
information on all users---the request is changed to the string {\tt ALL}
with no quotes around it.

If the originator already made a request, then this additional request
is placed in parentheses (though multiple requests violate the Finger
protocol).  If the request was for the {\tt full} format, then the
text ``{\tt (/W)}'' is appended to the request.  Finally, the request
is appended to the connection's \xreffield{addl}{connection} field.

\end{description}

The event engine generates an additional event that the predefined
{\tt finger} script does not handle:

\begin{description}
\indeventtype{finger_reply}{c: connection, reply_line: string}
Generated for each line of text sent in response to the originator's
request.

\end{description}

\index{analyzers!finger!event handlers|)}

\labsectsection{{The \module{frag} Module}}{\label{module-frag}}

The \indmodule{frag} utility module simply refines the capture filter
\pxref{capture-filter} so that Bro will capture and reassemble IP fragments.
Bro reassembles any fragments it receives; but normally it doesn't receive
any, except the beginnings of TCP fragments (see the \xrefmodule{tcp}
module), and UDP port 111 (per the \xrefmodule{portmapper} module).

\index{fragment reassembly}
\index{fragments!TCP vs. UDP}
\index{TCP!fragments}
\index{UDP!fragments}
So, to make Bro do fragment reassembly, you simply use ``\load {\tt frag}''.
It effects this by adding:
\begin{verbatim}
    (ip[6:2] & 0x3fff != 0) and tcp
\end{verbatim}
to the filter.  The first part of this expression matches all IP fragments,
while the second restricts those matched to TCP traffic.  We would \emph{like}
to use:
\begin{verbatim}
    (ip[6:2] & 0x3fff != 0) and (tcp or udp port 111)
\end{verbatim}
to also include portmapper fragments, but that won't work---the port
numbers will only be present in the first fragment, so the packet filter
won't recognize the subsequent fragments as belonging to a UDP port~111
packet, and will fail to capture them.

\index{NFS traffic!high volume fragments}
\note{ Alternatively, we might be tempted to use ``{\tt (tcp or udp)}''
and so capture \emph{all} UDP fragments, including port~111.  This would
work in principle, but in practice can capture very high volumes of
traffic due to NFS traffic, which can send all of its file data in
UDP fragments. }


\labsectsection{{The \module{hot-ids} Module}}{\label{module-hot-ids}}
\index{usernames!sensitive}
\index{sensitive usernames}
\index{hot usernames}

The \indmodule{hot-ids} module defines a number of redefinable variables
that specify usernames Bro should consider sensitive:

\begin{description}
\indvartype{forbidden_ids}{set[string]}
lists usernames that should never be used.  If Bro detects use of one,
it will attempt to terminate the corresponding connection.

\indttnotext{uucp}{username}
\indttnotext{daemon}{username}
\indttnotext{rewt}{username}
\indttnotext{nuucp}{username}
\indttnotext{EZsetup}{username}
\indttnotext{OutOfBox}{username}
\indttnotext{4Dgifts}{username}
\indttnotext{sgiweb}{username}
Default: {\tt \{ "uucp", "daemon", "rewt", "nuucp", "EZsetup", "OutOfBox", "4Dgifts", "ezsetup", "outofbox", "4dgifts", "sgiweb", \}}.  All of these
correspond to accounts that some systems have enabled by default
(with well-known passwords), except for {\tt "rewt"}, which corresponds
to a username often used by (weenie) attackers.
\index{attackers!weenie}

\deficiency{ The repeated definitions such as {\tt "EZsetup"} and
{\tt "ezsetup"} reflect that this variable is a {\tt set} and not
a {\tt pattern}.  Consequently, the exact username must appear in
it (with a pattern, we could use character classes to match both
upper and lower case).  \privatenote{The best way to fix this would be to add
a {\emph{case-insensitive} attribute}.}}

\indvartype{forbidden_ids_if_no_password}{set[string]}
Same as {\tt forbidden_ids} except only considered forbidden if
the login succeeded with an empty password.

Default: {\tt "lp"}, a default passwordless IRIX account.

\indvartype{forbidden_id_patterns}{pattern}
A pattern giving user ids that should be considered forbidden.
\deficiency{ This pattern is currently only used to check
Telnet/Rlogin user ids, not ids seen in other contexts, such
as FTP sessions.}

Default: \verb+/(y[o0]u)(r|ar[e3])([o0]wn.*)/+, a particularly
egregious style of username of which we've observed variants
in different break-ins.

\indvartype{always_hot_ids}{set[string]}
\label{always-hot-ids}
A list of usernames that should always be considered sensitive,
though not necessarily so sensitive that they should be terminated
whenever used.

Default: {\tt \{ "lp", "warez", "demos", forbidden_ids, \}}.  The
{\tt "lp"} and {\tt "demos"} accounts are specified here rather
than {\tt forbidden_ids} because it's possible that they might be
used for legitimate accounts.  {\tt "warez"} (for ``wares'', i.e.,
bootlegged software) is listed because its use likely constitutes
a policy violation, not a security violation.

\note{ {\tt forbidden_ids} is incorporated into {\tt always_hot_ids}
to avoid replicating the list of particularly sensitive ids by listing
it twice and risking inconsistencies.}

\indvartype{hot_ids}{set[string]}
User ids that generate alerts if the user logs in successfully.

Default: {\tt \{ "root", "system", always_hot_ids, \}}.  The
ones included in addition to {\tt always_hot_ids} are only considered
sensitive if the user logs in successfully.

\end{description}


\labsectsection{{The \analyzer{ftp} Analyzer}}{\label{analyzer-ftp}}
\index{FTP!analysis}
The \indanalyzer{ftp} analyzer processes traffic associated with
the FTP file transfer service \cite{rfc-ftp}.  Bro instantiates an
{\tt ftp} analyzer for any connection with service port {\tt 21/tcp},
providing you have loaded the {\tt ftp} analyzer, or defined a handler
for {\tt ftp_request} or {\tt ftp_reply}.

The analyzer uses a capture filter of ``{\tt port ftp}'' \pxref{capture-filter}.
It generates summaries of FTP sessions;
looks for sensitive usernames, access to sensitive files, and possible
FTP ``bounce'' attacks, in which the host specified in a ``{\tt PORT}'' or
``{\tt PASV}'' directive does not correspond to the host sending
the directive; or in which a different host than the server (client) connects
to the endpoint specified in a {\tt PORT} ({\tt PASV}) directive.

\subsection{The \indtt{ftp_session_info}{record} record}
\index{FTP!session information}

\begin{figure}
\label{fig:ftp-session-info}
\begin{verbatim}
type ftp_session_info: record {
    id: count;              # unique number associated w/ session
    user: string;           # username, if determined
    request: string;        # pending request or requests
    num_requests: count;    # count of pending requests
    request_t: time;        # time of request
    log_if_not_denied: bool;        # unless code 530 on reply, log it
    log_if_not_unavail: bool;       # unless code 550 on reply, log it
    log_it: bool;           # if true, log the request(s)
};
\end{verbatim}
\caption[Definition of the {\tt ftp_session_info} record]{
Definition of the \indtt{ftp_session_info}{record} record. }
\end{figure}

The main data structure managed by the {\tt ftp} analyzer is
a collection of {\tt ftp_session_info} records, where the
record type is shown in \f{fig:ftp-session-info}.
The corresponding fields are:

\begin{description}
\indfield{id}{ftp_session_info}
The unique session identifier assigned to this session.  Sessions
are numbered starting at~{\tt 1} and incrementing with each new session.

\indfield{user}{ftp_session_info}
The username associated with this session (from the initial FTP
authentication dialog), or an empty string if not yet determined.

\indfield{request}{ftp_session_info}
The pending request, if the client has issued any.  Ordinarily there
would be at most one pending request, but a client can in fact send
multiple requests to the server all at once,
and an attacker could do so attempting
to confuse the analyzer into mismatching responses with requests,
or simply forgetting about previous requests.

\indfield{num_requests}{ftp_session_info}
A count of how many requests are currently pending.

\indfield{request_t}{ftp_session_info}
The time at which the pending request was issued.

\indfield{log_if_not_denied}{ftp_session_info}
If true, then when the reply to the current request comes in,
Bro should log it, unless the reply code is {\tt 530} (``{\tt denied}'').

\indfield{log_if_not_unavail}{ftp_session_info}
If true, then when the reply to the current request comes in,
Bro should log it, unless the reply code is {\tt 550} (``{\tt unavail}'').

\indfield{log_it}{ftp_session_info}
If true, then when the reply to the current request comes in,
Bro should log it.

\end{description}

\subsection{{\tt ftp} variables}
\index{analyzers!ftp!variables|(}

The standard script defines the following redefinable variables:

\begin{description}
\indvartype{ftp_guest_ids}{set[string]}
A set of usernames associated with publicly accessible ``guest''
services.  Bro interprets guest usernames as indicating Bro should
use the authentication \emph{password} as the effective username.

Default: {\tt \{ "anonymous", "ftp", "guest", \}}.

\indvartype{ftp_skip_hot}{set[addr, addr, string]}
Entries indicate that a connection from the first given address to the
second given address, using the given string username, should not
be treated as hot even if the username is sensitive.

Default: empty.

Example: redefining {\tt ftp_skip_hot} using
\begin{verbatim}
redef ftp_skip_hot: set[addr, addr, string] += {
    [[bob1.dsl.home.net, bob2.dsl.home.net], bob.work.com, "root"],
};
\end{verbatim}
would result in Bro not alerting on FTP connections as user {\tt "root"}
from either {\tt bob1.dsl.home.net} or {\tt bob2.dsl.home.net} to the
server running on {\tt bob.work.com}.

\indvartype{ftp_hot_files}{pattern}
Bro matches the argument given in each FTP file manipulation
request (RETR, STOR, etc.)
against this pattern to see if the file is sensitive. If so,
and if the request succeeds, then the access is logged.

\indtt{eggdrop}{sensitive filename}
\index{filenames!sensitive}
Default: a pattern that matches various flavors of password files, plus
any string with {\tt eggdrop} in it.  \note{Eggdrop is an IRC management
tool often installed by certain attackers upon a successful break-in.}

\indvartype{ftp_not_actually_hot_files}{pattern}
A pattern giving exceptions to {\tt ftp_hot_files}.  It turns out
that a pattern like {\tt /passwd/} generates a lot of false hits,
such as from {\tt passwd.c} (source for the \emph{passwd} utility;
this can turn up in FTP sessions that fetch entire sets of utility sources
using {\tt MGET}) or {\tt passwd.html} (a Web page explaining how to enter
a password for accessing a particular page).

Default: {\tt /(passwd|shadow).*\verb+\.+(c|gif|htm|pl|rpm|tar|zip)/} .

\indvartype{ftp_hot_guest_files}{pattern}
Files that guests should not attempt to access.

\index{rhosts@{\tt .rhosts}}
\index{forward@{\tt .forward}}
Default: {\tt .rhosts} and {\tt .forward} .

\indvartype{skip_unexpected}{set[addr]}
If a new host (address) unexpectedly connects to the endpoint specified in a
{\tt PORT} or {\tt PASV} directive, then if either the original host
or the new host is in this set, no message is
generated.  The idea is that you can specify multi-homed hosts that
frequently show up in your FTP traffic, as these can generate innocuous
warnings about connections from unexpected hosts.

Default: some {\tt hp.com} hosts, as an example.  Most are specified
as raw IP addresses rather than hostnames, since the hostnames
don't always consistently resolve.

\indvartype{skip_unexpected_net}{set[addr]}
The same as {\tt skip_unexpected}, except addresses are masked to /24 and
/16 before looked up in this set.

Default: empty.

\end{description}

\begin{figure}
\label{fig:ftp-log-eg}
\begin{verbatim}
972499885.784104 #26 131.243.70.68/1899 > 64.55.26.206/ftp start
972499886.685046 #26 response (220 tuvok.ooc.com FTP server
    (Version wu-2.6.0(1) Fri Jun 23 09:17:44 EDT 2000) ready.)
972499886.686025 #26 USER anonymous/IEUser@ (logged in)
972499887.850621 #26 TYPE I (ok)
972499888.421741 #26 PASV (227 64.55.26.206/2427)
972499889.493020 #26 SIZE /pub/OB/4.0/JOB-4.0.3.zip (213 1675597)
972499890.135706 #26 *RETR /pub/OB/4.0/JOB-4.0.3.zip, ABOR (complete)
972500055.491045 #26 response (225 ABOR command successful.)
\end{verbatim}
\caption{ Example of FTP log file entries for a single FTP session. }
\end{figure}

\label{ftp-log-file}
\label{ftp-log}
\index{FTP!log file}
\index{log file!FTP}
\index{ftp session summary file@{\tt ftp} session summary file}
In addition, {\tt ftp_log} holds the name of the FTP log file to
which Bro writes FTP session summaries.  It defaults to
{\tt open_log_file("ftp")}.

\index{IEUser!useless FTP username}
\f{fig:ftp-log-eg} shows an example of
what entries in this file look like.  Here we see a transcript of
the 26th FTP session seen since Bro started running.  The first line
gives its start time and the participating hosts and ports.  The
next line (split across two lines above for clarity) gives the server's
welcome banner.  The client then logged in as user ``{\tt anonymous}'',
and because this is one of the guest usernames, Bro recorded their
password too, which in this case was ``{\tt IEUser@}'' (a useless
string supplied by their Web browser).  The server accepted this
authentication, so the status on the line is ``{\tt (logged in)}''.

The client then issues a request for the Image file type, to which the
server agreed.  Next they issued a {\tt PASV} directive, and received a
response instructing them to connect to the server on port {\tt 2427/tcp}
for the next transfer.  At this point, after issuing a {\tt SIZE} directive
(to which the server returned 1,675,597 bytes), they send {\tt RETR} to
fetch the file \emph{/pub/OB/4.0/JOB-4.0.3.zip}.  However, before the
transfer completed, they issued {\tt ABOR}, but the transfer finished
before the server processed the abort, so the log shows a status of {\tt
(completed)}.  Furthermore, because the client issued two commands without
waiting for an intervening response, these are shown together in the log
file, and the line marked with a ``{\tt *}'' so it draws the eye.  Finally,
because Bro paired up the {\tt (completed)} with the multi-request line, it
then treats the response to the {\tt ABOR} command as a reply by itself,
showing in the last line that the server reported it successfully carried
out the abort.

The corresponding lines in the \xreflog{red} file look like:
\begin{verbatim}
    972499885.784104 565.836 ftp 118 427 131.243.70.68 64.55.26.206
        RSTO L #26 anonymous/IEUser@
    972499888.984116 165.098 ftp-data ? 1675597 131.243.70.68 64.55.26.206
        RSTO L
\end{verbatim}
The first line summarizes the FTP control session (over which the client
sends its requests and receives the server's responses).  It includes
an \xreffield{addl}{connection} annotation of ``{\tt \#26 anonymous/IEUser@}'',
summarizing the session number (so you can find the corresponding records
in the \xreflog{ftp} log file) and the authentication information.

\index{connection size!undetermined for RST termination}
\index{RST termination!causing undetermined connection size}
The second line summarizes the single FTP data transfer, of 1,675,597 bytes.
The amount of data sent by the client for this connection is shown as
unknown because the client aborted the connection with a RST (hence
the state {\tt RSTO}).   For connections that Bro does not look inside
(such as FTP data transfers), it learns the amount of data transferred from
the sequence numbers of the SYN and FIN connection control packets, and
can't (reliably) learn them for the sender of a RST.  (It can for the
receiver of the RST.)

They also aborted the control session (again, state {\tt RSTO}), but
in this case, Bro captured all of the packets of the session, so it
could still assign sizes to both directions.

\privatenote{ Finally, the analyzer also sets the global {\tt have_FTP} to true so that
other modules can know that the {\tt ftp} analyzer has been
loaded.  \deficiency{ This is a clumsy way of signaling an
analyzer's presence; a cleaner scheme would be desirable. } }

\index{analyzers!ftp!variables|)}

\subsection{{\tt ftp} functions}
\index{analyzers!ftp!functions|(}

The standard {\tt ftp} script provides one function for external use:

\begin{description}
\itemindfunc{is_ftp_data_conn}{(c: connection): bool }
Returns true if the given connection matches one we're expecting
as the data connection half of an FTP session.  \note{ This function is
not idempotent: if the connection matches an expected one, then
Bro updates its state such that that connection is no longer expected.
It also logs a discrepancy if the connection appears to be usurping
another one that generated either a ``{\tt PORT}'' or a ``{\tt PASV}''
directive.}

Also returns true if the source port is {\tt 20/tcp} and there's currently
an FTP session active between the originator and responder, in case for
some reason Bro's bookkeeping is inconsistent.

\end{description}

\index{analyzers!ftp!functions|)}

\subsection{{\tt ftp} event handlers}
\index{analyzers!ftp!event handlers|(}

The standard script handles the following events:

\begin{description}
\indeventtype{ftp_request}{c: connection, command: string, arg: string}
Invoked upon the client side of connection {\tt c} having made the request
{\tt command} with the argument {\tt arg}.

The processing depends on the particular command:
\begin{description}
\item[{\tt USER}] ~ \\
Specifies the username that the client wishes to
use for authentication. If it is sensitive---in {\tt hot_ids} (which
the {\tt ftp} analyzer accesses via a \load of {\tt hot-ids})---then
the analyzer flags the FTP session as log-worthy.  In addition, if
the username is in {\tt forbidden_ids}, then the analyzer terminates
the session.

The analyzer also updates the connection's \xreffield{addl}{connection} field
with the username.

\item[{\tt PASS}] ~ \\
Specifies the password to use for authentication.

If the password is empty and the username appears in
{\tt forbidden_ids_if_no_password} (also from the {\tt hot-ids} analyzer),
then the analyzer terminates the connection.

If the username corresponds to a guest account ({\tt ftp_guest_ids}),
then the analyzer updates the connection's \xreffield{addl}{connection} field
with the password as additional account information.  Otherwise,
it generates an \xrefevent{account_tried} event to
facilitate detection of password guessing.

\item[{\tt PORT}] ~ \\
Instructs the FTP server to connect to the given
IP address and port for delivery of the next FTP data item.  The analyzer
first checks the address/port specifier for validity.  If valid, it
will generate an alert if either the address specified in the directive
does not match that of the client, or if the port corresponds to a
``privileged'' port, i.e., one in the range 0--1023.  Finally, it
establishes state so that {\tt is_ftp_data_conn} can identify a
subsequent connection corresponding to this directive as belonging to
this FTP session.

\item[{\tt ACCT}] ~ \\
Specifies additional accounting information associated with a session,
which the analyzer simply adds to the connection's \xreffield{addl}{connection}
field.

\item[{\tt APPE}, {\tt  CWD}, {\tt  DELE}, {\tt  MKD}, {\tt  RETR}, {\tt  RMD}, {\tt  RNFR}, {\tt  RNTO}, {\tt  STOR}, {\tt  STOU}] ~ \\
All of these manipulate files (and directories).  The analyzer checks
the filename against the policies to see if it is sensitive in the
context of the given username (i.e., guest or non-guest), and, if so,
marks the connection to generate an alert unless the operation fails.
The analyzer also checks for an excessively long filename, currently
by checking its length against a \deficiency{hardwired maximum of 250 bytes}.
\end{description}

\indeventtype{ftp_reply}{c: connection, code: count, msg: string, cont_resp: bool}
Invoked upon the server side of connection {\tt c} having replied to
a request using the given status code and text message.  {\tt cont_resp}
is true if the reply line is tagged as being continued to the next line.
The analyzer only processes requests when the last line of a continued
reply is received.

The analyzer checks the reply against any expected for the connection
(for example, ``{\tt log_if_not_denied}'') and generates alerts accordingly.
If the reply corresponds to a {\tt PASV} directive, then it parses the
address/port specification in the reply and generates alerts in an analogous
fashion as done by the {\tt ftp_request} handler for {\tt PORT} directives.

Finally, if the reply is not one that the analyzer is hardwired to skip
(code {\tt 150}, used at the beginning of a data transfer, and code
{\tt 331}, used to prompt for a password),
then it writes a summary of the request and reply to the FTP log file
\pxref{ftp-log-file}.  Also, if the reply is an ``orphan'' (there was
no corresponding request, perhaps because Bro started up after the
request was made), then the reply is summarized in the log file by
itself.

\end{description}

The standard {\tt ftp} script defines one other handler, an instance of
\xrefevent{connection_finished} used to flush FTP session information
in case the session terminates abnormally and no reply is seen to
the pending request(s).

\index{analyzers!ftp!event handlers|)}

\labsectsection{{The \analyzer{http} Analyzer}}{\label{analyzer-http}}
\index{HTTP!analysis}

The \indanalyzer{http} analyzer processes traffic associated with
the Hyper Text Transfer Protocol (HTTP) \cite{rfc-http-1-0,rfc-http-1-1},
the main
protocol used by the Web.  Bro instantiates an
{\tt http} analyzer for any connection with service port {\tt 80/tcp},
providing you have loaded the {\tt http} analyzer, or defined a handler
for {\tt http_request}.  It also instantiates an analyzer for
service ports {\tt 8080/tcp} and {\tt 8000/tcp}, as these are
often also used for Web servers.

The analyzer uses a capture filter of ``{\tt tcp dst port 80 or tcp dst port 8080 or tcp dst port 8000}'' \pxref{capture-filter}.  \note{ This filter excludes
traffic sent by an HTTP server (that would be matched by {\tt tcp src port 80},
etc.), because} \deficiency{ Bro doesn't yet have an analyzer for HTTP replies. }
It generates summaries of HTTP sessions (connections between the same
client and server) and looks for access to sensitive URIs (effectively, URLs).

\subsection{{\tt http} variables}
\index{analyzers!http!variables|(}

\begin{description}
\index{HTTP methods}
\indvartype{sensitive_URIs}{pattern}
Any HTTP method (e.g., \indtt{GET}{HTTP method}, \indtt{HEAD}{HTTP method},
\indtt{POST}{HTTP method}) specifying
a URI that matches this pattern is flagged as sensitive.

\index{etc/passwd@\emph{/etc/passwd}}
\index{passwd@\emph{passwd}}
\index{etc/shadow@\emph{/etc/shadow}}
\index{shadow@\emph{shadow}}
\index{Cold Fusion exploits}
Default: URIs with {\tt /etc/passwd} or {\tt /etc/shadow} embedded
in them, or {\tt /cfdocs/expeval} (used in some Cold Fusion exploits).
\note { This latter generates some false hits; it's mainly included
just to convey the notion of looking for direct attacks rather than
attacks used to exploit sensitive files like the first ones. }

\deficiency{ It would be very handy to have variables providing hooks
for more context when considering whether a particular access is
sensitive, such as whether the request was inbound or outbound. }

\indvartype{sensitive_post_URIs}{pattern}
Any {\tt POST} method specifying a URI that matches this pattern is flagged as
sensitive.

Default: URIs with \indtt{wwwroot}{sensitive POST URI} embedded in them.

\end{description}

\index{frogs!dissecting}
\begin{figure}
\label{fig:http-log-eg}
\begin{verbatim}
972482763.371224 %1596 start 200.241.229.80 > 131.243.2.12
%1596 GET /ITG.hm.pg.docs/dissect/portuguese/dissect.html
%1596 GET /vfrog/bottom.icon.gif
%1596 GET /vfrog/top.icon.gif
%1596 GET /vfrog/movies/off.gif
%1596 GET /vfrog/new.frog.small.gif
\end{verbatim}
\caption{ Example of HTTP log file entries for a single HTTP session. }
\end{figure}

\index{http session summary file@{\tt http} session summary file}
\label{http-log-file}
\label{http-log}
\index{HTTP!log file}
\index{log file!HTTP}
In addition, {\tt http_log} holds the name of the HTTP log file to
which Bro writes HTTP session summaries.  It defaults to
{\tt open_log_file("http")}.

\f{fig:http-log-eg} shows an example of
what entries in this file look like.  Here we see a transcript of
the 1596th HTTP session seen since Bro started running.  The first line
gives its start time and the participating hosts.  The
next five lines all correspond to {\tt GET} methods retrieving different
items from the Web server.  \deficiency{ Bro can't log whether
the retrievals succeeded or failed because it doesn't yet have
an HTTP reply analyzer. }

The corresponding lines in the \xreflog{red} file look like:
\begin{verbatim}
    972482762.872695 481.551 http 441 5040 131.243.2.12 200.241.229.80
        S3 X %10596
    972482764.686470 18.7611 http 596 7712 131.243.2.12 200.241.229.80
        S3 X %10596
    972482764.685047 ? http 603 2959 131.243.2.12 200.241.229.80
        S1 X %10596
\end{verbatim}
That there are three rather than five reflects \emph{(i)}~that the client
used persistent HTTP, and so didn't need one connection per item, but
also \emph{(ii)}~the client used three parallel connections (the maximum
the standard allows is only two) to fetch the items more quickly.  As with FTP
sessions, the \verb+%10596+
\xreffield{addl}{connection}
annotation lets you correlate the \xreflog{red} entries with the
\xreflog{http} log entries.

\note{ All three of the connections wound up in unusual states.  The first
two are in state {\tt S3}, which, as indicated by \tbl{tbl:conn-file-states}
means that the responder (in this case, the Web server) attempted to close
the connection, but their was no reply from the originator.  The last is
in state {\tt S1}, indicating that neither side attempted to close the
connection (which is why no duration is listed for the connection).
}

\index{analyzers!http!variables|)}

\subsection{{\tt http} event handlers}
\index{analyzers!http!event handlers|(}

The standard HTTP script defines one event handler:

\begin{description}
\indeventtype{http_request}{c: connection, request: string, URI: string}
Invoked whenever the client side of the given connection generates an
HTTP request.  {\tt request} gives the HTTP method and {\tt URI} the
associated resource.  The analyzer matches the URI against the ones
defined as sensitive, as given above.
\end{description}

\deficiency{ As mentioned above, the event engine does not currently
generate an {\tt http_reply} event.  This is for two reasons: first,
the HTTP request stream is much lower volume than the HTTP reply stream,
and I was interested in the degree to which Bro could get away without
analyzing the higher volume stream.  (Of course, this argument is
shallow, since one could control whether or not Bro should analyze
HTTP replies by deciding whether or not to define an {\tt http_reply}
handler.)  Second, matching HTTP replies in their full generality
involves a lot of work, because the HTTP standard allows replies to be
delimited in a number of ways.  That said, most of the work for implementing
{\tt http_reply} is already done in the event engine, but it is missing
testing and debugging.}

\index{analyzers!http!event handlers|)}

\labsectsection{{The \analyzer{ident} Analyzer}}{\label{analyzer-ident}}
\index{IDENT!analysis}

The \indanalyzer{ident} analyzer processes traffic associated with
the Identification Protocol \cite{rfc-ident}, which provides a simple
service whereby clients can query Ident servers to discover user information
associated with an existing connection between the server's host and
the client's host.  Bro instantiates an {\tt ident} analyzer for
any connection with service port {\tt 113/tcp}, providing you have loaded
the {\tt ident} analyzer, or defined a handler for {\tt ident_request},
{\tt ident_reply}, or {\tt ident_error}.

The analyzer uses a capture filter of ``{\tt tcp port 113}''
\pxref{capture-filter}.
The {\tt ident_reply} handler annotates the \xreffield{addl}{connection}
field of the connection for which the Ident client made its query with the
user information returned in the reply.  It also checks the user information
against sensitive usernames, because a match indicates that the connection
in the Ident query was initiated by a possibly-compromised account.

\subsection{{\tt ident} variables}
\index{analyzers!ident!variables|(}

The standard script defines the following pair of redefinable variables:

\begin{description}
\indvartype{hot_ident_ids}{set[string]}
usernames to flag as sensitive if they appear in an Ident reply.

Default: {\tt always_hot_ids} \pxref{always-hot-ids}.

\indvartype{hot_ident_exceptions}{set[string]}
usernames not to consider sensitive even if they appear in
{\tt hot_ident_ids}.

\indttnotext{uucp}{username}
\indttnotext{nuucp}{username}
\indttnotext{daemon}{username}
\index{daemons!as innocuous user names}
Default: {\tt \{ "uucp", "nuucp", "daemon", \}}.  These usernames
are exceptions because daemons sometimes run with the given user ids
and their use is often innocuous.

\end{description}

\index{analyzers!ident!variables|)}

\subsection{{\tt ident} event handlers}
\index{analyzers!ident!event handlers|(}

The standard script handles the following events:

\begin{description}
\indeventtype{ident_request}{c: connection, lport: port, rport: port}
Invoked when a client request arrives on connection {\tt c}, querying
about the connection from local port {\tt lport} to remote port
{\tt rport}, where local and remote are relative to the client.

\indeventtype{ident_reply}{c: connection, lport: port, rport: port, user_id: string, system: string}
Invoked when a server replies to an Ident request.  {\tt lport} and
{\tt rport} are again the local and remote ports (relative to the
client) of the connection being asked about.  {\tt user_id} is the
user information returned in the Ident server's reply, and {\tt system}
is information regarding the operating system (the Ident specification
\cite{rfc-ident} does not further standardize this information).

The handler annotates the queried connection with the user information,
which it also checks against {\tt hot_ident_ids} and {\tt hot_ident_exceptions}
as discussed above.  At present, it does nothing with the {\tt system}
information.

\indeventtype{ident_error}{c: connection, lport: port, rport: port, line: string}
Invoked when the given request yielded an error reply from the Ident
server.  The handler annotates the connection with
\indtt{ident/}{authentication annotation}\verb+<+\emph{error}\verb+>+,
where \emph{error} is the text given in {\tt line}.

\end{description}

\index{analyzers!ident!event handlers|)}

\labsectsection{{The \analyzer{login} Analyzer}}{\label{analyzer-login}}
\index{login session}
\index{Rlogin!sessions}
\index{Telnet!sessions}
\index{Unix analysis}
\index{keystrokes!analysis}
\index{input!analysis}
\index{user keystrokes!analysis}
The \indanalyzer{login} analyzer inspects interactive login sessions
to extract username and password information, and monitors user
keystrokes and the text returned by the login server.  It is one of
the most powerful Bro modules for detecting break-ins to Unix
systems because of the ability to look for particular commands that
attackers often execute once they have penetrated a Unix machine.

The analyzer is generic in the sense that it applies to more
than one protocol.  Currently, Bro instantiates a {\tt login}
analyzer for both Telnet \cite{rfc-telnet} and Rlogin \cite{rfc-rlogin}
traffic.  In principle, it could do the same for other protocols such as
SSH \cite{rfc-ssh} or perhaps X11 \cite{rfc-x11}, if one could write
the corresponding elements of the event engine to decrypt the
SSH session (naturally, this would require access to the encryption keys)
or extract authentication information and keystrokes from the
X11 event stream.  \note{ The analyzer does an exceedingly limited
form of SSH analysis; see \xrefvar{hot_ssh_orig_ports}. }

\index{authentication dialog}
\index{usernames!extracting}
\index{sniffing}
\index{passwords!sniffing}
\index{heuristics!extracting username information}
\index{Telnet!options}
\index{options!Telnet}
For Telnet, the event engine knows how to remove in-band Telnet option
sequences \cite{rfc-telnet-options}
from the text stream, and does not deliver these to
the event handlers, except for a few options
that the engine
analyzes in detail (such as attempts to negotiate authentication).
Unfortunately, the Telnet protocol does not include any explicit
marking of username or password information (unlike the FTP protocol,
as discussed in \xref{analyzer-ftp}).  Consequently, Bro employs a series
of heuristics that attempt to extract the username and password from the
authentication dialog the session is presumed to begin with.  The
analysis becomes quite complicated due to the possible use of
type-ahead and editing sequences by the user, plus the possibility
\index{evasion!authentication dialog}
that the user may be an attacker who attempts to mislead the heuristics
in order to disguise the username they are accessing.

\index{rhosts@{\tt .rhosts}}
Analyzing Rlogin is nominally easier than analyzing Telnet because Rlogin
has a simpler in-band option scheme, and because the Rlogin protocol
explicitly indicates the username in the initial connection dialog.
However, this last is not actually a help to the analyzer, because
for most Rlogin servers, if the initial username fails authentication
(for example, is not present in the {\tt .rhosts} file local to
the server), then the server falls back on the same authentication
dialog as with Telnet
(prompting for username and then password, or perhaps just
for a password to go with the transmitted username).
Consequently, the event engine employs the same set of heuristics
as for Telnet.

Each connection processed by the analyzer is in a distinct state:
user attempting to authenticate, user has successfully authenticated,
analyzer is skipping any further processing, or the analyzer is
confused \pxref{login-confused}.  You can find out the state of
a given connection using \xreffunc{get_login_state}.

The analyzer uses a capture filter of ``{\tt tcp port 23 or tcp port 513}''
\pxref{capture-filter}.  It annotates each connection
with the username(s) present in the authentication dialog.  If
the username was authenticated successfully, then it encloses
the annotation in quotes. If the authentication failed, then
the name is marked as {\tt failed/}\verb+<+\emph{username}\verb+>+.
So, for example, if user ``smith'' successfully authenticates,
then the connection's \xreffield{addl}{connection} field will have
{\tt "smith"} appended to it:
\begin{verbatim}
931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L "smith"
\end{verbatim}
while if ``smith'' failed to authenticate, the report will look like:
\begin{verbatim}
931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L fail/smith
\end{verbatim}
and if they first tried as ``smith'' and failed, and then succeeded
as ``jones'', the record would look like:
\begin{verbatim}
931803523.006848 254.377 telnet 324 8891 1.2.3.4 5.6.7.8 SF L fail/smith "jones"
\end{verbatim}

\index{passwords!inadvertently exposed}
\index{sensitive information!inadvertently exposed}
\note{ The event engine's heuristics can sometimes get out of synch
such that it interprets a password as a username; in addition, users
sometimes type their password when they should instead enter
their username.  Consequently, the connection logs sometimes include
passwords in the annotations, and so should be treated as very sensitive
information (e.g., not readable by any user other than the one running
Bro). }

\labsectsubsection{{{\tt login}} analyzer confusion}{\label{login-confused}}
\index{login analysis!confusion}
\index{confused login analysis}
\index{heuristics!confusion}
\index{confusion of heuristics}
\index{failure of heuristics}
\index{authentication dialog}
\index{usernames!extracting}
\index{heuristics!extracting username information}
\index{rhosts@{\tt .rhosts}}
Because there is no well-defined protocol for Telnet authentication
(or Rlogin, if the initial
{\tt .rhosts} authentication fails), the {\tt login} analyzer employs a set
of heuristics to detect the username, password, and whether the authentication
attempt succeeded.  All in all, these heuristics work quite well, but
it is possible for them to become confused and reach incorrect conclusions.

\begin{table}
\label{tbl:login-confusion}
\begin{center}
\begin{tabular}{|l|p{4.0in}|} 
\hline
{\bf Type of confusion} & {\bf Meaning} \\
\hline
\hline
\indextext{excessive_typeahead}{login confusion state}
\index{heuristics!type-ahead}
\index{type-ahead!maximum allowed}
		& The user has typed ahead 12~or more lines.
		  \deficiency{ The upper bound should be adjustable. } \\
\hline
\indextext{extra_repeat_text}{login confusion state}
\index{repeat text (VMS)}
\index{heuristics!VMS}
\index{VMS input editing}
		& The user has entered more than one VMS repeat sequence
		  (an escape followed by ``\verb+[A+'') on the same line.
		  \note{ Bro determines that a login session involves a
		  VMS server if the server prompts with ``\indextext{Username:}{VMS login prompt}''.
		  It then interprets VMS repeat sequences as indicating
		  it should replace the current line with the previous line. }
		  \\
\hline
\indextext{multiple_USERs}{login confusion state}
\index{heuristics!multiple usernames}
\index{usernames!repeated}
\index{multiple usernames}
		& The user has specified more than one username using
		  the \indenv{USER} environment variable.
		  \\
\hline
\indextext{multiple_login_prompts}{login confusion state}
\index{heuristics!multiple login prompts}
\index{login prompts!repeated}
\index{multiple login prompts}
		& The analyzer has seen several login prompts on
		  the same line, and has not seen a corresponding
		  number of lines typed ahead previously by the user.
		  \\
\hline
\indextext{no_login_prompt}{login confusion state}
\index{heuristics!missing login prompt}
\index{login prompts!missing}
\index{missing login prompts}
		& The analyzer has seen 50~lines sent by the server
		  without any of them matching \xrefvar{login_prompts}.
		  \deficiency{ The value of 50 should be adjustable. }
		  \\
\hline
\indextext{no_username}{login confusion state}
\index{heuristics!missing username}
\index{usernames!missing}
\index{missing username}
		& The analyzer is generating an event after having
		  already seen a login failure, but the user's input
		  has not provided another username to include with
		  the event.  \note{ If the analyzer's heuristics indicate
		  it's okay that no new username has been given, such
		  as when the event is generated due to one connection
		  endpoint closing the connection, then it instead uses
		  the username }
		  \protect\index{none@\verb+<none>+ username}
		  \protect\index{usernames!none@\verb+<none>+}
		  \verb+<none>+ .
		  \\
\hline
\indextext{no_username2}{login confusion state}
\index{heuristics!missing username}
\index{usernames!missing}
\index{missing username}
		& The analyzer saw an additional password prompt
		  without seeing an intervening username, and it has
		  no previous username to reuse.
		  \\
\hline
\indextext{non_empty_multi_login}{login confusion state}
\index{heuristics!multiple login prompts}
\index{login prompts!repeated}
\index{multiple login prompts}
		& The analyzer saw multiple adjacent login prompts,
		  with an apparently ignored intervening username
		  typed-ahead between them.
		  \\
\hline
\indextext{possible_login_ploy}{login confusion state}
\index{heuristics!attacker-induced confusion}
\index{authentication dialog!evasion}
\index{evasion!authentication dialog}
		& The client sent text that matches one of the
		  patterns reflecting text usually sent by the
		  server.  This form of confusion can reflect
		  an attacker attempting to evade the monitor.
		  For example, the client may have sent
		  the text ``{\tt login:}'' as a username so that
		  when echoed back by the server, the analyzer would
		  misinterpret it as reflecting another login prompt from
		  the server.
		  \\
\hline
\indextext{repeat_without_username}{login confusion state}
\index{repeat text}
\index{heuristics!VMS}
\index{VMS input editing}
		& The user entered a VMS repeat sequence but there is
		  no username to repeat.  (See {\tt extra_repeat_text}
		  for a discussion of the analyzer's heuristics for
		  dealing with VMS servers.)
		  \\
\hline
\indextext{responder_environment}{login confusion state}
\index{environment!responder}
\index{heuristics!environment}
\index{Telnet!options!environment}
\index{environment!Telnet options}
		& The responder (login server) has signaled a set
		  of environment variables to the originator (login
		  client).  This is in the opposite direction as to
		  what makes sense.
		  \\
\hline
\indextext{username_with_embedded_repeat}{login confusion state}
\index{repeat text}
\index{heuristics!VMS}
\index{VMS input editing}
		& The line repeated by a VMS server in response to
		  a repeat sequence itself contains a repeat sequence.
		  \\
\hline
\end{tabular}
\end{center}
\caption{
Different types of confusion that {\tt login} analyzer can report.
}
\end{table}

Bro attempts to detect such confusion.  If it does, then it generates a
\xrefevent{login_confused} event, after which the event engine will no
longer attempt to follow the authentication dialog.  In particular, it will
\emph{not} generate subsequent \xrefevent{login_failure} or
\xrefevent{login_success} events.  The {\tt login_confused} event includes
a string describing the type of confusion, using one of the values
given in Table~\ref{tbl:login-confusion}.

\subsection{{\tt login} variables}
\index{analyzers!login!variables|(}

The standard script defines a large number of variables for refining the
analysis policy:

\begin{description}
\indvartype{input_trouble}{pattern} lists patterns that the analyzer
should flag if they appear in the user's input (keystroke) stream.

\label{keystroke-editing}
\index{user keystrokes!editing}
\index{keystrokes!editing}
\index{input!editing}
The analyzer searches for these patterns both in the raw text typed
by the user and the same lines after applying \emph{editing}
using the \xreffunc{edit} function twice: once with interpreting
\emph{BS} (ctrl-H) as delete-one-character, and once with \emph{DEL}
as the edit character.  If any of these matches, then the analyzer
considers the pattern to have matched.

\indtt{eggdrop}{sensitive login input}
\indttnotext{rewt}{username}
\index{root!backdoors}
\index{Internet Relay Chat (IRC)!attacker subpopulation}
\index{IRC}
\indttnotext{loadmodule}{exploit}
\indtttwo{/bin/eject}{bin/eject}{exploit}
\indtttwo{eject}{eject}{exploit}
\index{exploits!Unix}
Default: a pattern matching occurrences of the strings
``{\tt rewt}'',
``{\tt eggdrop}'',
``{\tt loadmodule}'', or
``{\tt /bin/eject}''.  The first of these is a popular username attackers
use for root backdoor accounts.  The second reflects that one prevalent
class of attackers are devotees of Internet Relay Chat (IRC), who
frequently upon breaking into an account install the IRC {\tt eggdrop}
utility.

\indvartype{edited_input_trouble}{pattern} is the same
as {\tt input_trouble} except the analyzer only checks the edited
user input against the pattern, not the raw input (see above).

This variable is provided so you can specify patterns that can
occur innocuously as typos; whenever the user corrects the typo before
terminating the line, the pattern won't match, because it won't be present
in the edited version of the line.  In addition, for matches to
these patterns, the analyzer \emph{delays} reporting the match until
it sees the next line of output from the server.  It then includes
both the line that triggered the match and the corresponding response
from the server, which makes it easy for a human inspecting the logs
to tell if the occurrence of the pattern was in fact innocuous.

\index{filenames!sensitive}
\index{directory names!sensitive}
\index{sensitive filenames}
Here's an example of an innocuous report:
\begin{verbatim}
936723303.760483 1.2.3.4/21550 > 5.6.7.8/telnet
    input "cd ..." yielded output "ksh: ...:  not found."
\end{verbatim}
It was flagged because the user's input included
``{\tt ...}'', a name commonly used by attackers to surreptitiously
hide a directory containing their tools and the like.  However, we
see from the Telnet server's response that this was not actual access
to such a directory, but merely a typing mistake.

On the other hand:
\begin{verbatim}
937528764.579039 1.2.3.4/3834 > 5.6.7.8/telnet
    input "cd ..." yielded output "maroon# ftp sunspot.sunspot.noao.edu "
\end{verbatim}
shows a problem---the lines returned by the server was a root
prompt (``{\tt maroon\verb+#+}''), to which the user issued a command to
access a remote FTP server.

\deficiency{ The analyzer should decouple the notion of waiting
to receive the server's reply from the notion of matching only the
edited form of the line; there might be raw inputs for which it is
useful to see the server's response, and edited inputs for which the
server's response is unimportant in terms of knowing that the input
spells trouble. }

Default: the pattern
\begin{verbatim}
    /[ \t]*cd[ \t]+((['"]?\.\.\.)|(["'](\.[^"']*)[ \t]))/
\end{verbatim}
which looks for a ``{\tt cd}'' command to either a directory beginning
with ``{\tt ...}'' (optionally quoted by the user) or a directory
name beginning with ``{\tt .}'' that is quoted and includes an
embedded blank or tab.

\indvartype{output_trouble}{pattern} lists patterns that
the analyzer should flag if they occur in the output sent by the
login server back to the user.

\index{buffer overflow tools}
\index{exploits!buffer overflow}
\index{sniffer logs}
\index{trojaning}
\index{Linux!super exploit}
\index{smurf attacks}
\index{attacks!smurf}
\index{log file!altering}
\index{altering log files}
\indtt{PATH_UTMP}{sensitive pattern}
\indtt{smashdu.c}{exploit tool}
\index{root!setuid}
\index{setuid root}
\index{command shell!setuid root}
\index{ls@\emph{ls} utility}
\index{utilities!ls@\emph{ls}}
\index{lynx@\emph{lynx} utility}
\index{utilities!lynx@\emph{lynx}}
\index{fetch@\emph{fetch} utility}
\index{utilities!fetch@\emph{fetch}}
\index{anticode.com@\emph{anticode.com}}
\index{www.anticode.com@\emph{www.anticode.com}}
\index{TFreak}
Default: the pattern
\begin{verbatim}
      /^-r.s.*root.*\/bin\/(sh|csh|tcsh)/
    | /Jumping to address/
    | /smashdu\.c/
    | /PATH_UTMP/
    | /Log started at =/    
    | /www\.anticode\.com/
    | /smurf\.c by TFreak/
    | /Trojaning in progress/ 
    | /Super Linux Xploit/
\end{verbatim}
The first of these triggers any time the user inspects with the
\emph{ls} utility an executable whose pathname ends in {\tt /bin/} followed
by one of the popular command shells, and the \emph{ls} output shows
that the command shell has been altered to be setuid to root.
The remainder match either the output generated by some popular
exploit tools (for example, ``{\tt Jumping to address}'', present
in many buffer overflow exploit tools), exploit tool names (``{\tt smashdu.c}''),
text found within the tool source code (``{\tt smurf.c by TFreak}''),
or URLs accessed (say via the \emph{lynx} or \emph{fetch} utilities)
to retrieve attack software (``{\tt www.anticode.com}'').

\index{backdoor!prompts}
\indvartype{backdoor_prompts}{pattern} lists patterns that
the analyzer should flag if they are seen as the first line sent by the
server to the user, because they often correspond with
backdoors that offer a remote user immediate command shell access
without having to first authenticate.

Default: the pattern ``\verb+/^[!-~]*( ?)[#%$] /+'', which matches
a line that begins with a series of printable, non-blank characters and
ends with a likely prompt character, with a blank just after
the prompt character and perhaps before it.

\index{backdoor!avoiding false positives}
\indvartype{non_backdoor_prompts}{pattern} lists patterns
that if a possible backdoor prompt also matches, then the analyzer
should not consider the server output as indicating a backdoor prompt.
Used to limit false positives for {\tt backdoor_prompts}.

Default: the pattern ``\verb+/^ *#.*#/+'', which catches lines with
more than one occurrence of a \verb+#+.  Some servers generate such
lines as part of their welcome banner.

\index{backdoor!triggered by terminal type}
\index{magic terminal types}
\indvartype{hot_terminal_types}{pattern} lists ``magic''
terminal types sometimes used by attackers to access backdoors.
Both Telnet and Rlogin have mechanisms for negotiating a terminal
type (name; e.g., ``{\tt xterm}''); these backdoors trigger and skip
authentication if the name has a particular value.

\indtt{VT666}{terminal type backdoor}
Default: the name ``{\tt VT666}'', one of the trigger terminal types
we've observed in practice.

\index{backdoor!triggered by ephemeral port}
\index{ephemeral port!triggering a backdoor}
\index{client port!triggering a backdoor}
\indvartype{hot_telnet_orig_ports}{set[port]} Some Telnet backdoors
trigger if the ephemeral port used by the client side of the connection
happens to be a particular value.  This variable is used to list the
port values whose use should be considered as possibly indicating
a backdoor.  \note{ Clearly, this mechanism can generate false
positives when the client by chance happens to choose one of the
listed ports. }

Default: {\tt 53982/tcp}, one of the trigger ports we have observed
in practice.

\deficiency{ There should be a corresponding variable for Rlogin
backdoors triggered by a similar mechanism. }

\indvartype{hot_ssh_orig_ports}{set[port]} Similar to 
{\tt hot_telnet_orig_ports}, only for SSH.

Default: {\tt 31337/tcp}, a trigger port that we've observed in practice.

\indvartype{skip_authentication}{set[string]} A set of strings
that, if present in the server's initial output (i.e., its welcome banner),
indicates the analyzer should not attempt to analyze the session for an
authentication dialog.  This is used for servers that provide public
access and don't bother authenticating the user.

Default: the string {\tt "WELCOME TO THE BERKELEY PUBLIC LIBRARY"},
which corresponds to a frequently accessed public server in the
Berkeley area.  (Obviously, we include this default as an example,
and not because it will be appropriate for most Bro users!  But it
does little harm to include it.)

\deficiency{ It would be more natural if this variable and a number
of others listed below were of type
{\tt pattern} rather than {\tt set[string]}.  They are actually
converted internally by the event engine into regular expressions. }

\indvartype{direct_login_prompts}{set[string]} A set of strings
that if seen during the authentication dialog mean that the user will
be logged in as soon as they answer the prompt.

Default: {\tt "TERMINAL?"}, a prompt used by some terminal servers.

\indvartype{login_prompts}{set[string]} A set of strings
corresponding to login username prompts during an authentication
dialog.

Default: the strings
\begin{verbatim}
    Login:
    login:
    Name:
    Username:
    User:
    Member Name
\end{verbatim}
and the default contents of {\tt direct_login_prompts}.

\indvartype{login_failure_msgs}{set[string]} A set of strings
that if seen in text sent by the server during the authentication dialog
correspond to a failed login attempt.

Default: the strings
\begin{verbatim}
    invalid
    Invalid
    incorrect
    Incorrect
    failure
    Failure,
    User authorization failure,
    Login failed,
    INVALID
    Sorry,
    Sorry.
\end{verbatim}

\indvartype{login_non_failure_msgs}{set[string]} A set of strings
similar to {\tt login_failure_msgs} that if present mean that the
server text does not actually correspond to an authentication failure
(i.e., if {\tt login_failure_msgs} also matches, it's a false
positive).

Default: the strings
\begin{verbatim}
    Failures
    failures
    failure since last successful login
    failures since last successful login
\end{verbatim}

\indvartype{router_prompts}{set[string]} A set of strings
corresponding to prompts returned by the local routers when a
user successfully authenticates to the router.
For the purpose of this variable, see the next variable.

Default: empty.

\indvartype{login_success_msgs}{set[string]} A set of strings
that if seen in text sent by the server during the authentication dialog
correspond to a successful authentication attempt.

Default: the strings
\begin{verbatim}
    Last login
    Last successful login
    Last   successful login
    checking for disk quotas
    unsuccessful login attempts
    failure since last successful login
    failures since last successful login
\end{verbatim}
and the default contents of the {\tt router_prompts} variable.

\deficiency{ Since by default {\tt router_prompts} is empty, this
last inclusion does nothing.  In particular, if you redefine
{\tt router_prompts} then {\tt login_success_msgs} will \emph{not} 
pick up the change; you will need to redefine it to (again) include
{\tt router_prompts}, using: \mbox{\tt redef login_success_msgs += router_prompts}.
This is clearly a misfeature of Bro and will be fixed one fine day.
}

\indvartype{login_timeouts}{set[string]} A set of strings
that if seen in text sent by the server during the authentication dialog
correspond to the server having timed out the authentication attempt.

Default: the strings
\begin{verbatim}
    timeout
    timed out
    Timeout
    Timed out
    Error reading command input
\end{verbatim}
(This last is returned by the VMS operating system.)

\indvartype{non_ASCII_hosts}{set[addr]} A set of addresses
corresponding to hosts whose login servers do not (primarily) use
7-bit ASCII.  The analyzer will not attempt to analyze authentication
dialogs to such hosts, and will not complain about huge lines
generated by either the sender or receiver (per \xrefevent{excessive_line}).

Default: empty.

\indvartype{skip_logins_to}{set[addr]} A set of addresses
corresponding to hosts for which the analyzer should not attempt
to analyze authentication dialogs.

Default: the (empty) contents of {\tt non_ASCII_hosts}.

\indvartype{always_hot_login_ids}{set[string]} A set of usernames
that the analyzer should always flag as sensitive, even if they're seen in
a session for which the analyzer is \emph{confused} \pxref{login-confused}.

Default: the value of \xrefvar{always_hot_ids} defined by the
\xrefanalyzer{hot} analyzer.

\indvartype{hot_login_ids}{set[string]} A set of usernames
that the analyzer should flag as sensitive, unless it sees them
in a session for which the analyzer is \emph{confused} \pxref{login-confused}.

Default: the value of \xrefvar{hot_ids} defined by the
\xrefanalyzer{hot-ids} analyzer.

\index{rhosts@{\tt .rhosts}}
\indvartype{rlogin_id_okay_if_no_password_exposed}{set[string]}
A set of username exceptions to {\tt hot_login_ids} which the
analyzer should not flag as sensitive if the user authenticated without
exposing a password (so, for example, via {\tt .rhosts}).

Default: the username {\tt "root"}.

\end{description}

\index{analyzers!login!variables|)}

\subsection{{\tt login} functions}
\index{analyzers!login!functions|(}

The standard {\tt login} script provides the following functions for external use:

\begin{description}
\itemindfunc{is_login_conn}{(c: connection): bool }
Returns true if the given connection is one analyzed by {\tt login}
(currently, Telnet or Rlogin), false otherwise.

\itemindfunc{hot_login}{(c: connection, msg: string, tag: string) }
Marks the given connection as hot, logs the given message, and
demultiplexes \pxref{demux} the subsequent server-side contents of the
connection to a filename based on {\tt tag} and the client-side
to a filename based on the name {\tt "keys"}.  No return value.

\itemindfunc{is_hot_id}{(id: string, successful: bool, confused: bool): bool}
Returns true if the username id should be considered sensitive,
given that the user either did or did not successfully authenticate,
and that the analyze was or was not in a \emph{confused} state
\pxref{login-confused}.

\itemindfunc{is_forbidden_id}{(id: string): bool }
Returns true if the username id is present in \xrefvar{forbidden_ids}
or \xrefvar{forbidden_id_patterns}.

\itemindfunc{edit_and_check_line}{(c: connection, line: string, successful: bool): check_info}
Tests whether the given line of text seen on connection {\tt c} includes
a sensitive username, after first applying \emph{BS} and \emph{DEL}
keystroke editing \pxref{keystroke-editing}.  {\tt successful} should
be true if the user has successfully authenticated, false otherwise.

The return value is a {\tt check_info} record, which contains four
\indtt{check_info}{record}
fields:
\begin{description}
\indfield{expanded_line}{check_info}
All of the different editing interpretations of the line, separated
by commas.  For example, if the original line is
\begin{quotation}
{\tt "\verb+rob<+\emph{DEL}\verb+><+\emph{BS}\verb+><+\emph{BS}\verb+>ot+"}
\end{quotation}
then the different editing interpretations are
{\tt "\verb+ro<+\emph{BS}\verb+><+\emph{BS}\verb+>ot+"}
and \verb+"root"+, so the return value will be:
\begin{quotation}
{\tt "\verb+rob<+\emph{DEL}\verb+><+\emph{BS}\verb+><+\emph{BS}\verb+>ot+,\verb+ro<+\emph{BS}\verb+><+\emph{BS}\verb+>ot+,root"}
\end{quotation}

\deficiency{ Ideally, these values would be returned in a list of
some form, so that they can be accessed separately and unambiguously.
The current form is really suitable only for display to a person,
and even that can be quite confusing if {\tt line} happens to contain
commas already.  \emph{Or}, perhaps an algorithm of ``simply pick the
shortest'' would find the correct editing every time anyway.}

\item[{\tt hot: bool}]
\indttnotexttwo{hot}{check_info}{record}
True if any editing sequence resulted in a match against a sensitive username.

\item[{\tt hot_id: string}]
\indttnotexttwo{hot_id}{check_info}{record}
The version of the input line (with or without editing) that was considered
hot, or an empty string if none.

\item[{\tt forbidden: bool}]
\indttnotexttwo{forbidden}{check_info}{record}
True if any editing sequence resulted in a match against a username considered
``forbidden'', per \xreffunc{is_forbidden_id}.

\end{description}

\itemindfunc{edit_and_check_user}{(c: connection, user: string, successful: bool, fmt_s: string): bool}
Tests whether the given username used for authentication on connection {\tt c}
is sensitive, after first applying \emph{BS} and \emph{DEL}
keystroke editing \pxref{keystroke-editing}.  {\tt successful} should be
true if the user has successfully authenticated, false otherwise.

{\tt fmt_s} is a \xreffunc{fmt} format specifying how the username
information should be included in the connection's \xreffield{addl}{connection}
field.  It takes two {\tt string} parameters, the current value of the
field and the expanded version of the username as described in
\xreffield{expanded_line}{check-info}.

If \xreffunc{edit_and_check_line} indicates that the username is sensitive,
then {\tt edit_and_check_user} records the connection into its own
demultiplexing files \pxref{demux}.  If the username is \emph{forbidden},
then unless the analyzer is confused, we attempt to terminate the
connection using \xreffunc{terminate_connection}.

\index{connection!hot}
\index{hot connections}
Returns true if the connection is now considered ``hot,'' either
due to having a sensitive username, or because it was hot upon
entry to the function.

\index{connection!hot}
\index{hot connections}
\itemindfunc{edit_and_check_password}{(c: connection, password: string): bool}
Checks the given password to see whether it contains a sensitive username.
If so, then marks the connection as hot and logs the sensitive password.
No return value.

\note{ The purpose of this function is to catch instances in which the
event engine becomes out of synch with the authentication dialog and mistakes
what is, in fact, a username being entered, for a password being entered.
Such confusion can come about either due to a failure of the event
engine's heuristics, or due to deliberate manipulation of the event
engine by an attacker. }

\end{description}

\index{analyzers!login!functions|)}

\subsection{{\tt login} event handlers}
\index{analyzers!login!event handlers|(}

The standard {\tt login} script handles the following events:

\index{rhosts@{\tt .rhosts}}
\begin{description}
\indeventtype{login_failure}{c: connection, user: string, client_user: string, password: string, line: string}
Invoked when the event engine has seen a failed attempt to authenticate
as {\tt user} with {\tt password} on the given connection {\tt c}.
{\tt client_user} is the user's username on the client side of the
connection.  For Telnet connections, this is an empty string, but
for Rlogin connections, it is the client name passed in the initial
authentication information (to check against
{\tt .rhosts}).  {\tt line} is the
line of text that led the analyzer to conclude that the authentication
had failed.

The analyzer first generates an \xrefevent{account_tried}
event to facilitate detection of password guessing, and then checks for
a sensitive username or password.  If the username was not sensitive
and the password is empty, then no further analysis is applied, since
clearly the attempt was half-hearted and aborted.  Otherwise, the
analyzer annotates the connection's \xreffield{addl}{connection}
field with {\tt fail/\verb+<+\emph{username}\verb+>+} to mark the
authentication failure, and also checks the {\tt client_user} to
see if it is sensitive.  If we then find that the connection is
hot, the analyzer logs a message to that effect.

\indeventtype{login_success}{c: connection, user: string, client_user: string, password: string, line: string}
Invoked when the event engine has seen a successful attempt to authenticate.
The parameters are the same as for {\tt login_failure}.

The analyzer invokes \xreffunc{check_hot} with mode {\tt APPL_ESTABLISHED}
since the application session has now been established.  It generates
an \xrefevent{account_tried}
event to facilitate detection of password guessing, and then checks for
a sensitive username or password.  The event engine uses the special
password {\tt "\verb+<+none\verb+>+"} to indicate that no password
was exposed, and this mitigates the sensitivity of logins using particular
usernames per \xrefvar{rlogin_id_okay_if_no_password_exposed}.

The analyzer annotates the connection's \xreffield{addl}{connection}
field with {\tt "\verb+<+\emph{username}\verb+>+"} to mark the
successful authentication.  Finally, if we then find that the connection
is hot, the analyzer logs a message to that effect.

\indeventtype{login_input_line}{c: connection, line: string}
Invoked for every line of text sent by the client side of the login
session to the server side.  The analyzer matches the text against
\xrefvar{input_trouble} and \xrefvar{edited_input_trouble} and invokes
\xreffunc{hot_login} with a tag of {\tt "trb"} if it sees a match,
which will log an alert concerning the connection.  However, this
invocation is only done while the connection's \xreffield{hot}{connection}
field count is $\le 2$, to avoid cascaded alerts when an attacker gets
really busy and steps on a lot of sensitive patterns.

\indeventtype{login_output_line}{c: connection, line: string}
Invoked for every line of text sent by the server side of the login
session to the client side.  The analyzer checks
\xrefvar{backdoor_prompts} and any pending input alerts that
were waiting on the server output, per \xrefvar{edited_input_trouble}.
These last are then logged unless the output matched the pattern:
\begin{verbatim}
    /No such file or directory/
\end{verbatim}
\deficiency{ Clearly, this pattern should not be hardwired but instead
specified by a redefinable variable. }

Finally, if the line is not too long and the text matches
\xrefvar{output_trouble} and the connection's \xreffield{hot}{connection}
field count is $\le 2$ (to avoid cascaded alerts), the analyzer
invokes \xreffunc{hot_login} with a tag of {\tt "trb"}.
\deficiency{ ``Too long'' is hardwired to be a length $\ge 256$ bytes.
It, too, should be specifiable via a redefinable variable. } \note{ We might
wonder if not checking overly long lines presents an evasion threat: the
attacker can bury their access to a sensitive string in an excessive line
and thus avoid detection.  While this is true, it doesn't appear to cost
much.  First, some of the sensitive patterns are generated in server output
that will be hard to manipulate into being overly long.  Second, if the
attacker is trying to avoid detection, there are easier ways, such as
passing their output through a filter that alters it a good deal. }

\indeventtype{login_confused}{c: connection, msg: string, line: string}
Invoked when the event engine's heuristics have concluded that they
have become confused and can no longer correctly track the authentication
dialog \pxref{login-confused}.
{\tt msg} gives the particular problem the heuristics detected
(for example, {\tt multiple_login_prompts} means that the engine saw
several login prompts in a row, without the type-ahead from the client side
presumed necessary to cause them) and {\tt line} the line of text that
caused the heuristics to conclude they were confused.

Once declaring that it's confused, the event engine will no longer attempt
to follow the authentication dialog.  In particular, it will \emph{not}
generate subsequent {\tt login_failure} or {\tt login_success} events.

Upon this event, the standard
{\tt login} script invokes \xreffunc{check_hot} with
mode {\tt APPL_ESTABLISHED} since it could well be that the application
session is now established (it can't know for sure, of course, because
the event engine has given up).  It annotates the connection's
\xreffield{addl}{connection} field with
\indtt{confused/}{authentication annotation}\verb+<+\emph{line}\verb+>+ to mark the confused state,
and then logs to the \xreflog{weird} file the particulars of the
connection and the type of confusion ({\tt msg}).  \deficiency{ This should
be done by generating a \emph{weird}-related event instead. }

Finally, the analyzer invokes \xreffunc{set_record_packets} to specify
that all of the packets associated with this connection should be recorded
to the \xreflog{trace} file.
\note{ For the current {\tt login} analyzer, this call is not needed---it
records every packet of every login session anyway, because the generally
philosophy is that Bro should record whatever it analyzes, so that the
analysis may be repeated or examined in detail.  Since the current analyzer
looks at every input and output line via {\tt login_input} and {\tt
login_output}, it records all of the packets of every such analyzed session.
There is commented-out text in {\tt login_success} to be used if
{\tt login_input} and {\tt login_output} are not being used; it turns
off recording of a session's packets after the user has successfully logged
in (assuming the connection is not considered hot). }

\indeventtype{login_confused_text}{c: connection, line: string}
Invoked for every line the user types after the event engine has 
entered the \emph{confused} state.  If the connection is not already
considered hot, then the analyzer checks for the presence of sensitive
usernames in the line using \xreffunc{edit_and_check_line}, and, if
present, annotates the connection's \xreffield{addl}{connection} field
with \indtt{confused/}{authentication annotation}\verb+<+\emph{line}\verb+>+, logs that the connection
has become hot, and invokes \xreffunc{set_record_packets} to record
to the \xreflog{trace} file all of the packets associated with the connection.

\indeventtype{login_terminal}{c: connection, terminal: string}
Invoked when the client transmits a terminal type to the server.
The mechanism by which the client transmits the type depends on the
underlying protocol (Rlogin or Telnet).

The handler checks the terminal type against \xrefvar{hot_terminal_types}
and if it finds a match invokes \xreffunc{hot_login} with a tag of
{\tt "trb"}.

\index{tunneling}
\index{evasion!using tunneling}
\index{Napster!tunneled over Telnet or Rlogin}
\index{excessively long lines}
\indeventtype{excessive_line}{c: connection}
Invoked when the event engine observes a very long line sent by either
the client or the server.  Such long lines are seen as potential attempts
by an attacker to evade the {\tt login} analyzer; or, possibly, as
a Login session carrying an unusual application.  \note{ One example
we have observed occurs when a high-bandwidth binary payload protocol such
as Napster is sent over the Telnet or Rlogin well-known port in an
attempt to either evade detection or tunnel through a firewall. }

\index{Network Virtual Terminal (NVT)}
\index{NVT (Network Virtual Terminal)}
This event is actually generic to any TCP connection carrying
an application that uses the ``Network Virtual Terminal'' (NVT) abstraction,
which presently comprises Telnet and FTP.  But the only handler defined
in the demonstration Bro policy is for Telnet, hence we discuss it here.
For this reason, the handler first invokes \xreffunc{is_login_conn}
to check whether the connection is in fact a login session.  If so, then
if the connection is not hot, and if the analyzer finds the server
listed in \xrefvar{non_ASCII_hosts}, then it presumes the long line
is due to use of a non-ASCII character set; the analyzer invokes
\xreffunc{set_login_state} and \xreffunc{set_record_packets} to avoid
further analysis or recording of the connection.

Otherwise, if the connection is still in the authentication dialog, then
the handler generates a \xrefevent{login_confused} event with a
confusion-type of {\tt "excessive_line"}, and changes the connection's
state to \emph{confused}.

\deficiency{ The event engine is currently hardwired to consider a
line of $\ge 1024$ bytes as ``excessive''; clearly this should be
user-redefinable. }

\index{NVT options!inconsistent}
\index{Telnet!options!inconsistent}
\indeventtype{inconsistent_option}{c: connection}
NVT options are specified by the client and server stating which options
they are willing to support vs.\ which not, and then instructing one another
which in fact they should or should not use for the current connection.
If the event engine sees a peer violate either what the other peer has
instructed it to do, or what it itself offered in terms of options in 
the past, then the engine generates an {\tt inconsistent_option} event.

The handler for this event simply records an entry about it to the
\xreflog{weird} file.  \deficiency{ The event handler invocation does not
include enough information to determine what option was inconsistently
specified; in addition, it would be convenient to integrate the handling
of problems like this within the general ``weird'' framework. }

\note{ As for {\tt excessive_line} above, this event is actually a
generic one applicable to any NVT-based protocol.  It is handled here
because the problem most often crops up for Telnet sessions. }
\note{ Also, the handler does not check to see whether the connection
is a login session (as it does for {\tt excessive_line}); it serves
as the handler for any NVT session with an excessive line. }

\note{ Finally, note that this event can be generated if the session
contains a stream of binary data.  One way this can occur is when
the session is encrypted but Bro fails to recognize this fact. }
\index{encryption!leading to ``excessive lines''}

\index{NVT options!bad}
\index{Telnet!options!bad}
\indeventtype{bad_option}{c: connection}
If an NVT option is either ill-formed (e.g., a bad length field) or
unrecognized, then the analyzer generates this event.

The processing of this event (recording information to the \xreflog{weird}
file) and the various notes and deficiencies associated with it are
the same as those for {\tt inconsistent_option} above.

\index{NVT options!bad termination}
\index{Telnet!options!bad termination}
\indeventtype{bad_option_termination}{c: connection}
If an NVT option fails to be terminated correctly (for example,
a character is seen within the option that is disallowed for use
in the option), then the analyzer generates this event.

The processing of this event (recording information to the \xreflog{weird}
file) and the various notes and deficiencies associated with it are
the same as those for {\tt inconsistent_option} above.

\index{NVT options!authentication}
\index{Telnet!options!authentication}
\index{authentication!accepted}
\indeventtype{authentication_accepted}{name: string, c: connection}
The NVT framework includes options for negotiating authentication.
When such an option is sent from client to server and the server
replies that it accepts the authentication, then the event engine
generates this event.

The handler annotates the connection's \xreffield{addl}{connection} field
with \indtt{auth/}{authentication annotation}\verb+<+\emph{name}\verb+>+, unless that annotation is
already present.

\index{NVT options!authentication}
\index{Telnet!options!authentication}
\index{authentication!rejected}
\indeventtype{authentication_rejected}{name: string, c: connection}
The same as {\tt authentication_accepted}, except invoked when the
server replies that it rejects the attempted authentication.

The handler annotates the connection's \xreffield{addl}{connection} field
with \indtt{auth-failed/}{authentication annotation}\verb+<+\emph{name}\verb+>+.

\index{authentication!skipped}
\indeventtype{authentication_skipped}{c: connection}
Invoked when the event engine sees a line in the authentication dialog
that matches \xrefvar{skip_authentication}.

The handler annotates the connection's \xreffield{addl}{connection} field
with ``\indttparen{skipped}{authentication annotation}''
to mark that authentication was skipped,
and then invokes \xreffunc{skip_further_processing} and (unless the
connection is hot) \xreffunc{set_record_packets} to skip any further
analysis of the connection, and to stop recording its packets to
the \xreflog{trace} file.

\indeventtypenolabel{connection_established}{c: connection}
\xrefevent{connection_established} is a generic
event generated for all TCP connections; however, the {\tt login} analyzer
defines an additional handler for it.

The handler first checks (via \xreffunc{is_login_conn}) whether this is a Telnet
or Rlogin connection.  If so, it generates an
\xrefevent{authentication_skipped} event if the server's address occurs
in \xrefvar{skip_logins_to}, and also (for Telnet) checks whether the
client's port occurs in \xrefvar{hot_telnet_orig_ports}, invoking
\xreffunc{hot_login} with the tag {\tt "orig"} if it does.

For SSH connections, it likewise checks the client's port, but
in \xrefvar{hot_ssh_orig_ports}, marking the connection as hot and
logging a real-time alert if it is.

\indeventtypenolabel{partial_connection}{c: connection}
As noted earlier, \xrefevent{partial_connection} is a generic
event generated for all TCP connections.  The {\tt login} analyzer
also defines a handler for it, one which (if it's a Telnet/Rlogin
connection) sets the connection's state to \emph{confused} and
checks for \xrefvar{hot_telnet_orig_ports}.

\index{NVT options!encryption}
\index{Telnet!options!encryption}
\index{encrypted login sessions}
\indeventtype{activating_encryption}{c: connection}
The NVT framework includes options for negotiating encryption.  When such
a series of options is successfully negotiated, the event engine generates
this event.  \note{ The negotiation sequence is complex and can fail
at a number of points.  The event engine does not attempt to generate
events for each possible failure, but instead only looks for the
option sent after a successful negotiation sequence. }

The handler annotates the connection's \xreffield{addl}{connection} field
with ``{\tt (encrypted)}'' to mark that authentication was encrypted.
\note{ The event engine itself marks the connection as requiring no
further processing.  This is done by the event engine rather than the
handler because the event engine cannot do its job (regardless of the
policy the handler might desire) in the face of encryption. }

\end{description}

\index{analyzers!login!event handlers|)}

\labsectsection{{The \analyzer{portmapper} Analyzer}}{\label{analyzer-portmapper}}
\index{remote procedure call (RPC)}
\index{RPC (Remote Procedure Call)}
The \indanalyzer{portmapper} analyzer monitors one particularly
important form of remote procedure call (RPC) \cite{rfc-rpc,rfc-xdr}
traffic: the portmapper service, used to map between RPC program (and
version) numbers and the TCP or UDP port on which the service runs for a
particular host.  For example, \emph{rstatd} is an RPC service that provides
``remote host status monitoring'' so that a set of hosts can be informed
when any of them reboots.  \emph{rstatd} has been assigned a standard
RPC program number of 100002.  To find out the corresponding TCP or UDP
port on a given host, a remote host would usually first contact the
portmapper RPC service running on the host and request the port
corresponding to program 100002.

\begin{table}
\label{tbl:portmapper}
\begin{center}
% \begin{tabular}{|l|p{5.0in}|} 
\begin{tabular}{|l|l|} 
\hline
{\bf Call} & {\bf Meaning} \\
\hline
\hline
\indplain{NULL}{portmapper call}
		& A do-nothing call typically provided by all RPC services. \\
\hline
\indplain{GETPORT}{portmapper call}
		& Look up the port associated with a given RPC program. \\
\hline
\indplain{SET}{portmapper call}
		& Add a new port mapping (or replace an existing mapping)
		  for an RPC program. \\
\hline
\indplain{UNSET}{portmapper call}
		& Remove a port mapping. \\
\hline
\indplain{DUMP}{portmapper call}
		& Retrieve all of the RPC program mappings. \\
\hline
\indplain{CALLIT}{portmapper call}
		& Both look up a program and then directly call it. \\
\hline
\end{tabular}
\end{center}
\caption{
Types of calls to the RPC portmapper service.
}
\end{table}

All in all, clients can make six different types of calls to the portmapper,
as summarized in Table~\ref{tbl:portmapper}.  Attackers often use
GETPORT and DUMP to see whether a host may be running an RPC service
vulnerable to a known exploit.

The analyzer uses a capture filter of ``{\tt port 111}'' \pxref{capture-filter},
equivalent to ``{\tt tcp port 111 or udp port 111}'' (since the portmapper
service ordinarily accepts calls using either TCP or UDP, both on port~111).
It checks the different types of portmapper calls against policies
expressed using a number of different variables.

\note{ An important point not to overlook is that an attacker does
\emph{not} have to first call the portmapper service in order to
call an RPC program.  They might instead happen to know the port
on which the service runs \emph{a priori}, since for example it may
generally run on the same port for a particular operating system;
or they might scan the host's different TCP or UDP ports directly
looking for a reply from the service.  Thus, while portmapper monitoring
proves very useful in detecting attacks, it does \emph{not} provide
comprehensive monitoring of attempts to exploit RPC services. }

\subsection{{\tt portmapper} variables}
\index{analyzers!portmapper!variables|(}

The standard script provides the following redefinable variables:

\begin{description}
\indvartype{rpc_programs}{table[count] of string}
Maps RPC program numbers to a string used to name the service.
For example, the {\tt [100002]} entry is mapped to {\tt "rstatd"}.

Default: a large list of RPC services.

\index{NFS (Network File System)}
\index{Network File System (NFS)}
\indvartype{NFS_services}{set of string}
Lists the names of those RPC services that correspond to
Network File System (NFS) \cite{RFC-NFS2,RFC-NFS3} services.  This
variable is provided because it is convenient to express policies
specific to accessing NFS file systems.

Default: the services \emph{mountd}, \emph{nfs}, \emph{pcnfsd},
\emph{nlockmgr}, \emph{rquotad}, \emph{status}.

\deficiency{ Bro's notion of NFS is currently confined to just knowledge
of the existence of these services.  It does not analyze the particulars
of different NFS operations. }

\indvartype{RPC_okay}{set[addr, addr, string]}
Indexed by the host providing a given service and then by the host
accessing the service.  If an entry is present, it means that the 
given access is allowed.  For example, an entry of:
\begin{verbatim}
    [1.2.3.4, 5.6.7.8, "rstatd"]
\end{verbatim}
means that host {\tt 5.6.7.8} is allowed to access the \emph{rstatd}
service on host {\tt 1.2.3.4}.

Default: empty.

\indvartype{RPC_okay_nets}{set[net]}
A set of networks allowed to make GETPORT requests without complaint.
The notion behind providing this variable is that the listed
networks are trusted.  However, the trust doesn't extend beyond
GETPORT to other portmapper requests, because GETPORT is the only
portmapper operation used routinely by a set of hosts trusted by
another set of hosts (but that don't belong to the same group, and hence
are not issuing SET and UNSET calls).

Default: empty.

\index{walld@\emph{walld}}
\indvartype{RPC_okay_services}{set[string]}
A set of services for which GETPORT requests should not generate
complaints.  These might be services that are widely invoked and
believed exploit-free, such as \emph{walld}, though care should
be taken with blithely assuming that a given service is indeed
exploit-free.

Note that, like for {\tt RPC_okay_nets}, the trust does not
extend beyond GETPORT, because it should be the only portmapper
operation routinely invoked.

Default: empty.

\indvartype{NFS_world_servers}{set[addr]}
A set of hosts that provide public access to an NFS file system,
and thus should not have any of their NFS traffic flagged as
possibly sensitive.  (The presumption here is that such public
servers have been carefully secured against any remote NFS operations.)
An example of such a server might be one providing read-only
access to a public database.

Default: empty.

\indvartype{RPC_dump_okay}{set[addr, addr]}
Indexed first by the host requesting a portmapper dump, and second
by the host from which it's requesting the dump.  If an entry is
present, then the dump operation is not flagged.

Default: empty.

\indvartype{any_RPC_okay}{set[addr, string]}
Pairs of hosts and services for which any GETPORT access to the given
service is allowed.

\index{ypserv@\emph{ypserv}}
\indtt{sun-rpc.mcast.net}{reserved multicast address}
\index{RPC (Remote Procedure Call)!reserved multicast address}
Default:
\begin{verbatim}
    [NFS_world_servers, NFS_services],
    [sun-rpc.mcast.net, "ypserv"]
\end{verbatim}
The first of these allows access to any NFS service of any of the
{\tt NFS_world_servers}, using Bro's cross-product initialization
feature \pxref{cross-product-init}.  The second allows \emph{ypserv}
requests to the multicast address reserved for RPC multicasts.\footnote{
I don't know how much this type of access is actually used in practice, but
experience shows that requests for \emph{ypserv} directed to that address pop
up not infrequently. }

\index{walld@\emph{walld}}
\indvartype{suppress_pm_log}{table[addr, string] of bool}
Do not generate real-time alerts for access by the given address
for the given service.  Note that unlike most Bro policy variables,
this one is not {\tt const} but is modified at run-time to add
to it any host that invokes the \emph{walld} RPC service, so that
such access is only reported once for each host.

Default: empty, but dynamic as discussed above.

\end{description}

\index{analyzers!portmapper!variables|)}

\subsection{{\tt portmapper} functions}
\index{analyzers!portmapper!functions|(}

The standard script provides the following externally accessible functions:

\begin{description}
\itemindfunc{rpc_prog}{(p: count): string }
Returns the name of the RPC program with the given number,
if it's present in \xrefvar{rpc_programs}; otherwise returns
the text {\tt "unknown-\verb+<+\emph{p}\verb+>+"}.

\itemindfunc{pm_check_getport}{(r: connection, prog: string): bool }
Checks a GETPORT request for the given program against the policy expressed
by \xrefvar{RPC_okay_services}, \xrefvar{any_RPC_okay}, \xrefvar{RPC_okay},
and \xrefvar{RPC_okay_nets}, returning true if the request violates
policy, false if it's allowed.

\itemindfunc{pm_activity}{(r: connection, log_it: bool) }
A bookkeeping function invoked when there's been portmapper activity 
on the given connection.

The function records the connection via \xreffunc{record_connection},
unless it is a TCP connection (which will instead be recorded by
\xrefevent{connection_finished}).  If {\tt log_it} is true then the
function generates a real-time alert of the form:
\begin{quote}
    rpc:
    \verb+<+\emph{connection-id}\verb+>+
    \verb+<+\emph{RPC-service}\verb+>+
    \verb+<+\emph{r\$addl}\verb+>+
\end{quote}
For example:
\begin{verbatim}
    972616255.679799 rpc: 65.174.102.21/832 > 182.7.9.47/portmapper
	pm_getport: nfs -> 2049/udp
\end{verbatim}
However, it does not generate the alert if either the client host and
service are present in \xrefvar{suppress_pm_log}, or if it already generated
an alert in the past for the same client, server and service (to prevent
alert cascades).

\itemindfunc{pm_request}{(r: connection, proc: string, addl: string, log_it: bool) }
Invoked when the given connection has made a portmapper request of some
sort for the given RPC procedure {\tt proc}.  {\tt addl} gives an
annotation to add to the connection's \xreffield{addl}{connection} field.
If {\tt log_it} is true, then connection should be logged; it will also
be logged if the function determines that it is hot.

The function first invokes \xreffunc{check_scan} and \xreffunc{check_hot}
(with a mode of {\tt CONN_ESTABLISHED}),
unless {\tt r} is a TCP connection, in which case these checks have already
been made by \xrefevent{connection_established}.  The function then adds
{\tt addl} to the connection's {\tt addl} field, though if the field's
length already exceeds 80 bytes, then it just tacks on {\tt "..."}
(unless already present).  This last is necessary because Bro will sometimes
see zillions of successive portmapper requests that all use the same
connection ID, and these will each add to {\tt addl} until it
becomes unwieldy in size.  \deficiency{ Clearly, the byte limit of 80 should
be adjustable. }

Finally, the function invokes \xreffunc{check_hot} with a mode
of {\tt CONN_FINISHED}, and \xreffunc{pm_activity} to finish up
bookkeeping for the connection.

No return value.

\itemindfunc{pm_attempt}{(r: connection, proc: string, status: count, addl: string, log_it: bool) }
Invoked when the given connection attempted to make a portmapper request
of some sort, but the request failed or went unanswered.  The arguments
are the same as for \xreffunc{pm_request}, with the addition of
{\tt status}, which gives the RPC status code corresponding to why the
attempt failed (see below).

The function first invokes \xreffunc{check_scan} and \xreffunc{check_hot}
(with a mode of {\tt CONN_ATTEMPTED}),
unless {\tt r} is a TCP connection, in which case these checks have already
been made by \xrefevent{connection_attempt}.

\begin{table}
\label{tbl:portmapper-status}
\begin{center}
\begin{tabular}{|l|p{5.0in}|} 
\hline
{\bf Status description} & {\bf Meaning} \\
\hline
\hline
\indextext{ok}{RPC status code}
	& The call succeeded. \\
\hline
\indextext{prog unavail}{RPC status code}
	& The call was for an RPC program that
	has not registered with the portmapper. \\
\hline
\indextext{mismatch}{RPC status code}
	& The call was for a version of the RPC program that
	has not registered with the portmapper. \\
\hline
\indextext{garbage args}{RPC status code}
	& The parameters in the call did not decode correctly. \\
\hline
\indextext{system err}{RPC status code}
	& A system error (such as out-of-memory) occurred when
	processing the call. \\
\hline
\indextext{timeout}{RPC status code}
	& No reply was received within 24~seconds of the request. \\
\hline
\indextext{auth error}{RPC status code}
	& The caller failed to authenticate to the server,
	or was not authorized to make the call. \\
\hline
\indextext{unknown}{RPC status code}
	& An unknown error occurred. \\
\hline
\end{tabular}
\end{center}
\caption{
Types of RPC status codes.
}
\end{table}

The function then adds
{\tt addl} to the connection's {\tt addl} field, along with
a text description of the RPC status code, as given in
Table~\ref{tbl:portmapper-status}.

No return value.

\end{description}

\index{analyzers!portmapper!functions|)}

\subsection{{\tt portmapper} event handlers}
\index{analyzers!portmapper!event handlers|(}

The standard script handles the following events:

\begin{description}
\indeventtype{pm_request_null}{r: connection}
Invoked upon a successful portmapper request for the ``null'' procedure.
The script invokes {\tt pm_request} with {\tt log_it=F}.

\indeventtype{pm_request_set}{r: connection, m: pm_mapping, success: bool}
Invoked upon a nominally successful portmapper request to set the portmapper
binding {\tt m}.  The script invokes {\tt pm_request} with {\tt log_it=T}.
{\tt success} is true if the server honored the request, false otherwise;
the script turns this into an annotation of {\tt "ok"} or {\tt "failed"}.

The {\tt pm_mapping} type (for {\tt m}) has three fields,
{\tt program: count}, {\tt version: count} and {\tt p: port}, the
port for the mapping of the given program and version.
\indtt{pm_mapping}{portmapper mapping record}

\indeventtype{pm_request_unset}{r: connection, m: pm_mapping, success: bool}
Invoked upon a nominally successful portmapper request to remove a portmapper
binding.  The script invokes {\tt pm_request} with {\tt log_it=T}.
{\tt success} is true if the server honored the request, false otherwise;
the script turns this into an annotation of {\tt "ok"} or {\tt "failed"}.

\indeventtype{pm_request_getport}{r: connection, pr: pm_port_request, p: port}
Invoked upon a successful portmapper request to look up a portmapper
binding.  {\tt pr}, of type
\indtt{pm_port_request}{portmapper request}, has three fields:
{\tt program: count}, {\tt version: count}, and {\tt is_tcp: bool},
this last indicating whether the caller is request the TCP or UDP
port, if the given program/version has mappings for both.
The script invokes {\tt pm_request} with {\tt log_it} set
according to the return value of \xreffunc{pm_check_getport}
and an annotation of the mapping.

\indeventtype{pm_request_dump}{r: connection, m: pm_mappings}
Invoked upon a successful portmapper request to dump the portmapper
bindings.  The script invokes {\tt pm_request} with {\tt log_it=T}
unless \xrefvar{RPC_dump_okay} indicates that the dump call is allowed.
The script ignores {\tt m}, which gives the mappings as a
{\tt table[count] of pm_mapping}, where the table index simply reflects
the order in which the mappings were returned, starting with an index
of~1.  \deficiency{ What the script \emph{should} do, instead, is
keep track of the mappings so that Bro can identify the service associated
with connections for otherwise unknown ports. }

\index{walld@\emph{walld}}
\indeventtype{pm_request_callit}{r: connection, pm_callit_request, p: port}
Invoked upon a successful portmapper request to look up and call
an RPC procedure.  The script invokes {\tt pm_request} with {\tt log_it=T}
unless the combination of the caller and the
program are in \xrefvar{suppress_pm_log}.  Finally, if the program
called is \emph{walld}, then the script adds the caller to
\xrefvar{suppress_pm_log}.

The {\tt pm_callit_request} type has four fields:
\indtt{pm_callit_request}{portmapper call}
{\tt program: count}, {\tt version: count}, {\tt proc: count}, and
{\tt arg_size: count}.  These reflect the procedure being looked up and
called, and the size of the arguments being passed to it, respectively.
\deficiency{ Currently, the event engine does not do any analysis or
refinement of the arguments passed to the procedure (such as making
them available to the event handler) or the return value.}  {\tt p} is
the port value returned by the call.

\indeventtype{pm_attempt_null}{r: connection, status: count}
Invoked upon a failed portmapper request for the ``null'' procedure.
{\tt status} gives the reason for the failure.
The script invokes {\tt pm_attempt} with {\tt log_it=T}.

\indeventtype{pm_attempt_set}{r: connection, status: count, m: pm_mapping}
Invoked upon a failed portmapper request to set the portmapper
binding {\tt m}.  The script invokes {\tt pm_attempt} with {\tt log_it=T}.

\indeventtype{pm_attempt_unset}{r: connection, status: count, m: pm_mapping}
Invoked upon a failed portmapper request to remove a portmapper
binding.  The script invokes {\tt pm_attempt} with {\tt log_it=T}.

\indeventtype{pm_attempt_getport}{r: connection, status: count, pr: pm_port_request}
Invoked upon a failed portmapper request to look up a portmapper
binding.  {\tt pr}, of type {\tt pm_port_request}, has three fields:
{\tt program: count}, {\tt version: count}, and {\tt is_tcp: bool},
this last indicating whether the caller requested the TCP or UDP port.
The script invokes \indtt{pm_attempt}{portmapper attempt} with {\tt log_it} set
according to the return value of \xreffunc{pm_check_getport}.

\indeventtype{pm_attempt_dump}{r: connection, status: count}
Invoked upon a failed portmapper request to dump the portmapper
bindings.  The script invokes {\tt pm_attempt} with {\tt log_it=T}
unless \xrefvar{RPC_dump_okay} indicates that the dump call is allowed.

\index{walld@\emph{walld}}
\indeventtype{pm_attempt_callit}{r: connection, status: count, pm_callit_request}
Invoked upon a failed portmapper request to look up and call
an RPC procedure.  The script invokes {\tt pm_attempt} with {\tt log_it=T}
unless the combination of the caller and the
program are in \xrefvar{suppress_pm_log}.  Finally, if the program
called is \emph{walld}, then the script adds the caller to
\xrefvar{suppress_pm_log}.

\indeventtype{pm_bad_port}{r: connection, bad_p: count}
Invoked when a portmapper request or response includes an invalid
port number.  Since ports are represented by unsigned 4-byte integers,
they can stray outside the allowed range of 0--65535 by being $\ge$ 65536.
The script invokes \xrefevent{conn_weird_addl} with a \emph{weird tag}
of {\tt "bad_pm_port"}.

\end{description}

\index{analyzers!portmapper!event handlers|)}

\index{analyzers!application-specific|)}

\labsectsection{{The \analyzer{analy} Analyzer}}{\label{analyzer-analy}}
\index{statistical analysis}
\index{connection!analysis}
The \indanalyzer{analy} analyzer provides a limited mechanism to
use Bro to do statistical analysis on TCP connections.  Its primary
purpose is to demonstrate that Bro has applications to network
traffic analysis beyond intrusion detection.  It defines one
event handler:

\begin{table}
\label{tbl:endpoint-stats}
\begin{center}
\begin{tabular}{|l|p{5.0in}|} 
\hline
{\bf Field} & {\bf Meaning} \\
\hline
\hline
\indtt{num_pkts}{statistic} & The number of packets sent by the endpoint, as seen by
			the monitor.  The endpoint may have sent others
			that the network dropped upstream from the monitor. \\
\hline
\indtt{num_rxmit}{statistic}	& The number of packets retransmitted by the endpoint,
				as seen by the monitor.\\
\hline
\indtt{num_rxmit_bytes}{statistic}	& The number of bytes retransmitted by the endpoint.\\
\hline
\indtt{num_in_order}{statistic}	& The number of packets sent by the endpoint that
				arrived at the monitor in order, where
				``in order'' means in the same order
				as sent by the endpoint, rather than
				in sequence number.  (Thus, a retransmission
				can arrive in order, by this definition.)
				Bro determines if the packet arrived in
				\index{in-order delivery}
				\index{IP!identification field}
				order by applying heuristics to the IP
				identification (ID) field, which in general
				will increase by a small amount between
				successive packets transmitted by an
				endpoint. \\
\hline
\indtt{num_OO}{statistic}	& The number of packets sent by the endpoint that arrived
			at the monitor out of order.  See the previous
			\index{out-of-order delivery}
			entry for the definition of ``in order'',
			and hence ``out of order.'' \\
\hline
\indtt{num_repl}{statistic}	& The number of extra copies of packets
				\index{packets!replication}
				\index{replication of packets}
				sent by the endpoint that arrived at
				the monitor.  Bro considers a packet
				replicated if its IP ID field is the same
				as for the previous packet it saw from
				the endpoint.  Using this definition,
				a replication is most likely caused by
				a network mechanism such as duplication of
				a packet by a router, rather than a transport
				mechanism such as retransmission, though
				some TCPs fully reuse packets when
				retransmitting them, including their
				IP ID field. \\
\hline
\indtt{endian_type}{statistic}	& Whether the advance of the IP ID field
				\index{little endian}
				\index{big endian}
				\index{endian issues}
				as seen by the monitor was consistent with
				big-endian (network order) addition,
				little-endian, or undetermined. The
				three values are represented by the
				Bro constants \indtt{ENDIAN_BIG}{constant},
				\indtt{ENDIAN_LITTLE}{constant}, and
				\indtt{ENDIAN_UNKNOWN}{constant}.  In addition,
				the value can be \indtt{ENDIAN_CONFUSED}{constant},
				meaning that the monitor saw conflicting
				evidence for little- and big-endian. \\
\hline
\end{tabular}
\end{center}
\latexhtml{
\caption[ {\tt endpoint_stats} fields for summarizing connection endpoint statistics ]{ {\tt endpoint_stats} fields for summarizing connection endpoint statistics, all of type {\tt count}.  }}{
\caption{ {\tt endpoint_stats} fields for summarizing connection endpoint statistics, all of type {\tt count}.  }}
\end{table}

\begin{description}
\indeventtype{conn_stats}{c: connection, os: endpoint_stats, rs: endpoint_stats}
Invoked for each connection when it terminates (for whatever reason).
{\tt os} and {\tt rs} are the statistics for the originator endpoint and
the responder endpoint, respectively; Table~\ref{tbl:endpoint-stats} gives
the different record fields.

\end{description}

\section{The \module{weird} Module}
\index{weird events|(}
\index{events!exceptional|(}
\index{exceptional events|(}
\index{unusual events|(}

The \indmodule{weird} module processes unusual or exceptional
events.  A number of these ``shouldn't'' or even ``can't'' happen,
yet they do.  The general design philosophy of Bro is to check
for such events whenever possible, because they can reflect incorrect
assumptions (either Bro's or the user's), attempts by attackers to
confuse the monitor and evade detection, broken hardware, misconfigured
networks, and so on.

Weird events are divided into three categories, namely those pertaining
to: connections; flows (a pair of hosts, but for which a specific connection
cannot be identified); and network behavior (cannot be associated with a
pair of hosts).  These categories have a total of four event handlers:
\xrefevent{conn_weird}, \xrefevent{conn_weird_addl}, \xrefevent{flow_weird},
and \xrefevent{net_weird}, and in the corresponding sections below we
catalog the events handled by each.  In addition, we separately catalog
the events generated by the standard scripts themselves
\pxref{policy-script-events}.  Finally, two more weird events have their
own handlers, in order to associate detailed information with the event:
\xrefevent{rexmit_inconsistency} and \xrefevent{ack_above_hole}.

\index{weird event summary file@{\tt weird} event summary file}
\index{log file!weird events}
{\tt weird_file} is the logging file \label{weird-log} that
the module uses to record exceptional
events.  It defaults to {\tt open_log_file("weird")}.

\index{crud}
\index{unusual events!prevalence in actual network traffic}
\index{weird events!prevalence in actual network traffic}
\index{buggy implementations!causing ``weird'' events}
\index{diverse network use!causing ``weird'' events}
\index{Bro bugs/limitations!causing ``weird'' events}
\index{bugs!causing ``weird'' events}
\note{ While these events ``shouldn't'' happen, in reality they
often do.  For example, of the 73~listed below, a search of 10~months'
worth of logs at LBNL shows that 42~were seen operationally.  While
some of the instances reflect attacks, the great majority are simply
due to \emph{(i)}~buggy implementations, \emph{(ii)}~diverse use of
the network, or \emph{(iii)}~Bro bugs or limitations.  Accordingly,
you may initially be inclined to log each instance, but don't be surprised
to find that you soon decide to only record many of them in the {\tt weird}
file, or not record them at all.  (For further discussion, see the section
on ``crud'' in \cite{bro-comp-networks-99}.) }

\subsection{Actions for ``weird'' events}
\index{weird events!actions}

\begin{table}
\label{tbl:weird-action}
\begin{center}
\begin{tabular}{|l|p{5.0in}|} 
\hline
{\bf Action} & {\bf Meaning} \\
\hline
\hline
\indtt{WEIRD_UNSPECIFIED}{action} & No action specified. \\
\hline
\indtt{WEIRD_IGNORE}{action}	& Ignore the event.\\
\hline
\indtt{WEIRD_FILE}{action}	& Record the event to {\tt weird_file},
					if it has not been seen for
					these hosts before.  (But see
					\xrefvar{weird_do_not_ignore_repeats}.)\\
\hline
\indtt{WEIRD_LOG_ALWAYS}{action}	& Record the event to {\tt weird_file} and
					generate a real-time alert each
					time the event occurs. \\
\hline
\indtt{WEIRD_LOG_ONCE}{action}	& Record the event to {\tt weird_file};
					generate a real-time alert the first
					time the event occurs. \\
\hline
\indtt{WEIRD_LOG_PER_CONN}{action}	& Record the event to {\tt weird_file};
					generate a real-time alert the
					first time it occurs for a given
					connection. \\
\hline
\indtt{WEIRD_LOG_PER_ORIG}{action}	& Record the event to {\tt weird_file};
					generate a real-time alert the
					first time it occurs for a given
					originating host. \\
\hline
\end{tabular}
\end{center}
\caption{
Different types of possible actions to take for ``weird'' events.  }
\end{table}

The general approach taken by the module is to categorize for each event
the action to take when the event engine generates the event.
Table~\ref{tbl:weird-action} summarizes the different possible actions.

\subsection{{\tt weird} variables}

The standard {\tt weird} script provides the following redefinable variables:

\begin{description}
\indvartype{weird_action}{table[string] of count}
Maps different weird events to actions as given in Table~\ref{tbl:weird-action}.

Default: as specified in \xrefevent{conn_weird}, \xrefevent{conn_weird_addl},
\xrefevent{flow_weird}, \xrefevent{net_weird}, and
\xref{policy-script-events}.  As usual, you can change particular
values using refinement.  For example:
\begin{verbatim}
redef weird_action: table[string] of count += {
    [["bad_TCP_checksum", "bad_UDP_checksum"]] = WEIRD_IGNORE,
    ["fragment_overlap"] = WEIRD_LOG_PER_CONN,
};
\end{verbatim}
would specify to ignore TCP and UDP checksum errors (rather than the default
of {\tt WEIRD_FILE}), and to alert on fragment overlaps once per connection
in which they occur, rather than the default of {\tt WEIRD_LOG_ALWAYS}.

\indvartype{weird_action_filters}{table[string] of function(c: connection): count}
Indexed by the name of a weird event, yields a function that when called
for a given connection exhibiting the event, returns an action from
Table~\ref{tbl:weird-action}.  A return value of {\tt WEIRD_UNSPECIFIED} 
means ``no special action, use the action you normally would.''
This variable thus allows arbitrary
customization of the handling of particular events.

Default: empty, for the {\tt weird} analyzer itself.  The
\xrefanalyzer{portmapper} analyzer redefines this variable as follows:
\begin{verbatim}
    redef weird_action_filters += {
        [["bad_RPC", "excess_RPC", "multiple_RPCs", "partial_RPC"]] =
                RPC_weird_action_filter,
};
\end{verbatim}
where {\tt RPC_weird_action_filter} is a function internal to the
analyzer that returns {\tt WEIRD_FILE} if the originating host
is in \xrefvar{RPC_okay_nets}, and {\tt WEIRD_UNSPECIFIED} otherwise.

\indvartype{weird_ignore_host}{set[addr, string]}
Specifies that the analyzer should ignore the given weird event (named by
the second index) if it involves the given address (as either originator
or responder host).

Default: empty.

\indvartype{weird_do_not_ignore_repeats}{set[string]}
Gives a set of weird events that, if their action is {\tt WEIRD_FILE},
should still be recorded to the {\tt weird_file} each time they occur.

Default: the events relating to checksum errors, i.e.,
{\tt "bad_IP_checksum"},
{\tt "bad_TCP_checksum"},
{\tt "bad_UDP_checksum"}, and
{\tt "bad_ICMP_checksum"}.
These are recorded multiple times because it can prove handy to
be able to track clusters of checksum errors.

\end{description}

\subsection{{\tt weird} functions}

The {\tt weird} analyzer includes the following functions:

\begin{description}
\itemindfunc{report_weird}{(t: time, name: string, id: string, action: count, no_log: bool)}
Processes an occurrence of the weird event {\tt name} associated with
the connection described by the string {\tt id} (which may be empty
if no connection is associated with the event).  {\tt action} is the
action associated with the event.  For {\tt report_weird}, the only
distinctions made between the different actions are that {\tt WEIRD_IGNORE}
causes the function to do nothing; any of {\tt WEIRD_LOG_{\emph{}XXX}}
cause the function to log a message, unless {\tt no_log} is true; and {\tt
WEIRD_UNSPECIFIED} causes the function to look up the action in
\xrefvar{weird_action}.  If the function does \emph{not} find an action
for the event, then it uses {\tt WEIRD_LOG_ALWAYS} and prepends the log
message with a pair of asterisks (``{\tt **}'') to flag that this event
does not have a specified action.

For {\tt WEIRD_FILE}, {\tt report_weird} only
records the event once to the file, unless the given event is present
in \xrefvar{weird_do_not_ignore_repeats}.  Events with loggable actions
are always recorded to {\tt weird_file}.

\itemindfunc{report_weird_conn}{(t: time, name: string, id: string, c: connection)}
Processes an occurrence of the weird event {\tt name} associated with
the connection {\tt c}, which is described by the string {\tt id}.

If {\tt report_weird_conn} finds one of the hosts and the given event name
in \xrefvar{weird_ignore_host}, then it does nothing.  Then, if the event
is in \xrefvar{weird_action}, then it looks up the event in
\xrefvar{weird_action_filters} and invokes the corresponding function
if present, otherwise taking the action from {\tt weird_action}.
It then implements the various flavors of {\tt WEIRD_LOG_\emph{XXX}}
by not logging events more than once per connection, originator host,
etc., though the events are still written to {\tt weird_file}.
Finally, the function invokes \xreffunc{report_weird} to do the
actual recording and/or writing to {\tt weird_file}.

\itemindfunc{report_weird_orig}{(t: time, name: string, id: string, orig: addr)}
Processes an occurrence of the weird event {\tt name} associated with
the source address {\tt orig}.  {\tt id} textually describes the flow from
{\tt orig} to the destination, for example using \xreffunc{endpoint_id}.

The function looks up the event name in \xrefvar{weird_action} and
passes it along to \xreffunc{report_weird}.

\end{description}

\subsection{Events handled by {\tt conn_weird}}
\index{weird events!handled by {\tt conn_weird}}
\index{event handling!weird|(}

\begin{description}
\indeventtype{conn_weird}{name: string, c: connection}
Invoked for most ``weird'' events.
{\tt name} is the name of the weird event, and {\tt c} is the
connection with which it's associated.

\end{description}

\noindent {\tt conn_weird} handles the following events, all of which have
a default action of {\tt WEIRD_FILE}:

\begin{description}
\indweird{active_connection_reuse}
	A new connection attempt (initial SYN)
	was seen for an already-established connection that has not
	yet terminated.
\index{HTTP!weird events}
\indweird{bad_HTTP_reply}
	The first line of a reply from an HTTP
	server did not include \verb+HTTP/+\emph{version}.
\indweird{bad_HTTP_version}
	The first line of a request from an HTTP
	client did not include \verb+HTTP/+\emph{version}.
\index{ICMP!weird events}
\index{packets!corrupted}
\index{corrupted packets}
\index{checksum error!ICMP}
\index{ICMP!checksum error}
\indweird{bad_ICMP_checksum}
	The checksum field in an
	ICMP packet was invalid.
\index{Rlogin!weird events}
\indweird{bad_rlogin_prolog}
	The beginning of an Rlogin connection had
	a syntactical error.
\index{RPC (Remote Procedure Call)!weird events}
\indweird{bad_RPC}
	A Remote Procedure Call was ill-formed.
\indweird{bad_RPC_program}
	A portmapper RPC call did not include the
	correct portmapper program number.
\indweird{bad_SYN_ack}
	A TCP SYN acknowledgment (SYN-ack) did not acknowledge
	the sequence number sent in the initial SYN.
\index{TCP!weird events}
\index{checksum error!TCP}
\index{TCP!checksum error}
\indweird{bad_TCP_checksum}
	A TCP packet had a bad checksum.
\index{UDP!weird events}
\index{checksum error!UDP}
\index{UDP!checksum error}
\indweird{bad_UDP_checksum}
	A UDP packet had a bad checksum.
\indweird{baroque_SYN}
	A TCP SYN was seen with an unlikely
	combination of other flags (the URGent pointer).
\indweird{blank_in_HTTP_request}
	The URL in an HTTP request includes
	an embedded blank.
\indweird{connection_originator_SYN_ack}
	A TCP endpoint that originated
	a connection by sending a SYN followed this up by sending a SYN-ack.
\indweird{data_after_reset}
	After a TCP endpoint sent a RST to terminate
	a connection, it sent some data.
\indweird{data_before_established}
	Before the connection was fully
	established, a TCP endpoint sent some data.
\indweird{excessive_RPC_len}
	An RPC record sent over a TCP connection
	exceeded 8 KB.
\indweird{excess_RPC}
	The sender of an RPC request or reply included
	leftover data beyond what the RPC parameters or result value
	themselves consumed.
\indweird{FIN_advanced_last_seq}
	A TCP endpoint retransmitted a FIN with
	a higher sequence number than previously.
\indweird{FIN_after_reset}
	A TCP endpoint sent a FIN after sending a RST.
\index{packets!storms}
\index{storms}
\indweird{FIN_storm}
	The monitor saw a flurry of FIN packets all sent on
	the same connection.  A ``flurry'' is defined as 1,000 packets that
	arrived with less than 1~sec between successive FINs.
	\deficiency{ Clearly, this numbers
	should be user-controllable. }
\indweird{HTTP_unknown_method}
	The method in an HTTP request was
	not GET, POST or HEAD.
\indweird{HTTP_version_mismatch}
	A persistent HTTP connection sent a
	different version number for a subsequent item than it
	did initially.
\indweird{inappropriate_FIN}
	A TCP endpoint sent a FIN before the
	connection was fully established.
\indweird{multiple_HTTP_request_elements}
	An HTTP request included multiple
	methods.
\indweird{multiple_RPCs}
	A TCP RPC stream included more than one
	remote procedure call.
\index{NULs}
\indweird{NUL_in_line}
	A NUL (ASCII 0) was seen in a text stream
	that is expected to be free of NULs.  \updateme{ Currently,
	the only such stream is that associated with an FTP control
	connection. }
\indweird{originator_RPC_reply}
	The originator (and hence presumed client)
	of an RPC connection sent an RPC reply (either instead of a request,
	or in addition to a request).
\index{Finger!weird events}
\indweird{partial_finger_request}
	When a Finger connection terminated, it
	included a final line of unanalyzed text because the text was
	not newline-terminated.
\index{FTP!weird events}
\indweird{partial_ftp_request}
	When an FTP connection terminated, it
	included a final line of unanalyzed text because the text was
	not newline-terminated.
\indweird{partial_ident_request}
	When an IDENT connection terminated, it
	included a final line of unanalyzed text because the text was
	not newline-terminated.
\indweird{partial_portmapper_request}
	A portmapper connection terminated with
	an unanalyzed request because the data stream was incomplete.
\indweird{partial_RPC}
	An RPC was missing some required header information
	due to truncation.
\index{data!unanalyzed}
\index{unanalyzed data}
\indweird{pending_data_when_closed}
	A TCP connection closed even though
	not all of the data in it was analyzed due to a sequence hole.
\index{split routing}
\index{routing!split}
\index{vantage point}
\index{bidirectional vs. unidirectional analysis}
\index{analysis!bidirectional vs. unidirectional}
\index{undirectional analysis}
\index{scanning!stealth}
\index{stealth scans}
\indweird{possible_split_routing}
	Bro appears to be seeing only one
	direction of some bi-directional connections \pxref{split-routing}.
	This can also occur due to certain forms of stealth-scanning.
\index{connection!reuse}
\index{Maximum Segment Lifetime (MSL)}
\index{MSL (Maximum Segment Lifetime)}
\indweird{premature_connection_reuse}
	A TCP connection tuple is being
	reused less than 30~sec after its previous use.  (The standard
	requires waiting $2 \cdot \mbox{MSL} = 4$ minutes \cite[p. 27]{RFC793}.)
\indweird{repeated_SYN_reply_wo_ack}
	A TCP responder that replied to an
	initial SYN with a SYN-ack has subsequently sent a SYN \emph{without}
	an acknowledgment.
\indweird{repeated_SYN_with_ack}
	A TCP originator that sent an
	initial SYN has subsequently sent a SYN-ack.
\indweird{responder_RPC_call}
	The responder (and hence presumed server)
	of an RPC connection sent an RPC request (either instead of a reply,
	or in addition to a reply).
\indweird{rlogin_text_after_rejected}
	An Rlogin client sent additional text
	to an Rlogin server after the server already presumably rejected
	the client's service request.
\index{retransmission!inconsistent}
\index{inconsistent retransmission}
\index{evasion!inconsistent RPC retransmission}
\indweird{RPC_rexmit_inconsistency}
	An RPC call was retransmitted, and
	the retransmitted call differed from the original call.  This
	could reflect an attempt by an attacker to evade the monitor.
	\note{ This type of inconsistency checking is not available for
	RPC replies because the transmission of the reply in general
	marks the end of the RPC connection, and the monitor deletes
	the connection state shortly afterward. }
\indweird{RST_storm}
	The monitor saw a flurry of RST packets all sent on
	the same connection.  See {\tt FIN_storm} for the definition of
	``flurry.''
\indweird{RST_with_data}
	A TCP RST packet included data.  This actually
	is allowed by the specification \cite[4.2.2.12]{RFC1122}.
	\deficiency{ This event should include the data. }
\index{simultaneous open}
\index{connection!simultaneous open}
	\indweird{simultaneous_open}
	The monitor saw a TCP simultaneous open,
	i.e., both endpoints sent initial SYNs to one another at the same time.
	While the specification allows this \cite[p. 30]{RFC793}, none of the
	protocols analyzed by Bro should be using it.
\index{transients!startup}
\index{startup!transients}
\index{scanning!stealth}
\index{stealth scans}
\indweird{spontaneous_FIN}
	A TCP endpoint sent a FIN packet without
	sending any previous packets.  This event can reflect stealth-scanning,
	but can also occur when Bro has recently
	started up and has not seen other traffic on a connection and hence does
	not know that the connection already exists.
\indweird{spontaneous_RST}
	A TCP endpoint sent a RST packet without
	sending any previous packets.  As with {\tt spontaneous_FIN}, this
	event can reflect either stealth scanning or a Bro start-up
	transient.
\indweird{SYN_after_close}
	A TCP endpoint sent a SYN (connection
	initiation) after sending a FIN (connection termination),
	but before the connection fully closed.
\indweird{SYN_after_partial}
	A TCP endpoint in a ``partial'' connection
	\pxref{partial-connection} sent a SYN.
\indweird{SYN_after_reset}
	A TCP endpoint sent a SYN after sending a
	RST (reset connection).
\indweird{SYN_inside_connection}
	A TCP endpoint sent a SYN during a
	connection (or partial connection) on which it had already
	sent data.
\indweird{SYN_seq_jump}
	A TCP endpoint retransmitted a SYN or a
	SYN-ack, but with a different sequence number.
\index{T/TCP}
\index{TCP!transaction}
\index{transaction TCP}
\indweird{SYN_with_data}
	A TCP endpoint included data in a SYN packet
	it sent.  Note, this can legitimately occur for T/TCP connections
	\cite{RFC1644}.
\index{TCP!Christmas packet}
\index{Christmas packet}
\indweird{TCP_christmas}
	A TCP endpoint sent a SYN packet that
	included the RST flag (a nonsensical combination).  The
	term ``Christmas packet'' has been used in this context
	(particularly if other flags are set, too) because the
	packet's flags are ``lit up like a Christmas tree.''
\index{length mismatch!UDP}
\index{UDP!length mismatch}
\index{evasion!length mismatch}
\indweird{UDP_datagram_length_mismatch}
	The length field in a UDP header
	did not match the length field in the IP header.  This could
	reflect an attempt by an attacker to evade the monitor.
\indweird{unpaired_RPC_response}
	An RPC reply was seen for which no
	request was seen.  This event could reflect a Bro start-up
	transient (it started running after the request was sent).
\indweird{unsolicited_SYN_response}
	A TCP endpoint sent a SYN-ack without
	first receiving an initial SYN.  This event could reflect a
	Bro start-up transient.

\end{description}

\subsection{Events handled by {\tt conn_weird_addl}}
\index{weird events!handled by {\tt conn_weird_addl}}

\index{polymorphic functions!need for}
\begin{description}
\indeventtype{conn_weird_addl}{name: string, c: connection, addl: string}
Invoked for a few ``weird'' events that require an extra (string)
argument to help clarify the event. \deficiency{ It would likely be very
handy if the general ``weird'' event handling was more flexible, with
the ability to have various parameters associated with the events.  Doing
so will likely have to wait on general Bro mechanism for dealing with
default parameters and/or polymorphic functions and event handlers. }

\end{description}

\noindent {\tt conn_weird_addl} handles the following events, all of which
have a default action of {\tt WEIRD_FILE}:

\begin{description}
\index{IDENT!weird events}
\indweird{bad_ident_reply}
	A reply from an IDENT server was
	syntactically invalid.
\indweird{bad_ident_request}
	A request to an IDENT server was
	syntactically invalid.
\indweird{ident_request_addendum}
	An IDENT request included additional
	text beyond that forming the request itself.

\end{description}

\subsection{Events handled by {\tt flow_weird}}
\index{weird events!handled by {\tt flow_weird}}

\begin{description}
\indeventtype{flow_weird}{name: string, src: addr, dst: addr}
is invoked for ``weird'' events that cannot be associated with a
particular connection, but only with a pair of hosts, corresponding
to a flow of packets from {\tt src} to {\tt dst}. Presently, all of
these events deal with fragments.

\end{description}

\noindent {\tt flow_weird} handles the following events:

\begin{description}
\index{IP!fragments}
\index{fragments!excessively large}
\index{denial of service!excessively large fragments}
\indweird{excessively_large_fragment}
	A set of IP fragments reassembled
	to a maximum size exceeding 64,000 bytes.  \note{ Sizes between
	64,000 and 65,535 bytes are allowed, strictly speaking, but
	are highly unlikely in legitimate traffic.  Sizes above 65,535
	bytes generally represent attempted denial-of-service attacks,
	due to IP implementations that crash upon receiving such
	impossibly-large fragment sets. }

	Default: {\tt WEIRD_LOG_ALWAYS}.

\index{fragments!excessively small}
\index{evasion!excessively small fragments}
\indweird{excessively_small_fragment}
	A fragment other than the
	last fragment in a set was less than 64 bytes in size.
	\note{ The standard allows such small fragments, but their
	presence may reflect an attacker attempting to evade the
	monitor by splitting header information across multiple
	fragments. }

	Default: {\tt WEIRD_LOG_ALWAYS}.

\index{fragments!inconsistent}
\index{evasion!inconsistent fragments}
\indweird{fragment_inconsistency}
	A fragment overlaps with a previously
	sent fragment, and the two disagree on data they share in common.
	This event could reflect an attacker attempting to evade the
	monitor; it can also occur because Bro keeps previous fragments
	indefinitely (\deficiency{ it needs to provide a means for
	flushing old fragments, otherwise it becomes vulnerable to
	a state-holding attack}), and occasionally a fragment will
	overlap with one sent much earlier and long-since forgotten
	by the endpoints.

	Default: {\tt WEIRD_LOG_ALWAYS}.

\index{fragments!overlapping}
\indweird{fragment_overlap}
	A fragment overlaps with a previously
	sent fragment.  As for {\tt fragment_inconsistency}, this
	event can occur due to Bro keeping previous fragments
	indefinitely.  This event does not in general reflect a
	possible attempt at evasion.

	Default: {\tt WEIRD_LOG_ALWAYS}.

\index{fragments!inconsistent protocols}
\indweird{fragment_protocol_inconsistency}
	Two fragments were seen
	for the same flow and IP ID which differed in their transport protocol
	(e.g., UDP, TCP).  According to the specification, this is allowed
	\cite[p. 24]{RFC791}, but its use appears highly unlikely.

	Default: {\tt WEIRD_FILE}, because it is difficult to see how
	an attacker can exploit this anomaly.

\index{fragments!inconsistent sizes}
\index{evasion!inconsistent fragment size}
\indweird{fragment_size_inconsistency}
	A ``last fragment'' was
	seen twice, and the two disagree on how large the reassembled datagram
	should be.  This event could reflect an attacker attempting to evade
	the monitor.

	Default: {\tt WEIRD_FILE}, since it is more likely that this
	occurs due to a high volume flow of fragments wrapping the
	IP ID space than due to an actual attack.

\indweird{fragment_with_DF}
	A fragment was seen with the ``Don't Fragment''
	bit set in its header.  While strictly speaking this is not illegal,
	and not impossible (a router could have fragmented a packet and then
	decided that the fragments should not be further fragmented), its
	presence is highly unusual.

	Default: {\tt WEIRD_FILE}, because it's difficult to see how
	this could reflect malicious activity.

\indweird{incompletely_captured_fragment}
	A fragment was seen whose
	length field is larger than the fragment datagram appearing on the
	monitored link.

	Default: {\tt WEIRD_LOG_ALWAYS}.

\end{description}

\subsection{Events handled by {\tt net_weird}}
\index{weird events!handled by {\tt net_weird}}

\begin{description}
\indeventtype{net_weird}{name: string}
is invoked for ``weird'' events that cannot be associated with
a particular connection or set of hosts.  Except as noted, the
default action for all such events is {\tt WEIRD_FILE}.

\end{description}

\noindent {\tt net_weird} handles the following events:

\index{packets!corrupted}
\index{corrupted packets}
\index{IP!weird events}
\index{IP!checksum error}
\index{checksum error!IP}
\begin{description}
\indweird{bad_IP_checksum}
	A packet had a bad IP header checksum.

\index{TCP!corrupted header}
\indweird{bad_TCP_header_len}
	The length of the TCP header (which is
	itself specified in the header) was smaller than the minimum
	allowed size.

\index{headers!truncated}
\index{truncated headers}
\indweird{internally_truncated_header}
	A captured packet with a valid
	IP length field was smaller as actually recorded, such that the
	captured version of the packet was illegally small.  This event
	may reflect an error in Bro's packet capture hardware or software.

	Default: {\tt WEIRD_LOG_ALWAYS}, because this event can indicate
	a basic problem with Bro's packet capture.

\indweird{truncated_IP}
	A captured packet either was too small to
	include a minimal IP header, or the full length as recorded by
	the packet capture library was smaller than the length as indicated
	by the IP header.

\indweird{truncated_header}
	An IP datagram's header indicates a length
	smaller than that required for the indicated transport type (TCP,
	UDP, ICMP).

\end{description}

\labsectsubsection{{Events generated by the standard scripts}}{\label{policy-script-events}}
\index{weird events!generated by standard scripts}

The following events are generated by the standard scripts themselves:

\begin{description}
\indweird{bad_pm_port}
	See \xrefevent{pm_bad_port}.  Handled by \xrefevent{conn_weird_addl},
	where the extra parameter is the text
	\verb+"port <+\emph{bad-port}\verb+>"+.

\index{denial of service!Land attack}
\index{Land attack}
\indweird{Land_attack}
	A TCP connection attempt was seen with identical
	initiator and responder addresses and ports.  This event likely
	reflects an attempted denial-of-service attack known as a
	``Land'' attack.  See \xreffunc{check_spoof}.  Handled by
	\xrefevent{conn_weird}.

\end{description}

\subsection{Additional handlers for ``weird'' events}
\index{weird events!additional handlers}

In addition to the above, generalized events, Bro includes two specific
events that are defined by themselves so they can include additional
parameterization:

\index{evasion!inconsistent TCP retransmission}
\index{retransmission!inconsistent}
\index{inconsistent retransmission}
\begin{description}
\indeventtype{rexmit_inconsistency}{c: connection, t1: string, t2: string}
Invoked when a retransmission associated with connection {\tt c} differed
in its data from the contents transmitted previously.  {\tt t1} gives
the original data and {\tt t2} the different retransmitted data.

\index{bugs!appalling}
This event may reflect an attacker attempting to evade the monitor.
Unfortunately, however, experience has shown that
\emph{(i)}~inconsistent retransmissions do in fact
happen due to (appalling) TCP implementation bugs, and
\emph{(ii)}~once they occur, they tend to cascade, because often
the source of the bug is that the two endpoints have become
desynchronized.

The handler logs the message in the format
\verb+"+\emph{id}\verb+ rexmit inconsistency (<t1>) (<t2>)"+ .  However,
the handler only logs the first instance of an inconsistency, due to
the cascade problem mentioned above.

\deficiency{ The handler is not told which of the two connection
endpoints was the faulty transmitter. }

\index{packets!drops}
\index{acknowledgment holes}
\index{inconsistent acknowledgment}
\index{bugs!appalling}
\indeventtype{ack_above_hole}{c: connection, t1: string, t2: string}
Invoked when Bro sees a TCP receiver acknowledge data above
a sequence hole.  In principle, this should never occur.  Its
presence generally means one of two things: \emph{(i)}~a TCP
implementation with an appalling bug (these definitely exist),
or \emph{(ii)}~a packet drop by Bro's packet capture facility,
such that it never saw the data now being acknowledged.

Because of the seriousness of this latter possibility, the
handler logs a message \indextext{ack above a hole}{possible packet drop message}.  \note{ You can often distinguish between a truly broken TCP acknowledgment
and Bro dropping packets by the fact that in the latter case you generally
see a cluster of ack-above-a-hole messages among otherwise unrelated
connections. }

\deficiency{ The handler is not told which of the two connection
endpoints sent the acknowledgment. }

\end{description}

\index{event handling!weird|)}

\index{unusual events|)}
\index{exceptional events|)}
\index{events!exceptional|)}
\index{weird events|)}

\labsectsection{{The \analyzer{icmp} Analyzer}}{\label{analyzer-icmp}\label{icmp-analyzer-module}\label{icmp-conn}}

\labsectsection{{The \analyzer{stepping} Analyzer}}{\label{analyzer-stepping}\label{stepping-analyzer-module}}

\labsectsection{{The \analyzer{ssh-stepping} Module}}{\label{module-ssh-stepping}}

\labsectsection{{The \analyzer{backdoor} Analyzer}}{\label{analyzer-backdoor}\label{backdoor-analyzer-module}}

\labsectsection{{The \analyzer{interconn} Analyzer}}{\label{analyzer-interconn}}

\index{analyzers|)}
\index{scripts!standard|)}
\index{standard scripts|)}

%HEVEA\cutend

% \section{The \analyzer{scan} Analyzer}
% \label{analyzer-scan}
% \index{scan detection}
% \index{detecting scans}
% The \indanalyzer{scan} analyzer ...
% 
% \subsection{{\tt scan} variables}
% \index{analyzers!scan!variables|(}
% 
% \begin{description}
% \indvartype{}{}
% \end{description}
% 
% \index{analyzers!scan!variables|)}
% 
% \subsection{{\tt scan} functions}
% \index{analyzers!scan!functions|(}
% 
% \begin{description}
% \itemindfunc{\item[{\tt }]{(): }}
% \end{description}
% 
% \index{analyzers!scan!functions|)}
% 
% \subsection{{\tt scan} event handlers}
% \index{analyzers!scan!event handlers|(}
% 
% \begin{description}
% \indeventtype{}{}
% \end{description}
% 
% \index{analyzers!scan!event handlers|)}
