@need 3200

@node An Introduction to Posix Regexps
@appendix An Introduction to Posix Regexps

@geindex Posix
 @geindex Posix.2
 @geindex 1003.2
 @geindex ANSI/IEEE 1003.2
 @geindex ISO/IEDC 994502

@geindex regular expression notation

@geindex regexp

@geindex regular expression



This chapter describes the Posix-derived regexp pattern languages
understood by Rx.  The @emph{Posix.2} standard (@emph{ISO/IEC 994502: 1993
(ANSI/IEEE Std 1003.2 - 1992})), section @emph{2.8}) specifies the syntax
and semantics of a "Regular Expression Notation".  Appendix "B.5"
defines a "C Binding for RE Matching".


The current release of Rx follows this standard with several
extensions and two minor omissions (multi-character collating elements
and equivalence classes based on primary collation weights).


The languages understood by Rx are notations for describing text
patterns called @emph{regexps}.  Regexps are typically used by searching
within a string for a substring that matches the pattern.


In some documentation, pattern languages such as those understood by
Rx are called @emph{regular expressions} and in other documentation, @emph{regexps}.
We prefer the term @emph{regexps} because @emph{regular expression}
has a formal mathematical meaning which does not describe the
languages understood by Rx.  Moreover, the mathematical term does
describe a subset of what we will call regexps.  Sometimes
(particularly when talking about performance issues), it is
useful to distinguish the regular expression subset of regexps from
regexps as a whole.  Therefore, it is handy to have two separate terms
with distinct meaning: @emph{regexps} (the entire pattern language) and @emph{regular
expressions} (a subset of the language, recognizable in some
branches of mathematics).

@menu
* Basic Regexps and Extended Regexps::
* Literal Characters and Special Characters::
* Character Set Expressions::
* Parenthesized Subexpressions::
* Repeated Subexpressions::
* Optional Subexpressions::
* Counted Iteration-- Interval Expressions::
* Alternative Subexpressions::
* Anchors::
* Backreferences and Substitutions::
* Anonymous Subexpressions::
* The cut Operator::
* Regexps versus Regular Expressions::
* A Summary of Regexp Syntax::
* The Extended Regexp Syntax::
* The Leftmost Longest Rule::
@end menu

@need 3200

@node Basic Regexps and Extended Regexps
@section Basic Regexps and Extended Regexps

Posix defines two regexp languages which they call @geindex Basic Regular
Expressions
@dfn{Basic Regular
Expressions} (@geindex BRE
@dfn{BRE}) and @geindex Extended Regular Expressions
@dfn{Extended Regular Expressions} (@geindex ERE
@dfn{ERE}).  (We
would have preferred the terms @geindex Basic Regexp
@dfn{Basic Regexp} and @geindex Extended Regexp
@dfn{Extended Regexp}.)


This chapter describes BRE, and then describes how ERE differ from BRE.


The reason for having two regexp langauges is historical.  BRE is the
language traditionally used by @code{ed}; ERE is the language traditionally
used by @code{egrep}.


The two languages, as specified by Posix, have different capabilities.
There are some BRE operators with no corresponding ERE operator and
some ERE operators with no corresponding BRE operator.  Fortunately,
the syntaxes chosen by Posix are extensible: new operators can be
added to either language and, in fact, both can be extended to have
the same set of operators (but using slightly different syntaxes).


Rx does, in fact, extend both syntaxes in that way.  In the
introduction that follows, we have attempted to indicate which
operators are extensions: if you want to write regexps that are
maximally portable, avoid those extensions.




@need 3200

@node Literal Characters and Special Characters
@section Literal Characters and Special Characters

@geindex literal characters (in Posix regexps)

@geindex special characters (in Posix regexps)



In the simplest cases, a regexp is just a literal string that must
match exactly.  For example, the pattern:

@example
@group
     regexp
@end group
@group
@end group
@end example

@noindent
matches the string "regexp" and no others.


Some characters have a special meaning when they occur in a regexp.
They aren't matched literally as in the previous example, but instead
denote a more general pattern.  For example, the character @code{*} is used
to indicate that the preceding element of a regexp may be repeated @code{0},
@code{1}, or more times.  In the pattern:

@example
@group
     smooo*th
@end group
@group
@end group
@end example

@noindent
the @code{*} indicates that the preceding @code{o} can be repeated @code{0} or more
times.  So the pattern matches:

@example
@group
     smooth
     smoooth
     smooooth
     smoooooth
     ...
@end group
@group
@end group
@end example

The same pattern does not match these examples:

@example
@group
     smoth      -- The pattern requires at least two o's
     smoo*th    -- The pattern doesn't match the *
@end group
@group
@end group
@end example

Suppose you want to write a pattern that literally matches a special
character like @code{*}; in other words, you don't want to @code{*} to indicate
a permissible repetition, but to match @code{*} literally.  This is
accomplished by quoting @geindex quoting (in Posix regexps)
 the special character with a backslash.  The
pattern:

@example
@group
     smoo\*th
@end group
@group
@end group
@end example

@noindent
matches the string:

@example
@group
     smoo*th
@end group
@group
@end group
@end example

@noindent
and no other strings.


These characters are special (their meaning is described in this
chapter):

@example
@group
     . [ \ * ^ $
@end group
@group
@end group
@end example

In seven cases, the pattern is reversed - a backslash makes a normal
character special instead of making a special character normal.  The
characters @code{+}, @code{?}, @code{|}, @code{(}, and @code{)} are normal but the sequences
@code{\+}, @code{\?}, @code{\|}, @code{\(}, @code{\)}, @code{\@{}, and @code{\@}} are special (their
meaning is described later).


The remaining sections of this chapter introduce and explain the
various special characters that can occur in regexps.




@need 3200

@node Character Set Expressions
@section Character Set Expressions

This section introduces the special characters @code{.} and @code{[}.

@need 3200

@subsection The Dot Operator

@geindex dot operator (in Posix regexps)

@geindex . (in Posix regexps)



@code{.} ordinarily matches any character except the NULL character.  For
example:

@example
@group
     p.ck
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     pick
     pack
     puck
     pbck
     pcck
     p.ck
@end group
@group
@end group
@end example

@noindent
etc.


In some applications, @code{.} matches a newline @geindex newline (in Posix regexps)
 character, and in other
applications, it does not.  This is an application-specific paramter
and the documentation for a particular application should tell you
what @code{.} matches.




@need 3200

@subsection The Character Set Operator

@geindex character set operator (in Posix regexps)

@geindex [...] (in Posix regexps)



@code{[} begins a "character set".  A character set is similar to @code{.} in
that it matches not a single, literal character, but any of a set of
characters.  @code{[} is different from @code{.} in that with @code{[}, you define
the set of characters explicitly.


There are two basic forms a character set can take.


The first form is a plain character set:

@example
@group
     [<cset-spec>]      -- every character in <cset-spec> is in the set.
@end group
@group
@end group
@end example

@noindent
the second form is a negated character set: @geindex negated character set (in Posix regexps)


@example
@group
     [^<cset-spec>]     -- every character *not* in <cset-spec>
                           is in the set.
@end group
@group
@end group
@end example

In some applications, the newline @geindex newline (in Posix regexps)
 character (@code{'\n'}) is included in a
negated character set if it is not mentioned in '<cset-spec>'.  In
other applications, the newline character is not included.  This is an
application-specific paramter and the documentation for a particular
application should tell you whether newline is included in negated
character sets.


A @code{<cset-spec>} may be a more or less an explicit enumeration of a set
of characters.  It can be written as a string of individual
characters:

@example
@group
     [aeiou]            -- matches any of five vowels
@end group
@group
@end group
@end example

@noindent
or as a range of characters:

@example
@group
     [0-9]              -- matches any decimal digit
@end group
@group
@end group
@end example

These two forms can be mixed:

@example
@group
     [A-za-z0-9_$]      -- any letter (either case), any digit, 
                           `_' or `$'
@end group
@group
@end group
@end example

Because of that notation, if you want to include an unescaped @code{-} in a
character set, it must be the last character in the specification:

@example
@group
     [A-Za-]            -- all upper case letters, `a' and `-'
@end group
@group
@end group
@end example

If you want to include an unescaped @code{]} in a character set, it must be
the first character in the specification:

@example
@group
     [^])@}]             -- any character except `]', ')', and '@}'
@end group
@group
@end group
@end example

Characters that are special in regexps outside of character-set
specifictions (such as @code{*}) are @emph{not} special within a character set.
Character set syntax has its own, distinct set of special characters.
This is a four-character set:

@example
@group
     [*+/-]             -- the characters `*', '+', `/', and `-'
@end group
@group
@end group
@end example

A backslash within a character set specification is not special -- it
is a literal character:

@example
@group
     [\.]               -- the characters `\' and `.'
@end group
@group
@end group
@end example

Characters which are have special meaning in a character set can be
introduced as literal characters by surrounding them with @code{[.} and
@code{.]}:

@example
@group
     [a[.-.]z]          -- the characters `a', `-', and `z'
     [[.[.]-[.].]]      -- all characters from `[' to `]'
@end group
@group
@end group
@end example

A character set specification may also contain a "character class": @geindex character class (in Posix regexps)


@example
@group
    [[:class-name:]]    -- every character described by class-name 
                           is in the set
@end group
@group
@end group
@end example

The supported character class names are:

@example
@group
    any         - the set of all characters
    alnum       - the set of alpha-numeric characters
    alpha       - the set of alphabetic characters
    blank       - tab and space
    cntrl       - the control characters
@end group
@group
    digit       - decimal digits
    graph       - all printable characters except space
    lower       - lower case letters
@end group
@group
    print       - the "printable" characters
    punct       - punctuation
    space       - whitespace characters
    upper       - upper case letters
    xdigit      - hexadecimal digits
@end group
@group
@end group
@end example

Character classes can be combined with other kinds of character set
specification:

@example
@group
    [a-pr-y[:digit:]]   -- the telephone keypad alphabet
@end group
@group
@end group
@end example






@need 3200

@node Parenthesized Subexpressions
@section Parenthesized Subexpressions

@geindex subexpression (parenthesized, in Posix regexps)

@geindex parenthesized subexpression (in Posix regexps)

@geindex (...) (in Posix regexps)



A subexpression is a regular expression enclosed in @code{\(} and @code{\)}.  A
subexpression can be used anywhere a single character or character set
can be used.


Subexpressions are useful for grouping regexp constructs.  For
example, the repeat operator, @code{*}, usually applies to just the
preceding character.  Recall that:

@example
@group
     smooo*th
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     smooth
     smoooth
     ...
@end group
@group
@end group
@end example

Using a subexpression, we can apply @code{*} to a longer string:

@example
@group
     banan\(an\)*a
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     banana
     bananana
     banananana
     ...
@end group
@group
@end group
@end example

Subexpressions also have a special meaning with regard to
backreferences and substitutions. (See @ref{Backreferences and Substitutions}.)




@need 3200

@node Repeated Subexpressions
@section Repeated Subexpressions

@geindex repeated subexpression (in Posix regexps)

@geindex * (in Posix regexps)



As previously explained, @code{*} is the repeat operator.  It applies to
the preceding character, character set, subexpression or
backreference.  It indicates that the preceding element can be matched
@code{0} or more times:

@example
@group
     bana\(na\)*
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     bana
     banana
     bananana
     banananana
     ...
@end group
@group
@end group
@end example

@geindex + (in Posix regexps)

@code{\+} is similar to @code{*} except that @code{\+} requires the preceding element
to be matched at least once.  So on the one hand:

@example
@group
     bana\(na\)*
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     bana
@end group
@group
@end group
@end example

@noindent
but

@example
@group
     bana(na\)\+
@end group
@group
@end group
@end example

@noindent
does not.   Both match

@example
@group
     banana
     bananana
     banananana
     ...
@end group
@group
@end group
@end example

@strong{Note:} The @code{\+} operator is not present in standard BRE -- it is an
extension.  The corresponding @code{+} operator of ERE is standard.




@need 3200

@node Optional Subexpressions
@section Optional Subexpressions

@geindex optional subexpression (in Posix regexps)

@geindex ? (in Posix regexps)

@code{\?} indicates that the preceding character, character set, or
subexpression is optional.  It is permitted to match, or to be
skipped:

@example
@group
     CSNY\?
@end group
@group
@end group
@end example

@noindent
matches both

@example
@group
     CSN
@end group
@group
@end group
@end example

@noindent
and

@example
@group
     CSNY
@end group
@group
@end group
@end example

@strong{Note:} The @code{\?} operator is not present in standard BRE -- it is an
extension.  The corresponding @code{?} operator of ERE is standard.




@need 3200

@node Counted Iteration-- Interval Expressions
@section Counted Iteration-- Interval Expressions

@geindex counted iteration (in Posix regexps)

@geindex interval expression (in Posix regexps)

@geindex @{...@} (in Posix regexps)

An interval expression, @code{\@{m,n\@}} where @code{m} and @code{n} are decimal
integers with @code{0 <= m <= n <= 255}, applies to the preceding
character, character set, subexpression or backreference.  It
indicates that the preceding element must match at least @code{m} times and
may match as many as @code{n} times.


For example:

@example
@group
     c\([ad]\)\@{1,4\@}r
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     car
     cdr
     caar
     cdar
     ...
@end group
@group
     caaar
     cdaar
     ...
     cadddr
     cddddr
@end group
@group
@end group
@end example

@noindent
but it doesn't match:

@example
@group
     cdddddr
@end group
@group
@end group
@end example

@noindent
because that has five d's, and the pattern only permits four, and it
doesn't match

@example
@group
     cr
@end group
@group
@end group
@end example

@noindent
because the pattern requires at least one @code{a} or @code{d}.


A pattern of the form:

@example
@group
        R\@{M,\@}
@end group
@group
@end group
@end example

@noindent
matches @code{M} or more iterations of @code{R}.


A pattern of the form:

@example
@group
        R\@{M\@}
@end group
@group
@end group
@end example

@noindent
matches exactly 'M' iterations of 'R'.




@need 3200

@node Alternative Subexpressions
@section Alternative Subexpressions

@geindex alternative subexpressions (in Posix regexps)

@geindex | (in Posix regexps)



An alternative is written:

@example
@group
     regexp-1\|regexp-2\|regexp-3\|...
@end group
@group
@end group
@end example

It matches anything matched by some @code{regexp-n}.  For example:

@example
@group
     Crosby, Stills, \(and Nash\|Nash, and Young\)
@end group
@group
@end group
@end example

@noindent
matches

@example
@group
     Crosby, Stills, and Nash
@end group
@group
@end group
@end example

@noindent
and also

@example
@group
     Crosby, Stills, Nash, and Young
@end group
@group
@end group
@end example

@strong{Note:} The @code{\|} operator is not present in standard BRE -- it is an
extension.  The corresponding @code{|} operator of ERE is standard.




@need 3200

@node Anchors
@section Anchors

@geindex anchors (in Posix regexps)

@geindex left anchor (in Posix regexps)

@geindex right anchor (in Posix regexps)

@geindex ^ (in Posix regexps)

@geindex $ (in Posix regexps)



Anchor experssions are written @code{^} ("left anchor") and @code{$} ("right
anchor").  Both kinds of anchor match the empty string but only in
certain contexts.  A left anchor matches at the beginning of a string,
a right anchor at the end of a string.


In some situations, @code{^} matches the empty string immediately after a
newline character, and '$' matches immediately before a newline
character.  Whether or not this is the case is an application-specific
behavior: the documentation for each application that uses regexps
should state whether or not anchors match near newlines.




@need 3200

@node Backreferences and Substitutions
@section Backreferences and Substitutions

@geindex backreferences (in Posix regexps)

@geindex substitutions (in Posix regexps)

@geindex \DIGIT (in Posix regexps)

A backreference is written @code{\n} where @code{n} is some single digit, 1-9.
To be a valid backreference, there must be at least @code{n} parenthesized
subexpressions in the pattern prior to the backreference.


A backreference matches a literal copy of whatever was matched by the
corresponding subexpression.  For example,

@example
@group
     \(.*\)-\1
@end group
@group
@end group
@end example

@noindent
matches:

@example
@group
     go-go              -- .* matches the first "go"; 
                           \1 matches  the second
     ha-ha
     wakka-wakka
     ...
@end group
@group
@end group
@end example

@noindent
but does not match:

@example
@group
     ha-wakka           -- .* matches "ha", but "wakka" is not
                           the same as "ha"
@end group
@group
@end group
@end example

In some applications, subexpressions are used to extract substrings.
For example, Emacs @geindex emacs
 has the functions @code{match-beginning} and
@code{match-end} which report the positions of strings matched by
subexpressions.  These functions use the same numbering scheme for
subexpressions as backreferences, with the additional rule that
subexpression @code{0} is defined to be the whole regexp.


In some applications, subexpressions are used in string substitution.
This again uses the backreference numbering scheme. For example, this
sed command:

@example
@group
     s/From:.*<\(.*\)>/To: \1/
@end group
@group
@end group
@end example

@noindent
matches the entire line:

@example
@group
     From: Joe Schmoe <schmoe@@uspringfield.edu>
@end group
@group
@end group
@end example

@noindent
and subexpression @code{1} matches @code{schmoe@@uspringfield.edu}.  The command
replaces the matched line with @code{To: \1} after doing subexpression
substitution on it to get:

@example
@group
     To: schmoe@@uspringfield.edu
@end group
@group
@end group
@end example

@strong{Note:} The backreference operator is present in standard BRE, but not
in ERE.  Rx adds the backreference operator to ERE as an extension.




@need 3200

@node Anonymous Subexpressions
@section Anonymous Subexpressions

@geindex anonymous subexpression (in Posix regexps)

@geindex subexpression (anonymous, in Posix regexps)

@geindex [[:(...:)]] (in Posix regexps)

An anonymous subexpression is a subexpression that can not be
backreferenced.  In some situations, a pattern written using anonymous
subexpressions will run faster than the same pattern written with
ordinary subexpressions.


An anonymous subexpression is written as a regular expression enclosed
in @code{[[:(} and @code{):]]}, for example:

@example
@group
        sm[[:(oo):]]*th
@end group
@group
@end group
@end example

@noindent
will match all fanciful spellings of "smooth" that include an even
number of @code{o}'s.


An anonymous subexpression can be used anywhere a single character or
character set can be used.


@strong{Note:} The @code{[[:():]]} operator is not standard in either BRE or ERE
-- it is an Rx extension.




@need 3200

@node The cut Operator
@section The cut Operator

@geindex cut (in Posix regexps)

@geindex [[:cut n:]] (in Posix regexps)

The cut operator is used to indicate a point in the middle of a regexp
at which a match can succeed without matching the remaining pattern.
The cut operator takes a parameter which is an integer called the
@code{state label}. @geindex state label (in Posix regexps)
 When a pattern matches by reaching a cut operator, one
of the values that can be returned is the state label.


For example, this regexp can be used for lexical analysis @geindex lexical analysis (with Posix regexps)
:

@example
@group
        if[[:cut 2:]]\\|then[[:cut 3:]]\\|else[[:cut 4:]]
@end group
@group
@end group
@end example

If it matches "then", the state label returned will be @code{3}.  If it
matches "else", the state label will be @code{4}.  The default state label is
@code{1}.


This example illustrates using @code{cut} to terminate a match early:

@example
@group
        GNU[[:cut 2:]] project
@end group
@group
@end group
@end example

That expression matches either of two strings:

@example
@group
        GNU
        GNU project
@end group
@group
@end group
@end example

@noindent
with a state label of @code{2} in the first case and @code{1} in the second case.


A state label may be a decimal integer, or it may be @code{%}.  In the
latter case, an integer value is assigned automatically.
Automatically assigned integer values start with @code{1} and each successive
@code{[[:(cut %):]]} in an expression increments the automatically assigned
value.


@strong{Note:} The @code{[[:cut n:]]} operator is not standard in either BRE or ERE
-- it is an Rx extension.




@need 3200

@node Regexps versus Regular Expressions
@section Regexps versus Regular Expressions

@geindex regexp
 @geindex regular expression
 @geindex true regexp
 @geindex true regular expression

In the Rx documentation and source code we distinguish between
@emph{regexps} and @emph{regular expressions}.  Regular expressions are a
subset of regexps.  We call regexps which are not regular
expressions "true regexps" and if a regexp is a regular expression,
we say that that regexp is "regular" or a "true regular
expression".


A practial reason for making such a distinction is performance:
comparing text to a true regular expression is often much less
computationally expensive than comparing text to a true regexp.


Writing a true regular expression is easy.  Every regexp is regular
if it does @emph{not} use any of the following operators:

@example
@group
     ^re             - left anchor
     re$             - right anchor
     re\@{x,y\@}       - counted iteration
     \(re\)...\1     - parentheses with backreferences
@end group
@group
@end group
@end example

If you are mathematically inclined, you may notice that a regexp
of the form:

@example
@group
     re\@{x,y\@}       - counted iteration
@end group
@group
@end group
@end example

@noindent
is, in the mathematical sense, a regular expression.  For practical
reasons, however, Rx treats such expressions as true regexps.




@need 3200

@node A Summary of Regexp Syntax
@section A Summary of Regexp Syntax

@geindex regexp syntax summary (Posix)

In summary, regexps can be:

@example
@group
   `abcd'               - a string literal
@end group
@group
@end group
@end example
@example
@group
   `.'                  - the universal character set.
@end group
@group
@end group
@end example
@example
@group
   `[a-z_?]'            - a character set
   `[^a-z_?]'           - an inverted character set
   `[[:alpha:]]'        - a named character set
   `[[.c.]]'            - a quoted character set character
@end group
@group
@end group
@end example
@example
@group
   `\(subexp\)'         - an expression grouped to form a
                          parenthesized subexpression. 
@end group
@group
@end group
@end example
@example
@group
   `[[:(subexp:)]]'     - an expression grouped to form an
                          anonymous subexpression. (Non-standard)
@end group
@group
@end group
@end example
@example
@group
   `^'                  - a left anchor, matching the beginning
                          of a string.
@end group
@group
@end group
@end example
@example
@group
   `$'                  - a right anchor, matching the end
                          of a string.
@end group
@group
@end group
@end example
@example
@group
   `\n'                 - a backreference to the nth parenthesized
                          subexpression. (Non-standard in ERE)
@end group
@group
@end group
@end example
@example
@group
    `[[:cut n:]]'       - a point at which a regular expression 
                          match can immediately succeed.
                          (Non-standard)
@end group
@group
@end group
@end example

The following special characters and sequences can be applied to a
character, character set, subexpression, or backreference:

@example
@group
   `*'                  - repeat the preceding element 0 
                          or more times.
@end group
@group
@end group
@end example
@example
@group
   `\+'                 - repeat the preceding element 1 
                          or more times. (Non-standard in BRE)
@end group
@group
@end group
@end example
@example
@group
   `\?'                 - match the preceding element 0 
                          or 1 time. (Non-standard in BRE)
@end group
@group
@end group
@end example
@example
@group
   `\@{m,n\@}'            - match the preceding element at 
                          least `m', and as many as `n' times,
                          where 0 <= m <= n <= 255
@end group
@group
@end group
@end example
@example
@group
   `\@{m,\@}'             - match the preceding element at 
                          least `m' times, where 0 <= m <= 255
@end group
@group
@end group
@end example
@example
@group
   `\@{m\@}'              - match the preceding element exactly
                          `m' times, where 0 <= m <= 255
@end group
@group
@end group
@end example

Finally, a list of patterns built from the preceeding operators can be
combined using @code{\|}:

@example
@group
   `regexp-1\|regexp-2\|..'     - match regexp-1 or regex-2 or ...
                                  (Non-standard in BRE)
@end group
@group
@end group
@end example

A special character, like @code{.} or @code{*} can be made into a literal by
prefixing it with @code{\}.


A special sequence, like @code{\+} or @code{\?} can be made into a literal
character by dropping the @code{\}.




@need 3200

@node The Extended Regexp Syntax
@section The Extended Regexp Syntax

@geindex extended regexp syntax (Posix)
 @geindex basic regexp syntax (Posix)
 
@geindex ERE (Posix Regexps)
 @geindex BRE (Posix Regexps)



The regexp syntax described in the preceeding sections is called
"Basic Regular Expression Syntax" (BRE).  In that syntax, some
operators are preceeded by a backslash:

@example
@group
        one\|the other          -- the \| operator
        \(many\)\+              -- The \( \) and \+ operators
        re\@{x,y\@}               -- The \@{ \@} operator
@end group
@group
@end group
@end example

In that way, the BRE syntax is optimized for situations where we are
more likely to want to write @code{|}, @code{()}, @code{+}, or @code{@{@}} as literal
characters than as regexp operators.  The common case is handled
without the backslash, the less common case with the backslash.


Another syntax exists for situations where the regexp operators are
more likely to occur than the corresponding literal characters.  In
this, the "Extended Regexp Syntax" (ERE), none of the regexp operators
require a backslash and except in a character set specification, a
backslash always means to interpret the next character literally.

@example
@group
            BRE OPERATOR          ERE OPERATOR
                \|                      |
                \+                      +
                \( \)                   ( )
                \@{ x, y \@}              @{ x, y @}
@end group
@group
@end group
@end example

If an application uses the extended syntax, the documentation for that
application should tell you so.




@need 3200

@node The Leftmost Longest Rule
@section The Leftmost Longest Rule

@geindex leftmost longest rule (in Posix regexps)

@geindex regexp, ambiguous (Posix)

@geindex ambiguous regexp (Posix)

Sometimes a regexp appears to be ambiguous.  When
comparing an expression like:

@example
@group
        match
@end group
@group
@end group
@end example

@noindent
to a string like:

@example
@group
        match-match
@end group
@group
@end group
@end example

@noindent
there is an ambiguity about where the match begins.  Do the first @code{5}
characters match? or the last 5?


In every case like this, the leftmost match is preferred.  The first @code{5}
characters will match.


Sometimes the ambiguity is not about where a match will begin, but how
long a match will be.  For example, suppose we compare the pattern:

@example
@group
     begin\|beginning
@end group
@group
@end group
@end example

@noindent
to the string

@example
@group
     beginning
@end group
@group
@end group
@end example

@noindent
either just the first @code{5} characters will match, or the whole string
will match.


In every case like this, consistent with finding the leftmost match,
the longer match is preferred.  The whole string will match.


Sometimes there is ambiguity not about how many characters to match or
where a match begins, but where the subexpressions occur within the
match.  This can effect extraction functions like Emacs'
@code{match-beginning} or rewrite functions like sed's @code{s} command.  For
example, consider matching the pattern:

@example
@group
     b\([^q]*\)\(ing\)?
@end group
@group
@end group
@end example

@noindent
against the string

@example
@group
     beginning
@end group
@group
@end group
@end example

One possibility is that the first parenthesized subexpression matches
@code{eginning} and the second parenthesized subexpression is skipped.
Another possibility is that the first subexpression matches @code{eginn}
and the second matches @code{ing}.


The rule is that consistant with finding the leftmost and longest
overall match, the lengths of strings matched by subpatterns are
maximized, from left to right.


In the example, the subpattern @code{b} matches one character, the
subpattern @code{\([^q]*\)} matches the longest possible string,
@code{eginning}, and finally the third subpattern @code{\(ing\)?} matches what
is left over (the empty string).


The presence or absense of parentheses doesn't change what subpatterns
match.  If the pattern is:

@example
@group
     b[^q]*\(ing\)?
@end group
@group
@end group
@end example

The subpattern @code{[^q]*} still matches @code{eginning} and @code{\(ing\)?} still
matches the empty string.







