SWI-Prolog offers two comprehensive predicates for classifying characters and character codes. These predicates are defined as built-in predicates to exploit the C-character classification's handling of locale (handling of local character sets). These predicates are fast, logical and deterministic if applicable.
In addition, there is the library library(ctypes)
providing compatibility with some other Prolog systems. The predicates
of this library are defined in terms of code_type/2.
<ctype.h>
primitives. Neither the classes nor the case conversions are sensitive
to the active locale: they are derived from the Unicode
Character Database at build time (unicode_syntax_version)
rather than from the C library's <wctype.h>, so a
program classifies text the same way regardless of LC_CTYPE
and of the platform's C library. This also means the classes agree with
the classification used by read/1
and friends, which is where the types prolog_var_start,
prolog_atom_start, prolog_identifier_continue
and
prolog_symbol come from. The case (to_lower, to_upper, lower(Upper)
and
upper(Lower)) still use the C library and are thus still
locale conversions (to_lower, to_upper,
lower(Upper) and upper(Lower)) use the Unicode
simple case mapping, which is one code point to one code point
and thus length preserving; upcase_atom('straße', X)
yields 'STRAßE' rather than 'STRASSE'.
Because the mapping is Unicode's rather than the C library's, it does
not follow the Turkish and Azeri dotted/dotless i rules
even under a
tr_TR locale. Foreign code can reach the same classifier
and the same case mapping through PL_ctype_flags(), PL_toupper(),
PL_tolower()
and PL_totitle().
Every type is derived from the Unicode Character Database, from the
files UnicodeData.txt (general categories and the simple
case mappings), DerivedCoreProperties.txt (Alphabetic,
Uppercase, Lowercase, XID_Start, XID_Continue), PropList.txt
(White_Space, Sentence_Terminal, Pattern_Syntax) and EastAsianWidth.txt
(display width), for the Unicode version named by the
unicode_syntax_version
flag. The derivations are:
| Type | Derived from |
alnum | alpha, or
category Nd, Nl or No |
alpha | property Alphabetic |
ascii | code point < 128; not a Unicode notion |
cntrl | category Cc or Cf |
csym | ASCII letter, digit or _;
see below |
csymf | ASCII letter or _; see
below |
decimal | category Nd |
digit | U+0030..U+0039; POSIX fixes this to ASCII |
end_of_file | the code -1 |
end_of_line | the Unicode line terminators |
graph | alnum or punct |
lower | property Lowercase |
newline | U+000A |
paren | categories Ps and Pe, paired |
pattern_syntax | property Pattern_Syntax (UAX #31) |
period | property Sentence_Terminal |
print | graph, or white space
that is not Cc |
prolog_end_of_line | the Unicode line terminators |
prolog_layout | property Pattern_White_Space (UAX #31) |
punct | category P* or S*, not Alphabetic |
quote | ', ", `
and categories Pi, Pf |
space | property White_Space |
to_lower, to_upper | the simple case mappings |
upper | property Uppercase |
white | White_Space minus end_of_line |
width | property East_Asian_Width (UAX #11) |
xdigit | 0..9, a..f, A..F |
The types prolog_var_start, prolog_atom_start,
prolog_identifier_continue, prolog_symbol and
prolog_solo are the reader's own classification; see
section 2.15.1.9.
Note that the mode (-,+) is only efficient if the Type has
a parameter, e.g., char_type(C, digit(8)). If Type
is a atomic, the whole unicode range (0..0x1ffff) is generated and
tested against the character classification function.
alpha or a numeric character (Unicode
general category Nd, Nl or No). Note that
this is wider than digit: e.g. 0’
u0660 (ARABIC-INDIC DIGIT ZERO) and 0’
u00BD (VULGAR FRACTION ONE HALF) are alnum but not digit.
_).
_).
These two are ASCII because C identifiers are: C89 admits nothing
else, and the extended identifiers of C99 and later are written as
universal character names (\u00E9prolog_atom_start, prolog_var_start and
prolog_identifier_continue for those, or alnum
and
alpha for a script-independent notion of a word character.
space characters minus the seven line terminators (U+000A..
U+000D, U+0085, U+2028 and U+2029). That leaves U+0009 TAB, U+0020
SPACE, U+00A0 NO-BREAK SPACE, U+1680, U+2000..U+200A, U+202F, U+205F and
U+3000 IDEOGRAPHIC SPACE. This is what POSIX calls blank.
cntrl and space.
Note that U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are space,
not cntrl.
digit
class to hold exactly the ten ASCII digits, so the decimal digits of
other scripts are alnum and decimal, but not
digit. See also decimal.
char_type(X,
digit(6)) yields X = ’6’.
Useful for parsing numbers.
char_type(a, xdigit(X)) yields X = ’10’.
Useful for parsing numbers. As with digit(Weight) this is
ASCII only; there is no Unicode-wide counterpart because hexadecimal
notation is a source syntax rather than a script property.
graph, or white
space that is not a control character (U+0020, U+00A0, U+2028, U+3000,
... but not tab or newline).
alnum
or punct. Note that the space is not included! POSIX
requires graph and space to be disjoint, which
is why U+00A0 NO-BREAK SPACE is print but not graph.
graph
character that is not a letter or digit.
<wctype.h> implementations disagree here: glibc
denies that U+00A0 is white space and Darwin denies that U+0085 is. Note
also that U+00A0 is deliberately not layout for the reader; see prolog_layout.
space that is not white:
white space either stays within a line or ends it.
prolog_end_of_line is the set the reader acts on, which
happens to be the same today but is defined by the Prolog syntax rather
than by Unicode.
.!?", ', `
or one of the Unicode initial/final quotation marks (categories Pi
and Pf). Note that this holds for both sides of a pair,
where quote(Close) below holds only for the opening one.
(), []
and {}, plus every Unicode Ps/Pe
pair (about 60 pairs in Unicode 17, including angle, corner, ceiling,
floor, mathematical, ornamental, fullwidth and CJK brackets). The
mapping is reversible: with Close bound, Char
unifies with the matching open.
’, ",
and ‘ have
Close = Char; Unicode Pi/Pf
quote pairs (the guillemets, the standard left/right curly single and
double quotes, and the single/double angle and reversed quotation marks)
have Close different from Char. The mapping is
reversible.
position(-Pos) property.
Pattern_White_Space set used by read_term/2
to separate tokens. The eleven code points are U+0009..U+000D, U+0020,
U+0085, U+200E, U+200F, U+2028 and U+2029. Locale- independent; pinned
to unicode_syntax_version.
prolog_end_of_line is the seven-element line-terminator
subset.
Pattern_White_Space code points:
U+000A (LF), U+000B (VT), U+000C (FF), U+000D (CR), U+0085 (NEL), U+2028
(LINE SEPARATOR), and U+2029 (PARAGRAPH SEPARATOR). The same set
terminates %end_of_line, which is Unicode's, although the
two currently agree. See section
2.15.1.9.
=..\=!;%;
the same flag carries over to non-ASCII code points via the Unicode
syntax map (see
section 2.15.1.9).
Solo characters are written unquoted by writeq/1
and are accepted as single-character atoms by the reader.
Pattern_Syntax property
(UAX #31 R3). This is the immutable set of punctuation and
symbol code points whose classification is guaranteed not to change
across Unicode versions. Used by write_canonical/1
and the
pattern_syntax_solo option of write_term/2
to decide which single-character atoms can be printed bare with
round-trip safety across Unicode upgrades.
There is nothing in the Prolog standard for converting case in textual data. The SWI-Prolog predicates code_type/2 and char_type/2 can be used to test and convert individual characters. We have started some additional support:
\u0020) character. Out
uses the same conventions as with_output_to/2
and format/3.
A character counts as white space if it satisfies code_type(Code,
space), i.e., the C library function iswspace(). This
depends on the locale. Using a Unicode locale this includes the Unicode
separator characters such as \u00a0 (no-break space) and
\u3000 (ideographic space), while in the C
locale only the ASCII white space characters are recognised. Note that
the
bidi format characters \u200e and \u200f
are not white space and are thus preserved.
This section deals with predicates for language-specific string comparison operations.
The predicate collation_key/2 is used by locale_sort/2 from library(sort). Please examine the implementation of locale_sort/2 as an example of using this call.
The Key is an implementation-defined and generally unreadable string. On systems that do not support locale handling, Key is simply unified with Atom.