\input texinfo
@c %**start of header
@setfilename flex.info
@settitle Flex - a scanner generator
@c @finalout
@c @setchapternewpage odd
@c %**end of header

@set EDITION 2.5
@set UPDATED March 1995
@set VERSION 2.5

@c FIXME - Reread a printed copy with a red pen and patience.
@c FIXME - Modify all "See ..." references and replace with @xref's.

@ifinfo
@format
START-INFO-DIR-ENTRY
* Flex: (flex).         A fast scanner generator.
END-INFO-DIR-ENTRY
@end format
@end ifinfo

@c Define new indices for commands, filenames, and options.
@c @defcodeindex cm
@c @defcodeindex fl
@c @defcodeindex op

@c Put everything in one index (arbitrarily chosen to be the concept index).
@c @syncodeindex cm cp
@c @syncodeindex fl cp
@syncodeindex fn cp
@syncodeindex ky cp
@c @syncodeindex op cp
@syncodeindex pg cp
@syncodeindex vr cp

@ifinfo
This file documents Flex.

Copyright (c) 1990 The Regents of the University of California.
All rights reserved.

This code is derived from software contributed to Berkeley by
Vern Paxson.

The United States Government has rights in this work pursuant
to contract no. DE-AC03-76SF00098 between the United States
Department of Energy and the University of California.

Redistribution and use in source and binary forms are permitted
provided that: (1) source distributions retain this entire
copyright notice and comment, and (2) distributions including
binaries display the following acknowledgement:  ``This product
includes software developed by the University of California,
Berkeley and its contributors'' in the documentation or other
materials provided with the distribution and in all advertising
materials mentioning features or use of this software.  Neither the
name of the University nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE.

@ignore
Permission is granted to process this file through TeX and print the
results, provided the printed document carries copying permission
notice identical to this one except for the removal of this paragraph
(this paragraph not being relevant to the printed manual).

@end ignore
@end ifinfo

@titlepage
@title Flex, version @value{VERSION}
@subtitle A fast scanner generator
@subtitle Edition @value{EDITION}, @value{UPDATED}
@author Vern Paxson

@page
@vskip 0pt plus 1filll
Copyright @copyright{} 1990 The Regents of the University of California.
All rights reserved.

This code is derived from software contributed to Berkeley by
Vern Paxson.

The United States Government has rights in this work pursuant
to contract no. DE-AC03-76SF00098 between the United States
Department of Energy and the University of California.

Redistribution and use in source and binary forms are permitted
provided that: (1) source distributions retain this entire
copyright notice and comment, and (2) distributions including
binaries display the following acknowledgement:  ``This product
includes software developed by the University of California,
Berkeley and its contributors'' in the documentation or other
materials provided with the distribution and in all advertising
materials mentioning features or use of this software.  Neither the
name of the University nor the names of its contributors may be
used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE.
@end titlepage

@ifinfo

@node Top, Name, (dir), (dir)
@top flex

@cindex scanner generator

This manual documents @code{flex}.  It covers release @value{VERSION}.

@menu
* Name::                        Name
* Synopsis::                    Synopsis
* Overview::                    Overview
* Description::                 Description
* Examples::                    Some simple examples
* Format::                      Format of the input file
* Patterns::                    Patterns
* Matching::                    How the input is matched
* Actions::                     Actions
* Generated scanner::           The generated scanner
* Start conditions::            Start conditions
* Multiple buffers::            Multiple input buffers
* End-of-file rules::           End-of-file rules
* Miscellaneous::               Miscellaneous macros
* User variables::              Values available to the user
* YACC interface::              Interfacing with @code{yacc}
* Options::                     Options
* Performance::                 Performance considerations
* C++::                         Generating C++ scanners
* Incompatibilities::           Incompatibilities with @code{lex} and POSIX
* Diagnostics::                 Diagnostics
* Files::                       Files
* Deficiencies::                Deficiencies / Bugs
* See also::                    See also
* Author::                      Author
@c * Index::                       Index
@end menu

@end ifinfo

@node Name, Synopsis, Top, Top
@section Name

flex - fast lexical analyzer generator

@node Synopsis, Overview, Name, Top
@section Synopsis

@example
flex [-bcdfhilnpstvwBFILTV78+? -C[aefFmr] -ooutput -Pprefix -Sskeleton]
[--help --version] [@var{filename} @dots{}]
@end example

@node Overview, Description, Synopsis, Top
@section Overview

This manual describes @code{flex}, a tool for generating programs
that perform pattern-matching on text.  The manual
includes both tutorial and reference sections:

@table @asis
@item Description
a brief overview of the tool

@item Some Simple Examples

@item Format Of The Input File

@item Patterns
the extended regular expressions used by flex

@item How The Input Is Matched
the rules for determining what has been matched

@item Actions
how to specify what to do when a pattern is matched

@item The Generated Scanner
details regarding the scanner that flex produces;
how to control the input source

@item Start Conditions
introducing context into your scanners, and
managing "mini-scanners"

@item Multiple Input Buffers
how to manipulate multiple input sources; how to
scan from strings instead of files

@item End-of-file Rules
special rules for matching the end of the input

@item Miscellaneous Macros
a summary of macros available to the actions

@item Values Available To The User
a summary of values available to the actions

@item Interfacing With Yacc
connecting flex scanners together with yacc parsers

@item Options
flex command-line options, and the "%option"
directive

@item Performance Considerations
how to make your scanner go as fast as possible

@item Generating C++ Scanners
the (experimental) facility for generating C++
scanner classes

@item Incompatibilities With Lex And POSIX
how flex differs from AT&T lex and the POSIX lex
standard

@item Diagnostics
those error messages produced by flex (or scanners
it generates) whose meanings might not be apparent

@item Files
files used by flex

@item Deficiencies / Bugs
known problems with flex

@item See Also
other documentation, related tools

@item Author
includes contact information
@end table

@node Description, Examples, Overview, Top
@section Description

@code{flex} is a tool for generating @dfn{scanners}: programs which
recognized lexical patterns in text.  @code{flex} reads the given
input files, or its standard input if no file names are
given, for a description of a scanner to generate.  The
description is in the form of pairs of regular expressions
and C code, called @dfn{rules}. @code{flex} generates as output a C
source file, @file{lex.yy.c}, which defines a routine @samp{yylex()}.
This file is compiled and linked with the @samp{-lfl} library to
produce an executable.  When the executable is run, it
analyzes its input for occurrences of the regular
expressions.  Whenever it finds one, it executes the
corresponding C code.

@node Examples, Format, Description, Top
@section Some simple examples

First some simple examples to get the flavor of how one
uses @code{flex}.  The following @code{flex} input specifies a scanner
which whenever it encounters the string "username" will
replace it with the user's login name:

@example
%%
username    printf( "%s", getlogin() );
@end example

By default, any text not matched by a @code{flex} scanner is
copied to the output, so the net effect of this scanner is
to copy its input file to its output with each occurrence
of "username" expanded.  In this input, there is just one
rule.  "username" is the @var{pattern} and the "printf" is the
@var{action}.  The "%%" marks the beginning of the rules.

Here's another simple example:

@example
        int num_lines = 0, num_chars = 0;

%%
\n      ++num_lines; ++num_chars;
.       ++num_chars;

%%
main()
        @{
        yylex();
        printf( "# of lines = %d, # of chars = %d\n",
                num_lines, num_chars );
        @}
@end example

This scanner counts the number of characters and the
number of lines in its input (it produces no output other
than the final report on the counts).  The first line
declares two globals, "num_lines" and "num_chars", which
are accessible both inside @samp{yylex()} and in the @samp{main()}
routine declared after the second "%%".  There are two rules,
one which matches a newline ("\n") and increments both the
line count and the character count, and one which matches
any character other than a newline (indicated by the "."
regular expression).

A somewhat more complicated example:

@example
/* scanner for a toy Pascal-like language */

%@{
/* need this for the call to atof() below */
#include <math.h>
%@}

DIGIT    [0-9]
ID       [a-z][a-z0-9]*

%%

@{DIGIT@}+    @{
            printf( "An integer: %s (%d)\n", yytext,
                    atoi( yytext ) );
            @}

@{DIGIT@}+"."@{DIGIT@}*        @{
            printf( "A float: %s (%g)\n", yytext,
                    atof( yytext ) );
            @}

if|then|begin|end|procedure|function        @{
            printf( "A keyword: %s\n", yytext );
            @}

@{ID@}        printf( "An identifier: %s\n", yytext );

"+"|"-"|"*"|"/"   printf( "An operator: %s\n", yytext );

"@{"[^@}\n]*"@}"     /* eat up one-line comments */

[ \t\n]+          /* eat up whitespace */

.           printf( "Unrecognized character: %s\n", yytext );

%%

main( argc, argv )
int argc;
char **argv;
    @{
    ++argv, --argc;  /* skip over program name */
    if ( argc > 0 )
            yyin = fopen( argv[0], "r" );
    else
            yyin = stdin;

    yylex();
    @}
@end example

This is the beginnings of a simple scanner for a language
like Pascal.  It identifies different types of @var{tokens} and
reports on what it has seen.

The details of this example will be explained in the
following sections.

@node Format, Patterns, Examples, Top
@section Format of the input file

The @code{flex} input file consists of three sections, separated
by a line with just @samp{%%} in it:

@example
definitions
%%
rules
%%
user code
@end example

The @dfn{definitions} section contains declarations of simple
@dfn{name} definitions to simplify the scanner specification,
and declarations of @dfn{start conditions}, which are explained
in a later section.
Name definitions have the form:

@example
name definition