\input texinfo@c %**start of header@setfilename flex.info@settitle Flex - a scanner generator@c @finalout@c @setchapternewpage odd@c %**end of header@set EDITION 2.5@set UPDATED March 1995@set VERSION 2.5@c FIXME - Reread a printed copy with a red pen and patience.@c FIXME - Modify all "See ..." references and replace with @xref's.@ifinfo@formatSTART-INFO-DIR-ENTRY* Flex: (flex).         A fast scanner generator.END-INFO-DIR-ENTRY@end format@end ifinfo@c Define new indices for commands, filenames, and options.@c @defcodeindex cm@c @defcodeindex fl@c @defcodeindex op@c Put everything in one index (arbitrarily chosen to be the concept index).@c @syncodeindex cm cp@c @syncodeindex fl cp@syncodeindex fn cp@syncodeindex ky cp@c @syncodeindex op cp@syncodeindex pg cp@syncodeindex vr cp@ifinfoThis file documents Flex.Copyright (c) 1990 The Regents of the University of California.All rights reserved.This code is derived from software contributed to Berkeley byVern Paxson.The United States Government has rights in this work pursuantto contract no. DE-AC03-76SF00098 between the United StatesDepartment of Energy and the University of California.Redistribution and use in source and binary forms with or withoutmodification are permitted provided that: (1) source distributionsretain this entire copyright notice and comment, and (2)distributions including binaries display the followingacknowledgement:  ``This product includes software developed by theUniversity of California, Berkeley and its contributors'' in thedocumentation or other materials provided with the distribution andin all advertising materials mentioning features or use of thissoftware.  Neither the name of the University nor the names of itscontributors may be used to endorse or promote products derivedfrom this software without specific prior written permission.THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS ORIMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIEDWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULARPURPOSE.@ignorePermission is granted to process this file through TeX and print theresults, provided the printed document carries copying permissionnotice identical to this one except for the removal of this paragraph(this paragraph not being relevant to the printed manual).@end ignore@end ifinfo@titlepage@title Flex, version @value{VERSION}@subtitle A fast scanner generator@subtitle Edition @value{EDITION}, @value{UPDATED}@author Vern Paxson@page@vskip 0pt plus 1filllCopyright @copyright{} 1990 The Regents of the University of California.All rights reserved.This code is derived from software contributed to Berkeley byVern Paxson.The United States Government has rights in this work pursuantto contract no. DE-AC03-76SF00098 between the United StatesDepartment of Energy and the University of California.Redistribution and use in source and binary forms with or withoutmodification are permitted provided that: (1) source distributionsretain this entire copyright notice and comment, and (2)distributions including binaries display the followingacknowledgement:  ``This product includes software developed by theUniversity of California, Berkeley and its contributors'' in thedocumentation or other materials provided with the distribution andin all advertising materials mentioning features or use of thissoftware.  Neither the name of the University nor the names of itscontributors may be used to endorse or promote products derivedfrom this software without specific prior written permission.THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS ORIMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIEDWARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULARPURPOSE.@end titlepage@ifinfo@node Top, Name, (dir), (dir)@top flex@cindex scanner generatorThis manual documents @code{flex}.  It covers release @value{VERSION}.@menu* Name::                        Name* Synopsis::                    Synopsis* Overview::                    Overview* Description::                 Description* Examples::                    Some simple examples* Format::                      Format of the input file* Patterns::                    Patterns* Matching::                    How the input is matched* Actions::                     Actions* Generated scanner::           The generated scanner* Start conditions::            Start conditions* Multiple buffers::            Multiple input buffers* End-of-file rules::           End-of-file rules* Miscellaneous::               Miscellaneous macros* User variables::              Values available to the user* YACC interface::              Interfacing with @code{yacc}* Options::                     Options* Performance::                 Performance considerations* C++::                         Generating C++ scanners* Incompatibilities::           Incompatibilities with @code{lex} and POSIX* Diagnostics::                 Diagnostics* Files::                       Files* Deficiencies::                Deficiencies / Bugs* See also::                    See also* Author::                      Author@c * Index::                       Index@end menu@end ifinfo@node Name, Synopsis, Top, Top@section Nameflex - fast lexical analyzer generator@node Synopsis, Overview, Name, Top@section Synopsis@exampleflex [-bcdfhilnpstvwBFILTV78+? -C[aefFmr] -ooutput -Pprefix -Sskeleton][--help --version] [@var{filename} @dots{}]@end example@node Overview, Description, Synopsis, Top@section OverviewThis manual describes @code{flex}, a tool for generating programsthat perform pattern-matching on text.  The manualincludes both tutorial and reference sections:@table @asis@item Descriptiona brief overview of the tool@item Some Simple Examples@item Format Of The Input File@item Patternsthe extended regular expressions used by flex@item How The Input Is Matchedthe rules for determining what has been matched@item Actionshow to specify what to do when a pattern is matched@item The Generated Scannerdetails regarding the scanner that flex produces;how to control the input source@item Start Conditionsintroducing context into your scanners, andmanaging "mini-scanners"@item Multiple Input Buffershow to manipulate multiple input sources; how toscan from strings instead of files@item End-of-file Rulesspecial rules for matching the end of the input@item Miscellaneous Macrosa summary of macros available to the actions@item Values Available To The Usera summary of values available to the actions@item Interfacing With Yaccconnecting flex scanners together with yacc parsers@item Optionsflex command-line options, and the "%option"directive@item Performance Considerationshow to make your scanner go as fast as possible@item Generating C++ Scannersthe (experimental) facility for generating C++scanner classes@item Incompatibilities With Lex And POSIXhow flex differs from AT&T lex and the POSIX lexstandard@item Diagnosticsthose error messages produced by flex (or scannersit generates) whose meanings might not be apparent@item Filesfiles used by flex@item Deficiencies / Bugsknown problems with flex@item See Alsoother documentation, related tools@item Authorincludes contact information@end table@node Description, Examples, Overview, Top@section Description@code{flex} is a tool for generating @dfn{scanners}: programs whichrecognized lexical patterns in text.  @code{flex} reads the giveninput files, or its standard input if no file names aregiven, for a description of a scanner to generate.  Thedescription is in the form of pairs of regular expressionsand C code, called @dfn{rules}. @code{flex} generates as output a Csource file, @file{lex.yy.c}, which defines a routine @samp{yylex()}.This file is compiled and linked with the @samp{-lfl} library toproduce an executable.  When the executable is run, itanalyzes its input for occurrences of the regularexpressions.  Whenever it finds one, it executes thecorresponding C code.@node Examples, Format, Description, Top@section Some simple examplesFirst some simple examples to get the flavor of how oneuses @code{flex}.  The following @code{flex} input specifies a scannerwhich whenever it encounters the string "username" willreplace it with the user's login name:@example%%username    printf( "%s", getlogin() );@end exampleBy default, any text not matched by a @code{flex} scanner iscopied to the output, so the net effect of this scanner isto copy its input file to its output with each occurrenceof "username" expanded.  In this input, there is just onerule.  "username" is the @var{pattern} and the "printf" is the@var{action}.  The "%%" marks the beginning of the rules.Here's another simple example:@example        int num_lines = 0, num_chars = 0;%%\n      ++num_lines; ++num_chars;.       ++num_chars;%%main()        @{        yylex();        printf( "# of lines = %d, # of chars = %d\n",                num_lines, num_chars );        @}@end exampleThis scanner counts the number of characters and thenumber of lines in its input (it produces no output otherthan the final report on the counts).  The first linedeclares two globals, "num_lines" and "num_chars", whichare accessible both inside @samp{yylex()} and in the @samp{main()}routine declared after the second "%%".  There are two rules,one which matches a newline ("\n") and increments both theline count and the character count, and one which matchesany character other than a newline (indicated by the "."regular expression).A somewhat more complicated example:@example/* scanner for a toy Pascal-like language */%@{/* need this for the call to atof() below */#include <math.h>%@}DIGIT    [0-9]ID       [a-z][a-z0-9]*%%@{DIGIT@}+    @{            printf( "An integer: %s (%d)\n", yytext,                    atoi( yytext ) );            @}@{DIGIT@}+"."@{DIGIT@}*        @{            printf( "A float: %s (%g)\n", yytext,                    atof( yytext ) );            @}if|then|begin|end|procedure|function        @{            printf( "A keyword: %s\n", yytext );            @}@{ID@}        printf( "An identifier: %s\n", yytext );"+"|"-"|"*"|"/"   printf( "An operator: %s\n", yytext );"@{"[^@}\n]*"@}"     /* eat up one-line comments */[ \t\n]+          /* eat up whitespace */.           printf( "Unrecognized character: %s\n", yytext );%%main( argc, argv )int argc;char **argv;    @{    ++argv, --argc;  /* skip over program name */    if ( argc > 0 )            yyin = fopen( argv[0], "r" );    else            yyin = stdin;    yylex();    @}@end exampleThis is the beginnings of a simple scanner for a languagelike Pascal.  It identifies different types of @var{tokens} andreports on what it has seen.The details of this example will be explained in thefollowing sections.@node Format, Patterns, Examples, Top@section Format of the input fileThe @code{flex} input file consists of three sections, separatedby a line with just @samp{%%} in it:@exampledefinitions%%rules%%user code@end exampleThe @dfn{definitions} section contains declarations of simple@dfn{name} definitions to simplify the scanner specification,and declarations of @dfn{start conditions}, which are explainedin a later section.Name definitions have the form:@examplename definition@end exampleThe "name" is a word beginning with a letter or anunderscore ('_') followed by zero or more letters, digits, '_',or '-' (dash).  The definition is taken to begin at thefirst non-white-space character following the name andcontinuing to the end of the line.  The definition cansubsequently be referred to using "@{name@}", which willexpand to "(definition)".  For example,