## $Id: LWP.pm,v 1.152 2007/08/05 13:23:32 gisle Exp $package LWP;$VERSION = "5.808";sub Version { $VERSION; }require 5.005;require LWP::UserAgent;  # this should load everything you need1;__END__=head1 NAMELWP - The World-Wide Web library for Perl=head1 SYNOPSIS  use LWP;  print "This is libwww-perl-$LWP::VERSION\n";=head1 DESCRIPTIONThe libwww-perl collection is a set of Perl modules which provides asimple and consistent application programming interface (API) to theWorld-Wide Web.  The main focus of the library is to provide classesand functions that allow you to write WWW clients. The library alsocontain modules that are of more general use and even classes thathelp you implement simple HTTP servers.Most modules in this library provide an object oriented API.  The useragent, requests sent and responses received from the WWW server areall represented by objects.  This makes a simple and powerfulinterface to these services.  The interface is easy to extendand customize for your own needs.The main features of the library are:=over 3=item *Contains various reusable components (modules) that can beused separately or together.=item *Provides an object oriented model of HTTP-style communication.  Withinthis framework we currently support access to http, https, gopher, ftp, news,file, and mailto resources.=item *Provides a full object oriented interface ora very simple procedural interface.=item *Supports the basic and digest authorization schemes.=item *Supports transparent redirect handling.=item *Supports access through proxy servers.=item *Provides parser for F<robots.txt> files and a framework for constructing robots.=item *Supports parsing of HTML forms.=item *Implements HTTP content negotiation algorithm that canbe used both in protocol modules and in server scripts (like CGIscripts).=item *Supports HTTP cookies.=item *Some simple command line clients, for instance C<lwp-request> and C<lwp-download>.=back=head1 HTTP STYLE COMMUNICATIONThe libwww-perl library is based on HTTP style communication. Thissection tries to describe what that means.Let us start with this quote from the HTTP specification document<URL:http://www.w3.org/pub/WWW/Protocols/>:=over 3=itemThe HTTP protocol is based on a request/response paradigm. A clientestablishes a connection with a server and sends a request to theserver in the form of a request method, URI, and protocol version,followed by a MIME-like message containing request modifiers, clientinformation, and possible body content. The server responds with astatus line, including the message's protocol version and a success orerror code, followed by a MIME-like message containing serverinformation, entity meta-information, and possible body content.=backWhat this means to libwww-perl is that communication always take placethrough these steps: First a I<request> object is created andconfigured. This object is then passed to a server and we get aI<response> object in return that we can examine. A request is alwaysindependent of any previous requests, i.e. the service is stateless.The same simple model is used for any kind of service we want toaccess.For example, if we want to fetch a document from a remote file server,then we send it a request that contains a name for that document andthe response will contain the document itself.  If we access a searchengine, then the content of the request will contain the queryparameters and the response will contain the query result.  If we wantto send a mail message to somebody then we send a request object whichcontains our message to the mail server and the response object willcontain an acknowledgment that tells us that the message has beenaccepted and will be forwarded to the recipient(s).It is as simple as that!=head2 The Request ObjectThe libwww-perl request object has the class name C<HTTP::Request>.The fact that the class name uses C<HTTP::> as aprefix only implies that we use the HTTP model of communication.  Itdoes not limit the kind of services we can try to pass this I<request>to.  For instance, we will send C<HTTP::Request>s both to ftp andgopher servers, as well as to the local file system.The main attributes of the request objects are:=over 3=item *The B<method> is a short string that tells what kind ofrequest this is.  The most common methods are B<GET>, B<PUT>,B<POST> and B<HEAD>.=item *The B<uri> is a string denoting the protocol, server andthe name of the "document" we want to access.  The B<uri> mightalso encode various other parameters.=item *The B<headers> contain additional information about therequest and can also used to describe the content.  The headersare a set of keyword/value pairs.=item *The B<content> is an arbitrary amount of data.=back=head2 The Response ObjectThe libwww-perl response object has the class name C<HTTP::Response>.The main attributes of objects of this class are:=over 3=item *The B<code> is a numerical value that indicates the overalloutcome of the request.=item *The B<message> is a short, human readable string thatcorresponds to the I<code>.=item *The B<headers> contain additional information about theresponse and describe the content.=item *The B<content> is an arbitrary amount of data.=backSince we don't want to handle all possible I<code> values directly inour programs, a libwww-perl response object has methods that can beused to query what kind of response this is.  The most commonly usedresponse classification methods are:=over 3=item is_success()The request was was successfully received, understood or accepted.=item is_error()The request failed.  The server or the resource might not beavailable, access to the resource might be denied or other things mighthave failed for some reason.=back=head2 The User AgentLet us assume that we have created a I<request> object. What do weactually do with it in order to receive a I<response>?The answer is that you pass it to a I<user agent> object and thisobject takes care of all the things that need to be done(like low-level communication and error handling) and returnsa I<response> object. The user agent represents yourapplication on the network and provides you with an interface thatcan accept I<requests> and return I<responses>.The user agent is an interface layer betweenyour application code and the network.  Through this interface you areable to access the various servers on the network.The class name for the user agent is C<LWP::UserAgent>.  Everylibwww-perl application that wants to communicate should create atleast one object of this class. The main method provided by thisobject is request(). This method takes an C<HTTP::Request> object asargument and (eventually) returns a C<HTTP::Response> object.The user agent has many other attributes that let youconfigure how it will interact with the network and with yourapplication.=over 3=item *The B<timeout> specifies how much time we give remote servers torespond before the library disconnects and creates aninternal I<timeout> response.=item *The B<agent> specifies the name that your application should use when itpresents itself on the network.=item *The B<from> attribute can be set to the e-mail address of the personresponsible for running the application.  If this is set, then theaddress will be sent to the servers with every request.=item *The B<parse_head> specifies whether we should initialize responseheaders from the E<lt>head> section of HTML documents.=item *The B<proxy> and B<no_proxy> attributes specify if and when to go througha proxy server. <URL:http://www.w3.org/pub/WWW/Proxies/>=item *The B<credentials> provide a way to set up user names andpasswords needed to access certain services.=backMany applications want even more control over how they interactwith the network and they get this by sub-classingC<LWP::UserAgent>.  The library includes asub-class, C<LWP::RobotUA>, for robot applications.=head2 An ExampleThis example shows how the user agent, a request and a response arerepresented in actual perl code:  # Create a user agent object  use LWP::UserAgent;  $ua = LWP::UserAgent->new;  $ua->agent("MyApp/0.1 ");  # Create a request  my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search');  $req->content_type('application/x-www-form-urlencoded');  $req->content('query=libwww-perl&mode=dist');  # Pass request to the user agent and get a response back  my $res = $ua->request($req);  # Check the outcome of the response  if ($res->is_success) {      print $res->content;  }