-------------------------------------------------------------------------------$Id: README.txt 4955 2007-03-03 02:47:24Z gojomo $-------------------------------------------------------------------------------0.0 Contents1.0 Introduction2.0 Webmasters!3.0 Getting Started4.0 Developer Documentation5.0 Release History6.0 License1.0 IntroductionHeritrix is the Internet Archive's open-source, extensible, web-scale,archival-quality web crawler project. Heritrix (sometimes spelled heretrix, ormisspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic wordfor heiress (woman who inherits). Since our crawler seeks to collect andpreserve the digital artifacts of our culture for the benefit of futureresearchers and generations, this name seemed apt.2.0 Webmasters!Heritrix is designed to respect the robots.txt <http://www.robotstxt.org/wc/robots.html> exclusion directives and META robotstags <http://www.robotstxt.org/wc/exclusion.html#meta>. If you notice ourcrawler behaving poorly, please send us email at archive-crawler-agent *at*lists *dot* sourceforge *dot* net. 3.0 Getting StartedSee the User Manual at ./docs/articles/user_manual/index.html or at<http://crawler.archive.org/articles/user_manual/index.html>.4.0 Developer DocumentationSee ./docs/articles/developer_manual/index.html or<http://crawler.archive.org/articles/developer_manual/index.html>.5.0 Release HistorySee the Heritrix Release Notes in the local directorydocs/articles/releasenotes/index.html if this is a binary release orat http://crawler.archive.org/articles/releasenotes/index.html.6.0 LicenseHeritrix is free software; you can redistribute it and/or modify itunder the terms of the GNU Lesser Public License as published by theFree Software Foundation; either version 2.1 of the License, or anylater version.                                                                                Heritrix is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty ofMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See theGNU Lesser Public License for more details.                                                                                You should have received a copy of the GNU Lesser Public Licensealong with Heritrix (See LICENSE.txt); if not, write to the FreeSoftware Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA02111-1307  USA                                                                                For the licenses for libraries used by Heritrix and included in itsdistribution, see below in section '8.0 Dependencies'.