Flow-Portscan module documentation:Initial Discussions: Marc Norton Dan Roelker Chris GreenImplementation: Chris Green everything but sfxhashDocumentation to-dos: - explain the time domains - explain the scoring domainsDocumentation last updated: 2003-09-22  This is module is designed to detect rapid portscans based off flow  creation in the flow preprocessors.  The goals is to catch one->many  hosts and one->many ports scans.  The flow preprocessor to portscan recognizer is taken from  experience with spp_conversation/portscan2 by Jason Larsen & Jed  Haile and ipaudit by Jon Rifkin.  This subsystem became a bit more complicated than originally  intended but it does a good job of mitigating false positives from  devices such as squid proxies.  The new design is also a lot more  memory consistent than portscan1 or 2.  It also ignores single port  syn floods as they are a DoS, not a portscan.  Memory requirements should be way down from portscan2 architecture  though but there's slightly less information saved off.  The new  architecture operates similarly to a ring buffer.  When a scanner  has not been active in a long time, it's only reclaimed when there  is no more memory to use.  All of the prior methods for portscan detection in snort are  deprecated and will be removed in the near future.  If you have  custom code against conversation or one of the portscan  preprocessors, consider making it a module in flow or portscan.  Basic components:  2 Scoreboards ( One Talker, One Scanner )    Scoreboards contain information regarding timescales for a single    IP address.  There are two scoreboards, one for talkers (nodes    that are active on your network) and one for scanners (nodes that    have talked to a previously unknown port in your    server-watch-net)  1 Uniqueness tracker     The uniqueness tracker is used to determine if this connection     should count as something "new" for a particular IP.  It checks     if a connection is a new type of connection for a Source IP by     disregarding the source port.     Any change in (SIP,DIP,IP_PROTO,DPORT) indicates a new unique     connection and will be processed further for the server     statistics table and scoring.     This keeps things like a web page with 15 images from     rapidly increasing point scores with lots of accesses to the same     web server.  1 Server Statistics Tracker    This is used to track flows destined to the "server-watchnet"    and keep "hitcounts" on the number of times a particular service    has been requested with unique requests since snort has started.    This hitcount is tracked by (DIP,DPORT,PROTOCOL).    If a service is very popular, we can make connections to it be    ignored for scoring by comparing the hitcount to the    "server-ignore-limit". If we have more requests to this service    than the server-ignore-limit, then we will completely ignore this    service.  Similarly, the "server-scanner-limit" controls if a    request to a service counts as scanner points or as talker points.    If a request to a service is not in the server-watch-net, it will    count as talker points.    Caveat:    This does not perform validation that the service is connected    correctly so it is possible while learning that someone floods the    table with unique connections that it is possible to have    something become a service that you do not wish to be a service.    It's generally assumed that the learning time will occur at a time    where traffic is "typical". Future versions of snort should allow    this state to be saved and modifiable.    If this caveat is a concern in your environment, do not set a server    watchnet and rely only on talker scores.  Module Overview:  1) flow-portscan receives a new flow message from the flow module  2) The uniqueness tracker determines if message is a new type of      flow by looking for changes in (SIP,DIP,IP_PROTO,DPORT).  If this is     not unique, and the TCP flags are normal, exit out.  3) If this connection is to an Destination IP in the server-watchnet:      During the "server-learning-time", it increments the hitcounts      for service popularity.       If it's otherwise just get the stored hitcount.  If the hitcount      is greater than the server-ignore-limit, exit out.  If it's less      than the server-scanner-limit, mark the incremented points as      scanner points.  4) A connection is marked as either a talker or a scanner by step 3.     There are 4 time scales; 2 each for the IP Scanner and IP Talker.     The fixed timescales detect N events in M seconds.  This is the     typical type of portscan alert.     The sliding timescales adjust the "score reset point" on each     event after the first.  This adjusts the side of the window we're     detecting portscan events in by taking            end = end + ((end - start) * sliding-scale-factor)     Each time scale has it's own point tally that is incremented per     new flow.  Each set of points only touches either the         talker-fixed-score and talker-sliding-score     OR         scanner-fixed-score and scanner-sliding-score    5) Evaluate the score against individual thresholds, either talker     or scanner.      if(fixed_limit <= fixed_score)       generate_alert()flow-portscan options:General Note: higher row counts will take more memory away from thememory caps for a specific subsystem.  In the snort output, this isreferred to as "overhead bytes" and the percentage of overheadencountered will be shown.  Higher row counts provide a larger hashtable to minimize collisions and have a faster overall processing timeat the expense of memory.  The hash tables themselves use apseudorandom hardening salt that is picked at initialization time.scoreboard-memcap-talker     <bytes>  Number of bytes to use for the talker table    scoreboard-rows-talker       <count>  Number of rows to use for the talker tablescoreboard-rows-scanner      <count>  Number of rows to use for the scanner tablescoreboard-memcap-scanner    <bytes>  Number of bytes to use for the scanner tablescanner-fixed-threshold      <integer>  Number of points that a scanner must accumulate in the  scanner-fixed-window time range.  Set to 0 to disable this type of  alert.scanner-sliding-threshold    <integer>  Number of points that a scanner must accumulate in   scanner-sliding-window time range. Set to 0 to disable this type of  alert.scanner-fixed-window         <integer>  How many seconds we should go before resetting the fixed scanner scorescanner-sliding-window       <integer>  How many seconds we should go before resetting the sliding scanner scorescanner-sliding-scale-factor <float>  How much to increase the sliding window by each time we get a new  sliding scanner entry.  It's current size + (<scale factor> * current_size)talker-fixed-threshold       <integer>  Number of points that a scanner must accumulate in   talker-fixed-window time range. Set to 0 to disable this type of  alert.talker-sliding-threshold     <integer>  Number of points that a scanner must accumulate in   talker-sliding-window time range. Set to 0 to disable this type of  alert.talker-fixed-window          <integer>  How many seconds we should go before resetting the fixed talker scoretalker-sliding-window        <integer>  How many seconds we should go before resetting the sliding talker scoretalker-sliding-scale-factor  <float>  How much to increase the sliding window by each time we get a new  sliding talker entry.  It's current size + (<scale factor> * current_size)unique-memcap                <bytes>  How many bytes to allocate to the uniqueness tracker. The more  memory given, the less that connections to a busy server will appear  as a scan target on a popular service. unique-rows                  <integer>  How many rows to allocate for the uniqueness tracker.  server-memcap                <bytes>  How many bytes to allocate for server learningserver-rows                  <integer>  How many rows to allocate for server learningserver-watchnet              <ip list in snort notation>  The IP list of what machines to learn services on.  Busy servers  should be placed here to help the portscan detector learn what  services are requested on the network.src-ignore-net                   <ip list in snort notation>  The IP list of what Source IP's to ignore.dst-ignore-net                   <ip list in snort notation>  The IP list of what Destination IP's to ignore.tcp-penalties                <on|off>  If this is enabled, when a new tcp flow enters the portscan  detection set, check the TCP flags for non-standard session  initiators and assign penalty points for odd combinations such as  SYN+FIN  Flag mapping:  SYN or SYN+ECN bits                   == base_score ( defaults to 1 point )  SYN+FIN+TH_ACK and anything else      == 5 points  SYN+FIN and anything else without ack == 3 points  Anything else                         == 2 pointsserver-learning-time         <seconds>  How many seconds we should keep increment hitcounts of services on  IP's in the server-watchnetserver-ignore-limit          <hit count>  How many requests a port on an IP in the server-watchnet must see  before it is ignored for the purposes of portscans.server-scanner-limit         <hit count>  How many requests a port on an IP in the server-watchnet must see  before it is is treated as a talker rather than a scanner.  This is  a minimum number of requests that must be seen during the  server-learning-time for the flow to be treated as a talker  connection rather than as a scanner connection.alert-mode                   <once|all>  In once mode, alert only on the first time we get a scan entry hit.  This dramatically reduces clutter because the scan alert in the  first place tells one to look for other event types.  On All, alert each time the score increases beyond a threshold.output-mode                  <msg|pktkludge>  msg       - a variable text message with the scores included  pktkludge - generate a fake pkt and use the Logging output systemdumpall 1  When snort is exiting, dump the entire contents of the server table,  the uniqueness tracker table, and the scoreboard entries.  This is  ' useful if you suspect an underlying bug in the algorithms used or if  you would just like to see what it has learned.Example Configuration:preprocessor flow: stats_interval 0preprocessor flow-portscan: \server-watchnet [10.0.0.0/8] \unique-memcap 5000000 unique-rows 50000 \tcp-penalties on \server-scanner-limit 50 \alert-mode all \output-mode msg \server-learning-time 3600