<P><BR></P>

<TD>

<P>The total amount of CPU time used by the process since it began</P>

<TR>

<TD>

<P>COMD</P>

<TD>

<P><BR></P>

<TD>

<P>The command that generated the process</P></TABLE>

<P>If your problem is immediate performance, you can disregard processes that are sleeping, stopped, or waiting on terminal I/O, as these will probably not be the source of the degradation. Look instead for the jobs that are ready to run, blocked for disk 

I/O, or paging.

<BR></P>

<PRE>% ps -el

 F S   UID   PID  PPID  C PRI NI     ADDR     SZ    WCHAN TTY        TIME COMD

19 T     0     0     0 80   0 SY e00ec978      0          ?          0:01 sched

19 S     0     2     0 80   0 SY f5735000      0 e00eacdc ?          0:05 pageout

 8 S  1001  1382     1 80  40 20 f5c6a000   1227 e00f887c console    0:02 mailtool

 8 S  1001  1386     1 80  40 20 f60ed000    819 e00f887c console    0:28 perfmete

 8 S  1001 28380 28377 80  40 20 f67c0000   5804 f5cfd146 ?         85:02 sqlturbo

 8 S  1001 28373     1 80  40 20 f63c6000   1035 f63c61c8 ?          0:07 cdrl_mai

 8 S  1001 28392     1 80  40 20 f67ce800   1035 f67ce9c8 ?          0:07 cdrl_mai

 8 S  1001 28391 28388 80  40 20 f690a800   5804 f60dce46 ?         166:39 sqlturbo

 8 S  1001 28361     1 80  60 20 f67e1000  30580 e00f887c ?         379:35 mhdms

 8 S  1001 28360     1 80  40 20 f68e1000  12565 e00f887c ?         182:22 mhharris

 8 O  1001 10566 10512 19  70 20 f6abb800    152          pts/14     0:00 ps

 8 S  1001 28388     1 80  40 20 f6384800    216 f60a0346 ?         67:51 db_write

 8 S  1000  7750  7749 80  40 20 f6344800   5393 f5dad02c pts/2     31:47 tbinit

 8 O  1001  9538  9537 80  81 22 f6978000   5816          ?         646:57 sqlturbo

 8 S  1033  3735  3734164  40 20 f63b8800    305 f60e0d46 pts/9      0:00 ksh

 8 S  1033  5228  5227 80  50 20 f68a8800    305 f60dca46 pts/7      0:00 ksh

 8 S  1001 28337     1 80  99 20 f6375000  47412 f63751c8 ?         1135:50 velox_ga</PRE>

<P>The following are tips for using ps to determine why system performance is suffering.

<BR></P>

<P>Look at the UID (user ID) fields for a number of identical jobs that are being submitted by the same user. This is often caused by a user who runs a script that starts a lot of background jobs without waiting for any of the jobs to complete. Sometimes 
you can safely use kill to terminate some of the jobs. Whenever you can, you should discuss this with the user before you take action. In any case, be sure the user is educated in the proper use of the system to avoid a replication of the problem. In the 
example, User ID 1001 has multiple instances of the same process running. In this case, it is a normal situation, in which multiple processes are spawned at the same time for searching through database tables to increase interactive performance.

<BR></P>

<P>Look at the TIME fields for a process that has accumulated a large amount of CPU time. In the example, you can see the large amount of time acquired by the processes whose command is shown as velox_ga. This may indicate that the process is in an 
infinite loop, or that something else is wrong with its logic. Check with the user to determine whether it is appropriate to terminate the job. If something is wrong, ask the user if a dump of the process would assist in debugging it (check your UNIX 
system's reference material for commands, such as gcore, that can dump a process).

<BR></P>

<P>Request the -l option and look at the SZ fields for processes that are consuming too much memory. In the example you can see the large amount of memory acquired by the processes whose command is shown as velox_ga. You could check with the user of this 
process to try to determine why it behaves this way. Attempting to renice the process may simply prolong the problem that it is causing, so you may have to kill the job instead. SZ fields may also give you a clue as to memory shortage problems caused by 
this particular combination of jobs. You can use vmstat or sar -wpgr to check the paging and swapping statistics that are examined.

<BR></P>

<P>Look for processes that are consuming inordinate CPU resources. Request the -c option and look at the CLS fields for processes that are running at inappropriately high priorities. Use the nice command to adjust the nice value of the process. Beware in 
particular of any real-time (RT) process, which can often dominate the system. If the priority is higher than you expected, you should check with the user to determine how it was set. If he is resetting the priority because he has figured out the superuser 

password, dissuade him from doing this. (See Chapter 19 to find out more about using the nice command to modify the priorities of processes.)

<BR></P>

<P>If the processes that are running are simply long-running, CPU-intensive jobs, ask the users if you can nice them to a lower priority or if they can run them at night, when other users will not be affected by them.

<BR></P>

<P>Look for processes that are blocking on I/O. Many of the example processes are in this state. When that is the case, the disk subsystem probably requires tuning. The section &quot;Monitoring Disk Performance Using vmstat&quot; examines how to 
investigate problems with your disk I/O. If the processes are trying to read/write over NFS, this may be a symptom that the NFS server to which they are attached is down, or that the network itself is hung.

<BR></P>

<H3 ALIGN="CENTER">

<CENTER><A ID="I12" NAME="I12">

<FONT SIZE=4><B>Monitoring Memory Utilization</B>

<BR></FONT></A></CENTER></H3>

<P>You could say that one can never have too much money, be too thin, or have too much system memory. Memory sometimes becomes a problematic resource when programs that are running require more physical memory than is available. When this occurs UNIX 
systems begin a process called paging. During paging the system copies pages of physical memory to disk, and then allows the now-vacated memory to be used by the process that required the extra space. Occasional paging can be tolerated by most systems, but 

frequent and excessive paging is usually accompanied by poor system performance and unhappy users.

<BR></P>

<H4 ALIGN="CENTER">

<CENTER><A ID="I13" NAME="I13">

<FONT SIZE=3><B>UNIX Memory Management</B>

<BR></FONT></A></CENTER></H4>

<P>Paging uses an algorithm that selects portions, or pages, of memory that are not being used frequently and displaces them to disk. The more frequently used portions of memory, which may be the most active parts of a process, thus remain in memory, while 

other portions of the process that are idle get paged out.

<BR></P>

<P>In addition to paging, there is a similar technique used by the memory management system called swapping. Swapping moves entire processes, rather than just pages, to disk in order to free up memory resources. Some swapping may occur under normal 
conditions. That is, some processes may just be idle enough (for example, due to sleeping) to warrant their return to disk until they become active once more. Swapping can become excessive, however, when severe memory shortages develop. Interactive 
performance can degrade quickly when swapping increases since it often depends on keyboard-dependent processes (for example, editors) that are likely to be considered idle as they wait for you to start typing again.

<BR></P>

<P>As the condition of your system deteriorates, paging and swapping make increasing demands on disk I/O. This, in turn, may further slow down the execution of jobs submitted to the system. Thus, memory resource inadequacies may result in I/O resource 
problems.

<BR></P>

<P>By now, it should be apparent that it is important to be able to know if the system has enough memory for the applications that are being used on it.

<BR></P>

<HR ALIGN=CENTER>

<NOTE>

<IMG SRC="imp.gif" WIDTH = 68 HEIGHT = 35><B>TIP: </B>A rule of thumb is to allocate twice the swap space as you have physical memory. For example, if you have 32 MB of physical Random Access Memory (RAM) installed upon your system, you would set up 64 MB 

of swap space when configuring the system. The system would then use this diskspace for its memory management when displacing pages or processes to disk.

<BR></NOTE>

<HR ALIGN=CENTER>

<P>Both vmstat and sar provide information about the paging and swapping characteristics of a system. Let's start with vmstat. On the vmstat reports you will see information about page-ins, or pages moved from disk to memory, and page-outs, or pages moved 

from memory to disk. Further, you will see information about swap-ins, or processes moved from disk to memory, and swap-outs, or processes moved from memory to disk.

<BR></P>

<H4 ALIGN="CENTER">

<CENTER><A ID="I14" NAME="I14">

<FONT SIZE=3><B>Monitoring Memory Performance Using </B><B><I>vmstat</I></B>

<BR></FONT></A></CENTER></H4>

<P>The vmstat command is used to examine virtual memory statistics, and present data on process status, free and swap memory, paging activity, disk reports, CPU load, swapping, cache flushing, and interrupts. The format of the command is:

<BR></P>

<PRE>vmstat  <I>t</I> [<I>n</I>]</PRE>

<P>This command takes <I>n</I> samples, at <I>t</I> second intervals. For example, the following frequently used version of the command takes samples at 5-second intervals without stopping until canceled:

<BR></P>

<PRE>vmstat 5</PRE>

<P>The following screen shows the output from the SunOS variant of the command

<BR></P>

<PRE>vmstat -S 5</PRE>

<P>which provides extra information regarding swapping.

<BR></P>

<PRE> procs     memory            page            disk          faults           cpu

 r b w   swap  free  si  so pi po fr de sr s0 s3 s5 s5   in    sy   cs us sy id

 0 2 0  16516  9144   0   0  0  0  0  0  0  1  4 34 12  366  1396  675 14  9 76

 0 3 0 869384 29660   0   0  0  0  0  0  0  0  4 63 15  514 10759 2070 19 17 64

 0 2 0 869432 29704   0   0  0  0  0  0  0  4  3 64 11  490  2458 2035 16 13 72

 0 3 0 869448 29696   0   0  0  0  0  0  0  0  3 65 13  464  2528 2034 17 12 71

 0 3 0 869384 29684   0   0  0  0  0  0  0  1  3 68 18  551  2555 2136 16 14 70

 0 2 0 869188 29644   0   0  0  2  2  0  0  2  3 65 10  432  2495 2013 18  9 73

 0 3 0 869176 29612   0   0  0  0  0  0  0  0  3 61 16  504  2527 2053 17 11 71

 0 2 0 869156 29600   0   0  0  0  0  0  0  0  3 69  8  438 15820 2027 20 18 62</PRE>

<P>The fields in the vmstat report are the following:

<BR></P>

<TABLE BORDER>

<TR>

<TD>

<P>procs </P>

<TD>

<P>Reports the number of processes in each of the following states</P>

<TR>

<TD>

<P>r</P>

<TD>

<P>In the Run queue</P>

<TR>

<TD>

<P>b</P>

<TD>

<P>Blocked, waiting for resources</P>

<TR>

<TD>

<P>w</P>

<TD>

<P>Swapped, waiting for processing resources</P>

<TR>

<TD>

<P>memory </P>

<TD>

<P>Reports on real and virtual memory</P>

<TR>

<TD>

<P>swap</P>

<TD>

<P>Available swap space</P>

<TR>

<TD>

<P>free</P>

<TD>

<P>Size of free list</P>

<TR>

<TD>

<P>page </P>

<TD>

<P>Reports on page faults and paging, averaged over an interval (typically 5 seconds) and provided in units per second</P>

<TR>

<TD>

<P>re</P>

<TD>

<P>Pages reclaimed from the free list (not shown when the -S option is requested)</P>

<TR>

<TD>

<P>mf</P>

<TD>

<P>Minor faults (not shown when -S option is requested)</P>

<TR>

<TD>

<P>si</P>

<TD>

<P>Number of pages swapped in (only shown with the -S option)</P>

<TR>

<TD>

<P>so</P>

<TD>

<P>Number of pages swapped out (only shown with the -S option)</P>

<TR>

<TD>

<P>pi</P>

<TD>

<P>Kilobytes paged in</P>

<TR>

<TD>

<P>po</P>

<TD>

<P>Kilobytes paged out</P>

<TR>

<TD>

<P>fr</P>

<TD>

<P>Kilobytes freed</P>

<TR>

<TD>

<P>de</P>

<TD>

<P>Anticipated short-term memory shortfall</P>

<TR>

<TD>

<P>sr</P>

<TD>

<P>Pages scanned by clock algorithm, per second</P>

<TR>

<TD>

<P>disk</P>

<TD>

<P>Shows the number of disk operations per second</P>

<TR>

<TD>

<P>faults </P>

<TD>

<P>Shows the per-second trap/interrupt rates</P>

<TR>

<TD>

<P>in</P>

<TD>

<P>Device interrupts</P>

<TR>

<TD>

<P>sy</P>

<TD>

<P>System faults per second</P>

<TR>

<TD>

<P>cs</P>

<TD>

<P>CPU context switches</P>

<TR>

<TD>

<P>cpu </P>

<TD>

<P>Shows the use of CPU time</P>

<TR>

<TD>

<P>us</P>

<TD>

<P>User time</P>