project</B></A> of UC Berkeley is not focused on network simulation, it has 
  also come to our attention because of its similarity to the autonomous 
  component architecture in building component-based architectures for 
  general-purpose simulation. </LI></UL>
<BLOCKQUOTE>
  <P>Ptolemy mainly focuses on modeling and simulation of concurrent, real-time, 
  embedded systems, but shares similar philosophy with us in terms of 
  component-based models. They use well-defined <I>models of computation</I> to 
  govern the interaction between components. The models of computation they have 
  developed include <I>communicating sequential processes</I>, <I>continuous</I> 
  <I>time</I>, <I>discrete events</I>, <I>distributed discrete events </I>and so 
  on. These models, however, have their own communication semantics. Components 
  derived from different models can not be mixed together to form truly 
  heterogeneous systems. <I>J-Sim</I>, on the other hand, has a simple, unified 
  and concrete component model. Components developed in different domains of 
  application can easily be integrated as long as the contracts are 
matched.</P></BLOCKQUOTE>
<H2><A name=s2>2 Quantitative Comparison</A></H2>
<H3><A name=ex1>2.1 Experiment 1</A></H3><B>Scenario</B>: Multiple TCP 
connections through one bottleneck link, <I>n</I> TCP connections, 2<I>n</I> 
hosts, 2 routers and one bottleneck link with buffer 4<I>n</I> Kbytes, packet 
size=1Kbytes.<BR><IMG height=291 src="J-Sim Evaluation.files/setup.png" 
width=773><BR><B>Parameter</B>: Number of TCP connections, <I>n</I>; varied from 
10 to 10,000.<BR><B>Characteristics of the test</B>: 
<UL>
  <LI>The number of the packets being transmitted over the bottleneck link (and 
  therefore the number of packet events) is irrelevant to <I>n</I>. 
  <LI>Event queue is at least <I>n</I> (TCP timeout event). 
  <LI>Routing table size is (<I>n</I>+1). </LI></UL><B>Results collected</B>: 
Scenario setup time (time to create and configure the network before simulation 
starts) and time to complete 100-, 500-, and 1000-second 
simulation.<BR><B>Testee</B>: <I>J-Sim1.0</I>, Ns-2 v2.1b8a and SSFNET 
v1.3.<BR><B>Machine</B>:Dual PentiumIII 600MHz, 256MByte RAM, Linux 
2.4.8.<BR><B>JVM</B>: (for <I>J-Sim</I> and SSFNET) <A 
href="http://www.ibm.com/java">IBM JDK1.3</A> and <A 
href="http://www.blackdown.org/">Blackdown v1.3.1</A>.<BR><B>Arguments passed to 
JVM</B>: "<CODE>-Xms200000000 -Xmx200000000</CODE>". 
<HR>
<B>Result (Setup time)</B>: For Javasim, we set up the scenario in three 
different ways: Java program (J-Sim/Java), Tcl script (J-Sim/Tcl) and Python 
script (J-Sim/Python).<BR>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=565 src="J-Sim Evaluation.files/setup_ibm1.png" 
    width=695></TD>
    <TD><IMG height=559 src="J-Sim Evaluation.files/setup_ibm2.png" 
    width=684></TD></TR></TBODY></TABLE><I>* Results vary a lot when running SSFNET 
over 900 TCP connections probably due to the garbage collection process in IBM's 
JVM.</I> 
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=559 src="J-Sim Evaluation.files/setup_blackdown1.png" 
      width=693></TD>
    <TD><IMG height=558 src="J-Sim Evaluation.files/setup_blackdown2.png" 
      width=682></TD></TR></TBODY></TABLE><I>* Out-of-memory exception occurs when 
setting up the scenario for more than 7,500 TCP connections in SSFNET.</I> 
<HR>
<B>Result (Completion time, IBM JDK1.3)</B>: For J-Sim, we implement both a 
real-time simulation engine and a pure event-based simulation engine to observe 
the additional overhead incurred by the real-time simulation technique.<BR>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=555 src="J-Sim Evaluation.files/100_ibm1.png" 
width=693></TD>
    <TD><IMG height=560 src="J-Sim Evaluation.files/100_ibm2.png" 
  width=688></TD></TR></TBODY></TABLE><I>* Results vary a lot when running SSFNET 
over 900 TCP connections probably due to the garbage collection process in IBM's 
JVM. As such, the results of SSFNET is not shown in the following 
figures.</I><BR><I>* The sawtooth pattern in the ns-2 curves (including those in 
the figures below) is due to the instability of the calendar queue. That is, the 
performance of the calendar queue is sensitive to the enqueue pattern of the 
simulation events.</I> 
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=553 src="J-Sim Evaluation.files/500_ibm1.png" 
width=697></TD>
    <TD><IMG height=553 src="J-Sim Evaluation.files/500_ibm2.png" 
  width=678></TD></TR></TBODY></TABLE>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=551 src="J-Sim Evaluation.files/1000_ibm1.png" 
    width=696></TD>
    <TD><IMG height=554 src="J-Sim Evaluation.files/1000_ibm2.png" 
    width=682></TD></TR></TBODY></TABLE>
<HR>
<B>Result (Completion time, Blackdown1.3.1)</B>:<BR>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=558 src="J-Sim Evaluation.files/100_blackdown1.png" 
      width=697></TD>
    <TD><IMG height=553 src="J-Sim Evaluation.files/100_blackdown2.png" 
      width=682></TD></TR></TBODY></TABLE>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=552 src="J-Sim Evaluation.files/500_blackdown1.png" 
      width=693></TD>
    <TD><IMG height=558 src="J-Sim Evaluation.files/500_blackdown2.png" 
      width=679></TD></TR></TBODY></TABLE>
<TABLE>
  <TBODY>
  <TR>
    <TD><IMG height=555 src="J-Sim Evaluation.files/1000_blackdown1.png" 
      width=691></TD>
    <TD><IMG height=556 src="J-Sim Evaluation.files/1000_blackdown2.png" 
      width=681></TD></TR></TBODY></TABLE><I>* Out-of-memory exception occurs when 
running simulation for more than 3,310 TCP connections in SSFNET.</I> 
<HR>
<B>Conclusions from the experiment</B>:<BR>
<OL>
  <LI>Scenario setup time, often neglected, takes a non-negligible, and even 
  larger portion of the entire time of a simulation run. For example, it takes 
  about 393 seconds for ns-2 to set up the 4000 TCP connections scenario and 
  takes about 181 seconds to complete a 1000-second simulation. 
  <LI>J-Sim scales better in terms of memory usage. While Javasim can run 
  through 10,000 TCP connections without using up memory, ns-2 causes system to 
  thrash at about 4,500 TCP connections, and SSFNET throws out-of-memory 
  exception at about 3,310 TCP connections. 
  <LI>J-Sim suffers from inter-component communication overhead and JVM. And the 
  real-time simulation engine incurs more overhead than the pure event 
  simulation engine. The offsets of the completion time at small number of TCP 
  connections in all figures are the evidence. 
  <LI>As for the scalability in terms of the completion time v.s. number of 
  connections, J-Sim performs better than SSFNET as J-Sim's curves grow more 
  slowly than SSFNET's. J-Sim also performs better than ns-2 when simulation 
  time is 100 seconds, but worse than ns-2 when simulation time is 500 and 1000 
  seconds, before ns-2 uses up memory. </LI></OL><B>Scripts and modified source 
files</B>: 
<UL>
  <LI><I>J-Sim</I>: (v1.0) 
  <UL>
    <LI>Java: <A 
    href="http://www.j-sim.org/experiment/ex1/j-sim/OneBottleneck.java">OneBottleneck.java</A>. 
<PRE>Usage:
	java OneBottleneck &lt;#connections&gt; &lt;simulationTime&gt; ?interval&lt;nn&gt;? ?event? ?radix?
Argument:
	interval&lt;nn&gt;: interval (in simulation time) to report wall time elapsed
	event: use event simulation engine instead of real-time simulation engine (default)
	radix: use radix-tree routing table instead of multifield routing table (default)
Example:
	java -Xms200000000 -Xmx200000000 OneBottleneck 100 1000 interval200 event radix </PRE>
    <LI>Tcl: <A 
    href="http://www.j-sim.org/experiment/ex1/j-sim/onebottleneck.tcl">onebottleneck.tcl</A>. 
<PRE>Usage:
	java drcl.ruv.System -u onebottleneck.tcl &lt;#connections&gt; &lt;simulationTime&gt; ?interval&lt;nn&gt;? ?event? ?radix?
Argument:
	See "Java" above.
Example:
	java -Xms200000000 -Xmx200000000 drcl.ruv.System -u onebottleneck.tcl 100 1000 interval200 event radix </PRE>
    <LI>Python: <A 
    href="http://www.j-sim.org/experiment/ex1/j-sim/onebottleneck.py">onebottleneck.py</A>. 
<PRE>Usage:
	java drcl.ruv.System -u -s drcl.ruv.ShellPython onebottleneck.py &lt;#connections&gt; &lt;simulationTime&gt; ?interval&lt;nn&gt;? ?event? ?radix?
Argument:
	See "Java" above.
Example:
	java -Xms200000000 -Xmx200000000 drcl.ruv.System -u -s drcl.ruv.ShellPython onebottleneck.py 100 1000 interval200 event radix </PRE></LI></UL>
  <LI>ns-2: (v2.1b8a) 
  <UL>
    <LI>Script: <A 
    href="http://www.j-sim.org/experiment/ex1/ns2/ns2_test.tcl">ns2_test.tcl</A>. 
<PRE>Usage:
	ns2 ns2_test.tcl &lt;#connections&gt; &lt;simulationTime&gt;
	</PRE>
    <LI>Modified source files: (Look for "Tyan" in the files for modifications) 
    <UL>
      <LI><A 
      href="http://www.j-sim.org/experiment/ex1/ns2/ns-lib.tcl">ns-2.1b8a/tcl/lib/ns-lib.tcl</A>: 
      Added the "startTime_" argument in proceduer "Simulator instproc run" for 
      calculating the additional setup time. So for ns-2, the scenario setup 
      time is the sum of the time spent in the script before invoking "$ns_ run" 
      and the setup time collected above. 
      <LI><A 
      href="http://www.j-sim.org/experiment/ex1/ns2/scheduler.h">ns-2.1b8a/scheduler.h</A> 
      and <A 
      href="http://www.j-sim.org/experiment/ex1/ns2/scheduler.cc">ns-2.1b8a/scheduler.cc</A>: 
      added fields and codes to collect simulation engine information such as 
      number of events, maximum queue length, current queue length and so on. 
      </LI></UL></LI></UL>
  <LI>SSFNET: (v1.3) 
  <UL>
    <LI>Script: <A 
    href="http://www.j-sim.org/experiment/ex1/ssfnet/gen.tcl">gen.tcl</A> for 
    generating the DML file. <PRE>Usage:
	tclsh gen.tcl &lt;#connections&gt;
	</PRE>
    <LI><A 
    href="http://www.j-sim.org/experiment/ex1/ssfnet/Makefile">Makefile</A>: 
    executing the DML file generated by gen.tcl. Simulation time (1000 second) 
    is hard coded in the Makefile. 
    <LI>Modified source files: (Look for "Tyan" in the files for modifications) 
    <UL>
      <LI><A 
      href="http://www.j-sim.org/experiment/ex1/ssfnet/Net.java">src/SSF/Net/Net.java</A>: 
      added the "MyTimer" inner class and codes for printing out execution 
      profiles. </LI></UL></LI></UL></LI></UL>
<P>&nbsp;</P>
<P>&nbsp;</P></BODY></HTML>