\chapter{The connection protocol}

This is the protocol used for a single connection from
an output port to an input port.  We discuss how this
process gets initiated in the next section.
At the point of creation of a connection, the
following information is needed:

\begin{itemize}

\item An address -- the machine name and socket-port at which
the input port is listening.

\item The name of the input port.

\item The name of the output port associated with the
connection.  This name
needs to be retained for proper disconnection in some cases.
If the connection is not actually associated with a port,
but is initiated by an external entity, then the name is
not important and should be set to ``external'' (or any
name without a leading slash characher).

\end{itemize}

\section{Basic phases}

The connection protocol has several phases -- header,
index, and body.

\begin{itemize}

\item Initiation phase
  \begin{itemize}
  \item We begin once the sender has
    successfully opened a tcp socket connection to the receiver
    (assuming that is the carrier it is registered with).
  \end{itemize}

\item Header phase
  \begin{itemize}
  \item This phase follows immediately after the initiation phase.
  \item Transmission of protocol specifier
    \begin{itemize}
      \item Sender transmits 8 bytes that identify the carrier
	that will be used.  The header may be used to pass a 
	few flags also.
      \item Receiver expects 8 bytes, and attempts to find a
	carrier that is consistent with them.
    \end{itemize}
  \item Transmission of sender name
    \begin{itemize}
      \item Sender transmits the name of the output port
	it is associated with, in a carrier specific way.
      \item Receiver expects the name of the output port,
	transmitted in a carrier specific way.
    \end{itemize}
  \item Transmission of extra header material
    \begin{itemize}
      \item Sender may transmit extra information, depending on
      the carrier.
      \item Receiver may expect extra information, depending on
      the carrier.
    \end{itemize}
  \end{itemize}

\item Header reply phase

  \begin{itemize}
    
  \item This phase follows immediately after the header phase, and
    concludes the preamble to actual data transmission.  After this
    phase, the two ports are considered connected.
    
  \item Receiver may transmit some data, depending on the carrier.
    Receiver then may switch from the initial network
    protocol used to something else (udp, mcast, etc), again
    depending on the carrier.
    
  \item Sender may expect some data, depending on the carrier.
    Sender then may switch from the initial network
    protocol used to something else (udp, mcast, etc), again
    depending on the carrier.

  \end{itemize}

\item Index phase
    
  \begin{itemize}
    
  \item Sender sends carrier-dependent data describing properties of
    the payload data to come.
    
  \item Receiver expects carrier-dependent data describing properties
    of the payload data to come.
    
  \end{itemize}
  
\item Payload data phase
  
  \begin{itemize}
    
  \item Sender sends carrier-dependent expression of user data (maybe none).
    
  \item Receiver expects carrier-dependent expression of user data.
    
  \end{itemize}

\item Acknowledgement phase

  \begin{itemize}
    
  \item Receiver sends carrier-dependent acknowledgement of receipt of
    payload data (maybe none).
    
  \item Sender expects carrier-dependent acknowledgement of receipt of
    payload data (maybe none).
    
  \end{itemize}
  
\end{itemize}


\noindent
This is the basic pattern of YARP communication between ports.
Clearly different carriers have a lot of freedom in how they operate.


\section{The ``tcp'' carrier}


\begin{itemize}

\item{Header and header reply}

  \begin{itemize}
    
  \item The 8-byte protocol specifier for tcp is: \packet{'Y' 'A' 0xE4
    0x1E 0 0 'R' 'P'}.  Another possible variant is: \packet{'Y' 'A'
    0x64 0x1E 0 0 'R' 'P'}.  The first version identifies a connection
    that sends acknowledgements; the second is for connections that
    omit acknowledgements.
    
  \item The sender name is transmitted and expected to be in the
    following format: a 4 byte integer (little endian) giving the length
    of port, followed by the port name in characters, followed by the null
    character.

  \item There is no extra header material for this carrier.

  \item The header reply is as 8 bytes long: \packet{'Y' 'A' B1 B2 0 0
    'R' 'P'}, where (B1,B2) is a (little-endian) two-byte integer
    specifying a socket-port number (unused).  

  \item After the header reply, there is no switch in network protocol
    -- the initial tcp connection continues to be used.

    \end{itemize}

\item{Index, payload, and acknowledgement}

  \begin{itemize}

  \item The sender transmits 8 bytes: \packet{'Y' 'A' 10 0 0 0 'R' 'P'}.
    This identifies the length of the ``index header'' as 10.
    
  \item The sender transmits 10 bytes: \packet{LEN 1 255 255 255 255
    255 255 255 255}.  LEN is the number of blocks of user data need
    to be sent.  This byte-sequence says there are LEN send blocks, 1
    reply block expected, and that the sizes will be listed
    individually next (this odd format is for backward compatability
    with older YARP versions).
      
  \item The sender transmits LEN 4-byte little-endian integers, one
    for each of the LEN blocks of user data, giving the length of each
    block.
      
  \item The sender transmits 4 bytes: \packet{0 0 0 0}. This asks for
    a reply length of 0.

  \item If this is the variant of the tcp carrier that requires
  acknowledgments, then the receiver sends 8 bytes: \packet{'Y' 'A' B1
  B2 B3 B4 'R' 'P'}, where B1-4 is a little-endian integer giving a
  length (could be 0).  It then sends that number of extra bytes.

  \end{itemize}

\end{itemize}


\section{The ``udp'' carrier}


\begin{itemize}

\item{Header and header reply}

  \begin{itemize}
    
  \item The 8-byte protocol specifier for udp is: \packet{'Y' 'A' 0x61
    0x1E 0 0 'R' 'P'}.  The following variant of this should also be
    accepted: \packet{'Y' 'A' 0xE1 0x1E 0 0 'R' 'P'} (it is the same
    thing).

  \item Otherwise header and header reply are identical to the tcp case.
  \item After the header reply, both sides switch to a udp connection
    to the socket-port specified in the header reply.

  \end{itemize}

\item{Index, payload, and acknowledgement}

  \begin{itemize}

  \item Identical to tcp.  Data is split arbitrarily to fit into
    datagrams.
    
  \item Acknowledgments are not a possibility.

  \end{itemize}

\end{itemize}


\section{The ``mcast'' carrier}

\begin{itemize}

\item{Header and header reply}

  \begin{itemize}
    
  \item The 8-byte protocol specifier for mcast is: \packet{'Y' 'A' 0x62
    0x1E 0 0 'R' 'P'}.  The following variant of this should also be
    accepted: \packet{'Y' 'A' 0xE2 0x1E 0 0 'R' 'P'} (it is the same
    thing).

  \item The sender name is sent as for tcp.

  \item Extra header material is send -- 6 bytes.  The first 4 bytes
  specify a multicast IP address.  Next 2 bytes are a (bigendian)
  integer giving a socket-port number.  Note that producing these
  numbers can be helped by side communication with the name server.

  \item There is no header reply for mcast.
  
  \item Both sides switch to a multi-cast group on the specified IP
  and socket-port.

  \end{itemize}

\item{Index, payload, and acknowledgement}

  \begin{itemize}

  \item Identical to udp.

  \item But at most one connection from a given port with an mcast carrier
    should actually write to the multi-cast group.

  \end{itemize}
  
\end{itemize}



\section{The ``text'' carrier}

\begin{itemize}

\item This carrier is carefully designed to make it easy to type into
  a terminal.

\item{Header and header reply}

  \begin{itemize}
    
  \item The 8-byte protocol specifier for text is: \packet{'C' 'O' 'N' 'N' 'E' 'C' 'T' '\textvisiblespace{}'}.  

  \item The sender name is sent as plain text followed by the newline
  character `$\backslash$n'.

  \item There is no extra material

  \item There is no header reply expected for text.
  
  \item There is no network protocol switch.

  \end{itemize}

\item{Index, payload, and acknowledgement}

  \begin{itemize}

  \item There is no index.

  \item The payload is expected to be a series of lines of text
    terminated by the newline character `$\backslash$n'.

  \item There is no acknowledgement expected for text.

  \end{itemize}
  
\end{itemize}


\section{The ``shmem'' carrier}

This is essentially the same as the tcp carrier, except that there is no
header reply, and there is a shift in protocol after header transmission
on both sides to an ACE shared memory stream.  This carrier is 
currently being reworked to make its specification independent of ACE, 
and to further improve efficiency in an existing implementation.

The advantage of this carrier is that it is fast -- the best way to
send messages between processes on a single machine.  Of course, it
doesn't work for processes on different machines.


\section{The ``local'' carrier}

This is a new carrier designed specifically for communication between
threads in a single process.  Giving a specification for the protocol
it uses has low priority, since two such threads are unlikely to be
using different YARP implementations.


\section{Known protocol specifiers}

Here are the currently known protocol specifiers.
The ``shmem'' carrier is not yet documented, but is
implemented in the C++ version of YARP.

\begin{figure}[h]
\begin{tabular}{|cccccccc|l|l|}
\hline
\multicolumn{8}{|c|}{\bf 8-byte magic number} & {\bf protocol} & {\bf variant} \\ \hline\hline
`Y' & `A' & 0x61 & 0x1E & 0 & 0 & `R' & `P'  & udp & \\
`Y' & `A' & 0xE1 & 0x1E & 0 & 0 & `R' & `P'  & udp & \\
`Y' & `A' & 0x62 & 0x1E & 0 & 0 & `R' & `P'  & mcast & \\
`Y' & `A' & 0xE2 & 0x1E & 0 & 0 & `R' & `P'  & mcast & \\
`Y' & `A' & 0x63 & 0x1E & 0 & 0 & `R' & `P'  & shmem & \\
`Y' & `A' & 0xE3 & 0x1E & 0 & 0 & `R' & `P'  & shmem & \\
`Y' & `A' & 0x64 & 0x1E & 0 & 0 & `R' & `P'  & tcp & without acks \\
`Y' & `A' & 0xE4 & 0x1E & 0 & 0 & `R' & `P'  & tcp & with acks \\
`C' & `O' & `N'  & `N'  & `E' & `C' & `T' & `\textvisiblespace{}'  & text & \\
`L' & `O' & `C'  & `A'  & `L' & `I' & `T' & Y  & local & \\
\hline
\end{tabular}
\end{figure}