Sklower, et. al.            Standards Track                     [Page 6]RFC 1990                     PPP Multilink                   August 1996                      Figure 1.  Multilink Overview.     Network Layer     -------------                    ______           ______                   /      \         /      \                  |  NL 1  |       |  NL 2  |                   \______/         \______/                     | | |             | | |                     | | +-------------o-o-o-+                     | +------+  +-----+ | | |                     |        |  |       | | |                     | +------o--o-------+ + |                     | |      |__|_        | |                     | |     /      \      | |                     | |    |  MLCP  | <--- Link Layer                     | |     \______/    Demultiplexing                     | |        |          | |                     | |        |          | |                     | |        | <--- Virtual Link                     | |        |          | |                     | |        |          | |                     | |        |          | |                     | |        +          | |                  ___|_|        |       ___|_|                 /      \       |      /      \                |   LCP  |------+-----|  LCP   | <--- Link Layer                 \______/              \______/       Demultiplexing                    |                      |                    |                      |                  Link 1                 Link 23.  Packet Formats   In this section we describe the layout of individual fragments, which   are the "packets" in the Multilink Protocol.  Network Protocol   packets are first encapsulated (but not framed) according to normal   PPP procedures, and large packets are broken up into multiple   segments sized appropriately for the multiple physical links.   Although it would otherwise be permitted by the PPP spec,   implementations MUST NOT include the Address and Control Field in the   logical entity to be fragmented.  A new PPP header consisting of the   Multilink Protocol Identifier, and the Multilink header is inserted   before each section.  (Thus the first fragment of a multilink packet   in PPP will have two headers, one for the fragment, followed by the   header for the packet itself).Sklower, et. al.            Standards Track                     [Page 7]RFC 1990                     PPP Multilink                   August 1996   Systems implementing the multilink procedure are not required to   fragment small packets.  There is also no requirement that the   segments be of equal sizes, or that packets must be broken up at all.   A possible strategy for contending with member links of differing   transmission rates would be to divide the packets into segments   proportion to the transmission rates.  Another strategy might be to   divide them into many equal fragments and distribute multiple   fragments per link, the numbers being proportional to the relative   speeds of the links.   PPP multilink fragments are encapsulated using the protocol   identifier 0x00-0x3d.  Following the protocol identifier is a four   byte header containing a sequence number, and two one bit fields   indicating that the fragment begins a packet or terminates a packet.   After negotiation of an additional PPP LCP option, the four byte   header may be optionally replaced by a two byte header with only a 12   bit sequence space.  Address & Control and Protocol ID compression   are assumed to be in effect.  Individual fragments will, therefore,   have the following format:             Figure 2:  Long Sequence Number Fragment Format.                +---------------+---------------+   PPP Header:  | Address 0xff  | Control 0x03  |                +---------------+---------------+                | PID(H)  0x00  | PID(L)  0x3d  |                +-+-+-+-+-+-+-+-+---------------+   MP Header:   |B|E|0|0|0|0|0|0|sequence number|                +-+-+-+-+-+-+-+-+---------------+                |      sequence number (L)      |                +---------------+---------------+                |        fragment data          |                |               .               |                |               .               |                |               .               |                +---------------+---------------+   PPP FCS:     |              FCS              |                +---------------+---------------+Sklower, et. al.            Standards Track                     [Page 8]RFC 1990                     PPP Multilink                   August 1996             Figure 3:  Short Sequence Number Fragment Format.                +---------------+---------------+   PPP Header:  | Address 0xff  | Control 0x03  |                +---------------+---------------+                | PID(H)  0x00  | PID(L)  0x3d  |                +-+-+-+-+-------+---------------+   MP Header:   |B|E|0|0|    sequence number    |                +-+-+-+-+-------+---------------+                |    fragment data              |                |               .               |                |               .               |                |               .               |                +---------------+---------------+   PPP FCS:     |              FCS              |                +---------------+---------------+   The (B)eginning fragment bit is a one bit field set to 1 on the first   fragment derived from a PPP packet and set to 0 for all other   fragments from the same PPP packet.   The (E)nding fragment bit is a one bit field set to 1 on the last   fragment and set to 0 for all other fragments.  A fragment may have   both the (B)eginning and (E)nding fragment bits set to 1.   The sequence field is a 24 bit or 12 bit number that is incremented   for every fragment transmitted.  By default, the sequence field is 24   bits long, but can be negotiated to be only 12 bits with an LCP   configuration option described below.   Between the (E)nding fragment bit and the sequence number is a   reserved field, whose use is not currently defined, which MUST be set   to zero.  It is 2 bits long when the use of short sequence numbers   has been negotiated, 6 bits otherwise.   In this multilink protocol, a single reassembly structure is   associated with the bundle.  The multilink headers are interpreted in   the context of this structure.   The FCS field shown in the diagram is inherited from the normal   framing mechanism from the member link on which the packet is   transmitted.  There is no separate FCS applied to the reconstituted   packet as a whole if transmitted in more than one fragment.Sklower, et. al.            Standards Track                     [Page 9]RFC 1990                     PPP Multilink                   August 19963.1.  Padding Considerations   Systems that support the multilink protocol SHOULD implement Self-   Describing-Padding.  A system that implements self-describing-padding   by definition will either include the padding option in its initial   LCP Configure-Requests, or (to avoid the delay of a Configure-Reject)   include the padding option after receiving a NAK containing the   option.   A system that must pad its own transmissions but does not use Self-   Describing-Padding when not using multilink, MAY continue to not use   Self-Describing-Padding if it ensures by careful choice of fragment   lengths that only (E)nding fragments of packets are padded.  A system   MUST NOT add padding to any packet that cannot be recognized as   padded by the peer.  Non-terminal fragments MUST NOT be padded with   trailing material by any other method than Self-Describing-Padding.   A system MUST ensure that Self-Describing-Padding as described in RFC   1570 [11] is negotiated on the individual link before transmitting   any multilink data packets if it might pad non-terminal fragments or   if it would use network or compression protocols that are vulnerable   to padding, as described in RFC 1570.  If necessary, the system that   adds padding MUST use LCP Configure-NAK's to elicit a Configure-   Request for Self-Describing-Padding from the peer.   Note that LCP Configure-Requests can be sent at any time on any link,   and that the peer will always respond with a Configure-Request of its   own.  A system that pads its transmissions but uses no protocols   other than multilink that are vulnerable to padding MAY delay   ensuring that the peer has Configure-Requested Self-Describing-   Padding until it seems desireable to negotiate the use of Multilink   itself.  This permits the interoperability of a system that pads with   older peers that support neither Multilink nor Self-Describing-   Padding.4.  Trading Buffer Space Against Fragment Loss   In a multilink procedure one channel may be delayed with respect to   the other channels in the bundle.  This can lead to fragments being   received out of order, thus increasing the difficulty in detecting   the loss of a fragment.  The task of estimating the amount of space   required for buffering on the receiver becomes more complex because   of this.  In this section we discuss a technique for declaring that a   fragment is lost, with the intent of minimizing the buffer space   required, yet minimizing the number of avoidable packet losses.Sklower, et. al.            Standards Track                    [Page 10]RFC 1990                     PPP Multilink                   August 19964.1.  Detecting Fragment Loss   On each member link in a bundle, the sender MUST transmit fragments   with strictly increasing sequence numbers (modulo the size of the   sequence space).  This requirement supports a strategy for the   receiver to detect lost fragments based on comparing sequence   numbers.  The sequence number is not reset upon each new PPP packet,   and a sequence number is consumed even for those fragments which   contain an entire PPP packet, i.e., one in which both the (B)eginning   and (E)nding bits are set.   An implementation MUST set the sequence number of the first fragment   transmited on a newly-constructed bundle to zero.  (Joining a   secondary link to an exisiting bundle is invisible to the protocol,   and an implementation MUST NOT reset the sequence number space in   this situation).   The receiver keeps track of the incoming sequence numbers on each   link in a bundle and maintains the current minimum of the most   recently received sequence number over all the member links in the   bundle (call this M).  The receiver detects the end of a packet when   it receives a fragment bearing the (E)nding bit.  Reassembly of the   packet is complete if all sequence numbers up to that fragment have   been received.   A lost fragment is detected when M advances past the sequence number   of a fragment bearing an (E)nding bit of a packet which has not been   completely reassembled (i.e., not all the sequence numbers between   the fragment bearing the (B)eginning bit and the fragment bearing the   (E)nding bit have been received).  This is because of the increasing   sequence number rule over the bundle.  Any sequence number so   detected is assumed to correspond to a fragment which has been lost.   An implementation MUST assume that if a fragment bears a (B)eginning   bit, that the previously numbered fragment bore an (E)nding bit.   Thus if a packet is lost bearing the (E)nding bit, and the packet   whose fragment number is M contains a (B)eginning bit, the   implementation MUST discard fragments for all unassembled packets   through M-1, but SHOULD NOT discard the fragment bearing the new   (B)eginning bit on this basis alone.   The detection of a lost fragment, whose sequence number was deduced   to be U, causes the receiver to discard all fragments up to the   lowest numbered fragment with an ending bit (possibly deduced)   greater than or equal to U.  However, the quantity M may jump into   the middle of a chain of packets which can be successful completed.Sklower, et. al.            Standards Track                    [Page 11]RFC 1990                     PPP Multilink                   August 1996   Fragments may be lost due to corruption of individual packets or   catastrophic loss of the link (which may occur only in one   direction).  This version of the multilink protocol mandates no   specific procedures for the detection of failed links.  The PPP link   quality management facility, or the periodic issuance of LCP echo-   requests could be used to achieve this.   Senders SHOULD avoid keeping any member links idle to maximize early   detection of lost fragments by the receiver, since the value of M is   not incremented on idle links.  Senders SHOULD rotate traffic among   the member links if there isn't sufficient traffic to overflow the   capacity of one link to avoid idle links.   Loss of the final fragment of a transmission can cause the receiver   to stall until new packets arrive.  The likelihood of this may be   decreased by sending a null fragment on each member link in a bundle   that would otherwise become idle immediately after having transmitted   a fragment bearing the (E)nding bit, where a null fragment is one   consisting only of a multilink header bearing both the (B)egin and   (E)nding bits (i.e., having no payload).  Implementations concerned   about either wasting bandwidth or per packet costs are not required   to send null fragments and may elect to defer sending them until a   timer expires, with the marginally increased possibility of lengthier   stalls in the receiver.  The receiver SHOULD implement some type of   link idle timer to guard against indefinite stalls.   The increasing sequence per link rule prohibits the reallocation of   fragments queued up behind a failing link to a working one, a   practice which is not unusual for implementations of ISO multilink   over LAPB [4].4.2.  Buffer Space Requirements   There is no amount of buffering that will guarantee correct detection   of fragment loss, since an adversarial peer may withhold a fragment   on one channel and send arbitrary amounts on the others.  For the   usual case where all channels are transmitting, you can show that   there is a minimum amount below which you could not correctly detect   packet loss.  The amount depends on the relative delay between the   channels, (D[channel-i,channel-j]), the data rate of each channel,   R[c], the maximum fragment size permitted on each channel, F[c], and   the total amount of buffering the transmitter has allocated amongst   the channels.   When using PPP, the delay between channels could be estimated by   using LCP echo request and echo reply packets.  (In the case of links   of different transmission rates, the round trip times should be   adjusted to take this into account.)  The slippage for each channelSklower, et. al.            Standards Track                    [Page 12]