Time-Sensitive Networking
≈ 24 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Time-Sensitive Networking (TSN) is a set of standards under development by the Time-Sensitive Networking task group of the IEEE 802.1 working group. The TSN task group was formed in November 20121 by renaming the existing Audio Video Bridging Task Group and continuing its work. The name changed as a result of the extension of the working area of the standardization group.
The standards define mechanisms for the time-sensitive transmission of data over deterministic Ethernet networks. The majority of projects define extensions to IEEE 802.1Q Bridges and Bridged Networks, which describes virtual LANs and network switches. These extensions in particular address transmission with very low latency and high availability.
Applications include converged networks with real-time audio/video streaming and real-time control streams, which are used in automotive applications and industrial control facilities.
Time-Sensitive Networking (TSN) is a set of standards under development by the Time-Sensitive Networking task group of the IEEE 802.1 working group. The TSN task group was formed in November 2012 by renaming the existing Audio Video Bridging Task Group and continuing its work. The name changed as a result of the extension of the working area of the standardization group.
The standards define mechanisms for the time-sensitive transmission of data over deterministic Ethernet networks. The majority of projects define extensions to IEEE 802.1Q Bridges and Bridged Networks, which describes virtual LANs and network switches. These extensions in particular address transmission with very low latency and high availability.
Applications include converged networks with real-time audio/video streaming and real-time control streams, which are used in automotive applications and industrial control facilities.
AVB credit-based scheduler
\text{sendSlope} = \text{idleSlope} - \text{portTransmitRate}. The credit increases at a rate idleSlope if frames are waiting for other queues to be transmitted: \text{idleSlope} = \frac{\text{reservedBytes}}{\text{divFrameIntervalTime}}. Thus the idleSlope is the bandwidth reserved for the queue by the bridge, and the sendSlope is the transmission rate of the port MAC service.
If the credit is negative and no frames are transmitted, credit increases at idleSlope rate until zero is reached. If an AVB frame cannot be transmitted because a non-AVB frame is in transmission, credit accumulates at idleSlope rate, but positive credit is allowed.
Additional limits hiCredit and loCredit are derived from the maximum frame size and maximum interference size, the idleSlope/sendSlope, and the maximum port transmission rate. Reserved AV stream traffic frames are forwarded with high priority over non-reserved best-effort traffic, subject to credit-based traffic shaping rules which may require them to wait for a certain amount of credits.
This protects best-effort traffic by limiting the maximum AV stream burst. The frames are scheduled very evenly, though only on an aggregate basis, to smooth out the delivery times and reduce bursting and bunching, which can lead to buffer overflows and packet drops that trigger retransmissions.
The increased buffering delay makes re-transmitted packets obsolete by the time they arrive, resulting in frame drops, which reduces the quality of AV applications. Though credit-based shaper provides fair scheduling for low-priority packets and smooths out traffic to eliminate congestion, unfortunately, average delay increases up to 250 μs per hop, which is too high for control applications, whereas a time-aware shaper (IEEE 802.1Qbv) has a fixed cycle delay from 30 μs to several milliseconds, and a typical delay of 125 μs. Deriving guaranteed upper bounds on delays in TSN is non-trivial and is currently being researched, e.g., by using the mathematical framework Network Calculus.
IEEE 802.1Qat Stream Reservation Protocol
IEEE 802.1Qat Stream Reservation Protocol (SRP) is a distributed peer-to-peer protocol that specifies admission controls based on resource requirements of the flow and available network resources. SRP reserves resources and advertises streams from the sender/source (talker) to the receivers/destinations (listeners); it works to satisfy QoS requirements for each stream and guarantee the availability of sufficient network resources along the entire flow transmission path.
The traffic streams are identified and registered with a 64-bit StreamID, made up of the 48-bit MAC address (EUI) and 16-bit UniqueID to identify different streams from the same source. SRP employs variants of Multiple Registration Protocol (MRP) to register and de-register attribute values on switches/bridges/devices - the Multiple MAC Registration Protocol (MMRP), the Multiple VLAN Registration Protocol (MVRP), and the Multiple Stream Registration Protocol (MSRP).
The SRP protocol essentially works in the following sequence: Advertise a stream from a talker Register the paths along the data flow Calculate worst-case latency Create an AVB domain Reserve the bandwidth Resources are allocated and configured in both the end nodes of the data stream and the transit nodes along the data flow path, with an end-to-end signaling mechanism to detect the success/failure. Worst-case latency is calculated by querying every bridge.
Reservation requests use the general MRP application with the MRP attribute propagation mechanism. All nodes along the flow path pass the MRP Attribute Declaration (MAD) specification, which describes the stream characteristics so that bridges could allocate the necessary resources.
If a bridge is able to reserve the required resources, it propagates the advertisement to the next bridge; otherwise, a talker failed message is raised. When the advertisement message reaches the listener, it replies with listener ready message that propagates back to the talker. Talker advertise and listener ready messages can be de-registered, which terminates the stream.
Successful reservation is only guaranteed when all intermediate nodes support SRP and respond to advertise and ready messages; in Figure 2 above, AVB domain 1 is unable to connect with AVB domain 2. SRP is also used by TSN/AVB standards for frame priorities, frame scheduling, and traffic shaping
IEEE 802.1Qcc Enhancements to SRP
SRP uses a decentralized registration and reservation procedure; multiple requests can introduce delays for critical traffic. IEEE 802.1Qcc-2018 "Stream Reservation Protocol (SRP) Enhancements and Performance Improvements" amendment reduces the size of reservation messages and redefines timers so they trigger updates only when link state or reservation is changed.
To improve TSN administration on large-scale networks, each User Network Interface (UNI) provides methods for requesting Layer 2 services, supplemented by Centralized Network Configuration (CNC) to provide centralized reservation and scheduling, and remote management using NETCONF/RESTCONF protocols and IETF YANG/NETCONF data modeling. CNC implements a per-stream request-response model, where SR class is not explicitly used: end-stations send requests for a specific stream (via edge port) without knowledge of the network configuration, and CNC performs stream reservation centrally.
MSRP only runs on the link to end-stations as an information carrier between CNC and end-stations, not for stream reservation. Centralized User Configuration (CUC) is an optional node that discovers end stations, their capabilities and user requirements, and configures delay-optimized TSN features (for closed-loop IACS applications).
Seamless interop with Resource Reservation Protocol (RSVP) transport is provided. 802.1Qcc allows centralized configuration management to coexist with decentralized, fully distributed configuration of the SRP protocol, and also supports hybrid configurations for legacy AVB devices. 802.1Qcc can be combined with IEEE 802.1Qca Path Control and Reservation (PCR) and TSN traffic shapers.
IEEE 802.1Qch Cyclic Queuing and Forwarding (CQF)
While the 802.1Qav FQTSS/CBS works very well with soft real-time traffic, worst-case delays are both hop count and network topology dependent. Pathological topologies introduce delays, so buffer size requirements have to consider network topology.
IEEE 802.1Qch Cyclic Queuing and Forwarding (CQF), also known as the Peristaltic Shaper (PS), introduces double buffering, which allows bridges to synchronize transmission (frame enqueue/dequeue operations) in a cyclic manner, with bounded latency depending only on the number of hops and the cycle time, completely independent of the network topology. CQF can be used with the IEEE 802.1Qbv time-aware scheduler, IEEE 802.1Qbu frame preemption, and IEEE 802.1Qci ingress traffic policing.
IEEE 802.1Qci Per-Stream Filtering and Policing (PSFP)
IEEE 802.1Qci Per-Stream Filtering and Policing (PSFP) improves network robustness by filtering individual traffic streams. It prevents traffic overload conditions that may affect bridges and the receiving endpoints due to malfunction or Denial of Service (DoS) attacks.
The stream filter uses rule matching to allow frames with specified stream IDs and priority levels and apply policy actions otherwise. All streams are coordinated at their gates, similarly to the 802.1Qch signaling. The flow metering applies predefined bandwidth profiles for each stream.
IEEE 802.1Qbv Enhancements to Traffic Scheduling: Time-Aware Shaper (TAS)
The IEEE 802.1Qbv time-aware scheduler is designed to separate the communication on the Ethernet network into fixed length, repeating time cycles. Within these cycles, different time slices can be configured that can be assigned to one or several of the eight Ethernet priorities.
By doing this, it is possible to grant exclusive use - for a limited time - to the Ethernet transmission medium for those traffic classes that need transmission guarantees and can't be interrupted. The basic concept is a time-division multiple access (TDMA) scheme.
By establishing virtual communication channels for specific time periods, time-critical communication can be separated from non-critical background traffic. Time-aware scheduler introduces Stream Reservation Class CDT for time-critical control data, with worst-case latency of 100 μs over 5 hops, and maximum transmission period of 0.5 ms, in addition to classes A and B defined for IEEE 802.1Qav credit-based traffic shaper.
By granting exclusive access to the transmission medium and devices to time-critical traffic classes, the buffering effects in the Ethernet switch transmission buffers can be avoided and time-critical traffic can be transmitted without non-deterministic interruptions. One example for an IEEE 802.1Qbv scheduler configuration is visible in figure 1: In this example, each cycle consists of two time slices.
Time slice 1 only allows the transmission of traffic tagged with VLAN priority 3, and time slice 2 in each cycle allows for the rest of the priorities to be sent. Since the IEEE 802.1Qbv scheduler requires all clocks on all network devices (Ethernet switches and end devices) to be synchronized and the identical schedule to be configured, all devices understand which priority can be sent to the network at any given point in time.
Since time slice 2 has more than one priority assigned to it, within this time slice, the priorities are handled according to standard IEEE 802.1Q strict priority scheduling. This separation of Ethernet transmissions into cycles and time slices can be enhanced further by the inclusion of other scheduling or traffic shaping algorithms, such as the IEEE 802.1Qav credit-based traffic shaper.
IEEE 802.1Qav supports soft real-time. In this particular example, IEEE 802.1Qav could be assigned to one or two of the priorities that are used in time slice two to distinguish further between audio/video traffic and background file transfers.
The Time-Sensitive Networking Task Group specifies a number of different schedulers and traffic shapers that can be combined to achieve the nonreactive coexistence of hard real-time, soft real-time and background traffic on the same Ethernet infrastructure. The IEEE 802.1Qbv time-aware scheduler has to ensure that the Ethernet interface is not busy with the transmission of a frame when the scheduler changes from one time slice to the next.
The time-aware scheduler achieves this by putting a guard band in front of every time slice that carries time-critical traffic. During this guard band time, no new Ethernet frame transmission may be started; only already ongoing transmissions may be finished. The duration of this guard band has to be as long as it takes the maximum frame size to be safely transmitted.
For an Ethernet frame according to IEEE 802.3 with a single IEEE 802.1Q VLAN tag and including interframe spacing, the total length is: 1500 byte (frame payload) + 18 byte (Ethernet addresses, EtherType and CRC) + 4 byte (VLAN Tag) + 12 byte (Interframe spacing) + 8 byte (preamble and SFD) = 1542 byte. The total time needed for sending this frame is dependent on the link speed of the Ethernet network.
With Fast Ethernet and transmission rate, the transmission duration is as follows: t_\text{maxframe} = \frac{1542\text{ byte}}{12.5 \cdot 10^6\text{ byte} \cdot \frac{1}{s}} = 123.36 \cdot 10^{-6} s In this case, the guard band has to be at least 123.36 μs long. With the guard band, the total bandwidth or time that is usable within a time slice is reduced by the length of the guard band.
This is visible in figure 3 Note: to facilitate the presentation of the topic, the actual size of the guard band in figure 3 is not to scale, but is significantly smaller than indicated by the frame in figure 2. In this example, the time slice 1 always contains high-priority data (e.g. for motion control), while time slice 2 always contains best-effort data.
Therefore, a guard band needs to be placed at every transition point into time slice 1 to protect the time slice of the critical data stream(s). While the guard bands manage to protect the time slices with high priority, critical traffic, they also have some significant drawbacks: The time that is consumed by a guard band is lost - it cannot be used to transmit any data, as the Ethernet port needs to be silent. Therefore, the lost time directly translates in lost bandwidth for background traffic on that particular Ethernet link.
A single time slice can never be configured smaller than the size of the guard band. Especially with lower speed Ethernet connections and growing guard band size, this has a negative impact on the lowest achievable time slice length and cycle time. To partially mitigate the loss of bandwidth through the guard band, the standard IEEE 802.1Qbv includes a length-aware scheduling mechanism.
This mechanism is used when store-and-forward switching is utilized: after the full reception of an Ethernet frame that needs to be transmitted on a port where the guard band is in effect, the scheduler checks the overall length of the frame. If the frame can fit completely inside the guard band, without any infringement of the following high-priority slice, the scheduler can send this frame, despite an active guard band, and reduce the waste of bandwidth.
This mechanism, however, cannot be used when cut-through switching is enabled, since the total length of the Ethernet frame needs to be known a priori. Therefore, when cut-through switching is used to minimize end-to-end latency, the waste of bandwidth will still occur. Also, this does not help with the minimum achievable cycle time.
Therefore, length-aware scheduling is an improvement, but cannot mitigate all drawbacks that are introduced by the guard band.
IEEE 802.3br and 802.1Qbu Interspersing Express Traffic (IET) and Frame Preemption
To further mitigate the negative effects of the guard bands, the IEEE working groups 802.1 and 802.3 have specified the frame pre-emption technology. The two working groups collaborated in this endeavour since the technology required both changes in the Ethernet Media Access Control (MAC) scheme that is under the control of IEEE 802.3, as well as changes in the management mechanisms that are under the control of IEEE 802.1.
Due to this fact, frame pre-emption is described in two different standards documents: IEEE 802.1Qbu for the bridge management component and IEEE 802.3br for the Ethernet MAC component. Frame preemption defines two MAC services for an egress port: preemptable MAC (pMAC) and express MAC (eMAC). Express frames can interrupt the transmission of preemptable frames.
On resume, MAC merge sublayer reassembles frame fragments in the next bridge. Preemption causes computational overhead in the link interface, as the operational context shall be transitioned to the express frame. Figure 4 gives a basic example how frame pre-emption works.
During the process of sending a best effort Ethernet frame, the MAC interrupts the frame transmission just before the start of the guard band. The partial frame is completed with a CRC and will be stored in the next switch to wait for the second part of the frame to arrive.
After the high-priority traffic in time slice 1 has passed and the cycle switches back to time slice 2, the interrupted frame transmission is resumed. Frame pre-emption always operates on a pure link-by-link basis and only fragments from one Ethernet switch to the next Ethernet switch, where the frame is reassembled.
In contrast to fragmentation with the Internet Protocol (IP), no end-to-end fragmentation is supported. Each partial frame is completed by a CRC-32 for error detection. In contrast to the regular Ethernet CRC-32, the last 16 bits are inverted to make a partial frame distinguishable from a regular Ethernet frame.
In addition, the start of frame delimiter (SFD) is changed. The support for frame pre-emption has to be activated on each link between devices individually. To signal the capability for frame pre-emption on a link, an Ethernet switch announces this capability through the LLDP (Link Layer Discovery Protocol).
When a device receives such an LLDP announcement on a network port and supports frame pre-emption itself, it may activate the capability. There is no direct negotiation and activation of the capability on adjacent devices.
Any device that receives the LLDP pre-emption announcement assumes that on the other end of the link, a device is present that can understand the changes in the frame format (changed CRC-32 and SFD). Frame pre-emption allows for a significant reduction of the guard band.
The length of the guard band is now dependent on the precision of the frame pre-emption mechanism: how small is the minimum size of the frame that the mechanism can still preempt. IEEE 802.3br specifies the best accuracy for this mechanism at 64 bytes due to the fact that this is the minimum size of a still valid Ethernet frame.
In this case, the guard band can be reduced to a total of 127 bytes: 64 bytes (minimum frame) + 63 bytes (remaining length that cannot be pre-empted). All larger frames can be pre-empted again and therefore, there is no need to protect against this size with a guard band.
This minimizes the best effort bandwidth that is lost and also allows for much shorter cycle times at slower Ethernet speeds, such as and below. Since the pre-emption takes place in hardware in the MAC, as the frame passes through, cut-through switching can be supported as well, since the overall frame size is not needed a priori.
The MAC interface just checks in regular 64-byte intervals whether the frame needs to be preempted or not. The combination of time synchronization, the IEEE 802.1Qbv scheduler and frame pre-emption already constitutes an effective set of standards that can be utilized to guarantee the coexistence of different traffic categories on a network while also providing end-to-end latency guarantees.
This will be enhanced further as new IEEE 802.1 specifications, such as 802.1Qch are finalized.
IEEE 802.1Qcr Asynchronous Traffic Shaping
Credit-based, time-aware and cyclic (peristaltic) shapers require network-wide coordinated time and utilize network bandwidth inefficiently, as they enforce packet transmission at periodic cycles. The IEEE 802.1Qcr Asynchronous Traffic Shaper (ATS) operates asynchronously based on local clocks in each bridge, improving link utilization for mixed traffic types, such as periodic with arbitrary periods, sporadic (event driven), and rate-constrained.
ATS employs the urgency-based scheduler (UBS), which prioritizes urgent traffic using per-class queuing and per-stream reshaping. Asynchronicity is achieved by interleaved shaping with traffic characterization based on Token Bucket Emulation, a token bucket emulation model, to eliminate the burstiness cascade effects of per-class shaping.
The TBE shaper controls the traffic by average transmission rate, but allows a certain level of burst traffic. When there is a sufficient number of tokens in the bucket, transmission starts immediately; otherwise, the queue's gate closes for the time needed to accumulate enough tokens.
The UBS is an improvement on Rate-Controlled Service Disciplines (RCSDs) to control selection and transmission of each individual frame at each hop, decoupling stream bandwidth from the delay bound by separation of rate control and packet scheduling, and using static priorities and First Come - First Serve and Earliest Due - Date First queuing. UBS queuing has two levels of hierarchy: per-flow shaped queues, with fixed priority assigned by the upstream sources according to application-defined packet transmission times, allowing arbitrary transmission period for each stream, and shared queues that merge streams with the same internal priority from several shapers.
This separation of queuing has low implementation complexity while ensuring that frames with higher priority will bypass the lower priority frames. The shared queues are highly isolated, with policies for separate queues for frames from different transmitters, the same transmitter but different priority, and the same transmitter and priority but a different priority at the receiver.
Queue isolation prevents propagation of malicious data, assuring that ordinary streams will get no interference, and enables flexible stream or transmitter blocking by administrative action. The minimum number of shared queues is the number of ports minus one, and more with additional isolation policies.
Shared queues have scheduler internal fixed priority, and frames are transmitted on the First Come First Serve principle. Worst-case clock sync inaccuracy does not decrease link utilization, contrary to time-triggered approaches such as TAS (Qbv) and CQF (Qch).
IEEE 802.1Qca Path Control and Reservation (PCR)
IEEE 802.1Qca Path Control and Reservation (PCR) specifies extensions to the Intermediate Station to Intermediate Station (IS-IS) protocol to configure multiple paths in bridged networks. The IEEE 802.1Qca standard uses Shortest Path Bridging (SPB) with a software-defined networking (SDN) hybrid mode - the IS-IS protocol handles basic functions, while the SDN controller manages explicit paths using Path Computation Elements (PCEs) at dedicated server nodes.
IEEE 802.1Qca integrates control protocols to manage multiple topologies, configure an explicit forwarding path (a predefined path for each stream), reserve bandwidth, provides data protection and redundancy, and distribute flow synchronization and flow control messages. These are derived from Equal Cost Tree (ECT), Multiple Spanning Tree Instance (MSTI) and Internal Spanning Tree (IST), and Explicit Tree (ET) protocols.
IEEE 802.1CB Frame Replication and Elimination for Reliability (FRER)
IEEE 802.1CB Frame Replication and Elimination for Reliability (FRER) sends duplicate copies of each frame over multiple disjoint paths to provide proactive seamless redundancy for control applications that cannot tolerate packet losses. The packet replication can use traffic class and path information to minimize network congestion.
Each replicated frame has a sequence identification number, used to re-order and merge frames and to discard duplicates. FRER requires centralized configuration management and needs to be used with 802.1Qcc and 802.1Qca. Industrial fault-tolerance HSR and PRP specified in IEC 62439-3 are supported.
IEEE 802.1CS Link-Local Registration Protocol
MRP state data for a stream takes 1500 bytes. With additional traffic streams and larger networks, the size of the database proportionally increases and MRP updates between bridge neighbors significantly slow down. The Link-Local Registration Protocol (LRP) is optimized for a larger database size of about 1 Mbyte with efficient replication that allows incremental updates.
Unresponsive nodes with stale data are automatically discarded. While MRP is application specific, with each registered application defining its own set of operations, LRP is application neutral.
IEEE 802.1Qdd Resource Allocation Protocol
SRP and MSRP are primarily designed for AV applications - their distributed configuration model is limited to Stream Reservation (SR) Classes A and B defined by the Credit-Based Shaper (CBS), whereas IEEE 802.1Qcc includes a more centralized CNC configuration model supporting all new TSN features such as additional shapers, frame preemption, and path redundancy. IEEE P802.1Qdd project updates the distributed configuration model by defining new peer-to-peer Resource Allocation Protocol signaling built upon P802.1CS Link-local Registration Protocol.
RAP will improve scalability and provide dynamic reservation for a larger number of streams with support for redundant transmission over multiple paths in 802.1CB FRER, and autoconfiguration of sequence recovery. RAP supports the 'topology-independent per-hop latency calculation' capability of TSN shapers such as 802.1Qch Cyclic Queuing and Forwarding (CQF) and P802.1Qcr Asynchronous Traffic Shaping (ATS).
It will also improve performance under high load and support proxying and enhanced diagnostics, all while maintaining backward compatibility and interoperability with MSRP.
IEEE 802.1ABdh Link Layer Discovery Protocol v2
IEEE P802.1ABdh Station and Media Access Control Connectivity Discovery - Support for Multiframe Protocol Data Units (LLDPv2) updates LLDP to support IETF Link State Vector Routing protocol and improve efficiency of protocol messages.
YANG data models
The IEEE 802.1Qcp standard implements the YANG data model to provide a Universal Plug-and-Play (uPnP) framework for status reporting and configuration of equipment such as Media Access Control (MAC) Bridges, Two-Port MAC Relays (TPMRs), Customer Virtual Local Area Network (VLAN) Bridges, and Provider Bridges, and to support the 802.1X Security and 802.1AX Datacenter Bridging standards. YANG is a Unified Modeling Language (UML) for configuration and state data, notifications, and remote procedure calls, to set up device configuration with network management protocols such as NETCONF/RESTCONF.
DetNet
The IETF Deterministic Networking (DetNet) Working Group is focusing on defining deterministic data paths with high reliability and bounds on latency, loss, and packet delay variation (jitter), such as audio and video streaming, industrial automation, and vehicle control. The goals of Deterministic Networking are to migrate time-critical, high-reliability industrial and audio-video applications from special-purpose Fieldbus networks to IP packet networks.
To achieve these goals, DetNet uses resource allocation to manage buffer sizes and transmission rates in order to satisfy end-to-end latency requirements. Service protection against failures with redundancy over multiple paths and explicit routes to reduce packet loss and reordering.
The same physical network shall handle both time-critical reserved traffic and regular best-effort traffic, and unused reserved bandwidth shall be released for best-effort traffic. DetNet operates at the IP Layer 3 routed segments using a software-defined networking layer to provide IntServ and DiffServ integration, and delivers services over lower Layer 2 bridged segments using technologies such as MPLS and IEEE 802.1 AVB/TSN.
Traffic Engineering (TE) routing protocols translate DetNet flow specification to AVB/TSN controls for queuing, shaping, and scheduling algorithms, such as IEEE 802.1Qav credit-based shaper, IEEE802.1Qbv time-triggered shaper with a rotating time scheduler, IEEE802.1Qch synchronized double buffering, 802.1Qbu/802.3br Ethernet packet pre-emption, and 802.1CB frame replication and elimination for reliability. Also, the protocol interworking defined by IEEE 802.1CB is used to advertise TSN sub-network capabilities to DetNet flows via the Active Destination MAC and VLAN Stream identification functions.
DetNet flows are matched by destination MAC address, VLAN ID and priority parameters to Stream ID and QoS requirements for talkers and listeners in the AVB/TSN sub-network.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1371103021 under CC BY-SA 4.0; modifications have been made.
References
- ↩Wikipedia “Time-Sensitive Networking” date of formation (×1) · confidence 0.95 revision URL