Requirements for High-performance Wide Area Networks
ZTE Corporation
- PFC:
- Priority Flow Control
- ECN:
- Explicit Congestion Notification
- ECMP:
- Equal-Cost Multipath
- RTT:
- Round-Trip Time
- TCP:
- Transfer Control Protocol
- RDMA:
- Remote Direct Memory Access Round-Trip Time
- QUIC:
- Quick UDP Internet Connections
- WAN:
- Wide Area Networks
- MAN:
- Metropolitan Area Networks
Primary Goals
The primary goal of HP-WAN is to ensure the effective high-throughput
transmission of massive data with the performance indicators such as
ultra-high bandwidth utilization, zero packet loss ratio and low latency.
For instance, the computing method of throughput for TCP is as following
shown.
Throughput = min{BW,WindowSize/RTT,(MSS/RTT)*(1/P));
BW indicates the maximum bandwidth, WindowSize indicates the
size of the window, MSS indicates the maximum segment size,
RTT indicates the round time delay, P indicates the square
root of packet loss ratio.
Extremely Low or Zero Packet Loss Ratio
According to the throughput computing formula, the packet loss
negatively correlates with throughput. The lower the
packet loss rate, the higher the throughput. According to the
experimental data, for TCP, the throughput dramatically decreases
up to 89.9% when the packet loss ratio is 2%. For RDMA, the
throughput dramatically decreases with a packet loss ratio greater
than 0.1%, and a 2% packet loss ratio effectively reduces the
throughput to zero. It is important to ensure the extremely
low or zero packet loss ratio to achieve high-throughput data
transmission in HP-WAN.
Low Long-distance Delay and Jitter
According to the throughput computing formula, the RTT is
negatively correlated with throughput. The lower the RTT,
the higher the throughput. But the RTT delay is impacted
with the long-distance latency. For example, when the
distance between the data centers is 500 kilometers, the
RTT is 5ms. When the distance is 3000 kilometers, the
RTT is 30ms. According to the experimental data, when
the jitter is over 300~500us, the throughput will
dramatically decrease. So it is required to guarantee low
long-distance delay and jitter to achieve high-throughput
data transmission in HP-WAN.
Ultra-high Bandwidth Utilization
It is important to reserve sufficient bandwidth to
achieve high-throughput transmission for a single flow.
But for massive concurrent flows in a network with certain
resources, bandwidth utilization is the key aspect.
Ultra-high bandwidth utilization refers to the efficient
use of available network capacity to maximize data transfer
rates and minimize latency. This is particularly important
in scenarios where high volumes of data need to be transferred
quickly. So it is required to improve the bandwidth utilization
to achieve high-throughput data transmission for multiple
concurrent services in HP-WAN.
Requirements
Challenges of high-throughput transmission in HP-WAN come from
massive concurrent services and long-distance delays, jitter
and packet loss. The existing network technologies have various
problems and cannot meet the demands. This document outlines
the requirements for high-throughput data transmission in HP-WAN.
Support High-precision Flow Control
Flow control refers to a method for ensuring the data is
transmitted efficiently and reliably and controlling the rate
of data transmission to prevent the fast sender from overwhelming
the slow receiver and prevent packet loss in congested situations.
PFC (Priority-based Flow Control) [IEEE 802.1qbb] is a hop-by-hop
and priority-based flow control method which provides backpressure
mechanism for the receiver signals the sender to slow down the rate
of data transmission. For the long-distance link and transmission
delay in WAN, it is required to configure the reasonable threshold
and increase buffer for effective throughput without packet loss.
PFC creates 8 virtual channels on a link and assigns a
priority to each channel, allowing for individual pause
and restart of any one of the virtual channels. The existing
flow control mechanism is based on port and priority with
limited number, while there are multiple services with various
types and different traffic requirements. It is required to
provide fine-grained and high-precision flow control to reduce
the impact between different traffic flows.
Support Congestion Control based on End-network Coordination
Congestion control refers to a method for controlling the total
amount of data entering the network to maintain the traffic at an
acceptable level. The difference between congestion control and
flow control is that flow control acts on the receiver, while
congestion control acts on the network. As per , ECN
defines an end-to-end congestion notification mechanism based
on IP and transport layers. When the congestion occurred, the
device marks packets and transmits congestion information to
the server and the server sends packets to the client to notify
the source to adjust the transmission rate to achieve congestion
control.
The long-distance transmission of thousands of kilometers
results in extremely long link transmission delays and it will
delay the network state feedback. And it is inefficient that
1-bit ECN signal can not specify the detailed congestion information.
And it mainly relies on passive congestion control to adjust
the rate after receiving congestion signals. It is required to
improve the congestion control by enhancing the IP network
capability to achieve the end and network coordination in WAN.
For example, the device could initial the notification directly
to the source and provide precise notification information.
And the device in the network may further perform the proactive
congestion control.
Support Muti-path Load Balance
Load balance refers to a method for the allocation of load (traffic)
to multiple links for forwarding traffic. When transmitting intelligent
computing services, the traffic is mainly elephant flow and the network
resources is insufficient in WAN. Uneven network load will lead to a
decrease in network throughput and low link utilization. In order to
improve bandwidth utilization, it is required to implement multi-path
load balance to achieve low latency, zero packet loss and high-throughput
performance in WAN.
There are three optional methods such as flow-based ECMP,
flowlet-based load balance and packet-based load balance. As per ,
Link Aggregation Group (LAG) and Equal-Cost Multipath (ECMP) are used
for bandwidth scaling. ECMP uses 5-tuple for HASH load balancing to
achieve per-flow load balancing and link backup and it is applied
to scenarios with large number of flows. It will be challenging for
HASH conflict and poor network balancing with massive elephant flows.
For example, flow-based ECMP will distribute the elephant flows into
the same link, resulting in congestion and packet loss. Packet-based
load balance will result in out-of-order packets. Flowlet-based load
balance can distribute the sub-flows to different paths. The time
interval gap value between sub-flows needs be accurately configured
based on the delay information of multiple path. The deterministic
technology can be implemented to guarantee the latency and jitter.
Support the Differentiated Traffic Scheduling
Traffic scheduling refers to a method for managing and allocating
the flow of data packets within a network to optimize performance
and utilize network resources efficiently. Considering the multiple
services with various types and different traffic requirements, the
traffic is required to be scheduled to multiple paths and resources
to achieve differentiated QoS requirements. The existing technologies
such as resource reservation, network slicing, queuing-based solutions
to guarantee deterministic latency can be used for providing zero packet
loss, long-distance latency and jitter guarantees and high reliability
in WAN. For example, the flow-specific data may require low latency
or loose latency and it is required to provide different queuing and
scheduling functionalities.
Support Flow-based Network Monitoring
When a fault occurs in data transmission, it should
discover root causes of some of the hard-to-debug network and
identify the node which is dropping the packets. Bandwidth
monitoring is also important for network planning and service-level
assurance, which the network operators can predict bandwidth
availability and guarantee the high-throughput transmission. The
performance monitoring is a critical aspect of managing and optimizing
networks such as end-to-end measurement of packet loss, latency,
jitter and hop-by-hop node ID, node delay, queue and buffer information.
As per , network telemetry is a
technology for gaining network insight and facilitating efficient
and automated network management.It is required to provide flow-based
network monitoring based on telemetry, which makes it easier to
troubleshoot issues and monitor bandwidth, traffic and performance.
Security Considerations
This document lists the requirements for HP-WAN and does not raise
any security concerns or issues in addition to ones common to networking
which may have security considerations from both the use-specific
perspective and the technology-specific perspective.
IANA Considerations
This document makes no requests for IANA action.
Acknowledgements
The authors would like to acknowledge Yao Liu, Zheng Zhang and Bin Tan for
their thorough review and very helpful comments.
References
Normative References