]> Cisco

tievens@cisco.com

Cisco

trigaux@cisco.com

scotthenning00@gmail.com

Web and Internet Transport Media Over QUIC media over quic This document defines a comprehensive methodology for benchmarking Media over QUIC Transport (MOQT) relays to evaluate their performance under realistic conditions. The methodology utilizes configurable test profiles that simulate common media streaming scenarios including audio-only distribution, combined audio-video streaming, and multi-participant conferencing environments. The framework defines standardized message formats, synchronization mechanisms, and comprehensive metrics collection to ensure reproducible and comparable results across different relay implementations. Test scenarios cover both (QUIC) datagram and stream forwarding modes, enabling evaluation of relay performance characteristics such as throughput, latency variance, object loss rates, and resource utilization. The resulting benchmark data facilitates objective comparison and analysis of (MOQT) relay implementations, supporting informed deployment decisions and performance optimization efforts. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the Media Over QUIC Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction specifies the client-server interactions and messaging used by clients and relays to fan-out published data to one or more subscribers. Implementations employ several state machines to coordinate these interactions. Relays must serialize and deserialize data objects and their extension headers when forwarding to subscribers. Because runs over , relay performance is directly affected by receive/send processing, encryption/decryption, and loss-recovery ACK handling. To evaluate relay performance under realistic conditions, this document defines a benchmarking methodology that mimics real-world use cases.

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here. Commonly used terms in this document are described below.

Track Configurations:

Track configurations define the parameters and settings for individual media tracks used in benchmarking, including transmission patterns, object sizes, intervals, and delivery modes.

Config Profile:

A set of Track Configurations that are used to perform a series of tests to evaluate the performance of relays.

Test Scenario:

A test scenario is a set of config profiles that are used to perform a series of tests to evaluate the performance of relays.

Relay:

A relay is an intermediary that receives published media objects from publishers and forwards them to subscribers, providing fan-out distribution capabilities.

Publisher:

An entity that produces and sends media objects to a relay for distribution to subscribers.

Subscriber:

An entity that receives media objects from a relay by subscribing to specific tracks.

Track:

A sequence of related media objects identified by a namespace and track name, representing a single media stream such as audio or video.

Group:

A collection of related objects within a track that have dependencies on each other, typically used for video frames where objects within a group are interdependent.

Object:

An individual unit of media data transmitted within a track, such as an audio sample or video frame.

Namespace:

A hierarchical identifier that groups related tracks together, allowing for organized distribution of media content.

Datagrams:

datagram frames used for unreliable, low-latency transmission of small media objects, typically audio.

Streams:

streams used for reliable, ordered transmission of media objects, typically video or larger content.

Data Patterns Data transmission patterns are relative to the data that is being transmitted at the time of transmission.

Video Media Video media is a common use case. A video media track typically begins with a large data object, followed by a series of smaller data objects within the same group. The initial object is often significantly larger than the subsequent ones. Video is transmitted at regular intervals (e.g., every 33 ms for 30 fps), producing a bursty fan-out pattern at each interval. Each data object commonly carries at least three extension headers. Video has a direct dependency on previous objects within a group. If objects are lost in a group, it breaks the ability to render the video correctly. Video is often larger than MTU and cannot be transmitted as defined in using datagram. Unless video is very low quality, video uses stream forwarding instead of datagram forwarding.

Audio Media Audio media is a common use case. An audio media track typically maintains fixed intervals with similarly sized data objects. Like video, transmission occurs at each interval, producing a periodic fan-out burst. Each data object usually carries fewer than three extension headers. does not define how to handle larger than MTU sized data objects for datagrams. Audio does not have the same requirement as video where subsequent data objects are related to the previous object or group. Audio is a prime candidate for datagram considering the size of the data objects are less than MTU and there is no dependency on previous object within a group. Audio primarily uses datagrams for forwarding.

Fetch A fetch in is similar to a retrieval of a file where the data will be sent all at once and not paced on an interval. In the data will be sent using a single stream. The data will be sent as fast as the receiver connection can receive it.

Interactive Data Chat, metrics, and logging are use cases for interactive data. This data is sent based on interaction, such as a human typing a message or metrics event, or logging event. A unique characteristic of this data is that it is sent when needed and isn't always constant. Metrics might be an exception where metrics are sent at a constant interval, but metrics may also be sent based on a triggered event. Interactive data is often sent using streams but can also be sent via datagram considering the size of the data is often less than MTU size.

Group and Subgroup Impact to Relays In , when a group or subgroup changes a stream is created for the group/subgroup. This results in additional state for the relay implementation to handle multiple groups/subgroups in-flight. The data transmission use case drives how frequently groups/subgroups change, which can impact the performance of relays. Changing of group/subgroup is only impactful with stream tracks.

Scenarios A combination of data patterns are used to define a scenario. The scenario becomes a Config Profile. Various data patterns are combined to define a scenario. The scenario (Config Profile) is used to benchmark a relay. Below describes common scenarios that the benchmark and configuration profiles need to support. Other scenarios can be defined as needed using the configuration profiles.

Scenario 1: Single Publisher to Multiple Subscribers - Audio The above diagram shows a single publisher sending audio to multiple subscribers. A single audio track is published using datagram. The audio track is sent at a constant interval that is then sent to all subscribers. The benchmark is to see how many subscribers can be supported by the relay using datagram forwarding.

Scenario 2: Single Publisher to Multiple Subscribers - Audio and Video The above diagram shows a single publisher sending audio and video to multiple subscribers. Two tracks are published, an audio datagram track and a reliable stream track for video. The benchmark is to see how many subscribers can be supported by the relay using both datagram and stream forwarding tracks.

Scenario 3: Groups of Publishers and Subscribers (aka Meeting) The above diagram shows a set of publishers and subscribers. Each client in this scenario publishes two tracks, audio and video, and subscribe to the other subscribers audio and video tracks. In this scenario, the set of publishers and subscribers mimic a video conference meeting. The benchmark is to see how many sets (aka meetings) can be supported by the relay using datagram and stream forwarding tracks.

Methodology

Config Profile A configuration profile consists of per track configuration settings. The following track settings are defined: Textual NodeId Formats

Field	Notes
namespace	Namespace tuple of the track
name	Name of the track
reliable	True to use QUIC streams and false to use datagrams
mode	Is a value of 1 to indicate publisher, 2 to indicate subscriber, or 3 to indicate both. When both, the track namespace or name will often be interpolated based on the test script
priority	Priority of the track
ttl	Time to live for objects published
interval	Interval in milliseconds to send data objects
Objects Per Group	Number of objects within a group
First Object Size	First object of group size in bytes
Remaining Object Size	Remaining objects of group size in bytes
Duration	Duration in milliseconds to send data objects
Start Delay	Start delay in milliseconds

Interpolation can be used to template the configuration profile. Upon start, the track will start off with "First Object Size" of data bytes. Subsequent objects will use the Remaining Object Size in bytes. The remaining objects sent will be Objects Per Group number minus the first object. For example, if Objects Per Group is 10 and First Object Size is 100 bytes and Remaining Object Size is 50 bytes, then the first group will be 100 bytes and the remaining 9 groups will be 50 bytes each. Groups value will increment based on sending Objects Per Group number of objects. TTL is used to define how long published objects should remain in the transmit queue. Objects will be published at the Interval value. Objects and groups will continue for as long as the Duration value.

Synchronize Start of Benchmark Synchronization of the benchmark is required to ensure that all clients start at the same time as the publisher so that objects published can be compared to objects received. To synchronize the start of the benchmark, the publisher will publish a start message () repeatedly for the duration of Start Delay value in milliseconds. The start message will be published every every 1/10th interval of the Start Delay value in milliseconds. If 1/10th is less than 100ms, the start message will be published every 100ms. Clients will subscribe to the published track and will wait for the start message () to be received. If the start message is not received before data messages or completion message, the track benchmark is considered failed. Clients will ignore the repeated start messages after receiving the first one. The test begins on the first data message () received. Start message MUST NOT be sent after sending the first data message message.

Completion of benchmark The per track benchmark is completed when the publisher sends a completion message (). The completion message will be sent after the last data message () message is sent.

Messages Testing utilizes various messages to start testing, send data, and report metrics on completion. The following messages are defined:

START

DATA

milliseconds_since_first_object:

Number of milliseconds since the first object was sent. The first object starts at zero and is incremented by milliseconds from the publisher for each message sent. The publisher adds the time when it sends the message. If there are publisher delays, then this time would reflect the delay variance from expected interval.

COMPLETION

total_duration:

Total duration is the total duration in milliseconds from first object to last object sent

Benchmark Metrics

While Track is in Progress The following metrics are computed ongoing as the track is in progress:

Metric	Notes
Received Objects	Count of received objects
Received Groups	Count of received groups
Receive Delta	Delta time in milliseconds from last message to current message received
Average Delta	Average delta time in milliseconds from last message to current message received
Max Delta	Maximum delta time in milliseconds seen
Publisher Variance	Received milliseconds_since_first_object should be zero based on interval. This metric tracks the variance of publisher sending delay
Receive Variance	Received milliseconds_since_first_object should be zero based on interval. This metric tracks the variance of received objects based on expected interval
Average Bits Per Second	Average bits per second received

On completion of track The following result metrics are computed at the end of a track that has completed. Each track has the following metrics:

Metric	Notes
Objects Sent	Number of objects reported sent by the publisher
Groups sent	Number of groups reported sent by the publisher
Objects Received	Number of objects received
Groups Received	Number of groups received
Lost Objects	Computed based on objects received to objects sent
Total Duration	Total duration in milliseconds from the publisher
Actual Duration	Duration in milliseconds from received first object to last object received
Average Receive Variance	The average receive variance of received objects based on expected interval
Average Publisher Variance	The average publisher variance of objects published
Average Bits Per Second	The average bits per second received
Expected Bits Per Second	Computed bits per second rate based on first, remaining sizes and interval values

Running Benchmark Using a configuration profile, multiple instances of clients are ran to connect to the relay. They will publish and subscribe based on the configuration profile. Interpolation is used to dynamically modify the configuration profile namespace and name based on the test script. In the case of mode 3 (both), the namespace and name will be interpolated based on the test script to avoid multi-publisher and mirroring where the publisher subscribes to itself.

Test Environment Isolation Benchmarking should be conducted in isolated test environments to prevent:

• Interference with production traffic
• Exposure of test data patterns to unauthorized parties
• Resource exhaustion attacks on production systems

Authentication and Authorization Benchmark implementations SHOULD implement proper authentication and authorization mechanisms to:

• Prevent unauthorized participation in benchmark tests
• Ensure only authorized entities can initiate benchmark sessions
• Protect against malicious clients that could skew results

Resource Management Benchmark implementations MUST include safeguards to prevent:

• Excessive resource consumption that could lead to denial of service
• Memory exhaustion from unbounded metric collection
• CPU exhaustion from processing malformed benchmark messages

Data Privacy While benchmark data is typically synthetic, implementations SHOULD:

• Avoid including sensitive information in test data
• Provide mechanisms to sanitize benchmark results before sharing
• Consider the privacy implications of detailed performance metrics

Implementation Security Benchmark tools MUST follow secure coding practices and SHOULD undergo security review, as they may be deployed in sensitive network environments.

IANA Considerations This document does not require any IANA actions. The benchmarking methodology defined herein:

• Does not define new protocol elements that require IANA registration
• Does not create new registries or modify existing ones
• Uses existing and protocol mechanisms without extension
• Defines message formats for benchmarking purposes only, which are not part of the protocol specification

The message type values defined in this document (0x01, 0x02, 0x03) are used solely within the context of benchmarking implementations and do not conflict with or extend the protocol message space.

Security Considerations This document defines a benchmarking methodology and does not introduce new security vulnerabilities beyond those inherent in and . However, the following security considerations apply when implementing and running benchmarks.

References Normative References Cisco Google Google Meta This document defines the core behavior for Media over QUIC Transport (MOQT), a media transport protocol designed to operate over QUIC and WebTransport, which have similar functionality. MOQT allows a producer of media to publish data and have it consumed via subscription by a multiplicity of endpoints. It supports intermediate content distribution networks and is designed for high scale and low latency distribution. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm.

Example Benchmark Results Below are example results from running benchmark against existing relays.

Environment Each benchmark test was using the same set of EC2 instances in AWS. The publisher and relay ran on a one EC2 instance while the clients ran on one or more separate EC2 instances.

• EC2 Instance Type: c8g.2xlarge (Graviton4 ARM processors (2.7/2.8GHz), 8 cores, 16GB ram
• Networking: UP to 15Gbps. All instances on same VPC subnet and no NAT gateway was used

Benchmark Tool qperf benchmark implementation was used.

Relays Benchmark tests two open source relays using latest main branches on September 25th, 2025.

• LAPS
• Moxygen

Scenario 1: Single publisher audio

Config Profile =1 first_object_size = 120 ; size in bytes of the first object ; in a group object_size = 120 ; size in bytes of remaining objects ; in a group start_delay = 5000 ; start delay in ms - after control ; messages are sent and acknowledged total_transmit_time = 35000 ; total transmit time in ms, ; including start delay ]]>

Results Scenario 1 Relay Results

Relay	Number of Subscribers	Notes
LAPS	2000	No dropped datagrams and CPU was under 95% on a single core
Moxygen	1000	No dropped datagrams and CPU was under 95% on a single core

Scenario 2: Single publisher audio/video

Config Profile =1 first_object_size = 120 ; size in bytes of the first ; object in group object_size = 120 ; size in bytes of remaining ; objects in a group start_delay = 5000 ; start delay in ms - after control ; messages are sent and acknowledged total_transmit_time = 35000 ; total transmit time in ms [360p Video] namespace = perf/video/{} ; MAY be the same across tracks, ; entries delimited by / name = 1 ; SHOULD be unique to other tracks track_mode = stream ; (datagram|stream) priority = 3 ; (0-255) ttl = 5000 ; TTL in ms time_interval = 33.33 ; transmit interval in floating ; point ms objects_per_group = 150 ; number of objects per group >=1 first_object_size = 21333 ; size in bytes of the first object ; in a group object_size = 2666 ; size in bytes of remaining objects ; in a group start_delay = 5000 ; start delay in ms - after control ; messages are sent and acknowledged total_transmit_time = 35000 ; total transmit time in ms ]]>

Results Scenario 2 Relay Results

Relay	Number of Subscribers	Notes
LAPS	1000	No dropped datagrams or stream data. CPU was under 95% on a single core
Moxygen	250	Datagrams started to get lost after 250 and subscriptions started to close and fail after 250. CPU on a single core was less than 85%

Acknowledgments Thank you to Suhas Nandakumar for their reviews and suggestions.