Network Working Group A. Huang Feng
Internet-Draft Deutsche Telekom
Intended status: Standards Track P. Francois
Expires: 19 April 2027 INSA-Lyon
B. Claise
Everything OPS & Arrcus
T. Graf
Swisscom
J. Iurman
6WIND
16 October 2026
IOAM Extensions for On-Path Delay Measurement
draft-netana-ippm-ioam-on-path-delay-00
Abstract
This document defines extensions to In Situ Operations,
Administration, and Maintenance (IOAM) that enable IOAM nodes to
compute on-path delay samples. These samples are the basis for the
on-path delay statistics that can be exported using the IP Flow
Information Export (IPFIX) Information Elements defined in RFC 9951.
The document defines an On-Path Delay IOAM data field and extends the
IOAM Direct Export (DEX) Option-Type with a timestamp reference
inserted by the IOAM encapsulating node. These extensions enable
transit and decapsulating nodes to compute and locally aggregate on-
path delay. The resulting on-path delay can then be made available
by different mechanisms, for example IPFIX export or the YANG data
model for on-path telemetry.
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Huang Feng, et al. Expires 19 April 2027 [Page 1]
Internet-Draft IOAM On-Path Delay October 2026
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 19 April 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. On-Path Delay Measurement with IOAM DEX . . . . . . . . . . . 4
4. On-Path Delay Data Field . . . . . . . . . . . . . . . . . . 5
5. IOAM DEX Timestamp Extension . . . . . . . . . . . . . . . . 6
6. Export of On-Path Delay . . . . . . . . . . . . . . . . . . . 8
7. Security Considerations . . . . . . . . . . . . . . . . . . . 9
8. Operational Considerations . . . . . . . . . . . . . . . . . 9
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 10
11.1. Normative References . . . . . . . . . . . . . . . . . . 10
11.2. Informative References . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11
Huang Feng, et al. Expires 19 April 2027 [Page 2]
Internet-Draft IOAM On-Path Delay October 2026
1. Introduction
Network operators use delay measurements to detect and localize
performance degradation, including congestion, affecting customer
traffic. RFC 9951 defines IP Flow Information Export (IPFIX)
[RFC7011] Information Elements for exporting statistical views of the
on-path delay measured between an IOAM encapsulating node and
subsequent transit and decapsulating nodes.
To produce these statistics, the on-path delay needs to be computed
at the node performing the aggregation. In postcard mode, this
allows delay samples to be aggregated directly in the context of the
customer flow that triggered their collection, without requiring
subsequent correlation between separately exported telemetry data and
customer traffic.
IOAM Direct Export (DEX) [RFC9326] provides such a mechanism for
postcard mode. A packet carrying the DEX Option-Type triggers an
IOAM node to collect, locally aggregate, and/or export the IOAM data
selected by the IOAM Trace-Type. However, the DEX Option-Type
currently does not provide the timestamp reference required by an
IOAM transit or decapsulating node to compute the on-path delay
defined in RFC 9951.
This document defines two IOAM extensions to enable this computation:
* A new On-Path Delay IOAM data field, which identifies the on-path
delay as an IOAM telemetry metric and allows it to be selected by
the IOAM Trace-Type.
* An extension to the IOAM DEX Option-Type carrying a timestamp
reference inserted by the IOAM encapsulating node.
When processing a packet carrying these extensions, an IOAM transit
or decapsulating node computes the on-path delay as the difference
between its current timestamp and the timestamp reference carried in
the packet. The resulting delay sample can be locally aggregated in
the context of the customer flow and subsequently exported, for
example using the IPFIX Information Elements defined in RFC 9951.
This document defines the IOAM mechanisms required to compute and
identify the on-path delay metric. The aggregation and export
mechanisms are not redefined by this document.
2. Terminology
This document uses the following terms as defined in [RFC9197] and
[RFC9326].
Huang Feng, et al. Expires 19 April 2027 [Page 3]
Internet-Draft IOAM On-Path Delay October 2026
* IOAM domain
* IOAM encapsulating node
* IOAM transit node
* IOAM decapsulating node
* IOAM Direct Export (DEX) Option-Type
This document also uses the terms postcard mode and passport mode as
described in the Introduction of [RFC9951]. In postcard mode, IOAM
transit nodes also export on-path telemetry data. In passport mode,
only the IOAM decapsulating node of the IOAM domain reports metrics.
3. On-Path Delay Measurement with IOAM DEX
The IOAM Direct Export (DEX) Option-Type defined in [RFC9326]
provides a mechanism for postcard mode in which packets trigger IOAM
nodes to collect, locally aggregate, and/or export selected telemetry
data. The metrics to be processed by a DEX node are identified using
the IOAM-Trace-Type field carried in the DEX Option-Type.
Computing the on-path delay requires two pieces of information that
are not currently available together in IOAM DEX. First, the on-path
delay needs an allocation in the IOAM Trace-Type registry so that DEX
can identify it as a metric to be aggregated or exported. Second,
the DEX Option-Type needs to carry the timestamp of the IOAM
encapsulating node, which provides the reference from which a transit
or decapsulating node computes the delay.
The IOAM encapsulating node inserts the timestamp reference in the
DEX Option-Type. When a transit or decapsulating node processes the
packet, it subtracts this timestamp from its current timestamp to
obtain the on-path delay.
Huang Feng, et al. Expires 19 April 2027 [Page 4]
Internet-Draft IOAM On-Path Delay October 2026
IOAM Domain
.........................................
. .
. D1 .
. <------> .
. .
. D2 .
. <--------------------> .
. .
. D3 .
. <-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
IOAM IOAM IOAM IOAM
Host 1 Encapsulating Transit Transit Decapsulating Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
Figure 1: On-path delay measurement in an IOAM domain
D1, D2, and D3 represent the on-path delay from the IOAM
encapsulating node R1 to R2, R3, and R4, respectively.
Together, the On-Path Delay data field and the DEX timestamp
extension allow an in-flight customer packet to trigger computation
and local aggregation of the on-path delay at each participating
node. For the per-flow aggregation use case described in this
document, the IPFIX mechanism defined in [RFC9951] is the preferred
export method because the delay samples can be aggregated in the
context of the customer flow that triggered their collection,
avoiding subsequent correlation of telemetry records with customer
traffic.
The extensions defined here are not tied to IPFIX export. Other
mechanisms can make the resulting on-path telemetry available,
including the YANG data model and YANG-Push mechanisms described in
[I-D.ietf-ippm-on-path-telemetry-yang]. Export mechanisms are
outside the scope of this document and are discussed further in
Section 6.
4. On-Path Delay Data Field
This document defines a new IOAM Trace-Type data field for the on-
path delay. The field represents the delay between the IOAM
encapsulating node and the transit or decapsulating node where the
value is computed.
Huang Feng, et al. Expires 19 April 2027 [Page 5]
Internet-Draft IOAM On-Path Delay October 2026
The allocation in the IOAM Trace-Type registry is required for IOAM
DEX because the DEX Option-Type defined in [RFC9326] uses the IOAM-
Trace-Type field to identify which telemetry data is to be collected,
locally aggregated, and/or exported. Allocating an On-Path Delay
data field therefore allows a DEX node to select the computed delay
for processing in the same way as other IOAM telemetry metrics.
The same IOAM Trace-Type allocation can also be used by the Pre-
allocated and Incremental Trace Option-Types defined in [RFC9197].
Consequently, the allocation enables the on-path delay metric for
both postcard mode and passport mode, although the DEX use case
motivates the allocation in this document.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|O| On-Path Delay |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: On-Path Delay Data Field Format
The On-Path Delay data field is a 4-octet unsigned integer in the
range 0 to 2^31-1. It is the on-path delay between the IOAM
encapsulating node and the transit or decapsulating node, measured in
microseconds. Note that this unit differs from the nanoseconds used
by the transit delay field of [RFC9197].
The value is computed as specified in Section 5. If the delay
exceeds 2^31-1 microseconds (about 35 minutes), the top bit 'O' is
set to indicate overflow and the value is set to 0x80000000. If the
node cannot compute the delay, for example because the difference is
negative as a result of clocks of the IOAM encapsulating node and of
the computing node that are not sufficiently synchronized, the field
MUST be set to 0xFFFFFFFF to mean not populated. A delay sample with
either of these two values MUST NOT be included in local aggregation.
The data field is allocated by IANA as defined in Section 9.
5. IOAM DEX Timestamp Extension
In postcard mode, a packet carrying the DEX Option-Type can trigger
telemetry data to be exported directly or locally aggregated before
export, as defined in [RFC9326]. The telemetry metrics subject to
this processing are selected using the IOAM-Trace-Type field.
The On-Path Delay data field defined in this document identifies the
delay as one of these metrics. However, computing its value
additionally requires the timestamp of the IOAM encapsulating node.
Huang Feng, et al. Expires 19 April 2027 [Page 6]
Internet-Draft IOAM On-Path Delay October 2026
This document therefore extends the IOAM DEX Option-Type with a
timestamp reference. Two optional fields are defined: Timestamp
Seconds and Timestamp Fraction. The fields are enabled using the
IOAM DEX Extension-Flags.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Namespace-ID | Flags |Extension-Flags|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IOAM-Trace-Type | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flow ID (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp Seconds (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp Fraction (Optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: IOAM DEX Option-Type Format Including the Timestamps
As in [RFC9326], the bits of the IOAM DEX Extension-Flags are
numbered from 0, with bit 0 being the most significant bit of the
field. Bit [TBD1] of the Extension-Flags is used for Timestamp
Seconds. Bit [TBD2] is used for Timestamp Fraction. The Timestamp
Seconds and Timestamp Fraction fields MUST be used in combination.
The Timestamp Seconds and Timestamp Fraction fields use the encoding
formats defined in Section 5 of [RFC9197].
The timestamp reference is inserted at the IOAM encapsulating node.
At each subsequent IOAM transit or decapsulating node, the computing
node obtains the on-path delay, represented by the On-Path Delay data
field, by subtracting the timestamp reference from its current
timestamp, using the seconds and fraction fields of the format in use
at the full resolution of that format. The sign of the difference is
evaluated at this resolution, before any conversion. The difference
is then converted to microseconds and rounded to the nearest
microsecond. For the PTP truncated format, the fraction is in
nanoseconds. For the NTP 64-bit format, the fraction is in units of
2^(-32) seconds. For the POSIX format, the fraction is already in
microseconds.
Huang Feng, et al. Expires 19 April 2027 [Page 7]
Internet-Draft IOAM On-Path Delay October 2026
A packet that carries only one of the Timestamp Seconds and Timestamp
Fraction fields does not conform to this specification. A node
receiving such a packet MUST NOT use the timestamp reference to
compute the On-Path Delay data field value, and MUST set the field to
0xFFFFFFFF as described in Section 4.
Each of these fields is an optional 4-octet field as defined in
[RFC9326], so the length of the DEX Option-Type increases by 4 octets
for each field present. The Extension-Flags explicitly indicate this
length, as specified in [RFC9326]. For example, when the DEX Option-
Type is carried in IPv6 as specified in [RFC9486], the Opt Data Len
field MUST reflect the resulting length, the 4-octet alignment is
preserved because each field is 4 octets long, and the total length
of the IOAM Option-Types remains subject to the 255-octet limit
defined in that Other IOAM encapsulations, as presented in Section 5
of [RFC9378] have their own constraints, which this document does not
change. A node that does not support these fields ignores them as
specified in [RFC9326].
The computed value is meaningful only if the clocks of the IOAM
encapsulating node and of the computing node are synchronized, and
the accuracy of the result is bounded by the accuracy of that
synchronization. Nodes participating in this measurement MUST use
the same timestamp format, as determined by the management plane (see
[RFC9197]). Deployment guidance on clock synchronization is provided
in Section 8.
6. Export of On-Path Delay
This document does not define an export mechanism. It defines the
IOAM extensions required for an IOAM node to compute the on-path
delay and to identify that value as a telemetry metric that can be
locally aggregated and/or exported.
[RFC9951] defines IPFIX Information Elements and mechanisms for
exporting on-path telemetry, including statistical views of on-path
delay. This is particularly suitable when DEX processing aggregates
delay samples on a per-flow basis, because the resulting telemetry
can be exported in the context of the customer flow that triggered
the measurements.
Other export mechanisms can also be used. The YANG data model
defined in [I-D.ietf-ippm-on-path-telemetry-yang] provides on-path
telemetry data that can be published using YANG-Push. In addition,
[I-D.spiegel-ippm-ioam-rawexport] describes a mechanism for exporting
raw IOAM data. The choice of export mechanism is outside the scope
of this document.
Huang Feng, et al. Expires 19 April 2027 [Page 8]
Internet-Draft IOAM On-Path Delay October 2026
7. Security Considerations
The security considerations for the IOAM Trace-Type and IOAM DEX
Option-Type are defined in [RFC9197] and [RFC9326]. The additional
considerations below apply to the extensions defined in this
document.
The on-path delay is computed from a timestamp reference carried in
the packet and from the local clock of the computing node. An
attacker that can modify the timestamp reference in a packet, or that
can disturb the time synchronization of the IOAM nodes, can corrupt
the delay samples. The consequences include delay values that are
too small or too large, samples rejected as not populated, and
locally aggregated statistics that hide or exaggerate a performance
problem. Operators SHOULD protect the time source and the
synchronization protocol used by the IOAM nodes, as discussed in
[RFC9378], and use the extensions only within a limited IOAM domain,
with packets filtered at the domain boundary.
The Timestamp Seconds and Timestamp Fraction fields add 8 octets to
the DEX Option-Type for each packet that carries them. This
increases the per-packet overhead and the MTU and size considerations
of the encapsulation that carries the option, for example [RFC9486]
for IPv6, and the rate-limiting guidance of [RFC9326] continues to
apply.
8. Operational Considerations
The on-path delay is computed from the timestamps of two different
nodes, so the accuracy of the result depends on how closely the
clocks of the IOAM encapsulating node and of the computing node are
synchronized. Nodes participating in this measurement SHOULD
synchronize their clocks using a protocol such as PTP or NTP.
Operators need to ensure that the synchronization accuracy is small
compared to the delays being measured.
When synchronization degrades, the difference between the timestamps
can become negative, and the node then reports the On-Path Delay data
field as not populated, as described in Section 4. A higher rate of
such values can indicate a synchronization problem, so operators are
advised to monitor the clock status of the IOAM nodes that take part
in the measurement. The protection of the time source is discussed
in Section 7.
9. IANA Considerations
This document requests IANA to allocate the following bit in the
"IOAM Trace-Type" registry.
Huang Feng, et al. Expires 19 April 2027 [Page 9]
Internet-Draft IOAM On-Path Delay October 2026
Bit: [TBD3]
Description: On-path delay between the IOAM encapsulating node
and the transit or decapsulating node measured in
microseconds.
Reference: [This-RFC-to-be]
This document also requests IANA to allocate the following two bits
in the "IOAM DEX Extension-Flags" registry.
Bit: [TBD1]
Description: Timestamp Seconds
Reference: [This-RFC-to-be]
Bit: [TBD2]
Description: Timestamp Fraction
Reference: [This-RFC-to-be]
10. Acknowledgements
The authors would like to thank Greg Mirsky for the review and
valuable comments. The authors also thank Haoyu Song for the review
and valuable comments.
11. References
11.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi,
Ed., "Data Fields for In Situ Operations, Administration,
and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197,
May 2022, <https://www.rfc-editor.org/info/rfc9197>.
[RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T.
Mizrahi, "In Situ Operations, Administration, and
Maintenance (IOAM) Direct Exporting", RFC 9326,
DOI 10.17487/RFC9326, November 2022,
<https://www.rfc-editor.org/info/rfc9326>.
Huang Feng, et al. Expires 19 April 2027 [Page 10]
Internet-Draft IOAM On-Path Delay October 2026
[RFC9486] Bhandari, S., Ed. and F. Brockners, Ed., "IPv6 Options for
In Situ Operations, Administration, and Maintenance
(IOAM)", RFC 9486, DOI 10.17487/RFC9486, September 2023,
<https://www.rfc-editor.org/info/rfc9486>.
[RFC9951] Graf, T., Claise, B., and A. Huang-Feng, "Export of Delay
Performance Metrics in IP Flow Information Export
(IPFIX)", RFC 9951, DOI 10.17487/RFC9951, April 2026,
<https://www.rfc-editor.org/info/rfc9951>.
11.2. Informative References
[I-D.ietf-ippm-on-path-telemetry-yang]
Fioccola, G., Zhou, T., Zhu, Y., Zhang, W., Feng, A. H.,
and K. Zhu, "On-Path Telemetry YANG Data Models", Work in
Progress, Internet-Draft, draft-ietf-ippm-on-path-
telemetry-yang-07, 9 October 2026,
<https://datatracker.ietf.org/doc/html/draft-ietf-ippm-on-
path-telemetry-yang-07>.
[I-D.spiegel-ippm-ioam-rawexport]
Spiegel, M., Brockners, F., Bhandari, S., and R.
Sivakolundu, "In-situ OAM raw data export with IPFIX",
Work in Progress, Internet-Draft, draft-spiegel-ippm-ioam-
rawexport-07, 12 February 2024,
<https://datatracker.ietf.org/doc/html/draft-spiegel-ippm-
ioam-rawexport-07>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[RFC9378] Brockners, F., Ed., Bhandari, S., Ed., Bernier, D., and T.
Mizrahi, Ed., "In Situ Operations, Administration, and
Maintenance (IOAM) Deployment", RFC 9378,
DOI 10.17487/RFC9378, April 2023,
<https://www.rfc-editor.org/info/rfc9378>.
Authors' Addresses
Alex Huang Feng
Deutsche Telekom
Barcelona
Spain
Email: alex.huang-feng@t-systems.com
Huang Feng, et al. Expires 19 April 2027 [Page 11]
Internet-Draft IOAM On-Path Delay October 2026
Pierre Francois
INSA-Lyon
Lyon
France
Email: pierre.francois@insa-lyon.fr
Benoit Claise
Everything OPS & Arrcus
Belgium
Email: benoit@everything-ops.net
Thomas Graf
Swisscom
Binzring 17
CH-8045 Zurich
Switzerland
Email: thomas.graf@swisscom.com
Justin Iurman
6WIND
Email: justin.iurman@gmail.com
Huang Feng, et al. Expires 19 April 2027 [Page 12]