Graph Neural Networking Challenge 2023

Creating a Network Digital Twin with Real Network Data


Evaluation: Closed

Registration: Closed

ITU AI/ML in 5G Challenge

ITU invites you to participate in the ITU Artificial Intelligence/Machine Learning in 5G Challenge, a competition which is scheduled to run from now until the end of the year. Participation in the Challenge is free of charge and open to all interested parties from countries that are members of ITU.

Detailed information about this competition can be found on the Challenge website.


BNN-UPC is glad to announce the “Graph Neural Networking challenge 2023”, which is organized as part of the “ITU AI/ML in 5G Challenge.

Final ranking

1st m0b1us 20.001436
2nd BUPT_CMCC 22.680460
3rd LKN 35.389392
4th Yanbiyu 43.393626
5th CAU-Networks 43.439188


In recent years, the networking community has produced robust Graph Neural Networks (GNN) that can accurately mimic complex network environments. Modern GNN architectures enable building lightweight and accurate Network Digital Twins that can operate in real-time. However, and as a consequence of the lack of real-world data, current ML-based models have been mainly developed and trained using simulated data. This has strongly limited our understanding on how existing models perform in a real network.

This edition of the Graph Neural Networking challenge focuses on developing for the first time a GNN-based Network Digital Twin using a dataset from a real network. This includes modeling the input traffic as well as the relationship between traffic, topology, and configuration (e.g. routing) to produce accurate estimates of the network performance (e.g. delay).

Problem statement

The goal of this challenge is to create a Network Digital Twin based on neural networks which can accurately estimate QoS performance metrics given the network state and the input traffic. More in detail, solutions must predict the resulting per-flow mean delay given: (i) a network topology (L2 and L3), (ii) a set of input flow packet traces, and (iii) a routing configuration. The following figure presents a schematic representation.


Figure 1: Schematic representation of RouteNet.

As a baseline we provide an open-source implementation of RouteNet: RouteNet-Fermi. This model is the outcome of the three previous editions of the challenge and it has been developed and trained based on simulated data. More importantly for this challenge, it does not support arbitrary packet traces as input.

RouteNet-Fermi assumes that the input traffic is generated from a stationary stochastic process and uses the moments of such distributions (e.g. mean, variance, etc) as features describing the traffic. In addition, the simulation environment represents an ideal network scenario and RouteNet-Fermi has not been tested with real networks.

The objective of this challenge is to build a neural network that provides accurate estimates of the mean per-flow delay with a dataset generated from a real network. As a novelty, the dataset contains the input per-flow packet traces. We encourage participants to focus on how to summarize the per-flow packet traces and the non-ideal behavior of real networks. The proposed solution can be based on RouteNet-Fermi, but we also encourage participants to develop their own from-scratch ideas.


For this challenge, we provide two datasets generated with a real network testbed. Each dataset (Fig. 2) contains samples generated in several topologies and includes hundreds of network configurations (routing) and traffic flows. In the first dataset traffic is generated using Contstant-Bit Rate (CBR) while in the second traffic is generated using Multi-Burst Rate (MBR). In both cases packet size is randomly selected per flow and constant for all packets within the flow. The range of the packet size is between 100 and 1450 bytes. More information about this can be found later on.


Figure 2: Structure of the Training Dataset


Each sample is labeled with average per-flow performance measurements (mean per-flow and per-packet delay) resulting from running the traffic through the network. In particular, this challenge focuses on the prediction of the mean per-flow delay. This is the average delay of all the packets of each flow at the egress point of the network.

To rank solutions we will post a test dataset. This dataset will contain samples obtained from the same distribution as the training datasets, including a mix of CBR and MBR samples. This dataset will not contain any information related to the mean per-flow delay.

Information about the testbed and dataset can be found here:


All the main resources can be found in the following GitHub repository:

This repository includes a README file, which includes the instructions on how to download the dataset and use it to train the baseline models. We also included a guide on how you can modify the baseline model’s code in order to create your own submissions.

Below there are other useful resources for the challenge:

  • Summary slides [Link].
  • Technical report of RouteNet-Fermi [Link].
  • Mailing list for questions and comments related to the challenge [Link].

Kickoff webinar


The objective is to test the properties of the proposed solutions. To this end, before the end of the challenge, we will provide a test dataset.

This dataset will contain samples with similar distributions to the samples included in the provided dataset. Participants must label this dataset using their neural network models and send the results, the estimated mean per-flow delay for all flows, in CSV format. For the evaluation, we will use the Mean Absolute Percentage Error (MAPE) score computed over the delay predictions produced by candidate solutions:


Solutions with lower MAPE score will be the winners.

During the evaluation, participants will be able to submit automatically their solutions in our evaluation platform and see a provisional ranking with all the teams in real-time.

Note: The test dataset will not include any data related to flow performance measurements (i.e., flow delay, jitter, loss). It means that these features cannot be used as input for the proposed solutions. Users of the dataset API can assume that NO data under the performance_matrix can be used as input for the model, as well as the delay information in pkt_info.


As a baseline, we provide RouteNet-Fermi [link], a Graph Neural Network (GNN) architecture recently proposed to estimate end-to-end performance metrics in networks. RouteNet-Fermi is the outcome of three editions of the challenge, and it has an unprecedented ability to make accurate performance predictions even in network scenarios larger than those seen during the training phase. In addition, RouteNet-Fermi supports a wide range of queue scheduling policies and several stochastic traffic models. Notably, it does not support packet traces as input.

We provide an open-source implementation of RouteNet-Fermi used as the baseline, as well as a tutorial on how to execute the code and modify fundamental characteristics of the model [link]. Participants are encouraged to modify the model or design their own neural network architecture from scratch.

You can find at this [link] a deck of slides introducing the challenge and some tips from the organizers to have a good start.

Final submissions and awards

After the score-based evaluation phase (see “Evaluation” section), a ranking of all the teams will be posted on this website.
The top-5 teams must send to the organizers: (i) the code of the neural network solution proposed, (ii) the neural network model already trained, and (iii) a brief document describing the proposed solution and how to reproduce it (1-3 pages). This will be revised to check that the solutions comply with all the rules of the challenge.

Top teams will be invited to present their solutions at the 2nd GNNet workshop, co-located with ACM CoNEXT (December 2023) held in  Paris. Please, find more details about this workshop below:

The winning teams of the Graph Neural Networking challenge will receive the following cash prizes:

  • 1st prize: 2000 EUR
  • Runner-up: 500 EUR

Provided that the Organizing Team validates the solutions and the Judges Panel from the ITU AI/ML in 5G Challenge determines that the solution satisfies the judging criteria.

Also, top-3 teams will advance to the Grand Challenge Finale of the ITU AI/ML in 5G Challenge.

The organizers encourage the publication of papers describing the best solutions. Always respecting the desire of those participants that may want to keep their solutions private.


IMPORTANT NOTE: It is mandatory filling the registration form to officially participate in this challenge.

Please, note you should register additionally in the portal of the ITU AI/ML in 5G challenge (“Login” and “Create Team”):
All participants must satisfy the following rules:
To participate in this challenge, the following rules must be satisfied:

  • The proposed solution cannot use network simulation tools or solutions derived from them.
  • Solutions must be trained only with samples included in the training dataset we provide. Data augmentation techniques are allowed as long as they exclusively use data from the training dataset. However, it is not allowed to use additional data obtained from other sources (e.g., network testbeds or simulators) or public datasets.
  • During the evaluation process, each team can submit a maximum of 5 solutions (TXT files) per day. In total, teams can make up to 20 evaluation submissions. In case of receiving more submissions, only the first submissions up to these limits will be considered.
  • The challenge is open to all participants except members of the challenge organizing team, members of the challenge advisory board, and/or individuals belonging to or associated with the research group “Barcelona Neural Networking Center-UPC”.
  • Any dispute will be resolved by the Organizing Committee supported by at least one member of the ITU AI/ML in 5G organization.

Note: In the challenge, you may use any existing neural network architecture (e.g., the RouteNet-Fermi implementation we provide). However, it has to be trained from scratch and it must be clearly cited in the description of the solution. In the case of RouteNet-Fermi, it should be cited as it is in [link].

In case of any personal questions about the rules, please contact the organizers via email at gnnetchallenge .

Contact and updates

Subscribe to [Challenge2023 mailing list] for announcements and questions/comments.

Please, note that it is needed to subscribe to the mailing list before sending any email. Otherwise, emails are dropped by default.


  • Challenge duration: June-Nov 2023
  • Open registration: June 7th-Sep 30th
  • Release of tools and validation dataset: June 30th July 3rd
  • Score-based evaluation phase: Oct 2nd-Oct 17th 2023
  • Provisional ranking of all the teams: Oct 18th 2023
  • Top-5 teams submit the dataset, code and documentation: Nov 1st 2023
  • Final ranking and official announcement of top-3 teams: Nov 2023
  • Best solutions pitch in a 3-day event end of to determine the finalists: 28 – 30 November 2023
  • GNNet Challenge award ceremony and presentation: Dec 8th 2023 in Paris
  • ITU Award ceremony and presentations: Dec 13th, 2023

Please note that these dates are tentative and may change slightly over the course of the challenge. Stay tuned for further updates via the mailing list and also on this website.

Organizing Committee


Miquel Ferriol



Albert López



Carlos Güemes



Prof. Albert Cabellos



Prof. Pere Barlet-Ros


Scientific Advisory Board


krzysztof Rusek

AGH University of Science and Technology


Fabien Geyer

Technical University of Munich


José Suárez-Varela

Telefónica Research

Barcelona Neural Networking Center (BNN-UPC)

Universitat Politècnica de Catalunya

ITU AI/ML in 5G Challenge

ITU invites you to participate in the ITU Artificial Intelligence/Machine Learning in 5G Challenge, a competition which is scheduled to run from now until the end of the year. Participation in the Challenge is free of charge and open to all interested parties from countries that are members of ITU.

Detailed information about this competition can be found on the Challenge website.

Want to explore the application of GNNs to networks?


Framework for fast prototyping of GNNs for communication networks

Developed by networking enthusiasts for scientists and engineers of the field.

Build a custom GNN tailored to your networking problem.

Cite this competition

Plain text (IEEE format):

J. Suárez-Varela, et al., “The Graph Neural Networking challenge: a world-wide competition for education in AI/ML for networks,” ACM SIGCOMM Computer Communication Review, vol. 51, no. 3, pp. 9–16, 2021.


  title={The graph neural networking challenge: a worldwide competition for education in AI/ML for networks},
  author={Su{\'a}rez-Varela, Jos{\'e} and others},
  journal={ACM SIGCOMM Computer Communication Review},


[1] J. Suárez-Varela, P. Almasan, M. Ferriol-Galmés, K. Rusek, F. Geyer, X. Cheng, X. Shi, S. Xiao, F. Scarselli, A. Cabellos-Aparicio, P. Barlet-Ros, “Graph Neural Networks for Communication Networks: Context, Use Cases and Opportunities”, arXiv:2112.14792, 2021. [paper]

[2] K. Rusek, J. Suárez-Varela, A. Mestres, P. Barlet-Ros, A. Cabellos-Aparicio, “Unveiling the potential of Graph Neural Networks for network modeling and optimization in SDN,” In Proceedings of the ACM Symposium on SDN Research (SOSR), pp. 140-151, 2019. [ACM SOSR 2019] [arXiv]