A Reinforcement Learning-Based Scheduling Scheme for the IEEE 802.15.4e TSCH Network
Received: 10 May 2025 | Revised: 29 June 2025, 19 July 2025, and 22 July 2025 | Accepted: 27 July 2025 | Online: 8 August 2025
Corresponding author: Nadia Zerguine
Abstract
Time-Slotted Channel Hopping (TSCH), as defined in the IEEE 802.15.4e amendment, is currently considered the most widely used Medium Access Control (MAC) protocol in Industrial Internet of Things (IIoT) wireless networks. However, the absence of a defined scheduling process within the standard remains an open area of research. This paper presents a proposal for integrating Q-Learning (QL), a Reinforcement Learning (RL) method, into the TSCH protocol, allowing parent and channel selection in an intelligent manner. This intelligent allocation aims to optimize the performance of TSCH in the IIoT. The various in-depth simulations carried out show that the integration of QL significantly improves the performance of these networks, including radio reliability, Packet Delivery Ratio (PDR), and especially latency and energy consumption.
Keywords:
Time-Slotted Channel Hopping (TSCH), Industrial Internet of Things (IIoT), Reinforcement Learning (RL), scheduling, Q-Learning (QL)Downloads
References
E. Sisinni, A. Saifullah, S. Han, U. Jennehag, and M. Gidlund, "Industrial Internet of Things: Challenges, Opportunities, and Directions," IEEE Transactions on Industrial Informatics, vol. 14, no. 11, pp. 4724–4734, Nov. 2018.
IEEE Standard for Low-Rate Wireless Networks, Apr. 2016.
A. Vatankhah and R. Liscano, "Comparative Analysis of Time-Slotted Channel Hopping Schedule Optimization Using Priority-Based Customized Differential Evolution Algorithm in Heterogeneous IoT Networks," Sensors, vol. 24, no. 4, Feb. 2024, Art. no. 1085.
A. R. Raut, S. Khandait, and D. Theng, "A Priority based Self-Organised MAC Protocol for Real Time Wireless Sensor Network Applications," Engineering, Technology & Applied Science Research, vol. 14, no. 6, pp. 18600–18607, Dec. 2024.
M. Kherbache, O. Sobirov, M. Maimour, E. Rondeau, and A. Benyahia, "Decentralized TSCH scheduling protocols and heterogeneous traffic: Overview and performance evaluation," Internet of Things, vol. 22, Jul. 2023, Art. no. 100696.
A. R. Urke, Ø. Kure, and K. Øvsthus, "A Survey of 802.15.4 TSCH Schedulers for a Standardized Industrial Internet of Things," Sensors, vol. 22, no. 1, Jan. 2022, Art. no. 15.
F. Righetti, C. Vallati, S. K. Das, and G. Anastasi, "Analysis of Distributed and Autonomous Scheduling Functions for 6TiSCH Networks," IEEE Access, vol. 8, pp. 158243–158262, 2020.
P. Li, Y. Xiao, J. Yan, X. Li, and X. Wang, "Reinforcement Learning for Adaptive Resource Scheduling in Complex System Environments," in 2024 5th International Symposium on Computer Engineering and Intelligent Communications, Wuhan, China, 2024, pp. 92–98.
T. Chang, T. Watteyne, Q. Wang, and X. Vilajosana, "LLSF: Low Latency Scheduling Function for 6TiSCH Networks," in 2016 International Conference on Distributed Computing in Sensor Systems, Washington, USA, 2016, pp. 93–95.
I. F. Vieira Junior, J. Granjal, and M. Curado, "Insights for metrics in assessing TSCH scheduling efficiency," Internet of Things, vol. 29, Jan. 2025, Art. no. 101440.
P. Thubert, "An Architecture for IPv6 over the Time-Slotted Channel Hopping Mode of IEEE 802.15.4 (6TiSCH)," Internet Engineering Task Force, Request for Comments RFC 9030, May 2021.
F. F. Jurado-Lasso, C. Orfanidis, J. F. Jurado, and X. Fafoutis, "HRL-TSCH: A Hierarchical Reinforcement Learning-Based TSCH Scheduler for IIoT," IEEE Transactions on Cognitive Communications and Networking, vol. 10, no. 6, pp. 2102–2118, Dec. 2024.
H. Park, H. Kim, S.-T. Kim, and P. Mah, "Multi-Agent Reinforcement-Learning-Based Time-Slotted Channel Hopping Medium Access Control Scheduling Scheme," IEEE Access, vol. 8, pp. 139727–139736, 2020.
B. Zhang, W. Wu, X. Bi, and Y. Wang, "A Task Scheduling Algorithm Based on Q-Learning for WSNs," in Communications and Networking: 13th EAI International Conference, ChinaCom 2018, Chengdu, China, 2018, pp. 521–530.
M. Kherbache, M. Maimour, and E. Rondeau, "QL-TSCH-plus: A Q-learning Distributed Scheduling Algorithm for TSCH Networks," in 2023 IEEE 9th World Forum on Internet of Things, Aveiro, Portugal, 2023, pp. 1–6.
Y. H. Pratama, S.-H. Chung, and D. Z. Fawwaz, "Low-Latency and Q-Learning-Based Distributed Scheduling Function for Dynamic 6TiSCH Networks," IEEE Access, vol. 12, pp. 49694–49707, 2024.
S. Duquennoy, B. Al Nahas, O. Landsiedel, and T. Watteyne, "Orchestra: Robust Mesh Networks Through Autonomously Scheduled TSCH," in Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, Seoul, South Korea, 2015, pp. 337–350.
N. Zerguine, M. Mostefai, Z. Aliouat, and Y. Slimani, "Intelligent CW Selection Mechanism Based on Q-Learning (MISQ)," Ingénierie des Systèmes d’Information, vol. 25, no. 6, pp. 803–811, Dec. 2020.
C. J. C. H. Watkins and P. Dayan, "Technical Note: Q-Learning," Machine Learning, vol. 8, no. 3, pp. 279–292, May 1992.
C. Thomson, I. Romdhani, A. Al-Dubai, M. Qasem, B. Ghaleb, and I. Wadhaj, Cooja Simulator Manual, 2016.
G. Oikonomou, S. Duquennoy, A. Elsts, J. Eriksson, Y. Tanaka, and N. Tsiftes, "The Contiki-NG open source operating system for next generation IoT devices," SoftwareX, vol. 18, Jun. 2022, Art. no. 101089.
D. C. Montgomery and G. C. Runger, Applied Statistics and Probability for Engineers, 7th ed. Hoboken, NJ, USA: Wiley, 2018.
Downloads
How to Cite
License
Copyright (c) 2025 Nadia Zerguine, Zibouda Aliouat, Samia Kharchi
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
