RL-TAC: A Deep Reinforcement Learning And Trust-Aware Adaptive Clustering Framework For Heterogeneous, Mobile-Sink Wireless Sensor Networks

Abstract

Wireless Sensor Networks (WSNs) have been increasingly employed in various applications such as smart cities, the industrial Internet of Things (IoT) and environmental monitoring. However, there are a number of challenges that need to be addressed in order to improve the performance of WSNs. One of the main challenges in using WSNs is that the sensor nodes are powered by batteries which have limited energy and therefore the energy consumption of the nodes must be kept to a minimum. In addition, the topology of a WSN can change over time and there are a number of security threats to the network. Many of the existing clustering approaches in WSNs lack some key features such as adaptability, robust trust management, efficient energy harvesting and support for mobile sinks.

To support efficient and robust communication in large-scale and dynamic networks, an adaptive, trust-aware and energy-efficient clustering framework called RL-TAC for Reinforcement Learning-based Trust-Aware Adaptive Clustering is proposed. In RL-TAC, DRL, trust-based security, energy prediction for solar-powered sensor nodes and mobile-sink-aware routing are integrated.

In this paper, a novel framework called Reinforcement Learning-based Trust-Aware Adaptive Clustering (RL-TAC) has been introduced. The core of the proposed framework is a Deep Q-Network (DQN) that is trained to select the optimal cluster-heads in WSNs for extending the network lifetime while maintaining acceptable metrics for other performance metrics. To enhance the security of WSNs, trust-based security has been integrated into the framework to identify and isolate the behaviorally malicious nodes from the network using a Beta-distribution trust model enhanced by an Anomaly-detection module. The energy harvesting solar-powered sensor nodes benefit from an LSTM-based energy prediction model. The mobile-sink-aware routing is employed to optimize the routing of the collected data by sensor nodes to the mobile sink in the network. The performance evaluation of the proposed framework has been conducted by MATLAB/Python-based simulations for a 1000-node large-scale heterogeneous WSN. In addition, Contiki-NG/Cooja emulation has been used for validating the performance of the RL-TAC framework. Comparisons have been also made with well-established swarm intelligence approaches and state-of-the-art DRL-based clustering and routing techniques.

Results obtained by RL-TAC outperform a number of existing methods for managing of WSNs. The comparison was carried out in terms of the following metrics: saving of energy, extending of the network lifetime, clustering accuracy, packet delivery ratio and accuracy of detection of malicious nodes. The highest values of the aforementioned metrics have been achieved by RL-TAC: the amount of saved energy equals 18.4%, the network lifetime was extended up to 22.7%, clustering accuracy equals 97.9%, the packet delivery ratio equals 98.1% and the detection accuracy of malicious nodes equals 95.6%. False-positives ratio equals 3.2%. Introduced in the paper additional complexity of approach (caused by DRL, trust-based and energy-prediction by means of LSTM for each solar-powered sensor node) is acceptable for all considered scenarios. The proposed approach is also scalable and can be used in a wide variety of network environments.

The proposed framework is secure, adaptive and energy efficient clustering for the heterogeneous WSNs to prolong the network lifetime, to improve the communication reliability and to detect the malicious nodes. Therefore, the proposed framework is also suitable for future industrial IoT, smart city and environmental monitoring applications.