Small World IoT Networks using Deep-RL




This project aims to develop a predictive model for identifying faulty sensor nodes in small world IoT networks. The proposed approach uses Actor-Critic Reinforcement Learning to introduce small world characteristics in the network and leverages ML algorithms and network analysis techniques to detect and predict faulty sensor nodes. We conducted several experiments to validate our approach's effectiveness using simulated small world IoT networks and analyzed major network parameters like Lifetime, Energy Consumption, Latency, and Throughput. Our results came out to be promising and comparatively better than the existing state of the art.

[Code] [Slides] [Report]

Fault Node Detection


Sensor nodes possess low initial energies and tend to fail or die during data routing through a large IoT network. For efficient data transmission with optimal energy consumption and lifetime, we need to predict nodes that tend to fail so that we can eliminate them from the network. We prepare our own fault node dataset using a simulated IoT network by adding noise to network properties. Using that dataset, we trained various ML algorithms for anomaly/outlier detection and tested them on our network. The algorithms and obtained results are described below.

1. One Class SVM
2. Elliptic Envelope
3. Isolation Forest
4. Local Outlier Factor
5. Density-Based Spatial Clustering (DBSCAN)

From these results, we can see that DBSCAN is the most optimal approach for our task. It has outperformed the other popular approaches and has better values of shown metrics.

Hence, we selected DBSCAN for our pipeline and integrated it with data routing in our small world networks. Fault node prediction improved the network performance by increasing its lifetime and decreasing energy consumption, as discussed in later sections.

Small World Networks

Small World is a phenomenon, first observed in the social connectivity of people. Here we are using small world phenomenon in the context of WSNs. Introducing a small world in wireless sensor networks means introducing long-range links between nodes, decreasing the Average Path Length(APL) and Average Clustering Coefficient(ACC). It makes the network well-connected, efficiently transmitting data with high lifetime, throughput, low energy consumption, and latency.

While adding small world in WSNs, we introduce long- range links which brings the network in the small-world region. We need an algorithm which can efficiently introduce the links such that the network gets the most optimal small world characteristics.

Actor-Critic Reinforcement Learning

We are using the PPO algorithm to introduce long-range links in the network. We define the algorithm as follows:
1. Environment: The simulated WSN serves as the environment for PPO
2. State: The network graph serves as the state of the network
3. Action: The node IDs that need to be connected are actions
4. Reward: Reward is defined in terms of ACC and APL as follows:

Small World Results

Here we show results for a 500mX500m network with 400 nodes.

Links Introduced with Sink Node

APL-ACC Comparison with Small World

Links Introduced Within Nodes

APL-ACC Comparison with Small World


Data Routing

We analyze and compare the performance of obtained small-world networks with conventional netwworks by routing data across the network. We also tend to study the effect of fault node detection. We calculated the following metrics for comparison.

1. Energy Consumption
2. Network Lifetime
3. Throughput
4. Latency

Performance Analysis

Small World WSN vs Conventional WSN

Small World WSN vs Direct Transmission

Effect of Fault Node Prediction

This webpage template was borrowed from some colorful folks.