Latency and Lifetime Enhancements in IWSN: a Q-Learning Approach for Graph Routing

Industrial Wireless Sensor Networks usually havea centralized management approach, where a device known asNetwork Manager is responsible for the overall configuration,definition of routes, and allocation of communication resources.Graph routing is used to increase the reliability of the communicationsthrough path redundancy. Some of the state-of-the-art graph routing algorithms use weighted cost equationsto define preferences on how the routes are constructed. Thecharacteristics and requirements of these networks complicateto find a proper set of weight values to enhance networkperformance. Reinforcement Learning can be useful to adjustthese weights according to the current operating conditionsof the network. We present the Q-Learning Reliable Routingwith a Weighting Agent approach, where an agent adjusts theweights of a state-of-the-art graph routing algorithm. The statesof the agent represent sets of weights, and the actions changethe weights during network operation. Rewards are given tothe agent when the average network latency decreases or theexpected network lifetime increases. Simulations were conductedon a WirelessHART simulator considering industrial monitoringapplications with random topologies. Results show, in most cases,a reduction of the average network latency while the expectednetwork lifetime and the communication reliability are at leastas good as what is obtained by the state-of-the-art graph routingalgorithms.