A reinforcement learning approach for cost-efficient and fault-tolerant placement of stateful virtual network functions

Abstract

Network virtualization has become one of the key technologies for future networks. Soft- ware Defined Networking (SDN) makes the architecture of a network flexible, enabling the service providers to provide particular functions of the network and have fast response to- wards the service that they provide. Through network virtualization, networking nodes like firewalls, load balancers and intrusion detection systems, hitherto implemented as dedicated hardware, are transformed into software applications implemented inside vir- tual machines and containers hosted on general-purpose servers. Such functions can be easily migrated from one location to another, scaled-in/-out and can be flexibly activated or shut down depending on real time resource requirements. This enhances the revenue of network service providers because of efficient and flexible utilisation of resources and lowers both the capital and operation expenditure incurred by network service providers. Under such a softwarized environment, services are anticipated to be realised as a chain of virtualized network functions commonly referred to as service function chains. However, despite the numerous benefits of the virtualization paradigm, it presents a number of challenges in different dimensions such as service reliability due to the intro- duction of more sources of failure at both the software and hardware levels that in turn lead to losses due to penalties from the failure to meet the service level agreements (SLA). In this regard, this thesis proposed the application of Reinforcement learning to ad- dress the problem of service reliability and cost-efficient deployment of service chains in virtual network functions. The results show that the proposed Reinforcement Learning algorithm was able to optimize the service requests acceptance ratio, leading up to 27% improvement as well as 30% improvement in the placement cost of the service function chain (SFC) compared to the state-of-the art algorithm.