Monitoring and control of stochastic systems

The questions treated in this thesis are motivated by practical applications with a particular focus on the monitoring and control of telecommunication networks. The thesis is divided into three parts. Part I is about the static control of such stochastic systems with the objective of meeting certain performance criteria. In particular, we provide performance evaluation techniques for call centres based on queuing models. We find asymptotic expressions as well as provably efficient simulation algorithms for the purpose of estimating certain rare event probabilities of interest such as the probability of overload.
In Part II we consider a class of dynamic control problems known as restless multiarmed bandit problems, where the decision maker needs to select arms of a bandit (a slot machine) to play on, with the objective of maximising the rewards accumulated over time. We model control problems in telecommunications such as the problem of dynamically selecting transmission channels. For suitable bandit models we investigate the performance and structural properties of the celebrated Whittle index compared to a naive myopic policy.
The performance of any control policy derived from a stochastic model crucially depends on the validity of the imposed modelling assumptions. In Part III we therefore consider questions related to the monitoring and testing of data for the presence of ‘change points’ at which the assumed probability distribution has changed. We investigate novel criteria for the false alarm performance of such sequential tests, and propose methods for evaluating these criteria based on approximations.