Access Type

Open Access Dissertation

Date of Award

January 2015

Degree Type

Dissertation

Degree Name

Ph.D.

Department

Mathematics

First Advisor

George G. Yin

Abstract

In this dissertation, we present three problems arising in recent applications of stochastic approximation methods. In Chapter 2, we use stochastic approximation to analyze Particle Swarm Optimization (PSO) algorithm. We introduce four coefficients and rewrite the PSO procedure as a stochastic approximation type iterative algorithm. Then we analyze its convergence using weak convergence method. It is proved that a suitably scaled sequence of swarms converge to the solution of an ordinary differential equation. We also establish certain stability results. Moreover, convergence rates are ascertained by using weak convergence method. A centered and scaled sequence of the estimation errors is shown to have a diffusion

limit. In Chapter 3, we study a class of stochastic approximation algorithms with regime switching that is modulated by a discrete Markov chain having countable state spaces and two-time-scale structures. In the algorithm, the increments of a sequence of occupation measures are updated using constant step size. It is demonstrated that least squares estimations from the tracking errors can be developed. Under the assumption that the adaptation rates are of the same order of magnitude as that of times-different parameter, it is proven that the continuous-time interpolation from the iterates converges weakly to some system of ordinary differential equations (ODEs) with regime switching, and that a suitably scaled sequence of the tracking errors converges to a system of switching diffusion. This work is an extension of the work in [80]. In Chapter 4, we developed asynchronous stochastic approximation (SA) algorithms for networked systems with multi-agents and regime-switching topologies to achieve consensus control. There are several distinct features of the algorithms. (1)

In contrast to the most existing consensus algorithms, the participating agents compute and communicate in an asynchronous fashion without using a global clock. (2) The agents compute and communicate at random times. (3) The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. (4) The functions involved are allowed to vary with respect to time hence nonstationarity can be handled. (5) Multi-scale formulation enriches the applicability of the algorithms. In the setup, the switching process contains a rate parameter $\e > 0$ in the

transition probability matrix that characterizes how frequently the topology switches. The algorithm uses a step-size $\mu$ that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, it is shown that a continuous-time interpolation of the iterates converges weakly to a

system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain, or to a system of differential equations (an average with respect to certain measure). In addition, a scaled sequence of tracking errors converges to a witching diffusion or a diffusion. Simulation

results are presented to demonstrate these findings.

Download

Share

COinS