Delay Tolerance for Stable Stochastic Systems and Extensions Xiaofeng Zong , Tao Li , Senior Member, IEEE, George Yin , Fellow, IEEE, and Ji-Feng Zhang , Fellow, IEEE

2604 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021 Delay Tolerance for Stable Stochastic Systems and Extensions Xiaofeng Zong , Tao Li , Senior Member, IEEE, George Yin , Fellow, IEEE, and Ji-Feng Zhang , Fellow, IEEE

Abstract—This article establishes a “robustness” type I. INTRODUCTION result, namely, delay tolerance for stable stochastic systems under suitable conditions. We study the delay toler- HIS article presents an effort to address the important ance for stable stochastic systems and delayed feedback T question: How much delay can a stochastic system endures controls of such systems, where the delay can be state- so that the stability is still achieved. This, in fact, can be thought dependent or induced by the sampling-data. First, we con- of as a robustness consideration. The range of allowable delays sider systems with global Lipschitz continuous coefficients can be thought of as a “stability margin” in certain sense. Our and show that when the original stochastic system without consideration stems from the fact that stochastic systems are delay is pth moment exponentially stable, the system with used frequently in the real-world applications ranging from small delays is still pth moment exponentially stable. In p numerous dynamic systems in engineering, computer science, particular, when the th moment exponential stability is and network science to population biology, epidemiology, and based on Lyapunov conditions, we can obtain explicit delay bounds for moment exponential stability. Then, we consider economics [1]. While the study of stochastic systems has a long a class of stochastic systems with nonglobal Lipschitz con- history, one of the central issues drawing much attention in the ditions and find a delay bound for almost sure and mean literature is stability [2]–[5], which offers challenges and oppor- square exponential stability. As extension of the stability tunities to the automatic control theory [6]–[9]. With stochas- tolerance criteria, consensus, and tracking control of mul- tic perturbations, stability analysis involves various stochastic tiagent systems with measurement noises and nonuniform stability concepts such as stability in probability, stability in delays are studied. distribution, almost sure stability, and moment stability [3], [10]. Index Terms—Delay, multiagent system, stability, This is different from the systems with deterministic distur- stochastic system. bances in Hu et al. [11]. For many applications, delay is often unavoidable. This together with stochastic perturbations leads to the consideration of stochastic differential delay systems (SDDSs) [3], [12], whose future state depends not only on the present but also on the past history. Because stochastic Manuscript received August 9, 2018; revised February 2, 2020; ac- stability of such SDDSs is vital, much effort has been devoted cepted July 18, 2020. Date of publication July 28, 2020; date of current to the study; both Razumikhin methods and Lyapunov function version May 27, 2021. The work of Xiaofeng Zong, Tao Li, and Ji-Feng (or functional) methods are used for the stability analysis for Zhang was supported in part by the National Key R&D Program of SDDSs. Using Razumikhin methods, moment asymptotic and/or China under Grant 2018YFA0703800, in part by the National Natural Science Foundation of China under Grants 61703378 and 61977024, exponential stability were obtained [13]–[17], whereas using in part by the Shu Guang project of Shanghai Municipal Education Lyapunov function (or functional) methods, not only the moment Commission and Shanghai Education Development Foundation under stability but also the almost sure stability were obtained. By the Grant 17SG26, and in part by the Fundamental Research Funds for Lyapunov function method, Mao [3] gave delay-independent the Central Universities, China University of Geosciences (Wuhan). The work of George Yin was supported by the US National Science Foun- pth moment stability and obtained almost sure stability from dation under DMS-1710827. Recommended by Associate Editor Prof. the moment exponential stability under linear growth condition. C.-Y. Kao. (Corresponding author: Ji-Feng Zhang.) Rakkiyappan et al. [18] established the moment stability condi- Xiaofeng Zong is with the School of Automation, China University tions in terms of linear matrix inequalities (LMIs) for uncertain of Geosciences, Wuhan 430074, China, and also with the Hubei Key stochastic neural networks with delays. Gershon et al. [19] stud- Laboratory of Advanced Control and Intelligent Automation for Complex Systems, Wuhan 430074, China (e-mail: [email protected]). ied H∞ state-feedback control of stochastic delay systems using Tao Li is with the Shanghai Key Laboratory of Pure Mathemat- Lyapunov functions and LMIs. Shaikhet [20] also introduced ics and Mathematical Practice, School of Mathematical Sciences, many Lyapunov functionals to examine the stochastic stability East China Normal University, Shanghai 200241, China (e-mail: of different SDDSs. Using suitable Lyapunov functionals, Fei [email protected]). George Yin is with the Department of Mathematics, University of et al. [21] established the delay-dependent moment stability of Connecticut, Storrs, CT 06269-1009 USA (e-mail: [email protected]). the highly nonlinear hybrid stochastic system. Ji-Feng Zhang is with the Key Laboratory of Systems and Control, Although many important results have been obtained to date, Institute of Systems Science, Academy of Mathematics and Systems little is known about the delay tolerance for stable stochastic Science, Chinese Academy of Sciences, Beijing 100190, China, and systems. We consider the following stochastic system also with the School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100149, China (e-mail: [email protected]). d Color versions of one or more of the figures in this article are available online at https://ieeexplore.ieee.org. dy(t)=f(y(t))dt + gi(y(t))dwi(t),t≥ t0 (1) Digital Object Identifier 10.1109/TAC.2020.3012525 i=1

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. ZONG et al.: DELAY TOLERANCE FOR STABLE STOCHASTIC SYSTEMS AND EXTENSIONS 2605

n n with the initial data y(t0)=y0, f,gi : R → R , square stable stochastic systems were investigated under the T f(0) = gi(0) = 0, w(t)=(w1(t),w2(t),...,wd(t)) is a global Lipschitz conditions. However, stability tolerance under d-dimensional standard Brownian motion. We first assume that the general delays (time-vary or state-dependent) and general the stochastic system (1) (its trivial solution) is pth (p ≥ 2) Lyapunov conditions have not been examined. Moreover, the moment exponentially stable or almost surely exponentially issues under the nonglobal Lipschitz conditions have not been stable. That is, there are constants M>0 and γ>0, such that well understood. This article fills in these gaps. | | Inspired by the idea in [28], we study the delay tolerance for E| |p ≤ E| |p −γt log y(t) ≤− y(t) M y0 e , or lim sup γ a.s. moment or almost surely stable stochastic systems. According to t→∞ t the different information about the delay-free systems, different The fundamental question is under what conditions the fol- delay tolerance results are obtained. Under the global Lipschitz lowing delay system is still pth moment (or almost surely) condition, by assuming that we only know that the trivial solution exponentially stable of the delay-free systems is pth moment exponentially stable, d1 d we obtain a tolerance delay bound for the pth moment and dx(t)= fi(ϑi(xt))dt + gi(θi(xt))dwi(t) (2) almost sure exponential stability. We derive a weaker delay i=1 i=1 bound if the trivial solution of the delay-free system is moment { }d1 exponentially stable based on a Lyapunov condition. For such where the sequence fi i=1 is a decomposition of f n n d1 results, the concrete form of the delay need not be known, where with fi : R → R , fi(0) = 0, and f(x)= i fi(x), n=1 n xt = {x(t + u):u ∈ [−τ,0]}, ϑi,θi : C([−τi, 0], R ) → R the delays can be deterministic, random, or state-dependent. Under nonglobal Lipschitz conditions, we consider the case with satisfies |ϑi(ϕ)|∨|θi(ϕ)|≤sups∈ −τ , |ϕ(s)| for ϕ ∈ [ i 0] time-varying delays. It will be proved that if the time-varying − Rn ≥ d1∨d{ } C([ τ,0], ), τi 0, τ =maxi=1 τi . Here, delay terms delays are differentiable (or differentiable except at a sequence), can have various forms like the state-dependent delay studied then exponentially stable stochastic system can still be tolerant in [22], the deterministic delay (fixed or time-varying), and to time-varying delays with small derivatives. These results can random delay [23]. solve many control problems with delayed feedback control Considering the issue above leads to delay tolerance criteria. without the global Lipschitz and linear growth conditions. 1) First, it is important in the stability analysis of SDDSs. As an extension, the control of multiagent systems with In fact, Lyapunov functional is not unique for solving noises and time-varying nonuniform delays is investigated. The the stability of SDDSs. To choose an appropriate Lya- delay tolerance results aim to solve the consensus and tracking punov functional is a difficult work. Based on the delay problem of multiagent systems under the multiplicative noises tolerance criteria, one can find the delay bound directly and the nonuniform delays. This can be considered as a further without wasting time on trying to find suitable Lyapunov extension of our recent works [29], [30] from the uniform functionals. fixed delays to the time-varying nonuniform delays. In fact, 2) Second, it facilitates the design of delayed feedback the nonuniform delays for multiagent consensus have been investigated intensively for deterministic models (see [31]–[35] control for stochastic systems. With the delay tolerance for example). The Lyapunov functionals with derivatives of the criteria, we do not need to change the control gain when states and characteristic methods are two important tools for the delay appears and falls in an allowed region. In fact, designing the consensus control and establishing the consensus one only requires the stability of the delay-free system conditions. However, the two methods fail in the presence of and the structure of the delays. noises since the state of stochastic system is not differentiable 3) Finally, it can produce the stability of a class of semidis- and the characteristic methods are difficult to be applied. To date, crete stochastic systems. Most importantly, with the de- little is known about the models with measurement noises and lay tolerance criteria, we can obtain the design of the nonuniform delays. With the delay tolerance results obtained in sampled-data control and the explicit relationship be- this article, we have the ability to overcome the difficulty induced tween the sampling period and the systems parameters. by nonuniform delays and obtain the design of the consensus and Prior to this article, the delay tolerance has been considered tracking protocol. mainly for the almost surely stable stochastic systems. Mo- The rest of the article is arranged as follows. Section II hammed and Scheutzow [24] and Scheutzow [25] considered the addresses the delay tolerance under the global Lipschitz assump- almost sure exponential stability of the linear scalar stochastic tion, where the exponential stability condition and Lyapunov condition are studied, respectively. Section III examines the delay equation with pure diffusion dx(t)=σθ1(xt)dw1(t),σ > 0, and showed that the delay system is still almost sure stable for delay tolerance for the mean square and almost sure expo- sufficient small delay. Most recently, Mao et al. have engaged in nential stability under the nonglobal Lipschitz and local linear the study of almost sure stability of the general SDDSs [26] and growth conditions. Section IV applies the delay tolerance idea the associated switching cases [27]. These works are important to study consensus and tracking control of multiagent systems since they showed that almost surely stable system can be with nonuniform time-varying delays and measurement noises. tolerant with a small delay, where the small delay bound has been Section V concludes the paper with further remarks. revealed in [26] and [27] for the almost sure stability. One can Notation: We work with the n-dimensional Euclidean space Rn equipped with the Euclidean norm |·|. For a vector or a easily see from the references above, if the diffusion is not degen- T erate, then its delay can still contribute to the almost sure stabil- matrix A, its transpose is denoted by A . For a matrix A, denote T ity, but this is not the case for degenerated diffusions. Hence, one its trace norm by |A| = trace(A A); for a symmetric matrix has to consider the stability without taking the positive role of A with real entries, denote by λmax(A) and λmin(A) the largest noises into consideration. A representative work in this direction and smallest eigenvalues, respectively. Use a ∨ b to denote is Mao [28], where the fixed delay tolerance issues for the mean max{a, b} and a ∧ b to denote min{a, b}.Forτ>0, denote by

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2606 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

C([−τ,0]; Rn) the family of all Rn-valued continuous functions Itô process − | | on the interval [ τ,0] with the norm ϕ C =supt∈[−τ,0] ϕ(t) . d F P Let (Ω, , ) be a complete probability space with a filtration dX(t)=F (t)dt + Gi(t)dwi(t) {Ft}t≥ satisfying the usual conditions. That is, it is right 0 i=1 continuous and increasing while F0 contains all P-null sets. 1 n 2 n where F (t) ∈ L (R+; R ) and F (t) ∈ L (R+; R ). Then for 2,1 n any V ∈ C (R × R+; R+), we have the following Itô for- II. DELAY TOLERANCE UNDER GLOBAL mula [3] LIPSCHITZ CONDITIONS t V (X(t),t)=V (X(t0),t0)+ SV (X(s),s)ds In this section, we assume that the coefficients of (1) are t0 Lipschitz continuous. d t Assumption II.1: Assume that f,g satisfy f(0) = g(0) = 0 + Vx(X(s),s)Gi(s)dwi(s) and there exist positive constants K1 and K2i, such that t i=1 0 ∂V (x,t) d T |f(x) − f(y)|≤K |x − y|, |gi(x) − gi(y)|≤K i|x − y|, S 1 1 2 where V (x, t)= ∂t + 2 i=1 Gi(t) Vxx(x, t)Gi(t)+ ∂V (x,t) ∂V (x,t) Vx(x, t)F (t),Vx(x, t)=( ,..., ) Vxx(x, t) n ∂x1 ∂xn , and for all x, y ∈ R . ∂2V x =( ( ) ). Here, we need to remark that S is not the usual It is known that under Assumption II.1, the delay-free stochas- ∂xj ∂xl tic system (1) admits a unique global solution. We also assume operator associated with the stochastic differential equations that for each i, fi in the decomposition of f [see (2)] is Lipschitz but is merely a symbol; likewise SV (x, t) is just a notation. continuous. However, for stochastic system (1), we can obtain the true L 2,1 Rn × R R → R Assumption II.2: Assume that the decomposition f(x)= operator : C ( +; +) + associated with the d1 Itô diffusion i=1 fi(x) is a Lipschitz decomposition, that is, each compo- nent fi(x) satisfies |fi(x) − fi(y)|≤K1i|x − y| with K1i ≥ 0. ∂V (x, t) LV (x, t)= + Vx(x, t)f(x) Remark II.1: Assumption II.2 under Assumption II.1 is easy ∂t to verify. For example, if we consider the plant x˙ = Ax + Bu d with u = Kx, then the closed-loop system has the form x˙ = 1 T + g (x)Vxx(x, t)gi(x). (3) Ax + BKx. So we can let f(x)=Ax + BKx, which falls into 2 i the case of Assumption II.2. The corresponding issue can be i=1 considered as that if the timely control can make the system In what follows, we first consider the delay tolerance based stable, then how about for the delayed control. on the moment exponential stability. In this case, we may obtain For the purpose of stability, assume that f(0) = fi(0) = a delay bound for the delay system (2) to ensure moment 0,gj (0) = 0 for all i, j. This implies that each of the two stochas- exponential stability. However, this delay bound might be too tic systems (1) and (2) admits a trivial solution, respectively. conservative since we only know that the delay-free system Note that under the global Lipschitz assumptions above, the (1) is moment exponentially stable. To proceed, we study the moment exponential stability implies the almost sure exponen- delay tolerance issue under Lyapunov conditions imposed on tial stability [3]. So in this section, we only focus on the pth (3), which may guarantee the moment exponential stability of moment exponential stability. We consider the case p ≥ 2.For delay-free systems. In this case, we can get a relaxed delay bound p 1 the case p ∈ (0, 2), we can use the inequality (E|x(t)| ) p ≤ for system (2) to ensure moment exponential stability. 1 (E|x(t)|2) 2 to obtain the pth (p ∈ (0, 2)) moment exponential stability. A. Delay Tolerance Based on the Moment Remark II.2: The diffusion term (multiplicative noise) may Exponential Stability work positively for almost sure and pth (0 0. Consider the following

d1 d (32d+1)p−1 2 d t τ p/2 where γ =2p( K i + K ),M (τ)= + 1 i=1 1 i=1 2 2i 1 p−1 E | |2 2p+1 p p−1 d1 p p−1 d p p/2 +(2d) p0 gi(θi(xs)) ds 2 (τ d i K i + d i K iτ p0),p0 = t 1 =1 1 =1 2 i=1 [pp+1/2(p − 1)p−1]p/2. t+τ Proof: By the Itô formula, we have p ≤ C2 E(sup |x(u)| )ds t s−τ≤u≤s t d1 p p p−2 T | | | | | | i i s p γ1(t+τ−t0) x(t) = x(t0) + p [ x(s) x(s) f (ϑ (x )) ≤ E t t 3τC2 x 0 e (8) 0 i=1 p−1 p−1 d1 p p−1 p/2−1 d d where C =(2d ) τ K +(2d) τ p t − 2 1 i=1 1i 0 i=1 p 2 p−4 T 2 p p+1 − p−1 p/2 + p |x(s)| |x(s) gi(θi(xs))| ds K2i, p0 =[p /2(p 1) ] . This together with the deﬁ- t 2 i=1 0 nition of D(xt) gives d p p E|D(xt+τ )| = E(sup |x(t + τ + u) − x(t + τ)| ) 1 p−2 2 −τ≤u≤ + |x(s)| |gi(θi(xs))| ]ds + M(t) (6) 0 2 i=1 ≤ 2p−1E|x(t + τ) − x(t)|p d t p−2 T where M(t)= mi(t), mi(t)= |x(s)| x(s) p−1 p i=1 t0 +2 E(sup |x(t + u) − x(t)| ) t p E | | 0≤u≤τ gi(θi(xs))dwi(s).LetHt0 = (supt0≤u≤t x(u) ). Then, α β from Assumptions II.1–II.3 and the inequality x y ≤ p γ1(t+τ−t0) ≤ M (τ)E xt e . α α+β β α+β 1 0 α β x + α β y for x, y, α, β > 0,wehave + + That is, the desired assertion (5) follows. t p Ht ≤ E|x(t0)| + E(sup M(s)) In this lemma, we obtain the moment estimate of 0 p t0≤s≤t E | | (supt0−τ≤u≤t x(u) ) with the exponent γ1, and then based E| |p t d1 on this estimate, we get estimate (5) of D(xt+τ ) . Here, p p−1 E| | + p K iE(|x(s)| |ϑi(xs)|) we remark that the exponent γ1 for D(xt+τ ) can be es- 1 { } t0 i timated more accurately if the functionals ϑi,θi i=1,2 sat- =1 E| |p ∨ E| |p ≤ E| |p ∈ isfy ϑi(ϕ) θi(ϕ) sup−τ≤u≤0 ϕ(u) for all ϕ d p n − − R i − 2 p 1 p−2 2 LF0 (Ω; C([ τ,0]; ))(for example θ (ϕ)=ϕ( τ)). This is + K E(|x(s)| |θi(xs)| ) ds 2i 2 summarized in the following lemma. i=1 Lemma II.2: Let Assumptions II.1 and II.2 hold, p n d ∈ − R t ξ LF0 (Ω; C([ τ,0]; )), write x(t)=x(t; t0,ξ), and ≤ E| |p s E E| |p ∨ E| |p ≤ E| |p x(t0) + C1 Ht0−τ ds + (sup mi(s)) assume ϑi(ϕ) θi(ϕ) sup−τ≤u≤0 ϕ(u) for t t ≤s≤t p n 0 i=1 0 ∈ − R all ϕ LF0 (Ω; C([ τ,0]; )). Then we have the following (7) E| |p ≤ E p γ2(t−t0) E| |p ≤ estimates x(t) 2 xt0 e , D(xt+τ ) p γ t τ−t d1 d1 d E 2( + 0) 1 2 − M1(τ) xt0 e , where γ2 = p( i K1i + where C1 = p( i=1 K1i + 2 i=1 K2i(p 1)).Bythe =1 p−1 d 2 Burkholder–Davis–Gundy inequality, we have 2 i=1 K2i). Proof: From (6), we can obtain that E(sup mi(s)) t0≤s≤t t E| |p ≤ E| |p E| |p / x(t) x(t0) + C3 x(s) ds √ t 1 2 t 2p−4 T 2 0 ≤ p4 2 |x(s)| |x(s) gi(θi(xs))| ds d1 t0 t p i E| i s | t 1/2 + K1 θ (x ) ds √ t ≤ 2 E | |p | |p−2| |2 i=1 0 4 2pK2i sup x(u) x(s) θi(xs) ds t0≤u≤t t d 0 p − 1 t t 2 E| |p + K2i θ2(xs) ds 1 p 2 2 s ≤ E | | 2 t0 0.5 (sup x(u) )+16p dK2i Ht0−τ ds. i=1 d t0≤u≤t t 0 t t ≤ E| |p ≤ E| |p E| |p Substituting this into (7) yields Ht0 2 x(t0) + x(t0) + C4 sup x(u) ds t t −τ≤u≤s t s t t0 t 0 0 γ1 t Ht −τ ds. Note that Ht −τ ≤ Ht −τ + Ht ≤ 0 0 0 0 0 d d p t s − 1 1 2 − E t where C3 =(p 1)( i=1 K1i + i=1 K2i(p 2)) and 3 x 0 + γ1 t0 Ht0−τ ds. The Gronwall inequality yields 2 d1 p−1 d 2 p p γ1(t−t0) E(sup |x(u)| ) ≤ 3E xt e . Hence, the C4 = p( i=1 K1i + 2 i=1 K2i). Therefore, the Gronwall t0−τ≤u≤t 0 desired assertion (4) follows. By the Hölder inequality and the inequality leads to the desired assertion. Burkholder–Davis–Gundy inequality, we get The following lemma produces the estimate of the error − p e(t):=x(t) y(t). E(sup |x(t + u) − x(t)| ) Lemma II.3: Let Assumptions II.1, II.2, and II.3 hold, ξ ∈ ≤u≤τ p 0 L (Ω; C([−τ,0]; Rn)), and write x(t)=x(t; t ,ξ). Then, we F0 0 d1 t+τ have ≤ p−1 p−1 p E| |p (2d1) τ K1i ϑi(xs) ds E| |p ≤ − E p t e(t) J(τ,t t0) xt0 i=1

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2608 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

d d ∗ 1 − 2 M3(z−τ) Proof: We first prove that τ is well defined. For any fixed where J(τ,z)=( i=1 K1i +2(p 1) i=1 K2i) e M ∈ 1 γ1z − γ1τ ε (0, 1), define γ1 [e e ] with M1(τ) and γ1 defined in Lemma II.1, d d p−1 γ1(τ+h) γ1τ − 2 2 − 1 Q(τ):=2 M1(τ)e + εe M3 =2(p 1) i=1 K2i +(2p 1) i=1 K1i. d 1 − p−1 Proof: It is easy to see that de(t)= i=1[fi(ϑi(xt))) +4 J(τ,τ + h))) − 1. d − ε fi(y(t))]dt + i=1[gi(θi(xt)) gi(y(t))]dw(t). Apply the Itô log Note that Q(− ) > 0, limτ→0 = ε − 1 < 0, and Q (τ) > 0. formula and Assumptions II.1 and II.2 yields γ1 Hence, there is a unique root τ(ε) > 0 such that Q(τ ∗)=0, and ∗ d1 t τ =supε∈(0,1) τ(ε) is well defined. p p−1 ∗ E|e(t)| ≤ p K1i E(|e(s)| |ϑi(xs) − y(s)|)ds Let ε0 = arc supε∈(0,1) τ(ε), h = h(ε0), and τ<τ. Write t τ i=1 0+ x(t; t0,ξ):=x(t) for all t ≥ t0 and y(t0 + τ + h; t0 + τ,x(t0 + τ)) = y(t0 + τ + h). Bear in mind that − d p(p 1) 2 E| |p ≤ E| |p −γh + K2i y(t0 + τ + h) M x(t0 + τ) e . 2 i p =1 This together with Lemma II.1 implies E|y(t0 + τ + h)| ≤ p γ1τ−γh t 3ME ξ e . It follows from Lemma II.3 that p−2 2 × E(|e(s)| |θi(xs) − y(s)| )ds p p−1 p E|x(t0 + τ + h)| ≤ 2 E|x(t0 + τ + h) − y(t0 + τ + h)| t0+τ p−1 p d1 d t +2 E|y(t0 + τ + h)| 2 p ≤ i − E| | p K1 +(p 1) K2i e(s) ds p−1 γ1τ−γh p t τ ≤ 2 (3Me + J(τ,τ + h))E ξ . i=1 i=1 0+ Hence d1 t p p−1 p p−1 E xt τ h ≤ 2 E|D(xt τ h| + p K1i E(|e(s)| |ϑi(xs) − x(s)|)ds 0+ + 0+ + t τ i=1 0+ p−1 p +2 E|x(t0 + τ + h)| d ≤ E p − 2 J0(τ) ξ (9) + p(p 1) K2i p−1 γ1(τ+h) p−1 γ1τ−γh i=1 where J0(τ)=2 M1(τ)e +4 (3Me + t J(τ,τ + h))). From the definitions of τ and h, J0(τ) < 1. × E | |p−2| − |2 log J0(τ) ( e(s) θi(xs) x(s) )ds Denote γ0 = − τ h . Then, it follows from (9) that t τ + 0+ p −γ0(τ+h) p E xt τ h ≤ e E ξ . Then similar to the 0+ + t d1 derivation in [27], we can obtain that for k =1, 2,..., ≤ E| |p M3 e(s) ds + K1i E p ≤ −γ0k(τ+h)E p t τ xt0+k(τ+h) e ξ . 0+ i=1 t This implies the pth moment exponential stability of the delay p system (2). × E|ϑi(xs) − x(s)| ds t0+τ Theorem II.1 indicates that the moment exponentially stable stochastic system can be tolerant with a small delay with a d t ∗ 2 p bound τ . It is not required for us to know the exact stability +2(p − 1) K i E|θi(xs) − x(s)| ds. d 2 conditions on the coefficients f(x) and {gi(x)}i .Themore t0+τ =1 i=1 information about the systems we have, the more accurate results we would get. So it is a natural question that if we know some Applying the Gronwall inequality and Lemma II.1 lead to { }d additional information about the coefficients f, gi(x) i=1, and their decomposition, can we improve the delay bound obtained d1 t p M3(t−t0−τ) p in Theorem II.1? For example, all the information about the E|e(t)| ≤ e K1i E|ϑi(xs) − x(s)| ds t τ coefficients is available for the following linear system i=1 0+ dy(t)=−μy(t)dt + σy(t)dw(t),μ,σ>0. (10) d t − 2 E| − |p Most importantly for the pth moment stability analysis, one can +2(p 1) K2i θi(xs) x(s) ds p t τ resort to the Lyapunov function like V (y)=|y| and derive i=1 0+ − p p 1 2 p ≤ J(τ,t − t )E xt . LV (y)=−p(μ − σ )|y| (11) 0 0 2 which is a key in moment stability analysis and produces p p −p μ− p−1 σ2 t E|y(t)| = |y(0)| e ( 2 ) . That is, equation (10) is pth Theorem II.1: Let Assumptions II.1, II.2, and II.3 hold. 2 2 Assume that the original system (1) is pth moment exponentially moment exponentially stable for p<(2μ + σ )/σ . We hope p p −γt to find a delay bound τ better than that in Theorem II.1, such stable with E|y(t)| ≤ ME|y0| e for M,γ > 0. Then there is a positive number τ ∗, such that the delay system that the following delay system is still pth moment exponentially (2) is pth moment exponentially stable for any τ<τ∗. stable ∗ ∗ { − In fact, τ can be determined using τ =supε∈(0,1) τ> dx(t)= μθ1(xt)dt + σθ2(xt)dw(t),x0 = ξ (12) p−1 γ1(τ+h) γ1τ p−1 0|2 M1(τ)e + εe +4 J(τ,τ + h))) − 1= where w(t) is a scalar Brownian motion. That is the attention of } ε ∈ 0 , where h = h(ε)=log(3·4p−1M ) for ε (0, 1). the following subsection.

B. Delay Tolerance Based on Lyapunov Conditions Then, substituting (15) and (16) into (14) and taking expecta- tions, we get from condition 2) In this section, we assume the Lyapunov stability condition t like (11) as a precondition and examine the delay tolerance. The γt γs e EV (x(t)) ≤ EV (x(0)) + (γ − λ) e EV (x(s))ds following Lyapunov stability result is classical (see [3]). 0 ∈ 2 Rn R Theorem II.2: If there exist a function V (x) C ( ; +) t d1 d and two positive constants c1 and c2, such that γs p p + e E Θ1i(s)+ Θ2i(s) dt 1) c1|x| ≤ V (x) ≤ c2|x| L ≤−λ λ 0 i=1 i=1 2) V (x) V (x) for some ﬁxed > 0. (17) Then stochastic system (1) is pth moment exponentially where Θ1i(s)=Vx(x(s))[fi(ϑi(xs)) − fi(x(s))] and stable. T Θ2i(s)=0.5gi(θi(xs)) Vxx(x(s))[g(θi(xs)) − gi(x(s))] + To examine the delay tolerance under the above Lyapunov T conditions, the following integral version of the Halanay in- 0.5[gi(θi(xs)) − gi(x(s))] Vxx(x(s))gi(x(s)). It follows p−2 z ≤ 1 p − p 1 p equality [38] will be used. Then the delay tolerance under the that α(xy α ) p α(2x +(p 2)y )+ αp−1 z , for any Lyapunov function is given in Theorem II.3. x, y, z, α > 0, and we have Lemma II.4: Consider p−1 EΘ1i(s) ≤ c3K1iE(|x(s)| |ϑi(xs) − x(s)|) t ≤ −γ0(t−t0) −γ0(t−s) v(t) vt0 e f(t)+K e sup v(u)ds, p − 1 p 1 p t0 u∈[s−τ,s] ≤ c K i α E|x(s)| + E|ϑi(xs) − x(s)| 3 1 p pα ≥ where t t0, γ0 > 0,K >0, and f(t) is a nondecreasing pos- (18) itive function. If γ0 >K, then there exist positive constants γ and K, such that and − −γ(t−t0) c4 1 p p 2 p ≤ t E ≤ 2 E| | E| | v(t) v 0 Ke f(t) Θ2i(t) K2i α θi(xs) + α x(s) − γτ 2 p p where γ is the unique root of the equation γ = γ0 + Ke . Theorem II.3: Let Assumptions II.1 and II.2 hold, 1 p E| − |2 ∨ E| − |2 ≤ E| − − |2 + E|θi(xs) − x(s)| ϑi(ϕ) x θi(ϕ) x supu∈[0,τ] x ϕ( u) p pα for ϕ ∈ L (Ω; C([−τ,0]; Rn)) and x ∈ Rn. Assume that F0 − ∈ 2 Rn R c4 2 p 1 p there exist a function V (x) C ( ; +) and constants + K i α E|x(s)| ≥ { }4 2 2 p p 2, ci i=1 satisfying conditions 1), 2), and p−1 | x |≤ | | 3) V (x) c3 x 1 p | |≤ | |p−2 + E|θi(xs) − x(s)| . (19) 4) Vxx(x) c4 x . pα Then the delay system (2) is pth moment exponentially stable E| − |2 ∨E| − |2 ≤ if Note that ϑi(xs) x(s) θi(xs) x(s) supu∈[0,τ] E|x(s) − x(s − u)|2. Similarly to (8), we have that for any C5(τ) < λ (13) √ p−1 u ∈ [0,τ] p−1 p p−1 d1 p p/2 p−1 where C5(τ)=2 2 2 (τ d i K i + τ d p0 p pc1 1 =1 1 E|x(t + u) − x(t)| d p 1/2 d1 d 2 i K i) (c3 i K1i + c4 i K i), p0 = =1 2 =1 =1 2 d p+1 − p−1 p/2 1 t+u [p /2(p 1) ] . ≤ p−1 p E| |p (2d1τ) K1i ϑi(xs) ds Proof: Applying the Itô formula yields t γt i=1 de V (x(t)) d t+u p d1 p−1 p/2−1 E| |p γt γt +(2d) τ p0 K2i θi(xs) ds t = γe V (x(t))dt + e Vx(x(t))fi(ϑi(xt))dt + dM(t) i=1 i=1 p ≤ C6(τ)supE|x(u)| (20) d u∈[t−2τ,t] 1 T + gi(θi(xt)) Vxx(x(t))gi(θi(xt))dt (14) p−1 p p−1 d1 p p/2 p−1 d 2 where C6(τ)=2 (τ d i K i +τ d p0 i i=1 1 =1 1 =1 p C6(τ) d t γs K2i).Letα = p−1 . Combining (18), (19), and (20), we where M(t)= i=1 0 e Vx(x(s))gi(θi(xs)dwi(s) is a martingale with EM(t)=0because of the global Lipschitz condi- have tions. Note that 1 C6(τ) 2 EΘ1i(t) ≤ c3K1i (p − 1)α + sup E|x(u)| Vx(x(s))fi(ϑi(xs)) p α u∈[t−2τ,t] √ = Vx(x(s))f(x(s)) + Vx(x(s))[fi(ϑi(xs)) − fi(x(s))] p − 1 (15) ≤ 2 C6(τ)c3K1i sup EV (x(u)) pc1 u∈[t−2τ,t] and T and gi(θi(xs)) Vxx(x(s))gi(θi(xs)) E ≤ 1 2 − C6(τ) E| |2 T Θ2i(t) c4K2i (p 1)α + sup x(u) = gi(x(s)) Vxx(x(s))gi(x(s)) p α u∈[t−2τ,t] T √ + gi(θi(xs)) Vxx(x(s))[g(θi(xs)) − g(x(s))] − ≤ p 1 2 E 2 C6(τ)c4K2i sup V (x(u)). T pc u∈ t− τ,t +[gi(θi(xs)) − g(x(s))] Vxx(x(s))gi(x(s)). (16) 1 [ 2 ]

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2610 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

This together with (17) yields with the state space S = {1, 2 ...,m}, and Γ=[γij ]m×m is t generator of the Markov chain r(t). It is also interesting to extend γt γs e EV (x(t)) ≤ EV (x(0)) + (γ − λ) e EV (x(s))ds current results to the semi-Markov and singular stochastic sys- 0 tems in [39], which made a good contribution to extend the slide t model control from deterministic systems to the semi-Markov γs E + C5(τ) e sup V (x(u))ds, and singular stochastic versions. u∈ s− τ,s 0 [ 3 ] Remark II.4: Note that the above results do not require the (21) √ concrete form of the delay map ϑi and θi. In fact, these maps p−1 d1 d 2 i can include many types of delays. For example, the delay can where C5(τ)=2 pc1 C6(τ)(c3 i=1K1 +c4 i=1K2i). 2 λ − |x(t−τ)| Let γ = . Then the inequality (21) can be rewrit- be state-dependent like θi(xt)=x(t τ 1+|x(t−τ)|2 ), and state- −λt t −λ(t−s) ten as EV (x(t)) ≤ e EV (x(0)) + C5(τ) e independent like θi(xt)=x(t − τi(t)), where τi(t) is allowed E 0 supu∈[s−3τ,s] V (x(u))ds. Note that condition (13) implies to be not smooth. The nonsmooth delay can be used to describe λ >C (τ). Then by the integral version of Halanay inequality the delay induced by the sampled data, where the delay τi(t)= 5 − ∈ { }∞ (Lemma II.4), there exist M0 and η, such that t tk for t [tk,tk+1), tk k=1 are the sampling times. In this p 2 −ηt case, we can see that the sampled-data control problem falls in E|x(t)| ≤ M0 sup E|x(θ)| e θ∈[−τ,0] the delay tolerance issue. For deterministic systems, many works have contributed to this issue [40]. For stochastic systems, only where η is the unique root of the following equation η = −λ + ητ a few works have been achieved due to the nondifferentiability C (τ)e3 . Therefore, the desired assertion follows. 5 of the solution leading to the failure of many methods for the In Theorem II.3, we add two additional conditions 3) and 4) to deterministic systems. Mao and his coauthors [41], [42] proved get the delay tolerance. One may question whether there exists that the sampled-data control for stochastic system is applicable a Lyapunov function satisfying the two conditions. In fact, the T p/2 for sufficient small sampling period. You et al. [43] extended classical Lyapunov function V (x)=(x Px) with P>0 these results to get a better delay bound under the constant λ p/2 | |p/2 satisfies these conditions with c1 = min(P ) and ci = P period sampling. The results obtained in this article remove for i =2, 3, 4, and condition 2) in Theorem II.2 has the form the assumption of constant period sampling and introduce some d explicit conditions on time-varying sampling period. T p/2−1 T 1 T LV (x)=p(x Px) x f(x)+ gi(x) Pgi(x) For deterministic delay systems, the affine Bessel–Legendre 2 i=1 inequality [44] and Wirtinger’s inequality [45] can be used to d get the stability analysis. However, these methods cannot be p T p/2−1 T 2 extended to the stochastic version since these methods involve + p − 1 (x Px) |x Pgi(x)| 2 the derivatives of the state and the state of stochastic system is i=1 not differentiable. Note that for stochastic systems, there seems ≤−λ(xT Px)p/2. (22) to be no evidence that the Lyapunov functional is better than Moreover, we have the following corollary based on (22) for Lyapunov function. Moreover, one cannot design a Lyapunov p =2. functional without knowledge of the form of the delays. So the Corollary II.1: Assume that there exists a positive def- above analysis takes the Lyapunov function rather than Lya- inite matrix P , such that (22) holds for p =2. Then punov functional for the stability analysis for the delay system (2). Here, the Lyapunov function is from the stability analysis delay system (2) is mean square exponentially stable of the delay-free systems. 2 d1 2 d 2 d1 d if 2(τ d1 i=1 K1i + τdc2 i=1 K2i)( i=1 K1i + i=1 Note that all the delay tolerance results are based on the 2 |P | global Lipschitz conditions. One may question whether the K i) < λ. 2 λmin(P ) global Lipschitz assumption can be relaxed. At this point, we Now, for the delay tolerance concerning (10) and (12), we can do not have a way to relax this assumption for the case above obtain from Theorem II.3 that if τ satisfies √ without the knowledge of the concrete form and property of the p − 1 p − 1 2 2τ pμp +4τ p/2σp(μ + σ2)

0, there exist positive constants Hj , such that case with Markovian switching within a ﬁnite number of states. In fact, for the hybrid case, we only need to change the Lyapunov |f(x) − f(y)|∨|gi(x) − gi(y)|≤Hj |x − y| function V (x) to the switched Lyapunov function V (x, i).Inthis ∈ Rn | |∨| | m for all x, y with x y

d · Proof: Note that the delay system (23) admits a unique local under the decomposition f(x)= i=0 fi(x) with f0( ) being non-Lipschitz continuous, we consider the following delay sys- solution on t ∈ [−τ,ρe) under Assumption III.3, where ρe is the tem explosion time for the solution x(t). To show that this solution ∞ d1 is in fact global, we need only prove ρe = a.s. Let z(t)= d1 t − x(t)+ fi(x(s))ds dx(t)= f0(x(t)) + fi(x(t τi(t))) dt i=1 t−τi(t) . Then we have i=1 d1 d dz(t)= f(x(t)) + τ˙i(t)fi(x(t − τi(t))) dt + gi(x(t − τi(t)))dwi(t) (23) i=1 i=1 d { } Rn → Rn where τi(t) are time-varying delays and fi,gi : + gi(x(t − τi(t)))dwi(t) are continuous functions. We make the following assumptions i=1 on these delays and functions. d Assumption III.2: Each τi(t) ∈ [0,τ] is differentiable on =: F (t)dt + Gi(t)dwi(t). (27) [tk,tk+1) for k =1, 2,... with the derivative |τ˙i(t)|≤κ<1 for certain κ>0. i=1 Assumption III.3: Assume that all the coefficients in (23) are Hence, z(t) can be considered as an Itô process. We first prove local Lipschitz continuous, and there exist a matrix P>0 and that the explosion times for z(t) and x(t) are equal. Note that { } { } d1 some constants μ0,K1, K1i > 0, p>1, and β, β1, σ1i , K iτ<1 and { }≥ i=1 1 σ2i 0, such that d1 t T ≤− | |2 − | |p+1 2x Pf(x) μ0 x β x |x(t)|≤|z(t)| + |fi(x(s))|ds t−τ t T 2 p+1 i=1 i( ) gi(x) Pgi(x) ≤ σ1i|x| + σ2i|x| ,i=1,...,d p d1 |f(x)|≤K1|x| + β1|x| ≤|z(t)| + τ K1i sup |x(s)| t−τ≤s≤t |fi(x)|≤K1i|x|,i=1,...,d1. i=1 Remark III.1: Note that the general diffusion without the which implies nondegenerate property may not contribute to the stability. In 1 p | |≤ | |≤ | | − | | +1 x(t) sup x(s) d sup z(s) + C0 this case, high-order term producing β x in the drift is ≤s≤t − 1 ≤s≤t 0 1 τ i=1 K1i 0 required to suppress the growth induced by the high-order term d1 τ K1i in the diffusion. Hence, high-order terms in the drift have to i=1 | | where C0 = −τ d1 K sup−τ≤s≤0 x(s) .Ifwehaveanex- contribute positively to the stability. In fact, without such con- 1 i=1 1i dition, the corresponding system may be unstable. To illustrate, plosion time for z(t), denoted by ρze, then we must have consider the deterministic system dx(t)=(−x(t)+x3(t))dt, ρze ≤ ρe. Note also that T − | |2 | |4 which satisfies 2x f(x)= 2 x +2x . Then it is easy to d1 see that this system cannot tend to the trivial solution. Moreover, |z(t)|≤|x(t)| + τ K1i sup |x(s)| t−τ≤s≤t the similar conditions in Assumption III.3 were frequently used i=1 (see, for example [21]). d λ − d λ − d 1 Let = μ0 i=1 σ1i > 0 and 1 = β i=1 σ2i > T ≤ 1+τ K1i sup |x(s)|. 0. Note that Assumption III.3 implies 2x Pf(x)+ t−τ≤s≤t i=1 d T ≤−λ| |2 − λ | |1+p i=1 gi (x)Pgi(x) x 1 x . It follows from [3] ≥ | | that the trivial solution of the delay-free system is almost This implies ρze ρe. Hence, ρze = ρe. For each k> z(0) , λ λ define the stopping time ρk =inf{t ∈ [0,ρe):|z(t)|≥k}. surely and mean square exponentially stable if > 0, 1 > 0. →∞ → ≤ Before giving the stability analysis of the delay system, we first Clearly, ρk is increasing as k and ρk ρ∞ ρze = ρe a.s. If we can show ρ∞ = ∞ a.s., then ρe = ∞, which implies examine the regularity of the delay system (23). − Theorem III.1: Let Assumptions III.1, III.2, and III.3 hold that the solution x(t) is global. Define θi(xt)=x(t τi(t)). Consider the Itô process (27) and introduce a Lyapunov function with λ > 0 and λ1 > 0.If T d1 V1(x)=x Px. K1iτ<1 (24) Applying the Itô formula, we have i=1 dV1(z(t)) = SV1(z(t),t)dt + dM(t) (28) d1 d t T λ −| | 2 2 where M(t)= i 2z (s)Pgi(θi(xs))dwi(s) and J10 := P K1i + d1K1 τ + H(κ)κ>0 (25) =1 0 i=1 SV (z(t),t) 1 d1 d κ d1 J := λ − 2|P |β τ K i + σ i > 0 (26) T 20 1 1 1 1 − κ 2 =2z(t) P f(x(t)) + τ˙i(t)fi(θi(xt)) i=1 i=1 i=1 d d where H(κ)=|P |(d 1 K2 + d + 1+d1τ 1 K2 )+ 1 i=1 1i 1 1−κ i=1 1i d 1 d T T 1−κ i=1 σ1i. Then delay system (23) admits a unique global + gj (θj(xt))Pgj (θj(xt)) solution. j=1

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2612 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

d | | d1 2 | | 2 T T T where a = P (1 + d1κ) i=1 K1i, b = P d1(κ + τK1 ), j t j t =2x(t) Pf(x(t)) + gj (θ (x ))Pgj (θ (x )) and c = |P |(1 + d1τ). Deﬁne V (z(t),t)=V1(z(t)) + V2(t), j=1 where d1 d t T 1 T T +2x(t) P τ˙i(t)fi(θi(xt)) V2(t)= gj (x(s))Pgj (x(s))ds 1 − κ t−τ t i=1 j=1 i( ) d1 t 0 t T 2 +2 fi(x(s)) dsP f(x(t)) + a |x(θ)| dθds t−τ t −τ t s i=1 i( ) + d d1 t d 1 t T cκ 2 | |2 +2 fi(x(s)) dsP τ˙i(t)fi(θi(xt)) + − K1i x(s) ds t−τ t 1 κ t−τi(t) i=1 i( ) i=1 i=1 T d =: 2x(t) Pf(x(t)) + J1(t)+J2(t)+J3(t)+J4(t) | | 1 0 t 2 P β1 p+1 (29) + K1i |x(θ)| dθds. p +1 −τ t s T i=1 + where t ∈ [ti,ti+1), i =0, 1, 2 ... By the inequality 2x y ≤ 2 1 2 n ε|x| + ε |y| , ε>0, x ∈ R , we can obtain Note that V2(t) is differentiable and z(t) is an Itô process. Then, applying the Itô formula again yields that for t ∈ [ti,ti ) d1 +1 ≤| | | |2 2 | − |2 S J2(t) P τ˙i(t) x(t) + K1i x(t τi(t)) dV (z(t),t)= V (z(t),t)dt + dM(t) (30) i =1 where d1 ≤| | | |2 | | 2 | − |2 SV (z(t),t) P κd1 x(t) + P κ K1i x(t τi(t)) . i=1 S ˙ p q = V1(z(t),t)+V2(t) By the inequality xpyq ≤ xp+q + yp+q,forp, q, x, y > p+q p+q d 0,wehave T T T ≤ 2x(t) Pf(x(t)) + gj (x(t))Pgj (x(t)) d1 t ≤ | | | | | | j=1 J3(t) 2 P f(x(t)) fi(x(s)) ds t−τ i=1 d1 d κ 2 2 | |2 d1 t + aτ + b + c K1i + σ1i x(t) p 1 − κ i=1 i=1 ≤ 2|P | (K1|x(t)| + β1|x(t)| ) |fi(x(s))|ds t−τ i=1 d1 d κ d | | 2 | |p+1 1 t + 2 P β1τ K1i + − σ2i x(t) ≤| | 2 | |2 | | 2 | |2 i 1 κ i P d1K1 τ x(t) + P K1i x(s) ds =1 =1 t−τ i=1 2 p+1 ≤ h1(τ)|x(t)| + h2(τ)|x(t)| (31) d1 | | p | |p+1 −λ κ d1 2 d +2P β1 K1i( τ x(t) and h1(τ)= +[aτ + b + 1−κ (c i=1 K1i + i=1 σ1i)] p +1 d d i=1 −λ | | 1 κ and h2(τ)= 1 +(2P β1τ i=1 K1i + 1−κ i=1 σ2i). t E | | 1 p+1 Let U(t)= V (z(t),t). Then for any k> z(0) ,wehave + |x(s)| ds) ∈ p +1 from (30) that for t [ti,ti+1] t−τ and t∧ρk ∧ k i ∧ k E S d1 t U(t ρ )=U(t ρ )+ V (z(s),s)ds ≤| | 2 | |2 ti∧ρk J4(t) P κd1 K1i x(s) ds t−τ t∧ρk i=1 2 ≤ U(ti ∧ ρk) − h1(τ)E |x(s)| ds d1 ti∧ρk 2 2 | | | − i | + P d1κτ K1i x(t τ (t)) . t∧ρk i=1 p+1 − h2(τ)E |x(s)| ds Substituting the three inequalities above into (29) yields ti∧ρk SV (z(t),t) t∧ρk 1 2 ≤ U(ti−1 ∧ ρk) − h1(τ)E |x(s)| ds d 1 ti−1∧ρk ≤ T 2 | − |2 2x(t) Pf(x(t)) + cκ K1i x(t τi(t)) t∧ρk i p+1 =1 − h2(τ)E |x(s)| ds ≤ ··· t ∧ρ d1 i−1 k 2 pτ p+1 + J (t)+b|x(t)| +2|P |β K i|x(t)| t∧ρk 1 1 p +1 1 ≤ − E | |2 i=1 U(0) h1(τ) x(s) ds 0 d1 t t | | t∧ρ 2 P β1 p+1 2 k + K1i |x(s)| ds + a |x(s)| ds − E | |p+1 p +1 t−τ t−τ h2(τ) x(s) ds. i=1 0

This together with the deﬁnition of V (z(t),t) implies d1 2 where C8 =(d1 +1), C9 =( d1 +1) i=1 K1iτ + 1 d cκ d 2 EV1(z(t ∧ ρk)) ≤ U(t ∧ ρk) ≤ U(0). aτ + |P | σ i ∨ σ i + K , C = 1−κ j=1 1 2 1−κ i=1 1i 10 Note that 1 | | d ∨ | | 1 d1 1−κ P j=1 σ1i σ2i +2P β1 p+1 i=1 K1iτ. Applying E ∧ ≥ E ∧ γt V1(z(t ρk))) (V1(z(t ρk))1{ρk≤t}) the Itô formula to e V1(z(t)) and using V (z(t),t)= V (z(t)) + V (t), we have from (31) and (32) that for any 2 1 2 ≥ k P{ρk ≤ t}. γ>0, t ∈ [ti,ti+1] 2 Hence, k P{ρk ≤ t}≤U(0). Then, for any t>0, γt P{ ≤ } e V (z(t),t) limk→∞ ρk t =0, which together with the arbitrariness of t implies ρ∞ = ∞ a.s. Therefore, the solution x(t) is t γti γs global. = e V (z(ti),ti)+ e γV (z(s),s)ds Theorem III.1 shows that if the delay bound τ and its deriva- ti t t tive bound κ satisfy conditions (24), (25), and (26), then the γs γs solution of the delay system is still global. In other words, for + e SV (z(s),s)ds + e dM(s) ti ti the nonglobal Lipschitz stochastic system, small delays do not affect the existence of the global solution. In what follows, we t ≤ γti γs| |2 also show that small delays also do not affect the mean square and e V (z(ti),ti)+(C8γ + h1(τ)) e x(s) ds ti almost sure exponential stability of the solutions. The following lemma is important in the almost sure convergence analysis [3]. t γs| |p+1 Lemma III.1. Semimartingale Convergence Theorem: Let + h2(τ) e x(s) ds ti A1(t) and A2(t) be two Ft-adapted increasing processes on ≥ t s t 0 with A1(0) = A2(0) = 0 a.s. Let M(t) be a real-valued γs 2 + γC9 e |x(u)| duds local martingale with M(0) = 0 a.s. and ζ be a nonnegative F0- ti s−τ measurable random variable. Assume that Y (t) is nonnegative t s t and γs p+1 γs + γC10 e |x(u)| duds + e dM(s) Y (t)=ζ + A1(t) − A2(t)+M(t),t≥ 0. ti s−τ ti

If limt→∞ A1(t) < ∞ a.s., then for almost all ω ∈ Ω, ≤··· lim Y (t) < ∞ and lim A2(t) < ∞. t t→∞ t→∞ γs 2 ≤ V (z(0), 0) + (h1(τ)+γC8) e |x(s)| ds Theorem III.2: Let Assumptions III.1, III.2, and III.3 hold 0 with λ > 0 and λ1 > 0. If conditions (24), (25), and (26) hold, t s γs 2 then the trivial solution for the delay system (23) is almost surely + γC9 e |x(u)| duds and mean square exponentially stable, that is, there is a constant 0 s−τ γ>0, such that t t γs γs| |p+1 γs lim sup e |x(s)|2 < ∞ a.s. + h2(τ) e x(s) ds + e dM(s) t→∞ 0 0 and t s γs | |p+1 γs 2 + γC10 e x(u) duds (33) lim sup e E|x(s)| < ∞. 0 s−τ t→∞ Proof: We ﬁrst show that the trivial solution is almost surely where M(t) is deﬁned in (28). Note that for q =2, t s exponentially stable. Note that p +1, eγs |x(u)|qduds ≤ τeγτ 0 |x(u)|qdu + 0 s−τ −τ V (z(t),t) γτ t | |q τe 0 x(u) du. Hence, from (33), we get d1 t t t ≤ | |2 2 | |2 γt γs 2 γs (d1 +1)x(t) +(d1 +1) K1iτ x(u) du e V1(z(t)) ≤ C4 + J1(γ) e |x(s)| ds + e dM(s) t−τ i=1 0 0 t d | | t t γs| |p+1 P ∨ | |2 | |p+1 + J2(γ) e x(s) ds (34) + σ1i σ2i x(s) ds + x(s) ds 0 1 − κ t−τ t−τ j=1 where C = V (z(0), 0) + γC τeγτ 0 |x(u)|2du + γC t d t 11 9 −τ 10 0 | |2 cκ 2 | |2 τeγτ |x(u)|2du, J (γ)=h (τ)+γC + γC τeγτ + aτ x(s) ds + K1i x(s) ds −τ 1 1 8 9 t−τ 1 − κ t−τ γτ i=1 and J2(γ)=h2(τ)+γC10τe . It can be observed

that Ji(0) = hi(τ) < 0 and Ji(γ) > 0 for any γ>0, d1 t λ λ1 1 p+1 i =1, 2. Note that J1( C ) > 0 and J2( C τ ) > 0. Hence, +2|P |β1 K1iτ |x(s)| ds 8 10 p +1 t−τ there are two positive constants γ1 and γ2, such that i=1 ∗ Ji(γi)=0 and Ji(γ) < 0, i =1, 2,forγ<γ := γ ∧ γ . t t 1 2 t γs ∗ 2 2 p+1 | | | | | | Let Y (t)=C11 + 0 e dM(s). Note that for γ<γ, = C8 x(t) + C9 x(u) du + C10 x(u) du t t t−τ t−τ − γs| |2 − γs| |p+1 J1(γ) 0 e x(s) ds J2(γ) 0 e x(s) ds < Y (t). (32) Therefore, the semimartingale convergence theorem yields

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2614 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

∞ 2μ−σ2 lim supt→∞ Y (t) < ,a.s.and then 4σ4+2μ2( )2−2σ2 μ2+σ2 μ−σ2 t 2 μ2 < μ2 . That is, Corollary III.2 can − γs| |2 − 2 √ J1(γ) e x(s) ds J2(γ) provide a large delay bound for the case μ> 2. But for some 0 t special cases, we can obtain a tight delay bound. To this end, we γs p p =2 × e |x(s)| +1ds < ∞, a.s. consider in view of characteristic equation. By Itô formula 0 for (12), we have γs| |2 ∞ dE|x(t)|2 It is easy to see lim sups→∞ e x(s) < a.s., and then the = −2μE[x(t)x(t − τ )]ds + σ2E|x(t − τ )|2. almost sure exponential stability follows. dt 1 2 It remains to establish the mean square exponential stability. Let Z(t, s)=E[x(t)x(s)]. Then we have Let ρk be deﬁned in Theorem III.1. Then from (34), we have ˙ − − 2 − − that for γ<γ∗ Z(t, t)= 2μZ(t, t τ1)+σ Z(t τ2,t τ2). (36)

γ(t∧ρk) To examine the stability of the equation above, we assume that Ee V (z(t ∧ ρk),t∧ ρk) its solution has the form t∧ρk γt γs γs 2 Z(t, s)=e e . (37) ≤ EC11 + J1(γ)E e |x(s)| ds 0 We now examine the conditions on γ, such that (37) is indeed t∧ρk a solution to (36). In fact, substituting (37) into (36) admits the γs p+1 + J2(γ)E e |x(s)| ds. following characteristic equation 0 2γ = −2μe−γτ1 + σ2e−2γτ2 . (38) Noting that ρ∞ = ∞ a.s. was proved in Theorem III.1, applying − t γsE| |2 ≤ E ∀ That is, the delay equation (12) with θi(xt)=x(t − τi) is Fatou’s lemma yields J1(γ) 0 e x(s) ds C11, t> 0. This produces the mean square exponential stability. mean square exponentially stable if and only if all the infinitely Remark III.2: In Theorem III.1, the delay term in the drift many characteristic roots of the characteristic equation (38) have is assumed to be Lipschitz. To date, we have not found a way negative real parts. Especially, a) if τ1 =2τ2, by solving the to handle non-Lipschitz delay terms like dx(t)=(−μx(t) − characteristic equation (38), we know that the trivial solution 3 of (10) is mean square exponentially stable if and only if x(t − τ) )dt + σx(t)dw(t). The previous work [46] tells us that 2 − σ π the non-Lipschitz delay term may not affect the boundedness 0 < (μ 2 )τ1 < 2 . That is, the mean square stable stochastic 0.5π of the solution in any finite time. But it is still unclear how system (10) can tolerate the delay τ1 < σ2 .b)ifτ1 =0, then μ− 2 the non-Lipschitz delay term affects the stability property for solving (38) yields that the mean square stable stochastic system stochastic systems. (10) can tolerate any bounded delay τ2 in the diffusion. From Theorem III.2, we can obtain the following delay toler- The delay that the linear system (10) can tolerate established ance when the non-Lipschitz term f0(x) vanishes. above is better than that in Theorem II.1. But the characteristic Corollary III.1: Let Assumptions III.1, III.2, and III.3 hold equation above cannot be used for the nonlinear system and the λ with > 0 and β = β1 = σ2i =0. If conditions (24) and (25) general pth moment stability. So it is still an important direction { } hold, then delay system (23) is stable for small delays τi(t) to find a refined delay tolerance criterion in the future. with small derivative. In addition, delay system (23) is also Remark III.4: Razumikhin theorem is another important almost surely and mean square exponentially stable. technique in examining the stability of stochastic delay systems Especially, if the delay is fixed, and the delay system has the and was developed in [15]–[17] and [48] for different type following form stochastic systems. In this article, delay tolerance criteria are d discussed based on Lyapunov function and functional because dx(t)=f(x(t − τ))dt + gi(x(t − τ))dwi(t) (35) these issues are a class of delay-term dominated and delay- i=1 dependent stability, which cannot be solved by Razumikhin we have the following corollary. methods directly. The skills developed in this article would be Corollary III.2: Assume that there exist a function V (x) ∈ helpful for us to extend output feedback control studied in [49] 2 Rn R { }4 to delay output feedback control. C ( ; +) and constants p =2, ci i=1 satisfying conditions ≡ | | 2 1), 2), and Vxx(x) P for certain matrix P>0.If2 P K1 τ< λ, then the delay system (35) is almost surely and mean square IV. EXTENSIONS TO CONSENSUS AND TRACKING CONTROL exponentially stable. OF MULTIAGENT SYSTEMS UNDER MULTIPLICATIVE NOISES Remark III.3: Li and Mao [47] recently contributed an impor- AND NONUNIFORM DELAYS tant work on delay feedback stabilization of hybrid stochastic system. If we consider the degenerate system like dx(t)= In this section, we aim to use the delay tolerance idea to study u(t)dt + x(t)dw(t) with u(t)=−x(t − τ), our delay tolerance the multiagent consensus and tracking under multiplicative results provide a large delay bound since the stability rule (Rule noises and time-varying nonuniform delays. This is an important 3.5 in [47]) requires τ<1/4, but our Corollary III.2 only extension from the case with the uniform fixed time delays in our requires τ<1/2. previous works [30], [50] to the case with time-varying uninform Now, we continue to consider the stability tolerance of time delays. We consider N agents with the information flow structures (10) and (12) with θi(xt)=x(t − τi) based on Theorem III.1. In fact, from Theorem III.1, we can easily obtain that the among different agents being modeled as an undirected graph G = {V, E, A}, where V = {1, 2,...,N} is the set of nodes with mean square stable system (10) can tolerate a fixed delay E 2μ−σ2 i representing the ith agent, denotes the set of undirected edges, τ< 2 , such that the delay system (12) is still mean square N×N 2μ √ and A =[aij ]∈R is the adjacency matrix of G with element exponentially stable. We can see that if μ> 2, then aij =1or 0 indicating whether or not there is an information

N×N flow from agent j to agent i directly. The Laplacian matrix of G [lkij] ∈ R with D−A D is defined as L = , where = diag(deg1,...,degN ) and ⎧ N G ⎨−aij,j= i, τk(·)=τij (·) degi = j=1 aij , i =1,...,N.Note that is undirected. We lkji = 0,j= i, τk(·) = τij(·) denote Q =[φ2,...,φN ], where φi is the unit eigenvector of L ⎩ T T − lkip,j= i. associated with the eigenvalue λi = λi(L), that is, φi L = λiφi , p= i 1 φi =1, i =2,...,N. Then Q =(√ 1N , Q) is an orthogo- r N It can be observed that Lk is symmetric and k=1 Lk = L nal matrix. Let Λ:=diag(λ2, λ3,...,λN ). since τij(·)=τji(·). Moreover, we can see that each row sum T We consider a network of agents with the first-order dynamics of the matrix Lk is zero, that is, 1N Lk =0for all k =1,...,N. −1 x˙ i(t)=ui(t),i=1, 2,...,N, t∈ R+ (39) Define δ(t)=[(IN − JN ) ⊗ In]x(t)δ(t)=(Q ⊗ In)δ(t)= ∈ Rn ∈ T T T T T T ∈ where xi(t) denotes the state of i’s agent and ui(t) [δ1 (t),...,δN (t)] , and δ(t)=[δ2 (t),...,δN (t)] , δi(t) n R is the corresponding input control to be designed. Denote Rn −1 √1 1T ⊗ T T T T T T . Then, by the definition of Q ,wehaveδ1(t)= N ( N x(t)=[x1 (t),...,xN (t)] and u(t)=[u1 (t),...,uN (t)] . 1 T In)δ(t)= √ (1 (IN − JN ) ⊗ In)y(t)=0and We consider that the measurements of relative states by N N agent i have the following form zji(t)=Δxji(t − τji(t)) + r − ∈ − gji(Δxji(t τji(t)))ξji(t),j Ni, where Δxji(t)=xj(t) dδ(t)= − (Λk ⊗ K)δ(t − τk(t))dt ∈ xi(t), τji(t) [0,τ] is the time-varying delay depending the k=1 communication channel and satisfying τji(t)=τij (t), Ni de- N notes the neighbors of the ith agent, ξji(t) ∈ R denotes the − measurement noises and gji is the noise intensity. We need the + Gij (t τij (t))dwji(t) (42) following assumptions. i,j=1 Assumption IV.1: The noise process ξji(t) ∈ R satisfies T T where Λk = Q LkQ, and Gij (t)=aijQ (IN − JN )ηN,i ⊗ t ≥ ∈ 0 ξji(s)ds = wji(t), t 0,j Ni,i=1, 2,...,N, where (Kgji(Δxji(t))) with ηN,i denoting the N-dimensional col- { ∈ } wji(t), j Ni,i=1, 2,...,N are scalar independent Brow- umn vector with the ith element being 1 and others being nian motions. r zero. It is easy to see that k=1 Λk =Λ. Note that δi(t)= Assumption IV.2: For each (j, i), gji(0) = 0 and there exists a N xi − k xk(t)/N , i =1,...,N. Hence, mean square (or positive constant σji, such that |gji(x) − gji(y)|≤σji|x − y|, =1 n E| |2 for all x, y ∈ R . almost sure) weak consensus equals limt→∞ δ(t) =0 | | Based on the measurement zji(t), we aim to find a control u, (or limt→∞ δ(t) =0, a.s.) for any initial data. Note that 2 N−1 2 2 2 such that the multiagent system (39) achieves the mean square |Gji(t)| ≤ N |K| σjiaij |δj(t) − δi(t)| . Considering the 2 consensus or almost sure consensus, whose definitions are as Lyapunov function V (x)=|x| , we obtain for τij (t)=0 follows. T N Definition IV.1: We say that the control u(t) solves mean T K + K 2 LV (δ(t)) = 2δ(t) (Λ ⊗ )δ(t)+ |Gij (t)| square (or almost sure) weak consensus if it makes the agents 2 have the property that for any initial data ϕ and all distinct i, j ∈ i,j=1 2 V, limt→∞ E|xi(t) − xj(t)| =0[or limt→∞ |xi(t) − xj(t)| = 2 ≤−2λ (ΦK )|δ¯(t)| (43) 0 almost surely (a.s.)]. If, in addition, there is a random vec- min ∗ n ∗ ∈ R E 2 ∞ E − K+KT N−1 2 2 tor x , such that x < and limt→∞ xi(t) where ΦK =Λ⊗ − |K| σ¯ (Λ ⊗ In), σ¯ = ∗ 2 P{ ∗ ∞} − ∗ 2 N x =0(or x < =1and limt→∞ xi(t) x = maxi,j σji. That is, if ΦK > 0, then the system (42) without 0, a.s.), i =1, 2,...,N, then we say that the control u(t) solves time delays is mean square and almost surely exponentially mean square (or almost sure) strong consensus. stable [3]. And then the control u(t) defined by (40) We consider the following consensus control solves mean square and almost sure weak consensus for K ui(t)=ui (t)=K zji(t),i=1, 2,...,N (40) multiagent systems without time delays, which was also proved 2 N−1 2 2 2 in [36]. Note that |Gji(t)| ≤ |K| σ aij |δ(t)| . Let j∈Ni √ N ji 2 2 r 2 N C (τ)=|K| 2(τ r |Λi| + τN(N − 1) aij where K is the symmetrical control gain matrix to be designed. 12 i=1 i,j=1 2 1/2 r N−1 N 2 Consensus control under undirected topologies. Under con- σji) ( i |Λi| + N |K| i,j aijσji). By Corollary N − =1 =1 trol (40), system (39) has the form dxi(t)= j=1 aij Δxji(t II.1, if N τji(t))dt + aijgji(Δxji(t − τji(t)))dwji(t), which can j=1 C12(τ) < 2λmin(ΦK ) (44) be rewritten as r delay system (42) is mean square exponentially stable, and then dx(t)= − (Lk ⊗ K)x(t − τk(t))dt the control u(t) defined by (40) solves mean square weak con- k=1 sensus with an exponential rate. Note that all the coefficients in (42) are Lipschitz continuous. Then the mean square exponential N stability implies the almost sure exponential stability. That is, + [ηN,i ⊗ Kgji(Δxji(t − τji(t)))]dwji(t) under condition (44), the control u(t) defined by (40) also solves i,j=1 almost sure weak consensus with an exponential rate. Hence, (41) from Lemma 4.1 in [30], we have the following theorem. where r ≤ N(N − 1)/2, τk(·) ∈{τij (·):i, j =1,...,N} for Theorem IV.1: Let Assumptions IV.1 and IV.2 hold. If (44) k =1,...r, ηN,i denotes the N-dimensional column vector holds, then the control u(t) defined by (40) solves mean square with the ith element being 1 and others being zero, and Lk = and almost sure strong consensus.

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2616 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

From (44), we can see that if the delay-free multiagent system where Lk for k =1,...,r are defined above, Lr+i = with the condition ΦK > 0 can achieve the mean square consen- a0iηN,i, τr+i(t)=τi0(t), i =1,...,N, and Gij (t)= sus, then a small delay is allowed since the left side of (44) tends aij[ηN,i ⊗ Kgji(Δδji(t − τji(t)))]. Hence, mean square 2 to zero as τ → 0. That is, if the time-delay τ is small and satisfies (or almost sure) tracking equals limt→∞ E|δ(t)| =0 (or | | (44), we do not have to change the control gain K designed for limt→∞ δ(t) =0, a.s.) for any initial data. It can be proved the delay-free case. Then one may ask how about the case with r+N that k Lk = L + B, and large time delays. In fact, for large delays, we can adjust the =1 N N control gain to guarantee the mean square consensus. To see it 2 aij |ηN,i ⊗ Kgji(Δδji(t))| clearly, we choose the control gain K = kIn with k>0, and then (44) can be⎛ rewritten as ⎞ i=1 j=0 r − N N N ⎝ N 1 2 ⎠ 2 2 2 kb1 |Λi| + k aij σji ≤|K| aij σ |Δδji(t)| N ji i=1 i,j=1 i j =1 =0 N − 1 2 2 T < 2λ 1 − kσ¯2 (45) ≤|K| σ¯ δ(t) ((L + B) ⊗ In)δ(t). 2 N √ Considering the Lyapunov function V (x)=|x|2, we obtain for 2 r 2 N where b = 2(τ r |Λi| + τN(N − 1) 1 i=1 i,j=1 τij(t)=0 / N− N r 2 1 2 1 2 | | N aij σji) .Leta = N b1 i,j=1 aij σji, b = b1 i=1 Λi + N− T 2 1 λ 2 ∈ ¯ LV (δ(t)) = 2δ(t) ((L + B) ⊗ K)δ(t)+ |Gij (t)| N 2σ¯ . Then√ (45) can be guaranteed by the choice k (0, k), 2 i,j ¯ −b+ b +8aλ2 =1 where k = a . 2 ¯ 2 Tracking Control Under Leader-Following Topologies. We ≤−2λmin(ΦK )|δ(t)| aim to design the control, such that all the N agents can track K+KT 2 2 where ΦK =(L + B) ⊗ −|K| σ¯ ((L + B) ⊗ In). a leader denoted by 0. The state of the leader is assumed to 2 be a constant denoted by x .Fortheith follower, the dynamic That is, if ΦK > 0, then the system (46) without 0 time delays is mean square exponentially stable. And is described by (39) with ui(t) defined by (40). Note that this is different from the above since for each agent i, its neighbor then the control u(t) solves mean square tracking for the first-order multiagent systems without time set Ni may contain the leader 0. Assumptions IV.1 and IV.2 | |2 ≤| |2 2 | − |2 are also deemed to include the leader 0. Considering the in- delays. Note that √Gji(t) K σjiaij δ(t τji(t)) . | | r+N | |2 formation flow from the leader to the followers, we denote the Let C13(τ)= K 2τ(τr i=1 Li + N (N + G {V A} V { } N N 2 1/2 r+N | | | | N N topology graph by = , with = 0, 1, 2,...,N and 1) i=1 j=0 aijσji) ( i=1 Li + K i=1 j=0 aij 001×N (N+1)×(N+1) N×N 2 A =( ) ∈ R , where A =[aij ] ∈ R , σji). By Corollary II.1, if a0 A T C13(τ) < 2λmin(ΦK ), (47) a0 =[a10,...,aN0] , ai0 =1 if 0 ∈ Ni, otherwise ai0 =0. Let B = diag(a10,...,aN0) and Bi = diag(0,...,ai0,...). then the delay system (46) is mean square exponentially stable, We use G =(V, A) to represent the subgraph formed by the and then the protocol u(t) solves mean square tracking. N followers, where V = V\{ 0}. Theorem IV.2: Let Assumptions IV.1 and IV.2 hold. If (47) Definition IV.2: We say that the control u(t) solves mean holds, then the control u(t) defined by (40) solves mean square square (or almost sure) tracking problem if it makes the N +1 and almost sure tracking. agents have the property that for any initial data ϕ and all i ∈V, Similarly, we can choose the control gain K = kIn with k> 2 0, and then (47) can be rewritten as limt→∞ E|xi(t) − x0| =0(or limt→∞ |xi(t) − x0| =0a.s.). ⎛ ⎞ We impose the following assumption on the graph G and its r+N N N ⎝ 2 ⎠ 2 subgraph G. kb1 |Li| + k aij σji < 2λ01(1 − kσ¯ ), (48) Assumption IV.3: Assume that the graph G contains a span- i=1 i=1 j=0 G √ ning tree and its subgraph is undirected. r | |2 N where b1 = 2τ(τr i=1 Li + N(N +1) i,j=1 aij Let L0 = L + B. Under Assumption IV.3, we know that L0 2 1/2 N 2 r | | λ 2 is symmetric, and all eigenvalues of the matrix L0 are positive σji) .Leta = b1 i,j=1 aij σji, b = b1 i=1 Li + 01σ¯ . {λ }N ∈ ¯ ([51]), denoted by 0i i=1. Hence, there exists an unitary ma- Then (48)√ can be guaranteed by the choice k (0, k), where T 2 trix Φ, such that Φ L Φ=diag(λ ,...,λ N )=:Λ . Without −b+ b +8aλ2 0 01 0 0 k¯ = . loss of generality, we assume 0 < λ ≤ ...≤ λ N . 2a 01 0 Remark IV.1: Especially, if the leader-following topology Let δi(t)=xi(t) − x0 for i =1,...,N. Define δ(t)= T T T T T is a star, then the corresponding closed-loop system is de- [δ1 (t),...,δN (t)] .Letδ(t)=Φ δ(t), Φ(i)=Φ ηN,i. − − − − N coupled, that is, dδi(t)= kδi(t τ0i(t))dt + kg0i( δ0i(t Then we get dδi(t)= aij Δδji(t − τji(t))dt + j=0 τ0i(t)))dw0i(t). Applying Corollary II.1, we obtain that N − the above system is mean square exponentially stable if j=0 aij gji(Δδji(t τji(t)))dwji(t). Similarly to (41), √ we have k 2τ 2 +4τσ¯2(1 + kσ¯2) < 2 − kσ¯2. r+N Let us consider a scalar four-agent example under the dδ(t)= − (Lk ⊗ K)δ(t − τk(t))dt topology graph G = {V, E, A}, where V = {1, 2, 3, 4}, E = k =1 {(1, 2), (2, 1), (2, 3), (3, 4), (4, 3), (3, 2)} and A =[aij ]4×4 N N with a12 = a21 = a23 = a32 = a34 = a43 =1and other being + Gij (t)dwji(t) (46) zero. Moreover, we can obtain λ2 =0.5858, λ3 =2, and λ4 = T i=1 j=0 3.4142. The initial state is given by x(0) = [−7, 4, 3, −8] .

Fig. 1. Asymptotic behaviors of the four agents: k =0.05 and τ =0. Fig. 3. Asymptotic behaviors of the four agents: k =0.05, τ1 =0.2, τ1 =0.1,τ1 =0.15.

Fig. 2. Mean square errors of xi(t) − x1(t)| : k =0.05 and τ =0. Fig. 4. Mean square errors of |xi(t) − x1(t)| : k =0.05, τ1 =0.2, τ1 = 0.1,τ1 =0.15.

Assume fji(x)=σji with σji =0.5, i, j =1, 2, 3, 4, σ¯ = condition (45) holds. Hence, by Theorem IV.1, almost sure and maxi,j σji. mean square strong consensus can be achieved. In fact, Fig. 3 For the delay-free case, we choose k =0.05, and then it can shows that all the agents will tend to a common value. That is, N−1 2 λ 3 be seen that 0.01875 = 2 N kσ¯ < 2 2 =0.1716. That is, Φk the almost sure strong consensus is solved. Similarly, taking 10 2 defined in (43) is positive definite. Hence, the four agents achieve samples to approximate E|xi(t) − x1(t)| will produce Fig. 4, the mean square and almost sure strong consensus. The almost which depicts the mean square weak consensus. sure strong consensus is revealed in Fig. 1 and the mean square weak consensus is simulated in Fig. 2 by taking 103 samples to 2 V. C ONCLUSION approximate E|xi(t) − x1(t)| . Then we will reveal that a small delay is tolerated for This article establishes an important “robustness” type result, stochastic consensus. Let τ1 = τ12 = τ21 =0.2, τ2 = τ23 = namely, delay tolerance for stable stochastic systems under dif- τ32 =0.1 and τ2 = τ34 = τ43 =0.15. The initial data is x(t)= ferent Lipschitz type conditions. Under global Lipschitz condi- T [−2, 1, 2, −3] for t ∈ [−τ,0], τ =maxi τi. In this case, r =3 tions, we first show that if the delay-free stochastic system is pth ⎡ ⎤ ⎡ ⎤ moment exponentially stable, the corresponding stochastic delay 1 −100 00 00 version is still pth moment exponentially stable for sufficiently ⎢− ⎥ ⎢ − ⎥ ⎢ 1100⎥ ⎢ 01 10⎥ small delay. Then we prove that if the moment exponential L1 = ⎣ ⎦ ,L2 = ⎣ ⎦ 0000 0 −110 stability for delay-free system is based on Lyapunov conditions, 0000 00 00 a large delay is allowed for the delay system to be moment and exponentially stable. Without the global Lipschitz conditions, ⎡ ⎤ we studied the delay tolerance issues for mean square stable 00 0 0 stochastic systems under a class of one-sided linear growth ⎢ ⎥ 00 0 0 condition on the drift and polynomial growth condition on the L = ⎢ ⎥ . 3 ⎣00 1 −1⎦ diffusion. We also considered applications of delay tolerance 00−11 to the control design of multiagent systems with multiplicative noises and nonuniform delays. r | | Then it can be seen that 0.9614 = kb1( i=1 Λi + The results obtained can be used to examine the stochastic N−1 N 2 λ − N−1 2 N k i,j=1 σji) < 2 2(1 N kσ¯ )=1.1606. That is, systems with G-Brownians and improve the delay bound as in

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply. 2618 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 66, NO. 6, JUNE 2021

Ren et al. [52]. It can also be extended to the discrete-time case. [23] F. Wu, G. Yin, and L. Y. Wang, “Stability of a pure random delay system Based on this article, many related issues can be dealt with. For with two-time-scale Markovian switching,” J. Differ. Equ., vol. 253, no. 3, example, the delay tolerance for the boundedness and ergodicity pp. 878–905, 2012. [24] S.-E. A. Mohammed and M. K. R. Scheutzow, “Lyapunov exponents of stochastic systems. of linear stochastic functional differential equations. Part II. Examples and case studies,” Ann. Probability, vol. 25, no. 3, pp. 1210–1240, 1997. [25] M. Scheutzow, “Exponential growth rates for stochastic delay differential REFERENCES equations,” Stochastics Dyn., vol. 5, no. 02, pp. 163–174, 2008. [26] Q. Guo, X. Mao, and R. Yue, “Almost sure exponential stability of [1] A. Kohatsu-Higa, N. Privault, and S.-J. Sheu, Stochastic Analysis with stochastic differential delay equations,” SIAM J. Control Optim., vol. 54, Financial Applications (Progress in Probability). vol. 65, Berlin, Germany: no. 4, pp. 1919–1933, 2016. Springer, 2011. [27] M. Song and X. Mao, “Almost sure exponential stability of hybrid stochas- [2] H. J. Kushner, Stochastic Stability and Control, vol. 27. San Francisco, tic functional differential equations,” J. Math. Anal. Appl., vol. 458, no. 2, CA, USA: Academic, 2014. pp. 1390–1408, 2018. [3] X. Mao, Stochastic Differential Equations and their Applications (Hor- [28] X. Mao, “Robustness of exponential stability of stochastic differential wood Publishing Series in Mathematics & Applications). Chichester, U.K.: delay equations,” IEEE Trans. Autom. Control, vol. 41, no. 3, pp. 442–447, Horwood Publishing Limited, 1997. Mar. 1996. [4] V.Kolmanovskii and A. Myshkis, Introduction To the Theory and Applica- [29] X. Zong, T. Li, G. Yin, L. Y. Wang, and J. F. Zhang, “Stochastic tions of Functional Differential Equations, vol. 463. Norwell, MA, USA: consentability of linear systems with time delays and multiplicative Kluwer, 1999. noises,” IEEE Trans. Autom. Control, vol. 63, no. 4, pp. 1059–1074, [5] J. Bao, G. Yin, and C. Yuan, Asymptotic Analysis for Functional Stochas- Apr. 2018. tic Differential Equations (Briefs in Mathematics). Berlin, Germany: [30] X. Zong, T. Li, and J.-F.Zhang, “Consensus conditions for continuous-time Springer, 2016. multi-agent systems with time-delays and measurement noises,” Automat- [6] M. Shen, C. Fei, W. Fei, and X. Mao, “Boundedness and stability of highly ica, vol. 99, pp. 412–419, 2019. nonlinear hybrid neutral stochastic systems with multiple delays,” Sci. [31] P.-A. Bliman and G. Ferrari-Trecate, “Average consensus problems in China Inf. Sci., vol. 62, no. 10, 2019, Art. no. 202205. networks of agents with delayed communications,” Automatica, vol. 44, [7] D. Bresch-Pietri and M. Krstic, “Delay-adaptive control for nonlinear no. 8, pp. 1985–1995, 2008. systems,” IEEE Trans. Autom. Control, vol. 59, no. 5, pp. 1203–1218, [32] Y. G. Sun and L. Wang, “Consensus of multi-agent systems in directed May 2014. networks with nonuniform time-varying delays,” IEEE Trans. Autom. [8] M.-S. Koo and H.-L. Choi, “Output feedback regulation of a chain of Control, vol. 54, pp. 1607–1613, Jul. 2009. integrators with unknown time-varying delays in states and input,” Auto- [33] A. Papachristodoulou, A. Jadbabaie, and U. Münz, “Effects of delay matica, vol. 58, pp. 183–190, 2015. in multi-agent consensus and oscillator synchronization,” IEEE Trans. [9] F. J. Bejarano and G. Zheng, “Observability of linear systems with Autom. Control, vol. 55, no. 6, pp. 1471–1477, Jun. 2010. commensurate delays and unknown inputs,” Automatica, vol. 50, no. 8, [34] Y. Li, Y. Huang, P. Lin, and W. Ren, “Distributed rotating consensus of pp. 2077–2083, 2014. second-order multi-agent systems with nonuniform delays,” Syst. Control [10] T. Zhang, F. Deng, and W. Zhang, “Study on stability in probability of Lett., vol. 117, pp. 18–22, 2018. general discrete-time stochastic systems,” Sci. China Inf. Sci., vol. 63, [35] L. Li, X. Liu, and W. Huang, “Event-based bipartite multi-agent con- no. 5, 2020, Art. no. 159205. sensus with partial information transmission and communication delays [11] X. Hu, H. Yu, and F. Hao, “Lyapunov-based event-triggered control for under antagonistic interactions,” Sci. China Inf. Sci., vol. 63, no. 5, 2020, nonlinear plants subject to disturbances and transmission delays,” Sci. Art. no. 150204. China Inf. Sci., vol. 63, no. 5, 2020, Art. no. 150211. [36] T. Li, F. Wu, and J.-F. Zhang, “Multi-agent consensus with relative-state- [12] S.-E. A. Mohammed, Stochastic Functional Differential Equations. no. 99, dependent measurement noises,” IEEE Trans. Autom. Control, vol. 59, White Plains, NY, USA: Longman, 1986. no. 9, pp. 2463–2468, Sep. 2014. [13] X. Mao, “Razumikhin-type theorems on exponential stability of stochastic [37] P. H. Baxendale, “Lyapunov exponents and stability for the stochastic functional differential equations,” Stochastic Process. Appl., vol. 65, no. 2, Duffing-van der Pol oscillator,” in IUTAM Proceedings on Nonlinear pp. 233–250, 1996. Stochastic Dynamics, N. Sri Namachchivaya and Y. K. Lin, Eds., Dor- [14] F. Wu and S. Hu, “Razumikhin-type theorems on general decay stability drecht: Kluwer, 2003, pp. 125–135. and robustness for stochastic functional differential equations,” Int. J. [38] D. Sepúlveda, “Approximation of linear differential equations with vari- Robust Nonlinear Control, vol. 22, no. 7, pp. 763–777, 2012. able delay on an infinite interval,” 2016, arXiv:1607.06960. [15] S. Janković, J. Randjelović, and M. Jovanović, “Razumikhin-type ex- [39] W. Qi, G. Zong, and H. Karimi, “Sliding mode control for nonlinear ponential stability criteria of neutral stochastic functional differential stochastic singular Semi-Markov jump systems,” IEEE Trans. Autom. equations,” J. Math. Anal. Appl., vol. 355, no. 2, pp. 811–820, 2009. Control, vol. 65, no. 1, pp. 361–368, Jan. 2020. [16] F. Wu, G. Yin, and L. Y. Wang, “Razumikhin-type theorems on moment [40] L. Hetel et al., “Recent developments on the stability of systems with exponential stability of functional differential equations involving two- aperiodic sampling: An overview,” Automatica, vol. 76, pp. 309–335, time-scale Markovian switching,” Math. Control Related Fields,vol.5, 2017. no. 3, pp. 697–719, 2015. [41] X. Mao, “Technical communique: Stabilization of continuous-time hy- [17] W. Hu, Q. Zhu, and H. Karimi, “Some improved Razumikhin stability brid stochastic differential equations by discrete-time feedback control,” criteria for impulsive stochastic delay differential systems,” IEEE Trans. Automatica, vol. 49, no. 12, pp. 3677–3681, 2013. Autom. Control, vol. 64, no. 12, pp. 5207–5213, Dec. 2019. [42] X. Mao, W. Liu, L. Hu, Q. Luo, and J. Lu, “Stabilization of hybrid stochas- [18] R. Rakkiyappan, P. Balasubramaniam, and S. Lakshmanan, “Robust sta- tic differential equations by feedback control based on discrete-time state bility results for uncertain stochastic neural networks with discrete interval observations,” Syst. Control Lett., vol. 73, pp. 88–95, 2014. and distributed time-varying delays,” Phys. Lett. A, vol. 372, no. 32, [43] S. You, W. Liu, J. Lu, X. Mao, and Q. Qiu, “Stabilization of hybrid systems pp. 5290–5298, 2008. by feedback control based on discrete-time state observations,” SIAM J. [19] E. Gershon, U. Shaked, and N. Berman, “H∞ control and estimation Control Optim., vol. 53, no. 2, pp. 905–925, 2015. of retarded state-multiplicative stochastic systems,” IEEE Trans. Autom. [44] W. I. Lee, S. Y. Lee, and P. Park, “Affine Bessel–Legendre inequality: Control, vol. 52, no. 9, pp. 1773–1779, Sep. 2007. Application to stability analysis for systems with time-varying delays,” [20] L. Shaikhet, Lyapunov Functionals and Stability of Stochastic Functional Automatica, vol. 93, pp. 535–539, 2018. Differential Equations. Berlin, Germany: Springer, 2013. [45] K. Liu and E. Fridman, “Wirtinger’s inequality and Lyapunov-based [21] W. Fei, L. Hu, X. Mao, and M. Shen, “Delay dependent stability of highly sampled-data stabilization,” Automatica, vol. 48, no. 1, pp. 102–108, 2012. nonlinear hybrid stochastic systems,” Automatica, vol. 82, pp. 165–170, [46] X. Zong, F. Wu, and C. Huang, “Theta schemes for SDDEs with 2017. non-globally lipschitz continuous coefficients,” J. Comput. Appl. Math., [22] F. Hartung, T. Krisztin, H. O. Walther, and J. Wu, “Functional differential vol. 278, no. 1, pp. 258–277, 2015. equations with state-dependent delays: Theory and applications,” Hand- [47] X. Li and X. Mao, “Stabilisation of highly nonlinear hybrid stochastic dif- book of Differential Equations Ordinary Differential Equations,vol.3, ferential delay equations by delay feedback control,” Automatica, vol. 112, Amsterdam, The Netherlands: Elsevier, 2006, pp. 435–545. 2020, Art. no. 108657.

[48] W. Hu, Q. Zhu, and H. R. Karimi, “On the pth moment integral input-to- George Yin (Fellow, IEEE) received the B.S. state stability and input-to-state stability criteria for impulsive stochastic degree in mathematics from the University of functional differential equations,” Int. J. Robust Nonlinear Control, vol. 29, Delaware, Newark, DE, USA, in 1983, and no. 16, pp. 5609–5620, 2019. the M.S. degree in electrical engineering and [49] J. Zhai and H. R. Karimi, “Global output feedback control for a class of the Ph.D. degree in applied mathematics from nonlinear systems with unknown homogenous growth condition,” Int. J. Brown University, Providence, RI, USA, in 1987. Robust Nonlinear Control, vol. 29, no. 7, pp. 2082–2095, 2019. He joined the Department of Mathematics, [50] X. Zong, G. Yin, L. Y.Wang, T. Li, and J. F. Zhang, “Stability of stochastic Wayne State University, Detroit, MI, USA, in functional differential systems using degenerate lyapunov functionals and 1987, and became Professor, in 1996, and Uni- applications,” Automatica, vol. 91, pp. 197–207, 2018. versity Distinguished Professor, in 2017. He [51] C. Ma, T. Li, and J.-F. Zhang, “Consensus control for leader-following moved to the University of Connecticut, Storrs, multi-agent systems with measurement noises,” J. Syst. Sci. Complexity, CT, USA, in 2020. His research interests include stochastic processes, vol. 23, no. 1, pp. 35–49, 2010. stochastic systems theory, and applications. [52] Y. Ren, W. Yin, and R. Sakthivel, “Stabilization of stochastic differential Dr. Yin is a Fellow of IFAC and a Fellow of SIAM. He was the Chair equations driven by G-Brownian motion with feedback control based on of the SIAM Activity Group on Control and Systems Theory, and served discrete-time state observation,” Automatica, vol. 95, pp. 146–151, 2018. on the Board of Directors of the American Automatic Control Council. He is the Editor-in Chief for SIAM Journal on Control and Optimization, was a Senior Editor for IEEE CONTROL SYSTEMS LETTERS, and is an Associate Editor for ESAIM: Control, Optimisation and Calculus of Vari- ations, Applied Mathematics and Optimization and many other journals. He was an Associate Editor for Automatica from 2005 to 2011 and IEEE Xiaofeng Zong received the B.S. degree in TRANSACTIONS ON AUTOMATIC CONTROL from 1994 to 1998. mathematics from Anqing Normal University, Anhui, China, in 2009, and the Ph.D. degree in probability theory and mathematical statistics from the School of Mathematics and Statistics, Huazhong University of Science and Technol- ogy, Hubei, China, in 2014. He was a Postdoctoral Researcher with the Academy of Mathematics and Systems Science Ji-Feng Zhang (Fellow, IEEE) received the B.S. (AMSS), Chinese Academy of Sciences (CAS), degree in mathematics from Shandong Univer- Beijing, China, from July 2014 to July 2016, and sity, Jinan, China, in 1985, and the Ph.D. degree a Visiting Assistant Professor with the Department of Mathematics, in operations research and control theory from Wayne State University, Detroit, MI, USA, from September 2015 to Octo- the Institute of Systems Science (ISS), Chinese ber 2016. Since October 2016, he has been a Professor with the School Academy of Sciences (CAS), Beijing, China, in of Automation, China University of Geosciences, Wuhan, Hubei, China, 1991. where now he is an Associate Dean with the School of Automation. His Since 1985, he has been with the ISS, CAS, current research interests include stochastic approximations, stochastic and now is the Director of ISS. His current systems, delay systems, and multiagent systems. research interests include system modeling, adaptive control, stochastic systems, and multiagent systems. Dr. Zhang is a Fellow of the International Federation of Automatic Control (IFAC), member of the European Academy of Sciences and Arts, and Academician of the International Academy for Systems and Tao Li (Senior Member, IEEE) received the B.E. Cybernetic Sciences. He was the recipient of the Second Prize of the degree in automation from Nankai University, State Natural Science Award of China, in 2010 and 2015, respectively. Tianjin, China, in 2004, and the Ph.D. degree He is a Vice-President of the Chinese Association of Automation, Vice- in systems theory from the Academy of Mathe- President of the Chinese Mathematical Society, Associate Editor-in- matics and Systems Science (AMSS), Chinese Chief for Science China Information Sciences, and Senior Editor for Academy of Sciences (CAS), Beijing, China, in IEEE CONTROL SYSTEMS LETTERS. He was a Vice-Chair of the IFAC 2009. Technical Board, member of the Board of Governors, IEEE Control Since January 2017, he has been with Systems Society; Convenor of Systems Science Discipline, Academic East China Normal University, Shanghai, China, Degree Committee of the State Council of China; Vice-President of where now he is a Professor and Director of the the Systems Engineering Society of China; and Editor-in-Chief, Deputy Department of Mathematical Intelligence Sci- Editor-in-Chief or Associate Editor of more than ten journals, including ences, School of Mathematical Sciences. His current research inter- Journal of Systems Science and Mathematical Sciences, IEEE TRANS- ests include stochastic systems, cyber-physical multiagent systems, and ACTIONS ON AUTOMATIC CONTROL, and SIAM Journal on Control and game theory. Optimization etc. He was General Co-Chair or IPC Chair of many control Dr Li is a member of the International Federation of Automatic Control conferences. (IFAC) Technical Committee on Networked Systems and a member of Technical Committee on Control and Decision of Cyber-Physical Sys- tems, Chinese Association of Automation. He was the recipient of the 28th “Zhang Siying” Chinese Control and Decision Conference (CCDC) Outstanding Youth Paper Award, in 2016, the Best Paper Award of the 7th Asian Control Conference, in 2009, and honourable mention as one of ﬁve ﬁnalists for Young Author Prize of the 17th IFAC Congress, in 2008. He was the recipient of the 2009 Singapore Millennium Foun- dation Research Fellowship and the 2010 Australian Endeavor Re- search Fellowship. He was entitled Dongfang Distinguished Professor by Shanghai Municipality in 2012, was the recipient of the Excellent Young Scholar Fund from the National Natural Science Foundation of China (NSFC) in 2015 and was elected to the Chang Jiang Scholars Program, Ministry of Education, China, Youth Scholar, in 2018. He now serves as an Associate Editor for Science China Information Science, Systems and Control Letters and IEEE CONTROL SYSTEMS LETTERS,etc.

Authorized licensed use limited to: East China Normal University. Downloaded on May 28,2021 at 02:32:25 UTC from IEEE Xplore. Restrictions apply.