Biologically-Inspired Intelligent Flocking Control for Networked Multi-UAS with Uncertain Network Imperfections

Abstract

In this paper, a biologically-inspired distributed intelligent control methodology is proposed to overcome the challenges, i.e., networked imperfections and uncertainty from the environment and system, in networked multi-Unmanned Aircraft Systems (UAS) flocking. The proposed method is adopted based on the emotional learning phenomenon in the mammalian limbic system, considering the limited computational ability in the practical onboard controller. The learning capability and low computational complexity of the proposed technique make it a propitious tool for implementing in real-time networked multi-UAS flocking considering the network imperfection and uncertainty from environment and system. Computer-aid numerical results of the implementation of the proposed methodology demonstrate the effectiveness of this algorithm for distributed intelligent flocking control of networked multi-UAS.

1. Introduction

1.1. Motivation

Distributed coordination of networked multi-Unmanned Aircraft Systems (UAS) has been studied by diverse research communities in recent years [1,2,3,4]. Due to the broad applications of flocking in real-world scenarios, most of the networked multi-UAS control methodologies are adopted based on the mathematical model of flocking [5,6,7]. In general, three basic rules (i.e., separation, alignment, and cohesion) are considered for simulating the flocking behavior [1] which are observed in many living beings (i.e., birds, fish, bacteria, and insects) [8].

Several research groups have been contributed for improving the flocking behavior of networked multi-UAS in recent years [9,10,11,12]. Groups of aircrafts, in many applications involving networked multi-UAS, require communicating in order to successfully accomplish their assigned tasks. Network imperfection, e.g., delay, commonly exists in communication due to the limited communication resource and heavy traffic in the network [13,14]. As a result, it is of paramount importance to address the challenges of network-induced delay and taking the influence of network-induced delays into account in designing control algorithms for multi-unmanned aircraft systems. Besides network imperfections, the uncertainty from the complex environment and system dynamics is another critical challenge and cannot be ignored in advanced applicable control development. Therefore, it is important to consider the uncertainties from the environment and system in designing control algorithms.

1.2. Related Works

Diverse research groups have attempted to address the issues arising from the effects of network-induced delay in flocking control of multi-unmanned aircraft systems/multi-agent systems. For example, delay-independent flocking control of multi-agent systems have been addressed in [15,16]. Closely related, a distributed control design for the discrete-time directed multi-agent systems with distributed network-induced delay has been proposed in [17]. The authors in [18] presented a distributed algorithm for the sensor networks by considering the effects of the imperfect communication such as link failures and channel noise. In [19], the authors studied the design of distributed formation recovery control for nonlinear heterogeneous multi-agent systems. A distance constrained based adaptive flocking control for the multi-agent system with network-induced delay was investigated in [20]. Recently, coordinated control of two-wheel mobile robots with input network-induced delay was presented in [21]. Although all of these proposed methods perform well dealing with the effects of network-induced delay, they still need the detailed information of the system. In this regard, the development of control strategies for dealing with the network-induced delay with less dependency on the full knowledge of the system dynamics is of paramount importance.

In recent years, intelligent approaches have been extensively utilized for successfully solving diverse complex problems [22,23,24]. Among them, biologically-inspired intelligent approaches have received tremendous interest by many researchers because of their inherent potential to deal with computationally complex systems. Emotional Learning is one such approach, which takes advantage of a computational model of the amygdala in the mammalian limbic system [25]. This model, known as Brain Emotional Learning Based Intelligent Controller (BELBIC), consists of the Amygdala, Orbitofrontal Cortex, Thalamus, and Sensory Input Cortex as its main components. From a control systems point of view, BELBIC is a model-free controller (i.e., model dynamics are fully or partially unknown) that has shown promising performance under noise and system uncertainty [26].

Sensory Inputs ($SI$) and Emotional Signal ($ES$) are two main inputs to the BELBIC model, and it is shown that the multi-objective problems could be solved by defining appropriate $SI$ and $ES$ [27,28]. The flexibility in assigning different $SI$ and $ES$ makes this controller a practical tool for implementation in real-time applications. Furthermore, BELBIC could effectively control a system even when the states of the system and the controller performance feedback are the only available information [26]. In addition, compared to other existing learning-based intelligent control methods, the computational complexity of BELBIC is on the order of $O\left(n\right),$ which makes it a suitable approach for real-time implementation.

1.3. Main Contributions

The main contribution of this paper is to develop a model-free distributed intelligent control methodology to overcome the challenges including the network-induced delay and uncertainties from the environment and system in networked multi-UAS. To this end, we propose a biologically-inspired distributed intelligent controller, which takes advantage of the computational model of emotional learning in the mammalian limbic system. Our recent work in [29] introduced the theoretical framework for the flocking control for networked multi-UAS using BELBIC. These results are improved upon and extended in this paper for accomplishing an intelligent and practical flocking of networked multi-UAS, when subjected to impacts from uncertain network imperfections. The proposed methodology has a low computational complexity, which makes it a promising method for real-time applications. Furthermore, keeping the system complexity in a practically achievable limit, the proposed method delivers a controller with multi-objective properties (i.e., control effort optimization, network-induced delay handling, and noise/disturbance rejection). Moreover, we provided the Lyapunov stability analysis to demonstrate that our proposed methodology guarantees the convergence of the designed control signals as well as maintaining the system stability during the learning. The learning capability of the proposed approach is validated for flocking control of multi-unmanned aircraft systems influenced by the network-induced delay with promising performance. Computer-based numerical results of the implementation of the proposed methodology demonstrate the effectiveness of this algorithm for distributed intelligent flocking control of networked multi-UAS.

In other words, the solution proposed in this paper is a model-free distributed intelligent controller, which provides the following benefits:

• The knowledge of system dynamics is not fully or partially required.
• It has the capabilities of overcoming the network-induced delay, handling the uncertainties, and noise/disturbance rejection.
• It is appropriate for real-time implementation due to its low computational complexity (i.e., the developed algorithm is a real-time applicable learning technique).
• It ensures the stability of the system.

The rest of the paper is organized as follows. Section 2 presents the problem formulation and some preliminaries about flock modeling with network-induced delay and emotional learning. Our main contribution is introduced in Section 3, which consists of a distributed intelligent flocking control strategy based on emotional learning. Section 4 presents numerical simulation results. The conclusions of the paper and future directions of our work are provided in Section 5.

2. Problem Formulation and Preliminaries

In this section, some preliminaries are provided and the problem formulation is briefly discussed. First, the dynamic of the networked multi-UAS is given; then, flock topology is modeled by means of a dynamic graph. Next, network-induced delay and BELBIC model are introduced, and, ultimately, the problem is formulated.

2.1. Flock Modelling

Assuming the movement of the flock in an m–dimensional space $(m=2,3)$, the equation of motion of the ith agent with continuous-time double integrator dynamics could be described according to the following set of equations:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\dot{q}}_i\left(t\right)=p_i\left(t\right),\hfill \\ {\dot{p}}_i\left(t\right)=u_i\left(t\right),i=1,2,\dots ,n,\hfill \end{array}\right.\end{array}$$

(1)

where $u_i\left(t\right)\in \mathrm{I}{\mathrm{R}}^{\mathrm{m}}$ is the control input, $\{q_i\left(t\right),p_i\left(t\right)\}\in \mathrm{I}{\mathrm{R}}^{\mathrm{m}}$ are position, and velocity of the ith agent, respectively. Consider a dynamic graph $\mathcal{G}(\upsilon ,\epsilon (t\left)\right)$ that consists of a set of vertices $\upsilon =\{1,2,\dots ,n\}$, and edges $\epsilon \left(t\right)\subseteq \left\{\right(i,j):i,j\in \upsilon ,j\ne i\}$. Each vertex represents an agent of the flock while a communication link between a pair of agents is represented by an edge.

$N_i^{\alpha }\left(t\right)=\left\{j\in {\upsilon }_{\alpha }:\parallel q_j\left(t\right)-q_i\left(t\right)\parallel <r,j\ne i\right\}$ is the neighborhood set of agent i, where the range of interaction between agents i and agent j is defined by a positive constant r, and $\parallel ·\parallel$ is the Euclidean norm in $\mathrm{I}{\mathrm{R}}^{\mathrm{m}}$. Solving the set of algebraic conditions: $\parallel q_j\left(t\right)-q_i\left(t\right)\parallel =d\forall j\in N_i^{\alpha }\left(t\right)$, we could describe the geometric model of the flock, i.e., the $\alpha$-lattice [3], where the distance between two neighbors i and j is represented by a positive constant d.

To avoid the singularity of the collective potential function at $q_i\left(t\right)=q_j\left(t\right)$, the $\sigma$-norm (i.e., ${\parallel ·\parallel }_{\sigma }$) is defined where ${\parallel z\parallel }_{\sigma }=\frac{1}{ϵ}[\sqrt{1+ϵ{\parallel z\parallel }^2}-1]$, and $ϵ$ is a positive constant. To resolve the singularity problem, the set of algebraic conditions can be rewritten as: ${\parallel q_j\left(t\right)-q_i\left(t\right)\parallel }_{\sigma }=d_{\alpha }\forall j\in N_i^{\alpha }\left(t\right)$.

A smooth collective potential function $V\left(q\right)=\frac{1}{2}{\sum }_i{\sum }_{j\ne i}{\psi }_{\alpha }\left({\parallel q_j\left(t\right)-q_i\left(t\right)\parallel }_{\sigma }\right)$ can be obtained by considering the above-mentioned constraints, where ${\psi }_{\alpha }\left(z\right)={\int }_{d_{\alpha }}^z{\varphi }_{\alpha }\left(s\right)ds$ is a smooth pairwise potential function with ${\varphi }_{\alpha }\left(z\right)={\rho }_h(z/r_{\alpha })\varphi (z-d_{\alpha })$, $\varphi \left(z\right)=\frac{1}{2}[(a+b){\sigma }_1(z+c)+(a-b)]$, and ${\sigma }_1\left(z\right)=z/\sqrt{1+z^2}$.

A possible choice for defining $\rho \left(z\right)$, which is a scalar bump function that smoothly varies between [0,1], is [3]:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}1,z\in [0,h),\hfill \\ \frac{1}{2}\left[1+cos\left(\pi \frac{(z-h)}{(1-h)}\right)\right],z\in [h,1],\hfill \\ 0,\mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

(2)

$u_i\left(t\right)=u_i^{\alpha }+u_i^{\beta }+u_i^{\gamma }$ is the flocking control algorithm introduced in [3], which consists of three main terms:

(i).

$u_i^{\alpha }$ is the interaction component between two $\alpha$-agents and is defined as follows:

$$\begin{array}{cc}\hfill u_i^{\alpha }& =c_1^{\alpha }{\displaystyle \sum _{j\in N_i^{\alpha }}}{\varphi }_{\alpha }\left({\parallel q_j\left(t\right)-q_i\left(t\right)\parallel }_{\sigma }\right){\mathbf{n}}_{i,j}\hfill \\ & +c_2^{\alpha }{\displaystyle \sum _{j\in N_i^{\alpha }}}{a}_{ij}\left(q\right)(p_j\left(t\right)-p_i\left(t\right)),\hfill \end{array}$$

(3)

where $c_1^{\alpha }$ and $c_2^{\alpha }$ are positive constants. The terms ${\mathit{n}}_{i,j}$ and $a_{ij}\left(q\right)$ are vector and the elements of the spatial adjacency matrix $A\left(q\right)$, respectively, which are described as follows:

$$\begin{array}{cc}\hfill {\mathbf{n}}_{i,j}& =\frac{q_j\left(t\right)-q_i\left(t\right)}{\sqrt{1+ϵ{\parallel q_j\left(t\right)-q_i\left(t\right)\parallel }^2}},\hfill \\ \hfill a_{ij}\left(q\right)& ={\rho }_h({\parallel q_j\left(t\right)-q_i\left(t\right)\parallel }_{\sigma }/r_{\alpha })\in [0,1],j\ne i,\hfill \end{array}$$

where $r_{\alpha }={\parallel r\parallel }_{\sigma }$, and $a_{ii}\left(q\right)=0$ for all i and q.

(ii).

$u_i^{\beta }$ is the interaction component between the $\alpha$-agent and an obstacle (named the $\beta$-agent) and is defined as follows:

$$\begin{array}{cc}\hfill u_i^{\beta }& =c_1^{\beta }{\displaystyle \sum _{k\in N_i^{\beta }}}{\varphi }_{\beta }\left({\parallel {\widehat{q}}_{i,k}-q_i\parallel }_{\sigma }\right){\widehat{\mathbf{n}}}_{i,k}\hfill \\ & +c_2^{\beta }{\displaystyle \sum _{k\in N_i^{\beta }}}{b}_{i,k}\left(q\right)({\widehat{p}}_{i,k}-p_i),\hfill \end{array}$$

(4)

where $c_1^{\beta }$ and $c_2^{\beta }$ are positive constants. ${\widehat{q}}_{i,k}$ and ${\widehat{p}}_{i,k}$ are position, and velocity of the kth obstacle (i.e., $\beta$-agent), respectively. The terms ${\widehat{\mathit{n}}}_{i,k}$ and $b_{i,k}\left(q\right)$ are vector and the elements of the heterogeneous adjacency matrix $B\left(q\right)$, respectively, which are defined as follows:

$$\begin{array}{cc}\hfill {\widehat{\mathbf{n}}}_{i,k}& =\frac{{\widehat{q}}_{i,k}-q_i}{\sqrt{1+ϵ{\parallel {\widehat{q}}_{i,k}-q_i\parallel }^2}},\hfill \\ \hfill b_{i,k}\left(q\right)& ={\rho }_h({\parallel {\widehat{q}}_{i,k}-q_i\parallel }_{\sigma }/d_{\beta }).\hfill \end{array}$$

${\varphi }_{\beta }\left(z\right)={\rho }_h(z/d_{\beta })({\sigma }_1(z-d_{\beta })-1)$ is a repulsive action function and $N_i^{\beta }=\left\{k\in {\upsilon }_{\beta }:\parallel {\widehat{q}}_{i,k}-q_i\parallel <r^{\prime }\right\}$ is the set of $\beta$-neighbors of an $\alpha$-agent i, where the range of interaction of an $\alpha$-agent with obstacles is the positive constant $r^{\prime }$. Here, $d_{\beta }={\parallel d^{\prime }\parallel }_{\sigma }$, and $r_{\beta }={\parallel r^{\prime }\parallel }_{\sigma }$

(iii).

$u_i^{\gamma }$ is a goal component that consists of a distributed navigational feedback term and is defined as follows:

$$\begin{array}{c}\hfill u_i^{\gamma }=-c_1^{\gamma }{\sigma }_1(q_i-q_r)-c_2^{\gamma }(p_i-p_r),\end{array}$$

(5)

where $c_1^{\gamma }$ and $c_2^{\gamma }$ are positive constants.

More detailed studies about flocking control algorithms can be found in [3].

Remark 1.

In practical networked multi-UAS flocking control systems, due to the complexity of the overall system, autonomous agents are commonly described by the double integrator dynamics. Since our analysis is focused on developing an intelligent distributed controller for flocking of networked multi-UAS, in this paper, the double integrator dynamics is adopted. Considering the fact that the double integrator dynamics is a very reduced dynamics of the quad rotorcrafts, one can extend our results by employing model-free inner-loop controllers in [23,30], etc.

2.2. Network-Induced Delays

Assuming that the state of agent i gets to agent j after passing through a communication channel with network-induced delay ${\tau }_{ij}>0$, the $u_i^{\alpha }$ could be rewritten as:

$$\begin{array}{cc}{u}_i^{\alpha }& =c_1^{\alpha }{\displaystyle {\sum }_{j\in N_i^{\alpha }}}{\varphi }_{\alpha }\left({\parallel q_j(t-{\tau }_{ij})-q_i(t-{\tau }_{ij})\parallel }_{\sigma }\right){\mathbf{n}}_{i,j}\hfill \\ & +c_2^{\alpha }{\displaystyle {\sum }_{j\in N_i^{\alpha }}}{a}_{ij}\left(q\right)(p_j(t-{\tau }_{ij})-p_i(t-{\tau }_{ij})).\hfill \end{array}$$

(6)

In this paper, we consider the case where the network-induced delays in all channels are equal to $\tau >0$. Although the delay is deterministic, it is unknown. The proposed method can effectively handle this unknown delay.

2.3. Brain Emotional Learning-Based Intelligent Controller

Brain Emotional Learning Based Intelligent Controller (BELBIC) is one of the neurobiologically-motivated intelligent methodologies, which is based on the computational model of emotional learning observed in the mammalian limbic system proposed in [25]. This model (depicted in Figure 1), has two main parts: Amygdala, and Orbitofrontal Cortex. Amygdala is responsible for immediate learning, while Orbitofrontal Cortex is responsible for inhibition of any inappropriate learning happening in the Amygdala. Sensory Inputs ($SI$) and Emotional Signal ($ES$) are two main inputs to the BELBIC model.

The output of the BELBIC model ($MO$) can be defined as

$$\begin{array}{c}\hfill MO={\displaystyle \sum _l}{A}_l-{\displaystyle \sum _l}O{C}_l,\end{array}$$

(7)

which is calculated by the difference between the Amygdala outputs ($A_l$) and the Orbitofrontal Cortex outputs ($O{C}_l$). Here, l is the number of sensory inputs.

The Orbitofrontal Cortex and the Amygdala outputs are calculated by the summation of all their corresponding nodes, where the output of each node is described as:

$$\begin{array}{c}\hfill A_l=V_{l}S{I}_l,\end{array}$$

(8)

$$\begin{array}{c}\hfill O{C}_l=W_{l}S{I}_l,\end{array}$$

(9)

where $S{I}_l$ is the lth sensory input, $V_l$ is the weight of the Amygdala, and $W_l$ is the weight of the Orbitofrontal Cortex. The following equations are employed for updating $V_l$ and $W_l$, respectively:

$$\begin{array}{c}\hfill \Delta V_l=K_{v}S{I}_{l}max\left(0,ES-{\displaystyle \sum _l}{A}_l\right),\end{array}$$

(10)

$$\begin{array}{c}\hfill \Delta W_l=K_{w}S{I}_l\left(MO-ES\right),\end{array}$$

(11)

where $K_w$ and $K_v$ are the learning rates.

The maximum of all $SIs$ is another input considered in the model. This signal (i.e., $A_{th}$), which is directly sent from the Thalamus to the Amygdala, is defined as:

$$\begin{array}{c}\hfill A_{th}=V_{th}max\left(S{I}_l\right),\end{array}$$

(12)

where $V_{th}$ is the weight and the corresponding update law is the same as Equation (10).

Several techniques have been adopted for tuning the BELBIC parameters [26,31,32,33,34,35]. In this paper, to significantly reduce the computational complexity, a heuristic approach is utilized for tuning the BELBIC parameters.

2.4. Objectives

The objective is to design a biologically-inspired distributed intelligent controller for flocking control of multi-unmanned aircraft systems (i.e., $u_i$, $i=1,\dots ,n$ and n is the number of UASs), specifically, in the events of network-induced delay. The proposed intelligent control method is leveraging the computational model of emotional learning in the mammalian limbic system (i.e., BELBIC) introduced in Section 2.3, and is applied to the flocking model of networked multi-UAS described in Section 2.1.

In other words, the solution proposed in this paper is a model-free distributed intelligent controller, which is designed to maintain the motion of all agents in the flock in the events of network-induced delay.

3. Distributed Intelligent Flocking Control of Networked Multi-UAS Using Emotional Learning

3.1. System Design

Specifically, our focus is on the design of a biologically-inspired distributed intelligent controller for flocking of networked multi-UAS by using BELBIC because the implementation of it could be accomplished without increasing the complexity of the overall system. The BELBIC architecture implemented in this work is shown in Figure 2. This figure demonstrates a closed loop configuration that consists of the following blocks: (i) BELBIC block, (ii) Sensory inputs (SI) function block, (iii) Emotional signal (ES) generator block, and finally (iv) a block for the plant. This architecture implicitly demonstrates the overall emotional learning based control concept, which consists of the action selection mechanism, the critic, and the learning algorithm [26].

3.2. Emotional Signal and Sensory Input Development

Fundamentally, BELBIC is an action selection technique, in which action is produced based on the sensory input ($SI$) and the emotional signal ($ES$). The general forms of $SI$ and $ES$ are given as follows:

$$\begin{array}{c}\hfill SI=G\left(e,r,y,u\right),\end{array}$$

(13)

$$\begin{array}{c}\hfill ES=F\left(e,r,y,u\right),\end{array}$$

(14)

where e is the system error, r is the system input, y is the system output, and u is the control effort.

The control objectives (e.g., reference tracking and optimal control) could implicitly be decided by choosing the adequate $ES$. For example, it is possible to choose the $ES$ for achieving a better reference tracking performance, for reducing the overshoot, and/or for the energy expense minimization, among  others.

Aiming at designing a model-free distributed intelligent control for flocking control of multi-unmanned aircraft systems, the proposed biologically-inspired technique will focus on improving: (i) reference tracking performance, (ii) network-induced delay handling, and (iii) disturbance rejection.

To accomplish these objectives, for each of the control inputs (i.e., {$u_1$,...,$u_n$}), the $S{I}_i$ and $E{S}_i$, will be designed as:

$$\begin{array}{c}\hfill S{I}_i=K_{SI}^{\alpha }{u}_i^{\alpha }+K_{SI}^{\beta }{u}_i^{\beta }+K_{SI}^{\gamma }{u}_i^{\gamma },\end{array}$$

(15)

$$\begin{array}{c}\hfill E{S}_i=K_{ES}^{\alpha }{u}_i^{\alpha }+K_{ES}^{\beta }{u}_i^{\beta }+K_{ES}^{\gamma }{u}_i^{\gamma },\end{array}$$

(16)

where $i=\{1,\dots n\}$, $K_{SI}^{\alpha }$, $K_{SI}^{\beta }$, $K_{SI}^{\gamma }$, $K_{ES}^{\alpha }$, $K_{ES}^{\beta }$, and $K_{ES}^{\gamma }$ are positive gains. The $ES$ will change its impact on the system behavior by assigning different values to these positive gains. In this work, different gains are assigned for each one of the control inputs (i.e., $u_i$, $i=1,\dots ,n$) of the system.

It should be mentioned that we designed the $ES$ in such a way that the increase in reference tracking error will generate a negative emotion in the system, which is then taken as evidence for the unsatisfactory performance of the system. Therefore, the proposed controller will behave in such a way that it will always minimize the negative emotion, which leads to the satisfactory performance of the  system.

3.3. Learning-Based Intelligent Flocking Control

In flocking control of networked multi-UAS, multiple performance considerations have to be taken into account all at the same time; therefore, it is a very interesting case for using biologically-inspired learning-based multi-objective methodologies like BELBIC. Designing a model-free distributed intelligent control for flocking of networked multi-UAS by considering the network-induced delay, in addition to designing a suitable controller for real-time implementation, encourages us to take advantage of the computational model of emotional learning in the mammals’ limbic system, i.e., BELBIC.

From Equations (15) and (16), the BELBIC-inspired distributed intelligent flocking control strategy for networked multi-UAS is defined as

$$\begin{array}{cc}\hfill u_i^{BEL}=& {\displaystyle \sum _i}{V}_i·S{I}_i-{\displaystyle \sum _i}{W}_i·S{I}_i\hfill \\ \hfill =& {\displaystyle \sum _i}{V}_i·\left(K_{SI}^{\alpha }·u_i^{\alpha }+K_{SI}^{\beta }·u_i^{\beta }+K_{SI}^{\gamma }·u_i^{\gamma }\right)\hfill \\ & -{\displaystyle \sum _i}{W}_i·\left(K_{SI}^{\alpha }·u_i^{\alpha }+K_{SI}^{\beta }·u_i^{\beta }+K_{SI}^{\gamma }·u_i^{\gamma }\right).\hfill \end{array}$$

(17)

Here, $i=1,\dots ,n$ makes reference to each control input and n is the number of UASs. Considering the results obtained in Theorem 1 and by substituting the Emotional Signal with Equation (16), the BELBIC model output of the distributed intelligent control for flocking of networked multi-UAS could be obtained as follows:

$$\begin{array}{c}\hfill MO=E{S}_l=K_{ES}^{\alpha }{u}_i^{\alpha }+K_{ES}^{\beta }{u}_i^{\beta }+K_{ES}^{\gamma },u_i^{\gamma },\end{array}$$

(18)

which clearly satisfies our goal of distributed intelligent flocking control. In other words, the model output accomplished all the flocking properties i.e., Collision Avoidance, Obstacle Avoidance, and Navigational Feedback.

The overall model-free biologically-inspired distributed intelligent flocking control methodology proposed in this paper is summarized as a pseudo-code in Algorithm 1.

Algorithm 1 : The BELBIC-inspired methodology for distributed intelligent flocking control of networked multi-UAS.

Initialization:

Set $W_i=0$, $V_{th}=0$, and $V_i=0$, for $i=1,\dots ,n$.

Define $\tau =$ network-induced delay.

Define $E{S}_i=$ Objective function, for $i=1,\dots ,n$.

for each iteration $t=t_s$ do

for each agent i do

Compute $S{I}_i=K_{SI}^{\alpha }{u}_i^{\alpha }+K_{SI}^{\beta }{u}_i^{\beta }+K_{SI}^{\gamma }{u}_i^{\gamma }$

Compute $E{S}_i=K_{ES}^{\alpha }{u}_i^{\alpha }+K_{ES}^{\beta }{u}_i^{\beta }+K_{ES}^{\gamma }{u}_i^{\gamma }$

Compute $A_i=V_{i}S{I}_i$

Compute $A_{th}=V_{th}max\left(S{I}_i\right)$

Compute $O{C}_i=W_{i}S{I}_i$

Compute $BE{L}_i={\sum }_i{A}_i-{\sum }_{i}O{C}_i$

Update $V_i$

Update $V_{th}$

Update $W_i$

end for

end for

3.4. Stability Analysis

Theorem 1 is presenting the convergence of the weights of the Amygdala ($V_l$) and the Orbitofrontal Cortex ($W_l$). Theorem 2 is providing the closed-loop stability of the proposed controller and Remark 2 explains how the proposed method converges to distributed intelligent flocking control of networked multi-UAS.

Theorem 1.

Given the BELBIC design as Equations (15)–(18), there exists the positive BELBIC tuning parameters, $K_v$, $K_w$ satisfying

I. 

$\left|\left[1-K_v{\left(S{I}_l\right)}^2\right]\right|<1,$

II. 

$\left|\left[1-K_w{\left(S{I}_l\right)}^2\right]\right|<1,$

such that the BELBIC estimated weights of the Amygdala ($V_l$) and the Orbitofrontal Cortex ($W_l$) converge to the desired targets asymptotically.

Proof. 

See Appendix A. ☐

Theorem 2.

(Closed-loop Stability): Given that the initial networked multi-UAS state $x\left(0\right)$ and the BELBIC estimated weights of the Amygdala ($V_l\left(0\right)$) and the Orbitofrontal Cortex ($W_l\left(0\right)$) are bounded in the set Λ, let the BELBIC be tuned and estimated control policy be given as Equations (10), (11) and (17), respectively. Then, there exist positive constants, $K_v$, $K_w$, satisfying Theorem 1 such that networked multi-UAS state, $x\left(t\right)$ and BELBIC weight estimation errors are all asymptotically stable.

Proof. 

See Appendix B. ☐

Remark 2.

Based on the BELBIC theory [26] and (Equation (17)), the distributed intelligent flocking control of networked multi-UAS can be obtained while the estimated weights of the Amygdala ($V_l$) and the Orbitofrontal Cortex ($W_l$) are converging to desired targets. According to Theorem 1, estimated weights converge to desired targets asymptotically. Therefore, the designed BELBIC input $u_i^{BEL}$ (i.e., Equation (17)) converges to distributed intelligent flocking control of networked multi-UAS asymptotically.

4. Simulation Results

This section presents computer-based simulation results showing the performance of the proposed biologically-inspired distributed intelligent flocking control of multi-unmanned aircraft systems (Section 4.2) and multi-unmanned ground systems (Section 4.1) in an obstacle-free environment. A total of 50 unmanned aircraft and 150 unmanned ground vehicles (UGVs) where employed, with initial velocities equal to zero, and positions randomly distributed in a squared area. The following parameters are used through the simulation: $d_{\alpha }=7$, $r=1.2d_{\alpha }$, $d^{\prime }=0.6d_{\alpha }$, $r^{\prime }=1.2d^{\prime }$. For the $\sigma$-norm, the parameter $ϵ=0.1$, for $\varphi \left(z\right)$ the parameters $a=b=5$, for the bump functions ${\varphi }_{\alpha }\left(z\right)$ and ${\varphi }_{\beta }\left(z\right)$, $h=0.2$ and $h=0.9$, respectively. The same network-induced delays (i.e., ${\tau }_{i,j}=\tau =0.3$, $\forall i$) are considered for all unmanned aircrafts and unmanned ground vehicles. All simulations are carried out on a platform with following specifications: Windows Server 2012 R2 standard, Processor: Intel(R) Xeon(R) CPU E5-2680 0 @ 2.70 GHz (4 processors), RAM: 8.00 GB.

4.1. Flocking of UGVs in an Obstacle-Free Environment

Figure 3 shows two snapshots of the simulation of the multi-unmanned ground systems in the obstacle-free environment. Figure 3 Left shows the 150 UGVs in their initial positions at $t=0$ s while Figure 3 Right shows the UGVs at $t=40$ s where they are flocking and have successfully formed a connected network.

For comparison purposes, two similar experiments were performed, but using the flocking algorithm introduced in [3], and Multirobot Cooperative Learning for Predator Avoidance (MCLPA) flocking algorithm introduced in [36] instead of the proposed algorithm. Figure 4 shows the velocities of all UGVs on the x- and y-axis for all methods in an obstacle-free environment under the influence of the networked-induced delay. The plot shows that, although the delay is deterministic, it is unknown and the proposed method could effectively handle the influences of the network-induced delay.

Figure 5 shows the Mean Square Error (MSE) $\frac{1}{n}{\sum }_{i=1}^n{\left(p_i-p_d\right)}^2$ of the velocities of the overall group of agents, for the flocking methods in [3], the MCLPA flocking strategy in [36], and the BELBIC-based flocking, when evolving in an obstacle-free environment.

Table 1. Characteristics of the MSE of all flocking strategies for UGVs in the obstacle-free environment.

	Flocking in [3]	MCLPA [36]	BELBIC-Based
Mean Value on the x-axis	0.802	0.641	0.602
Standard Deviation on the x-axis	6.025 × 10−5	5.566 × 10−5	0.814 × 10−5
Mean Value on the y-axis	0.371	0.245	0.176
Standard Deviation on the y-axis	6.869 × 10−4	3.068 × 10−4	1.324 × 10−4

shows the characteristics of the MSE of velocities of all flocking strategies for UGVs in the obstacle-free environment.

4.2. Flocking of UASs in an Obstacle-Free Environment

Figure 6 shows two snapshots of the simulation of the multi-unmanned aircraft systems in the obstacle-free environment. Figure 6 Left shows the 50 UASs in their initial positions at $t=0$ s while Figure 6 Right shows the UASs at $t=40s$ where they are flocking and have successfully formed a connected network.

For comparison purposes, two similar experiments were performed, but using the flocking algorithm introduced in [3], and Multirobot Cooperative Learning for Predator Avoidance (MCLPA) flocking algorithm introduced in [36] instead of the proposed algorithm. Figure 7 shows the velocities of all UASs on the x- and y-axis for all methods in an obstacle-free environment under the influence of the networked-induced delay. The plot shows that, although the delay is deterministic, it is unknown and the proposed method could effectively handle the influences of the network-induced delay.

Figure 8 shows the Mean Square Error $\frac{1}{n}{\sum }_{i=1}^n{\left(p_i-p_d\right)}^2$ of the velocities of the overall group of agents, for the flocking methods in [3], the MCLPA flocking strategy in [36], and the BELBIC-based flocking, when evolving in an obstacle-free environment.

Table 2. Characteristics of the MSE of all flocking strategies for UAVs in the obstacle-free environment.

	Flocking in [3]	MCLPA [36]	BELBIC-Based
Mean Value on the x-axis	0.762	0.581	0.512
Standard Deviation on the x-axis	5.726 × 10−5	5.065 × 10−5	0.764 × 10−5
Mean Value on the y-axis	0.356	0.225	0.147
Standard Deviation on the y-axis	6.439 × 10−4	2.805 × 10−4	1.042 × 10−4

shows the characteristics of the MSE of velocities of all flocking strategies for UAVs in the obstacle-free environment.

5. Conclusions

The challenges of network-induced delay in flocking of networked multi-unmanned aircraft systems have been studied in this paper. A biologically-inspired distributed intelligent control methodology based on the emotional learning phenomenon in the mammalian limbic system has been proposed to overcome these challenges. Considering the influence of network-induced delay in real-time networked multi-unmanned aircraft systems flocking, the proposed technique demonstrated to be a promising tool for practical implementation because of its learning capability and low computational complexity. Computer-based numerical results of the implementation of the proposed methodology show the effectiveness of this algorithm for distributed intelligent flocking control of networked multi-UAS.

Future work will consider the implementation of a biologically-inspired intelligent control strategy for addressing real-world scenarios such as tracking, search and rescue, underwater explorations, etc.