## I Introduction

With recent breakthroughs in design and production of unmanned aerial vehicles (UAVs), UAVs are increasingly used in many applications such as military surveillance [orfanus2016self], disaster response [erdelj2017help], oil gas pipe inspection [hausamann2005monitoring], precision agriculture [tokekar2016sensor], and delivery of goods [murray2015flying]

. The Federal Aviation Administration (FAA) estimates that commercial UAVs will grow 10-fold between 2016 and 2021, and the global market revenue of UAVs will rise to $11.2 billion by 2020

[uav_trend]. One of the major obstacles that constrains the huge potential of UAVs is the limited flight time. This is a significant problem as more and more UAV applications require coverage of a large geographical area, lengthening the mission duration well beyond the battery capacity of UAVs [gong2018flight].Numerous solutions have been developed to extend the flight time of UAVs focusing on utilization of charging stations that recharge/replace the battery of UAVs [swieringa2010autonomous][suzuki2012automatic][lu2018wireless]. While these solutions have successfully increased the operation time of UAVs, there is still a significant knowledge gap on how to deploy charging stations to maximize the system performance. For example, existing approaches are based on a simplifying assumption for deploying charging stations, *e.g.,* deploying them at fixed locations [tseng2017autonomous][yu2018algorithms][wei2018coverage] like the center of a field [trotta2018joint] and the base station [scherer2017short].

In this paper, we address the problem of deploying charging stations that maximizes the system performance specifically concentrating on generating optimal UAV trajectories, while minimizing the number of deployed charging stations. More specifically, we formulate the problem of jointly optimizing UAV Trajectories and Locations of Battery swap Stations (TLBS) as an optimization problem, and develop an optimization framework to solve the problem. Especially, since finding a shortest trajectory for a single UAV visiting each region of interest (ROI) and returning to the base station, without even considering charging stations, is essentially the travelling salesperson problem (TSP) which is NP-Hard, *we propose a heuristic solution called UBAT based on Ant Colony Optimization (ACO) motivated by the fact that ACO is well known to solve TSP very effectively [dorigo1997ant2].* The technical contributions of this paper are that the standard ACO is adapted to account for unique requirements for solving the TLBS

problem especially in calculating the probabilities for path selection and updating the pheromone trails: (1) constrained energy of UAVs, (2) simultaneous coverage of ROIs by multiple UAVs, and (3) joint optimization of UAV trajectories and number of charging stations.

The proposed framework is fine-tuned with novel algorithms that are designed to enhance the convergence speed and the quality of solutions. More specifically, an algorithm is developed to minimize the number of candidate locations for deploying charging stations, thereby reducing the search space and lowering the computational overhead. We also design an algorithm that effectively tunes the parameters for the proposed framework to enhance the quality of solutions. Additionally, the 2-OPT local search [voudouris1999guided] is effectively incorporated to further improve the quality of solutions.

Extensive simulations are conducted to evaluate the performance of UBAT as well as the supporting algorithms. The results demonstrate that the proposed approach effectively performs joint optimization of UAV trajectories and placement of charging stations that are within 8.3% and 7.3% of the true optimal solutions, respectively.

## Ii TLBS Problem

### Ii-a System Model

Consider a target area with an arbitrary shape in . The target area is divided into a two-dimensional grid of square cells, *i.e.,* the target area is represented as a set where each element is a subarea (cell), and is the number of subareas. Among the cells are regions of interests (ROIs) denoted by a set , where is the number of ROIs. We introduce the function which defines the Euclidean distance between two cells. More specifically, means the Euclidean distance between the centers of two cells and .

There are UAVs denoted by a set . Assume a typical UAV application where UAVs fly over a target area, perform sensing, and transmit sensed data to the base station. We assume that UAVs fly meters high from the ground. As such, the sensing coverage is where is the angle of the sensing cone. The sensing coverage is sufficiently large to cover a whole ROI when a UAV is above the center of the ROI. Figure 1 illustrates a target area divided into cells, three ROIs in the target area, and the sensing coverage. The time is discretized into time slots where the length of a time slot is denoted by . In particular, let be the time slot when all ROIs have been fully covered.

The flight time of a UAV is limited due to its energy capacity. To support continuous operation of UAVs, charging stations are deployed in the target area which are denoted by a set . More precisely, we assume that these charging stations automatically replace the battery of a UAV when it is landed on the charging station.

Let be the maximum energy capacity of a UAV. Also, let be the remaining energy of a UAV at time slot . Assume that all UAVs are fully charged before operation, *i.e.,* . We define be the energy loss rate while a UAV is flying. Each UAV is in one of these states at each time slot. The state indicates that a UAV is flying, and the state indicates that a UAV is landed on a charging station to get its battery replaced.

A function is defined to specify whether a UAV is flying over a certain cell at a certain time slot. For example, if UAV is flying over cell at time slot , and if it is landed on a charging station located in a cell . The residual energy of a UAV at time slot , *i.e.,* can then be defined as follows.

(1) | ||||

Here, is the residual energy at the previous time slot. represents the energy loss for flying during the time slot, and indicates replacement of the battery during the time slot.

We introduce the function indicating whether the UAV is located in a certain cell at a certain time slot. For example, means that UAV is in a cell at time slot , and otherwise . We also define the function indicating that a charging station is deployed in a certain cell. More precisely, if a charging station is in the cell , otherwise . Let us define another function which indicates the Euclidian distance moved during a time slot. For example means the distance moved by UAV during time slot . Let be the maximum speed of the UAV.

### Ii-B Problem Formulation

Having defined all notations and functions, the problem of jointly optimizing UAV Trajectories and Locations of Battery swap Stations (TLBS) is formulated as follows.

(2) |

(3) |

(4) |

(5) |

(6) |

(7) | |||

Equation 1 | (8) |

Constraint 3 asserts that each UAV can be located in only one cell at each time slot. Constraint 4 means that each ROI is covered by at least one UAV when the UAV operation is finished. Constraint 5 states that each UAV does not deplete its energy completely during operation, and the residual energy should not be greater than the maximum energy capacity. Constraint 6 assures that each UAV does not move faster than the maximum speed . Constraint 7 means that if a UAV is not flying, *i.e.,* landed in a cell for recharging, there should be a charging station in that cell. Finally, Constraint 8 specifies how the residual energy of each UAV is updated at each time slot.

The TLBS problem is NP-Hard because it contains, as a simplified instance, the traveling salesman problem (TSP) [dorigo1997ant]. As such, we develop a heuristic framework based on the ant colony optimization (ACO) especially motivated by the fact that ACO is particularly effective in solving the TSP problem [dorigo1997ant2].

## Iii Proposed Approach

This section explains how ACO is adapted to solve the TLBS problem satisfying all constraints of the problem. We omit the background on ACO due to the limited page count. Readers are referred to [dorigo2010ant] for a complete review of ACO.

### Iii-a Path Selection

We represent UAVs and ROIs as ants and food sources, respectively. For simplicity, we assume that UAVs fly at a speed of (Constraint 6

), which can be easily replaced by a vector to represent the time-varying speed. We follow the energy model defined in Eq.

1 to respect Constraint 8. Compared to ACO with ants, we leverage two advantages of UAVs. First, UAVs have memory; they can keep track of visited ROIs so that they will visit only ROIs that have not been covered by any other UAV. Second, UAVs have computational capabilities; they can compute the distance between two ROIs. Based on these advantages, the probability that a UAV in cell a selects a ROI to visit is defined as follows.(9) |

Here is the set of ROIs that have not been visited by any UAV (Constraint 4). is the pheromone intensity of an edge connecting two cells and . is the inverse of the Euclidean distance between two cells and namely , which is used to ensure that a shorter edge is selected in finding a path. Two parameters and are introduced to adjust the influence of the pheromone and the distance between two cells, respectively. After calculating the probabilities for all ROIs, a UAV selects a ROI with the highest probability and moves to the selected ROI.

If a UAV does not have enough energy to reach the selected ROI, the state of the UAV is changed to the charging mode, and the UAV finds a set of cells that it can reach with its residual energy (Constraint 5). A cell is selected from those candidate cells using a slightly modified probability model compared to Eq. 9, and then the UAV moves to the selected cell in which a charging station is placed (Constraint 7). Specifically for the modified probability model, is set to the Euclidean distance between two ROIs (i.e., ), instead of the inverse of it as specified in Eq. 9, to minimize the number of deployed charging stations by assuring that a longer edge is preferred in the charging mode, *i.e.,* we allow UAVs to fly as far as possible with the given residual energy because the battery will be replaced anyways.

### Iii-B Trail Update

The pheromone trails are updated in two ways: evaporation and reinforcement. The former mimics the natural evaporation of pheromone. More specifically, pheromone between two cells and are reduced if based on the formula: , where is a parameter that controls the speed of pheromone reduction, and is the initial pheromone. Evaporation is particularly useful for preventing one path from dominating other possible solutions. Due to evaporation, other paths are explored, potentially leading to a better solution.

Once a path is discovered, pheromone trails on that path are strengthened, which is called the reinforcement. The reinforcement process is used to encourage the use of shorter paths and to increase the chance of using the edges of the currently known shortest path in finding potentially better paths subsequently. Formally, the pheromone trail of each edge of the selected path is reinforced as follows: , where is the length of path ; is a system parameter – Details on determining an appropriate value for the parameter is discussed in Section V-B.

### Iii-C Multiple UAVs with Minimum Number of Charging Stations

We now extend the algorithm to account for general scenarios where multiple UAVs collaborate to cover ROIs with an objective of minimizing the number of deployed charging stations. More specifically, each UAV visits a portion of ROIs and ends up with having an individual path denoted by . Denote the length of path for UAV be . Then, to minimize , we try to minimize the maximum path length of all UAVs, *i.e.,* , rather than minimizing . Consequently, the pheromone updating formula is rewritten as follows: .

We then take into account the number of deployed charging stations in the pheromone updating rule for reinforcement. If we do not constrain the number of charging stations in finding a path, it may increase arbitrarily. To this end, UBAT tries to make as many of the charging stations be shared by multiple UAVs such that the path length is also optimized. The idea to implement this is to include the number of deployed charging stations in the pheromone updating formula. More specifically, we aim to find cells to deploy charging stations such that both and the number of deployed charging stations are minimized, as specified in the objective function (Eq. 2). In order to encourage the use of a path that minimizes both and , we can rewrite the pheromone updating formula as follows: .

Algorithm 1 summarizes the operation of UBAT. Starting from the current cell , each UAV selects the next cell to visit using Eq. 9 (Line 6). If the selected cell is reachable with the remaining energy , the edge connecting and , namely is added to the path of UAV (Lines 7-9). On the other hand, if is not reachable, the algorithm locates a cell to place a charging station based on the modified version of the probability model Eq. 9 (Lines 11-14). More precisely, a charging station is selected such that both and are minimized, and the path segment connecting cells and becomes part of . Since a new charging station has been placed in cell , the number of charging stations is incremented by one. This process is repeated until all ROIs have been visited by UAVs, and as a result, we obtain the trajectories for all UAVs and the cells that have charging stations. Given the discovered paths , the pheromone trails are updated based on evaporation and reinforcement (Lines 15-16). This whole process is repeated until we reach (Line 3).

## Iv Enhancement Techniques

Reducing Search Space: We develop an algorithm to improve the execution time of UBAT by reducing the overhead for finding a set of candidate cells for deploying a charging station (Lines 10-14 of Algorithm 1). The key idea is to make the search space smaller based on the observation that a charging station can be deployed only within the convex hull of ROIs. Consequently, only the cells within the convex hull are considered in finding .

We can easily prove that deploying charging stations only inside the convex hull of ROIs does not influence the quality of the solution. Assume in contradiction that a charging station is placed outside the convex hull, *e.g.,* in Figure 3 to allow the UAV to move from to . We can always find such that . Furthermore, is a better choice because it is closer to other ROIs considering that it may be used by other UAVs.

Tuning Parameters: Another key factor that affects the performance of UBAT is the parameters and . If we choose too large values for and , *i.e.,* , the pheromone intensity of a new path would become too strong. As a result, it will take long for the pheromone of a newly found path to evaporate down to the level that other potential paths can be explored. On the other hand, if the values for the parameters are too small, *i.e.,* , the pheromone intensity of a new path would become too small leading to the situation where a potentially good solution is dropped too early, and as such, UBAT misses a chance to validate that it was actually a good solution.

We aim to find adequate values for and such that . To this end, we adopt an iterative approach. More specifically, we set to 1 and start UBAT with arbitrary values for and . After only a few iterations, we get intermediate solutions and for and , respectively. We use these intermediate solutions as new values for the parameters (*i.e.,* ), and then, we repeat the same process to obtain new values for the parameters. Through experiments in Section V-B, we validate that this approach is quick and effective requiring only one or two iterations for finding adequate values of the parameters.

Improving paths: To make UBAT run faster and obtain a higher quality solution, we perform correction on a path found in each iteration by applying the 2-OPT local search algorithm [voudouris1999guided]. The algorithm was first proposed by Croes for improving the quality of a solution for TSP [croes1958method]. The idea is to detect a path that crosses over itself and perform reordering of cells so that the cross over is removed. More precisely, two edges that cross each other are removed from a path, and then the resulting two path segments after removal of the edges are reconnected such that the cross over no longer exists. For example in Figure 3, an edge is replaced by ; and an edge is replaced by , resulting in a shorter path with no cross over. Unlike the traditional 2-OPT local search, however, we ensure that the output of the algorithm does not result in increased path length for other UAVs and deployment of additional charging stations.

## V Evaluation

We implemented UBAT using C++ on a PC equipped with a 2.5GHz dual-core Intel processor, and 16GByte RAM. A field with dimensions of 2020km was created which was divided into cells with dimensions of 11km, *i.e.,* 400 cells in the field. Random scenarios were created by selecting 10 ROIs randomly. UAVs started operation from one of the randomly selected ROIs. We conducted simulations with different numbers of UAVs. The batteries of all UAVs were fully charged. The maximum distance that a fully charged UAV can fly was set to 5km. The sensing range of a UAV was configured to fully cover a cell when it hovers over the cell. The parameters for ACO were set as , and . The max iteration count was set to 30,000. The main metrics measured were (1) the length (km) of the longest path of all UAVs, *i.e.,* (which will be referred to as ‘path length’ hereafter), and (2) the number of deployed charging stations (CS). The path length and number of CS of UBAT were compared with the true optimal solutions. The effectiveness of the proposed enhancement techniques was also evaluated.

### V-a Comparison with Exact Algorithm

The performance of UBAT was compared with the exact algorithm which finds the solution using the brute force search. A challenge for running the exact algorithm is the huge computation time. Fortunately, the true optimal solutions for a single-UAV scenario can be easily and quickly calculated [liao2016electric]. Specifically, the optimal path of a single UAV is simply the output of TSP with a set of ROIs as input, reducing the search space from 400 (number of cells) to 10 (number of ROIs). Calculating path lengths for all possible permutations for 10 ROIs completes in a few seconds. Given the shortest path, the minimum number of CS can be computed in linear time by deploying a CS at a point on the shortest path that is farthest from the current position of a fully charged UAV [liao2016electric].

Figures 5 and 5 show the path length and number of CS for UBAT and the exact algorithm. On average, the path length of UBAT was 6.3% greater with STDEV of 2.6% compared with the true optimal solutions, and the number of CS was 4.3% greater with STDEV of 6.0%. Putting in different terms, UBAT had a 3.1km longer path and required 0.5 additional charging stations on average in comparison with the true optimal solutions under single-UAV scenarios.

We then measured the path length and the number of CS for multi-UAV scenarios. Unlike single-UAV scenarios, however, we cannot simply apply the TSP algorithm to find the true optimal solution for multi-UAV scenarios resulting in huge computation time. A quick, yet effective solution to this challenge is to test the algorithms with ‘semi’ random scenarios. Specifically, 5 ROIs are randomly selected from the top of the field, and the other 5 ROIs are randomly selected from the bottom of the field such that these two sets of ROIs are individually covered by each UAV. This way the 2-UAV scenarios can be divided into two separate single-UAV scenarios that allows us to apply the TSP algorithm to obtain the true optimal solutions quickly, while UBAT is not aware of this configuration and running normally. A similar simulation setting was used for 4-UAV scenarios, *i.e.,* we deployed 10 ROIs randomly in NE, NW, SE, SW regions of the field such that ROIs in each region can be independently covered by each UAV. We leave the work of obtaining the true optimal solutions for more complicated scenarios with a large number of UAVs using a high performance computing system as our future work.

Figures 7 and 7 show the impressive performance of UBAT for 2-UAV scenarios. The average path length of UBAT was 2.2% greater with STDEV of 1.1% in comparison with the exact algorithm. The average number of CS was 7.3% greater with STDEV of 8.4% compared with the exact algorithm. The larger percentage difference for the number of CS compared with the results for the single-UAV scenarios can be attributed to the fact that fewer charging stations were deployed for the 2-UAV scenarios because only a half of the field was covered by a UAV. Also, even though the percentage difference for the number of CS may seem much greater than that for the path length, it means only 0.8 additional charging stations on average in comparison with the true optimal solutions.

Similarly, Figures 9 and 9 show the results for the 4-UAV scenarios. As shown, the average path length and the number of CS for the proposed framework were 8.3% (STDEV 2.8%) and 6.7% (STDEV of 3.6%) greater in comparison with the true optimal solutions, respectively. In other words, the average path length of UBAT was 1.3km longer with 0.8 additional charging stations compared with the true optimal solutions. Overall, the simulation results demonstrate that UBAT produces very high-quality solutions with slightly longer path length and less than 1 more charging stations compared with the true optimal solutions in our simulation environments.

### V-B Effect of Enhancement Techniques

Effect of Max Iterations: We investigate the effect of the maximum iteration count (the default is set to 30,000). It is an interesting analysis because the longer we run UBAT, the solution keeps improving. Specifically, we increased the max iteration count to 400,000 while fixing other settings for a 2-UAV scenario. We observed the following from the results (Figure 11): (1) a sufficiently good solution is obtained within the first few iterations; (2) after that, the solution keeps improving but very slightly. In this particular example, the path length was significantly decreased in the first 750 iterations, and it was reduced only by 0.9% between 750th and 68,432th iterations (and virtually no improvement after that); similarly the number of CS stayed the same after the first few iterations, although in some cases, we observed that the number of CS was decreased after a very large number of iterations at the cost of the increased path length. Overall, while the solution keeps improving as we increase the maximum iteration count, the advantage is not obvious considering that major progress is made in the first few iterations and using a large maximum iteration count significantly increases the execution time.

Effect of Search Space Reduction: To understand the effect of the search space reduction algorithm, we measured the execution time (*i.e.,*

the time it takes to run a single iteration for UBAT) with and without applying the algorithm under the 2-UAV scenarios. The cumulative distribution function (CDF) of the execution time is shown in Figure

11. The results show that the average time for running a single iteration without the algorithm was 0.44 seconds, and it was decreased to 0.33 seconds when the search space reduction algorithm was used, indicating 25% faster execution time.Effect of Parameter Tuning: We evaluate the performance of the parameter tuning algorithm. We started with some arbitrary values for the parameters and . We then executed UBAT only for a few iterations, *i.e.*, 1,000 iterations. As observed in the experiments for the max iteration count, a sufficiently good solution is obtained in the first few iterations. The obtained intermediate solution, *i.e.,* the path length, and the number of CS were used as new values for parameters , and , respectively. We then ran UBAT with the new parameters and compared the results with those obtained with arbitrarily selected parameters. Figure 13 displays the results. As shown, the path length and the number of CS were improved by up to 24.5% and 21.4% (10.5% and 6.3% on average), respectively, when the parameter tuning algorithm was applied.

We then evaluate the performance gain when the parameter tuning algorithm is applied repeatedly. An interesting observation was that the performance gain was very marginal even from the second time running the parameter tuning algorithm. Specifically, the path length and the number of CS were improved only by up to 0.3% on average. In the third time for tuning the parameters, virtually no improvement was observed, indicating that running the parameter tuning process once or twice suffices to obtain good parameters. Note that all simulation results were obtained using this parameter tuning method, showing the effectiveness and practicality of the proposed approach.

Effect of Path Correction: We analyze the effect of the path improvement algorithm. We measured the path length with and without applying the algorithm, while fixing all other settings for the 2-UAV scenarios. It should be noted that the number of CS was not measured because the path improvement algorithm does not increase the number of CS at the cost of improving the path. The results are depicted in Figure 13. The path length was decreased by 2.5% on average with STDEV of 1.6% when the path improvement algorithm was applied. The results indicate that by skipping low-quality solutions (*e.g.,* paths with self-crossings), the search space is explored more effectively, leading to better solutions.

## Vi Conclusion

We have presented UBAT, an optimization framework that jointly optimizes UAV trajectories and locations of charging stations. Enhancement techniques are developed to improve the convergence time and quality of solutions. Through extensive simulations, we demonstrate that UBAT effectively calculates the trajectories and placement of charging stations. The results of this research are expected to benefit and facilitate numerous UAV applications that require coverage of a large geographic area. We also expect that various techniques developed for UBAT will be useful resources for development of new solutions for sustainable UAV operation.

Comments

There are no comments yet.