Personal Sign In. For IEEE to continue sending you helpful information on our products and services, please consent to our updated Privacy Policy. Email Address. Sign In. Access provided by: anon Sign Out. This assessment is carried out according to the following steps: a run hundreds of executions or thousands, if more time is available , where each execution implies the generation of random values for each random demand according to the associated probability distribution; b assess the performance of each solution by estimating the TEC as the average of the total costs obtained at each execution; and c use the simulation feedback to better guide the searching process inside the metaheuristic.

### Reinforcement Learning for Solving the Vehicle Routing Problem

The correlation is expected to be stronger or more significant as the demand variances tend to zero. A safety stock is a certain amount of the vehicle capacity that is not considered while designing the routes. Then, if the final routes' demands surpass their expected values, this stock can be employed to try to satisfy them. Thus, the aim of considering safety stocks is to reduce the probability of a route failure.

The flowchart diagram of our approach is depicted in Fig. Each customer i has associated a demand that follows a known probability distribution with an existing mean. Determine a set K of percentages, where each element k is the percentage of the vehicle capacity W that can be used during the route design phase; in other words, represents a fixed level of safety stock. For each of these elements, follow the steps 3—9. In the case of the stochastic problem, there is also a variable cost that depends on the corrective actions undertaken.

### Original Research ARTICLE

Use MCS to estimate the expected cost due to corrective actions for each route j of the aprioristic solution, , where , being o the number of routes. Then, aggregate the TEC for all routes,. In this phase, a short simulation i.

- Introduction to Topology, Differential Geometry and Group Theory for Physicists.
- A GRASP for the Vehicle Routing Problem with Time Windows | ORSA Journal on Computing.
- Power Laws, Scale-Free Networks and Genome Biology (Molecular Biology Intelligence Unit).
- Vehicle Routing Problem | OR-Tools | Google Developers.
- How to Solve Vehicle Routing Problems: Route Optimization Software and their APIs | AltexSoft?
- I Capacitated Vehicle Routing Problem.

Then, the TEC of the solution is calculated as follows:. Set a base solution as the initial solution. Employ an ILS metaheuristic framework, which starts an improvement process that will continue until a stopping condition, based on time or a fixed number of iterations, is reached. At each iteration, the following steps are implemented. First, a perturbation is applied to the base solution to generate a new one, which is then improved by means of a local search.

If the FC of the new solution is lower than the FC of the current base solution, then the list of the best deterministic solutions BDSs is updated only if it is not full or if the worst solution has a higher cost, then a swap is performed and the TEC of the new solution is estimated with a short simulation. Otherwise, an acceptance criterion is used to decide whether the base solution is replaced by the new one, which has a worse performance.

Before that, if the FC of the new solution is higher, then that solution is discarded. Try to improve all solutions in the list with an intensive routing algorithm. Use a long simulation i. Large samples are required to obtain estimates with small confidence intervals. Finally, return the top BSSs considering all solutions found with the different values in K , and the corresponding samples they will be used for completing a risk analysis. Some key issues of our approach should be explained in detail. Initially, a customer—depot assignment map is set, and then the CWS heuristic is applied to obtain a fast routing plan.

The first step is performed by computing a priority list of potentially eligible customers for each depot. Then, two policies are iteratively considered to assign customers to depots: the first allows the depot with the most remaining serving capacity to choose the next customer from its priority list, while the second employs a round robin tournament criterion to select which depot chooses next. This criterion consists in following a given sequence of depots, which has been randomly generated and is repeatedly applied—as long as the corresponding depot has enough capacity to satisfy the demand required.

The second step consists in solving each resulting CVRP independently by employing the savings heuristic. Regarding safety stocks, it is expected that lower values of k will provide more reliable routes i. However, a high FC will result too, since more vehicles will be needed to cover all the customers' demands.

## Vehicle routing problem — LocalSolver documentation

In the worst case, the problem instance could become unsolvable. On the other hand, a high value of k is related to a lower FC but a higher variable cost due to the elevated risk of having to return to the depot to reload. The cost due to corrective actions step 5 is computed as follows. In case of route failure, it includes the cost of returning to the depot first and then to the customer being served.

Here, we consider that the vehicle has to return to the same depot where its route starts and ends. It is assumed that the vehicle delivers all the remaining stock before going back to reload. A restocking is carried out when the expected demand of the next customer is higher than the current remaining stock.

## A GRASP for the Vehicle Routing Problem with Time Windows

The cost of this strategy incorporates the costs on the edges that link a customer with the depot and the depot with the next customer minus the cost of the edge linking both customers. Other restocking policies could be tested. The perturbation operator step 7 employed in the iterative part of the metaheuristic framework modifies the current solution by reallocating the customers assigned to a given percentage p of depots randomly selected, considering the remaining capacity of all the depots, and following the allocation procedure employed to generate new solutions.

Then, the savings heuristic is applied again to design the routes. It allows the base solution to be replaced by a new solution with a greater TEC with a probability equal to , where rpd is the relative percentage difference between both solutions and is computed as follows: TEC of minus that of , divided by this last value and multiplied by The reason to implement this acceptance criterion is to reduce the risk of getting trapped in a local optimum. This algorithm is based on a randomized version of the savings heuristic that employs a geometric distribution to guide the random search, and a cache and splitting techniques to make it more efficient.

This algorithm has been adapted for the stochastic solutions. The set of samples will allow us to compare the solutions not only focusing on the TEC, but also on the distribution of the total cost. The algorithm described in the previous section has been implemented as a Java application. The best known solutions BKSs have been extracted from these works. It includes the instance name, the number of customers, the maximum number of vehicles per depot, the number of depots, the maximum route length allowed, and the vehicles' maximum capacity.

The demand of each customer has been considered as a random variable following a lognormal distribution with mean and variance. Three different scenarios have been considered, each one with a, respectively, different variance: 0. In order to choose the percentage of the vehicle capacity in the route design phase k , equally spaced values varying from 0. This allows us to compare them in terms of performance.

In particular, the following frameworks have been considered: An MS framework: The base solution and, as a consequence, the perturbation procedure is erased, so that a new solution is created in each iteration of the loop. A VNS framework: While in the ILS the percentage of destruction is fixed, here it is progressively increased thus, exploring a variable neighborhood until arriving to a maximum value before resetting that percentage to its minimum value. The number of iterations is set to , The number of seeds is set to 10, and only the best result is stored note that these runs can be executed in parallel, and thus a decision maker is mainly interested in the best solution.

Concerning the number of iterations in each simulation, we have employed runs observations for the short simulations this allows us to obtain rough estimates of the stochastic costs in a reasonable amount of computing time and runs for the long simulations which allows us to obtain more accurate estimates of the stochastic costs for each of the returned solutions.

The selection of these values, as well as of the number of solutions stored in the list of top solutions four in our experiments , is mainly driven by the total computing time available. Biased randomization techniques are used in the generation and repair of solutions, and in the intensive routing algorithm.

These techniques rely on two geometric distributions one for mapping and one for routing and, therefore, they require distribution parameters: and , respectively. Additionally, there is the parameter p used in the perturbation procedure. Each of these tables represents a specific scenario.

The last column is the gap between the TECs of both solutions. There is a case in which the gap reaches the instance p 08 with high variance. The reason is that the deterministic solution is not balanced, and a high variance results in an increasing of the TEC. The vehicle capacity is The numbers in the nodes reveal the expected customer demands, while the numbers in the center of each route are the total demands.

The gaps related to the TECs are higher, on average, and get higher as the variability increases. This is due to the fact that those costs are the sum of both the variable and FCs. It is illustrated on a specific case, the instance p 09 with high variance.

Thus, Fig.

- The vehicle routing problem: An overview of exact and approximate algorithms?
- Capacity Constraints | OR-Tools | Google Developers.
- Vehicle Routing.

In Fig. It can be stated that the variability of total costs associated to the BDS is the highest Fig. In other words, the probability of having a total cost equal to or lower than a given value is usually higher with this solution. Nevertheless, the minimum values are provided by the deterministic solution, which makes sense since this solution will be the one selected in scenarios where the customer demands are similar to the corresponding mean of the distributions.

Since the normality assumption is not fully satisfied, we apply the Kruskal—Wallis test by ranks.

The internet development access is really very fast and change all aspects for life activities include buying and selling transactions of goods or services that can arrange online or also called e-commerce which courier service influence. The courier basic operational is logistics of a supply chain. The purpose of this study is to find out what kind of vehicle routing problem VRP is used for courier service so can be used as reference for further research. Collect and selection process found 40 science journals for analyse.

There has been a lot of research about the shortest route problem for courier service or can also called city logistics with VRP which the optimum solution obtained with heuristic and metaheuristic algorithm.