Author(s)

Liqiang Wu ¹

Affiliation(s)

¹ Department of Industry Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China,

Corresponding Author

Liqiang Wu

ABSTRACT

Managing real-time road conditions and satisfying stochastic customer demands pose significant challenges for optimizing the Dynamic Vehicle Routing Problem with Stochastic Requests (DVRPSR) in urban grocery delivery settings. Most existing approaches generate solutions offline as static plans, which are only applicable to the specific scenarios they were optimized for, making it difficult to efficiently plan and operate a dynamic urban grocery delivery system for last-mile delivery. In this study, we introduce a new dynamic optimization model for DVRPSR. Our approach combines a multi-attention mechanism with reinforcement learning and incorporates a customer point update strategy to enhance efficiency in urban E-grocery delivery. To validate the effectiveness of our method, we conducted experiments across small (50 customers and 5 vehicles), medium (100 customers and 10 vehicles), and large (200 customers and 20 vehicles) data scales. The results demonstrate that our method outperforms current routing methods, reducing total path length, improving customer service coverage, and maintaining efficient computation time. This provides a promising strategy for enhancing the efficiency of urban E-grocery delivery and reducing operational costs.

KEYWORDS

Dynamic Vehicle Routing Problem; Reinforcement Learning; Stochastic Optimization; Urban E-grocery Deliveries

CITE THIS PAPER

Liqiang Wu, Enhancing Urban E-grocery Delivery Efficiency: Adaptive Reinforcement Learning for Dynamic Vehicle Routing. Industrial Engineering and Innovation Management (2025) Vol. 8: 120-134. DOI: http://dx.doi.org/10.23977/ieim.2025.080217.

REFERENCES

[1] Rahmanifar, G., M. Mohammadi, A. Sherafat, M. Hajiaghaei-Keshteli, G. Fusco, C. Colombaroni. Heuristic approaches to address vehicle routing problem in the Iot-based waste management system. Expert Systems with Applications. 2023. 220:119708.
[2] Mo, P., Y. Yao, A. D’Ariano, Z. Liu. The vehicle routing problem with underground logistics: Formulation and algorithm. Transportation Research Part E: Logistics and Transportation Review. 2023. 179:103286.
[3] Liu, M., Q. Zhao, Q. Song, Y. Zhang. A Hybrid Brain Storm Optimization Algorithm for Dynamic Vehicle Routing Problem With Time Windows. IEEE Access. 2023. 11:121087-121095.
[4] Wang, S., W. Sun, M. Huang. An adaptive large neighborhood search for the multi-depot dynamic vehicle routing problem with time windows. Computers & Industrial Engineering. 2024. 191:110122.
[5] Teng, Y., J. Chen, S. Zhang, J. Wang, Z. Zhang. Solving dynamic vehicle routing problem with time windows by ant colony system with bipartite graph matching. Egyptian Informatics Journal. 2024. 25:100421.
[6] Liu, T., B. Zou, M. He, Y. Hu, Y. Dou, T. Cui, P. Tan, S. Li, S. Rao, Y. Huang, S. Liu, K. Cai, D. Wang. LncReader: identification of dual functional long noncoding RNAs using a multi-head self-attention mechanism. Briefings in Bioinformatics. 2023. 24(1):bbac579.
[7] Zahin, M. A. Multi-Headed Self-Attention Mechanism-Based Transformer Model for Predicting Bus Travel Times Across Multiple Bus Routes Using Heterogeneous Datasets [Master's Thesis]. Columbia, MO: University of Missouri-Columbia; 2023.
[8] Chen, J., J. Luo, editors. Enhancing Vehicle Routing Solutions Through Attention-Based Deep Reinforcement Learning. 2023 5th International Conference on Data-driven Optimization of Complex Systems (DOCS); 2023 22-24 Sept. 2023; Tianjin, China: IEEE.
[9] Sun, Y., J. Platoš. Abstractive text summarization model combining a hierarchical attention mechanism and multiobjective reinforcement learning. Expert Systems with Applications. 2024. 248:123356.
[10] Dantzig, G. B., J. H. Ramser. The Truck Dispatching Problem. Management Science. 1959. 6(1):80-91.
[11] Maroof, A., B. Ayvaz, K. Naeem. Logistics Optimization Using Hybrid Genetic Algorithm (HGA): A Solution to the Vehicle Routing Problem With Time Windows (VRPTW). IEEE Access. 2024. 12:36974-36989.
[12] Legrand, C., D. Cattaruzza, L. Jourdan, M.-E. Kessaci, editors. Investigation of the Benefit of Extracting Patterns from Local Optima to Solve a Bi-objective VRPTW; 2024; Cham: Springer Nature Switzerland.
[13] Speidel, V., editor EDP-assisted fleet scheduling in tramp and coastal shipping. Proceedings of the 2nd International Ship Operation Automation Symposium, Washington, DC, August 30-September 2, 1976 Proceedings expected to be available about December; 1976; 1976.
[14] Wilson, N. H. M., N. J. Colvin. Computer control of the Rochester dial-a-ride system: Massachusetts Institute of Technology, Center for Transportation Studies; 1977.
[15] Psaraftis, H. N. Vehicle routing: Methods and studies.  Dynamic Vehicle Routing Problems. 16. Amsterdam: North Holland Publishing; 1988. p. 223-248.
[16] Ojeda Rios, B. H., E. C. Xavier, F. K. Miyazawa, P. Amorim, E. Curcio, M. J. Santos. Recent dynamic vehicle routing problems: A survey. Computers & Industrial Engineering. 2021. 160:107604.
[17] Pillac, V., M. Gendreau, C. Guéret, A. L. Medaglia. A review of dynamic vehicle routing problems. European Journal of Operational Research. 2013. 225(1):1-11.
[18] Bekta?s, T., P. P. Repoussis, C. D. Tarantilis. Chapter 11: Dynamic Vehicle Routing Problems.  Vehicle Routing. MOS-SIAM Series on Optimization: Society for Industrial and Applied Mathematics; 2014. p. 299-347.
[19] Psaraftis, H. N., M. Wen, C. A. Kontovas. Dynamic vehicle routing problems: Three decades and counting. Networks. 2016. 67(1):3-31.
[20] Garrido, P., M. C. Riff. DVRP: a hard dynamic combinatorial optimisation problem tackled by an evolutionary hyper-heuristic. Journal of Heuristics. 2010. 16(6):795-834.
[21] Hanshar, F. T., B. M. Ombuki-Berman. Dynamic vehicle routing using genetic algorithms. Applied Intelligence. 2007. 27(1):89-99.
[22] Vinyals, O., M. Fortunato, N. Jaitly. Pointer networks. Advances in neural information processing systems. 2015. 28.
[23] Nazari, M., A. Oroojlooy, L. Snyder, M. Takác. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems. 2018. 31.
[24] Sutton, R. S., A. G. Barto. Reinforcement learning: An introduction. Cambridge: MIT press; 2018.
[25] Mnih, V., K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis. Human-level control through deep reinforcement learning. Nature. 2015. 518(7540):529-533.
[26]  Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin. Attention is all you need. Advances in neural information processing systems. 2017. 30.
[27] Zhang M, Li P. Nested graph neural networks. Advances in Neural Information Processing Systems 34 (2021): 15734-15747.
[28]  Solomon, M. M. Algorithms for the vehicle routing and scheduling problems with time window constraints. Operations research. 1987. 35(2):254-265.

Sponsors, Associates, and Links

---

• Information Systems and Economics
  
  
• Accounting, Auditing and Finance
  
  
• Tourism Management and Technology Economy
  
  
• Journal of Computational and Financial Econometrics
  
  
• Financial Engineering and Risk Management
  
  
• Accounting and Corporate Management
  
  
• Social Security and Administration Management
  
  
• Population, Resources & Environmental Economics
  
  
• Statistics & Quantitative Economics
  
  
• Agricultural & Forestry Economics and Management
  
  
• Social Medicine and Health Management
  
  
• Land Resource Management
  
  
• Information, Library and Archival Science
  
  
• Journal of Human Resource Development
  
  
• Manufacturing and Service Operations Management
  
  
• Operational Research and Cybernetics