Document Type : Review Article

Authors

1 Department of Industrial Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran.

2 Department of Industrial Engineering and Management Systems, Amirkabir University of Technology, Tehran, Iran.

Abstract

Todays, human health is in the best situation ever. The progress of vaccines, development of hospitals, new medicines, advanced medical equipment, and new treatments preventing death will place the health system indicators at its best state in all ages and centuries. In addition, healthcare is one of the biggest industries in developed and developing countries and is a service-oriented industry that is significant, high-quality, and safe in medical services. Healthcare has become one of the biggest sectors in terms of income and employment. Health care involves hospitals, medical devices, clinical trials, outsourcing, telemedicine, medical tourism, health insurance, and medical equipment. Nowadays, the application of operations research in various fields, including health, is on increase. Although many issues face operations research in healthcare, such issues are not analytically different from the issues in other industries. Operations research, as a quantitative systemic method, can considerably solve the problems related to the health system. The present study aimed to evaluate the application of operations research models based on the research process in health systems including Markov decision-making processes (MDPs) and partially observable Markov decision process (POMDP), etc., and compare these methods with each other. The basis of this study was evaluating the publication of scientific studies on operations research models in the health systems.

Keywords

Alexander, B., Nallathambi, G., and Selvaraj, N., (2018). "Screening of heart sounds using hidden Markov and gammatone filterbank models", In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 1460-1465). IEEE.          
Araz, C., Selim, H., and Ozkarahan, I., (2007). "A fuzzy multi-objective covering-based vehicle location model for emergency services", Computers & Operations Research, Vol. 34, No. 3.           
Arruda, E. F., Pereira, B. B., Thiers, C. A., and Tura, B. R. (2018). "Optimal testing policies for diagnosing patients with intermediary probability of disease", Artificial intelligence in medicine.
Ashton, R., Hague, L., Brandreth, M., Worthington, D., and Cropper, S., (2005). "A simulation-based study of a NHS Wilkin centre", Journal of the Operational Research Society, Vol. 56, No. 2, pp. 153–161.
Ayer, T., Alagoz, O., and Stout, N. K., (2012). "OR Forum—A POMDP approach to personalize mammography screening decisions", Operations Research, Vol. 60, No. 5, pp. 1019-1034.
Ayer, T., Alagoz, O., Stout, N. K., and Burnside, E. S., (2015). "Heterogeneity in women’s adherence and its role in optimal breast cancer screening policies", Management Science, Vol. 62, No. 5, pp. 1339-1362.
Beech, A. J., (2001). "Market-based demand forecasting promotes informed strategic financial planning", Healthcare Financial Management, Vol. 55, No. 11, pp. 46-56.
Belciug, S., and Gorunescu, F., (2020). How Can Intelligent Decision Support Systems Help the Medical Research?, In Intelligent Decision Support Systems—A Journey to Smarter Healthcare (pp. 71-102). Springer, Cham.
Boloori, A., Saghafian, S., Chakkera, H. A., and Cook, C. B., (2020). "Data-driven management of post-transplant medications: an ambiguous partially observable Markov decision process approach", Manufacturing & Service Operations Management.
Bonifonte, A. J., (2018). Data-driven healthcare management an analytics approach to blood pressure control and cardiovascular disease prevention (Doctoral dissertation, Georgia Institute of Technology).
Brandeau, M. L., Sainfort, F., and Pierskalla, W. P. (Eds.). (2004). Operations research and health care: a handbook of methods and applications (Vol. 70). Springer Science & Business Media.
Bruni, M. E., Conforti, D., Sicilia, N., and Trotta, S., (2006). "A new organ transplantation location–allocation policy: a case study of Italy", Health care management science, Vol. 9, No. 2, pp. 125-142.
Capan, M., Khojandi, A., Denton, B. T., Williams, K. D., Ayer, T., Chhatwal, J., and Zhang, S., (2017). From data to improved decisions: Operations Research in healthcare delivery. Medical Decision Making, Vol. 37, No. 8, pp. 849-859.
Choi, S. E., Brandeau, M. L., and Basu, S., (2017). "Dynamic treatment selection and modification for personalised blood pressure therapy using a Markov decision process model: a cost-effectiveness analysis", BMJ open, Vol. 7, No. 11.
Cipriano, L. E., Goldhaber-Fiebert, J. D., Liu, S., and Weber, T. A., (2018). "Optimal Information Collection Policies in a Markov Decision Process Framework", Medical Decision Making, Vol. 38, No. 7, pp. 797-809.
Cissé, M., Yalçındağ, S., Kergosien, Y., Şahin, E., Lenté, C., and Matta, A., (2017). "OR problems related to Home Health Care: A review of relevant routing and scheduling problems", Operations Research for Health Care, Vol. 13, pp. 1-22.
Cote, M. J., and Tucker, S. L., (2001). "Four methodologies to improve healthcare demand forecasting", Healthcare Financial Management, Vol. 55, No. 5, pp. 54-54.
Crown, W., Buyukkaramikli, N., Sir, M. Y., Thokala, P., Morton, A., Marshall, D. A., and Pasupathy, K. S., (2018). Application of constrained optimization methods in health services research: report 2 of the ISPOR Optimization Methods Emerging Good Practices Task Force. Value in health, 21(9), 1019-1028.
Dai, T., and Tayur, S., (Eds.). (2018). Handbook of Healthcare Analytics: Theoretical Minimum for Conducting 21st Century Research on Healthcare Operations. John Wiley & Sons.
Daskin, M. S., and Dean, L. K., (2004). Location of Health Care Facilities,” chapter 3 in the Handbook of OR/MS in Health Care: A Handbook of Methods and Applications, F. Sainfort, M. Brandeau and W. Pierskalla, editors.
Davoodi, S. M. R., and Goli, A. (2019). "An integrated disaster relief model based on covering tour using hybrid Benders decomposition and variable neighborhood search: Application in the Iranian context", Computers & Industrial Engineering, Vol. 130, pp. 370-380.
Denton, B. T., (2018). "Optimization of sequential decision making for chronic diseases: from data to decisions", In Recent Advances in Optimization and Modeling of Contemporary Problems (pp. 316-348), INFORMS.
Drougard, N., Ponzoni Carvalho Chanel, C., Roy, R. N., and Dehais, F., (2017). Mixed-initiative mission planning considering human operator state estimation based on physiological sensors.
Eldabi, T., Paul, R. J., and Young, T., (2007). "Simulation modelling in healthcare: reviewing legacies and investigating futures", Journal of the Operational Research Society, Vol. 58, No. 2, pp. 262-270.
Erenay, F. S., Alagoz, O., and Said, A., (2014). "Optimizing colonoscopy screening for colorectal cancer prevention and surveillance", Manufacturing & Service Operations Management, Vol. 16, No. 3, pp. 381-400.
Feinberg, E. A., and Shwartz, A. (Eds.). (2012). Handbook of Markov decision processes: methods and applications (Vol. 40). Springer Science & Business Media.
Fikar, C., and Hirsch, P., (2017). "Home health care routing and scheduling: A review", Computers & Operations Research, Vol. 77, pp. 86-95.
Finarelli Jr, H. J., and Johnson, T., (2004). "Effective demand forecasting in 9 steps: shifts in demand for a hospital's services can occur unexpectedly. Demand forecasting can help you prepare for these shifts and avoid strategic missteps", Healthcare Financial Management, Vol. 58, No. 11, pp. 52-58.
Fischer, G. S., da Rosa Righi, R., de Oliveira Ramos, G., da Costa, C. A., and Rodrigues, J. J., (2020). "ElHealth: Using Internet of Things and data prediction for elastic management of human resources in smart hospitals", Engineering Applications of Artificial Intelligence, Vol. 87.
Goli, A., Zare, H. K., Tavakkoli-Moghaddam, R., and Sadeghieh, A., (2019). "Hybrid artificial intelligence and robust optimization for a multi-objective product portfolio problem Case study: The dairy products industry", Computers & Industrial Engineering, Vol. 137.
Goli, A., Tirkolaee, E. B., and Sangaiah, A. K., (2020). "Hybrid neural network and improved cuckoo optimization algorithm for forecasting thermal comfort index at urban open spaces", Advances in Edge Computing: Massive Parallel Processing and Applications, Vol. 35, No. 264.
Gordon, L. G., Tuffaha, H. W., James, R., Keller, A. T., Lowe, A., Scuffham, P. A., and Gardiner, R. A., (2018, March). Estimating the healthcare costs of treating prostate cancer in Australia: A Markov modelling analysis. In Urologic Oncology: Seminars and Original Investigations (Vol. 36, No. 3, pp. 91-e7). Elsevier.
Grover, G., Swain, P. K., Goel, K., and Singh, V., (2018). "Multistate Markov Modelling for Disease Progression of Breast Cancer Patients Based on CA15-3 Marker", Thailand Statistician, Vol. 16, No. 2, pp. 129-139.
 Günal, M. M., and Pidd, M., (2009). "Understanding target-driven action in emergency department performance using simulation", Emergency medicine journal, Vol. 26, No. 10, pp. 724-727.
Günal, M. M., and Pidd, M., (2010). "Discrete event simulation for performance modelling in health care: a review of the literature", Journal of Simulation, Vol. 4, No. 1, pp. 42-51.
Güneş, E. D., and Örmeci, E. L., (2018). "OR applications in disease screening", In Operations Research Applications in Health Care Management (pp. 297-325). Springer, Cham.
Hagberg, J. M., Park, J. J., and Brown, M. D., (2000). "The role of exercise training in the treatment of hypertension", Sports medicine, Vol. 30, No. 3, pp. 193-206.
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., and Zhang, K., (2019). "The practical implementation of artificial intelligence technologies in medicine", Nature medicine, Vol. 25, No. 1, pp. 30-36.
Hudson, J., (2019). A Partially Observable Markov Decision Process for Breast Cancer Screening.
Hynninen, Y., Linna, M., and Vilkkumaa, E., (2019). "Value of genetic testing in the prevention of coronary heart disease events", PloS one, Vol. 14, No. 1.
Jacobson, S. H., (2006). Discrete-event simulation of health care systems In: Hall RW, editor. Patient flow: Reducing delay in healthcare delivery vol. 91.
Jean-Baptiste, E. M., and Mihailidis, A., (2017). Analysis and comparison of two task models in a partially observable Markov decision process based assistive system. In 2017 IEEE 4th International Conference on Soft Computing & Machine Intelligence (ISCMI) (pp. 183-187). IEEE.
Jenkins, P. R., Lunday, B. J., and Robbins, M. J., (2019). "Robust, multi-objective optimization for the military medical evacuation location-allocation problem", Omega.
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., and Wang, Y., (2017). "Artificial intelligence in healthcare: past, present and future", Stroke and vascular neurology, Vol. 2, No. 4, pp. 230-243.
Jones, S. S., Thomas, A., Evans, R. S., Welch, S. J., Haug, P. J., and Snow, G. L., (2008). "Forecasting daily patient volumes in the emergency department", Academic Emergency Medicine, Vol. 15, No. 2, pp. 159-170.
 Jun, J. B., Jacobson, S. H., and Swisher, J. R., (1999). "Application of discrete-event simulation in health care clinics: A survey", Journal of the operational research society, Vol. 50, No. 2, pp. 109-123.
Kaveh, M., Kaveh, M., Mesgari, M. S., and Paland, R. S., (2020). "Multiple criteria decision-making for hospital location-allocation based on improved genetic algorithm", Applied Geomatics, pp. 1-16.
LaRosa, J. C., He, J., and Vupputuri, S., (1999). "Effect of statins on risk of coronary disease: a meta-analysis of randomized controlled trials", Jama, Vol. 282, No. 24, pp. 2340-2346.
Latha, R., and Vetrivelan, P., (2019, January). Blood Viscosity based Heart Disease Risk Prediction Model in Edge/Fog Computing. In 2019 11th International Conference on Communication Systems & Networks (COMSNETS) (pp. 833-837). IEEE.
Lomas, J., Asaria, M., Bojke, L., Gale, C. P., Richardson, G., and Walker, S., (2018). "Which costs matter? Costs included in economic evaluation and their impact on decision uncertainty for stable coronary artery disease", PharmacoEconomics-open, Vol. 2, No. 4, pp. 403-413.
Liu, Y., Dong, B., Guo, B., Yang, J., and Peng, W., (2015). "Combination of cloud computing and internet of things (IOT) in medical monitoring systems", International Journal of Hybrid Information Technology, Vol. 8, No. 12, pp. 367-376.
Madadizadeh, F., Montazeri, M., and Bahrampour, A., (2016). "Predicting of liver disease using Hidden Markov Model", Razi Journal of Medical Sciences, Vol. 23, No. 146, pp. 66-74.
Marynissen, J., and Demeulemeester, E., (2019). "Literature review on multi-appointment scheduling problems in hospitals", European Journal of Operational Research, Vol. 272, No. 2, pp. 407-419.
Meenaxi, D. S., and Singh, N., (2018). "A Reliability Model for the Progression of Chronic Heart Failure", International Journal of Applied Engineering Research, Vol. 13, No. 21, pp. 15351-15355.
Mostafaeipour, A., Goli, A., and Qolipour, M., (2018). "Prediction of air travel demand using a hybrid artificial neural network (ANN) with Bat and Firefly algorithms: A case study", The Journal of Supercomputing, Vol. 74, No. 10, pp. 5461-5484.
Munir, B. A., Hafeez, S., Rashid, S., Iqbal, R., and Javed, M. A., (2019). "Geospatial assessment of physical accessibility of healthcare and agent-based modeling for system efficacy", GeoJournal, pp. 1-16.
Myers, C., and Green, T., (2004). "Forecasting demand and capacity requirements", Healthcare financial management: journal of the Healthcare Financial Management Association, Vol. 58, No. 8, p. 34.
Ndiaye, M., and Alfares, H., (2008). "Modeling health care facility location for moving population groups", Computers and Operations Research, Vol. 35, No. 7, pp. 2154-2161.
Neter, J. E., Stam, B. E., Kok, F. J., Grobbee, D. E., and Geleijnse, J. M., (2003). "Influence of weight reduction on blood pressure: a meta-analysis of randomized controlled trials", Hypertension, Vol. 42, No. 5, pp. 878-884.
Onar, S. C., Oztaysi, B., and Kahraman, C., (2018). "A Comprehensive Survey on Healthcare Management", In Operations Research Applications in Health Care Management (pp. 23-51). Springer, Cham.
Parthiban, G., Rajesh, A., and Srivatsa, S. K., (2011). "Diagnosis of heart disease for diabetic patients using naive bayes method", International Journal of Computer Applications, Vol. 24, No. 3, pp. 7-11.
Peterse, E. F., Meester, R. G., Siegel, R. L., Chen, J. C., Dwyer, A., Ahnen, D. J., ... and Lansdorp‐Vogelaar, I., (2018). "The impact of the rising colorectal cancer incidence in young adults on the optimal age to start screening: microsimulation analysis I to inform the American Cancer Society colorectal cancer screening guideline", Cancer, Vol. 124, No. 14, pp. 2964-2973.
Petousis, P., Han, S. X., Hsu, W., and Bui, A. A. (2018, July). Generating Reward Functions Using IRL towards Individualized Cancer Screening. In International Workshop on Artificial Intelligence in Health (pp. 213-227). Springer, Cham.
Rahman, S. U., and Smith, D. K., (2000). Use of location-allocation models in health service development planning in developing nations.
Rais, A., and Viana, A., (2011). "Operations research in healthcare: a survey. International transactions in operational research", Vol. 18, No. 1, pp. 1-31.
Rauner, M. S., and Bajmoczy, N., (2003). "How many AEDs in which region? An economic decision model for the Austrian Red Cross", European journal of operational research, Vol. 150, No. 1, pp. 3-18.
Sangaiah, A. K., Goli, A., Tirkolaee, E. B., Ranjbar-Bourani, M., Pandey, H. M., and Zhang, W., (2020). "Big Data-Driven Cognitive Computing System for Optimization of Social Media Analytics", IEEE Access, Vol. 8, pp. 82215-82226.
Shishvan, O. R., Zois, D. S., and Soyata, T., (2018). "Machine intelligence in healthcare and medical cyber physical systems: A survey", IEEE Access, Vol. 6, pp. 46419-46494.
 Smith, H. K., Harper, P. R., Potts, C. N., and Thyle, A., (2009). "Planning sustainable community health schemes in rural areas of developing countries", European Journal of Operational Research, Vol. 193, No. 3, pp. 768-777.
Smith‐Daniels, V. L., Schweikhart, S. B., and Smith‐Daniels, D. E., (1988). "Capacity management in health care services: Review and future research directions", Decision Sciences, Vol. 19, No. 4, pp. 889-919.
Steimle, L. N., and Denton, B. T., (2017). "Markov decision processes for screening and treatment of chronic diseases", In Markov Decision Processes in Practice (pp. 189-222). Springer, Cham.
Tighe, P. J., Bzdega, M., Fillingim, R. B., Rashidi, P., and Aytug, H., (2016). "Markov chain evaluation of acute postoperative pain transition states", Pain, Vol. 157, No. 3, pp. 717.
Tirkolaee, E. B., Goli, A., Pahlevan, M., and Malekalipour Kordestanizadeh, R., (2019). "A robust bi-objective multi-trip periodic capacitated arc routing problem for urban waste collection using a multi-objective invasive weed optimization", Waste Management and Research, Vol. 37, No. 11, pp 1089-1101.
Tirkolaee, E. B., Goli, A., Faridnia, A., Soltani, M., and Weber, G. W. (2020a). "Multi-objective optimization for the reliable pollution-routing problem with cross-dock selection using Pareto-based algorithms", Journal of Cleaner Production.
Tirkolaee, E. B., Goli, A., and Weber, G. W., (2020b). "Fuzzy Mathematical Programming and Self-Adaptive Artificial Fish Swarm Algorithm for Just-in-Time Energy-Aware Flow Shop Scheduling Problem with Outsourcing Option", IEEE Transactions on Fuzzy Systems.
Toes-Zoutendijk, E., van Leerdam, M. E., Dekker, E., van Hees, F., Penning, C., Nagtegaal, I. and Biermann, K., (2017). "Real-time monitoring of results during first year of Dutch colorectal cancer screening program and optimization by altering fecal immunochemical test cut-off levels", Gastroenterology, Vol. 152, No. 4, pp. 767-775.
Tsoukalas, A., Albertson, T., and Tagkopoulos, I., (2015). "From data to optimal decision making a data-driven, probabilistic machine learning approach to decision support for patients with sepsis", JMIR medical informatics, Vol. 3, No. 1.
 Verter, V., and Lapierre, S. D., (2002). "Location of preventive health care facilities", Annals of Operations Research, Vol. 110, pp. 123-132.
Wang, Z., (2017, July). Unconscious decision minds and divergent thinking model. In 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) (pp. 1879-1886). IEEE.
WHO, (2007). Prevention of cardiovascular disease: guidelines for assessment and management of cardiovascular risk.
Wu, W. Y. Y., Törnberg, S., Elfström, K. M., Liu, X., Nyström, L., and Jonsson, H., (2018). "Overdiagnosis in the population-based organized breast cancer-screening program estimated by a non-homogeneous multi-state model: a cohort study using individual data with long-term follow-up", Breast Cancer Research, Vol. 20, No. 1, p. 153.
Wu, B., Ahmadi, M., Bharadwaj, S., and Topcu, U., (2019, July). Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes. In 2019 American Control Conference (ACC) (pp. 1216-1223). IEEE.
Xue, J. L., Ma, J. Z., Louis, T. A., and Collins, A. J., (2001). "Forecast of the number of patients with end-stage renal disease in the United States to the year 2010", Journal of the American Society of Nephrology, Vol. 12, No. 12, pp. 2753-2758.
Yaylali, E., and Karamustafa, U., (2019). "A Markov Decision Process Approach to Estimate the Risk of Obesity Related Cancers", In Industrial Engineering in the Big Data Era (pp. 489-502). Springer, Cham.
Yuan, Y. B., Qiu, W. Q., Wang, Y. J., Gao, J., and He, P., (2017, July). Classification of heart failure with polynomial smooth support vector machine. In 2017 International Conference on Machine Learning and Cybernetics (ICMLC) (Vol. 1, pp. 48-54). IEEE.
Zois, D. S., (2016). Sequential decision-making in healthcare IoT: Real-time health monitoring, treatments and interventions. In 2016 IEEE 3rd World Forum on Internet of Things (WF-IoT) (pp. 24-29). IEEE.