Technical Efficiency of Public Hospitals in Mongolia
Sodnom Munkhsaikhan, Alan C Tsai, Ming-Cheng Chang
Department of Healthcare Administration, Asia University, 500 Liufeng Road, Wufeng, Taichung 41354, Taiwan.
Corresponding Author: Mr.Sodnom Munkhsaikhan, M.D., M.Sc.
Department of Healthcare Administration, Asia University, 500 Liufeng Road, Wufeng, Taichung 41354, Taiwan.
E-mail address: Munkhsaikhan@gmail.com.
Telephone number: (976) 91190001; (886) 958119001.
Abstract
The purpose of this study was to evaluate the technical efficiency of public hospitals in Mongolia and to test the link between technical efficiency and various factors such as level of hospital, location of hospital and time period observed. In this study non-parametric data
envelopment analyses (DEA) was used to assess the technical efficiency of hospitals. The original data set consists of almost all secondary and tertiary level public hospitals in Mongolia making the study nationwide and help to get the overall information on technical efficiency of public hospitals in Mongolia. The DEA results in this research provide efficiency scores for each of the hospitals through three year of period. For each inefficient hospital the reductions in inputs and/or increases in outputs needed to bring the hospital up to full efficiency. The findings of this
research indicate that the secondary level hospitals in provinces on average are significantly less efficient (p<0.05) than secondary level hospitals in cities. Decision-makers should reevaluate hospital performance and reallocate resources among across regions based on needs and necessities to maximize system-wide efficiency.
Keywords: Data envelopment analysis; technical efficiency; resource allocation; Mongolia
1. Introduction
Hospitals represent a significant proportion of health expenditures. Improvement in the technical efficiency of hospitals may result in large savings in healthcare expenditures, which might, in turn, be devoted to other services such as prevention and education. Technical efficiency describes the relationship between output and the quantity of resources used to produce that output. Technical efficiency can be assessed without the need to consider complex issues such as cost and quality (1). Technical efficiency studies treat labor, capital, and technology as resource inputs used to create outputs (goods and services). Measuring the level of technical efficiency involves comparing all decision-making units (i.e. hospitals) to identify the most efficient organizations (2, 3).
Data envelopment analysis (DEA) is a powerful decision making tool and an ideal technique for measuring technical efficiency, particularly in the public sector, because it uses a non-parametric, non-subjective approach by empirically building an achievable best-practice production function from observed inputs and outputs (4). DEA has been widely used for the evaluation of resource consumption among healthcare organizations and can lead to improved hospital efficiency and reductions in healthcare costs (5). A hospital is judged to be technically efficient if it operates at optimal levels in comparison with peer hospitals in the sample with the same resources (6, 7). DEA has evolved as a worldwide accepted operational research tool. Ali et al. (2008) have identified more than 4000 research articles published in journals or book chapters.
Mongolia is a central Asian country with population about 2.6 million. Administratively, Mongolia is divided into 21 provinces and the capital city, Ulaanbaatar which has 9 districts.
Moreover, about 1 million people reside in Ulaanbaatar city (8). The health care system in
Mongolia is characterized by three levels of care and services and its prevailing principle is to
deliver equitable, accessible and quality health care and services for every person. Primary care and services are mainly placed in family practice facilities in Ulaanbaatar city, and in sub provincial hospitals. Secondary care and services take place in district general hospitals in Ulanbaatar city, and provincial general hospitals. Tertiary care and services are placed in major hospitals and specialized professional centers in Ulaanbaatar city, except for three Regional Diagnostic and Treatment Centers (RDTCs). By 2006, 15 specialized hospitals, 3 regional diagnostic and treatment centers, 18 provincial general hospitals, 9 district general hospitals, 4 rural general hospitals have been delivering health care services to the Mongolian population (9).
A relatively large number of hospitals in Mongolia including all 18 provincial general hospitals, all tertiary level general hospitals and regional diagnostic and treatment centers and all district
hospitals with outpatient visits of Ulaanbaatar city makes the study nationwide and help to get the overall information on technical efficiency of public hospitals in Mongolia.
The purpose of this study is to evaluate the technical efficiency of the public hospitals in Mongolia using data envelopment analysis methodology and to find out the relationships between technical efficiency of hospitals and other factors such as level of hospital, location of hospital and time period observed.
2. Methods
2.1. Study scope and data analysis
Data used in this study were from the Ministry of Health (MoH) of Mongolia. This data set is maintained by the National Center for Health Development (NCHD) and Ulaanbaatar City Health Department, which require that the data are aggregated and tabulated at the hospital level.
The study is based on cross-sectional data for 3 years (2004-2006).
In this study non-parametric DEA super efficiency model is used to evaluate the technical efficiency of hospitals. The efficiency scores were calculated for the pooled sample including all year groups. The set of Decision Making Units (DMUs) in a DEA efficiency study should be a homogeneous set, with activity measurements for the same period, so that the
comparison is meaningful and the differences identified are sensible. In this study the selected DMUs are the 30 general hospitals owned by and operated under the control of the Ministry of Health in Mongolia. Specialty hospitals, (such as maternity, pediatric and psychiatric hospitals), and very small hospitals, (i.e. primary health care centers enhanced with beds) and private hospitals were excluded in order to have a homogeneous group of hospitals. The selected hospitals are reasonably homogeneous and having a common role and objective. They use the same technology (inputs, outputs and processing procedures are identical) and operate under similar environments.
The technical efficiency score were computed using KonSi Data Envelopment Analysis for Benchmarking software, professional edition. The relationships between technical efficiency of hospitals and other factors such as level of hospital, location of hospital and time period observed were then examined using appropriate parametric and non-parametric test. We tested for variation in efficiency score by hospital level, location and the time period. Parametric and non- parametric tests were employed to determine the statistical significance of differences between the technical efficiency of secondary and tertiary level hospitals, hospitals in cities and hospitals in provinces, hospitals in west, east, khangai and central regions of Mongolia, and the stability (or instability) of hospital's efficiency over time period observed.
Statistical software package SPSS 13 was used (Statistical Package for the Social Sciences,
SPSS Inc, Chicago, Ill). Statistical significance for all analyses was accepted at α = 0.05.
2.2. DEA model specification
In this study we used the variable return to scale (VRS) framework of DEA, as it allows for varying returns to scale and pure model efficiency and it is the most appropriate model for real benchmarking researches (10). DEA super-efficiency model that was first proposed by Andersen and Petersen (11) was used as a method of discriminating among efficient DMUs.
For the purpose of this study the output maximization model has been adopted
because public hospitals in Mongolia are subject to budget constraints imposed by the central government and the inputs are nearly fixed, on the other hand Mongolian Ministry of Health decisions reflects their interest to maximize services to the population.
In order to compare the technical efficiency of hospitals during time period observed window analysis was used. The frontier is constructed from the total number of hospitals in a three-year period.
2.3. Selection of inputs and outputs data
Inpatient days and number of outpatient visits were used as an output in this study.
Inpatient days is a common measure of hospital productivity in the literature and is a widely accepted measure of inpatient workload (12). Number of outpatient visits or outpatient workload is a widely accepted measure of hospital output (13). Inpatient days and outpatient workload provide a comprehensive measure of hospital productivity (14).
In DEA, inputs are any factor used as a resource to produce something of value (15). Since we are not interested in cost analysis in this study, we used number of beds as capital input. The
number of hospital beds is an accepted indicator of capital investment (16, 17). Also the number
of physicians and number of other personnel as inputs was used. The disaggregation of staff to physicians and others personnel are due to the specialized role of staff particularly physicians in treatment of patients (18).
2.4. Sensitivity analysis and super efficiency model
To test for the robustness of the DEA technical efficiency scores, the Jackknife analysis was used. We performed a two-phase diagnostic test to measure influence and detect potential data errors. In the first phase, the score of the efficiency was calculated using VRS super efficiency model for all DMUs. In second phase DMUs with extreme observations, namely hospital ARH of the year 2005 (1.14) and hospital GVS of the year 2006 (1.07) were omitted and three models of DEA were run: one with hospital ARH of the year 2005 removed, one with hospital GVS of the year 2006 removed and one with both hospitals removed examining the efficiency score change in all DMUs.
3. Results
3.1. Descriptive statistics of the data
Table 1 shows the brief descriptive statistics of the variables. On average, tertiary level
hospitals have higher number of beds, number of physicians, number of other staff, inpatient days,
outpatient visits than secondary level hospitals. There are big variations between city and province
hospitals on the secondary and tertiary levels. In general, tertiary level hospitals in the cities are bigger
than tertiary level hospitals in the provinces and vice versa secondary level hospitals in cities are
smaller than in provinces.
3.2. Evaluating the robustness of DEA using sensitivity analysis
Three models had a strong agreement on efficiency with the original model and DEA results are consistent in all models. Even with both hospitals removed, none of the efficiency scores of the hospitals changed significantly. Removing both of these hospitals only affected the efficiency scores change by 0.0107 (± 0.0196) on average. In a general statistical setting,
observations are regarded as influential if their omission from the data results in substantial changes to important features of the analysis. A large drop in efficiency score suggested either a potential error or an unusual distinctive competency, whereas a small decrease implied that DMU was efficient with respect to multiple dimensions (19, 20). Examination of the efficiency score change in DMUs shows that original DEA model is insensitive for omitting hospitals with extreme
observations.
3.3. Technical efficiency of hospitals in Mongolia
The technical efficiency score of hospitals are shown in Table 2. DEA result indicates that 5 hospitals are efficient (score equal or more than 1.00) during the all three-year period, 3 hospitals are efficient during two-year period and 4 hospitals efficient during one year period.
In other words 25 hospitals are efficient out of 90 hospitals. Of 22 efficient secondary level
hospitals 9 were city and 13 are provincial hospitals. From 3 efficient tertiary level units 2 are
city and 1 is provincial hospital. In year 2004 there are 11 units (9 are secondary and 2 are
tertiary level), in year 2005 there are 6 (all are secondary level) units and in year 2006 there
are 8 (7 are secondary and 1 is tertiary level) units with efficient status.
3.4. Variation of technical efficiency of hospitals by level, location and time period
The mean efficiency scores by hospital level and location are reported in Table 3. For both groups (city and province), and for pooled data there is no significant difference in means of efficiency scores between secondary and tertiary level hospitals. However, on average secondary level hospitals in city have a higher efficiency score than secondary level hospitals in provinces (p<0.05), but there is no significant difference in average efficiency score of the tertiary level hospitals in cities and provinces according to t-test and Mann-Whitney test. Also there is no significant difference in efficiency score between hospitals in west, east, Khangai and central regions of Mongolia. The means of efficiency scores in all groups of hospitals during the time period 2004-2006 have the similar pattern (Table 2). For all groups of hospitals (secondary, tertiary, city and, province) and for pooled data there is no significant difference in means of efficiency scores over the time observed according to Anova and Kruskal-Wallis test. Here we have to note that some hospitals show change in efficiency score over time.
4. Discussion
Because the scope of this study helps to get the overall information on technical efficiency of public hospitals in Mongolia it may provide policy makers with much needed information when planning for the future of health care systems in general, and for the hospital industry in particular.
The DEA results in this research provide efficiency scores for each of the hospitals through
three year of period, together with suggestions for potential improvements for those which are
inefficient. For each inefficient hospital the reductions in inputs and/or increases in outputs
needed to bring the hospital up to full efficiency. The MoH need to look at the hospitals with
very low efficiency more closely, to find out more details about them and to explore any
changes that would enhance their efficiency.
The hospital technical efficiency score showed that secondary level hospitals in provinces on average are significantly less efficient (p<0.05) than secondary level hospitals in cities. This result can play an important role in shaping administrative decisions for hospitals.
Decision-makers should reevaluate hospital performance and reallocate resources among across regions based on needs and necessities in order to improve the system-wide efficiency.
In addition, MoH should establish a database that contains more comprehensive health care information as well as other important pertinent data in order to conduct more comprehensive and qualitative research to measure the real total efficiency of hospitals. Policymakers at the regional and national levels should consider efficiency analysis as they explore ways to coordinate the increasing demand and lacking accessibility for healthcare services in Mongolia.
References
1. Alshare K, Chowdhury M. Data envelopment analysis (DEA): A review of concepts and applications. International Journal of Business and Public Administration 2006; 3:1-14.
2. Fare R, Grosskopf S, Lovell C. The Measurement of Efficiency of Production. Kluwer-Nijhoff Publishing, Boston 1985.
3. Grosskopf S. The role of reference technology in measuring productive efficiency. Economic Journal 1986; 96:499-513.
4. Ozcan YA, McCue MJ, Okasha AA. Measuring the technical efficiency of psychiatric hospitals.
Journal of Medical Systems 1996; 20:141-150.
5. Sherman H. Hospital efficiency measurement and evaluation empirical test of a new technique.
Medical Care 1984; 22:922-938.
6. Grosskopf S, Valdmanis V. Measuring hospital performance: A non-parametric approach.
Journal of Health Economics 1987; 6:89-107.
7. Fare R, Grosskopf S. Profit efficiency, Farrell decompositions and the Mahler inequality.
Economics Letters 1997; 57:283-287.
8. National Statistical Office. Mongolian Statistical Yearbook 2006. National Statistical Office, Ulaanbaatar 2007.
9. Ministry of Health and National Centre for Health Development. Health indicators 2004-2006.
MoH and NCHD, Ulaanbaatar 2005-2007.
10. Cooper W, Seiford L, Tone K. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software. Springer Science Business Media, LLC 2007.
11. Andersen P, Petersen N. A procedure for ranking efficient units in data envelopment analysis.
Management Science 1993; 39:1261-1264.
12. Harrison J, Coppola MN, Wakefield MD. Efficiency of federal hospitals in the United States.
Journal of Medical Systems 2004; 28:411-422.
13. Bilodeau D, Cremieux P, Jaumard B, Ouellette P, Vovor Ts. Measuring hospital performance in the presence of quasi-fixed inputs: An analysis of Quebec hospitals. Journal of Productivity Analysis 2004; 21:183-99.
14. Wang BB, Ozcan YA, Wan TT, Harrison J. Trends in hospital efficiency among metropolitan markets. Journal of Medical Systems 1999; 23:83-97.
15. Jeffrey PH, Christopher S. The improving efficiency frontier of religious not-for-profit hospitals. Hospital Topics 2006; 84(1):2-10.
16. Hu JL, Huang YF. Technical efficiencies in large hospitals: A managerial perspective.
International Journal of Management 2004; 21:506-514.
17. Worthington AC. Frontier efficiency measurement in health care: A review of empirical techniques and selected applications. Medical Care Research Review 2004; 61:135-170.
18. Jensen GA, Morrisey MA. The role of physicians in hospital production. Review of Economics and Statistics 1986; 68:432-442.
19. Efron B. The Jackknife, the Bootstrap, and other Resampling Plans. Society for Industrial and Applied Mathematics, Philadelphia 1982.
20. O’Neill L. Multifactor efficiency in data envelopment analysis with an application to urban
hospitals. Health Care Management Science 1998; 1:19-27.
Table 1
Descriptive statistics of input and output variables (Mean ± SD).
_____________________________________________________________________________________________
Variable Secondary Tertiary
City (N=18) Province (N=54) City (N=9) Province (N=9) Inputs
No. of beds 115 ± 44 212 ± 67 363 ± 136 263 ± 57
No. of physicians 73 ± 18 49 ± 15 113 ± 26 64 ± 80
No. of other staff 179 ± 46 206 ± 69 387 ± 64 284 ± 70
Outputs
Inpatient days 37921 ± 12962 59580 ± 22470 119620 ± 48406 83075 ± 18233 Outpatient visits 187034 ± 95439 89045 ± 42725 115421 ± 37440 113106 ± 20824
Table 2
Technical efficiency score of hospitals.
_________________________________________________________________________________________
Hospital 2004 2005 2006 Hospital 2004 2005 2006
Secondary City ORH PGH 1.02* 0.95 0.90
NLH HC 0.90 0.89 0.71 UMG PGH 0.85 0.66 0.84
BGN HC 0.89 0.86 0.85 SBT PGH 0.89 0.82 0.96
CHG HC 0.82 0.87 0.84 SLG PGH 0.86 0.97 1.01*
KHU HC 1.02* 1.00* 1.00* TUV PGH 0.74 0.59 0.47
SKH HC 1.00* 1.01* 1.06* UVS PGH 0.68 0.63 0.79
BGL HC 1.04* 1.03* 1.00* KHG PGH 0.95 0.85 0.84
Mean 0.95 0.94 0.91 KHT PGH 1.05* 0.91 0.78
Secondary Province Mean 0.88 0.86 0.90
ARH PGH 1.03* 1.14* 1.01* Tertiary City
BOL PGH 0.89 0.91 0.89 SCH1 1.02* 0.98 1.06*
BKH PGH 0.96 0.89 0.93 SCH2 0.92 0.90 0.94
BLG PGH 0.85 0.75 0.93 SCH3 0.74 0.86 0.73
GVA PGH 0.81 0.97 0.96 Mean 0.89 0.91 0.91
GVS PGH 1.00* 1.01* 1.07* Tertiary Province
DRU PGH 1.05* 0.98 1.05* DRN RDTC 0.98 0.97 0.87
DOG PGH 1.05* 1.00* 0.88 UVH RDTC 0.83 0.81 0.83
DUG PGH 0.65 0.78 0.97 KHD RDTC 1.06* 0.91 0.95
ZVH PGH 0.55 0.72 0.88 Mean 0.96 0.90 0.88
*The technically efficient hospitals. HC, health center; PGH, provincial general hospital; SCH, state clinical hospital; RDTC, regional diagnostic and treatment center.
Table 3
Efficiency score (Mean ± SD) arranged by hospital level and location.
______________________________________________________________________________________________
Location Secondary Tertiary Pooled
N Score N Score N Score
City 18 0.97 ± 0.08* 9 0.91 ± 0.11 27 0.94 ± 0.10*
Province 54 0.88 ± 0.14 9 0.91 ± 0.08 63 0.88 ± 0.13
West 12 0.81 ± 0.13 3 0.97 ± 0.08 15 0.84 ± 0.14
East 6 0.90 ± 0.10 3 0.94 ± 0.06 9 0.92 ± 0.08
Khangai 15 0.93 ± 0.09 3 0.82 ± 0.02 18 0.91 ± 0.09
Central 21 0.88 ± 0.17 0 21 0.88 ± 0.17
All 72 0.89 ± 0.13 18 0.91 ± 0.10 90 0.90 ± 0.13
*The secondary city hospitals are significantly more efficient than secondary province hospitals, and the city hospitals are significantly more efficient than province hospitals according to t-test and Mann-Whitney test (p<0.05).
