Validation of the Maslach Burnout Inventory – Human Services Survey to assess burnout of prison officers working in Sri Lankan Prisons

Introduction: Prison officers undergo a high amount of work-stress and consequently suffer from burnout, which is detrimental to their health, the organization and the inmates. There are no studies conducted on burnout of prison officers in Sri Lanka, partly due to the non-availability of a validated scale. Objectives: To validate the Maslach Burnout Inventory – Human Services Survey (MBI-HSS) for assessing burnout among prison officers working in Sri Lankan prisons Methods: A cross-sectional validation study was carried out among a representative sample of 267 correctional and rehabilitation officers working in the Colombo Remand Prison and Colombo New Magazine Prison, selected through probability-proportionate-to-sample size and random sampling methods. The Sinhala-translated MBIHSS was judgmentally validated and pre-tested prior to administration. Confirmatory factor analysis (CFA) was performed using LISREL 9.3 software to assess the construct validity of MBI-HSS. The presence of job-related neurasthenia in participants, diagnosed by a psychiatrist was taken as the gold standard to assess the criterion validity. Internal consistency and test-retest methods were used to assess the reliability. Results: A three-factor model with items 6, 13, 16 and 22 of the instrument deleted, was identified as the model best fitting the data during CFA, achieving the best goodness of fit parameters (RMSEA=0.0748; 2=185.2, p=0.001; 2/df=1.4; CFI=0.974; NNFI=0.97; GFI=0.985; SRMR=0.056). Clinical burnout was seen in 32.6% of the sample. According to the cut-off values achieved through receiver operating characteristics (ROC) curves for burnout and its three sub-scales of emotional exhaustion, depersonalization and diminished personal accomplishment, the sensitivity and specificity were 94.3% & 87.8%; 82.8% & 80.6%; 72.4% & 73.9%; and 77% & 77.2%, respectively. The reliability of the tool was satisfactory. Conclusions: Translated and validated MBI-HSS was found to be a valid and reliable tool for assessing burnout among Sri Lankan prison officers. Using it during their medical examinations is recommended.


Introduction
Burnout is 'a state of physical, emotional and mental exhaustion resulting from long-term involvement in work situations that are emotionally demanding' (1). It is a well-accepted common concept occurring among employed people, subjected to exploration by various researchers (2). Long-term occupational stress can lead to burnout (3), where high work demands combined with low work resources may cause burnout among employees of all occupations, particularly among employees of human services (4). Thus, burnout is considered a syndrome of emotional exhaustion and cynicism, which occurs mainly among individuals who are engaged in 'people-work'. It is described under three main dimensions (5). Emotional exhaustion describes overwhelming exhaustion, depletion of energy and fatigue during work, while depersonalization includes feelings of cynicism, withdrawal, irritability and inappropriate thoughts about one's clients. Reduced personal accomplishment, the third dimension, implies low morale, inefficiency and incapability in the job (6).
The Maslach Burnout Inventory is considered the most popular, and has been most extensively used, to measure burnout among people who are working in human services since its development (5). Following the development of alternate forms of the MBI for educators and for professions other than human services, the original MBI has been renamed as the Maslach Burnout Inventory -Human Services Survey (MBI-HSS) (7).
Many studies have categorized the employees in to high and low levels of the three subscales of burnout separately, as implied by the MBI-HSS, while some studies have given cut-offs for the total burnout score, to categorize the participants as being either 'burned out' and 'non burned out'. However, the single construct syndromic approach to burnout, which considers burnout as a syndrome can be taken in favour of using a single score (4).
The MBI-HSS consists of 22 items to assess the three dimensions of burnout. Nine items evaluate emotional exhaustion, while depersonalization and personal accomplishment are assessed by five and eight items, respectively (7). The items assessing emotional exhaustion describe feelings of being emotionally overextended and exhausted by one's work. Unfeeling and impersonal response towards the recipients under one's care or service is described in the items assessing depersonalization. Items assessing personal accomplishment describe feelings of competence and successful achievement in one's work with people (4). All these items are assorted together to make one complete scale. Each item is expected to be rated on a seven-point, fully anchored Likert scale. The frequency ranges from 'never' to 'every day' (8). The items assessing personal accomplishment are worded positively, where higher mean scores indicate a lower level of burnout (9).
The validity and reliability of the MBI-HSS have been explored and proven through many studies (7,10). When the initial questionnaire developed by Maslach and colleagues was subjected to exploratory factor analysis and CFA, a three-factor structure was the ultimate result (7,9). However, subsequently, certain researchers have suggested removing the weakerperforming items from the scale in order to improve the goodness of fit parameters, while some others believe that factor structures different to the threedimensional one would be more suitable (11). Items 6, 12-13, 16 and 20 have been found to either deviate from their original sub-scale or the model has been found to be better with those deleted (11)(12).
In the Sri Lankan context, the MBI-HSS has been previously translated to Sinhala language and validated among the nursing officers (13), where a three-factor model with item four deleted has been identified as the best model for nurses in Sri Lanka following construct validation. It is identified that prison officers undergo a significant amount of stress and subsequent burnout during their work, in a context where their service recipients are prison inmates who are held in prisons against their wish. Prison officers come into direct contact with the inmates during their work, and their work setting makes them a distinct group of employees. Therefore, the study instruments already validated for other professions and the cut-off values derived from such validation studies cannot be directly adopted for this occupational category. Thus, this study aimed to validate the MBI-HSS for correctional and rehabilitation officers working in Sri Lankan prisons, in terms of judgmental, construct and criterion validity, to produce a validated tool to assess their burnout.

Methods
A descriptive cross-sectional validation study was conducted to assess both criterion and construct Wijegoonewardene N et al. JCCPSL 2019, 25 (1) Open Access validity of the MBI-HSS, where correctional and rehabilitation officers working in the Colombo Remand Prison and Colombo New Magazine Prison were taken as the study population. In this study, the category of correctional officers was represented by the prison guards since they had the highest amount of contact with the inmates. Officers who have been working for more than six months duration were included, while those not having direct and frequent (at least four days a week) contact with the inmates, on maternity leave or other long-term leave at that time, diagnosed of a mental disorder at the time and those who could not read and write in Sinhala language were excluded from the study. The minimum required sample size was 267 according to the minimum number of participants needed for testing criterion validity. This was based on the expected sensitivity or specificity of the instrument, the required level of precision and the confidence interval (14).
Since the number of rehabilitation officers was a minority, all of them employed in the two prisons and fulfilling the eligibility criteria were included in the study. The remainder was taken from the correctional officers, proportionate to the total number of eligible correctional officers in each prison. The correctional officers to be selected from each prison were chosen randomly using lists of their names.
The English version of MBI-HSS was translated to Sinhala language, which was identified as the conversant language among the vast majority of the correctional and rehabilitation officers. Forward and backward translation processes were adopted. The word 'inmate' was considered in place of the word 'patient/client' when translating the items. The instrument was pre-tested prior to administration, and the duration necessary for completing the questionnaire was estimated at 10-15 minutes.
The judgmental validity was assessed in terms of face, content and consensual validity. The content validity was assured with the meticulous process followed in translating the instrument. This was complemented by the assessment carried out by a panel of multi-disciplinary experts consisting of a psychiatrist, a psychologist, a sociologist, a public health expert and a high-rank prison official. This panel also assessed the consensual validity of each item for its relevance in assessing burnout among prison officers, the appropriateness of the wording used, and the acceptability of the items to the local context. Necessary modifications were carried out to achieve conceptual equivalence of the original item to the maximum possible level.
Construct validity of the instrument was assessed using CFA to assess how well the underlying, already established three-factor model was replicated in the data of the study. As for criterion validity, the clinical diagnosis of burnout made by a psychiatrist was taken as the gold standard to test against the outcome of the instrument. Previous studies have used the International Classification of Diseases (ICD-10) criteria for neurasthenia as the diagnostic guideline for assessing burnout (6,15). Since the symptoms needed to be work-related, job-related neurasthenia has been considered the equivalent for clinical burnout (15).
All the participants faced a clinical interview conducted by the psychiatrist who diagnosed each participant as 'having burnout' or 'not having burnout' according to the ICD-10 criteria, independent of the details they provided in the questionnaire. The participants were administered the translated MBI-HSS prior to attending this clinical interview. The completeness of the questionnaires was assured before receiving them.

Data analysis
In order to assess the construct validity of the instrument, confirmatory factor analysis (CFA) was carried out using the software package Lisrel 9.3, during which a multitude of goodness of fit measures were evaluated to assess the model fit. These included chi-squared statistic (  2 ), normed  2 value, root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) as the residual matrix based indices; goodness of fit index (GFI) as the absolute fit measure; and non-normed fit index (NNFI) or the Tucker Lewis index (TLI) and comparative fit index (CFI) as the incremental fit indices.
In order to assess the criterion validity, scores received for the MBI-HSS were compared with the clinical diagnosis provided by the psychiatrist using Statistical Package for Social Sciences (SPSS) version 20.0. For this purpose, the appropriately summed scores of the overall MBI-HSS and the subscales, and Open Access the diagnosis of the psychiatrist were used to draw the ROC curves. From the ROC curves, the optimal cut-off values were derived to categorize the respondents as 'burned out' and 'not burned out', and also for the presence or absence of each of the three dimensions of burnout. Criterion validity was assessed by comparing the presence and absence of burnout identified by the MBI-HSS, as well as the presence or absence of each subscale, with the clinical diagnosis.
Internal consistency was assessed using the Cronbach's Alpha, where values higher than 0.7 were considered satisfactory (16). Test-retest reliability was assessed by Spearman's r correlation coefficient and achieving values of 0.7 or above was considered as having at least a moderate correlation between the scores (17).

Results
In total, 267 prison officers including 261 correctional officers and six rehabilitation officers participated in the study. The response rate was 100%. Only five female officers were working in the selected prisons, all of whom participated. More than one third of the participants (n=96; 36%) were between 25-34 years. A total of 146 participants were working in the Colombo Remand Prison, while the remaining 121 were recruited from the Colombo New Magazine Prison.
Following the assessment of judgemental validity, it was assured that the instrument had adequate judgmental validity. Prior to the assessment of construct validity using CFA, several statistics were performed to ensure that the statistical assumptions required for this analysis were fulfilled. The sample size was assured to be adequate, where the ratio of cases to items was 12.1:1, in a context where the sample size should be at least 5-10 times the number of items in the instrument (18). The data were found to be non-normal with the support of histograms, calculation of standardized skewness and kurtosis values, and by performing Kolmogorov-Smirnov and Shapiro-Wilk tests of normality. Thus, the robust maximum likelihood method was adopted in CFA while the Satorra-Bentler scaled chi-squared statistic was considered in the analysis. A random sample of bivariate scatter plots was examined to ensure the linearity of the data. There were no outliers in the dataset. The data were found to be factorable, depending on the results from the correlation matrix, anti-image correlation matrix, Bartleet test of sphericity and the Kaiser-Meyer-Olkin measure of sample adequacy. It was also assured that multi-collinearity was not present.
Following the assurance that the dataset had the eligibility to undergo CFA, the goodness of fit parameters were assessed for several models, in order to identify the model best fitting the data. Firstly, it was assessed whether a three-factor model depicted in many studies using the MBI-HSS was fitting the data well. In the current study, the three-factor model with items measuring emotional exhaustion, depersonalization and personal accomplishment loaded into three different factors was found to be superior to the one-factor and two-factor models. It further showed considerable improvement in the parameters of goodness of fit than the other two models, as depicted in Table 1.
In further obtaining a good fit of data, certain items were deleted from the original instrument and the threefactor model was subjected to further CFA. Accordingly, items 4, 6, 12, 13, 16 and 22 were deleted in different combinations and assessed for their goodness of fit, taking previous validation studies as evidence (11,(19)(20). When the items 6, 13, 16 and 22 were deleted from the three-factor model, it yielded the best goodness of fit parameters out of all (  2 =185.2;  2 /df=1.4; RMSEA=0.0748; CFI=0.974; NNFI=0.970; SRMR=0.0554; GFI=0.985). Thus, this 18-item model was selected as the best model fitting the data and used for the subsequent components of the study. Factor loadings of all the items included in the final instrument were equal to or greater than 0.4 for the particular factor concerned.
The final instrument comprising six items in the emotional exhaustion subscale, four items in the depersonalization subscale and eight items in the personal accomplishment subscale was assessed for criterion validity. Since the personal accomplishment was inversely related to presence of burnout and to the presence of emotional exhaustion and depersonalization sub-scales, its scoring was reverse scored, and the total score was calculated accordingly for burnout, in addition to taking the three-subscale scores separately. The mean score of the sample for MBI-HSS was 34.7, with a minimum score of two and a maximum score of 94. The ROC curve was obtained for different cutoff values of the total score and the sub-scale scores of the modified MBI-HSS. The diagnosis of the psychiatrist showed that 87 (32.6%) of the participants were suffering from clinical burnout. This diagnosis was taken as the state variable, and the total score of the modified MBI-HSS and the subscale scores as the test variables. Figure 1 depicts the ROC curve for the total score of MBI-HSS against the clinical diagnosis of the psychiatrist. The area under the curve was found to be 0.964, which was statistically different from the diagonal reference line.
The cut-off value corresponding to the minimum area under the curve was obtained from the calculations of the d 2 for each cut-off value, where d 2 was    and (1-specificity). The cut-off value obtained from the SPSS output for the lowest d 2 value was 36.5. At this cut-off value, the sensitivity was 94.3% and the specificity was 87.8%. Cut-off values for the three subscales of the MBI-HSS were also determined. Table  2 describes the summarized view of the cut-off values for each sub-scale score and the total score, with the relevant sensitivity, specificity, positive predictive value, negative predictive value, and the positive and negative likelihood ratios.
In the reliability assessment, the Chronbach's Alpha value for all the subscales of the MBI-HSS were above 0.7, while it was 0.862 for the total scale. All the correlation coefficients were above 0.7 in the assessment of test-retest reliability. Wijegoonewardene N et al. JCCPSL 2019, 25 (1) Open Access

Discussion
This study was conducted to validate the MBI-HSS for prison correctional and rehabilitation officers in Sri Lanka. It has not been validated for this occupational group previously. As it is understood that Sri Lankan prison officers undergo significant stress and subsequent burnout due to their distinct work setting and service recipients, previously validated cutoffs relevant to other professions and countries could not be applied to assess their burnout. The MBI-HSS was selected as the screening tool, since it was the most extensively used burnout scale among researchers worldwide. A meticulous translation procedure was adhered to, in order to ensure the content validity of the instrument. The panel of experts who assessed the content and consensual validity was carefully selected to cover all areas of interest with regard to the instrument. The calculated sample size was considered satisfactory since it was above the expected ratio of ten to one. All the pre-requisites for carrying out CFA were fulfilled, while using the Satorra-Bentler scaled chi-squared statistic to compensate for nonnormality of the data. Since using one psychiatrist had been the practice of previous studies validating instruments related to burnout using the diagnosis of job-related neurasthenia (13,(21)(22), this was followed in the current study as well, due to time and logistic constraints, while acknowledging the fact that the final agreed diagnosis of two psychiatrists would have been ideal.
The prisons selected for the study housed a vast variety of inmates, representing the inmate community in all kinds of prisons in the country. Therefore, it was assumed that the work conditions and duties in the prison correctional and rehabilitation officers in the country were up to a great extent represented by the prison officers of these prisons. Both correctional and rehabilitation officers were used in the study, since these were the two categories of officers who were directly, regularly and frequently involved in providing the routine services to the inmates.
The resultant superiority of the three-factor model over the one-and two-factor models was in accordance with other research conducted for assessing the construct validity of MBI-HSS among different occupational groups in other countries (15,20,(23)(24), as well as in Sri Lanka among nursing officers (13). The final item structure of the validated instrument which has the items 6, 13, 16 and 22 deleted, is similar to that developed in a Finnish study among nursing staff (23). The RMSEA value of 0.0748 in the current study was higher than that in the Finnish study, and above the close fit cut-off of 0.05, but managed to get in to the range of fair fit defined by Browne & Cudeck (25), and MacCullum, Browne & Sugawara (26).
The sensitivity and specificity values at the defined cut-offs for burnout and its three sub-scales were satisfactory, with all values above 70%. The Cronbach's Alpha values and correlation coefficient values above 0.7 proved the reliability of the instrument.

Conclusions and Recommendations
The validated and modified 18-item Sinhala version of the MBI-HSS was found to be a valid and reliable instrument to assess burnout among Sri Lankan prison correctional and rehabilitation officers. It is recommended that this instrument be used as a screening tool for measuring burnout in them and in future research as well as during their medical examinations conducted in the prison setting.