you can imagine” to “the best health you can imagine”. The EQ-5D-5L was introduced in Vietnam in 2012 with a wide range of target populations for example, people living with HIV, the elderly, people with non-communicable diseases, and young people suffering from internet addiction. However, until 2020 the EQ-5D-5L was developed value set based on social preferences obtained from a nationally representative sample in Vietnam (Mai et al., 2020). The researcher got permission to apply the EQ-5D-5L Vietnamese version in the present study by the author Vu Quynh Mai.
4.3.9. The Patient Health Questionnaire (PHQ-9)
The Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001) is a self-administered version of PRIME-MD diagnostic instrument for common mental health disorders. The PHQ-9 is the depression module which scores each of the 9 DSM-IV criteria as 0=Not at all, 1=Several days, 2=More than half the days, 3=Nearly every day in the past two weeks. In Vietnam, the validity and reliability of the Vietnamese version of PHQ-9 were assessed by Dang Duy Thanh et al. in 2011 (Dang D.T, 2011). The internal consistency of the scale returned to a Cronbach’s alpha coefficient of around 0.82 to 0.86 both in the community and hospital. We only used this questionnaire for validation purposes.
stage of study. We got permission to apply the instruments of BSRS-5, CIQ-R, and BRCS in the Vietnamese population from all the investigator authors (Callaway et al., 2016; Lee et al., 2003; Sinclair & Wallston, 2004). The patients’ characteristics, suicidality information, and help-seeking behavior were developed based on a literature review and experiences by the primary researcher and the supervisor. The main researcher had fifteen years’ experience in psychiatric clinical settings, and her supervisor - Professor Chia-Yi Wu, Vice CEO of the Taiwan Suicide Prevention Center and an expert in suicide prevention and treatment-resistant depression.
The questionnaire has been translated forward to Vietnamese and backward to English by three Vietnamese professionals in the medical field and one in the business profession. They are fluent in English, with IELTS scores from 6.5 – 8.0. The BSRS-5 was translated from Taiwanese by two Vietnamese who have been living in Taiwan for more than ten years to persuade undergraduate and master's degrees and currently working in Taiwan as government officers. The final version has agreements from all translators in each process (Figure 3). The content validity index (CVI) was assessed by a Vietnamese panel of five experts (including two psychiatrists and two psychiatric nurses with a clinical background and above master's degree level, and one community mental health researcher) who all have an average of 15 years of working experience in Vietnamese mental healthcare system. The experts were asked to give their viewpoints on each item and total scales. The CVI was calculated for all individual items (I-CVI) and all the scale (S-CVI). For CVI, the panel of experts was asked to rate each scale item in terms of its relevance (1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4
high clarity) and whether to keep it (1 = should not use this item, 2 = should keep this item). Prior to the calculation of CVI, the relevance rating must be recorded as 1 (representing a scale of 3 or 4) or 0 (representing to a scale of 1 or 2) (Yusoff, 2019).
There are two methods for calculating S-CVI: the average of the I-CVI scores for all items on the scale (S-CVI/Ave – a sum of I-CVI scores/number of items) and the proportion of items on the scale that receive a relevance score of 3 or 4 from all experts (S-CVI/UA = sum of UA score/number of item) are the two approaches (Yusoff, 2019).
I-CVI is determined by dividing the number of experts who rated each item as
"quite relevant" or "very relevant" by the total number of experts (Polit & Beck, 2021).
If I-CVI is greater than 0.79, the item is considered relevant; if it falls between 0.70 and 0.79, it requires revisions; and if it falls below 0.70, it is eliminated. S-CVI prioritizes the average quality of items over the average performance of experts. Recommendations call for a minimum S-CVI of 0.8 to indicate content validity (Polit & Beck, 2021; Yusoff, 2019).
As shown in Tables 3, 4, and 5, the overall CVI for both the relevance and clarity in the translation in the Vietnamese version was high. We have revised some questions as comments and suggestions from experts to make the content more appropriate for the Vietnamese cultural and healthcare system.
The researcher and her supervisor finalized the questionnaire with comments from the committee members before interviewing face to face five patients with TRD to assess the sustainability of the questionnaire and their comments about whether the questionnaire was difficult for them to answer. Each interview lasted around 20-25
minutes. The final version of the questionnaire was then approved to use in the main study (Appendix).
The researcher also assessed 100 nursing students at Hanoi Medical University and 154 depressive patients (including 53 TRD patients) in NIMH to validate the CIQ-R, BSRS-5, and BRCS scales. In total, the 254 participants were assessed for test-retest after one week of the first assessment (Table 6).
As we can see in Table 7, the Cronbach’s alpha values in the two groups differed.
The values for the students were higher in BSRS-5 and lower in CIQ-R, and BRCS compared to the depressive patients’ group. The Cronbach’s alpha values for the overall questionnaire was 0.83 (CIQ-R), 0.70 (BSRS-5), and 0.90 (BRCS), respectively. These values in our study are considered as sufficiently reliable (Nunnally & Bernstein, 1994).
In the test-retest reliability, the value for Pearson’s correlation coefficient varied across two groups ranging from 0.70 – 0.98 (p<0.05) which mean high reliability.
Concerning construct validity, we measured two groups, one of which is known to have a better scores and the other not (Parmenter & Wardle, 2000). Table 8 shows that the depressive patients’ group scored consistently higher than the students on BSRS-5 and lower on CIQ-R and BRCS (p<0.001). For example, the depressive patients had an average BSRS-5 score (mean±SD) of 9.12±3.35, and the students’ group had an average score of 5.57±3.10 out of the maximum score of 20. Moreover, depressive patients showed significantly lower scores compare to the students’ group in average CIQ-R (12.51± 6.92) and BRCS (11.97 ± 4.27) score with 22.96 ± 2.99 and 14.83 ± 2.06, respectively.
Table 9 shows the result of concurrent validity that people who had higher CIQ-R scores and BRCS also had a higher score in quality of life (EQ-5D-5L, EQ-VAS) and lower depressive levels (PHQ-9), who had a higher score in BSRS-5 had lower score in quality of life and higher score in a depressive level. These significant correlations indicate that CIQ-R, BSRS-5, BRCS and criterion tools administered at the same time have achieved concurrent validity. Hence, the three measurements can discriminate health-related outcomes among different expected groups and were valid to use in the next step of the present study.
Table 3. The relevance rating on the item scale for CIQ-R
Table 4. The relevance rating on the item scale for BSRS-5
Table 5. The relevance rating on the item scale for BRCS
Table 6. Gender and age of the sample participated in validation process
Variables Undergraduates (n=100)
Depressive patients (n=154)
Total (N=254)
n (%)/mean±SD n (%)/mean±SD n (%)/mean±SD
Gender
Male 6 (6%) 49 (31.8%) 55 (21.7%)
Female 94 (94%) 105 (68.2%) 199 (78.3%)
Age (years) min – max
20.22±1.26 18 – 26
46.39±12.67 18 – 69
36.09 ±16.18 18 – 69
Table 7. Internal consistency, test-retest reliability of CIQ-R, BSRS-5, and BRCS
Scales (no. of items)
Internal consistency reliability Test-retest reliability
(Cronbach’s alpha) Intraclass correlation coefficient (ICC) (95%CI) Undergraduates
(n=100)
Depressive patients (n=154)
Total (n=254)
Undergraduates (n=100)
Depressive patients (n=154)
Total (n=254) CIQ-R (18) 0.58 0.79 0.83 0.85 (0.77 – 0.90)* 0.97 (0.96 – 0.98)* 0.98 (0.97 – 0.98)*
BSRS-5 (6) 0.72 0.63 0.70 0.76 (0.59 – 0.85)* 0.62 (0.15 – 0.84)* 0.70 (0.19 – 0.85)*
BRCS (4) 0.71 0.91 0.90 0.82 (0.73 – 0.88)* 0.92 (0.89 – 0.94)* 0.92 (0.89 – 0.94)*
*p<0.001, CIQ-R = Community integration questionnaire revised, BSRS-5 = 5-item Brief symptom rating scale, BRCS = Brief resilience coping skill.
Table 8. Mean scores of CIQ-R, BSRS-5, and BRCS among two test groups
Undergraduates (n=100)
Depressive patients (n=154)
Differences between two groups
min – max mean±SD min – max mean±SD Mean difference p
CIQ-R 16 – 30 22.96±2.99 0 – 29 12.51±6.92 -10.45 0.00
BSRS-5 1 – 15 5.57±3.10 0 - 18 9.12±3.35 3.55 0.00
BRCS 8 – 20 14.83±2.06 4 – 20 11.97±4.27 -2.86 0.00
CIQ-R = Community integration questionnaire revised, BSRS-5 = 5-item Brief symptom rating scale, BRCS = Brief resilience coping skill.
Table 9. Bivariate correlation between CIQ-R, BSRS-5, and BRCS with criterion variables (N=254) Correlations
CIQ-R BSRS-5 BRCS EQ-5D-5L EQ-VAS PHQ-9
CIQ-R 1
BSRS-5 -.458** 1
BRCS .522** -.458** 1
EQ-5D-5L -.679** .636** -.533** 1
EQ-VAS .617** -.577** .351** -.727** 1
PHQ-9 -.455** .776** -.504** .687** -669** 1
**p<0.01, CIQ-R = Community integration questionnaire revised, BSRS-5 = 5-item Brief symptom rating scale, BRCS = Brief resilience coping skill, EQ-5D-5L = EuroQoL 5-Dimensions 5-Levels, EQ-VAS = Visual analogue scale, PHQ-9 = The Patient Health Questionnaire.
Figure 5. Instrument validation procedure (1) Forward
translation: Two bilingual translators whose first language is Vietnamese
independently
translated the English and Chinese version to Vietnamese. Each produced a Vietnamese translation of the questionnaire.
(2) Compilation of a single translated versions: The research team worked with one the translator in (1) to compare the two translated version and compile a single
translated version which is understandable of words and sentences. It produced an initial Vietnamese version.
(3) Blind back-translation: A third, Vietnamese-native, bilingual translator reverse-translated the initial translation from Vietnamese to English.
This translator was unable to comprehend the original text. It produced a blind back-translated version.
(4) Comparison of the back-translated with original versions: The research team compared the original text with a blind back-translation.
This process involved evaluating the
differences between two versions of the original questionnaire items and their relevance. It produced a pre-final Vietnamese translation.
(5) Evaluation: The pre-final Vietnamese translation were evaluated and adapted in terms of the
instruction, items and response format. Five experts adapted the questionnaire based on local information, context, and culture.
(6) Content validity:
On a scale, the five experts were asked to assess the content-related validity of each translated instrument item. The expert’s endorsement was collected and the content validity index (CVI) score was estimated.
(7) Pilot interview: Five patients were chosen for the pilot interview to assess the sustainable of the questionnaire and their comments about whether the questionnaire difficult for them to answer.
(8) Final version: Research team produced the final Vietnamese instrument use in this study.
Report validation scores:
Reliability and validity.