eISSN: 1897-4317
ISSN: 1895-5770
Gastroenterology Review/Przegląd Gastroenterologiczny
Bieżący numer Archiwum Artykuły zaakceptowane O czasopiśmie Rada naukowa Bazy indeksacyjne Prenumerata Kontakt Zasady publikacji prac
Panel Redakcyjny
Zgłaszanie i recenzowanie prac online
NOWOŚĆ
Portal dla gastroenterologów!
www.egastroenterologia.pl
SCImago Journal & Country Rank
3/2023
vol. 18
 
Poleć ten artykuł:
Udostępnij:
Artykuł oryginalny

Translation and validation of the Ukrainian version of the visceral sensitivity index for patients with irritable bowel syndrome

Artem Neverovskyi
1
,
Vadym Shypulin
1
,
Nataliya Mikhnova
1

1.
Department of Internal Medicine No. 1, Bogomolets National Medical University, Kyiv, Ukraine
Gastroenterology Rev 2023; 18 (3): 313–319
Data publikacji online: 2023/09/22
Plik artykułu:
Pobierz cytowanie
 
Metryki PlumX:
 

Introduction

Irritable bowel syndrome (IBS) is one of the most prevalent functional gastrointestinal diseases in the world, affecting around 11% of the population [1], with regional variations. In Ukraine the estimated prevalence of IBS is 16–20%, which is higher than in other European countries [2]. There are a lot of possible proposed mechanisms of development of functional gastrointestinal (GI) disorders, including IBS: dysmotility [3], visceral hypersensitivity [4, 5], immune dysregulation, inflammation, barrier dysfunction, altered gut microbiota [6], and gut-brain axis disturbances [7]. Psychological distress plays an important role in the development of IBS and in aggravating its symptoms [8]. It was shown that comorbidity of functional GI disorders with depression is seen in about 30% of patients, and with anxiety – in about 30–50% [8, 9]. Moreover, patients with functional GI disorders have reduced quality of life and unsatisfactory experience with health care providers [10]. It should also be noted that there is a bidirectional impact: psychological distress, depression, and anxiety might be both the causes and consequences of IBS, leading to a vicious circle [8].

Anxiety by itself doubles the risk of development of IBS [11], but at the same time a lot of patients with IBS do not have diagnosed clinical anxiety or depression [9]. Hence, it was suggested that anxiety about specific GI symptoms and situations related to gut functioning is more suitable for assessing symptoms, severity, and outcomes in functional GI disorders including IBS [12]. Labus et al. [13] defined the term gastrointestinal-specific anxiety (GSA) as “the cognitive, affective, and behavioural response stemming from fear of GI sensations, symptoms, and the context in which these visceral sensations and symptoms occur”. In 2004 Labus et al. [14] developed a specific scale allowing assessment of gastrointestinal-specific anxiety (GSA): the Visceral Sensitivity Index (VSI). The VSI is a reliable and valid brief tool that can be used to assess the severity of IBS and response to treatment in routine GI practice [13, 14]. The VSI was translated into Norwegian [15] and Japanese [16], and the validity and reliability of these versions were confirmed. It should be emphasized that cross-cultural factors in functional GI-diseases are important, especially the language that patients speak, which influences the different expression and understanding of symptoms and feelings [17]. Therefore, the development and translation of different tools into various languages is very important for adequate assessment of IBS in particular.

Unfortunately, there is no Ukrainian version of the VSI. However, Ukrainian internists and gastroenterologists are faced with the need for such an important tool in the management of patients with IBS and GSA, considering the possible future increase of their prevalence.

Aim

The aim of present study was to translate the original English version of the VSI into Ukrainian language and to test its validity and reliability in patients with IBS.

Material and methods

The present study was conducted in accordance with Ukrainian law, the requirements of Good Clinical Practice, and the ethical principles of the Declaration of Helsinki. All the participants signed a written informed consent form before the start of the investigation. The protocol of the trial was approved by the Bioethical Committee of Bogomolets National Medical University, Kyiv, Ukraine.

Study population

Patients of both sexes, aged 18–44 years, with diagnosed IBS. The Rome IV criteria [18] were used for the diagnosis of IBS. Participants were enrolled in the study from November 2021 to April 2022. The inclusion criteria were as follows: men and women, aged 18–45 years, with diagnosed IBS (using a positive strategy of making the diagnosis), normal levels of faecal calprotectin, negative faecal transferrin and/or haemoglobin test, negative celiac serology, negative pregnancy test result, and signed written informed consent. The exclusion criteria were as follows: patients aged > 45 years, organic disorders of the GI tract, especially IBD, elevated levels of faecal calprotectin, positive tests on faecal transferrin and/or haemoglobin, positive celiac serology, positive pregnancy test, and lactation.

The study was conducted on a cross-sectional design. A total of 121 patients were selected to participate in the investigation. Each participant it was asked to complete the following questionnaires: translated Ukrainian version of the VSI, Ukrainian versions of the Patient Health Questionnaire-9 (PHQ-9) [19, 20], Beck’s Depression Inventory (BDI) [21, 22], and the Hospital Anxiety and Depression Scale (HADS) [20, 23]. Thirteen patients refused to fill in all the questionnaires used in the study due to personal circumstances and prejudices. A total of 108 patients completed all the questionnaires and were included in the statistical analysis.

Translation of the Ukrainian version of the VSI (VSI-UA)

Before starting the translation of the VSI into Ukrainian, consent from the author (Dr. Emeran A. Mayer) of the original English version was obtained via e-mail.

The VSI is considered as a reverse scale and includes 15 questionnaire items, which range from 1 – “strongly agree” to 6 – “strongly disagree”. These are then reverted to 5 – 0 points (Table I) So, the higher the VSI score, the more severe the GSA [14].

Table I

Original English version of the VSI [14]

ItemStrongly agreeModerately agreeMildly agreeMildly disagreeModerately disagreeStrongly disagree
1. I worry that whenever I eat during the day, bloating and distension in my belly will get worse.123456
2. I get anxious when I go to a new restaurant.123456
3. I often worry about problems in my belly.123456
4. I have a difficult time enjoying myself because I cannot get my mind off the discomfort in my belly.123456
5. I often fear that I won’t be able to have a normal bowel movement.123456
6. Because of fear of developing abdominal discomfort, I seldom try new foods.123456
7. No matter what I eat, I will probably feel uncomfortable.123456
8. As soon as I feel abdominal discomfort I begin to worry and feel anxious.123456
9. When I enter a place I haven’t been before, one of the first things I do is to look for a bathroom.123456
10. I am constantly aware of the feelings I have in my belly.123456
11. I often feel discomfort in my belly could be a sign of a serious illness.123456
12. As soon as I awake, I worry that I will have discomfort in my belly during the day.123456
13. When I feel discomfort in my belly, it frightens me.123456
14. In stressful situations, my belly bothers me a lot.123456
15. I constantly think about what is happening inside my belly.123456

The translation and cultural adaptation of the Ukrainian version of the VSI was performed according to recommendations of the Professional Society for Health Economics and Outcomes Research (ISPOR) [24]. The translation and adaptation process included the stages of direct and back translation with the next pilot testing and cognitive analysis of the obtained data. The direct translation from English to Ukrainian was performed by 2 Ukrainian gastroenterologists with basic fluency in English (B2 level according to requirements of CEFRL) and native Ukrainian language. During the translation, the conceptual adaptation of the text was preferred over the literal translation. Then, a bilingual physician with native English conducted back-translation of the Ukrainian version into English. The obtained version was analysed and compared to the original one. After corrections of the linguistic differences, pilot testing on 10 patients with IBS, and cognitive analysis of the obtained results, the final Ukrainian version of the VSI was established and accepted by consensus agreement between the translators (Table II).

Table II

The Ukrainian version of the VSI (VSI-UA)

ПитанняПовністю згоднийПомірно згоднийТрохи згоднийТрохи не згоднийПомірно не згоднийПовністю не згодний
1. Я переживаю, що кожного разу, коли я вживаю їжу, здуття і розпирання в животі будуть посилюватися.123456
2. Я хвилююся, коли йду в новий ресторан.123456
3. Я часто переживаю про свої проблеми з животом123456
4. Мені важко насолоджуватись чимось, тому що я не можу відволіктися від дискомфорту в животі.123456
5. Я часто боюся, що не зможу нормально випорожнити кишечник.123456
6. Я рідко пробую нові продукти харчування через страх виникнення дискомфорту в животі.123456
7. Незалежно від того, що я їм, ймовірно, я відчуватиму себе дискомфортно.123456
8. Як тільки я відчуваю дискомфорт у животі, я починаю хвилюватися і відчувати тривогу.123456
9. Коли я відвідую місце, де раніше не був/була, то першим ділом шукаю туалетну кімнату.123456
10. Я постійно відслідковую свої відчуття в животі.123456
11. Я часто думаю, що дискомфорт у животі може бути симптомом серйозного захворювання.123456
12. Одразу після прокидання я починаю хвилюватись, що протягом дня відчуватиму дискомфорт у животі.123456
13. Коли я відчуваю дискомфорт у животі, це мене лякає.123456
14. У стресових ситуаціях мене дуже турбує живіт.123456
15. Я постійно думаю про те, що відбувається в моєму животі.123456

Patient Health Questionnaire-9 (PHQ-9)

PHQ-9 is a 9-item self-assessment scale which is effective in the diagnosis of depression disorders [19]. The Ukrainian version of this scale is approved by the Ministry of Health of Ukraine [20]. The total score can range from 0 (absence of depression) to 27 (severe depression). It was shown that PHQ-9 is one of the screening tools to assess depression in patients with IBS [25], and it can be used for the assessment of the construct validity of the VSI-UA.

Beck’s Depression Inventory (BDI)

The BDI is 21-item self-assessment scale effective in the diagnosis and evaluation of the severity of depression [21]. The Ukrainian version of this scale has been validated in previous studies [22]. The total score can be from 0 (absence or minimal depression) to 63 (severe depression). In previous investigations the BDI was used for assessment of depression in patients with IBS [26] and may be used for the evaluating of construct validity of the VSI-UA.

Hospital Anxiety and Depression Scale (HADS)

The HADS is a 14-item self-assessment scale effective in the diagnosis and evaluation of depression and anxiety in the general population of patients (non-psychiatric) [23]. Half of the questions from this scale relate to depression (HADS-Dep) and the other half to anxiety (HADS-Anx). Total score of each ranges from 0 (absence of depression/anxiety) to 21 (severe clinical cases of depression/anxiety). Previous studies on the validation of the English version of the VSI showed correlation with HADS-Dep and HADS-Anx [13]. Therefore, HADS may be used for evaluation of the construct validity of the VSI-UA.

Statistical analysis

Statistical analysis was performed using SPSS (version 23, IBM Corp., Armonk, NY, USA). The obtained data were presented as mean and standard deviation (mean ± SD) in the case of normal distribution, or as median with first and third quartiles (median (Q1–Q3)) in the case of non-normal distribution of the obtained data. Assessment of reliability and validity of translated Ukrainian version of the VSI was conducted according to the ‘Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine’ [27].

Reliability was checked by the following:

  1. Assessment of the internal consistency with coefficient α (Cronbach’s α) [28].

  2. Method of test-retest reliability [29] with checking of the intraclass correlation coefficient (ICC) between the first and 4 weeks after administration of VSI-UA to the same participants.

Validity was checked by the following:

  1. Assessment of content validity by Lawshe method [30]. Twelve Ukrainian experts in gastroenterology and functional GI diseases were proposed to answer questions about each item from the VSI-UA: “Is this item ‘essential’, ‘useful, but not essential’, or ‘not necessary’ to the assessment of gastrointestinal-specific anxiety?”. Then, the content validity ratio (CVR) was calculated for each item using the following equation: CVR = (neN/2)/(N/2), where ne = number of experts that answer on the question “essential”, and N = total number of experts. If the CVR ≤ 0.56, such an item is unlikely measure the content – GSA of the VSI. Then, to measure the overall content validity of the VSI-UA, the content validity index (CVI) was calculated as the mean CVR of all items of the VSI-UA.

  2. Assessment of construct validity by estimation of correlation coefficients (Pearson’s r) between the VSI-UA and PHQ-9, BDI, HADS-Anx, and HADS-Dep.

For all the statistical methods used in the study, the difference between variables was considered statistically significant at p < 0.050.

Results

The flow-chart of the study is represented in Figure 1. Demographic and clinical data are represented in Table III.

Figure 1

Flow chart of the study

/f/fulltexts/PG/51452/PG-18-51452-g001_min.jpg
Table III

Demographic and clinical data of the study population

VariableResult*
Age30.5 (25.5–37)
Sex [n (%)]
Male
31 (28.7%)
VSI-UA baseline35.5 (31–41)
VSI-UA after 4 weeks35.5 (31–40.5)
PHQ95.7 ±1.8
BDI15 (12–18)
HADS-Anx6 (5–9)
HADS-Dep4 (2–6)

* Normally and non-normally distributed data were presented as mean ± standard deviation (SD) and median (Q1–Q3), respectively.

The mean VSI-UA baseline was 35.5 (31–41), without any significant difference between men (35.9 ±8.7) and women (35.9 ±6.4), p = 0.98. A similar tendency was seen after repeated administration of VSI-UA 4 weeks later: the mean level was 35.5 (31–40.5), without a significant difference between men (34 (29–42)) and women (36 (31–40)), p = 0.55. There were no differences in the VSI level between the IBS subtypes, p > 0.05.

Reliability

The obtained Cronbach’s α for VSI-UA was 0.84, indicating good internal consistency of the questionnaire. The intraclass correlation coefficient (ICC) between the first and repeated 4 weeks after administration of VSI-UA was 0.92 (95% CI: 0.87–0.95), indicating good reliability and stability of the VSI-UA.

Content validity

The calculated CVR for each item of the VSI-UA was higher than the minimal critical value of 0.56, meaning that all items are likely measure the content of the VSI-UA. The calculated CVI was 0.94.

Construct validity

The Pearson’s correlation coefficients (correlation matrix) are represented in Table IV. Moderate positive correlation was found between VSI-UA and PHQ-9 (r = 0.65), BDI (r = 0.69), HADS-Anx (r = 0.61), and HADS-Dep (r = 0.48), p < 0.05 in all correlations (Table IV).

Table IV

Correlation matrix (Pearson’s r) for construct validation of the VSI-UA (p < 0.05 in all the coefficients)

VSI-UAPHQ-9BDIHADS-AnxHADS-Dep
VSI-UA0.650.690.610.48
PHQ-90.650.770.730.8
BDI0.690.770.690.83
HADS-Anx0.610.730.690.66
HADS-Dep0.480.80.830.66

Discussion

In the present study we translated the original English version of the VSI into Ukrainian language and checked its reliability and validity in a cohort of IBS patients. We revealed that the mean level of the VSI-UA among the IBS patients was 35.5, which is comparable with the data from previous studies (from 25.1 to 38) [13, 15, 16].

We showed that Cronbach’s α for the VSI-UA was 0.84, which is comparable with the original version (α = 0.93) [14], further validation of the English version (α = 0.9 and 0.92 on 2 samples) [13], as well as translated versions of the VSI (α = 0.93 and α = 0.93) [15, 16]. This means that the VSI-UA has good internal consistency, confirming its reliability. Also, with the aim of additionally checking the reliability of the VSI-UA, we used the method of test-retest reliability. Four weeks after the first administration of the VSI-UA the same patients with IBS were asked to fill in the same questionnaire again. It was found that the interclass correlation coefficient between the first and second administration of the VSI-UA was 0.92, meaning very strong correlation. These findings are comparable with the results from a previous study on the validation of the Norwegian version of the VSI (ICC = 0.86) [15], and they confirm the stability of the VSI-UA and its reliability.

Validation of the Ukrainian version of VSI was done by the methods of content and construct validity. Content validity means the degree to which a questionnaire assesses the construct that is developed to evaluate, for example, gastrointestinal specific anxiety in the present investigation [27]. In our study, content validation was performed with the involvement of 12 gastroenterologists who are experts in functional GI diseases, and who evaluated all 15 items of the VSI-UA. It was revealed that the content validation ratio for each item exceeded the minimal critical value, meaning that all items are suitable for assessment of construct – GSA and may be used together in one questionnaire. The overall content validation index, i.e. the mean of CVR of all items, was 0.94, indicating a satisfactory level of content validity of the VSI-UA. Construct validity means the degree to which a questionnaire assesses the concept that it is developed to evaluate [27]. In our study we used the evaluation of convergent validity by assessment of the correlation of the VSI-UA with the internationally accepted scales of evaluating anxiety and depression – PHQ-9, BDI, HADS-Anx, and HADS-Dep. Theoretically, they must correlate positively, and before starting the trial we made the assumption about such a correlation. It was shown that the VSI-UA had moderate positive correlation with all these scales, with higher correlation indexes with PHQ-9, BDI, and HADS-Anx (0.65, 0.69, 0.61, respectively) and lower with HADS-Dep (0.48). Previous studies on validation of the original English version of the VSI also showed positive correlation with HADS-Anx (from 0.34 to 0.73) [13, 14] and HADS-Dep (0.25) [13], while studies on the validation of translated versions of the VSI revealed positive correlation only with HADS-Anx (from 0.16 to 0.48) [15, 16] without significant correlation with HADS-Dep. It should be noted that in previous studies on the validation of the original and translated versions of the VSI, the authors used other questionnaires to assess the construct validity, e.g. the Anxiety Sensitivity Index (ASI), Irritable Bowel Syndrome Symptom Questionnaire (IBS-SQ), IBS Severity Index (IBS-SI), and Symptom Severity Indices [1316]. Unfortunately, there are no Ukrainian versions of these scales, so we did not have the opportunity to use them in our study and consequently compare them with previous studies. On the other hand, in our study we used the PHQ-9 and BDI scales, which measure depression, and which were not used in previous studies on validation of the VSI [1316]. Several investigations have shown that depression is quite prevalent among patients with IBS and that it correlates with anxiety [8, 9], and PHQ-9 and BDI may be used as screening tools for assessment of depression in IBS patients [25, 26]. So, in our opinion, these scales are suitable for the assessment of construct validity of the VSI that was shown in our study.

Due to the presence of some limitations of this study, like the absence of involvement of healthy adults, quite a small study population, and the inability to use some scales to evaluate construct validity, its results should be confirmed and expanded in future trials with additional assessment of reliability and validity. The Visceral Sensitivity Index is a good tool for evaluating anxiety about GI symptoms, which is helpful in real practice for the management of patients with IBS, making the decision about next line of treatment options (e.g. neuromodulators and psychotherapy), and assessment of the effectiveness of therapeutic interventions, including in clinical trials etc. So, the adaptation and implementation of the national versions of the VSI, like the Ukrainian one, would be an effective approach in improving the medical care for IBS patients and theoretically other functional GI disorders associated with anxiety and GSA.

Conclusions

The translated Ukrainian version of the Visceral Sensitivity Index is a reliable and valid tool for the assessment of gastrointestinal-specific anxiety in Ukrainian-language patients with irritable bowel syndrome. The VSI-UA may be widely implemented in routine clinical practice among Ukrainian GI specialists when managing patients with IBS, and assessing the effectiveness of current treatment options and newly developed approaches.

Acknowledgments

We are grateful to all the experts and translators who participated in the translation and content validation of the VSI-UA.

Conflict of interest

The authors declare no conflict of interest.

References

1 

Canavan C, West J, Card T. The epidemiology of irritable bowel syndrome. Clin Epidemiol 2014; 6: 71-80.

2 

Dumitrascu DL, Babin A, Bordin DS, et al. Recent data on irritable bowel syndrome from some Central and East European countries. J Gastrointest Liver Dis 2020; 29: 247-50.

3 

Boeckxstaens G, Camilleri M, Sifrim D, et al. Fundamentals of neurogastroenterology: physiology/motility–sensation. Gastroenterology. 2016; S0016-5085(16)00221-3. doi:10.1053/j.gastro.2016.02.030

4 

Mayer EA, Gebhart GF. Basic and clinical aspects of visceral hyperalgesia. Gastroenterology 1994; 107: 271-93.

5 

Farzaei MH, Bahramsoltani R, Abdollahi M, Rahimi R. The role of visceral hypersensitivity in irritable bowel syndrome: pharmacological targets and novel treatments. J Neurogastroenterol Motil 2016; 22: 558-74.

6 

Drossman DA. Functional gastrointestinal disorders: history, pathophysiology, clinical features and Rome IV. Gastroenterology 2016; S0016-5085(16)00223-7. doi:10.1053/j.gastro.2016.02.032

7 

Ancona A, Petito C, Iavarone I, et al. The gut-brain axis in irritable bowel syndrome and inflammatory bowel disease. Dig Liver Dis 2021; 53: 298-305.

8 

Van Oudenhove L, Crowell MD, Drossman DA, et al. Biopsychosocial aspects of functional gastrointestinal disorders. Gastroenterology 2016; S0016-5085(16)00218-3. doi:10.1053/j.gastro.2016.02.027.

9 

Hu Z, Li M, Yao L, et al. The level and prevalence of depression and anxiety among patients with different subtypes of irritable bowel syndrome: a network meta-analysis. BMC Gastroenterol 2021; 21: 23.

10 

Houghton LA, Heitkemper M, Crowell M, et al. Age, gender and women’s health and the patient. Gastroenterology 2016; S0016-5085(16)00183-9. doi:10.1053/j.gastro.2016.02.017.

11 

Sibelli A, Chalder T, Everitt H, et al. A systematic review with meta-analysis of the role of anxiety and depression in irritable bowel syndrome onset. Psychol Med 2016; 46: 3065-80.

12 

Jerndal P, Ringström G, Agerforz P, et al. Gastrointestinal-specific anxiety: an important factor for severity of GI symptoms and quality of life in IBS. Neurogastroenterol Motil 2010; 22: 646-e179.

13 

Labus JS, Mayer EA, Chang L, et al. The central role of gastrointestinal-specific anxiety in irritable bowel syndrome: further validation of the visceral sensitivity index. Psychosom Med 2007; 69: 89-98.

14 

Labus JS, Bolus R, Chang L, et al. The Visceral Sensitivity Index: development and validation of a gastrointestinal symptom-specific anxiety scale. Aliment Pharmacol Ther 2004; 20: 89-97.

15 

Lind R, Lied GA, Lillestøl K, et al. Do psychological factors predict symptom severity in patients with subjective food hypersensitivity? Scand J Gastroenterol 2010; 45: 835-43.

16 

Saigo T, Tayama J, Hamaguchi T, et al. Gastrointestinal specific anxiety in irritable bowel syndrome: validation of the Japanese version of the visceral sensitivity index for university students. Biopsychosoc Med 2014; 8: 10.

17 

Fang X, Francisconi CF, Fukudo S, et al. Multicultural aspects in functional gastrointestinal disorders (FGIDs). Gastroenterology 2016; S0016-5085(16)00179-7.

18 

Mearin F, Lacy BE, Chang L, et al. Bowel disorders. Gastroenterology 2016; S0016-5085(16)00222-5. doi:10.1053/j.gastro.2016.02.031.

19 

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606-13.

21 

Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry 1961; 4: 561-71.

22 

Нікітчук У.. Психометричні властивості української версіїопитувальника депресії А. Бека (BDI-I) на студентськійвибірці. Psychol J 2020; 6: 56-68.

23 

Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983; 67: 361-70.

24 

Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health 2005; 8: 94-104.

25 

Snijkers JTW, van den Oever W, Weerts ZZRM, et al. Examining the optimal cutoff values of HADS, PHQ-9 and GAD-7 as screening instruments for depression and anxiety in irritable bowel syndrome. Neurogastroenterol Motil 2021; 33: e14161.

26 

Kopczyńska M, Mokros Ł, Pietras T, Małecka-Panas E. Quality of life and depression in patients with irritable bowel syndrome. Gastroenterology Rev 2018; 13: 102-8.

27 

Tsang S, Royse CF, Terkawi AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth 2017; 11 (Suppl 1): S80-9.

28 

Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297-334.

29 

Crocker L, Algina J. Introduction to Classical and Modern Test Theory. Mason, Ohio: Cengage Learning; 2008.

30 

Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975; 28: 563-75.

31 

Cronbach L, Meehl P. Construct validity in psychological tests. Psychol Bull 1955; 52: 281-302.

Copyright: © 2023 Termedia Sp. z o. o. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License (http://creativecommons.org/licenses/by-nc-sa/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material, provided the original work is properly cited and states its license.
 
© 2024 Termedia Sp. z o.o.
Developed by Bentus.