Quality and content analyses of YouTube videos related to medication-related osteonecrosis of jaws

Introduction

Marx [1] first defined bisphosphonate-related osteonecrosis of the jaw (BRONJ) in 2003, and since 2014, the condition’s scope has been expanded under the term ‘medication-related osteonecrosis of the jaw’ (MRONJ) [2]. MRONJ is defined as the presence of exposed jaw bones that do not heal in patients with no history of radiation therapy, which last longer than eight weeks, and is associated with the use of anti-angiogenic and anti-resorptive drugs [3]. MRONJ can occur at different times and intensities depending on the duration, dosage, route of administration (oral/i.v.), and nitrogen content of the bisphosphonate drug as well as the presence of an accompanying systemic disease, patient’s oral hygiene, and dental interventions. MRONJ may develop spontaneously at a very high-rate of 40%, generally observed after dento-alveolar surgeries, such as tooth extraction [4]. However, the prognosis for spontaneously developed MRONJ lesions has been reported to be bad [4]. Bisphosphonates, which are important in the development of MRONJ, are chemotherapeutic anti-resorptive drugs, which affect morphology and activity of bone cells [5]. Bisphosphonates are used in the treatment of osteoporosis, metabolic bone diseases, prostate cancer, and breast cancer [6]. Bisphosphonates affect bone metabolism by suppressing osteoblastic activity responsible for bone destruction. In addition, bisphosphonates have anti-tumoral and anti-angiogenic proprieties by inhibiting tumor invasion and inducing apoptosis of tumor cells [7]. Recently, other drugs (e.g., denosumab) applied in osteoporosis and cancer treatments have been associated with MRONJ [6, 8]. Denosumab is used in osteoporosis and the treatment of bone metastases of solid tumors, such as breast and prostate cancers [8]. It is noted that increase in the incidence of drugs’ complications has been associated with increased prescription rates by oncologists in recent years [9]. Today, Internet has become a popular source of health-related information. With increasing accessibility, Internet and social media play an increasingly major role in ways people communicate [10]. YouTube is an online public communications website owned by Google, where registered users can upload, share, view, and comment videos [11]. It is known that visual information sources, such as YouTube, are much appealing than readable sources [12]. Although the original intent of YouTube creation was entertainment rather than educational purposes, it later became a subject of research for academicians due to the supply of medical information and interest from the patients.

Objectives

Patients use the Internet not only to find answers to medical problems, but also to research treatment alternatives for their conditions. However, the presence of inaccurate, incomplete, and unregulated information on the Internet is one of its biggest disadvantages. People who access health-related information on the Internet do not focus on reliability or accuracy of videos or their creators [10, 11]. The aim of this study was to investigate the quality and accuracy of information provided by YouTube videos about MRONJ. Material and methods Videos of interest were retrieved from YouTube (https://www.youtube.com) on June 9, 2020, using two search terms, including ‘MRONJ (medication-related osteonecrosis of the jaw)” and ‘bisphosphonate-related osteonecrosis of the jaw’. Sort by relevance was used as default filter setting for YouTube search. It has been stated that more than 90% of YouTube users select a video that appears in the first three pages of search results to obtain the information they are searching for, and 79% of users who do not find what they are looking for on the first page, look at other pages [13]. Out of 49,000 results, the first 150 videos were recorded for evaluation. Videos not related to the subject and non-English language videos were excluded. Videos with audio or visual quality problems as well as duplicate videos were also excluded. Distribution of excluded videos is summarized in Figure 1. A total of 80 videos were included in the study, and their sources were divided into six groups: 1) academic health institutions/professional organizations; 2) doctors; 3) health sites; 4) news broadcasted channels; 5) unknown sources, and 6) other (e.g., medical advisor, nurse, patient, legal advisor). Moreover, videos were classified in terms of their content: 1) education/training videos; 2) patient experiences; 3) general information about MRONJ for dentists; 4) physician- patient experiences; 5) surgery and dental treatments related to MRONJ; 6) informing patients; 7) other (e.g., drugs, advertising, legal/financial issues). All videos were fully viewed. Data of upload, number of views, length, numbers of likes and dislikes, number of comments, and target audience (specialist or dentist vs. patient) were recorded. Interaction scores were calculated using the following formulas for interaction index and viewing rate, respectively. Interaction index = Number of likes − Number of dislikes × 100% Number of views Viewing rate = Number of views × 100% Number of days since upload
The YouTube content was evaluated using a six-item list of evaluation criteria: A) definition; B) risk factors; C) diagnosis; D.) findings/symptoms; E) prognosis; F) treatment methods. For each content area, videos were scored as 1 point (included) or 0 points (not included). The sum of points accrued resulted in total content score. Videos scoring 0-3 points were classified as low-content videos, and videos scoring 4-6 points were classified as high- content videos. Video information and quality index (VIQI) was used to evaluate the overall quality of video. VIQI uses a five-point Likert scale, where 1 represents ‘poor quality’ and 5 represents ‘high quality’. Criteria of VIQI are listed below: flow of information (flow); information accuracy (accuracy); video quality (one point each for each interview with people in a community, presence of video captions, use of still images, use of animation, and presence of a report summary) (quality); level of consistency between content and video title (precision). Videos were categorized into three groups according to their usefulness (highly useful, moderately, or not useful). If the YouTube videos had information about the definition, diagnostic criteria, clinical symptoms, etiology, and treatment methods of MRONJ/ BRONJ, then the videos were categorized into highly useful group. If they contained one or two of the above-mentioned criteria, they were classified as moderately useful. If none of the criteria were mentioned and the relationship with the terms was low, they were considered as not useful. Ethical approval was not required for this study because it contained publicly available data only. Two researchers (GA: an oral and maxillofacial radiologist with 7 years’ experience, and DY: a maxillofacial surgeon with 7 years’ experience) assessed the videos for scientific accuracy (MRONJ and BRONJ definition, etiology, diagnosis, treatment modalities) and video’s content. Results were compared, and then consolidated. The American Association of Oral and Maxillofacial Surgeons position paper on medication-related osteonecrosis of the jaw [9] has been accepted as scientific data guide for evaluation of the videos. Furthermore, the videos were viewed on two separate occasions, approximately two weeks apart by both viewers, using website links stored at the time of first evaluation. Disagreements between the two researchers were arbitrated by a third researcher (KG), with knowledge about MRONJ.

Statistical analysis

All statistical analyses were performed using IBM SPSS Statistics, version 23.0 program (IBM; Armonk, New York, USA). Significance level was set as p = 0.05. Normality of data was analyzed using Shapiro-Wilk and Kolmogorov-Smirnov goodness-of-fit tests. Due to violation of normality assumption, non-parametric statistical comparison methods were applied. To compare low- and high-content score videos concerning properties of the videos, Mann-Whitney U-test was used. Comparison of low- and high-content score videos was performed with Fisher’s exact test. Mann-Whitney U-test was applied to compare scores of these videos concerning quantitative characteristics and VIQI. Effect of usefulness on video’s views and popularity was analyzed using Kruskal- Wallis test. Also, Kruskal-Wallis test was applied to analyze the effect of usefulness on quantitative characteristics and VIQI of videos. Comparison of categorical variables was performed with c2 test and Fisher’s exact test, and correlation between characteristics of videos was analyzed using Kendall’s tau-b correlation coefficient.

Results

A total of 80 videos were examined and analyzed. VIQI results of YouTube videos and their quantitative characteristics are listed with descriptive analysis in Table 1. 23.8% of the videos were prepared to inform patients, and 22.5% of them were educational videos for healthcare professionals/doctors. Most of the videos were found to be uploaded by medical doctors and dentists, or maxillofacial surgeons (38.8%), followed by academic health institution/professional organizations (22.5%; Table 2). Concerning country of origin of the videos, USA was the country that produced the largest number of videos on MRONJ (65%). Other videos mainly came from UK (6.3%), India (3.8%), Greece (3.8%), other countries (10%), and in 11.1%, the country of origin was not determined (Table 3). The test results showed that most of the high-content score videos were loaded by academic health institution/professional organizations (p < 0.05). Significant differences in the length of videos, the number of views, and likes were identified between low and high score videos (p < 0.05). Moreover, VIQI values were compared between both video groups, and high-content videos were found statistically significant over 5 parameters (flow, information, quality, precision, and total score), with p < 0.05. The effect of usefulness (high, moderate, or not useful) on quantitative characteristics and VIQI of videos was analyzed. These results are presented in Table 4. The videos for academic health institution/ professional organizations had the highest score in terms of usefulness. For the length of videos, likes, interaction index, and viewing rate, significant differences were observed between three levels of usefulness (p < 0.05). The correlation matrix is shown in Table 5. According to the correlation coefficients, VIQI was positively correlated to the length of video, the number of views/ likes, interaction index and viewing rate, total content score, and usefulness degree of YouTube videos. Among these parameters, the relationship between VIQI and usefulness (r = 0.801, p = 0.000) was observed as high. Moreover, a positive correlation was found between the length of video and the interaction index (r = 0.226, p = 0.005). Additionally, the length of video correlated positively with the number of likes (r = 0.216, p = 0.008).

Discussion

In the recent literature, surgical procedures, such as tooth extraction or dental implant operations in patients using anti-resorptive and anti-angiogenic agents, are considered major risk factors for the development of MRONJ [14]. MRONJ is a condition that can cause pain and poor quality of life, requires a professional approach for the maintenance of the affected area, and is difficult to treat [15]. In our study, most of the analyzed videos referred to drugs associated with the development of MRONJ, and videos focusing on issues related to MRONJ treatments. Among the drugs presented in the videos, bisphosphonates were the most frequently mentioned drug group. In the reviewed videos, the side effects of these drugs, which are usually prescribed for the treatment of osteoporosis or cancer, were explained. The second most frequently mentioned drug was denosumab. Videos recording of scientific meetings examined the effectiveness of denosumab and bisphosphonate drugs in disorders, such as multiple myeloma, and their relationships with MRONJ were investigated by comparing their side effects and benefits. Today, YouTube’s video sharing network provides its’ users with a wide variety of content. The fact that video sharing is free and easy to access, is one of the important advantages provided to its’ users. However, no standardization has been developed for videos uploaded to YouTube by healthcare professionals and individual users [16]. Accessing health information on the Internet is of indispensable importance to patients’ health, and a well-informed patient presents greater compliance with treatment and increases its’ effectiveness [17]. This study is the first to investigate the content and quality of YouTube videos about osteonecrosis developing in the jaw depending on prescribed drug treatment.
The use of social media networks, such as YouTube™, has positive impacts on professional networking and education as well as patient’s care and education, but creates potential hazards, including poor information quality, loss of professional image, or violations of patient’s privacy and licensing [18]. In previous studies, YouTube video content about oral health issues, such as oral lesions, oral cancers, Sjögren’s syndrome, and burning mouth syndrome, has been researched and evaluated [17, 19-21]. In a study by Hassona et al. investigating oral cancers, useful videos related to the subject were listed [20]; however, patients searching for the topic of oral cancers on YouTube were accordingly more likely to encounter videos that were less relevant and/or less useful. The results of the present study showed that the majority of searched videos were low-content videos. YouTube has become one of the most important sources of information for general population, but misleading videos accessed from this source are challenging for healthcare organizations. Therefore, patients should not use these videos as their sole source of information about MRONJ and BRONJ. On the other hand, the superiority of high-content videos over low-content videos was confirmed by applying specified criteria, identifying video length, number of views, and number of likes as higher in high-content videos. This can be interpreted in two ways. The examination of these videos may indicate that students and employees in the field of healthcare or YouTube users (i.e., patients, non-healthcare professionals) who use these videos for educational purposes pay attention not only to the videos but also to the expertise of people who upload the videos. However, previous studies investigated if people accessing health-related information on the Internet examined whether videos were uploaded by subject-matter experts, their last update, and how much of the information was correct [10, 11]. Creating custom segments, such as YouTube Medicine or YouTube Healthcare, could facilitate searches and increase access to high quality medical and dental videos [21].
Different criteria have been used to evaluate YouTube video quality. In various studies, videos were generally analyzed by determining quality of video, focusing on pathophysiology, signs and symptoms, prevention, and treatment options, source of the video (e.g., health personnel, patients, lecture notes), and target audience [11]. As a result of this research on videos about oral medicine published on YouTube, it was revealed that higher quality videos should be produced, and that there was a lack of information in the existing videos [22]. Madathil et al. [23] concluded that YouTube contains misinformation and misleading healthcare data. In the literature, different scoring systems have used to evaluate YouTube videos’ quality. VIQI, involving the contents of global quality scale (GQS) was used to determine overall quality of the videos. Also, VIQI was found as more general index for evaluating video quality [24]. In this study, VIQI was used to determine the video quality, flow, information accuracy, and precision. Potential viewers of videos uploaded to YouTube and related to MRONJ can be divided into two groups. While one group includes medical staff/experts, the other involves patients/individuals seeking medical information. When the audience was evaluated in our study, it was determined that 55% of the videos were prepared for patients and 45% were created for specialists/dentists. In addition, 22.5% of MRONJ-related YouTube videos were found to be high-content videos. When the usefulness of YouTube videos was examined, it was determined that only 20% were considered very useful, and 37.5% were deemed not useful. The VIQI values (i.e., flow, information, quality, and precision) of the most useful videos were observed to be statistically higher than those of the moderately useful and not useful groups. In addition, highly useful videos were found to be uploaded by academic health institutions/professional organizations more often. Samuel et al. [25] support the results of our study, and emphasize that videos produced by professional organizations are of higher quality. Again, when the reliability of information of the videos and the scope of information were evaluated in this study, we found that the most adequate and accurate information was provided by academic health institutions/professional organizations. We observed that videos with high VIQI scores were recorded from seminars of professional institutes and prestigious universities. However, when we examined the video sources in our study, we determined that only 22.5% of them were produced by health institutions/professional organizations, and 38.8% were produced by doctors. Hassona et al. [20] reported that 50% of YouTube videos related to oral cancers were uploaded by professional organizations, and Kovalski et al. [19] found that 75% of YouTube videos on oral leukoplakia were uploaded by independent users and advertising companies. According to the results of our study, healthcare providers should be more engaged in providing accurate and reliable evidence-based information, and present their best practices information about MRONJ on YouTube.
In addition to medical and surgical treatment methods, alternative minimally invasive treatment methods are recommended to support and accelerate healing process of MRONJ treatment [3, 26]. These alternative minimally invasive treatment methods consist of hyperbaric oxygen therapy, ozone therapy, platelet-rich plasma, and low-level laser therapy [27-30]. It is stated that tissue recovery accelerates, inflammation decreases, stem cell migration is induced, and bone cycle suppression is controlled, and it is thought to play an effective role in the treatment of MRONJ [29]. It was found that only ten of all videos examined in our study shared a detailed section on this topic. In addition, only one of these videos showed a successful patient’s experience with hyperbaric oxygen therapy in MRONJ. In the present study, videos with scientific content focusing on radiographic detection of osteonecrosis and methods used were also identified. The most commonly used imaging methods for radiological detection of MRONJ were periapical and panoramic radiographs as well as computed tomography [31]. It was found that only four of the videos examined in our study contained detailed sections on this subject. The main limitation of this study was the difficulty in achieving repeatable results due to changes in YouTube content over time, and the use of different filters [24]. In addition, this study was limited to a direct YouTube search, and was not able to account for YouTube videos viewed on other sites that linked videos. Finally, another limitation of the study was the exclusion of videos that were not in English, and searching by the level of relevance for the evaluated videos. Therefore, the results had to be determined according to the limited selection of video examples.

Conclusions

The results of our study showed that the majority of videos related to MRONJ contain videos with low-content/scores. Therefore, YouTube videos cannot be used to share dependable MRONJ information with individuals. There is a need for good practices and new studies that provide access to accurate information, raise awareness of large audiences, and provide data standardization in videos about MRONJ.

Conflict of interest

The authors declare that they have no conflict of interests.

References