INTRODUCTION

Telemedicine is a set of tools for remote healthcare that has existed as a concept since the 1950s¹. As technology develops, the means that enable care through telemedicine also develop and, consequently, healthcare professionals and institutions, as well as regulatory agencies, governments and legislation itself, have to adapt to keep up with the speed of change. In 2005, the World Health Organization (WHO) created an observatory for “electronic health” (eHealth) and has since advocated for the regulation of such activities^1),(2.

Telemedicine in psychiatry, known as telepsychiatry, has the main advantage of expanding access to the specialty. In places where there is a shortage of professionals, telepsychiatry may be the only possible alternative, improving the quality of care provided to patients in crisis and reducing the length and number of psychiatric hospitalizations^3)-(7.

Telepsychiatry can be used in clinical assessments, for psychotherapy, psychoeducation and medication management, and can also be applied in a variety of contexts, such as outpatient consultations, interconsultations or emergencies^5),(9)-(17. The most widely used and recommended tool today is videoconferencing.³ Video consultations are the closest thing to face-to-face consultations and are accessible to a large part of the population, with the spread of smartphones and the popularization of audio and video platforms⁸.

Despite the reported advantages, the widespread use of telepsychiatry and telemedicine in general still faces technical, legal, ethical, regulatory and cultural challenges worldwide. The problems depicted by the studies tend to be similar, such as situations in which the patient or psychiatrist are not familiar with or knowledgeable about the technology or when internet access is limited, for example. Technical problems can make it difficult to engage during the consultation, compromise the creation of a bond between doctor and patient, and reduce the evaluator’s ability to perceive nonverbal signs and symptoms, such as eye contact, gestures, and posture⁷.

Even in the face of the possible adversities of remote psychiatric care, studies show a high degree of satisfaction among doctors and patients regarding telepsychiatry care⁶. The advantages of the method and the very nature of the psychiatric anamnesis are considered important factors in acceptance, being possible to perform good listening and mental examination through video^18),(19.

In Brazil, the discussion about the regulation of telemedicine has been going on since the 1990s²⁰. In a country with a large territorial extension, which has historically faced problems related to social inequality, the use of telemedicine is seen as something with the potential to change paradigms, increasing the population’s access to health care and living up to the guiding principle of the Brazilian Unified Health System (SUS, Sistema Único de Saúde) of universality and, operationally, decentralization^1),(20.

With the COVID-19 pandemic declared by the WHO on March 11, 2020, laws and ordinances on the subject were updated in Brazil^21)-(27. In April 2022, with the resolution of the Federal Council of Medicine (CFM) Number 2,314/2022, the final text that defines and regulates telemedicine in Brazil was finally attained²⁶. Since then, telepsychiatry has spread and gained prominence, making it important to study how Brazilian psychiatrists have been using such tools^28),(29. The method chosen for this research was through a KAP (knowledge, attitude, and practice) questionnaire.

KAP (Knowledge, Attitude, and Practice) questionnaires are tools used to measure what a certain group of people know, think, and how they act in a given scenario. The logic behind such research is based on the assumption that knowledge acquired in a scientifically appropriate manner tends to create an attitude favorable to correct practice and, ultimately, the adoption of correct practices^30)-(32. Therefore, the KAP questionnaire was the method adopted in this study to evaluate the use of telepsychiatry by psychiatrists in Brazil.

METHODS

The study was carried out in two stages. The first stage was a methodological study for the construction and validation of an instrument (KAP questionnaire) and the second stage was a cross-sectional study of the Knowledge, Attitude and Practice (KAP) questionnaire with an analytical component (application of the KAP questionnaire to psychiatrists and medical residents in psychiatry working in Brazil). This research complied with the terms recommended by the National Health Council (Resolution number 466/2012) for research involving human beings. The project was approved by the IMIP Research Ethics Committee on 11/17/2021 (CAAE 52584521.20000.5201, Opinion number 5,124,348). For the first stage (methodological study), a first questionnaire was prepared in accordance with Ordinance Number 467/2020, Ordinance and Law Number 13,989/2020, the General Law on the Protection of Personal Data (Law Number 13,709/2018), CFM Resolutions Number 2,314/2022, 2,299/2021, 1,821/2007 and 1,643/2002, in addition to the current scientific literature on telepsychiatry. The questionnaire was submitted to appearance and content validation, that is, to verification of the relevance of the presented items and their representativeness in adequately capturing the proposed concepts, through the electronical application of the Delphi method^33),(34.

The Delphi method aims to obtain consensus in the experts’ opinion on a subject through rounds of structured questionnaires. The method precepts were respected, including participant anonymity; feedback on individual contributions; construction and presentation of the group’s response as a whole; and the possibility of reviewing and changing responses^33),(34. The questionnaire was presented to 13 psychiatrists with theoretical and practical expertise in the area of ​​psychiatry and telepsychiatry (judges) through Google Forms. The selection criteria for the composition of the panel of judges were: psychiatrists with at least three years’ experience in direct patient care and/or teaching and research in the area of ​​psychiatry and experience in telepsychiatry, respecting the requirements for defining content judges by Jasper³⁵.

The calculation of the number of experts found in the literature varies, with studies indicating that the ideal number should not be less than ten, which could compromise the results regarding effective consensus and relevance of the obtained information. On the other hand, a larger number of experts would generate too much data and make the administration of the rounds and the data analysis very complex, without necessarily bringing benefits³⁴.

The 13 experts were asked to judge each question (there were 37 questions, initially) using a Likert scale³⁶ (“I fully agree with the inclusion of this question in the final questionnaire”, “I partially agree with the inclusion of this question in the final questionnaire”, “I am indifferent/have no opinion about the inclusion of this question in the final questionnaire”, “I partially disagree with the inclusion of this question in the final questionnaire” and “I fully disagree with the inclusion of this question in the final questionnaire”) and to comment on what they considered pertinent about the questions (e.g.: “I did not understand the question”, “Confusing question”, “I believe that almost no one knows this information, difficult question”) or on the opinions they had about the presented subjects (e.g.: “I did not know this information”, “Electronic prescribing should be expanded to blue (model B) and yellow (A, for psychostimulants) forms” or “I believe that some applications of telemedicine were already allowed before the pandemic. It might be interesting to include in what aspects they were permitted before and under what circumstances they were expanded after COVID-19.”).

The experts had ten days to answer each round of questionnaires, and three rounds were necessary until the maximum level of agreement was reached, with the questionnaire ending up with 33 questions, validated by ten experts (three experts were lost to follow-up because they did not answer the second round). The excluded questions were those considered redundant, confusing, unnecessary or too difficult/specific. The final comments on the set of questions were that the final questionnaire was more fluid to read, easier to answer, more intelligible and covered the main topics on telepsychiatry.

The second stage (cross-sectional study) was carried out by sending the KAP questionnaire in Google Forms to psychiatrists and medical residents in psychiatry throughout Brazil, using social media and by posting it on the website of the Brazilian Psychiatry Association (ABP, Associação Brasileira de Psiquiatria). Data collection took place between June 2022 and March 2023.

The questions about knowledge had their answers coded as: “true”, “false” and “I don’t know”. Regarding attitude, the answers could be “I fully disagree (1)”, “I partially disagree (2)”, “I am indifferent/I have no opinion (3), “I partially agree (4)”, “I fully agree (5)”. The questions related the psychiatrists’ opinions about the practice of telepsychiatry, its advantages and disadvantages. Regarding the practice, the answers could be “never (1)”, “rarely (2)”, “sometimes (3)”, “frequently (4)” and “always (5)”, and the proportion of telepsychiatry consultations by each respondent in the last year was also recorded.

After the collection of responses, the data were exported from Google Forms to an Excel database and then analyzed using the Stata 12.1 software. For data analysis, knowledge was considered adequate when the answer was “true” for true statements or “false” for false statements; inadequate when the answer was “false” or “I don’t know” for true statements or “true” or “I don’t know” for false statements.

Regarding attitude: it was considered adequate when the answer corresponded to “I fully agree” (5) or “I partially agree” (4) for the statements. The respondents’ attitude was questioned regarding Brazilian laws on telemedicine and on the use of telemedicine for care in psychiatric emergencies, interconsultations, outpatient consultations, care for children, adolescents, adults and the elderly. As there is evidence supporting the use of telemedicine in care for all asked forms, the attitude was considered inadequate when the answer corresponded to “I fully disagree” (1), “I partially disagree” (2) or “I am indifferent/I have no opinion” (3).

The practice was considered inadequate when the answer was “never” (1) or “rarely” (2) and adequate when the answer was “always” (5), “frequently” (4) or “sometimes” (3). The statements asked about the use of telemedicine for psychiatric emergency care, interconsultations, outpatient consultations, and care for children, adolescents, adults and the elderly. Regarding the proportion of telepsychiatry care in the last year, the possible answers were “more telepsychiatry”, “50% of each modality” or “more in-person care”.

For each domain of the questionnaire, a minimum of 70% of correct answers/expected answers was defined as the cut-off point (mean 7.0, also used in other studies with similar methodology)^31,32. The frequency of adequate answers was calculated for each of the domains.

Continuous data were presented using mean and standard deviation. For the multivariate Poisson analyses Wald’s test was performed to search for factors associated with the outcomes of adequate knowledge, attitude and practice, estimating the crude and adjusted prevalence ratios (PR), with 95% confidence intervals and significance levels. The variables that reached a p-value <0.20 in the univariate analysis were selected for the final stage of construction of the multivariate models, considering them as statistically significant when the p-value was <0.05.

RESULTS

The validation of the KAP questionnaire appearance and content was performed by ten psychiatrists with theoretical and practical expertise in the area of ​​psychiatry and telepsychiatry and occurred in three phases. In the first phase, with 37 questions, the convergence was 97.7%, with 25 notes on the appearance and content of the questions. Four questions were eliminated in the second phase, totaling 33 questions, of which 15 had their text changed, and the convergence was 96.3%. A third, final phase, included the same 33 questions from the previous phase and changes in the text of ten of them, resulting in a 100% consensus (score 5 = maximum agreement) of all judges in the totalization of the questions on knowledge, attitude and practice).

The judges’ mean age ranged from a minimum of 32 to a maximum of 64 years, with a mean of 38 years (SD ± 10). The female gender predominated (60%), with professional experience ranging from a minimum of three to a maximum of 39 years, with an average of ten years (SD ± 10.8). All judges reported working in the capital and metropolitan region, 80% work with students (undergraduate, postgraduate or residents) and 100% use telemedicine at work.

Of approximately 15,000 psychiatrists and medical residents in psychiatry in Brazil, data corresponding to the KAP questionnaire were collected from 101 professionals (0.7% of the population)³⁸. A total of 53.5% were under 40 years of age, with a mean age of 40.5 years (SD ± 10). More than half, 60.4% of the respondents, reported being female. The majority (83.2%) lived in the capital or metropolitan region, with 27.7% working in the interior. Most respondents (60.4%) lived and worked in the Northeast of Brazil, with 14.9% from the South, 18.8% from the Southeast, 5% from the Midwest and 1% from the North region of Brazil. The flowchart for recruiting and monitoring the study participants is shown in Figure 1.

Figure 1 Flowchart for recruiting participating psychiatrists and psychiatry residents. 

Regarding training, 63.4% of the respondents completed the medical course more than ten years ago, with the average among respondents being 15 years since graduation (SD ± 10). Regarding the highest qualification, 56.4% declared having finished medical residency, 21.8% declared having a Master’s Degree, 4% a Ph.D. and 1% a Post-doctorate degree. Just over 16% declared having a specialization or Board Certification title.

Regarding access to information and updates, 66.3% declared having access to research databases, 64.4% declared being medical residency preceptors, 57.4% declared having access to medical journal subscriptions and, the vast majority, 90.1% declared participating in conferences in the area. Almost 30% declared being teachers. Only 12.9% declared having had formal training in telemedicine.

Among the respondents, 65.3% reported working for the SUS, 95% provide care in the private sector, 24.8% provide care through health insurance providers, and 6.9% provide care voluntarily. A total of 98% of the respondents reported providing care in a private or public clinic/outpatient clinic, 34.7% work in Psychosocial Care Centers (CAPS, Centros de Atenção Psicossocial), 28.7% provide psychiatric interconsultations (psychiatric Opinions), 21.8% provide care in general hospitals, 14.9% provide care in psychiatric emergencies, and 13.9% work in psychiatric hospitals.

All respondents provide care to adults, 73.3% provide care to the elderly, 64.4% provide care to adolescents, and only 24.8% provide care to children. Most respondents reported providing care to people who were familiar with telemedicine tools (97%), while 3% reported providing care to people with complete lack of this knowledge. Most psychiatrists and residents reported working in places with infrastructure for telemedicine care (81.1%), while 14.9% of the respondents reported working in places without any infrastructure.

Knowledge was adequate (more than 70% of responses with correct answers) for 68.3% of the survey participants. The study variables that participated in the univariate analysis for adequate knowledge of Brazilian psychiatrists and psychiatry residents regarding telemedicine are shown in Table 1. In the multivariate analysis for adequate knowledge of Brazilian psychiatrists and psychiatry residents, none of the studied variables were statistically significant (Table 2).

Regarding the attitude, considering that for all enquired categories (emergency care, medical office/outpatient clinic, interconsultation, care for children, adolescents, adults and the elderly) there is scientific evidence that supports the use of telemedicine, the attitude was considered adequate when the participant fully agreed (5) or partially agreed (4) with 70% or more of what was asked. For 80.1% of the people, the attitude was considered adequate.

The study variables that participated in the univariate analysis regarding the adequate attitude of Brazilian psychiatrists and psychiatry residents in relation to telemedicine are shown in Table 3. The final model of the multivariate analysis regarding the attitude of Brazilian psychiatrists and psychiatry residents is shown in Table 4. Doctors who provide office care were 74% less likely to demonstrate an adequate attitude when compared to those who do not provide office/outpatient clinic care. Those who do not provide care for children were 77% less likely to demonstrate an adequate attitude when compared to those who do.

The practice was considered adequate when at least 70% of the responses to the questions about practice corresponded to always (5), frequently (4) or sometimes (3). The questions addressed the frequency of telemedicine use in psychiatric emergency care, office/outpatient clinic, interconsultation, care for children, adolescents, adults and the elderly. The practice was considered adequate for 81.2% of psychiatrists and residents, with the modalities with the lowest adherence to telemedicine care being psychiatric emergencies (17.8% of adequate responses), interconsultation (22.8% of adequate responses) and care for children and adolescents (34.7% of adequate responses).

The study variables that participated in the univariate analysis for the adequate practice of Brazilian psychiatrists and psychiatric residents in relation to telemedicine are shown in Table 5. The final model of the multivariate analysis for the adequate practice of Brazilian psychiatrists and psychiatric residents is shown in Table 6. Physicians who did not care for adolescents were 65% less likely to demonstrate adequate practice when compared to those who did. Those who did not care for elderly patients were 78% less likely to demonstrate adequate practice when compared to those who did.

In total, ten psychiatrists answered that they had not provided telemedicine services in the last year. Of these, eight had adequate knowledge and five had an appropriate attitude. The ten answered that they did not have tools for providing telemedicine services in their workplaces.

DISCUSSION

The KAP questionnaire was validated by ten experts in a total of three rounds, reaching maximum convergence (all fully agreeing with the inclusion of the 33 questions in the final questionnaire). The questionnaire was answered by 101 Brazilian psychiatrists and medical residents in psychiatry, with 68.3% of them achieving adequate knowledge, 80.1% of them achieving adequate attitude, and 81.2% of them achieving adequate practice.

The research indicates knowledge close to what was considered adequate (70%). Although the sample is satisfactorily heterogeneous in many points, no sociodemographic factor, related to access to information and updates in the area, time since graduation, highest degree, place of work, place of residence, target audience, and infrastructure for telemedicine care, was shown to be statistically significant in achieving adequate knowledge.

One piece of data that may partially explain this issue is the fact that only 12.8% of the professionals answered that they had formal training in telemedicine. A 2022 French study of medical students and residents found that only 14% of the students reported having had practical training in telemedicine during their training³⁷. Telemedicine has been widely disseminated in Brazil since the COVID-19 pandemic and was often the only possible way to provide care^2),(20. Without formal training in telemedicine, whether in undergraduate, postgraduate or other courses, it can be inferred that professionals in active practice had to seek updates in the area on their own to be able to work during the pandemic. The lack of training has a significant impact on the professional’s perception of their preparedness to actually provide care via telemedicine. The same French study found that 97.9% of the students did not feel prepared to apply telemedicine to practice³⁷.

Although some specific modalities, such as psychiatric emergency care and child care, were seen as more unwilling in terms of attitude (56.4% of the respondents had an appropriate attitude towards psychiatric emergency care via telemedicine and 60.3% for care of children and adolescents), the overall attitude of 80.1% of psychiatrists and psychiatry residents was in agreement that there is scientific evidence for telepsychiatry care.

Some subspecialties of psychiatry may be seen as more difficult to apply to telemedicine^9),(10. In the case of psychiatric emergencies, the presence of multiple companions and other professionals, such as first-aid workers, in addition to the psychomotor agitation of the patient in crisis, may be seen as complicating factors for remote care^6),(7. In the case of care for children and adolescents, the dynamics of consultations themselves, which also involve the presence of companions and dialogue with multiple people, may be seen as hindering remote care¹⁵.

Despite the apparent distrust regarding telemedicine care for children and adolescents, providing care to these age groups showed to be a statistically significant factor for appropriate attitude and practice. Probably, precisely because this is a specific public and a subspecialty of psychiatry (child and adolescent psychiatry), the professional who provides care to them must be aligned with the evidence in the area, bringing telepsychiatry care into practice. Something similar can be inferred about appropriate practice among psychiatrists who provide care to the elderly, with psychogeriatrics also being a subspecialty. The participation of companions in consultations with children, adolescents and the elderly can also increase the chances that someone is familiar with telemedicine and can help with the fluidity of care⁹.

Not providing care in an office/outpatient clinic showed to be a factor associated with appropriate practice; however, only two of the 101 participants stated that they did not provide care in an office/outpatient clinic. The difference in the number between those who provide care (99 people, 68 of which have adequate knowledge) versus those who do not provide care (two people, with these people having the level of knowledge considered to be adequate) must have been the factor responsible for this variable appearing as statistically significant - which would probably not be extrapolated if the sample of professionals who do not provide care in offices/outpatient clinics had been larger.

To achieve quality practice, the gap between knowledge and appropriate attitude and practice should be smaller^30)-(32. Formal training in telemedicine can benefit professionals who wish to provide care in this modality. It is possible that new professionals in the field, still in training, receive training in telemedicine during their undergraduate studies, completely changing the results of a similar study carried out in the near future.

The present study has limitations. Only 0.7% of Brazilian psychiatrists and psychiatry residents answered the questionnaire. The difference in the number of responses between the regions of the country was large, not corresponding to the proportion of professionals who actually work in each region of the country³⁸. It is not known how widely the research was disseminated among the expected population, but it is possible that more people interested in the topic answered the questionnaire, which would overestimate the knowledge, attitude and adequate practice on the subject. It is also estimated that, over time, the percentage of knowledge, attitude and adequate practice in telepsychiatry will increase³⁹.

The search for other studies using the KAP questionnaire in telepsychiatry up to June 2023 in the PubMed, Bireme and Scielo databases shows that this study is a pioneering one, bringing a series of new data to be explored. The pioneering nature of this study makes it impossible to directly compare it with other studies; however, there are studies with KAP questionnaires aimed at physicians in the evaluation of telemedicine and tools, as well as of the population in relation to telemedicine, also showing interesting and similar results, pointing to similar needs, both for formal training in telemedicine for physicians and for good acceptance by the general population of the method^37),(40.

FINAL CONSIDERATIONS

Qualitative studies can help both in identifying difficulties in the application of telemedicine in practice and in creating strategies to disseminate knowledge in the area among active professional More studies that show evidence for telemedicine care in the various modalities of psychiatry can also help to improve knowledge, attitude and practice. Continuing education programs promoted by the Federal Council of Medicine, professional associations, medical residencies and training courses should also improve the rates of knowledge, attitude and adequate practice in telepsychiatry.