Introduction

For several years now, the concept of competencies has been integrated into the curriculum of higher education institutions (HEI) ⁽¹⁾ in Mexico and in the world; thus, the Tuning project was defined at the European level, which was developed in accordance with the existing educational needs of 1999 and in response to very specific objectives. It became a project that transcended European borders and was adapted to the needs of HEIs in Latin America ^(2,3).

However, there is no standard definition of competencies, but it is a topic that has been addressed in different contexts and sectors of higher education. Specifically, in the nursing area, studies have been conducted that demonstrate the importance of defining the qualities that nursing professionals should acquire enfermería ⁽⁴⁾, and educational institutions are encouraged to incorporate student-centered learning through competency-based curricula enfermería ⁽⁵⁾.

Despite the progress in this area, there are still discrepancies among the disciplines that address it, since there are different approaches, such as the functionalist, constructivist and behaviorist, which do not fully integrate the management of competencies in the educational field and reflect a fragmentation by privileging the cognitive aspects over the affective aspects enfermería ⁽⁶⁾ when it is transferred to work competencies, an essential aspect in the nursing profession.

Nursing is no stranger to competencies, largely due to the requirement of having competent professionals capable of carrying out care in a humanistic and altruistic manner for the benefit of the person, the family and the community. Therefore, for nursing, the need to be competent starts from university education, where HEIs become the main foundation for the achievement of this objective, which is reinforced by the Tuning project, where competencies represent a combination of knowledge and skills, which allows having professionals who can meet the needs of society with generic and specific competencies enfermería ⁽³⁾.

The basic competencies that nurses should have are defined as their ability to perform their practice based on the set of general and specific knowledge associated with the skills and attitudes that result not only from their knowledge, but also from their actions as professionals with a human sense enfermería ⁽⁷⁾.

Nursing competencies, therefore, require that their daily work, care, be based on the cognitive level (knowledge and skills) and on the axiological level, with values that allow efficient and responsible professional performance with the possibility of applying their autonomy and flexibility to individualize care enfermería ⁽⁸⁾.

In this sense, the competency-based approach acquired by HEIs is a model that not only forms part of the curricular structure, but also becomes an essential element that has a direct impact, in terms of nursing, on health services through the assurance of quality of care, patient safety and cost-benefit enfermería ⁽⁹⁾. For this reason, nursing professionals need to demonstrate, through certification processes, the competencies that validate their professional performance and allow them to competently address the health problems they confront.

It should be noted that nurses undergo a process of competence acquisition that begins during their undergraduate training and increases significantly during their social service, a transcendental period of time for social service providers (SSP), who are inserted in first and second level health care units, schools, companies, among others, for a year, allowing them to put into practice what they have learned during their educational training, solve the needs of society and acquire specific competencies of the profession ⁽¹⁰⁾.

To give an idea of the importance of social service providers, in Mexico in 2011, 23 313 SSPs were registered in the National Health System distributed among the Ministry of Health (17 206), Mexican Social Security Institute (3 931), Institute of Security and Social Services of State Workers (578), National System for the Integral Development of the Family (DIF-Desarrollo Integral de la Familia in Spanish) national (132), among others ⁽¹¹⁾.

Although it is true that the SSPs should not cover the labor needs of the health institutions to which they are assigned, in reality they become a fundamental human resource of the workforces ⁽¹²⁾ and perform, with the particularities of each institution, practically all the functions and activities of a nurse, which is why it is essential to evaluate the competencies they acquire during the year in which the students perform their professional internships or social service.

Consequently, the evaluation of learning in the SSPs is considered essential for HEIs that train human resources in health, since it has an impact on both the process and the results of the Social Service (SS). Likewise, doing so allows students to identify the degree of competencies they have acquired, the skills they possess for the resolution of tasks and the identification of weaknesses and strengths in their performance in the care, administrative, teaching and research areas ⁽¹³⁾.

In this regard, it should be noted that for several years in Mexico the National Center for Higher Evaluation (CENEVAL- Centro Nacional para la Evaluación Superior in Spanish) has been conducting a valid and reliable national evaluation of the professional competencies of recent nursing graduates. However, there are not enough published antecedents that offer evidence of the clinical aptitude that students have at the beginning and at the end of the SS ⁽¹²⁾.

Consequently, in nursing it is important to have instruments or scales that allow objective evaluation of the attributes that make up the increasingly complex constructs and dimensions of the competencies of nursing trainees ⁽¹⁴⁾ and that meet the required criteria of validity and reliability, in such a way that the performance and professional action scenarios are precisely defined, allowing comparability in all senses, inter- and intra-institutional ⁽¹⁵⁾.

Currently, there is no reference instrument available for measuring competencies in SSPs in care, administrative, teaching and research practice. One reason is the difficulty in measuring competencies in a comprehensive manner; another is the complexity of measuring care due to the "holistic conception of competencies" and "the complexity of disciplinary action". Therefore, from the educational point of view, there is a need to define "effective evaluation strategies or models" in SSPs that make it possible to demonstrate the acquisition of competencies ⁽⁹⁾. Specifically, it is necessary to identify the best existing evidence on instruments with the theme of competencies in nursing.

Therefore, the objective of this work was to identify empirical evidence on the construction, validation, reliability and/or cross-cultural adaptation of instruments that measure competencies in nursing students and professionals.

Methodology

This study was a systematic review following the PRISMA-P 2015 guidelines, searching Cochrane, PubMed, CONRICyT, BVS, LILACS, Science Direct and Redalyc databases published during the period from 2000 to 2020. The research question was elaborated using the CPC (Concept, Population and Context) format proposed by the Joanna Brigs Institute, where the Concept is "instruments that measure comprehensive professional competencies", the Population is "nursing students and professionals" and the Context is "reliability and validity".

The search strategy was performed with the keywords in Spanish, English and Portuguese with the roots "competencies in nursing" and "measurement instruments"; secondary descriptors "social service" and "nursing students"; and the marginal descriptors "construction and validation", "competency measurement scales" and "validity and reliability" according to the DeCS and MeSH descriptors.

The articles consulted for this study were found in the Science Direct, PubMed and Redalyc databases; in addition to the "snowball" search method. The search strategies used were as follows: in PubMed the algorithm "((competence nurse) AND (validation OR reliability)) AND test nursing AND development" was used (8 studies); in the Science Direct database the algorithm "(Competencies nurses AND students nurses) AND (scale OR validation AND reliability) AND development" was used (2 studies); the Redalyc database used the algorithm "competencies in nursing" AND "validation" AND "instruments" (1 study); the Scielo database was searched with the algorithm "competencies" AND "nursing" [Words] AND instruments [Words] (1 study); and, finally, 3 studies were located using the snowball method.

The inclusion criteria were defined according to Del Pino's proposal Pino ⁽¹⁶⁾. For the participants: nurses and nursing students; for the study variable: instruments or scales to measure nursing competencies, instruments or scales with description of psychometric properties (reliability and validity) and articles with the development of the transcultural adaptation process of the scale; selection period: 2000 to 2020; languages: Spanish, English and Portuguese; articles in open access; and, regarding the type of publications: only journal articles.

Articles that merely described nursing competencies, qualitative studies or essays were excluded; theses, books, book chapters, conferences and government documents were also discarded.

Results

In total, 1088 articles were retrieved with the defined search strategies in addition to four references obtained by the snowball method; 77 articles were chosen for assessment; after the abstracts were read independently by two experts, 18 were eliminated because they did not include the instrument, 20 did not measure nursing competencies and 22 did not correspond to the profession (Figure 1). Finally, the sample consisted of 15 articles that met the established criteria (Table 1).

Source: Adapted from PRISMA-P 2009 model (Moher et al⁽⁴⁰⁾).

Figure 1 Article selection process for this study  

Most of the studies assessed that describe competency instruments or scales were published in English (80%) and the remainder in Spanish (Spain and Mexico) and Portuguese (Brazil).

As time goes by, it is seen that the development of instruments on nursing competencies increases, and in this review 60% of them were published in the last five years.

According to the country of the article, most were produced in Asian countries (Taiwan and Korea [33%]), Nordic countries (Sweden, Norway and Finland [20%]) and Brazil (20%); the rest include countries such as Spain, Mexico, Canada and the United States (6.6%).

Regarding the study population, most of the instruments (46.6%) were directed to general nurses and to a lesser extent to nursing specialties such as pediatrics, oncology and psychiatry; 26.6% focused on students; one of the studies sampled both populations (nurses and students) and another one studied social service providers.

Due to the need for specific measurement scales in this area, 66.6% of the authors of the review developed scales regarding competencies, while the remaining studies presented the cross-cultural adaptation with translation-retrotranslation of existing instruments using the methodology recommended by Guillermin et al ⁽¹⁷⁾ and Beaton et al ⁽¹⁸⁾; most of the adaptations were made from English to other languages (Spanish ⁽¹⁹⁾, Portuguese ⁽²⁰⁾ and Korean ⁽²¹⁾, although there was also an instrument adapted from Swedish to Norwegian ⁽²²⁾ and another in Spanish for physicians that was adapted for nurses ⁽²³⁾.

Although all the instruments or scales measure nursing competencies, differences were found in terms of specialty; the majority focused on clinical competencies at the first and second levels of care (26. 6%) ^{(20,22,24,25)}, while the remaining ones aimed to study competencies in diverse topics, including informatics⁽²¹⁾, emergency ⁽⁷⁾, patient education ⁽²⁶⁾, case management ⁽²⁷⁾, oncology ⁽²⁸⁾, ambulance nursing⁽²⁹⁾, Evidence-Based Nursing (EBN) ⁽¹⁹⁾ and pediatric EBN ⁽²³⁾. On the other hand, studies that measure competencies in a comprehensive manner were also found, one in social service providers ⁽¹⁰⁾, another in registered nurses ⁽³⁰⁾ and, finally, one that evaluates self-efficacy in nursing students' competencies ⁽³¹⁾.

In relation to scale length, a x =41.5 (SD=8) items were reported, with ranges from 11 to 81; in addition, most authors structured the instruments with dimensions or categories that fluctuated from two to eight, and only two scales had no dimensions.

Regarding the measurement properties of the scale, the authors conducted psychometric evaluations focused on validity and reliability (or trustworthiness). Regarding reliability, different methods were used to evaluate it: one study applied inter-rater or inter-observer reliability ⁽²³⁾, two others performed test-retest ^(26,27) and the others used Cronbach's alpha statistical test to determine internal consistency.

Moreover, to determine the validity of the use of the scales, various types were measured. It was reported that 26.6% of the researchers used content, construct and criterion validity ^{(21,24,26,30)}; the same percentage of the studies used content and construct validity ^(23,25,29); 6.6% used construct and criterion validity ⁽³¹⁾; 20% of the authors used only content validity ^(7,10,28); the same percentage used only construct validity ^(19,20,22); and finally, only one study used content, construct and discriminant validity ⁽³²⁾. It should be noted that ten of the studies performed exploratory or confirmatory factor assessment ^{(19,21,22,24-26,29-32)}.

Regarding the use of a conceptual approach to systematically guide the process of measuring competencies, four of the authors considered Patricia Benner ^{(20,24,30,32)} as a theoretical reference, and one more referred in his study to Alberto Bandura's theory ⁽³¹⁾.

Discussion

This systematic review made it possible to identify the instruments that measure nursing competencies; it was found that most of them are aimed at professionals, which can be explained because the labor market demands highly trained personnel and it is important to evaluate them constantly. In addition, in general, recently graduated nurses have adequate competencies to carry out their practice despite being considered new to specialized care, as pointed out by Benner ⁽³³⁾.

One aspect that stands out in this review is the insufficient creation of measurement instruments for the evaluation of nursing competencies, despite the local, national and international demand for the development of skills and attitudes that improve the professional practice of care ⁽²⁰⁾.

However, the immediate incorporation of nursing students into the workplace means that they must be prepared to provide adequate responses to labor demands. Therefore, the training of nursing students is complex, since they must possess and acquire knowledge and skills that guarantee the performance with quality, efficiency and effectiveness in the promotion, prevention, diagnosis, healing and rehabilitation of the person they care for, so it is already a requirement to evaluate the competencies in a comprehensive manner and in all the contexts that integrate it ^(15,34).

In addition, it is to be expected that the evaluation methods in nursing with a focus on competencies that integrate a global vision will increase ⁽²⁰⁾, especially in Spanish and Latin America, because currently this was not observed in the systematic review, where the scales were developed mainly in Anglo-Saxon and Asian countries.

One result to highlight is the cross-cultural adaptation process to which the instruments in this review were subjected, which respected the phases of translation, retrotranslation, expert committee, pilot testing and validation. This process is essential to ensure that the translated measurement instruments can be applied safely and efficiently in another country and culture ⁽³⁵⁾.

It is observed that the instruments that exist to measure integral competencies in nursing students and professionals agglomerate diverse specialties of nursing practice, which strengthens the need to have instruments that identify competencies in at least three moments: one, when students go to clinical and community practices; the second would be during social service; and the third for nurses during their professional practice. In other words, it becomes very important and significant to distinguish the uniqueness of the actions for a safe, humane and professional practice without risks for the patient, the nurse, the institution and the community through instruments that measure specific competencies of nurses in their field of specialization based on the actions that professionals develop in their workplace combined with the verification of the assessment capabilities of the different situations that may be occur to them ⁽³⁶⁾.

With regard to the extent of the scales, there was a great variability in the number of items, even though brevity is recommended, given that short questionnaires are more likely to be returned complete than long ones, so that in the development of the instruments, questions that are interesting for the researcher, but not necessarily relevant or indispensable for the research project, should be avoided ⁽³⁷⁾.

The competency measurement instruments in this systematic review have focused on reliability and validity, two measurement properties most commonly used to evaluate the quality of a measure. In that sense, psychometricians and most nursing researchers accept that test-retest reliability (stability reliability), interobserver reliability (or interobserver reliability), and internal consistency (homogeneity reliability) are measured for reliability ⁽³⁸⁾. For the scales in this review, the authors used Cronbach's alpha (α) coefficient, which is the most widely used measure of internal reliability for scales with multiple items ⁽³⁹⁾.

Regarding validity, the researchers used several categories or types of the most described in nursing to determine the degree that each scale possesses. The literature indicates content validity (face validity, expert validity, factorial validity), construct validity (contrast groups, convergence and divergence) and criterion validity (predictive and concurrent validity) ⁽³⁹⁾.

Finally, it was found that Patricia Benner's theory was used as a research model in most of the articles analyzed, in addition to Bandura's social learning theory, which supports the idea that studies should not be developed in a research vacuum, but that there should be an underlying conceptualization of people's behaviors and characteristics ⁽³⁹⁾.

Conclusions

This systematic review found that a large percentage of the studies evaluated competencies in areas such as clinical competencies and evidence-based nursing; others were even more specific to areas of knowledge such as oncology, pediatrics and informatics, as well as aimed at the population of students or professional and specialist nurses; however, there is little evidence of instruments that evaluate competencies globally and in the essential areas of nursing care, teaching, research and administration.

It is also noteworthy that only one of the instruments is aimed at social service providers. This is noteworthy because in Mexico many HEI curricula still consider them as students, in addition to the fact that SSPs have become a fundamental pillar in the human resources staff of the health sector.

We propose the creation of specific instruments or scales for the PSS due to the specific characteristics of this population and that comply with the methodological and psychometric process of elaboration.

Conflicts of interest

None of the authors presented any conflict of interest.

Funding

There was no funding for this research.