Servicios Personalizados
Revista
Articulo
Indicadores
-
Citado por SciELO
-
Accesos
Links relacionados
-
Similares en SciELO
Compartir
Problemas del desarrollo
versión impresa ISSN 0301-7036
Prob. Des vol.53 no.210 Ciudad de México jul./sep. 2022 Epub 05-Dic-2022
https://doi.org/10.22201/iiec.20078951e.2022.210.69807
Articles
Universities and socioeconomic development: a proposal for evaluating their social linkage projects
aTechnical University of Ambato (Universidad Técnica de Ambato), Ecuador and DeTEI Research Group and University of the Basque Country (UPV/EHU), Spain. Email address: ra.zamora@uta.edu.ec.
bUniversity of the Basque Country, Spain. Email addresses: arturo.rodriguez@ehu.eus and jon.barrutia@ehu.eus, respectively.
In compliance with the "third mission of universities," the "University Society Linkage Projects" (USLPS) are currently undergoing an important development in many countries. However, there is no comprehensive model for evaluating these projects' development and impact. This article proposes a model to remedy this deficiency, one which factors in the agents involved, the dimensions and factors relevant to their development, and the variables and indicators to be considered. This model can therefore be useful for universities, public supervisory bodies, and the beneficiary entities of the projects themselves.
Key Words: University Society Linkage Projects (USLPS); social impact; impact evaluation; indicators; project management
En cumplimiento de la "tercera misión de las universidades", los Proyectos Universitarios de Vinculación con la Sociedad (PUVS) experimentan, en la actualidad, un importante desarrollo en muchos países. Sin embargo, no existe un modelo completo para la evaluación del desarrollo e impacto de este tipo de proyectos. Para remediar esta carencia, el presente trabajo propone un modelo al respecto, considerando tanto a los agentes que intervienen en ellos, como a las dimensiones y a los factores relevantes en su desarrollo, y las variables e indicadores a considerar. Así pues, este modelo puede resultar útil tanto a las universidades como a los organismos públicos de supervisión y a las propias entidades beneficiarias de los proyectos.
Palabras clave: Proyectos Universitarios de Vinculación con la Sociedad (PUVS); impacto social; evaluación de impacto; indicadores; gestión de proyectos
Clasificación JEL: A22; I25; I31
INTRODUCTION
Universities, just like other entities, have to adapt to the demands of their surroundings in order to serve the needs of the community. This leads to expanding beyond its two traditional missions of education and research by adding a “third mission.” This new mission consists of contributing to the scientific, technological, economic and social development of the territories where they are located (Laredo, 2007).
However, universities not only play an important role in educating people, but also in preparing for their integration into social environments where they can put into practice the knowledge they have acquired, thus becoming agents of social change.
Nevertheless, this “third mission” has recently been consciously and purposefully pursued. In order to carry out the mission effectively, the cooperation of other agents, both private and public, is required. This is the basis for approaches like the “Triple Helix” approach (Etzkowitz and Leydesdorff, 2000): collaboration between the university, companies and public administrations. Going further, if the objective is developing the immediate surroundings, one should opt for a “Quadruple Helix” (Carayannis and Campbell, 2009), by integrating a new actor: “civil society.”
When proposing the object of this work, we use with this last approach University Society Linkage Projects (USLPs) as they are currently undergoing an important development in Latin America. Their goal is precisely to solve problems of a social, cultural and business nature that arise in the university’s surroundings. One must keep in mind that, in Latin America, the majority of universities’ collaborative experiences with social agents and public administration are still framed within the context of the “Triple Helix,” while in USLPs civil society plays a more important role.
On the other hand, the evaluation of USLPs, when it exists, usually focuses exclusively on general aspects of their implementation and not on a detailed analysis of their development, which would allow improvements to be made. It measures neither its effective impact on improving people’s quality of life nor the efficiency of the organizations supported. In summary, there is no comprehensive model for evaluating USLPs and their actual impact.
Therefore, the objective of this analysis is to present a proposal for an evaluation model which takes on USLPs’ development and measures their impact. We intend for the model to be rigorous, albeit easy to understand, and flexible so that it may adapt to the various circumstances in which this type of project can be developed, when put into practice.
As such, this proposal fills a gap both theoretical and practical in the literature on the subject. At the theoretical level, the interest that the “Quadruple Helix” has in socioeconomic development in specific territorial environments becomes evident, given the central role played by civil society, as is the case with USLPs. Likewise at the practical level, as there has been heretofore no comprehensive model for evaluating the development of these type of projects, nor for measuring their impact.
We expect our proposed model to be of use for universities given that a comprehensive evaluation of USLPs’ development and an adequate measurement of their impact will allow them to perfect USLPs’ design and execution. Furthermore, it will allow for public bodies to carry out their supervisory work more effectively.
To facilitate an understanding of our proposal, this article is divided into five sections, including the introduction. The second section analyzes the concepts of “social linkage” and the USLPs which it engenders, as well as their presence in Latin American countries. Then we analyze the concepts and references in the literature on impact measurement, as they may be of use for the model to be built. The process employed in elaborating the model is presented in the fourth section along with the model itself and its components: agents, dimensions of its development, variables and indicators. Finally, the conclusions are presented in the fifth section.
2. UNIVERSITIES’ SOCIAL LINKAGE PROJECTS (USLPs)
As dynamic entities, universities have undergone various transformations throughout history, brought about by the need to adapt to the demands of their surroundings, often manifested as new social needs (Altbach, 2008). Thus, was a “third mission” added to those already traditionally accepted for over a century, education and research, consisting of a contribution to the economic and social development of their surroundings (Laredo, 2007; Rodríguez-Castellanos and Zamora-Sánchez, 2020). Therefore, they are no longer only creators and transmitters of scientific and technological knowledge, but also generators of innovation who contribute to the development of regions and countries (Altbach, 2008; Valero and Van Reenen, 2019).
However, this “third mission” includes not only support for companies and economic organizations, but also a commitment to improve the social conditions in their surroundings. While activities aimed at solving problems in companies both transmit and apply generated knowledge (Bueno and Casini, 2007), actions aimed at the community in its immediate surroundings emphasize and reaffirm the university’s stance of “civic commitment,” (Goddard, 2009; Sánchez Ambríz and Pérez Balbuena, 2018).
In a complex and changing world like the one we live in now, facing the problems derived from relationships with one’s surroundings requires collaborating with different agents and stakeholders (Gray and Purdy, 2018). Therefore, in order to carry out the “third mission,” universities have created methods of linking themselves more closely with public or private organizations (Pugh et al., 2016; Manrique, 2019). This collaboration helps universities to be seen as “catalysts of change,” with an active role in contributing to the regions’ socio-economic development (Aranguren et al., 2016; Fonseca and Nieth, 2021).
The connection between universities and other agents for the fulfillment of the “third mission” gave rise to various approaches, the best known being the “Triple Helix” model (Etzkowitz and Leydesdorff, 2000) made up of universities, companies and public administrations. However, a stricter implementation of that mission requires enhancing the economic and social development of the communities in the immediate vicinity, especially those most vulnerable. This approach opens up the possibility for a “Quadruple Helix,” where a new agent, civil society, is added to the agents that already make up the “Triple Helix” (Carayannis and Campbell, 2009; Miller et al., 2018; Urra, 2018). It is understood that the university’s social linkage actions fit best within this model, the object of this work.
Although there is no single concept of social linkage, it can be understood as “the set of activities which creates a relationship between one or more entities with social organizations, so they may cooperate to achieve certain ends” (Zamora-Sánchez et al., 2017, p. 973). In other words, it is a matter of putting into action the activities developed by the university in conjunction with companies or other social entities and the State. This linkage manifests itself in the USLPs.
In Latin America we find, albeit in a limited fashion, various manifestations of this social linkage, mainly in productive approaches, though social approaches are also actively being developed, overlapping with the “Quadruple Helix” approach. There are studies for both specific countries and groups of countries (Ramírez and García, 2010; Morales et al., 2012; Urra, 2018).
Brazil was one of the first Latin American countries to introduce “business incubators” as a means of generating greater socioeconomic development. These are aimed at promoting not only technology and innovation, but also cultural entrepreneurship and satisfying the needs of society and indigenous peoples (Chandra, 2007). Another standout feature is socially focused pharmaceutical research (Morales et al., 2012; Pereira da Veiga et al., 2016).
Next is Chile, a country with great wealth in natural products. Innovation systems were created for this wealth as a result of the collaboration between universities, companies and public administrations (Giuliani and Bell, 2005; Bas et al., 2008; Vera, 2009; Gómez-Gajardo, 2017), with high-impact “enabling innovations” already extant in other countries (Ramírez and García, 2010).
When it comes to Colombia, the University Research Results Transfer Offices (OTRIs)2 has found success in solving social and business problems (Pineda et al., 2011) as has the State Enterprise University Committees (CUEE)3 (Morales-Gualdrón and Giraldo Gómez, 2015).
In Ecuador they prefer to channel university-business-administration-society collaborations through USLPs (Zamora-Sánchez et al., 2017; Brito-Gaona et al., 2018). Studies such as that of Macías et al. (2017) or Rueda et al. (2020) analyze these projects and show the commitment of Ecuadorian universities in this regard. They also highlight the need to establish a clear diagnosis of the needs they wish to meet and the process to be followed in bringing the projects to fruition, as well as the lack of a model for a comprehensive evaluation.
Ending with Mexico, there is limited linkage between companies and the academic sector (Saavedra, 2009), though this has improved recently (Pavón-Silva et al., 2007; Morales et al., 2012). However, there are several works which evaluate regional innovation systems (Moctezuma et al., 2017; Ordóñez, 2017). Lastly, González et al. (2020) investigate the links between the agents of the “Quadruple Helix” in the state of Tamaulipas and find that there is a network of regular relations between universities, administrations and businesses, but civil society has yet to be fully integrated.
As one can see, the university-society link in Latin American countries, though growing, is limited. This is especially true in projects of a social or community nature and, in the case of the latter, we must add the lack of a model for its comprehensive evaluation.
Therefore, one of the biggest problems universities face with this type of activity is the need to accurately evaluate their development and the positive impact had on the community. This evaluation is of interest as it allows one to identify the effectiveness of the project’s implementation in productive development, its contribution to social welfare, the relationship between transmitted knowledge and new knowledge generated, or the impact on the environment.
From the extant literature we learn that evaluations of the link with society - when it exists - usually focus on some useful yet general aspects with little specificity (Drucker and Goldstein, 2007), or on elements related to its execution, and lack the detailed analysis of the projects’ development needed to propose any changes. We likewise note the absence of any measure of the projects’ actual impact on the affected people’s quality of life or, where appropriate, on the efficiency of organizational management. This means that there is as yet no comprehensive and concrete model for measuring USLPs’ development and their effective impact.4
It was precisely these shortcomings that spurred us to create the proposal presented in this article.
3. IMPACT MEASUREMENT
Although there is no literature on models for evaluating development or measuring impact when it comes to USLPs specifically, there is abundant literature on impact measurement, both in general and for various types of projects. This section will analyze those contributions considered to be especially useful in elaborating the proposed model.
The concept of “impact” is widely used in the study of environmental sciences. “Impact” is generally considered to be the effect produced by a given event within a specific context. From a social perspective, the term “impact” can be employed to refer to the consequences that a proposed intervention has on the community in general when it seeks not only to solve an identified problem, but also to improve the well-being of people in general through a beneficial result while avoiding any possible negative effects (Esteves et al., 2012).
For Cohen and Martínez (2002), the impact of a social project or program is the magnitude of the change in the situation of a target population as a result of delivering goods or services to it after accounting for external influences. In the case of USLPs, people can be beneficiaries at the individual level, although these are often organized groups, such as artisan guilds and companies. Therefore, depending on the scope in which the people or organizations benefiting from the projects operate, there may be a variety of economic, environmental and technical impacts, among others (Kroeger and Weber, 2014).
Stufflebeam and Shinkfied (1993) point out that measuring impact should provide information on the effectiveness of the methods used for executing actions, serve to make better decisions in the future and be useful for the people and/or entities which benefit. This means establishing the general and specific objectives of the evaluation, as well as the sources of information to be used; the quality of the available data influences the effectiveness of the project evaluation and the reliability of the results obtained. Likewise, the evaluation must be differentiated according to whether it is for people, companies, other organizations or society as a whole, as each group has their own unique characteristics and needs so the questions to be asked need to reflect this fact.
Measuring the impact of a project is vital for evidence-based policymaking, which is useful for identifying groups of interest, those responsible, verifying project quality and effectiveness and making decisions for future interventions, i.e., guiding the design of future projects and allocating budgets (Castro-Martínez et al., 2016). Evidence is a key factor in making higher education institutions transparent and accountable to interested parties, especially the public sector.
The first consideration taken into account is the cause-effect relationships between the program or project implementation and changes in the target population. In this case, despite the fact that the USLPs are different from each other, the establishment of common variables and indicators was sought so that the results of the projects could be analyzed within a global context (Gertler et al., 2017). Thus, changes in individuals, companies or civil society that can be attributed to a particular program, project or policy must be adequately recorded; clear evidence of the management and development of the project, program or measures applied should also be provided (Kroeger and Weber, 2014). It is advisable to carry out periodic evaluations of the projects, so that it can be ascertained if the design and execution have shortcomings that affect the results, thereby reducing uncertainty regarding the projects’ efficiency and effectiveness.
As can be deduced from the above, a comprehensive impact measurement should also include an evaluation of the project’s development. This is what was elaborated for USLPs in the proposal presented below.
4. MODEL FOR EVALUATING THE DEVELOPMENT OF USLPS AND MEASURING THEIR IMPACT
Approach and its elaboration
An evaluation of USLPs’ development and measurement of their impact must take into account that their purpose is to solve social, economic or business problems in the universities’ geographical surroundings. On the other hand, agents of different types can participate in this type of projects: people who direct and execute the projects (professors and students), beneficiary entities (people, communities, organizations or companies which are the target of the projects’ actions), and government agencies, at different levels, which can facilitate the linkage and also supervise the process. Obviously, the fundamental agents are the entities benefitted as the ultimate goal of the USLPs to achieve in them an immediate positive change as a direct consequence of the executed actions. Yet there is also a long-term effect, both in the beneficiary entities and the universities’ surroundings and conditions (Rodríguez-Castellanos and Zamora-Sánchez, 2020).
Developing a model for monitoring and measuring the impact of USLPs poses several difficulties. The most important of these is that it must be able to give a clear answer to the question: “Have the objectives of the project been achieved?” As such, those responsible for the projects’ planning and execution must always be aware of said objectives and the activities required to achieve them. It is important to select variables and indicators that allow the project and its effective impact to be properly evaluated, as well as to identify the beneficiary entities which, obviously, expect to be “affected” positively. Thus another question may be answered: “what positive change has taken place in the beneficiary entities and their surroundings as a result of participating in a USLP?”
On the other hand, as we are dealing with projects, their evaluation must decidedly be based not only on the literature on impact measurement, but also on Project Management methods and techniques (Kerzner and Kerzner, 2017). Furthermore, a proper evaluation must involve a complete and comprehensive study which contemplates both internal processes (budget, efficiency, compliance with standards, meeting objectives, etc.), and external ones (participants’ satisfaction, project’s usefulness, etc.) This means data needs to be compiled, not only from files and records, but also by means of surveys, interviews or other methods.
Based on these criteria, the USLPs carried out at the Technical University of Ambato, in Ecuador, were taken as a reference. The large number and diverse nature of projects executed there allow us to propose a sufficiently general model (Zamora-Sánchez et al., 2017). The methodology for its elaboration is detailed in Figure 1.
![](/img/revistas/prode/v53n210//0301-7036-prode-53-210-181-gf3.gif)
Source: created by the authors.
Figure 1 Process for elaborating a model for the evaluation of USLPs and measuring their impact.
As can be observed, the first step was to identify the elements, of which there are of four types: first are the agents participating in the USLPs, their characteristics and the actions to be carried out during their execution; second is the dimensions of the projects’ development, along with the agents and factors associated with them; third is the intervening variables, whose identification is made possible by identifying the first two elements; finally are the indicators which will be developed to help establish the variability in the value of each variable. The second step was an initial proposal of an evaluation model where we identify variables and indicators, especially those of impact, which, as will be demonstrated later, will differ according to the area of intervention where each project takes place. That initial proposal was then validated by a group of experts, after which the final proposal was developed.
Taking into account the characteristics of these projects, we believe it appropriate that the evaluation be aimed at the groups of agents indicated above. The first groups - faculty, students and heads of the Linkage with Society Unit5 - in charge of the direction and execution, will provide information on the guidelines used for identifying the problem to be solved, the planning, elaboration and execution of the projects, and identifying any shortcomings, as well as the agents’ view regarding satisfaction levels and the positive effects generated by participating in the projects’ execution; the group benefitted will provide information on its general opinion of how its situation improved, or not, after taking part; finally, the supervisory group - government representatives - will verify that the project is executed according to plan and that there are no conflicts in its implementation.
An initial proposal was prepared and submitted for validation by a group of experts. This consisted of 26 people: three heads of the Linkage with Society Unit, two USLP coordinators, two teachers responsible for USLPs, two students participating in USLPs, twelve beneficiaries, two professors of the University’s Chair of Entrepreneurship and Innovation and two analysts as government representatives and a consultant.
The validation process consisted of two stages: in the first, we gathered the experts and gave them material consisting of the model’s structure, together with the questionnaires prepared for carrying out interviews and surveys with the various types of agents. They were then asked to evaluate both the model and the information-gathering tools based on their experience, responding to questionnaires and identifying errors. In the second phase, a report was prepared with the observations and suggestions gathered; based on this, the documents were modified and emailed to each expert for a final review. Using all of this, the final proposal was created.
Next, we will present the final proposal’s deployment according to the proposed scheme.
Agents, characteristics and actions
The first step in building the model consisted of identifying the agents involved in the USLPs. These must meet a series of qualities or characteristics, and carry out a series of actions correctly.
Identifying these elements in each group of agents allows for the systematic collection of information on the activities, characteristics and responsibilities they exercise in the project’s execution. This makes it possible to accurately judge the project’s planning and execution, the fulfillment of objectives and the impact achieved, as well as to evaluate its efficiency in terms of development, effectiveness, viability and sustainability.
All these aspects are summarized in Table 1. Thus, in accordance with that set forth in the previous subsection, the group of agents participating in the USLPs are in the first column, along with their qualification or main function in parentheses; in the second column, the characteristics, qualities or capabilities that they must meet for the process’s optimal execution, as shown below; and, finally, the actions to be carried out in the process by each group of agents.
Table 1 Agents, characteristics and actions in USLPs
Agents | Characteristics | Actions |
Linkage with Society Department´s directors (those in charge) | - Experience | - First contact with the beneficiary entities or government agencies |
-Abilities | - Acceptance, where appropriate, of the request for assistance | |
- Decision on the project´s development | ||
- Assignation of work teams | ||
- Signing of the commitment agreement | ||
- Auditing the USLPs | ||
Professors (executors) | - Education | - Elaborating the Project |
- Experience | - Elaborating the material | |
- Motivation | - Project development | |
- Consequence | - Evaluating the Project | |
Students (executors) | - Education | - Implementing their education and training |
- Motivation | - Elaborating the Project | |
- Consequence | - Elaborating the material | |
- Project development | ||
Beneficiary entities | - Awareness of the problema | - Contact with the university or goverment agencies |
- Need for a solution | - Signing of agreement | |
- Motivation | - Providing physical infrastructure for the USLP´s execution | |
- Consequence | - Participation in the USLP | |
- Communication | ||
- Evaluation of USL´s implementation | ||
Government agencies (supervisors) | - Social recognition | - First contact whith beneficiaries |
- Putting beneficiaries in contact with the university | ||
- Provision of physical infrastructure for the signing of the agreement | ||
USLP follow-up | ||
- Socialization of USLP |
Source: created by the authors.
Development: Dimensions, agents and factors
Once the basic characteristics and the main actions of the USLP’s agents have been established, we then proceed to identify the dimensions that make up its development. Following Project Management methodology (Kerzner and Kerzner, 2017) (see Figure 2), three dimensions have been identified: input, process and outcome. Likewise, the link with the aforementioned agents and with other factors relevant to the success of the projects is taken into account for each dimension. This link is conditioned by the way in which the university carries out the activities that influence the aid provided to the beneficiary entities.
We believe it necessary to emphasize two factors relevant in USLP development: the area of intervention and the surrounding conditions (see Figure 2).
The first, included in the input dimension, corresponds to the types of problems to be solved by the USLPs. Clearly, the projects executed in each field must be aligned both with the actual needs of the social environment and with the kind of knowledge the executing personnel have, i.e. in accordance to their field of study. Using the university analyzed as a reference, six areas were identified and are developed in Table 2.
Table 2 USLPs' área of intervention
Area of intervention | Goal |
Community Training | Provide knowledge to communities on citizens' rights, duties and regulations ז as well as on issues of common interest. |
Other community services | Positively modify people's living conditions, especially the most disadvantaged social groups. |
Technical training | Provide training in specific areas/topics to the personnel of specific organizations. |
Consulting/Advisory | Help organizations identify and solve specific problems, main-taining long-term relationships. |
Other technical support services | Solve specific problems that hinder the normal development of an organization's activities. |
Entrepreneur Program | Evaluate the economic profitability of business ideas, analyze the conditions, capabilities and means available for their implementation. |
Source: created by the authors based on Zamora-Sánchez et al. (2017, pp. 976 and 977).
As one can see, the first two areas are “social” in nature, aimed at solving the needs of the communities, while the other four are of a more “entrepreneurial” persuasion, providing support to companies and organizations in their activities. This classification is broad enough to cover the different possibilities of university-society linkage.
The surrounding conditions included in the last two points of the input dimension refer on the one hand to the environment surrounding the agents involved in the projects’ implementation and, on the other, to the applicable regulations, which may correspond to different territorial levels.
Variables
As has been demonstrated, the executed projects’ effectiveness and efficiency in each of the identified dimensions must be defined through the different agent groups’ actions and interactions, while taking into consideration the key factors.
To do this, it is necessary to identify the variables whose measurement, in each of the dimensions and in relation to the agents and factors involved, will allow for the comprehensive analysis of the USLPs execution’s impact. In the analysis process, six categories of variables were identified (see Table 3).
Table 3 Variable categories
Categories | Explanation | Variables |
Internal (internal agents and conditioոs) | Refers to the higher educa-tion institution that carries out the USLP, as well as its agents (professors and student). | Problem identification: |
- Initial assessment | ||
- Basic information gathering | ||
- Decision to carry out the USLP | ||
Developing the project | ||
Internal communication | ||
Professors | ||
Students | ||
Physical resources | ||
Time management (professors/students) | ||
Space management | ||
External (external agents and conditions) | Separate from the educational institution, but connected to it through the execution of lhe USLP. | Government agencies |
Beneficiary entities | ||
Time management (activities): matching the work team's schedule with availability of the beneficiary entities | ||
Area of intervention | ||
Communication with beneficiary entities | ||
External communication: external to the executed USLP, between the university and the collaborating public body | ||
Geographic (external environment) | A single variable, corresponds to the location where the USLP is carried out. | Determining the area: rural or urban |
Legal (external conditions) | Also a single variable corresponds to the establishment of regulations, rules or laws in effect in a specific area. | Regulations: |
- Territorial areas | ||
- University | ||
- Collaboration commitments | ||
Financial | Also singular refers to the resources available for executing the USLP | Financial resources |
Consequence | Allows one to know if the USLP were carried out in a timely manner and with a positive effect on the beneficiary entities. | Meeting objectives |
Solving the problem | ||
Performance evaluation |
Fuente: elaboración propia.
The table’s first column indicates the variables’ type or category, and in parentheses, where appropriate, whether they refer to the agents involved in the projects or the environment in which they are carried out. The second column expresses the essential characteristics of each type. The third indicates the specific variables identified in each of the types.
To formulate the indicators, which we address in the following subsection, we took the table into account while making the following necessary observations:
“Government Agencies” will be given special treatment in the evaluation since, as will be shown, no indicators have been formalized for them. In this case we believed it more convenient to obtain information through semi-structured interviews.
Neither have indicators been formalized for the variable “External communication: socialization” since, as is indicated in the table, the ways of socializing the project’s results depend largely on government agencies’ collaboration. As such, the relevant information will be obtained from their interviews.
In the end it was convenient to group some variables into a single reference variable in spite of being presented separately in Table 3. Such is the case with “Time management (professors/students)” and “Time management (activities)” which, as can be seen in the following subsection, were grouped into a single variable: “Time management.” Likewise, “Internal communication” and “Communication with beneficiary entities” were grouped into the more general variable of “Communication.”
On the other hand, as indicated in the table, we grouped into the variable “Regulations” or rules applicable to different territorial areas and levels, including regional and national, as well as the clauses applicable to universities, and the commitment agreements between the parties involved in the USLPs.
Now that these changes have been addressed, we can move on to the topic of the indicators.
Indicators
Approach
The last phase of building a model for evaluating USLPs is proposing a panel of indicators.
Indeed, both the development of the projects and their effects on the beneficiaries need to be quantified as far as possible (Gibbon and Dey, 2011). Indicators are used to quantify based on information gathered from various sources, both documentary in nature and through interviews or surveys of the agents concerned.
In spite there being no specific references in the literature to USLP-related indicators, a review of the literature on impact evaluation of social projects and on university-society linkages yields considerations and suggestions which are useful for this endeavor. We thereby created a panel of indicators based both on the grounds set forth by the literature we reviewed, and on our analysis of the USLPs’ agents, dimensions, factors and variables. Furthermore, as stated, these were validated by a panel of experts just like the model as a whole.
Below we present the proposed indicators, both those for the projects’ development and execution, as well as for their impact.
Development and performance indicators
Tables 4, 5 and 6 have the indicators related to the development and execution of USLPs, linked to the three dimensions identified, and which cover the process from the moment of identifying the problem to the complete fulfillment of the actions associated with each project.
Table 4 Development and execution indicators: input dimension
Agents/Factors | Variables | Indicators | Scales | Verification method |
Director of the linkage department | Project development | - Ability to execute USLPs | - Numerical | - Interviews |
Companies/ Artisans/ Government bodies | Problem identification | - Recognizing the problem | - Opción múltiple | - Interviews |
- Effects on. Beneficiary entities. | - Open | |||
- Time - duration of the problem | - Open | |||
Beneficiaries | Beneficiaries | - Applicant companies | - Numericals | - Interviews |
- Applicant communities | - Records | |||
- Applicant entrepreneurs | ||||
Surrounding conditions | Determining location | - Identifying the area | - Numericals | - Interviews |
- Distance | - Records | |||
- Locations | ||||
Regulations | - National legislation | - Dichotomous/ Open | - Interviews | |
- Regional regulations | ||||
- Clauses | ||||
- Responsibility | ||||
Work team | Professors | - Number professors participating | - Numerical | - Interviews |
Students | - Number of students participating | - Numerical | - Interviews | |
Physical resources | - Those iո charge of materials | - Open | - Interviews | |
Time management | - Professors' hours | - Numerical | - Interviews | |
- Students' hours | - Records | |||
- Total hours executing USLPs | ||||
- Valuation of time management | - Likert scale | |||
Area of intervention | Area of intervention | - Activities carried out according to area of intervention | - Numerical | - Records |
Source: created by the authors.
Table 5 Development and execution indicators: process dimension
Factors | Variables | Indicators | Scala | Verificación method |
USLP Development | Space management | - Environment | - Likert scale | - Surveys - Observation |
Financial resources | - Resource allocation | - Dichotomous/open | - Records | |
- Project cost | - Numerical | - Interviews | ||
- Surveys | ||||
Achieving objectives | - Percentage of US LP's objectives met | - Numerical | - Records | |
- Interviews | ||||
- Plan Compliance Percentage | - Numerical | - Surveys | ||
Surrounding conditions | Communication | - Participants' relationship with each other | Likert scale | - Interviews |
- Complaints and/or grievances | - Dichotomous/open | - Surveys |
Source: created by the authors.
Table 6 Development and execution indicators: outcome dimension
Factors | Variables | Indicators | Scale | Verification method |
Outcome | Salving the problem | - Level of success in solving the problem | - Likert scale | - Records |
- Number of beneficiary entities served | - Numerical | - Interviews | ||
- Contribution/Profitability of the project for the beneficiary entities | - Multiple choice | - Surveys | ||
Performance evaluation | - Beneficiary entities' level of satisfaction | - Likert scale | - Records | |
- Percentage of beneficiary entities satisfied | - Numerical | - Surveys |
Source: created by the authors.
In the three tables, the first column corresponds to the agents that intervene in the USLPs, and to the factors relevant in their development. Both give rise to the variables indicated in the second column, for which we elaborated the indicators that appear in the next column. The last two columns correspond, first, to the type of scales used to represent the indicators, and lastly, to the method used to compile the information related to the indicator (interviews, surveys, records from the USLPs executed during the period analyzed or from direct observations).
The variables and indicators detailed in the previous tables show the development, management and effectiveness of the USLPs, with the goal of easily reaching conclusions through an analysis of the information obtained. This will facilitate the proposal of new and better university-society-State actions.
Impact indicators
The culmination of the model is the proposal of indicators for measuring the impact of USLPs. However, as the impact depends largely on the area of intervention in which each project is carried out, differentiated indicators are presented accordingly.
Table 7 details the areas of intervention, the indicators proposed for each of them and the verification method used.
Table 7 Impact indicators according to área of intervention
Area of intervention | Indicators | Verification method |
Community training | Number of training sessions; number of informative events; number of beneficiaries trained; impact on the lives of beneficiaries; level of satisfaction. | - Records |
- Surveys | ||
Other community services | Number of works carried out; improvement in quality of life of the beneficiary entities; number of beneficiary entities served; level of satisfaction. | - Records |
- Surveys | ||
Technical training | Number of training sessions in companies/associations/guilds; perceived usefulness of the training received; level of satisfaction. | - Records |
- Surveys | ||
Consulting/ advisory | Number of consultations, amount of advice given; perceived usefulness of the consultation/advice given; level of satisfaction. | - Records |
- Surveys | ||
Other technical support services | Number of activities; company's/association's/guild's perception of the solution provided; level of satisfaction. | - Records |
- Surveys | ||
Entrepreneur program | Number of development plans; number of feasibility plans; number of business plans; perception of the activities carried out for lhe undertaking; level of satisfaction. | - Records |
- Surveys |
Source: created by the authors.
We expect to ascertain if USLP execution contributes to improving the conditions of persons or organizations meant to benefit by participating in the USLPs. The question: “would you participate in a USLP again?” also needs to be answered. As the reasons for participating in these projects are linked to the benefits obtained, these impact indicators are particularly
5. CONCLUSIONS
This article presents a model for evaluating the development USLPs and measuring their impact. This entails four groups of elements: the agents, along with their characteristics and the actions they carry out; the dimensions of USLP development, with the agents and factors linked to them; the variables whose measurements are of interest, obtained from the first two groups; and indicators, provided by these measurements.
It is evident that the main groups of agents involved in the projects’ execution are those in the universities responsible for society linkage, the faculty and students in charge of the execution, the governmental organizations and the beneficiary entities (companies/artisans/communities).
The faculty, as well as the students, must have the requisite education and experience for executing the USLPs; furthermore, they are responsible for both planning and executing the project and for evaluating its success and compliance. On the other hand, government organizations help to identify the need to be met, have experience in collaborating with companies and universities, have social recognition and monitor the development of the project. Finally, the beneficiary entities, fundamental agents without which there would be no need for USLPs, are those that identify shortcomings, report to the university, provide the space for the project’s execution and are committed to its development.
As for the dimensions of USLPs’ development, we established the following: input, process and outcome. For each one, agents and related factors were established. Two groups stand out in the latter: the areas of intervention, corresponding to the types of problems the USLPs are meant to solve - six fields, ranging from mostly social activities to training, advice, consulting and entrepreneurship - and the surrounding conditions - applicable regulations and geographical and social environments.
In each of the dimensions, and for each group of agents or factors, a number of key variables were considered. Development indicators were also proposed for each variable; there is also a series of impact indicators for each of the areas of intervention considered.
This makes it possible for this endeavor to contribute to the literature on the approaches and actions within the “Quadruple Helix” approach, where we find USLPs, by demonstrating their advantages in promoting socioeconomic development in regional areas. It is here after all that civil society is an indispensable agent.
On the other hand, we understand that the model presented is purpose-made to make up for the fact that there is currently no model for the evaluation of a USLP’s execution and impact. This means it can be useful not only to universities which carry out this type of projects, so that they may evaluate their execution and propose ways to improve them, but also to the supervisory bodies which can now have comprehensive information on the projects, and even to the beneficiary entities as their opinion will be available for the evaluation and impact measurement. The result will be an impact on obtaining greater benefits from future USLPs.
It is evident that with the application of the model in a variety of circumstances, both its strengths and areas for improvement will come to light. This process has already started with its application at the Technical University of Ambato, and we hope that its continued practical application will soon verify its value.
REFERENCES
Altbach, P. G. (2008). The complex roles of universities in the period of globalization. En Global University Network for Innovation: Higher Education in the World 3: New Challenges and Emerging Roles for Human and Social Development (pp. 5-14). Palgrave MacMillan. [ Links ]
Aranguren, M. J., Guibert, J. M., Valdaliso, J. M. y Wilson, J. R. (2016). Academic institutions as change agents for territorial development. Industry and Higher Education, 30(1). https://doi.org/10.5367/ihe.2016.0289. [ Links ]
Bas, T., Amoros, E. y Kunc, M. (2008). Innovation, entrepreneurship and clusters in Latin America natural resource: implication and future challenges. Journal of Technology Management and Innovation, 3(4). https://doi. org/10.4067/S0718-27242008000200005. [ Links ]
Brito-Gaona, L., Quezada-Abad, C. y Arzola de la Rosa, L. (coords.) (2018). La universidad y la vinculación con la sociedad. Una condición impostergable. Editorial UTMACH. [ Links ]
Bueno, E. y Casini, F. (2007). La tercera misión de la universidad. Enfoques e indicadores básicos para su evaluación. Revista Economía Industrial, 366. https://dialnet.unirioja.es/servlet/articulo?codigo=2510911. [ Links ]
Carayannis, E. G. y Campbell, D. F. (2009). Mode 3 and the Quadruple Helix: Toward a 21st century fractal innovation ecosystem. International Journal of Technology Management, 46(3/4 ). https://doi.org/10.1504/IJTM.2009.023374. [ Links ]
Castro-Martínez, E., Olmos-Peñuela, J. y Fernández-de-Lucio, I. (2016). La vinculación ciencia-sociedad: estereotipos y nuevos enfoques. Journal of Technology Management & Innovation, 11(2). http://dx.doi.org/10.4067/ S0718-27242016000200012. [ Links ]
Chandra, A. (2007). Business incubation in Brazil: creating an environment for entrepreneurship. Networks Financial Institute Working Paper, n° 2007-WP-25. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1058901. [ Links ]
Cohen, E. y Martínez, R. (2002). Manual: formulación, evaluación y monitoreo de proyectos sociales. División de Desarrollo Económico, Naciones Unidas - CEPAL. https://dds.cepal.org/redesoc/archivos_recursos/242/Manual_dds_200408.pdf. [ Links ]
Drucker, J. y Goldstein, H. (2007). Assessing the regional economic development impacts of universities: A review of current approaches. International Regional Science Review, 30(1). https://doi. org/10.1177/0160017606296731. [ Links ]
Esteves, A. M., Franks, D. y Vanclay, F. (2012). Social impact assessment: the state of the art. Impact Assessment and Project Appraisal, 30(1). https://doi.org/10.1080/14615517.2012.660356. [ Links ]
Etzkowitz, H. y Leydesdorff, L. (2000). The dynamics of innovation: from National Systems and 'Mode 2' to a Triple Helix of university-industry-government relations. Research Policy, 29(2). https://doi.org/10.1016/S0048-7333(99)00055-4. [ Links ]
Fonseca, L. y Nieth, L. (2021). The role of universities in regional development strategies: A comparison across actors and policy stages. European Urban and Regional Studies, 28(3). https://doi.org/10.1177/0969776421999743. [ Links ]
Frondizi, R., Fantauzzi, C., Colasanti, N. y Fiorani, G. (2019). The evaluation of universities' third mission and intellectual capital: Theoretical analysis and application to Italy. Sustainability, 11(12 ). https://doi.org/10.3390/su11123455. [ Links ]
Gertler, P. J., Martínez, S., Premand, P., Rawlings, L. B. y Vermeersch, C. M. J. (2017). La evaluación de impacto en la práctica. World Bank Publications. https://openknowledge.worldbank.org/bitstream/handle/10986/25030/9781464808883.pdf. [ Links ]
Gibbon, J. y Dey, C. (2011). Developments in social impact measurement in the third sector: scaling up or dumbing down? Social and Environmental Accountability Journal, 31(1). https://doi.org/10.1080/0969160X.2011.556399. [ Links ]
Giuliani, E. y Bell, M. (2005). The micro-determinants of meso-level learning and innovation: evidence from a Chilean wine cluster. Research Policy , 34(1). https://doi.org/10.1016/j.respol.2004.10.008. [ Links ]
Goddard, J. (2009). Re-inventing the civic university. NESTA Provocation No. 12, NESTA Foundation. https://media.nesta.org.uk/documents/reinventing_the_civic_university.pdf. [ Links ]
Gómez-Gajardo, F. (2017). Proyectos de innovación como estrategia de vinculación universidad empresa. La experiencia de la Universidad Tecnológica de Chile INACAP y el Nodo Osorno Natural Beef, Región de Los Lagos, Chile. En A. Cazorla y R. Stratta (coord.). La universidad: motor de transformación de la sociedad (pp. 130-140). Universidad Politécnica de Madrid. http://sedici.unlp.edu.ar/handle/10915/60117. [ Links ]
González, A., Lavin, J. y Pedraza, N. A. (2020). El papel de los actores de la cuádruple hélice en el emprendimiento tecnológico de Tamaulipas. Paradigma Económico, 12(2).https://paradigmaeconomico.uaemex.mx/article/view/15521. [ Links ]
Gray, B. y Purdy, J. M. (2018). Collaborating for our future: Confronting complex problems through multi-stakeholder partnerships. Oxford University Press. [ Links ]
Kerzner, H. y Kerzner, H. R. (2017). Project management: a systems approach to planning, scheduling, and controlling. Wiley & Sons. [ Links ]
Kroeger, A. y Weber, C. (2014). Developing a conceptual framework for comparing social value creation. Academy of Management Review, 39(4), https://doi.org/10.5465/amr.2012.0344. [ Links ]
Laredo, P. (2007). Revisiting the third mission of universities: toward a renewed categorization of university activities? Higher Education Policy, 20(4). https://doi.org/10.1057/palgrave.hep.8300169. [ Links ]
Macías, C. A. M., Vélez, T. G. J. y Loor, M. E. V. (2017). La Participación de la Comunidad Universitaria y su Vinculación con la Sociedad. Revista Científica Hallazgos 21, 2(3).https://revistas.pucese.edu.ec/hallazgos21/article/view/192. [ Links ]
Manrique, S. (2019). Exploring the impact of university-firm collaboration on regional development: The Spanish case. Working Paper 02/2019, RUNIN Project: The Role of Universities in Innovation and Regional Development. https://doi.org/10.3990/4.2535-5686.2019.02. [ Links ]
Miller, K., McAdam, R. y McAdam, M. (2018). A systematic literature review of University Technology transfer from a quadruple helix perspective: Toward a research agenda. R&D Management, 48(1). https://doi. org/10.1111/radm.12228. [ Links ]
Moctezuma, P., López, S. y Mungaray, A. (2017). Innovación y desarrollo: programa de estímulos a la innovación regional en México. Problemas del Desarrollo. Revista Latinoamericana de Economía, 48(191 ). https://doi.org/10.1016/j.rpd.2017.11.007. [ Links ]
Morales, M., Pineda, K. y Ávila, K. (2012). Organizaciones innovadoras a partir de la interacción con la universidad: casos exitosos. Estudios Gerencíales, 28. https://doi.org/10.18046/j.estger.2012.1493. [ Links ]
Morales-Gualdrón, S. T. y Giraldo Gómez, A. S. (2015). Análisis de una innovación social: el Comité Universidad Empresa Estado del Departamento de Antioquia (Colombia) y su funcionamiento como mecanismo de interacción. Innovar, 25(56 ). http://dx.doi.org/10.15446/innovar.v25n56.48996. [ Links ]
Ordóñez, S. (2017). Sistemas de innovación y conocimiento: el caso de Jalisco, México. Problemas del Desarrollo. Revista Latinoamericana de Economía , 48(191 ). https://doi.org/10.1016/j.rpd.2017.11.008. [ Links ]
Pavón-Silva, T., Barrera-Díaz, C., Pacheco-Salazar, V., Sánchez-Meza, J., Gómez-Beltrán, G. y Moreno-Colín, R. (2007). Beneficios de la vinculación universidad-sector productivo. Revista Ingeniería, Investigación y Tecnología, 8(1).http://www.revistas.unam.mx/index.php/ingenieria/article/view/13462. [ Links ]
Pereira da Veiga, C. R., Pereira da Veiga, C., Del Corso, J. M. y Vieira da Silva, W. (2016). Vaccine R&D in Brazil: The effectiveness of push and pull regulations. World Patent Information, 46(C). https://doi.org/10.1016/j.wpi.2016.05.006. [ Links ]
Pineda, K., Morales, M. y Ortiz, M. (2011). Modelos y mecanismos de interacción universidad-empresa-Estado: retos para las universidades colombianas. Equidad y Desarrollo, 15. https://dialnet.unirioja.es/servlet/articulo?codigo=5166538. [ Links ]
Pugh, R., Hamilton, E., Jack, S. y Gibbons, A. (2016). A step into the unknown: universities and the governance of regional economic development. European Planning Studies, 24(7). https://doi.org/10.1080/09654313.2016.1173201. [ Links ]
Ramírez, M. y García, M. (2010). La alianza universidad-empresa-estado: una estrategia para promover innovación. Revista EAN, 68. https://www.redalyc.org/pdf/206/20619844010.pdf. [ Links ]
Rodríguez-Castellanos, A. y Zamora-Sánchez, R. (2020). ¿Qué pueden hacer las universidades para remediar la desigualdad socioeconómica? En A. M. Gil-Lafuente (coord.). Los confines de la equidad y desigualdad en la prosperidad compartida (pp. 105-134). Real Academia de Ciencias Económicas y Financieras. [ Links ]
Rueda, I., Acosta, B. y Cueva, F. (2020). Las universidades y sus prácticas de vinculación con la sociedad. Educação & Sociedade, 41. https://doi.org/10.1590/ES.218154. [ Links ]
Saavedra, M. L. (2009). Problemática y desafíos actuales de la vinculación universidad empresa: El caso mexicano. Actualidad Contable FACES, 12(19 ).https://www.redalyc.org/pdf/257/25715409009.pdf. [ Links ]
Sánchez Ambríz, G. y Pérez Balbuena, J. J. (2018). La tercera misión de las universidades: innovación, emprendimiento y compromiso social. Revista Universitaria Digital en Ciencias Sociales (RUDICS), 9(17 ).https://virtual.cuautitlan.unam.mx/rudics/?p=2514. [ Links ]
Secundo, G., Perez, S. E., Martinaitis, Ž. y Leitner, K. H. (2017). An intellectual capital framework to measure universities' third mission activities. Technological Forecasting & Social Change, 123. https://doi.org/10.1016/j.techfore.2016.12.013. [ Links ]
Stufflebeam, D. L. y Shinkfied, A. J. (1993). Evaluación sistemática: guía teórica y práctica. Editorial Paidós. [ Links ]
Urra, M. (2018). Estado, mercado, academia... y comunidad. Una cuádruple hélice para el desarrollo integral y la innovación [Tesis doctoral, Universidad Pontificia de Comillas]. https://repositorio.comillas.edu/xmlui/hand-le/11531/26826. [ Links ]
Valero, A. y Van Reenen, J. (2019). The economic impact of universities: evidence from across the globe. Economics of Education Review, 68. https://doi.org/10.1016/j.econedurev.2018.09.001. [ Links ]
Vera, J. (2009). Cluster del salmón en Chile: análisis de los factores de competitividad a escala internacional. Revista Venezolana de Gerencia, 14(47 ). https://doi.org/10.31876/revista.v14i47.10538. [ Links ]
Zamora-Sánchez, R., Rodríguez-Castellanos, A. y Barrutia-Güenaga, J. (2017). Acercamiento de la universidad a la problemática social: los Proyectos Universitarios de Vinculación con la Sociedad. El caso de la Universidad Técnica de Ambato. En P. Laguna Sánchez y A. Blanco González (coords.). XXXI Congreso Anual de AEDEM. Empresa y Sociedad: Investigación e Innovación Responsable (pp. 967-985). European Academic Publisher. [ Links ]
3While the “intellectual capital” approach (Secundo et al., 2017; Frondizi et al., 2019) is of interest, it is also beyond the scope of this study.
Received: July 30, 2021; Accepted: February 16, 2022