ARTÍCULO ORIGINAL

 

Design of the 2009 evaluation of the Medical Insurance for a New Generation program

 

Diseño de la evaluación 2009 del programa Seguro Médico para una Nueva Generación

 

 

Ricardo Pérez-Cuevas, MD, MSc, DrSc^I; Onofre Muñoz-Hernández, MD, MSc^II; Evelyne Rodríguez-Ortega, MPP^III; Luis Jasso-Gutiérrez, MD, MSc^IV; Samuel Flores-Huerta, MD^V; Luis Durán-Arenas, MD, PhD^I; Mireya Pasillas-Torres, MSc^IV; Juan Garduño-Espinosa, MD, MSc, DrSc^II; Gabriel Cortés-Gallo, MD.^VI

^ICentro de Estudios Económicos y Sociales en Salud, Hospital Infantil de México Federico Gómez. México
^IIDirección de Investigación, Hospital Infantil de México Federico Gómez. México
^IIIDepartamento de Investigación en Políticas de Salud, Hospital Infantil de México Federico Gómez. México
^IVDepartamento de Evaluación y Análisis de Medicamentos, Hospital Infantil de México Federico Gómez. México
^VDepartamento de Investigación en Salud Comunitaria, Hospital Infantil de México Federico Gómez. México
^VISeguro Médico para una Nueva Generación, Sistema de Protección Social en Salud, Secretaría de Salud. México

Corresponding author

 

 

---

ABSTRACT

OBJECTIVE: To describe the mixed-method approach to evaluate the Medical Insurance for a New Generation (Seguro Médico para una Nueva Generación, SMNG).
MATERIALS AND METHODS: The program has been comprehensively evaluated. It has four key domains: 1) SMNG design; 2) children's health status and socio-demographic characteristics; 3) performance by measuring coverage, efficiency and productivity; 4) family health expenditure. Quantitative and qualitative research approaches have been used. This included reviews of existing databases and clinical charts, collection of empirical data through in-depth interviews with healthcare providers, and a nation-wide household survey.
CONCLUSION: The results should serve as baseline data of the health status of SMNG children and the current staus of the program.

Keywords: evaluation, health services, social programs, access, and coverage

---

RESUMEN

OBJETIVO: Describir el abordaje de métodos mixtos que se utilizó para evaluar el SMNG Seguro Médico para una Nueva Generación.
MATERIAL Y MÉTODOS: la evaluación integral del Programa se llevó a cabo en cuatro áreas clave: 1) el diseño del SMNG, 2) el estado de salud y las características sociodemográficas de los niños afiliados, 3) el desempeño del SMNG mediante la medición de la cobertura y de procesos clave, 4) los gastos en salud de la familia. La metodología mixta combina enfoques de investigación cuantitativa y cualitativa y la recolección y análisis de datos. La evaluación incluyó la revisión de bases de datos y de expedientes clínicos y la recolección de datos empíricos mediante entrevistas a profundidad con actores clave, una encuesta de hogares en todo el país y cuestionarios a personal de salud.
CONCLUSIÓN: Los resultados sirven como referencia sobre el estado de salud de los niños afiliados al SMNG y de las condiciones actuales del programa; además permiten identificar áreas clave que pueden abordarse para mejorar la calidad, eficiencia y eficacia del SMNG.

Palabras clave: evaluación; servicios de salud; programas sociales; acceso; equidad

---

 

 

Evaluation is "the use of social research procedures to systematically investigate the effects of social intervention programs".¹ Research-based evaluation provides a systematic and objective analysis of public policies, budgetary programs, and institutional performance. Evaluation addresses different dimensions of healthcare programs and deals with efficiency, efficacy, quality, and effectiveness.

The most recent large-scale public policy for healthcare in Mexico is the System for Social Protection in Health, which is commonly called Seguro Popular (Popular Insurance). In 2003, the government initiated it to offer subsidized, publicly provided health insurance to the people without social security to achieve equitable access and coverage of health services. Seguro Popular is the operational component of the health sector reform that has several objectives, namely to: 1) increase the public spending in health; 2) stimulate efficient allocation, and 3) protect families from excessive health expenditures.² Seguro Popular affiliates are not supposed to pay for health care, neither for the drugs they receive.

Seguro Popular policies provide evidence of progress, such as mobilization of public resources, availability of drugs, service utilization and financial protection.³ In 2011, 48 million people were affiliated to Seguro Popular. Furthermore, nationwide health programs aimed at improving the health of children of <5 years of age are currently in place. Despite these efforts, a number of children's health-needs remain unmet.⁴ Inequities in access to healthcare between rural and urban populations and asymmetries in the health status of this age group persist.^5,6 The pervasively high rates of neonatal mortality^7,8 malnutrition,^9,10, anemia,¹¹ communicable diseases,¹² low birth weight^13,14 and the unsatisfied demand for high cost health problems remain unsolved. The access to neonatal intensive care units¹⁵ of premature newborns with respiratory distress syndrome, and the increasing rates of chronic conditions such as cancer (mainly leukemia), point to the need for further resources.¹⁶

In 2007, the Federal Government through the Ministry of Health (MoH) launched Seguro Médico para una Nueva Generación (SMNG) as part of the System for Social Protection for Health (Seguro Popular). SMNG is a program that guarantees funding for health services to children born after December 2006, that are not affiliated with social security institutions. It is a capitated payment system in which the resources are allocated to the decentralized Ministries of Health (MoH) in each Mexican state. The funds are allocated in accordance to the number of affiliated children. SMNG funding complements MoH healthcare expenditure for ambulatory (primary care) and hospital health care (secondary care). Another fund, the Fund for Protection against Catastrophic Expenditures (FPGC) of Seguro Popular covers the treatment of high cost diseases, including cancer, congenital malformations and neonatal respiratory distress syndrome, among others.

SMNG has five major objectives, namely to: a) reduce the incidence of morbidity and mortality among neonates, infants and preschool children; b) enhance children's healthy growth and development; c) ensure access to health care to its members, thus improving equity; d) provide a package of health benefits congruent with the epidemiological profile that can respond to the health needs and demands of the covered population; e) avoid impoverishment of families due to healthcare expenditure.

According to the mandate for evaluating public policy programs, SMNG authorities commissioned to Hospital Infantil de Mexico Federico Gomez (HIMFG) run an evaluation of SMNG. This initiative might serve well as the starting point for subsequent evaluations aimed at ascertaining the achievements of the goals.

SMNG is a social program that has been planned, organized and carries out a set of activities (i.e., promotion of the program to affiliate members, acts as an active stakeholder in the design of health policies and financing of health care. For example, SMNG financed rotavirus and pneumococcal vaccines to improve the health status of children <5 years of age. SMNG is not a direct healthcare provider. This evaluation addressed four key program domains:

1. Evaluation of the design of SMNG.

2. Evaluation of children's health status and their socio-demographic characteristics.

3. Evaluation of SMNG's performance by measuring coverage, productivity and efficiency.

4. Health expenditure of the family.

This paper describes the design of the evaluation. It addresses the conceptual framework, rules of decision, and particular methods to evaluate these domains. Individual papers analyze the results of each domain and have been published within this series. The paper discusses the importance of scientific credibility in carrying out evaluations of nationwide health programs, and the relevance in answering the questions and needs of the stakeholders.

 

Materials and methods

Evaluation of the design of SMNG

There were six sections: a) SMNG characteristics; b) contribution of the SMNG to the strategic objectives of Seguro Popular and congruence with the objectives of the National Health Plan 2007-2012 of Mexico;¹⁷ c) evaluation of the logical framework matrix; d) definition of the target population; e) links to the operating rules; f) coincidences, complementarities and duplicities.

The evaluation included the revision of SMNG documentation of its normative and legal framework, and in-depth interviews with key informants, such as SMNG and Seguro Popular decision-makers. This was done in accordance with the norms stated by the National Council for the Evaluation of Social Development Policy (CONEVAL). CONEVAL is an autonomous institution responsible for evaluating social development policies and programs. Two ad-hoc data collection instruments were designed. One instrument described the general characteristics of the program. The other instrument was a questionnaire designed to collect information about SMNG design, policies and operations procedures.

Evaluation of children's health status and their socio-demographic characteristics

The conceptual framework for the evaluation of the health status of children followed the perspective of evaluating health systems performance proposed by the World Health Organization.¹⁸ It also considered the rules and objectives of SMNG.¹⁹ This allowed clear definition of the background, as well as the justification and goals of the evaluation.

HIMFG signed an agreement with the Instituto Nacional de Estadística y Geografía (INEGI) to conduct a nationwide urban-rural representative survey of children <2 years of age who were affiliated to SMNG at the time of the study. The Committee of Ethics and Research of the HIMFG reviewed and approved the protocol. Informed consent of the mother or caretaker of the child was obtained according to the Ethical Principles for Medical Research in Human Beings" as stated in the Helsinki Declaration.

The general objective of the survey was to:

1. Present a diagnosis of the SMNG members in the following aspects: a) health status of the child; b) socio-economic characteristics of the household; c) access and quality of healthcare services, and d) out-of-pocket and catastrophic health expenditure of SMNG children.

Table I describes the main components of the survey. The units of analysis were children affiliated with SMNG who were born between December 1^st, 2006 and September 30^th, 2008. The units of observation were households, which were identified through the address list in the SMNG registries.

Statistical design

Geographical coverage: the 2009 SMNG survey was designed to provide precise and reliable information at two levels: national level representativeness, and urban-national and rural-national representativeness. An urban setting was defined as a community with >2500 inhabitants. The sample followed a probabilistic design to allow generalization of the results to the population under study and calculate the magnitude of the error in the estimates. In each Mexican state, the sample was proportionally distributed in accordance with the size of the population under study.

Sampling frame

SMNG provided an updated database of children that served as the sampling frame for the survey. The database contained information on 1545717 children registered during the study period. The sampling design included a 3-stage cluster sampling approach. The primary sampling units were the Municipios (counties), the secondary sampling units were the villages or communities, and the tertiary sampling units were SMNG children. The subsample of children from whom anthropometric measurements were taken was proportional to the size of the main sample. A simple random sampling technique served for this component. INEGI undertook a review of SMNG registries and a pilot study to identify potential problems to find both households and interviewees. It was discovered that 40% of registries had problems, such as incomplete information, duplicates, etc. The original sample size was estimated using a standard demographic formula for surveys that resulted in 8040 children. Nevertheless, due to the problems identified in the quality of the registries, the sample increased to 12240 children. This sample size assured the provision of results with national and rural/urban representativeness to estimate proportions with p=0.05. The subsample for anthropometric measurements involved 6240 children and the subsample to measure hemoglobin was 760 children.

Data collection

A pilot study was conducted in the Mexican states of Veracruz and Mexico to test the feasibility of finding the participants, test the electronic questionnaire, and improve the training of the interviewers, particularly in collecting anthropometric measurements.

The household survey was conducted from March 16 to April 10, 2009. Data was collected by interviewing the mother or primary caretaker (in cases where the mother was permanently absent). The interviewers used the electronic questionnaire. Data were entered on site into a mini-laptop. The equipment used for the anthropometric measurements were electronic scales with 10 g of precision (Seca 354) and portable infantometers (Seca 210). The Z-scores were used to ascertain the nutritional status: weight for age, height for age, and body mass index. To measure hemoglobin, a hemocue was used. The prevalence of anemia was adjusted according to the altitude above sea-level of the communities, using the Ruiz-Arguelles' formula. The cut-off values to ascertain anemia were those proposed by the WHO.

The data analysis was carried out using descriptive and inferential statistics. The proportions and measures of central tendency and dispersion were estimated with the corresponding expansion factors of the respective sample and subsamples. Chi-square and student's t tests were used to compare groups. Multiple logistic regression analyses were carried out to identify potential associations i.e., socioeconomic factors and anemia. The statistical packages STATA 8.5 and SPSS 17 were used to analyze the data.

Evaluation of health services performance for SMNG children

The evaluation of performance comprised productivity and efficiency. Productivity was measured by analyzing the supply and coverage of specific services within the package of benefits regarding the target population and their geographical distribution (Table II). Efficiency included two aspects, clinical efficiency and management of financial resources (Table III).

 

 

The evaluation was based on the indicators defined in the 2008-SMNG operating rules, the Catalog of SMNG Essential Health Services and the Catalog of Diseases (focusing on neonatal health care) covered by the Fund for Catastrophic Expenditures.

Three methods were used to collect the required information to evaluate performance:

a. Population survey. Selected information from the 2009 SMNG survey was used to evaluate some of the indicators.

b. Document review of the databases of SMNG children receiving healthcare during 2008. These databases comprised: 108 health conditions covered by the SMNG package (108-SMNG), conditions not included in the package (non-108-SMNG) and health services that the Fund for Catastrophic Expenditure (FPGC) covers particularly neonatal intensive care unit services (FPGC-NICUs). The authorities of SMNG and FPGC provided the 2008 databases. The database of FPGC-NICUs had information about premature and term newborns with diagnosis of respiratory distress syndrome and/or sepsis. Within these two groups, five conditions were identified: aspiration of meconium, newborn transitory tachypnea, respiratory distress syndrome, perinatal asphyxia and non-specified respiratory distress.

c. Health care services surveys. The surveys comprised 8 data collection instruments applied in a representative sample of primary care clinics, secondary and tertiary care hospitals, and health providers. The instruments were designed to collect information based on the 2008 SMNG operating rules. Table IV describes their main content.

The evaluation also included five tracers to analyze in-depth the infrastructure, quality and training of health providers. The tracers for primary care were iron supplementation, and monitoring of growth and development of children; for secondary care, these were low birth weight and detection of congenital hypothyroidism, and for tertiary care neonatal respiratory distress syndrome.

Sampling design. The units of analysis were health providers working in accredited medical facilities in which SMNG children were born after December 1^st, 2006.

A national representative sample of units was estimated. A 3-stage sampling design was used: the first stage comprised five regions and its corresponding states: North (Chihuahua, Coahuila, San Luis Potosí and Tamaulipas), Center (Guanajuato, Hidalgo, Estado de México, Michoacán, Querétaro and Tlaxcala), South (Campeche, Quintana Roo y Veracruz), West (Jalisco) and Mexico City. The regions were defined geographically, socially, by economic development and use of services criteria. The second stage comprised a random selection of medical facilities. The third stage comprised a random selection of medical doctors selected from the medical facility payrolls. Only medical doctors working the morning shift were included: 67 primary care doctors; 73 working in secondary care and 97 in tertiary care facilities.

The assumptions in estimating the sample size for the five tracers were based upon the rate of service utilization per person per year reported in the 2006 National Health and Nutrition Survey (ENSANUT 2006), which permitted calculation of the number of clinical charts needed to be reviewed.

The interviewers were trained previously and the data collection instruments were pre-tested in a pilot study. The data analysis used only descriptive statistics to calculate rates, measures of central tendency and proportions.

Evaluation of health expenditures of the family

The object was to estimate the effect of SMNG on out-of-pocket and catastrophic expenditures due to healthcare for SMNG children. The definition of out-of-pocket expenditure included all claims made to obtain healthcare for a child. These included: honoraria of medical doctors and medications, diagnostic tests and hospitalization expenses, and payments to alternative medicine procedures or traditional healers.

The above-mentioned nationwide survey collected the following information. Expenditure on child's health care included four components:

1. 3-month period of expenditure on a child's ambulatory healthcare, comprising cost of consultations, medications and diagnostic procedures;

2. 12-month period of expenditure on a child's hospitalization (when applicable) –the total amount spent when the child was hospitalized;

3. Source of the economic resources for health expenditure: savings, loans, donations, etc., besides the regular income of the family to afford healthcare;

4. Delay or lack of healthcare due to economic constraints in the last 12 months.

Household income and expenditure were broken down as follows:

1. Last month expenditure on food, beverages, public transportation, alcoholic beverages and cigarettes;

2. 3-month expenditure on clothes, education (tuition, stationery, etc.), leisure activities, rent of the household, services (water, electricity, communication), and healthcare;

3. 6-month period expenditure on furniture, maintenance or repair of the household, transportation, purchase of motor vehicles and repairs.

In addition, the survey collected information about the affiliation of any family member with social support programs such as Oportunidades, Liconsa, Procampo or other.

 

Discussion

To gain knowledge about the actual condition of a program, its achievements, weaknesses and future action-directedness, it was worthwhile assessing several key domains: the need for the program, its design, implementation, service delivery, effect and efficiency.20,21 SMNG aims to reinforce Seguro Popular; thus, it should be viewed within a set of co-existing social and health policies that aim to improve the well-being of children <5 years old.

Evaluation of the design of SMNG

SMNG is a tax-based health insurance. It aims to strengthen the financial capability of the decentralized MoH in each Mexican state in providing healthcare, and simultaneously to ease access to health services. The opening strategy was to include children born after December 2006 who were not affiliated with the social security. The SMNG is fully linked to the National Development Plan 2007-2012 and the Health National Plan 2007-2012. The analysis of the matrix of indicators showed that the aim of SMNG is to contribute to the universal health insurance, and its purpose is to facilitate access to all actions of socially protected health. Its target population is well-defined in terms of its vulnerability. Social exclusion, poverty and lack of access to basic services and social security, such as healthcare, are the main determinants of the vulnerability of this group. One of every 10 deaths occurs before a child reaches the age of 5, 40% of deaths occurring within the neonatal period.²²

The evaluation of SMNG's design showed that the program is well aligned with current MoH and Seguro Popular policies. Although the decision to only affiliate children born after December 2006 to SMNG instead of all children under 5 should be analyzed in-depth. Affiliating all children <5 years, instead of only those born after December 2006 would be an opportunity to hasten the decline morbidity and mortality rates of under-5s. Nevertheless, 40% of under-5 mortality happens in newborns and infants. Therefore, focusing on this group was a good starting point of the program. Furthermore, SMNG has the potential of complementing the actions of Seguro Popular, given that all members of a family should be affiliated with this insurance once the child becomes an SMNG member. From the perspective of health policy, the evaluation answers important questions that help the government to accomplish its commitment of using research-based evidence to reinforce social programs and manage results.

Survey to evaluate the children's health status and their socio-demographic characteristics

The survey collected complete information on 75.3% of the original sample size (9211 children). The non-response percentage was 21.7%: addresses not located 7.1%; families not located 8.5%; families who had moved 6.1%; and 1.1% "other", such as duplicated registries. The subsample of hemoglobin was taken in 74.1% of the estimated sample size (563 children).

The sample was representative at a national level of 1 520 494 children <=2 years old. Regarding the division of urban and rural, the sample represented 1095116 and 425378 children, respectively. The distribution by sex was similar between both groups. (Table V).

Evaluation of health services performance for SMNG affiliates

The survey collected information to evaluate health service performance. The mothers were not completely clear as to which insurance their children were affiliated with and were confused by the name of the program: 30.9% knew that the child was with SMNG; 13.3% answered that the child was with SMNG and Seguro Popular; and 47.6% that the child was with Seguro Popular. About 8% were unaware of the affiliation of the child.

Regarding the survey at medical facilities, information from all 82 medical facilities included in the sample was obtained: 29 primary care clinics, 30 secondary care hospitals and 21 tertiary care hospitals. The distribution of medical doctors interviewed was as follows: 67 general practitioners; 73 physicians working in secondary care hospitals, among which 60% were pediatricians; and 91 pediatricians working in tertiary care hospitals.

Evaluation of the health expenditure of a family

The household survey provided the information to evaluate the effect of SMNG on the health expenditure. The calculations comprised out-of-pocket and catastrophic expenditure. The analysis showed that 63% of households spent on their children's healthcare, and between 4.3 and 11.6% had catastrophic expenditure. Furthermore, 15.5% delayed healthcare and 10.0% did not seek healthcare due to economic constraints.²³

Strengths and weaknesses of the study

This study has several strengths and weaknesses. Among the strengths, the study used a triangulated²⁴ or mixed-method approach. This means a combination of quantitative and qualitative approaches in the research methods, data collection and data analysis.²⁵ This included review of existing databases and clinical charts, and collection of empirical data through in-depth interviews with key stakeholders, household surveys, and questionnaires to healthcare providers. These methods complemented each other in providing an in-depth understanding of the evaluated program domains. The collaboration of INEGI in the design and implementation of the nationwide survey was key to obtaining the best possible information. Another strength is that the study was carried out with the support of different stakeholders, such as the central authorities of Seguro Popular, the local state representatives of Seguro Popular, and the authorities and medical doctors of each medical facility, which helped to accomplish the project goals.

Among the weaknesses of the study was that it did not investigate cause-and-effect relationships, and therefore the design was not experimental; thus the study lacks a comparison group. Also, selecting the comparison group would be a challenge. There are many programs and/or institutions in Mexico that provide healthcare, intervention and support to families and children; thus, choosing a suitable comparison group and differentiating its attributable effect was beyond the objectives of this study.

Evaluation of efficiency also has shortcomings. Ideally, the allocative and production efficiencies should be part of the SMNG evaluation. Allocative efficiency would identify whether the mix of services maximizes the expected health outcome for the available share of resources. Production efficiency evaluates whether SMNG contributes in producing health services at minimum cost.²⁶ The perspective of the study focused on evaluating the use of different funds, such as per capita allocation, reimbursement to the health provider and allocation of funds for vaccines. In-depth analysis to learn about allocative and production efficiency would strengthen the results.

We can conclude that, given the recent implementation of the program, this evaluation is an unbiased assessment providing baseline data that should allow future comparisons with the results of subsequent evaluations. It also identifies the areas that should be addressed in the near future.

 

Acknowledgment

Realization of this evaluation was achieved thanks to the financial support from the Sistema Nacional de Protección en Salud and the Seguro Médico para una Nueva Generación. The authors also extend their most sincere gratitude to Instituto Nacional de Estadística y Geografía (INEGI) for its technical guidance and collaboration.

Declaration of conflict of interests: The authors declare not to have conflict of interests.

 

References

1. Rossi P, Freeman H, Lipsey M. Evaluation: a systematic approach. 6^th ed California. Sage Publications, 1999.         [ Links ]

2. Knaul F, Frenk J. Health Insurance In Mexico: Achieving Universal Coverage Through Structural Reform. Health Affairs 2005; 24(6):1467-1476.         [ Links ]

3. Gomez-Dantes O, Knaul F, Frenk J. The democratization of health in Mexico: financial innovations for universal coverage. Bull World Health Organ 2009;87:542–548 doi:10.2471/BLT.08.053199        [ Links ]

4. OECD Reviews of Health Systems Mexico. Paris: Organization for Economic Cooperation and Development, 2005.         [ Links ]

5. Barraza-Llorens M, Bertozzi S, González-Pier E, Gutiérrez JP. Addressing inequity in health and health care in Mexico. Health Affairs 2002; 21(3): 47-57.         [ Links ]

6. Reyes-Morales H, Gómez-Dantés H, Torres-Arreola LP, Tomé-Sandoval P, Galván-Flores G, González- Unzaga M. Necesidades de salud en áreas urbanas marginadas de México. Rev Panam Salud Publica. 2009;25(4):328-236.         [ Links ]

7. Perdigón-Villaseñor G, Fernández-Cantón S. La mortalidad neonatal y postneonatal en México, 1980-2005. Bol Med Hosp Infant Mex. 2008; 65: 412-414.         [ Links ]

8. Consejo Nacional de Población. Proyecciones de la población de México, 2000-2050. México: CONAPO, 2002.         [ Links ]

9. Ávila-Curiel A, Shamah-Levy T. Diagnóstico de la magnitud de la desnutrición en México. Consejo Nacional de Población: México ante los desafíos de desarrollo del milenio, México: Secretaría de Gobernación, 2005.         [ Links ]

10. Rivera-Dommarco J, Shamah-Levy T, Villalpando-Hernández S, González-de Cossío T, Hernández-Prado B, Sepúlveda B J. Encuesta Nacional de Nutrición 1999. Estado nutricio de niños y mujeres en México. Cuernavaca: Instituto Nacional de Salud Pública, 2001.         [ Links ]

11. Olaiz G, Rivera J, Shamah T, Rojas R, Villalpando S, Hernández M, et al. Encuesta nacional de salud y nutrición 2006. Cuernavaca: Instituto Nacional de Salud Pública; 2006.         [ Links ]

12. Oláiz-Fernández G, Rivera-Dommarco J, Shamah-Levy T, Villalpando-Hernández S, Hernández-Ávila M, Sepúlveda-Amor J. Encuesta Nacional de Salud y Nutrición 2006. Cuernavaca: Instituto Nacional de Salud Pública, 2006.         [ Links ]

13. OECD Family database - Social Policy Division - Directorate of Employment, Labour and Social Affairs. CO1.3: Low birth weight. [Accessed 2010 September 23]. Available at: www.oecd.org/els/social/family/databaseOECD.         [ Links ]

14. Frank R, Pelcastre B, Salgado-de Snyder V, Frisbie WP, Potter JE, Bronfman-Pertzovsky M. Low birth weight in Mexico: New evidence from a multi-site postpartum hospital survey. Salud Publica Mex 2004;46:23-31.         [ Links ]

15. Jasso L. Mortalidad perinatal y neonatal: Neonatologia practica. 7a edición. Mexico: El Manual Moderno, 2008.         [ Links ]

16. Rivera-Luna R. Conceptos generales del cáncer infantil en México. Oncología Pediátrica. México: Editorial Intersistemas, 2002.         [ Links ]

17. Secretaría de Salud. Plan Nacional de Salud 2007-2012 México: Secretaría de Salud, 2009.         [ Links ]

18. Murray Ch, Evans D. Health systems performance assessment: debates, methods and empiricism. Geneva: World Health Organization, 2003.         [ Links ]

19. Diario Oficial de la Federación. Reglas de Operación del Programa Seguro Médico para una Nueva Generación para el Ejercicio Fiscal 2009, México: Gobierno Federal; Martes 30 de diciembre de 2008.         [ Links ]

20. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR 1999; 48(No. RR-11).         [ Links ]

21. Minister of Public Works and Government Services. Program evaluation methods. Measurement and attribution of program results. Third Edition. Deputy Comptroller General Branch. Canada: Treasury Board of Canada, 2002.         [ Links ]

22. Perdigón-Villaseñor G, Fernández-Cantón S. La mortalidad neonatal y postneonatal en México, 1980-2005. Bol Med Hosp Infant Méx 2008; 65: 412-414.         [ Links ]

23. Rodriguez-Ortega E, Pasillas-Torres M. Family health care expenditure with children affiliated to the New Generation Health Insurance Program. Salud Publica Mex 2012;54 supl 1: S65-S72.         [ Links ]

24. Bowling A. Research methods in health. Investigating health and health services. 2^nd. Edition. Buckingham UK: Open University Press, 2002.         [ Links ]

25. Johnson R, Onwuegbuzie A, Turner L. Toward a Definition of Mixed Methods Research. J Mixed Methods Research 2007; 1; 112-133.         [ Links ]

26. Aday LA, Begley Ch, Lairson D, Slater C. Evaluating the health care system. Effectiveness, efficiency and equity. Chicago: Health Administration Press, 1998.         [ Links ]

 

 

Corresponding author:
Dr. Ricardo Pérez-Cuevas
Coordinación de Asesores, Subsecretaría de Prevención y Promoción de la Salud, Secretaría de Salud
Lieja 9 col. Juárez. 06600 México, D.F. México
E-mail: ricardo_perezcuevas@prodigy.net.mx

Received on: February 10, 2011
Accepted on: October 11, 2011