Artículos

 

The impact of technical parameters such as video sensor technology, system configuration, marker size and speed on the accuracy of motion analysis systems

 

Carlos Diaz Novo¹, Sultan Alharbi², Megan Fox³, Elaine Ouellette⁴, Edmund Biden⁵, Maureen Tingley⁶ y Victoria Chester⁷

 

¹ Institute of Biomedical Engineering. Faculty of Mechanical Engineering. University of New Brunswick, Fredericton, NB, Canada.

² Institute of Biomedical Engineering. Faculty of Mechanical Engineering. University of New Brunswick, Fredericton, NB, Canada.

³ Faculty of Kinesiology. University of New Brunswick, Fredericton, NB, Canada.

⁴ Faculty of Mechanical Engineering. University of New Brunswick, Fredericton, NB, Canada.

⁵ Faculty of Mechanical Engineering. University of New Brunswick, Fredericton, NB, Canada.

⁶ Department of Mathematics and Statistics. University of New Brunswick, Fredericton, NB, Canada.

⁷ Andrew and Marjorie McCain Human Performance Laboratory Richard J. CURRIE CENTER¨. Faculty of Kinesiology. University of New Brunswick, Fredericton, NB, Canada.

 

Fecha de recepción: 14-03-2014
Fecha de aceptación: 25-06-2014

 

Abstract

In biomechanical assessments, minimizing errors and achieving accurate representations of human movements are the main goals of motion capture systems. Although 3D camera-based motion capturing systems are effective for accurate acquisition of motion, their performance is highly dependent on various parameters. This paper examines the how variations in independent technical parameters such as video camera sensor technology, system configuration, marker size and speed influence the accuracy of the kinematic measurements. A method was developed to systematically assess accuracy and precision of motion capture systems by measuring the mean absolute inter-marker distance errors and standard deviations respectively. A custom-made dynamic measurement protocol was used to test the performance of three motion capture systems; two different Vicon motion capture systems and a low-cost motion capture system implemented in the Santiago de Cuba Hospital (SCH) using common video cameras. For the two Vicon labs, the individual effects and interactions between the parameters and the spatial measurements of the motion analysis systems were able to be determined. However, the Santiago lab was unable to accurately track small changes in the elements of the measurement system and therefore was not recommended for small human movements. Although the Santiago lab reported to be significantly less accurate than the two other labs, results showed that the mean absolute inter-marker distance error was minimized in the center of the capture volume. This information is essential in the implementation of this system as a clinical assessment tool and is a factor that should be considered for low accuracy systems.

Keywords: precision, video, capturing systems, human movement, biomechanics, measurements accuracy, video camera, inter-marker distance.

 

Resumen

El objetivo de los sistemas de captura de movimiento es la disminución de los errores y la representación exacta de los movimientos humanos. A pesar de que la captura 3D de los movimientos basada en cámaras de video es efectiva, su desempeño es dependiente de varios parámetros. Este artículo examina como impactan en la precisión de las mediciones cinemáticas la tecnología de las cámaras y sus configuraciones, el tamaño de los marcadores y la velocidad. Se propuso un método para la evaluación de la exactitud y precisión de los sistemas de captura mediante la medición de la media de los errores de la distancia inter-marcadores absoluta y la desviación estándar. Se implementó un protocolo de medición dinámico con el cual se estudiaron tres sistemas de captura del movimiento; dos con diferentes tecnologías de la firma Vicon y un sistema de bajo costo desarrollado en el Hospital de Santiago de Cuba que emplea cámaras de video convencionales. Los efectos individuales y de interacción entre parámetros fueron obtenidos en los dos laboratorios Vicon. Por el contrario, en el laboratorio de Santiago no fue posible determinar con precisión el efecto de pequeños cambios inducidos en los parámetros de los elementos del sistema de medición por lo que no se recomienda la evaluación de pequeños rangos de movimientos humanos con este sistema. El laboratorio de Santiago es significativamente menos preciso que los otros dos de la firma Vicon, obteniéndose los menores errores inter-marcadores en la porción central del volumen de captura. Esta información es esencial en la implementación y uso de este sistema como herramienta para la asistencia clínica, factor que pudiera tenerse en cuenta en los sistemas de bajo costo.

Palabras clave: precisión, video, sistema de captura, movimientos humanos, biomecánica exactitud de mediciones, cámaras de video, distancia entre-marcadores.

 

Introduction

Video-based motion capturing is widely used for clinical applications. This type of analysis requires accurate measurements in order to provide correct and precise information for clinicians (Mündermann 2006, Harris 2004, Billington 2008, Taylor 2006). The performance of measurement systems is subject to change due to a large number of influencing factors. These factors include the adequacy and quality of the system itself, but also parameters related to the laboratory set-up and equipment (Unal 2007). Parameters such as the temporal-spatial resolution and configuration of the cameras, the nature of the illumination, marker properties such as size, shape, speed and inter-marker distance and also the care of the user in performing the calibration procedure have individual effects on the measurements (Leardini 2005, Chen 2005, Ehara 1994, Chiari 1993). Measures of accuracy vary between studies and the calculations are generally based on two methods. Classic methods consist of static or dynamic recording of a rigid bar carrying at least two markers placed at a known distance (Gorton 2009, Holden 2003, Ehara 1997, Lewis 2007, Delia 2000). This calibration bar is either translated throughout the capture volume by the researcher or manually placed at different locations on the floor. The second method consists of a combination of procedures that use robots designed to perform repeatable dynamic measurements to determine the resultant system accuracy and precision of small motion magnitudes (Windolf 2008, Piazza 2012). A wedge comparator with a resolution of 0.25 mm to provide measured marker displacements in three orthogonal have also been used to quantify marker-location accuracy in small capture volumes (Liu 2007).

Due to the variety of sensors and technologies available, most authors who perform motion capture studies do not report the details of their experimental protocol for the characterization of the marker-location accuracy. They also do not take into account the impact of the size, speed and inter-marker distance of the markers, as well as the configuration of the optoelectronic sensors in different laboratory settings. The purpose of this study was to develop and apply a method to systematically assess the accuracy and precision of motion capture systems by controlling several independently parameters in the defined workspace. The methodology combines the advantages of the motorized methods to assess the effects of the laboratory configuration, as well as marker diameter size, and speed. The accuracy of the system with respect to the location of the markers within the capture volume was also evaluated.

 

Materials and Method

Testing was performed in three different laboratories with three different motion capture systems. Two of these laboratories use VICON motion analysis technology (Institute of Biomedical Engineering and Andrew and Marjorie McCain Human Performance Laboratory (HPL)) which automatically processes the marker data. VICON is a world leader in the design of motion capture systems. The third laboratory custom-made in the Santiago de Cuba Hospital (SCH) is a low cost motion analysis system design in the hospital for kinematic gait analysis of patients that attends the neurological service.

To achieve a repeatable analysis of the motion capture systems, systematic measurements were acquired by successively positioning a cluster of markers at predefined grid points, following a random order within capture volume. For the comparison of the results of the study, a common capture volume was defined for three motion capture systems.

First, the individual capture volume sizes for each of the laboratories were determined by positioning the cameras in each laboratory at their maximum height and directing it toward the center of the laboratory. With a meter stick containing three reflective markers, live monitors were used to observe the limits of the capture volume. These limits were determined by marking the outer-most position where all three markers were visible. Based on the smallest capture volume of the three laboratories, (SCH) the dimensions of the grid to be used for all measurements were found to be (2.5 x 1.0 x 1.0 m). Within this volume, 20 sub-volumes (0.5 x 0.5 x 0.5 m) were defined for the locations of data collection.

Details about the system configurations and technologies used in each of them are described in Table 1.

To test the effects of marker size on the accuracy and precision of the systems, two marker clusters with identical configurations were created. The two sizes of retro-reflective markers used were 0.014 m and 0.025 m in diameter. Each cluster consisted of three pairs of two markers attached at the ends of three dowels of known lengths (0.1, 0.2 and 0.3 m). Each dowel was inserted through a Styrofoam ball (0.050 m diameter) at a different angle in order to maintain a constant and controlled marker configuration. This configuration allowed for a comparison between the three fixed inter-marker distances and their significance with respect to the instrumental errors.

For each marker size, the custom-made cluster was mounted on the end of a rigid arm supported by a tripod base. The arm was made to be adjustable in order to position the cluster in the center of each defined sub-volume. A DC motor was also mounted on the end of the arm to rotate the entire cluster at two different speeds (20 and 40 rpm) to investigate the impact of slow and fast marker movement. The configuration of the designed measurement apparatus along with the capture volume grid and marker cluster are shown in Figure 1 at the IBME.

The data for each trial was collected at 60 Hz for a total of ten seconds for each marker size and speed combination. This resulted in four sets of position data for each sub-volume of the predefined capture volume grid. All three motion capture systems provided X, Y, and Z position coordinates for each marker. The error for marker-location accuracy and precision was determined by calculating the difference between the estimated value from the 3D data set collected by the system and the known distance value from the marker cluster which was measured with slider callipers prior to the experiment. For each sampled instant in time, the absolute value of the error was computed. From this, the mean absolute values of the error and the standard deviations for each measurement trial were calculated and used as representations of accuracy and precision respectively for each system. Based on these results, an increase in mean error indicated a decrease in the accuracy of the system and similarly, an increase in standard deviation indicated a decrease in measurement precision.

An independent, 3-way ANOVA (analysis of variance test) was employed on the data collected to investigate differences in the accuracy and precision of the systems (dependent variables). A Tukey post-hoc test was also applied to determine significant differences in the results by grouping the information based the independent variables. The factors and levels of the independent variables and fixed parameters used for the statistical tests are shown according to Table 2.

 

Results

The effects of the independent variables were investigated based on the calculated absolute errors allowing the accuracy of the three systems to be evaluated. For the purpose of this study, the results were analyzed for each lab and also for each fixed inter-marker distance separately.

While taking into consideration the different configurations and characteristics of each motion capture system, the relationships between the size and speed of the markers and the absolute errors are shown in Figures 2 and 3 respectively. The data from both IBME and HPL labs (wherein referred to as the Vicon labs) demonstrate very similar results for both the big and small marker sizes. The absolute error was lowest for the smaller marker diameter and was generally constant for each inter-marker distance for both Vicon labs. For the larger marker diameter, the error was slightly lower for the largest inter-marker distance. On the other hand, the results for the SCH differed significantly from the pattern of errors shown for the Vicon labs. Although the smaller marker diameter presented the lowest error of the two sizes, the relationships between the inter-marker distances and the marker size did not indicate any logical trends.

Similar observations were noted for the data shown for the speed of the markers. The Vicon labs once again showed cohesive patterns and the absolute errors decreased with increasing inter-marker distance. No particular trends were demonstrated between the speed of the marker and the absolute errors for the SCH. Also, for both the size and speed parameters (Figures 2 and 3), the standard deviations were much larger for the SCH whereas the other two laboratories showed very slight deviations from the mean values. Focusing on the Vicon labs, the largest inter-marker distance presented the smallest standard deviations in both cases.

This study also analyzed the distribution between the position along the major axis of the capture volume (X-axis) and the mean absolute error for each of the three inter-marker distances. With fixed capture volume dimensions and grid resolution for each laboratory, it was possible to make a valid comparison between the three motion capture systems as shown in Figure 4.

As can be seen for the Vicon labs, the position of the marker cluster along the X-axis did not prove to be an influential factor on the mean absolute error of the inter-marker distance. The error values remain generally constant throughout the capture volume and only a slight increase in error can be seen as the inter-marker distance decreases. For the Santiago lab, although no distinct pattern was identified across the three inter-marker distances, the smallest error for each case tended to be midway along the X-axis of the capture volume. Also, the standard deviation values for each inter-marker distance were smallest at the center of the capture volume (1.25 mm) and generally largest along the borders.

For each of the three independent variables, a Tukey post-hoc test was employed in order to statistically evaluate the various levels as described in Table 2. All tests used a 95% confidence interval to determine the significance of the difference between the mean absolute errors. When comparing the three laboratory configurations, each using different sensor technologies, the Santiago lab was found to be significantly different from the two Vicon labs where the magnitude of the difference was approximately 9 mm. The differences between the mean absolute errors for the big and small marker diameters and the fast and slow marker speeds were both reported to be significant with mean differences in magnitude of 4.4 mm and 1.7 mm respectively.

 

Discussion

From the analysis of mean absolute inter-marker distance errors with respect to the laboratory configuration, the marker size and speed, along with the position within the capture volume, several observations were made. The similarities between the patterns of errors for the two Vicon labs, which both use Vicon motion capture technologies, confirmed that both of these systems are highly accurate. Although the results from the two systems were very much alike, the HPL proved to be slightly more accurate than the IBME in all cases. As the HPL used a higher number of higher resolution cameras and a more advanced software system, the small difference in accuracy was expected. The variation in error noted between the marker sizes and speeds as well as between the inter-marker distances proved that these systems were capable of tracking very small changes in the technical parameters across the trials. With this level of technology, any variation of the elements of the measurement system were described in a logical way and therefore demonstrates that both of these systems are sensitive to the different levels of these independent factors.

The low-cost motion capture system installed in the Santiago de Cuba Hospital on the other hand, did not provide results with the same tendencies as were shown in both of the Vicon labs. As previously mentioned, no logical trends in the mean absolute errors were able to be identified for the marker size or speed. Although of the two marker sizes the smallest diameter presented the lowest error, the inconsistency of the results throughout the trials prevented this finding from being reliable. The variations in the elements of the measurement system such as the size and speed of the markers were unable to be identified due to the lack of sensitivity of the system and therefore the influence of these independent variables on the overall accuracy of the system was inconclusive.

While the proposed methodology did not necessarily reveal the specific relationship between the independent parameters and the inter-marker distance errors measured with a low-resolution motion capture system (SCH), the ability to identify the influence of these parameters employing the Vicon systems was useful as a reference during the comparison of the measurement systems. Similar methodology was used by Piazza 2012. where inter-marker distance errors were measured using a rotating device containing markers at known distances. The results obtained from this study are in accordance with those obtained with the Vicon systems of this study although limited information was given about the technical characteristics of the motion capture system that they tested. Similar results were also found by Ehara 1995 for both high and low-resolution camera-based systems.

Results regarding the effect of marker size and shape on the accuracy of calculating inter-marker distances often vary from one study to another. In the cases of both Windolf 2008 and Liu 2007, it was determined that the use of larger markers increases the accuracy of inter-marker distance measurements. According to these studies, larger markers increase the number of pixels projected on the sensor which enhances the resolution and the larger area of the marker, the higher the accuracy of calculating the central point of the marker. Although these statements contradict the findings of this study, consideration should be taken on the ability of these systems to detect the location of the center of the marker. The motion of the markers often creates uneven silhouettes as mentioned by Windolf 2008, which can lead to unstable approximations of the centers by the system. Similar issues were observed during the tracking of the markers due to the combination of speed and rotation of the marker clusters. Another important consideration to note for the SCH lab is the fact that marker centers were manually approximated. Based on this, it is believed that a combination of these factors has contributed to the differences between the results of this study and previous studies focusing on marker size.

When examining the effect of the position along the major axis (X-axis), the errors for the two Vicon labs were very small and consistent throughout the capture volume for each inter-marker distance which further confirmed their accuracy. For the SCH lab, the results of this analysis provided very meaningful information. For each inter-marker distance, the smallest mean absolute error was found to be at the midway point along the X-axis. This was an indication that within the defined workspace, there was a distinct area where the results obtained were more accurate. The system in the SCH lab uses only three low-resolution cameras and therefore the configuration of the cameras is not symmetric and does not permit a homogeneous distribution of motion capture capability. The standard deviations tended to be larger at the extremities of the axis for the SCH lab which implies that more consistent and precise measurements were obtained in the center of the capture volume. On the other hand considering the number of cameras and the symmetry of the configuration of the two Vicon labs, it is logical that the accuracy is consistent along all axes of the capture volume.

 

Conclusions

The IBME and HPC laboratories, which both used Vicon motion capture technology, demonstrated very consistent and similar patterns of mean absolute inter-marker distance error for marker size and speed. These two systems were able to track small changes in the elements of the measurement system in a logical way and provide accurate results.

The SCH lab was unable to demonstrate any distinct patterns that correlated with theoretical concepts on the impact of marker size and speed on the accuracy of motion capture systems. Due to the lack of sensitivity in this technology, this could be used for kinematic evaluation of big ranges of movements (e. human gait studies) but was not recommended for tracking small human movements (e. in hands, toes fingers and also facial movements).

The results of the relationship between the mean absolute inter-marker distance errors and the position along the x-axis on the other hand, presented very useful information about this laboratory configuration. The smallest mean absolute errors and standard deviations were found to be midway along the x-axis of the capture volume. Therefore, to increase the feasibility of the analysis of human motion, studies should focus on tracking small ranges of motion in the most accurate area within the capture volume, which in this case was the center of the defined workspace.

Locating the most homogeneous spot within the capture volume where errors are minimized is an important consideration to take for low-resolution systems. As the results obtained by these systems have a significant impact on the diagnosis and treatment of patients, it is necessary to ensure that all significant factors are considered and the most effective and reliable analysis is achieved.

This study was able to show the individual effects and interactions between the parameters and the spatial measurements of three motion analysis systems, two sophisticated technologies from VICON and one low cost, custom made at the hospital.

The methodology provided powerful information on the performance of the cost effective system in Santiago de Cuba. This methodology could be generalized to the accuracy characterization of motion captures laboratories, based on reflexive markers endowed with video cameras.

 

Acknowledgments

The authors would like to thank the Government of Canada's Canadian International Development Agency (CIDA) for funding through the Students for Development (SFD) and University Partnerships in Cooperation and Development (UPCD) programs.

 

References

Billington H C, Helbostad JL, Nilssen R. "Should trunk movement or footfall parameters quantify gait asymmetry in chronic stroke patients?" Gait & Posture, 2008; 27:552-558.         [ Links ]

Cappozzo A, Delia Croce U, Catani F, Leardini A, Fioretti S, Maurizi M, et al. "Stereometric system accuracy tests: Measurement and data processing methodology in clinical movement analysis-preliminary". CAMARC II Internal Report, 1993.         [ Links ]

Chen L Armstrong CW, Raftopolus DD. "An investigation on the accuracy of the tree dimensional space reconstruction using the direct linear transformation technique". Journal of Biomechanics, 1994; 27:493-500.         [ Links ]

Chiari L, Croce U D, Leardini A, Cappozzo A; "Human movement analysis using stereophotogrammetry. Part 2": Instrumental errors. Gait and Posture, 2005; 21:197-211.         [ Links ]

Della Croce U, Cappozzo A. "A spot-check for estimating stereophotogrammetric errors". Medical and Biological Engineering Computer, 2000; 38:260-6        [ Links ]

Ehara Y, Fujimoto H, Miyazaky S, Tanaka S, Yamamoto S. "Comparison of the performance of 3-D camera systems". Gait Posture, 1995; 3:166-9.         [ Links ]

Gorton G. E. III, Hebert D. A, Gannotti M E." Assessment of the kinematic variability among 12 motion analysis laboratories". Gait & Posture, 2009; 29:398-402.         [ Links ]

Harris M. O., Siegel KL, Scott M. P, Benson K. "Rehabilitation management of Friedreich Ataxia: lower extremity force-control variability and gait performance". Neurorehabilitation and Neural Repair, 2004; 18:2.         [ Links ]

Holden JP, Selbie S, Stanhope SJ."A proposed test to support the clinical movement analysis laboratory accreditation process". Gait and Posture, 2003; 17:205-13.         [ Links ]

Leardini A, Chiari L, Dela Croce U, Cappozzo A, "Human movement analysis using stereophotogrammetry Part 3". Soft tissue artifact assessment and compensation. Gait and Posture, 2005;21:212-225.         [ Links ]

Lewis A, Stewart C, Postans N, Trevelyan J." Development of an instrumented pole test for use as a gait laboratory quality check". Gait and Posture 2007; 26:317-322        [ Links ]

Liu H, Holt C, Evans S. "Accuracy and repeatability of an optical motion analysis system for measuring small deformations of biological tissues". Journal of Biomechanics, 2007; 40:210-214        [ Links ]

Mündermann L, Corazza S, Andriacchi T P. "The evolution of methods for the capture of human movement leading to marker less motion capture for biomechanical applications". Journal of NeuroEngineering and Rehabilitation, 2006; 1:3-6.         [ Links ]

Piazza S. J, Chou Li-Shan, Denniston N. L, McMulkin M. L, Quigley E. J, Richard J. G, Schwartz M. H. "A proposed standard for assessing the marker-location accuracy of video-based motion analysis systems". Retrieved March 2012, from the Gait and Clinical Movement Analysis Society: http://www.gcmas.org/sites/default/files/samsa_abstract_0.pdf        [ Links ]

Taylor D, Stretton CM, Mudge S, Garrett N. "Does clinic-measured gait speed differ from gait speed measured in the community in people with stroke?" Clinical Rehabilitation, 2006; 20:438-44.         [ Links ]

Unal G, Yezzi A, Soatto S, Slabaugh G. "A variational approach to problems in calibration of multiple cameras". IEEE Trans Patt Anal Mach Intell, 2007; 29:1322-38.         [ Links ]

Windolf M, Gotzen N, Morlock M. "Systematic accuracy and precision analysis of video motion capturing systems— exemplified on the Vicon-460 system". Journal of Biomechanics, 2008; 41:2776- 2780        [ Links ]