2.1 Pulsed thermography imaging acquisition system

For the thermal analysis of the samples, a testing bench of pulsed thermography was built, following the scheme shown in Figure 1, which is an embedded system (Raspberry Pi 3 Model B, Raspberry Pi Foundation) with Python 3 installed as a development language. The images were acquired by a radiometric thermal module without a shutter (FLIR Lepton 2.5, FLIR Systems, Wilsonville, OR, US) mounted on a 10 cm long by 10 cm high and 10 cm deep three-dimensional printed frame (Figure 2).

Figure 1 General configuration of experiments in active thermography: a) heat source, (b) specimen, (c) infrared camera, and (d) embedded system for displaying, recording and processing data. 

Figure 2 Testing bench design overview: (a) testing bench side views and isometric views and (b) part assembly. 

The FLIR Lepton module has a resolution of 80 x 60 pixels, a spectral range of 8-14 μm and a thermal sensitivity of 0.050 °C. A servomotor with a black paper attached is activated when starting the experiment to obtain a reference image in the Lepton module. Four 7W (46823/FI-50T, Voltech, Mexico) crown-like incandescent spotlights mounted on the top lid of the frame provide thermal stimulation. A piece of glass covered with polyethylene foam was chosen as the background for the scene. The polyethylene foam is a thermal insulator, which reduces its heating because of the reflector and maximizes the contrast between the sample and the background. A solid-state relay module (G3MB, Omron, Kyoto, Japan) was used to activate and deactivate the heat source.

In general terms, the system works as follows: The Lepton module is activated, and a black image is acquired as a reference image. The image acquisition is then started, and the servo motor is activated, exposing the module lens while an image of the specimen is captured before stimulation (Figure 3 (a)). The relay is then activated, which turns on the heat source for 11 s. to know the heating time, tests were done at different times and found that the contrast background begins to gain heat in t > 11 s. It is necessary to ensure that the background does not heat up to not introduce errors in the measurements. In this practical way, we determine that 11 seconds is the maximum warm-up time to avoid this phenomenon. Besides, the relay is then deactivated, and the thermal stimulation is completed (Figure 3 (b)). The process finishes 33 s after the heat pulse ends by activating the servomotor, covering the lens, deactivating the Lepton module, and exporting approximately 44 s of video (Figure 3 (c)).

Figure 3 General scheme of active thermography where as can be seen (a) thermal images of the specimen prior to the heat pulse, (b) the heat pulse and (c) the surface response of the product to thermal stimulation. 

A software application was developed in Python language to automatically operate the system, which consists of the following: activation and deactivation of the reflector, activation and deactivation of the servomotor, thermal camera operation and memory management for storing video.

2.1.1 Sample preparation

This study can be done using several dehydrated agricultural products such as thin layers of fruits, thin layers of vegetables, or leaves. However, Neem (Azadirachta indica) leaves were selected because of their availability in the region and their use in traditional medicine to treat skin infections. Another use of Neem is vector control and its potential disease transmission (^{El et al., 2003}). Sixty leaves were collected from the facilities of Tecnológico Nacional de México / I T Tuxtla Gutiérrez in Tuxtla Gutiérrez, Chiapas, in September 2019. The samples were stored for two hours in sealed bags at a constant temperature of 3 °C ± 0.10 °C to stabilize their temperature and humidity. The leaves were classified according to a size of approximately 7 cm ± 0.3 cm in length, and the damaged, withered, or defective leaves were discarded. The selected samples were of similar sizes and had no visible defects, stains, or pests.

For the study, 48 samples were retained. Samples were processed approximately three hours after collection. Two treatment groups were created: the first with 44 samples and the second with four samples as a control group. The first group was dehydrated at 50 °C for 150 min; four samples were taken randomly from every 15 min and evaluated on the testing bench. Finally, the four samples were discarded, and the procedure continues.

2.1.2 Moisture content determination of Neem leaves

To know the moisture of the samples, the leaves were evenly distributed in a tray of a commercial dryer (Hamilton Beach 32100 Food Dehydrator, Hamilton Beach, US) at 50 °C of temperature. A digital scale was used with ± 0.01 g precision (^{Mwithiga & Olwal, 2005}) to measure the weight of the samples. The samples were dried until the weight readings were constant according to the methodology proposed by (^{A. O. A. C., 1995}). The readings were taken in three replicates, and the average values were used for further analysis.

Usually, the drying curves are expressed by the moisture content (in grams) X against time t. It is obtained directly from the weight loss and time during drying. Another way to construct the curve is using the moisture ratio (MR) against time t. The MR can be calculated with Eq. 2 (^{Grumezescu & Holban, 2018}):

where X₀ is the initial moisture content (in grams) and X_eq is the moisture content in balance (in grams). If X_eq is very small compared with X₀, Eq. 2 is simplified as Eq.3.

2.2. Processing of thermal images

The processing of the information was carried out in four stages: a) image capture (Figure 4(a)), b) extraction of the region of interest (Figure 4 (b)), c) plotting of the thermal kinetics curve and curve feature extraction (Figure 4(c)), and d) classification (Figure 4(d)). Each of these steps are detailed below.

Figure 4 General overview of information processing. 

2.2.1. Imaging capture

The duration of the obtained videos was 44 s. One second after calibrating the module with the black image and starting the recording, the servomotor is activated by uncovering the lens and activating the heat source. At 12 s, the heat source is deactivated, resulting in a total heat pulse duration of 11 s. After 43 s of starting the recording, the servomotor is activated again to cover the lens, giving 31 s of a cooling curve. The recording is finished with an additional second of image capture to avoid unintentional truncation of information. The eight first samples information captured was intertwined: First, a video was captured from the control sample (fresh leaves), followed by the sample from the dry leaves group. The rest of the information capture was taken from the samples of the group of dry leaves. One video was obtained per sample, producing 48 videos in total where each video was captured at 20 FPS, yielding approximately 42,000 images to process.

2.2.2. Extraction of the region of interest

The extraction of the region of interest aims to highlight the relevant areas in the scene. The image segmentation favors the algorithm's simplicity, the speed of execution, the computing charge, and the correct segmentation to avoid modifying the FLIR module's original information. Finally, the results obtained with the aid of segmentation minimize the noise presented in the region of interest instead of not segmenting the image.

The segmentation process was performed using the scikit image library for Python. A three-dimensional matrix T(x, y, z) ∈ R ^mxnxp was obtained from the sequence of captured images. All the pixels of each image were averaged (Eq.4), and the maximum temperature value index (Eq.5) was sought:

The image that matches the peak temperature 𝑃(𝑥, 𝑦, 𝑡_𝑚𝑎𝑥) was segmented by thresholding, given the transformation function (Eq.6) where the transition level given by the parameter 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 = 10 In the inspection of 48 samples, we found that, on average, the contour of the leaves had a mean value of 10 on a grayscale, which allowed us to delimit the edges for the binarization and information extraction process. This value must be calibrated for each different product.

The number of white pixels (WP) was counted to know the total of pixels in the region of interest with Eq. 7.

The segmented image (q(x,y)) was multiplied by each frame in the video to extract only the pixels of the leaves according to Eq.8. All the pixels of the object are averaged together with Eq.9, which produces an average value for each frame. In addition, the temperature variation (Eq.10) was plotted with respect to time. The set of values that were obtained from Eq. 10 were stored in CSV formatted files for analysis.

2.2.3. Thermal curve profile, amplitude, and phase extraction

Pulsed phase thermography (PPT) is an analytical technique for analyzing the thermal response from the object to the heat pulse. In this technique, the grayscale intensity change of each pixel of the object was considered as a temporary thermal signal. The series of input signals is known as the thermal profile curve. The Fourier discrete-time transformation (DFT) decomposes the input signal given by the thermal profile curve into a sum of sinusoidal components, each having a different frequency, amplitude, and phase delay. The infinite integral of exponential functions expresses the continuous Fourier transform (^{Baranowski et al., 2009}) as shows in Eq. 11:

where 𝑗² = −1. When a finite series of signal samples (𝑇₀, 𝑇₁, 𝑇₂, …, 𝑇_𝑁−1) is analyzed, it can be transformed into a fundamental frequency (F₀) and harmonic series (𝐹₁, 𝐹₂, 𝐹₃, …, 𝐹_𝑁−1) (^{Brown & Puckette, 1993}) by using the Eq. 12:

where Re and Im are the real and imaginary components of the transform, respectively, j is an imaginary unit, n is the number of harmonic components (n = 0,1, …, N), k is the signal sample's value. The real and imaginary part of the DFT can be used to calculate the amplitude with Eq. 13 and with Eq. 14 the phase delay of the thermal profile curve. The phase delay that corresponds to the fundamental frequency is of interest among researchers as characteristics of the samples' discrimination (^{Brown & Puckette, 1993}; ^{Kamarainen et al., 2002}; ^{Kuzy & Li, 2017}).

Fourier's analysis of the samples was done using the Python NumPy library. The Fourier's analysis input signal was thermal profile waves shown in Figure 6. Followed by the Fourier decomposition, several complex components equal to the number of frames in each video were generated. The Hermite function's symmetry properties are reflected in the amplitude and phase transformation, which are even and odd, respectively, concerning ƒ = 0 Hz.

Therefore, for a sequence of 𝑁 thermograms, there are N/2 proper frequencies (10 Hz); the other half of the spectrum provides redundant information that can be dismissed. The fundamental frequency phase is interesting since it is less affected by the typical problems of active thermography such as the noise present in the measurements, reflections from the environment, variations in emissivity, and non-uniform heating, among others (^{Ibarra-Castanedo et al., 2014}).

2.2.4. Feature selection and estimation of MR

In previous studies, (^{Kuzy & Li, 2017}) determined that the set of characteristics in the frequency domain is suitable for discriminating between materials. For this study, the phase of the fundamental frequency obtained from Eq. 14 was selected to estimate leaf moisture during the dehydration process. Moreover, when the amplitude and phase information was analyzed, it was found that the amplitude does not provide enough information to discriminate degrees of dehydration; that is why it was considered to take only the phase information. The estimation was made using a curve adjusted to the values given by the phase of the fundamental frequency and the moisture percentage given in Figure 5.

Figure 5 Neem’s drying curve at 50°C. 

3.1. Drying of leaves

The weight difference of the leaves as a function of time throughout the drying process was monitored. Figure 5 shows the drying curve for Neem leaves at 50 °C. The initial drying speed was high because of the product's high moisture content and the high temperature inside the dryer. The drying speed decreased continuously in proportion to the decrease in the humidity of the leaves. Drying time for the constant temperature of 50 °C was 165 min When compared to previous studies on fruits, vegetables and leaves, this curve is in good agreement (^{Ali et al., 2014}).

Figure 5 shows the moisture content of the product concerning the drying time, at constant temperature and constant airflow. The curve fit was made with a second-order polynomial regression, giving the equation %MR = 0.0033t² − 1.1114t + 96 with R² of 0.97.

3.2. Analysis of the thermal profile curve

Concerning the curves of Figure 6 during the stage where the drying speed decreases, most of the curves of thermal kinetics are found. There is a difference from one another, and in the phase of high drying speed a wider difference is shown between curves of thermal kinetics.

As can be seen, Figure 6 shows the response of the Neem leaves to thermal stimulation. Each dehydration degree's thermal profile wave shown in Figure 6 presents a difference between the curves at different dehydration levels of the product. For example, the peak gray value of the sample MR 6%, with a mean value of 14.8 ≈ 15 in grayscale, was higher than MR 48% that had a maximum average value of 10.07 ≈ 10 in grayscale. It can be observed that the samples between MR 48 % and MR 21% reach similar maximum levels; however, the heating and cooling zone have different gradients.

From the set of curves of Figure 6, it is observed that at the beginning of the thermal kinetic curves from t₀ = 0 s to t₁ ≈ 1.4 s, there is an abrupt rise in intensity. This directly corresponds to the reference image's capture and to the activation of the servomotor to discover the lens of the FLIR module.

Figure 6 Thermal profile waves of all study samples. 

Similarly, between the time t₂ ≈ 42 s and t₃ = 44.05 s, a rapid drop is observed in the average gray value within the region of interest, and this drop corresponds to the shutter closure of the FLIR module.

3.3. Leaves moisture estimation

The fundamental frequency phase shifts between the thermal kinetic curves are evidenced by the time variation between the maximum gray values within the region of interest. The samples that reach high temperatures the fastest are the most dehydrated. The phase change is represented by a variation in the time axis of the thermal kinetic curve.

As was previously mentioned, the phase of the fundamental frequency was selected as an estimation parameter, and the drying curve (Figure 5) was related to each percentage of dehydration corresponding to a value of the phase of the fundamental frequency of each test. This relation was plotted, and nonlinear regressions were calculated to find the one that best fits the data. The equation that best fits the data is the one proposed in this study, Eq.15, had a R-square of 0.8937 and RMSE of 8.8313, where %MR is the estimated moisture percentage and Φ_n is the phase calculated by Eq. 14. The data and the fit curve are shown in Figure 7.

Figure 7 Drying estimation curve. 

To validate the estimation equation, 36 Neem leaves were dried at 50 °C, and groups of three leaves were removed every 15 min and submitted to the estimation system. The precision in the estimation given by Eq. 15 was 85.3%.

With the algorithm implemented in the proposed device, the average processing time of the sequence of images (approximately 880 images) was 0.8812 s. On the other hand, the average time of capturing images was 44 s, giving an overall average of the estimation process of approximately 45 s. Furthermore, because of the time of capture and image processing is very short compared to the time during the drying process, the method presented in this work could be used in a processing line for dehydrated products.

The methodology developed in this work could be applied to other agricultural products. Thermal curves will be obtained that relates the variation of the moisture of the product during the drying process and the response in phase according to each product.