Introduction

The cognitive process known as attention is made possible by selecting, zooming, and maintaining the processing of certain pieces of information ^[1]. Whatever is now being processed by sensorial or information systems or reaction possibilities generated by ongoing cognitive activity might serve as objects of attention. It helps us to focus on the information that is relevant to us and discriminate against the rest. In addition, it is essential to have it before beginning the learning process, as it serves as its foundation. There have been a variety of methods of measurement, including recording response times or clicks made on specific software, measuring eye contact time from movies, doing Magnetic resonance imaging (MRI) or Functional magnetic resonance imaging (fMRI) research, and so on.

Attention is fundamental for university students, so measuring it is essential. Some research works are related, like ^{[2] [3]}, where primary school students are observed doing math and language tests using electroencephalogram (EEG). Figure 1 shows the framework followed for the preprocessing and classification stages in ^{[2] [3]}. They use 64-channel EEG data and process only the signal from frontal (6, 12, 60) and parietal (28, 34, 42) electrodes. They use the average Welch's Power Spectrum Density (PSD) for the 21-45 Hz band and the K-Nearest Neighbor as a feature and classifier for these EEG channels. The preprocessing for the EEG signals was performed with the Python MNE library, and the PSD was obtained with the SciPy library. Five-and-a-half percent did poorly on the arithmetic test, while seventy-four percent did poorly on the language test. Based on the results of the tests, the average sensitivity and specificity for each fold were satisfactory. Similar sensitivity was found when EEG data was combined with socio-demographic and home environment characteristics. The results obtained were: Math = 58.7 %, Language = 66.3 %, but more specificity was found Math = 43.4 % to 50.6 %, Language = 32 % to 60 %.

Figure 1 Framework for the preprocessing and classification stages followed by researchers in ^{[2] [3]}. Taken from ^{[2] [3]}. 

In ^[4], the authors examine the effects of exercise on the math test scores and anxiety levels of 68 sixth-graders from two primary schools in New South Wales, Australia. In this study, trait anxiety (low vs. high) and condition (activity break vs. control) are the between-subjects factors in a 22-between-subjects design. The dependent variables were math exam scores, mental effort expended, task perceived difficulty, and a three-time state anxiety measure. IBM SPSS Statistics 25 was used to do an ANOVA and ANCOVA on the collected data. As a result, authors found that physical activity break before a test examination does not deteriorate test anxiety and math test performance.

In ^[5], the authors consider the eye movements of 30 participants' suggesting that they unconsciously focus on the numbers (operands, solution) they are now processing. Faster performance before fixing on the relevant numbers and making fewer trips back to the first operand in the computation resulted in shorter latencies. These telltale signs of superior task performance were most obvious for addition and visually ordered numbers and for subtraction and visually ordered numbers in the opposite direction. In this case, the eye movements were done with EyeLink 1000, a video-based eye tracker, and the acquired information was processed with SR Research Data Viewer software. 2 + 5, 5 + 2, 3 + 5, 5 + 3, 2 + 6, 6 + 2, 3 +6, 6 + 3, 3 +8, 8 + 3, 4 + 8, 8 + 4, 3 + 9, 9 + 3, and 9 + 4 were the addition problems utilized in this study. Each addition problem was transformed into a corresponding subtraction problem with the answer to the addition problem as the first operand (7 + 5 = 7 became 7 - 5 = 2), and so on. Four white numbers, set against a black background, were displayed horizontally in ascending or descending numerical sequence (see Figure 2). The numerals were 18 points tall, had a visual angle of 0.5 degrees, and were set in Times New Roman font type. On an invisible 44 grid (shown as a dotted line in Figure 2), each number may take one of sixteen possible placements, with a single digit occupying each column and row. Two numerical images corresponded to the orally provided operands of the arithmetic problem; another was a distractor operand, and the final image was the answer (correct or incorrect). Eye tracking data showed that people naturally gaze at the numbers they are currently processing (operands, solution). Performance improvements were seen in shorter latency before fixating the relevant integers and fewer returns to the first operand during solution computation. In particular, these hallmarks of high-quality task performance were more prominent for addition and visually-arranged numbers in ascending order and for subtraction and visually-arranged numbers in descending order (compared to the opposite pairings). Our findings demonstrate that the "visual number world"-paradigm provides real-time insight into mental arithmetic, is sensitive to visual layout modifications that are not reflected in response time measures and can capture variability in arithmetic performance.

Figure 2 An example of a trial sequence is used in the research, taken from ^[5]. 

Authors found that comparing the proportions of fixating the first operand between small-and large-operand-first problems in the computational phase (paired test) revealed a higher proportion for large-operand-first problems, t(28) = −2.256, p = .033. The fixation proportion of the first operand was 7.2% increased for these problems (SEM = 1.6). Thus, eye movements do not reflect a possible mental rearrangement of operands from a small-large into a large-small order but, rather, the activation of the larger number to which the smaller number is added.

Research ^[6] presents a proposed methodology that evaluates "Attention" and "Meditation" levels in children in virtual classes with a Mindwave BCI, a brainwave-reading EEG headset. They obtain a level of "Attention" and "Meditation" from 0 to 100 % using the software provided with the BCI. Their methodology was evaluated in two situations: with a 6-year-old child and a 15-year-old during virtual classes. This one-channel BCI uses the FP1 electrode, and the software measures the Attention and Meditation levels during a session and displays their average, as is shown in Figure 3. At the beginning of the class, the 6-year-old child's Attention average was 10 % and a Meditation average of 40 %. When the child had to solve an exercise and turn it in, "Attention" rose to 72 %, and "Meditation" a 50 %. For the 15-year-old student, in the beginning, the "Attention" average was 0 %, and the "Meditation" average of almost 100 %. His "Attention" average in Math class was 60 %, and the "Meditation" average was nearly 40 %. While he was analyzing the math exercises, the "Attention" average was 90 %, and the "Meditation" average was 15 %. They conclude that the two students' attention levels are different when performing similar tasks, with the 6-year-old student showing a maximum attention level of 72 % reaching 100 % due to the student at that age being very distracted, and the 15-year-old student showing a maximum attention level of 100 % when performing class activities. They concluded that taking a break to exercise before an exam did not lower performance on arithmetic tests or increase test-related anxiety.

Figure 3 Attention and Meditation averages displayed by the Mindwave BCI software. The illustrative Figure is taken from: https://apkpure.com/brainwave-visualizer/com.neurosky.unitythinkgear. 

Also, Machine-Learning is used to diagnose and study different neurodevelopmental and neurodegenerative disorders. In ^[7], they use the retina's electrical response (electroretinogram, ERG) to light for autism spectrum disorder (ASD) detection. They collected and analyzed ERG signals from 47 control and 96 ASD individuals. As features, they use four well-known time-domain indices, specifically the amplitude of a "Va" wave, the timing of its "Ta" peak, the amplitude of a "Vb" wave, and the timing of its "Tb" peak. These time domain and statistical features are used for this research. Then they obtain the Power Spectrum Density (PSD), and the Discrete Wavelet Transform of the ERG signals is used with the Least Absolute Shrinkage and Selection Operator (LASSO) regression technique for feature extraction. The machine learning models used included a Decision Tree (DT), a Support Vector Machine (SVM), a Gradient Boosting (GB), and Random Forest (RF). They detected ASD using Random Forest, which obtained the best classification accuracy of 86 % and 98 % sensitivity with time domain and spectral features.

In reference ^[8], authors suggest using deep convolutional neural networks (CNNs), a sparse coding-based feature mapping methodology, and the DouglasPeucker (DP) algorithm to spot ASD. This research used the King Abdulaziz University Hospital, Jeddah, Saudi Arabia dataset. This dataset contains 20 children with ASD (ages 6-20) and a healthy group with nine children without neurological conditions. Figure 4 shows the block diagram of the proposed method. They employ a g.tec EEG cap with Ag/AgCl electrodes, G.tech USB amplifiers, and BCI2000 software to acquire signals without artifacts. Using the worldwide 10-20 system with AFz as GND and the right ear lobe, they work with 16 channels (FP1, FP2, F7, F3, Fz, F4, F8, T3, C4, Cz, C3, T5, Pz, O1, Oz, and O2). A popular method for simplifying lines, the Douglas-Peucker (DP) algorithm can reduce curve complexity and storage requirements by eliminating unimportant nodes and isolating key nodes The goal of the DP method is to create a new data series with fewer and more significant points while keeping the original data series within an acceptable range of variance. Using the DP algorithm, they reduce from 47088 original samples to only 2462 with a compression parameter of 20. Next, EEG rhythms are extracted with a Wavelet Transform using Daubechies 4th order decomposition to obtain Gamma, Beta, Alpha, Theta, and Delta band power and coded with sparse representation. All rhythms are neatly lined up, and the sparse presentation is encoded with the help of a histogram. Histograms of the sparsely coded rhythms for each EEG channel are added to form a matrix representation of the input EEG information. Before applying the dB power scale (20log()), the matrix data must be normalized into the [0,1] interval. Scaled color representations of the matrix are generated using the imagesc command in Matlab and then saved as images. Autoencoders (AE) based on extreme learning machines (ELM) are used during data augmentation. Next, pre-trained deep CNN models are used to classify the EEG signals from people with ASD and those without the disorder. When applied to the automatic diagnosis of ASD, the proposed method achieved a perfect 100 % sensitivity, 96.4 % specificity, and a perfect F1-score of 99.19 %.

Figure 4 Block diagram of the proposed method. Taken from ^[8]. 

Deep-Learning models are also applied to EEG signals to detect schizophrenia, as is presented in reference ^[9]. In this instance, they utilized information gathered from Warsaw, Poland's Institute of Psychiatry and Neurology. Models from the Machine Learning (M-L) family employed include the Naive Bayes, Support Vector Machine, K-Nearest Neighbors, Decision Tree, Extremely Randomized Trees, Random Forest, and Bagging. Long Short-Term Memories (LSTMs), OneDimensional Convolutional Networks (1D-CNNs), and One-Dimensional Convolutional Networks-LSTMs are just a few examples of the Deep-Learning (DL) models they employ. The initial step in processing is slicing the EEG signals into 25-second chunks. Following this, both DL and FCF are applied concurrently in the feature extraction phase. Using a CNN-LSTM network, the DL model employs functional connectivity methods, including synchronization likelihood (SL), Fuzzy SL (FSL), and simplified interval type-2 FSL (SIT2FLS). Here, they use a concatenate layer to merge the DL features with those of each functional connectivity type before passing the resulting dataset over to a sigmoid activation layer for classification. K-Fold with K = 5 was utilized in the categorization stage to evaluate the outcomes. As a result, they found that the best performance was for the CNN-LSTM model, with an accuracy percentage of 99.25 %.

Also, reference ^[10] presents a method for identifying overconfidence patterns by analyzing EEG power spectrum bands. Students solve mathematical tasks and receive feedback about their mistakes in the solved exercises. Twenty healthy engineering students (13 males and seven females, with a mean age of 18.73 ± 0.65 years) were monitored by EEG as they performed mathematical calculations. Before, during, and after problem-solving, the subjects' Delta and Theta band activity was evaluated. The graded work included ten multiple-choice exercises on topics including algebraic fraction simplification, factoring, and the usage of radicals, typically covered in the intermediate years of high school. Each exercise has one correct answer and four incorrect ones, as shown in Figure 5. If they answer correctly, the next exercise is presented. Still, if an incorrect answer is selected, feedback is given to the student, including the probable cause of the error and the correct procedure. Next, a second similar exercise is presented, again with one correct answer and three wrong answers.

Figure 5 The mathematical task presented. Taken from ^[10]. 

The authors used an electroencephalograph (EEG) with a 200 Hz sampling rate and 19 electrodes positioned according to the 10-20 International System. This means electrodes Fp1/2, F3/4, F7/8, C3/4, P3/4, T3/4, T5/6, O1/2, Fz, Cz, and Pz are used for EEG data acquisition.

Authors analyzed signals from four discrete times within a single second: (a) just before the math problem was shown (V1), (b) immediately after it was shown (V2), (c) just before the answer was selected (V3), and (d) just as feedback appears in response to the answer chosen (V4). These moments of interest are depicted in Figure 6.

Figure 6 Example of a mathematical exercise and the structure of the one-second windows for analysis. Taken from ^[10]. 

The relative band energy is obtained by estimating the relative energy of each band, and they estimate the average PSD of each window using equation (1).

Where EBr represents the relative energy of the interesting band, EB is the absolute energy of the interesting band, and Eδ, Eθ, Eα, and Eβ the absolute energies of bands Delta, Theta, Alpha, and Beta, respectively.

Those who got the answers right saw an uptick in Delta band activity when the correct solutions were shown, whereas those who got them wrong saw a decrease. Subjects who got an exercise wrong were given feedback and given a second chance to get it right. Subjects' Theta energy levels increased when they answered correctly and decreased when they answered incorrectly. The authors of this study did not employ any Machine-Learning techniques and found that overconfidence may be quantified by measuring the fluctuations in subject energy during mathematical task errors.

In ^[11], our previous work uses EEG signals and Machine-Learning Algorithms for attention measurement in an ASD user. For EEG signal acquisition, we used an Epoc+ headset manufactured by Emotiv to acquire EEG signals from a 13 years old boy with ASD diagnosed while he performed learning activities. As features, we use the PSD with two seconds windows. Theta, which goes from 4 to 8 Hz, Alpha, from 8 to 12 Hz; and Beta, from 12 to 30 Hz, are the frequency bands that comprise the Power Spectrum Density. Theta Relative Power, Alpha Relative Power, and Beta Relative Power are all attained and used with the capabilities of this band. Theta-Beta, Theta-Alpha, and Theta/(Alpha + Beta) Ratios are all obtained using the relative powers presented in ^[12]. These features are obtained for the F3, F4, P7, and P8 channels of the Epoc+ headset. Naive Bayes (NB), Decision trees (DT), k-nearest neighbors (KNN), Support Vector Machine (SVM)-RBF, Stochastic Gradient Descent (SGD), Random Forest (RF), Extra trees (ET), and Multi-Layer Perceptron Neural Network (MLP-NN) from the Scikit- learn library (https://scikit-learn.org/stable/) were among the Machine-Learning models analyzed. An AUC of 0.9299 indicates that the best model is the multi-layer perceptron neural network (MLP-NN). Table 1, compares the state of the art presented in the Introduction section.

Listed below are the sections of this document. The second section, Materials and Methods, presents the proposed approach. Here we present the Activity Sheet, the BCI, the Data Acquisition Process, Signal Processing, Feature Extraction, and the Dataset obtained. Results and Discussion, the third section, summarizes this paper's findings and where the debate takes place. Our findings are summarized in the Conclusions section.

Materials and methods

This study explains how to measure and process the brain's electrical activity and assess attention levels when engaging in cognitive activities and interacting with various software systems or applications. EEG signals from an Epoc+ Brain-Computer Interface (BCI) can be used in this research for detecting when a user has high attention levels while solving mathematical problems, as in a class. The user's "Attention" and "No Attention" states are classified using ML techniques in this paper. This study uses EEG readings and machine learning algorithms to classify the attention of a ten-engineering student sample with an average age of 22.4 years and a standard deviation of 2.2 years, six males and four females. People with Asperger's Syndrome can be diagnosed based on their ability to focus on tasks and interact with computer programs, according to a study published in the Journal of Autism and Developmental Disorders.

This project was registered as POSG/020-1-04 with the University of Baja California's Ethics Committee and Research for Undergraduates and Graduates on October 8, 2020. Data was collected using an Epoc+ BCI connected to the Emotiv Pro platform while every engineering student in the sample solved some math exercises for about 15 to 18 minutes. Then, a similar time EEG register of no attention activities like checking cell phone, playing a videogame, and talking to the staff, among other activities. Matlab 2019a and Emotiv Pro, Student Edition were used for signal processing of the recorded EEG data. Figure 7 shows the Emotiv Epoc+ headset (right) and the electrode placement (left).

Figure 7 Emotiv Inc.'s Epoc+ headset's electrode placement (left side) (right side), retrieved on December 29, 2021, from the firm's site at https://emotiv.gitbook.io/epoc-usermanual/. 

The following is a description of the planned data collection procedure. First, we put the headset on the subject and wet the electrodes. The video recording and EEG data collection can begin at this point. Next, we give the worksheet with the instructions to the test subject. Then, we allow the test participant to begin the exercise, like in a typical math school session. The attention sampling process ends when the subject finishes the exercises or fifteen to eighteen minutes have elapsed.

The Epoc+ BCI uses a 50 Hz/60 Hz dual notch filter and a 64 Hz low-pass filter for data acquisition. Then, the signal was downsampled to 128 Hz before transmission, and next, multiply the signal by 0.51x10^-6 to convert it into a voltage reading.

A new sampling process begins at the same time as the attention recording, but now non-academic activities, like chattering with the staff, checking cell phones, or playing a video game.

Activity sheet

Appendix A shows the activity sheet used for this experiment. It has three scalar vector problems, four algebraic multiplications, three algebraic simplification problems, two synthetic division problems, and five fraction integrals. The idea of these problems is to measure the attention for 15 to 18 minutes while the students solve these problems, whether they were correctly solved. Figure 8 shows how the Activity Sheet looks.

Figure 8 The activity sheets are used for measuring attention. Source: self-made. 

Data acquisition process

The data acquisition process for the math-solving stage was:

1. First, the individual will be fitted with the headgear containing the hydrated electrodes.

2. Initiate EEG data collection.

3. Give the test subject the worksheet and instructions.

4. Allow the test subject to begin solving math exercises for about 15 to 20 minutes.

5. Stop the data acquisition before 20 minutes of testing.

Figure 9 shows the headset placement, the data acquisition starts, the solving math exercises stage, and the EEG signals acquired. Next, for the non-attention stage, we repeat the same process for the math-solving stages. Still, in step 3, we let the user do any other activity, like talking to the staff, playing video games, checking social networks, and at the same time as the math-solving stage. This EEG data acquisition process was presented in reference ^[11].

Figure 9 EEG acquisition process with the Epoc+ BCI. Source: self-taken. 

Signal processing

The signal's processing procedure block diagram is shown in Figure 10 and is presented with more details in reference ^[11]. It begins with preprocessing and then calculating the power density of EEG data to split them into bands.

Figure 10 The method proposed a block diagram. Source: self-made. 

After the signal is preprocessed, the band power is separated, and features are extracted. After that comes the Attention Quantification result, followed by Validation Process for the Features and the ML Training stage.

Features listed in Section 2.3 will be obtained and verified next and used to teach algorithms for ML. We will go into greater detail about these actions in the following section.

To determine the PSD in absolute values, in V2/Hz, the Emotiv Pro Software estimates them using two-second windows. This 256-sample window spans two seconds ^[17]. Figure 11 illustrates power bands used in EEG. Delta is 1-4 Hz, Theta is 4-8 Hz, Alpha is 8-12 Hz, Beta is 12-30 Hz, and Ram (also known as Gamma) is 30-50 Hz.

Figure 11 Example of band power separation used in EEG signal processing. 

Results and discussion

After training ML models on the datasets, we evaluate using the parameters listed in Table 4. True Positive values range from 7027 for Naive Bayes to 14968 for Random Forest. True Negative values range from a high of 17219 in Naive Bayes to a low of 12682 in Decision Trees. Again, Naive Bayes had the highest False Positives value at 13818, while Random Forest had the lowest at 5877. False Negatives ranged from a high of 7797 for Decision Trees to a low of 3260 for Naive Bayes. Accuracy was maximized by Random Forest (0.7392) and minimized by Naive Bayes (0.088). Random Forest achieved an F1 Score of 0.7430, while SGD managed only 0.6215 %. Random Forest had the highest Specificity/Accuracy (0.7261), while Naive Bayes (0.5547) had the lowest. Naive Bayes had the greatest Sensitivity/Recall score of 0.8408, while Decision Trees had the lowest at 0.6192.

Table 5, shows the results of the parameters of the evaluated ML algorithms. Table 5, also included in, presents the performance metrics obtained with the ML algorithms evaluated. Random Forest obtains the highest AUC value with 0.7394, and Naïve Bayes has the lowest with 0.5889. The highest Cohen's Kappa coefficient was for Random Forest with 0.4787, while the lowest was 0.1771 for Naïve Bayes. Naïve Bayes had the highest Hamming Loss with 0.41327, and Random Forest had the lowest with 0.2607. Random Forest had Matthew's Correlation Coefficient of 0.4792, while Naive Bayes and Support Vector Machine had values as low as 0.2057.

Table 5 shows how the Area Under the Curve (AUC) of the ML models used in this paper for Attention Classification behaves. It is noticed that Random Forest presents the best performance among the evaluated models with a 0.7394 value. The second best AUC was for Extra Trees, with 0.7335, followed by K-NN, with 0.6917. The worst performance was for Naïve Bayes, with a 0.5889 value. Comparing the results obtained from this research with those obtained in ^[11], we observe that the ML model performance is lower due to the use of several individuals in the dataset elaboration. Using many test individuals for dataset conformation reduces the performance of the ML models evaluated compared to the same models trained with datasets from one individual.

Conclusions

In this research, it is observed that when forming the dataset with ten users, the results are lower than when working with only one user. However, the results obtained are considered acceptable. It is observed that the Random Forest model presented the best performance for the parameters: F1 Score, Accuracy, Area Under the Curve, and Specificity/Precision. In Sensitivity/Recall, Naive Bayes had the best results. We also conclude that using several individuals for dataset conformation reduces the performance compared to a single-user dataset. Increasing the number of test subjects is necessary to increase the dataset. We conclude that this kind of ML system is better if personalized for a specific user than making it general for more users because the ML models tested performance decreases. This performance reduction is caused by the variability of the data obtained from different users. So it is better to obtain personalized features for the training datasets than to obtain generalized features for more than one user.

In future work, it is necessary to perform more tests to compare the performance of the ML models trained with datasets formed with single-user samples or personalized and other datasets made with the samples of more users and compare the performance of the trained models. Also, it is important to program these models on embedded systems and evaluate their performance.

Author Contributions

J.J.E.E. conceptualized the experiments, supervised the feature extractions, and wrote the original draft; L.J.B. participated in the methodology and formal analysis. A.S.T. contributed to data curation. M.Z.A. collaborated with the dataset elaboration and validation. D.A.T.T. oversaw the review and editing of the manuscript. O.R.L.B. supervised the data acquisition, G.M.G.A. performed the validation of the process, and the writing and editing of the paper. J.R.J.R. participated in the software validation. A.L.R. and A.S.M.V. realized the EEG data acquisition process and feature extraction. M.M.M.L and I.A.R.P. also participated in signal preprocessing and feature extraction.