1 Introduction

Research on autonomous vehicles is intensively explored due to their increasing scope of application, such as surveillance tasks, space exploration, delivery, transportation, and others [¹, ², ³, ⁴]. These vehicle systems require information about the environment to correctly perform the various tasks and avoid collisions with various obstacles in the environment. The various tasks that these vehicles perform are classified into three main problems: trajectory tracking, path tracking, and point stabilization [⁵,⁶].

All information is collected by the sensors these systems are equipped with, such as B. LiDAR, ultrasonic sensors, GPS, and cameras. The information of the environment is processed and interpreted to perform the above-mentioned navigation tasks and avoid accidents and collisions [⁴, ⁷]. Therefore, the main focus of this study consists of the precise control of the vehicle’s movement along a desired path, known as trajectory tracking, as well as the generation of such paths, known as trajectory generation. These topics have attracted significant attention in the field of autonomous vehicles [⁸]. A wide range of studies have been carried out on the generation of trajectories.

Prior studies [⁹, ¹⁰, ¹¹, ¹²] have shown that it is possible to implement intelligent techniques for generating trajectories and segmenting lanes. A technique for lane line identification is proposed by the authors in [¹³]. This approach utilizes the RANSAC algorithm to aid in the detection of lane lines and is predicated on an adaptive region of interest extraction strategy.

In addition, convolutional neural network-based algorithms for lane line detection are proposed by Haixia and colleagues [¹⁴]. For training and validation purposes, their algorithms employ the TuSimple dataset. The implementation of a Neural Network (NN) using an on-board camera in [⁹] leads to favorable outcomes in the trajectory generation process.

Hart et al. [¹⁰] describe the implementation of intelligent methodologies to create a viable trajectory generator. The study’s findings indicate that it is feasible to create trajectories using intelligent methodologies. However, Bellusci et al. [¹¹] present a new approach to lane segmentation using neural networks and computer vision techniques to create a map of the surroundings. Nevertheless, it does not explain the process of generating trajectories.

Besides, in [¹⁵] the system utilizes a convolutional neural network along with an auxiliary layer to detect the borders of lanes. In addition, the authors suggest a straightforward algorithm to rectify the vehicle's orientation by utilizing the centroid of the drivable area.

Unfortunately, there is no process for generating trajectories. In addition, in [¹⁶] was developed an independent navigation system using machine learning and computer vision techniques on a scaled vehicle.

The system also incorporates a depth camera for localization. However, the algorithm’s performance is reduced in low-brightness scenarios. Furthermore, a study conducted by Neven et al. [¹⁷] focuses on the development of a control system for a Car-Like robot equipped with a vision system.

This system enables the robot to detect and monitor lanes on a road. The study’s findings indicate that the utilization of vision techniques leads to effective lane detection in practical situations.

After addressing the trajectory generation problem, the subsequent critical task is solving trajectory tracking. Numerous studies have tackled this challenge, but there are lingering issues in the field. Various control schemes, including feedback control strategies [¹⁸], Sliding Mode Control (SMC) [¹⁹], and decoupled approaches [²⁰], have been explored.

Furthermore, these controllers use different scenarios that encompass diverse convergence rates [²⁰, ²¹], the effect of disturbances [¹⁹, ²², ²³], and different kinematic models [¹⁸]. Cui et al. [²⁴] introduced an adaptive control law within SMC, demonstrating exponential convergence to the trajectory.

The proposed methodology implements a decoupled approach to position and orientation tracking. Experimental results indicated its effectiveness for trajectory tracking despite disturbances. Nevertheless, the proposed methodology implements a simplified kinematic model, and the convergence rate is low. Qun Lu et al [²¹] proposed a fixed-time controller coupled with an observer under kinematic disturbances.

The controller accounted for signal saturation to prevent slipping, yielding satisfactory tracking results. However, it is noteworthy that the convergence exhibited a gradual pace. The employed kinematic model was the simplified version, and the control structure is complex. In [²⁵], the authors presented a prescribed-time containment controller coupled with a prescribed-time observer to achieve leader-follower tasks.

This study employs the effect of uncertainties and external disturbances by using a chain of integrators for the model. The results demonstrated good tracking performance and disturbance rejection. The research on trajectory tracking has been tackled from different perspectives, like finite-time stability, fixed-time stability, and simplified kinematic models.

However, prescribed-time stability has not been widely studied on WMRs, but in [²⁵] it was proved that this methodology can be implemented in these systems by achieving a fast convergence rate and low tracking errors. Thus, it is important to tackle this problem in WMRs because these systems need to attain a fast response in different scenarios where convergence time is crucial.

Furthermore, trajectory generation is a problem that has been studied from different perspectives, however, there are not many studies with onboard cameras on this crucial task.

2 Trajectory Generation

The generation of trajectories is an essential undertaking for WMRs owing to the diverse outcomes it has in real-world situations and the limitations it must satisfy to ensure its tracking is feasible. To achieve this goal, we propose the use of the methodology presented in Fig. 1, which integrates computer vision techniques and neural networks. To achieve this objective, we provide a detailed description of the proposed strategy.

Fig. 1 Trajectory generation methodology 

2.1 HybridNets Neural Network

The Neural Network employed is HybridNets1^fn [²⁷, ²⁸], which is an end-to-end perception neural network based on PyTorch. The objective is to address the multi-task issue by employing segmentation and box detection classification networks. Its main architecture comprises two networks, as depicted in Fig. 2.

Fig. 2 HybridNets architecture [29] 

The initial part of the system is the backbone, which uses the EfficientNet-B3 convolutional neural network architecture to extract characteristics from the input. This architecture scales the dimensions of depth, width, and resolution using a composite coefficient to obtain feature maps of the image.

The information extracted by the backbone is then passed on to the neck network, called EfficientDet, which uses a Weighted Bi-directional Feature Pyramid Network (BiFPN) module for image segmentation and object detection. The BiFPN module achieves this by creating bidirectional interconnections between network nodes. Each input feature is assigned an additional weight, allowing the network to determine the individual significance of each feature^fn.

2.2 Proposed Algorithm

To attain the trajectory generation task, we perform a series of steps detailed in Fig. 1, which allows us to finally obtain a feasible trajectory for a WMR using an on-board camera.

To this end, we provide a deep description of this algorithm in this section by using the Gazebo simulator of the Autominy vehicle [³¹].

Image acquisition. The first step entails reading the input image captured by the vehicle’s built-in camera, as shown in Fig. 3.

Fig. 3 Image acquired from the on-board camera of the Autominy simulator 

This step can be realized through the application of the available topics and the Robotic Operating System (ROS) to control the vehicle.

Intelligent Methods for Lane Detection. HybridNets NN application is used for lane segmentation and lane detection. This scheme identifies and separates the lane observed by the vehicle’s on-board camera. An important advantage of this NN is its ability to accurately identify the lane directly above the vehicle, as shown in Fig. 4.

Fig. 4 Output image with HybridNets NN 

Perspective Transformation. The output NN image undergoes a perspective transformation technique called “bird’s eye view”. This technique simulates the image being viewed from a higher angle, similar to a bird’s viewpoint [³²].

Leveraging this transformation provides a comprehensive perspective of the lane, ensuring both lanes remain parallel to facilitate the implementation of lane detection and segmentation processes. This simplifies the detection and segmentation processes. Figure 5 illustrates the use of the technique from a bird’s eye view.

Fig. 5 Bird-Eye-View perspective transformation 

Maximum Drivable Area Acquisition. In our pursuit to determine the largest navigable region, we first examine the area divided by the NN and thus determine the track with the largest extent to generate the desired path. To achieve this, a mask is implemented, which makes use of the segmentation color indicated by the intelligent method results (blue area).

All resulting available segmented areas are then found and the maximum available area is calculated. Subsequent to this phase, the largest area is selected and highlighted in green to proceed with the trajectory generation method.

The algorithm then identifies all currently available segmented areas and uses them to calculate the largest possible area. Next, the algorithm identifies the largest area and marks it with a green marker to proceed with trajectory creation.

Centroid Calculation and Reference Point Determination. The subsequent procedure entails determining the centroid of the drivable area that has been identified through the green contour in the segmented results, as depicted in Fig. 6. Afterward, five points are positioned to create the intended trajectory. Initially, two immobile points are positioned at an equivalent elevation as the ArUco marker on the Autominy vehicle. These points are fixed and serve as the initial positions for displaying the trajectory. In addition, two extra points are included, one located blow, and one above the calculated centroid. The reference points are adjusted according to the area determined by the NN, and they help to determine the trajectory when navigating a curved lane based on the captured image.

Fig. 6 Procedure for determining the centroid of the available drivable area 

Afterward, the visual representation of the trajectory that corresponds to the five points mentioned earlier is displayed in Fig. 7. Additionally, the homography transformation converts the image representing the aerial perspective into the initial image perspective [³³].

Fig. 7 Reference points for path visualization 

This transformation is dependent on the ArUco position, which serves as a reference point. This procedure utilizes the marker location and its position within the pixel coordinates on the bird-eye view image to accurately convert them into the corrected perspective pixel coordinates.

Figure 8 illustrates the process of translating the first trajectory to its equivalent on the vehicle’s perspective.

Fig. 8 Perspective correction 

Path generation. Ultimately, we derive the equation that defines the reference trajectory. The outcomes of the trajectory generation phase utilizing our suggested approach are depicted in Fig. 9. The trajectory is determined by analyzing the image captured by the on-board camera.

Fig. 9 Results for trajectory generation 

This visual observation is used to determine the trajectory. Figure 10 illustrates the reference trajectory produced using the proposed methodology in conjunction with the Autominy simulator. The controller will receive this trajectory as the reference input.

Fig. 10 Path generated using the proposed trajectory generation method 

3 Kinematic Model and Control Design

To tackle the trajectory tracking problem, we aim to design a controller that achieves prescribed-time stability to the desired trajectory. To this end, let us consider the full kinematic model a Car-Like robot depicted in Fig. 11, where q(t)=[x(t),y(t),θ(t),ϕ(t)]∈ℝ4 is the system’s vector configuration, the x(t), y(t) is the position on the plane with respect to the world frame {x,y}, θ(t) is the orientation of the vehicle with respect to the x axis, ϕ(t) is the steering angle of the front wheels. Now, the kinematic model of the WMR is described as:

Fig. 11 Description of the WMR’s kinematic model in the x, y plane 

With v(t)=[v1(t),v2(t)]T∈ℝ2 is the control input vector with v1(t), v2(t) being the linear and angular velocities, and d(t)=[d1(t),d2(t),d3(t),d4(t)]T∈ℝ4 encompasses the disturbances, which are bounded and smooth until its first time derivative [³⁴].

Furthermore, the actuators generate bounded control signals. Hence, based on the previous statement we are able to consider the following assumption.

Assumption 1. There exist some positive constants D¯, d¯, V1+, V2+, and Φ such that:

Then, we perform a coordinate transformation by defining the new output variable:

With an arbitrary δ≠0, which will be used to design a controller. Furthermore, to attain the trajectory tracking problem, a reference kinematic model is required, which is described by:

where qd(t)=[xd(t),yd(t),θd(t), ϕd(t)]T∈ℝ4, vd1(t), vd2(t)∈ℝ are the reference signals of q(t) and v1(t), v2(t). Moreover, we perform the same coordinate transformation as in (4):

By following the procedure described in [²⁰] along the coordinate transformation (5) and the reference model (9), we define the tracking error ξ¯(t)=ξ(t)−ξd(t), which yields to the following dynamic error structure:

where the structure of A(θ,ϕ), A¯(θ,ϕ), u¯(t), Γ˙(t), u¯d(t), A˙(θd,ϕd), and vd(t) can be consulted in [²³].

Now we can state the control objective by implementing a hybrid control scheme that makes the tracking error signal ξ˜(t) to converge to zero in a prescribed time despite disturbances.

Control objective. Considering the WMR’s kinematic model (1), the coordinate transformation (4), and the error’s dynamics (7), design a control input v(t) such that the tracking error converges to zero in a prescribed time.

4 Control Design

In order to design the proposed controller, depicted in Fig. 12, we begin by presenting a general structure for the controller, which is then applied to the kinematic model (1) by employing the tracking error ξ˜(t), and the error's dynamics (7) to achieve prescribed time stability to the desired trajectory.

Fig. 12 General scheme of the proposed methodology 

5 Numerical Results

To assess the proposed approaches for trajectory generation and trajectory tracking tasks, we conducted simulations using MATLAB/SIMULINK® for trajectory tracking and the Autominy simulator for trajectory generation [⁴]. The trajectory generated via the methodology described in Section 2 is illustrated in Fig. 13.

Fig. 13 Reference points for path visualization 

To assess the performance of the proposed methodology, named PTC, we consider the controllers described in references [¹⁸] and [²⁰]. We choose the dynamic feedback controller [¹⁸] (named DFC), due its simple structure and because it implements the complete kinematic model of a Car-Like robot.

Furthermore, we also employ [²⁰] (referred to as FTC) because it utilizes the full kinematic model of the Car-Like robot, attains the trajectory tracking problem by using a decoupling approach, and achieves Finite-time stability. The following cases are considered for the validation of the PTC controller:

The gains considered for the DFC are kpi=343, kvi=147, kai=21, and for the FTC are k1=35, k2=25, k3=5 and k4=7. Finally, the parameters for the PTC's time-varying gains are T1i=1, k11=5, k12=5, k21=25, k22=15, m_i=1, and for the twisting controller are c11=275, c12=495, c21=770, c22=715 withδ=0.03. The simulation in Matlab/Simulink was performed using the sampling time of 1×10−4 seconds and Runge-Kutta algorithm as the solver.

In Fig. 13 is depicted the final trajectory generated with the procedure detailed in Section 2. The red marks are the samples gathered from the Autominy simulator, and the smooth trajectory generated with the time intervals and the red marks is represented by the blue line. The equations that represent the motion at each time interval are shown in Table 3 (see Appendix A) and are depicted in generalized coordinates in Fig. 14.

Fig. 14 Desired trajectory represented in generalized coordinates x(t), y(t), θ(t) and ϕ(t) 

In Fig. 15 are depicted the tracking errors in generalized coordinates. It can be observed that the FTC and the PTC controllers converge faster than the DFC. Also, the proposed controller reaches the vicinity near the origin in t<1[s], as highlighted with the orange area; during this transitory stage the time-varying feedback control scheme is used, which afterwards is switched to the twisting controller.

Fig. 15 Tracking errors in x˜, y˜, θ˜ and ϕ˜ for case C1 

This behavior is also present in y˜, where the FTC and the PTC present the fastest responses, followed by the DFC. The PTC controller exhibits a slight overshoot; however, it approaches the vicinity of the origin at t<1[s] within the orange region.

Finally, in ϕ˜ S the PTC controller obtains the fastest convergence compared with the DFC and FTC controllers; however, it also exhibits the highest level of overshoot. Furthermore, in 10≤t≤14.15 the proposed controller achieves the lowest overshoots compared with the FTC and DFC controllers; this behavior can be attributed to the alterations in the reference signal for ϕ(t) as depicted in Fig. 14 during this specific period.

Furthermore, in Fig. 16 are depicted the control signals generated by the controllers in comparison to the desired control inputs v1(t) and v2(t), which are represented by the black-colored dashed lines. In v1(t), it is evident that the FTC controller exhibits the highest overshoot when compared to the PTC and DFC controllers; this behavior is associated with the fastest convergence of the FTC’s tracking x˜ in Fig. 15, which is attributed to the presence of the SMC in the x coordinate.

Fig. 16 Control signals v1(t) and v2(t) generated for case C1 

However, the PTC achieves the fastest response, which implies the fastest tracking response. For v2(t), the PTC controller generates a large control signal before the commutation of the twisting controller, then it generates a noisy control signal, which is directly related to the tracking error of ϕ(t) in Fig. 15. Moreover, it is evident that the PTC controller transitions to the twisting control scheme in a time period less than 1 second.

As observed in the signal, the PTC controller generates the effect of chattering due to the presence of the twisting controller. In addition, all the control signals remain with similar behaviors after the transient stage.

Figure 17 depicts the tracking errors for case C2 in the x, y, θ and ϕ coordinates. It can be observed that x˜ behaves similarly, with the FTC and PTC controllers achieving the fastest response, even when disturbances are considered. Regarding y˜(t), it is evident that the FTC and PTC controllers also achieve the fastest response; nevertheless, the performance of the FTC and DFC controllers is diminished due to the generation of greater oscillations.

Fig. 17 Tracking errors in x˜, y˜, θ˜ and ϕ˜ for case C2 

Furthermore, it is noteworthy that the time-varying stage of the PTC commutes to the twisting controller before the orange-colored area ends, where the twisting control signal is introduced, and thereafter the error signal keeps a neighborhood in zero.

For θ˜, the fastest response is achieved by the PTC and FTC control schemes, but the PTC controller keep less oscillations. It can also be observed that the DFC controller generates a greater error at the end of the simulation.

For ϕ˜(t), the largest overshoot was generated by the PTC due to the time-varying stage and the control signal v2(t), which is observed at the beginning of the orange-colored area.

The PTC maintains the lowest tracking errors in this coordinate with the lowest oscillations even in the presence of disturbances.

Conversely, the DFC and FTC controllers present the large oscillations during the trajectory tracking. The tracking error ϕ˜(t) is associated with the control signal v2(t) depicted in Fig. 18, where it is corroborated that the DFC controller generated large control signals. On the other hand, the PTC controller generated the largest overshoot while the time-varying gains were active, i.e., in the orange-colored area at t<1[s]; afterwards, the proposed scheme presents the effect of chattering due to the properties of the twisting controller and the SMC, however, its magnitude is small.

Fig. 18 Control signals v1(t) and v2(t) generated for case C2 

For v1(t), the PTC controller achieved the fastest response with a slight overshoot compared with the FTC controller. The DFC controller generated the lowest overshot and achieved the lowest response. In addition, it is clear that the proposed PTC controller achieves the fastest response and keeps the lowest oscillations in comparison with the DFC and FTC controllers.

Hence, the aforementioned findings illustrate that the PTC controller achieves the fastest response and the lowest oscillations, despite the existence of disturbances. Therefore, it can be inferred that the PTC controller excels over the DFC and FTC controllers.

In addition, a quantitative analysis was con-ducted to supplement the qualitative analysis. Tables 1 and 2 expose the IAE (Integral of Absolute Error) and ITSE (Integral of Time-weighted Squared Error) for evaluating the tracking error of the controllers, while the ISV (Integral of Squared Control Signal) is used to quantify the control signals [²⁰]. Table 1 displays the quantitative tracking errors and ISV for case C1 while Table 2 showcases the performance indexes for case C2.

Regarding case C1, it is evident that the PTC controller achieves the lowest tracking errors for each coordinate, except for ϕ(t), where the FTC controller achieves the lowest tracking errors in terms of ITSE. These quantities indicate that the PTC controller outstands with the best performance by achieving the lowest tracking errors with respect to the DFC and FTC controllers.

In addition, for case C2, the FTC controller demonstrated the smallest tracking error in θ(t), while the PTC controller exhibited the smallest tracking errors in the remaining coordinates, thus demonstrating its superior performance in comparison with the other controllers.

Based on the preceding qualitative and quantitative analysis, we can deduce that the PTC controller demonstrates superior performance in both disturbed and undisturbed scenarios by achieving the fastest response and lowest oscillations with respect to the DFC and PTC controllers. Moreover, the proposed control strategy achieved the lowest tracking errors with ITSE and IAE in general, thus outstanding its performance. Furthermore, we have observed that the trajectory generation algorithm is an effective solution for the task of generating trajectories.

This is because it successfully fulfills the non-holonomic constraints of WMRs. The simulations conducted with the controllers have demonstrated the algorithm’s ability to track trajectories in real-world scenarios, even in the presence of disturbances.

6 Trajectory Generation

The trajectory generation algorithm obtained specific points from the rail detections, which were subsequently analyzed to calculate two polynomials in the x,y plane. In order to achieve a feasible trajectory for a Car-Like robot [¹⁷], it is necessary for these polynomials to be both smooth and continuous. The procedure for designing the equations is described in [37], resulting in the equation's trims presented in Table 3.

7 Conclusions

The manuscript introduced an innovative approach for generating trajectories in vehicular systems by utilizing an on-board camera, computer vision techniques, and intelligent algorithms.

The proposed methodology was evaluated by employing three controllers that successfully achieved the trajectory tracking task. In addition, a prescribed-time controller was introduced for the purpose of trajectory tracking tasks. The performance of this controller was evaluated by comparing it to two controllers previously discussed in the literature.

Employing the simulation capabilities of MATLAB/SIMULINK, the prescribed-time controller demonstrated be superior, showcasing its resilience and adaptability in maintaining the desired trajectory, irrespective of the presence of disturbances.

The proposed methodology demonstrated its key feature of adjusting the convergence rate and its ability to withstand disturbances. Therefore, this study introduced a novel approach that integrates two approaches to tackle significant challenges in vehicular systems: the prescribed-time controller and the trajectory generation algorithm.

Future research ought to emphasize on enhancing the prescribed time controller by means of reducing or modifying the control structure to mitigate chattering and minimize the overshoot of the control signal of v2(t).

Furthermore, the trajectory generation algorithm can be enhanced by utilizing optimization algorithms to enhance its desired control signals in relation to generalized coordinates.