Time headway distribution analysis of naturalistic road users based on aerial datasets

Abstract

Purpose

Time headway (THW) is an essential parameter in traffic safety and is used as a typical control variable by many vehicle control algorithms, especially in safety-critical ADAS and automated driving systems. However, due to the randomness of human drivers, THW cannot be accurately represented, affecting scholars’ more profound research.

Design/methodology/approach

In this work, two data sets are used as the experimental data to calculate the goodness-of-fit of 18 commonly used distribution models of THW to select the best distribution model. Subsequently, the characteristic parameters of traffic flow are extracted from the data set, and three variables with higher importance are extracted using the random forest model. Combining the best distribution model parameters of the data set, this study obtained a distribution model with adaptive parameters, and its performance and applicability are verified.

Findings

In this work, two data sets are used as the experimental data to calculate the goodness-of-fit of 18 commonly used distribution models of THW to select the best distribution model. Subsequently, the characteristic parameters of traffic flow are extracted from the data set, and three variables with higher importance are extracted using the random forest model. Combining the best distribution model parameters of the data set, this study obtained a distribution model with adaptive parameters, and its performance and applicability are verified.

Originality/value

The results show that the proposed model has a 62.7% performance improvement over the distribution model with fixed parameters. Moreover, the parameter function of the distribution model can be regarded as a quantitative analysis of the degree of influence of the traffic flow state on THW.

Citation

Yu, R., Zhang, Y., Wang, L. and Du, X. (2022), "Time headway distribution analysis of naturalistic road users based on aerial datasets", Journal of Intelligent and Connected Vehicles, Vol. 5 No. 3, pp. 149-156. https://doi.org/10.1108/JICV-01-2022-0004

1. Introduction

In the traffic safety domain, time headway (THW) is defined as the elapsed time between successive vehicles in a single lane of traffic (Biswas et al., 2021). During Safety of The Intended Functionality (SOTIF) development, THW distribution can provide references for requirement metric extraction, scenario-based testing and quantitative evaluation. In concept phased of Functional Safety (FuSa) development, THW distribution can be used to determine key time indicators, e.g. fault tolerant time interval and fault reaction time interval and controllability (C) to avoid a specified harm or damage (Mao et al., 2021; Zhu et al., 2020; Chen et al., 2021; Shangguan et al., 2021). What’s more, it can also provide indicator references for longitudinal-related ADAS and AD standards, e.g. ACC, AEB and ALKS.

However, the observed THW values are not constant even if the environment is completely identical due to factors such as differences of driver’s ability in perceptions, data processing, taking actions and heterogeneity of vehicle performance (Li and Chen, 2017). That’s why it is difficult to obtain THW characteristics of drivers. Therefore, many researchers use distribution model to describe THW.

Buckley (1968) proposed the semi-Poisson model and verified its good fitting performance using peak-hour THW data from a 6-lane highway in Sydney. Based on Buckley’s study, Wasielewski (1979) used a semi-Poisson model to explore the relationship between the variation of THW and traffic flow of following vehicles on highways. Tolle (1976) explored the fit performance of the composite exponential distribution, the Pearson Type III distribution and the lognormal distribution to THW. By analyzing data from Interstate 71 in Ohio, USA, Tolle verified that the lognormal distribution fits the THW better than the other two distributions. Based on Berthon distribution, Cowan (1975) obtained the M3 distribution by improving the distribution model. Cowan evaluated the model by collecting 1,324 THW samples on a two-lane highway in both directions in Sydney, and the results showed that the M3 model could fit the naturalistic driving data well. Griffiths and Hunt (1991) proposed a double-shifted negative exponential distribution model to describe the THW distribution by analyzing data from 86 groups of single-lane THW samples collected in the UK with a total of 82,388 cases. The test results show that the double-shifted negative exponential distribution model can fit most of the data. Mei and Bullen (1993) investigated the applicability of the THW distribution model in a high-density traffic flow environment. He fitted the THW data collected on interstate 279 during the morning peak hour with a normal distribution, a Pearson distribution, a lognormal distribution and a shifted lognormal distribution. The final test results showed that the shifted lognormal distribution had the best performance.

Sun and Benekohal (2005) demonstrated that the lognormal, inverse Gaussian and Weibull distributions have better performance in describing THW distribution among 10 commonly used mathematical models. Some studies (Ye and Zhang, 2009) also used normal and shifted exponential distribution models to describe THW under normal traffic conditions. Burr distribution is also widely used in the transportation domain (Taylor, 2017), it is used in the description of the travel time data and the calculation of its reliability ratio.

Current analysis of the THW distribution model found that many papers improve the goodness-of-fit of their models by dividing different applicability ranges of average speed or flow rate. The introduction of the segmentation function improves the performance but introduces the limitations of the segmentation function itself. Weng et al. (2013) investigated the THW distribution in work zones by including construction intensity, lane location, traffic flow rate and truck occupancy as variables and considered their effects on the distribution model parameters, which could be used to obtain an adaptive parameter model. However, the above four variables were artificially given and were not validated for their influence on THW.

Traffic flow data used in related papers were collected on a small scale; most of them are less than 10,000 samples, and data quality is relatively low. Finally, selected distribution models in these works also fit the best for these small amounts of data. Most of the research in the past decade used the NGSIM data set collected by the Federal Highway Administration (2004). The vehicle trajectory data in this data set is extracted from videos captured by multiple digital cameras installed in different places near the highway. The vehicles in these videos are detected by a feature-based vehicle detection algorithm and tracked based on a zero-mean correlation matching algorithm. The lane markers are manually identified to find the lane number of the vehicle. However, more than 10% of vehicles were not successfully detected, and missing trajectories in the detected data are also a severe problem. In addition, the vehicle acceleration data often show unrealistically large additions and subtractions (Punzo et al., 2011; Montanino and Punzo, 2015). It can be found that the quality of the NGSIM data set is not very satisfactory due to the limitations of data acquisition and processing techniques.

Referring to the above-mentioned research gap, we aim to build a more reliable THW distribution model using high-quality trajectory data set, the HighD data set (Krajewski et al., 2018) and the AD4CHE data set, and considering the effects of multiple scenario elements. The main contributions of this paper are as follows:

• We proposed a parametric adaptive methodology for describing the THW distribution and verified its performance over previous work.

• The parameter functions computed in the methodology can be used for the quantitative study of the relationship between traffic flow parameters and THW.

The rest of this paper is organized as follows. Section 2 proposes the method for obtaining the THW distribution model with adaptive parameters; Section 3 gives a detailed description of the data processing; Section 4 gives the analysis results and their application in quantitative studies; concluding remarks and future applications are discussed in Section 5.

2. Methodology

The flow chart of the methodology is shown in Figure 1. To obtain the distribution model with the best goodness-of-fit on the data set, 18 THW distribution models were considered. The best distribution parameters will be determined using maximum likelihood estimation (MLE). After obtaining the best parameters for all candidate distribution models, a goodness-of-fit test is required to select the best-performing model from the candidate distribution models.

In transportation engineering, the chi-square test and K-S test are usually used to verify the goodness-of-fit of the distribution model. As a nonparametric method, the K-S test is widely used due to its robustness, insensitive to scaling, independence of the location of the mean and more effectiveness than the chi-square, therefore, the K-S test is chosen in this paper. For each sample value x examined, the K-S test compares the proportion of values less than x with the expected number predicted by the hypothetical distribution, and then uses the maximum difference of all x values as its test statistic. It can be expressed as the following equation:

(1) T(x) =max(|F(x)−G(x)|)

where F (x) is the proportion of data set values less than or equal to x and G(x) is the standard hypothetical cumulative distribution function (CDF) evaluated at x. Therefore, the smaller the value of the statistic, the better the performance of the test distribution model.

Second, for these models, the one with the lowest statistic in K-S test is considered to be the best distribution model. Then, 60 data files of the aerial data set will be divided into a training set and a test set. For the training set, the distribution model parameters will be estimated using MLE technique.

After that, the traffic scenario variables of each data file, such as congestion level, and truck–car ratio, are extracted, and the random forest model selects the parameters that have the most significant influence on the distribution model. Many indicators can be used to measure the importance of variables, and in this paper, we choose out-of-bag permuted predictor delta error, as shown in equations (2), (3):

(2) Hoob(x)=argmaxy∈γ∑t=1TI(ht(x)=y)⋅I(x∉Dt)

(3) eoob=1|D|∑(x,y)∈DI(Hoob(x)≠y)

where e^oob denotes out-of-bag permuted predictor, D_t is the training sample set that classifier h_t actually use, H^oob(x) is the out-of-bag prediction to the sample x, y indicates THW, and I represents indicator function.

The selected parameters are used as input and multiple regression technique is performed to obtain the parameter functions. Finally, the performance of distribution model with adaptive parameters is verified by the test set.

Although Weng et al. (2013) have verified that vehicle type has a significant effect on THW distribution, in real traffic scenarios, pure Car–Car, Car–Truck, Truck–Truck, Truck–Car scenarios are rare, and these scenarios are often mixed in the whole traffic flow. That is why the THW distribution model is not analyzed separately for a different vehicle type in this paper. However, if heterogeneity is not adequately considered, biased coefficient estimates and erroneous predictions may be obtained (Yu and Abdel-Aty, 2013; Mannering et al., 2016). Therefore, in this paper, the heterogeneity of traffic flow is taken into account by including the truck–car ratio as one of the traffic flow characteristic variables in fitting the parameter function.

3. Data processing

3.1 Data set

Following two aerial-based naturalistic vehicle trajectory data sets will be used in this paper: HighD (The Highway Drone Data set), AD4CHE (Aerial Data set for China Congested Highway & Expressway). HighD, captured from German highways using drone, will be used as experimental data. The drone measurements provide an average of 17 min of each recorded video, covering approximately 420 m of highway section (Figure 2). It was conducted on six different highway sections (divided into 60 periods) for 44,500 km total traveled distance. The position, speed, acceleration and vehicle type data of every vehicle extracted from the 40,244,960 frames comprised 112,122 trajectories.

The AD4CHE data set, which was established by DJI Automotive and Jilin University, collects data in a similar way as HighD data set in four cities in China (Changchun, Xi’an, Hefei, Shenzhen). In addition to straight roads, videos are collected curved roads and on/off ramps. Therefore, its data structure is also more comprehensive than HighD. The AD4CHE data set provides an average of 5 min of each recorded video, covering approximately 130 m of highway section (Figure 3). It was conducted on different highway sections (divided into 68 periods) for 6,540 km total traveled distance.

Unlike HighD data set with an average speed of 100.7 km/h and an occupancy of 0.08, the vehicle’s average speed and occupancy in AD4CHE are 29.25 km/h and 0.22. In the next quantitative analysis process, HighD data set will be used as the main object of analysis. In the qualitative analysis process, HighD data set and AD4CHE data set are compared to study the behavioral characteristics of drivers in the following scenarios at high and low speeds.

3.2 Data filtering

These data sets contain three scenarios: free driving, following and lane changing; this paper will focus on the THW distribution in following scenario. Therefore, we need to filter the data and exclude the lane changing and free driving scenarios. The flow is shown in Figure 4.

Firstly, the THW data of the data set is used to eliminate the free driving scenarios and the false detection trajectories, that is, the vehicles with THW equal to 0 or exceed 10 can be considered as free driving scenarios, and the vehicles with THW less than 0 are error data, which need to be eliminated.

Subsequently, according to the number of lane-changing data in the data set, all the lane-changing vehicles within the video shooting range are identified and their IDs are obtained. The longitudinal coordinates of the lane-changing vehicles are retrieved according to their IDs, and the specific lane changing times are calculated with the positions of lane markers. The surrounding vehicles 3 s before and after the lane-changing vehicle are considered vehicles affected by the lane changing behaviour and excluded. Then, as the HighD data set is large and contains 40,244,960 THW samples, the THW data needs to be simplified. A trajectory is divided into several segments according to whether its surrounding vehicle ID has changed, and the THW value of each segment is averaged as valid and independent THW data.

3.3 Data set division

With only 60 valid data files, to obtain a parameter function with sufficient confidence interval, it is necessary to limit the number of random forest predictors, that is, the number of traffic flow characteristics variables. A minimum of 59 valid samples is required for a multivariate regression with three variables, 0.8 statistical power level, 0.2 effect size and 0.05 probability level as calculated by a dedicated calculator (Soper, 2013). Therefore, three parameters from the traffic flow characteristics variables need to be selected for multiple regression.

4. Results

4.1 Best distribution model selection

This work will be compared and selected among the 18 distribution models mentioned above. Using the processed THW data, the MLE of each of the 18 distribution models was followed by the K-S test, and the results are shown in Table 1.

As shown in Table 1, the Burr distribution has the lowest K-S statistic, and the result of the Burr distribution model fitting is shown in Figure 5. For the data set used in this paper, the Burr distribution is the best fit and will be used in the next analysis The probability density function is as follows:

(4) f(x;α,c,k)=kcα(xα)c−1(1+(xα)c)k+1

where c and k are the shape parameters and α is the scale parameter.

4.2 Parameter function fitting

The best distribution model derived in the previous subsection can achieve satisfactory results only in the traffic flow with similar characteristics. However, for different traffic flows, the model's performance cannot be guaranteed. Therefore, the model parameters must adapt to the traffic flow characteristic variables. Due to the lack of interpretability of the data-driven approach, a multiple regression approach is chosen to fit the parameter functions in this paper.

The traffic flow characteristics variables are extracted from the training set: traffic flow rate, average velocity, occupancy rate, truck-car ratio and average vehicle length. Using the MLE technique, Burr distribution parameters are obtained: α (scale), c (first shape parameter) and k (second shape parameter).

This paper uses the random forest model to evaluate the importance of five traffic flow characteristics variables and their inverse and natural logarithms on the Burr distribution parameters. First, appropriate hyperparameter values for the random forest model should be selected. The mean squared error is used as an evaluation metric to measure the model’s performance, and a gradient descent method is used to obtain the hyperparameter values that make the model perform best.

The importance of the three parameters of the Burr distribution is then evaluated separately for each of the identical, logarithm and inverse forms of the five traffic flow characteristics variables. Due to the small gap between different forms of the same variable and the random nature of the random forest model, the results of a few tests could not be trusted. Therefore, the random forest algorithm was repeated 100 times to count the average value of the importance index for the three forms of the five traffic flow characteristic variables, and the results are shown in Table 2.

Based on Table 2, the selected traffic flow characteristic variables are shown in Table 3. Using multiple regression technique, the fitted parameter functions are shown in Table 4. These variables and functions will be used in THW distribution model fitting in HighD data set or the other German highway data set.

4.3 Distribution model validation

After the adaptive parameter model obtained, it is necessary to investigate and validate the performance of the model with the test set. The traffic flow characteristics variables of the test set are firstly calculated. Then the Burr distribution model fitted directly from the training set, the test set, and the Burr distribution with adaptive parameters are compared. The traffic flow characteristics variables, their cumulative density functions (CDFs) and the K-S test results are shown in Figure 6.

It can be seen that, compared with the fixed parameters model trained from the training set, the adaptive distribution model has a CDF function that is closer to the empirical distribution and reduces the K-S statistic by 62.7%. In addition, its K-S statistic is only 0.0196 higher than the distribution trained from the test set, which shows that the adaptability and performance of the distribution model with adaptive parameters are greatly improved.

4.4 Application to qualitative research

The above analysis has demonstrated the advantages of the proposed method for THW quantitative studies, and the next analysis will demonstrate the applicability of the method for qualitative studies with the HighD data set and the AD4CHE data set. However, the chosen Burr distribution itself is a three-parameter model, and indicators such as mean and variance, which are commonly used in qualitative studies, are difficult to express. After considering the model’s goodness-of-fit to the data (Table 5) and the simplicity of the model parameters, the inverse Gaussian distribution will be used in next analysis. The probability density function is as follows:

(5) f(x;μ,λ)=(λ2πx3)12e−λ(x−μ)22μ2x

where µ is the scale parameter and λ is the shape parameter. The inverse Gaussian distribution has a mean of µ and a variance of λµ³.

The above method was then applied to each of the two data sets, and the results are shown in Tables 6–8.

In this paper, we will use the model mean which indicates the steady following characteristics of the driver for subsequent qualitative analysis.

As we can see in Table 8, the mean value of THW will decrease as the number of trucks in the traffic flow increases in the high-speed scenario (Above 80 km/h), which means that when the front vehicle is a truck, the driver of the car will try to approach the truck and try to overtake it. In contrast, on low-speed congested roads on China highway (0–80 km/h), the average THW value rises as the number of trucks rises. This may be because in congestion, trucks, like cars, produce more excellent acceleration and deceleration, which makes trucks with large masses avoided by surrounding drivers.

It can also be found that the mean THW value decreases as the average vehicle speed increases on low-speed congested roads. The reason is that most drivers maintain a constant distance between cars during low-speed passing to prevent being overtaken.

5. Conclusion

Our findings demonstrate a new method to describe the THW distribution, which can be used in traffic flow analysis and automated driving system safety design. Based on the vehicle data following the scenario in the HighD data set, it is found that the Burr distribution model fits the data set best among 18 commonly used distribution models. Through the random forest model, three traffic flow characteristics variables, traffic flow rate, occupancy rate and truck-car ratio, have the most significant influence on the Burr distribution model. Subsequently, an adaptive parameter model is fitted based on the test set, and the results show that, with the test set, its performance is validated to be better than fixed parameters models. Subsequently, by fitting the two data sets with the inverse Gaussian model and comparing its parameter functions, we showed the application value of the method to qualitative analysis. It can be found that compared to traditional piecewise function, with this parameter adaptive distribution model, we can express the THW distribution more concisely and at the same time achieve higher goodness-of-fit. Moreover, by analyzing the coefficient changes in the parameter functions, we can better explore the potential information hidden in a large amount of data.

The method proposed in this paper is not limited to THW but is also applicable to other safety indicators. For example, the parameter describing the degree of urgency in the following scenario: TTC, the parameter describing the driver’s driving behavior characteristics: acceleration; yaw rate, etc.

Consequently, we plan to extract more safety-related metrics from the aerial data set in the future. We will also conduct data application research in automated driving FuSa and SOTIF development, e.g. safety requirement metric extraction, scenario-based testing and quantitative evaluation.

Figures

Figure 1

Workflow of THW adaptive parameter model

Figure 2

HighD data set

Figure 3

AD4CHE data set

Figure 4

Data filtering

Figure 5

Result of the distribution model fitting

Figure 6

Comparison of different models

Table 1

Statistical results for different distribution models. (HighD data set)

Models	Birnbaum Saunders	Burr	Exponential	Extreme Value	Gamma	Generalized Pareto
K-S statistic	0.0494	0.0333	0.1788	0.2924	0.0868	0.1606
	Half Normal	Inverse Gaussian	Logistic	Loglogistic	Lognormal	Nakagami
	0.1095	0.0363	0.1508	0.0370	0.0435	0.1336
	Normal	Poisson	Rayleigh	Rician	tLocationScale	Weibull
	0.1641	0.2629	0.2506	0.2506	0.1535	0.0853

Table 2

Mean value of variable importance

Traffic flow characteristic variables		Traffic flow rate	Average velocity	Occupancy rate	Truck−car ratio	Average vehicle length
α	Identical form	0.5783	0.0919	0.2248	0.2334	0.1275
	Inverse form	0.4458	0.1033	0.2057	0.3081	0.1566
	Logarithm form	0.5932	0.0969	0.2014	0.2429	0.1259
c	Identical form	0.5603	0.175	0.3161	0.3974	0.2374
	Inverse form	0.6678	0.1669	0.3742	0.3474	0.2054
	Logarithm form	0.5605	0.1915	0.3239	0.3891	0.2478
k	Identical form	0.5603	0.1057	0.2056	0.2765	0.1665
	Inverse form	0.4579	0.1204	0.2321	0.3196	0.1927
	Logarithm form	0.5596	0.0724	0.2132	0.2532	0.1768

Table 3

Selected traffic flow characteristics variables

Burr distribution parameter	Traffic flow characteristic variable
α	In(f), o, 1/P
c	1/f, 1/o, P
k	f, 1/o, 1/P

Table 4

Parameter functions of the best distribution models

Burr distribution parameter	Parameter function	Adjusted R2
α	α = 32.5231 –4.5391In(f) +0.0451o −0.0040(1/P)	0.7585
c	c = 4.4 –1392.5(1/f) +3.5(1/o) −2.5 P	0.7836
k	k = 4.3787 –0.0025f −16.2797(1/o) +0.0555(1/P)	0.5406

Table 5

Statistical results for different distribution models (AD4CHE data set)

Models	BirnbaumSaunders	Burr	Exponential	ExtremeValue	Gamma	GeneralizedPareto
K-S statistic	0.0335	0.0224	0.2427	0.2541	0.0613	0.2192
	HalfNormal	InverseGaussian	Logistic	Loglogistic	Lognormal	Nakagami
	0.1808	0.0352	0.0960	0.0230	0.0267	0.1084
	Normal	Poisson	Rayleigh	Rician	tLocationScale	Weibull
	0.1375	0.2379	0.1188	0.1189	0.0976	0.0868

Table 6

Mean value of variable importance (HighD)

Traffic flow characteristic variables		Traffic flow rate	Average velocity	Occupancy rate	Truck–car ratio	Average vehicle length
µ	Identical form	0.7296	0.0763	0.2927	0.1973	0.1129
	Inverse form	0.6185	0.0934	0.2393	0.2117	0.1542
	Logarithm form	0.7170	0.0783	0.2937	0.1890	0.1196
λ	Identical form	0.3550	0.3392	0.3115	0.1611	0.0567
	Inverse form	0.3079	0.3079	0.3549	0.1959	0.0668
	Logarithm form	0.3562	0.3392	0.3167	0.1661	0.0492

Table 7

Mean value of variable importance (AD4CHE)

Traffic flow characteristic variables		Traffic flow rate	Average velocity	Occupancy rate	Truck–car ratio	Average vehicle length
µ	Identical form	0.5991	0.2517	0.0956	0.2077	0.1755
	Inverse form	0.4756	0.2045	0.0969	0.2527	0.1773
	Logarithm form	0.5940	0.2541	0.0925	0.2160	0.1789
λ	Identical form	0.3967	0.3533	0.2917	0.3452	0.1568
	Inverse form	0.3019	0.3424	0.2713	0.4109	0.2104
	Logarithm form	0.3955	0.3583	0.2977	0.3353	0.1640

Table 8

Parameter functions of Inverse-Gaussian models

Dataset	Distribution parameter	Parameter function	Adjusted R2
HighD	µ	µ = 2.8465 + 0.3275*ln(o)+0.0136*(1/P)-0.0015*f	0.9546
AD4CHE	µ	µ = 3.8319 –0.51391*ln(v)-0.0121*(1/P) −0.0004*f	0.8198