October 11, 2018

Did BC’s speed limit increase lead to more deaths?

Don MacKenzie

 

I spent Wednesday driving in my home province of British Columbia (BC), Canada, and I noticed something that had also struck me on my last visit: the speed limits on many of the province’s rural highways seemed surprisingly high relative to the physical design (geometry, sight lines) of the roads. Apparently this is not (as I had previously assumed) just a matter of my getting cautious with age or turning soft after years of driving on American Interstates; the speed limits were increased on many BC highway segments in 2014. Interestingly, the Minister of Transportation at the time had a well known need for speed, having had his driver’s license suspended after receiving five speeding tickets within just one year.

Coincidentally, Wednesday also marked the release of a new research study assessing the effects of the speed limit increase on highway safety in BC. The study was widely covered on the news here in BC, and understandably: it claimed to have identified a huge increase in crashes, including fatal crashes, as a result of the speed limit increase.

I commend the authors for addressing this important question, but after reading the study in full, I have serious concerns about its quality and the soundness of its conclusions. In short, I would not have recommended publication if I had been a peer reviewer, due to many unusual and unexplained analytical decisions, inadequate description of methods, and a lack of robustness checks to ensure that the claimed results were not just a statistical quirk.

I am not claiming that the speed limit increase had no effect on safety. Indeed, we know that higher speeds mean less time for drivers to react to problems, which can increase the likelihood of some types of crashes. Higher speeds also mean more kinetic energy, increasing accident severity. So I find it entirely plausible that the speed limit increase may have increased crashes and fatalities. But, this particular study does not provide adequate evidence to support such a conclusion. My major concerns with the study are outlined below.

Assigns crashes from nearby roads to affected highway segments

Adverse outcomes on affected highway segments (where the speed limit was increased) included not only those on the highway proper, but also those within 500m of the highway. The authors explain that this was “to account for possible errors in mapping crashes to exact location.” However, this could lead to an over-estimate or under-estimate of the true effect of the speed limit increase. It is impossible to judge how serious a problem this is without seeing additional data or analysis. One simple possibility would be to vary the buffer distance between 50m and 1000m (rather than using only 500m) and see how this affects the results.

Uses gasoline sales instead of actual kilometers of travel

It is important to control for increases in the total amount of vehicle travel, since more travel would lead to more crashes even the per-kilometer crash risk remained constant. The authors “included taxable and tax-exempt sales for gasoline and ethanol blended gas as a surrogate for vehicle travel.” The problem is that average fuel consumption per km likely decreased over the study period, due to (1) BC’s carbon tax and (2) national-level fuel economy policy, which generally mirrors US CAFE policy. This means that vehicle kilometers traveled in BC likely increased more quickly than would be estimated based only on gasoline sales, and failing to account for this would lead to over-estimates of crash rates in later years of the study.

The use of gasoline sales is an especially peculiar choice since as the authors note, “permanent count stations record the number of vehicles passing the count station every hour (for each travel direction). Between 2005 and 2016, there were 336 permanent count stations that recorded traffic volumes at 185 unique sites, with 11 of these sites located on one of the affected segments.” The authors do not explain why they didn’t simply use these vehicle counts as a measure of the overall level of vehicle travel, although this would seem to be a more suitable direct measure than gasoline sales.

Claims increase in fatal crashes, but not in ambulance dispatches

If there were a systematic increase in crash frequency and/or severity, we would expect to see an increase in fatal crashes, ambulance dispatches, and total losses. While the study reports an increase in all claims, casualty claims, and fatal crashes, it does not find any change in ambulance dispatches. This is surprising, but the authors do not reconcile this inconsistency. This points to the possibility that the results for claims and fatal crashes were a statistical quirk.

Speed increases were smaller on affected segments than non-affected segments

Table 4 in the paper shows that observed speeds at traffic count stations on the affected segments (where speed limits increased) increased by less than the observed speeds on non-affected segments. This is true for mean, median, and 85^th percentile speeds, and it seems inconsistent with a theory that the increase in speed limits led to higher actual travel speeds and therefore to more adverse outcomes.

Shortcomings in statistical modeling

The authors used linear regression models with the dependent variables being the average rates of events per month per million residents. This is a curious modeling choice that gives up much of the richness in the original data, which could have helped to identify the effect of the speed limit increase more convincingly. When dealing with rare events like crashes, a more conventional approach would be to use a count regression model such as a negative binomial model. While a linear regression model might be acceptable in this case, the authors offer no explanation of why they chose a linear regression over a count model.

The aggregation of data gives up a great deal of granularity, which could have provided a much more convincing analysis. Instead of aggregating the crashes to all affected and all non-affected segments in the province, segments could have been treated as separate units of analysis. This would have exploited the variation in speed limits both across time and from segment to segment in order to identify more convincingly the causal effect of the speed limit increase.

An additional deficiency of the paper is that it does not report the full results of the statistical model; it only reports the estimated treatment effects of the speed limit increase. However, to properly assess the quality of the analysis it is helpful to see the full model results. In addition, the authors did not report any robustness checks to help establish that their results are not merely an artifact of the particular model specification that they chose. It would have been better to show how the estimated treatment effects and other coefficients in the model depend on which covariates are included, and on certain key assumptions such as the distance buffer discussed earlier.

Summary

Overall, this paper contains a number of red flags that demand further explanation. While none of these necessarily imply that the speed limit increase was harmless, they do cast serious doubt on the estimated impacts reported in the paper.