Ethical Challenges and Professional Responses of Travel Demand Forecasters

P. Anthony Brinkman, a PhD candidate at the University of California, Berkeley, produced a very interesting dissertation on this issue in late 2003. The complete document can be downloaded from the UC archives.

It is a fairly dense dissertation of nearly 200 pages, with much discussion, survey/interview data and references. Its density and length should not be allowed to detract from its interest as a reminder of the pressures that can be placed on the people who produce forecasts and the ethics that they must maintain.

The archive contains a short review which may make the findings more generally accessible.

My own comments (from the review) on Brinkman's findings are as follows.

technical issues are dismissed

The thesis assumes that the reasons for bias are not technical, and it does not address technical aspects of forecasting. Its perspective is that forecasts are consistently biased, and therefore the people who produce them are deliberately distorting the truth.

Technical limitations of modelling are still a matter of public and professional debate in Australia, and most practitioners believe that the quality of their forecasts will increase over time as continuing technical deficiences are remedied. Brinkman might dismiss this as an "excuse" but it may be a better explanation for any bias than that the modellers are crooks or do not know what they are doing.

North America is not necessarily Australia

The dissertation was based on a survey of and interviews with North American, mainly US, practitioners. A similar exercise in Australia might not produce the same findings, because of:

• our different cultural background
• our different institutional structures
• our different transport policies
• lower resources devoted to planning in general and modelling in particular, requiring more ingenuity and innovation than in US
• the different composition of our modelling pool
• the relatively strong influence in Australia of British rather than American modelling practice

significance of British influence

The basis of transport modelling was developed in the United States in the 1960's, and models conforming to the basic "four-step" modelling structure have been developed and applied all over the world. This is not to say that they are all the same.

Mode shares to public transport are much higher in Britain than they are in North America. British modelling practice displays much more experience and interest in estimating demand levels for public transport both in isolation and in a multi-modal context.

Australian cities are less car-dependent than North American ones, though more so than British/European cities. Thanks partly to the influence of this British experience, and also to the absence of federal largesse as a prize for the most optimistic forecasts, Britain/Europe tends to produce more realistic forecasts for transport projects than those reviewed by Brinkman for North America. Significantly, Table 2 showed a greater proportion of "similar to actual" forecasts for Europe (47%) than for North America (9%).

the reducing opportunity for peer or public scrutiny of modelling

The habit of both public and private sector clients of requiring modellers to sign legally enforceable confidentiality agreements before modelling commences might inhibit investigation here.

standards, practice guidelines and quality assurance

The difficulty of assessment of the quality of demand forecasts has long been recognized as a barrier to credibility in Australia. The UK in particular is more advanced in this respect than we are.

There is a lack of "Practice Manuals" or "standard guidelines" for modellers in Australia. (In contrast, the UK has quite extensive practice manuals already). Without such professionally endorsed standards and practices, quality assurance for modelling forecasts becomes impossible. Hence the establishment of both standards for practice and reporting/checking procedures are essential precursors to adequate quality assurance.

Another issue is the lack of range (confidence intervals) often attached to forecasts, with a resultant obsession with comparing single forecast numbers rather than ranges. Remedying this could be incorporated in any "practice guidelines" developed as above.

how good are our forecasts anyway?

Brinkman noted: There appears to be little research on the accuracy of forecasts.

This is true in Australia also. One inquiry (the Advisory Committee on the relocation of the Hume Freeway between Craigieburn and the Metropolitan Ring Road) said in 1999:

A common feature of many environmental assessments is the doubt which submittors seek to cast on the reliability and accuracy of traffic forecasts. In the light of the importance placed on traffic volumes by VicRoads ... the Committee finds it surprising that so little attention is apparently paid by VicRoads to monitoring the outcomes of its traffic predictions.

(VicRoads is the state road planning agency in this case, but the point is a general one). An ex-post investigation into the accuracy of forecasts made in Australia 10-20 years ago would be a significant addition to our understanding of forecasting bias, if any.

how to make progress in Australia

There seems to be a prima facie case in Australia for an independent professional organisation to take up the issues of training and general credibility of modellers that Brinkman raises, and quality assurance for modelling. This would improve both professional standards and the potential policy impact of modelling, the two factors whose poor state Brinkman identified as leading to disillusionment and complacency in North America.

The National Committee for Transport of Engineers Australia may be an appropriate body to do this for Australia. This will be investigated further over the coming months.

January 2004

Back to "Planning Methods" index