Level crossing
signs and signals
Sawbridgeworth Level Crossing by Therofr, CC BY-SA 4.0 , via Wikimedia Commons
01 The mission
Since the introduction of level crossings, signs and signals have evolved and proliferated. There are now a large number of signs that the road user may be confronted with as they approach level crossings.
Road signs and signals are an important way of controlling risk at level crossings, because road user behaviour is a key source of risk at these locations.
AIM
This project aimed to evaluate, from first principles and from a road user’s perspective, the designs of the signs, signals and markings placed on the road approaches to level crossings. Where design improvements may reduce road user errors, the study aimed to develop and evaluate new signs.
While the shop included the full range of signs, signals and markings, this case study will focus only on the advance warning signs and the light signals at level crossings.
MY ROLE
I took on the role of Lead Researcher for this work
-
Led the design of the methodology and production of the winning proposal document
-
Carried out the literature review
-
Developed the study plan, briefed a small team and led the collection of data
-
Commissioned and worked with a design agency to come up with and refine new sign designs
-
Took an active part in data collection activities
-
Developed the analysis plan and undertook quantitative analysis
-
Wrote research reports
-
Led meetings with the client’s project steering group: representatives from road and railway companies and authorities
-
Worked with the project manager to provide the client with weekly project updates and monthly progress reports
I changed role, moving client-side, before this piece of research was completed, meaning the final phases of research and final reporting are not mine.
02 The game plan
A mixed-methods approach comprising:
-
Review of scientific literature on the design of information and warnings, incident reports, level crossing and sign design Standards - to build on existing knowledge from rail and other domains.
-
Observation of road users at different types of level crossing - ethnographic approach to
-
Experience different types of level crossing & how design standards are implemented.
-
Describe road user behaviours (which signs and signals aim to influence) at level crossings.
-
Describe the context in which the signs and signals sit.
-
Record local factors that could affect road user behaviour and performance.
-
-
Postal questionnaire of 200+ road users to gather an initial indication of sign comprehension (based broadly on ISO9186-1:2014)
- Online survey of 14,000+ road users to quantify their comprehension of two traffic signs
-
Lab-based eyetracking trials to record and describe patterns of visual behaviour in relation to signs and signals, and highlight any issues with their conspicuousness.
-
Descriptive statistics and hypothesis testing, including t-tests, ANOVA. A range of non-parametric statistical tests where the data did not meet the assumptions of t-tests.
Tabulated task, human error and information needs analysis
-
to bring together many strands of the research
-
to provide an overview of how current level crossing design supports the cognitive processes of sensation, perception, comprehension, decision making and action
Synthesis workshop
-
to list the gaps and problems with the current sources of information based on the research
-
to triangulate evidence from different strands of the research to identify stronger conclusions
-
Work with a design agency to generate a range of ideas (diverge)
-
Converge through user reviews (20 road users), including a short questionnaire to report initial comprehension of the designs.
-
Face to face survey of 200 road users to measure and explore comprehension of the new designs and old counterparts (based broadly on ISO9186-1:2014)
-
A computer-based choice reaction time task to assess the conspicuousness, visibility and priming effect of the designs. 61 participants, 363 images, of which 121 included a level crossing
-
Virtual reality trials to gauge road users' situation awareness (SAGAT methodology) and comprehension of the final designs. 120 participants.
-
Desk-based assessment (by relevant experts) of each measure, including cost, engineering feasibility and legal and standards issues.
-
An overall assessment for each promising measure which took into account user behaviour, cost benefit, wider implications and safety considerations.
-
A stakeholder analysis was conducted to identify those parties that any of the measures would affect, and stakeholder interviews with representatives from those groups,
-
Soft Systems Methodology to characterise how the changes would affect stakeholders, and build a road map for implementation of the measures.
03 The world 'as is'
IT'S A COMPLICATED WORLD FOR ROAD USERS
There are many different types of level crossings.
Many have lights that flash lights and barriers that come down when a train is approaching.
Others have lights only (no barriers)....
...or a gate but no lights.
Still others have neither lights or barriers, so road users look left and right to check for trains and cross if it is safe.
ARE LEVEL CROSSING LIGHT SIGNALS ANY GOOD?
The subtext here is: wouldn't we be better off replacing these with standard traffic lights?
Level crossing signals (also called wig wags) need to be different to standard traffic lights because they have a different meaning. Emergency services cannot legally cross a red wig wag aspect but they can cross a red aspect at standard lights.
The design of the wig wag has advantages. There is a dynamic element which improves how conspicuous they are, and road users are able to read the aspect through their peripheral vision.
Although people might be less familiar with wig wags than standard traffic lights, the comprehension survey showed that the wig wag stop aspect was well-understood. Whether the signals were presented on a white background (out of context) or within a level crossing scene (in context) - all respondents said that the flashing red aspect means stop.
Research conclusion
This research found no reason to redesign the light signals.
WHAT'S THE DEAL WITH THE WARNING SIGNS?
Problem 1: There is a weak link between stimiulus (the sign) and response (desired behaviour from road users)
The sign used at each crossing depends on whether the crossing has a barrier. The road user’s task sequence differs not according to whether there are barriers at the crossing, but according to whether they have to look up and down the track for trains. So, the relationship between the stimulus (sign) and desired response is weak.
Sign for crossings with gates or barriers
Sign for crossings without gates or barriers
Problem 2: Many road users can't identify the difference in the meaning of the two signs
We asked over 16,000 people from an online survey panel about the meaning of the two level crossing advance warning signs.
If the difference between the signs was well-understood you'd expect the orange (left hand) bar for 'Barriers' to be at zero and the blue (right hand) bar to be at 100%.
But instead they are much closer to each other, at 17% and 38% respectively, meaning the difference between the signs is not well-understood.
Problem 3: These signs don't seem to draw the eye as much as many other signs at level crossings
We analysed eye tracking data from 286 approaches to (virtual reality) level crossings or road junctions.
We recorded whether or not participants looked at each of the signs in the moving scene.
The level crossing warning signs were among the worst performers - people didn't even glance at them once in half of the scenes.
Research conclusion
These three issues supported our decision to design a new advance warning sign for level crossings.
04 Design objective
Create a new design for a single, universal, advance warning sign to be used at all level crossings. Follow:
-
Guidance on the design of visual information (detailed in the literature review report)
-
Current traffic sign design conventions (Chapter 7 of the Traffic Signs Manual)
The new design would be intended for use at all level crossings and other signs that tell users how to use the crossing will remain in place.
05 Redesign
ROUND 1 - CASTING THE NET WIDE... AND BRINGING IT IN
"To me it says ‘train’, whoa, stop"...
"It’s a lot more up to date with the design of modern trains. It has a lot more meaning to it."
"Looks like a tram"
"My first impression was that the train was coming towards me, and I thought am I standing on the tracks?"
...
"I can’t figure what that means. Does it mean one way across the track, because it’s an arrow going one way across the track? Don’t stop on the other side of the track?"
....
"It’s a steam train."
"But it might not be a steam train that passes."
"It says chance of collision, and that actually grabs me."
"I don’t like the car. You’re already sitting in a car, I don’t need to know that, I just need to see the train."
Round 1 Verdict
Keep in the running. Look at ways to make the pictogram look more like a modern train (eg change light configuration)
Discard. Scored poorly in the ratings. Qualitative data is also not favourable.
Retain to compare proposed designs to the current sign. Comments suggest it does not accurately represent the hazard at level crossings. Resulted in the lowest hazard ratings.
Keep in the running. High hazard ratings. Mixed comments. Play with the orientation, giving side-on view of train. Look at the impact on risk ratings of depicting a collision.
ROUND 2 - DO ROAD USERS UNDERSTAND THE DESIGNS?
About this graph:
We asked interviewees to describe what they thought each sign meant. This graph shows how many referred to a level crossing.
What the data means:
Signs A2 and A3 were the most likely designs to elicit references to a level crossing ahead.
Sign A5, a sign that is in current use, was least likely to be associated with a level crossing.
About this graph:
Interviewees were asked to rate the level of risk they thought each sign implied on a scale with 1 being no risk and 10 being extreme risk.
What the data means:
All of the differences in risk ratings were statistically significant, except for A2 and A4.
Sign A3 was associated with the most risk, followed by signs A2 and A4.
Sign A5, the current gated level crossing sign, was associated with the least risk.
Half of the participants saw the advance warning sign with a white background and half of the sample saw the sign with a yellow background. T-tests showed that a yellow background had no effect on risk perception.
About this table:
At the end of the session participants were told that each of the signs were intended to mean the same thing; that they were advance warning signs for level crossings and light signals and gates or barriers may or may not be present. Participants were then asked to rank how clear and understandable each of the signs were.
What the data shows:
Participants ranked sign A3 and A2 joint first (the differences between these two signs were not statistically significant).
Round 2 Verdict
-
Designs A2 and A3 are the most promising in terms of comprehension.
-
The additional ‘collision’ element depicted on Design A3 would make the sign less legible if implemented on the road.
ROUND 3 - DO THE SIGNS PRIME ROAD USERS TO EXPECT A LEVEL CROSSING?
Round 3 Verdict
When compared with the current signs (A5 and A6), new sign A2 performed neither worse nor better in priming people about the presence of a level crossing, even though participants would have been more familiar with signs A5 and A6.
The car in Sign A2 looks old fashioned and we should update it before the final round of trials.
ROUND 4 - DO THE FINDINGS HOLD IN A MOVING ROAD SCENE?
-
120 participants
-
Each saw one approach to a level crossing.
-
Each video paused 4 times and a blank screen presented. After each pause, the video was played from the beginning. The advance warning sign was visible before the 2nd, 3rd and 4th pause.
-
Situational awareness tested (SAGAT methodology)
About this graph:
This graph shows the % of participants who recalled seeing the sign after the 2nd, 3rd and 4th pauses in the video. Each participant had one session and only saw one video, so the % of sessions is the same as the % of participants.
What the data means:
Participants’ awareness and recall of the sign was high, particularly in the third and fourth video presentations.
Almost all those who said they saw the sign said they thought it was a warning for a level crossing (96% after the second pause and 97% after the third and fourth pauses).
Round 4 Verdict
In a moving scene, this sign delivers high levels of recall and comprehension. It is a strong design that should be considered for implementation.
06 Handover
Outputs:
A series of research reports including
-
The final design
-
The data and a narrative of the anlaysis
-
A feasibility assessment of each measure, including cost, engineering feasibility and legal and standards issues
-
A stakeholder analysis
-
A road map for implementation of the measures
Impact: Following an open consultation in 2022, the UK Government is now considering adopting new level crossing signs.
07 Reflections
HOW DID THIS PROJECT CHALLENGE ME?
-
I enjoyed being hands-on with all of the tasks but I needed to protect some of my time for planning, management and technical leadership.
-
The client’s project steering group comprised over 15 people representing different interests, including the rail infrastructure owners, level crossing managers, rail regulator, Department for Transport, interest groups for hauliers and many others. It was hard to reconcile views.
-
Scope creep was a real risk during these meetings, and given the complexity of the project and research aims.
-
Having confidence in a robust methodology and being able to express how the evidence would address their issues was critical in preventing the project from being hijacked by dominant opinions.
-
While I felt more comfortable with hard, quantitative data, this study was better suited to a qualitative approach overall, but with some aspects quantified through questionnaires and trials.
-
I came up with the idea of using a task analysis as the basis for systematically recording, organising and tracking our findings.
WHAT COULD I DO BETTER NEXT TIME?
-
Some stakeholders are more easily persuaded by data, and others by seeing users approach the task. I would potentially look to include more in the method to support the latter - perhaps recording in-vehicle videos as people use level crossings.
-
Eye tracking: We’d be in a better position to estimate the effort for manual coding of eyetracking videos. However technology has moved on and can automate much of the analysis.
-
I would look to allocate more budget to the initial comprehension questionnaire so that it could be conducted face to face and responses could be probed.