For a handful of years now, the benefits of augmented reality (AR) have been widely communicated across the leisure and entertainment sectors – but for the rail industry, harnessing these capabilities is still a new and exciting prospect. Quadrant Transport talks to some of the leading public and private sector minds behind bringing AR to the railways
Expected to eclipse $160bn as an industry by 2023, AR has rapidly become one of the most prominent emerging technologies in the business and leisure sectors. Whether it’s using specially designed headsets to overlay images and visual stimulants onto real-world environments, or going one step further to completely immerse users in those simulated environments, the potential uses for virtual and augmented reality really are boundless.
Most recently: AR is being utilised in applications such as the Healium AR app helping military veterans to manage anxiety; US magazine The New Yorker is dipping its toe into the AR waters, using the technology to bring comic strips to life; and tech giant Apple plans to release an AR headset in the middle of next year, encouraging ‘third-party brand collaboration’ and working with developers to bring innovative solutions to the market. Rival companies are also identifying AR as an area that many industries will look to invest in during 2020, with titans such as Google, Microsoft, Samsung, and Oculus VR all developing their hardware to compete in the burgeoning industry.
Outside of the leisure sector, although slow on the uptake, industries have been embracing AR capabilities in the construction, engineering, and architecture fields, to name just a few – and no more has this recent wave of innovation been felt than in the rail sector. Earlier this year, the RSSB released a brief report on the current uses of AR in rail, on the industry’s readiness to roll out AR more comprehensively, and where the sector could improve in adapting to emerging and game-changing immersive technologies.
But some scepticism remains regarding AR’s use in the UK rail industry, with the report arguing that the roll-out of immersive technologies “should be evaluated on a case-by-case basis” for effective implementation – and noting that despite the Technology Readiness Level of AR being at the highest level, Level 9, the Rail Industry Readiness Level has only been rated as a 7. The RSSB claimed that AR puts a demand on device processors due to the number of components that it relies on, such as displays, communication links, cameras, and speech recognition – and acknowledged that devices such as smart glasses may lack 4G/5G connectivity, hampering their capabilities in areas without access to Wi-Fi. But the organisation did emphasise that collaboration over common AR platforms can take advantage of economies of scale and potential standardisation, and thus could result in a smoother transition to onsite AR technologies.
Capturing imaginations
AR usage is being expanded in the rail division in some cases, however – most notably in the benefits it can provide to the recruitment and training of the next generation of rail professionals. In March this year, Innovate UK and the DfT fully funded a FOAK AR Stations Digital Twin project led by digital tech SME PAULEY. The new partnership between HS2, the National College for Advanced Transport & Infrastructure (NCATI), Inventya, WSP, and PAULEY pioneered the use of AR to train staff members at Old Oak Common – one of the high-speed line’s flagship stations connecting the southeast with the rest of the UK, and destined to become a UK “super hub” serving both HS2 and the Elizabeth Line. Though Old Oak Common is yet to be completed, through AR team members are able to visualise the future station and its innerworkings in absolute detail, allowing them to deliver greater customer experience, station operations, and safety.
After seeing the AR technologies used in action, HS2 team members noted wide-ranging benefits of the technology applying to both stations and the linear asset, from a maintenance and asset management perspective. Looking beyond just HS2 assets, observers could see the technology extended to areas including Rapid Response, competence and familiarisation, defect rectification, and reference to technical documentation.
Taking a holistic view, the rail industry shifting towards increasingly utilising AR solutions is a welcome one for Philip Pauley, PAULEY’s founder and director. For Philip, the untapped capabilities of AR and digital twinning for the infrastructure industry will be a heavy focus both for major tech companies and the rail industry in the UK in the coming years. “We are pioneering digital twinning through AR,” he told Quadrant Transport. “Big tech companies are pouring millions into wearable AR headwear with a vision to empower 1.4 billion frontline workers to drive design, construction, and operational efficiencies with this technology. This holographic functionality facilitates all levels of ability and software knowledge to collaborate in multiple workspaces (office or onsite) for accelerated upskilling, safety, and knowledge share.”
Philip went on to note that Building Information Modelling (BIM) seems to have limited collaboration opportunities, which has led to a land grab or ‘God Complex’ within project teams; according to him, neither the client, infrastructure owner, operator, or supply chain is being informed on best practice.
Bringing through innovative solutions to the rail market has been extremely challenging for some. The ethos behind PAULEY’s technology applies the Gemini Principles – a set of criteria establishing common values and goals for the effective management of real-time assets through digital twin technologies, including value creation, accelerated learning, enhanced safety, security, and clear ownership and regulation – and is something heavily supported by the digital firm’s founder. Philip called on national policymakers to directly engage UK SMEs to streamline that process and encourage private-public collaboration on the railways, as well as to reduce the influence of global tech corporations in a bid to bring through and catalyse some of the frontline innovators like his company.
Learning to better engage with SMEs
With many years of experience in the industry – his company boasts a history of industry firsts in bringing immersive technology to various parts of the rail sector – the PAULEY owner welcomed efforts by the DfT and Innovate UK, for example, in bringing PAULEY, HS2, WSP, and NCATI together. But he argued most major UK organisations like Network Rail “still can’t engage with SMEs working on innovation directly.”
“We were strongly advised to partner with Tier 1s on tenders and frameworks,” he explained. “I believe the UK has always been an innovation powerhouse, but innovative UK SMEs need to be nurtured and supported within the UK procurement process,” Philip added, believing that a greater commitment to support the small business market can lead to more “success stories, attract the best talent into the industry, and generate more export opportunities” within the UK rail landscape.
He continued: “Ironically, even after being invited to join the bid team which won a multimillion-pound procurement, we have been inexplicably dropped from the new ‘alliance’ and have been asked to apply as a supplier. Having spent the last two years aligning to Tiers 1 for tenders, and having been on the winning team for CP6 track renewals and subsequently both the Design Services Framework and StarTwo Framework as part of the supply chain with other partners, it has now become very clear there is no guarantee we will be given any work as a result of all our effort.
I believe the UK has always been an innovation powerhouse, but innovative UK SMEs need to be nurtured and supported within the UK procurement process
“What’s more frustrating is having now built up a world-leading AR development team, we are having to start to disband our young talent in the face of a very uncertain future in the rail industry.”
Despite these pressing issues, Philip did praise Network Rail for its renewed engagement with smaller companies and highlighted its SME Action Plan as a key milestone. The infrastructure owner has also appointed SME champions relative to the new UK rail routes, an initiative which Philip sees as beneficial to the SME market. “This seems like a really fantastic initiative which I sincerely hope will safeguard SME innovators and help us identify the right route to working with Network Rail and its designers and contractors,” he said.
It is also hoped that bringing innovations like augmented and virtual realities to the classroom and workplace can spark imagination and aspiration for future engineers. It’s well known by those in the industry that the engineering sector could be facing a dearth in talented and ambitious engineers coming through the ranks in the coming years. That sentiment was echoed by the UK Government’s 2018 engineering study, which found that for the engineering sector to gain enough candidates to curb the skills shortage, a massive 186,000 recruits will be needed to fill jobs each year until 2024.
For the PAULEY founder, would bringing through these digitally-savvy young professionals make a considerable positive impact on major projects like HS2? “I would definitely say yes! Spatial computing through wearable AR is the future of industry today,” he commented. “Early adopters and pioneers may fall by the wayside, but both HS2 and NCATI identified early on in the project that a new wave of digitally-skilled engineers would be needed to support the whole of the UK infrastructure supply chain,” Philip added – ensuring that UK PLCs stay at the forefront of digital skills and infrastructure against budding foreign industries.
Bringing AR to the commuter
For passengers, developers and transport planners are unlocking AR capabilities by allowing customers to visualise what new developments might be on the way to their local rail station. In October, Network Rail – in partnership with Wood (a company providing solutions for the delivery of project, engineering, and technical services) and AR smartphone application ARki (developed by Darf Design) – made one of the first steps into the AR world by installing design data capable of allowing passengers to see replacement footbridges at stations through an app.
Having developed three different footbridge designs that blend, in the words of Network Rail, “forward-thinking architecture with creative engineering,” prototypes dubbed The Beacon, The Ribbon, and The Frame will give users unique insight on how new installations in their local stations could look in a 1:1 scale, allowing travellers to give feedback and offer suggestions on station feature design without the need to wait for laborious architecture draw-ups of how a new footbridge may look. Aligning with Network Rail’s drive of ‘Putting Passengers First’ announced earlier this year, planners behind the development are hoping the new tech capabilities can keep the shrewdly knowledgeable local passengers in the loop and help introduce more community-led features in their stations.
Anthony Dewar, professional head of buildings and architecture at Network Rail, said the collaboration with the app is part of a wider incentive to elevate passion and interest in design across the rail industry. “We have a number of strands of work underway to raise the importance of good design, get the foundations of what good design looks like, and how we’re going to monitor and improve the design across a railway’s estate,” Anthony told Quadrant Transport. “So the AR app has multiple uses – it allows us to showcase to the public the new work that we’re doing, and it’s something that we believe has not been done before. If it has, then it hasn’t been done particularly well.
“It’s a way of bringing together infrastructure design that affects millions of people around the UK on a daily basis, and showing to the travelling public what the future of their passenger journey is going to look like.”
Frank Anatole, principal architect at Network Rail, stressed the potential benefits for every single stakeholder in the railway. “As architects we do tend to visualise in 3D, but those visualisations are more in the form of animations, so you would have to have a computer to run them to show to the client,” he clarified. “This is a way of actually having control of an animation, in the same way I see it, and then putting that into the hands of a project manager, passenger, client, or sponsor, and inviting them to walk around the site with the model in whatever device they’re carrying and actually be part of the animation. I think that’s quite a powerful tool.”
When Network Rail is installing infrastructure across the country – around 20,000 miles of track, 30,000 bridges, tunnels and viaducts, and thousands of signals and stations – a lot of the designs “do not get past 2D” on a drawing, with actual visualisation proving difficult to some until the full structure is created, Frank noted. Earlier knowledge of what works and what might not can improve final customer satisfaction and potentially reduce the cost of man-hours for both Network Rail and the wider supply chain working on a project. “We really want to get to the point where we’re able to review projects at a very, very early stage, and visualise that in an early stage. I see this as helping us in a way to do this,” he said. “The hardware has been waiting to catch up with it, but I think we’re getting there, so I’m quite excited by it.”
It’s that new thinking around interconnected urbanisation for sectors working together, and transport is quite frankly one of the key ones, because it physically connects all of those sectors together
Compared to other industries around the world, the AEC (architecture, engineering, and construction) sector ranks as one of the slowest industries to adapt to digitisation and digital technologies. The 2018 JBKnowledge annual construction report, a US-based construction survey with 2,825 respondents, found that a sizeable 42% of the companies surveyed did not have a defined R&D process – and amongst those that did, a third assign this budget to existing staff and resources rather than dedicating them full-time to R&D. Many are pitting AR as the potential solution to allow companies to see value in committing to investment in digital construction services.
For the designers of the new footbridge feature on ARki, AR can improve project delivery and enhance the mobility sector far beyond the current projects being worked on. “Because we’ve operated in all these different sectors, from defence and nuclear to transport, our own business team strategy is to tie all those sectors together. That’s kind of been the catalyst behind looking at engaging with passengers in the transport sector, because happy passengers using an effective transport system are able to get to work and live in the area that they want to live in,” said Mark O’Connor, of Wood.
“It’s a whole connected work and connected environment, and it’s across those themes that we see the need to bring this technology forwards – because it’s a new way of thinking about something, and you need a new technology to better engage that thinking. It’s a very complicated and interconnected world. It’s that new thinking around interconnected urbanisation for sectors working together, and transport is quite frankly one of the key ones, because it physically connects all of those sectors together.”
But what is it specifically that has led to this piqued interest from the construction industry into what AR can offer? Sahar Fikouhi, founder of Darf Design, explained that the technology itself has been around for some time – Darf Design has been making AR applications for around a decade now – but it’s only since Apple and Google released their AR developer kit in the past two years that app developers have been able to take the technology to the general public.
“Before Apple and Google released their AR solutions, it was kind of impossible to take models onsite and do 1:1 scale AR,” Sahar said. “It was mainly limited to image tracking, and so we did a lot of work where image tracking was blown up to a 1:1 scale, but that’s not always suitable when you go onsite. It’s only really been since the launch of Apple’s AR kit over the last two years and Google’s AR core that has allowed developers to make huge leaps with the capabilities of AR.”
Bringing the application to the smartphone landscape will do wonders in terms of connecting with the younger generation, too. Sahar explained that the app is being used by thousands of architecture students, who have praised its ability in helping them understand the intricacies of a building’s innerworkings.
There’s no age limit on who can benefit from the learning experience, as Mark noted: “From Wood’s side, we are heavily invested in STEM education and I would think something like this is no longer a ‘nice to have.’ It should now be a mandatory activity that must be taken, in order to engage with the future professionals that we want to attract to Wood and attract to the industry.
“My wife has gotten bored of me playing with this app with my son, who’s three years old,” he joked. “We’re running it on the big screen with the iPad Pro, where you drive a little matchbox car through my personal favourite, which is The Frame.
“That’s with a three-year-old all the way up to the very top end of Network Rail – the chief executive, the chairman, and designers onsite now saying, ‘This is incredible.’ We can actually show the general public very, very quickly the potential this could have at their railway station, whereas before it could have at times taken up to half a year to see images of what’s going to change at a station.”
Ultimately, with passenger in mind, success for Network Rail would be to see a spike in customer satisfaction as part of this new community-led approach to working on stations. “Upgrades such as putting a footbridge in a station are often the biggest changes these smaller railway stations have seen since they were built,” Anthony explained.
“Therefore, they can drive increased socioeconomic benefits with increases in housing and people’s quality of life. These footbridges, we believe, are far more diverse and inclusive as a piece of great design, and they encourage much greater accessibility to the railway, which then encourages much greater usage of the network and much better connectivity between human beings.”
With the app receiving positive feedback from “99.99%” of users, as Anthony put it, the development of AR could do wonders to improve an at-times fragmented relationship between passenger and rail supplier and encourage local interaction with the rail sector. For Network Rail, bringing in innovations like the ARki app will be a common theme for the industry going forward.
Crunching costs
Providing AR-led solutions to a variety of projects in the rail industry, developers behind the Enable My Team platform have been utilising AR in a variety of formats to benefit the rail supply chain. Having worked on Crossrail, HS2, and more widely on the River Humber pipeline, perhaps the most exciting project has been leading an Innovate UK project focusing on predicting and preventing faults before they occur.
Working on a two-year research project in collaboration with Costain and the University of the West of England, iRAMP (IoT-enabled Platform for Rail Assets Monitoring and Predictive Maintenance) seeks to address the significant challenge of infrastructure faults and the strain it can have on a workforce. Using machine learning to anticipate potential faults before they occur, the team leading on the project believe AR can provide cost savings in reducing man-hours and the number of staff out on track.
“One of the focus areas for Network Rail is to reduce boots on the ground. It’s to get access to information much quicker for the network as a whole, so you can perform predictive analysis and predict or spot faults quicker than how it’s done now,” commented Sandeep Jain, CEO of Enable My Team.
Through developing automated asset recognition from high-resolution imagery for rail assets, and by introducing data analytics into the vast datasets collected by Network Rail (via satellite imagery, weather forecasts, and trackside sensors, amongst others), the team behind iRAMP can identify trends and faults before they occur. Sandeep said Network Rail is a leader in collecting track and trackside information, noting that the sheer amount of information collected is “astounding.” “We have information on track, lots of sensors all over the place, specialised guiding instruments for tunnels and OLE, and helicopters with lasers capable of performing specialised scans, so we are certainly ahead of other countries in that regard,” he added.
For Sandeep, maintaining and inspecting OLE is a key example where AR provides value. According to him, cameras and lasers mounted on the train roof can monitor the contact point of the contact wire, the health of the pantograph, and the force at which the wire is being contacted. “Where does predictive maintenance come into this picture? Obviously, what you want to do is spot problems and compare the contact wire thickness from two months to now, observe how quickly it could be degrading, and say, ‘Okay, that needs to be looked into because it’s ready for replacement,’” he explained.
One of the focus areas for Network Rail is to reduce boots on the ground. It’s to get access to information much quicker for the network as a whole, so you can perform predictive analysis and predict or spot faults quicker than how it’s done now
“But this information only informs Network Rail that there’s a potential maintenance issue. When getting to the actual maintenance of the wire, where AR kicks in is when someone goes onsite. AR technology with the right hardware peripheral has the potential to inform staff about all data, analysis, and real-time information in the field, on demand at the precise location, by merging it with reality.
“Staff can be deployed in higher-priority areas, hence increasing their effectiveness. They are also in and out of a repair job quicker, since relevant information about the defect and its precise location is projected onto their AR peripherals. In my view, that’s where AR can really bring all the benefits of the data science and automation and be helpful for people doing maintenance and repairs.”
One of the prototypes the team is working on harnesses the Microsoft HoloLens, using real-time sensor data to be displayed on the maintenance team’s headsets, therefore allowing inspectors to assess an area of track, for example – with the ability to see real-time data in their headsets without the need for paperwork or tablets. “That’s how we’re starting to use some of the AR capabilities in that project with the maintenance teams,” Luke Holbrook, digital delivery manager at Enable My Team, pointed out. “Obviously, [the application] could be much wider in the rail industry as a whole; in terms of design teams, passenger experience, training simulations, that sort of thing – but at the moment, our main focus is more around the maintenance side of the sector.”
Looking more expansively at how the UK rail industry fares in its adaptation to emerging technologies like AR, Sandeep believes more could be done: “Where we are lacking, I believe, is processing and understanding that information quickly and effectively. Also making that information available to the right people in a collaborative fashion – and that’s where I feel a lot of progress needs to be made.”
Luke called on greater industry collaboration to create a consistent stream of data for developers, highlighting that because the UK railway system is one of the oldest in the world, new technologies marrying up with sometimes decades-old signalling equipment can lead to peaks and troughs of data coming through to maintenance teams – therefore potentially hampering the overall efficiency of maintenance works.
“It’s one of the most complicated railway systems in the world. Some of the equipment that we’re putting in now with electrification is brand-new, cutting-edge, has sensors and monitoring, but we still have signalling equipment which is decades old and there’s no way of capturing data from those,” elaborated Luke.
“Having said that, Network Rail does have world-leading technologies in terms of some of the maintenance equipment. Some of its maintenance trains have some really cutting-edge equipment which is a world-first, so we are very good at capturing that information. It’s now a case of collaborating and getting it all into one platform so that actually, whether it’s a five-year-old asset or a 100-year-old asset, we still have similar levels of information on both and can therefore do our maintenance on it.”
When you take a step back and consider the inroads that the commercial sector is making to introduce AR-led solutions to the rail industry, it’s easy to get caught up in the grandiosity of the technology and its many bells and whistles – but the consistent theme throughout all of the innovations brought to the market is the focus on the customer, the end passenger. Whether it’s educating station team members of the intricacies of new stations to help provide prompt and helpful information for passengers, or encouraging users to visualise new developments in their station and reducing maintenance and inspection times to reduce closures of lines and keep people moving, AR is here to stay.
The $160bn industry may be solving some major global issues in its time in operation, but it’s these hyper-local uses of AR that can really make the difference.