Construction Site of the Future

IoT Computer Vision Demo Lab Setup

Construction Site of the Future

Capturing the Correct Use of PPE

The COVID-19 crisis has led to many headlines about PPE (Personal Protective Equipment). We have also had cancellation of a number of events, including AWS Summits, where we had intended to demonstrate our work around PPE. Our customer Balfour Beatty had a Strategic Supplier Conference, which was also postponed. At this event, we were due to unveil a computer vision demo showing detection of PPE in construction, e.g. safety jackets and helmets. We built the solution in AWS, working with their leading computer vision partner ADLINK. ADLINK Technology specializes in edge computing hardware and software solutions, with a mission to be a catalyst for industry, powered by AI. We describe in this blog our experience creating the PPE Detection service, with a focus on deploying models to the Edge. Edge computing is the concept of running software directly at the location where it is needed. The global edge computing market is forecasted to reach 1.12 trillion dollars by 2023, according to this Forbes report. We believe  the technology is mature and the time is right to roll out these kind of initiatives across the construction industry and others. We believe this work can promote correct PPE usage, and thus reduce the risks of illness and injury.

Building the Computer Vision model

Our main focus in the project was to design, build and train a computer vision model. For this, we chose to use Tensorflow. Balfour Beatty supplied us a selection of PPE they use. Then we collected our own static images at first to use for training. This initial set of images were labelled with bounding boxes around the three classes of objects: people, hi-visibility jackets and helmets.

Data Labelling

Data Labelling example bounding boxes

Next, we augmented the images by flipping them, cropping, blurring and applying colour adjustments. The aim was to increase the number and vary the quality of training data to simulate demo conditions. This helped to eventually provide a set of a few thousand labelled images. These were used to cross-train a model that was selected for its ability to already recognise people.

Rapid Architecture Setup

We based our AWS Cloud architecture on the Inawisdom Rapid Analytics & Machine-learning Platform (RAMP). RAMP provides a secure and structured analytics working environment very quickly. We mainly used a single SageMaker instance for this project to access the training data from an S3 bucket and develop the model.

PPE Detection Demo Architecture

PPE Detection Demo Architecture based on RAMP

The Left hand side of the architecture is an optional extension of RAMP deployed on premises. It shows the ADLINK inference engine and the NVidia-based Neon-i smart camera. This equipment would be installed within an industrial environment in the real world. ADLINK provide the connections between sensors and devices, enabling vision data to flow. Specifically, the solution relies on the ADLINK Edge™ software platform and ADLINK Data River™ technology. The next section covers these components in more detail. Our other component on the AWS side is the model converter, which automatically takes the output from SageMaker and converts it to a run time inference engine that can be deployed. We took a rapid prototyping and iterative approach to model training and optimisation. Our best performing model reached over 90% accuracy of identification of helmets and jackets on trials conducted so far. This video shows pure object detection, note how the boxes expand and contract and occasionally disappear as the helmet is rotated upside down (which is expected). We can also add logic to ensure the helmet is actually worn on the head, not being carried.

Computer Vision at the Edge

Alongside building the model, we worked with ADLINK to deploy to their Edge device for inference on site. The main photo for this blog  shows our demo setup in the lab, with the laptop controlling and coordinating the other components. The Neon-i camera can be clearly seen on the left and next to it is the small demo inference engine, the ADLINK Vizi-AI. For interest, a layout of the Vizi-AI is shown in the following close up picture.

The Vizi-AI Module

The Vizi-AI Module Used for Inference

The Vizi-AI was running Intel OpenVINO. This is Intel’s open source computer vision technology. The model converter, mentioned above, reads in the Tensorflow model and creates the artefacts for deployment on to the Vizi-AI module. The Vizi-AI takes the feed from the camera and runs our inference on it to add the bounding boxes and the object detection confidence levels as a percentage value.

Health & Safety of Construction

Health & Safety at work is a top priority for Balfour Beatty and in construction generally. The estimated cost to the UK economy is over a billion pounds (UK Government HSE statistics).

Injuries Statistics Construction

Statistics of Lost Working Days in Construction and economic costs

What we have shown in this prototype demo is that it is possible for AI at the Edge to detect PPE. We have the opportunity to alert and advise construction health and safety leaders to ensure onsite corrective action can be taken. Limited statistics exist about how many workplace injuries and health issues are related to incorrect PPE, but the potential to improve the situation is clear. Based upon the success of this 1st phase, the next steps are to proceed to further model optimization and trials on real construction sites. Ultimately, we anticipate a global roll out so that these AI tools become standard across industries. If you would like more information and a more detailed demo please contact us.

Andrew Smale

Elderly Geek. Arsenal fan. Guitar-playing marathon runner. PhD in AI and Medical Informatics from Oxford in 1991.

  • John Smith
    Posted at 17:27h, 09 May Reply

    Technology is going to bring lots of changes in the construction industry after this COVID-19 pandemic ends. I hope everything will be back to normal soon..

    • Andrew Smale
      Posted at 08:00h, 11 May Reply

      Thank you John. Yes, new processes and practices will be needed

  • James
    Posted at 15:50h, 22 July Reply

    Engineers are working on many aspects to make a comfortable life for individuals and professionals. The construction industry has also enhanced a lot due to advanced methods and technology. Andrew Smale, thank you so much for sharing this information.

  • James
    Posted at 15:53h, 22 July Reply

    It is very important for everyone to know about the importance of the latest technology in construction industry. Andrew Smale, thank you because you have done a great job.

Post A Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.