December 2023

Robotics for Asset Maintenance and Inspection (RAMI) 2023 competition

Gabriele Ferri1, Alessandro Faggiani2, Tommaso Fabbri2, Fausto Ferreira3

1RAMI 2023 Technical Director 2RAMI2023 Deputy Technical Director  3VP Workshops & Symposia

NATO-STO Centre for Maritime Research and Experimentation (CMRE) has been organising Student AUV Challenge – Europe (SAUC-E), the premier European student competition for underwater vehicles, since 2010. Over the years, we have been fostering autonomy and robotics in different events such as euRathlon 2015 Grand Challenge and the European Robotics League Emergency 2017, involving the use of multi-domain cooperative teams of robots in search and rescue scenarios. euRathlon 2015 Grand Challenge, organised in the framework of the euRathlon EU project, was the first world’s multi-domain robotics competition. Disaster response tasks were proposed to teams composed of land, sea and air robots and were held at a real power plant site, specifically Tor del Sale in Piombino, Italy. The inspiration came from the Fukushima 2011 accident.

This formula of multi-domain competitions has continued with the European Robotics League (ERL) Emergency. ERL is a common framework for robotics competition funded by the European Union. ERL was launched in 2016 in three vibrant fields of robotics: industrial, service and emergency robots. The first season culminated with the ERL Emergency 2017, a multi-domain competition again locally organised by CMRE at the Tor del Sale power plant site in Piombino. ERL Emergency 2018 and 2019 took place again in La Spezia and were planned to take place in 2020 and 2021 but the COVID-19 pandemic made it impossible to have physical competitions.

Following this tradition, we have started in 2022 a new competition named Robotics for Asset Inspection and Maintenance (RAMI). RAMI is part of the ERL Emergency League and is organised in the framework of the EU-funded METRICS project (https://metricsproject.eu/) aiming at addressing Inspection & Maintenance (I&M) tasks achieved in risky and/or hostile environments where human intervention is challenging or impossible, where direct link with an operator could not be guaranteed and where autonomous decisions are necessary to reduce operational time of the inspection tasks and ensure repeatability while maintaining an appropriate safety level for the mission. RAMI proposes events for aerial robots, organised by CATEC (Spain), and for underwater vehicles organised by CMRE.

RAMI consists of both virtual and field competitions. At CMRE, we have organised two virtual competitions (https://metricsproject.eu/inspection-maintenance/rami-cascade-campaign-marine-2023/). In such competitions a dataset of images of underwater objects of potential interest (OPIs) are provided to teams, which are requested to develop the adequate software in order to classify, identify and localise images of OPIs of a test dataset. These virtual or cascade competitions are important to prepare teams for the field events and to attract researchers from other domains to tackle with underwater image processing challenges.

The first field event for marine robots was organised at CMRE seawater basin in July 2022. In this report, we present the second RAMI event, recently held at CMRE in July 2023.

The competition

CMRE hosted this year the RAMI23 robotics competition from 16 to 21 July. The competition took place in CMRE seawater basin. The areas were prepared to simulate an Oil & Gas plant in a harbour connecting it to the general theme of the RAMI competition: Inspection & Maintenance (I&M). To support teams’ growth, we proposed tasks similar to those present in ERL Emergency 2019 and in RAMI22.

The RAMI marine competition tasks are inspired by the following user story:

An Oil&Gas offshore site has to be investigated after that a malfunction has been reported by the plant safety systems. From what is known, a pipeline has started leaking and an explosion may occur soon.

A robotic team composed of underwater robots (AUVs) is ready to intervene. It is time for the emergency team to act. The priorities are to reach the area of the accident, and to assess and quantify the entity of the leak. Then the robots have to reach the pipe assembly area for quantifying the damage to the plant and for identifying which pipe has been damaged and is responsible for the leak. Finally, the robots must intervene on the plant itself by closing a valve to stop the leak to prevent the explosion and further damages to the environment.

Underwater robots were challenged to inspect a pipe structure, reach several waypoints and pass through a validation gate. In addition, several buoys of four possible colors had to be detected, localised and their colour recognised. A different action was requested to the teams depending on the buoy color: for instance, turning around the buoy in a clock-wise circle or stopping for 30 seconds increasing the depth. The objective was to force teams to integrate perception with adaptive mission planning in a realistic environment such as CMRE water basin. Finally, three manipulation tasks were proposed, specifically staying into contact with the pipe, valve closing and picking a stick, bringing that to the surface in a controlled way. These tasks were the only ones allowed to be performed through teleoperation (ROV mode), while all other tasks had to be performed autonomously. As in the previous editions, the challenges were held at the CMRE waterfront sea basin, which is a sheltered harbor, providing participants with the opportunity to grapple with real-life sea conditions, including limited visibility and salty water, but within a safe, controlled environment. The limited visibility added severe difficulties to object recognition by AUVs, even if the targets were bright orange or red in color.

Figure 1 (Left) The OUBOT’s team during the deployment. (Right) Judges controlling a mission from the Judge Tent.

We have noticed that, despite the COVID-19 interruption and related issues for the students, teams’ performance generally has been improving over the past years, especially for entries with experience in our competitions. UNIFI Robotics Team (Italy), in particular, accomplished several tasks, showing a good maturity in navigation, survey, buoy localization and manipulation. For the first time in our competitions, a buoy was classified correctly in real-time and the appropriate autonomous behaviour was triggered (the AUV changed depth in a controlled way). The other buoys were also localised, classified and identified in post-processing with good accuracy. All the teams were able to deploy their AUV in the water and to score some achievements. ITU AUV Team from the Istanbul Technical University (Turkey), for instance, was a rookie team in RAMI events. The team was the winner of the Singapore SAUVC 2022. SAUVC, sponsored by OES as well, is held in a swimming pool and targets at more junior teams. Despite some problems and difficulties in adapting to the sea scenario, they were able to accomplish manipulation tasks in the last day, showing good capabilities of adaptation and improvement in performance over the event week. This is extremely encouraging showing how competitions all around the world can be a common training ground for teams to grow and improve.

Further work has to be done to improve the performance of teams in real-time perception and related adaptive mission planning. Cascade competitions can help in this aspect.

Finally, for the first time in our event, we organised a student poster competition. This was an opportunity to further increase contacts and involvement between teams and the international experts present at the event.

The participant teams

This year 5 teams attended the event. Four teams attempted the competition tasks, while a fifth team only practiced in the arenas. It is to be noticed that after the COVID19 pandemic, some teams lost continuity as students graduate and there was no overlap between older generations and younger ones. This is why it is important to organise annual competitions and to support team participation, facilitating the succession with the required handover of experience. To start again after the COVID-19 period, we think it is pivotal to involve people with the aim to strengthen the community, and to contact and involve as much as possible participants and potential sponsors. This year, we had the participation of three teams at their first participation in our competitions, and the other two had attended many of our past events.  This good mixture of new and veteran teams shows how RAMI/ERL is today a fixed appointment for several European research groups, and continues to attract new venues.

The participant teams were:

  1. ITU AUV Team from the Istanbul Technical University in Turkey was the winner of the Singapore SAUVC 22, a competition held in a swimming pool. They participated in RAMI23 for the first time with their Taluy AUV.
  2. 2. NAUTILUS Team consisted of three people, from ESTACA School of Engineering in France. They were at their first participation in one of our competitions.
  3. UNIFI Robotics Team, from the University of Florence, Italy. A veteran of our competitions (first participation in 2012). They participated in RAMI23 with their FeelHippo AUV.
  4. OUBOT, from the Obuda University, Budapest, Hungary. This team had participated in euRathlon 2015 and ERL Emergency 2017. In these competitions, they participated thanks to an AUV loaned by the organisation. Then, they built their own robot to participate in our events.
  5. Team ERGO from the University of Pisa, Italy. Their robot, a Zeno vehicle manufactured by MDM Team company, was used mostly for practicing in the area and collecting optical and acoustic imaging. This is important in view of a future participation of the team in our events.

The winners

Figure 2 (Left) Identification of a yellow buoy by UNIFI Team’s robot. (Right) The manipulation console as imaged by UNIFI. The valve and the pole are visible, together with the number. Photo credit: NATO STO CMRE and UNIFI Team.

RAMI 2023 proposed one Task Benchmark (TBM) in a different competition day: Pipeline area inspection on Wednesday, Intervention on the pipeline structure on Thursday and finally the Complete mission at the plant on Friday. TBMs are oriented to task fulfillment and are judged by judges in real-time, or scrutinizing the data produced by teams within one hour from the end of their time-slot. On the other hand, Functionality Benchmarks (FBMs) aim to evaluate robot functionalities, such as mapping capabilities. They are evaluated after the competition by analyzing the output data from the teams.

UNIFI Robotics Team stood out in the three TBMs, winning all of the TBMs and having a draw in the Intervention on the pipeline with OUBOT team.  UNIFI Robotics Team was finally awarded with the SAUC-E 2023 award, the best team over all the proposed TBMs.

Furthermore, UNIFI Robotics Team also was first in rank in the FBMs involving mapping the area (detecting and localizing the coloured buoys) and object recognition (classifying and identifying the coloured buoys).

Other prizes awarded were:

  • “Best team SAUC-E award” (best team of the rankings over the three TBMs): UNIFI Robotics Team – voucher from Breaking the Surface sponsor.
  • “Best presentation” (teams were asked to give a presentation on their robot and team): ITU AUV Team – voucher from Breaking the Surface sponsor.
  • IEEE OES Innovation Award Plaque – UNIFI Robotics Team – 300 $ from IEEE OES sponsor.
  • “Best rookie team”: NAUTILUS Team.
  • “Most improved vehicle award”: OUBOT – ping sonar from Blue Robotics sponsor.
  • “Best student poster” – Simone Tani “Comparison of Monocular and Stereo Vision Approaches for Structure Inspection using Autonomous Underwater Vehicles,” ERGO Team – voucher from Breaking the Surface sponsor.

Feedback

The feedback from both the participants and the judges was very positive. We have noticed a general improvement in teams’s performance. In particular, we had good results in tasks involving the combination of real-time perception with autonomous behaviours. For instance, UNIFI Team was capable, for the first time in our competitions, to detect a buoy in real-time, to determine its colour and to react with the required behaviour. This witnesses how proposing the same tasks over different editions can support the gradual growth of participant teams. At the same time, it is felt the need to keep or create some simple tasks for the newcomers (we had two rookie teams this year). We will try to achieve this trade-off between more complex and more basic tasks in the future, to be attractive for both new entries, and for more experienced teams, which require to be pushed by more challenging missions. The realistic scenario, which attracts participants, is also central to involve end-users and companies. This year, for instance, we had one judge from ROSEN group and many judges from the U.S., interested in our realistic set-up. Results of RAMI23 showed, for most of the teams, good performance in autonomous navigation, mapping and imaging. However, in general, more work and support are needed for real-time perception and autonomy. This will be addressed in the future campaigns. In the ongoing cascade campaign, we provided teams with an underwater image dataset for developing image processing software. This is important to prepare teams to challenge with real-world conditions; especially when there is a lack of available datasets in the underwater domain to support the development and the validation of image processing algorithms.

The judges

Figure 3 Teams and the Organising Committee during the award ceremony at CMRE.

We had more than 30 judges and observers from Europe and U.S., both from companies, academia and research institutions. In particular, this year’s edition saw a large presence of judges and experts from the U.S. We had people from key-players in the Oil&Gas field, such as ROSEN group (Germany), DLR (Germany), FINCANTIERI and Leonardo (Italy). From the U.S. we  had the strong presence and support of the Office of Naval Research (ONR). Other judges were present from NRL, Spawar and RoboNation. As in previous competitions, CMRE provided the full engineering and logistical support during the event execution. Teams were supported by CMRE logistics, having access to a good support, since engineers and technicians helped the teams with suggestions and supply parts. This was appreciated  by teams and is crucial in helping teams to solve the unavoidable technical issues. Teams’ general feedback was very positive regarding the marine trial area and the way the scenario was setup.

The local community was also engaged. Local schools from the La Spezia area were also involved in the framework of Giona’s Project. This allowed around 20 students close to high school graduation to come and interact with the teams.

Sponsors

Figure 4 The RAMI23 Organising Committee awarded by Bill Kirkwood with a plaque in recognition by IEEE OES of the excellence in the organisation of robotics competitions over the last 10 years.

As in previous years, IEEE OES played a fundamental role as the Main Sponsor. IEEE OES supported team travel and participation. This is essential to attract teams and allow them to take part in such complex events. One of the RAMI goals is to educate future multi-disciplinary engineers. Therefore, the sponsorship of ‘Breaking the Surface’ 2023, the 15th Interdisciplinary Field Workshop of Marine Robotics and Applications, is well aligned with our mission. The organisers of ‘Breaking the Surface’ provided complimentary registrations for three students.  These students had the opportunity to participate in the 15th edition of this multi-disciplinary and educational workshop that since 2019 counts with the IEEE OES sponsorship.

As in the past, we again had the marine robotics commercial sector involved. Blue Robotics gave a Ping sonar to the team that won the ‘Most improved vehicle’ award. Other sponsors were Fincantieri and ROV-Expert.

Our aim is to continue to develop RAMI as a unique event that challenges teams with realistic conditions, with an emphasis on precise metrological evaluation. These achievements were made possible thanks to the fundamental support of IEEE OES and all our other sponsors. We thank all the teams, judges, presenters, visitors, staff and everyone involved, who made RAMI 2023 such as a successful event.