June 2021 OES Beacon OES Beacon


Nadir Kapetanović, Igor Kvasić, Anja Babić, Ivan Lončar      

Despite the challenging time that we’re faced with, IEEE OES University of Zagreb Student Branch Chapter (UNIZG SBC) members are giving their best to continue with the usual research activities. One of the essential parts of these activities are periodic tests of the equipment, marine vehicles and various algorithms developed for those vehicles. At the beginning of April, our members participated in diver-robot interaction trials at our soon-to-be-opened research pool in the newly renovated Laboratory for Underwater Systems and Technologies space. The end of April was marked with our trials regarding underwater communication and localization in Split, Croatia. During those trials, our members also participated in a webinar organized by Blueye company from Norway to show the integration of their Blueye Pro ROV with our newly developed autonomous catamaran. Dissemination of our knowledge and experience and promotion of our SBC are also very important for us, so we participated in two such events in May. On May 13 our members presented their research on underwater human-robot interaction modalities on the seventh in the series of events “Coffee with EUMR” – EUMarineRobots – Marine robotics research infrastructure networks, this time hosted by the University of Zagreb. As every year our research activities, underwater vehicles and ongoing projects were presented at the 2021 Science Festival with the goal of making the public eye more aware of the areas of our work and research laboratories.

First remote trials in the new LABUST facilities – a diver in a pool in Auckland on the top left, the D2 AUV on the top right, a control and camera streaming PC on the bottom left and right

Adriatic project remote trials (March 29th to April 8th)

Marine robotics is one of the research fields that largely depends on experimental work. COVID restrictions and lockdown measures in 2020 brought physical collaboration possibilities, joint experiments, equipment sharing and field trials close to impossible. 2020 was the year that coincided with long-term LABUST plans for expanding and upgrading its laboratory facilities. With most of the people stuck working from home and project partners unable to travel abroad, a lot of initial concerns in designing a new experiment workspace went into addressing these challenges. One of the solutions aimed at enabling more work to be done remotely is equipping the indoor pool with underwater cameras, ceiling cameras and ultra-wideband localization systems. The idea of implementing such a setup is streaming as much data as possible and providing situational awareness to a remote user.

The first remote trials that put the described setup in the new laboratory to the test took place in late March and the beginning

of April 2021. These trials were performed in collaboration with the Auckland Bioengineering Institute (ABI) from the University of Auckland, New Zealand. The goal of the trials was to test novel diver-robot interaction modalities developed through ONR-G project Adriatic. The scenario consisted of a diver equipped with sensors embedded in the diving suit and a gesture recognizing diving glove in a pool in Auckland, and the D2 AUV in the LABUST pool in Zagreb collaborating together. The diver used the smart diving glove to recognize gestures and send them acoustically to a poolside PC. The commands are then sent over a http server to a poolside PC in Zagreb and again translated to acoustic commands transmitted to the underwater vehicle. The vehicle executed simple movement actions mapped to these commands. The underwater cameras, as well as the cameras mounted on the vehicle, were streamed to Auckland to provide situational awareness, and the diver received a haptic feedback through the glove when a gesture was recognized and sent successfully.

 Considering the distance between ABI and LABUST laboratories and the current travel and collaboration restrictions in force, it was practically impossible to do a joint experiment in person. Introducing existing and new technologies towards enabling such remote collaboration experiences promotes equipment and infrastructure sharing and helps crossing barriers standing in the way of science.

CUV-ME trials (April 19th to 24th)

One of the significant challenges of underwater robotics is underwater localization on which many high-level functionalities such as mission/path planning and following rely. Trials that were organized from the 19th to 24th of April, 2021, in Split, Croatia, had inter-vehicle underwater localization and communication testing as the main objective. Degaussian station at Marjan peninsula, managed by the Republic of Croatia Armed Forces (OSRH), was a great location to work at with a full logistical support (location, work tent, divers, boats, etc.) provided by OSRH.

Degaussian station at Marjan peninsula, Split, Croatia
Logistical support provided by the OSRH with boats and divers

These trials were organized in the scope of the CUV-ME project (Cooperative Unmanned Vehicles in the Maritime Environment). Sea trials were a great opportunity for all vehicles to be deployed at the same time and to work cooperatively. The primary objective was to  assess  the  current navigational  accuracy of all the mentioned underwater  platforms, and the ability  to communicate between surface and underwater platforms.

It was planned that the ASV would track the AUVs in its downward looking camera as a ground truth, but also provide a USBL beacon for acoustic underwater localization of those vehicles. A firehose was spread along a 200m long transect along which  LAUV Lupis traversed and detected the firehose in its downward looking camera and in its side-scan sonar.

Morning briefing at the Degaussian station
ASV Proteus with mounted SeaTrac USBL and downward looking camera

The D2 AUV was used to detect and track LAUV Lupis in its forward looking sonar imagery. All these data will be post-processed by a sensor fusion method in order to quantify how much localization precision improvement does each and every additional sensor bring compared to the initial dead reckoning (with a DVL in the case of LAUV Lupis) navigation.

Deployment of LAUV Lupis from the side of a boat
Screenshot from Neptus mission planning and analysis software used for controlling LAUV Lupis and ASV Proteus

Another objective of these trials was to record datasets for diver detection in forward looking sonar imagery and for the detection of diver presence by detecting diver breathing/bubbles sounds recorded by a hydrophone.

Experiment setup for recording the dataset of diver detection in forward looking sonar
Experiment setup for diver breathing/bubbles detection recorded by a simple hydrophone

The trials were successful at providing a proof of concept for cross-country USV-AUV communication and data exchange. Data collection was completed successfully with shown reproducibility and repeatability. Additionally, capability to exchange underwater mapping data formats between the University of Zagreb and OSRH was validated.

Group photo of all participants of the trials and the equipment

Potential future testing should focus on research in fully autonomous detection, acquisition and tracking of AUV targets in sonar imaging. Autonomous visual tracking from the surface would provide ground truth for evaluating navigation precision of AUVs. Including OSRH autonomous vehicles into this framework would be beneficial for showing easy transfer of technology between different vehicles and to evaluate the current readiness and navigation precision of OSRH vehicles.

 HEKTOR project field tests (April 21st)

In the scope of the HEKTOR project (Heterogeneous autonomous robotic system in viticulture and mariculture, http://hektor.fer.hr/) our SBC members got a chance to acquire a Blueye Pro ROV from the Norwegian robotics company of the same name (https://www.blueyerobotics.com/).

HEKTOR is conceived as a modular and autonomous robotic system, adapted for various missions in viticulture and mariculture with the anticipated possibility of human intervention while performing various work, inspection and intervention tasks. The main objective of the HEKTOR project is to provide a systematic solution for the coordination of smart heterogeneous robots/vehicles (marine, land and air) capable of autonomously collaborating and distributing tasks in open unstructured space/waters.

One of the tasks of the HEKTOR project is to develop an autonomous surface vehicle (ASV) that could also be a docking station for the ROV, as well as provide a landing platform for an unmanned aerial vehicle (UAV). Because of these significant payload requirements, our previously developed ASV model (e.g., ASV Proteus mentioned above) had to be redesigned into a larger catamaran-shape ASV.

Left: Blueye ROV, middle: ASV Korkyra, right: ASV Proteus
Maiden voyage of the ASV Korkyra on the 21st of April, 2021, deployed at sea together with the Blueye ROV integrated with it

Since our members were already in at the Degaussian station, they also took time to launch the ASV Korkyra on its maiden voyage on the 21st of April, 2021, and test its manual and automatic controls and integration of the surveillance IP camera. Also, the ROS2 package that was developed by our members, which interfaces Blueye’s SDK with ROS2 for future autonomy tasks, was tested, i.e., data sending between the ROV and the operator’s PC with the ASV acting as a data relay in between the two.

Blueye webinar (April 22nd)

On the 22nd of April,, 2021 the Blueye company organized a one-hour webinar named “Customize your setup with the Blueye SDK and API”

(https://www.blueyerobotics.com/webinar/customize-your-setup-with-the-blueye-sdk-and-api) hosted by Andreas Viggen, company’s Senior Software Engineer. Blue ROVs are normally controlled by the Blueye app running on either iOS or Android devices. However, some of their customers have unique use-cases where they want to control the drones with their own software. Therefore, the Blueye company has developed the Blueye SDK — a python library that allows you to control the drone programmatically within minutes.

Screen capture of the webinar’s presentation

During the webinar, Andreas Viggen and Nadir Kapetanović (IEEE OES UNIZG SBC) have shown how to get started with the SDK. Andreas presented some stories from their customers who have utilized the SDK for controlling multiple drones simultaneously and others that have improved their post-processing sequence of the media and log files via the API.

Nadir presented the HEKTOR project use case for the Blueye ROV, and described the Blueye SDK-ROS2 integration as a necessary functionality for any kind of future autonomous inspection tasks. Also, the guests of the webinar had the chance to watch a live demo of the ASV Korkyra and Blueye Pro ROV deployment and manual control, where the ASV was basically a data relay between operator’s PC and the ROV. The complete recording of the webinar can be found on https://youtu.be/243JWUpxBY8.

Coffee with EUMR webinar (May 13th)

 As partners in the European Union’s Horizon 2020 project EUMarineRobots (EUMR), the Faculty of Electrical Engineering and Computing, University of Zagreb, hosted the Seventh Coffee with EUMR webinar series, an online educational and training session with invited talks, exhibits and TNA experiments.

Screen capture of the live demo. Left: Blueye ROV-ROS2 graphical user interface used for video streaming and controlling the ROV. Right: Video stream from ASV Korkyra’s surveillance IP pan-tilt-zoom camera showing the ROV in front of it
Screen capture of the “Coffee with EUMR” webinar

The scope of the EUMR project is to provide an access-infrastructure for the deployment of a full-range of aerial, surface, and subsurface marine robotic assets. UNIZG is part of the consortium that comprises 15 partners from 10 European countries who, collectively, can deploy a comprehensive portfolio of marine robotic assets with required associated support assets and expertise with a capital value well in-excess of €500M. Igor Kvasić presented LABUST’s approach on underwater human-robot interaction capabilities. Part of the presentation was dedicated to acquainting the audience with the soon to be opened laboratory facilities and subsystems that enable remote access work, as well as the latest joint remote experiment results with TNA partners.