December 2023 OES Beacon

Sebastian, the AUV Won First Place in Underwater Robot Convention in JAMSTEC 2023!

Hiroshi Kasuga, Lisa Hakataya, Hiroki Yokohata, The University of Tokyo  

Fig.1. The group photo of “MakiLabM1” Team and the AUV, Sebastian


On August 26 and 27, students and enthusiasts gathered at JAMSTEC headquarters in Yokosuka, Japan, to participate in the Underwater Robot Convention in JAMSTEC 2023. The convention, hosted by NPO Japan Underwater Robot Network, serves as a place for participants to exchange technical ideas and make connections through the competition and presentations of self-made underwater robots. The overview of the convention can be found on the official website [1] (in Japanese), and the conventions in previous years are reported in [2], [3], [4], [5], and [6].

We, the authors, are master’s course students in Prof. Maki’s laboratory at the University of Tokyo.  We participated in the AI-challenge division as team “MakiLabM1” [Figure 1]. Our main objective was to acquire basic knowledge and skills on underwater robotics in preparation for future research works. We also learned about teamwork, which is important in the development and operation of underwater robots.

Fig.2. Overview of water tank setup

Competition Rules

The AI-challenge division was established to encourage the introduction of AI (Artificial Intelligence) into underwater robotics. Robots were required to autonomously break balloons placed in the water tank. There were three types of balloons: red, yellow, and blue, and multiple balloons were set up for each. The height from the bottom of the water tank at which the balloons were placed, and the score obtained by breaking them, differed depending on their color [Table 1]. The arrangement of them was random. Figure 2 shows the overview of the setup of the water tank. The only tool allowed to break them was one thumbtack. Each team was given 4 minutes to perform, and the higher the score obtained, the higher the competition score. In addition to the competition score, the judges also evaluated the degree of autonomy of the robots and the presentations in the poster session, and the final ranking was determined by the total of these scores.


Table.1. Height and score of balloons

Color Height [m] Score
Red 0.3 30
Yellow 0.8 20
Blue 0.6 -10


Fig.3. The process of breaking balloons


Sebastian is an AUV that can move nimbly in surge, heave, and yaw directions. We implemented various algorithms in order to break more red balloons within the time limit and get a higher score. The following is a description of the algorithm implemented along with the competition flow. At the beginning of the competition, Sebastian is at the edge of the pool. At the signal to start the competition, Sebastian moves to the center of the pool where the balloons are densely packed. Once in the center of the pool, Sebastian uses the heave thruster to land on the bottom of the pool. Sebastian, with an altitude of 0 cm at this landing point, rises to an altitude of 30 cm, where the red balloons are located. This makes it easier to recognize the red balloons with the highest score. After surfacing to an altitude of 30 cm, Sebastian starts autonomous navigating and searching for balloons. Fig. 3 shows a conceptual diagram of the balloon search mode. Sebastian remembers the nearest balloon A and the nearest balloon B from balloon A. While approaching A, another balloon C is seen. If balloon C is farther away than balloon A and closer than balloon B, the second target is changed from B to C. Otherwise, the second target remains B. Even if Sebastian lost sight of balloon B, the location information of B is preserved, and after breaking A, it rotates in the direction of B. The direction of rotation is determined by the x and y coordinates of balloons A and B, and the relationship between yA/xA and yB/xB. Similarly, if a new balloon D is seen after balloon A is broken, the distance to balloon D is compared with the distance to balloon B, and the closer balloon B is selected as the next target. When breaking a balloon, it rushes toward the center coordinates of the balloon. Sebastian approaches the balloon, and when the balloon is too close to be recognized, Sebastian accelerates to gain momentum and break the balloon with the forward pin. The above is the basic flow of breaking a balloon, but in order to more accurately aim for a high score, the following algorithm was implemented. Based on the RGB value of the balloon’s center coordinates and the altitude at which the balloon exists, the algorithm recognizes what color the balloon is, and prioritizes high-scoring red balloons, while blue balloons with negative points are excluded from the target. Similarly, if the altitude at which a balloon exists exceeds the altitude of the water surface, it recognizes that the balloon is in a situation of total reflection on the water surface and does not aim this as a target. Similarly, the weight holding the balloon at the bottom of the pool is also excluded from the target because it is not at the altitude where the balloon should be, thus preventing Sebastian from mistaking the weight for a balloon.

Fig.4. General arrangement of AUV Sebastian


This mission used the original cruising autonomous underwater vehicle “Sebastian” [Fig. 4]. This AUV is based on the wAriel AUV that participated in last year’s competition.

Sebastian is equipped with two Heave and two Surge thrusters and can be controlled in four degrees of freedom (Surge, Heave, Pitch and Yaw). The acrylic hull, which is a buoyant body, is positioned on the left and right sides and the center of gravity is designed in the center downwards to maintain stability in the roll direction. As sensors, a depth sensor for depth estimation and a camera module for recognizing balloons in the water were used. The AUV was also equipped with leg parts for landing on the bottom of the pool and guides for pushing the balloon against the pushpins from the front to destroy it.

Sebastian is controlled using a Raspberry Pi 4 computer and a Teensy driver. As a control tool, Sebastian uses a common open source tool for robotics control called Robot Operation System (ROS) on the Raspberry pi, and sends PWM commands to the motor drivers with the Teensy driver.

In this mission, circle detection is performed by OpenCV Hough transform of the USB camera image [Fig. 5]. The Hough gradient method can be used to recognize objects by changing the parameters and the processing of the image data to be captured. The results of balloon recognition using the blue part of the BGR image were output to an external PC for confirmation and showed sufficient recognition results to control the system. This shows that circle recognition using the Hough transform is effective for recognizing balloons in water.

Fig.5 The system in Sebastian detecting a balloon using a USB camera.

The position of the balloon in the camera coordinate system was calculated from the size and position of the balloon as seen from the camera [2]. Assuming a balloon diameter of 20 cm, the relative position was calculated using the following equations (1), (2) and (3) [Fig. 6]. The actual depth and horizontal distance of the balloon was calculated by calculating the following equations (4), (5) and (6) using the internal IMU sensor and depth sensor.

For color recognition of balloons, the BGR score of the center of the balloon was used due to the computational limitations of the Raspberry Pi. If the center of the balloon was outside the camera’s field of view, the point in the field of view closest to the center was used for color recognition [Fig. 7]. The depth of the balloon was also used to aid color estimation.

The competition rules were presented in June 2023 and a development project was initiated. Software development was divided into balloon recognition and target determination algorithms, vehicle control and state transition algorithms, and firmware development. by July, hardware improvements and control code for minimum functions were completed. As of early August, balloons could be destroyed under a certain condition.

In August, debugging, parameter adjustments and exception processing were added to improve the robustness of the system. In parallel, the algorithm for determining which balloon to break next to the currently approached balloon was also implemented.

Finally, the color recognition success rate and the approach success rate were confirmed to be more than 90% against blue, red, yellow, green and pink balloons.

Fig.6. Calculate the position of the balloon considering the attitude of the AUV


On the first day of the convention, each team presented a poster session and answered questions from the judges and other participants. We also tested the operation of Sebastian in the water tank. The second day of the convention consisted of a water tank competition, which concluded with an awards ceremony. We were awarded first place out of all four teams for breaking the most balloons and for our high degree of autonomy to operate without tether cables.

Through this competition, we were able to realize the difficulty of underwater robot development, the importance of teamwork, and networking with people in the community. The experience we gained through our three months of intensive development and participation in the competition will certainly be a great source of inspiration for our future research activities.


The Underwater Robot Convention in JAMSTEC in 2023 was supported by The Japan Society of Naval Architects and Ocean Engineers, IEEE/OES Japan Chapter, MTS Japan Section, Techno-Ocean Network, Kanagawa Prefecture, Yokosuka City, Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Center for Integrated Underwater Observation Technology at Institute of Industrial Science, the University of Tokyo, FullDepth Co., Ltd., Nortek Japan LLC, Japan Branch of Robotiz, Inc, ARAV Co., Ltd., Sea challenge Co., Ltd., Space Entertainment Laboratory Co., Ltd., Matsuyama Industry Co., Ltd., IWAKITEC Co., Ltd., Chick Co., Ltd., Misago Co., Ltd., and Aqua Modelers Meeting. We would like to express our sincere appreciation to the sponsors for their strong support and cooperation in realizing this convention.

Fig.7. Color detection point in a camera view field


Hiroshi Kasuga: I am pleased that we won, but there are many points that can be improved upon. I hope to make use of this experience in my research.

Lisa Hakataya: I’m glad we won. However, there were many things we could not do. We would like to increase what we can do in the future.

Hiroki Yokohata: I’m very happy to have achieved another great result of winning the first place this year, as we did last year. I would like to thank my teammates for leading the team to this height.


[1] Underwater Robot Convention in JAMSTEC 2023 (Japanese).

[2] A. Toriyama, M. Ohashi, H. Yokohata, wARIEL, the AUV Won First Place in Underwater Robot Convention in JAMSTEC 2022!  IEEE OES Beacon Newsletter, 11(4), 83-86 (2022.12)

[3] K. Yamamoto, S. Chun, Y. Sekimori, C. Kawamura, ARIEL, the AUV Won First Place in Underwater Robot Convention in JAMSTEC 2021! IEEE OES Beacon Newsletter, 10(4), 70-73 (2021.12)

[4] Y. Sekimori, T. Maki, Underwater Robot Convention in JAMSTEC 2020 – All Hands on Deck! Online!!, IEEE OES Beacon Newsletter, 10(1), 39-42 (2021.3)

[5] K. Fujita, Y. Hamamatsu, H. Yatagai, Reflection for Singapore Autonomous Underwater Vehicle Challenge – the Comparison Between SAUVC and a Competition Held in Japan, IEEE OES Beacon Newsletter, 8(2), 64-67 (2019.6)

[6] H. Yamagata, T. Maki, Underwater Robot Convention in JAMSTEC 2018 – from an Educational Perspective, IEEE OES Beacon Newsletter, 7(4), 68-72 (2018.12)