Floating Ball Alert! The navigation was abruptly halted by a warning sound during the maiden voyage of “Dolphin 1”. Patches of floating balls and vast marine farms posed unexpected challenges below.

HEU Professor ZHANG Zhi, leader of the maiden voyage of China’s inaugural digital twin intelligent test ship “Dolphin 1”, felt a chill as there was no indication of a breeding farm on the nautical chart. Navigating through this uncharted territory became the only option.

Could “Dolphin 1” navigate around these floating balls with unknown coordinates?

This question raced through ZHANG Zhi’s mind. However, the advanced equipment loaded on “Dolphin 1”, including lidar, solid-state radar, panoramic infrared vision system, and its data processing speed of hundreds of megabits per second, reassured ZHANG Zhi that he was overthinking.

With its high-end configuration and decision-making capabilities, “Dolphin 1” swiftly addressed challenges during its maiden voyage, expertly maneuvering through the “dangerous beach”.

At the World Navigation Equipment Conference on October 12th, HEU’s showcased “Dolphin 1” garnered widespread attention. Attendees, upon learning about the ship’s intelligence in information fusion, situational awareness, and digital twins, couldn’t help but laud its capabilities.

Led by “Dolphin 1”, China has taken a significant stride toward the high-quality development of intelligent ships.

Endowing ships with the superpower of “foretelling accurately”

Since the International Maritime Organization introduced the concept of e-Navigation in 2006, many countries, including Europe, America, Japan and South Korea, have embarked on large-scale plans to enhance ship intelligence.

“With the deepening application of technologies like artificial intelligence and digital twins, ship intelligence has far surpassed the initial definition of ‘informatization’”, explained by Professor XIA Guihua, leader of the “Head Goose Team” of Heilongjiang Province and the Digital and Intelligent Technology Innovation Team at Harbin Engineering University. The test ship is named “Dolphin” because the dolphin is a symbol of intelligence.

“Dolphin 1” not only exhibits remarkable precision in detecting small targets on the water surface within 2 nautical miles but is also equipped with artificial intelligence technologies for situational intelligence perception and pre-judgment.

“On land, intelligent connected vehicles are changing transportation safety and energy efficiency, moving towards intelligent and even unmanned driving. In the ocean, the transformation of intelligent ships is also catching up,” said XIA Guihua.

“Dolphin 1” has pioneered the development of “smart eyes, smart ears, and smart brains” for future intelligent ships. After further enhancements, it aims to empower Chinese shipbuilding and lay a solid foundation for fully autonomous technological capabilities in the main systems and key equipment of future smart ships.

“To achieve such a leap, it is necessary to equip ‘Dolphin 1’ with a true ‘super brain’”, stated Professor ZHU Qidan, the technical leader of the HEU Digital and Intelligent Technology Innovation Team. The team, with extensive research achievements in intelligent ship design, manufacturing, modeling, and calculation, provides high-precision information for intelligent ships through real-time ship dynamics modeling, navigation hydrodynamic performance simulation, power system simulation, wind and wave flow field simulation, etc. “It is equivalent to being able to conduct predrills in an environment close to a real ship, giving ‘Dolphin 1’ the ability to anticipate, think, and make judgments”, added ZHU Qidan.

To create this model, the team meticulously collected tens of thousands of real ship images, covering various scenarios, accurately annotated and analyzed them, and formed a dataset for machine learning of artificial intelligence. With the support of the core module, “Dolphin 1” can collect and process real-time multi-source information data at a speed of hundreds of megabytes per second, enabling it to autonomously navigate obstacles and plan the optimal route in real time.

“Dolphin 1” possesses a “digital clone”

Externally, “Dolphin 1” appears ordinary, measuring only 25 meters in length with a displacement of 100 tons. However, its distinctive feature lies in the virtual world—a “digital twin” capable of making predictions.

The advantage of a virtual world is its low trial and error cost. “Dolphin 1” has a virtual ship tailored for it, which can sail synchronously and rehearse the voyage even when the physical ship is stationary.

“The physical ship has stopped, and the virtual ship can continue to sail”, explained ZHANG Zhi. This capability enables the virtual ship to predict the next navigation strategy, anticipate potential dangers, and experiment with operations in risky situations to ensure the safe operation of the real ship.

“We hope to ‘replicate’ the wisdom of ‘Dolphin 1’ to countless ships. However, we cannot teach one by one. The role of the ship digital twin integration software platform and its application system is to ‘teach’ like a teacher”, said XIA Guihua. The virtual-real interaction platform provides a diverse ‘exercise field’ for intelligent ship testing, surpassing the limitations of the real environment.

In the future, the efficient testing, verification, operation, scheduling, and control platform that “Dolphin 1” relies on may be applied to the early research and development, mid-term testing, and later operation of autonomous intelligent ships, expediting the application and deployment of China’s autonomous intelligent ships in reachable waters.

The ocean environment is constantly changing, and numerous factors affect ship navigation. How to build a deduction platform that maps the real world, including virtual ships, virtual oceans, and virtual weather environments, and has precise simulation functions?

“In the digital twin system, virtual and real are dynamic dialectical relationships, which can be summarized as ‘from real to virtual’ and ‘from virtual to real’, and finally achieve ‘controlling real with virtual’”, explained XIA Guihua. The team constructs digital twin ships and a digital “cyber” marine environment based on real physical ships and the ocean environment they navigate. Information and artificial intelligence technology enable real-time interaction and connectivity between the virtual and real.

In the cabin of “Dolphin 1” and at remote digital support control centers in Huangdao and Harbin, real-time “virtual ships” synchronized with “Dolphin 1” can be observed. These virtual ships digitally reproduce the mechanical structure, host system, perception system, equipment components, and pipeline circuit system of the physical ship, fully connecting with the perception and control system of “Dolphin 1”.

“Digital modeling in virtual ships can remotely support and control physical ships, achieving autonomous navigation, automatic berthing, and automatic collision avoidance for intelligent ships”, stated ZHANG Zhi. “Dolphin 1” has successfully addressed the key challenge of remote digital support for autonomous ship navigation based on digital twin technology. Furthermore, it provides essential technical support for the autonomous navigation and operation of intelligent ships and marine engineering equipment through remote support.

Two innovations solidify the technological foundation

“This is not a completely new field”, noted Professor CAI Chengtao, the chief system leader of the team. Industrial digitization is part of new industrialization, and many countries began developing digital twin intelligent ships many years ago. South Korea, for instance, initiated the construction of smart ships in 2012 to promote the informatization, automation, intelligence, and green development of ships.

China opted for a meticulous plan before taking action.

In 2015, HEU and the 702 Institute of CSSC jointly led the research and development of the world’s first numerical water tank virtual testing system V1.0. This system, in collaboration with multiple research institutes and universities with technological advantages in shipbuilding and maritime science, received dual certifications from China Classification Society and France BV Classification Society.

The system employs large-scale and efficient parallel computing with billions of grids, providing extremely precise hydrodynamic performance simulations for ship and ocean engineering design, trial navigation, and operation.

“At that time, the trend was data-driven, akin to an alpha dog playing Go. With algorithmic support, it trained on massive datasets to generate ‘intelligence’ in fixed tasks”, recalled ZHANG Zhi. However, XIA Guihua deviated from this path.

“Data is important, but models are the soul of performance. Only by blending the two can high-precision calculations be achieved on the basis of a certain amount of computing power”, emphasized XIA Guihua. For digital systems to be applied in the shipbuilding industry, they must meet two conditions: establishing precise models and iteratively upgrading models through data-driven modeling.

In the realm of information technology, mastery of the code determines the pace of iteration. Therefore, when the implementation of intelligent ships and digital twin technology commenced in 2019, XIA Guihua once again chose source innovation.

“I had a debate with XIA before. I thought the architecture and models on open-source platforms were polished by many AI enthusiasts, and were already rich and solid enough. However, XIA insisted on innovation at the source”, shared ZHANG Zhi. In retrospect, this decision proves visionary. The underlying code, as the foundation of industrial digitization, serves long-term development and national strategy, requiring steadfastness.

The decision to innovate at the source not only solidified the foundation of digital twin technology but also enabled the flourishing application of technology.

“In terms of ship digital twin technology, we are already recognized as the ‘leader’ in the shipbuilding industry”, affirmed ZHANG Zhi. The team’s hands-on approach to typing out the code for the digital twin system provides a thorough understanding of the underlying and core mechanisms and laws.

Future remote “proxy driving” holds promise

A gust of strong wind struck, and the waves rose one after another. The sea, initially calm and peaceful, transformed into turbulent waves and undercurrents, causing the ship to sway violently.

“The wind was quite strong on the voyage from Weihai to Rushan, and ‘Dolphin 1’ was violently rolling. Almost all the researchers on board were seasick and lay still, afraid to move”, recounted ZHANG Zhi. Despite the discomfort, there was no cause for concern. “Dolphin 1” had a digital clone to assist the crew in “driving”.

Currently, “Dolphin 1” functions as a quadruplet, and its performance closely resembles that of the digital twin system in the cabin. It directly obtains all perception data of “Dolphin 1”, becoming the “strongest assistant” for crew members during navigation.

However, at the Intelligent Ship Remote Digital Support Center, located thousands of miles away from the sea, limitations in satellite data transmission and network bandwidth restrict the amount of data that can be transmitted. Presently, only nearshore remote “proxy driving” is feasible under favorable sea conditions. Nonetheless, the team envisions that remote “proxy driving” can genuinely navigate through challenging conditions in the near future. The team members express confidence in the potential of this technology.

“In the future, with the assistance of the intelligent ship situational perception system, we can accurately grasp the surrounding environmental situation and easily navigate the ship”, asserted Professor LIU Zhilin, a member of the “Dolphin 1” team. The team has developed a comprehensive information fusion software system that integrates data from hardware such as panoramic vision, lidar, solid-state radar, with information from electronic nautical charts, BeiDou navigation, real-time satellite remote sensing, etc. This system analyzes and calculates synchronized with a digital ocean environment, achieving intelligent perception of ship navigation situations and guiding the remote control of ship navigation.

The lidar on “Dolphin 1” carries the genes of the Chang’e lunar probe. Developed by the Shanghai Institute of Technical Physics, CAS, this lidar has been successfully applied to the Chang’e 3, 4, and 5 lunar probes. The lidar for “Dolphin 1” is designed to detect targets up to 4 kilometers away, significantly surpassing the 100-meter detection range of space lidar.

On the maiden voyage of “Dolphin 1”, the “maze” encountered in the breeding farm was navigated using a device called “solid-state radar”, developed by Beijing Hailanxin Data Technology Co., Ltd. According to LI Changwei, the chief engineer of the company, solid-state radar effectively marks floating balls in the breeding area by suppressing clutter characteristics.

“Dolphin 1’s” visual system includes both infrared and visible light. Comprising two sets of 360° panoramic vision systems, each with 12 spliced high-resolution cameras, it quickly captures visual information during navigation. The information is processed and transmitted to the remote measurement and control center and the onboard navigation perception system as a seamless 360° panoramic image.

High-precision sensing devices, real-time information transmission, and advanced algorithmic models collectively signal a transformative change for future ships. Remote “proxy driving” might just be a typical application among these innovations.

From ZHENG He’s seven voyages to Columbus’s global journey, humanity’s conquest of the sea persists. “The Art of Harnessing the Sea” has entered the AI era, enabling humans to achieve maximum operational efficiency while mitigating sea risks and reducing energy consumption. With self-learning, self-adaptation, and self-evolutionary algorithms, ship systems can continuously iterate and optimize, much like smartphones, simulating “hypothetical” scenarios in the future and generating predictive response strategies.

Exploring the Blue Ocean of Intelligent Manufacturing for Ships

Ninety percent of world trade transportation relies on sea freight, making ship design and manufacturing crucial for the shipbuilding industry’s high-quality development.

In July, China’s independently developed, designed, and built pure battery-powered container ship N997, with independent intellectual property rights, undertook its maiden voyage.

In September, China’s first large-scale cruise ship, equipped with an integrated operating system and 5G coverage, completed its trial voyage.

Presently, the ship design and manufacturing industry emphasizes green and intelligent practices to promote transformation and upgrading.

“Ships differ from land transportation vehicles like automobiles, as their production and manufacturing involve multiple varieties, small batches, and complex structural systems, resulting in low standardization and modularity. Digital systems can compensate for this disadvantage”, highlighted Professor CAI Chengtao. Digital twin technology facilitates the formation of universal software modules for intelligent ships, such as perception, decision-making, control, environment, etc. This greatly promotes the modular “flexible customization” assembly of intelligent ships, accelerating the rapid development of ship intelligent technology.

“Dolphin 1” aims to establish a mobile maritime laboratory for the development of intelligent ship technology in China, expediting the iterative process of intelligent ship technology. “Dolphin 1” can be equipped with various intelligent systems, providing efficient and optimized solutions for the design of future intelligent ships through data accumulation and model analysis.

“The ‘Dolphin 1’ cabin is equipped with interfaces for connecting experimental equipment and necessary tools for making experimental tools”, introduced LIU Zhilin. The ship’s open system and modular design enable interfaces for all internal systems to meet the experimental needs of ship and marine technology equipment.

With the support of this “mobile laboratory”, the team successfully demonstrated and verified key technologies for intelligent ships, such as autonomous navigation, remote control, and multi-scale and multimodal intelligent ship perception. In the future, it will offer on-site testing and verification conditions for the independent research and development of software in the shipbuilding industry.

HEU is collaborating with units possessing industry advantages to jointly establish an industry alliance for intelligent navigation equipment. In the future, member units within the alliance can use “Dolphin 1” for experiments, actively promoting the development of the intelligent shipbuilding industry.

“Digitization is the basis of intelligence, industrial software is the means to achieve digitalization, and ship digital and intelligent control are the main direction of future ship development”, stated XIA Guihua. The upgrading of ship design, including green, intelligent, and other key core technologies, not only opens up a new blue ocean for the development of China’s shipbuilding industry but also showcases the “China created” style of green and intelligent high-end ship types in the international shipbuilding market.