How we’re using machine learning to detect coral-eating COTS
The Great Barrier Reef is one of Australia’s most diverse and unique landscapes. Covering more than 2,900 individual reefs, it is home to unmatched marine biodiversity. A multidisciplinary group of researchers from Australia’s national science agency, CSIRO, have been working on projects using innovative science and technology to help combat some of the threats facing our reef.
The team have worked with a range of stakeholders, most recently joining forces with Google and the international Kaggle community to explore ways to help with the monitoring and detection of crown-of-thorns starfish.
Meet the team and learn more about their work here.
Dr Brano Kusy
Dr Brano Kusy is an internationally respected scientist and research group leader with CSIRO’s Data61. Dr Kusy’s work focuses on new frontiers in networked embedded systems, mobile and wearable computing, and Internet of Things.
Brano, tell us about your work on the Great Barrier Reef and what attracted you to it?
My research interests are at an intersection between digital technology and the physical world. Digital technology delivers high value in land environments, however, coastal ecosystems such as coral reefs remain poorly understood. This is due to their size, that seawater hides detail from remote sensing methods in all but the shallowest marine ecosystems, and the general difficulty of operating digital technology in remote marine environments.
I have championed a multi-pronged approach to solve reef challenges that relied on CSIRO’s in-house technologies, such as Internet of Things, robotics, machine learning, and computer vision.
We have developed biosensors that can monitor feeding of coral trouts and physiology of oysters, a new underwater hyperspectral imaging platform, and a robust method for detecting Irukandji jellyfish based on eDNA contained in seawater.
Can you tell us more about the machine learning technology behind the crown-of-thorns starfish surveys?
The COTS monitoring application is the culmination of edge ML (machine learning) and imaging technologies developed over the past four years.
It is based on a close collaboration of CSIRO computer vision and edge ML experts with Google and Kaggle and it is a shining example of ML technology helping to protect the environment.
We have built an edge ML platform for oceans that can analyse underwater images as they are collected by marine scientists in the field and basically uncover the hidden world under the surface through an intuitive touch-screen interface.
In the COTS monitoring use case, the ML platform processes the images in real-time and shows the survey team on the boat how many COTS have been detected and their whereabout.
The beauty of this approach is that it is not locked in – it generalises too many applications and devices. We demonstrated it works amazingly well for mapping COTS on coral reefs, but the method can be adapted for sea cucumbers in a sustainable aquaculture context, seagrass biomass for carbon accounting, or surveying condition, health, and diversity of sea life on coral reefs for climate impact assessment.
Additionally, our platform works with many different data collection technologies and supports multiple ML software frameworks, all you need is a wired or wireless connection from your data collection platform. We demonstrated the platform with in-house data collection technology, real-time GoPro camera streams, commercial Pro Squid platform, and will be adding more in the future
What role can digital sciences play in ensuring the sustainability of our natural environments?
One of our major objectives was to scale our invention to increase global impact. This was achieved by allowing fellow researchers to use our technology to explore the plethora of opportunities in this space.
In collaboration with Google TensorFlow team, we open sourced the COTS ML model and workflows under Apache license. This allowed students, scientists, and entrepreneurs worldwide to evaluate our ML technology with their own image datasets and extend it to suit their application.
The ML model training toolchain will be released soon to retrain the ML models for other species or object identification. By democratising ML capabilities in this space, we can make a tangible difference in ocean and marine life protection.
How important are partnerships to this kind of work?
It’s impossible to overstate the importance of partnerships and open sharing of scientific ideas in this line of work. In addition to our technology being inherently multi-disciplinary (designed by computer geeks like me, but used and interpreted by marine scientists), careful planning is required to deploy the technology prototypes reliably and safely at sea. Conditions can change in an instant and internet connectivity is non-existent.
The project team needs to work as a tight-knit unit. We are very fortunate to have worked with some of the most competent and experienced crews in Australia. Shout out to University of Queensland’s Heron Island and Moreton Bay research stations, University of Sydney’s One Tree Island research station, Blue Planet Marine, and GBR Marine Park Authority.
It was also a great privilege to work with Google Tensorflow and Kaggle teams. Having access to the latest ML expertise and hardware resources coupled with the global reach of both brands was pivotal in getting the message out. Over 2,000 international ML teams participated in the competition, the video was viewed 26 million times, and we were featured in a keynote atGoogle’s annual I/O conference.
Dr Joey Crosswell
Dr Joey Crosswell is a biogeochemist with broad research interests across oceanography and engineering. His research includes diverse environments around the word, ranging from mangroves and mesoscale eddies, to arid tropical estuaries in northern Australia and fjords in Patagonia.
Joey, tell us about your work on the Great Barrier Reef and what attracted you to it?
My research focuses on the connectivity of coastal systems, particularly carbon and nutrient cycling between land, ocean, and atmosphere. The Reef is particularly interesting in this regard because it is one of the largest and most complex coastal ecosystems in the world. For example, human activities far up in river catchments and oceanic processes that start on the other side of the Pacific come together in the GBR to affect the health and resilience of the Reef.
My work looks at untangling these processes across the multiple time and space scales by using novel observation methods combined with advanced modelling tools, such as eReefs. This multi-scale understanding is important for managing the Reef because it informs where meaningful local actions can be taken, such as restoration, through to needs for larger-scale efforts such as global climate action.
I have worked in estuaries and coral reefs along the entire coast of the GBR, but I am particularly interested in those further afield. That is, the more remote, the better. These systems provide a valuable comparison that help us gauge the impact of coastal development and future change.
The lack of existing data in many of these remote environments also presents the challenge of building a holistic understanding from the ground up, a task for which I think CSIRO’s research disciplines, researchers, and partnerships are uniquely suited. I also have a keen interest in extreme events such as cyclones and floods that are relatively brief but have lasting impacts.
We currently have a limited ability to resolve these events, and the development of new observational tools, methods and models for extreme conditions is one of my long-term research passions.
How important is multidisciplinary science and collaboration between different groups in this space?
Put simply, it is the only effective way forward.
Like the Reef faces combined threats from rising sea temperature, water quality, COTS and coastal development, so too must we employ cross-cutting science to support Reef resilience to these threats.
The benefits of multidisciplinary research are being widely recognised through programs like eReefs the Reef Restoration and Adaptation Program, and the COTS Control and Innovation Program. The COTS ML model that we recently developed through a cross disciplinary multi-institutional collaboration clearly shows how integrative research can drive a step change in technical methods that have otherwise made little progress for decades.
Moreover, closer coupling of research and management disciplines allows technical innovations to have a ripple effect that drives systemic change. In the COTS application, more and better data collected using ML COTS detection will enable more efficient decision support for active control measures.
New data dimensions unlocked by computer vision will also feedback to research on key relationships and thresholds, such as triggers for COTS outbreaks that can be proactively managed. Even more exciting than the potential of multidisciplinary science to mitigate threats is the potential to maximise benefits and ecosystem services.
Last but not least is the more personal aspect of multidisciplinary science. This blog highlights only a few members and accomplishments of much larger teams of which I am a part. Not only is it fun and fulfilling to learn from diverse expertise, backgrounds and perspectives, but it also expands the impact of our research to new environments and cultures.
Multidisciplinary research teams are a big part of why I enjoy what I do and who I do it with, which is particularly important when you spend a lot of time on small boats at sea!