Multi-resolution mapping revolution in space

It’s the early hours of 22 March 2024. A rocket blasts off from the Cape Canaveral Space Force Station in Florida, USA. On board is a small device.

It’s labelled “Multi-resolution scanning”, and it’s packed with CSIRO smarts.

Its goal: Test a new CSIRO-developed 3D sensing and mapping payload for a NASA robot called Astrobee, which operates inside the International Space Station (ISS).

Dr Marc Elmouttie is leading the project. For Marc, this launch was the culmination of years of hard work and extensive international collaboration.

But we’re getting ahead of ourselves. Let’s take a step back. What is the Astrobee anyway? Why does it need multi-resolution scanning? And what are they doing up in space?

Let’s start with Astrobee.

This system of three free-flying robots moves around the ISS autonomously, or via remote control, to document experiments or even move cargo. As modular platforms, they can be loaded with other technologies, one of which will be multi-resolution scanning.

Our multi-resolution scanning device brings together two pieces of CSIRO tech: Stereo-Depth Fusion and Wildcat Simultaneous Localisation and Mapping.

The CSIRO-designed payload consists of an array of sensors arranged in a special way – two cameras, three time-of-flight sensors and an inertial measurement unit.

You can think of this as the “body”. The “brains” consist of the onboard computer running our algorithms. They process the data coming from these sensors to generate 3D maps. They also track the motion of the robot through the ISS by plotting its trajectory.

It’s designed to slot into the Astrobee robot platform and roam the station, creating detailed internal maps.

Marc said integrating these two technologies makes for significantly more accurate data.

What will these high-resolution maps of the ISS be used for?

At this stage, we are demonstrating how multi-resolution scanning technology would work.

The team’s goal is to showcase that the device will produce reliable data to support various uses. This includes real-time localisation of robots, robot-astronaut interactions, or monitoring and tracking inventory including equipment, science experiments, and hull damage from micrometeoroids. 

The value of these tasks lies in their potential to significantly enhance operational efficiency and safety aboard spacecraft. For instance, by ensuring precise robot positioning, crew members can better coordinate tasks and interactions.

Meanwhile, monitoring for hull damage could enable proactive maintenance and reduces the risk of critical breaches caused by micrometeoroids.

Stereo-Depth Fuson and Wildcat SLAM were not originally developed for spaceflight. They were actually created for use about as far away from space as you can get – in underground mining and other terrestrial applications.

Wildcat SLAM won the most accurate object detection prize in the DARPA Subterranean Challenge, a global robotics competition for underground exploration.

This smart tech could have simply stayed underground. But a chance conversation with aerospace manufacturer Boeing changed that.

“We’ve been collaborating with Boeing on 3D imaging for a number of years. After one of our project reviews, we received a call asking, ‘could you do this on the ISS?’” Marc said.

“That request initiated a number of meetings between us and Boeing to flesh out the concept of operations for interior space vehicle operations, exterior space vehicle scanning and even off-world roving.”

“That’s when we really knew this project had the potential to not only get to space, but actually advance human exploration out to the Moon and beyond,” Marc said.

Once the goal of spaceflight was locked in, Marc brought together a multi-disciplinary team to prepare the payload for the ISS. He pitched a very different proposition to the technology’s previous use case.

The team needed to make sure the device could survive the stress of launch and handle the unique environment of the ISS.

Lauren Hanson is our Senior Mechanical Engineer for the project.

Lauren and the rest of the team spent many long days revising, improving, and preparing the device for its maiden space voyage.

The team relied on experienced researchers across robotics, software engineering, systems integration and more.

"Space is something wonderful in that it really unites all of us behind a common goal and passion,” Lauren said.

“It’s been great to be involved in a project where we can really take experts from their everyday jobs and pull them together in a common fashion.”

After months of hard work, the payload was hand-delivered to the NASA Ames Research Center in Silicon Valley, California. Here, it would undergo the final phase of testing and certification.

“The Boeing and NASA test facilities have been used heavily throughout the project and have really put the payload through its paces,” Marc said.

“There are tests for just about everything you can imagine – launch stresses, payload-robot integration, electromagnetic interference and so on.

These tests ensure the payload's secure attachment, deployment, and operation. They are crucial for its functionality in environments filled with electromagnetic fields, among other challenges.

At the end of January, the team finally got the word – multi-resolution scanning had passed all of NASA’s safety checks.

Marc and another team member Dave Haddon, travelled to Florida to see the launch up close.

"Knowing that our precious payload, the product of several years of effort from an amazing team across CSIRO, Boeing and NASA, was en-route to the ISS made it quite an emotional experience," Marc said.

"Just so grateful Dave Haddon and I got a chance to witness this live!"

With the payload safely on board the ISS, our team now waits for the device to enter the experiment schedule. It will begin by mapping Japan’s experimental Kibō module, with the potential to map the rest of the station afterwards.

“At this point, it’s out of our hands. We’ve done everything we can,” Marc said.

He and the team are waiting for the first on-orbit mission to be scheduled. It will be designed to test and validate their technology in the unique conditions of space.

“The astronauts will pair our payload to the robot and we’ll wake it up. Then flying and scanning can begin!” Marc said.

The multi-resolution scanning story is only just beginning. The data we receive will inform future iterations of this technology.

“This is really a jumping-off point for us. Once we’ve confirmed this type of payload can do the job, that opens up many new opportunities,” Marc said. 

“Unlike the ISS, the Gateway isn’t expected to always have a crew onboard. Multi-resolution scanning is a great way to ensure that station is continually monitored, so that astronauts know any areas of concern or action well before they arrive,” Marc said.

Lauren is looking forward to other deployments that could come further down the line, such as in support of bases on the Moon or Mars, or on rovers.

“Being able to have an accurate scan of that space to make sure that it’s human-ready before we send people back there is really exciting,” Lauren said.

“We’re seeing a growth of robotics and AI supporting those robotics, and that’s going to increase as we establish a permanent human presence on the Moon – and eventually, Mars too,” Marc said.