Last spring, the world experienced a rare moment of collective joy and wonder when NASA’s Artemis II mission, the first crewed flight to the Moon since 1972, completed its historic lunar flyby. On April 6, astronauts Reid Wiseman, Christina Koch, Victor Glover and Jeremy Hansen flew further from Earth than any humans before them. Today, as NASA prepares for the next Artemis mission and beyond, the space agency is working with IBM to provide scientists with new tools to study the Moon.
On Thursday, the two organizations released the NASA-IBM Lunar Foundation model. It is an open source AI system available for download from Hugging Face. As a basic model, it can do different things, says Dr Juan Bernabé-Moreno, director of IBM Research Europe, UK and Ireland. For starters, NASA and IBM found that they were particularly good at identifying areas of the lunar surface where there might be ice.
The two organizations conducted a test in which they compared the model’s predictions to a map they had created using a published scientific workflow integrating terrain, thermal, and other environmental data. They then compared the model to SwinV2-B, a vision system trained by Microsoft for processing high-resolution images. The NASA-IBM model reduced errors by 23 percent. NASA and IBM compared the model to SwinV2-B because the tool is often used as a baseline for many image analysis tasks.
NASA and IBM also evaluated their model’s ability to identify and classify craters. There, the new model outperformed SwinV2-B by 19% while using half the training data. NASA and IBM recently had the opportunity to check out the model’s capabilities when a SpaceX Falcon 9 rocket crashed into the Moon on August 5. When IBM fed an image of the impact to the model, it correctly identified the crash site as a new crater, although it closely overlapped with an existing crater. “It worked wonderfully,” says Bernabé-Moreno, with the model successfully identifying on its first attempt.
Bernabé-Moreno says training the model was a challenge. “In Earth observation, you have impeccable images,” he explains. Our planet’s atmosphere diffuses sunlight, filling shadows with ambient light that softens their edges. Shadows on the Moon appear rather sharp and black, meaning that whenever they are photographed, the shadowed pixels contain no information. “On the Moon, you have the sun at different times of the day, it illuminates and cuts shadows, and you have to manage those shadows,” he says. The result is that a crater can look different in one image and completely different in the next image.
Another challenge arises from the way computer vision models are typically trained and tested. Most often, this process involves removing certain parts of an image and then instructing the model to reconstruct what is missing based on the patterns learned during training. “You can remove 90 percent of a crater and with the (remaining) 10 percent you ask the model to reconstruct the remaining 90 percent,” explains Bernabé-Moreno. This doesn’t work for the Moon. As you can imagine, the reason is that many craters look very similar when photographed from orbit.
In fact, IBM’s attempts to train the model the traditional way were a “total disaster,” says Bernabé-Moreno, but the research team ultimately found a creative solution. “We took the Moon and divided it into quarters, like an orange, and we completely separated the training quarters from the test quarters. That way we were able to give a lot of consistency to the model,” he says.
One of the reasons NASA and IBM are excited about this release is that it includes a first-of-its-kind open source dataset that other researchers can use to create future models. It integrates tens of thousands of images and instrument data from NASA’s Lunar Reconnaissance Orbiter (LRO) and Gravity Recovery and Interior Laboratory (GRAIL) missions, as well as Japan’s Selenological and Engineering Explorer (SELENE).
“One of the reasons we haven’t seen a complete, powerful lunar model before is that we didn’t have the data (organized) in the right way,” says Bernabé-Moreno. In this case, organization meant creating a grid in which each pixel of imaging data corresponds to other data modalities that a model can use to inform its processing. “This alone represents a massive scientific contribution, because apart from the model: the community now has a co-registered dataset with more than two million (data) points,” explains Bernabé-Moreno. “And we know that AI models are being replaced by other models. It’s the data that really creates the AI model industry.”
