The Rosetta orbiter gave humanity one of its closest looks at a comet, ever. The data collected during Rosetta's orbit of 67P/Churyumov-Gerasimenko was incredibly exciting, with a dozen different instruments competing for resources to collect data streams, including multiple cameras, microwave, ultraviolet, spectrometer, and more.

One of the key elements of spacecraft instrumentation is power and resource allocation; there isn't enough juice to keep everything measuring and sending data back all the time; you have to pick and choose. In the old days (and even still now for many missions), scientists would debate and schedule time, power, and bandwidth for their instrument to collect and send back data. But, what some people (such as JPL's Artificial Intelligence group) have been pushing for is to have a more adaptive decision-making process, to maximize interesting data.

In collaboration with a researcher in the machine learning group, William Huffman, we aimed to show that image processing could be robustly used for a uniquely shaped object like the multi-lobed comet 67P to identify and track plumes - transient targets of high interest, because of the insight that can be gained from measuring the expelled contents. The main idea: automatically identify plumes using a broad field-of-view camera, so that a narrow field-of-view instrument like microwave (MIRO) could hone in on them to measure their contents in real time.

The goal of this project was to show that image processing could be robustly used for a uniquely shaped object like the multi-lobed comet 67P to identify and track plumes - transient targets of high interest, because of the insight that can be gained from measuring the expelled contents. The main idea: automatically identify plumes using a broad field-of-view camera, so that a narrow field-of-view instrument like microwave (MIRO) could hone in on them to measure their contents in real time.

Some unique challenges of this project:

• Plumes come in all shapes and sizes

• The contrast between light and dark in space with no atmosphere is extreme; the dark parts of a comet are darker than surrounding space, making traditional image segmentation challenging

• There is no ground truth of whether a plume exists and how far it extends

• Geometric estimates of the body of the comet are dependent on timing, spacecraft distance from the body, and angle of the camera, and the latest models - making automatic detection of the body based on geometry alone not completely reliable