I faltered quite a bit in my paper reading plans. I had thought to take a step back and read some book about machine-learning-and-friends basics instead. That went ok until first free time ran out for a bit, then I started a new job on top of that, and then I moved cities. You know, life. My non fiction reading plans fell a bit by the wayside. I got some new motivation seeing this paper pass by in my day job and decided to have a look. The paper is an application of machine learning.

Speaking of which, I never really thought I would be writing this, but given the connection with my new job, I feel like I should. I now work at the Space Telescope Science Institute. We deal with the ingestion, processing, archiving, and organising a bunch of things around Hubble Space Telescope, James Webb Space Telescope, and the upcoming Nancy Grace Roman Space Telescope. Gosh that still sounds cool.

Anyway, all the opinions expressed here are my own, not my employer’s. With that in mind…

Today’s paper is Identifying astrophysical anomalies in 99.6 million source cutouts from the Hubble legacy archive using AnomalyMatch by David O’Ryan and Pablo Gómez. Published in “Astronomy & Astrophysics”, volume 704, December 2025. DOI 10.1051/0004-6361/202555512.

Caveat before continuing: I am new to all the scientific astrophysics terminology, so double check anything I say.

What’s It About?

Space is big [Citation Needed]. Despite being largely empty, there are a lot of objects in every little corner of the sky you look from our point of view. If you have a massive telescope hanging in space, constantly taking pictures (for some definition of pictures) of all different parts of the sky, then it turns out there is a lot of data to look at. All that data that has been flowing since Hubble launched on 24 April 1990 has been getting stored and made available to researchers and the public. Going through it all manually is hard. Going through it all manually to find potentially interesting objects in the sky, anomalies that deserve further research, is even harder.

In comes this paper, using “AnomalyMatch”. AnomalyMatch is a tool and technique the authors introduce in a companion paper (“AnomalyMatch: Discovering Rare Objects of Interest with Semi-supervised and Active Learning” by Pablo Gómez, Laslo E. Ruhberg, Maria Teresa Nardone, David O’Ryan). I did not read the companion paper. The premise of the tool is as follows. We are interested in anomalies. Anomalies are here defined as astrophysical objects with morphological characteristics that deviate from the general population. “It’s in space and it looks different”. Anomalies, by their nature, do not occur very often, making it harder to train a model to discover them for you. AnomalyMatch can start from a small set of labelled anomalous samples and go from there. Using the small starting set (and a bunch of data), it spews out what it thinks is anomalous too, i.e., it gives a 0 to 1 rating for every image you give it. It then requires a feedback loop with someone with knowledge on the subject. That person will mark what they think also counts as anomalous, then the updated model goes again. Do that a few times till you are happy with the result.

In the paper I’m describing here, they apply this tool on data from the Hubble Space Telescope. Now, Hubble comes with a few instruments. The paper only considers data from the Advanced Camera for Surveys (ACS) Wide Field Channel with the F814W filter. I know some of those words. The paper also only uses “observations at Calibration Level 3” which are “science ready mosaics”. Don’t know what that means either. Look, I’m new here, okay? Just think of it as photos.

The takeaway here is that they use 99.6 million “source cutouts” where a source cutout seems to be a 150×150 pixel greyscale image of a certain part of the sky. The part of the sky is 7.5”×7.5” large, where the " is an arcsecond (1/60 of a ' arcminute, 1/3600 of a ° degree), so this basically indicates how big (as perceived by us) the slice of sky is. Keep in mind here that the sky is a sphere around us, composed of just shy of 41 253 square degrees. Also, to try to picture that slice in the night sky. According to this website, if you hold your pinky at arms length, then its width is about 1–1.5 arc degrees. So 7.5 arcseconds is 7.5/3600 to 7.5×1.5/3600 meaning 0.2–0.3% of the width of your pinky at arms length. Tiny.

The authors started out with 128 labelled images. In the active learning phase they build that out to 1400 labelled images, i.e., the model throws images at them, the expert marks those they consider good and bad. Of those 1400 labelled images, 375 were anomalies, 1025 were nothing special to them. Of the anomalies, 178 were galaxy mergers and 63 were gravitational lenses. In the end, the training happened with those 1400 labelled images and about 99 000 unlabelled images.

After this training, they run AnomalyMatch on a subset of the images: 500 000 of them. The idea here is to have a look at what kinds of images the model gives high scores and low scores. A sort of sanity check before running it on all almost 100 million images. Part of their approach here is to overlay “saliency plots” with “SHapely Additive exPlanations (SHAP)” by Lundberg and Lee, 2017. In normal language: the overlay colours parts of the image red that were more important to the score and blue for parts that were less important. How you derive that out of your model, I do not know. I guess technically it could be derived from your internal matrices? Just spitballing here. In case it was not clear yet, the paper includes pictures, so for the full experience, you are really going to want to pop it open. It looks cool.

Finally it is time to run AnomalyMatch on the entire dataset. For this, as well as the training before it, they use the European Space Agency’s “ESA Datalabs” platform. ESA Datalabs has all the data from Hubble and several missions loaded in and enables a researcher to run their code within the platform. This avoids the need to download all that data and there is a decent chance that ESA’s hardware is better than what the researcher can bring to the table. In particular, the authors note that ESA Datalabs recently added an integrated GPU cluster that worked wonders for their purpose. The training had completed in under four hours. Running it on all 99.6 million images took 2.5 days. I just had a look, you can’t just give ESA Datalabs a spin yourself: to get an account you have to send in a proper request. I don’t think “I just want to have a peek and see what I can do” will count as a good reason.

The Results

Results here are simple enough: what got marked as anomalous? Is it really anomalous? As I mentioned earlier, AnomalyMatch gives a score of 0 to 1 to images. The higher the score, the more anomalous. For evaluation, the authors selected the 5000 highest scored images. They deduplicated those 5000 images by looking at their coordinates. If two anomalies are close enough (10 arcseconds) together, then assume it is a duplicate. It is unclear to me which image they then keep. Do they look at a duplicate set together? Do they keep just one and don’t think about it later? Space changes very slowly, so presumably the picture really does look the same across different images. Either way, deduplication leaves them with 1339 unique images.

For those 1339 unique images, the expert looks at them and gives their own assessment of the image. Each image gets classified into what type of anomaly it is or as a false positive of some sort. The literature is also consulted to determine whether any papers had discussed the object before (the “Referenced” column here). Apparently you can somewhat easily search for that using SIMBAD and ESASky.

Anyway, time for some numbers! This is how they classified those 1339 images.

Classification	Nr Found	Referenced
Merging Galaxy	629	212
Gravitational Lenses	140	54
Odd Galaxy	164	60
Nominal Galaxy (false positives)	163	60
Unknown Morphology	43	0
High Redshift Galaxy	28	7
Jellyfish Galaxy	35	17
Overlapping Galaxy	39	26
Gravitational Arc	39	19
Clumpy Galaxy	11	5
Galaxy Hosting a Jet	13	5
Galaxy Hosting an AGN	8	4
Ring Galaxy	12	5
Lensed Quasar	5	5
Collisional Ring Galaxy	2	0
Edge-On Protoplanetary Disk	2	2
Galaxy Hosting Supernova	2	2
Submillimetre Galaxy	1	1

Here a brief description of each, often taken from the paper.

Merging Galaxy: two or more galaxies lying at ± same redshift (overly simplifying, but same redshift boils down to same distance from us, so their proximity is not just an optical illusion of our night sky, redshift is an elegant concept once I got my head around it) and showing signs of gravitational effect on one another.
Gravitational Lenses: lucky alignment of two galactic systems. The gravity of the front one bends the back one’s light around itself in a ring. We can reconstruct the original image in higher detail than without the front one.
Odd Galaxy: some weird shape, but they cant fit it in one of the main categories. Possibly the result of one or more interactions. No literature.
Nominal Galaxy: false positives. Looks like just a regular galaxy.
Unknown Morphology: “completely defies classification based on morphology”, may not be galaxies to begin with.
High Redshift Galaxy: seems to be used by the authors like a bit of a catch all for things where they have trouble distinguishing what is what. Some of the findings are also described as “high-redshift galaxy” in literature. Going by what I understand of redshift: really far galaxies, their light travelled a long time to get to us, them being high redshift means what we see of them happened a long time ago.
Jellyfish Galaxy: due to its presence in a denser area of space, it experiences ram pressure stripping. Think of that as bumping into things as it moves through the denser area. The front of the galaxy gets condensed into a bow shock: bumping into things slows the front down and pushes it together. A tail forms of parts getting left behind after they bump into things.
Overlapping Galaxy: look close together in 2D, but actually nowhere near in 3D.
Gravitational Arc: think this is the same idea as gravitational lenses, but not a complete ring? Found the explanation a bit confusing.
Clumpy Galaxy: gas rich systems with large and luminous star forming clumps.
Galaxy Hosting a Jet: while a suppermassive black hole grows, jets form on either side of its galaxy. Relatively common in other wavelengths (radio, x-ray), rare at optical wavelengths that Hubble handles.
Galaxy Hosting an AGN: active galactic nucleus. Central supermassive black holes of galaxies, undergoing accretion and growth. I kinda assumed they all had black holes at their center, so guess that assumption was wrong.
Ring Galaxy: systems with large empty gap in the center, their formation is active research.
Lensed Quasar: another type of gravitational lensing, the system getting magnified is a quasar, resulting in typical brighter spots in the lens.
Collisional Ring Galaxy: somewhat relating to ring galaxies? Result of two galaxies merging in a specific direction. Explanation a bit vague to me.
Edge-On Protoplanetary Disk: exhibit a single, dark dust lane across their centres with a butterfly shape extending in perpendicular directions. Often host a jet, also perpendicular to the dust lane.
Galaxy Hosting Supernova: not explained, but guessing there’s a supernova taking up part of the picture. :)
Submillimetre Galaxy: not explained.

Anyway, that was mostly just me repeating their results because I thought it was cool. I wanted to more easily find back the different weird stuff in the future, so dumping it here seemed like the way to go.

They release their results for further investigation. Who gets the credit if any of these turns out to be important, I wonder?

Mostly, they point out that Hubble is used for specific investigations. A (team of) researcher(s) decides on a part of the sky they want to look at, they send a proposal to my company (as far as I understand), and a panel decides on its viability and determines which projects get executed. So anything weird that is found in this study is the result of it accidentally showing up in those other projects. As the authors point out, there are more survey oriented missions (Euclid or the Vera C. Rubin observatories) where the goal is to just take pictures of everything and dump that information out there for researchers to play with. The authors figure applying their tool on those datasets might prove more interesting in the long run.

My Rushed Conclusion

Now, I had originally planned to delve a bit more into (minor) questions I had. Maybe see what I can figure out on my own and add that here. However, I started these notes back in May and then things fell by the wayside again. It is now August and I just want to get this post out there. Soooo. Rushed ending. Here’s some final remarks and questions.

If there is already a “link this object or piece of sky to papers” connection, would it not be easy to go from that to labelled images? Giving a bigger starting set of examples, potentially letting the model discover more objects in the “find something new and fun now” phase. I don’t think I saw this option discussed. I’m also not sure why that would not have been made part of the database by now.
Could have maybe used a comparison of where the false positives / non interesting objects fell in the score distribution. Was that nearer the lower scoring ones of the top 5000?
Why not go for 2000 anomalies to handle manually after deduplication or some other round number? So keep grabbing from the top till you reach 2000 unique ones. Or some other roundish number that isn’t 1339.
Or use an anomaly score cutoff rather than a number of objects.
I do not like how this scientific community handles their end of text references. They are of the format Authors year, journal abbreviation, volume, id. So for example “Muylaert, W. 2026. ApJ, 500, 4”. Just put a title in there so I can more easily skim for interesting followups where I found the body text not clarify enough how they used this particular reference. Inline refs are fine of the format Author et al. (year).
My little laptop / PDF reader had some real lag trying to render some pages, especially after annotations on my tablet.
This shit is cool.
I need more free time.