Finding pins with computer vision

26-12-2025

Although it seems like overkill is the name of the game nowadays, I tried a few different approaches at a computer vision application which picked out macromollecular crystallography pins from a background, and this is how it went:

Preface

All "final" images on which processing was run were the result of averaging over 10 images, since the pins are submerged in a LN2 bath. This means the images are "blurrier", making some of the process more difficult.

Attempting to run the process on each one of the 10 images yielded no better results, as the fog clouded all images.

Attempt 1: Hough transforms

Hough transforms felt like the most natural choice, as they are "the" go-to for finding circles in images. However, the uneven shape of the pins (which usually look circular and metallic, but not completely) made this difficult.

It would work "OK" for pucks (larger circles upon which pins are placed), but pins themselves would not pop against the background once the image was greyed and blurred further to reduce noise.

No matter what, parameters, DP ratio, I could not achieve a sufficiently low false positive/negative ratio, nor could I reliably detect unpopulated pins (which are only 1 or 2 cm smaller in radius than a populated pin)

Attempt 2: Blob detection

If I could run an edge detection algorithm, I could calculate the area of the blob (which may not be a circle), and then verify if the area is closer to a pin/non-pin.

Unfortunately, the same issue that plagued Hough transforms arose: not enough contrast. Even by bumping up the contrast artificially, introducing more aggressive mechanisms that would make features pop, it would not work.

Attempt 3: KNN clustering/colour averaging

Instead of obtaining the bounds of each pin, what if I selected the areas in which I knew pins sat, and got the average colour from that? Darker colours would mean no pin, and lighter colours would mean presence of metal, meaning a pin was likely to be present.

Yet again, that was foiled, as noise/different puck colours would mess with the average colour. So instead, I took to KNN clustering to find the dominant colours (similar to my mosaic building adventures).

That wasn't reliable either - sometimes the fog would be too strong, and the puck/pin would meld into one, just enough to be discernible for a human, but not for a machine.

Attempt 4: Single class YOLO model

Although I avoided ML like the plague, this felt like the best option. I trained a single class YOLO model to detect pins, and if said pins fell under the area under which a pin normally sat, I would consider that area to have a pin.

Since single class models are a pain to train due to the lack of negative examples, that also had less-than-optimal yield, with plenty of false negatives.

Attempt 5: Binary classification model

Binary classification seemed like the best fit - either something had a pin or didn't, and would make my life much easier. I could crop the area in which I suspected a pin may reside, and predict the presence of a pin in there.

That went remarkably well - 98% accuracy, only a dozen of false positives/negatives over >350 pins, but that still wasn't accurate enough.

The reason being that between new loads of pucks being loaded, the pins/pucks would shift ever so slightly in the camera's FOV, making hardcoding their positions impossible. I had to then assess where these sat, and build out my matrix of pin positions at runtime.

Puck positioning correction

I went back to the original method for this step - Hough transforms. They performed well enough for larger circles with more prominent features and edges, so that was enough for pucks/the outer bounds of the LN2 bath.

So far so good, I could find the centre of the bath reliably, but a problem presented: the lid would sometimes overlap with the edges of the bath, making the detected circle off center, but ever so slightly so.

Because of it, I've decided to search for the "centre puck" in a 200x200 area in what I knew to be the centre, and use that as a reference to correct for the error induced in the original measurement (as caused by the positioning of the lid).

Since all distances will always be constant relative to the centre of the bath, regardless of where the camera was positioned, we only need this one reference point to perform "dead reckoning" to find each puck. However, the camera's angle and distance from the bath may alter the pixel distance, so although the actual distance doesn't change, the perceived distance does.

We can correct this by taking at least one puck from each one of the concentric puck circles, and once again performing Hough transforms to get its actual distance and angle offset from the centre, and we can infer the positions of all other pucks in that one circle based on the first one (given the pucks follow a periodic positioning pattern).

Conclusion

This is just a silly project, but it's an example of how one can coordinate ML-based approaches to computer vision alongside more traditional techniques to get the best out of both worlds, all in accordance to the law of least power. I couldn't have done this without models, nor would it be wise to try and bruteforce this with models.

As a matter of fact, I'd have to get the positions/names of each pin regardless, so I would have to find a non-model approach for positioning pins anyway.