3 Ways to Remove a Background From a Photo (and When Each One Fails)

"Remove the background" sounds like one problem. In practice it's solved three completely different ways, and knowing which one you're using explains why a tool that works perfectly on a product photo falls apart on a photo of hair, glass, or smoke.

1. Chroma key (color-based)

This is the oldest method: pick a color (usually green or blue), and cut out every pixel close to it. It's how weather forecasters stand in front of a map.

• Works when: the background is a single, flat, controlled color that doesn't appear in the subject.
• Fails when: the subject wears that color, or the background isn't uniform — a photo taken against a white wall with uneven lighting will have shadows that chroma key either keeps (grey isn't "white enough") or eats into the subject's edge.

It's fast and predictable, but useless for a normal photo with a real-world background, because real backgrounds are never one flat color.

2. Edge detection (contrast-based)

This method looks for sharp boundaries in contrast or color gradient and assumes the subject is the region enclosed by the strongest edges. It doesn't need a special-colored background, so it works on ordinary photos.

• Works when: the subject has clean, high-contrast edges against the background — a coin on a dark table, a printed document on a desk.
• Fails when: the edge is soft or ambiguous. Hair is the classic failure case: each strand is a separate low-contrast edge, so the algorithm either cuts a hard outline around the whole head (looks like a helmet) or gives up and leaves a halo of background pixels.

3. ML segmentation (learned)

Modern background removers use a neural network trained on millions of photos with hand-labeled subject/background masks. Instead of following a rule ("cut at the edge," "cut at this color"), the model has learned what subjects — people, animals, products — statistically look like, pixel by pixel, and predicts a mask directly.

• Works when: the subject is a common category the model was trained on (people, pets, everyday objects), even with messy or low-contrast backgrounds.
• Fails when: the "background" isn't really separable — a photo through a window, smoke, motion blur, or fine translucent material (a veil, a wine glass) still confuses it, because the model has to guess a hard yes/no mask for pixels that are genuinely a blend of foreground and background.

This is what Spellkit's Background Remover uses. It runs the segmentation model in your browser via WebAssembly, so the photo never leaves your device — there's no upload step, which also means no size limit imposed by a server timeout.

Why the output sometimes has a faint edge

Even ML segmentation outputs a mask as a probability per pixel, then thresholds it into "keep" or "cut." Semi-transparent or blurred edges (hair, fur, fabric) sit right at that threshold, which is why a cutout can show a faint white or dark fringe — it's not a bug, it's the model expressing genuine uncertainty about a handful of edge pixels. Zooming in and cleaning a few pixels manually usually fixes it faster than trying to find a setting that removes it entirely.

Picking the right tool for the job

If you control the shot, chroma key is the highest-quality option — a real green screen with even lighting beats any algorithm run after the fact. If the photo is already taken and unstaged, ML segmentation is the only method of the three that scales to that mess.