ASCII Art

The image above is made entirely of text. Up close, every mark is a keyboard character: letters, punctuation and spaces. There is no picture underneath. So how does the computer decide which character goes where?

First rule: match the brightness

Start by splitting the picture into a grid of cells, one cell per character. Measure how bright each cell is on average.

Every character has a brightness too. Drawn in white on a black background, a dense character like @ or M lights up a lot of its cell; a sparse one like . or ' barely lights any; a space lights none. So the simplest rule is: for each cell, pick the character whose average brightness is closest. Dense letters for light areas, sparse marks for dark ones.

This is how most ASCII art converters work, and it gets the overall picture right. But it has a blind spot.

Brightness ignores shape

Two cells can have the same average brightness and look completely different: one is a flat patch of mid-gray, the other is a sharp diagonal edge between dark and light. Brightness-only matching gives them the same letter. Edges get filled in with texture instead of being drawn.

Second rule: match the shape too

So try a different rule. For each cell, render all ninety-five printable ASCII characters, from space to ~, and compare each one with the cell:

• Shape score: blur both the cell and the character slightly, remove their average brightness, and measure how different the remaining patterns are. This asks, “do the light and dark parts sit in the same places?”
• Brightness score: the squared difference between their average brightness, as before.

Then combine them: shape score plus a weighted brightness penalty. The lowest combined score wins.

The weight matters. With too little brightness penalty, characters follow edges well, but the picture’s tones drift and the portrait gets harder to recognize. In the reel’s version, the weight was chosen as the smallest value that kept the brightness error within strict limits, so the picture stays recognizable while the shapes improve.

Now a slash can follow a diagonal instead of just filling a shade. In the reel’s close-ups of the eyes and the smile, slashes, backslashes and parentheses were picked exactly where the painting has slanted or curved edges.

What the reel measured

The reel used the Mona Lisa, 120 columns by 90 rows, with the same font, grid and colors for both rules. On that grid, the shape-matching version had about 20% lower edge error, while its brightness stayed almost the same as the brightness-only version.

That is a measurement on one painting and one setting, not a universal quality score. The method is a simplified version of an idea from research on structure-based ASCII art (Xu, Zhang and Wong, 2010). It scores every cell independently and ignores how neighboring characters line up with each other.

What the controls change

• Columns sets how many characters fit across the image, from 16 to 240; the rows follow from the photo’s shape. More columns bring out fine detail, like eyes and hair. Fewer columns make each character big enough to read, which is fun for posting. Very tall photos are capped at 120 rows, so the actual column count may come out lower.
• Characters chooses the candidate set. All ASCII characters lets the shape score pick from all 95 printable characters, so slashes, brackets and underscores can follow edges. Simple limits the choice to the classic ramp .:-=+*#%@, which mostly express brightness. Switch between the two on the same photo to see how much the extra shapes help along edges.
• Invert flips the brightness the characters are matched against, so dense characters go where the photo is dark instead of where it is light.
• Color paints each character with the photo’s color at the center of its cell. The choice of character doesn’t change, only its color.

In every case, the brightness weight is picked the same way as in the reel: the smallest weight from a fixed list that keeps the tone error within the limits.

Try this

• Pick a photo with strong lines: a face in side light, a building, a pet with clear outlines. Shape matching shows its advantage on edges, not on flat areas.
• Use the split view and drag across an outline. You should see slashes, brackets and underscores tracing the edge.
• Lower the number of columns until you can read individual characters on your phone, then save the image for a story.

Your photo is processed on your own device. Nothing is uploaded.