Procedural shapes (circles, lines, masks, text-like glyphs) often look “jagged” because shader maths produces perfectly sharp transitions, but screens are made of discrete pixels. This guide explains practical, shader-friendly anti-aliasing methods so your edges stay crisp in stills and stable in motion.
What “aliasing” looks like in shaders
When you draw a shape in a fragment shader you usually end up with a hard threshold: “inside the shape = 1, outside = 0”. On a pixel grid, a hard threshold means the edge can only land in a few discrete places, so you see stair-stepping (jaggies). When the camera moves, the edge hops between pixels and you get shimmering or crawling.
Anti-aliasing (AA) is the act of smoothing those transitions so pixels near the edge become partially covered, producing a visually smoother boundary. In real-time rasterisation you’ll often hear about MSAA/FXAA/TAA, but for procedural shader art the most reliable wins usually come from analytic smoothing: you control edge softness based on how quickly your edge changes across the pixel footprint.
Important distinction: WebGL antialiasing vs shader anti-aliasing
If you’re rendering WebGL to a canvas, the browser may apply multisample anti-aliasing to polygon edges when you create the context. In WebGL, this is controlled by context attributes (for example requesting an antialiased drawing buffer). That helps with geometry edges, but it does not automatically fix jaggies created inside your fragment shader (like a hard step() mask for a circle). For reference, see the MDN section on getContext() parameters when you request a WebGL context.
The core trick: replace hard thresholds with a soft transition
The simplest AA upgrade is: don’t use a hard step() at the boundary. Use a small blend region so edge pixels become fractional. In GLSL this is typically smoothstep().
From step() to smoothstep()
Here’s a typical “inside/outside” mask:
// Hard edge (aliased)
float mask = step(0.0, -sd); // sd = signed distance; inside when sd < 0
Replace it with a smooth transition around zero:
// Soft edge (basic anti-aliasing)
float aa = 0.002; // a small thickness in "shape space"
float mask = smoothstep(aa, -aa, sd);
This already looks better, but there’s a catch: a fixed aa value works only at one scale. Zoom in/out (or change resolution) and the edge will become too blurry or too sharp again.
Resolution-independent AA with fwidth()
To keep edges consistently crisp across resolutions and zoom levels, you want the smoothing width to match the pixel footprint of your function. GLSL gives you screen-space derivatives, and the helper fwidth(x) (roughly abs(dFdx(x)) + abs(dFdy(x))). For signed distance fields (SDFs) and similar edge functions, this is a great default.
Analytic AA for an SDF edge
// sd: signed distance to the shape boundary (negative inside, positive outside)
float w = fwidth(sd); // estimates how much sd changes across a pixel
float mask = smoothstep(w, -w, sd); // soft edge proportional to pixel size
Why it works: if sd changes rapidly across the pixel, the smoothing width becomes larger to prevent flickering. If it changes slowly (big shape / zoomed-in), the smoothing width becomes smaller so the edge stays sharp.
Circle example (SDF)
// p: centred UV in aspect-corrected space
float circleSDF(vec2 p, float r) {
return length(p) - r;
}
float circleAA(vec2 p, float r) {
float sd = circleSDF(p, r);
float w = fwidth(sd);
return smoothstep(w, -w, sd);
}
This pattern generalises: if you can express your edge as “distance-like” value that crosses 0 at the boundary, fwidth() + smoothstep() is a strong starting point.
Anti-aliasing without an SDF
Not every procedural edge begins life as a signed distance field. You might have a “height” function, noise threshold, or a pattern that creates thin stripes. The same principle still applies: smooth the threshold based on how fast the function changes across the pixel.
Anti-aliasing a thresholded value
// v: some scalar field, threshold at t
float w = fwidth(v);
float mask = smoothstep(t - w, t + w, v);
Use this when you’d otherwise write step(t, v). It reduces shimmer when v animates over time (for example, scrolling noise or rotating patterns).
Lines and strokes: thickness + AA
For strokes, you usually want a band around an SDF. A common approach is “distance to curve/edge” and then build a stroke mask with two smooth edges: outer and inner. The key is to use the same pixel-aware width for both edges so the stroke doesn’t flicker when it gets thin.
Stroke from an SDF
// sd: signed distance to the centre line (0 at the line)
// halfWidth: stroke half-thickness in shape space
float strokeAA(float sd, float halfWidth) {
float w = fwidth(sd);
float a = smoothstep(halfWidth + w, halfWidth - w, abs(sd));
return a;
}
If your stroke becomes very thin, you may still see popping. In that case, clamp the effective smoothing a bit so you always get at least a tiny blend region:
float w = max(fwidth(sd), 1e-4);
Why your edges still look “wrong” sometimes
1) Gamma and colour blending
Edge pixels are blends between inside/outside colours. If you blend in a non-linear colour space, edges can look too dark or too bright. If you notice “dirty” outlines, try doing your blending in linear space (or keep your palette simple). For many shader GIFs, the most important part is consistency: avoid extreme contrast at subpixel edges unless it’s a deliberate style.
2) Tiny features below one pixel
No AA method can magically represent detail smaller than a pixel. If you have micro-stripes or very high-frequency noise, the best fix is to reduce the frequency (larger features) or use a filtered version of the signal. For animated patterns, consider limiting contrast at the smallest scales to reduce shimmering.
3) High-frequency procedural noise
Hard-thresholded noise is a classic shimmer factory. Prefer smooth noise (value noise / simplex) and avoid “knife-edge” thresholds. If you need a binary look, apply the derivative-aware thresholding method shown above.
4) Export/downscaling artefacts
When exporting to GIF, you often reduce resolution or colour depth. Both can exaggerate aliasing. If your tool supports it, a practical workflow is: render at a higher internal resolution (or supersample), then downscale for export. This is effectively brute-force anti-aliasing and can noticeably improve edges—at the cost of render time.
Practical recipe you can reuse
If you’re not sure where to start, follow this checklist:
- Find the edge function: the value that crosses a boundary (SDF distance, a thresholded noise value, a stripe function).
- Estimate pixel footprint: use
fwidth(edgeValue). - Smooth the transition: replace
step()withsmoothstep()using the footprint as width. - Clamp if needed: add a minimum width to avoid “too sharp” edges at extreme scales.
- Test in motion: aliasing problems often only show when animating.
If you want a quick partner for iterating on GLSL snippets (SDF helpers, stroke functions, and AA wrappers), this breakdown of the top AI coding tools covers which assistants stay reliable with shader-style code and longer prompts.
Example: AA rounded rectangle (SDF)
Rounded rectangles are common in UI-like shader designs. Here’s an SDF-based approach with proper AA:
float roundRectSDF(vec2 p, vec2 b, float r) {
// p: centred position
// b: half-size of the rectangle
vec2 q = abs(p) - b;
return length(max(q, 0.0)) + min(max(q.x, q.y), 0.0) - r;
}
float fillAA(float sd) {
float w = fwidth(sd);
return smoothstep(w, -w, sd);
}
// Usage:
vec2 p = uv * 2.0 - 1.0;
p.x *= ratio; // keep shapes correct on wide canvases
float sd = roundRectSDF(p, vec2(0.6, 0.35), 0.12);
float rect = fillAA(sd);