Texture Synthesis by Non-parametric Sampling

A Unity implementation of the Efros & Leung (1999) texture synthesis algorithm, featuring both a CPU reference implementation and a GPU-accelerated compute shader version capable of synthesizing textures at resolutions up to 2048×2048.

---

Results

Sample (Input)	Synthesized (Output)
WindowSize: 31 x 31px, MaxErrorThreshold: 0.1	OutputSize: 128 x 128px
WindowSize: 27 x 27px, MaxErrorThreshold: 0.1	OutputSize: 128 x 128px
WindowSize: 15 x 15px, MaxErrorThreshold: 0.1	OutputSize: 128 x 128px

---

Algorithm Overview

The algorithm synthesizes a new texture by growing it one pixel at a time, always matching the local neighbourhood of each unfilled pixel against patches from the sample image.

Pipeline:

1. Seed — A small random patch (3×3) is copied from the sample image and placed at the centre of the output.
2. Grow — On each iteration, all unfilled pixels that are adjacent to at least one filled pixel are collected into a candidate list.
3. Neighbourhood extraction — For each unfilled pixel, a W×W window is extracted from the (padded) output image.
4. Gaussian weighting — A Gaussian mask is applied to the neighbourhood so that pixels closer to the centre contribute more to the match score. Only already-filled pixels participate in the distance calculation.
5. Patch matching — Every valid W×W patch from the sample image is compared against the neighbourhood using a weighted SSD (sum of squared differences). All patches within (1 + ε) × d_min of the best match are kept.
6. Stochastic selection — One patch is chosen at random from the shortlist and its centre pixel is written to the output.
7. Error relaxation — If no pixel is placed in an entire pass, the maximum error threshold is relaxed by 10% and the pass is retried.

---

Project Structure

Assets/
├── Shaders/
│   ├── TextureSynthesis.compute   # GPU kernels (GetCandidates, FindMatches, etc.)
│   └── GaussianBlur.shader        # Utility Gaussian blur shader
├── Scripts/
│   ├── TextureSynthesis.cs        # CPU reference implementation
│   ├── TextureSynthCS.cs          # GPU compute shader controller
│   └── GrowImage.cs               # (Experimental) growth utility
├── Materials/                     # Scene materials
├── Textures/                      # Sample input textures
└── Scenes/                        # Unity scene files

---

Reference

• Efros, A. A., & Leung, T. K. (1999). Texture Synthesis by Non-parametric Sampling. Proceedings of the International Conference on Computer Vision (ICCV). PDF