Sketchy
MAT 594CM
Qian Liu
Drawing
Research Project
2010S
Sketchy Rendering
1. Generate intermediate rendering results that represent edges and surface colors of 3D geometry.
2. Apply uncertainty values in image space to sketch these textures nonuniformly.
3. Adjust depth information in such a way that the resulting sketchy drawing can be merged with general 3D scene contents.
Edges and Color Patches
In addition, render surface colors of 3D geometry to texture to produce striking color patches that appear flat, cover all surface details, and emulate a cartoon-like style (shade map). Figure (b)
-- Applying Uncertainty
2. Apply an additional texture whose texels represent uncertainty values.
3. Introduce a degree of uncertainty to control the amount of perturbation, which is a user-defined 2*2 matrix. Multiply uncertainty values derived from the Perlin noise texture by texture coordinates to weight all these values uniformly and then use the resulting offset vector to translate the texture coordinates.
Determined uncertainty values (offs , offt ) by using the turbulence function (based on Perlin noise):
offs turbulence(s, t); and offt turbulence(1 - s, 1 - t);
4. To enhance the sketchiness effect: perturb texture coordinates for accessing the edge and shade maps differently.
Apply two different 2*2 matrices, resulting in different degrees of uncertainty for each map.
One degree of uncertainty shifts texture coordinates of the edge map.
One shifts texture coordinates of the shade map—that is, shift them in opposite directions.
texture values correspond to fragments of 3D geometry = interior regions
texture values do not correspond to fragments of 3D geometry = exterior regions
combine texture values of the edge and the shade maps. Multiply the intensity values derived from perturbing the edge map with the color values derived from perturbing the shade map.
When rendering the quad textured with textures of 3D geometry:
1. z-values of the original geometry are not present in interior regions, and in particular.
2. no z-values of the original geometry are present in exterior regions when uncertainty has been applied.
Thus the sketchy rendering cannot correctly interact with other objects in the scene.
Solutions:
Apply depth sprite rendering by considering previous perturbations.
Additionally texture the quad with the high-precision depth map (already available) and access this texture twice using perturbed texture coordinates.
First perturbation, apply the degree of uncertainty used for accessing the edge map;
Second perturbation, apply the degree of uncertainty used for accessing the shade map.
The minimum value of both these texture values produces the final z-value of the fragment for depth testing. Figure c' shows the result of both perturbations applied to the depth map. The interior region of the perturbed depth map matches the combination of the interior regions of both the perturbed edge map and the perturbed shade map. Even those spots produced for the shade map appear in the perturbation of the depth map.
Download Video Example
Apply randomly chosen noise values; hence, adjacent texture values of the noise texture are uncorrelated. Consequently, the degrees of uncertainty that apply to the texture coordinates of adjacent fragments are also uncorrelated. The roughness and granularity, in particular for edges, vary as though the pressure had varied as it does when drawing with chalk. This effect depends on the amount of uncertainty that applys in image space.
Repeated Edges:
A fundamental technique in hand drawings is to repeatedly draw edges to draft a layout or design. Exclude the shade map but apply the edge map multiple times using different degrees of uncertainty and possibly different edge colors. Edges will then overlap nonuniformly as if the edges of the 3D geometry have been sketched repeatedly, as in Figure c.
1. Achieve frame-to-frame coherence, for instance, when interacting with the scene (because neighboring uncertainty values are correlated).
2. Access interior regions from beyond exterior regions and, vice versa, sketch beyond the boundary of 3D geometries.
Uncertainty values remain unchanged in image space and have no obvious correspondence with geometric properties of the original 3D geometry.
Consequently, sketchy drawings tend to "swim" in image space (known as the shower-door effect).
To overcome these limitations:
• Preserve geometric properties such as surface positions, normals, or curvature information to determine uncertainty values.
• Render these geometric properties directly into a texture to generate an additional G-buffer.
• Texture the screen-aligned quad with that additional texture, and then access geometric properties using texture coordinates (s, t).
• Finally, calculate uncertainty values based on a noise function, using the geometric properties as parameters.
Mathematically, use the following function to determine the perturbed texture coordinates (s', t'):
f : (s, t) {-- (s', t')
f (s, t) = p(s, t, g(s, t))
where (s, t) represent texture coordinates of a fragment produced when rasterizing the screen-aligned quad, g() provides the geometric properties available in the additional texture, and p() determines the perturbation applied to (s, t) using g() as input.
• Continue to provide uncertainty values in exterior regions, at least close to the 3D geometry.
• Enlarge the original 3D geometry to generate geometric properties in its surroundings in image space.
• Slightly shifting each vertex of the mesh along its vertex normal in object space. For this technique to work as expected, the surface must at least form a connected component and each of its shared vertices must have an interpolated normal.
• Render the geometric properties into a texture for calculating uncertainty values in interior regions as well as in the exterior regions (nearby the original 3D geometry). So interior regions can be sketched beyond the boundary of the 3D geometry and exterior regions can penetrate interior regions. Then apply perturbations based on uncertainty values that do have an obvious correspondence to the underlying 3D geometry.
Download Video Example
Reducing the Shower-Door Effect
Render enlarged 3D geometry with its object-space positions as color values into a texture.
To do so, determine the object-space positions for each of the displaced vertices and provide them as texture coordinates to the rasterization process.
Then the rasterizer produces interpolated object-space positions for each fragment. A specialized fragment shader then outputs them as high-precision color values to a high-precision texture. Thus, g(s, t) preserves object-space positions.
Based on g(s, t), determine texture coordinates f (s, t) using p(). In our example, the function p() calculates the perturbation by a user-defined 2 x 2 matrix and by a Perlin noise function encoded into a 3D texture. Then access the 3D texture using g(s, t) as texture coordinates. Multiply the resulting texture value by the 2 x 2 matrix to obtain a degree of uncertainty. The function f (s, t) then applies the degree of uncertainty to perturb (s, t), resulting in (s', t').
Calculating fEdge (s, t) and fShade (s, t) using different matrices results in the sketchy depiction. The accompanying video on the book's CD illustrates that the shower-door effect has been significantly reduced. The overview below illustrates the process flow of sketchy rendering by considering the underlying geometric properties.
Download Video Example
References
Sketchy Rendering
Non-photorealistic rendering
Stylized Depiction in Computer Graphics
Perlin Noise
Shower Door Effect
Sketchy Rendering Example