Separating gradients from geometry


Table of Contents

Introduction
Different kinds of mappings
Pulling gradients apart
Advanced gradients
Scattered points
Grids
Meshes by mirroring grids
Diffusion Curves
Further considerations
Reweighting
Spread methods
Conclusion
Bibliography

SVG 1 is maturing, so now focus is shifting towards SVG 2. This next version will bring us a much needed revision of SVG's functionality. One area which seems to have caught everyone's attention is "advanced gradients". Meant to make more flexible shading/coloring possible, so-called advanced gradients are an important tool in vector graphics. Many proposals for different schemes have come forward. Gradient meshes will almost certainly be a part of SVG 2 ([Tav2011-1], [Tav2011-2]), but more exotic schemes like diffusion curves and the <replicate> tag have also been mentioned ([Orzan2008], [Dailey2010], [Grasso2011]). But instead of just "plugging" these new features in like any other gradient, how about rethinking gradients altogether?

So what is a "gradient"? Usage differs a bit between contexts, but in SVG it used for (certain) patterns of (slowly) changing colors. In other contexts the gradient is only the sequence of colors that is mapped to the image, not the resulting pattern. I will adopt this latter terminology as much as possible. This is illustrated in Figure 1, “Gradient definition.”.


So what can we gain from separating the gradient from its geometry? Quite a lot. We will see that this separation will bring us new transforms, as well as new possibilities for gradients. In a sense we can see some of the potential in the feDisplacementMap filter primitive. Although this primitive is not very easy to use, it does show the desire for more general image deformations in SVG.

A simple example of the kind of thing that can be accomplished with the concepts outlined in this paper is composing a linear transformation with a "radial" transformation to get either an angular/conical gradient, a radial gradient or a spiral gradient. With the help of some of the more advanced "gradients" we would be able to create very cool deformations based on meshes or diffusion curves. And all this just by stepping back a bit and recognizing that we can separate gradients from their geometry.

This discussion would not be complete without taking a closer look at the different kinds of mappings we have. Broadly speaking there are two categories: "source to target" and "target to source" (see Figure 2, “Two types of maps.”). Traditionally, transforms are more of the source to target type, you have an object in a local coordinate space and you specify a transformation matrix that maps positions in the current coordinate system to a new coordinate system. The transforms involved in specifying how gradients are mapped to an object are also of this type (or can at least be easily thought of in the same way). Diffusion curves, on the other hand, behave more like target to source maps, they map each position in the image to a color.


Having a target to source map can seem to have some advantages, as we then have mapping information at the resolution of the target and can choose any source position we want for each of the target positions, what more could we want? Mathematically this is true, but practically it is often much more difficult to think about transforms in this way. For example, viewing a radial gradient as a source to target map is easy, almost trivial in fact, just envisage a number of concentric rings and you are done. A little high school math tells us that (u,v) maps onto (u cos(v),u sin(v)), see Figure 3, “A radial gradient.” (note that in practice u and v need to be properly scaled). Viewing a radial gradient as a target to source map, is, not so easy. Something similar holds for many of the discussed mappings.


It should also be noted that in real life either option is just fine. After all, we cannot bend an object and have the same end appear in two places at once. The main application where we can (seem) to see the same image twice (although it will often be a slightly different image in reality, as photons generally do not split), is optics. In such cases it might be enough to simply use different "lenses" for the same object (think <use>/clones).

The main problem with source to target maps is what to do with "overlaps". In many cases these can be solved by specifying an order on the source pixels, but obviously more schemes are possible.

Now we come to the "separating" part of this story, we are going to pull our beloved gradients apart by their limbs. Their limbs being the colour space used and the image (or at least the fill of the object referencing the gradient). First we define a (one dimensional) gradient with no regard for how it is going to be used in the image, a target to source map:

<gradient>
  <stop offset="0" stop-color="#ffcc00" />
  <stop offset="0.5" stop-color="#00aa00" />
  <stop offset="1" stop-color="#ff00ff" />
</gradient>

This is illustrated in Figure 4, “Just a gradient.”.


Just like a traditional SVG gradient the above does not cause anything to be rendered yet. For that we need to use the gradient as a fill, and in doing so, define its geometry. Traditionally this is done (for a linear gradient) through the x1, y1, x2 and y2 attributes, in combination with the gradientTransform attribute (we ignore gradientUnits, as this can be taken to work mostly the same as before). Here we conceptually place the gradient along the x-axis and extend it vertically. Then we use the transform attribute (source to target map) on the gradient element to align the gradient with the shape in question:

<gradient id="gradient" transform="matrix(1,1,-1,1,0,0)">
  <stop offset="0" stop-color="#ffcc00" />
  <stop offset="0.5" stop-color="#00aa00" />
  <stop offset="1" stop-color="#ff00ff" />
</gradient>
<path d="M 0,0 A 0.707107,0.5 45 0 0 1,1 A 0.707107,0.5 45 0 0 0,0" fill="url(#gradient)"/>

This is illustrated in Figure 5, “A linear gradient applied to an object.”.


Alternatively, and perhaps even more usefully, the transform could be moved to the object referencing the gradient. One advantage would be that it would become much easier to reuse a gradient in different objects. Currently this requires creating a new gradient for each object, usually all referencing the same "base" gradient. Moving the transform to the referencing object will require new attributes though (or an expanded syntax for the gradient). It could look roughly like this (ignoring gradientUnits):

<gradient id="gradient">
  <stop offset="0" stop-color="#ffcc00" />
  <stop offset="0.5" stop-color="#00aa00" />
  <stop offset="1" stop-color="#ff00ff" />
</gradient>
<path d="M 0,0 A 0.707107,0.5 45 0 0 1,1 A 0.707107,0.5 45 0 0 0,0" fill="url(#gradient)" fillTransform="matrix(1,1,-1,1,0,0)"/>

For advanced gradients this makes less sense, as those are usually much more tailored to a specific object. Also, it should probably remain possible to specify a transform on the gradient itself. So in the remainder of this paper I will mostly stick to the more familiar convention of specifying the transform on the gradient.

As it is a bit inconvenient to have to specify a matrix like I did above, I propose introducing another transform: linear. Also, the above just covers linear gradients, and we would obviously also like to be able to use radial gradients. This is where separating gradients from their geometry starts to really pay off. We now introduce two new transforms (note that the second is non-linear):

  • linear(<x2>,<y2>[,<x1>,<y1>[,<x3>,<y3>]]), which is the linear transformation that transforms (0,0) into (x1,y1), (1,0) into (x2,y2) and (0,1) into (x3,y3). One way to find this transformation is to use barycentric interpolation of the points [(x0,y0),(x1,y1),(x2,y2)] on the triangle [(0,0),(1,0),(0,1)] and extend it to the entire plane. In terms of transformation matrices it is given explicitly by [x2-x1 y2-y1 x3-x1 y3-y1 x1 y1]. If x1 and y1 are not given they are assumed to be zero. If x3 and y3 are not given they are assumed to correspond to the point (-y2,y1).

  • polar(<cx>,<cy>,<r>,<p>,<fx>,<fy>]), which essentially causes the current coordinate system to be treated as a polar coordinate system. It maps a point (x,y) to (fx,fy) + x (cx-fx,cy-fy) + r x (cos(2 pi y/p),sin(2 pi y/p)). Note that there are different ways to deal with positions for which x<0 and y is outside the range [0,p), the easiest being to just ignore all such positions.

It should be clear that these transforms in combination with the gradient element used above will allow us to create the traditional linear and radial gradients. However, they allow much more. For one thing, it would be possible to not just use them on gradients, but on any object that can be transformed. This would allow easy polar plots for example, while the linear transformation given above would likely be easier to use in many circumstances than specifying a full matrix transformation. Furthermore, when combined they can be used to create spiral or conical gradients, as illustrated in Figure 6, “"Polar" gradients.”.


The first thing to note about most of the "advanced" gradients is that they are inherently two dimensional. Traditional gradients are of course mapped to a two dimensional object, but their specification is one dimensional. In one dimension it is not too difficult to think of a decent specification, just use a piece-wise linear function. If you want to get fancy you can even specify some more interesting transition functions with relative ease. However, when going to two dimensions things are not so clear cut, and a number of options present themselves. In the next few subsections I will present a few of these options.

A central theme in the following sections is the realization that gradient definitions and transforms have much in common, especially if you consider two dimensional gradients. After all, both map from a gradient to something else (either a color space or an image). So it makes sense to have a similar syntax for both. In addition, unifying the syntax for gradients and transforms makes it very easy to combine them. For example, if we allowed gradients to be defined using a syntax similar to transforms we could do something like this:

<path d="M 1,1 L -1,2 L 3,4" fill="linear(#f00,#0f0,#00f)" fillTransform="linear(1,1, -1,2, 3,4)"/>

Figure 7, “Trilinear interpolation.” shows what this would look like. Note that defining both the gradient and the transform using a map from the gradient makes it trivial to match up the colors in the fill with the positions in the fillTransform. In general one does have to be careful though, there are subtle differences between colors and positions that can complicate things. For example, colors are usually constrained to assume values in a limited range and taken to lie in a three (or more) dimensional space.


This directly generalizes the traditional gradient stop mechanism. To define a two dimensional gradient the user would simply specify a number of stops and their colors, and the renderer would fill in the rest of the space. This could be implemented using something like Sibson's/natural neighbour interpolation, although a detailed study of the different possibilities and their (dis)advantages should be undertaken before making any definitive choice of course. This could look like this:

<gradient id="gradient" interpolation="linear">
  <stop offset="0.5 0" stop-color="#ffcc00" />
  <stop offset="0 1" stop-color="#00aa00" />
  <stop offset="1 0.5" stop-color="#ff00ff" />
  <stop offset="0.5 0.5" stop-color="#0000aa" />
</gradient>
<path d="M 0,0 A 0.707107,0.5 45 0 0 1,1 A 0.707107,0.5 45 0 0 0,0" fill="url(#gradient)"/>

See Figure 8, “Gradient using interpolation of scattered points.” for a simple example of the use of this gradient. Note that I have explicitly ignored what happens outside the convex hull of the given points, I will come back to this later. Also, I have omitted the color space (and the corresponding links) for clarity of presentation, as what was a line in a 3D color space is now a surface.


As announced we will be trying to unify gradient and transform syntax as much as possible, so we would like to introduce syntax for specifying transforms based on scattered points. However, it should be noted that the feasibility of this would depend on the kind of interpolations supported. Also, does the above really need an additional transform? Perhaps not, but we might want to use it to allow for very, very flexible deformations. It would look something like this to get a uniform scaling by a factor of two (obviously done just for demonstration purposes):

<transform id="tr" interpolation="linear">
  <stop offset="0.5 0" stop-position="1 0" />
  <stop offset="0 1" stop-position="0 2" />
  <stop offset="1 0.5" stop-position="2 1" />
  <stop offset="0.5 0.5" stop-position="1 1" />
</transform>
<path d="..." transform="url(#tr)" />

On the surface perhaps not as flexible as the above, but grid-based interpolation can definitely be useful, especially in combination with further processing. Also, we need not limit ourselves to the rectangular grid (although I will), and it is relatively straightforward to use higher order interpolation. A simple syntax, similar to HTML tables, could look like this:

<gridGradient>
  <gridRow>
    <stop id="a" stop-color="#ffcc00" />
    <stop id="b" stop-color="#00aa00" />
  </gridRow>
  <gridRow>
    <stop id="d" stop-color="#ff00ff" />
    <stop id="c" stop-color="#0000aa" />
  </gridRow>
</gridGradient>

Figure 9, “Grid gradient.” shows what this might look like in practice. Note that the above enables an efficient implementation, but would otherwise be equivalent to:

<gradient>
  <stop offset="0 0" stop-color="#ffcc00" />
  <stop offset="1 0" stop-color="#00aa00" />
  <stop offset="0 1" stop-color="#ff00ff" />
  <stop offset="1 1" stop-color="#0000aa" />
</gradient>

In contrast, the proposed syntax for mesh gradients ([Tav2011-1], [Tav2011-2]) considers each cell (patch) to be the basic unit. This is more natural if you also wish to specify information on the edges of the grid. This looks roughly like this:

<gridGradient>
  <gridRow>
    <gridCell>
      <stop id="a" stop-color="#ffcc00" ... />
      <stop id="b" stop-color="#00aa00" ... />
      <stop id="c" stop-color="#ff00ff" ... />
      <stop id="d" stop-color="#0000aa" ... />
    </gridCell>
    <gridCell>
      <stop ... />
      <stop id="e" stop-color="..." ... />
      <stop id="f" stop-color="..." ... />
    </gridCell>
    ...
  </gridRow>
  <gridRow>
    <gridCell>
      <stop ... />
      <stop id="g" stop-color="..." ... />
      <stop id="h" stop-color="..." ... />
    </gridCell>
    <gridCell>
      <stop ... />
      <stop id="i" stop-color="..." ... />
    </gridCell>
    ...
  </gridRow>
</gridGradient>

Each grid cell only specifies stops (and stop-colors) where necessary. Of course this could also be done by specifying edges explicitly, like this:

<gridGradient>
  <gridRow>
    <gridCell>
      <stop id="a" stop-color="#ffcc00" />
      <edge ... />
      <stop id="b" stop-color="#00aa00" />
      <edge ... />
      <stop id="c" stop-color="#ff00ff" />
      <edge ... />
      <stop id="d" stop-color="#0000aa" />
      <edge ... /> <!-- Connects d and a -->
    </gridCell>
    <gridCell>
      <edge ... /> <!-- Connects b and e -->
      <stop id="e" stop-color="..." />
      <edge ... />
      <stop id="f" stop-color="..." />
      <edge ... /> <!-- Connects f and c -->
    </gridCell>
    ...
  </gridRow>
  <gridRow>
    <gridCell>
      <edge ... /> <!-- Connects d and g -->
      <stop id="g" stop-color="..." />
      <edge ... />
      <stop id="h" stop-color="..." />
      <edge ... /> <!-- Connects h and c -->
    </gridCell>
    <gridCell>
      <edge ... /> <!-- Connects f and k -->
      <stop id="i" stop-color="..." />
      <edge ... /> <!-- Connects i and g -->
    </gridCell>
    ...
  </gridRow>
</gridGradient>

See Figure 9, “Grid gradient.” for what the above grids look like.

For specifying a grid the first syntax presented here is obviously the more compact one, but a cell-centered syntax also has its merits. One could imagine setting properties of the interpolation on edges for example. Also, a cell-centered syntax might allow specifying stops with (non-integer) offsets, where the offset is interpreted as being along the boundary of a cell (with the four corners corresponding to offsets 0, 1, 2 and 3). Such a scheme could be given meaning in any number of ways (which I will not discuss here) and allow for a relatively easy and efficient generalization of grid gradients.

Finally, one might wonder whether it is actually necessary to have a special syntax for grid gradients. In principle it should not be too hard to detect that grid offsets lie on integer coordinates and make use of this. This could even be made more efficient by allowing stop offsets to be specified relative to the previous stop offset. Then it would be possible to simply define a number of interpolation types that only support integer offsets. The main issue would be figuring out how to deal with things like holes in the data and non-integer offsets, should those cause errors, or can we define reasonable behaviour without making things overly complex?

Traditionally a number of different mesh based gradients are available to the artist. And just like with other gradient types they tend to combine the gradient with its "geometry". However, internally they can all be seen as consisting of a gradient and a transform, in fact, quadrilateral meshes are even defined that way (in the PDF reference for example). This lets us pull apart these gradients as well, leading to very interesting possibilities.

First consider grid-like meshes. These meshes have a topology that is the same as that of a (regular) grid, so we can just use any method for generating a grid-based gradient to get the "gradient part". The transform part mirrors this, except that the map is defined the other way around. For this we introduce the following syntax (which is meant to mirror whatever grid gradient syntax we have):

<transform id="tr">
  <stop offset="0 0" stop-position="0 0" />
  <stop offset="1 0" stop-position="1 -0.2" />
  <stop offset="0 1" stop-position="0.6 0.9" />
  <stop offset="1 1" stop-position="1.6 0.7" />
</transform>
<gradient transform="url(#tr)">
  <stop offset="0 0" stop-color="#ffcc00" />
  <stop offset="1 0" stop-color="#00aa00" />
  <stop offset="0 1" stop-color="#ff00ff" />
  <stop offset="1 1" stop-color="#0000aa" />
</gradient>

See Figure 10, “A mesh gradient.” for how these two kinds of maps work together (you can also compare to Figure 7, “Trilinear interpolation.”).


Here I just deal with piece-wise (bi)linear transforms and gradients, for Coons patch meshes, see [Tav2011-1] and [Tav2011-2]. Further generalizations could build upon the idea of specifying path data along edges and/or introduce "control stops" of some kind. Such generalizations should be considered to apply to both gradients and transforms whenever possible.

Defining transforms independently of gradients and being able to apply them not just to gradients, but also to ordinary objects gives us unprecedented power. Also, it allows us to mix and match interpolation methods. For example, a Coons patch mesh gradient with Sibson's interpolation for the gradient would be smoother than the same mesh with the traditional bilinear interpolation of colors. However, for efficiency and ease of use we could define a combined gradientTransform element that allows both color and position properties to be set on stops:

<gradientTransform>
  <stop offset="0 0" stop-color="#ffcc00" stop-position="0 0" />
  <stop offset="1 0" stop-color="#00aa00" stop-position="1 -0.2" />
  <stop offset="0 1" stop-color="#ff00ff" stop-position="0.6 0.9" />
  <stop offset="1 1" stop-color="#0000aa" stop-position="1.6 0.7" />
</gradientTransform>

Note that when a transform maps two different source pixels to the same destination pixel we need to specify what to do. One option (the one adopted in the PDF reference) is that we define an ordering on the source pixels and say that the "higher" pixel wins.

For free-form meshes (that consist of triangles and/or quadrilaterals in some arbitrary arrangement) and/or grid gradients with discontinuities at vertices we need something more than can be offered by the above. One option is to allow the specification of multiple colors/positions per stop (where "missing" values would be filled in by, for example, repeating the last value, but this would require further research into use cases). For example (assuming bilinear interpolation on a grid, replication and order of values as in Figure 11, “A freeform mesh gradient.”):

<gradientTransform>
  <stop offset="0 0" stop-color="#ffcc00" stop-position="0,0" />
  <stop offset="1 0" stop-color="#00aa00 #ff0000" stop-position="1,-0.2 0.5,-0.5" />
  <stop offset="2 0" stop-color="#ffaa00" stop-position="1.5,-0.5" />
  <stop offset="0 1" stop-color="#ff00ff" stop-position="0.6,0.9" />
  <stop offset="1 1" stop-color="#0000aa #009bff" stop-position="1.6,0.7 0.5,0.5" />
  <stop offset="2 1" stop-color="#7e7e7e" stop-position="1.5,0.5" />
</gradientTransform>

Alternatively, we could borrow the current convention of defining the same stop multiple times to specify a discontinuity. Then the above would become:

<gradientTransform>
  <stop offset="0 0" stop-color="#ffcc00" stop-position="0,0" />
  <stop offset="1 0" stop-color="#00aa00" stop-position="1,-0.2" />
  <stop offset="1 0" stop-color="#ff0000" stop-position="0.5,-0.5" />
  <stop offset="2 0" stop-color="#ffaa00" stop-position="1.5,-0.5" />
  <stop offset="0 1" stop-color="#ff00ff" stop-position="0.6,0.9" />
  <stop offset="1 1" stop-color="#0000aa" stop-position="1.6,0.7" />
  <stop offset="1 1" stop-color="#009bff" stop-position="0.5,0.5" />
  <stop offset="2 1" stop-color="#7e7e7e" stop-position="1.5,0.5" />
</gradientTransform>

Diffusion curves are a relatively recent addition to the family of gradients (see [Orzan2008] for the original paper, see this bibliography for more). In essence they take curves instead of points as stops. Along the curves colors can be controlled using (again) gradients, which can be different on either side of a curve to allow introducing discontinuities in the image. For an example, see Figure 12, “Diffusion curves.”.


Due to their definition diffusion curves are most suited as defining a "gradient", they map every point in the plane to some color. In principle they could also be used as a transform, mapping every point in the (gradient) plane to some position in the user space of the referencing object, yielding a very flexible deformation method. When used as a gradient, like in the example above, the syntax could look like this:

<gradient id="left">
  <stop offset="0" stop-color="rgb(57,225,137)" />
  <stop offset="1.1" stop-color="rgb(255,255,255)" />
  <stop offset="3.0" stop-color="rgb(170,255,0)" />
</gradient>
<gradient id="right">
  <stop offset="0" stop-color="rgb(85,13,255)" />
  <stop offset="3.0" stop-color="rgb(85,170,255)" />
</gradient>
<gradient interpolation="diffusion">
  <stop offset-path="M 45,110 C 20,50 55,20 90,30 C 125,40 120,110 160,120 C 200,130 220,70 180,20" stop-color="url(#left) url(#right)" />
</gradient>

Here I have opted for allowing two stop "colors", as specifying the same stop again seems a bit inefficient here. Alternatively it could be made easier to specify that we want the same stop again. Also, it might be good to make it possible to specify the gradients along the curves as children of the associated stop, both for efficiency and ease of use.

In the above I have explicitly ignored the spreadMethod attribute. For one dimensional gradients it can remain pretty much as is, except that it might be good to just repeat the range of offsets specified, instead of the fixed range [0,1]. For two dimensional gradients we need to be a bit more creative.

Some "interpolation" methods have the potential to work over the entire plane (diffusion curves for example), so in principle these would not require any "spread method". Other methods are only valid within the convex hull of the stops. In such cases spreadMethod=pad can be interpreted as giving points outside the convex hull the color of the closest point on the convex hull.

The reflect and repeat spread methods are of a different nature. Instead of "clamping" values to the given range they essentially tile the range. Specifically, the reflect and repeat methods correspond to one dimensional symmetry groups, so it might make sense to use two dimensional symmetry groups for two dimensional gradients (the famous "wallpaper" groups), like in Figure 13, “Two dimensional repeat.”. In addition to the traditional repeat and reflect you would not have a whole host of options corresponding to the different ways in which a two dimensional pattern can be tiled (17 in total).


Note that rather than tiling some preinterpolated area it would probably be best to "tile" the stops before interpolation. Otherwise we would need to define the boundaries of the "tile". It would still be necessary to specify some parameters though. For example, the group pmm depicted in the example above would require a width and height of the tile.

This paper is not about advocating yet another "advanced" gradient. Rather, it is about making a clear separation between the gradient itself and the transform that maps it to the image, explicitly exposing this, and using it to fit "advanced" gradients into one scheme together with traditional gradients. This separation of concerns comes with the benefit of getting to mix and match transforms and gradients in ways that are usually impossible. In addition, it leads us to see new and interesting ways to define both gradients and transforms.

The ideas discussed in section 3 (defining an "abstract" gradient element in combination with the linear and polar transforms) would make an excellent and safe addition to SVG. With some small changes to the current syntax we would have: a convenient way to define general linear transforms, spiral and angular/conical gradients; and polar transforms on objects. Implementation of these features should not be especially difficult, at least not much more difficult than support for linear and radial gradients, so it would mainly be a matter of deciding on some of the details that would have to be settled.

In the section on advanced gradients I have shown how such gradients can be seen to fit in the same scheme as normal gradients. Just like traditional gradients they can be defined in terms of gradients and transforms, although for example diffusion curves are traditionally only defined as a gradient, the corresponding transform can be seen as a bonus of exploiting the similarity in definition of gradients and transform. By recognizing how these different types of gradients naturally fit within one framework we see how adding them to SVG can be done in an elegant, controlled and very powerful manner.