Smart Fill

Table of Contents

Use Smart Fill to remove large objects from images with complex background details. Smart Fill is unique; rather than directly cloning parts of an image, the filter intelligently samples image chunks around the area to be removed, automatically detects details such as color and lighting, then recombines the sampled image details to create a contiguous background texture where the original object once appeared.


This chapter contains the following sections:

Using Smart Fill

1. Using a selection tool, preferably the lasso, make a selection around the defect or object that you want to remove.

2. Run the Smart Fill filter.

3. Click the Arrow button to view and adjust the Texture Source Box control. Most of the time, placing a Texture Source Box directly around the area of the selection will do the trick. Sometimes, however, you may wish to sample a texture from another location in the filter preview. In this case, click and drag the spot in the center of the texture source box to the desired location in your image. For more information on using Texture Source Boxes, see the “Texture Source Boxes” section in this chapter.

4. Adjust the Contract or Expand Selection slider to make sure you have completely selected the the object you wish to remove.

5. Adjust the Texture Feature Size slider to sample appropriately sized image details. See “Texture Feature Size” on page 30 for more information.

6. Adjust Texture Regularity to taste. See “Texture Regularity” on page 30 for more information.

7. Click Regenerate Texture a few times to randomize the resulting fill pattern. In many cases, you can simply run the Smart Fill filter, click “Regenerate Texture” a few times, and get great results.

Slider Descriptions and In-Preview Controls

Contract or Expand Selection (pixels)

Use this slider to contract or expand your original selection from within the Image Doctor interface.
Range: -20 – (20)

Why? If the initial preview reveals remnants of your original selection, you can expand your selection to knock out the leftovers. If you find that you are removing some nearby desirable image detail, you can contract your selection a couple of pixels to protect that area. It’s best to make a good, close selection before you run the filter, however.

Texture Feature Size

Use this slider to specify the texture sample size that Smart Fill uses for texture replacement. If, for example, the blades of grass that make up the sampled texture are roughly 15 pixels tall, pump up the feature size to about 15 pixels.
Range: 3 – 100

Note: The Feature Size slider works in conjunction with the hash marks in the Texture Source Box in-preview control. When you increment the feature size slider, the hash marks spread apart to reflect the increasing size of texture chunks being sampled.

Texture Regularity

These radio buttons control how closely Smart Fill matches the image details generated in your selection. For more predictable textures, such as bricks, use higher values. For irregular patterns, such as grass, sometimes lower values will do just fine. Using lower Texture Regularity values helps the filter render faster and randomizes the appearance of image details generated in your selection.
Options: Low, Medium, High

Stitch into Background

Selecting this box forces Smart Fill to attempt a seamless placement of fill textures in your selected area.

Note: Checking this box disables the “Feather Radius” slider.

Feather Radius

Feather Radius creates an opacity drop off around the edge of the affected area of your image—helping blend the filtered area with the surrounding unaffected pixels. To use stitching, clear the “Stitch into Background” check box. See “When to Use Stitching or Feathering” for detailed information on the subject.
Range: 0 - 4 pixels.

Feathering can cause an unnatural loss of detail at the edge of your selection if used in excess.

Regenerate Texture

Use this button to randomly rearrange the fill details generated by Smart Fill. In many cases, you can simply run the Smart Fill filter, click “Regenerate Texture” a few times, and get great results.

When to Use Stitching or Feathering

This section provides detailed information on how to properly use the “Stitch into Background” and “Feather Radius” controls.

Stitching and Feathering: How they Differ

“Stitch into Background” analyzes the appearance of image details that surround your selection and attempts to match the edges of the composite fill image to those details. Stitching almost always provides the best Smart Fill results.

“Feather Radius” specifies an opacity drop-off (1 to 4 pixels) around the outside of your selected fill area and the surrounding image detail. Feathering disables stitching, however, usually with results inferior to those produced by stitching.

Always Try Stitch into Background First

Stitch into Background is the best option for seamlessly removing objects from a background. If you are not getting the desired result, try the following steps, before using feathering, to improve the quality of your fill results.

  • Expand your texture source box.
  • Reduce the “Feature Size” slider value.
  • Use a precise selection tool, such as the lasso.
  • Use multiple selections when the object you wish to remove traverses multiple different backgrounds.
  • Rearrange the fill details in your selection by clicking “Rearrange Texture”.


When to use Feather Radius

When a selection falls on a sharply contrasting edge, such as a horizon behind a desert landscape, stitching can introduce a rough edge. Feathering can help soften these edges.

Stitching writes outside of your selection. Sometimes, as a result, Smart Fill overwrites parts of an image you wish to preserve. If you want to preserve the areas outside of your selection, use feathering.

In-Preview Controls

Texture Source Boxes

Texture Source Boxes specify the areas from which Smart Fill samples the small image details used to remove an object.

Texture Source Box

Hash Marks

The hash marks on your texture source control box expand and contract as you adjust the Texture Feature Size slider.

Arrow Button

Click the Arrow button to control the placement, size, and inclination of Texture Source Boxes in Smart Fill. Click and drag any corner of a Texture Source Box to resize it. Click and drag the spot in the center of the box to move it. Click and drag the triangle above the box to rotate the box.

Arrow Button

Note: Rotating the Texture Source Box rotates the angle at which fill details will appear in your image. Rotation is rarely needed, but can be useful in cases where you are sampling image details far from the object that you are removing. In far off locations, the perspective of sampled image details may differ.

Using Multiple Texture Source Boxes

If you are trying to fill an area with image details that exist in various parts of an image, use more than one Texture Source Box. To initialize a new Texture Source Box, click any part of your preview image using the Arrow tool.

Deleting Texture Source Boxes

To delete a Texture Source Box, click and drag it out of the preview area.

Tutorials

Simple Object Removal

Use this tutorial to learn how to remove an object from an uninterrupted, homogenous background texture.

We begin with a bottle lying in a background texture of grass.

1. Using virtually any selection tool, preferably the lasso, make a rough selection around the object that you wish to remove.


2. Run Smart Fill.


3. Click on the in-preview Arrow control to adjust your Texture Source Box.

Note: Clicking and dragging any corner resizes the frame. Clicking and dragging the dot in the center of the frame moves the frame around the interface. The bottle will not be sampled. Smart Fill ignores any selected pixels inside of the Texture Source Box.

4. Center the frame around the bottle and resize to sample a generous amount of the surrounding background texture.

5. Adjust the Texture Feature Size slider to match the physical characteristics of the background texture. In this example, blades of grass are about 50 pixels long.

Note: The hash marks on your texture source control box expand and contract as you change Texture Feature Size.

6. Try the Texture Regularity radio buttons to and see which one works best.

Note: For less complex background textures, such as concrete, sand, or asphalt, a low Texture Regularity would work just fine.

7. Select the “Stitch Into Background” check box to seamlessly place the sampled grass swatches into the selected area. If the stitching effect leaves a rough edge, click the “Regenerate Texture” button a few times. If you still can’t get the desired effect, clear “Stitch into Background” and try using feathering to soften the effect edge.

Note: “Stitch into Background” is the best way to generate a seamless fill texture in your selection. For the most “invisible” result, stitching is nearly always preferable to feathering.

8. Preview your results. If they are close but not quite perfect, click the “Randomize Texture” button a few times until you get the best result.

9. When you are satisfied with the preview, click OK.

The final result

Using Multiple Selections to Remove Objects from Complex Backgrounds

Sometimes an object or defect you wish to remove will overlie several different background textures such as in the fire hydrant example below.

The fire hydrant stands in front of two distinct background textures: bushes and mulch.

To get the best results when removing the fire hydrant, use two selections, with the selection boundary between the two located roughly between the bushes and mulch.

1. Select the upper part of the fire hydrant using the lasso tool.

2. Run Smart Fill.

3. Click on the in-preview Arrow control to adjust your Texture Source Box.

4. Make sure the frame is positioned to sample a generous amount of the bush texture.

5. Adjust the Texture Feature Size slider to match the physical characteristics of the background texture.

Note: The hash marks on your texture source control box expand and contract as you change Texture Feature Size.

6. Try the Texture Regularity radio buttons to and see which one works best.

Note: For less complex background textures, such as concrete, sand, or asphalt, a low Texture Regularity would work just fine.

7. Select the “Stitch Into Background” check box to seamlessly place the sampled bush swatches into the selected area. If the stitching effect leaves a rough edge, click the “Regenerate Texture” button a few times. If you still can’t get the desired effect, clear “Stitch into Background” and try using feathering to soften the effect edge.

Note: “Stitch into Background” is the best way to generate a seamless fill texture in your selection. For the most “invisible” result, stitching is almost always preferable to feathering.

8. Preview your results. If they are close but not quite perfect, click the “Randomize Texture” button a few times until you get the best result.

9. Click OK.

10. Select the lower part of the fire hydrant using the lasso tool.

11. Run Smart Fill.


12. Repeat Steps 3-9.

Here’s your result!

Tips and Tricks

The following tips will help you get the best results when using Smart Fill.

Use Multiple Selections to Remove Objects from Complex Backgrounds

When you wish to remove an object that traverses several different background textures, you should you use multiple selections and applications of Smart Fill to get the best effect. The fire hydrant example at the beginning of this chapter required two separate selections and applications of Smart Fill to get the best result.

Using Smart Fill on Long or Narrow Selections

With long or narrow objects (like a car or phone booth), dividing your work into small selections gives you more control and better results when using Smart Fill—especially if the object you are removing lies on a background of multiple textures—grass, asphalt, and dirt, for example.

When applying Smart Fill to long objects, it’s best to start by selecting the left most part of the object and then work toward the right. With tall objects, start at the top left and work your way down. Because of the way Smart Fill analyzes a selection, this method will result in the best fill quality.

Making Good Selections

Great image corrections start with good selections. Try to get a selection that includes only the defect you wish to remove. Specifically, we recommend that you do not use a geometric selection tool, such as Photoshop's Marquee tool to make your selections. Instead, Photoshop users should use the Lasso tool.