Context

For an image in-painting task (filling in missing regions in an image, see image below; missing region printed red) I’d like to try a template matching approach.

The idea is to use parts of the image surrounding the missing region and finding similar patches (in the same or different images) to use them and fill-in the missing regions. This boils down to sliding a kernel containing the missing region and some border pixels over the image (or different images) to identify similar regions. Note that the missing regions in the kernel have to be masked and not used for comparing image patches with the kernel.

Actual question

How do I use ImageCorrelate with a kernel that is masked? As far as I am aware there is no build-in way to provide a mask for the kernel. Compare this to openCV’s matchTemplate which accepts an optional mask parameter.

Using the third argument of ImageCorrelate to provide a custom distance function e.g. masked L1 norm works, but is horribly slow. See example below.

Example

As example image/kernel take (found in the documentation of ImageCorrelate):

For a mask take for instance

mask = BoxMatrix[{10, 10}, {50, 30}] // ReplaceAll[{1 -> 0, 0 -> 1}] //  Flatten // N;  HighlightImage[ker, Image@ArrayReshape[mask, {50, 30}]] 

together with a custom distance function

maskedL1Norm = Function[{patch, ker}, Total[Abs[patch - ker]*mask] ] 

Note that ImageCorrelate passes flat lists of pixel values to it’s third argument, not arrays. Hence I flattened mask as well in its definition.

Now compare

ImageCorrelate[img, ker, maskedL1Norm ] //ImageAdjust //AbsoluteTiming 

and for instance

ImageCorrelate[img, ker, EuclideanDistance ] //ImageAdjust //AbsoluteTiming 

On my machine the first takes 0.4s and the latter 0.005s! Is there a way to speed this up? I already tried compiling my distance function

maskedL1NormCompiled =  Compile[{{patch, _Real, 1 }, {ker, _Real, 1}},  Apply[Plus, mask*Abs[(patch - ker)]],  CompilationTarget -> "C",  RuntimeOptions -> "Speed",  CompilationOptions -> {"InlineExternalDefinitions" -> True}] 

and noticed that it is about twice as fast when called on it’s own (for instance maskedL1NormCompiled[N@Range[1500], N@Range[1500]]). For some reason it does not work as a drop-in replacement for the uncompiled version however. Also it is only about twice as fast while the performance penalty for using the third argument of ImageCorrelate is a factor of 80.

ImageCorrelate::bdarg3: Applying the distance function Compiled  Function[{patch,ker},Total[Abs[patch-ker] mask]] did not yield a real number. )