Context
For an image in-painting task (filling in missing regions in an image, see image below; missing region printed red) I’d like to try a template matching approach.
The idea is to use parts of the image surrounding the missing region and finding similar patches (in the same or different images) to use them and fill-in the missing regions. This boils down to sliding a kernel containing the missing region and some border pixels over the image (or different images) to identify similar regions. Note that the missing regions in the kernel have to be masked and not used for comparing image patches with the kernel.
Actual question
How do I use ImageCorrelate
with a kernel that is masked? As far as I am aware there is no build-in way to provide a mask for the kernel. Compare this to openCV’s matchTemplate which accepts an optional mask parameter.
Using the third argument of ImageCorrelate
to provide a custom distance function e.g. masked L1 norm works, but is horribly slow. See example below.
Example
As example image/kernel take (found in the documentation of ImageCorrelate
):
For a mask take for instance
mask = BoxMatrix[{10, 10}, {50, 30}] // ReplaceAll[{1 -> 0, 0 -> 1}] // Flatten // N; HighlightImage[ker, Image@ArrayReshape[mask, {50, 30}]]
together with a custom distance function
maskedL1Norm = Function[{patch, ker}, Total[Abs[patch - ker]*mask] ]
Note that ImageCorrelate
passes flat lists of pixel values to it’s third argument, not arrays. Hence I flattened mask
as well in its definition.
Now compare
ImageCorrelate[img, ker, maskedL1Norm ] //ImageAdjust //AbsoluteTiming
and for instance
ImageCorrelate[img, ker, EuclideanDistance ] //ImageAdjust //AbsoluteTiming
On my machine the first takes 0.4s and the latter 0.005s! Is there a way to speed this up? I already tried compiling my distance function
maskedL1NormCompiled = Compile[{{patch, _Real, 1 }, {ker, _Real, 1}}, Apply[Plus, mask*Abs[(patch - ker)]], CompilationTarget -> "C", RuntimeOptions -> "Speed", CompilationOptions -> {"InlineExternalDefinitions" -> True}]
and noticed that it is about twice as fast when called on it’s own (for instance maskedL1NormCompiled[N@Range[1500], N@Range[1500]]
). For some reason it does not work as a drop-in replacement for the uncompiled version however. Also it is only about twice as fast while the performance penalty for using the third argument of ImageCorrelate
is a factor of 80.
ImageCorrelate::bdarg3: Applying the distance function Compiled Function[{patch,ker},Total[Abs[patch-ker] mask]] did not yield a real number. )