While writing various image processing programmes with OpenCV, I found the remap function extremely useful.
What the remap function does, is to … remap your image pixels to other areas of the image, according to the chosen mapping functions. In other words, it moves the pixels to different locations in the same image, shuffles, rearranges the pixels. Here is the standard OpenCV tutorial, a very good starting point.
The amount of geometric transformations you can do with the remap function is practically limitless and it all depends to the relocation functions you will be using. The word function here is of particular importance as it allows you to do whatever geometrical transformation that can be described by a mathematical function.
Rotating, mirroring, flipping etc are nothing but the simplest things you can do.
In this tutorial we will show the power of the function with a less common and hopefully more interesting transformation.
We will try to apply a ripple effect on an image, creating a waving flag, similar to what happens to a flag when the wind blows.
The image above is the result of what we will be showing.
We will be writing in C++ ( I tried that in python and some very … interesting observations came out. I will make a new post about that).
Here you can download the complete code (it is not that much, which is normal because this is one of the virtues of simple yet powerful api functions).
Please keep in mind that this code was written and executed on a MacOS. It will work pretty much unchanged on Linux, but beyond that I have no clue. You will be able to get the idea though.
For reference, let me also paste the code here as well:
//
// main.cpp
// RippleEffect
//
// Created by peeknpoke.net on 18/9/16.
// Copyright ... no copyright
#include <iostream>
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
float f = 0.25f*CV_PI;
float p = 3.0f*CV_PI/4.0f;
static void rippleEffect(Mat const & inputImage, Mat & outputImage, Mat const & mapX, Mat const & mapY)
{
remap(inputImage, outputImage, mapX, mapY, CV_INTER_LINEAR, BORDER_REPLICATE);
}
int main(int argc, const char * argv[])
{
// Check usage
if (argc<2)
{
cerr<<"Usage "<<argv[0]<<" <filename>"<<endl;
exit(EXIT_FAILURE);
}
// Read input image
Mat inputImage = imread(argv[1]);
imshow("Input image", inputImage);
waitKey(0);
// Create output image
Mat outputImage;
outputImage.create(inputImage.rows, inputImage.cols, inputImage.type());
// Create remap maps
Mat mapX, mapY;
mapX.create(inputImage.rows, inputImage.cols, CV_32FC1);
mapY.create(inputImage.rows, inputImage.cols, CV_32FC1);
// Constant ripple paramters
float const A = 10.0f; // Ripple magnitude
float const dir = (1.0f/5.0f)*CV_PI/2.0f; // Ripple direction
float const omega = CV_PI*(2.0f/inputImage.rows); // Ripple frequency
unsigned int const rippleFrames = 100; // Ripple frames
while(1)
{
for (unsigned int m=0; m<rippleFrames; m++)
{
float p = (float)m*2.0f*CV_PI/rippleFrames+CV_PI; // Ripple phase
for (int j=0; j<inputImage.rows; j++)
{
for (int i=0; i<inputImage.cols; i++)
{
mapX.at<float>(j,i) = i+A*(i/inputImage.cols)*sin(omega*j+p)*cos(dir);
mapY.at<float>(j,i) = j+A*sin(omega*i+p)*sin(dir);
}
}
rippleEffect(inputImage, outputImage, mapX, mapY);
imshow("Output image", outputImage);
waitKey(3);
}
}
return 0;
}Up to and including line 45, all the code does is to read the input file, create the destination image matrix (outputImage) and create the two maps (mapX, mapY) that will be used during the remapping process. Why two maps? Because we need one for the x axis remapping and another for the y axis.
Basically, each of these maps is a discrete function, in the mathematical sense. Their inputs are x and y coordinates:
x’ = mapX(x, y)
y’ = mapY(x, y)
They do not have to be the same, meaning that the remap can function in an entirely different manner along each axis.
x’ and y’ are the coordinates where the original x and y coordinates have been remapped to. Whatever was at location (x’, y’) in the original image, is now at (x, y).
Quiz
Allow me a small break here for a quiz! I would expect the function to work the other way around, to take what was at (x, y) and put it at (x’, y’). But there is a very good reason why it is working this way, a reason that has nothing to do with image processing, OpenCV or computers and programming in general! I am letting you guess that (you can use the comments section if you care to answer that 🙂 )
Having said that, let’s get back to our program. Lines 46-50 define constants that will be used to form the mapping functions (maps). Here they are again
// Constant ripple paramters
float const A = 10.0f; // Ripple magnitude
float const dir = (1.0f/5.0f)*CV_PI/2.0f; // Ripple direction
float const omega = CV_PI*(2.0f/inputImage.rows); // Ripple frequency
unsigned int const rippleFrames = 100; // Ripple framesIn order to explain what they are doing, I have to explain first what kind of remap I had in mind and that was the ripple effect of wind on a flag, or of a rock falling on water. All these bring sines and cosines to mind, so I thought that perhaps I could get what I wanted by making the two maps to be sine waves.
The function of a sine wave is
f(t) = A*sin(2πft+φ), with respect to time t, or
f(x) = A*sin(2πfx+φ) with respect to a coordinate x.
So, A is the magnitude of the sine wave. A value of 10.0f gives a nice, clear, visible ripple.
f is called the ordinary frequency and the whole 2πf the angular frequency ω (omega)
ω = 2πf
The chosen value for the angular frequency is
ω = 2π/image.rows
The reason behind that is that I wanted the period of the frequency to be proportional to the image dimensions, in order for the effect to look more natural. Since we are talking about space coordinates, the period I am referring to is not time but a length. I chose it to be the smallest dimension, the image rows.
The dir parameter is the direction of the ripple and it is an angle. Do you wish the ripple to be applied in one axis only, either x or y, or both? I chose both and thought of the ripple as a vector in the two dimensional space. In that case the ripple on each axis is the projection of that vector
x axis: ripple * cos(dir)
y axis: ripple * sin(dir)
where ripple is the total calculated ripple magnitude at a certain point. This will become clearer later when we actually apply the calculation.
Finally, the rippleFrames, is just how many frames we wish our animation to have. We could skip that and apply the ripple just once and we would still demonstrate the remap function, but making an animation is much more appealing.
Now, let’s get to the loops. The while loop simply tells our animation to run forever and the loop at line 54 creates our animation frames.
What is interesting here is that at line 56 we introduce a phase to our sine waves. This is because we do not want to apply the same ripple in all frames. Think of it as our ripple traveling on the flag. The main idea is that we want our ripple to complete when the frames complete, so that it restarts seamlessly in the next while iteration. Therefore the phase should start from 0 and end just before 2π, but not at 2π, so that we do not have two consecutive frames with the same phase (a phase of 2π is the same as 0 in sine waves). If you wonder why I have added an extra π there, this is only because I wanted it to start smoother compared to the still image. It is not that important and I did not even spend time to calculate the exact starting phase. I just got that with a bit of trial and error. Do not pay too much attention to that.
Now that we have all in place we must calculate our maps. The effect of the sine wave comes from adding the ripple values to our coordinates, in order to get the x’ and y‘ coordinates.
The cos(dir) and sin(dir) just project the ripple magnitude on the two axes. The rest of the sinusoidals compute the sine waves along each axis. Another customization that I did here was that for the x axis remapping I multiplied the magnitude with i/inputImage.cols. The reason for this is that I just wanted the ripple magnitude to grow as I am moving to the right on the image. This was an arbitrary choice and you can omit it or change it.
Finally, while applying the ripple I chose OpenCV linear interpolation and border replication. You can experiment with that as well, but it is not in the scope of this tutorial.
That was it. Let me apologise for the math in this post and especially if some of them are not too accurate.
Playing with the parameters in this code is a good way to practice with the remap function. Trying to explain what you are getting is a great way towards understanding how remap works, particularly if you plan to make non trivial geometric transformations.
Thanks for following!
