From your code; you are trying to calculate gradient from RGB but there is nothing to indicate how RGB is stored in your image. A complete guess is that in your image you have BGRBGRBGR...etc.
In that case your code is getting the gradient from the green channel, then storing it in the red of the gradient image. You don't show the gradient image being cleared to 0 - if you don't do this then it will probably be full of junk.
My suggestion is to convert to a greyscale image first; then you can use your original code.Or calculate a gradient for each colour channel.