I would remove any previous installation and start again by following this page. First of all open a shell and launch:
identify -version
which will give you the IM version installed on your system.
Depending on how You installed IM, find the way to remove It completely from the system. For instance if you used apt-get, try:
sudo apt-get remove ImageMagick
If you installed IM from sources, go to where you have them stored (I mean the sources path/folder) and type:
make uninstall
You can then reinstall ImageMagick, compiling it from the sources:
cd
wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick.tar.gz
tar zxf ImageMagick.tar.gz
cd ImageMagick-*/
./configure --prefix=$HOME --without-perl
make
make install
Then you have to add $HOME/bin to the beginning of your $PATH
cd
echo "export PATH=$HOME/bin:\$PATH" >> .bash_profile
source .bash_profile
Now it's time to gem install RMagick:
export LD_LIBRARY_PATH=$HOME/lib
gem install rmagick