If you don't want to use a package manager, then don't use a package manager.
apt is a great tool and is there to help, but you aren't required to use it.
On Ubuntu, you might want to just use dpkg instead of apt and so that none
of the dependencies will be automatically updated and avoid upgrading the
libraries that are causing problems. Or, just install everything directly
from source. If you go that route, install to /usr/local or some
non-standard location. (ie, do NOT configure --prefix=/usr &&
make && make install. Use the default /usr/local or $HOME for prefix.)
Note that doing this is a heinous kludge, and you have bigger problems. If you
are relying on libraries that are unstable, then you probably ought to consider
removing your dependency on those libraries. Also, mixing usage of dpkg with
apt will cause maintenance headaches. If you do that, only do it on your
developmental boxes (eg, not on production servers.) Your primary concern
should be to get your package working correctly with the package management
system, and one part of this may involve fixing the packaging of all the
libraries you depend on.