This issue was addressed in this question. The way to do this is to create a minor mode with your bindings. Your minor mode bindings will shadow any major mode's bindings.
I like your example, since just within the last couple of weeks, I rebound M-h,j,k,l to their equivalent vim movements, and made a minor mode to do it (It turned out to be a great idea. Emacs's default bindings really are terrible). Here's a sample of some of my code:
(defvar kirkland-minor-mode-map (make-keymap) "kirkland-minor-mode keymap.")
(define-key kirkland-minor-mode-map (kbd "M-h") 'backward-char)
(define-key kirkland-minor-mode-map (kbd "M-l") 'forward-char)
(define-key kirkland-minor-mode-map (kbd "M-j") 'next-line)
(define-key kirkland-minor-mode-map (kbd "M-k") 'previous-line)
(define-minor-mode kirkland-minor-mode
"A minor mode so that my key settings aren't shadowed by other major/minor modes"
t " kirkland" 'kirkland-minor-mode-map)
One other thing I should mention is that while this will override any major mode bindings, it can still be overridden by other minor modes which are loaded later.