I have a very similar setup to yours (Emacs 22.1, 22.2, 23.1 on various Linux versions with and without X and Windows with and without Cygwin). My setup includes ELPA, auctex, emacsw32, CEDET, JDEE, nxml and various other elisp packages. I do not use whatever comes with the system but keep copies of those packages in subversion.
Majority of setup just works in all environments. Regarding paths, I think that majority of stuff one wants to call, such as aspell, can be called outside Emacs from command line too, so it`s worth putting them in $PATH thus avoiding having to specify full paths in Emacs.
For other things, I do
In .emacs:
; Load system-specific library and setup system-specific things that
; must be setup before main setup
(cond ((eq system-type 'windows-nt) (load-library "ntemacs-cygwin"))
((eq system-type 'gnu/linux) (load-library "linux"))
(t (load-library "default")))
(system-specific-setup)
; Set up things as usually, no more system-type queries.
Where in linux.el:
(defun system-specific-setup()
; Default font
(add-to-list
'default-frame-alist
'(font . "-Misc-Fixed-Medium-R-Normal--14-130-75-75-C-70-ISO8859-1"))
(setq my-frame-width 95)
(setq my-frame-height 56)
; Not much else
)
And in ntemacs-cygwin.el:
(defun system-specific-setup()
;; EmacsW32
(setq emacsw32-root (concat private-elisp-lib "EmacsW32"))
(add-to-load-path emacsw32-root)
;; Work around XSymbol initialization bug
;; ("C:\\ImageMagick\\convert" instead of system $PATH? Seriously?)
(setq x-symbol-image-convert-program "convert")
;; etcetera...
)
Basically it is a matter of setting things up on one system, trying them on another and factoring out whatever needs to be different to the system-specific-setup.
And the Steve Yegge´s article in Noufal´s answer is a very good one.