Portable applications can be run from USB-drives and all and are thus very convenient, but unfortunately they are slow (as a USB drive is usually slower).
How exactly does the installation of a portable app differ from that of a normal app?
I know they do not create registries and all, but then how do they achieve the same thing as other, 'normal' apps?
They don't store anything in the registry or on the hard disk. Application configuration options and other settings are saved on the USB drive, usually (but not always) in either an .INI file or an XML file.
Any application that stores all required information in a self-contained way can be made "portable".
For exmaple, eclipse doesn't require installation and keeps all preferences within the workspace, so it could be considered portable.
An application which any of the following isn't immediately portable:
In order to make these applications portable they can be processed or run within a mini-VM (like ThinApp) so that calls that registry calls and file accesses are modified to refer to locations within the USB.
They don't have any dependencies on system resources (such as the registry) that require a higher level of security to access, nor do they have any dependencies on any libraries not shipped with the application on the thumb drive (unless the dependencies are commonly found in a typical install).
Most simple apps meet these requirements and could hypothetically be run off a thumb drive.