Non-full screen playback of video is possible in iPhone OS 3.2 for iPad.
If you are porting an application that uses the MPMoviePlayerController class of the Media Player framework, you must change your code if you want it to run in iPhone OS 3.2. The old version of this class supports only full-screen playback using a simplified interface. The new version supports both full- and partial-screen playback and offers you more control over various aspects of the playback.
On the full-screen question, see this section of the iPad Programming Guide for further information, and this property of MPMoviePlayerController specifically.
Bear in mind that you can still only play one video at a time, and that this is only currently possible for iPad.