To upload files via Ajax, you need to use a workaround using iframes or Flash.
I recommend you Uploadify, it uses flash to do the uploading, and uses jQuery. I like the fact that it has many settings, and it comes with a nice default style for the queue handling. Just follow the instructions to set it up in your site.
In the PHP script you'll receive the file in the $_FILES array, just like if it were a normal submission via a form, the default name is 'Filedata', so you can access your file via $_FILES['Filedata']. Then just echo the response for the Ajax request (JSON or XML), and execute some JS code for the Uploadify 'onComplete' event.