You could use a third-party library such as Aspose.NET for document conversion, but I'm afraid - if high-fidelity rendering is critical - there is no way around using the original application.
Microsoft Office provides a converter API which allows conversions without Office being installed. However, not only might you be facing license issues (IANAL), this API only supports conversions of text-processing formats that don't require rendering the document (e.g. RTF -> DOC, DOC -> DOCX), so it is not really an option for you.
Update: Probably the best option would be to have a look at the SharePoint 2010 conversion engine, which is exactly made for automated (server-side) document conversions. It's quite heavy though (both hardware and pricing) so maybe it is overkill for your use-case.