You will need to implement a new massAction in the admin product controller. A good way to start is to have a look at the product controller and see how the other massActions are implemented.
Let's have a look... First you will need to add a declaration of a massAction to a grid. This can be done in app/code/core/Mage/Adminhtml/Block/Catalog/Product/Grid.php. You need to add the following in the _prepareMassaction method:
$this->getMassactionBlock()->addItem('export', array(
'label' => Mage::helper('catalog')->__('Export to CSV'),
'url' => $this->getUrl('*/*/massExport', array('_current'=>true)),
));
Now you will have to implement this new action in the product controller (app/code/core/Mage/Adminhtml/controllers/Catalog/ProductController.php):
public function massExportAction()
{
$productIds = $this->getRequest()->getParam('product');
if (!is_array($productIds)) {
$this->_getSession()->addError($this->__('Please select product(s).'));
$this->_redirect('*/*/index');
}
else {
//write headers to the csv file
$content = "id,name,url,sku\n";
try {
foreach ($productIds as $productId) {
$product = Mage::getSingleton('catalog/product')->load($productId);
$content .= "\"{$product->getId()}\",\"{$product->getName()}\",\"{$product->getProductUrl()}\",\"{$product->getSku()}\"\n";
}
} catch (Exception $e) {
$this->_getSession()->addError($e->getMessage());
$this->_redirect('*/*/index');
}
$this->_prepareDownloadResponse('export.csv', $content, 'text/csv');
}
}
The code is mostly copied from the massDeleteAction, but instead of deleting products you should add to the $content variable. After you are done creating the content of your csv export (you will probably need to add other fields to it) you need to call the _prepareDownloadResponse method of the controller class. That's it, you have your custom export in place!
On a final note, once you are happy with the changes remember to move them to your local code pool so that your magento installation remains update-proof :)