Simply name your template category-X.php, where X is the specific category name or id you want a custom template for, and place it in a /single folder in your main theme directory. Now any time a single post is called and it matches an existing template in that folder, it will use that to display the post instead of the regular single.php. If no match is found then single.php is used.
in functions.php:
define(SINGLE_PATH, TEMPLATEPATH . '/single');
add_filter('single_template', 'force_cat2single_template');
function force_cat2single_template($single) {
global $wp_query, $post;
foreach((array)get_the_category() as $cat) :
if(file_exists(SINGLE_PATH . '/single-cat-' . $cat->slug . '.php'))
return SINGLE_PATH . '/single-cat-' . $cat->slug . '.php';
elseif(file_exists(SINGLE_PATH . '/single-cat-' . $cat->term_id . '.php'))
return SINGLE_PATH . '/single-cat-' . $cat->term_id . '.php';
endforeach;
if(file_exists(SINGLE_PATH . '/single.php'))
return SINGLE_PATH . '/single.php';
elseif(file_exists(SINGLE_PATH . '/default.php'))
return SINGLE_PATH . '/default.php';
return $single;
}