Any text that is ran trough t() is translatable.
E.g. the code in your theme:
<title><?php print "my title"; ?></title>
is not translatable, while
<title><?php print t("my title"); ?></title>
is ran trough the translation system.
For any string that is ran trough the t() function Drupal allows several ways of overriding them. Almost all Drupal core interface strings are ran trough the translation interface. With exceptions for any user-inputted data (manually added menu-items, site-slogans, titles etceteras).
To override, you can use the on-site interface in admin (admin > site building > localisation) or use the method you describe, to hardcode them in the settings.php.
You should try to avoid the latter on a fully multilingual site. That method is only meant to be used if you don't want to enable the entire translation interface just to change one or two strings.
But to start at the beginning: you should make sure the string is translatable at all. Only then can you change this on a per-language base.
If, in addition to the core -t()-ed strings you want to translate user-inputted strings, you will need the i18n module. A large module, but it allows to translate things such as the slogan and site title.