Yes, you can:
string formatted = dt.ToString("dd'.'MM'.'yyyy");
Now in this case the quotes aren't actually required, as custom date/time format strings don't interpret dot in any special way. However, I like to make it explicit - if change '.' for ':' for example, then while it's quoted it will stay with the explicit character, but unquoted it would be "the culture-specific time separator". It wasn't entirely obvious to me whether "." would be interpreted as "the culture-specific decimal separator" or not, hence the quoting. You may feel that's over the top, of course - it's entirely your decision.
You may also want to specify the invariant culture, just to remove any other traces of doubt:
string formatted = dt.ToString("dd'.'MM'.'yyyy", CultureInfo.InvariantCulture);
(At that point the quotes around the dot become less relevant, as "." is the decimal separator in the invariant culture anyway.)