Default-value features of a programming language, if actually a documented part of the standard rather than just an accident of the implementation (which many past "default initializations" have been), are no different from any other features of a programming language. You might as well ask if it's wise to rely on anything else in the language, and the answer regardless of wisdom is that there's no choice -- you have to rely on something, and anything could hypothetically be changed in a future version.
Of course, if the thing that you're relying on is a commonly-used feature of the language, rather than an odd corner case, then there's a lot more chance that it will be retained in future versions. In addition, if you're concerned about such things, it's wise to choose a well-established language that has a history of maintaining backwards compatibility. Some languages take great pains to make sure that older code runs in the new version of the language, and some less so.
So, that's the general answer. The specific answer about default values is that it depends on the particular case, the language in question, and so forth. You can have absolute ironclad reliance on the fact that global static variables in C will be zero at program start. Some other cases are rather notably less reliable.