I'm aware of the possibility to mark untrusted objects as tainted, but what's the underlying purpose and why should I do it?
It used to be a pretty standard practice when writing CGIs in Perl. There is even a FAQ on it. The basic idea was that the run time could guarantee that you did not implicitly trust a tainted value.
One tracks taint as a security precaution, in order to ensure that untrusted data isn't mistakenly used for calculations, transactions, or interpreted as code.
Tracking taint via a built-in language feature is more clear and more reliable than tracking via coding conventions or relying on code review.
For example, input from the user can generally be considered 'untrusted' until it has been sanitized properly for insertion into the database. By marking the input as tainted, Ruby ensures satisfactory sanitation takes place and prevents a potential SQL injection attack.
For an example of an "ancient" (2005) coding practice that demonstrates how taint was tracked without such Perl and Ruby modules, read some good old Joel: