Is it acceptable for an API to have bad behaviors (like segfault, bus error, memory leak) if the condition that would cause the bad behaviors is documented? Or should it always fail "gracefully" in all known conditions?
not at all , it shouldnt crash , when it failed , it should return a value which indicate it
I would personally prefer a graceful fail over a crash any day, especially if the cause of the crash scenario is known and documented. Hopefully since it's documented it will get fixed at some time in the future .. yes?? ;)
Absolutely. It's called undefined behavior (link), but you should only use it in cases where condition verification results in an unacceptable runtime performance hit or massively over complicated code and the documentation can clearly state what those conditions are.
As others, I believe in graceful fail over a crash. If the nature of the failure is totally dependent on values calculated with the inside of the API implementation, it should be a graceful failure. However one must also consider that when the failure is a consequence of bad input, you may have to just rely on documentation, as prevention may hurt the performance of common case for the safety of the odd ball case. This may be worthwhile if your software is to be run in mission-critical circumstances.
So...
If the API receives a pointer, it is likely that either it is impossible (usual case) or too computationally expensive to determine the validity of the pointer (in the cases where the API controls the lifetime of the memory pointed to, because it would have to be tracked and searched)
It is easy to fend off most "sigbus" due to alignment from the input arguments, by checking the modulus of the address with the alignment for the platform. Note that verification will be mostly platform dependent. If you work on multiple platforms this will probably will become bug prone.
As far as memory leaks, the best you can hope for is to document really well the ownership protocol of the memory. Hard to "gracefully" fail there. You can actually think of the memory leak as the "graceful" failure.