If adding duplicate songs is not an expected use scenario for your API, you should throw an Exception, yes.
However by looking at the given example, since you're implementing an album, "duplicate song" seems to be a problem that should be avoided by API design.
First, I think you should have a method to check if the album contains a certain song is in the library; ContainsSong
for instance. The client should know that duplicate songs are not expected and they need to be provided with necessary tools to avoid that by themselves.
This way client can check for duplicates before the actual call. In fact, a client in this case may not even allow "Add"ing in the first place, by disabling the "Add" button or not showing it at all. (Or providing nice hint messages). The point is, if this is something you don't expect, your design should be shaped in a way to avoid it.
Then after having this kind of nice layout, you can still throw your exception, because in this case, it would have really been "unexpected" to get a duplicate song from the client.
Otherwise having just the Exception narrows down client's options to handle the situation smoothly. It eventually may turn out to be a problem of "call Open inside a try block to see if file exists", which promotes drug use.