The first problem I had with Prolog is that it isn't a logic programming language. It lacks the three-valued standard logic of "true", "false", and "don't know", conflating the two latter ones. In other words, the two truth values are actually "can be shown" and "cannot be shown". This gives Prolog real problems with the idea of "not", which is pretty basic to logical reasoning.
In normal logic, it's perfectly reasonable to prove a proposition by disproving its negation, this being called "reductio ad absurdam" (unless I've misspelled it). (Yes, there are people who have tried to reconstruct mathematics without using it, but that's getting a bit esoteric.) This simply doesn't work in Prolog, since there's no distinction between proved false and not proved anything.
Therefore, when I did a class project in Prolog, I got into trouble whenever I thought of it as programming logic. I'd always wind up doing something that required actual negation. Perhaps other people don't do that, but I wound up thinking of it as a pattern-matching language, and then had little difficulty finishing the project.
It's not possible to have a true logic-based language where the programmer can write things and really rely on the results. First-order predicate calculus (i.e., logic with variables, true-or-false functions, "and", "or", "not", "for all", and "there exists") is positively undecidable. (There are reasons why we keep pouring coffee into mathematicians rather than generate all possible theorems mechanically, after all.) There is no way for a programmer to know a priori whether a given proposition will be proved or not, even if the programmer already knows it to be true or false.
Edit: I also forgot the critical necessity of ordering clauses properly. In logic, it doesn't matter in what order you write things down. In Prolog, I kept getting into infinite loops, until I stopped treating it as a logic-based language. Again, it has some nice features as a pattern-matching language, but it isn't logic, and it seemed to me like a one-trick pony of a language. YMMV, but some other people seem to agree with me.