Actually, you said it reasonably well right there.
The truth is that the "instance of" comb is almost always a bad idea (the exception happening for example when you're marshaling or serializing, when for a short interval you may not have all the type information at hand.) As josh says, that's a sign of a bad class hierarchy otherwise.
The way that you know it's a bad idea is that it makes the code brittle: if you use that, and the type hierarchy changes, then it probably breaks that instance-of comb everywhere it occurs. What's more, you then lose the benefit of strong typing; the compiler can't help you by catching errors ahead of time. (This is somewhat analogous to the problems caused by typecasts in C.)
Update
Let me extend this a bit, since from a comment it appears I wasn't quite clear. The reason you use a typecast in C, or instanceof, it that you want to say "as if": use this foo as if it were a bar. Now, in C, there is no run time type information around at all, so you're just working without a net: if you typecast something, the generated code is going to treat that address as if it contained a particular type no matter what, and you should only hope that it will cause a run-time error instead of silently corrupting something.
Duck typing just raises that to a norm; in a dynamic, weakly typed language like Ruby or Python or Smalltalk, everything is an untyped reference; you shoot messages at it at runtime and see what happens. If it understands a particular message, it "walks like a duck" -- it handles it.
This can be very handy and useful, because it allows marvelous hacks like assigning a generator expression to a variable in Python, or a block to a variable in Smalltalk. But it does mean you're vulnerable to errors at runtime that a strongly typed language can catch at compile time.
In a strongly-typed language like Java, you can't really, strictly, have duck typing at all: you must tell the compiler what type you're going to treat something as. You can get something like duck typing by using type casts, so that you can do something like
Object x; // A reference to an Object, analogous to a void * in C
// Some code that assigns something to x
((FoodDispenser)x).dropPellet(); // [1]
// Some more code
((MissleController)x).launchAt("Tehran"); // [2]
Now at run time, you're fine as long as x is a kind of FoodDispenser at [1] or MissleController at [2]; otherwise boom. Or unexpectedly, no boom.
In your description, you protect yourself by using a comb of else if and instanceof
Object x ;
// code code code
if(x instanceof FoodDispenser)
((FoodDispenser)x).dropPellet();
else if (x instanceof MissleController )
((MissleController)x).launchAt("Tehran");
else if ( /* something else...*/ ) // ...
else // error
Now, you're protected against the run-time error, but you've got the responsibility of doing something sensible later, at the else.
But now imagine you make a change to the code, so that 'x' can take the types 'FloorWax' and 'DessertTopping'. You now must go through all the code and find all the instances of that comb and modify them. Now the code is "brittle" -- changes in the requirements mean lots of code changes. In OO, you're striving to make the code less brittle.
The OO solution is to use polymorphism instead, which you can think of as a kind of limited duck typing: you're defining all the operations that something can be trusted to perform. You do this by defining a superior class, probably abstract, that has all the methods of the inferior classes. In Java, a class like that is best expressed an "interface", but it has all the type properties of a class. In fact, you can see an interface as being a promise that a particular class can be trusted to act "as if" it were another class.
public interface VeebleFeetzer { /* ... */ };
public class FoodDispenser implements VeebleFeetzer { /* ... */ }
public class MissleController implements VeebleFeetzer { /* ... */ }
public class FloorWax implements VeebleFeetzer { /* ... */ }
public class DessertTopping implements VeebleFeetzer { /* ... */ }
All you have to do now is use a reference to a VeebleFeetzer, and the compiler figures it out for you. If you happen to add another class that's a subtype of VeebleFeetzer, the compiler will select the method and check the arguments in the bargain
VeebleFeetzer x; // A reference to anything
// that implements VeebleFeetzer
// Some code that assigns something to x
x.dropPellet();
// Some more code
x.launchAt("Tehran");