- An app written in silverlight makes it harder to use the browsers built in functionality (tabs, history, favorites, etc.)
- Silverlight makes the app slower to start.
- Silverlight won't work on all platforms
In general, I think a lot of the arguments for "Desktop app vs. Web app" are going to apply SL. You could very well ask the same question, "Why would you ever use silverlight instead of a click-once application?"
As an aside, silverlight isn't the first thing of it's kind. Before it there was Java applets, flash, and even javascript frameworks to make rich client side interaction possible within a browser. Currently Silverlight competes in the corporate sector with Adobe Flex, which also is showing potential.
In the end, HTML is always there. All of these technologies ride in on HTML, so there is no danger of HTML becoming outdated anytime soon.