Hard to say without seeing the XAML, but my first thoughts are either 1) you have not set the DataContext property to the ViewModel or 2) you have some syntax issue in the Binding itself.
You should use ObservableCollection instead of IEnumerable<DirectoryModel> to support DataBinding. I'm also not sure the implementation of your DirectoryDetails getter is beneficial. Your setter sets the private variable directly and fires the PropertyChanged event - this is proper. But your getter also sets the variable directly, bypassing the PropertyChanged event. Not to mention that you have a getter doing the work of a setter, which is probably a bad idea on several levels. I think you would be better to simplify your getter and have it just return the private variable. Do you really need to retrieve teh details everytime or can you use the local variable?
I would also point out that you do not need to implement INotifyPropertyChanged on your Model: the ViewModel needs this interface to support DataBinding, but there is no real value in adding it to the Model class.