There are some strategies for dealing with large Entity Framework models in this article. You could consider using them. However, I have found that most of the pain with the regeneration of the EDMX comes from changes made via dragging and dropping on the GUI designer. On the other hand, doing Update Model From Database or via the properties window tends to make changes in a fairly sensible manner, and does not tend to be difficult to merge.
The biggest problem, as far as I can see, is that the layout information for the visual object model in the conceptual/mapping/storage models are in the same file. In other words, the problem is not so much the size of the file itself or the changes made to the entity model itself, but the wholesale rearrangement which happens when you drag and drop an object on the GUI designer. I wish the GUI designer layout and the conceptual/mapping/storage models were into different files. I believe this would eliminate most of the pain with merging changes to the model.
Therefore, we have a semi-official policy of not making changes to the graphical layout of the model. This is not much of a loss, because when you have more than a couple dozen entities in your model, the one-page-only GUI designer is not really useful anyway. And it certainly makes merges a lot easier.
Version 4 of the Entity Framework will have an option to do artifact generation based on T4 templates. I'm no expert, but it may be possible to coax the GUI layout information into a different file using a T4 template.