I'd use CRM to store the data. You can stick the new fields in a separate pane in the UI so that it won't clutter. You can even add some Javascript to the UI to hide pane/fields from users who are not part of the group that requires them. I know this sounds a little hacke-ty, but it's a lot less work than coming up with an entirely different app and users will get a consistent experience. Having the data in one place is also a boon for reporting and such.
I can't say for sure, but I don't think adding a few columns to an entity (which already has a bazzilion columns) will deteriorate performance much further. I'd go over the installation and check for the usual performance pitfalls.