Is it OK to have a single
configuration, rather than separating
Debug and Release (in our case)?
It may be OK - it depends heavily on your case (but depending on your details I think it is very not OK).
We don't have a QA team, and don't use assertions.
Assertions are not the issue with a debug build. They are another tool you can use (or not).
Having a QA team or not should not influence (heavily) the decision between debug and release builds (but if you do have a QA team, sooner or later you will probably want to have a debug version of your product).
A QA team will affect the quality of your product heavily. Without dedicated QA (by someone other than the people who develop the application) you have no guarantee of the quality or stability of your product, you can provide no guarantee it does what it's supposed to do (or that it's fit for any purpose) and you cannot make meaningful measurements on your product in lots of areas).
It may be you actually don't need a QA team, but in most cases you're just depriving your development team and customers (internal or not) of a lot of necessary data.
A debug build should make it easier to - well - debug your product and track issue and fix them. If you are doing no organized QA, you may not even know what your main issues to fix are.
Methinks that you actually have a QA team, you just don't see it as such: your internal customers (that may even be you) are your QA team. It's a bad idea, to the degree your application's function is important.
Working with no QA team is like creating a car by yourself and taking it on the road for testing: you have no idea if the wheels are held together OK, or if the breaks work until you are in traffic. It may be you don't kill anyone, but I wouldn't put the critical data in your company in your untested application, unless it's not really critical.
Performance is important, application size isn't.
If performance is important, who measures it? Does the measurement code belong to your released application? Do you add it and remove it in the released code?
It sounds like you're doing ad-hoc development and with a performance-critical application with no QA team and no dedicated debugging I'd have lots of doubts your team can actually deliver.
I don't know your situation and there may be a lot I don't see in this so maybe it's OK.
Are there any cons to this approach?
Yes: you will either end up with diagnostics code in your release version, or have to remove the diagnostics code after fixing each problem and add it again when working on the next problem.
You should not remove the debug version only for optimization though. That's not a valid argument, since you can optimize your release version and leave the debug version as is.