How to ensure that a desktop program works correctly after a clean-slate installation?

Question

Motivation — I had a new version of my Cocoa application ready that worked fine on all beta testers' machines. So I released it. Turns out that a crucial feature simply doesn't work on anybody else's computer. Yikes! Yes, read that again: I released software that didn't work.

Cause — Users who had used previous versions my app (read: all my loyal beta testers) already had a folder ~/Application Support/MyApp/ from an older version. Due to the critical bug in the new release, this folder was necessary for the software to work. And for everybody else, because the folder did not exist, it didn't work.

As you can imagine, this is extremely embarrassing, and I want this to never, ever, happen again.

Remedies? — The straight-forward way to ensure this, of course, is to actually download and install it on a "clean" machine just before you publish a new release. But this seems impractical, because in time I will run out of friends with a Mac who have never tried my app yet (eventually all will have ;-)), and because I'm not eager to "format c:" my Mac before every single release…

This is where I need your help:

How can I ensure that a user who has never used my software before will get the same results as someone who has?

Answer 1

Virtual machines (VMWare Workstation, etc.) can be useful for testing clean installs of applications. You can start up a new virtual OS, install your stuff, test it, then delete the VM when you're done. There are ways to automate the spinning-up of a VM as well, which can make your life even easier.

Another thing to do is to determine all the prerequisites your app requires, and add checks for these things at startup. If something is not setup right, you can either attempt to set it up within your code, or inform the user to do it.

Answer 2

A more lightweight approach might be using tools like AppZapper to get rid of everything — temporary files, preferences, cache, history, etc – related to the app to test.