I would like to create a folder that users who do not have privileges to view the rest of the site can see. This user group would be granted access to the site, but I only want them to be able to view one particular page.
Is this possible to do without going to every single page and removing the new user group's access?
yeah, you should be able to create a new group and add the users to that list/subweb/whatever and just that. This is assuming that you didn't grant access to all users somewhere. If you did, then hopefully the default access is granted to a default user group (like sharepoint visitors) and you can alter that group to exclude the users you only want to access the limited part of the site.
If created correctly the new group shouldn't have access to the rest of the site.
If you are getting thrown off by the fact that the user/group is listed as having "Limited Access" on the ACLs on, say, the parent site/web. That's just a placeholder SharePoint uses to make sure people have access to at least the bare minimum set of objects (e.g. theme and other UI files and the parent web itself) to get to the list or item you actually want them to have access to.
As long as the group only has access on a single list, you should have to worry about them having access to anything else.