You are doing more work on the server and the client so it is normal that response times increase. On low bandwidth connections you might make it good with reduced transfer times.
If you are on a high bandwidth connection then the compression will have not have a significant impact on the transfer delay as it is already short uncompressed. However you will pay 100% of the CPU penalty.
Now zipping large responses takes quite some CPU power, if the server CPU's are already loaded the response times might even get worse.
My advice : check the server CPU and if it is non-negligeable then either turn off zipping or buy a bigger box. If you have a large population on mobile or in remote locations with poor internet connectivity then zipping might be useful, otherwise it will make little difference.
You might also look in using a reverse proxy to reduce the load of the server.