Yes.
The GAC supports side-by-side versioning of dlls, so multiple versions of System.Web.Extensions.dll can exist and run together peacefully.
If you're not doing development on the web server, to install ASP.NET Ajax Extensions 1.0 for ASP.NET 2.0, all you need to do is copy the System.Web.Extensions.dll (Version=1.0.61025.0) into the GAC.
(In your situation, you might have to install the .msi on a different server to get the System.Web.Extensions.dll out in the first place)
Make sure in the application to use the fully qualified assembly name when referencing ASP.NET Ajax Extensions 1.0 for ASP.NET 2.0. You'll need to check your web.config carefully to make sure all System.Web.Extensions references point to the right version. Anyways, The fully qualified assembly name will be:
System.Web.Extensions, Version=1.0.61025.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
So, I'm not sure why the installer didn't work for you, but since you don't need to actually execute the .msi on every machine you want to use it on (only if you need access to the source code), you should be ok with just the dll in the GAC.