How to compress/decompress a long query string in PHP?

Question

I doubt if this is encryption but I can't find a better phrase. I need to pass a long query string like this:

http://test.com/test.php?key=[some_very_loooooooooooooooooooooooong_query_string]

The query string contains NO sensitive information so I'm not really concerned about security in this case. It's just...well, too long and ugly. Is there a library function that can let me encode/encrypt/compress the query string into something similar to the result of a md5() (similar as in, always a 32 character string), but decode/decrypt/decompress-able?

Answer 1

base64_encode makes the string unreadable (while of course easily decodable) but blows the volume up by 33%.

urlencode() turns any characters unsuitable for URLs into their URL-encoded counterparts. If your aim is to make the string work in the URL, this may be the right way for you.

If you have a session running, you could also consider putting the query string into a session variable with a random (small) number, and put that random number into the GET string. This method won't survive longer than the current session, of course.

Note that a GET string should never exceed 1-2 kilobytes in size due to server and browser limitations.

Answer 2

Update:
gzcompress() won't help you. For example if you take Pekka's answer:

String length: 640
Compressed string length: 375
URL encoded string length: 925
(with base64_encode, it is only 500 characters ;) )

So this way (passing the data via the URL) is probably not the best way...

If you don't exceed the URLs limits with the string, why do you care about how the string looks like? I assume it gets created, sent and processed automatically anyway, doesn't it?

But if you want to use it as e.g. some kind of confirmation link in an email, you have to think about something short and easy to type for the user anyway. You could, e.g. store all the needed data in a database and create some kind of token.

---

Maybe gzcompress() can help you. But this will result in not allowed characters, so you will have to use urlencode() too (which makes the string longer and ugly again ;) ).

Answer 3

For long/very long string values, you would like to use POST method instead of GET !

for a good encoding you might wanna try urlencode()/urldecode()

or htmlentities()/html_entity_decode()

Also be carefull that '%2F' is translated to the browser as the '/' char (directory separator). If you use only urlencode you might wanna do a replace on it.

i don't recommend gzcompress on GET parameters.

Answer 4

The basic premise is very difficult. Transporting any value in the URL means you're restricted to a subset of ASCII characters. Using any sort of compression like gzcompress would reduce the size of the string, but result in a binary blob. That binary blob can't be transported in the URL though, since it would produce invalid characters. To transport that binary blob using a subset of ASCII you need to encode it in some way and turn it into ASCII characters.

So, you'd turn ASCII characters into something else which you'd then turn into ASCII characters.

But actually, most of the time the ASCII characters you start out with are already the optimal length. Here a quick test:

$str = 'Hello I am a very very very very long search string';
echo $str . "\n";
echo base64_encode(gzcompress($str, 9)) . "\n";
echo bin2hex(gzcompress($str, 9)) . "\n";
echo urlencode(gzcompress($str, 9)) . "\n";

Hello I am a very very very very long search string
eNrzSM3JyVfwVEjMVUhUKEstqkQncvLz0hWKUxOLkjMUikuKMvPSAc+AEoI=
78daf348cdc9c957f05448cc554854284b2daa442772f2f3d2158a53138b9233148a4b8a32f3d201cf801282
x%DA%F3H%CD%C9%C9W%F0TH%CCUHT%28K-%AAD%27r%F2%F3%D2%15%8AS%13%8B%923%14%8AK%8A2%F3%D2%01%CF%80%12%82

As you can see, the original string is the shortest. Among the encoded compressions, base64 is the shortest since it uses the largest alphabet to represent the binary data. It's still longer than the original though.

For some very specific combination of characters with some very specific compression algorithm that compresses to ASCII representable data it may be possible to achieve some compression, but that's rather theoretical. Update: Actually, that sounds too negative. The thing is you need to figure out if compression makes sense for your use case. Different data compresses differently and different encoding algorithms work differently. Also, longer strings may achieve a better compression ratio. There's probably a sweet spot somewhere where some compression can be achieved. You need to figure out if you're in that sweet spot most of the time or not.

Something like md5 is unsuitable since md5 is a hash, which means it's non-reversible. You can't get the original value back from it.

I'm afraid you can only send the parameter via POST, if it doesn't work in the URL.

Answer 5

You could try a combination of gzdeflate (raw deflate format) to compress your data and base64_encode to use only those characters that are allowed without Percent-encoding (additionally exchange the characters + and / by - and _):

$output = rtrim(strtr(base64_encode(gzdeflate($input, 9)), '+/', '-_'), '=');

And the reverse:

$output = gzinflate(base64_decode(strtr($input, '-_', '+/')));

Here is an example:

$input = 'Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.';

// percent-encoding on plain text
var_dump(urlencode($input));

// deflated input
$output = rtrim(strtr(base64_encode(gzdeflate($str, 9)), '+/', '-_'), '=');
var_dump($output);

The savings in this case is about 23%. But the actual efficiency of this compression precedure depends on the data you are using.