Multibyte Safe Url Title Conversion in PHP

Question

I'm trying to create a multibyte safe title => url string converter, however I've run into the problem of not knowing how to allow legal asian (and other) characters in the url when removing others. This is the function set at the moment.

public static function convertAccentedCharacters($string)
{                            
    $table = array(
            'Œ'=>'CE', 'œ'=>'ce', '¥'=>'Y', 'Ÿ'=>'Y', 'µ'=>'u', 'ü'=>'u', 
            'Š'=>'S', 'š'=>'s', 'Đ'=>'Dj', 'đ'=>'dj', 'Ž'=>'Z', 'ž'=>'z', 'Č'=>'C', 'č'=>'c', 'Ć'=>'C', 'ć'=>'c',
            'À'=>'A', 'Á'=>'A', 'Â'=>'A', 'Ã'=>'A', 'Ä'=>'A', 'Å'=>'A', 'Æ'=>'A', 'Ç'=>'C', 'È'=>'E', 'É'=>'E',
            'Ê'=>'E', 'Ë'=>'E', 'Ì'=>'I', 'Í'=>'I', 'Î'=>'I', 'Ï'=>'I', 'Ñ'=>'N', 'Ò'=>'O', 'Ó'=>'O', 'Ô'=>'O',
            'Õ'=>'O', 'Ö'=>'O', 'Ø'=>'O', 'Ù'=>'U', 'Ú'=>'U', 'Û'=>'U', 'Ü'=>'U', 'Ý'=>'Y', 'Þ'=>'B', 'ß'=>'Ss',
            'à'=>'a', 'á'=>'a', 'â'=>'a', 'ã'=>'a', 'ä'=>'a', 'å'=>'a', 'æ'=>'a', 'ç'=>'c', 'è'=>'e', 'é'=>'e',
            'ê'=>'e', 'ë'=>'e', 'ì'=>'i', 'í'=>'i', 'î'=>'i', 'ï'=>'i', 'ð'=>'o', 'ñ'=>'n', 'ò'=>'o', 'ó'=>'o',
            'ô'=>'o', 'õ'=>'o', 'ö'=>'o', 'ø'=>'o', 'ù'=>'u', 'ú'=>'u', 'û'=>'u', 'ý'=>'y', 'ý'=>'y', 'þ'=>'b',
            'ÿ'=>'y', 'Ŕ'=>'R', 'ŕ'=>'r',
        );

    return str_replace(array_keys($table), array_values($table), $string); 
}

public static function convertStringToSafeCharacters($string)
{                 
    $string = self::convertAccentedCharacters($string);
    if (function_exists('mb_convert_encoding') === true)
    {
        $string = mb_convert_encoding($string, 'UTF-8', 'auto'); 
    }
    else if(function_exists('iconv') === true && ($iconvstr = @iconv('', 'UTF-8', $string)) !== false)
    {
        $string = $iconvstr;
    }
    else
    {
        $string = utf8_decode($string);
    }
    return strip_tags(trim($string));
}

public static function convertToUrlsafe($string, $options=array())
{    
    if(isset($options['separator']) === false || $options['separator'] === false)
    {
        global $_SITE;
        if(strpos($_SITE->urlsafe_format, 'underscore') !== false)
        {
            $options['separator'] = '_';
        }
        else
        {
            $options['separator'] = '-';
        }
    }
    if(isset($options['case']) === false || $options['case'] === false)
    {
        global $_SITE;
        $format = substr($_SITE->urlsafe_format, 0, 5);
        if($format === 'lower')
        {
            $options['case'] = 'lower';
        }
        else if($format === 'upper')  
        {
            $options['case'] = 'upper';
        }
        else
        {
            $options['case'] = 'mixed';
        }
    } 

    $string = self::convertStringToSafeCharacters($string);

    $separator_in_use = $options['separator'];
    $separtor_convert = $options['separator'] == '-' ? '_' : '-';
    $preg_changes = array(
//              convert other seperators into the seperator being used
        '/\\'.$separtor_convert.'/i'            => $separator_in_use, 
//              remove any none legal chars
        '/[^\-\_a-zA-Z0-9&\s]/i'                => '',
        '/\&\#\d+?\;/'                          => '', 
        '/\&\S+?\;/'                            => '',
//              replace spaces with separator
        '/\s+/'                                 => $separator_in_use,
//              replace amersands with and
        '/\&/'                                  => 'and',
//              now finally remove any last chars
        '/[^A-Za-z0-9\/'.$separator_in_use.']/' => '',
//              remove any double paddinging separators, ie ----
        '/'.$separator_in_use.'+/'              => $separator_in_use, 
//              remove separator padding from start and end of string
        '/'.$separator_in_use.'$/'              => '',
        '/^'.$separator_in_use.'/'              => ''
    );
    $string = preg_replace(array_keys($preg_changes), array_values($preg_changes), $string);
    switch($options['case'])
    {
        case 'lower' :
            $string = strtolower($string);
            break;
        case 'upper' :
            $string = strtoupper($string);
            break;
        case 'mixed' :
            // do nothing
    }
    return $string;
}

Please bear in mind that there is some CMS related functionality in there and that the functions are actually part of a class. The main function call would be

convertToUrlsafe($string);

And I'm pretty sure the problem lies with the regex replaces in that function itself but I'm not quite sure what to change without breaking some existing functionality.

I want to be able to take the following titles and have them return urlsafe titles.

"The Very First Blog!" => "The-Very-First-Blog"

"スーザンは本日、グラスゴーのベラヒューストン・パークでローマ法王の" (no idea what this says) but it needs to be converted to urlsafe and at the moment it just returns an empty string.

Answer 1

There are no "legal asian" characters in URLs: URLs can contain only ASCII characters.

Western characters you can normalize using your method (or e.g. iconv(); here is another alternative). For everything else, use percent encoding (i.e. urlencode). Modern browsers will automatically display the proper characters instead where appropriate.

Background: Unicode characters in URLs