How should I store and retrieve UK pound symbols?

Question

Hi all,

This is a problem I come across every so often; I always end up writing some horrible bodge, but I'm sure there must be a correct way to deal with it (after all, it's surely not uncommon to want to work with UK pound symbols).

The problem: the user enters a UK pound symbol (£) into a textfield in the CMS. They click save, and the form field value is escaped using the JavaScript escape() function and POSTed using AJAX (jQuery). However, at some point the pound symbol becomes a question mark (incorrect character encoding?).

I can't just convert the symbol to it's HTML entity before saving to the database, because when the values are retrieved for display on the front end of the web site they are HTML encoded (so the entity will just appear as is).

To clear this up for me once and for all, what should I be doing here?

Answer 1

You don't need to use escape function. jQuery already handles encoding:

$.ajax({
    url: '/somepage',
    data: { param1: $('#textfield').val() },
    success: function() { }
});

Just make sure your application is set up for UTF-8 encoding:

<system.web>
    <globalization requestEncoding="utf-8" responseEncoding="utf-8" />
</system.web>

And HTML pages also:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />

Answer 2

Try to save UK pound symbol as £ so that when it is displayed in html response it will show the UK pound symbol.

Answer 3

the form field value is escaped using the JavaScript escape() function

There's your problem. escape is the wrong way to create URL parameter values, and should in general never be used. You are looking for encodeURIComponent, which does it correctly. One of the things escape messes up is non-ASCII characters like the £.

If you are using jQuery as you mention, you should never need to create a POST data string manually; just pass in a lookup like {x: '£'} and it will take care of the encoding for you.

There may well be further problems storing and retrieving a £ if your application isn't set up to handle Unicode correctly. Ideally you should be serving your pages as UTF-8 and using UTF-8 IO as mentioned by Darin, plus using NATIONAL characters (NVARCHAR) to store Unicode strings in SQL Server.

I can't just convert the symbol to it's HTML entity before saving to the database, because when the values are retrieved for display on the front end of the web site they are HTML encoded

Good! That's absolutely correct. You should never store HTML-encoded data in the database, it's utterly the wrong way to cope with output escaping issues. Continue to HTML-encode all text going through to output.