String Encoding Issue

Question

Hi guys,

I am having a very weird issue in the way strings get stored in my database, and as a result, I am getting these "unterminated string literal" errors in Javascript.

Here's an overview of what I am doing:

Platform: C#/ASP.NET MVC 1.0, SQL Server 2005, SparkViewEngine, YUI 2

In my view, I serialize an object into a JSON data structure using Json.NET from NewtonSoft.

<script type="text/javascript">
    // <![CDATA[ 
    var data = YAHOO.lang.JSON.parse("${Newtonsoft.Json.JsonConvert.Serialize(Model)}");
    ....
</script>

Normally, this works, but I noticed that one of the fields I pull from the database contains the following data, which causes the string to not be formed properly.

The database field is an NVARCHAR(2000).

For some of the entries, I get these weird characters in the string when I copy and paste from the database to notepad.

Compile ?Analysis ?& ?Recommendations? deck

In Firebug, it shows as a bunch of line breaks:

Analysis & Recommendations deck","StartDate":"1/19/10","FinishDate":"1/26/10","Duration":6.0,"

Compile 
Analysis 
& 
Recommendations
 deck

UPDATE After talking to the user, I discovered that they were using the copying and pasting from a word document onto the HTML form.

The form itself uses the YUI Connection Manager to make an asynchronous POST call (AJAX) to save the form values. The database saved the form field value along with any encoding associated with it.

It seems like there are characters in Word that are printable, but not in ASCII. Is there any way to detect this and encode correctly?

Answer 1

This appears to be a case of you not handling the quote characters properly. You have several ways to go. The way I use for comments for our web site is to make sure all quotes are encoded, both single and double, so that both your SQL, ASPX & Javascript won't hiccup. Our method is for customer comment fields so we're overboard on the conversions.

Function SafeComment(ByVal strInput)
' Renders Any Comment Codes Harmless And Leaves Them HTML readable In An eMail Or Web Page
' Try: SafeComment("`~!@#$%^&*()_+=-{}][|\'"";:<>?/.,")
    SafeComment = ""
    If Len(strInput) = 0 Then Exit Function
    SafeComment =   Replace(Replace(Replace( _
                    Replace(Replace(Replace( _
                    Replace(Replace(Replace( _
                    Replace(Replace(Replace( _
                        Server.HtmlEncode(Trim(strInput)), _
                    "-", "&#45;"), ":", "&#58;"), "|", "&#124;"), _
                    "`", "&#96;"), "(", "&#40;"), ")", "&#41;"), _
                    "%", "&#37;"), "^", "&#94;"), """", "&#34;"), _
                    "/", "&#47;"), "*", "&#42;"), "'", "&#39;")
End Function

It's not pretty, but it does the job.