Get correct output from UTF-8 stored in VarChar using Entity Framework or Linq2SQL?

Question

Borland StarTeam seems to store its data as UTF-8 encoded data in VarChar fields. I have an ASP.NET MVC site that returns some custom HTML reports using the StarTeam database, and I would like to find a better solution for getting the correct data, for a rewrite with MVC2.

I tried a few things with Encoding GetBytes and GetString, but I couldn't get it to work (we use mostly Delphi at work); then I figured out a T-SQL function to return a NVarChar from UTF-8 stored in a VarChar, and created new SQL views which return the data as NVarChar, but it's slow.

The actual problem appears like this: “descriptionâ€ instead of “description”, in both SSMS and in a webpage when using Linq2SQL

Is there a way to get the proper data out of these fields using Entity Framework or Linq2SQL?

Answer 1

Well, once you get the data out, you could always try this:

Encoding.UTF8.GetString(Encoding.Default.GetBytes(item.Description))

assuming the field is encoded in the system ANSI page. You might have to create the right encoding with Encoding.GetEncoding() if for some reason it isn't (looked up from DB type, for example).