encoding

How is utf8 data supposed to look when stored in a database?

I need a bit of help understanding how utf8 data is supposed to look when stored inside the database. I'm using mysql and php, the database is set to utf8, the collation on the column "p_name" is set to "utf8_unicode_ci". When I insert the data I pass it through this function function convert_charset($in_str) { $cur_encod...

How do I encode something in PHP with a key?

I see base64_encode can encode values, but without a key. I need a function that takes a salt and data, encodes it, and can be decoded with the same salt (but if you try to decode it without the salt, it gives you gibberish). Is there a PHP function for this (can't find it, only modified versions of base64_encode). EDIT: Found the answ...

How to send special characters via mail from a shell script?

I have a script that runs on cron that outputs some text which we send to the 'mail' program. The general line is like this: ./command.sh | mail -s "My Subject" [email protected] -- -F "Sender Name" -f [email protected] The problem is that the text generated by the script has some special characters - é, ã, ç - since it is not ...

Error with urlencode in python

I have this: a = {'album': u'Metamorphine', 'group': 'monoku', 'name': u'Son Of Venus (Danny\xb4s Song)', 'artist': u'Leandra', 'checksum': '2836e33d42baf947e8c8adef48921f2f76fcb37eea9c50b0b59d7651', 'track_number': 8, 'year': '2008', 'genre': 'Darkwave', 'path': u'/media/data/musik/Leandra/2008. Metamorphine/08. Son Of Venus (Danny\xb4...

Is this base40 and if so how can I decode it in C#?

I have several encoded strings and I need to decode them, my guess is they could be base40 encoding. I really need to decode, but it would be nice to do the encoding as well all in C#. BTW, I have tried all standard types of decoding before coming to the conclusion that it appears to be base40. I couldn't find anything about base40 enc...

Do I need to html-encode title attributes (tooltips)?

In my markup I am using HTML title attributes which I set by the Tooltip property of various ASP.NET controls like an asp:Label. The content of those titles come from a database and I use data binding syntax, for instance: <asp:Label ID="PersonLabel" runat="server" Text='<%# HttpUtility.HtmlEncode(Eval("PersonShortName")) %>' T...

How to reverse the process of this C# Decode method to create an Encode method?

I would like to know how to reverse the process of the below DecodeBinaryBase64 so that I can have a matching Encode method. In short C# code that if given the output of this method it would return the same string that it took as input. private static string DecodeBinaryBase64(string stringToDecode) { StringBuilder builder = new S...

Problems displaying French accented characters in UTF-8

I'm working on a French language site built in CakePHP. I have tried multiple functions to try and convert the text into UTF-8 and display properly, but have had no success so far - any accented letters are displaying as a black diamond with a question mark. They do display correctly when I change the char set in the browser to ISO-885...

Java/XSLT character encoding

I am using XSLT to display some &#160; (space) and &#163; (£) signs but they all come out as question marks (unrecognised characters). My encoding is set to UTF-8 in the HTML and the XSLT header looks like this <xsl:stylesheet version="1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns="http://www.w3.org/1999/xhtml" xmlns:xs...

PHP DomDocument, DomXPath encoding issue

Hi, I'm having a problem with encoding from a wordpress feed that I just can't seem to figure out. I was loading my feed with DOMDocument->load but then did a file_get_contents and am now using ->XMLload with the same results. I did the XMLload so I could manipulate the feed if needed. The correct output that I'm looking for is - ‘ £....

Using an StringIO object for holding uu.encode/uu.decode data

Hello, I would like to do the following: import StringIO, uu my_data = StringIO.StringIO() # this is a file-like object uu.encode(in_file, my_data) # do stuff with my data (send over network) uu.decode(my_data, out_file) # here I finally write to disk The above code works. However, if I implement the previous step as a property in an...

xml parse error on illegal character

SO, I am asking as a last resort, as I am completely out of ideas. I have a Windows ASP.NET ASMX web services app that returns a serialized Person object with a -- name, address, email... etc but some attributes in the xml are encoded very weirdly, for instance- &#x1a (I dont know where the encoding takes place. I assume in the seria...

How Can I Get Around this EOutOfMemory Exception When Encoding a Very Large File?

I am using Delphi 2009 with Unicode strings. I'm trying to Encode a very large file to convert it to Unicode: var Buffer: TBytes; Value: string; Value := Encoding.GetString(Buffer); This works fine for a Buffer of 40 MB that gets doubled in size and returns Value as an 80 MB Unicode string. When I try this with a 300 MB Buffer...

Unescaping HTML in Django

I have html encoded text which reads like this: RT <a href="http://twitter.com/freuter"&gt;@freuter&lt;/a&gt;... I want this displayed as html but I am not sure if there is a filter which i can apply to this text to convert the html-encoded text back to html ... can someone help? ...

Jekyll does not parse UTF-8

I created a page in notepad and selected UTF-8 as the encoding while saving. Jekyll does not parse this page. It renders the liquid extensions in the page as they are. Now I saved the same page using ANSI encoding. Jekyll parses that easily and my site is up and running. But it is limited only to ANSI and some characters appear as a que...

Android: can not import org.apache.harmony.xml.ExpatReader

I need to parse an UTF-8 encoded input stream, so I think the most appropriate method is to use XMLReader reader = new ExpatReader(); InputSource source = new InputSource(in); source.setEncoding(encoding.expatName); reader.parse(source); For that I need to import org.apache.harmony.xml.ExpatReader but I cannot figure ...

Python base64.decode does not seem to work on windows

I am consuming a webservice (written in java) - that basically returns a byte[] array (the SOAP equivalent is base64 encoded binary data). I am using the python suds library and the following code works for me on my mac (and on cygwin under windows), but the decoding does not work on vanilla windows (python 2.6.5). I am primarily a jav...

php problem with encoding in a cli script reading id3 tags

i am trying to get a php cli script to go through a folder, get the id3 tags, which are in utf8 in cyrillic and but it in the database. when i execute the script i get the raw utf in the DB fields like "&#192;&#235;&#225;&#229;&#237;&#224;" here is the script <? set_time_limit(0); include('classes/adodb5/adodb.inc.php'); include ('cla...

Varchar to UniVarchar: What are the tradeoffs?

We need to tranfer our database which has a current encoding of utf 8 to an iso 1 database server. As not to do any more cleaning of data I am thinking of converting varchar to univarchar. So in this way unicode encoding can be preserved in the iso 1 database server. Aside from the size(1byte for varchar vs 2 for univarchar) what are th...

special character in php soap request wrongly displayed in xml out package

Hi, I'm using a webservice that in my request includes a ¥ symbol (chr 0165). I have set the encoding on my soap client to: $soap = new MySoapClient('address.wsdl', array('trace' => 1, 'encoding'=>'ISO-8859-1')); But when I look at my outgoing soap package the "¥" is changed to "Â¥" I'm not good at encoding but I've tried a number...