character-encoding

How to handle string encoding in java ?

I was really discouraged by java's string encoding. There are many auto conversions in it. and I can't found the regular. Anyone have good idea? for example: In a jsp page, it has such link http://localhost:8080/helloworld/hello?world=凹ㄉ And then we need to process it, so we do this: String a = new String(request.getParameter("world"...

Why the character is corrupted when use request.getParameter() in java?

I have such a link in JSP page with encoding big5 http://hello/world?name=婀ㄉ And when I input it in browser's URL bar, it will be changed to something like http://hello/world?name=%23%24%23 And when we want to get this parameter in jsp page, all the characters are corrupted. And we have set this: request.setCharacterEncoding("UTF-8"), s...

How is transformation of code point to final character implemented in Unicode?

Characters included in BMP as specified by 4 digits, and those characters outside of BMP contains 5 or 6 digits. But my doubt is: how is the finanal character drawed from value of code point? Are the pictures of each character restored in each computer and when displaying just show the matching picture? Or the final glyph is a comput...

I do replace literal \xNN with their character in Perl?

I have a Perl script that takes text values from a MySQL table and writes it to a text file. The problem is, when I open the text file for viewing I am getting a lot of hex characters like \x92 and \x93 which stands for single and double quotes, I guess. I am using DBI->quote function to escape the special chars before writing the value...

question about Character encoding in Web

let's say I have a JSP Page(i just list part of it, please don't mind): <%@ page language="java" contentType="text/html;charset=UTF-8"%> <form> <input type=input> </input> 中華<!--character with BIG5 encoding> </form> and In server side I use this request.setCharacterEncoding("UTF-8"); my problem is: If i use...

Character Encoding Issue - Strange Behaviour From Pound Signs (£) with UTF-8 IE6 / ASP / XML

Hi folks, I am having a very strange problem with pound signs displaying incorrectly (or not at all) on a web page. I am keying text in a textbox, which then gets (briefly) stored in XML before being displayed in a new IE(6) window. The worst part is that this is inconsistent. I have three different things happening: 1. Pound sign doe...

Encoding problems with ogr2ogr and Postgis/PostgreSQL database

Hi! In our organization, we handle GIS content in different file formats. I need to put these files into a PostGIS database, and that is done using ogr2ogr. The problem is, that the database is UTF8 encoded, and the files might have a different encoding. I found descriptions of how I can specify the encoding by adding an options param...

What is "ANSI as UTF-8" and how can I make fputcsv() generate UTF-8 w/BOM?

I made a PHP script that generates CSV files that were previously generated by another process. And then, the CSV files have to be imported by yet another process. The import of the old CSV files works fine, but but when importing the new CSV files there are issues with special characters. When I open old CSVs with Notepad++, it says t...

How can I retrieve an Oracle record when my key field contains a mu ('μ') character?

I have a small table of measurement units in Oracle (10.2.0.4). It's defined as CREATE TABLE Units ( UNIT_ID number, UNIT varchar2(12) ) It's populated with a few records, and one of those records has a unit value of 'μL'. When I try to query for that record using this query... select * from units where unit = 'μL' .. I g...

How can we save special symbols in xml

Hi, I am saving "Children’s Music & videos" to xml so i used the following code to convert the special symbols string str = "Children’s Music & videos"; string temp = HttpUtility.HtmlEncode(str); but the xml is not readable after saving the encoded value. How can i save this special symbols to xml Thanks, ...

How do I reverse escape backslash ecodings like "\ " and "\303\266" in bash?

I have a script that records files with UTF8 encoded names. However the script's encoding / environment wasn't set up right, and it just recoded the raw bytes. I now have lots of lines in the file like this: .../My\ Folders/My\ r\303\266m/... So there are spaces in the filenames with \ and UTF8 encoded stuff like \303\266 (which is ö...

Overcoming text encoding from database to filesystem?

Hi folks, in my SQL 2000 database, the model names for my vehicles have accents on some of the letters, like so Scénic 2.0 On the filesystem, the filenames look correct on the server if I browse the folder on the local machine, like so: Car-2005-Renault-Scénic-2_0--Multi-Purpose Vehicle-1157-01tn.jpg When I use the filename in a li...

Good resources for learning the different types of Character Encoding and converting between them

One thing I have never truly understood is the concept of character encoding. The way encoding is handled in memory and code often baffles me in that I just copy an example from the internet without truly understanding what it does. I feel it's a really important and much overlooked subject that more people should take the time to get ri...

Accentuated characters look funny in html passed in ajax call..

Hi guys, I've run into issues here I notice that some accentuated characters if I try to pass them as value sin an ajax call they end up all funny LIke for example: Adana Şakirpaşa turns into Adana %u015Eakirpa%u015Fa WHats wrong here :( EDIT================== The problem is that once the characters are recieved by my php script on...

How do I recover a document that has been sent through the character encoding wringer?

Until recently, my blog used mismatched character encoding settings for PHP and MySQL. I have since fixed the underlying problem, but I still have a ton of text that is filled with garbage. For instance, ï has become ï. Is there software that can use pattern recognition and statistics to automatically discover broken text and fix it? ...

MySQL Database has non escaped single quotes in entires ... How to show them?

Hello, The database has a ton of entries that were not escaped because they were inputted manually when they were inserted so they look like: Don't inside of the entry, but when I try to display them they have a weird characters when I output in PHP. Before I would put anything into the database I would usually use mysqli_real_escape_s...

Writing a connection string when password contains special characters

I'm using SQLalchemy for a Python project, and I want to have a tidy connection string to access my database. So for example: engine = create_engine('postgres://user:pass@host/database') The problem is my password contains a sequence of special characters that get interpreted as delimiters when I try to connect. I realize I could ju...

Charsets in general, with specific questions regards Zend DB - incorrect display of data.

Hello, I'm having trouble in my Zend Framework App displaying the correct characters. EG/ rue de l'Odéon Displays as: rue de l'Od�on When displaying information through a webservice and also by displaying it through the applications view. I have tryed the following things: 1) Set the character encoding in the view. $view->setEn...

Encoding problem with nordic characters

I'm trying to send HL7 messages to a receiver. The receiver now says they got question marks instead of 'ä' and 'ö'. What should I do? Right now I do it about like this: I initialize System.Net.Sockets.TcpClient and then Dim data(payload.Length) As Byte Dim stream As System.Net.Sockets.NetworkStream = _tcpClient.GetStream() System.Tex...

PHP/HTML display hidden characters

So, I have a bunch of code that I'm pulling from a column in MySQL. This code includes hidden characters, such as "\t" and "\n". I'm trying to get that raw code to display in a DIV. I was hoping to actually show the hidden characters. I have the following so far, without much luck: <?php // Simple example: replace all newlines with the...