what's the least frequently used characters for web users?

Question

I need this to be used as a delimiter,

has anyone known about this statistics?

Answer 1

I'm sure there are tons of strange unicode characters that don't get used much, but that's probably not what you're looking for.

Why do you want something "rare" for a delimiter? How will it be used?

Answer 2

What about using a string of characters as delimiter?

Answer 3

Pick any character, then pick a mechanism to escape that character to handle the case where the user wants to type it. For example, in comma delimited files the comma is the separator:

1,2,fred,john

Unless the data itself contains a comma, then you quote it:

1,2,"Bloggs, Fred",john

And if you need use a quote:

1,2,"Bloggs, Fred","Jean-Luc \"Make it so\" Picard"

Answer 4

I don't think it matters what character you use, you shouldn't just hope that no-one will type your delimiter. Use a comma and handle the users adding their own commas.

Answer 5

You could prefix whatever data you have on the web with the length.. that's how HTTP-Chunked encoding sends things across the web.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html

Answer 6

In such cases, I like the use the vertical bar | character.

• It's easy to spot when looking at a text file.
• It clearly marks a separation.
• It's rarely used.
• And, since it has no intrinsic meaning in English grammar, it is easy to either just disallow it or blindly change it to something else (like a dash) if it appears in the column text.

Answer 7

You sound like you're trying to convert a list of strings into a single string in such a manner that you can later turn it back into a list of strings.

There are several traditional approaches to this, most of them already mentioned in this thread:

• Use an unusual character as a delimiter, and simply don't allow it in your input, either by rejecting input containing the delimiter, or by replacing the delimiter with "?" or "." or similar. For this, I agree with the person who suggested the vertical bar (|)
  • Advantage: dirt simple to code, in a wide variety of languages
  • Disadvantage: You lose some expressiveness and chances for future expansion by eliminating the possibility of input containing your delimiter.
• Use a delimiter, and an escape mechanism when the delimiter appears in input. There are actually a few variants to this:
  • The "just like C code" method, where you prepend an escape character to every occurence in your data of your delimiter or your escape character. For example: the string «Greetings,Hey,Hello\,World,Hello \\ Backslash» contains four elements, using , as the delimiter and \ as the escape character. (The last element has one backslash originally)
    • This is actually a royal pain to code and implement correctly in many languages
    • Even once you do implement it, it's generally much slower compared to other methods
  • The "like URL parameters" method where your escape mechanism is to convert your delimiter into a multi-character sequence that does not contain your delimiter. You then also need to convert the first character of whatever your delimiter turns into to its own multi-character sequence. For example, if you decided to use , as your delimiter, and decide to represent , as «\1» and \ as «\2», you could write the last example as: «Greetings,Hey,Hello\1World,Hello \2 Backslash»
    • This is usually not too hard to implement. The advantage is that you can do the "splitting" and "unescaping" parts of going from string to list-of-strings in separate steps. The unescaping process might be a tiny bit tricky, since you have to do it as a scan of each string.
  • Like CSV files, with quotes around items that contain your delimiter, and the quotes escaped according to some obscure mechanism. (Such as by doubling)
    • Avoid this unless you can just throw it at a pre-existing library.
    • This has all the disadvantages of the "Like C code" method, plus extra confusing state to screw up when implementing it.
  • One of the above methods, but with a multi-character delimiter. This is harder than you'd think; the extra characters actually significantly complicate the logic of what exactly should be escaped.
• Prefix each item with its length, then include the item unchanged
  • This is used by HTTP in its "Chunked" encoding, by bencoding (the wire format bittorrent uses), and by Google's protocol buffers.
  • Implementing this can be a tiny bit tricky, and is very prone to off-by-one errors. I still think it's easier to implement than the "like C code" method, especially in low-level languages.
  • Once you do implement it correctly, it's generally much faster than the other schemes, even the lossy scheme that just forbids input containing the delimiter. (The exception is if you're working in a high-level language that has a built-in "split" routine)