remove non ascii characters in a file

Question

How do i remove non ascii characters from a file?

Answer 1

tr -dc [:graph:][:cntrl:] < input-file > cleaned-file

That's assuming you want to retain "control" characters and "printable" characters. Fiddle as required.

Answer 2

My 2 cents might not solve your problem but may give you some hints.
file command tells you file encoding i.e. utf, ascii, etc and iconv can convert a file between different encodings.

Answer 3

perl -pe's/[[:^ascii:]]//g' < input.txt > output.txt

Answer 4

what language do you want to use? You can do this in perl like this:

perl -pi -e 's/[[:^ascii:]]//g' filename

Answer 5

You can write a C program like this:

#include <stdio.h>
#include <ctype.h>

int main(int argc, char **argv)
{
   FILE *fin = fopen("source_file", "rb");
   FILE *fout = fopen("target_file", "w");
   int c;
   while ((c = fgetc(fin)) != EOF) {
       if (isprint(c))
          fputc(c, fout);
   }
   fclose(fin);
   fclose(fout);
   return 0;
}

Note: error checks were avoided for simplicity.

Compile it with:

$ gcc -W source_code.c -o convert

Run it with:

$ ./convert