I'm trying to split a string by a delimiter but it doesn't work, why?

Question

Hi All,

I wrote below code to readin line by line from stdin ex.

city=Boston;city=New York;city=Chicago\n

and then split each line by ';' delimiter and print each record.

But for some reason the "record" pointer comes back always null. Why?

    char    del = ';';
    char    input[BUFLEN];

    while(fgets(input, BUFLEN, fp)) {

            input[strlen(input)-1]='\0';
            char* record = strtok(input, &del);

            while(record) {
                    printf("Record: %s\n",record);
                    record = strtok(NULL, &del);
            }
}

Answer 1

Try char * del = ";";

The second parameter should be a null terminated string, i.e. an array of chars with a null termination.

And then pass in del to strtok.

   char*    del = ";";
   char    input[BUFLEN];

    while(fgets(input, BUFLEN, fp)) {

            input[strlen(input)-1]='\0';
            char* record = strtok(input, del);

            while(record) {
                    printf("Record: %s\n",record);
                    record = strtok(NULL, del);
            }
     }

Answer 2

strtok needs a string with a nul (zero byte) at the end for its second argument. Change del to be

 char * del = ";"

Here is a full program (minus the input reading part):

#include <string.h>
#include <stdio.h>

int main ()
{
   char*    del = ";";
   char * input = strdup ("city=Boston;city=New York;city=Chicago\n");
   char* record = strtok (input, del);
   while (record) {
       printf("Record: %s\n",record);
       record = strtok (NULL, del);
   }
}

Try it: http://codepad.org/tzzxjOJE

There is also a strsep function which has the advantage of handling more than one string at a time.

Answer 3

Your call to strtok() is very likely to segfault. The char * pointed to by &del does not have a NULL terminator.

Answer 4

I'm not sure but I think because your del won't have the proper '\0' at the end.

You have to make sure the string is NULL terminated or most of the string function are gonna read the memory after your variable which could leads to LOTS of problems

something like this would be better:

    char    *del = ";";
    char    input[BUFLEN];

    while(fgets(input, BUFLEN, fp)) {

            input[strlen(input)-1]='\0';
            char* record = strtok(input, del);

            while(record) {
                    printf("Record: %s\n",record);
                    record = strtok(NULL, del);
            }
}

Answer 5

strtok() expects the delimiters string to be exactly that - a null-terminated C string. You're passing it a single character, which obviously isn't a null-terminated string since a semicolon is not a null character. Instead, try this:

char del[] = ";";