how to tokenize string to array of int in c?

Question

Anyone got anything about reading a sequential number from text file per line and parsing it to an array in C?

What I have in a file:

12 3 45 6 7 8
3 5 6 7
7 0 -1 4 5

What I want in my program:

array1[] = {12, 3, 45, 6, 7, 8};
array2[] = {3, 5, 6, 7};
array3[] = {7, 0, -1, 4, 5};

I've been through several ways to read it, but the only matter is only when i want to tokenize it per line. Thank you.

Answer 1

The following code will read a file a line at a time

char line[80]
FILE* fp = fopen("data.txt","r");
while(fgets(line,1,fp) != null)
{
   // do something
}
fclose(fp);

You can then tokenise the input using strtok() and sscanf() to convert the text to numbers.

From the MSDN page for sscanf:

Each of these functions [sscanf and swscanf] returns the number of fields successfully converted and assigned; the return value does not include fields that were read but not assigned. A return value of 0 indicates that no fields were assigned. The return value is EOF for an error or if the end of the string is reached before the first conversion.

The following code will convert the string to an array of integers. Obviously for a variable length array you'll need a list or some scanning the input twice to determine the length of the array before actually parsing it.

char tokenstring[] = "12 23 3 4 5";
char seps[] = " ";
char* token;
int var;
int input[5];
int i = 0;

token = strtok (tokenstring, seps);
while (token != NULL)
{
    sscanf (token, "%d", &var);
    input[i++] = var;

    token = strtok (NULL, seps);
}

Putting:

char seps[]   = " ,\t\n";

will allow the input to be more flexible.

I had to do a search to remind myself of the syntax - I found it here in the MSDN

Answer 2

What I would do is to make a function like this:

size_t read_em(FILE *f, int **a);

In the function, allocate some memory to the pointer *a, then start reading numbers from the f and storing them in *a. When you encounter a newline character, simply return the number of elements you've stored in *a. Then, call it like this:

int *a = NULL;
FILE *f = fopen("Somefile.txt", "r");
size_t len = read_em(f, &a);
// now a is an array, and len is the number of elements in that array

Useful functions:

• malloc() to allocate an array.
• realloc() to extend a malloc()ed array
• fgets() to read a line of text (or as much as can be stored).
• sscanf() to read data from a string (such as a string returned by fgets()) into other variables (such as an int array created by malloc() - hint hint)

Answer 3

Does your file have a specific number of lines or do you need to be able to read an arbitrary number into random arrays?

Here's code to read in a file line by line.

#include <stdio.h>

int main()
{
    char *inname = "test.txt";
    FILE *infile;
    char line_buffer[BUFSIZ];

    infile = fopen(inname, "r");
    if (!infile) {
        printf("Couldn't open file %s for reading.\n", inname);
        return 0;
    }

    while (fgets(line_buffer, sizeof(line_buffer), infile)) {
        // process line
    }

    return 0;
}

You can use sscanf or any of a number of tokenizing/converting functions to extract the numbers. BUFSIZ is a good constant from stdio.h that is designed to make stream I/O efficient on a target system.

Answer 4

I'd strongly suggest NOT to use sscanf and friends when the number of fields is variable. Use strtok and atoi. Just make sure to read the strtok manpage well, many programmers I know find its syntax a bit surprising in the beginning. Also not that it will modify the input string, so you may want to work on a copy.

Answer 5

The following code may be what you're looking for. Hopefully you won't need too much of a description given the comments but, if you have questions, feel free to ask.

It basically uses an fgets loop to read each line in and strtok to separate that line into fields. It constructs a linked list of integer arrays which contain the actual data - you can see the use of that linked list in the code at the end that dumps out the table.

It also has a means by which it can handle arbitrary-sized lines in the input file without buffer overflow (subject to memory constraints of course). Keep in mind that the strtok only expects one space between each field on the line although that could be recoded to handle multiple spaces or even any amount of white space. I've kept that bit simple since the code was already getting a little big :-)

The atoi function is used to convert the individual word on each line into integers. If you want error checking on those, I'd call your own variant which also checks that all characters in the word are numeric.

Using your input file of:

12 3 45 6 7 8
3 5 6 7
7 0 -1 4 5

it produces output along the lines of:

0x97b5170, size = 6:
   12 3 45 6 7 8
0x97b51d0, size = 4:
   3 5 6 7
0x97b51e0, size = 5:
   7 0 -1 4 5

Here's the code that produced that output:

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <errno.h>

// This is the linked list of integer arrays.

typedef struct _tIntArray {
    int size;
    int *array;
    struct _tIntArray *next;
} tIntArray;
static tIntArray *first = NULL;
static tIntArray *last = NULL;

// Add a line of integers as a node.

static int addNode (char *str) {
    tIntArray *curr;  // pointers for new integer array.
    char *word;       // word within string.
    char *tmpStr;     // temp copy of buffer.
    int fldCnt;       // field count for line.
    int i;

    // Count number of fields.

    if ((tmpStr = strdup (str)) == NULL) {
        printf ("Cannot allocate duplicate string (%d).\n", errno);
        return 1;
    }
    fldCnt = 0;
    for (word = strtok (tmpStr, " "); word; word = strtok (NULL, " "))
        fldCnt++;
    free (tmpStr);

 

    // Create new linked list node.

    if ((curr = malloc (sizeof (tIntArray))) == NULL) {
        printf ("Cannot allocate integer array node (%d).\n", errno);
        return 1;
    }

    curr->size = fldCnt;
    if ((curr->array = malloc (fldCnt * sizeof (int))) == NULL) {
        printf ("Cannot allocate integer array (%d).\n", errno);
        free (curr);
        return 1;
    }
    curr->next = NULL;

    for (i = 0, word = strtok (str, " "); word; word = strtok (NULL, " "))
        curr->array[i++] = atoi (word);

    if (last == NULL)
        first = last = curr;
    else {
        last->next = curr;
        last = curr;
    }

    return 0;
}

 

int main(void) {
    int lineSz;       // current line size.
    char *buff;       // buffer to hold line.
    FILE *fin;        // input file handle.
    long offset;      // offset for re-allocating line buffer.
    tIntArray *curr;  // pointers for new integer array.
    int i;

    // Open file.

    if ((fin = fopen ("qq.in", "r")) == NULL) {
        printf ("Cannot open qq.in, errno = %d\n", errno);
        return 1;
    }

    // Allocate initial line.

    lineSz = 2;
    if ((buff = malloc (lineSz+1)) == NULL) {
        printf ("Cannot allocate initial memory, errno = %d.\n", errno);
        return 1;
    }

    // Loop forever.

    while (1) {
        // Save offset in case we need to re-read.

        offset = ftell (fin);

 

        // Get line, exit if end of file.

        if (fgets (buff, lineSz, fin) == NULL)
            break;

        // If no newline, assume buffer wasn't big enough.

        if (buff[strlen(buff)-1] != '\n') {
            // Get bigger buffer and seek back to line start and retry.

            free (buff);
            lineSz += 3;
            if ((buff = malloc (lineSz+1)) == NULL) {
                printf ("Cannot allocate extra memory, errno = %d.\n", errno);
                return 1;
            }
            if (fseek (fin, offset, SEEK_SET) != 0) {
                printf ("Cannot seek, errno = %d.\n", errno);
                return 1;
            }
            continue;
        }

        // Remove newline and process.

        buff[strlen(buff)-1] = '\0';
        if (addNode (buff) != 0)
            return 1;
    }

 

    // Dump table for debugging.

    for (curr = first; curr != NULL; curr = curr->next) {
        printf ("%p, size = %d:\n  ", curr, curr->size);
        for (i = 0; i < curr->size; i++)
            printf (" %d", curr->array[i]);
        printf ("\n");
    }

    // Free resources and exit.

    free (buff);
    fclose (fin);
    return 0;
}

Answer 6

Use strtol() to parse each line:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    static char buffer[1024];
    static long values[256];

    while(fgets(buffer, sizeof buffer, stdin))
    {
        char *current = buffer;
        size_t i = 0;
        while(*current && *current != '\n' &&
            i < sizeof values / sizeof *values)
        {
            char *tail = NULL;
            errno = 0;
            values[i] = strtol(current, &tail, 0);

            if(errno || tail == current)
            {
                fprintf(stderr, "failed to parse %s\n", current);
                break;
            }

            ++i, current = tail;
        }

        // process values
        printf("read %i values\n", i);
    }
}