Splitting String C++

Question

Hi in C++ if i have a string, how can I split this into tokens?

Answer 1

It depends on how complex the token delimiter is and if there are more than one. For easy problems, just use std::istringstream and std::getline. For more complex tasks or if you want to iterate the tokens in an STL-compliant way, use Boost's Tokenizer. Another possibility (although messier than either of these two) is to set up a while loop that calls std::string::find and updates the position of the last found token to be the start point for searching for the next. But this is probably the most bug-prone of the 3 options.

Answer 2

Try using stringstream:

std::string   line("A line of tokens");
std::stringstream lineStream(line);

std::string token;
while(lineStream >> token)
{
}

Check out my answer to your last question:
http://stackoverflow.com/questions/275355/c-reading-file-tokens#275405

Answer 3

this works nicely for me :), it puts the results in elems. delim can be any char.

std::vector<std::string> &split(const std::string &s, char delim, std::vector<std::string> &elems) {
    std::stringstream ss(s);
    std::string item;
    while(std::getline(ss, item, delim)) {
     elems.push_back(item);
    }
    return elems;
}

Answer 4

You can use the C function strtok:

/* strtok example */
#include <stdio.h>
#include <string.h>

int main ()
{
  char str[] ="- This, a sample string.";
  char * pch;
  printf ("Splitting string \"%s\" into tokens:\n",str);
  pch = strtok (str," ,.-");
  while (pch != NULL)
  {
    printf ("%s\n",pch);
    pch = strtok (NULL, " ,.-");
  }
  return 0;
}

The Boost Tokenizer will also do the job:

#include<iostream>
#include<boost/tokenizer.hpp>
#include<string>

int main(){
   using namespace std;
   using namespace boost;
   string s = "This is,  a test";
   tokenizer<> tok(s);
   for(tokenizer<>::iterator beg=tok.begin(); beg!=tok.end();++beg){
       cout << *beg << "\n";
   }
}

Answer 5

With this Mingw distro that includes Boost:

#include <iostream>
#include <string>
#include <vector>
#include <iterator>
#include <ostream>
#include <algorithm>
#include <boost/algorithm/string.hpp>
using namespace std;
using namespace boost;

int main() {
    vector<string> v;
    split(v, "1=2&3=4&5=6", is_any_of("=&"));
    copy(v.begin(), v.end(), ostream_iterator<string>(cout, "\n"));
}

Answer 6

See also boost::split from String Algo library

string str1("hello abc-*-ABC-*-aBc goodbye");
vector<string> tokens;
boost::split(tokens, str1, boost::is_any_of("-*")); 
// tokens == { "hello abc","ABC","aBc goodbye" }