Endless problems with a very simple python subprocess.Popen task

Question

I'd like python to send around a half-million integers in the range 0-255 each to an executable written in C++. This executable will then respond with a few thousand integers. Each on one line. This seems like it should be very simple to do with subprocess but i've had endless troubles. Right now im testing with code:

// main()
u32 num;
std::cin >> num;

u8* data = new u8[num];
for (u32 i = 0; i < num; ++i)
    std::cin >> data[i];

// test output / spit it back out
for (u32 i = 0; i < num; ++i)
    std::cout << data[i] << std::endl;

return 0;

Building an array of strings ("data"), each like "255\n", in python and then using:

output = proc.communicate("".join(data))[0]

...doesn't work (says stdin is closed, maybe too much at one time). Neither has using proc.stdin and proc.stdout worked. This should be so very simple, but I'm getting constant exceptions, and/or no output data returned to me. My Popen is currently:

proc = Popen('aux/test_cpp_program', stdin=PIPE, stdout=PIPE, bufsize=1)

Advise me before I pull my hair out. ;)

Answer 1

This works perfectly for me:

#include <iostream>

int main()
{
    int num;
    std::cin >> num;

    char* data = new char[num];
    for (int i = 0; i < num; ++i)
        std::cin >> data[i];

    // test output / spit it back out
    for (int i = 0; i < num; ++i)
        std::cout << data[i] << std::endl;

    return 0;
}

python:

In [9]: from subprocess import Popen, PIPE
In [10]: a = Popen('./a.out', stdin=PIPE, stdout=PIPE)
In [11]: a.stdin.write("2\nab")
In [12]: a.stdout.read()
Out[12]: 'a\nb\n'

Note that I added a delimiter (\n) between the number of bytes to write, this is the safest if you do not want your c++ cin to fail on sending something like 3,1,2,3 which would concatenate to 3123, expecting so many arguments.

Answer 2

In C++, when you read a char or unsigned char from cin, it reads a single byte from stdin. However, you expect it to read a decimal representation of a number from 0 to 255. If you read an int instead it should read it correctly:

unsigned int n;
std::cin >> n;
data[i] = static_cast<u8>(n);

Or instead, you can have the Python code write the values as a sequence of bytes rather than digits, by using the chr function.