Working around NSFileHandle NSTask blocking when passing large amounts of data

Question

I've got a bit of code that handles exporting data from my application. It takes in an NSString full of XML and runs it through a PHP script to generate HTMl, RTF, etc. It works well unless a user has a large list. This is apparently due to it overrunning the 8k or so buffer of NSPipe.

I worked around it (I think) in the readPipe and readHandle, but I'm not sure how to handle it in the writeHandle/writePipe. The application will beachball at [writeHandle writeData:[in... unless I break on it in gdb, wait a few seconds and and then continue.

Any help on how I can workaround this in my code?

- (NSString *)outputFromExporter:(COExporter *)exporter input:(NSString *)input {
  NSString *exportedString = nil;
  NSString *path = [exporter path];
  NSTask *task = [[NSTask alloc] init];

  NSPipe *writePipe = [NSPipe pipe];
  NSFileHandle *writeHandle = [writePipe fileHandleForWriting];
  NSPipe *readPipe = [NSPipe pipe];
  NSFileHandle *readHandle = [readPipe fileHandleForReading];

  NSMutableData *outputData = [[NSMutableData alloc] init];
  NSData *readData = nil;

  // Set the launch path and I/O for the task
  [task setLaunchPath:path];
  [task setStandardInput:writePipe];
  [task setStandardOutput:readPipe];

  // Launch the exporter, it will convert the raw OPML into HTML, Plaintext, etc
  [task launch];

  // Write the raw OPML representation to the exporter's input stream
  [writeHandle writeData:[input dataUsingEncoding:NSUTF8StringEncoding]];
  [writeHandle closeFile];

  while ((readData = [readHandle availableData]) && [readData length]) {
    [outputData appendData:readData];
  }

  exportedString = [[NSString alloc] initWithData:outputData encoding:NSUTF8StringEncoding];
  return exportedString;
}

Answer 1

To be clear, this isn't just slow but it's actually freezing until you break in with a debugger? It's not a problem with your sub-process?

One would expect NSFileHandle to handle any data you throw at it, but perhaps you can split your data into smaller chunks using -subdataWithRange: to see what effect that has. You could also grab the fileDescriptor and use the POSIX APIs (fdopen, fwrite, etc.) to write to the stream. The POSIX APIs would provide more flexibility, if indeed that's what you need.

Answer 2

The simple, painful truth is that writing a lot of data to a subprocess and then reading a lot of data back from it is not something you can do in a single function or method without blocking the UI.

The solution is just as simple, and is certainly a painful-looking prospect: Make the export asynchronous. Write data as you can, and read data as you can. Not only are you then not blocking the UI, you also gain the ability to update a progress indicator for a really long export, and to do multiple exports in parallel (e.g., from separate documents).

It's work, but the UI payoffs are big, and the result is a cleaner design both internally and externally.

Answer 3

I believe what was happening was I was running into a deadlock caused by the [writeHandle writeData:[input dataUsingEncoding:NSUTF8StringEncoding]]; line filling overfilling its buffer and causing the application to hang until it is (never) emptied.

I worked around it by dispatching the write operation off to a separate thread.