views:

584

answers:

2

Hello--here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:

#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>

int main(int argc, char **argv) {
    int pipefd[2];
    int result;
    FILE *in_file;
    FILE *out_file;

    result = pipe(pipefd);

    in_file = fopen(argv[1], "rb");
    out_file = fopen(argv[2], "wb");

    result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
    printf("%d\n", result);

    result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
    printf("%d\n", result);

    if (result == -1)
        printf("%d - %s\n", errno, strerror(errno));

    close(pipefd[0]);
    close(pipefd[1]);
    fclose(in_file);
    fclose(out_file);

    return 0;
}

When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?

Thanks!

+2  A: 

From the splice manpage:

   EINVAL Target  file  system  doesn't  support  splicing; target file is
          opened in append mode; neither of the descriptors  refers  to  a
          pipe; or offset given for non-seekable device.

We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.

bdonlan
That was the problem. Thanks! I should have read the man page all the way through, and didn't realize that splice was filesystem-dependent. In my case I was copying to an NFS filer.I'm trying to find the fastest way to copy many files (which average at about 10MB) from one NFS filesystem to another. mmap-ing the source file and using write() aren't performing spectacularly.Thanks for the pointers!
ldamerow
+1  A: 

What kind of file system(s) are you copying to/from?

Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.

Duck
NTFS would make sense - that's implemented via FUSE, and the actual filesystem driver runs as a userspace process. With other filesystems, otoh, the filesystem can futz around with the page cache directly. It's a shame `splice()` doesn't have a clean automatic fallback to a copy loop though...
bdonlan
It is NTFS though splice doesn't appear to work on DOS either. Agree about the fallback. I'm tempted to do some simple benchmarks as it seems impressive to the naked eye test.
Duck