Problems With Sending Larger Files Over Ruby's TCPServer

Question

I've set up a simple, single-service server which has been working just fine with small test files, but when I tried running larger, more practical files, things go wrong.

Sending a 5.2 MB file works fine. Sending a 30.3 MB file works, but takes a long time (Like 15 minutes or so). When sending a 38.5 MB file, the server receives it, but then it throws the error: msconvert_server.rb:20:in 'write': Invalid argument - 179.raw (Errno::EINVAL) (179.raw being the name of the file).

I'm at a loss as to why this is happening. This forum seemed to have the answer, and while it did speed up the send and receive time and also get pass line 20, it failed at another point. I don't believe that TCP has any file size limit, which leads me to believe that the problem lies in the Ruby code. Has anyone seen this problem, or have an idea as to what might be going on here?

Here's the code.

Server:

require 'socket'

server = TCPServer.open(2000)
loop {
  client = server.accept

  filename = client.gets.chomp

  puts "Reading contents of #{filename}.raw"
  raw_data = client.gets("\r\r\n\n").chomp("\r\r\n\n")

  (Line 20, where error occurs) File.open(filename + ".raw", 'wb') {|out| out.print raw_data}

  puts "Converting #{filename}"

  #It's lame to have a script run a script, but it's the only way to get this to work.
  system "scriptit.bat " + filename + ".raw"

  puts "Sending contents of #{filename}.mzML"
  client.print IO.read(filename + ".mzML")
  client.print "\r\r\n\n"
  puts "Done"
  client.close
}

Client:

    host = config_value("//Host/@ip")
    port = 2000

    client = TCPSocket.open(host, port)

    fileName = @file.split("/")[-1].chomp(File.extname(@file))

    puts "Sending raw file"
    client.puts fileName
    client.print(File.open("#{@file}", "rb") {|io| io.read})
    client.print("\r\r\n\n")  #This is the delimiter for the server

    puts "Receiving mzML file"
    File.open("#{$path}../data/spectra/#{fileName}.mzML", 'wb') {|io| io.print client.gets("\r\r\n\n")}
    client.close

Answer 1

I found a solution to the problem. By applying the same solution from a previous question, I changed the code to this:

Server: require 'socket'

server = TCPServer.open(2000)
loop {
  client = server.accept

  filename = client.gets.chomp

  puts "Reading contents of #{filename}.raw"
  raw_data = client.gets("\r\r\n\n").chomp("\r\r\n\n")
  file = File.open(filename + ".raw", 'wb')
  file.print raw_data
  file.close

  puts "Converting #{filename}"
  #It's lame to have a script run a script, but it's the only way to get this to work.
  system "scriptit.bat " + filename + ".raw"

  puts "Sending contents of #{filename}.mzML"
  data = IO.read(filename + ".mzML")
  client.print data
  client.print "\r\r\n\n"

  puts "Done"
  client.close
}

Client:

host = config_value("//Host/@ip")
port = 2000

client = TCPSocket.open(host, port)

fileName = @file.split("/")[-1].chomp(File.extname(@file))

puts "Sending raw file"
client.puts fileName
data = IO.read("#{@file}")
client.print data
client.print "\r\r\n\n"  #This is the delimiter for the server

puts "Receiving mzML file"
file = File.open("#{$path}../data/spectra/#{fileName}.mzML", 'wb')
data = client.gets("\r\r\n\n")
file.print data

client.close

It seems that expanding IO one-liners solves most of my large file Ruby problems.