Can i deal with many files at the same time in Haskell ?

Question

Hi

I have to solve a following problem.

there are many files let's say 3 for example, with the following content

file1

a1

a2

a3

a4

a5

a6

......

file2

b1

b2

b3

b4

b5

b6

......

file3

c1

c2

c3

c4

c5

c6

......

my program has to take filenames in parameter, read those files and print the following result

"a1 b1 c1"

"a2 b2 c2"

"a3 b3 c3"

"a4 b4 c4"

"a5 b5 c5"

"a6 b6 c6"

......

I've already wrote one version of the program, but it start by reading all the files, and that is very inneficient because those file can be 200Mb in size.

how to write a program to retrieve a line in each file and displays my matches before moving on to the following lines in the files. This certainly will avoid loading all the files and make good use of the garbage collector?

profile images

sorry i dont know how to insert images here, it always fails, but while profiling, memory usage looks like stairs from the top to the bottom

ok it's works 1

thanks for reply

Answer

Thanks to FUZxxi his answer really help me, but there was a problem when files did not have the same number of line, to solve this problem, I've re-write his program this way

printLines :: [[String]] -> IO ()

printLines [] = return ()

printLines ss = do

    ss' <- printFirstLine ss

    if and $ map null ss' then putStrLn "finish" else printLines ss'





printFiles :: [FilePath] -> IO ()

printFiles paths = do

  files <- mapM readFile paths

  let fileLines = map lines files

  printLines fileLines



sliceFirstRow :: [[String]] -> ([String],[[String]])

sliceFirstRow list = unzip $ map getFirst list



printFirstLine :: [[String]] -> IO ([[String]])

printFirstLine ss = do

  let (fline,lline) = sliceFirstRow ss

  mapM_ putStrLn fline

  return lline    



getFirst :: [String] -> (String, [String])

getFirst [] = ("",[])

getFirst (x:xs) = (x,xs)

Thanks again

Answer 1

I would do something like this:

printFiles :: [FilePath] -> IO ()
printFiles paths= do
  files <- mapM readFile paths
  let fileLines = map lines files
  printLines fileLines
  where
  sliceFirstRow :: [[String]] -> ([String],[[String]])
  sliceFirstRow list = unzip $ map helper list where
    helper (x:xs) = (x,xs)
    helper []     = "" -- Your behaviour here
  printFirstLine :: [[String]] -> IO ([[String]])
  printFirstLine ss = do
    let (fline,lline) = sliceFirstRow ss
    mapM_ putStrLn fline
    return lline
  printLines :: [[String]] -> IO ()
  printLines [] = return ()
  printLines ss = do
    ss' <- printFirstLine ss
    printLines ss'

Although not tested, it should be good for memory.