Convert one file into ten by splitting the data

Question

I have a file that has > 10 rows. I want to create 9 files that each contains equal number or rows, with the 10th file containing that equal number plus any left over.

I currently get the data from the main file by the following:

Dim sData As String

Using sr As New StreamReader(vsFilePath)
    sData = sr.ReadToEnd
End Using

Dim oDataList As New List(Of String)
oDataList.AddRange(sData.Split(Convert.ToChar(ControlChars.Lf)))

How can I take oDataList and parse it into 10 files without having to loop through each file and oDataList to write line by line?

Is there a better way them what I am currently doing?

Answer 1

If the order of the lines is not important, you can do the following to separate the content in 10 equal portions, otherwise, a double loop through the data (one for counting and storing the lines, the other for writing them) is probably necessary.

Dim sData As String
Dim counter As Integer
Dim sw(10) As StreamWriter
For i As Integer = 0 To 9 Step 1
    sw(i) = New StreamWriter("path" + something)
Next

Using sr As New StreamReader("path")
    sData = sr.ReadLine()
    While sData IsNot Nothing
        sw(1 Mod counter).WriteLine(sData)
        counter += 1
        sData = sr.ReadLine
    End While
End Using

'add a finally block for closing the 10 streams

Answer 2

• Open your 10 output files.
• Iterate through every line in the input file doing the following:
  • Split the line up its row elements, then slice them appropriately
  • For every slice, write it to the appropriate file

Violà. You only need to iterate through oDataList once.

Answer 3

If the number of lines in the original file is a relatively small number, you can read them all into an array in a line of code. From there, producing the 10 output files is a simple operation. Here is one such method.

Dim path As String = "C:\Temp\test\input.txt"
Dim outputPath As String = "C:\Temp\test\output{0}.txt"

Dim lines As String() = File.ReadAllLines(path)

Dim files As Integer = 10
Dim linesPerFile As Integer = lines.Length \ 10
Dim currentLine As Integer = 0

For i As Integer = 0 To files - 1
   Dim iterations As Integer = linesPerFile
   If i = files - 1 Then
       iterations = lines.Length - currentLine
   End If

   Using writer As New StreamWriter(String.Format(outputPath, i))
       For j As Integer = 0 To iterations - 1
            writer.WriteLine(lines(currentLine))
            currentLine += 1
       Next
   End Using
Next

...

string path = @"C:\Temp\test\input.txt";
string outputPath = @"C:\Temp\test\output{0}.txt";

string[] lines = File.ReadAllLines(path);

int files = 10;
int linesPerFile = lines.Length / 10;
int currentLine = 0;

for (int i = 0; i < files; ++i)
{
    int iterations = linesPerFile;
    if (i == files - 1)
    {
        iterations = lines.Length - currentLine;
    }

    using (StreamWriter writer = new StreamWriter(string.Format(outputPath, i)))
    {
        for (int j = 0; j < iterations; j++)
        {
            writer.WriteLine(lines[currentLine++]);
        }
    }
}