Separating data from a constantly appended file into a new file

Question

I am using a macro to export a table in a Microsoft Access database to a csv file in order to import into a mysql database. I ended up using a batchfile that would place a marker in the text file before the exporting took place, and then place everything after the last marker into a new file. This works fine, except for the fact that access does not append, but will recreate the file each time, so it is impossible to use any kind of marker.

Is there any way, using access or batch files or whatever, to either a) force access to append to a file, or to place a marker of its own, or b) export to a different file each time, possibly the filename being a variable such as the date, or c) overcome this behavior with outside manipulation

Answer 1

a) force access to append to a file, or to place a marker of its own

No. The exports is expecting to write a completely new file each time, and has problems if the file already exists.

b) export to a different file each time, possibly the filename being a variable such as the date

Yes, build the path/filename up as a string, and append the date/time to the end

c) overcome this behavior with outside manipulation

Well, you could dump out what's new to a dummy file, and then with a batch script or system call that does:

echo marker >> goodfile.csv
type dummy >> goodfile.csv
del dummy

However, if you just want to add the new records, a better way over all would be to just process the dummy file instead trying to seek the last marker and process everything below that.

Answer 2

You could also use VBScript to execute the query you use to export, and append those records to an existing file.

Answer 3

Instead of using a macro to export the table you could simply create some code to open the file, and append the data to it.

How to use

Simply copy the code to a VBA module in your application and call it like this:

' Export the Table "Orders" to "orders.csv", appending the data to the       '
' existing file if there is one.                                             '
ExportQueryToCSV "Orders", "C:\orders.csv", AppendToFile:=True

' Export the result of the query to "stock.csv" using tabs as delimiters     '
' and no header or quotes around strings                                     '
ExportQueryToCSV "SELECT * FROM Stock WHERE PartID=2", _
                 "C:\stock.csv", _
                 AppendToFile:=False, _
                 IncludeHeader:=False, _
                 Delimiter:=chr(9), _
                 QuoteString:=false

Code

'----------------------------------------------------------------------------'
' Export the given query to the given CSV file.                              '
'                                                                            '
' Options are:                                                               '
' - AppendToFile : to append the record to the file if it exists instead of  ' 
'                  overwriting it (default is false)                         '
' - Delimiter    : what separator to use (default is the coma)               '
' - QuoteString  : Whether string and memo fields should be quoted           '
'                  (default yes)                                             '
' - IncludeHeader: Whether a header with the field names should be the first '
'                  line (default no)                                         '
' Some limitations and improvements:                                         '
' - Memo containing line returns will break the CSV                          '
' - better formatting for numbers, dates, etc                                '
'----------------------------------------------------------------------------'
Public Sub ExportQueryToCSV(Query As String, _
                            FilePath As String, _
                            Optional AppendToFile As Boolean = False, _
                            Optional Delimiter As String = ",", _
                            Optional QuoteStrings As Boolean = True, _
                            Optional IncludeHeader As Boolean = True)
    Dim db As DAO.Database
    Dim rs As DAO.RecordSet

    Set db = CurrentDb
    Set rs = db.OpenRecordset(Query, dbOpenSnapshot)
    If Not (rs Is Nothing) Then
        Dim intFile As Integer

        ' Open the file, either as a new file or in append mode as required '
        intFile = FreeFile()
        If AppendToFile And (Len(Dir(FilePath, vbNormal)) > 0) Then
            Open FilePath For Append As #intFile
        Else
            Open FilePath For Output As #intFile
        End If

        With rs
            Dim fieldbound As Long, i As Long
            Dim record As String
            Dim field As DAO.field

            fieldbound = .Fields.count - 1

            ' Print the header if required '
            If IncludeHeader Then
                Dim header As String
                For i = 0 To fieldbound
                    header = header & .Fields(i).Name
                    If i < fieldbound Then
                        header = header & Delimiter
                    End If
                Next i
                Print #intFile, header
            End If

            ' print each record'
            Do While Not .EOF
                record = ""
                For i = 0 To fieldbound
                    Set field = .Fields(i)
                    If ((field.Type = dbText) Or (field.Type = dbMemo)) And QuoteStrings Then
                        record = record & """" & Nz(.Fields(i).value, "") & """"
                    Else
                        record = record & Nz(.Fields(i).value)
                    End If
                    If i < fieldbound Then
                        record = record & Delimiter
                    End If
                    Set field = Nothing
                Next i
                Print #intFile, record
                .MoveNext
            Loop
            .Close
        End With
        Set rs = Nothing
        Close #intFile
    End If
    Set rs = Nothing
    Set db = Nothing
End Sub

Note that it's not perfect and you may have to adapt the code to reflect how you want the data to be formatted, but the defaults should be fine in most cases.