Useful F# Scripts

Question

I have been investigating the use of F# for development and have found (for my situations) building scripts to help me simplify some complex tasks is where I can get value from it (at the moment).

My most common complex task is concatenating files for many tasks (mostly SQL related).

I do this often and every time I try to improve on my F# script to do this.

This is my best effort so far:

open System.IO

let path = "C:\\FSharp\\"
let pattern = "*.txt"
let out_path = path + "concat.out"

File.Delete(out_path)
Directory.GetFiles(path, pattern)
 |> Array.collect (fun file -> File.ReadAllLines(file))
 |> (fun content -> File.WriteAllLines(out_path, content) )

I'm sure others have scripts which makes their sometimes complex/boring tasks easier. What F# scripts have you used to do this or what other purposes for F# scripts have you found useful?

I found the best way for me to improve my F# was to browse other scripts to get ideas on how to tackle specific situations. Hopefully this question will help me and others in the future. :)

I have found an article on generating F# scripts that may be of interest: http://blogs.msdn.com/chrsmith/archive/2008/09/12/scripting-in-f.aspx

Answer 1

I use F# in a similar way when I need to quickly pre-process some data or convert data between various formats. F# has a great advantage that you can create higher-order functions for doing all sorts of similar tasks.

For example, I needed to load some data from SQL database and generate Matlab script files that load the data. I needed to do this for a couple of different SQL queries, so I wrote these two functions:

// Runs the specified query 'str' and reads results using 'f'
let query str f = seq { 
  let conn = new SqlConnection("<conn.str>");
  let cmd = new SqlCommand(str, conn)
  conn.Open()
  use rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)
  while rdr.Read() do yield f(rdr) } 

// Simple function to save all data to the specified file
let save file data = 
  File.WriteAllLines(@"C:\...\" + file, data |> Array.ofSeq)

Now I could easily write specific calls to read the data I need, convert them to F# data types, does some pre-processing (if needed) and print the outputs to a file. For example for processing companies I had something like:

let comps = 
  query "SELECT [ID], [Name] FROM [Companies] ORDER BY [ID]" 
        (fun rdr ->  rdr.GetString(1) )
let cdata = 
  seq { yield "function res = companies()"
        yield "  res = {"
        for name in comps do yield sprintf "   %s" name
        yield "  };"
        yield "end" }
save "companies.m" cdata

Generating output as a sequence of strings is also pretty neat, though you could probably write a more efficient computation builder using StringBuilder.

Another example of using F# in the interactive way is described in my functional programming book in Chapter 13 (you can get the source code here). It connects to the World Bank database (which contains a lots of information about various countries), extracts some data, explores the structure of the data, convert them to F# data types and calculates some results (and visualizes them). I think this is (one of many) kinds of tasks that can be very nicely done in F#.

Answer 2

Sometimes if I want a brief of an XML structure (or have a recursive list to use in other forms such as searches), I can print out a tabbed list of nodes in the XML using the following script:

open System
open System.Xml

let path = "C:\\XML\\"
let xml_file = path + "Test.xml"

let print_element level (node:XmlNode) = [ for tabs in 1..level -> "   " ] @ [node.Name]
                                            |> (String.concat "" >> printfn "%s")

let rec print_tree level (element:XmlNode) = 
        element 
        |> print_element level
        |> (fun _ -> [ for child in element.ChildNodes -> print_tree (level+1) child ])
        |> ignore

new XmlDocument()
 |> (fun doc -> doc.Load(xml_file); doc)
 |> (fun doc -> print_tree 0 doc.DocumentElement)

I am sure it can be optimised/cut down and would encourage by others' improvements on this code. :)

Answer 3

(For an alternative snippet see the answer below.) This snippet transforms an XML using an XSLT. I wasn't sure of hte best way to use the XslCompiledTransform and XmlDocument objects the best in F#, but it seemed to work. I am sure there are better ways and would be happy to hear about them.

(* Transforms an XML document given an XSLT. *)

open System.IO
open System.Text
open System.Xml
open System.Xml.Xsl

let path = "C:\\XSL\\"

let file_xml = path + "test.xml"
let file_xsl = path + "xml-to-xhtml.xsl"

(* Compile XSL file to allow transforms *)
let compile_xsl (xsl_file:string) = new XslCompiledTransform() |> (fun compiled -> compiled.Load(xsl_file); compiled)
let load_xml (xml_file:string) = new XmlDocument() |> (fun doc -> doc.Load(xml_file); doc)

(* Transform an Xml document given an XSL (compiled *)
let transform (xsl_file:string) (xml_file:string) = 
      new MemoryStream()
        |> (fun mem -> (compile_xsl xsl_file).Transform((load_xml xml_file), new XmlTextWriter(mem, Encoding.UTF8)); mem)
        |> (fun mem -> mem.Position <- (int64)0; mem.ToArray())

(* Return an Xml fo document that has been transformed *)
transform file_xsl file_xml
    |> (fun bytes -> File.WriteAllBytes(path + "out.html", bytes))

Answer 4

After clarifying approaches to writing F# code with existing .net classes, the following useful code came up for transforming xml documents given xsl documents. The function also allows you to create a custom function to transform xml documents with a specific xsl document (see example):

let transform = 
    (fun xsl ->
        let xsl_doc = new XslCompiledTransform()
        xsl_doc.Load(string xsl)

        (fun xml -> 
            let doc = new XmlDocument() 
            doc.Load(string xml)
            let mem = new MemoryStream()
            xsl_doc.Transform(doc.CreateNavigator(), null, mem)
            mem
        )
    )

This allows you to transform docs this way:

let result = transform "report.xml" "report.xsl"

or you can create another function which can be used multiple times:

let transform_report "report.xsl"

let reports = [| "report1.xml"; "report2.xml" |]
let results = [ for report in reports do transform_report report ]