F# - performing non-deterministic groupings on lists

Question

I’m working on a problem which I know I can solve with C#. I would like to prove to my boss that F# would be able solve in a more succinct way. However my understanding of functional programming is still fairly immature.

The Problem:

I’m working with a list of ‘Trade’ classes. The definition of the class is as follows:

type Trade(brokerId : string, productId : string, marketId : string, buySideId : string, tradeDate : string, ruleId : int) = class

    member this.BrokerId = brokerId
    member this.ProductId = productId
    member this.MarketId = marketId
    member this.BuySideId = buySideId
    member this.TradeDate = tradeDate
end

I need to be able to group the trades and then apply a rule to each of the resulting groups of data.

However I cannot guarantee the grouping of the data i.e. the rule for determining the grouping will potentially change every time the program is run - so for instance I may have to group by:

• TradeDate, BrokerId
• TradeDate only
• TradeDate, BrokerId, AccountId

... and so on.

Once I have the distinct groups it would be easy (I think) to apply a rule (such as ‘is the total TradeAmount greater than 10,000’).

Any help / pointers with creating a functional orientated solution to this problem would be very welcome.

Many thanks.

Answer 1

If I understand the problem correctly, then you essentially want to call the Seq.groupBy function. The problem is that you don't quite know the lambda function that you want to pass it as an argument when writing the code, because the function may vary depending on the choice of keys that should be used for grouping. Here is one relatively simple way to do this...

We'll create a dictionary of functions that gives us a function for reading the specified property of the Trade (this could be, in principle, constructed automatically, but it is probably easier to just write it):

let keyfunctions : IDictionary<string, Trade -> obj> = 
  dict [ "TradeDate", (fun t -> box t.TradeDate);  
         "BrokerId", (fun t -> box t.BrokerId);
         "MarketId", (fun t -> box t.MarketId); ]

Now, if we wanted to use multiple keys, we need a way to combine two functions that give us parts of the key into a single function. We can write a combinator that takes two functions and returns a single one that produces boxed tuple as the key:

let combine f1 f2 = (fun t -> box (f1 t, f2 t))

If you have a list of strings that specifies your keys, then you just need to pick function from the dictionary for each of the keys and combine them into a single function using combine:

let grouping = [ "TradeDate"; "MarketId" ]
let func = grouping |> Seq.map (fun n -> keyfunctions.[n]) |> Seq.reduce combine

And now you have a function that can be used as an argument to Seq.groupBy:

trades |> Seq.groupBy func

There are probably other ways to do this in F#, but I think this is a relatively simple approach that could convince your boss :-). As a side-note, you could write essentially the same thing in C# 3.0, although it would look a bit uglier due to more heavy syntax...

EDIT 1: A nice thing about this approach is that you don't need to use any reflection. Everything runs as compiled code, so it should be pretty efficient. The composed function just calls several other functions (.NET methods) and boxes the returned values...

EDIT 2: Regarding the order - this approach will work (when comparing tuples, the first elements are compared first), but I'm not entirely sure in which order the items are aggregated when using Seq.reduce, so maybe this example works the other way round...

Answer 2

How about something like this?

open System.Reflection

let getProp obj prop =
  obj.GetType().GetProperty(prop).GetValue(obj,null)

let groupByProps props =
  Seq.groupBy (fun obj -> List.map (getProp obj) props)

Then you can do trades |> groupByProps ["BrokerId"; "RuleId"], etc.

EDIT

For a slightly less concise but more performant solution, you could try this instead:

open System.Reflection
open System.Linq.Expressions

let propReader<'t> (prop:PropertyInfo) =
  let param = Expression.Parameter(typeof<'t>, "x")
  Expression.Lambda<System.Converter<'t,obj>>(Expression.Convert(Expression.Property(param, prop),typeof<obj>), [| param |]).Compile()
  |> Microsoft.FSharp.Core.FuncConvert.ToFSharpFunc

let propMap<'t>() =
  typeof<'t>.GetProperties()
  |> Seq.map (fun prop -> prop.Name, propReader<'t> prop)
  |> dict

let tradeMap = propMap<Trade>()

let groupByProps =
  fun props -> Seq.groupBy (fun obj -> List.map (fun prop -> tradeMap.[prop] obj) props)

This avoids using reflection each time the groupByProps function is called by creating functions ahead of time (like Tomas's solution), but uses reflection to create those functions so that you don't have to type in any boilerplate.