How can I keep track of how many calls I make to other 3rd party API's using .NET?

Question

Hi folks,

In a number of our web applications, we make calls to 3rd part API's like Twitter, etc. Nothing new there.

What we would like to do is to grab some statistics about our website usage -> how many calls we are making every hour and day to our various free and paid 3rd party API's.

I'm not sure how we can do this.

Now, I've used Performance Counters before in the past with great success, but they are very limiting (unless I am not using them correctly). Firstly, they increment/decrement int values. So if we need to see how much traffic is going via a particular API for a time period, we need to have the PerfMon running ... which is not possible (ie. our managers don't/shouldn't/will never get access to the live web server(s).

So I'm not sure what other people do in the scenario?

Should I have an application state variable that records the time a request was made? then u can possible use linq or something to SUM/COUNT by GROUP ? What about a web farm, now? (Application state is per machine). Add AppFabric then?

The next typical response is to use a database. Well, that's fine ... until you have a lot of external api requests .. which means there's a lot of overhead traffic from server to db .. which can slow/bottleneck things.

After this, i thought maybe it's a case of mixing options. Using the application cache and then every 5 mins, doing a dump to the db if the number has changed. This might alivate some load issues, but it still adds way-way-extra complexity into the system.

So then I go back to the Performance Counters idea .. which I like (because it's been coded into the core OS to handle high traffic without impacting (or impacting at the least possible amount) performance .... but now i'm back at step 1 - how can i use perf counters to make time based results.

maybe use application state again and every hour/whenever fixed time dump the result into the perf counter (instead of a db) ?

Anyone have any thoughts, please :)

Answer 1

You can create your own performance counters, install them to the host server, and then hit them with every call you make. Then you get a per host count of your usage. You could then write a script (PowerShell) that would allow you to collect the counter stats from the various servers in the farm. Or you could use another Windows Service doing the same thing as the powershell script and dumping the statistics into a database for reporting purposes.

This is hypothetical. I've created and use performance counters before, but only for a single web service machine. The process doing this would need administrative privileges.

MSDN Performance Counters

Answer 2

Do your web servers send outgoing requests through a proxy of any sort? You might be able to use the logs of that to get what you want.