Writing efficient .Net/SQL Server code

Question

Let me preface by saying I am not knocking .Net (it helps me earn my living)...

I recently heard of a site were the developer/owner bragged that he had minimal hardware to support his site (plentyoffish.com). This site is listed as being in the top 50 in terms of traffic on Alexa. I am struggling to wrap my head around how something like that can be acomplished by doing what the developer/owner claims, a single developer writing a site (it took him 2 weeks to get the intial site up) using minimal hardware that gets 2 million hits per hour (peak). Larger companies have huge web farms to handle traffic like that...

How do you write code so efficient, with limited hardware resources (he claims he uses only three DB servers)? Has anyone worked on or developed a site that efficient?

Answer 1

With web sites, some pretty amazing things can be done by caching. The language being used matters much less if you can cache things efficiently.

Answer 2

As jle said, caching is key. Also, it may simply be the nature of the user interaction on the web site. I may be wrong but I think the example, plentyoffish.com, is not a particularly resource/user-intensive site. There is plenty of data in the user profiles, presumably, but (in my limited experience checking it out) the purpose of the site isn't to manipulate or correlate or transform data, so much as to simply return profiles matching the user's criteria -- but not really to massage that data in any way.

Answer 3

Compared to the alternatives, .Net is actually quite efficient. What else is there? PHP? Java? Ruby? Sure you could code up your web application in C++, but that's going to be a major pain to implement and maintain. Also, as others have mentioned, with proper caching, database tuning, and algorithms, you can get very good performance. Way more performance than just picking a fast langauge and writing code that isn't tuned.

Answer 4

This is a tough one to answer with specifics, but not all choices one makes in software development are created equal, even using the same language. The more layer upon layer of frameworks you slap ontop of something, the farther removed you get from "what really matters", and if what "really matters" is performance, the person that can program at that "lower level" can do things many, many times faster than people that develop programs by piecing together controls. Sometimes ease of development is more important than raw speed, so its not always the right choice to right at a lower level - ease of maintaining the code is also important and sometimes outweighs raw performance.

I once went head to head with another consultant trying to win a high 6 figure contract for a major financial services firm, we were both selling the idea of a BI/EIS system. He used bloated technology and every step of the way, but it was all "new and shiny" technology that he was very proud of and contained all the right buzzwords.

I wrote mine in asp (asp classic, .net was not out yet) and use custom written, native SQL Server communication methods, with all hand coded routines to generate reports and other similar performance enhancing decisions. He used a report writer plugin and plug-and-play controls to build his system. I did my entire demo over a 28.8 dialup line against my development webserver in my home office 200 miles away - his was done against a much more powerful development server in the next room connected via 100Mbit ethernet and it was still many, many times slower.

When the demo was over I told my clients that my entire demo was done over a dialup line (they assumed it was local). That sealed the deal since most of the users of the system were scattered throughout the country and broadband was uncommon at the time.

Answer 5

It's more than just the code: doing something like this requires a cooperative effort from developers, DBAs and IT. Each member of the team has to recognize the impact that their work has on the other areas of the team.

When a developer writes a query that does a table scan on a heavy OLTP system, he has to understand why it won't scale. When the DBA sees a table scan, he has to understand how to fix it at the database level or at the code level. When the sysadmin sees heavy read activity on the drives, he has to understand the underlying causes of it (things like table scans) in order to decide whether to fix it with better code or throwing more drives at it.

Well-performing systems are the result of each person doing a great job, AND communicating really well with the other team members. If just one member of the team is out to throw everybody else under the bus, the system won't perform. And it sounds like you're already doing it by throwing .NET and SQL Server under the bus. ;-)

Answer 6

The Microsoft stack is remarkably fast if you avoid all the drag/drop garbage that gives ASP.NET a bad name.

• IIS is a good web server.
• SQL Server is a good database.
• C# compiles to fast code. (qualifications probably needed here :)
• ASP.NET can be quite lightweight if you use it correctly

That's the whole stack. It's all good. You can say what you like about Microsoft, its business practices and its consumer products, but their development tools and servers are top notch.

My personal experience with their web stack includes a site that sees ~500 pageviews/second and seldom spikes the server above 15%. That's fast enough for most operations.