benchmarking

Benchmark: VMware vs Virtualbox

I tested VirtualBox 2.1.4 and VMware Workstation 6.5.1 for later use as a development VM Host: Ubuntu64, 4GB RAM, Core2Duo E6600, Samsung HD502IJ Guest: Windows XP Home, 1GB RAM, 8GB virtual disk Benchmark Software: SiSoft Sandra Light, HD Tune What is your experience or suggestion for a VM mostly used for development tools (not as...

How to prove that code isn't broken, but the hardware is?

I'm sure it repeats everywhere. You can 'feel' network is slow, or machine or slow or something. But the server/chassis logs are not showing anything, so IT doesn't believe you. What do you do? Your regressions are taking twice the time ... but that's not enough Okay you transfer 100 GB using dd etc, but ... that's not enough. Okay yo...

How do I accurately time how long it takes to call a function on the iPhone?

I would like to know how long it's taking to send a message to an object with at least a 1ms accuracy. How do I do this? ...

When benchmarking, what causes a lag between CPU time and "elapsed real time"?

I'm using a built-in benchmarking module for some quick and dirty tests. It gives me: CPU time system CPU time (actually I never get any result for this with the code I'm running) the sum of the user and system CPU times (always the same as the CPU time in my case) the elapsed real time I didn't even know I needed all that informa...

When benchmarking what would make the elapsed CPU time LESS than the user CPU time

Following up on a question I posed earlier: I ended up with a User CPU time and Total CPU time that was about 4% longer in duration than the elapsed real time. Based on the accepted answer to my earlier question, I don't understand how this could be the case. Could anyone explain this? ...

How to do "performance-based" (benchmark) unit testing in Python

Let's say that I've got my code base to as high a degree of unit test coverage as makes sense. (Beyond a certain point, increasing coverage doesn't have a good ROI.) Next I want to test performance. To benchmark code to make sure that new commits aren't slowing things down needlessly. I was very intrigued by Safari's zero tolerance poli...

Illogical benchmarking?

Hello, I witnessed the following weird behavior. I have two functions, which do almost the same - they measure the number of cycles it takes to do a certain operation. In one function, inside the loop I increment a variable; in the other nothing happens. The variables are volatile so they won't be optimized away. These are the functions...

benchmarking PHP vs Pylons

I want to benchmark PHP vs Pylons. I want my comparison of both to be as even as possible, so here is what I came up with: PHP 5.1.6 with APC, using a smarty template connecting to a MySQL database Python 2.6.1, using Pylons with a mako template connecting the the same MySQL database Is there anything that I should change in that set...

Understanding Ruby on Rails render times

I am working on an "optimization" on my application and I am trying to understand the output that rails (version 2.2.2) gives at the end of the render. Here is the "old" way: Rendered user/_old_log (25.7ms) Completed in 466ms (View: 195, DB: 8) | 200 OK And the "new" way: Rendered user/_new_log (48.6ms) Completed in 337ms (View: 192...

Server benchmarking: how many http requests can the server process?

I am creating an application which sends the measurement data every 200 ms from mobile phone to the server. The data have to be processed in real-time, but I noticed that the difference between the sent time and the time of process start is getting bigger and bigger so I have to find the point where the the requests get stuck. I am send...

Trivial mathematical problems as language benchmarks

Why do people insist on using trivial mathematical problems like finding numbers in the Fibonacci sequence for language benchmarks? Don't these usually get optimized to relativistic speeds? Isn't the brunt of the bottlenecks usually in I/O, system API calls, operations on strings and structures, processing large quantities of data, abstr...

How to find the cause of RESTful service bad performance?

Hi, I am creating a service which receives some data from mobile phones and saves it to the database. The phone is sending the data every 250 ms. As I noticed that the delay for data storing is increasing I tried to run WireShark and write a log as well. I noticed that the web requests from mobile phone are being made without the dela...

Why is List<T>.ForEach faster than standard foreach?

Consider this: Requisite: //The alphabet from a-z List<char> letterRange = Enumerable.Range('a', 'z' - 'a' + 1) .Select(i => (Char)i).ToList(); //97 - 122 + 1 = 26 letters/iterations Standard foreach: foreach (var range in letterRange) { Console.Write(range + ","); } Console.Write("\n"); Inbuilt foreach: letterRange.ForEach(r...

Rails App/webserver benchmarking

Does anyone know the best lightweight Rails benchmarking tool? I need to get performance statistics of the webserver and simulate authentication + page navigation per session. I've been trying to use httperf, but been encountering TamperingWithCookie exception in the application. Ideally, I'd like to separate the application and databa...

TPC or other DB benchmarks for SSD drives

I have been interested in SSD drives for quite sometime. I do a lot of work with databases, and I've been quite interested to find benchmarks such as TPC-H performed with and without SSD drives. On the outside it sounds like there would be one, but unfortunately I have not been able to find one. The closest I've found to an answer was t...

SQLite Performance Benchmark -- why is :memory: so slow...only 1.5X as fast as disk?

Why is :memory: in sqlite so slow? I've been trying to see if there are any performance improvements gained by using in-memory sqlite vs. disk based sqlite. Basically I'd like to trade startup time and memory to get extremely rapid queries which do not hit disk during the course of the application. However, the following benchmark giv...

Java: Why do two consecutive calls to the same method yield different times for execution ...

Here is a sample code: public class TestIO{ public static void main(String[] str){ TestIO t = new TestIO(); t.fOne(); t.fTwo(); t.fOne(); t.fTwo(); } public void fOne(){ long t1, t2; t1 = System.nanoTime(); int i = 10; int j = 10; int k = j*i; System.out.println(k); ...

Is this scenario a valid stress test?

I have a server application that handles clients requests in different manner. I want to know how many users can be served with minimal latency, so I made a small stress test application that simulate the users requests; at the same time another application monitor the memory/CPU utilization. The stress test tool creates thread every s...

How do I measure response time in seconds given the following benchmarking data?

We recently got some data back on a benchmarking test from a software vendor, and I think I'm missing something obvious. If there were 17 transactions (I assume they mean successfully completed requests) per second, and 1500 of these requests could be served in 5 minutes, then how do I get the response time for a single user? Is this so...

Performance Evaluation of Linux Scheduler

Hi I have done some simple changes to the scheduler in the Linux Kernel. Now, I would like to see how those changes affect the response time of the system; in other words, I would like to know how long a context switch takes with my modifications compared to the original scheduler. A straightforward approach would be to use the time st...