performance

Python: Replace values in list

Hello Stack Overflow Community, I have a list where I want to replace values with None where condition() returns True. [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] For example, if condition checks bool(item%2) should return: [None, 1, None, 3, None, 5, None, 7, None, 9, None] What is the most efficient way to do this? ...

ASP.NET Retrieving text via System.IO.File vs WebRequest

I have a text file that I need to retrieve to populate a web page. My first instinct was to use System.IO.File to open and read the file contents. then it occured to me I could also use a web request (since the file lives on the webserver). I'm wondering which one is a better choice. I figure using the file system would be faster, sinc...

Why are PostgreSQL Text-Search GiST indexes so much slower than GIN indexes?

I'm testing out the PostgreSQL Text-Search features, using the September data dump from StackOverflow as sample data. :-) The naive approach of using LIKE predicates or POSIX regular expression matching to search 1.2 million rows takes about 90-105 seconds (on my Macbook) to do a full table-scan searching for a keyword. SELECT * FROM P...

Hibernate implementation of simple many-to-many relationships - Dictionary or List?

I have 3 DB Tables: Person, Address, and PersonAddress. Person Address is a simple join table (only stores the IDs of Person and Address). In my domain model, I have Person and Address. Person is configured to have a many-to-many relationship to Address (through PersonAddress). In code, this is implemented with List<Address>. I've been...

MySQL: optimizing a JOIN query

Say I have two MyISAM tables: tab_big: id1, id2, id_a, ord (5 billion records) tab_small: id1, id2, id_b (1 billion records) CREATE TABLE IF NOT EXISTS `tab_big` ( `id_a` int(10) unsigned NOT NULL, `id1` int(10) unsigned NOT NULL, `id2` int(10) unsigned NOT NULL, `ord` int(10) unsigned NOT NULL DEFAULT '...

Can I atomically increment a 16 bit counter on x86/x86_64?

I want to save memory by converting an existing 32 bit counter to a 16 bit counter. This counter is atomically incremented/decremented. If I do this: What instructions do I use for atomic_inc(uint16_t x) on x86/x86_64? Is this reliable in multi-processor x86/x86_64 machines? Is there a performance penalty to pay on any of these a...

Server Benchmarking: What tools to use with my real-world test data

I want to benchmark a new server using historical HTTP-request data. I have a textfile that contains one day's worth of real historical requests to a production server. What is the best tool for sending that list of requests on the server I'm testing? The tool I use should be able to configure the following: Number of threads making th...

Win32Exception @ ServiceHost.Open() for WCF service...

I am working on writing BDD specifications for a broad set of WCF service infrastructure I am writing. I have noticed that each specification I write that involves a call to ServiceHost.Open(), that line takes a good 2 - 6 seconds to execute (the time keeps growing as I add more and more specs). I noticed that when this method is called,...

Will this code be evaluated on every iteration?

I have this for-each-loop: for (Component c : container.getComponents()) { // Loop code } Is getComponents called on every iteration? Does it make sense to call getComponents before the look and only work on a cached array? ...

ASP.NET MVC application running II6 with wildcard mapping using only 3 threads

During the performance testing of our ASP.NET MVC application I discovered interesting bottle neck. The application is using only 3 managed threads. I checked maximum thread pool size. It's 200 and we are having 197 available threads. I checked connection limit of web site and it's unlimited. I tried to run stress test locally agains...

Database: SQL Pagination - How many pages?

I'm using MySQL LIMIT to display 50 rows (pages) at a time, like so: SELECT cols FROM table LIMIT 49, 50 What's the most efficient way for me to determine how many "pages" I have in my result set (how many total records my query is producing)? Before I run the code above, should I just run a quick and dirty: SELECT COUNT(cols) FROM ...

Long running Jobs Performance Tips

I have been working with SQL server for a while and have used lot of performance techniques to fine tune many queries. Most of these queries were to be executed within few seconds or may be minutes. I am working with a job which loads around 100K of data and runs for around 10 hrs. What are the things I need to consider while writing ...

What are the methods for identifying unnecessary columns within a covering index?

What methods are there for indentifying superfluous columns in covering indices: columns which are never searched against, and therefore may be extracted into Include's, or even removed completely without affecting the applicability of the index? ...

MySQL performance using IN predicate

If I run the following queries each one returns quickly (0.01 sec) and gives me my desired result. SELECT tagId FROM tag WHERE name='programming' SELECT COUNT(DISTINCT workcode) FROM worktag WHERE tagId=123 OR tagId=124 (assume the two tagId numbers were the results from the first query) I would like to combine these queries so I onl...

silverlight math performance question

Is there some reason that identical math operations would take significantly longer in one Silverlight app than in another? For example, I have some code that takes a list of points and transforms them (scales and translates them) and populates another list of points. It's important that I keep the original points intact, hence the sec...

What performance overhead do IoC containers involve?

Hi, Loose coupling is wonderful of course, but I have often wondered what overhead wiring up dynamically using an IoC container (for example Castle Windsor) has over a tightly coupled system? I know that a detailed answer would depend on what the IoC was being used for, but I'm really just trying to get a feel for the magnitude of effo...

Putting performance optimization (with examples) on developer's resume?

Two (somewhat separate) questions I have are: Is it a good approach to put performance optimization as a separate skill-set backed up by line items (instead of just 2 words in "skills")? Do the items i put as examples below sound like useful/braggable data to distinguish oneself to a hiring technical manager (whether I put them on a r...

Creating the most efficient fullscreen color-shifting gradient

For a project I'm working on I'm trying to create a Flash file with a constantly color-shifting gradient that fills up the entire browser window. For a better explanation please see the following link: GradientTest Example Don't worry about the moving balls, those are just there to challenge the frameRate of the movie, which has a maxi...

PHP MVC - How to design?

I'm relatively well versed in designing websites. I mostly go for LAMP where I already have a small "framework" of my own, that I use. In short, it separates the logic from the layout and I have basically one logic file to one or many layout files depending on what views are supported in the layout. There's an admin section and there are...

Profiling PHP app, 50% of time busy constructing classes -- is this OK?

I have a web application written in PHP. It uses MySQL for data storage. Today I decided to profile it to find bottlenecks in code and find which parts of it are running slower that the others. The usual stuff. I did a lot of work and now my page loads in less than 0.05 seconds on my desktop. But now my profiler tells me that half of th...