optimization

Why is MySQL with InnoDB doing a table scan when key exists and choosing to examine 70 times more rows?

Hello, I'm troubleshooting a query performance problem. Here's an expected query plan from explain: mysql> explain select * from table1 where tdcol between '2010-04-13 00:00' and '2010-04-14 03:16'; +----+-------------+--------------------+-------+---------------+--------------+---------+------+---------+-------------+ | id | select_ty...

How to implement Excel Solver functionality in C#?

Hi, I have an application in C#, I need to do some optimization calculations, like Excel Solver Add-in does, one option is certainly to write my own solver implementation, but I'm kind of short of time, so I'm looking into libraries that already exist that can help me with this. I've been trying the Microsoft Solver Foundation, which s...

How do you control what your C compiler Optimizes?

I am writing the firmware for an embedded device in C using the Silicon Labs IDE and the SDCC compiler. The device architecture is based on the 8051 family. The function in question is shown below. The function is used to set the ports on my MCU to drive a stepper motor. It gets called in by an interrupt handler. The big switch statemen...

Optimizing division/exponential calculation

I've inherited a Visual Studio/VB.Net numerical simulation project that has a likely inefficient calculation. Profiling indicates that the function is called a lot (1 million times plus) and spends about 50% of the overall calculation within this function. Here is the problematic portion Result = (A * (E ^ C)) / (D ^ C * B) (where A-C...

PHP Increasing write to page speed.

I'm currently writing out xml and have done the following: header ("content-type: text/xml"); header ("content-length: ".strlen($xml)); $xml being the xml to be written out. I'm near about 1.8 megs of text (which I found via firebug), it seems as the writing is taking more time than the script to run.. is there a way to increase this...

c#, can you think of anymore optimization in this code?

Hi All: Look at the following code: StringBuilder row = TemplateManager.GetRow("xyz"); // no control over this method StringBuilder rows = new StringBuilder(); foreach(Record r in records) { StringBuilder thisRow = new StringBuilder(row.ToString()); thisRow.Replace("%firstName%", r.FirstName) .Replace("%lastName%", r....

How to get REALLY fast python over a simple loop

Edit2: I finally got it to pass at 10.54 seconds. I used nearly all of your answers to get there and thus it was hard to choose one as 'correct' but I believe the one I chose sums it up the best. Thanks to you all. Final passing code is below. Edit: I included some of the suggested updates in the included code. I'm working on a spo...

"Anagram solver" based on statistics rather than a dictionary/table?

My problem is conceptually similar to solving anagrams, except I can't just use a dictionary lookup. I am trying to find plausible words rather than real words. I have created an N-gram model (for now, N=2) based on the letters in a bunch of text. Now, given a random sequence of letters, I would like to permute them into the most likely...

Calculating Growth-Rates by applying log-differences

I am trying to transform my data.frame by calculating the log-differences of each column and controlling for the rows id. So basically I like to calculate the growth rates for each id's variable. So here is a random df with an id column, a time period colum p and three variable columns: df <- data.frame (id = c("a","a","a","c","c","d","...

Finding perfect numbers (optimization)

I coded up a program in C# to find perfect numbers within a certain range as part of a programming challenge . However, I realized it is very slow when calculating perfect numbers upwards of 10000. Are there any methods of optimization that exist for finding perfect numbers? My code is as follows: using System; using System.Collections....

Profiling short-lived Java applications

Is there any Java profiler that allows profiling short-lived applications? The profilers I found so far seem to work with applications that keep running until user termination. However, I want to profile applications that work like command-line utilities, it runs and exits immediately. Tools like visualvm or NetBeans Profiler do not even...

How to optimize an database suggestion engine

Hi, I`m making an online engine for item-to-item recommending movies. I have made some researches and I think that the best way to implement that is using pearson correlation and make a table with item1, item2 and correlation fields, but the problem is that after each rate of item I have to regenerate the correlation for in the worst cas...

Linux PDF/Postscript Optimizing

So I have a report system built using Java and iText. PDF templates are created using Scribus. The Java code merges the data into the document using iText. The files are then copied over to a NFS share, and a BASH script prints them. I use acroread to convert them to PS, then lpr the PS. The FOSS application pdftops is horribly ineff...

Optimize a MySQL count each duplicate Query

I have the following query That gets the city name, city id, the region name, and a count of duplicate names for that record: SELECT Country_CA.City AS currentCity, Country_CA.CityID, globe_region.region_name, ( SELECT count(Country_CA.City) FROM Country_CA WHERE City LIKE currentCity ) as counter FROM Country_CA LEFT JOIN glo...

Combine query results from one table with the defaults from another

This is a dumbed down version of the real table data, so may look bit silly. Table 1 (users): id INT username TEXT favourite_food TEXT food_pref_id INT Table 2 (food_preferences): id INT food_type TEXT The logic is as follows: Let's say I have this in my food preference table: 1, 'VEGETARIAN' and this in the users table: 1, '...

How to log slow queries in shared hosting MySQL?

I have a shared hosting where I have my website and MySQL database. I've installed a open source script for statistics (phpMyVisites) and it started to work very slow lately. It's written using some kind of framework and has many PHP files. I know that to find slow queries I can use slow query log functionality in MySQL. But on this shar...

Should I use integer primary IDs?

For example, I always generate an auto-increment field for the users table, but I also specify a UNIQUE index on their usernames. There are situations that I first need to get the userId for a given username and then execute the desired query, or use a JOIN in the desired query. It's 2 trips to the database or a JOIN vs. a varchar inde...

PHP Performance Metrics

I am currently developing a PHP MVC Framework for a personal project. While I am developing the framework I am interested to see any notable performance by implementing different techniques for optimization. I have implemented a crude BenchMark class that logs mircotime. The problem is I have no frame of reference for execution times....

How to increase my Web Application's Performance??

I have a ASP.NET web application (.NET 2008) using MS SQL server 2005, I want to increase the performance of the web site, If anyone have an article contains steps to do that, step by step , In SQL(Indexes, ..... etc.) and in the code. Thanks in advance Best Regards ...

Most optimized way to calculate modulus in C

I have minimize cost of calculating modulus in C. say I have a number x and n is the number which will divide x when n == 65536 (which happens to be 2^16): mod = x % n (11 assembly instructions as produced by GCC) or mod = x & 0xffff which is equal to mod = x & 65535 (4 assembly instructions) so, GCC doesn't optimize it to this ...