optimization

Optimization! - What is it? How is it done?

Its common to hear about "highly optimized code" or some developer needing to optimize theirs and whatnot. However, as a self-taught, new programmer I've never really understood what exactly do people mean when talking about such things. Care to explain the general idea of it? Also, recommend some reading materials and really whatever y...

Help optimize my RPN evaluation function

Guys, My parser evaluates PEMDAS expressions by first converting from infix to postfix then uses the standard postfix evaluation rules. I parse the expression and store the tokens in a list. This precompilation is ok for me since I plan on caching the precompiled functions. I am trying to optimize the evaluate function (See Evaluate0...

How can I speed up Perl's processing of fixed-width data?

We've got a mature body of code that loads data from files into a database. There are several file formats; they are all fixed-width fields. Part of the code uses the Perl unpack() function to read fields from the input data into package variables. Business logic is then able to refer to these fields in a 'human-readable' way. The file...

best time to SELECT value in Oracle nested query

From an optimization standpoint, I want to know when it's best to SELECT some arbitrary data as columns. Specifically, I have this query: Where do I put this part? SELECT 'String 1' AS segment FROM DUAL UNION ALL SELECT 'String 2' AS segment FROM DUAL UNION ALL SELECT 'String 3' AS segment FROM DUAL In Outermost Query I need access...

How should I optimize this filesystem I/O bound program?

I have a python program that does something like this: Read a row from a csv file. Do some transformations on it. Break it up into the actual rows as they would be written to the database. Write those rows to individual csv files. Go back to step 1 unless the file has been totally read. Run SQL*Loader and load those files into the data...

Dictionary/Client VS Application Variables

Hi i got a question about my server performance ... i got a classic asp cms hosting ~250 websites, for each website we build a Classic ASP dictionary using set dict = CreateObject("Scripting.Dictionary") dict.add "test1","Value Test1" dict.add "test2","Value Test2" dict.add "test3","Value Test3" that dictionary is then loaded on eve...

Optimizing COUNT(*) query with 4 MM relations and larger table

I am struggling (again) with the table from this question: http://stackoverflow.com/questions/1555561/how-to-optimize-this-query-4-mm-tables-involved It is one main table product_table which has four MM relations via the lookup tables mm1 to mm4. The lookup tables have the fields uid_local containing the uid of the product_table and ...

Profiling DLL/LIB Bloat

I've inherited a fairly large C++ project in VS2005 which compiles to a DLL of about 5MB. I'd like to cut down the size of the library so it loads faster over the network for clients who use it from a slow network share. I know how to do this by analyzing the code, includes, and project settings, but I'm wondering if there are any tools...

UNION ALL versus CONNECT BY LEVEL for generating rows

I was wondering which is a better/faster/more efficient way of turning arbitrary strings into columns: UNION ALL SELECT my_field, CASE WHEN my_field = 'str1' THEN ... ... END, ... FROM ( SELECT 'str1' AS my_field FROM DUAL UNION ALL SELECT 'str2' AS my_field FROM DUAL ...

Confusion with the problems of inline function

In the problems of inline functions in wikipedia : http://en.wikipedia.org/wiki/Inline%5Fexpansion#Problems it says : "# A language specification may allow a program to make additional assumptions about arguments to procedures that it can no longer make after the procedure is inlined." Could somebody elaborate this point? How do you p...

Optimizing A* Pathfinding iPhone - Will NSDictionary do the trick?

I've got a pretty big A* pathfinding function that gets called frequently and has to be put in another thread because otherwise it will make my game stutter. I come from a Java background, and recently read a discussion about the speed of HashMap's (essentially the equivalent of NSDictionary) and the different implementations you can use...

Fastest way to obtain the largest X numbers from a very large unsorted list?

Hello everyone, I'm trying to obtain the top say, 100 scores from a list of scores being generated by my program. Unfortuatly the list is huge (on the order of millions to billions) so sorting is a time intensive portion of the program. Whats the best way of doing the sorting to get the top 100 scores? The only two methods i can th...

How do I optimize a database for superstring queries?

So I have a database table in MySQL that has a column containing a string. Given a target string, I want to find all the rows that have a substring contained in the target, ie all the rows for which the target string is a superstring for the column. At the moment I'm using a query along the lines of: SELECT * FROM table WHERE 'my supe...

Best books to optimize C++ code

What are the best books on how to optimize C++ code? ...

Java NIO FileChannel versus FileOutputstream performance / usefulness

Hello, I am trying to figure out if there is any difference in performance (or advantages) when we use nio FileChannel versus normal FileInputStream/FileOuputStream to read and write files to filesystem. I observed that on my machine both perform at the same level, also many times the FileChannel way is slower. Can I please know more de...

Why Does This Improve Performance?

I've two for loops that basically look up in two different arrays (each having a size around 2-4k at peak) and set a value in a 3rd array based on these values. For some weird reason there is a factor two difference between the performance of this piece of code depending on in which order I put the two for loops. This is the first setup...

Branchless code that maps zero, negative, and positive to 0, 1, 2

Write a branchless function that returns 0, 1, or 2 if the difference between two signed integers is zero, negative, or positive. Here's a version with branching: int Compare(int x, int y) { int diff = x - y; if (diff == 0) return 0; else if (diff < 0) return 1; else return 2; } Here's a versio...

Is it okay to have a method declared an inline method if its has a for loop in C++

I have a method like the one shown below. Will the for loop always make the compiler for go the "inline request" ? inline void getImsiGsmMapFrmImsi ( const string& imsiForUEDir, struct ImsiGsmMap& imsiGsmMap ) { for (int i = 0 ; (unsigned)i < imsiForUEDir.length() - 1 ; i++) { imsiGsmMap.value[i] = imsiForU...

Vector initializing slower than array...why?

I tried 2 things: (pseudo code below) int arr[10000]; for (int i = 0; i < 10000; i++) { for (int j = 0; j < 10000; j++) { arr[j] = j; } } and vector<int> arr(10000); for (int i = 0; i < 10000; i++) { for (int j = 0; j < 10000; j++) { arr[j] = j; } } I ran both the programs and timed it using the "tim...

MySQL not using indexes

Hi, I just enabled the slow-log (+not using indexes) and I'm getting hundreds of entries for the same kind of query (only user changes) SELECT id , name FROM `all` WHERE id NOT IN(SELECT id FROM `picks` WHERE user=999) ORDER BY name ASC; EXPLAIN gives: +----+--------------------+-----...