optimization

Saving memory when fetching large result sets with PDO

Hi everybody. I have written a tool for database replication in PHP. It's working fine but there's one issue: I'm using PDO to connect to the different databases to keep it independent of any specific RDBMS, which is crucial to this application. The tool does some analysis on the tables to decide how to convert certain types and some ...

Help me do this SQL update query correctly.

I have 3 values that I need to copy from one table to another table. Here is my amateur attempt at it, I know it is horribly ineffecent, what would the correct way of doing this query? update [IDAT_PATIENTS] set TargetRabiesSerial = (select top 1 SERIAL_NUMBER from [IDAT_RABIESHISTORY] as rab where TargetPetAccount...

Help with an algorithm

I need help with making this bit of code faster: UnitBase* Formation::operator[](ushort offset) { UnitBase* unit = 0; if (offset < itsNumFightingUnits) { ushort j = 0; for (ushort i = 0; i < itsNumUnits; ++i) { if (unitSetup[i] == UNIT_ON_FRONT) { if (j == offset) unit = unitFormation[i]; ++j; } } } el...

Visual Studio 2010 Optimization & Tips?

I've noticed Visual Studio 2010 is a lot slower than my Visual Studio 2008 EDI, I've found several nice tips and optimization suggestions for VS2008, however I want to know if people have any tips for me and VS2010 ...

Python library for experimenting with compiler optimizations

I want to learn about compilers and some optimization techniques, and I thought it would be helpful to do some quick implementations of the algorithms. Is there a library/framework for Python that can make things easier (like the Natural Language Toolkit) - generating the parse tree, manipulating loops, methods? I saw that Microsoft...

Performance-wise: request JSON and render in JS, or request the entire HTML?

IF I send an AJAX request to a PHP file, what would result in faster rendering of HTML: Sending the completely formatted HTML straight from PHP, or: Just send JSON data and let Javascript do the HTML rendering? I have a rather complex HTML structure, and this puts download time of a large HTML chunk vs. the times Javascript (jQuery) ...

Why the most natural query(i.e. using INNER JOIN (instead of LEFT JOIN)) is very slow.

This query takes too long... explain analyze select c.company_rec_id, c.the_company_code , c.company from tlist t -- it is questionable why this query become fast when using left join, the most natural query is inner join... join mlist m using(mlist_rec_id) join parcel_application ord_app using(parcel_application_rec_id) join par...

Most efficient way to calculate Levenshtein distance

I just implemented a best match file search algorithm to find the closest match to a string in a dictionary. After profiling my code, I found out that the overwhelming majority of time is spent calculating the distance between the query and the possible results. I am currently implementing the algorithm to calculate the Levenshtein Dista...

Sending huge vector to a Database in R

Good afternoon, After computing a rather large vector (a bit shorter than 2^20 elements), I have to store the result in a database. The script takes about 4 hours to execute with a simple code such as : #Do the processing myVector<-processData(myData) #Sends every thing to the database lapply(myVector,sendToDB) What do you think is ...

Correct way of getting multiple records from database in php.

I always used the loop method to get all records from resource received from mysql_query(), but I wonder is there a easier way to accomplish this task? e.g.: SELECT table_schema, table_name FROM information_schema.tables; I suppose it should either way return more, than just one record. Question is short, but I think I explained mys...

convert number to slash divided hex path

Hey, I need to generate a path string from a number (in C) e.g: 53431453 -> 0003/2F4/C9D what I have so far is this: char *id_to_path(long long int id, char *d) { char t[MAX_PATH_LEN]; sprintf(t, "%010llX", id); memcpy(d, t, 4); memcpy(d+5, t+4, 3); memcpy(d+9, t+7, 4); d[4] = d[8] = '/'; return ...

optimizing jQuery Selectors - Which is faster?

Are selectors or functions faster in jQuery? Example: $('#something div.else'); // find multiple divs with class .else or $('#something').children('div.else'); I'm trying my best to optimize a page with hundreds of returned elements that seems to hang on a certain crappy computer here in the office (that must use Internet Explore...

Clang vs GCC - which produces better binaries?

I'm currently using GCC, but I discovered Clang recently and I'm pondering switching. There is one deciding factor though - quality (speed, memory footprint, reliability) of binaries it produces - if gcc -O3can produce a binary that runs 1% faster or takes 1% less memory, it's a deal-breaker. Clang boasts better compile speeds and lower...

Partially initialize variable defined in other module.

I'm considering a certain solution where I would like to initialize a cell of an array that is defined in other module (there will be many modules initializing one table). The array won't be read before running main (so there is not problem with static initialization order). My approach: /* secondary module */ extern int i[10]; // th...

gcc -O3 question, calling same function from different file yields different performance

Hey, I'm running the following benchmark: int main(int argc, char **argv) { char *d = malloc(sizeof(char) * 13); TIME_THIS(func_a(999, d), 99999999); TIME_THIS(func_b(999, d), 99999999); return 0; } with normal compilation, the results are the same for both functions % gcc func_overhead.c func_overhead_plus.c -o func_overhead...

Batch script to pngcrush all files in all subfolders

I have a folder structure with one main parent folder containing many subfolders, and in these some PNGs, something like: .../data .../data/013523/ .../data/345343/ .../data/395338/ .../data/013523/filex.png .../data/013523/filey.png .../data/345343/filea.png .../data/345343/fileb.png .../data/345343/filec.png I'd like to crush all ...

Which compiler (Sun Studio or gcc ) to choose while porting from Solaris Sparc to Linux x86

We have decide to port our application that currently runs on solaris sparc to linux x86 system. Which compiler, Sun Studio or GNU gcc would be more beneficial to use on linux? Which one would produce at a good level of optimized binaries? Any help or directions would be appreciated! Thanks in advance. ...

How to reduce code duplication in this example

I need to loop through a number (xx). xx always starts at zero. My problem is that if the moveDirection variable is +1 then xx increases until it reaches the positive of range. If moveDirection is -1, then xx decreases until reaching the negative of range. In the code below, I have done this by having an if statement test for moveDirect...

How to Optimize Queries in a Database - The Basics

It seems that all questions regarding this topic are very specific, and while I value specific examples, I'm interested in the basics of SQL optimization. I am very comfortable working in SQL, and have a background in hardware/low level software. What I want is the tools both tangible software, and a method to look at the mysql databas...

Regex optimization question

As a personal learning exercise, I wrote this regex to split a unary string into parts whose length is increasing powers of two (see also on ideone.com): for (String s : new String(new char[500]) .split("(?=(.*))(?<=(?<=(^.*))\\G.*)(?<=(?=\\2\\2.\\1)^.*)") ) { System.out.printf("%s ", s.length()); } ...