efficiency

In SQL can a sequenced range selection be done more efficiently than my algorithm (see code) that uses a cursor?

I need to collapse multiple ranges of sequential numbers (1 or more) to sets of their minimum and maximum values. I have unique integers (no duplicates) stored in a table column. The obvious way (to me) to solve this problem is to use a cursor (see my algorithm below) and iterate through every integer. However, it seems inefficient to ...

Deeper understanding on loops (for, while, do while, foreach, recursion, etc.)

If givin some situation that you can do a loop of a certain event or function that needed to be solved using loops, where you can achieve these by any kind of loop. How can we determine the difference between each loops that can be used based on their speed, efficiency, and memory usage? Like for example, you have these loops for(int...

Problems with HUGE XML files

I have 16 large xml files. When I say Large, I am talking in gigabytes. One of these files is over 8 GB. Several of them are over 1 gb. These are given to me from an external provider. I am trying to import the XML into a database so that I can shred it into tables. Currently, I stream 10,000 records at a time out of the file into mem...

Efficient Multiple SQL insertion

Hey guys, What is the best/most time efficient way to insert 1000 rows into one table (jdbc/connector-mysql database)? (it is a buffer and need to be dumped into the database everytime it is full) 1- One auto generated/concanated SQL statement? 2- for (int i = 0; i<1000; i++) { con.prepareStatement(s.get(i)); } con.commit(); 3- stored...

Query optimization - VARCHAR equality vs Numeric equality

I have a large SQL Server database with about 40 columns and hundreds of millions of rows. This table is supposed to be loose in schema so I have a lot of columns as VARCHAR(MAX) even where it could have been BIGINT, DATETIME, INT etc. Does this have an impact on querying time/efficiency? e.g. will SELECT TOP 100 * FROM CustomerId = 3...

Python - Is my code effecient? Or are people going to have fun killing my server...?

Aight, basically, I have a database that looks like this: id | parentid | type | name --------------------------------------------- 1 | 0 | heading | this is my heading --------------------------------------------- 2 | 1 | child | this is one of many child elements I'm using Mako to go through that list of data an...

The most efficient way to setup MySQL on Ubuntu Server 10 LTS?

Very soon I will setting up a web server for a client who has a very busy site. He has a forum (run on Simple Machines Forum SMF written in PHP) that cannot really be changed. I want to move this to a separate virtual private server on it's own, as it's very busy at around 40,000 hits a day with 400+ posts a day. With the version of SMF ...

I have a huge set of elements and jQuery's each() function is killing my browser - How can I improve my code?

I'm using two plugins I wrote to find all the radio/checkbox inputs and select boxes in a form and them style them. I now have a large form which a lot of checkboxes and Firefox is hanging as my plugin tries to style each of them. Here's the plugin code: (function($) { $.fn.stylecheck = function(options) { /* Para...

Better site performance: multiple simple mySQL queries or one complex mySQL query later manipulated in PHP?

I am new to web development and am used to extensive use of various data structures in getting things done. I've also heard that having lots of queries is worse than few queries (but that complex queries can be even slower). I'm wondering whether lots of simple queries would perform better or worse than one complex query that is later p...

Is SQL equal test fast for large numbers?

I run the very same query on two almost identical database. The only difference is that the first database has ID entries from 1 to 9000 for 2 tables while the other is in the 458231044 and 103511044 range for the 2 same tables. (for the same 9000 entries) The query compares ID and UNIX time numerous times. Running it on first database...

Algorithm to get changes between two arrays

I needed to create an algorithm which will (efficiently) take an old array and a new array and give me back the changes between the two (which items added, which removed). It happens to need to be in JavaScript (to run in the browser) but the algorithm's more important than the language. This is what I came up with: http://jsbin.com/ose...

How to efficiently identify a binary file

What's the most efficient way to identify a binary file? I would like to extract some kind of signature from a binary file and use it to compare it with others. The brute-force approach would be to use the whole file as a signature, which would take too long and too much memory. I'm looking for a smarter approach to this problem, and I'...

loop through queries

//display faction userlevel: $level_boss = $req_faction_info['f_boss']; $level_uboss = $req_faction_info['f_uboss']; $level_rhm = $req_faction_info['f_rhm']; $level_lhm = $req_faction_info['f_lhm']; $level_r1 = $req_faction_info['f_r1']; $level_r2 = $req_faction_info['f_r2']; if($level_boss == $username){ $u_level = 'Boss'; } elsei...

Which is generally faster, a yield or an append?

I am currently in a personal learning project where I read in an XML database. I find myself writing functions that gather data and I'm not sure what would be a fast way to return them. Which is generally faster: yields, or several append()s within the function then return the ensuing list? I would be happy to know in what situati...

What is better for performance: Update specific cells or replace entire table ?

My project requires that I update the contents of a specific column in an HTML table via an AJAX call to a PHP script. In terms of workload: The table contains about 4 columns and about 20 rows (more rows could be added with time). What would be more (or most) efficient: Replace ONLY the column/specific cells that need to be updated...

(almost) Non-colliding simple hashing function for use in a switch

Hi, I am writing an advanced calculator in C. As you can guess, it currently has many functions, and I use a switch to do the proper operation for every function name. It goes something like this: switch(hash_of(function_name_currently_being_parsed)) { case HASH_COS: // do something break; case HASH_SIN: // do something...

Detect how much stress the mysql database is currently under with PHP

I am developing a large web app and want it to alter itself dependent on a factor that relates to the stress the database is currently under. I am not sure what would me most accurate/effective/easiest. I am considering maybe the number of current connections or server response time or CPU useage? What would be best suited and possible...

ServiceHost Efficiency

Hiya, Hoping you could help me please.. I am using WCF in my program. Part of this includes using ServiceHost in a self-service type scenario using a netNamedTypeBinding. When I include ServiceHost in my class and then instantiate that class (ServiceHost is set to null at this point), the memory usage increases by approx 9Mb. If I c...

C - Glib GINT_TO_POINTER portability

I'm dealing with large numbers coming from the hash table. I'm wondering what would be a good way of adding them to a constant (100) taking into account portability. Glib's documentation highlights that using GINT_TO_POINTER is not portable in any way. Any ideas would be appreciated! gpointer v, old_key; gint value; // ? if(g_hash_tab...

Php / MySQL search function: there must be an easier way of achieving this?

//lets get the auto name set $accepted = 0; $sql = mysql_query("SELECT * FROM ".TBL_FACTIONS." WHERE f_name='no_name'"); $numrows = mysql_num_rows($sql); //default name 1 if($numrows == 0){ $auto_name = 'no_name'; $accepted = 1;} //default name 2 if ($accepted == 0){ $sql = mysql_query("SELECT * FROM ".TBL_FACTIONS." WHERE f_name='no_na...