Combining query rows in a loop

Question

I have the following ColdFusion 9 code:

<cfloop from="1" to="#arrayLen(tagArray)#" index="i">
    <cfquery name="qryGetSPFAQs" datasource="#application.datasource#">
        EXEC searchFAQ '#tagArray[i]#'
    </cfquery>
</cfloop>

The EXEC executes a stored procedure on the database server, which returns rows of data, depending on what the parameter is. What I am trying to do is combine the queries into one query object. In other words, if it loops 3 times and each loop returns 4 rows, I want a query object that has all 12 rows in one object. How do I acheive this?

Answer 1

You might want to take a different approach (modify your stored procedure to accept multiple arguments or use a list and fnSplit) and return the dataset all at once. However, to directly answer your question, this is how you could combine the queries as you're asking to:

You can use UNION in a Query of Queries to combine all of the datasets.

<cfloop from="1" to="#arrayLen(tagArray)#" index="i">
    <cfquery name="qryGetSPFAQs#i#" datasource="#application.datasource#">
        EXEC searchFAQ '#tagArray[i]#'
    </cfquery>
</cfloop>

<cfquery name="combined" dbtype="query">
    <cfloop from="1" to="#arrayLen(tagArray)#" index="i">
        select * from qryGetSPFAQs#i#
        <cfif i lt arrayLen(tagArray)>UNION</cfif>
    </cfloop>
</cfquery>

Answer 2

A more direct way might be something like this:

<cfset bigQ = queryNew("column")>
<cfloop from="1" to="#arrayLen(tagArray)#" index="i">
    <cfquery name="qryGetSPFAQs" datasource="#application.datasource#">
        EXEC searchFAQ '#tagArray[i]#'
    </cfquery>
    <cfset queryAddRow(bigQ)>
    <cfset querySetCell(bigQ, "column". qryGetSPFAQs)>
</cfloop>

You will need a querySetCell() assignment for each column. Check out the query functions in the live docs for more information.

Answer 3

Here is an out of the box solution, abandoning the StoredProc for a SQL View (I'll explain).

Disclaimer: without seeing the SP source code, I can't tell if my solution fits. I'm assuming that the SP is fairly basic, and I admit I usually prefer the compiled execution of an SP over a view, but the one-time execution of a SQL View should outperform the looping of the SP x times.

First make a view that looks like the SELECT statement in the SP (minus the parameterization, of course -- you'll cover that in a WHERE clause within the CFQUERY of your new view.

Second, set up your loop to do no more than build a data set we're going to use for the WHERE clause. You'll need to use ArrayToList and a little bit of string manipulation to tidy it up, with the end product being a string stored in a single CF variable looking like this:

('ValueOfArrayElement1','ValueOfArrayElement2','Value_And_So_On')

Building the string is pretty easy, using the delimeter attribute of ArrayToList, and after the loop is complete, append a Left Parenthesis & Single Quote to the Left most position of the string; and append a Single Quote & Right Parenthesis to the Right most position in the string.

Now, write the CFQUERY statement to SELECT the columns you need from your view (instead of executing your SP). And instead of passing a parameter to the SP, you're going to put a WHERE clause in the CFQUERY.

Oh, BTW, I am stating you need a SQL View, but the entire SELECT could be built in CFQUERY. Personally, when I have a multi-table JOIN, I like to define that in a SQL View where it's executed more quickly than a JOIN in CFQUERY. Ultimately a StoredProc is even faster, but our WHERE clause is much friendlier to code and read like this than it would be to send into StoredProc without looping in and out of SQL.

It's a good goal to make only one trip out to the database and back if possible. That's why we looped through the array to write a string equating to all the values in the dataset. This way, we'll only execute one query, one time.

SELECT Col1, Col2, Col_etc
FROM SQL_View_Name
WHERE ColumnName in #BigStringWeMadeFromArrayToList#

when our CFQUERY is rendered, the clause will look just like this in SQL:

WHERE ColumnName in 
     ('ValueOfArrayElement1','ValueOfArrayElement2','Value_And_So_On')

So there you have it. Like I said, this is nice because it makes only one trip to the DB, and since we are building a view, the performance will still be pretty good -- better than running a StoredProc 4+ times. (no offense)

I'll must repeat... without having seen the SP code, I'm not sure if this is do-able. Plus, it's kind of odd to abandon a StoredProc for a SQL View, a "lesser" entity in the RDBMS, but I'm sure we will achieve greater performance and I think it's pretty readable, too.