using cfhttp in multiple files taking too much time

Question

I don't know if its possible but just want to ask if we can cfhttp or any other thing to read selected amount of data instead of putting whole file in CFHTTP.FileContent.

I am using cfhttp and want to read only last two lines from a remote xml files(about 20 of them) and read middle two lines from some text files (about 7 of them). Is there any way I could just read that specific data instead of getting all files because its taking a lot of time right now(about 15-20 seconds). I just want to reduce the run time of my .cfm page. Any suggestions ???

Answer 1

Hmm, not really any special way to get just parts of the remote files.

Do you have to do it every time? Could you fetch the files in the background, write them locally, and have your actual incoming requests just read those files? Make the reading of the remote files asynchronous to the incoming requests?

If not, and you're using CF8+, you could use CFTHREAD to thread out the various requests to run in parallel: http://livedocs.adobe.com/coldfusion/8/htmldocs/help.html?content=Tags_t_04.html

You can use the "join" action in the end to make wait for all the threads to complete.

Edit:

Here's a great tutorial by Ben Nadel on using CFThread to parallelize CFHTTP requests:

http://www.bennadel.com/blog/749-Learning-ColdFusion-8-CFThread-Part-II-Parallel-Threads.htm

There's something else, though:

27-30 sequential http requests should not take 20-30 seconds. It really shouldn't even take 1-2 seconds - so you may have some serious other issue going on here.

Answer 2

HTTP does not have the ability to read a file in that manner. This has nothing to do with ColdFusion.

You can use some smart caching to reduce the time somewhat at the cost of a longer time the first time you run it using CFHTTP's method="HEAD" which does not.

Do you have a local copy of the page?

• No, use CFHTTP method="GET" to grab and store it
• Yes, use CFHTTP method="HEAD" to check the timestamp and compare it to the cached version. If cache is newer, use it, else CFHTTP method="GET" to grab and parse the file you want.

method="HEAD" will only grab the http headers and not the entire file which will speed things up ever so slightly. Either way, you are making almost 30 file requests, so this isn't going to be instantaneous either way you cut it.

Answer 3

How about ask CF to only serve that chunk of file with URL params?

Since it is XML, I guess you can use xmlSearch() and return only the result?

as for text file, u can pass in the startline & numOfLines and return only those lines as string?