Optimized Image Loading in a UIScrollView

Question

I have a UIScrollView that has a set of images loaded side-by-side inside it. You can see an example of my app here: http://www.42restaurants.com. My problem comes in with memory usage. I want to lazy load the images as they are about to appear on the screen and unload images that aren't on screen. As you can see in the code I work out at a minimum which image I need to load and then assign the loading portion to an NSOperation and place it on an NSOperationQueue. Everything works great apart from a jerky scrolling experience.

I don't know if anyone has any ideas as to how I can make this even more optimized, so that the loading time of each image is minimized or so that the scrolling is less jerky.

- (void)scrollViewDidScroll:(UIScrollView *)scrollView{
    [self manageThumbs]; 
}

- (void) manageThumbs{
    int centerIndex = [self centerThumbIndex];
    if(lastCenterIndex == centerIndex){
     return;
    }

    if(centerIndex >= totalThumbs){
     return;
    }

    NSRange unloadRange;
    NSRange loadRange;

    int totalChange = lastCenterIndex - centerIndex;
    if(totalChange > 0){ //scrolling backwards
     loadRange.length = fabsf(totalChange);
     loadRange.location = centerIndex - 5;
     unloadRange.length = fabsf(totalChange);
     unloadRange.location = centerIndex + 6;
    }else if(totalChange < 0){ //scrolling forwards
     unloadRange.length = fabsf(totalChange);
     unloadRange.location = centerIndex - 6;
     loadRange.length = fabsf(totalChange);
     loadRange.location = centerIndex + 5;
    }
    [self unloadImages:unloadRange];
    [self loadImages:loadRange];
    lastCenterIndex = centerIndex;

    return;
}

- (void) unloadImages:(NSRange)range{
    UIScrollView *scrollView = (UIScrollView *)[[self.view subviews] objectAtIndex:0];
    for(int i = 0; i < range.length && range.location + i < [scrollView.subviews count]; i++){
     UIView *subview = [scrollView.subviews objectAtIndex:(range.location + i)];
     if(subview != nil && [subview isKindOfClass:[ThumbnailView class]]){
      ThumbnailView *thumbView = (ThumbnailView *)subview;
      if(thumbView.loaded){
       UnloadImageOperation *unloadOperation = [[UnloadImageOperation alloc] initWithOperableImage:thumbView];
       [queue addOperation:unloadOperation];
       [unloadOperation release];
      }
     }
    }
}

- (void) loadImages:(NSRange)range{
    UIScrollView *scrollView = (UIScrollView *)[[self.view subviews] objectAtIndex:0];
    for(int i = 0; i < range.length && range.location + i < [scrollView.subviews count]; i++){
     UIView *subview = [scrollView.subviews objectAtIndex:(range.location + i)];
     if(subview != nil && [subview isKindOfClass:[ThumbnailView class]]){
      ThumbnailView *thumbView = (ThumbnailView *)subview;
      if(!thumbView.loaded){
       LoadImageOperation *loadOperation = [[LoadImageOperation alloc] initWithOperableImage:thumbView];
       [queue addOperation:loadOperation];
       [loadOperation release];
      }
     }
    }
}

EDIT: Thanks for the really great responses. Here is my NSOperation code and ThumbnailView code. I tried a couple of things over the weekend but I only managed to improve performance by suspending the operation queue during scrolling and resuming it when scrolling is finished.

Here are my code snippets:

//In the init method
queue = [[NSOperationQueue alloc] init];
[queue setMaxConcurrentOperationCount:4];


//In the thumbnail view the loadImage and unloadImage methods
- (void) loadImage{
    if(!loaded){
     NSString *filename = [NSString stringWithFormat:@"%03d-cover-front", recipe.identifier, recipe.identifier]; 
     NSString *directory = [NSString stringWithFormat:@"RestaurantContent/%03d", recipe.identifier];  

     NSString *path = [[NSBundle mainBundle] pathForResource:filename ofType:@"png" inDirectory:directory];
     UIImage *image = [UIImage imageWithContentsOfFile:path];

     imageView = [[ImageView alloc] initWithImage:image andFrame:CGRectMake(0.0f, 0.0f, 176.0f, 262.0f)];
     [self addSubview:imageView];
     [self sendSubviewToBack:imageView];
     [imageView release];
     loaded = YES;  
    }
}

- (void) unloadImage{
    if(loaded){
     [imageView removeFromSuperview];
     imageView = nil;
     loaded = NO;
    }
}

Then my load and unload operations:

- (id) initWithOperableImage:(id<OperableImage>) anOperableImage{

    self = [super init];
    if (self != nil) {
     self.image = anOperableImage;
    }
    return self;
}

//This is the main method in the load image operation
- (void)main {
    [image loadImage];
}


//This is the main method in the unload image operation
- (void)main {
    [image unloadImage];
}

Answer 1

One option, although less visually pleasing, is to only load images when the scrolling stops.

Set a flag to disable image loading in:

-scrollViewWillBeginDragging:

Re-enable loading images when the scrolling stops using the:

-scrollViewDidEndDragging:willDecelerate:

UIScrollViewDelegate method. When the willDecelerate: parameter is NO, the movement has stopped.

Answer 2

I'm a little puzzled by the "jerky" scrolling. Since NSOperationQueue runs operations on separate thread(s) I'd have expected at worst you might see empty UIImageViews showing up on the screen.

First and foremost I'd be looking for things that are impacting the processor significantly as NSOperation alone should not interfere with the main thread. Secondly I'd be looking for details surrounding the NSOperation setup and execution that might be causing locking and syncing issues which could interrupt the main thread and therefore impact scrolling.

A few items to consider:

1. Try loading your ThumbnailView's with a single image at the start and disabling the NSOperation queuing (just skip everything following the "if loaded" check. This will give you an immediate idea whether the NSOperation code is impacting performance.

2. Keep in mind that -scrollViewDidScroll: can occur many times during the course of a single scroll action. Depending on how for the scroll moves and how your -centerThumbIndex is implemented you might be attempting to queue the same actions multiple times. If you've accounted for this in your -initWithOperableImage or -loaded then its possible you code here is causing sync/lock issues (see 3 below). You should track whether an NSOperation has been initiated using an "atomic" property on the ThumbnailView instance. Prevent queuing another operation if that property is set and only unset that property (along with loaded) at the end of the NSOperation processes.

3. Since NSOperationQueue operates in its own thread(s) make sure that none of your code executing within the NSOperation is syncing or locking to the main thread. This would eliminate all of the advantages of using the NSOperationQueue.

4. Make sure your "unload" operation has a lower priority than your "load" operation, since the priority is the user experience first, memory conservation second.

5. Make sure you keep enough thumbnails for at least a page or two forward and back so that if NSOperationQueue falls behind, you have a high margin of error before blank thumbnails become visible.

6. Make sure your load operation is only loading a "pre-scaled" thumbnail and not loading a full size image and rescaling or processing. This would be a lot of extra overhead in the middle of a scrolling action. Go even further and make sure you've converted them to PNG16 without an alpha channel. This will give at least a (4:1) reduction in size with hopefully no detectable change in the visual image. Also consider using PVRTC format images which will take the size down even further (8:1 reduction). This will greatly reduced the time it takes to read the images from "disk".

I apologize if any of this doesn't make sense. I don't see any issues with the code you've posted and problems are more likely to be occurring in your NSOperation or ThumbnailView class implementations. Without reviewing that code, I may not be describing the conditions effectively.

I would recommend posting your NSOperation code for loading and unloading and at least enough of the ThumbnailView to understand how it interacts with the NSOperation instances.

Hope this helped to some degree,

Barney

Answer 3

the problem is here:

UIImage *image = [UIImage imageWithContentsOfFile:path];

It seems that threaded or not when you load a file from disk (which maybe that happens on the main thread regardless, I'm not totally sure) everything stalls. You normally don't see this in other situations because you don't have such a large area moving if any at all.

Answer 4

While researching this problem, I found two more resources that may be of interest:

Check out the iPhone sample project "PageControl": http://developer.apple.com/iphone/library/samplecode/PageControl/index.html

It lazy loads view controllers in a UIScrollView.

• and -

Check out the cocoa touch lib: http://github.com/facebook/three20 which has a 'TTPhotoViewController' class that lazy loads photos/thumbnails from web/disk.