Getting an 'out of memory' exception in this relatively simple program.

Question

Here's my Picture.cs class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Drawing;

namespace SharpLibrary_MediaManager
{
    public class Picture:BaseFile
    {
        public int Height { get; set; }
        public int Width { get; set; }
        public Image Thumbnail { get; set; }

        /// <summary>
        /// Sets file information of an image from a given image in the file path.
        /// </summary>
        /// <param name="filePath">File path of the image.</param>
        public override void  getFileInformation(string filePath)
        {
            FileInfo fileInformation = new FileInfo(filePath);
            if (fileInformation.Exists)
            {
                Name = fileInformation.Name;
                FileType = fileInformation.Extension;
                Size = fileInformation.Length;
                CreationDate = fileInformation.CreationTime;
                ModificationDate = fileInformation.LastWriteTime;
                Height = calculatePictureHeight(filePath);
                Width = calculatePictureWidth(filePath);                
            }
        }

        public override void getThumbnail(string filePath)
        {            
            Image image = Image.FromFile(filePath);
            Thumbnail = image.GetThumbnailImage(40, 40, null, new IntPtr());            
        }

        private int calculatePictureHeight(string filePath)
        {
            var image = Image.FromFile(filePath);
            return image.Height;
        }

        private int calculatePictureWidth(string filePath)
        {
            var image = Image.FromFile(filePath);
            return image.Width;
        }
    }
}

And here, I'm using that class to pull information from every file in a given folder:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;

namespace SharpLibrary_MediaManager
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        string folderPath = @"D:\Images\PictureFolder";

        private void button1_Click(object sender, EventArgs e)
        {
            DirectoryInfo folder = new DirectoryInfo(folderPath);
            List<Picture> lol = new List<Picture>();
            foreach (FileInfo x in folder.GetFiles())
            {
                Picture picture = new Picture();
                picture.getFileInformation(x.FullName);
                lol.Add(picture);
            }

            MessageBox.Show(lol[0].Name);
        }
    }
}

I'm getting an Out Of Memory exception and I don't really know why. This is the first time I'm doing something like this so I'm pretty new to batch file processing, etc.

Any help guys? :)

Edit: I opened the Task Manager to see memory usage and when I press the Button to run the method I notice my memory usage increases by 100mb~ every second.

Edit 2: In my folder I have about 103 images, each image being ~100kb. I need a solution where it doesn't matter how many images are in a folder. Someone recommended opening an image, doing my magic, then close it. I don't really understand what he meant by 'close'.

Can someone recommend a different approach? :)

Edit 3: Still getting the out of memory exception, I've changed the code in Picture.cs based on recommendations, but I'm out of ideas. Any help?

public override void  getFileInformation(string filePath)
        {
            FileInfo fileInformation = new FileInfo(filePath);

            using (var image = Image.FromFile(filePath))
            {
                if (fileInformation.Exists)
                {
                    Name = fileInformation.Name;
                    FileType = fileInformation.Extension;
                    Size = fileInformation.Length;
                    CreationDate = fileInformation.CreationTime;
                    ModificationDate = fileInformation.LastWriteTime;
                    Height = image.Height;
                    Width = image.Width;
                }
            }
        }

Also, should I open a new question now that this one has grown a bit?

Answer 1

How many pictures do you have in that folder. I see you looping through all the pictures and loading them into memory directly and storing them in a list. It may be the cause.

Answer 2

How big are your picture files? How much files are in your directory? Depending on those parameters I could see how you could get an OutOfMemory. You may want to recheck your approach. Instead of loading everything into memory, you should load each picture individually, perform your action and then continue (after you disposed your previous picture, of course).

Answer 3

You should not store all images in a single list, you should do what you have with one picture at a time and dispose each picture after each iteration.

Check for Image.Dispose() method.

Answer 4

You are not calling Dispose on your Image instances. Also create your image once and then extract your data.

See also:

http://msdn.microsoft.com/en-us/library/8th8381z.aspx

EDIT If copied your code and tested it with my Picture Library. My avg. FileSize is 2-3 MB per File. I've executed your program and it did exactly what it should. The GC did exactly what I've expected.

Memory of your Program was always about 11-35 MB Private Working set, Commit Size was stable at 43 MB.

I've aborted the program after 1156 files with a total picture size of 2.9 GB.

So there must be another reason for you out of memory exception.

Here's my program output and code:

1133: Total Size = 2.842,11 MB
1134: Total Size = 2.844,88 MB
1135: Total Size = 2.847,56 MB
1136: Total Size = 2.850,21 MB
1137: Total Size = 2.853,09 MB
1138: Total Size = 2.855,86 MB
1139: Total Size = 2.858,59 MB
1140: Total Size = 2.861,26 MB
1141: Total Size = 2.863,65 MB
1142: Total Size = 2.866,15 MB
1143: Total Size = 2.868,52 MB
1144: Total Size = 2.870,93 MB
1145: Total Size = 2.873,64 MB
1146: Total Size = 2.876,15 MB
1147: Total Size = 2.878,84 MB
1148: Total Size = 2.881,92 MB
1149: Total Size = 2.885,02 MB
1150: Total Size = 2.887,78 MB
1151: Total Size = 2.890,57 MB
1152: Total Size = 2.893,55 MB
1153: Total Size = 2.896,32 MB
1154: Total Size = 2.898,92 MB
1155: Total Size = 2.901,48 MB
1156: Total Size = 2.904,02 MB

Sourcecode:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Drawing;

namespace SharpLibrary_MediaManager
{
    public abstract class BaseFile
    {
        public string Name { get; set; }
        public string FileType { get; set; }
        public long Size { get; set; }
        public DateTime CreationDate { get; set; }
        public DateTime ModificationDate { get; set; }

        public abstract void getFileInformation(string filePath);

    }


    public class Picture : BaseFile
    {
        public int Height { get; set; }
        public int Width { get; set; }
        public Image Thumbnail { get; set; }

        public override void getFileInformation(string filePath)
        {
            FileInfo fileInformation = new FileInfo(filePath);

            using (var image = Image.FromFile(filePath))
            {
                if (fileInformation.Exists)
                {
                    Name = fileInformation.Name;
                    FileType = fileInformation.Extension;
                    Size = fileInformation.Length;
                    CreationDate = fileInformation.CreationTime;
                    ModificationDate = fileInformation.LastWriteTime;
                    Height = image.Height;
                    Width = image.Width;
                    Thumbnail = image.GetThumbnailImage(40, 40, null, new IntPtr());
                }
            }
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            string folderPath = @"C:\Users\arthur\Pictures";

            DirectoryInfo folder = new DirectoryInfo(folderPath);
            List<Picture> lol = new List<Picture>();
            double totalFileSize = 0;
            int counter = 0;
            foreach (FileInfo x in folder.GetFiles("*.jpg", SearchOption.AllDirectories))
            {
                Picture p = new Picture();
                p.getFileInformation(x.FullName);
                lol.Add(p);
                totalFileSize += p.Size;
                Console.WriteLine("{0}: Total Size = {1:n2} MB", ++counter, totalFileSize / 1048576.0);
            }

            foreach (var p in lol)
            {
                Console.WriteLine("{0}: {1}x{2} px", p.Name, p.Width, p.Height);
            }
        }
    }
}

Answer 5

A few problems I see off the bat. Firstly, you are loading each Image twice with your subsequent calls to CalculatePictureWidth and CalculatePictureHeight. Secondly, you're never actually doing anything with the thumbnail, it appears. Thirdly, you should call Dispose on the Image instances once you are done gathering information from them.

Answer 6

The Image file contains a handle to a large block of unmanaged data. This data has to be removed and you do this by explicitly calling Image.Dispose. Because you don't call Dispose in the calculatePictureHeight and calculatePictureWidth methods, the Image files are kept in memory, which is causing your OOM exceptions.

[Update]: Perhaps I little more background is useful, why this is actually happening. After all, isn't there the GC to clean up this mess for us? Yes there is, and the GC is pretty effective. It will eventually also clean up objects, that we had to dispose.

The Image class contains a finalizer. This finalizer will ensure that all resources are cleared, even when you forget to call Dispose. However, what happens here is that you run out of memory, which immediately triggers a GC.Collect. During this collect, the GC finds all your unreferenced Images. However, because the GC will not run the finalizer method immediately, all your objects are promoted to Gen1 (they are kept in memory) and so are all your resources (the native memory). So even after the GC has kicked in, there is still too little memory and you get an OOM exception.

This is why you should always dispose objects that implement IDisposable.

Answer 7

You have to free the resources used when opening an Image object.

Either you can call Dispose, or create your image in a Using statement

e.g.

public override void getThumbnail(string filePath)
{
    using (Image image = Image.FromFile(filePath))
    {
        Thumbnail = image.GetThumbnailImage(40, 40, null, new IntPtr());
    }
}

And as your class contains an Image you should implement the IDisposable interface, so you can also use it in a using statement.

Answer 8

Try looking at your images one at a time. I have found certain .jpg's cause this - the image manipulating libraries have problems with certain images, not sure why. But we have had certain .jpg's saved from photoshop some odd way cause this. I'll bet you'll find it's one bad apple.

Answer 9

Several people have mentioned calling dispose once done with the images. However, this is not enough and something the .net framework will do for you as memory usage increases. Since you've added each picture to a linked list the .NET framework can't dispose of it, as you implicitly keep a reference.

This means that all data stored within the picture class is kept during the foreach loop. From the code fragment I can't see what it is, it may include a compressed or even an uncompressed version of the images which could explain your memory explosion.

[Edit] removed part of anwer which others have pointed out to be irrelevant; I assumed picture class to be the XNA picture class.

Answer 10

Can you do this?

public class Picture:BaseFile
{
    public int Height { get; set; }
    public int Width { get; set; }
    public Image Thumbnail { get; set; }

    /// <summary>
    /// Sets file information of an image from a given image in the file path.
    /// </summary>
    /// <param name="filePath">File path of the image.</param>
    public override void  getFileInformation(string filePath)
    {
        FileInfo fileInformation = new FileInfo(filePath);
        if (fileInformation.Exists)
        {
            /*
            Name, FileType, Size, etc
            */
            using (Image image = Image.FromFile(filePath))
            {
                Height = image.Height;
                Width = image.Width;
                Thumbnail = image.GetThumbnailImage(40, 40, new Image.GetThumbnailImageAbort(ThumbnailCallback), default(IntPtr));
            }
        }
    }

    public bool ThumbnailCallback()
    {
        return false;
    }
}