Memory alignment on modern processors?

Question

I often see code such as the following when, e.g., representing a large bitmap in memory:

size_t width = 1280;
size_t height = 800;
size_t bytesPerPixel = 3;
size_t bytewidth = ((width * bytesPerPixel) + 3) & ~3; /* Aligned to 4 bytes */
uint8_t *pixelData = malloc(bytewidth * height);

(that is, a bitmap allocated as a contiguous block of memory having a bytewidth aligned to a certain number of bytes, most commonly 4.)

A point on the image is then given via:

pixelData + (bytewidth * y) + (bytesPerPixel * x)

This leads me to two questions:

1. Does aligning a buffer like this have a performance impact on modern processors? Should I be worrying about alignment at all, or will the compiler handle this?
2. If it does have an impact, could someone point me to a resource to find the ideal byte alignment for various processors?

Thanks.

Answer 1

• Does aligning a buffer like this have a performance impact on modern processors?

Yes. For instance if memcpy is optimized using SIMD instructions (like MMX/SSE) some operations will be faster with aligned memory. In some architectures there are (processor) instructions that fail if the data is not aligned, thus something might work on your machine but not in another one.

With aligned data you also make a better use of the CPU caches.

• Should I be worrying about alignment at all, or will the compiler handle this?

I should worry about alignment when I use dynamic memory and the compiler cannot handle this (see the reply to this comment).

For other stuff in your code you have the -malign flag and aligned attribute to play with.

Answer 2

Yes, alignment does have a performance impact on modern-- let's say x86--processors. Generally, loads and stores of data happen on natural alignment boundaries; if you're getting a 32-bit value into a register, it's going to be fastest if it's aligned on a 32-bit boundary already. If it's not, the x86 will "take care of it for you", in the sense that the CPU will still do the load, but it will take a significantly larger number of cycles to do it, because there will be internal wrangling to "re-align" the access.

Of course, in most cases, this overhead is trivial. Structures of binary data are frequently packed together in unaligned ways for transport over the network or for persistence on disk, and the size benefits of the packed storage outweigh any perf hit from operating occasionally on this data.

But particularly with large buffers of uniform data that get accessed randomly and where performance in the aggregate really is important, as in your pixel buffer above, keeping data structures aligned can still be beneficial.

Note that in the case of the example you give above, only each "line" of pixel data is aligned. The pixels themselves are still 3 bytes long and often unaligned within the "lines", so there's not much benefit here. There are texture formats, for example, that have 3 bytes of real data per pixel, and literally just waste an extra byte on each one to keep the data aligned.

There's some more general information here: http://en.wikipedia.org/wiki/Data%5Fstructure%5Falignment

(The specific characteristics vary between architectures, both in what the natural alignments are, whether the CPU handles unaligned loads/stores automatically, and in how expensive those end up being. In cases where the CPU doesn't handle access magically, often the compiler/C runtime will do what it can to do this work for you.)

Answer 3

It depends on a lot of factors. If you're only accessing the pixel data one byte at a time, the alignment will not make any difference the vast majority of the time. For reading/writing one byte of data, most processors won't care at all whether that byte is on a 4-byte boundary or not.

However, if you're accessing data in units larger than a byte (say, in 2-byte or 4-byte units), then you will definitely see alignment effects. For some processors (e.g. many RISC processors), it is outright illegal to access unaligned data on certain levels: attempting to read a 4-byte word from an address that's not 4-byte aligned will generate a Data Access Exception (or Data Storage Exception) on a PowerPC, for example.

On other processors (e.g. x86), accessing unaligned addresses is permitted, but it often comes with a hidden performance penalty. Memory loads/stores are often implemented in microcode, and the microcode will detect the unaligned access. Normally, the microcode will fetch the proper 4-byte quantity from memory, but if it's not aligned, it will have to fetch two 4-byte locations from memory and reconstruct the desired 4-byte quantity from the appropriate bytes of the two locations. Fetching two memory locations is obviously slower than one.

That's just for simple loads and stores, though. Some instructions, such as those in the MMX or SSE instruction sets, require their memory operands to be properly aligned. If you attempt to access unaligned memory using those special instructions, you'll see something like an illegal instruction exception.

To summarize, I wouldn't really worry too much about alignment unless you're writing super performance-critical code (e.g. in assembly). The compiler helps you out a lot, e.g. by padding structures so that 4-byte quantities are aligned on 4-byte boundaries, and on x86, the CPU also helps you out when dealing with unaligned accesses. Since the pixel data you're dealing with is in quantities of 3 bytes, you'll almost always being doing single byte accesses anyways.

If you decide you instead want to access pixels in singular 4-byte accesses (as opposed to 3 1-byte accesses), it would be better to use 32-bit pixels and have each individual pixel aligned on a 4-byte boundary. Aligning each row to a 4-byte boundary but not each pixel will have little, if any, effect.

Based on your code, I'm guessing it's related to reading the Windows bitmap file format -- bitmap files require the length of each scanline to be a multiple of 4 bytes, so setting up your pixel data buffers with that property has the property that you can just read in the entire bitmap in one fell swoop into your buffer (of course, you still have to deal with the fact that the scanlines are stored bottom-to-top instead of top-to-bottom and that the pixel data is BGR instead of RGB). This isn't really much of an advantage, though -- it's not that much harder to read in the bitmap one scanline at a time.

Answer 4

Buffer alignment has an impact. The question is: is it a significant impact? The answer can be highly application specific. In architectures which do not natively support unaligned access—for example, the 68000 and 68010 (the 68020 adds unaligned access)—it's truly a performance and/or maintenance problem since the CPU will fault, or maybe trap to a handler to perform unaligned access.

The ideal alignment for various processors can be estimated: 4-byte alignment is appropriate for architectures with a 32-bit data path. 8-byte alignment for 64-bit. However, L1 caching has an effect. For many CPUs this is 64 bytes though it will no doubt change in the future.

Too high of an alignment (that is, eight byte where only two byte is needed) causes no performance inefficiency for any narrower system, even on an 8-bit microcontroller. It simply wastes (potentially) a few bytes of storage.

Your example is rather peculiar: the 3-byte elements have a 50% chance of individually being unaligned (to 32 bits), so aligning the buffer seems pointless—at least for performance reasons. However, in the case of a bulk transfer of the whole thing, it optimizes the first access. Note that an unaligned first byte might also have a performance impact in the transfer to a video controller.