Databases use various disk-based data-structures, such as B-trees, Linear-Hash-Tables, and Heap Files, to store data on disk. It is an incredibly complex topic that requires a lot of study, but essentially depending on the type of data you want to store (whether it is fixed length or variable length), a common strategy used by many databases is to store variable length data on pages stored in a heap file.
The heap file is basically a sequence of fixed-length pages, each page containing slots for different records. Pages are stored on disk, but are loaded into memory using a paging sub-system, similar to the way a file-system works. In order to facilitate quick lookup of records stored on pages, an index structure (usually a B-tree or B+tree, but also possibly a hash table) is used to find the correct page and record number of the record you're looking for stored in the heap file.
The SQL query-engine is layered on top of the underlying data structures. The query engine parses the query, and then looks up the appropriate records using indexes. (There's many other facets involved here, but that's the gist of it.)
Needless to say, implementing a relational database in C++ (or any language) requires an enormous amount of work. If you're really interested, I would recommend reading Database Management Systems.