Tools for understanding large codebase

Question

Hi!

My whole life I have been programming in simple plain text editor. Lately, I was contemplating about joining an open source project which is fairly large and written in C.

I downloaded the sources, started to look around, read this, forget that...

Then I thought to myself: this can't be true. This is 21st century there must be some tool which would help me to understand the code, perhaps some kind of IDE or "code navigator". What flows from here to where, this typedef struct is just interface to that private type, this function is just #define from above, function called in this file is defined in that file, ... you get the idea.

Dear Stack Overflow, is this 21st century? Is there something like this?

Answer 1

Many IDEs have an option to "go to definition" when you right click a keyword, like a function call or variable. This will take you to where the function or variable was declared. You can also use a search to find all instances of that keyword. This is helpful in understanding code in that you can incrementally understand what is happening by exploring each function.

You can also use a debug mode to step through the code line by line, "stepping in" to the definition of function calls, and view the values of variables. This can help you gain an understanding of a code base.

However, if the code base is using patterns, technologies, or APIs that you are very unfamilier with, you might be overwhelmed to the point that you make very little headway in understanding the code base. You may need to take time working with and playing with those concepts on a small project of your own making, and then come back to the open source project later.

Answer 2

While IDEs are the current way of finding definitions, etc. the way it was done in the 1980s was using the visual editor vi to navigate a tags file generated by a utility like ctags.

In vi, the command keystroke Ctrl-] goes to the definition of whatever symbol is under the cursor if it is defined in tags. Ctrl-^ goes back to the previous file and position. Modern versions of vi build on this in useful ways.

Create a tags file and begin editing whatever module contains function main() with

$ ctags *.h *.c */*.h */*.c */*/*.h */*/*.c  (etc.)
$ vi -t main

Answer 3

IDE's are basically text-editors on steroids and for programmers. IDE's just help you navigate your code much MUCH easier then just using the text editor, like for instance:

• Autofinish redundant lines
• Some might include debugger's (which are EXTREMELY helpful)
• color's/ highlights keywords for easy readability

I went from an IDE to a text editor an didn't know where to go. I just kinda looked a my code and got lost, mainly because I couldn't destinguish any words from each other. So from there... I went right straight back to the IDE. I am actually using X-Code but it really doesn't matter which one you use. I would just suggest you really consider getting one and making sure you feel comfortable with that one and it will be your best friend forever.(bff)

Answer 4

Cscope is nice for navigating large C code bases as it generates an index (like ctags does), so you have an upfront cost to generate the index, but it can give you a list of search results almost instantly (except if you are searching for certain things like literal strings).

Unlike ctags, Cscope can be used via an interactive textual user interface, so you can get good results without having to move to another editor; if you do want to move to another editor/IDE, Cscope integrates with Emacs at least.

The "screen shot" at the Cscope page linked above will give you an idea of the kind of searches you can perform and the results you get; in that screen shot, hitting one of the digits 0 through 6 would open your editor (selected via an environment variable) on the specified file, at the correct line if possible.

Edit: Doxygen can also be nice for understanding C code bases. It will give you a set of static HTML files and will create links wherever names it recognises (e.g. functions) appear. It is best if the code base has Doxygen-style /** ... */ comments, as Doxygen can format these specially, but even without these, you can configure Doxygen to generate pages with the source code for each file marked up with links to other functions, etc. If you use Emacs, Doxymacs is useful - when the cursor is in the middle of an identifier, you can use the key sequence C-c d ? to load up your web browser with the Doxygen documentation for that identifier.

Answer 5

I personally (without any affiliation) recommend the CppDepend/NDepend/XDepend series of tools. If you want to understand how your code is structured from a high level, what the dependencies are between components, and get a bucket full of metrics then it is invaluable. I own a copy of NDepend and couldn't be happier with it...

CPPDepend

NDepend

Answer 6

If you feel uncomfortable using sluggish IDEs, you could always use a powered up text editor like vi or emacs.

To navigate the source code you can use emacs along with ctags that scans your source code and indexes the symbols into a TAGS file.

To generate a TAGS file, do this in the root of your code tree (stick this in a script or Makefile):

#ETAGS=/cygdrive/c/emacs-21.3/bin/etags.exe

ETAGS=etags # Exuberant ctags

rm TAGS

find . -name '*.cpp' -o -name '*.h' -o -name '*.c' -print0 \

| xargs $(ETAGS) --extra=+q --fields=+fksaiS --c++-kinds=+px --append

Some of the features that you get is

• M-. goes to the symbol definition
• M-0 M-. goes to the next matching definition
• M-* return to your starting point
• M-x tags-search initiate a search
• M-, go to the next match

Ebrowse, GNU id-utils are two other ctags alternatives

Answer 7

you could consider Eclipse with CDT..never used it personally though (I'm not an active C programmer).

Answer 8

I use Visual Studio and Source Insight

Answer 9

Maybe this product may help: http://www.scitools.com/products/understand/

I used it for large C/C++ projects and it seems to be pretty good.

Answer 10

I really like Structure101: http://www.headwaysoftware.com.

Answer 11

Concerning .NET code, you can use NDepend to re-engineer any large .NET existing code base; NDepend comes with a dependency graph, a dependency matrix, a treemap/metric view, and a Code Query Language dedicated to explore the code base, see a few screenshots below:

Answer 12

For large .NET applications analysis I developed Runtime Flow - showing real time interactions in code as your program is running.