Coding Pratice: What are your thoughts on a 1.7 Million LOC project?

Question

I am listening to a panel discussion where a person mentions their 'engine' is not 1.3 but now is 1.7 lines of code.

That frightens me. I cant imagine that, the amount of modules, etc. I always felt that C++ doesnt handle modules as well as other languages can. I felt large projects are harder to manage and preferred to reasonably keep lines of code down. My question is

What practices do you have while developing and maintaining projects of this size? I feel discomfort when i hit 10k, i cant imagine how 20k, 50k, 500k nor 1 million would feel like.

Answer 1

What practices do you have while developing and maintaining projects of this size?

Divide and conquer, thus not having a monolithic project of this size.

Answer 2

For me it hasnt been the number of lines, but how modular the design is, how well encapsulated the modules are. After a certain point, if I can zoom in so to speak on a module, figure out what its design is, and write features and fix bugs, then number of lines of code doesnt matter. Arguably I havent worked on a system bigger than 1 million lines of code.

Answer 3

What practices do you have while developing and maintaining projects of this size?

Well, that's when you evolve from developer to architect.

With large software project, a project leader's concern shouldn't narrowed at implementation but on structure level: Properly and correctly modularise your components/libraries, well decoupling them, utilise design patterns.

Answer 4

What type of Engine is that? If it's a Game Engine, it's surely modularized into Graphic, Sound, Physics, Map Handling and dozens of other modules. Every module surely contains several submodules, i.e. Graphics is split into Font-Rendering, Effects, GPU-specific parts etc.

Answer 5

I guess it depends. You could give a bad chef the same amount of ingredients (i.e. lots) as a good chef and the bad chef will give you something that will make you throw up whereas the good chef can make it taste good and be easily digested.

I don't think of a project in terms of number of lines but (as you have eluded to) the maintainability of the project itself. As long as the project is modularized and well refactored it might not be too bad.

So to answer your question, practices that I use to maintain a large code base would be the same one to maintain any size code base - I would look at integrating an automated build with a high code coverage (preferably with system level tests), include tools to help you have readable (e.g. checkstyle) and no duplicated code (e.g. simian) and things such as that.

If the project continues to grow and you reach a large number of lines and you develop the code base properly, I guess it can only mean that your clients are happy and want more features.

Answer 6

One million lines of code is past the point that most mortals can keep it all in their heads. That means that team members will each be carrying around incomplete mental maps of the system, which can make design discussions difficult.

To mitigate multiple, incomplete understandings, you need maps, in the form of an appropriate set of architectural diagrams. These will usually include a very high-level block diagram of the system's architecture, with more detailed lower-level diagrams for key parts, and possibly sequence diagrams for describing key interactions at an appropriate level of detail. Having such diagrams within reach help the team be "on the same page" when discussing the system.

'Dependencies between subsystems' diagrams can also point out areas of messiness (of the "Wuh? Why is that bit of the persistence framework dependent on the UI?!?" type) that need to be cleaned up. Best if if you can figure out a way to automate the generation of these diagrams. Graphviz can be your friend.

Answer 7

What practices do you have while developing and maintaining projects of this size?

There are many school of thoughts for managing large software projects, and obviously it largely depends on the type of project:

• Funded software development (that might cover some Open Source projects to which corporation are contributing, think Eclipse),
• "Hobbyist" software development that's been wildly successful (think Linux).

Funded software development will typically tend to use formal process from the beginning, where "hobbyist" project tend to adopt only the required practices to address paint points as they appear.

As it has been pointed out by others a project of that size is necessarily a team effort, and a lot of the challenges faced come from the need to coordinate activities and limit chaos.

But they will all tend to use the same "tactical" practices to handle the complexity. I suggest you read this introduction to the Joel Test to get a sense of some useful/required practices.