I'm studying about algorithms which can help me write smaller but more complex code. Instead of writing 150 lines of if-else statements, I can design an algorithm that does it in 20 lines. The problem is a lot of these algorithms can be complex and require a lot of math to understand them. I'm also the only one around here who understands them.
For maintainability of the code, would it be better to just write code like everyone else does, or is it better to use the algorithms?
Simpler is usually better. Remember that other people will probably have to maintain it someday.
Write the code in a way that is easiest to maintain and understand for the average programmer who works on your code.
Remember that code should primarily be understood by humans...compilers take care of making the computer understand.
The level of "complex" can be a little dicey here, but as long as the algorithm doesn't require a PhD in mathematics to solve,
I'd say go ahead and use the algorithms. Make sure to put a decent level of documentation about the name of the algorithm, and maybe a short description of how it works or a reference to a document about it.
It will keep the amount of code down, hopefully give a little extra performance to the application, and it will hopefully encourage a few programmers around you to learn some new things to add to their repertoire. Plus there's always the chance that a programmer that does know these things won't be coming in later on.
Unneeded complexity is always bad. Yes it could be fun to write a brilliant one liner that does the work of a 200 line function. But remember that you should maintain the code. And short complex code is hell to maintain.
It needs to be as complex as necessary. What it can't be is complicated, big difference.
As Einstein said:
Make everything as simple as possible, but not simpler.
It applies to code as well as to physics.
Use your judgement - will it get easier to maintain? Often by reducing large if/else messes into something smaller you remove corner cases that didn't need to be there, preventing bugs that may have popped-up in the future. Of course by reducing a clear set of conditions into an obscure twist of boolean logic that only happens to work, you may be making things much more difficult to maintain when something changes.
EDIT:
With the case you cited, if the formula is going to work, then it's probably the better option - you can even leave a comment citing the source. It's entirely possible though, that the formula was there before, but was dropped to allow a specific case to be worked around. This is the sort of thing comments in your versioning repository should help with.
Since no one's posted a link yet, here's a description of the PID algorithm he's referring to.
The risk of making it complex is that nobody else will understand it. In this case, comment it clearly and cite some sources that will help someone else learn and figure it out.
The risk of making it simple is that someone won't want to understand it because it is lengthy.
What's driving the need to make it more complex? Complexity for its own sake is a disaster waiting to happen. If you have a specific need for high performance, then you might need complex code.
I think the question is - Is the complex code any better? Is it faster, more reliable? The goal is to write the best programs, if the complex code is the best solution then you just have to write better comments so that your successor can manage the code in the future. In principal the code should be as simple as possible in any given situation.
Code is written once, and read 10 times. So you should try to make it as easy to understand as possible.
Also, debugging is much harder than writing code. So how can you ever debug your code when you already put all of your brainpower into writing complex code?
Just try to follow The Three Laws of Software Development:
The other risk of making the code simpler is that you'll get lumped into the pile of average coders. The line you're looking for is readable/unreadable code. If you can come back in a week and still easily read the code, then I would say it's simple enough.
Interestingly enough, one of the most common measures of code complexity is the depth of nesting. Your "If-then-else" solution may well be classified as more complex by an automated code analyzer than your formulaic one.
However, I've seen enough of the "WTF?" coding that you describe that I would generally go for the if-then-else approach. But consider this; is is possible to use your more complex, formulaic approach but to split out particularly difficult components into well-named methods? If you can do so (possibly even refactoring to eliminate redundancy) you may be able to avoid the worst aspects of your algorithm while avoiding a multi-level if-then-else structure as well.
Good old quote....
Any fool can write code that a computer can understand. Good programmers write code that humans can understand
Well, code can get pretty hard to understand and maintain just due to sheer volume if it gets large enough (see most Java code). If all else (performance, etc.) is equal and the difference in length between the simple and complex algorithm is really drastic, I'd always go with the complex but concise and elegant algorithm over the simple but verbose one. Assuming both algorithms are proven to be correct, the one with less if statements and less code overall is less likely to have subtle implementation bugs and therefore less likely to ever require maintenance. If I were a maintainer, I'd much rather spend my time learning a new algorithm than learning how someone implemented some ridiculously long but boring algorithm.
The Zen of Python does a pretty good job of addressing this issue:
...
Simple is better than complex.
Complex is better than complicated.
...
If the implementation is hard to explain, it's a bad idea
...
In other words (and as others have said), the simplest algorithm that will get the job done under the imposed constraints (time, memory, etc.) is the best one. Obviously a simpler algorithm that doesn't get the job done isn't good enough. If your code is shorter because it uses complex ideas, it's complex. If no one else can understand your code, it's complicated and if you have a hard time explaining it to them (even if they would understand it with a PhD in math), it's a bad idea.
...
Special cases aren't special enough to break the rules.
Although practicality beats purity.
...
There's often a lot of special case code that creeps into originally "pure" code during its lifetime. You should fight this building complexity, but accept it when it's necessary.
Depends. I wouldn't want to have to maintain 150 lines of if-then-else statements making up a function, particularly if the decision tree is dense. 20 lines of complicated math may or may not be better. It may take time to understand, but the longer solution might take even longer to verify.
Ideally, you'd find a way to do it in fewer and simpler lines, but not all functions work that way.
If you do use the 20 lines, use some of the 130 lines saved on comments to explain what algorithm you're using. If you can't explain it well, go with the 150-line solution.
It really depends on what complex means. If your "simple" algorithm is 150 lines of almost the same code, my eyes will glaze over and I won't be able to understand it. If you put those conditions in a matrix or something and your code is then a loop to read the matrix and make decisions, I will understand it better even though the loop may be less "simple" then a bunch of if/else statements.
If basically you are talking about a Perl program where you do everything in one line making huge use of the default variable $_ I would say stick with the longer more verbose version.
If there is an accepted formula to do something then you can use the accepted formula. If there is a simple algebraic formula that takes 3 or 4 lines and there is a complex differential equation that takes one line, you should use the simple algebraic formula. In the case mentioned in the comments, I think a PID Algorithm is "simpler" than 150 lines of if/else code. I don't think you are using the algorithm to obscure anything, but rather you are using standard techniques in the domain of the problem. Just make sure to comment it well and maybe even include a link to a web page describing the formula.
Finding the way through 100 lines of if/else statements is often (in my opinion) harder on the maintainer's stamina than spending the same time understanding a better algorithm (which should be explained or linked to in the comments) and finally just verifying that the 20 lines of the implementation do indeed execute that one. It also has the benefit of education on the job, which makes it that much more interesting (in a positive sense), and often also better execution profile (less use of resources).
The cleverness you should avoid is in the form of "clever hacks" that abuse the language without actual benefit. The cleverness you should embrace is always using the best algorithm for the job.
edit: Regarding the PID example: I find it very hard to imagine that the functionality of a PID can be sanely substituted by a bunch of if-else statements. The if-else solution will always work worse (less smooth transitions), be very hard to maintain, and very hard to tune (tuning the PID parts is very important to get the desired behaviour). I'd like to add that understanding PID is not too hard, even if you do not know the math, which can be easily looked up.
Complex does not necessarily mean more or less lines of code to me.
The perfect system is never built the first time. All you can do is try not to make too many complex decisions that tie you into doing things one way.
For that reason I like to keep complexity low during the initial versions of any project. The reason you built something (new flexibility) is severly impacted. If you make it as complex as possible fewer people will understand it at the beginning. That could be a good or bad thing.
If you make it too simple (and 50 to 70% more code) it may have performance issues.
As a system ages and matures, the complexity seems to come in through re-factoring. By then you can reach a point where some code may never be touched again, and if you ever do, the costs to understand the complexity will be lower due to the lower frequency of touching it.
I like solving complex problems with simple steps. Where it isn't possible, the complexity increases accordingly. There was a point in another question about knowing when its "good enough". Sometimes a bit more code (5-20%) can offset the complexity significantly, which may be more expensive to re-learn or understand by someone.
Needing a better algorithm is usually a good problem because it means your stuff is being used and theres new demands to be dealt with.
This is the same sort of complexity that applies to Database Abstraction for me, you have to know when to make it more flexible, and when to keep it simple, and its best learnt through building it, and scrapping it a lot before you write a single line of anything.
If you can do it on 20 lines and then properly comment those lines I would say go for it. Not only are you making it easier to maintain by having less code to maintain, but you are also contributing to making your fellow programmers smarter.
Pre-optimizations and clever hacks is one thing but smart algorithms is and have always been fair game.
But keep those algorithms isolated in their own functions and remember to explain the input and output variables!
Obligatory quote:
“Controlling complexity is the essence of computer programming.” (Brian Kernigan)
Complex algorithms are good only if the speed or space boost is actually necessary. Don't bother writing a fancy factorial algorithm that runs in O(1) because a factorial about 25 is basically never going to be used. However if the code is in the innermost loop and a speedup for 25% in this code will improve the entire application by 20% then go for it.
I've seen very optimized code run very slowly, and the unoptimized code run quickly, and that's because compilers know how the computer wants to work, and compiler writers focus on the easy to optimize code first.
It may be that your 150 line algorithm compiles into something that's much faster at run time than your 20 line version. Your 20 line algorithm may end up being slower because the compiler doesn't know how to optimize it.
I would also suggest that you might want to put the if-else version in a comment block above the 20-line algorithm version (or vice versa) so that maintainers can understand what your complex version is trying to do. Having them both in the code makes it easy to performance test the two as well (and once you've typed them both in, there's no reason to delete the other). This will also allow for easy migration to other languages/platforms/compilers going forward. You may not expect it, but if the code works, it's likely to live on for decades and see many different compilers and platforms.
Try and have your "more complex" version reviewed by others, a few weeks after you've written it. Judge the complexity on how difficult is it for you to read and explain it to others, and their reaction to the code.
I can't believe anybody thinks 150 lines of anything is simpler than 20 lines of something.
Go with the 20 lines, and protect your coworkers as follows:
Document every data-structure invariant and every nontrivial loop invariant. Document these not in comments but in actual code that checks the invariants. The check can be turned off during production.
If you use math formulas in your code or to derive in your code, include references in comments (or static strings). Ideally include two references: a Web reference that's likely to be available in seconds and a well-regarded textbook that's likely to stay in print a long time and be found in university libraries.
If understanding your code requires special expertise (understanding of partial differential equations, a degree in physics, Galois theory, you name it), then to protect yourself you may wish to go to your management and say "I have this special expertise which makes it possible for me to write code that is smaller (and therefore faster, more reliable, and easier to maintain), but when you have to replace me you are going to have to hire someone with similar expertise. What do you want me to do?" It will be helpful if you can tell your management how easily such expertise can be obtained. For example, lots of people with engineering degrees can learn partial differential equations, but Galois theorists are rather thin on the ground.
P.S. Here's an example comment from a hobby project so I can debug my own code later:
/*
* Compute the position of a point partially along the geodesic from
* pt1.lat,pt1.lon to pt2.lat,pt2.lon
*
* Ref: http://mathworld.wolfram.com/RotationFormula.html
*/
Anything that fits 40-50 lines could not border complexity (on 1280x800, Visual Studio default settings, default fonts has 47 lines). Sometimes we can learn from books better than reading online because books can contain more lines, moreso because we can see two pages at a time. Some programs which doesn't fit the screen, when printed, can be understood better.
I think it is important to separate the domain complexity from the underlying technical complexity.
Some domain-specific functionality for which you would like to use the computer is intrinsically complex, some is not. For example, accounting problems are full of weird rules, but most of the computations are fairly simple. Valuating financial securities on the other hand, depending on the type, can include hugely complex mathematical formulas and lots and lots of simulations. Most computer systems are really just collecting large amounts of data, but many of them have a few underlying complex algorithms.
Technology on the other hand often imposes its own complexity. Writing a big program in C++ for a PC can be a challenge. Writing a web based application for the Internet can be far worse. Trying to insure fault-tolerance, performance or security often causes a great deal of added complexity. Each different technology helps or hinders the system.
What we really want to do, is express the code in it's simplest possible form that is closest to its inherent domain or technical specification. If you're writing an operating system, then C is an easier language to use. If you're working out complex risk probabilities for insurance than a matrix-oriented language like APL may be more appropriate. If you're just creating massive reports, than something with a simple syntax oriented towards reporting is best choice.
So if your 150 lines of if/else stuff "matches" the way the problem is expressed far better than 20 clever lines, it is a far more maintainable and expendable bit of code. If you take a long-term perspective, writing running code is easy, it's keeping it that way that is the real challenge ....
Paul.