views:

1296

answers:

21

Possible Duplicates:
As a programmer without formal CS training or a CS degree, what am I missing?
How to make up for lack of a computer science degree?

I have 7 years experience as a computer programmer doing high level languages like C#, VB, Java and Ruby on Rails. I know these technologies pretty well, but I got into programming by accident and am totally self taught. I majored in business at university and never took a CS class, so I feel like I don't understand a lot of the lower level concepts like registers, gates, stack, heap, assembly language, algorithms, embedded systems, UNIX, functional languages, and so on.

Are there any good books or online resources that will help me fill in these gaps? The basic CS books I have seen are aimed at people with no previous programming experience at all. The ideal book for me would use the high level knowledge I already have to help explain the low level stuff.

+3  A: 

The ideal book for me would use the high level knowledge I already have to help explain the low level stuff.

Not sure many, if any text's exist that teach from this perspective, but I'm curious if someone has links or resources to one.

If you could find the patience and time to work through something like Assembly Language for Intel-Based Computers. I think it would open your mind to a lot of CS fundamentals. By Chapter 9 it starts to get Windows specific, but the prior 8 chapters are full of excellent information. Granted this is very low level.

I'm starting to stray here, but I just keep thinking back to really great books I've read in my travels:

The Art of UNIX Programming

Programming Pearls

Mr-sk
-1: Jon Bentley's 'Programming Pearls' is no longer a useful tome. http://stackoverflow.com/questions/406760/whats-your-most-controversial-programming-opinion/1330062#1330062
Jim G.
Jim: per (http://www.amazon.com/review/R3BISEOUSFRYDQ/ref=cm_cr_rdp_perm) is ONE person's opinion. So your now the authority on which books hold value and which ones do not? hah
Mr-sk
I just read the book review referenced above. What a load of crap. To address some of the statements in the review: My background is in biology, not computer science. I do not program in C or a dialect of it, I have no trouble with the examples. I have read **Programming Pearls'** cover-to-cover a few times. On each reading I gain deeper insight into the art of programming. Pearls are found in oysters, oysters are fundamentally hard to crack - but once you get to the pearl it is truly a thing of beauty. Cracking this oyster only means you have to *think and read* at the same time.
NealB
@NealB: Can you describe a time (within the past 3 years) when you incorporated the insight you gained from 'Programming Pearls' into your professional work? // Emphasis: I understand why 'Programming Pearls' *was* a useful tome. I'm asserting that Moore's Law, modern programming frameworks, and modern libraries have reduced the need for the principles and optimizations espoused in the book. // In Short - Should programmers be aware of various sorting techniques? Absolutely! Will programmers likely need to write their own List or LinkedList class and worry about optimizations? Probably not.
Jim G.
@Jim: the asker specifically wants to learn *low-level* details. In this context the objections you've raised against Programming Pearls are ridiculous. The asker is clearly interested in algorithmic efficiency and low-level implementation of data structures. +1 to counteract an absurd downvote.
Derrick Turk
@Derrick Turk: I'm arguing two different things to two different people. To @Dave Horton: Look elsewhere. You can find more readable books on those subjects. [I'd take issue with anyone who claims that 'Programming Pearls' is still the best of breed.] // To @NealB: I'm arguing that the book is no longer *generally* useful, because while the book discusses algorithms, the explanations apply almost exclusively to C. // FYI: I readily acknowledge that some programmers still program in this space (i.e. in C w/o libraries), but I'd argue that they number less than when PP was penned.
Jim G.
@Jim: You get a few things from Programming Pearls: 1) A down-to-earth approach to reasoning about and analyzing problems in computer science. 2) What the original question ask for - some insight into the'lower level concepts' used to develop 'modern libraries'. Here itit is the journey, not the destination that is important.
NealB
A: 

There is a great book Called write great code that will help to explain how the machine works. For CS topics the gold standard is The art of computer programming, but if you don't have some higher level math in your background you might find it a bit challenging. Reading the algorithms and pattern question here will give you a set of topics to research.

rerun
+14  A: 

If I had to pick one area that would give you the most coverage of the topics that you mentioned, it would be compiler theory. Writing a compiler will require you to understand how programming languages interact with the machine - what they are doing with the registers, how they use the stack and heap, how they translate to assembly language. A good compilers book will also provide a bit of coverage on functional languages as well, and of course you will need to also understand some type theory.

I would recommend buying one of Andrew Appel's Modern Compiler Implementation books and working through it (I am doing this right now). It walks you through building a fully-functional compiler step by step without quite spoon-feeding it to you. You will also need a supplementary resource on assembly language; there are many available.

IMO, the most important area that you mentioned not covered in writing a compiler is a good understanding of algorithms. For this, Steven Skiena's Algorithm Design Manual is very good for a beginner. My recommendation here would be to go through the book and implement the algorithms it discusses in whatever programming language you like. For more advanced reading, there is also the quintessential algorithms text Introduction to Algorithms (CLRS).

Edit: looking at the larger number of answers that have been posted (with surely more to come), I'd like to emphasize that "CS fundamentals" can encompass a very broad range of topics, depending on your definition of fundamentals. I would advise you not to get overwhelmed, and instead focus on one specific area at a time. Give yourself a project (one of the two I mentioned, for instance), work through it fully, and then figure out where you'd like to go next.

danben
+1: Building a compiler (albeit a simple one) was the best thing I've ever done to improve my CS/programming skills. If you want to really get into it, you should read the The Dragon Book by Aho, Sethi and Ullman.
Callum Rogers
excellent answer
Neil N
This! Do this. I completely agree.
unforgiven3
Absolutely. Compilers is all of computer science in one shrink-wrapped fairly simple concept!
just_wes
Pretty ironic for me. I have never taken a CS class in my life, and I recently became interested in the fundamentals of assembly as well. I'm working on a compiler *at this moment* (well, I was until I got distracted by this ;D) to become more familiar with this stuff.
Wallacoloo
A: 

I don't know of any specific resources to answer your question, but I can give you some pointers on where to look.

If you are interested in low-level concepts (registers, gates, stack, assembly language, embedded systems), you'd find most of the knowledge in a computer engineering curriculum that's heavy on electrical engineering concepts. You might try trawling the websites of great universities to see what lecture notes are available. Any of the books that I used in college would be overkill for what you probably want, so I won't recommend them.

For the algorithms, UNIX, and functional language topics, you can try the above strategy for the computer science curriculum. MIT has some particularly good podcasts.

LectureFox also has a lot of good resources, mostly videos and notes.

If you have specific questions about these topics, post them here and see what kind of an education StackOverflow can provide!

Pat
A: 

I agree with Mr-sk that Irvine's book on assembly is a good book. If you're interested in learning more about some additional topics such as computer architecture, and how linking, memory management, and some other more non-language specific parts of how computer science works, take a look at Computer Systems: A Programmer's Perspective. I think it's one of the best books I had as part of my undergraduate (and graduate) CS education.

jwir3
+3  A: 

This book is cheap, recent, compact, good, and has course material/code/emulators online:
http://www1.idc.ac.il/tecs/

You build circuits from logic gates, a simple computer, assembler, os, compiler for a high level language, etc.

To be clear, you build virtual hardware using the software simulators they supply on the book's home page.

z5h
+6  A: 

Work through Structure and Interpretation of Computer Programs. It will teach you everything that you're missing at the moment, and it's fun!

It doesn't assume that you know how to program, but it also won't treat you like an idiot. From your background, I think that virtually everything in there will be new and interesting to you.

Skilldrick
+5  A: 

Donald Knuth's The Art of Computer Programming will give you all the theory you can handle. I've just started reading it with hopes to learn theory as well and have found it surprisingly approachable.

In the introduction he recommends the book to anyone who has written "at least three programs". It is structured in such a way that all the hardcore math / proofs are at the end of the chapter so that they can be skipped if that isn't what you want to focus on. There are also huge amounts of exercises all rated based on difficulty and amount of time they should take to complete - so you can learn by doing if so inclined.

I found the clause about the recommended prerequisites on the first page of the preface: "These four prerequisites can perhaps be summed up into the single requirement that the reader should have already written and tested at least, say, four programs for at least one computer."

Winder
A: 

I will agree with Mr-sk about SICP. Be sure the read Peter Norvig's review on amazon to understand what you are getting into before you buy the book. Working through it is an challenging mind expanding experience.

rschuler
A: 

Echoing on a comment attached to this answer, I highly recommend David Malan's CS50 Video Podcast. [The production quality and editing is superb!]

I think it's superior to other competitive offerings in this space, and I think he (quite possibly better than anyone else on Earth) fulfills the promise of online education.

Jim G.
+3  A: 

If you live near a major university, you might inquire about being allowed to audit certain CS courses intended for junior undergrads. You'll learn a lot more hands-on this way.

Another alternative would be to acquire a syllabus for such a course at a school nearby where you could purchase the book and potential lab materials if necessary. You're going to find that your knowledge of high level programming is going to be moot when it comes to learning about the underpinnings of the architecture. However, once you learn the architecture, a lot of the "why this makes sense" junk is going to fall into place.

Joel Etherton
A: 

The ideal book for me would use the high level knowledge I already have to help explain the low level stuff.

I think you should stick with the 'beginner mind' and approach things as an (almost) absolute beginner. I found this to speed up learning. I'm also self taught (got a CS degree, but just because it seemed that piece of paper is useful, I didn't learn that much in 'official' studies).

As for a book recommendation, there's a book I tend to recommend a lot: SICP, http://mitpress.mit.edu/sicp/ (full text available online).

It is both high and low level - and I also believe it is hard enough to keep you entertained. As a bonus, your practical experience has probably taught you little of what they are describing. Just don't expect to be using what you learn from it directly - it teaches a lot of stuff that's indirectly extremely useful by changing the way you look at what you know or have to approach.

There are also some nice videos at http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussman-lectures/, but the essential experience here is reading the book and working on the exercises.

Miron Brezuleanu
A: 

IMO you're interested in too low level, that information isn't so much relevant to software development at all any longer unless you're going to work with embedded systems or write drivers perhaps. I would recommend learning original C instead so you can learn pointer arithmetic and memory management. That is far more important to understand especially since you've worked with managed programming languages that have handled all of the memory management.

Aside from that, object orientated design and understanding how to apply patterns and learn new tools and frameworks is substantially more relevant to modern software development than being able to code in assembler.

Chris Marisic
It never hurts to understand how the low level works, even if you can't apply it directly.
Ken Liu
I'm not saying it hurts just that the value would be minimal to a software developer except in very specific circumstances whereas understanding C is globally beneficial for almost any type of developer.
Chris Marisic
I've heard horror stories from a friend about a programmer who didn't understand the difference between "pass-by-reference" and "pass-by-value" -- or, really, how pointers work at all. It seems unlikely that the poster is one of these people, but I would definitely support reading a good book about Operating Systems -- virtual memory, file systems, all that.
khedron
@khedron yes exactly not understanding what a managed programming language is abstracting in terms of memory allows for very poor decisions to be made.
Chris Marisic
+2  A: 

Introduction to Algorithms by Cormen et al, is a decent and widely available text on algorithms.

comingstorm
Sedgewick's books (Algorithms in C, Java or C++) are as good with a more practical bent. Here's the C++ version for sorting and searching: http://search.barnesandnoble.com/Algorithms-in-C-Parts-1-4/Robert-Sedgewick/e/9780201350883/?itm=5
Joel
A: 

Try writing a language interpreter, or other complex program. This requires different types of components, and so much code, that you begin to see parts of your own code, as other programmers do, when you go revisit parts. You will see how having a clear design pattern for a component makes it extensible and flexible. You will need some form of source code version control as you put aside and resume parts of the program.

Write the interpreter in C. You can get the C Programming Language, by Kernighan and Ritchie. This book is a minimalist masterpiece. Get the oldest version you can get of this book, before ANSI C is included, and bloat begins to invade.

mrsmoothie
A: 

I think learning at least some assembly language is very useful in understanding how many languages work. I remember people having a lot of problem with things like pointers in C, but if you've done assembly language it all makes sense.

Eno
A: 

There are a lot of good suggestions (e.g. SICP), but most of them are pretty heavy (e.g. Knuth, Dragon Book). If you want to start with the heavy, definitely grab a book on Discrete Math and possibly algorithms first.

If you're looking to start a bit lighter, I found Write Great Code I and II to offer a great introduction to the concepts you're interested in, and it's designed for precisely for someone in your position.

Shannon Weyrick
A: 

I think the whys are very important to a self-taught programmer learning new tricks. You can't just read a book that says "do it this way" because the unconscious question is always there; why should I do it this way, instead of the current way I do it, when a new method will take longer? How do I integrate my strengths with a new methodology? What will I get out of doing so?

I can't suggest specific books at the moment, but if you vet your books by checking that they get into the reasons behind the theory and comparison between methods, you may have some luck.

quillbreaker
A: 

You might want to check out Write Great Code: Understanding the Machine. I own this book but have not yet read it, so I can't say for sure it meets your needs. The Amazon.com reviews speak highly of it however.

Joshua Carmody
A: 

For a completely different angle of attack, I'd suggest The Pragmatic Programmer. It's less about the hardware details, and more about being a software developer. Good stuff.

khedron
A: 

MIT and Berkeley (and others) publish podcasts of undergraduate CS courses.

For example http://webcast.berkeley.edu/course_feeds.php

You can find them on itunes.

Tim