I'm reading Charles Petzold's Code and would like to build and use my own very simple computer in order to internalize better some of the concepts. I was told about CARDIAC by a friend, but it seems it isn't available anymore. So, I'm wondering if any one has suggestions on similar kits which would help to get me acquainted with similar low level concepts, operations, etc.
You could go ahead and build your own Apple II or Commodore 64, but there are a number of hard to source parts required to build a fully pin-compatible version. That would be an educational project for sure, but it would be a huge undertaking.
What might be better to cut your teeth on is the Arduino platform, a microcontroller kit similar in power to computers of that vintage. There is a large community that supports it, and it interfaces with modern equipment using standard connectors such as USB and Ethernet. Some versions even have Bluetooth and Wi-Fi to make communication even easier.
If you program this device in assembly language, instead of the default high-level language, that's about as low-level as you can get. You can build up from a blank slate to explore all the fundamentals.
The arduino is inexpensive and wildly popular. The armmite pro, similar, ARM based instead of AVR (you will want to use the C interface probably). msp430 from ti...in fact a long list of microcontrollers that let you get your hands as dirty as you like. Maybe something like the stellaris eval boards which are loaded with peripherals that you can learn about reading by reading schematics and datasheets. Or something simpler like the AVR Butterfly if those are still available, enough onboard stuff to fiddle with without having to buy and solder things on.
A path that you can pursue at any time is simulation. The hardware design language (HDL) Verilog is not hard for software engineers to pick up and learn, you just have to think more in parallel execution than sequential execution. Verilator is a good free simulator that makes it very easy to connect the edges of your logic design to C/C++ so you can simulate things in C++ that you cannot or do not want to in Verilog. An HDL that I would like to see get more exposure is the cyclicy design language, cyclicity-cdl.sourceforge.net. Unlike Verilog and VHDL languages which both allow you to write synthesizable logic, that which can be turned into hardware, and behavioral logic, that which cannot but is used to test synthesizable logic. cdl is synthesizable only, the test benches you write are instead in a script language or in C/C++. Cdl code once compiled produces synthesizable verilog (for testing then turning into hardware) and C++ (for simulating/testing). Icarus Verilog is another good, free, verilog simulation platform, probably easier to use than verilator from a command line perspective but not as easy to integrate with C++ which I know I prefer, for testing the logic or just plain integration with the logic for educational purposes. You can get lower than assembler, with HDL you can create trees of connected or gates, use flip-flops, etc, directly implementing things discussed in the book.
Another approach is simulation without a hardware language, just write your own simulator of whatever type of system you like, simulate the logic, simulate a processor, etc. From personal experience I find that simulating, disassembling, etc teach you considerably more about the platform you are interested in than simple examination (reading about, etc) or writing programs for on someone elses simulator or hardware.
BTW thanks for dropping Petzolds name, I was about to buy this book online when reading your question, as I remember seeing it in stores, when I realized I had bought it when a book store was closing, buried it on my bookshelf without opening it. My programming windows 3.1 and windows 95 Petzold books are well worn from all the use back in the day. Flipping through this Code book to respond to this question, I am going to have add it to my stack of next to be read books.