Let's consider symmetric encryption. This is how you convert some data into something unreadable, except for those who have some secret knowledge which enables to reverse the operation. This is "symmetric" because the knowledge needed to encrypt is the same than the knowledge needed to decrypt.
Historically, the "secret knowledge" consisting in the description of the steps to be performed to scramble the data. At some point, it became interesting to split the "encryption method" into two parts, one being the algorithm (the generic description of the method steps) and the second being the key (the precise parameters used in that occasion). On a computer, the algorithm becomes the program which the machine runs; the key is a numeric parameter, a sequence of zeros and ones. Computers exacerbate the need for that split: a computer program is very hard to keep confidential. A program is bulky, it is written to a harddisk... on the other hand, a key is short enough to fit in a human brain, to be typed when used, to be stored in a device as puny as a cheap smartcard, and so on.
In a way, the key concentrates the secret. Everything which is secret in the encryption lies in that key.
In you XOR example, the method is then: "we XOR the data with the key" and the key is the actual sequence of zeros and ones with which to XOR. The encryption-by-XOR method is known to be extremely difficult to use properly (basically, the key has to be as long as the data to encrypt, which is very often highly impractical; but using a shorter key means reusing some parts of the key, which opens many deadly weaknesses). Usual symmetric encryption systems are much more complex, and strive at providing "your money worth" from the key.
How much a key is worth ? Well, since the key is the only part of the whole thing which is secret, all the attacker has to do is to guess the key. The most basic form of guessing is exhaustive search: the attacker simply tries all possible keys. The number of possible keys grows quite fast with the key length. For instance, a 10-bit key has 1024 possible keys. With a 20-bit key, there are 1048576 possible keys. Each additional bit doubles the number of possible keys.
An algorithm is deemed "robust" if the best known attack is the exhaustive search. In that sense, a robust algorithm is an algorithm which offers no better attack path than the brutal exhaustive search, which is always "doable" but can be prohibitively expensive. If the algorithm is robust and the used key is large enough to thwart exhaustive search, then the encryption is secure. "Large enough", in practical terms, means "a hundred bits, or more". The largest key which has been attacked by exhaustive search is a 64-bit key. The number of combinations was huge (18446744073709551616) by could be achieved with existing technology (it took several years and thousands of computers). The continuous progress in computer speed makes exhaustive search better and better, but not as fast as could be imagined. Basically, exhaustive search increases by 1 bit every 12 to 18 months (this is known as "Moore's law"). There are many good reasons why Moore's law cannot be upheld beyond about year 2020 (that's what Gordon Moore was predicting himself, actually), at which point computing power for a given price will somehow stagnate. A 100-bit key should then remain safe (that's way too many combinations for what computer will be able to do in 10 years). Usual encryption algorithms use a 128-bit key because "128" looks good when written in binary.
There are also asymmetric encryption, and digital signatures. The basic principle remains the same: the key is the part which is secret, as opposed to the method (the program) which is known by everybody. Asymmetric encryption and digital signatures try to do complicated things, in which, for instance, someone who knows how to encrypt does not have enough information to know how to decrypt. This requires some elaborate mathematical structures. Keys are then mathematical objects, which need more bits when they are encoded. There again, the number of possible keys grows with the key size, but not as fast, because very few sequence of bits will have the required mathematical structure. Also, that very same structure allows for faster attack paths, implying then again larger keys to achieve security. This is why we hear about things like "a 1024-bit RSA key". A RSA key consists in a few integers, the most important being such that it should be impractical to factor it into a product of prime integer. A 1024-bit integer (an integer which, when written in binary, consists in 1024 zeros and ones -- that would be a bit more than 300 digits in decimal) seems big enough to thwart factorization (current record is for a 768-bit integer).