This is more of a theoretical question than an actual problem I have.
If I understand correctly, the sequence number in the TCP header of a packet is the index of the first byte in the packet in the whole stream, correct? If that is the case, since the sequence number is an unsigned 32-bit integer, then what happens after more than FFFFFFFF = 4294967295 bytes are transferred? Will the sequence number wrap around, or will the sender send a SYN packet to restart at 0?
The sequence number loops back to 0. Source:
TCP sequence numbers and receive windows behave very much like a clock. The receive window shifts each time the receiver receives and acknowledges a new segment of data. Once it runs out of sequence numbers, the sequence number loops back to 0.
Also see chapter 4 of RFC 1323.
It wraps. RFC 793:
It is essential to remember that the actual sequence number space is finite, though very large. This space ranges from 0 to 2*32 - 1. Since the space is finite, all arithmetic dealing with sequence numbers must be performed modulo 2*32. This unsigned arithmetic preserves the relationship of sequence numbers as they cycle from 2*32 - 1 to 0 again. There are some subtleties to computer modulo arithmetic, so great care should be taken in programming the comparison of such values. The symbol "=<" means "less than or equal" (modulo 2*32).
Read more: http://www.faqs.org/rfcs/rfc793.html#ixzz0lcD37K7J
The sequence number is not actually the "index of the first byte in the packet in the whole stream" since sequence numbers deliberately start at a random value (this is to stop a form of attack known as the TCP Sequence Prediction Attack).
No SYN is required, the sequence number simply loops back to zero again once it gets to the limit.