God-Emperor
Deity
But to get 0xFFFD you don't need an FF and an FD character in the input stream. It is the replacement character that you should get for any invalid UTF-8 character.
UTF-8 is somewhat misnamed since it is not an 8-bit encoding. It is a variable byte count encoding. All valid 1-byte UTF-8 characters are actually 7-bit characters that match the 7-bit ASCII characters. Any other character is encoded with more than 1 byte. All valid 2 byte UTF-8 encoded characters start with a byte that is theoretically in the range from 192 to 223 (decimal; C0 through DF hex; in the format of 110xxxxx in binary where any combination of 0s and 1s for the xxxxx part is valid, except that decimal 192 and 193, where the binary values would be 00000 and 00001, are not valid bytes because the bits encoded can be done as a 1 byte character instead and therefore would be a 1 byte character) - note that these are, of course, all valid Latin-1 characters (mostly capital letters with accents). In order to be a valid UTF-8 encoding, the next byte must be of the form 10xxxxxx (binary). If a UTF-8 parser hits a 2 byte sequence where the first byte is in the valid range but the second byte is not in the valid format you will end up with the Unicode character 0xFFFD since that is what it is for (although some parsers use different values to represent "invalid character" values). So you can get the 0xFFFD character when none of the bytes in the incoming data stream is 0xFF or 0xFD.
It is also important that any single byte that is in the 245 to 255 (decimal) range is invalid in UTF-8. There is no way to encode anything in UTF-8 that will produce them so there is no way for a UTF-8 parser to parse them into a valid character. So if you try to parse some Latin-1 encoded data as UTF-8 and it includes a character in this range you will get the 0xFFFD character instead. One relatively common character in that range is "ö", which sometimes appears in English text in words taken from other languages.
There are also 3 and 4 byte encodings for characters in UTF-8. These also have specific values for what range each byte can be in and will also result in the same 0xFFFD character if the first byte in the sequence indicates it is in one of these two types but one or more of the following bytes is not the a valid range. The 3-byte values all have a first byte starting (high order) with 3 1s and a 0 (binary; 1110xxxx) and the 4 byte encodings with 4 1s and a 0 (11110xxx).
The extension bytes for all 2, 3, and 4 byte encodings are all in the 10xxxxxx (binary) format (if they are valid) except the 4th byte in a 4 byte encoding which has an upper limit to the value.
(UTF-8 as originally proposed also had 5 and 6 byte character encodings with the first byte having 5 and 6, respectively, 1 bits then a 0 bit followed by the data bits in the first byte, but they were done away with along with part of the 4-byte range. The original format could encode 31 bits of Unicode character, to match the original range of the Unicode character set, but it ended up being limited to covering the UTF-16 encoding's character set instead.)
Never had to do this stuff with Win 7?
