MS-OXMSG specifies the use of CRC-32 checksums without enough details

Question

MS-OXMSG specifies the use of CRC-32 checksums without enough details

Destruction Elemental 96

(Going to quickly preface this by saying sorry if this is the wrong category, please point me to the right one if it is not there already)

I'm working on implementing parsing on .MSG files, and am currently looking at the specification in MS-OXMSG, and it specifies that for String Named Properties the Stream ID uses the CRC-32 of the Unicode name. However, I have been unable to find a combination of Unicode (a very broad definition) and CRC-32 algorithm that matches the results I have found for what they should be (using process of elimination I have deduced the correct Stream ID which contains a copy of what the CRC-32 should be).

Can someone please provide the details for the CRC-32 algorithm used as well as which form of Unicode (I.E. UTF-8, UTF-16LE, UTF-16BE, etc.) the name should be in before calculating the CRC-32 of it?

Anonymous

2021-10-02T02:48:10.397+00:00

Hello DestructionElemental,

Thank you for posting your question. One of our protocols engineers will respond soon.

Best Regards,
Jeff McCashland
Microsoft Open Specifications
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-04T16:27:50.75+00:00

Hi @Destruction Elemental ,

I'll assist with this question and get back to you shortly with information.

Best regards,
Tom Jebo
Sr Escalation Engineer
Microsoft Open Specifications
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-04T17:11:08.923+00:00

@Destruction Elemental , a couple of questions. What version of Outlook created the .msg file and did you take into consideration the following behavior note:

<1> Section 2.2.3.2.2.1: If the string named property belongs to the PS_INTERNET_HEADERS
property set ([MS-OXPROPS] section 1.3.2), then the Office Outlook 2003, Office Outlook 2007,
Outlook 2010, and Outlook 2013 implementations of the .msg File Format will convert the Unicode
property name to lower case before computing the equivalent CRC-32 for it.

Tom
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-07T17:01:07.663+00:00

@Destruction Elemental , it looks like we're using the lookup table from [MS-OXRTFCP] 2.1.2.2.1 CRC Lookup Table in computing the CRC for the string property ID in [MS-OXMSG] 2.2.3.2.2.1 Stream ID Equation. Could you give that a try and let me know if that yields the correct result? Remember to take into account the lower case note for the versions of Outlook indicated in the above behavior note.

Tom
Destruction Elemental 96 Reputation points

2021-10-31T23:17:42.783+00:00

Appologies, it appears I am not getting any notifications here which I will have to look into. Yes, I took into account that they would be lowered, and tried encoding into many different encodings and using a combination of upper, lower, no change, etc. I tried UTF-8, utf-16 (LE and BE), utf-32 (again, LE and BE) but stopped there after not being able to get the result using a multitude of crc32 algorithms.

I'll check the documentation you specified now.

Answer accepted by question author

0 additional answers

Your answer

Anonymous

2021-10-02T02:48:10.397+00:00

Hello DestructionElemental,

Thank you for posting your question. One of our protocols engineers will respond soon.

Best Regards,
Jeff McCashland
Microsoft Open Specifications
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-04T16:27:50.75+00:00

Hi @Destruction Elemental ,

I'll assist with this question and get back to you shortly with information.

Best regards,
Tom Jebo
Sr Escalation Engineer
Microsoft Open Specifications
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-04T17:11:08.923+00:00

@Destruction Elemental , a couple of questions. What version of Outlook created the .msg file and did you take into consideration the following behavior note:

<1> Section 2.2.3.2.2.1: If the string named property belongs to the PS_INTERNET_HEADERS
property set ([MS-OXPROPS] section 1.3.2), then the Office Outlook 2003, Office Outlook 2007,
Outlook 2010, and Outlook 2013 implementations of the .msg File Format will convert the Unicode
property name to lower case before computing the equivalent CRC-32 for it.

Tom
Tom Jebo 2,451 Reputation points Microsoft Employee Moderator

2021-10-07T17:01:07.663+00:00

@Destruction Elemental , it looks like we're using the lookup table from [MS-OXRTFCP] 2.1.2.2.1 CRC Lookup Table in computing the CRC for the string property ID in [MS-OXMSG] 2.2.3.2.2.1 Stream ID Equation. Could you give that a try and let me know if that yields the correct result? Remember to take into account the lower case note for the versions of Outlook indicated in the above behavior note.

Tom
Destruction Elemental 96 Reputation points

2021-10-31T23:17:42.783+00:00

Appologies, it appears I am not getting any notifications here which I will have to look into. Yes, I took into account that they would be lowered, and tried encoding into many different encodings and using a combination of upper, lower, no change, etc. I tried UTF-8, utf-16 (LE and BE), utf-32 (again, LE and BE) but stopped there after not being able to get the result using a multitude of crc32 algorithms.

I'll check the documentation you specified now.

Answer 1

Destruction Elemental 96

Thank you @Tom Jebo for providing the location for the information about the CRC. To be specific, when converting to Unicode it appears (based on my first every successful crc32 computation that matched what was found in the files) to be using the UTF-16-LE format to represent it and then using the CRC32 lookup table defined in [MS-OXRTFCP] 2.1.2.2.1 CRC Lookup Table.

I do still believe this to be an oversight that this data was not specified in the section where it would be very relevant, but I can now finally begin implementing the necessary code for this.

DK-6481 21 Reputation points

2022-02-21T18:20:46.253+00:00

I stumbled across this while researching something Exchange protocol-related, and I wanted to mention that the default encoding for the .NET String class on Windows is UTF-16LE, which might be why that's the encoding that worked.