Microsoft fixes Y2K22 Exchange bug that disrupted email worldwide on New Years

Lavanya Rao K

January 12, 2022

The turn of the new year triggered errors in Microsoft Exchange mail servers, causing thousands of emails around the world not to get sent and staying stuck on email transport queues causing entire servers to crash. The server administration community dubbed this bug “Y2K22” due to its similarity to the infamous Y2K bug. This date-related bug was feared to cause many computers to collapse at the turn of this century. Microsoft Exchange 2016 and 2019 servers have an integrated malware-scanning engine called FIP-FS. The engine uses a signature file that stores dates in a 32-bit integer format. The largest possible number that can be stored is 2147483647. It worked fine for the dates in the year 2021 because it was 211231XXXX. However, with the turn of the new year, January 1st, 2022, it got converted to 2201010001, but this was larger than the maximum number allowed when trying to format in 32 bits, causing the date/time validations on the server software to fail. This led to the piling up of emails on servers. Exchange administrators discovered one quick fix: to disable FIP-FS since it left servers vulnerable and open to attacks unless the servers had another external antivirus software running. The next day, Microsoft released a fix. It came in two forms: an automated PowerShell script or a manual solution that didn’t work correctly, as reported by some admins. In either case, the fixes had to be performed on every on-premises Exchange 2016 and Exchange 2019 server inside an affected organization. The automated script can run on multiple servers in parallel. This fix is not an automatic update and requires some action from its customers. Microsoft states that this is only a short-term fix that sets the date on the signature file as 2112330001 (December 33rd, 2021), which lets the server continue to operate normally, despite not being an actual date, while Microsoft develops a longer-term solution. The software maker said the automated script might take some time to run and requested admins to be patient.

Abridged fromIt's Foss News