"TenantInboundAttribution" rejections for Inline mode customers

Minor incident Mailflow CH Instance Mailflow EU Instance CH Instance EU Instance External Services M365 Mailhosting
2024-05-17 12:00 CEST · 4 days

Updates

Retroactive

Due to an unannounced change by Microsoft, various seppmail.cloud customers using M365 as the target server in inline mode had some inbound messages rejected starting Friday, May 17, 2024 during daytime and ending Tuesday, May 21, 2024 before noon. The error message contained the keyword “TenantInboundAttribution”.

Messages rejected during this time due to “TenantInboundAttribution” will need to be resent.

The problem did not affect all customers, and not all customers from the same point in time. Customers were affected in the order in which Microsoft rolled out the change.

Microsoft’s change was not announced. The symptoms initially suggested a (temporary) Microsoft problem, rather than a change in behaviour. The suspected temporary problem could be countered with equally temporary measures (removing entries from the recipient cache, see technical details below).

Since it was a staged rollout, even different systems assigned to the same customers were affected at different times, which made the analysis more difficult. When it became clear that this was not a temporary problem but that a change in behavior, the actual fix could be developed, tested and put into production by the morning of Tuesday, May 21, 2024.

Due to the slowly developing situation we unfortunately missed creating an incident report. We apologize for the lack in communication and we will revise our process in this regard to enable timely and transparent information under such circumstances.

Technical details

When receiving messages, the seppmail.cloud system must know whether a recipient address exists. This is realized with the so-called call ahead system, which collects the information (“recipient exists (does not)”) from various sources. Due to the behaviour of (especially) ExchangeOnline - under certain circumstances, ExO does not provide reliable information after the MAIL FROM - this system has some surprising complexity.

The result of the call ahead system is stored in the so-called recipient cache. This cache can be viewed by seppmail.cloud administrators in the domain settings (see https://docs.seppmail.com/ch/cloud/c06_scf_03_sped__settings-per-email-domain.html#inbound_recipient_cache).

One of the elements in the Call Ahead system establishes a connection to the target system (M365 / ExchangeOnline) independently of the message delivery in order to check the recipient address. With Microsoft’s change from Friday, May 17, 2024, a tenant-specific instead of a general certificate is now required for this connection. Otherwise a “user unkown” response will be detected.

With our change on May 21, 2024, the tenant-specific certificate is now used for the call ahead connection as well as for message delivery.

May 22, 2024 · 14:04 CEST

← Back