Database inconsistency

After the individual users had been identified and fixed, and after no new cases have been reported by partners and customers, we consider the immediate issue leading to this incident to be resolved.

We will continue to closely monitor the situation and are following up with further analysis into the root cause and improvements to avoid such a scenario in the future.

A post-incident report will be published here in the coming days, depending on the further investigation progress.

A small number of domains had continued issues this morning which could only be found when users in these domains started to send mails again. The effects were error notifications with could not apply policy or rule engine reject. These domains have now been fixed and we can confirm successful handling and delivery of messages.

We are still aware of a number of individual users who may have continued issues. We are actively identifying and fixing such cases as and when we become aware of them.

Again, we sincerely apologise for the inconvenience caused.

Note: The incident was de-escalated from major to minor in one of the updates yesterday. We are now properly setting it to major again.

We noticed that ca 3% of users have some settings not yet restored. These settings are currently being restored.

Mail queues are mostly back to normal, with a certain number of mostly error notifications still pending delivery.

The assessment of the user impact has not been finalized yet. However we are aware that some messages have been rejected with a message of 555 User does not exist and "user_management" is not set to automatic. Unfortunately, affected senders would need to resend such messages.

We apologise for the inconvenience this incident has caused and will provide a more thorough post-incident report in the coming days.

We continue to monitor the situation and will provide a next update by tomorrow, Feb 3 2026 10:00 CET.

Most of the data restore is done, some additional items should be done in the next 15 to 20 minutes. We will then perform additional consistency checks on the data.

We see decreasing mail queues and will continue to monitor the situation.

The database inconsistency is mostly fixed, however an updates is still running to restore individual user settings to their original state.

We see that mailflow is resuming with some speed, however there are still quite a few error notifications in the queues which need to be cleaned out.

We see that messages are processed, however with some delay. We are investigating the root cause and working towards temporary or final fix.

Further, we are investigating possible negative side effects on messages which have already been processed.

Login to the cloud portal should be possible, however some features may be missing / not being shown.

We are currently investigating an issue of possible database inconsistency which partially impacts mailflow and portal login in the CH instance.