In the absolute worst case, a totally malicious Signal Server can perform traffic analysis to correlate the IP address assigned to the messages arriving with the delivery token for a recipient.

However, this requires actively malicious, or actively compromised, Signal Servers in order to perform. If an attacker tries to retroactively determine who sent a message, and to which recipient, there are a lot of cryptographic mechanisms built-in that prevent the Signal Server from learning any of this information.

This serie of blog post was full of insight. Some cryptography are not yet understandable for me such as the double ratchet.
I will look forward to understand it.