Federation has become the established term for an old concept, the idea that everyone should run her own server and applications connect the appropriate servers as they need,
just like SMTP and HTTP have always done.
By now federation is heavily criticized for its habit of exchanging massive amounts of private data on servers in the clear. This could theoretically be reduced, but it doesn't really happen, as it's trying to introduce privacy as an add-on. That has never worked, privacy needs to be planned for by design.
Even if end-to-end encryption were properly provided on all levels, the metadata of who is friends with whom and subscribed to which pubsubs, revealing each user's personal or business interests and social network, remains more or less easily available for surveillance agencies. As discussed on the Matrix page, the more servers participate in a pubsub, the more likely one of them is under surveillance.
Another problem of federation is that it implies users having some trust in their choice of server, but most users are terrified at the idea of having their school, company or nerd friends get access to their personal data thus they usually choose some large company's offering. So when your significant other will talk with friends about you, he or she will rather use Facebook than your server – doesn't matter how much you promise your server doesn't read into the messages. So, all in all, federation doesn't even protect those who use it.
There's also a problem with the scalability of federation protocols. As shown in the list below most have missed the chance to properly support multicast and thus function efficiently for everyone. This opens up an opportunity for large cloud technology offerings to improve on message distribution internally and thus offer a more speedy service than the free Internet. As an example, Google is likely to be able to distribute a mail message to a million Gmail recipients in less than a second while your average linux server will be busy for hours or days. This is because cloud technology has multicast trees inside while SMTP hasn't. In theory (if PSYC had become popular maybe), multicasting over federation is possible – but the cloud industry has a strong business interest in this not happening, combined with a lack of understanding of the issue by the majority of Internet developers.
For the two preceding reasons, scalability and perceived privacy, federation technologies have always degenerated into a slippery slope leading people from free services into the dependency of centralized offerings. It comes as no surprise that at some point the so-called open standards lose their relevance and the big companies procede to lock their users in for good. The open XMPP federation is nearing this condition, whereas with email Google is contempt to host the largest number of participants.
It is time to understand that the antique ideal of federation is instead a very bad ideology and needs to be done with.
Still some developers view backwards compatibility with the existing infrastructure more importantly than privacy and constitutionality of communications. We don't have hard numbers, but any attempt to ask end-users whether they care about backwards compatibility to e-mail, Facebook or XMPP has resulted in end-users being definitely more concerned about metadata privacy and welcoming a new system that completely breaks with the past, anytime. This we intend when we say, we'll make ourselves a GNU Internet.
1 Time to move beyond federation
PSYC developers have decided to move on and are working on a fully distributed (not just decentralized) and end-to-end privacy-enabled solution in secushare, which operates on top of public-key routing technology. That means, PSYC as a syntax and protocol lives on, but the federation architecture is gone. This addresses all the issues above. See PSYC2 for more on this ongoing project.
2 History of chat federation
In the domain of chat and messaging running your own server isn't enough. You almost always end-up requiring an interserver protocol, which has led to different constellations:
- IRC - a system where all servers need to trust each other, so they cannot openly federate and let everyone join in with their own server
- NNTP, which isn't realtime. Other than that it was pretty neat.
- PSYC, which promoted the concept of federated servers since 1993 (the [synconf] paper discussing problems of IRC and how to address them) or 1991 (brainstorm session with BuGless at the Aachen relay party).
- XMPP (Jabber), which in 1998 took a much simpler approach leaving out the multicast concept and just got going.
- Centralistic systems occasionally offered federation to those willing to pay for it using the SIP protocol.
Related Issues: