Recently, The cybersecurity, development and technology communities were taken aback when it was disclosed that a backdoor (CVE-2024-3094) had been uncovered in XZ, a widely employed open-source data compression utility found on most Linux systems. This vulnerability enabled threat actors to establish remote connections and execute commands on affected systems with administrative privileges. Given the critical reliance of many internet infrastructures on Linux, the discovery of this backdoor was of significant concern and could have led to a major security breach if not detected promptly.

I will discuss the backdoor's impact, exploit details and mitigation strategies in this article.

Vulnerability Unveiled

On March 29, Microsoft engineer Andres Freund published the details regarding the Backdoor affecting. xz/liblzma.

Post on Mastodon by Andres
Further disclosure detail can be found here.

Initially, the Backdoor was believed to be an SSH authentication bypass. However, further analysis revealed a more severe threat: Remote Code Execution (RCE). This grants attackers complete control over the affected system, potentially compromising sensitive data and disrupting operations.

How the exploit works:

This extremely sophisticated exploit has multiple stages and involves multiple steps chaining throughout the process.

  • A series of malicious code disguised as test files would be invoked at some stage of the build process by a script m4/build-to-host.m4 . The output of this script de-obfuscates the cleverly hidden malicious code and executes it.
The de-obfuscated code
  • The above code, once executed, extracts hidden data from a compressed file good-large_compressed.lzma and prepares it for execution by applying specific steps to decode and decompress it.
  • The resulting decompressed data is an elaborate bash script (infected.txt as attached in original disclosure), which checks for a few conditions, i.e. whether the system is x86-64 Linux or not, whether the build process is manual or a part of package build (Debian or RPM), etc. If conditions are met, the compilation process is modified to include the Backdoor; if not, the backdoor installation process is simply skipped.
  • The Backdoor hooks the PLT (Procedure Linkage Table), specifically the one used for SSH login RSA_public_decrypt@....plt , to point to its own exploit code. This allows the Backdoor to run its malicious code whenever SSH authentication is attempted.
  • The Backdoor only activates if the login attempt includes specific data sets signed and encrypted by a private ED448 key controlled by the attacker. Once the payload is decrypted, it is executed with root privileges in the host system.

For detailed technical analysis, refer to the following resources
- https://gynvael.coldwind.pl/?lang=en&id=782

The Heist: How It Was Pulled Off

Now that we know how the exploit works let's discuss how it was pulled off on one of the most popular open-source projects. If not for its early discovery, this could easily be one of history's most devastating Supply Chain attacks. This attack was planned and executed over years of meticulous planning and deception and involved multiple steps.

Originally the concerned project was being maintained solely by Lasse Collin. Being the sole maintainer of such a huge project (unpaid hobby) for years understandably burned him out. Taking this as an opportunity, the attacker, with Jigar Kumar and Dennis Ans persona, began pressuring the maintainer into accepting another person as the maintainer, presumably Jia Tan, to reduce the burden and further boost the project's development.

The whole thread for this mail interaction can be found here.

Here, in these emails by Lasse, we can also see Jia Tan being referred to as a possible maintainer in the future.

After these discussions and pressure, Jia Tan was finally appointed maintainer in early 2023, and he merged his first PR on January 6, 2023.

Outside of the above XZ discussion, these two persona i.e Jigar and Dennis is never seen again.

After gaining trust and gaining privilege, Jia Tan started slowly unloading the exploit infrastructure required for the Backdoor.

Testing Infrastructure required for Backdoor - I
Testing Infrastructure required for Backdoor - II
Testing Infrastructure required for Backdoor - III

As seen above, the final piece required for the Backdoor was added to the repository on March 9, 2024. Then, the poisoned tarball was released as versions 5.6.0 and 5.6.1.

Timeline of Events

The Timeline of events leading up to the discovery of the vulnerability.
The Timeline of events leading up to the discovery of the vulnerability.

An attacker, operating under the alias Jia Tan (Jia Cheong Tan or JiaT75) on GitHub, gained the community's trust. Here's the timeline:

  • Aug 2021: Jia Tan creates a GitHub account.
  • Late 2021/Early 2022: Jia Tan contributes to related open-source projects, building credibility.
  • Early 2022: Jia Tan joins the XZ project mailing list and contributes minor code.
  • Jan 2023: Jia Tan eventually earns maintainer privileges for XZ Utils.
  • Jun 2023 - Mar 2024: Jia Tan begins unloading the exploit infrastructure disguised as test files.
  • Feb 2024, Mar 9, 2024: The malicious code is introduced into versions 5.6.0 and 5.6.1.
  • Mar 29, 2024: An engineer stumbles upon the exploit while benchmarking Postgres and notices an abnormal spike in CPU usage caused by SSHD processes.
Post on Mastodon By Andres regarding the Finding
For detailed timeline of events refer to this great post.

Checking Exposure and Mitigation

The impact of this vulnerability was not widespread and is mostly limited to bleeding age and experimental systems. The following table shows the affected systems.

Distribution Advisory
Fedora Rawhide https://www.redhat.com/en/blog/urgent-security-alert-fedora-41-and-rawhide-users
Fedora 41 https://www.redhat.com/en/blog/urgent-security-alert-fedora-41-and-rawhide-users
Debian testing, unstable and experimental distributions versions 5.5.1alpha-0.1 to 5.6.1-1. https://lists.debian.org/debian-security-announce/2024/msg00057.html
openSUSE Tumbleweed and openSUSE MicroOS https://news.opensuse.org/2024/03/29/xz-backdoor/
Kali Linux https://www.kali.org/blog/about-the-xz-backdoor/
Arch Linux https://archlinux.org/news/the-xz-package-has-been-backdoored/

You can also check your installed version by running the command. 

 strings which xz` | egrep '\(XZ Utils\)

If your installed XZ util is version 5.6.0/5.6.1 then immediately downgrade to a known safe version, i.e 5.4.6.

Conclusion and Lessons Learned

While the Backdoor itself was sophisticated and involved many techniques, this supply chain attack was ultimately possible due to the one overworked developer maintaining this critical infrastructure, whom the threat actors could manipulate into giving them the required privileges for this attack.

Dependency
Comic: Dependency by xkcd

This incident serves as a reminder of the complex challenges faced in maintaining the security of open-source software - especially those widely used in critical infrastructures like Linux systems, and the community's collective responsibility to uphold and improve security practices. The trust placed in such tools requires continuous vigilance and scrutiny to prevent vulnerabilities and attacks. Let this incident be a lesson for the open-source community as a whole. As software users, we must actively support projects we rely on, not just through bug reports and feature requests but also by contributing code, documentation, and financial resources.

Further Reading and References

Thank you for reading this article. Stay vigilant, stay safe, stay curious!