From CLI to C: Integrating XZ Utils and liblzma into Your Software Stack

tukaani-project/xz

Let’s break down XZ Utils and see why it’s a staple in the Dev world.

XZ Utils is a set of free lossless data compression software which includes the xz and lzma commands. It’s based on the LZMA2 algorithm, which is famous for producing incredibly high compression ratios—often much better than gzip or bzip2.

Space Efficiency
It shrinks binaries and tarballs significantly, saving bandwidth and disk space.

Decompression Speed
While compressing can be CPU-intensive and slow, decompressing is very fast and requires relatively low memory.

Ubiquity
It’s the standard for packaging Linux kernel images and many distro packages (.deb, .rpm).

Most Unix-like systems have it pre-installed, but if you're setting up a fresh environment

macOS
brew install xz

Ubuntu/Debian
sudo apt-get install xz-utils

CentOS/RHEL
sudo yum install xz

As an engineer, you'll likely interact with XZ in two ways
via the CLI for automation/DevOps, or via the C Library (liblzma) for application development.

Commonly used in CI/CD scripts to compress build artifacts.

# Compress a file (original is replaced by file.txt.xz)
xz file.txt

# Decompress a file
xz -d file.txt.xz

# Compress with maximum compression (level 9)
xz -9 huge_log_file.log

If you're building a system that needs to handle .xz files natively, you'll use liblzma. Here is a simplified look at how you might initialize a "one-shot" encoding

#include <lzma.h>
#include <stdio.h>
#include <stdbool.h>

bool simple_compress(const uint8_t *src, size_t src_size, uint8_t *dst, size_t *dst_size) {
    // Setting up the stream with a preset (6 is the default)
    uint32_t preset = 6; 
    
    // One-shot buffer-to-buffer compression
    lzma_ret ret = lzma_easy_buffer_encode(preset, LZMA_CHECK_CRC64, NULL, 
                                           src, src_size, dst, dst_size, *dst_size);

    return (ret == LZMA_OK);
}

Note
Always link with -llzma when compiling!

Pro Tip
Use XZ when you are distributing files that will be downloaded many times but only compressed once (like a software release). If you need to compress real-time logs quickly, gzip or zstd might be better choices.