Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
372 changes: 321 additions & 51 deletions Cargo.lock

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,13 @@ embed-uruntime = []
embed-mkdwarfs = []

[dependencies]
backhand = { version = "0.25.5", default-features = false, features = ["zstd"] }
clap = { version = "4.6.1", features = ["derive", "env"] }
libc = "0.2.186"
sha2 = "0.10.9"
sha2 = "0.11.0"
thiserror = "2.0.18"
ureq = "3.3.0"
zsync-rs = "0.3.0"

[build-dependencies]
sha2 = "0.10.9"
sha2 = "0.11.0"
32 changes: 30 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

A Rust implementation of `appimagetool` for the [Anylinux-AppImages](https://github.com/pkgforge-dev/Anylinux-AppImages) project.

It takes a prepared AppDir and produces a finished `.AppImage` using DWARFS compression and the [uruntime](https://github.com/VHSgunzo/uruntime) AppImage runtime. Single binary, no Python, no C++ deps.
It takes a prepared AppDir and produces a finished `.AppImage` using DWARFS compression and the [Anylinux-uruntime](https://github.com/pkgforge-dev/Anylinux-uruntime) AppImage runtime. Single binary, no Python, no C++ deps.

## Quick start

Expand All @@ -25,7 +25,8 @@ That's it. The tool will:
## Features

- DWARFS compression for small, delta-friendly AppImages.
- Drop-in uruntime integration with automatic download and ELF section patching.
- Drop-in Anylinux-uruntime integration with automatic download and ELF section patching.
- A fixed-size SquashFS metadata sidecar (`.DirIcon`, desktop entry, and a stub `AppRun`) so SquashFS-only thumbnailers and desktop integration tools work with DwarFS AppImages.
- Optional profile-guided optimization (DWARFS hotness profiling) for faster launches.
- Built-in zsync generation, no `zsyncmake` shell-out.
- No central repository, no daemon, no extra runtime deps.
Expand Down Expand Up @@ -131,6 +132,33 @@ appimagetool --dwarfs-comp 'zstd:level=22 -S26 -B2' ./AppDir

An explicit `--dwarfs-comp` is passed to `mkdwarfs` unchanged.

### SquashFS metadata sidecar

Each AppImage gets a fixed **128 KiB** SquashFS sidecar between the runtime and
the DwarFS payload. It contains the AppDir's `.DirIcon`, main `.desktop` file,
and a stub `AppRun` that tells tools such as `appimage-run` that the image must
be mounted as DwarFS. The sidecar is compressed with zstd level 22.

The sidecar makes SquashFS-only thumbnailers and desktop-integration tools work
without understanding DwarFS. The stub `AppRun` does not launch the application;
the real payload is the following DwarFS image.

The sidecar is zero-padded to a constant size. This keeps the DwarFS payload at
a stable offset across releases, which is what preserves zsync delta-update
efficiency. Changing the slot size shifts the payload and costs a large update;
the first release that adopts the sidecar will also have a large update from
pre-sidecar versions.

The build fails if the metadata image does not fit in the slot. The 128 KiB
slot covers typical 256x256 and 512x512 icons and most SVGs, including
translation-heavy desktop files.

The pinned runtime is the
[Anylinux-uruntime](https://github.com/pkgforge-dev/Anylinux-uruntime) fork,
which skips the sidecar and mounts the DwarFS payload. A custom `--runtime`
must include the same metadata-image support, otherwise the runtime will try to
mount the metadata SquashFS as the application.

### AppDir requirements

The AppDir must contain:
Expand Down
17 changes: 13 additions & 4 deletions src/appimage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ use crate::config::Config;
use crate::desktop::{self, DesktopEntry};
use crate::dwarfs;
use crate::error::Result;
use crate::metadata;
use crate::uruntime;
use crate::util;

Expand Down Expand Up @@ -129,15 +130,23 @@ pub fn build(config: &Config) -> Result<()> {
.clone()
.unwrap_or_else(|| dwarfs::default_compression(&config.appdir));

// Build the final AppImage
dwarfs::build_appimage(
// Build the final AppImage. The header is the configured runtime followed
// by a small metadata SquashFS so SquashFS-only thumbnailers and desktop
// integration tools can read .DirIcon and the desktop entry.
crate::log_info!("Building metadata SquashFS sidecar...");
let sidecar = metadata::build(&config.appdir, &desktop.path)?;
let header_path = metadata::header_path(&config.tmpdir);
metadata::write_header(&header_path, &runtime_path, &sidecar)?;
let build_result = dwarfs::build_appimage(
&mkdwarfs,
&config.appdir,
&runtime_path,
&header_path,
&output_path,
&compression,
profile.as_deref(),
)?;
);
let _ = std::fs::remove_file(&header_path);
build_result?;

// Make output executable
use std::os::unix::fs::PermissionsExt;
Expand Down
7 changes: 4 additions & 3 deletions src/dwarfs.rs
Original file line number Diff line number Diff line change
Expand Up @@ -150,11 +150,12 @@ fn mkdwarfs_source(config: &Config) -> (String, Option<&'static str>) {
(url, expected_checksum)
}

/// Build a DWARFS AppImage. The runtime is embedded via `--header`.
/// Build a DWARFS AppImage. `header` is prepended via `--header`; it is the
/// configured runtime followed by the metadata SquashFS sidecar.
pub fn build_appimage(
mkdwarfs: &Path,
appdir: &Path,
runtime: &Path,
header: &Path,
output: &Path,
compression: &str,
profile: Option<&Path>,
Expand All @@ -171,7 +172,7 @@ pub fn build_appimage(
.arg("--no-history")
.arg("--no-create-timestamp")
.arg("--header")
.arg(runtime)
.arg(header)
.arg("--input")
.arg(appdir);

Expand Down
16 changes: 16 additions & 0 deletions src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,22 @@ pub enum Error {
#[error("mksquashfs failed: {0}")]
SquashfsFailed(String),

/// Failed to build the metadata SquashFS sidecar.
#[error("failed to build metadata SquashFS sidecar: {0}")]
MetadataSidecar(String),

/// The metadata SquashFS is larger than its fixed slot.
#[error(
"metadata SquashFS is {size} bytes, exceeding the fixed {limit}-byte sidecar slot\n \
hint: use a smaller .DirIcon (for example a 256x256 PNG) or reduce the desktop entry size"
)]
MetadataSidecarTooLarge {
/// Size of the built metadata image.
size: usize,
/// Fixed slot size available for the sidecar.
limit: usize,
},

/// Failed to generate the `.zsync` control file.
#[error(
"zsync generation failed: {0}\n \
Expand Down
2 changes: 2 additions & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ pub mod elf;
pub mod embed;
pub mod error;
pub mod log;
/// Builds the metadata SquashFS sidecar placed before the DwarFS payload.
pub mod metadata;
/// Pinned upstream artifact URLs and SHA-256 digests, shared with `build.rs`.
pub mod pinned;
pub mod uruntime;
Expand Down
211 changes: 211 additions & 0 deletions src/metadata.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,211 @@
//! Build the small SquashFS metadata sidecar placed between the runtime and
//! the DwarFS payload.
//!
//! Many thumbnailers and desktop-integration tools only know how to read a
//! SquashFS at the end of the runtime ELF. The sidecar carries `.DirIcon`, the
//! main `.desktop` entry, and a stub `AppRun` so those tools work against a
//! DwarFS AppImage. The runtime skips this image and mounts the following
//! DwarFS payload.
//!
//! The image is always padded to [`SIDECAR_SLOT_BYTES`]. Keeping the slot size
//! constant keeps the DwarFS payload at a stable offset across releases, which
//! preserves zsync delta-update efficiency.

use std::io::Cursor;
use std::path::{Path, PathBuf};

use backhand::compression::{CompressionOptions, Compressor, Zstd};
use backhand::{BackhandError, FilesystemCompressor, FilesystemWriter, NodeHeader};

use crate::error::{Error, Result};

/// zstd level used for the sidecar.
const SIDECAR_ZSTD_LEVEL: u32 = 22;

/// Fixed size of the metadata slot between the runtime and the DwarFS payload.
///
/// The image is zero-padded to this size. The patched runtime scans the fixed
/// 128 KiB slot for the following DwarFS payload.
pub const SIDECAR_SLOT_BYTES: usize = 128 * 1024;

/// Stub `AppRun` for tools that mount or extract the metadata image instead of
/// using uruntime. It must never be mistaken for the real DwarFS payload.
const DUMMY_APPRUN: &[u8] = b"#!/bin/sh\n\
>&2 echo 'This is not the real AppImage payload, this is a DwarFS AppImage, please use dwarfs to mount it instead'\n\
exit 1\n";

/// Build a padded metadata SquashFS image containing `.DirIcon`, the main
/// desktop entry, and a stub `AppRun` from `appdir`.
///
/// `desktop_path` must be the final desktop file (after X-AppImage metadata
/// has been added), so the sidecar advertises the same metadata as the real
/// payload.
pub fn build(appdir: &Path, desktop_path: &Path) -> Result<Vec<u8>> {
let dir_icon = std::fs::read(appdir.join(".DirIcon"))?;
let desktop = std::fs::read(desktop_path)?;
let desktop_name = desktop_path
.file_name()
.ok_or_else(|| Error::MetadataSidecar("desktop entry has no file name".to_string()))?
.to_owned();

let mut fs = FilesystemWriter::default();
// A fixed timestamp and root ownership keep the sidecar reproducible.
fs.set_time(0);
fs.set_only_root_id();
fs.set_root_mode(0o755);
fs.set_root_uid(0);
fs.set_root_gid(0);
fs.set_no_padding();
let compressor = FilesystemCompressor::new(
Compressor::Zstd,
Some(CompressionOptions::Zstd(Zstd {
compression_level: SIDECAR_ZSTD_LEVEL,
})),
)
.map_err(metadata_error)?;
fs.set_compressor(compressor);
// The zstd level is a build-time choice; readers only need the compressor
// id, so don't spend bytes storing the options in the image.
fs.set_emit_compression_options(false);

let file_header = NodeHeader::new(0o644, 0, 0, 0);
fs.push_file(Cursor::new(dir_icon), Path::new(".DirIcon"), file_header)
.map_err(metadata_error)?;
fs.push_file(Cursor::new(desktop), Path::new(&desktop_name), file_header)
.map_err(metadata_error)?;
fs.push_file(
Cursor::new(DUMMY_APPRUN),
Path::new("AppRun"),
NodeHeader::new(0o755, 0, 0, 0),
)
.map_err(metadata_error)?;

let mut output = Cursor::new(Vec::new());
let (superblock, _) = fs.write(&mut output).map_err(metadata_error)?;
let mut image = output.into_inner();
let actual_size = usize::try_from(superblock.bytes_used).map_err(|_| {
Error::MetadataSidecar("metadata image size does not fit usize".to_string())
})?;

if actual_size > SIDECAR_SLOT_BYTES {
return Err(Error::MetadataSidecarTooLarge {
size: actual_size,
limit: SIDECAR_SLOT_BYTES,
});
}

image.resize(SIDECAR_SLOT_BYTES, 0);
crate::log_debug!(
"metadata sidecar: {} bytes, padded to {} bytes",
actual_size,
SIDECAR_SLOT_BYTES
);
Ok(image)
}

/// Write `runtime || sidecar` to `dest` for use as `mkdwarfs --header`.
pub fn write_header(dest: &Path, runtime: &Path, sidecar: &[u8]) -> Result<()> {
use std::io::Write;

let runtime_bytes = std::fs::read(runtime)?;
let mut file = std::fs::File::create(dest)?;
file.write_all(&runtime_bytes)?;
file.write_all(sidecar)?;
file.flush()?;
Ok(())
}

/// Temporary header path for a single build.
pub fn header_path(tmpdir: &Path) -> PathBuf {
crate::util::process_unique_path(tmpdir, "appimage-header")
}

fn metadata_error(error: BackhandError) -> Error {
Error::MetadataSidecar(error.to_string())
}

#[cfg(test)]
mod tests {
use super::*;

fn temp_dir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!(
"appimagetool-metadata-{name}-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(&dir).unwrap();
dir
}

#[test]
fn builds_fixed_size_squashfs_with_expected_entries() {
let dir = temp_dir("build");
std::fs::write(dir.join(".DirIcon"), b"fake png bytes").unwrap();
std::fs::write(dir.join("TestApp.desktop"), b"[Desktop Entry]\nName=Test\n").unwrap();

let image = build(&dir, &dir.join("TestApp.desktop")).unwrap();
assert_eq!(image.len(), SIDECAR_SLOT_BYTES);
assert_eq!(&image[..4], b"hsqs");

let bytes_used = u64::from_le_bytes(image[40..48].try_into().unwrap());
assert!(bytes_used <= SIDECAR_SLOT_BYTES as u64);

let reader =
backhand::FilesystemReader::from_reader(Cursor::new(image.as_slice())).unwrap();
let paths: Vec<_> = reader.files().map(|node| node.fullpath.clone()).collect();
assert!(paths.iter().any(|path| path == Path::new("/.DirIcon")));
assert!(
paths
.iter()
.any(|path| path == Path::new("/TestApp.desktop"))
);
assert!(paths.iter().any(|path| path == Path::new("/AppRun")));

let apprun = reader
.files()
.find(|node| node.fullpath == Path::new("/AppRun"))
.unwrap();
assert_eq!(apprun.header.permissions, 0o755);

let _ = std::fs::remove_dir_all(&dir);
}

#[test]
fn is_reproducible() {
let dir = temp_dir("repro");
std::fs::write(dir.join(".DirIcon"), b"fake png bytes").unwrap();
std::fs::write(dir.join("TestApp.desktop"), b"[Desktop Entry]\nName=Test\n").unwrap();

let first = build(&dir, &dir.join("TestApp.desktop")).unwrap();
let second = build(&dir, &dir.join("TestApp.desktop")).unwrap();
assert_eq!(first, second);

let _ = std::fs::remove_dir_all(&dir);
}

#[test]
fn rejects_metadata_larger_than_slot() {
let dir = temp_dir("large");
// Pseudo-random bytes resist zstd better than a repeating pattern.
let mut state = 0x9e37_79b9_u32;
let large: Vec<u8> = (0..SIDECAR_SLOT_BYTES * 2)
.map(|_| {
state ^= state << 13;
state ^= state >> 17;
state ^= state << 5;
(state & 0xff) as u8
})
.collect();
std::fs::write(dir.join(".DirIcon"), &large).unwrap();
std::fs::write(dir.join("TestApp.desktop"), b"[Desktop Entry]\nName=Test\n").unwrap();

let error = build(&dir, &dir.join("TestApp.desktop")).unwrap_err();
assert!(matches!(error, Error::MetadataSidecarTooLarge { .. }));

let _ = std::fs::remove_dir_all(&dir);
}
}
Loading
Loading