Skip to content

linux-gzip-tutorial.md

Latest commit

 

History

History
302 lines (204 loc) · 11.5 KB

linux-gzip-tutorial.md

File metadata and controls

302 lines (204 loc) · 11.5 KB
Raw
title Linux Command Tutorial: gzip
date 2026-09-12 00:00:00 +0000
categories
Technology
tags
Linux
GNU gzip
gzip
Linux Command Tutorial
draft false
slug linux-gzip-tutorial
description Authoritative reference tutorial for gzip (GNU gzip), detailing DEFLATE algorithm compression, archive decompression, compression level tuning, integrity testing, and stream pipelines.
upstream_suite gnu-gzip
upstream_version GNU gzip 1.13
posix_standard None
research_date 2026-09-12

The Linux Command Tutorial series provides rigorous, upstream-verified references for essential system commands across Linux distributions and UNIX-like environments. Each article focuses on a single executable, combining exhaustive option documentation, verified real-world examples, security boundaries, and best practices directly derived from official source documentation and POSIX standards.

1. Introduction

Upstream: gnu-gzip (GNU gzip 1.13) | POSIX: None (De-facto Standard / RFC 1952) | Safety Tier: unprivileged-filesystem-write | Scope: stream-compression

gzip (GNU zip) is a standard file compression and decompression utility utilizing the DEFLATE algorithm (a combination of LZ77 and Huffman coding). It reduces the size of individual files, replaces original files in-place with .gz archives, and preserves file timestamps, permissions, and ownership.

  • Upstream Project & Provenance: Maintained by the GNU Project under GNU gzip (gzip), originally developed by Jean-loup Gailly and Mark Adler.
  • Portability & Standards Baseline: De-facto standard compression format governed by RFC 1952 (GZIP file format specification) and RFC 1951 (DEFLATE); not standardized in IEEE Std 1003.1-2024 (POSIX.1-2024).
  • Target Research Implementation: Audited against GNU gzip 1.13 (gzip(1)).
  • Applicability & Lifecycle: The ubiquitous compression standard for Linux distribution packages, tarballs (.tar.gz), HTTP content encoding (Content-Encoding: gzip), and log rotation (logrotate).

2. Syntax and Command Model

2.1 Canonical Synopsis

gzip [options] [file...]

2.2 Execution and In-Place File Replacement Model

A critical behavioral characteristic of gzip is in-place file replacement:

  • Compression: Executing gzip file.txt creates file.txt.gz and deletes the original file.txt upon successful compression.
  • Decompression: Executing gzip -d file.txt.gz restores file.txt and deletes file.txt.gz.
  • Preservation: To retain original files, you must explicitly supply -k (--keep) or stream through stdout (-c).
  • Single-File Scope: gzip does not package multiple files into an archive. Archiving directory hierarchies requires pairing with tar (producing .tar.gz).

3. Options

3.1 Primary Operational Flags

Option Long Option Description Default
-d --decompress Decompress specified .gz files (equivalent to gunzip). Compress
-c --stdout Write output to stdout; leave original files untouched. Replace file
-k --keep Keep (do not delete) original input files during processing. Delete originals
-f --force Force overwrite of existing destination files or compress symlinks. Prompt / Abort
-l --list List compressed size, uncompressed size, and ratio for archives. Normal run
-t --test Test the cryptographic and checksum integrity of compressed files. Decompress
-v --verbose Display compression percentage reduction and filenames. Silent
-r --recursive Traverse directory trees recursively, compressing every file inside. Top-level only
-q --quiet Suppress all non-fatal warning messages. Normal
-n --no-name Do not save or restore original filename and timestamp. Save name/mtime

3.2 Compression Levels

gzip allows regulating the trade-off between compression speed and file size reduction using numeric flags:

Flag Mode Description
-1 --fast Fastest compression speed; lower compression ratio.
-6 Default balance of speed and compression efficiency.
-9 --best Maximum compression ratio; slowest execution speed.

4. Basic Usage

4.1 Quick-Reference Cheatsheet Card

Operation Command Pattern Copyable One-Liner Notes
Compress keep original gzip -k [file] gzip -kv access.log Compresses file without deleting original
Decompress keep archive gzip -dk [file.gz] gzip -dk access.log.gz Restores original file, keeps .gz archive
Fast compression gzip -1 [file] gzip -1k backup.sql Minimum compression time, larger output
Maximum compression gzip -9 [file] gzip -9k backup.sql Best compression ratio, higher CPU usage
Test archive integrity gzip -t [file.gz] gzip -tv archive.tar.gz Verifies internal CRC32 checksums
Inspect metrics gzip -l [file.gz] gzip -l access.log.gz Displays uncompressed size and savings ratio
Stream stdout gzip -c [file] > [out.gz] gzip -c file.txt > file.txt.gz Safe stream output leaving source untouched

4.2 Compressing Files while Retaining Originals (-k)

Compress a file with progress reporting while ensuring the original source file is preserved:

gzip -kv access.log

Output:

access.log:	 84.2% -- replaced with access.log.gz

Verify that both files remain present:

ls -lh access.log*

Output:

-rw-r--r-- 1 user user 100M Sep 12 18:00 access.log
-rw-r--r-- 1 user user  16M Sep 12 18:00 access.log.gz

4.3 Decompressing Files

Decompress an archive, restoring the original uncompressed file:

gzip -d -k access.log.gz
# Or use the standard symlink
gunzip -k access.log.gz

4.4 Inspecting Archive Metrics with -l

Query compressed size, uncompressed size, and space savings without decompressing:

gzip -l access.log.gz

Output:

         compressed        uncompressed  ratio uncompressed_name
           16580412           104857600  84.2% access.log

5. Practical Operations

5.1 Verifying Archive Integrity (-t)

Validate that a compressed file is not corrupted or truncated before removing external backups:

gzip -tv access.log.gz

Output:

access.log.gz:	 OK

When corruption exists, gzip -t immediately flags CRC check or data integrity failures:

$ gzip -tv corrupted.log.gz
corrupted.log.gz:	 invalid compressed data--crc error

5.2 Streaming Standard Input Pipelines

Compress stdout from arbitrary commands directly to disk without creating intermediate uncompressed files:

# Stream database dump directly through maximum compression
mysqldump --all-databases | gzip -9 > db_backup_2026_09_12.sql.gz

Decompress directly into a processing pipeline:

gzip -dc db_backup_2026_09_12.sql.gz | mysql

5.3 Viewing Compressed Files Directly (Gz-Tools Integration)

Pairing gzip with standard utility wrappers allows inspecting compressed files without manual decompression:

# View compressed text page-by-page
zless access.log.gz

# Search inside compressed logs without unpacking
zgrep "ERROR 500" access.log.gz

# Compare two compressed files
zdiff file1.txt.gz file2.txt.gz

5.4 Recursive Log Directory Compression

Compress all historical log files within a nested directory hierarchy:

gzip -r -k /var/log/archive/

6. Advanced Usage

6.1 Concatenated Stream Handling

The GZIP specification (RFC 1952) permits multiple compressed members to be concatenated together into a single file. gzip -d transparently decompresses all members sequentially:

# Compress two files independently
gzip -c part1.txt > combined.gz
gzip -c part2.txt >> combined.gz

# Decompressing combined.gz unpacks the concatenated text of both parts
gzip -dc combined.gz > full.txt

6.2 Speed Optimization with Modern Multithreaded Alternatives (pigz)

On modern multi-core systems, standard gzip is CPU-bound to a single core. The drop-in compatible tool pigz (Parallel Implementation of GZip) splits compression work across all available CPU threads while generating fully compliant gzip files:

# Compress using all CPU cores with pigz
pigz -k -9 huge_disk_dump.raw

7. Exit Status, Environment, and Configuration

7.1 Exit Status

Exit Code Meaning
0 Success: compression, decompression, or integrity test completed without errors.
1 Error: invalid file format, CRC failure, or file read/write error.
2 Warning: warning condition encountered (e.g. file not compressed because it would expand).

7.2 Environment Variables

Variable Description
GZIP Historical environment variable holding default options (deprecated by upstream due to security hazards with filenames containing spaces).

8. Safety, Security, and Portability

8.1 In-Place Deletion Hazard

Warning

In-Place Source Deletion: By default, gzip deletes the uncompressed source file upon successful compression, and deletes the .gz archive upon decompression. Always pass -k (--keep) or stream with -c to preserve input files.

8.2 Denial of Service via "Zip Bombs"

Caution

Denial of Service via Zip Bombs: Highly repetitive byte sequences can compress gigabytes of data into small megabyte archives. Check uncompressed dimensions using gzip -l before unpacking untrusted archives.

8.3 Portability Constraints

gzip is universal across all Linux distributions, BSD systems, and macOS. Decompressed .gz files comply with RFC 1952 and can be unpacked by any standard deflation tool (e.g. 7-Zip, WinRAR, Python's gzip module).

9. Best Practices

9.1 Always Supply -k (--keep) in Production Scripts

Tip

Always Supply -k (--keep) in Production Scripts: Preserving input files until external backup integrity verification succeeds prevents catastrophic data loss if subsequent tasks fail.

Upstream Rationale: gzip(1) explicitly defines destructive source deletion as standard default behavior. In automated scripts, unlinking source files before verifying backup integrity invites data loss. Always specify -k to retain the source until external verification succeeds.

9.2 Always Validate with -t Before Deleting Source Data

Important

Always Validate with -t Before Removing Originals: Run gzip -t <file>.gz to calculate and verify internal CRC32 checksums before cleaning up original uncompressed data.

Upstream Rationale: Silent disk write failures or network interruptions during streaming can produce truncated .gz archives. Running gzip -t <file>.gz calculates and verifies the internal CRC32 checksum before original files are removed.

9.3 Combine With tar for Directory Archiving

Upstream Rationale: gzip is strictly a file compression filter; it does not record directory structures, permissions, or multi-file hierarchies. Always bundle directory structures with tar (tar -czf archive.tar.gz directory/) rather than attempting to run gzip -r.

References

  1. gzip(1) — GNU gzip reference manual: https://www.gnu.org/software/gzip/manual/gzip.html
  2. RFC 1952 — GZIP File Format Specification version 4.3: https://datatracker.ietf.org/doc/html/rfc1952
  3. GNU gzip Source Repository: https://git.savannah.gnu.org/cgit/gzip.git