Questions tagged [compression]
Shrinking (compressing) and restoring (decompressing) of data.
440 questions
0
votes
2
answers
135
views
Why did my backup folder with large amounts of repeated data compress so poorly?
I have a folder with around seventy subfolders, each containing a few tarballs which are nightly backups of a few directories (the largest being /home) from an old Raspberry Pi. Each is a full backup; ...
- 261
2
votes
0
answers
129
views
Add error correction to SquashFS images as part of a backup strategy
My backup strategy currently primarily consists of daily backups of all of my machines with Borg Backup, stored on different storage devices in different locations, following the 3-2-1 strategy. These ...
- 41
0
votes
1
answer
79
views
A document format for grey scanned pages with better lossless compression (i.e., a smaller file) than PDF+Zip?
I have page_1.pnm, …, page_6.pnm, which represent 6 pages of a scanned document, all in gray PNM produced by scanimage and manually postprocessed with GIMP. The command
convert $(for i in 1 2 3 4 5 6; ...
- 1
0
votes
1
answer
107
views
From GIMP, how to store images losslessly at maximum compression when creating a PDF?
Assume an image opened in GIMP in Debian 12. From this image, you would like to create a single-page PDF file with maximum lossless compression. How? As of 2024-12-19, https://docs.gimp.org/en/gimp-...
- 1
0
votes
1
answer
147
views
GitLab CI/CD: Create tar.gz with content inside current working dir and save tar.gz inside current working dir: tar: .: file changed as we read it
I'm trying to create an archive e.g. archive.tar.gz inside the current working directory e.g. /builds/project/ without saving the archive.tar.gz inside archive.tar.gz.
To prevent this I'm trying to ...
- 41
1
vote
1
answer
596
views
`pzstd` = parallelized Zstandard, how to watch progress in 4TB large file/disk?
I am brand new to zstd/pzstd, trying out its features, compression, benchmarking it, and so on. (I run Linux Mint 22 Cinnamon.) This computer has 32 GB RAM.
The basic command appears to be working, ...
- 31.3k
1
vote
1
answer
106
views
Storing large amount of small files in HDD pool
I have a Z6 HDD pool with 6 x 18T drives and a SLOG, LZ4 compression is enabled by default. Now I need to store large amount of small files and I'm worried about fragmentation.
The files:
70K files ...
- 121
0
votes
1
answer
113
views
How do I extract a single file from a large (1 TB) afio archive?
I have to restore a single file from a fairly large (~ 1 TB) afio archive I created by using the following script (debug messages omitted):
#!/bin/bash
#
SRCDIR=/bak
BAKDEV=/dev/disk/by-partlabel/...
- 364
0
votes
2
answers
92
views
Unrar through multiple directories - Ubuntu 20
folks, need a hand with unpacking a big batch of rars. Don't want to do this one-by-one. I have several directories inside subdirectories of 'dir' with many .rar's in each.
dir
subdir_a
...
- 63
0
votes
1
answer
192
views
mod_deflate Logging on Apache/2.4.62 (Debian) does not work
Basics
I am running the Apache/2.4.62 on my Raspberry Pi 4B (ARM64/AArch64) with the Debian GNU/Linux 12 (bookworm).
Some HW/OS info:
# neofetch
_,met$$$$$gg. root@rpi4
,g$$$$$$$$...
- 31.3k
1
vote
2
answers
1k
views
How can I calculate potential space savings with btrfs compression?
I'm not currently using compression with my btrfs-formatted disk, but am wondering how much space I'd save if I did enable it. Short of actually enabling compression on the disk and comparing the ...
- 974
0
votes
1
answer
199
views
How to verify size of pigz (parallel gzip) archive contents? [duplicate]
I created some pigz (parallel gzip) - home page - compressed archives of my SSD disk drives. (compiled version 2.8)
I called one of them 4TB-SATA-disk--Windows10--2024-Jan-21.img.gz which says the ...
- 31.3k
0
votes
3
answers
164
views
Is it possible to obtain .tar from .tar.bz2?
I need to extract a specific folder from .tar.bz2 (34G). The issue is that it takes 1 hour. I guess that this is due to the compression. I guess that w/o compression extraction of a specific folder ...
- 757
0
votes
1
answer
235
views
Failed to run init with error: jffs2: compression type 0x06 not available
I am working on an embedded Linux system (5.10.24), and I am using jffs2 as the rootfs.
Now I changed the kernel configuration of jffs2 as follows,
# CONFIG_JFFS2_FS_WBUF_VERIFY is not set
# ...
- 651
1
vote
1
answer
664
views
Creating a compressed image of an exFAT USB drive using dd
My system is running Ubuntu 22.04.3 LTS. I plug a USB drive into that system. That USB drive (/dev/sdb) contains an Ubuntu installation, mostly in an ext4 partition (/dev/sdb3). That installation ...
- 123
2
votes
0
answers
57
views
OCR high res images & combine OCR data later, after image compression?
I have a large number of .tif's coming out of ScanTailor. Is there a way that I might OCR those .tif's with tesseract, holding the OCR data separate from the images; then compress the images, and ...
- 740
0
votes
1
answer
637
views
What file compression software/algorithm is best for mixed types of files? [closed]
I am looking to backup a large amount (~400GB) of different types of files (50% images and videos, 30% audio, 20% text). I hesitate to give an amount by which I'd like the size to be reduced, since I ...
- 77
6
votes
1
answer
424
views
Download a file and piping to uncompress is not as fast as expected
I need to download and decompress a file as quickly as possible in a very latency sensitive environment with limited resources (A VM with 1 cpu, 2 cores, 128MB RAM)
Naturally, I tried to pipe the ...
- 113
2
votes
1
answer
225
views
How can I have rotated log files without a numbered suffix using Linux logrotate?
I have on my server more file rotate, but when I want to compress it, i Have an extension 1.gz, but I just want .gz
These files are automatically rotate by the application:
server.log.2023-03-16
...
- 21
10
votes
1
answer
945
views
Issue viewing compressed file with "zless", but not with "zmore" or "gunzip -c"
On my FreeBSD 13.2 system, the zless utility cannot view text files compressed with gzip or compress, warning that they may be binary files and then showing garbage if I say I want to see the contents ...
- 356k
12
votes
4
answers
2k
views
+100
How to compress disk swap?
I am looking for a way to compress swap on disk. I am not looking for wider discussion of alternative solutions. See discussion at the end.
I have tried:
Using compressed zfs zvol for swap is NOT ...
- 149
0
votes
2
answers
111
views
What POSIX utility can I use to decompress files with the .xz extension?
Instead of using non-POSIX tools such as unxz, what POSIX utility can I use to decompress files with the .xz extension?
Neither xz nor unxz is a POSIX command, so if I want to run only POSIX commands, ...
1
vote
1
answer
3k
views
Normal gz file not extractable by tar [duplicate]
I have a gz archive but for some reason tar said that the format is incorrect even though I can double click it in mac Finder and extract it normally, and file command shows the same format just like ...
- 2,442
0
votes
1
answer
244
views
See the file name and size within a (.Z) compressed file without uncompressing it
I have tons of .Z compressed files scattered across various directores and need to see the size of the file within it. I don't plan on uncompressing all the .Z files. Is there a way to see the content ...
- 103
0
votes
1
answer
378
views
Extract first n bytes from .tar.gz and output as a .tar.gz in a single command
I have a .tar.gz as input and want to extract the first 128 MiB of it and output as a .tar.gz in a single command. I tried:
sudo tar xzOf input.tar.gz | sudo dd of=output bs=1M count=128 iflag=...
- 3,606