zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

Question

I'm trying through Windows terminal to decompress a large number of compressed files with zstd v1.4.0 and then 'ag' search over :

zstd -dc -r . | ag -z -i "term"

It gives me the following error while proceeding :

zstd: error 70 : Write error : Broken pipe (cannot write decoded block)

I have spent hours looking for a solution, tried different options for the zstd command but can't solve this.

Mitchmitchael

This may indicate that ag is quitting, can you normally check if ag -z "term" <file> is working for you? – Lynden

It gives me errors : ERR: Cannot decompress zipped file or ERR: Zip files not yet supported – Mitchmitchael

Yes, so zip and gzip is different and I believe ag supports gzip not zip – Lynden

@Mitchmitchael you should try to narrow the problem, is it working with 1 compressed file ? maybe using powershell is different (in the past cmd pipe was: write to a temporary file and reading it) ? – Predator

Answer 1

If you like to search over uncompressed files, you should use :

zstd -dc -r . | ag -i "term"

zstd will uncompress the supported files and ag will search over it (your command try to uncompress twice, once with zstd, once with ag)

You can get the supported format using zstd -vV that should give you something like :

*** zstd command line interface 64-bits v1.4.0, by Yann Collet ***
*** supports: zstd, zstd legacy v0.5+, gzip

Note: The zstd version I tried doesnot support zip.

You can get the supported compressed format by ag using ag -V, that should give something like :

ag version 2.1.0

Features:
  +jit +lzma +zlib

Note: The ag version I tried doesnot support zip.

Predator

Thanks for this lengthy answer, it's really interesting. I've tried first solution, ie. removing -z from formula and I have same error. – Mitchmitchael

@titibouboul: This is strange because in decompress.c the message Zip files not yet supported is printed in method decompressand this method is called only when -z is used search.c – Predator

I'm referring to the original error I'm trying to fix : zstd: error 70 : Write error : Broken pipe (cannot write decoded block) – Mitchmitchael