How do you loop thru a list of files?
For instance you want to archive than delete all pdf documents in the current directory :
Bad practice :
tar cvf f.tar *.pdf rm *.pdf
There are multiple issue with the command above
1) new files could come during the tar, so the rm will delete files that have not been archived
filelist=$(ls *.pdf) tar cvf f.tar $filelist rm $filelist
2) if there is no file, tar and rm will return an error
filelist=$(ls|grep '\.pdf') if [ -n "$filelist" ] then tar cvf f.tar $filelist rm $filelist fi
3) this will not work for long list (above 100k documents)
filelist=/tmp/filelist.$(date "+%Y%m%d%H%M%S").$$.$RANDOM ls|grep '\.pdf' > $filelist if [ -s "$filelist" ] then tar cvfL f.tar $filelist for f in $(<filelist) do rm $f done fi
As you see, this require special handling. tar for instance use the -L option to accept a list of files, rm could delete files one by one (or in bunches with xargs -L).
This 100’000 limit (the limit may vary for your shell/os) is something that often gets forgotten.
Typical error that could occur are
ksh: no space bash: Arg list too long