So I created a simple PHP script, which will do :
# List millions of files with no problem
# default output : machine parseable, for easy post-processing
# human-readable output option also available
The result turned out to be quite powerful. I used it to process tens of millions of files, no sweat.
An example :
cd /mnt/tmp/tmp1 ; bigls.php | while read OUT ; do cond=20130901 ; x=`echo $OUT | cut -b 1-8` ; f=`echo $OUT | cut -b 10-255` ; if [ $cond -ge $x ]; then mv $f /mnt/tmp/tmp2/ >> /tmp/files-moved.log ; fi ; done
That single line of commands will move all files created before 1 Sep 2013 from /mnt/tmp/tmp1/ to /mnt/tmp/tmp2/
Again, pretty powerful stuff 🙂 now we can process million of files with no problem.
So here it is, enjoy !
bigls website : https://github.com/sufehmi/bigls
note: you'll need at least PHP 5.3.0 to run bigls
Post imported by Google+Blog for WordPress.