Skip to content

UNIX Script for Directory Maintenance

April 9, 2013

In is a common practice in data warehousing to receive high volumes of data files from data sources. It is necessary monitor and clean up old files to avoid running out of disk space.

Here is a template of a ksh script to automate the clean up process; it has been tested in Solaris 10. Please note that this script remove all files older that 365 days from the main directory and empty the temporary directory before and after the compression of the archive files.

The 2 main requirements are:

  1. Compress all files older than a week.
  2. Remove all files older that a year.

To fully automate the clean up process, you’ll need to create a cron job that uses a script base on this template.

Disclaimer – Use of this script is as your own risk, the author assumes no responsibility for data loss or damage.

 

#!/usr/bin/ksh

###########################################################
# Script name : extractfilespurger.sh
# Description : Find all the files older than 7 days,
# : Move the files to the save directory,
# : Compress the files. Delete all the files
# : older that a year
# Author : Uvernes Somarriba
# Creation date: April 9, 2013
###########################################################

DATE=`date +%Y%m%d%H%M`
COMPRESSDAYS=7
DELETEDAYS=365

#++++++++++++++++++++++++++++++++++++++
# Process SMS files
#++++++++++++++++++++++++++++++++++++++

MYDIR=/dw/extracts/archive/MainDirectory
MYCOMPRESSDIR=/dw/extracts/archive/temp_compress
NEWCOMPRESSFILE=ARCHIVE_${DATE}.tar

cd ${MYCOMPRESSDIR}

# Delete files older that 365 days

find ${MYDIR} -type f -mtime +365 -exec rm {} \;

# Move all files older than 7 days to the temporary directory, compress them
# and move the compressed file back to the main directory

rm ${MYCOMPRESSDIR}/*

find ${MYDIR} -type f ! -name *.Z -mtime +7 -exec mv {} ${MYCOMPRESSDIR} \;
echo ${NEWCOMPRESSFILE} > ${MYCOMPRESSDIR}/EXCLUDE_FILE
tar cfX ${MYCOMPRESSDIR}/${NEWCOMPRESSFILE} ${MYCOMPRESSDIR}/EXCLUDE_FILE *
compress ${MYCOMPRESSDIR}/${NEWCOMPRESSFILE}
mv ${MYCOMPRESSDIR}/${NEWCOMPRESSFILE}.Z ${MYDIR}

rm ${MYCOMPRESSDIR}/*
 

Advertisements

From → UNIX

Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: