Moving data between quota protected directories on Isilon

Updated version of the script here: https://unscrupulousmodifier.wordpress.com/2015/10/08/moving-data-between-quota-protected-directories-on-isilon-take-ii/

In the current versions of Isilon OneFS, it is impossible to move files and directories between two directories with quotas on them (regardless of the directory quota type; even if it’s advisory, it won’t allow it). This is really annoying, and although I’ve put in a feature request for it, who knows if it will ever be fixed. So I wrote this script that will make a note of the quota location and threshold (if it’s a hard threshold), remove the quota, move the items, and reapply the quotas.

#!/bin/bash

#Tests whether there is a valid path
testexist () {
        if [ ! -r $1 ]; then
                echo "$1 is an invalid path. Please try again."
                exit
        fi
}

#Iterates through path backwards to find most closely related quota
findquota () {
        RIGHTPATH=0
        i=`echo $1 | awk -F'/' '{print NF}'` #define quantity of fields
        while [ $RIGHTPATH -eq 0 ]; do
                QUOTA=`echo $1 | cut -d"/" -f "1-$i"`
                if [ -n "`isi quota list | grep $QUOTA`" ]; then
                        RIGHTPATH=1
                fi
                i=$(($i-1))
        done
        echo $QUOTA
}

testquota () {
        if [ "$1" = "-" ]; then
                echo "No hard directory quota on this directory."
                exit
        fi
}

if [[ $# -ne 2 ]]; then
        #Gets paths from user
        echo "Enter source:"
        read SOURCE
        echo "Enter target:"
        read TARGET
else
        SOURCE=$1
        TARGET=$2
fi

testexist $SOURCE
testexist $TARGET

#Verifies paths with user
echo "Moving $SOURCE to $TARGET. Is this correct? (y/n)"
read ANSWER
if [ $ANSWER != 'y' ] ; then
        exit
fi

#Defines quotas
SOURCEQUOTA=$(findquota $SOURCE)
TARGETQUOTA=$(findquota $TARGET)

#Gets size of hard threshold from quota
SOURCETHRESH=$(isi quota view $SOURCEQUOTA directory | awk -F" : " '$1~/Hard Threshold/ {print $2}')
TARGETTHRESH=$(isi quota view $TARGETQUOTA directory | awk -F" : " '$1~/Hard Threshold/ {print $2}')
testquota $SOURCETHRESH
testquota $TARGETTHRESH

echo $SOURCEQUOTA $SOURCETHRESH
echo $TARGETQUOTA $TARGETTHRESH

isi quota quotas delete --type=directory --path=$SOURCEQUOTA -f
isi quota quotas delete --type=directory --path=$TARGETQUOTA -f

isi quota quotas view $SOURCEQUOTA directory
isi quota quotas view $TARGETQUOTA directory

mv $SOURCE $TARGET

isi quota quotas create $SOURCEQUOTA directory --hard-threshold=$SOURCETHRESH --container=yes
isi quota quotas create $TARGETQUOTA directory --hard-threshold=$TARGETTHRESH --container=yes

isi quota quotas view $SOURCEQUOTA directory
isi quota quotas view $TARGETQUOTA directory

Here’s how I use it:

bash /ifs/data/scripts/qmv /ifs/source/path /ifs/target/path

First I’ve got some functions in there:
testexist (): test if it’s a sane path
findquota (): find the quota info for the given path
testquota (): check if it’s a hard quota. If it’s not, the script fails, because that’s all we use around here. Feel free to fix it up and post something better in the comments.

Then we get to the bit where if it’s not given two arguments for source and target, it asks for them. It then tests if the source and target both exist. Please note that this script expects a fully qualified path including the bit you want to move for the source, and the place you want to move it for the target (ie, not source=/ifs/data/somethingdir/something target=/ifs/data/otherdir/something).

Of course, there’s a bit of error checking you’ll pretty much start ignoring and answering y to all the time…

Then we find the quotas for the directories. What the findquota () function does is it iterates back through the path until it finds an actual quota on it. I think this will break if you have nested quotas, but again, feel free to fix it up and let me know. It’ll then throw out which quota applies. Once it’s found both the quota paths, it saves the hard threshold in a variable. Now we’ve got variables for the source quota directory, the target quota directory, and both of their hard thresholds.

From there, it’s an easy move to delete the quotas, move the actual data, and then put the quotas back.

Don’t forget to use the –container=yes flag on those isi quota quotas create commands if you don’t want to show your end users the entire size of the filesystem.

**** Please note, and I found this after I made this post… if you comment out the echo $QUOTA line in the findquota () function, it kinda breaks the whole script. And then deletes all of your quotas without asking you. So, uh, don’t comment that out. That echo is what populates the $SOURCEQUOTA and $TARGETQUOTA variables. ****

This script works as of OneFS 7.1. I make no guarantees they won’t switch around the isi commands again in their quest to make commands as long and convoluted as possible.

Advertisements

About kcarlile
Twitter: @overclockdlemon

5 Responses to Moving data between quota protected directories on Isilon

  1. Pingback: Moving data between quota protected directories on Isilon, take II | Unscrupulous Modifier

  2. Paul says:

    Is this issue prevalent in OneFS 8.0 as well?

  3. Spencer says:

    Don’t know much about isilon but I was wondering if the same thing was possible with sub-directories

    • kcarlile says:

      Hi Spencer,

      I’m not sure I follow the question. The script will happily move directories as well as files. Let me know if I’ve misunderstood (which is certainly possible!)

      –Ken

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: