public class HDFSSynchronization extends java.lang.Object implements ExternalSynchronization
ExternalSynchronization
which registers locks in the HDFS.
Requires locksDir
to be specified. This directory MUST be different that directory
which is possibly stored under "mapreduce.output.fileoutputformat.outputdir"
key.
Otherwise setup of job will fail because the directory will exist before job setup.
Constructor and Description |
---|
HDFSSynchronization(java.lang.String locksDir)
Creates instance of
HDFSSynchronization . |
Modifier and Type | Method and Description |
---|---|
org.apache.hadoop.mapreduce.TaskAttemptID |
acquireTaskAttemptIdLock(org.apache.hadoop.conf.Configuration conf,
int taskId)
Creates unique
TaskAttemptID for given taskId. |
org.apache.hadoop.mapreduce.TaskID |
acquireTaskIdLock(org.apache.hadoop.conf.Configuration conf)
Creates
TaskID with unique id among given job. |
void |
releaseJobIdLock(org.apache.hadoop.conf.Configuration conf)
Deletes lock ids bounded with given job if any exists.
|
boolean |
tryAcquireJobLock(org.apache.hadoop.conf.Configuration conf)
Tries to acquire lock for given job.
|
public HDFSSynchronization(java.lang.String locksDir)
HDFSSynchronization
.locksDir
- directory where locks will be stored. This directory MUST be different that
directory which is possibly stored under "mapreduce.output.fileoutputformat.outputdir"
key. Otherwise setup of job will fail
because the directory will exist before job setup.public boolean tryAcquireJobLock(org.apache.hadoop.conf.Configuration conf)
ExternalSynchronization
tryAcquireJobLock
in interface ExternalSynchronization
conf
- configuration bounded with given job.true
if the lock was acquired, false
otherwise.public void releaseJobIdLock(org.apache.hadoop.conf.Configuration conf)
ExternalSynchronization
releaseJobIdLock
in interface ExternalSynchronization
conf
- hadoop configuration of given job.public org.apache.hadoop.mapreduce.TaskID acquireTaskIdLock(org.apache.hadoop.conf.Configuration conf)
ExternalSynchronization
TaskID
with unique id among given job.acquireTaskIdLock
in interface ExternalSynchronization
conf
- hadoop configuration of given job.TaskID
with unique id among given job.public org.apache.hadoop.mapreduce.TaskAttemptID acquireTaskAttemptIdLock(org.apache.hadoop.conf.Configuration conf, int taskId)
ExternalSynchronization
TaskAttemptID
for given taskId.acquireTaskAttemptIdLock
in interface ExternalSynchronization
conf
- configuration of given task and jobtaskId
- id of the taskTaskAttemptID
for given taskId.