org.apache.hadoop.mapreduce.filecache

Class ClientDistributedCacheManager

java.lang.Object

org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager

@InterfaceAudience.Private
public class ClientDistributedCacheManager
extends Object

Manages internal configuration of the cache by the client for job submission.

Constructor Summary

Constructors

Constructor and Description
ClientDistributedCacheManager()

Method Summary

Methods

Modifier and Type	Method and Description
static void	determineCacheVisibilities(org.apache.hadoop.conf.Configuration job, Map<URI,org.apache.hadoop.fs.FileStatus> statCache) Determines the visibilities of the distributed cache files and archives.
static void	determineTimestamps(org.apache.hadoop.conf.Configuration job, Map<URI,org.apache.hadoop.fs.FileStatus> statCache) Determines timestamps of files to be cached, and stores those in the configuration.
static void	determineTimestampsAndCacheVisibilities(org.apache.hadoop.conf.Configuration job) Determines timestamps of files to be cached, and stores those in the configuration.
static void	getDelegationTokens(org.apache.hadoop.conf.Configuration job, org.apache.hadoop.security.Credentials credentials) For each archive or cache file - get the corresponding delegation token

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClientDistributedCacheManager

public ClientDistributedCacheManager()

Method Detail

determineTimestampsAndCacheVisibilities

public static void determineTimestampsAndCacheVisibilities(org.apache.hadoop.conf.Configuration job)
                                                    throws IOException

Determines timestamps of files to be cached, and stores those in the configuration. Determines the visibilities of the distributed cache files and archives. The visibility of a cache path is "public" if the leaf component has READ permissions for others, and the parent subdirs have EXECUTE permissions for others. This is an internal method!

Parameters:

job -

Throws:

IOException

determineTimestamps

public static void determineTimestamps(org.apache.hadoop.conf.Configuration job,
                       Map<URI,org.apache.hadoop.fs.FileStatus> statCache)
                                throws IOException

Determines timestamps of files to be cached, and stores those in the configuration. This is intended to be used internally by JobClient after all cache files have been added. This is an internal method!

Parameters:

job - Configuration of a job.

Throws:

IOException

getDelegationTokens

public static void getDelegationTokens(org.apache.hadoop.conf.Configuration job,
                       org.apache.hadoop.security.Credentials credentials)
                                throws IOException

For each archive or cache file - get the corresponding delegation token

Parameters:

job -

credentials -

Throws:

IOException

determineCacheVisibilities

public static void determineCacheVisibilities(org.apache.hadoop.conf.Configuration job,
                              Map<URI,org.apache.hadoop.fs.FileStatus> statCache)
                                       throws IOException

Determines the visibilities of the distributed cache files and archives. The visibility of a cache path is "public" if the leaf component has READ permissions for others, and the parent subdirs have EXECUTE permissions for others

Parameters:

job -

Throws:

IOException