org.finra.herd.dao

Interface EmrDao

All Known Implementing Classes:

EmrDaoImpl

public interface EmrDao

A DAO for Amazon AWS EMR.

Method Summary

Modifier and Type	Method and Description
List<String>	addEmrMasterSecurityGroups(String clusterId, List<String> securityGroups, AwsParamsDto awsParams) Add Security groups to the master node of EMR cluster.
String	addEmrStep(String clusterId, com.amazonaws.services.elasticmapreduce.model.StepConfig emrStepConfig, AwsParamsDto awsParamsDto) Add an EMR Step.
String	createEmrCluster(String clusterName, EmrClusterDefinition emrClusterDefinition, AwsParamsDto awsParams) Create the EMR cluster.
com.amazonaws.services.elasticmapreduce.model.ClusterSummary	getActiveEmrClusterByName(String clusterName, AwsParamsDto awsParams) Get an Active EMR cluster by the cluster name.
com.amazonaws.services.elasticmapreduce.model.StepSummary	getClusterActiveStep(String clusterId, AwsParamsDto awsParamsDto) Gets the active step on the cluster if any.
com.amazonaws.services.elasticmapreduce.model.Step	getClusterStep(String clusterId, String stepId, AwsParamsDto awsParamsDto) Gets the step on the cluster.
com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient	getEmrClient(AwsParamsDto awsParamsDto) Create the EMR client with the given proxy and access key details.
com.amazonaws.services.elasticmapreduce.model.Cluster	getEmrClusterById(String clusterId, AwsParamsDto awsParams) Get EMR cluster by cluster Id.
String	getEmrClusterStatusById(String clusterId, AwsParamsDto awsParams) Get EMR cluster status by cluster Id.
com.amazonaws.services.elasticmapreduce.model.Instance	getEmrMasterInstance(String clusterId, AwsParamsDto awsParams) Gets the master instance of the EMR cluster.
com.amazonaws.services.elasticmapreduce.model.ListInstanceFleetsResult	getListInstanceFleetsResult(String clusterId, AwsParamsDto awsParams) Get the instance fleets
void	terminateEmrCluster(String clusterId, boolean overrideTerminationProtection, AwsParamsDto awsParams) Terminates the EMR cluster.

Method Detail

addEmrMasterSecurityGroups

List<String> addEmrMasterSecurityGroups(String clusterId,
                                        List<String> securityGroups,
                                        AwsParamsDto awsParams)
                                 throws Exception

Add Security groups to the master node of EMR cluster.

Parameters:

clusterId - EMR cluster Id.

securityGroups - the security groups list.

awsParams - the proxy details.

Returns:

the security groups that were added.

Throws:

Exception

addEmrStep

String addEmrStep(String clusterId,
                  com.amazonaws.services.elasticmapreduce.model.StepConfig emrStepConfig,
                  AwsParamsDto awsParamsDto)
           throws Exception

Add an EMR Step. This method adds the step to EMR cluster based on the input.

Parameters:

clusterId - EMR cluster ID.

emrStepConfig - the EMR step config to be added.

awsParamsDto - the proxy details.

There are four serializable objects supported currently. They are 1: ShellStep - For shell scripts 2: HiveStep - For hive scripts 3: HadoopJarStep - For Custom Map Reduce Jar files and 4: PigStep - For Pig scripts.

Returns:

the step id

Throws:

Exception

createEmrCluster

String createEmrCluster(String clusterName,
                        EmrClusterDefinition emrClusterDefinition,
                        AwsParamsDto awsParams)

Create the EMR cluster.

Parameters:

awsParams - AWS related parameters for access/secret keys and proxy details.

emrClusterDefinition - the EMR cluster definition that contains all the EMR parameters.

clusterName - the cluster name value.

Returns:

the cluster Id.

getActiveEmrClusterByName

com.amazonaws.services.elasticmapreduce.model.ClusterSummary getActiveEmrClusterByName(String clusterName,
                                                                                       AwsParamsDto awsParams)

Get an Active EMR cluster by the cluster name. Cluster only in following states are returned: ClusterState.BOOTSTRAPPING, ClusterState.RUNNING, ClusterState.STARTING, ClusterState.WAITING

Parameters:

awsParams - AWS related parameters for access/secret keys and proxy details.

clusterName - the cluster name value.

Returns:

the ClusterSummary object.

getClusterActiveStep

com.amazonaws.services.elasticmapreduce.model.StepSummary getClusterActiveStep(String clusterId,
                                                                               AwsParamsDto awsParamsDto)

Gets the active step on the cluster if any.

Parameters:

clusterId - the cluster id.

awsParamsDto - AWS related parameters for access/secret keys and proxy details.

Returns:

the step summary object.

getClusterStep

com.amazonaws.services.elasticmapreduce.model.Step getClusterStep(String clusterId,
                                                                  String stepId,
                                                                  AwsParamsDto awsParamsDto)

Gets the step on the cluster.

Parameters:

clusterId - the cluster id.

stepId - the step id to get details of.

awsParamsDto - AWS related parameters for access/secret keys and proxy details.

Returns:

the step object.

getEmrClient

com.amazonaws.services.elasticmapreduce.AmazonElasticMapReduceClient getEmrClient(AwsParamsDto awsParamsDto)

Create the EMR client with the given proxy and access key details.

Parameters:

awsParamsDto - AWS related parameters for access/secret keys and proxy details.

Returns:

the AmazonElasticMapReduceClient object.

getEmrClusterById

com.amazonaws.services.elasticmapreduce.model.Cluster getEmrClusterById(String clusterId,
                                                                        AwsParamsDto awsParams)

Get EMR cluster by cluster Id.

Parameters:

clusterId - the job Id returned by EMR for the cluster.

awsParams - AWS related parameters for access/secret keys and proxy details.

Returns:

the cluster status.

getEmrClusterStatusById

String getEmrClusterStatusById(String clusterId,
                               AwsParamsDto awsParams)

Get EMR cluster status by cluster Id.

Parameters:

clusterId - the job Id returned by EMR for the cluster.

awsParams - AWS related parameters for access/secret keys and proxy details.

Returns:

the cluster status.

getEmrMasterInstance

com.amazonaws.services.elasticmapreduce.model.Instance getEmrMasterInstance(String clusterId,
                                                                            AwsParamsDto awsParams)
                                                                     throws Exception

Gets the master instance of the EMR cluster.

Parameters:

clusterId - EMR cluster id.

awsParams - the proxy details.

Returns:

the master instance of the cluster.

Throws:

Exception

terminateEmrCluster

void terminateEmrCluster(String clusterId,
                         boolean overrideTerminationProtection,
                         AwsParamsDto awsParams)

Terminates the EMR cluster.

Parameters:

clusterId - the cluster Id.

awsParams - AWS related parameters for access/secret keys and proxy details.

getListInstanceFleetsResult

com.amazonaws.services.elasticmapreduce.model.ListInstanceFleetsResult getListInstanceFleetsResult(String clusterId,
                                                                                                   AwsParamsDto awsParams)

Get the instance fleets

Parameters:

clusterId - the cluster Id.

awsParams - AWS related parameters for access/secret keys and proxy details.

Returns:

list instance fleets result