org.finra.herd.service.impl

Class EmrServiceImpl

java.lang.Object

org.finra.herd.service.impl.EmrServiceImpl

All Implemented Interfaces:

EmrService

@Service
 @Transactional(value="herdTransactionManager")
public class EmrServiceImpl
extends Object
implements EmrService

The EMR service implementation.

Constructor Summary

Constructors

Constructor and Description
EmrServiceImpl()

Method Summary

Modifier and Type	Method and Description
EmrMasterSecurityGroup	addSecurityGroupsToClusterMaster(EmrMasterSecurityGroupAddRequest request) Adds security groups to the master node of an existing EMR Cluster.
protected EmrMasterSecurityGroup	addSecurityGroupsToClusterMasterImpl(EmrMasterSecurityGroupAddRequest request) Adds security groups to the master node of an existing EMR Cluster.
Object	addStepToCluster(Object request) Adds step to an existing EMR Cluster.
protected Object	addStepToClusterImpl(Object request) Adds step to an existing EMR Cluster.
EmrCluster	createCluster(EmrClusterCreateRequest request) Creates a new EMR Cluster.
protected EmrCluster	createClusterImpl(EmrClusterCreateRequest request) Creates a new EMR cluster based on the given request if the cluster with the given name does not already exist.
EmrCluster	getCluster(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto, String emrClusterId, String emrStepId, boolean verbose, String accountId, Boolean retrieveInstanceFleets) Gets details of an existing EMR Cluster.
protected EmrCluster	getClusterImpl(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto, String emrClusterId, String emrStepId, boolean verbose, String accountId, Boolean retrieveInstanceFleets) Gets details of an existing EMR Cluster.
EmrCluster	terminateCluster(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto, boolean overrideTerminationProtection, String emrClusterId, String accountId) Terminates the EMR Cluster.
protected EmrCluster	terminateClusterImpl(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto, boolean overrideTerminationProtection, String emrClusterId, String accountId) Terminates the EMR Cluster.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

EmrServiceImpl

public EmrServiceImpl()

Method Detail

getCluster

@NamespacePermission(fields="#emrClusterAlternateKeyDto?.namespace",
                     permissions=READ)
 @Transactional(propagation=REQUIRES_NEW)
public EmrCluster getCluster(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto,
                                                                                                                                                                                   String emrClusterId,
                                                                                                                                                                                   String emrStepId,
                                                                                                                                                                                   boolean verbose,
                                                                                                                                                                                   String accountId,
                                                                                                                                                                                   Boolean retrieveInstanceFleets)
                                                                                                                                                                            throws Exception

Gets details of an existing EMR Cluster.

This implementation starts a new transaction.

Specified by:

getCluster in interface EmrService

Parameters:

emrClusterAlternateKeyDto - the EMR cluster alternate key

emrClusterId - the id of the cluster to get details

emrStepId - the step id of the step to get details

verbose - parameter for whether to return detailed information

accountId - the optional AWS account that EMR cluster is running in

retrieveInstanceFleets - parameter for whether to retrieve instance fleets

Returns:

the EMR Cluster object with details

Throws:

Exception - if there were any errors

getClusterImpl

protected EmrCluster getClusterImpl(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto,
                                    String emrClusterId,
                                    String emrStepId,
                                    boolean verbose,
                                    String accountId,
                                    Boolean retrieveInstanceFleets)

Gets details of an existing EMR Cluster.

Parameters:

emrClusterAlternateKeyDto - the EMR cluster alternate key

emrClusterId - the cluster id of the cluster to get details

emrStepId - the step id of the step to get details

verbose - parameter for whether to return detailed information

accountId - the optional AWS account that EMR cluster is running in

retrieveInstanceFleets - parameter for whether to retrieve instance fleets

Returns:

the EMR Cluster object with details.

createCluster

@NamespacePermission(fields="#request?.namespace",
                     permissions=EXECUTE)
 @Transactional(propagation=NOT_SUPPORTED)
public EmrCluster createCluster(EmrClusterCreateRequest request)
                                                                                                                                                                 throws Exception

Creates a new EMR Cluster.

This implementation starts a new transaction.

Specified by:

createCluster in interface EmrService

Parameters:

request - the EMR cluster create request

Returns:

the created EMR cluster object

Throws:

Exception - if there were any errors while creating the cluster

createClusterImpl

protected EmrCluster createClusterImpl(EmrClusterCreateRequest request)
                                throws Exception

Creates a new EMR cluster based on the given request if the cluster with the given name does not already exist. If the cluster already exist, returns the information about the existing cluster.

The request must contain:

• A namespace and definition name which refer to an existing EMR cluster definition.
• A valid cluster name to create.

The request may optionally contain:

• A "dry run" flag, which when set to true, no calls to AWS will occur, but validations and override will. Defaults to false.
• An override parameter, which when set, overrides the given parameters in the cluster definition before creating the cluster. Defaults to no override.

A successful response will contain:

• The ID of the cluster that was created, or if the cluster already exists, the ID of the cluster that exists. This field will be null when dry run flag is true.
• The status of the cluster that was created or already exists. The status will normally be "Starting" on successful creations. This field will be null when dry run flag is true
• The namespace, definition name, and cluster name of the cluster.
• The dry run flag, if given in the request.
• An indicator whether the cluster was created or not. If the cluster already exists, the cluster will not be created and this flag will be set to false.
• The definition which was used to create the cluster. If any overrides were given, this definition's values will be the values after the override. This field will be null if the cluster was not created.

Notes:

• At any point of the execution, if there are validation errors, the method will immediately throw an exception.
• Even if the validations pass, AWS may still reject the request, which will cause this method to throw an exception.
• Dry runs do not make any calls to AWS, therefore AWS may still reject the creation request even when a dry run succeeds.

Parameters:

request - - EmrClusterCreateRequest The EMR cluster create request

Returns:

EmrCluster the created EMR cluster object

Throws:

Exception - when the original EMR cluster definition XML is malformed

terminateCluster

@NamespacePermission(fields="#emrClusterAlternateKeyDto?.namespace",
                     permissions=EXECUTE)
 @Transactional(propagation=REQUIRES_NEW)
public EmrCluster terminateCluster(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto,
                                                                                                                                                                                            boolean overrideTerminationProtection,
                                                                                                                                                                                            String emrClusterId,
                                                                                                                                                                                            String accountId)
                                                                                                                                                                                     throws Exception

Terminates the EMR Cluster.

This implementation starts a new transaction.

Specified by:

terminateCluster in interface EmrService

Parameters:

emrClusterAlternateKeyDto - the EMR cluster alternate key

overrideTerminationProtection - parameter for whether to override termination protection

emrClusterId - the id of the cluster

accountId - the optional AWS account that EMR cluster is running in

Returns:

the terminated EMR cluster object

Throws:

Exception - if there were any errors while terminating the cluster

terminateClusterImpl

protected EmrCluster terminateClusterImpl(EmrClusterAlternateKeyDto emrClusterAlternateKeyDto,
                                          boolean overrideTerminationProtection,
                                          String emrClusterId,
                                          String accountId)

Terminates the EMR Cluster.

Parameters:

emrClusterAlternateKeyDto - the EMR cluster alternate key

overrideTerminationProtection - parameter for whether to override termination protection

emrClusterId - The EMR cluster ID

accountId - The account Id

Returns:

the terminated EMR cluster object

addStepToCluster

@NamespacePermission(fields="#request?.namespace",
                     permissions=EXECUTE)
 @Transactional(propagation=REQUIRES_NEW)
public Object addStepToCluster(Object request)
                                                                                                                                                               throws Exception

Adds step to an existing EMR Cluster.

There are four serializable objects supported currently. They are 1: ShellStep - For shell scripts 2: HiveStep - For hive scripts 3: HadoopJarStep - For Custom Map Reduce Jar files and 4: PigStep - For Pig scripts.

This implementation starts a new transaction.

Specified by:

addStepToCluster in interface EmrService

Parameters:

request - the EMR steps add request

Returns:

the EMR steps add object with added steps

Throws:

Exception - if there were any errors while adding a step to the cluster.

addStepToClusterImpl

protected Object addStepToClusterImpl(Object request)
                               throws Exception

Adds step to an existing EMR Cluster.

Parameters:

request - the EMR steps add request

Returns:

the EMR step add object with added steps

Throws:

Exception - if there were any errors while adding a step to the cluster.

addSecurityGroupsToClusterMaster

@NamespacePermission(fields="#request?.namespace",
                     permissions=WRITE)
 @Transactional(propagation=REQUIRES_NEW)
public EmrMasterSecurityGroup addSecurityGroupsToClusterMaster(EmrMasterSecurityGroupAddRequest request)
                                                                                                                                                                                             throws Exception

Adds security groups to the master node of an existing EMR Cluster.

This implementation starts a new transaction.

Specified by:

addSecurityGroupsToClusterMaster in interface EmrService

Parameters:

request - the EMR master security group add request

Returns:

the added EMR master security groups

Throws:

Exception - if there were any errors adding the security groups to the cluster master

addSecurityGroupsToClusterMasterImpl

protected EmrMasterSecurityGroup addSecurityGroupsToClusterMasterImpl(EmrMasterSecurityGroupAddRequest request)
                                                               throws Exception

Adds security groups to the master node of an existing EMR Cluster.

Parameters:

request - the EMR master security group add request

Returns:

the added EMR master security groups

Throws:

Exception - if there were any errors adding the security groups to the cluster master.