com.spotify.helios.master

Class ZooKeeperMasterModel

java.lang.Object

com.spotify.helios.master.ZooKeeperMasterModel

All Implemented Interfaces:

MasterModel

public class ZooKeeperMasterModel
extends Object
implements MasterModel

The Helios Master's view into ZooKeeper.

Field Summary

Fields

Modifier and Type	Field and Description
static com.fasterxml.jackson.core.type.TypeReference<com.spotify.helios.common.descriptors.AgentInfo>	AGENT_INFO_TYPE
static Map<com.spotify.helios.common.descriptors.JobId,com.spotify.helios.common.descriptors.TaskStatus>	EMPTY_STATUSES
static com.fasterxml.jackson.core.type.TypeReference<com.spotify.helios.common.descriptors.HostInfo>	HOST_INFO_TYPE
static com.fasterxml.jackson.core.type.TypeReference<List<String>>	STRING_LIST_TYPE
static com.fasterxml.jackson.core.type.TypeReference<Map<String,String>>	STRING_MAP_TYPE

Constructor Summary

Constructors

Constructor and Description
ZooKeeperMasterModel(ZooKeeperClientProvider provider, String name, List<EventSender> eventSenders, String deploymentGroupEventTopic) Constructor.

Method Summary

Modifier and Type	Method and Description
void	addDeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup deploymentGroup) Create a deployment group.
void	addJob(com.spotify.helios.common.descriptors.Job job) Adds a job into the configuration.
void	deployJob(String host, com.spotify.helios.common.descriptors.Deployment job)
void	deployJob(String host, com.spotify.helios.common.descriptors.Deployment deployment, String token) Creates a config entry within the specified agent to un/deploy a job, or more generally, change the deployment status according to the Goal value in Deployment.
void	deregisterHost(String host) Undoes the effect of registerHost(String, String).
com.spotify.helios.common.descriptors.AgentInfo	getAgentInfo(String host)
com.spotify.helios.common.descriptors.Deployment	getDeployment(String host, com.spotify.helios.common.descriptors.JobId jobId) Returns the current deployment state of jobId on host.
com.spotify.helios.common.descriptors.DeploymentGroup	getDeploymentGroup(String name)
List<String>	getDeploymentGroupHosts(String name)
Map<String,com.spotify.helios.common.descriptors.DeploymentGroup>	getDeploymentGroups() Returns a Map of deployment group name to DeploymentGroup objects for all of the deployment groups known.
com.spotify.helios.common.descriptors.DeploymentGroupStatus	getDeploymentGroupStatus(String name)
Map<String,String>	getHostLabels(String host) Returns labels for host.
com.spotify.helios.common.descriptors.HostStatus	getHostStatus(String host) Returns the current status of the host named by host.
com.spotify.helios.common.descriptors.Job	getJob(com.spotify.helios.common.descriptors.JobId id) Returns the job configuration for the job specified by id as a Job object.
List<com.spotify.helios.common.descriptors.TaskStatusEvent>	getJobHistory(com.spotify.helios.common.descriptors.JobId jobId) Given a jobId, returns the N most recent events in its history in the cluster.
List<com.spotify.helios.common.descriptors.TaskStatusEvent>	getJobHistory(com.spotify.helios.common.descriptors.JobId jobId, String host) Given a jobId and host, returns the N most recent events in its history on that host in the cluster.
Map<com.spotify.helios.common.descriptors.JobId,com.spotify.helios.common.descriptors.Job>	getJobs() Returns a Map of JobId to Job objects for all of the jobs known.
com.spotify.helios.common.descriptors.JobStatus	getJobStatus(com.spotify.helios.common.descriptors.JobId jobId) Returns the current job status as a JobStatus object.
List<String>	getRunningMasters() Returns a list of the host names of the currently running masters.
boolean	isHostUp(String host)
List<String>	listHosts() Returns a list of the hosts/agents that have been registered.
void	registerHost(String host, String id) Registers a host into ZooKeeper.
void	removeDeploymentGroup(String name) Remove a deployment group.
com.spotify.helios.common.descriptors.Job	removeJob(com.spotify.helios.common.descriptors.JobId jobId)
com.spotify.helios.common.descriptors.Job	removeJob(com.spotify.helios.common.descriptors.JobId id, String token) Deletes a job from ZooKeeper.
void	rollingUpdate(com.spotify.helios.common.descriptors.DeploymentGroup deploymentGroup, com.spotify.helios.common.descriptors.JobId jobId, com.spotify.helios.common.descriptors.RolloutOptions options)
void	rollingUpdateStep()
void	stopDeploymentGroup(String deploymentGroupName)
com.spotify.helios.common.descriptors.Deployment	undeployJob(String host, com.spotify.helios.common.descriptors.JobId jobId)
com.spotify.helios.common.descriptors.Deployment	undeployJob(String host, com.spotify.helios.common.descriptors.JobId jobId, String token) Undeploys the job specified by jobId on host.
void	updateDeployment(String host, com.spotify.helios.common.descriptors.Deployment deployment)
void	updateDeployment(String host, com.spotify.helios.common.descriptors.Deployment deployment, String token) Used to update the existing deployment of a job.
void	updateDeploymentGroupHosts(String groupName, List<String> hosts)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EMPTY_STATUSES

public static final Map<com.spotify.helios.common.descriptors.JobId,com.spotify.helios.common.descriptors.TaskStatus> EMPTY_STATUSES

HOST_INFO_TYPE

public static final com.fasterxml.jackson.core.type.TypeReference<com.spotify.helios.common.descriptors.HostInfo> HOST_INFO_TYPE

AGENT_INFO_TYPE

public static final com.fasterxml.jackson.core.type.TypeReference<com.spotify.helios.common.descriptors.AgentInfo> AGENT_INFO_TYPE

STRING_MAP_TYPE

public static final com.fasterxml.jackson.core.type.TypeReference<Map<String,String>> STRING_MAP_TYPE

STRING_LIST_TYPE

public static final com.fasterxml.jackson.core.type.TypeReference<List<String>> STRING_LIST_TYPE

Constructor Detail

ZooKeeperMasterModel

public ZooKeeperMasterModel(ZooKeeperClientProvider provider,
                            String name,
                            List<EventSender> eventSenders,
                            String deploymentGroupEventTopic)

Constructor.

Parameters:

provider - ZooKeeperClientProvider

name - The hostname of the machine running the MasterModel

eventSenders - EventSender

Method Detail

registerHost

public void registerHost(String host,
                         String id)

Registers a host into ZooKeeper. The id is initially generated randomly by the Agent and persisted on disk. This way, in the event that you have two agents attempting to register with the same value of @{code host}, the first one will win.

Specified by:

registerHost in interface MasterModel

listHosts

public List<String> listHosts()

Returns a list of the hosts/agents that have been registered.

Specified by:

listHosts in interface MasterModel

getRunningMasters

public List<String> getRunningMasters()

Returns a list of the host names of the currently running masters.

Specified by:

getRunningMasters in interface MasterModel

deregisterHost

public void deregisterHost(String host)
                    throws HostNotFoundException,
                           HostStillInUseException

Undoes the effect of registerHost(String, String). Cleans up any leftover host-related things.

Specified by:

deregisterHost in interface MasterModel

Throws:

HostNotFoundException

HostStillInUseException

addJob

public void addJob(com.spotify.helios.common.descriptors.Job job)
            throws JobExistsException

Adds a job into the configuration.

Specified by:

addJob in interface MasterModel

Throws:

JobExistsException

getJobHistory

public List<com.spotify.helios.common.descriptors.TaskStatusEvent> getJobHistory(com.spotify.helios.common.descriptors.JobId jobId)
                                                                          throws JobDoesNotExistException

Given a jobId, returns the N most recent events in its history in the cluster.

Specified by:

getJobHistory in interface MasterModel

Throws:

JobDoesNotExistException

getJobHistory

public List<com.spotify.helios.common.descriptors.TaskStatusEvent> getJobHistory(com.spotify.helios.common.descriptors.JobId jobId,
                                                                                 String host)
                                                                          throws JobDoesNotExistException

Given a jobId and host, returns the N most recent events in its history on that host in the cluster.

Specified by:

getJobHistory in interface MasterModel

Throws:

JobDoesNotExistException

addDeploymentGroup

public void addDeploymentGroup(com.spotify.helios.common.descriptors.DeploymentGroup deploymentGroup)
                        throws DeploymentGroupExistsException

Create a deployment group.

If successful, the following ZK nodes will be created:

• /config/deployment-groups/[group-name]
• /status/deployment-groups/[group-name]
• /status/deployment-groups/[group-name]/hosts

These nodes are guaranteed to exist until the DG is removed.

If the operation fails no ZK nodes will be created. If any of the nodes above already exist the operation will fail.

Specified by:

addDeploymentGroup in interface MasterModel

Throws:

DeploymentGroupExistsException - If a DG with the same name already exists.

getDeploymentGroup

public com.spotify.helios.common.descriptors.DeploymentGroup getDeploymentGroup(String name)
                                                                         throws DeploymentGroupDoesNotExistException

Specified by:

getDeploymentGroup in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

removeDeploymentGroup

public void removeDeploymentGroup(String name)
                           throws DeploymentGroupDoesNotExistException

Remove a deployment group.

If successful, all ZK nodes associated with the DG will be deleted. Specifically these nodes are guaranteed to be non-existent after a successful remove (not all of them might exist before, though):

• /config/deployment-groups/[group-name]
• /status/deployment-groups/[group-name]
• /status/deployment-groups/[group-name]/hosts
• /status/deployment-groups/[group-name]/removed
• /status/deployment-group-tasks/[group-name]

If the operation fails no ZK nodes will be removed.

Specified by:

removeDeploymentGroup in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException - If the DG does not exist.

updateDeploymentGroupHosts

public void updateDeploymentGroupHosts(String groupName,
                                       List<String> hosts)
                                throws DeploymentGroupDoesNotExistException

Specified by:

updateDeploymentGroupHosts in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

rollingUpdate

public void rollingUpdate(com.spotify.helios.common.descriptors.DeploymentGroup deploymentGroup,
                          com.spotify.helios.common.descriptors.JobId jobId,
                          com.spotify.helios.common.descriptors.RolloutOptions options)
                   throws DeploymentGroupDoesNotExistException,
                          JobDoesNotExistException

Specified by:

rollingUpdate in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

JobDoesNotExistException

rollingUpdateStep

public void rollingUpdateStep()

Specified by:

rollingUpdateStep in interface MasterModel

stopDeploymentGroup

public void stopDeploymentGroup(String deploymentGroupName)
                         throws DeploymentGroupDoesNotExistException

Specified by:

stopDeploymentGroup in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

getDeploymentGroups

public Map<String,com.spotify.helios.common.descriptors.DeploymentGroup> getDeploymentGroups()

Returns a Map of deployment group name to DeploymentGroup objects for all of the deployment groups known.

Specified by:

getDeploymentGroups in interface MasterModel

getDeploymentGroupStatus

public com.spotify.helios.common.descriptors.DeploymentGroupStatus getDeploymentGroupStatus(String name)
                                                                                     throws DeploymentGroupDoesNotExistException

Specified by:

getDeploymentGroupStatus in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

getDeploymentGroupHosts

public List<String> getDeploymentGroupHosts(String name)
                                     throws DeploymentGroupDoesNotExistException

Specified by:

getDeploymentGroupHosts in interface MasterModel

Throws:

DeploymentGroupDoesNotExistException

getJob

public com.spotify.helios.common.descriptors.Job getJob(com.spotify.helios.common.descriptors.JobId id)

Returns the job configuration for the job specified by id as a Job object. A return value of null indicates the job doesn't exist.

Specified by:

getJob in interface MasterModel

getJobs

public Map<com.spotify.helios.common.descriptors.JobId,com.spotify.helios.common.descriptors.Job> getJobs()

Returns a Map of JobId to Job objects for all of the jobs known.

Specified by:

getJobs in interface MasterModel

getJobStatus

public com.spotify.helios.common.descriptors.JobStatus getJobStatus(com.spotify.helios.common.descriptors.JobId jobId)

Returns the current job status as a JobStatus object.

Specified by:

getJobStatus in interface MasterModel

removeJob

public com.spotify.helios.common.descriptors.Job removeJob(com.spotify.helios.common.descriptors.JobId jobId)
                                                    throws JobDoesNotExistException,
                                                           JobStillDeployedException

Specified by:

removeJob in interface MasterModel

Throws:

JobDoesNotExistException

JobStillDeployedException

removeJob

public com.spotify.helios.common.descriptors.Job removeJob(com.spotify.helios.common.descriptors.JobId id,
                                                           String token)
                                                    throws JobDoesNotExistException,
                                                           JobStillDeployedException,
                                                           TokenVerificationException

Deletes a job from ZooKeeper. Ensures that job is not currently running anywhere.

Specified by:

removeJob in interface MasterModel

Throws:

JobDoesNotExistException

JobStillDeployedException

TokenVerificationException

deployJob

public void deployJob(String host,
                      com.spotify.helios.common.descriptors.Deployment job)
               throws HostNotFoundException,
                      JobAlreadyDeployedException,
                      JobDoesNotExistException,
                      JobPortAllocationConflictException

Specified by:

deployJob in interface MasterModel

Throws:

HostNotFoundException

JobAlreadyDeployedException

JobDoesNotExistException

JobPortAllocationConflictException

deployJob

public void deployJob(String host,
                      com.spotify.helios.common.descriptors.Deployment deployment,
                      String token)
               throws JobDoesNotExistException,
                      JobAlreadyDeployedException,
                      HostNotFoundException,
                      JobPortAllocationConflictException,
                      TokenVerificationException

Creates a config entry within the specified agent to un/deploy a job, or more generally, change the deployment status according to the Goal value in Deployment.

Specified by:

deployJob in interface MasterModel

Throws:

JobDoesNotExistException

JobAlreadyDeployedException

HostNotFoundException

JobPortAllocationConflictException

TokenVerificationException

updateDeployment

public void updateDeployment(String host,
                             com.spotify.helios.common.descriptors.Deployment deployment)
                      throws HostNotFoundException,
                             JobNotDeployedException

Specified by:

updateDeployment in interface MasterModel

Throws:

HostNotFoundException

JobNotDeployedException

updateDeployment

public void updateDeployment(String host,
                             com.spotify.helios.common.descriptors.Deployment deployment,
                             String token)
                      throws HostNotFoundException,
                             JobNotDeployedException,
                             TokenVerificationException

Used to update the existing deployment of a job.

Specified by:

updateDeployment in interface MasterModel

Throws:

HostNotFoundException

JobNotDeployedException

TokenVerificationException

getDeployment

public com.spotify.helios.common.descriptors.Deployment getDeployment(String host,
                                                                      com.spotify.helios.common.descriptors.JobId jobId)

Returns the current deployment state of jobId on host.

Specified by:

getDeployment in interface MasterModel

getHostStatus

public com.spotify.helios.common.descriptors.HostStatus getHostStatus(String host)

Returns the current status of the host named by host.

Specified by:

getHostStatus in interface MasterModel

getHostLabels

public Map<String,String> getHostLabels(String host)

Description copied from interface: MasterModel

Returns labels for host. Returns an empty map for hosts not found in the store.

Specified by:

getHostLabels in interface MasterModel

isHostUp

public boolean isHostUp(String host)

Specified by:

isHostUp in interface MasterModel

getAgentInfo

public com.spotify.helios.common.descriptors.AgentInfo getAgentInfo(String host)

Specified by:

getAgentInfo in interface MasterModel

undeployJob

public com.spotify.helios.common.descriptors.Deployment undeployJob(String host,
                                                                    com.spotify.helios.common.descriptors.JobId jobId)
                                                             throws HostNotFoundException,
                                                                    JobNotDeployedException

Specified by:

undeployJob in interface MasterModel

Throws:

HostNotFoundException

JobNotDeployedException

undeployJob

public com.spotify.helios.common.descriptors.Deployment undeployJob(String host,
                                                                    com.spotify.helios.common.descriptors.JobId jobId,
                                                                    String token)
                                                             throws HostNotFoundException,
                                                                    JobNotDeployedException,
                                                                    TokenVerificationException

Undeploys the job specified by jobId on host.

Specified by:

undeployJob in interface MasterModel

Throws:

HostNotFoundException

JobNotDeployedException

TokenVerificationException