com.spotify.helios.master.reaper

Class JobHistoryReaper

java.lang.Object

com.google.common.util.concurrent.AbstractIdleService

com.spotify.helios.agent.InterruptingScheduledService

com.spotify.helios.master.reaper.JobHistoryReaper

All Implemented Interfaces:

com.google.common.util.concurrent.Service

public class JobHistoryReaper
extends InterruptingScheduledService

Removes job histories whose corresponding jobs don't exist anymore. There are two race conditions where jobs can be deleted but their histories are left behind in ZooKeeper:

1. The master deletes the job (in ZooKeeperMasterModel and then deletes its history. During this deletion, the agent creates a znode. The master's deletion operations fail.

2. The master deletes all relevant history znodes successfully. The agent still hasn't undeployed its job and continues writing history to ZooKeeper. This will recreate deleted history znodes via TaskHistoryWriter.

Solve both of these cases by scheduling an instance of this class. It runs once a day once scheduled.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.common.util.concurrent.Service

com.google.common.util.concurrent.Service.Listener, com.google.common.util.concurrent.Service.State

Constructor Summary

Constructors

Constructor and Description
JobHistoryReaper(MasterModel masterModel, ZooKeeperClient client)

Method Summary

Modifier and Type	Method and Description
protected void	runOneIteration()
protected ScheduledFuture<?>	schedule(Runnable runnable, ScheduledExecutorService executorService)

Methods inherited from class com.spotify.helios.agent.InterruptingScheduledService

beforeIteration, shutDown, startUp

Methods inherited from class com.google.common.util.concurrent.AbstractIdleService

addListener, awaitRunning, awaitRunning, awaitTerminated, awaitTerminated, executor, failureCause, isRunning, serviceName, startAsync, state, stopAsync, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

JobHistoryReaper

public JobHistoryReaper(MasterModel masterModel,
                        ZooKeeperClient client)

Method Detail

runOneIteration

protected void runOneIteration()

Specified by:

runOneIteration in class InterruptingScheduledService

schedule

protected ScheduledFuture<?> schedule(Runnable runnable,
                                      ScheduledExecutorService executorService)

Specified by:

schedule in class InterruptingScheduledService