org.finra.herd.service.helper

Class StorageFileHelper

java.lang.Object

org.finra.herd.service.helper.StorageFileHelper

@Component
public class StorageFileHelper
extends Object

A helper class for StorageFile related code.

Constructor Summary

Constructors

Constructor and Description
StorageFileHelper()

Method Summary

Modifier and Type	Method and Description
StorageFile	createStorageFileFromEntity(StorageFileEntity storageFileEntity) Creates a storage file from the storage file entity.
List<StorageFile>	createStorageFilesFromEntities(Collection<StorageFileEntity> storageFileEntities) Creates a list of storage files from the collection of storage file entities.
List<StorageFile>	createStorageFilesFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Creates a list of storage files from the list of S3 object summaries.
List<StorageFile>	getAndValidateStorageFiles(StorageUnitEntity storageUnitEntity, String s3KeyPrefix, String storageName, BusinessObjectDataKey businessObjectDataKey) Retrieves and validates a list of storage files registered with the specified storage unit.
List<StorageFile>	getAndValidateStorageFilesIfPresent(StorageUnitEntity storageUnitEntity, String s3KeyPrefix, String storageName, BusinessObjectDataKey businessObjectDataKey) Retrieves and validates a list of storage files registered with the specified storage unit.
List<String>	getFilePathsFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Returns a list of file paths extracted from the specified list of S3 object summaries.
List<String>	getFilePathsFromStorageFiles(List<StorageFile> storageFiles) Returns a list of file paths extracted from the specified list of storage files.
List<File>	getFiles(List<StorageFile> storageFiles) Returns a list of files extracted from the specified list of storage files.
Map<String,StorageFileEntity>	getStorageFileEntitiesMap(Collection<StorageFileEntity> storageFileEntities) Returns a map of file paths to the storage file entities build from the list of storage file entities.
Map<String,StorageFile>	getStorageFilesMapFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries) Returns a map of file paths to the storage files build from the list of S3 object summaries with map iteration order matching the original list order.
void	validateCreateRequestStorageFiles(List<StorageFile> storageFiles) Validates a list of storage files.
void	validateDownloadedS3Files(String baseDirectory, String s3KeyPrefix, List<StorageFile> storageFiles) Validate downloaded S3 files per specified list of storage files.
void	validateDownloadedS3Files(String baseDirectory, String s3KeyPrefix, StorageUnit storageUnit) Validate downloaded S3 files per storage unit information.
void	validateRegisteredS3Files(List<StorageFile> expectedStorageFiles, List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries, String storageName, BusinessObjectDataKey businessObjectDataKey) Validates registered S3 files per list of expected storage files.
void	validateStorageFile(StorageFile storageFile, String s3BucketName, Map<String,StorageFile> actualS3Keys, boolean validateFileSize) Validates storage file against the actual S3 objects reported by S3.
void	validateStorageFileEntity(StorageFileEntity storageFileEntity, String s3BucketName, Map<String,StorageFile> actualS3Keys, boolean validateFileSize) Validates storage file entity against the actual S3 objects reported by S3.
void	validateStorageFilePaths(Collection<String> storageFilePaths, String s3KeyPrefix, BusinessObjectDataKey businessObjectDataKey, String storageName) Validates a list of storage file paths.
void	validateStorageUnitS3Files(StorageUnit storageUnit, List<String> actualS3Files, String s3KeyPrefix) Validates S3 files per storage unit information.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageFileHelper

public StorageFileHelper()

Method Detail

createStorageFileFromEntity

public StorageFile createStorageFileFromEntity(StorageFileEntity storageFileEntity)

Creates a storage file from the storage file entity.

Parameters:

storageFileEntity - the storage file entity

Returns:

the storage file

createStorageFilesFromEntities

public List<StorageFile> createStorageFilesFromEntities(Collection<StorageFileEntity> storageFileEntities)

Creates a list of storage files from the collection of storage file entities.

Parameters:

storageFileEntities - the collection of storage file entities

Returns:

the list of storage files

createStorageFilesFromS3ObjectSummaries

public List<StorageFile> createStorageFilesFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Creates a list of storage files from the list of S3 object summaries.

Parameters:

s3ObjectSummaries - the list of S3 object summaries

Returns:

the list of storage files

getAndValidateStorageFiles

public List<StorageFile> getAndValidateStorageFiles(StorageUnitEntity storageUnitEntity,
                                                    String s3KeyPrefix,
                                                    String storageName,
                                                    BusinessObjectDataKey businessObjectDataKey)

Retrieves and validates a list of storage files registered with the specified storage unit. This method makes sure that the list of storage files is not empty and that all storage files match the expected s3 key prefix value.

Parameters:

storageUnitEntity - the storage unit entity the storage file paths to be validated

s3KeyPrefix - the S3 key prefix that storage file paths are expected to start with

storageName - the storage name

businessObjectDataKey - the business object data key

Returns:

the list of storage files

getAndValidateStorageFilesIfPresent

public List<StorageFile> getAndValidateStorageFilesIfPresent(StorageUnitEntity storageUnitEntity,
                                                             String s3KeyPrefix,
                                                             String storageName,
                                                             BusinessObjectDataKey businessObjectDataKey)

Retrieves and validates a list of storage files registered with the specified storage unit. This method returns an empty list if storage unit has no storage files.

Parameters:

storageUnitEntity - the storage unit entity the storage file paths to be validated

s3KeyPrefix - the S3 key prefix that storage file paths are expected to start with

storageName - the storage name

businessObjectDataKey - the business object data key

Returns:

the list of storage files

getFilePathsFromS3ObjectSummaries

public List<String> getFilePathsFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Returns a list of file paths extracted from the specified list of S3 object summaries.

Parameters:

s3ObjectSummaries - the list of of S3 object summaries

Returns:

the list of file paths

getFilePathsFromStorageFiles

public List<String> getFilePathsFromStorageFiles(List<StorageFile> storageFiles)

Returns a list of file paths extracted from the specified list of storage files.

Parameters:

storageFiles - the list of storage files

Returns:

the list of file paths

getFiles

public List<File> getFiles(List<StorageFile> storageFiles)

Returns a list of files extracted from the specified list of storage files.

Parameters:

storageFiles - the list of storage files

Returns:

the list of files

getStorageFileEntitiesMap

public Map<String,StorageFileEntity> getStorageFileEntitiesMap(Collection<StorageFileEntity> storageFileEntities)

Returns a map of file paths to the storage file entities build from the list of storage file entities.

Parameters:

storageFileEntities - the collection of storage file entities

Returns:

the map of file paths to storage file entities

getStorageFilesMapFromS3ObjectSummaries

public Map<String,StorageFile> getStorageFilesMapFromS3ObjectSummaries(List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries)

Returns a map of file paths to the storage files build from the list of S3 object summaries with map iteration order matching the original list order.

Parameters:

s3ObjectSummaries - the list of S3 object summaries

Returns:

the map of file paths to storage files

validateCreateRequestStorageFiles

public void validateCreateRequestStorageFiles(List<StorageFile> storageFiles)

Validates a list of storage files. This will validate and trim appropriate fields.

Parameters:

storageFiles - the list of storage files

validateDownloadedS3Files

public void validateDownloadedS3Files(String baseDirectory,
                                      String s3KeyPrefix,
                                      StorageUnit storageUnit)
                               throws IllegalStateException

Validate downloaded S3 files per storage unit information.

Parameters:

baseDirectory - the local parent directory path, relative to which the files are expected to be located

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3

storageUnit - the storage unit that contains a list of storage files to be validated

Throws:

IllegalStateException - if files are not valid

validateDownloadedS3Files

public void validateDownloadedS3Files(String baseDirectory,
                                      String s3KeyPrefix,
                                      List<StorageFile> storageFiles)
                               throws IllegalStateException

Validate downloaded S3 files per specified list of storage files.

Parameters:

baseDirectory - the local parent directory path, relative to which the files are expected to be located

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3

storageFiles - the list of storage files

Throws:

IllegalStateException - if files are not valid

validateRegisteredS3Files

public void validateRegisteredS3Files(List<StorageFile> expectedStorageFiles,
                                      List<com.amazonaws.services.s3.model.S3ObjectSummary> s3ObjectSummaries,
                                      String storageName,
                                      BusinessObjectDataKey businessObjectDataKey)

Validates registered S3 files per list of expected storage files. The validation ignores (does not fail) when detecting unregistered zero byte S3 files.

Parameters:

expectedStorageFiles - the list of expected S3 files represented by storage files

s3ObjectSummaries - the list of actual S3 files represented by S3 object summaries

storageName - the storage name

businessObjectDataKey - the business object data key

validateStorageFile

public void validateStorageFile(StorageFile storageFile,
                                String s3BucketName,
                                Map<String,StorageFile> actualS3Keys,
                                boolean validateFileSize)

Validates storage file against the actual S3 objects reported by S3.

Parameters:

storageFile - the storage file to be validated

s3BucketName - the S3 bucket name

actualS3Keys - the map of storage file paths to storage files as reported by S3

validateFileSize - specifies whether file size validation is required or not

validateStorageFileEntity

public void validateStorageFileEntity(StorageFileEntity storageFileEntity,
                                      String s3BucketName,
                                      Map<String,StorageFile> actualS3Keys,
                                      boolean validateFileSize)

Validates storage file entity against the actual S3 objects reported by S3.

Parameters:

storageFileEntity - the storage file to be validated

s3BucketName - the S3 bucket name

actualS3Keys - the map of storage file paths to storage files as reported by S3

validateFileSize - specifies whether file size validation is required or not

validateStorageFilePaths

public void validateStorageFilePaths(Collection<String> storageFilePaths,
                                     String s3KeyPrefix,
                                     BusinessObjectDataKey businessObjectDataKey,
                                     String storageName)

Validates a list of storage file paths. This method makes sure that all storage file paths match the expected s3 key prefix value.

Parameters:

storageFilePaths - the storage file paths to be validated

s3KeyPrefix - the S3 key prefix that storage file paths are expected to start with

businessObjectDataKey - the business object data key

storageName - the name of the storage that storage files are stored in

validateStorageUnitS3Files

public void validateStorageUnitS3Files(StorageUnit storageUnit,
                                       List<String> actualS3Files,
                                       String s3KeyPrefix)

Validates S3 files per storage unit information.

Parameters:

storageUnit - the storage unit that contains S3 files to be validated

actualS3Files - the list of the actual S3 files

s3KeyPrefix - the S3 key prefix that was prepended to the S3 file paths, when they were uploaded to S3