Package flyteidl.core

Class Tasks.DataLoadingConfig

  • All Implemented Interfaces:
    com.google.protobuf.Message, com.google.protobuf.MessageLite, com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder, Tasks.DataLoadingConfigOrBuilder, Serializable
    Enclosing class:
    Tasks
    public static final class Tasks.DataLoadingConfig
extends com.google.protobuf.GeneratedMessageV3
implements Tasks.DataLoadingConfigOrBuilder
     This configuration allows executing raw containers in Flyte using the Flyte CoPilot system.
 Flyte CoPilot, eliminates the needs of flytekit or sdk inside the container. Any inputs required by the users container are side-loaded in the input_path
 Any outputs generated by the user container - within output_path are automatically uploaded.
    Protobuf type flyteidl.core.DataLoadingConfig
    See Also:
    Serialized Form

    • Constructor Detail

      • DataLoadingConfig

        private DataLoadingConfig​(com.google.protobuf.GeneratedMessageV3.Builder<?> builder)

      • DataLoadingConfig

        private DataLoadingConfig()

    • Method Detail

      • newInstance

        protected Object newInstance​(com.google.protobuf.GeneratedMessageV3.UnusedPrivateParameter unused)
        Overrides:
        newInstance in class com.google.protobuf.GeneratedMessageV3

      • getUnknownFields

        public final com.google.protobuf.UnknownFieldSet getUnknownFields()
        Specified by:
        getUnknownFields in interface com.google.protobuf.MessageOrBuilder
        Overrides:
        getUnknownFields in class com.google.protobuf.GeneratedMessageV3

      • getDescriptor

        public static final com.google.protobuf.Descriptors.Descriptor getDescriptor()

      • internalGetFieldAccessorTable

        protected com.google.protobuf.GeneratedMessageV3.FieldAccessorTable internalGetFieldAccessorTable()
        Specified by:
        internalGetFieldAccessorTable in class com.google.protobuf.GeneratedMessageV3

      • getEnabled

        public boolean getEnabled()
         Flag enables DataLoading Config. If this is not set, data loading will not be used!
        bool enabled = 1;
        Specified by:
        getEnabled in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The enabled.

      • getInputPath

        public String getInputPath()
         File system path (start at root). This folder will contain all the inputs exploded to a separate file.
 Example, if the input interface needs (x: int, y: blob, z: multipart_blob) and the input path is '/var/flyte/inputs', then the file system will look like
 /var/flyte/inputs/inputs.<metadata format dependent -> .pb .json .yaml> -> Format as defined previously. The Blob and Multipart blob will reference local filesystem instead of remote locations
 /var/flyte/inputs/x -> X is a file that contains the value of x (integer) in string format
 /var/flyte/inputs/y -> Y is a file in Binary format
 /var/flyte/inputs/z/... -> Note Z itself is a directory
 More information about the protocol - refer to docs #TODO reference docs here
        string input_path = 2;
        Specified by:
        getInputPath in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The inputPath.

      • getInputPathBytes

        public com.google.protobuf.ByteString getInputPathBytes()
         File system path (start at root). This folder will contain all the inputs exploded to a separate file.
 Example, if the input interface needs (x: int, y: blob, z: multipart_blob) and the input path is '/var/flyte/inputs', then the file system will look like
 /var/flyte/inputs/inputs.<metadata format dependent -> .pb .json .yaml> -> Format as defined previously. The Blob and Multipart blob will reference local filesystem instead of remote locations
 /var/flyte/inputs/x -> X is a file that contains the value of x (integer) in string format
 /var/flyte/inputs/y -> Y is a file in Binary format
 /var/flyte/inputs/z/... -> Note Z itself is a directory
 More information about the protocol - refer to docs #TODO reference docs here
        string input_path = 2;
        Specified by:
        getInputPathBytes in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The bytes for inputPath.

      • getOutputPath

        public String getOutputPath()
         File system path (start at root). This folder should contain all the outputs for the task as individual files and/or an error text file
        string output_path = 3;
        Specified by:
        getOutputPath in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The outputPath.

      • getOutputPathBytes

        public com.google.protobuf.ByteString getOutputPathBytes()
         File system path (start at root). This folder should contain all the outputs for the task as individual files and/or an error text file
        string output_path = 3;
        Specified by:
        getOutputPathBytes in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The bytes for outputPath.

      • getFormatValue

        public int getFormatValue()
         In the inputs folder, there will be an additional summary/metadata file that contains references to all files or inlined primitive values.
 This format decides the actual encoding for the data. Refer to the encoding to understand the specifics of the contents and the encoding
        .flyteidl.core.DataLoadingConfig.LiteralMapFormat format = 4;
        Specified by:
        getFormatValue in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The enum numeric value on the wire for format.

      • getFormat

        public Tasks.DataLoadingConfig.LiteralMapFormat getFormat()
         In the inputs folder, there will be an additional summary/metadata file that contains references to all files or inlined primitive values.
 This format decides the actual encoding for the data. Refer to the encoding to understand the specifics of the contents and the encoding
        .flyteidl.core.DataLoadingConfig.LiteralMapFormat format = 4;
        Specified by:
        getFormat in interface Tasks.DataLoadingConfigOrBuilder
        Returns:
        The format.

      • isInitialized

        public final boolean isInitialized()
        Specified by:
        isInitialized in interface com.google.protobuf.MessageLiteOrBuilder
        Overrides:
        isInitialized in class com.google.protobuf.GeneratedMessageV3

      • writeTo

        public void writeTo​(com.google.protobuf.CodedOutputStream output)
             throws IOException
        Specified by:
        writeTo in interface com.google.protobuf.MessageLite
        Overrides:
        writeTo in class com.google.protobuf.GeneratedMessageV3
        Throws:
        IOException

      • getSerializedSize

        public int getSerializedSize()
        Specified by:
        getSerializedSize in interface com.google.protobuf.MessageLite
        Overrides:
        getSerializedSize in class com.google.protobuf.GeneratedMessageV3

      • equals

        public boolean equals​(Object obj)
        Specified by:
        equals in interface com.google.protobuf.Message
        Overrides:
        equals in class com.google.protobuf.AbstractMessage

      • hashCode

        public int hashCode()
        Specified by:
        hashCode in interface com.google.protobuf.Message
        Overrides:
        hashCode in class com.google.protobuf.AbstractMessage

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(ByteBuffer data)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(ByteBuffer data,
                                                com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(com.google.protobuf.ByteString data)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(com.google.protobuf.ByteString data,
                                                com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(byte[] data)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Tasks.DataLoadingConfig parseFrom​(byte[] data,
                                                com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                         throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • newBuilderForType

        public Tasks.DataLoadingConfig.Builder newBuilderForType()
        Specified by:
        newBuilderForType in interface com.google.protobuf.Message
        Specified by:
        newBuilderForType in interface com.google.protobuf.MessageLite

      • toBuilder

        public Tasks.DataLoadingConfig.Builder toBuilder()
        Specified by:
        toBuilder in interface com.google.protobuf.Message
        Specified by:
        toBuilder in interface com.google.protobuf.MessageLite

      • newBuilderForType

        protected Tasks.DataLoadingConfig.Builder newBuilderForType​(com.google.protobuf.GeneratedMessageV3.BuilderParent parent)
        Specified by:
        newBuilderForType in class com.google.protobuf.GeneratedMessageV3

      • getParserForType

        public com.google.protobuf.Parser<Tasks.DataLoadingConfig> getParserForType()
        Specified by:
        getParserForType in interface com.google.protobuf.Message
        Specified by:
        getParserForType in interface com.google.protobuf.MessageLite
        Overrides:
        getParserForType in class com.google.protobuf.GeneratedMessageV3

      • getDefaultInstanceForType

        public Tasks.DataLoadingConfig getDefaultInstanceForType()
        Specified by:
        getDefaultInstanceForType in interface com.google.protobuf.MessageLiteOrBuilder
        Specified by:
        getDefaultInstanceForType in interface com.google.protobuf.MessageOrBuilder