Object

org.apache.spark.storage.StorageLevel

All Implemented Interfaces:
Externalizable, Serializable

:: DeveloperApi :: Flags for controlling the storage of an RDD. Each StorageLevel records whether to use memory, or ExternalBlockStore, whether to drop the RDD to disk if it falls out of memory or ExternalBlockStore, whether to keep the data in memory in a serialized format, and whether to replicate the RDD partitions on multiple nodes.

The StorageLevel singleton object contains some static constants for commonly useful storage levels. To create your own storage level object, use the factory method of the singleton object (StorageLevel(...)).

See Also:

  • Constructor Summary

    Constructors

  • Method Summary

    apply(boolean useDisk, boolean useMemory, boolean useOffHeap, boolean deserialized, int replication)

    :: DeveloperApi :: Create a new StorageLevel object.

    apply(boolean useDisk, boolean useMemory, boolean deserialized, int replication)

    :: DeveloperApi :: Create a new StorageLevel object without setting useOffHeap.

    apply(int flags, int replication)

    :: DeveloperApi :: Create a new StorageLevel object from its integer representation.

    :: DeveloperApi :: Read StorageLevel object from ObjectInput stream.

    clone()

    boolean

    DISK_ONLY()

    boolean

    :: DeveloperApi :: Return the StorageLevel object with the specified name.

    int

    hashCode()

    boolean

    isValid()

    NONE()

    Various StorageLevel defined and utility functions for creating new storage levels.

    OFF_HEAP()

    void

    int

    int

    toInt()

    toString()

    boolean

    useDisk()

    boolean

    useMemory()

    boolean

    void

  • Constructor Details

    • StorageLevel

      public StorageLevel()

  • Method Details

    • NONE

      Various StorageLevel defined and utility functions for creating new storage levels.

      Returns:
      (undocumented)

    • DISK_ONLY

    • DISK_ONLY_2

    • DISK_ONLY_3

    • MEMORY_ONLY

    • MEMORY_ONLY_2

    • MEMORY_ONLY_SER

    • MEMORY_ONLY_SER_2

    • MEMORY_AND_DISK

    • MEMORY_AND_DISK_2

    • MEMORY_AND_DISK_SER

    • MEMORY_AND_DISK_SER_2

      public static StorageLevel MEMORY_AND_DISK_SER_2()

    • OFF_HEAP

    • fromString

      :: DeveloperApi :: Return the StorageLevel object with the specified name.

      Parameters:
      s - (undocumented)
      Returns:
      (undocumented)

    • apply

      public static StorageLevel apply(boolean useDisk, boolean useMemory, boolean useOffHeap, boolean deserialized, int replication)

      :: DeveloperApi :: Create a new StorageLevel object.

      Parameters:
      useDisk - (undocumented)
      useMemory - (undocumented)
      useOffHeap - (undocumented)
      deserialized - (undocumented)
      replication - (undocumented)
      Returns:
      (undocumented)

    • apply

      public static StorageLevel apply(boolean useDisk, boolean useMemory, boolean deserialized, int replication)

      :: DeveloperApi :: Create a new StorageLevel object without setting useOffHeap.

      Parameters:
      useDisk - (undocumented)
      useMemory - (undocumented)
      deserialized - (undocumented)
      replication - (undocumented)
      Returns:
      (undocumented)

    • apply

      public static StorageLevel apply(int flags, int replication)

      :: DeveloperApi :: Create a new StorageLevel object from its integer representation.

      Parameters:
      flags - (undocumented)
      replication - (undocumented)
      Returns:
      (undocumented)

    • apply

      :: DeveloperApi :: Read StorageLevel object from ObjectInput stream.

      Parameters:
      in - (undocumented)
      Returns:
      (undocumented)

    • useDisk

      public boolean useDisk()

    • useMemory

      public boolean useMemory()

    • useOffHeap

      public boolean useOffHeap()

    • deserialized

      public boolean deserialized()

    • replication

      public int replication()

    • clone

    • equals

      public boolean equals(Object other)

      Overrides:
      equals in class Object

    • isValid

      public boolean isValid()

    • toInt

      public int toInt()

    • writeExternal

      Specified by:
      writeExternal in interface Externalizable

    • readExternal

      Specified by:
      readExternal in interface Externalizable

    • toString

      Overrides:
      toString in class Object

    • hashCode

      public int hashCode()

      Overrides:
      hashCode in class Object

    • description

      public String description()