All Known Implementing Classes:
SparkConf

public interface ReadOnlySparkConf

  • Method Summary

    <T> T

    catchIllegalValue(String key, scala.Function0<T> getValue)

    Wrapper method for get() methods which require some specific value format.

    boolean

    Does the configuration contain a given parameter?

    boolean

    contains(org.apache.spark.internal.config.ConfigEntry<?> entry)

    Does the configuration have the typed config entry?

    Get a parameter; throws a NoSuchElementException if it's not set

    Get a parameter, falling back to a default if not set

    <T> T

    get(org.apache.spark.internal.config.ConfigEntry<T> entry)

    Retrieves the value of a pre-defined configuration entry.

    getAll()

    Get all parameters as a list of pairs

    boolean

    getBoolean(String key, boolean defaultValue)

    Get a parameter as a boolean, falling back to a default if not set

    double

    getDouble(String key, double defaultValue)

    Get a parameter as a double, falling back to a default if not ste

    By using this instead of System.getenv(), environment variables can be mocked in unit tests.

    int

    getInt(String key, int defaultValue)

    Get a parameter as an integer, falling back to a default if not set

    long

    getLong(String key, long defaultValue)

    Get a parameter as a long, falling back to a default if not set

    Get a parameter as an Option

    long

    Get a size parameter as bytes; throws a NoSuchElementException if it's not set.

    long

    getSizeAsBytes(String key, long defaultValue)

    Get a size parameter as bytes, falling back to a default if not set.

    long

    Get a size parameter as bytes, falling back to a default if not set.

    long

    Get a size parameter as Gibibytes; throws a NoSuchElementException if it's not set.

    long

    Get a size parameter as Gibibytes, falling back to a default if not set.

    long

    Get a size parameter as Kibibytes; throws a NoSuchElementException if it's not set.

    long

    Get a size parameter as Kibibytes, falling back to a default if not set.

    long

    Get a size parameter as Mebibytes; throws a NoSuchElementException if it's not set.

    long

    Get a size parameter as Mebibytes, falling back to a default if not set.

    long

    Get a time parameter as milliseconds; throws a NoSuchElementException if it's not set.

    long

    Get a time parameter as milliseconds, falling back to a default if not set.

    long

    Get a time parameter as seconds; throws a NoSuchElementException if it's not set.

    long

    Get a time parameter as seconds, falling back to a default if not set.

  • Method Details

    • catchIllegalValue

      <T> T catchIllegalValue(String key, scala.Function0<T> getValue)

      Wrapper method for get() methods which require some specific value format. This catches any NumberFormatException or IllegalArgumentException and re-raises it with the incorrectly configured key in the exception message.

      Parameters:
      key - (undocumented)
      getValue - (undocumented)
      Returns:
      (undocumented)

    • contains

      boolean contains(String key)

      Does the configuration contain a given parameter?

    • contains

      boolean contains(org.apache.spark.internal.config.ConfigEntry<?> entry)

      Does the configuration have the typed config entry?

    • get

      Get a parameter; throws a NoSuchElementException if it's not set

    • get

      Get a parameter, falling back to a default if not set

    • get

      <T> T get(org.apache.spark.internal.config.ConfigEntry<T> entry)

      Retrieves the value of a pre-defined configuration entry.

      - This is an internal Spark API. - The return type if defined by the configuration entry. - This will throw an exception is the config is not optional and the value is not set.

      Parameters:
      entry - (undocumented)
      Returns:
      (undocumented)

    • getAll

      Get all parameters as a list of pairs

    • getBoolean

      boolean getBoolean(String key, boolean defaultValue)

      Get a parameter as a boolean, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      IllegalArgumentException - If the value cannot be interpreted as a boolean

    • getDouble

      double getDouble(String key, double defaultValue)

      Get a parameter as a double, falling back to a default if not ste

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as a double

    • getInt

      int getInt(String key, int defaultValue)

      Get a parameter as an integer, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as an integer

    • getLong

      long getLong(String key, long defaultValue)

      Get a parameter as a long, falling back to a default if not set

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as a long

    • getOption

      Get a parameter as an Option

    • getSizeAsBytes

      long getSizeAsBytes(String key)

      Get a size parameter as bytes; throws a NoSuchElementException if it's not set. If no suffix is provided then bytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsBytes

      long getSizeAsBytes(String key, String defaultValue)

      Get a size parameter as bytes, falling back to a default if not set. If no suffix is provided then bytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsBytes

      long getSizeAsBytes(String key, long defaultValue)

      Get a size parameter as bytes, falling back to a default if not set.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as bytes

    • getSizeAsGb

      long getSizeAsGb(String key)

      Get a size parameter as Gibibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Gibibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Gibibytes

    • getSizeAsGb

      long getSizeAsGb(String key, String defaultValue)

      Get a size parameter as Gibibytes, falling back to a default if not set. If no suffix is provided then Gibibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Gibibytes

    • getSizeAsKb

      long getSizeAsKb(String key)

      Get a size parameter as Kibibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Kibibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Kibibytes

    • getSizeAsKb

      long getSizeAsKb(String key, String defaultValue)

      Get a size parameter as Kibibytes, falling back to a default if not set. If no suffix is provided then Kibibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Kibibytes

    • getSizeAsMb

      long getSizeAsMb(String key)

      Get a size parameter as Mebibytes; throws a NoSuchElementException if it's not set. If no suffix is provided then Mebibytes are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the size parameter is not set
      NumberFormatException - If the value cannot be interpreted as Mebibytes

    • getSizeAsMb

      long getSizeAsMb(String key, String defaultValue)

      Get a size parameter as Mebibytes, falling back to a default if not set. If no suffix is provided then Mebibytes are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as Mebibytes

    • getTimeAsMs

      long getTimeAsMs(String key)

      Get a time parameter as milliseconds; throws a NoSuchElementException if it's not set. If no suffix is provided then milliseconds are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the time parameter is not set
      NumberFormatException - If the value cannot be interpreted as milliseconds

    • getTimeAsMs

      long getTimeAsMs(String key, String defaultValue)

      Get a time parameter as milliseconds, falling back to a default if not set. If no suffix is provided then milliseconds are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as milliseconds

    • getTimeAsSeconds

      long getTimeAsSeconds(String key)

      Get a time parameter as seconds; throws a NoSuchElementException if it's not set. If no suffix is provided then seconds are assumed.

      Parameters:
      key - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NoSuchElementException - If the time parameter is not set
      NumberFormatException - If the value cannot be interpreted as seconds

    • getTimeAsSeconds

      long getTimeAsSeconds(String key, String defaultValue)

      Get a time parameter as seconds, falling back to a default if not set. If no suffix is provided then seconds are assumed.

      Parameters:
      key - (undocumented)
      defaultValue - (undocumented)
      Returns:
      (undocumented)
      Throws:
      NumberFormatException - If the value cannot be interpreted as seconds

    • getenv

      By using this instead of System.getenv(), environment variables can be mocked in unit tests.

      Parameters:
      name - (undocumented)
      Returns:
      (undocumented)