Constructor Summary

Method Summary

• Methods inherited from class java.lang.Object

  equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

• Frame

  public Frame​(FrameObject frameObject,
               SparkExecutionContext sparkExecutionContext)

• Frame

  public Frame​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
               FrameMetadata frameMetadata)

  Convert a Spark DataFrame to a SystemDS binary-block representation.

  Parameters:

  dataFrame - the Spark DataFrame

  frameMetadata - frame metadata, such as number of rows and columns

• Frame

  public Frame​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame,
               long numRows,
               long numCols)

  Convert a Spark DataFrame to a SystemDS binary-block representation, specifying the number of rows and columns.

  Parameters:

  dataFrame - the Spark DataFrame

  numRows - the number of rows

  numCols - the number of columns

• Frame

  public Frame​(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame)

  Convert a Spark DataFrame to a SystemDS binary-block representation.

  Parameters:

  dataFrame - the Spark DataFrame

• Frame

  public Frame​(org.apache.spark.api.java.JavaPairRDD<Long,​FrameBlock> binaryBlocks,
               FrameMetadata frameMetadata)

  Create a Frame, specifying the SystemDS binary-block frame and its metadata.

  Parameters:

  binaryBlocks - the JavaPairRDD<Long, FrameBlock> frame

  frameMetadata - frame metadata, such as number of rows and columnss

Method Detail

• toFrameObject

  public FrameObject toFrameObject()

  Obtain the frame as a SystemDS FrameObject.

  Returns:

  the frame as a SystemDS FrameObject

• to2DStringArray

  public String[][] to2DStringArray()

  Obtain the frame as a two-dimensional String array

  Returns:

  the frame as a two-dimensional String array

• toJavaRDDStringIJV

  public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringIJV()

  Obtain the frame as a JavaRDD<String> in IJV format

  Returns:

  the frame as a JavaRDD<String> in IJV format

• toJavaRDDStringCSV

  public org.apache.spark.api.java.JavaRDD<String> toJavaRDDStringCSV()

  Obtain the frame as a JavaRDD<String> in CSV format

  Returns:

  the frame as a JavaRDD<String> in CSV format

• toRDDStringCSV

  public org.apache.spark.rdd.RDD<String> toRDDStringCSV()

  Obtain the frame as a RDD<String> in CSV format

  Returns:

  the frame as a RDD<String> in CSV format

• toRDDStringIJV

  public org.apache.spark.rdd.RDD<String> toRDDStringIJV()

  Obtain the frame as a RDD<String> in IJV format

  Returns:

  the frame as a RDD<String> in IJV format

• toDF

  public org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> toDF()

  Obtain the frame as a DataFrame

  Returns:

  the frame as a DataFrame

• toBinaryBlocks

  public org.apache.spark.api.java.JavaPairRDD<Long,​FrameBlock> toBinaryBlocks()

  Obtain the frame as a JavaPairRDD<Long, FrameBlock>

  Returns:

  the frame as a JavaPairRDD<Long, FrameBlock>

• getFrameMetadata

  public FrameMetadata getFrameMetadata()

  Obtain the frame metadata

  Returns:

  the frame metadata

• toString

  public String toString()

  Overrides:

  toString in class Object

• hasBinaryBlocks

  public boolean hasBinaryBlocks()

  Whether or not this frame contains data as binary blocks

  Returns:

  true if data as binary blocks are present, false otherwise.

• hasFrameObject

  public boolean hasFrameObject()

  Whether or not this frame contains data as a FrameObject

  Returns:

  true if data as binary blocks are present, false otherwise.