Class

org.apache.spark.whylogs

DatasetProfileAggregator

Related Doc: package whylogs

Permalink

case class DatasetProfileAggregator(datasetName: String, sessionTimeInMillis: Long, timeColumn: String = null, groupByColumns: Seq[String] = Seq(), sessionId: String = UUID.randomUUID().toString) extends sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile] with Serializable with Product

A dataset aggregator. It aggregates Row into DatasetProfile objects underneath the hood.

Linear Supertypes
Product, Equals, sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile], Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DatasetProfileAggregator
  2. Product
  3. Equals
  4. Aggregator
  5. Serializable
  6. Serializable
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DatasetProfileAggregator(datasetName: String, sessionTimeInMillis: Long, timeColumn: String = null, groupByColumns: Seq[String] = Seq(), sessionId: String = UUID.randomUUID().toString)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. def bufferEncoder: Encoder[DatasetProfile]

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  7. val datasetName: String

    Permalink

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  10. def finish(reduction: DatasetProfile): ScalaDatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  12. val groupByColumns: Seq[String]

    Permalink

  13. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  14. def merge(profile1: DatasetProfile, profile2: DatasetProfile): DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

  15. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  18. def outputEncoder: Encoder[ScalaDatasetProfile]

    Permalink

    To understand the detailed implementation of this class, see ExpressionEncoder.

    To understand the detailed implementation of this class, see ExpressionEncoder.

    We use some internal Spark API here.

    Definition Classes
    DatasetProfileAggregator → Aggregator

  19. def reduce(profile: DatasetProfile, row: Row): DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

  20. val sessionId: String

    Permalink

  21. val sessionTimeInMillis: Long

    Permalink

  22. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  23. val timeColumn: String

    Permalink

  24. def toColumn: TypedColumn[Row, ScalaDatasetProfile]

    Permalink
    Definition Classes
    Aggregator

  25. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  26. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  27. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  28. def zero: DatasetProfile

    Permalink
    Definition Classes
    DatasetProfileAggregator → Aggregator

Inherited from Product

Inherited from Equals

Inherited from sql.expressions.Aggregator[Row, DatasetProfile, ScalaDatasetProfile]

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped