[core] Introduce truncatedString for representations of large collections

[Zouxxyy]

Purpose

• truncatedString is similar to Spark's truncatedString, and its default value also references Spark's default of 25.

  def truncatedString[T](
      seq: Seq[T],
      start: String,
      sep: String,
      end: String,
      maxFields: Int): String = {
    if (seq.length > maxFields) {
      if (truncationWarningPrinted.compareAndSet(false, true)) {
        logWarning(
          "Truncated the string representation of a plan since it was too large. This " +
            s"behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.")
      }
      val numFields = math.max(0, maxFields - 1)
      seq.take(numFields).mkString(
        start, sep, sep + "... " + (seq.length - numFields) + " more fields" + end)
    } else {
      seq.mkString(start, sep, end)
    }
  }

• Use it in the toString of the IN predicate, as the IN predicates generated by DPP typically contain a large number of partition values.

Tests

API and Format

Documentation