Skip to content

[AutoSparkUT] log10 - Function has different output on GPU #13738

@GaryShen2008

Description

@GaryShen2008

Describe the bug

The log10 function test fails on GPU during expression evaluation.


Steps/Code to reproduce bug

Spark-shell Reproduction Code

Environment Setup:

export SPARK_HOME=/path/to/spark-3.3.0
export RAPIDS_JAR=rapids-4-spark_2.12-25.12.0-SNAPSHOT-cuda12.jar

Reproduction Script: test-failures/reproduce-math-expressions-log10.scala

// Test: log10 function with positive and negative values

// Positive values
val df1 = spark.range(1).selectExpr(
  "log10(0.1) as test1",
  "log10(1.0) as test2",
  "log10(10.0) as test3",
  "log10(100.0) as test4"
)
df1.show(false)

// Non-positive values (should return null)
val df2 = spark.range(1).selectExpr(
  "log10(0.0) as zero",
  "log10(-0.5) as negative",
  "log10(-1.0) as negative2"
)
df2.show(false)

Run Command (with all 32 configuration parameters):

cd test-failures
$SPARK_HOME/bin/spark-shell \
  --master local[2] \
  --conf spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding \
  --conf spark.rapids.sql.enabled=true \
  --conf spark.plugins=com.nvidia.spark.SQLPlugin \
  --conf spark.sql.queryExecutionListeners=org.apache.spark.sql.rapids.ExecutionPlanCaptureCallback \
  --conf spark.rapids.sql.explain=ALL \
  --conf spark.rapids.sql.test.isFoldableNonLitAllowed=true \
  --conf spark.rapids.sql.csv.read.decimal.enabled=true \
  --conf spark.rapids.sql.format.avro.enabled=true \
  --conf spark.rapids.sql.format.avro.read.enabled=true \
  --conf spark.rapids.sql.format.hive.text.write.enabled=true \
  --conf spark.rapids.sql.format.json.enabled=true \
  --conf spark.rapids.sql.format.json.read.enabled=true \
  --conf spark.rapids.sql.incompatibleDateFormats.enabled=true \
  --conf spark.rapids.sql.python.gpu.enabled=true \
  --conf spark.rapids.sql.rowBasedUDF.enabled=true \
  --conf spark.rapids.sql.window.collectList.enabled=true \
  --conf spark.rapids.sql.window.collectSet.enabled=true \
  --conf spark.rapids.sql.window.range.byte.enabled=true \
  --conf spark.rapids.sql.window.range.short.enabled=true \
  --conf spark.rapids.sql.expression.Ascii=true \
  --conf spark.rapids.sql.expression.Conv=true \
  --conf spark.rapids.sql.expression.GetJsonObject=true \
  --conf spark.rapids.sql.expression.JsonToStructs=true \
  --conf spark.rapids.sql.expression.StructsToJson=true \
  --conf spark.rapids.sql.exec.CollectLimitExec=true \
  --conf spark.rapids.sql.exec.FlatMapCoGroupsInPandasExec=true \
  --conf spark.rapids.sql.exec.WindowInPandasExec=true \
  --conf spark.rapids.sql.hasExtendedYearValues=false \
  --conf spark.unsafe.exceptionOnMemoryLeak=true \
  --conf spark.sql.session.timeZone=America/Los_Angeles \
  --jars $RAPIDS_JAR \
  < reproduce-math-expressions-log10.scala

Expected behavior

The log10 function should compute base-10 logarithm correctly for positive values and return null for non-positive values.

Test cases:

  • log10(x) for x in (1 to 20).map(_ * 0.1) should work correctly
  • log10(x) for x in (-5 to 0).map(_ * 0.1) should return null

Actual behavior

GPU output:
+-------------------+-----+-----+-----+
|test1 |test2|test3|test4|
+-------------------+-----+-----+-----+
|-0.9999999999999998|0.0 |1.0 |2.0 |
+-------------------+-----+-----+-----+

+----+--------+---------+
|zero|negative|negative2|
+----+--------+---------+
| | | |
+----+--------+---------+

CPU:
+-----+-----+-----+-----+
|test1|test2|test3|test4|
+-----+-----+-----+-----+
|-1.0 |0.0 |1.0 |2.0 |
+-----+-----+-----+-----+

+----+--------+---------+
|zero|negative|negative2|
+----+--------+---------+
|null|null |null |
+----+--------+---------+


Environment details

  • Spark Version: 3.3.0
  • RAPIDS Version: 25.12.0-SNAPSHOT
  • CUDA Version: 12.x
  • Test Suite: MathExpressionsSuite
  • Test Name: log10

Additional context

  • Original Spark Test Location: spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala (line 435)
  • Test Logic: Validates log10 function with positive and negative inputs

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions