2024-04-03 15:20:56 WARN  NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-04-03 15:20:56 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerApplicationStart'
2024-04-03 15:20:56 INFO  log - Logging initialized @1551ms to org.sparkproject.jetty.util.log.Slf4jLog
2024-04-03 15:20:56 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:56 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:58 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionStart'
2024-04-03 15:20:58 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:58 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:58 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobStart'
2024-04-03 15:20:58 INFO  JobMetricsHolder - Added stages for job 0: [0]
2024-04-03 15:20:58 INFO  JobMetricsHolder - State of jobStages: {0=[0]}
2024-04-03 15:20:58 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 0: org.apache.spark.executor.TaskMetrics@225b89d7
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {0=[0]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {0=org.apache.spark.executor.TaskMetrics@225b89d7}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed stages for job 0: [0]
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed metrics for stage 0: org.apache.spark.executor.TaskMetrics@225b89d7
2024-04-03 15:20:59 INFO  JobMetricsHolder - Mapped output metrics for job 0: {WRITE_BYTES=489, WRITE_RECORDS=10}
2024-04-03 15:20:59 INFO  JobMetricsHolder - Cleaned up stages for job 0: null
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added stages for job 1: [1]
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {1=[1]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 1: org.apache.spark.executor.TaskMetrics@65b28bea
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {1=[1]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {1=org.apache.spark.executor.TaskMetrics@65b28bea}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed stages for job 1: [1]
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed metrics for stage 1: org.apache.spark.executor.TaskMetrics@65b28bea
2024-04-03 15:20:59 INFO  JobMetricsHolder - Mapped output metrics for job 1: {WRITE_BYTES=489, WRITE_RECORDS=10}
2024-04-03 15:20:59 INFO  JobMetricsHolder - Cleaned up stages for job 1: null
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added stages for job 2: [2]
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {2=[2]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 2: org.apache.spark.executor.TaskMetrics@40467444
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {2=[2]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Cleaned up stages for job 2: [2]
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed stage 2 from jobStages
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added stages for job 3: [3]
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {3=[3]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 3: org.apache.spark.executor.TaskMetrics@45decde3
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {3=[3]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Cleaned up stages for job 3: [3]
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed stage 3 from jobStages
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobStart'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added stages for job 4: [4]
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {4=[4]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 4: org.apache.spark.executor.TaskMetrics@350cca53
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {4=[4]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3, 4=org.apache.spark.executor.TaskMetrics@350cca53}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerTaskEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Added metrics for stage 4: org.apache.spark.executor.TaskMetrics@5b256a4f
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {4=[4]}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3, 4=org.apache.spark.executor.TaskMetrics@5b256a4f}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerJobEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed stages for job 4: [4]
2024-04-03 15:20:59 INFO  JobMetricsHolder - Removed metrics for stage 4: org.apache.spark.executor.TaskMetrics@5b256a4f
2024-04-03 15:20:59 INFO  JobMetricsHolder - Mapped output metrics for job 4: {WRITE_BYTES=510, WRITE_RECORDS=10}
2024-04-03 15:20:59 INFO  JobMetricsHolder - Cleaned up stages for job 4: null
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerSQLExecutionEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  OpenLineageSparkListener - Handling event of type 'SparkListenerApplicationEnd'
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of jobStages: {}
2024-04-03 15:20:59 INFO  JobMetricsHolder - State of stageMetrics: {2=org.apache.spark.executor.TaskMetrics@40467444, 3=org.apache.spark.executor.TaskMetrics@45decde3}
2024-04-03 15:20:59 INFO  ShutdownHookManager - Shutdown hook called
2024-04-03 15:20:59 INFO  ShutdownHookManager - Deleting directory /tmp/spark-75a4101e-8727-45a2-9a20-4ce6c2133b19
2024-04-03 17:20:59 INFO  tc.quay.io/openlineage/spark:spark-3.3.4-scala-2.12 - Container quay.io/openlineage/spark:spark-3.3.4-scala-2.12 started in PT5.507S
2024-04-03 15:20:59 INFO  ShutdownHookManager - Deleting directory /tmp/spark-d423a5c6-fdfe-413f-b5b2-4ff62fe84824