SPark 3.5 int96 config

Question

SPark 3.5 int96 config

Preeti Choraria 0 Microsoft Employee

We understand that spark is setting int64 as defauult values for timestamp. however we need t use int96 as default for config. We came across spark docs and I tried the below configs to be set in spark pool, and added these configs also -

"spark.sql.parquet.int96RebaseModeInWrite": "CORRECTED",

"spark.sql.parquet.int96RebaseModeInRead": "CORRECTED",

"spark.sql.parquet.outputTimestampType": "INT96"

However, still we are unable to get parquet files with int96 format.

Could you help us with the exact configs that can be used to generate parquet files with Timestamp format as int96.

Preeti Choraria 0 Reputation points Microsoft Employee

2026-04-06T07:36:59.1166667+00:00

Thank you for your response. Will try this.
Manoj Kumar Boyini 11,685 Reputation points Microsoft External Staff Moderator

2026-04-06T20:02:34.02+00:00

Hi @Preeti Choraria

Just checking in to see if the information provided earlier was helpful. Please let us know if you need any additional details or assistance, and we’ll gladly continue supporting you.

1 answer

Your answer

Preeti Choraria 0 Reputation points Microsoft Employee

2026-04-06T07:36:59.1166667+00:00

Thank you for your response. Will try this.
Manoj Kumar Boyini 11,685 Reputation points Microsoft External Staff Moderator

2026-04-06T20:02:34.02+00:00

Hi @Preeti Choraria

Just checking in to see if the information provided earlier was helpful. Please let us know if you need any additional details or assistance, and we’ll gladly continue supporting you.

Answer 1

Hi @Preeti Choraria

Spark can write Parquet files with INT96 timestamps in Synapse Spark 3.5, but the configs must be applied to the same Spark session that performs the write.

Recommended settings (before df.write.parquet(...)):

spark.sql.parquet.outputTimestampType = INT96

spark.sql.parquet.writeLegacyFormat = true

These ensure Spark uses INT96 as the physical timestamp type. The int96RebaseMode* settings only control calendar rebasing (CORRECTED vs LEGACY); they do not decide INT96 vs INT64.

Apply the configs via Synapse Studio → Manage → Apache Spark configurations, attach them to the Spark pool/notebook, and restart the session if needed. You can verify they’re applied with:

spark.conf.get("spark.sql.parquet.outputTimestampType")

Important notes:

To get INT96, write plain Parquet (df.write.parquet(...)). Delta tables have their own timestamp/precision constraints.
Verify the output using parquet-tools schema <file>.parquet—you should see int96 for timestamp columns.

If INT64 still appears, double‑check that the column being written is actually TimestampType (not already a LongType).

Hope this helps, Please let us know if you have any questions and concerns.

Smaran Thoomu 34,880 Reputation points Microsoft External Staff Moderator

2026-04-10T02:54:01.38+00:00

Hi @Preeti Choraria

Just to add to the above - even with the correct config, in Spark 3.5 (especially in Synapse), INT96 is treated as a legacy format and may not always be honored depending on the execution path.

If you’re still seeing INT64 after applying:

spark.sql.parquet.outputTimestampType = INT96

please confirm:

How the data is being written (notebook vs pipeline)

Whether this is plain Parquet or via another layer (like Delta)

This will help narrow down if it’s a config issue or runtime behavior.

Share via

SPark 3.5 int96 config

1 answer

Your answer