Skip to content

fix(manifest): encode decimal partitions with declared precision#1129

Open
officialasishkumar wants to merge 1 commit into
apache:mainfrom
officialasishkumar:fix/1027-decimal-manifest-precision
Open

fix(manifest): encode decimal partitions with declared precision#1129
officialasishkumar wants to merge 1 commit into
apache:mainfrom
officialasishkumar:fix/1027-decimal-manifest-precision

Conversation

@officialasishkumar
Copy link
Copy Markdown

Summary

  • track decimal partition precision separately from decimal scale while building manifest field metadata
  • use the declared Avro decimal precision to determine fixed-size encoding width
  • add regression coverage for a decimal(10, 2) partition value whose textual length is smaller than the required fixed size

Fixes #1027

Testing

  • go test . -run TestManifest/TestWriteManifestDecimalPartitionUsesDeclaredPrecision -count=1
  • go test . ./codec -count=1

Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Comment thread manifest.go
Comment on lines -422 to +424
return result, logicalTypes, fixedSizes
return result, logicalTypes, decimalScales, decimalPrecisions
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we combine the decimalScales and decimalPrecisions into a single that maps the fieldID to a combined scale and precision? Compacting that would be better than having two separate maps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug(manifest): convertDecimalValue uses string length instead of declared precision for Avro fixed-size encoding

2 participants