[AURON #1924] [BUILD] Add -Xlint:_ to scala-maven-plugin and resolve related warnings#2216
Open
1fanwang wants to merge 3 commits intoapache:masterfrom
Open
[AURON #1924] [BUILD] Add -Xlint:_ to scala-maven-plugin and resolve related warnings#22161fanwang wants to merge 3 commits intoapache:masterfrom
-Xlint:_ to scala-maven-plugin and resolve related warnings#22161fanwang wants to merge 3 commits intoapache:masterfrom
Conversation
…esolve related warnings
Enable `-Xlint:_` in both the base scala-maven-plugin config and the
scala-2.13 profile, then resolve the warnings it surfaces:
- lint:adapted-args (3 sites in metric-map construction): replace
`:+ ("k", v)` with `:+ ("k" -> v)` so the Tuple2 creation is explicit
instead of silent argument-list adaptation.
- lint:nonlocal-return (2 sites): drop `return` from inside closures by
using `Iterable.exists` for `matchVersion`, and an early-exit `while`
over an iterator in `SparkOnHeapSpillManager.spill`.
- lint:type-parameter-shadow / pattern-shadow (3 sites): rename pattern
bindings that shadowed enclosing-scope vals (`case in: ... =>`,
`case expr => expr`, `case nativeXScanExec`).
- lint:inaccessible: silenced project-wide via
`-Wconf:cat=lint-inaccessible:s` in both profiles, with a comment
explaining why -- Auron implements Spark plugin SPIs whose contracts
use `private[spark]` types (ElementTrackingStore, SparkUI,
IndexShuffleBlockResolver, ShuffleWriteMetricsReporter); exposing
them is required by Spark, not a defect.
Verified with `./auron-build.sh --pre --sparkver <v> --scalaver <s>
-DskipBuildNative` for spark-3.0/3.1/3.5 + scala-2.12 and spark-3.5 +
scala-2.13.
Closes apache#1924
-Xlint:_ to scala-maven-plugin and resolve related warnings-Xlint:_ to scala-maven-plugin and resolve related warnings
Contributor
There was a problem hiding this comment.
Pull request overview
This PR tightens Scala compilation by enabling full Scala linting (-Xlint:_) in the Maven Scala compiler configuration, and updates a handful of Scala sources to eliminate the newly surfaced lint warnings while preserving existing runtime behavior.
Changes:
- Enable
-Xlint:_for Scala compilation (base + Scala 2.13 profile) and silencelint-inaccessiblewarnings project-wide via-Wconf. - Replace non-local returns in Scala collection loops with
exists/whileearly-exit patterns to avoidNonLocalReturnControl. - Fix/avoid lint issues such as adapted-args tuple auto-tupled append and pattern binding shadowing.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| spark-version-annotation-macros/src/main/scala/org/apache/auron/sparkver.scala | Replaces loop+return with exists to avoid non-local return lint. |
| spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/shuffle/AuronBlockStoreShuffleReaderBase.scala | Renames pattern binders to avoid shadowing warnings in nested matches. |
| spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/plan/NativeParquetInsertIntoHiveTableBase.scala | Makes appended metric tuples explicit using -> to avoid adapted-args lint. |
| spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/plan/NativeBroadcastExchangeBase.scala | Makes appended metric tuples explicit using -> to avoid adapted-args lint. |
| spark-extension/src/main/scala/org/apache/spark/sql/execution/auron/plan/ConvertToNativeBase.scala | Makes appended metric tuple explicit using -> to avoid adapted-args lint. |
| spark-extension/src/main/scala/org/apache/spark/sql/auron/memory/SparkOnHeapSpillManager.scala | Removes non-local return from foreach by switching to an iterator while loop. |
| spark-extension/src/main/scala/org/apache/spark/sql/auron/SparkUDAFWrapperContext.scala | Renames pattern binder to avoid shadowing (case other => other). |
| spark-extension-shims-spark/src/test/scala/org/apache/auron/AuronEmptyNativeRddSuite.scala | Renames pattern binders in tests to avoid shadowing warnings. |
| pom.xml | Adds -Xlint:_ and suppresses lint-inaccessible warnings (base + Scala 2.13 profile). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
@1fanwang Thanks for the contribution! LGTM |
yew1eb
approved these changes
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #1924. Sets up the next sibling under the [BUILD] epic #1869 (#1925) without changing its scope.
Rationale for this change
-Xlint:_enables Scala's bundled lint checks, several of which catch real bug classes that the existing-Ywarn-unused+-Xfatal-warningsdo not. This turns the flag on in both the basescala-maven-pluginconfig and thescala-2.13profile, then resolves every warning it surfaces so the matrix build stays green.What changes are included in this PR?
pom.xml -- add
-Xlint:_and-Wconf:cat=lint-inaccessible:sto bothscala-maven-pluginarg lists. Thelint-inaccessiblesilencer is commented inline: Auron implements Spark plugin SPIs (AppHistoryServerPlugin, theShimstrait,AuronRssShuffleWriterBase) whose contracts takeprivate[spark]types (ElementTrackingStore,SparkUI,IndexShuffleBlockResolver,ShuffleWriteMetricsReporter). Exposing them in our overrides is required by Spark, not a defect, so silencing the category project-wide is cleaner than annotating every override site.Source fixes for the warnings that are real bug classes (8 sites across 8 files):
lint:adapted-args--:+ ("k", v)on aSeq[(String, V)]was being silently auto-tupled. Made theTuple2explicit via->. Files:ConvertToNativeBase,NativeBroadcastExchangeBase,NativeParquetInsertIntoHiveTableBase.lint:nonlocal-return--returnfrom inside afor { ... }/foreach { ... }body desugars to throw/catch viaNonLocalReturnControl. Replaced withIterable.existsinsparkver.matchVersion, and an early-exitwhile-loop over an iterator inSparkOnHeapSpillManager.spill.pattern shadowing (
lint:type-parameter-shadow/ pattern-shadow) -- pattern bindings reusing names from the enclosing scope. Renamed:case in: ... =>->case limited: ... =>/case fileIn: ... =>inAuronBlockStoreShuffleReaderBase;case expr => expr->case other => otherinSparkUDAFWrapperContext;case nativeXScanExec->case scaninAuronEmptyNativeRddSuite.Are there any user-facing changes?
No -- build-only.
How was this patch tested?
Local
./auron-build.sh --pre --sparkver <v> --scalaver <s> -DskipBuildNative -Dspotless.skip=truewas green for:Spark 3.3 / Scala 2.12 left for the matrix CI to cover (middle of the range, unlikely to surface different lint categories). Happy to iterate on any warning CI catches that I missed locally.