fix: "\%" early stop

[wjiayis]

Previously:

• % is used to remove comments, which interfered with literal \% in LaTeX documents.

Now:

• Handle % properly. Tested for %, \% and \\%.

[4ndrelim]

Left some comments and suggestions for your consideration, but overall lgtm.

[4ndrelim]

This should cover most cases. Think its fair not to expect multiple chained \ since it hardly serves practical purpose beyond 2 consecutive backslashes which represents newline. 3 consecutive followed by a % would denote newline followed by % which is hardly the intent.

But a more general solution that guards against malicious input is to count the number of backslashes preceding % and only treating the remaining as comment if the count is even. This logic is actually implemented in the xtramcp backend in LatexParser since it is not uncommon to encounter cases e.g.\\\{. https://github.com/PaperDebugger/xtramcp/blob/main/common/latex_parser.py#L780

That said, I personally feel it is reasonable enough and am also ok with this solution.

[4ndrelim]

Actually, on a separate note, do you know why are we removing comments before passing to the LLM? I don't know the specifics, but assume this is the case. In general, even if it is a user comment not meant to be displayed, it might serve as good contextual cues or auxiliary information. The output may be informed with user's thoughts in the form of comments.

Might it be because some comments might be too long / redundant? So this is an attempt to save tokens?

[wjiayis]

@4ndrelim Sure, I can generalise this solution to count odd and even backslashes.

@Junyi-99 Was there more context as to why comments were removed previously?

[wjiayis]

@4ndrelim Wait actually this already handles 3 or more backslashes before the % as well. I've improved the test cases to cover 3 or more backslashes regardless.

[4ndrelim]

Oh, i did not realise it already covers. nice.