Save multi resolution image (#311)

[Gtortellier]

• create an image with high resolution on bboxes part and low resolution elsewhere

• save the image with the avif format and lower overall quality to save image weight

[MateoLostanlen]

Hey! Thanks a lot for the PR, really interesting ideas here

First, sorry for the delay on the review, I’ll try to be more reactive going forward.

On the AVIF part, I’m a bit concerned about compatibility. Firefighters are not always using up to date browsers or environments, so we need to be extra careful here. From what I’ve seen, support is improving but still not fully reliable depending on the browser and especially the tooling we use. Do you know if you checked how well AVIF is handled on the frontend side, in our current stack? (display, zoom, annotations, etc.)

On the variable resolution idea (high res on bbox, lower elsewhere), I think it’s a very interesting optimization, but I’m a bit worried about using only this format for the dataset. It could introduce biases during training, since the model might learn artifacts from the resolution differences instead of actual smoke patterns. So I’d prefer not to replace the current images entirely with this approach.

Maybe a good compromise could be:

keep standard JPEG images for training and platform usage

optionally use AVIF for long term storage or archival to reduce size

Also, this is just a discussion, I still think it’s worth testing this in practice. I’d like to set this up on a test tower to capture:

• classic JPEG
• high resolution JPEG crops on detections
• your proposed image

Curious to hear your thoughts, especially on the AVIF compatibility side

[MateoLostanlen]

Hi @Gtortellier , thanks a lot for your PR. So, I've done quite a bit of testing and, although I really like the idea, unfortunately the raw approach is going to be a problem because the encoding time on Raspberry Pi is way too long and therefore not suitable for our use case. It's a shame because the camera quality with the crop was really interesting, but unfortunately we can't apply it to our use case.
That said, it's still a good idea to increase the quality at the detection level, especially for small smoke plumes in the distance. So what we've put in place (there were PRs opened for this purpose) is, in addition to the image, to send an uncompressed 4K crop around the detection. Then, on the platform, we'll display both the main image and the 4K crop in order to help better understand what the model has detected. You can see it in this PR: #359