Add the OpenEXR recommendation

[doug-walker]

This is the proposed recommendation for reading and writing OpenEXR files, based on many conversations during the Color Interop Forum meetings.

Please note that this will not be merged until after the Interop ID recommendation, but I would like to get it approved and ready to go while it is fresh in people's minds from recent meeting discussions.

Implements issue #4.

Based on this Google doc:
https://docs.google.com/document/d/1MTH1bq2L67ifvdDf64Amhzg4AbkIM5LG6yPHrB96Vwo/edit?usp=sharing

And Sean's initiative to create Mermaid diagrams:
#9 (comment)

[peterhillman]

That seems all very sensible and clear.

I agree with the restriction that all genuine color channels within the same file be in the same space. It could more explicitly state that if a file contains both data channels and color channels, then the data channels should be in different parts to the color channels, and those parts have an colorInteropID attribute set to "data". That way, if you have no choice but to encode non-color data in R,G,B channels, there's a way to indicate that.

Similarly, if a file contains a mixture of color channels in known and unknown spaces, the unknown channels should be in parts with an "unknown" colorInteropID attribute.

You could interpret this as recommending that data channels and unknown color channels be explicitly tagged as being in the same colorspace as the color channels. It should be permitted to have different values for colorInteropID across a multipart file, but all parts which are not set to "data" or "unknown" should have the same value. Code reading images should anticipate that, too.

[zachlewis]

Similarly, if a file contains a mixture of color channels in known and unknown spaces, the unknown channels should be in parts with an "unknown" colorInteropID attribute.

tl;dr --this makes me uneasy. It's not clear to me what "unknown" intends to communicate here, or what to make of the reliability and validity of metadata indicative of an EXR that is explicitly partially color managed.

What does it mean for a file to contain a mixture of color channels in known and unknown spaces? How is the writer able to distinguish known color channel layers from unknown color channel layers, and unknown color channel layers from data channel layers? Is the writer working in a color-unmanaged environment or not? And if not, should the writer be actively tagging channel layers without colorInteropID as explicitly "unknown" in the first place? What level of "certainty" does "unknown" suggest? Does the writer actively want downstream applications to treat channel layers differently because it knows something about the difference between known and unknown color layers that it isn't able to better articulate? Or is the writer working in a color-unmanaged environment, drawing from multiple sources authored under different conditions, and just persisting whatever colorInteropID metadata happens to already exist? If some channels are known, and others are unknown, should downstream processes attempt to resolve all unknown layers to the same color space? If a writer indicates that the main R,G,B channels are known, but others are explicitly "unknown", what should downstream processes do if they must reconcile known and unknown color layers to a single working space? Does it still make sense for downstream readers to parse the filename for hints toward resolving "unknown" channels? Does it make any more or less sense to fail over to a general default color space for EXRs, compared to a known color space as indicated by one of the other channel layers?

[peterhillman]

tl;dr --this makes me uneasy.

Totally - those are valid concerns. My comment was mainly about "data" rather than "unknown". E.g. Cryptomatte and other schemes repurpose R,G,B channels for data. If a file contains both a real color layer and data-in-RGB channels, the data layers should be in a separate part with a colorInteropID set to "data". Ideally, data channels wouldn't be labelled as R, G, or B (and perhaps also Y) so could be in any part with any colorInteropID.

Maybe discussing some parts being in an known space and others unknown just confuses matters. The "multiple sources in different spaces" could well happen if a file contains a reference image for comparison with the main RGB image (and the color of the reference may not be important, just the content). It's a tenuous example, though, and allowing for it in the recommendation could lead to misunderstandings.

One rule should be "don't lie" - don't tag channels as being in a space that is known to be incorrect just to satisfy the recommendations. That just decreases trust in the metadata. A playback tool may well ignore the per-part attribute when displaying channels from the "unknown" part, but at least it would be there in the file to help debug why something looks wrong.

[zachlewis]

What happens if an OpenColorIO config contains a color space explicitly named "unknown"?

If the colorInteropID is explicitly set to "unknown", and other mechanisms have failed to resolve a more appropriate color space in the OCIO config, do readers...

A. Fail over to assigning the "unknown" color space?
B. Fail over to assigning the "default" color space?

The phrasing of rule 3 seems to imply that color spaces named "unknown" in the config should be ignored (B); but I think we should clarify this point before finalizing this document.

[zachlewis]

Maybe discussing some parts being in an known space and others unknown just confuses matters. The "multiple sources in different spaces" could well happen if a file contains a reference image for comparison with the main RGB image (and the color of the reference may not be important, just the content). It's a tenuous example, though, and allowing for it in the recommendation could lead to misunderstandings.

Oh, to be clear, I don't think this is a tenuous example at all -- it's a real-world edge case that I think is worth discussing, and I'm glad you brought it up. I know I've certainly interchanged scene-referred plates with gamma-encoded color-graded reference living in the same EXR. It's not exactly the same thing, but I know Baselight does something similar with BLG files, which carry both the source encoding as the main channel layer, and a display-referred reference in an alternate channel layer...! In fact, I think they even encode a preview that splits the frame diagonally and shows both at the same time!

But I do think, in general, that per-part colorInteropID metadata is a tricky prospect. I think it's certainly useful to have if can be done correctly and automatically; but I also think getting DCCs and users on the same page re: manually managing per-part colorInteropID metadata is going to prove difficult for the immediate future; and I'm concerned that the complexity involved will throw the reliability and validity of such metadata into question before it has a chance to really get off the ground...! That said, I'm less concerned about the validity and reliability of per-part colorInteropID metadata for distinguishing between color and non-color data channels, because it's a lot easier and more immediately apparent to debug when things go wrong. (And per Murphy's Law, things will, of course, go wrong).

[scoopxyz]

Comment carryover from today's CIF meeting.

There was some discussion about the Unknown CIF ID and whether a config could/would define a colorspace called "Unknown".

I just wanted to point out that as described/visualized the data and unknown IDs are checked first and treated as escape hatches for external systems (non-OCIO) to make decisions before OCIO is queried. Though in both cases the OCIO color management system driven DCC would have subtle caveats.

In the case of data we say "don't color manage". However, in the OCIO CMS case there will be instances where the more sensible approach will be to assign the data a OCIO Color Space where isdata: true as a method of ensuring no color management happens to those pixels. This is a possible point of confusion for implementation.

As with data, unknown triggers the escape hatch. In this sense it already is a "reserved" word as Larry brought up because it potentially triggers behavior before OCIO is ever queried. Only after this would the File Rules or other mechanism search for an OCIO color space called unknown. This reduces the potential control of default behavior for DCC that would attempt to do something "fancy" like looking to chromaticity coordinates or similar. It's debatable if that is a good or bad thing.

Is this the correct interpretation of expected behavior and, if so, are we ok with the present level of ambiguity/specificity in the data and unknown ID "escape hatch" and external handling?

[doug-walker]

@scoopxyz, I like your framing of the data/unknown issue that Zach raised. I have made some revisions based on the discussion today.

@peterhillman, that was great feedback, I have adopted one of your suggestions and reworked the discussion around multi-part files.

Thank you to everyone who participated in the meeting today. Apologies that we ran out of time.

[zachlewis]

Thanks guys -- I appreciate the discussion.

I just wanted to clarify CIF's intent here, because it wasn't clear whether letting an OCIO config author define an "unknown" color space implied that facilities may use an OCIO config to (somewhat) reliably steer different certain "switching" behavior in DCCs, depending on whether written colorInteropID metadata is left unset (readers fail over to the config's "default" role), versus explicitly set to "unknown" (readers fail over to the "unknown" space).

If we didn't want an application to clobber a config author's / pipeline TD's / user's intent, this would mean tightening the guidelines around reading and writing as well -- we'd have to caution applications against automatically assuming an EXR that doesn't have the attribute set can be interpreted the exact same way as an EXR that has it explicitly set to "unknown"; and likewise, we'd have to provide more nuanced guidance for using OCIO color space "interop_id" attributes and user preferences to determine whether or not to leave the colorInteropID metadata unset write "unknown" metadata instead. In turn, we'd also have to advise exercising caution about using the string "unknown" to represent an internal state, if there's a chance doing so might accidentally leak into written metadata against the config author's wishes.

The specific use cases I have in mind pertain to VFX commercials workflows, where, even today, color management is sometimes considered optional right up until the moment it isn't. Perhaps surprisingly, facilities large enough to have pipeline departments may struggle more than smaller facilities with correct and reliable commercials color pipelines and workflows. Where color management is concerned, tooling built into getting plates into the pipeline may demand knowledge and decision-making ability of those who might not be equipped with either. The people responsible for communicating with clients and vendors are often not the same people responsible for receiving the actual data from the client and shepherding it to production storage; often, the person responsible for knowing and making decisions about how the data need to be prepped before artists can start working isn't the same as the person responsible for conforming the media and/or ingesting plates into the pipeline. And through all of this, there's immense pressure to get plates turned over before a certain time of day, so everything an artist needs is ready for them by the time they're onboarded.

So, it would be very useful to have a way to use metadata to differentiate un-color-managed media that hasn't been ingested or touched by the pipeline yet; versus un-color-managed media that has been touched by the pipeline, but whose encoding was not yet known at the time of ingestion (but must be prepped before certain disciplines get started); versus media that intentionally and explicitly should not be color managed. Without too much effort, it would allow a pipeline to orchestrate application / task-specific "breakpoints", gating parts of the pipeline that require upstream decision-making from the parts that do not.

As you can see, I burnt my brain out on this, so I have no idea how crazy all of this sounds. But, as I see it, almost everything is in place to facilitate such workflows; and we really wouldn't have to change much about the current wording of the guidelines, apart from encouraging OCIO hosts not to write explicitly "unknown" colorInteropID metadata unless the user wants to, or an OCIO color space interop_id says to. That's it, really; everything else needed kind of happens automatically.

On the other hand, if we were to ignore any OCIO color spaces named "unknown" when parsing colorInteropID metadata, that would preempt the possibility of such workflows, and vastly simplify the scope of what we're purporting to support; and would free applications to treat missing and explicitly "unknown" metadata the same way on both read / write, while still permitting OCIO configs to surface user interfaces that provide an "unknown" option.

Anyway, to be honest, I'm fine with whatever direction we take, as long as we provide guidance and manage expectations duly.

[KevinJW]

I'm in the camp that there is a clear distinction between:

• unspecified (lack of any attribute)
• specified (attribute present)
  • colorInteropID provided explicit colour space
  • 'unknown'

The unknown case is a positive assertion and should only be actioned by the user/configuration choosing to make that assertion, not by a hidden application/library default assumption.

Not clear if the specified case where an attribute is present needs to handle an 'indeterminate' option where no attempt has been made to decide as a distinct option vs the user/system decided/asserted it is unknowable

[doug-walker]

Zach and Kevin, thanks for elaborating on the issue of the "unknown" ID. I'm not clear on this:

The unknown case is a positive assertion and should only be actioned by the user/configuration choosing to make that assertion, not by a hidden application/library default assumption.

By "assertion" I believe you are referring to what a piece of software writes into the EXR file, not what the config author includes in the config?

The main reason we added the "unknown" CIF ID was because some applications seem to feel that they always need to write something (even if they don't know the color space) and we didn't want that to be a hard-coded value such as "lin_rec709_scene", which is one of the main factors that caused a loss of trust in chromaticities. I'm worried about the difficulty of trying to get developers on board with anything too complicated about when to write "unknown" vs. not writing anything.

In addition, one could imagine different types of apps wanting to do different things in this case, so I worry about being too prescriptive. As Carol said during the meeting, we may want to aim to keep it simpler to start with. That said, I'm not opposed to "adding a feature" around how "unknown" should work, but I'm not clear on what you are proposing exactly. I feel that having a concrete proposed change to the PR would be needed to move forward on your feedback on this issue.

[peterhillman]

Some thought about multipart files is required if "no colorInteropID specified" is to be handled differently to "colorInteropID set to unknown"

The convention in OpenEXR is the first part's metadata refers to the entire file, and other parts' metadata applies only to that part. So, if part 0 has a colorInteropID of lin_ap0_scene, and part 1 has no colorInteropID, then it would likely also be considered to be lin_ap0_scene. The recommendation might say all part should have a colorInteropID, but I don't know how consistently that would be followed

For a single part file, you could make a distinction between a "known unknown" of a colorInteropID set to "unknown" and an "unknown unknown" of a missing colorInteropID, but you can't do that for multipart files which only have some parts in a known space.

[lgritz]

I worry about being too prescriptive.

I would vote for being as non-prescriptive as possible. At present, all I'm really advocating for is a reserved name for the state of not knowing, and license to not worry about anybody else using that same name for something that is known or expecting identical behavior across all apps (which seems like an impossible goal to me -- an interactive can decide an unknown space needs to ask the user to make a choice, whereas a library or a scripting app cannot do that).

I'm not sure I understand the argument against each part of an openexr potentially having its own color space. I do agree that as a practical matter, it seems burdensome to deal with multiple color spaces within a part.

[zachlewis]

What if we disallowed writing un-namespaced unknown metadata?

[doug-walker]

@peterhillman, thanks for that feedback, I've reworked the recommendations for multi-part files.

@zachlewis, I'm not sure I follow, but the goal is really to keep this as simple as possible for implementers unless there is a strong reason to do otherwise.

I'm not sure I understand the argument against each part of an openexr potentially having its own color space. I do agree that as a practical matter, it seems burdensome to deal with multiple color spaces within a part.

@lgritz, there is no mechanism to support multiple color spaces within a part. Even to support that across multiple parts in a given file seems like it would impose a large burden on application or pipeline developers to implement the necessary UI and test all the possible scenarios.

[lgritz]

@lgritz, there is no mechanism to support multiple color spaces within a part.

Maybe I was unclear. I am agreeing with this. I think it would be very burdensome on all parties to allow multiple color spaces within a part.

Even to support that across multiple parts in a given file seems like it would impose a large burden on application or pipeline developers to implement the necessary UI and test all the possible scenarios.

I'm afraid I don't understand that notion very well. From my point of view, it is more natural to think of each part as having an independent set of metadata (including color space) than to force them all to be the same. My (and OIIO's) mental model for a multi-part file is simply a collection of separate images stored in one container, and its preferred semantics are that the parts can be split into separate images at will, and also that a set of separate images can be combined at will into a multi-image file. Constraints that make multiple parts semantically different from multiple files are additional burden, as far as I'm concerned.

[lgritz]

Don't get me wrong, I'm not demanding separate color spaces per part if it's problematic for others. It's not something I'm losing sleep over.

But I am here to say that I don't see it as problematic to allow it, and in most respects, things get easier for me and my software the more multiple parts resemble multiple files (except for the convenience of being stored in one file).