1.1 Pan and Scan Specification The pan and scan metadata track shall contain the information to allow for a “pan and scan” version of the image. Instead of storing multiple versions of the image to accommodate different aspect ratios of the feature (for example, 2.39:1, 1.78:1 and a 1.33:1), this feature allows for the IMP to contain one version of image and along with pan and scan metadata, create multiple aspect ratio versions of the image track.

1.1.1.1 Basic Pan and Scan Requirements The pan and scan metadata track shall contain information derived from pan and scan composition equipment/software in a standardized format. There are two specific types of pan and scan information that will be discussed here: 1. Pixel-for-Pixel Pan and Scan Metadata 2. Spatial Pan and Scan Metadata

Pixel-for-Pixel Pan and Scan Metadata includes coordinate information for changing the composition of an image but does not include any resizing (zooming in or out) of the image. Instead, the total image area has a certain raster and the pan and scan information moves around this raster, leaving the size of each pixel the same or pixel-for-pixel. This data includes basic coordinate information that allows for simple movements in the picture without the need to factor in any filters for zooming or re-scaling of the image.

Spatial Pan and Scan Metadata includes both coordinate information and scaling information for changing the composition of an image. With spatial changes, the area depicted by the pan and scan information does not have to stay pixel-for-pixel to the original total image area. Instead, the pan and scan area could be smaller or larger than the final output size, causing a re-scaling of the image in the pan and scan area. Due to the re-scaling, filters need to be factored in in order to properly create the re-size without causing too many image artifacts such as aliasing or “ringing.”

The format shall include the basic, common capabilities of pan and scan in an image for Pixel-for- Pixel pan and scan including: . Displaying only certain areas of an image . Zoom in . Zoom out/Scale/Windowbox . Tilt (up and down movement) . Pan (left and right movement) . Horizontal squeeze/stretch . Vertical squeeze/stretch

In addition to the above, the format should include basic Spatial pan and scan information including: . Zoom in . Zoom out/Scale/Windowbox . Horizontal squeeze/stretch . Vertical squeeze/stretch

Rotate and Flip/Flop (horizontally/vertically) have been intentionally excluded from the dynamic pan and scan metadata.

1.1.1.2 Time Code and Movement Identification The pan and scan metadata track shall be used in conjunction with the time code metadata track in order to create the proper movements from one area of the image to another area of the image at specific time codes. Any movement from one area to another area of the image shall be depicted as Pan Scan Events that occur at specific frame counts. For simple cut changes, the pan-scan size and location shall be described at specific frame counts numbers. The movement could be either a constant, linear movement or a dynamically-changing, non-linear movement; however, movements shall not be defined – only the resulting changes in image size and location per frame count.

1.1.1.3 Image Container, Active Image and Pan/Scan Image Area In order to output the pan and scan image information properly, the source image area and the final destination area shall be defined. The Image Container shall hold the entire canvas that is to be panned and scanned over and the Pixel Aspect Ratio of the Image Container is taken into consideration for the proper pan and scan image output. Each set of pan and scan metadata shall contain a specific Active Image area that defines the final destination image area and size for the pan and scan output.

In addition to the Image Container and the final Active Image area, a specific Pan/Scan Image Area needs to be defined. This area specifies the location of the actual pan and scan image information in regards to the Image Container. The Pan/Scan Image Area can move around, but the final Active Image area shall determine the final destination size and pixel aspect ratio that will be output. Other parameters in the pan and scan metadata could cause the Pan/Scan Image Area to be altered such as changing the overall aspect ratio or the size of the image to be displayed, but the Active Image shall always determine the final output size.

If pan and scan metadata exists, the Active Image area for the pan and scan metadata shall supersede the global Active Image area set for the IMF for the particular output of the pan and scan version. This allows an IMF to contain more than one set of pan and scan metadata. Other types of metadata should be included such as fields that would define certain types of scaling/re- size filters and dithering that should be used during image re-sizing. At the time of this writing, more investigation is needed to determine if universal scaling and dithering filters should be included in the pan and scan metadata.

The output image settings and/or Output Profile List (OPL) shall determine the overall scaling of the images during the output stage of the IMF. The Active Image area created by the pan and scan metadata shall be scaled to fit the output resolution depending on the OPL parameters. For example, a 1.78 pan and scan could be output as a 4x3 letterboxed 1.78 if the OPL specified the settings to create the 4x3 letterbox from the 1.78 pan and scan metadata. If no OPL exists, then the output of the Pan and Scan Metadata Track shall be the Active Image size.

1.1.1.4 Pan and Scan Metadata Required Fields In order to define the areas of the image that should be shown and to create any pan and scan movements, specific metadata is required to accurately identify the pan and scan information. At the time of this writing, however, the actual metadata fields cannot be defined. Instead, the data elements shown in Error: Reference source not found, shall be the minimum amount of information supported by the IMF for pan and scan metadata. These data elements shall be converted into specific metadata fields once the specification is complete. Some of these data elements have sub-element fields to help define the overall data element.

Table 1 - Pan and Scan Metadata Data Elements

Data Element Data Element Definition Examples Final Pan/Scan The fixed aspect ratio for the final output 2.40, 2.39, 2.35, 2.20, Aspect Ratio 1.78, 1.33 Fill Color (Fill Image) In cases where the Active Image is smaller Format TBD; could than the output size, this field determines the be R-G-B values color that should be used to fill. dependent on bit depth of IMF Data Element Data Element Definition Examples Active Image Width Total number of horizontal pixels used for the 1920, 720 final Active Image area Active Image Height Total number of vertical pixels used for the 1080, 576, 486, 480 final Active Image area Active Image Pixel Shape of the pixel expressed in a ratio of width 1:1, could be any Aspect Ratio divided by height of the pixel, specifically for ratio the final Active Image area Pan/Scan Event ID Identifies the event within the list of pan and Numeric, starting with scan events. 0, number of digits TBD Frame count Sub-element of Pan/Scan Event ID, frame 123456 count of where event is located. Pan-Scan Image Sub-element of Pan/Scan Event ID, start of x and y can be any Area Upper-Left Pan-Scan Image Area within the Image number between 00- (x,y) Container area expressed in an (x,y) 100, including coordinate value that is placed in relation to the fractional numbers. Image Container Top Left Coordinate; can The data type is include non-integer numbers single-precision 32-bit IEEE 754 floating point. Pan-Scan Image Sub-element of Pan/Scan Event ID, end of x and y can be any Area Lower- Pan-Scan Image Area within the Image number between 00- Right (x,y) Container area expressed in an (x,y) 100, including coordinate value that is placed in relation to the fractional numbers. Image Container Bottom Right Coordinate; can The data type is include non-integer numbers single-precision 32-bit IEEE 754 floating point. Scale x % Sub-element of Pan/Scan Event ID, denotes 100% = No change the percentage in change in the horizontal <100 = Smaller direction for the Active Image for output >100 = Larger Scale y % Sub-element of Pan/Scan Event ID, denotes 100% = No change the percentage in change in the vertical <100 = Smaller direction for the Active Image for output >100 = Larger

Examples of how the metadata fields work together to create the pan and scan image areas and movements are shown below. Example 1: Simple cuts from one area to another

In the above example, two people are having a conversation. The Image Container/full aperture image is 4096x3112 (with a square PAR for simplicity), and the red boxes show the Active Image area that should be displayed to create a 1.33 version of the image. The image in Shot #1 would be displayed starting at a specific time code, while one person is talking and then cut to the image in Shot #2 when the other person starts talking. This example is an example of Pixel-for-Pixel pan and scan metadata. A possible scenario of the pan and scan metadata is shown below:

Metadata Field Value Comments Pan/Scan AR 1.33 Fill Color 64-64-64 Assumes 10-bit RGB content Active Image H 1440 Final output of PS meant for Active Image W 1080 1440x1080 Active Image PAR 1:1 Square pixel PS Event ID 0000 Pan/Scan Event for Shot #1 Frame Count 123456 Upper-Left x,y 0, 10.345 Top left of red box Lower-Right x,y 55.2, 62.4 Bottom right of red box Scale x 100 No change Scale y 100 No change PS Event ID 0001 Pan/Scan Event for Shot #2 Frame Count 234567 Upper-Left x,y 47.64, 10.276 Top left of red box Lower-Right x,y 100, 62.4 Bottom right of red box Scale x 100 No change Scale y 100 No change

Example 2: Pan from one area to another

In the above example, two people are having a conversation. The Image Container/full aperture image is 4096x3112 (with a square PAR), and the red boxes show the Active Image area that should be displayed to create a 1.33 version of the image. The image in Shot #1 would be displayed starting at a specific time code, while one person is talking and then pan over to the image in Shot #2 when the other person starts talking. The pan could either be linear or non-linear in speed. Example 2 is similar to example Example one1, but instead of only having two Pan/Scan events, there would be a separate Pan/Scan Event for each subsequent frame as the red box area changes along the pan. During each time code frame, the x,y coordinates of the red box would change according to where the image should be based on whether the pan is a constant, linear speed or a dynamic, non-linear pan. This example is an example of Pixel-for-Pixel pan and scan metadata.

Example 3: Change in Active Image size

In this example, Shot #1 starts out with a smaller Active Image area that slowly zooms out to show the entire area in Shot #2. The zoom out could be linear or non-linear in speed. Similar to example 2, there would be a separate Pan/Scan Event for each subsequent frame as the red box area changes along the zoom out. During each time code frame, the x,y coordinates of the red box would change according to where the image should be based on whether the zoom out is a constant, linear speed or a dynamic, non-linear zoom out.

Because the image sizes are different for each shot’s Active Area, during the output of the IMP, the images would be scaled to the final output resolution. This could result in Shot #1 being scaled up to the final resolution or Shot #2 being scaled down to the final resolution. An OPL could be created to determine the parameters of how this re-scaling could be done. This example is an example of Spatial pan and scan metadata.

Example 4: Squeezing or Scaling Shots

1

2

In this example, the shot contains a 1.78 main title that needs to be modified in order to display in a 1.33 version. However, after the main titles and credits, the rest of the feature will be full-frame 1.33. This example shows two ways to make widescreen credits fit into a 1.33 aspect ratio: a. Squeeze the credit horizontally to fit into the 4x3 area b. Letterbox the credit and keep the same aspect ratio

In either case, feature content could either cut back to a full-screen 4x3 image. In the letterbox case, the letterboxing could slowly “scroll” outwards to reveal a full-screen 4x3 image by zooming in on the credit. All of these examples are examples of Spatial pan and scan metadata. A possible scenario of the pan and scan metadata is shown below for the Squeeze method:

Metadata Field Value Comments Pan/Scan Track ID 1.33 Fill Color 64-64-64 Assumes 10-bit RGB content Active Image H 1440 Final output of PS meant for Active Image W 1080 1440x1080 Active Image PAR 1:1 Square pixel PS Event ID 0000 Pan/Scan Event for Squeeze Frame Count 123456 Upper-Left x,y 0, 0 Top left of red box Lower-Right x,y 100, 100 Bottom right of red box Scale x 100 Use full image Scale y 100 Use full image

A possible scenario of the pan and scan metadata is shown below for the Letterbox method:

Metadata Field Value Comments Pan/Scan Track ID 1.33 Fill Color 64-64-64 Fills blank areas with Black Active Image H 1440 Final output of PS meant for Active Image W 1080 1440x1080 Active Image PAR 1:1 Square pixel PS Event ID 0000 Pan/Scan Event for Initial Letterbox Frame Count 123456 Upper-Left x,y 0, 0 Use full image Lower-Right x,y 100, 100 Use full image Scale x 75 Reduce the horizontal to 75% of original Scale y 75 Reduce the vertical to 75% of original PS Event ID 0001 Pan/Scan Event for start of zoom in Frame Count 234567 Upper-Left x,y 0, 0 Top left of red box Lower-Right x,y 100, 100 Bottom right of red box Scale x 77 Increases the horizontal to 77% of original Scale y 77 Increases the vertical to 77% of original PS Event ID 0002 Pan/Scan Event for next part of zoom in Frame Count 234568 Upper-Left x,y 0, 0 Top left of red box Lower-Right x,y 100, 100 Bottom right of red box Scale x 79 Increases the horizontal to 79% of original Scale y 79 Increases the vertical to 79% of original PS Event ID 0003 Pan/Scan Event for next part of zoom in Frame Count 234569 Upper-Left x,y 0, 0 Top left of red box Lower-Right x,y 100, 100 Bottom right of red box Scale x 81 Increases the horizontal to 81% of original Scale y 81 Increases the vertical to 81% of original …

Note: Pan/Scan Events are continued until the zoom in is complete.