Real + Virtual world Real + Virtual world • Two software required for this composition technique. – AVIsynth – VirtualDub / VirtualDubMod • AVIsynth is just a set of video frame server without any GUI. • AVIsynth combined with VirtualDub/Mod can be a very powerful video editing tools • With just an simple text file with AVIsynth script and extension .avs, we should able process the script using VirtualDub/Mod program.

Real + Virtual world Real + Virtual world • Compose both real and virtual world is not • Let look at some AVIsynth code that will help something novel. us to combine the stereo movie generated from Amira into our stereo movie from 3D camera. – Virtual rendered world → Real 3D Capture world • First, let us look at the video frame of both videos? Is there a problem ? Minoru’s 3D camera Amira rendered video .asf .avi 1280 x 480 128080 x48x 4800 • The weather channel has been used this technology since 1970s, and so, today’s most of the blockbuster’s movie.

Real + Virtual world Real + Virtual world • Crop It • To read in video file, we can use: – We will need to crop the video from each left + – AVI format: right image • Simply any avi filename • Resize It – avisource(“filename.avi”) – We will need to resize each left + right images to • Open with specific codec (e.g XVID) 640x480 each to 1280x480 – avisource(“filename.avi”, fourcc=“XVID”) • Still one problem? – Any other than avi format (such as asf, wmv …) – The exact position of the viewing L/R images of the • directshowsource(“filename.asf”).converttorgb24() video generated from 3D camera is not constant • directshowsource(“filename.wmv”) from one video to another. – • Remember to put the path along with the Every time we align the camera, the position of the viewing images changed as well to conform to the filename if the video is not in the same folder as alignment. the avs script file. – We will have to find out the position manually.

Real + Virtual world Real + Virtual world • Now, we know how to load both avi and asf • Load the asf video using AVIsynth video, how do we get them compatible in – directshowsource(“filename.asf”) terms of the size of actual displayed video? • Use the time frame slider at the bottom, drag to the 1280px frame with visible corner

480px

• UseU “Copy the source frame to clipboard” underu Video • PasteP it into MS-Paint or any graphics editinge software that allow you to point at

480px specifics pixel co-ordinate.

1280px Real + Virtual world Real + Virtual world • Manually locate the first colour (visible) pixel on the • Always remember to set the Compression following position: before saving, make sure the video stream is – Top-Left corner of Left + Right image set to Full Processing Mode under Video. – Bottom-Right corner of Left + Right image (TLx, TLy) • If use VirtualDubMod, you can set (TRx, TRy) CompressionCompression uundernder Save-As dialogbox.

(BLx, BLy) (BRx, BRy)

• Crop with AVIsynth: video3D = directshowsource(“filename.asf”) .ConverttoRGB24() left_clip = crop(video3D, TLx, TLy, BLx – TLx +1, BLy – TLy +1 ) right_clip = crop(video3D, TRx, TRy, BRx – TRx +1, BRy – TRy +1)

Real + Virtual world Real + Virtual world • Resize with AVIsynth • ColorKeyMask left_clip = left_clip.BilinearResize(640,480) – Allows us to layer one video on to another video via right_clip = right_clip.BilinearResize(640,480) the specific colour map. • We need to stack the left + right back to into • If we want to layer vidA on top of vidB, then we side-by-side, we use StackHorizontal will have the background of vidA with colour to StackHorizontal ( left_clip, right_clip ) be transparent. • We can open the script + save the end result • For our case, we want to have Amira video on into AVI with VirtualDub/Mod, remember to the Real video, so we will need to have the set the Compression encoder. Amira video to have ONE colour background. • To summarise: Typically, this colour will be Green, or Blue if it m3Dvideo = directshowsource(“filename.asf”).ConverttoRGB24() involves people, basically any colour that is left_clip = crop(m3Dvideo, TLx , TLy, width, height).BilinearResize(640,480) NOT going to exist on the foreground objects in right_clip = crop(m3Dvideo, TRx , TRy, width, height).BilinearResize(640,480) the video. StackHorizontal ( left_clip, right_clip ) Real + Virtual world Real + Virtual world • What colour screen would you use if you • Make sure it is set Uniform background. are the director of this film ?

Drag the colour over

Real + Virtual world Real + Virtual world • So what colour backgroundckground should wewe useu for • Create the “Blue screen” video via our Amira video? MovieMaker object.

• Set background in Amira under View menu…

• Get the Real world video ready and applied the following avisynth script: topClip=directshowsource(“AmiraVid.avi").converttorgb32() transparent= resetmask(topClip).colorkeymask($3a3bf7, 50) backClip=avisource(“camera3D_video.avi”).converttorgb32() layer(backClip, transparent) Real + Virtual world Real + Virtual world • Different colormask colour + threshold value • Other useful feature to use in VirtualDub/Mod results differently : – Use the feature forward or backward frame by – colorkeymask($3a3bf7, 0) frame. – Begin/End Markers for delete or crop part of the video, or even save a segment of it as different avi.

– colorkeymask($0000ff, 50), need at least 100 to work

Real + Virtual world Real + Virtual world • Overlay • Always work in segment of videos rather one – We can watermarked or put a transparent whole big video. background logo (png) on to our video via AVIsynth • It is quicker to test out the segment than full – Remember we need to do twice, once for Left and video. one for the Right images originalClip = avisource(“Combine_Real_Virtual.avi”) • We can either use Append AVI segment under the_image = ImageReader(“KIS.png”, pixel_type=“RGB32”) VirtualDub/Mod to append to existing AVI files, alpha_image = the_image.ShowAlpha(pixel_type=“RGB32”) or to use AVIsynth script to append more first_overlay_clip = Overlay(originalClip, the_image,mask=alpha_image, 5, 5) Overlay ( first_overlay_clip, the_image,mask=alpha_image, 645, 5) videos. vidA = avisource(“vidA.avi”) vidB = directshowsource(“vidB.avi”) vidC = avisource(“vidC.avi”) vidA + vidB + vidC

Real + Virtual world • What’s wrong with this two chunks of AVIsynth scripts? Code 1: videoA = directshowsource(“videoAavi”).ConverttoRGB24() leftVid = crop(videoA , 46, 38, 550, 404).BilinearResize(640,480) rightVid = crop(videoA , 686, 38, 550, 404).BilinearResize(640,480) SideBySideVid = StackHorizontal(leftVid, rightVid)

Code 2: 3Dvideo = directshowsource(“videoAavi”).ConverttoRGB32() transparent= resetmask(3Dvideo).colorkeymask($3a3bf7, 50) backClip=avisource(“camera3D_video.avi”).converttorgb32() layer(backClip, transparent)

Real + Virtual world • What software to display the 3D stereo? – GUI option: • Run nVidia 3D Vision Stereoscopic Player • Make sure to set Side-by-Side Left Image first – Command Line option: play3D.exe video.avi –sbs –quad –full –quit –full: use Fullscreen –quite: Quit after play finished (optional), else press Q can also quit – Amira:: PrPressess “Use“UUse Stereo”Stereo”o checkboxcheckbox to selectselect “Raw“RRawa Stereo”S