Projection Mapped Graphics for Automotive Design

Virtual Automotive: Projection Mapped Graphics for Automotive Design

Roy C. Anthony Kevin Moule

2 Who we are • Roy C. Anthony • Leads research initiatives creating new concepts and technologies designed to benefit Christie's extensive, global customer base.

• Kevin Moule • Focused on developing machine vision based system for automatically warping and blending multi-projector displays.

3 Christie at a Glance

At Christie we help our customers create and share the world’s best visual experiences.

4 Christie at a Glance

5 Christie at a Glance

Entertainment Solutions Business Products

Visual Environment - Visualization Visual Environment - Simulation

6 Projection Mapping ? 7 8 Projection Mapping • What can Projection Mapping offer?

• A “wow-factor”. • Cost-effective transformation of any space • The projected image is mapped to fit the surface and space .

9 Projection Mapping • What can Projection Mapping offer?

• A “wow-factor”. • Cost-effective transformation of any space • The projected image is mapped to fit the surface and space .

10 Projection Mapping • What can Projection Mapping offer?

• A “wow-factor”. • Cost-effective transformation of any space • The projected image is mapped to fit the surface and space .

11 Projection Mapping • What can Projection Mapping offer?

• A “wow-factor”. • Cost-effective transformation of any space • The projected image is mapped to fit the surface and space .

12 13 14 CONFIDENTIAL 15 25 March 2014 CONFIDENTIAL 16 25 March 2014 Projection Mapping • For traditional megapixel displays Christie develops technologies like TWIST and AutoSTACK • Allow projection images to be warped and blended automatically • Supports typical screen shapes • Supports techniques such as overlaying multiple projected images for brightness

17 18 19 Projection Mapping

20 CONFIDENTIAL Projection Mapping

21 Projection Mapping • Problems?

• Approach and tools can be ad-hock, using a custom production pipeline • Every job may be uniquely built, with few reusable parts • Workflow is “whatever works”

Projection Mapping • So given this, we tried it!

• Used commonly accepted popular toolkit • COTS projector and an irregular surface to project onto. • Enabled 2D Mask workflow with custom masking pipeline

23 Projection Mapping • We wanted something reusable…

• We wanted higher quality • We wanted to augment existing design tools and technologies • We wanted to integrate a pipeline and design solution that enables tool creation

24 Projection Mapping • We wanted something reusable…

• We wanted higher quality • We wanted to augment existing design tools and technologies • We wanted to integrate a pipeline and design solution that enables tool creation

25 Projection Mapping • We wanted something reusable…

• We wanted higher quality • We wanted to augment existing design tools and technologies • We wanted to integrate a pipeline and design solution

that enables tool creation Retrieved from GPUTech 2013 On Demand, Ralph Gilles SVP Chrysler talk http://on-demand.gputechconf.com/gtc/2013/video/S3902-Gilles-SRT-GPU-Motorsports-Chrysler.mp4

26 Projection Mapping • We wanted something reusable…

• We wanted higher quality • We wanted to augment existing design tools and technologies • We wanted to integrate a pipeline and design solution

that enables tool creation Retrieved from GPUTech 2013 On Demand, Ralph Gilles SVP Chrysler talk http://on-demand.gputechconf.com/gtc/2013/video/S3902-Gilles-SRT-GPU-Motorsports-Chrysler.mp4

27 WE BUILT SOMETHING ELSE…

28 29 Inspiration • Automotive and other design applications frequently use scale prototypes as a design verification step

Retrieved from Oobject, a Curations Creation Retrieved from Oobject, a Curations Creation http://www.oobject.com/category/12-clay-car-mockups/ http://www.oobject.com/category/12-clay-car-mockups/

30 Inspiration • Clay models for automotive design in particular are ubiquitous • Static and Expensive to produce • Frequently finished, alterations can take quite a bit of time to execute • Tools have evolved for generating the models, opportunities still exist for taking this further...

Retrieved from Car Body Design http://www.carbodydesign.com/archive/2008/06/02-mercedes-benz-new-design-studio-california/

31 Inspiration • Clay models for automotive design in particular are ubiquitous • Static and Expensive to produce • Frequently finished, alterations can take quite a bit of time to execute • Tools have evolved for generating the models, opportunities still exist for taking this further...

Retrieved from Oobject, a Curations Creation http://www.oobject.com/category/12-clay-car-mockups/

32 Inspiration • Clay models for automotive design in particular are ubiquitous • Static and Expensive to produce • Frequently finished, alterations can take quite a bit of time to execute • Tools have evolved for generating the models, opportunities still exist for taking this further...

Photo Credit, Aaron Gold, 2007 Los Angeles Auto Show http://cars.about.com/od/autoshows/ig/2007-LA-Auto-Show-pics/Mazda-Clay-Concept-LAAS07.htm

33 Inspiration • Clay models for automotive design in particular are ubiquitous • Static and Expensive to produce • Frequently finished, alterations can take quite a bit of time to execute • Tools have evolved for generating the models, opportunities still exist for taking this further...

Retrieved from Oobject, a Curations Creation http://www.oobject.com/category/12-clay-car-mockups/

34 Inspiration • Clay models for automotive design in particular are ubiquitous • Static and Expensive to produce • Frequently finished, alterations can take quite a bit of time to execute • Tools have evolved for generating the models, opportunities still exist for taking this further...

Retrieved from Oobject, a Curations Creation http://www.oobject.com/category/12-clay-car-mockups/

35 Inspiration • Reproduce and augment the clay experience through Projected Light • Add dynamic details with the use of projection • Provide a path for iterative design • Rendered visuals and background to generate an immersive experience and integrate the model within a virtual space

36 Retrieved from AutoBlog, Jaguar designers deconstruct XF Clay model -- http://www.autoblog.com/2012/06/06/jaguar-designers-deconstruct-xf-clay-model-with-axes/

37 Problem/Challenges • Display Design • Megapixel canvas support • Resolution in PPI required • Image quality from DEP • Interactivity • 3D printing a car • Rendering, clustering, graphics • Warping/blending

38 Problem/Challenges • Display Design • Megapixel canvas support • Resolution in PPI required • Image quality from DEP • Interactivity • 3D printing a car • Rendering, clustering, graphics • Warping/blending

39 Designing a megapixel canvas • Placing the car with in an enclosed environment provides a more immersive experience • MicroTiles provide excellent pixel density and good ambient light rejection

40 Designing a megapixel canvas • Projector location needs to be chosen that fit physically within in the display but also provide sufficient coverage • Many other factors come into play • Brightness • Pixel shape • Warping required • Shadows

41 Designing a megapixel canvas • A simulation of the car and projectors is an essential design tool in optimizing image quality • Achieving a desired PPI density is crucial in creating a compelling life-like experience

42 Picture Quality • Pixel Per Inch (PPI) • Target: 50-100 PPI

Channel 3

Channel 1

Channel 2

43 Display Development • Pixel pitch dictates projection coverage, from an initial Design Eye Point • 43 Microtiles used to aid in integrating 120 degree H-FOV from the User DEP • Further optimization for coverage considered increased horizontal viewer positions

Design Engineering, Shawn Mills

44 Display Development • Pixel pitch dictates projection coverage, from an initial Design Eye Point • 43 Microtiles used to aid in integrating 120 degree H-FOV from the User DEP • Further optimization for coverage considered increased horizontal viewer positions

Design Engineering, Shawn Mills

45 3D Printing a model for projection • Cost effective • Starts with the existing 3D Asset • 3D Asset is also required for the graphics application • Some processes are need to craft a good model for printing • “shell”-ing • Internal support structure • Shadow elimination

46 Adapting a model for 3D printing and projection • Removing details to avoid shadows when projecting • Details will be added back via projection • Simplifies the model for 3D printing

47 Adapting a model for 3D printing and projection • Removing details to avoid shadows when projecting • Details will be added back via projection • Simplifies the model for 3D printing

48 Rendering/Clustering • Computing power • 7x HP z620 / z820 • Cluster is spit into two, for rendering the car versus the backdrop. • Balancing and synchronizing the car and backdrop • 12 NVIDIA k5000 with Quadro gSync

49 Rendering/Clustering • Real-Time Graphics engine • We use RTT DeltaGen r11 for generating visuals, RTT is a market leader in design visualization within automotive • For simplicity both clusters are running DeltaGen Master/Slave and share the asset database and just suppress some information at init time

50 RTT DeltaGen Virtual World Building • The virtual world • Allowed for Pre Visualization of all display surfaces, and final 3D Asset in its final position • This quality virtual rig aids in display calibration prior to runtime • Helps the artists develop the scene prior to final integration with the display product

51 RTT DeltaGen Virtual World Building • The virtual world • Allowed for Pre Visualization of all display surfaces, and final 3D Asset in its final position • This quality virtual rig aids in display calibration prior to runtime • Helps the artists develop the scene prior to final integration with the display product

52 RTT DeltaGen r11 Scene Configuration

53 R&I: Audi R8 Projection Map Interactive Experience

Asset Design &PrintAsset Design &Print Virtual World Design Final Rendered Result

54 Warping and Blending • Ideally the projected image is rendered from the projector’s location • Not always possible • Limitations in application software • Lighting and reflections may be wrong

55 Warping and Blending • The viewer’s location is where the image needs to look right • Rendering the image from the viewer’s location is where typical application do this

56 Warping and Blending • Warping is the transformation that corrects the projector’s image for the viewer. • Allows us to account for many factors • Physical geometry errors • Intermediate view points • Lens distortion

57 Warping and Blending • Careful planning on the design side is needed to minimize the amount of warping necessary • Optimizing the eye point and other view parameters (FOV, etc.) can also help.

58 Warping and Blending • Careful planning on the design side is needed to minimize the amount of warping necessary • Optimizing the eye point and other view parameters (FOV, etc.) can also help.

59 Warping and Blending • Careful planning on the design side is needed to minimize the amount of warping necessary • Optimizing the eye point and other view parameters (FOV, etc.) can also help.

60 Warping and Blending • NVIDIA Warp + Intensity API enables the warping and blending • The API is a low level warping engine that need additional software support • Ideal for us because we can layer our existing projector based warping tools (Twist/AutoCal) on top • Ideal for the application because the warping support is baked into the driver, no need for software changes

61 Results – 3D Surface in a Virtual World

62 Results – Projector/Viewer Calibration

63 Results – Reflection and Colour, Final

64 Integrated Result – Reflection and Colour, Final

65 Future work • The car looks great but requires a lot of time • Both in setup and design • How to make it faster and better? • Aim to formalizing workflow, enabling broader use

66 Automation • Machine vision and structured light • Reduced manual calibration of warping and blending • Expand our existing AutoCal solution to handle more complex geometry

67 Automation • Feed key information up stream • Projection locations • Which model to use • Model orientation • Create a unified system, a software platform to tie the virtual and physical worlds together

68 Advanced canvas correction • Front projection leads to global illumination effects • Ruins contrast • Does not give a faithful reproduction of the content • Attempt to compensate for indirect lighting contribution • Use white and black level blending for correction Full Light

69 Advanced canvas correction • Front projection leads to global illumination effects • Ruins contrast • Does not give a faithful reproduction of the content • Attempt to compensate for indirect lighting contribution • Use white and black level blending for correction Direct Light

70 Advanced canvas correction • Front projection leads to global illumination effects • Ruins contrast • Does not give a faithful reproduction of the content • Attempt to compensate for indirect lighting contribution • Use white and black level blending for correction Indirect Light

71 Advanced canvas correction

72 Previsualization • CAD For Projection Mapping • Tailored tools, for: • Lumens, brightness • Coverage, shadowing • Warping, blending • Leads to simulation and eventually virtualization of whole process

73 Conclusion • Effective 3D Projection Mapping • Includes a fully 3D workflow • Uses 2D Masking and warping sparingly • Integrates Content Creators needs and where possible full DCC solutions • Create opportunities for further tool and technology development

74 Cheers!