Augmenting Film/Video Footage with Sensor Data

Total Page：16

File Type：pdf, Size：1020Kb

Augmenting Film/Video Footage with Sensor Data

Center for Embedded Networked Sensing AugmentingAugmenting Film/VideoFilm/Video FootageFootage withwith SensorSensor DataData Norman Makoto Su, Heemin Park, Eric Bostrom, Jeff Burke, Mani B. Srivastava and Deborah Estrin Networked & Embedded Systems Laboratory (NESL) & The Hypermedia Studio Introduction: Augmenting Film with Sensor Data Why an Intelligent Film Set/Stage? An Ideal Marriage: WSN + Theater/Film • Traditional Sets are Heavily Scripted and Not Dynamic • Wireless Sensor Networks Once a decision in the film process has been made (e.g. location of actors, color – Can monitor and localize data on film sets. temperature of lights, etc it is very expensive and cumbersome to change later. – Are small and unobtrusive to performers, directors and technicians. Post-production insertion of computer graphics or other effects often requires – Are easily deployed in various filming environments. careful preplanning and precision from the camera crew. A First Step: ARS, Augmented Recording System – Augmented Footage is sensor data synchronized with each film or video • As a result performer creativity and freedom is stifled. frame. We want actors and actresses to be able to interact with their environment. – The crew can view recorded film and observe its corresponding data in In addition, we want the film set to actively record useful non-visual real-time. information about a shot. – Define a new level of seamless integration between computer graphics and real world photography. Problem Description: Issues in Developing a Synchronized Data Recording System Timecode Generator Recording Data Audio signal System Design We augment SMPTE (an industry standard Mechanisms for time synchronization 29.97 frames/sec timecode for video) synchronized sensor data to compensate for clock drift. Serial Port Server with film frames. Data is needed for each frame, 0:05:11:27 timecode decoder Custom MAC protocol for high speed 0:05:11:28 which is approximately every 1/30 seconds. We 0:05:11:29 transfers with PALOS. 0:05:12:00 29.97 frames/sec 0:05:12:01 0:05:12:02 filter redundant data and buffer readings at : Middleware software Sylph for 80 bits / frame sensor registration. the sensors-side. Timecode hh:mm:ss:frame(0-29) Proposed Solution: Augmented Recording, a Subset of the Intelligent Film Set System Architecture Evaluation in TV Studio & Experimental Setup Mote Neighborhood Serial Port Server Timecode Generator SQL Database Server Sylph Server Middleware Film Camera Jini Client Snapshot of Frame 11 and 45 UCLA – UCR – Caltech – USC – CSU – JPL – UC Merced.

Copy Link

Recommended publications

The General Idea Behind Editing in Narrative Film Is the Coordination of One Shot with Another in Order to Create a Coherent, Artistically Pleasing, Meaningful Whole
Chapter 4: Editing Film 125: The Textbook © Lynne Lerych The general idea behind editing in narrative film is the coordination of one shot with another in order to create a coherent, artistically pleasing, meaningful whole. The system of editing employed in narrative film is called continuity editing – its purpose is to create and provide efficient, functional transitions. Sounds simple enough, right?1 Yeah, no. It’s not really that simple. These three desired qualities of narrative film editing – coherence, artistry, and meaning – are not easy to achieve, especially when you consider what the film editor begins with. The typical shooting phase of a typical two-hour narrative feature film lasts about eight weeks. During that time, the cinematography team may record anywhere from 20 or 30 hours of film on the relatively low end – up to the 240 hours of film that James Cameron and his cinematographer, Russell Carpenter, shot for Titanic – which eventually weighed in at 3 hours and 14 minutes by the time it reached theatres. Most filmmakers will shoot somewhere in between these extremes. No matter how you look at it, though, the editor knows from the outset that in all likelihood less than ten percent of the film shot will make its way into the final product. As if the sheer weight of the available footage weren’t enough, there is the reality that most scenes in feature films are shot out of sequence – in other words, they are typically shot in neither the chronological order of the story nor the temporal order of the film.
[Show full text]
American Vandal Episode
American Vandal Episode 101: The Crime Vandalism, Vulgarity and Destruction on March 15, 2016 written by Tony Yacenda and Dan Perrault TEASER EXT. THE HOME OF DYLAN MAXWELL - ESTABLISHING - DAY A modest southern California house with a patchy yard and a 2001 Acura parked in front of a one car garage. SUPER: April 2, 2016 INT. THE HOME OF DYLAN MAXWELL - KITCHEN - DAY DYLAN MAXWELL, 17, sits in a chair facing the camera. The framing is awkward; we see a light at the top of the frame as PETER ANDERHOLM, 15, slips a lav mic down Dylan’s Etnies T- shirt. PETER (O.C.) (PRELAP) Tell us your name and who you are. JUMP CUT: INT. KITCHEN - MOMENTS LATER A well composed interview setup. Dylan looks right into the camera. DYLAN My name is Dylan Maxwell. And I’m... I don’t know. I’m Dylan. What do you mean who am I? PETER It’s okay. You can look at me and not the camera. Dylan, why am I here interviewing you? DYLAN Because I didn’t fucking do it. PETER Didn’t do what? DYLAN The dicks. Cue music. Begin montage: 2. LOCAL NEWS REPORT - EXT. STAFF PARKING LOT - DAY The camera pans across the staff parking lot at Hanover High School. We see graffiti on dozens of cars. It’s all blurred. FEMALE REPORTER (V.O.) It was supposed to be a quiet “Administrative Day” at Hanover High - a day off for students as teachers took part in a series of mandatory faculty meetings. But there was one student activity that did, in fact, take place: vandalism.
[Show full text]
(12) United States Patent (10) Patent No.: US 6,778,252 B2 Moulton Et Al
USOO6778252B2 (12) United States Patent (10) Patent No.: US 6,778,252 B2 Moulton et al. (45) Date of Patent: Aug. 17, 2004 (54) FILM LANGUAGE 5,880,788 A 3/1999 Bregler 5,884.267 A 3/1999 Goldenthal et al. (75) Inventors: William Scott Moulton, Kentfield, CA 6,097,381 A 8/2000 Scott et al. (US); Steven Wolff, Woodacre, CA OTHER PUBLICATIONS (US); Rod Schumacher, Los Angeles, CA (US); Andrew Bryant, San Diego, Bregler et al., “Video Rewrite: Driving Visual Speech with CA (US); Marcy Hamilton, Los Audio,” ACM Siggraph 97, Interval Research Corporation, Angeles, CA (US); Strath Hamilton, No.97, (Mar. 8, 1997). Los Angeles, CA (US); Dana Taschner, Ezzat et al., “Mike Talk: A talking Facial Display Based on Sunset Beach, CA (US) Morphing Techniques,” Proceedings of Computer Anima tion Conference, (Jun. 8, 1998). (73) Assignee: Film Language, Los Angeles, CA (US) Ezzat et al., “Visual Speech Synthesis by Morphing Visemes.” A.I.Memo, MIT, No. 165, (Mar. 8, 1999). (*) Notice: Subject to any disclaimer, the term of this Brand et al., “Voice-Driven animation.” TR-98-20, Mitsub patent is extended or adjusted under 35 ishi Electric Research Laboratory, (Mar. 8, 1998). U.S.C. 154(b) by 53 days. Burnett et al., “Direct and Indirect Measures of Speech Articulator Motions. Using Low Power EM SEnsors,” XIV (21) Appl. No.: 10/027,191 International CongreSS of Phoenetic Sciences, Lawrence (22) Filed: Dec. 20, 2001 Livermore National Laboratory, (Mar. 8, 1999). e a Vs (65) Prior Publication Data (List continued on next page.) Primary Examiner Rodney Fuller US 2002/0097380 A1 Jul.• 4-225, 2002 74Y A ttorney,y, AgAgent, or Firm-SanfordFirm-Sanfor AstO Related U.S.
[Show full text]
The Front Runner
The Front Runner Written by Matt Bai & Jay Carson & Jason Reitman July 27th, 2017 Blue Revisions 8/28/17 Pink Revisions 9/10/17 Yellow Revisions 9/15/17 ii. Note: The following screenplay features overlapping dialogue in the style of films like The Candidate. The idea is to create a true-to-life experience of the Hart campaign of 1987. CAST OF CHARACTERS THE HARTS GARY HART, SENATOR LEE HART, HIS WIFE THE CAMPAIGN TEAM BILL DIXON, CAMPAIGN MANAGER BILLY SHORE, AIDE-DE-CAMP KEVIN SWEENEY, PRESS SECRETARY JOHN EMERSON, DEPUTY CAMPAIGN MANAGER DOUG WILSON, POLICY AIDE MIKE STRATTON, LEAD ADVANCE MAN IRENE KELLY, SCHEDULER AT THE WASHINGTON POST BEN BRADLEE, EXECUTIVE EDITOR ANN DEVROY, POLITICAL EDITOR AJ PARKER, POLITICAL REPORTER DAVID BRODER, CHIEF POLITICAL CORRESPONDENT BOB KAISER, MANAGING EDITOR AT THE MIAMI HERALD KEITH MARTINDALE, EXECUTIVE EDITOR JIM SAVAGE, EDITOR TOM FIEDLER, POLITICAL REPORTER JOE MURPHY, INVESTIGATIVE REPORTER ROY VALENTINE, PHOTOGRAPHER THE TRAVELING PRESS JACK GERMOND, BALTIMORE SUN COLUMNIST IRA WYMAN, AP PHOTOGRAPHER ALAN WEINBERG, PHILADELHIA ENQUIRER ANN MCDANIEL, NEWSWEEK MIKE SHANAHAN, AP MIAMI DONNA RICE, MODEL AND ACTRESS BILLY BROADHURST, HART’S PERSONAL FRIEND LYNN ARMANDT, RICE’S FRIEND “1984” EXT. SAINT FRANCIS HOTEL, SAN FRANCISCO. NIGHT. We open inside a NEWS VAN. Four monitors show different competing feeds. A waiting reporter. Color Bars. A political commercial. One monitor is cueing up a debate clip. A light pops on the reporter and he springs to life. TV REPORTER Yes, we learned just a few minutes ago that Senator Hart will soon be leaving this hotel back to the convention hall, where he will concede -- yes, he will concede -- to former vice president Walter Mondale.
[Show full text]
Video Rewrite: Driving Visual Speech with Audio Christoph Bregler, Michele Covell, Malcolm Slaney Interval Research Corporation
ACM SIGGRAPH 97 Video Rewrite: Driving Visual Speech with Audio Christoph Bregler, Michele Covell, Malcolm Slaney Interval Research Corporation ABSTRACT Video Rewrite automatically pieces together from old footage a new video that shows an actor mouthing a new utterance. The Video Rewrite uses existing footage to create automatically new results are similar to labor-intensive special effects in Forest video of a person mouthing words that she did not speak in the Gump. These effects are successful because they start from actual original footage. This technique is useful in movie dubbing, for film footage and modify it to match the new speech. Modifying example, where the movie sequence can be modified to sync the and reassembling such footage in a smart way and synchronizing it actors’ lip motions to the new soundtrack. to the new sound track leads to final footage of realistic quality. Video Rewrite automatically labels the phonemes in the train- Video Rewrite uses a similar approach but does not require labor- ing data and in the new audio track. Video Rewrite reorders the intensive interaction. mouth images in the training footage to match the phoneme Our approach allows Video Rewrite to learn from example sequence of the new audio track. When particular phonemes are footage how a person’s face changes during speech. We learn what unavailable in the training footage, Video Rewrite selects the clos- a person’s mouth looks like from a video of that person speaking est approximations. The resulting sequence of mouth images is normally. We capture the dynamics and idiosyncrasies of her artic- stitched into the background footage.
[Show full text]
Using Variable Frame Rates on the AU-EVA1 (Excerpted from “A Guide to the Panasonic AU-EVA1 Camera”)
Using Variable Frame Rates On The AU-EVA1 (excerpted from “A Guide To The Panasonic AU-EVA1 Camera”) The AU-EVA1 allows variable-frame-rate shooting in a wide selection of frame rates and frame sizes. The variable frame rates provide you with a variety of creative choices. To start with understanding why variable frame rates even matter, let’s reference back to how movie film gets shot. In film, slow motion is shot by running the camera at a faster frame rate. Film normally runs at 24 frames per second (fps), but for slow motion the camera operator might shoot it at something like 48 fps. When those 48 frames are played back at the 24 fps speed, it’ll take twice as long to play back, so everything will be moving at half speed, giving that superb film-style slow-motion look. Shooting at a faster frame rate is called “overcranking,” because in the early days cinematographers used a hand-crank to drive the camera, and for slow-mo they would actually crank the film faster. Similarly, shooting at a slower-than-normal frame rate results in a “fast motion” effect – think of the Keystone Kops or an old Charlie Chaplin movie and you’ll get the idea. If you only shoot 12 frames in a second, but you play those frames back at the 24fps speed, it’ll only take 1/2 second to play back action that took a full second to record – accordingly, the motion will be twice as fast as normal. This is referred to as “undercranking.” Using actual overcranking and undercranking can yield dramatically smoother, superior off-speed effects in your productions.
[Show full text]
Avid Film Composer and Universal Offline Editing Getting Started Guide
Avid® Film Composer® and Universal Offline Editing Getting Started Guide Release 10.0 a tools for storytellers® © 2000 Avid Technology, Inc. All rights reserved. Film Composer and Universal Offline Editing Getting Started Guide • Part 0130-04529-01 • Rev. A • August 2000 2 Contents Chapter 1 About Film Composer and Media Composer Film Composer Overview. 8 About 24p Media . 9 About 25p Media . 10 Editing Basics . 10 About Nonlinear Editing. 10 Editing Components. 11 From Flatbed to Desktop: Getting Oriented. 12 Project Workflow . 13 Starting a Project . 14 Editing a Sequence . 15 Generating Output . 16 Chapter 2 Introduction Using the Tutorial. 17 What You Need . 19 Turning On Your Equipment . 19 Installing the Tutorial Files . 20 How to Proceed. 21 Using Help. 22 Setting Up Your Browser . 22 Opening and Closing the Help System . 22 Getting Help for Windows and Dialog Boxes. 23 Getting Help for Screen Objects . 23 Keeping Help Available (Windows Only) . 24 3 Finding Information Within the Help . 25 Using the Contents List . 25 Using the Index . 25 Using the Search Feature . 26 Accessing Information from the Help Menu. 27 Using Online Documentation . 29 Chapter 3 Starting a Project Starting the Application (Windows). 31 Starting the Application (Macintosh). 32 Creating a New User . 33 Selecting a Project . 33 Viewing Clips . 34 Using Text View. 35 Using Frame View. 36 Using Script View . 37 Chapter 4 Playing Clips Opening a Clip in the Source Monitor. 39 Displaying Tracking Information . 40 Controlling Playback. 44 Using the Position Bar and Position Indicator . 45 Controlling Playback with Playback Control Buttons . 46 Controlling Playback with Playback Control Keys .
[Show full text]
A Supercut of Supercuts: Aesthetics, Histories, Databases
A Supercut of Supercuts: Aesthetics, Histories, Databases PRACTICE RESEARCH MAX TOHLINE ABSTRACT CORRESPONDING AUTHOR: Max Tohline The genealogies of the supercut, which extend well past YouTube compilations, back Independent scholar, US to the 1920s and beyond, reveal it not as an aesthetic that trickled from avant-garde [email protected] experimentation into mass entertainment, but rather the material expression of a newly-ascendant mode of knowledge and power: the database episteme. KEYWORDS: editing; supercut; compilation; montage; archive; database TO CITE THIS ARTICLE: Tohline, M. 2021. A Supercut of Supercuts: Aesthetics, Histories, Databases. Open Screens, 4(1): 8, pp. 1–16. DOI: https://doi.org/10.16995/os.45 Tohline Open Screens DOI: 10.16995/os.45 2 Full Transcript: https://www.academia.edu/45172369/Tohline_A_Supercut_of_Supercuts_full_transcript. Tohline Open Screens DOI: 10.16995/os.45 3 RESEARCH STATEMENT strong patterning in supercuts focuses viewer attention toward that which repeats, stoking uncritical desire for This first inklings of this video essay came in the form that repetition, regardless of the content of the images. of a one-off blog post I wrote seven years ago (Tohline While critical analysis is certainly possible within the 2013) in response to Miklos Kiss’s work on the “narrative” form, the supercut, broadly speaking, naturally gravitates supercut (Kiss 2013). My thoughts then comprised little toward desire instead of analysis. more than a list; an attempt to add a few works to Armed with this conclusion, part two sets out to the prehistory of the supercut that I felt Kiss and other discover the various roots of the supercut with this supercut researchers or popularizers, like Tom McCormack desire-centered-ness, and other pragmatics, as a guide.
[Show full text]
THE NINES Written by John August FINAL SCRIPT November 2006
THE NINES written by John August FINAL SCRIPT November 2006 READER NOTE One thing that will be obvious to viewers, but not to readers, is that the nine principal parts in this film are played by only three actors: 1) Gary, Gavin, Gabriel 2) Sarah, Susan, Sierra 3) Margaret, Melissa, Mary It'll make more sense in context. Promise. 100 A MAN’S HAND 100 unwinds a short length of green string. We’re extremely close, with a shallow, blurry focus. It’s like the first moments after a dream -- just fragments. Scissors cut the string. The man wraps it around his left wrist. A loop. A bracelet. We see the man’s teeth, the edge of his chin as he pulls the knot tight. His fingers pull against the string. Solid. It won’t break easily. FADE OUT. 101 PITCH BLACK 101 There’s no music. No sound at all, really, except for some distant birds CHIRPING. Then a SQUEAK. A SQUEAL as rusty springs engage. A GARAGE DOOR LIFTS, revealing GARY BANKS in silhouette. He’s 30, effortlessly fit, with movie-star good looks. (Although for now, he’s merely a TV star.) Like most Laurel Canyon garages, this one has never held a car. Instead, it’s the resting place for all the detritus of bachelordom: shitty Ikea furniture, a drum set, a styrofoam snowman, and the Harley he always meant to get running. Gary spots what he was looking for. CUT TO: 102 EXT. BACK PATIO / LAUREL CANYON HOUSE - MAGIC HOUR (DAWN) 102 Gary drags a beaten Weber kettle.
[Show full text]
Advance Digital Non-Linear Editing Technology
Vol-1 Issue-4 2015 IJARIIE-ISSN(O)-2395-4396 Advance Digital Non-Linear Editing Technology SONU SHARMA Assistant Professor, Department of Mass communication & Television-Film Technology NIMS University Jaipur, Rajasthan, India ABSTRACT Non Linear Editing technological changes have significantly simplified digital video editing. Digital video editing in its basic form can be completed by subsequent simple events and through the use of hardware and software normally found on personal computer. In order to produce excellent and intelligible audio-videos the editing process is generally directed according to the superior rules called video gramma. Computer system demonstrates promising results on large video groups and is a first step towards increased automation in non-linear video editing. This paper proposes a method to automatically section the raw video resources into valuable sections and useless sections based on the video grammar as a part of the audio video editing support system using the camerawork and cut point information. This paper justifies the steps to follow in digital video editing. It’s describes some reasons that made digital video editing simple for some but difficult for others. It’s explains the steps complicated in digital video editing and the obtainable technological alternatives in each step. We provide an overview of the use of digital non-linear editing application. Keywords: Digital Video, Linear Editing, Non Linear Editing Introduction: - Digital non-linear editing we individuals can follow simple steps to edit digital videos through their basic information of computer operations and application software. Video editing has several unique challenges not found with other media. Digital video is a time-based medium.
[Show full text]
Editing A/B Rolling
Editing Continuity editing - The majority of film sequences are edited so that time seems to flow, uninterrupted, from shot to shot. Within a ‘continuity editing’ sequence, only cuts will be used. Continuity editing can also involve ‘cross-cutting’, where a sequence cuts between two different settings where action is taking place at the same time. Montage - In montage, different images are assembled to build up an impression. This is often used in title sequences. The most famous example of this technique is the Odessa Steps sequence from Battleship Potemkin. Transitions describe the way in which one shot replaces the previous one: Cut - One image is suddenly replaced by another, without a visible transition. Cross-dissolve One image dissolves into another. This can be used to make a montage sequence - eg the title sequence - flow smoothly; it can also be used in continuity editing to show that we have moved forwards in time and/or space. Fade up - An image gradually fades in Fade out - An image gradually fades out. (Fades to and from black usually mean that time has passed ) Wipe - One image replaces another without dissolving, with the border between the images moving across or around the screen. A/B Rolling A/B rolling is a technique used in film editing to hide ugly splices. When a splice (two pieces of film stuck together with splicing tape or glued with cement) rolls through a projector, the area of tape or glue will appear fuzzy. To eliminate this nasty blip in a finished film, the negative is a/b rolled.
[Show full text]
ABSTRACT Editing Closet Memories: Theory and Methodology Grant M. Hall, M.A. Advisor: Christopher J. Hansen, M.F.A. Editing Is
ABSTRACT Editing Closet Memories: Theory and Methodology Grant M. Hall, M.A. Advisor: Christopher J. Hansen, M.F.A. Editing is a unique aspect of the filmmaking process that involves crafting production footage and audio into a cohesive story. The editor must also approach the film with an objective eye in order to craft the best narrative possible. This study examines the creative editing process of Closet Memories, a short film, from preproduction through post-production as well as the relationship between the editor and director and how the vision of each shaped the film from the rough cut to the final cut. Editing Closet Memories: Theory and Methodology by Grant M. Hall, B.A. A Thesis Approved by the Department of Communication David W. Schlueter, Ph.D., Chairperson Submitted to the Graduate Faculty of Baylor University in Partial Fulfillment of the Requirements for the Degree of Master of Arts Approved by the Thesis Committee Christopher J. Hansen, M.F.A., Chairperson James Kendrick, Ph.D. DeAnna M. Toten Beard, Ph.D. Accepted by the Graduate School May 2013 J. Larry Lyon, Ph.D., Dean Page bearing signatures is kept on file in the Graduate School. Copyright © 2013 by Grant M. Hall All rights reserved TABLE OF CONTENTS Chapter 1: Storytelling and Professional Goals ..................................................................1 Chapter 2: Literature Review ...............................................................................................5 Film Literature .........................................................................................................5
[Show full text]