US 201401 11670A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2014/0111670 A1 LOrd et al. (43) Pub. Date: Apr. 24, 2014 (54) SYSTEMAND METHOD FOR ENHANCED Publication Classification IMAGE CAPTURE (51) Int. Cl. (71) Applicant: NVIDIA CORPORATION, Santa EN 2. 30.8 Clara, CA (US) (52) U.S. Cl. (72) Inventors: Nathan Lord, Santa Clara, CA (US); USPC ......... 348/240.99; 348/207.99; E. Patrick Shehane, Fremont, CA (US) (57) ABSTRACT A systemand method for image capture. The method includes (73) Assignee: NVIDIA CORPORATION, Santa configuring an image sensor to capture at a full resolution of Clara, CA (US) the image sensor and automatically capturing a first image with the image sensor irrespective of a shutter button of a camera. The method further includes receiving an image cap (21) Appl. No.: 13/658,117 ture request and accessing a second image after the receiving of the image capture request. The first image is captured prior to the receiving of the image capture request. The first image (22) Filed: Oct. 23, 2012 and the second image may then be stored. 704 < 3 > 706 Pre-Shutter Burst Count < 2 > 708 Pre-Shutter Skip Count < 5 > Post-Shutter 710 Burst Count < 1 > Post-Shutter 712 Skip Count Patent Application Publication Apr. 24, 2014 Sheet 1 of 10 US 2014/011 1670 A1 115 LOCal Graphics Memory FIG. 1 Patent Application Publication Apr. 24, 2014 Sheet 2 of 10 US 2014/011 1670 A1 |elauueo eZOZ Patent Application Publication Apr. 24, 2014 Sheet 3 of 10 US 2014/011 1670 A1 OO Capture Preview Display Preview Image Image 302 304 NO Take Picture Request? 306 Yes Flush Outstanding Preview Captures 308 Change Sensor Resolution 310 Capture Image 312 Change Sensor Resolution 314 FIG. 3 Patent Application Publication Apr. 24, 2014 Sheet 4 of 10 US 2014/011 1670 A1 2s. 2. 3 3f.S C w Patent Application Publication Apr. 24, 2014 Sheet 5 of 10 US 2014/011 1670 A1 Change Sensor Resolution 502 Capture A First Take Picture Image Request? 508 504 Display Preview Yes Image 506 Capture A Second Image 510 Access The First Image And The Second Image 512 Display The First Image And The Second Image 514 FIG. 5 Patent Application Publication Apr. 24, 2014 Sheet 6 of 10 US 2014/011 1670 A1 Patent Application Publication Apr. 24, 2014 Sheet 7 of 10 US 2014/011 1670 A1 70/ 90/ 80/ 0|/ Z||/ ?unOO?Sung Patent Application Publication Apr. 24, 2014 Sheet 8 of 10 US 2014/011 1670 A1 S Patent Application Publication Apr. 24, 2014 Sheet 9 of 10 US 2014/011 1670 A1 S. Patent Application Publication Apr. 24, 2014 Sheet 10 of 10 US 2014/011 1670 A1 Computer Readable Storage Medium 1004 Removable Storage Imaging Module 1008 OO6 Non-Removable mage careCapture moduleModul Storage 1010 Image Sensor Image Capture Image Sensor Congon Request Module Control Module 1024 1026 1022 Output Device(s) 1016 Image Storage Image Scaling Module Module 1028 O3O Input Device(s) 1014 Interface Module 1O4O Communication Graphical User Image Selection Connection(s) 1012 Interface Module Module 1042 1044 Image Encoder Module 1050 Processing Unit Image Storage Module 10O2 106O Fig. 10 US 2014/011 1670 A1 Apr. 24, 2014 SYSTEMAND METHOD FOR ENHANCED with the image sensor irrespective of a shutter button of a IMAGE CAPTURE camera. In one embodiment, the first image is stored in a circular buffer. The method further includes receiving an FIELD OF THE INVENTION image capture request and accessing a second image after the 0001 Embodiments of the present invention are generally receiving of the image capture request. The first image is related to image capture. captured prior to the receiving of the image capture request. The image capture request may be based on a shutter button BACKGROUND OF THE INVENTION press, received from a camera application, or received via an 0002 AS computer systems have advanced, processing application programming interface (API). The first image and power and speed have increased substantially. At the same the second image may then be stored. The method may further time, the processors and other computer components have include displaying the first image and the second image in a decreased in size allowing them to be part of an increasing graphical user interface. In one embodiment, the graphical number of devices. Cameras and mobile devices have ben user interface is operable to allow selection of the first image efited significantly from the advances in computing technol and the second image for storage. The method may further ogy. The addition of camera functionality to mobile devices include Scaling the first image to a preview resolution where has made taking photographs and video quite convenient. the preview resolution is less than the full resolution of the 0003. The timing of the image capture can be critical to image sensor. capturing the right moment. If a user presses a shutter button to capture an image too early or too late the intended picture 0008. In one embodiment, the present invention is imple may be missed. For example, hesitation of a user in pressing mented as a system for image capture. The system includes an the shutter button while at a sporting event could result in image sensor configuration module operable to configure an missing a key play of the game, such as a goal in Soccer. image sensor to capture at a full resolution of the image sensor 0004. The timing of the capture of an image may also be and an image capture request module operable to receive an impacted by the speed of the camera. A request from the image capture request. The image capture request module is shutter button may go through a software stack having a operable to receive the image capture request from a shutter corresponding delay or latency before reaching hardware button, from an application (e.g., camera application), or an which also has a corresponding delay. The hardware delay application programming interface (API). The system further may be partially caused by delay in reading the pixels of a includes an image sensor control module operable to signal camera sensor. Thus, even if the user is able to press the the image sensor to automatically capture a first image irre shutter button at the desired moment in time, the delay of the spective of a shutter button of a camera. In one embodiment, camera may result in capturing an image too late thereby the first image is stored in a buffer. The image sensor control missing the desired shot. Conventional Solutions have module is further operable to signal the image sensor to focused on making image capture faster by reducing the delay capture a second image, where the first image is captured after the time the shutter button is pressed. Unfortunately, prior to the image capture request. The system may further while a faster camera may have a reduced delay, this fails to include an image selection module operable to generate of a solve issues related to the timing of the shutter button press graphical user interface operable for selection of the first and the delay from the camera is still present. image and the second image for at least one of storage and 0005 Thus, a need exists for a solution to allow capture of deletion. The system may further include a scaling module an image at the desired moment irrespective of device hard operable to scale the first image and the second image to a ware delays or timing of a shutter button press. second resolution, where the second resolution is lower than the full resolution of the sensor. SUMMARY OF THE INVENTION 0009. In another embodiment, the present invention is 0006 Embodiments of the present invention are operable directed to a computer-readable storage medium having to continually capture full resolution images, irrespective of a stored thereon, computer executable instructions that, if shutter button of a camera, to memory such that when a user executed by a computer system cause the computer system to presses or pushes a shutter button, images that have been perform a method of capturing a plurality of images. The captured prior to the shutter button press are available for a method includes automatically capturing a first plurality of user to select and save (e.g., to storage). A user thereby has images with an image sensor operating in a full resolution access to images captured prior to the shutterbutton press and configuration and receiving an image capture request. The thereby can overcome reaction time delays and device delays capturing of the first plurality of images is irrespective of a (e.g., Software and hardware delays). Embodiments of the shutter button of a camera. The first plurality of images is present invention are further operable to provide images that captured prior to receiving the image capture request. In one are captured after the shutter button press (e.g., a burst of embodiment, the first plurality of images is captured continu images). Embodiments of the present invention are also oper ously and stored in a circular buffer. In one exemplary able to allow a user to navigate and select images that were embodiment, the number of images in the first plurality of captured before and after the shutter button press. Embodi image is configurable (e.g., by a user). The method further ments of the present invention thus allow a user to select the includes accessing a second plurality of images after the most desired image(s) captured before and after the shutter image capture request and displaying the first plurality of button press.