Video Content Analysis for Assessment of the Advertisement

Research Article Volume 6 Issue No. 3

Video Content Analysis for Assessment of the Advertisement Jigar Gajjar1, Aayush Khator2, Pooja Patil3 Department of Computer Engineering Mumbai University, Vidyalankar Institute of Technology, India. [email protected], [email protected], [email protected]

Abstract: This paper is a literature survey paper of our system which will help advertisers to get some sort of feedback about their advertisement. Advertisement is a form communication which helps to promote a product in an attempt to induce consumers purchase that product. Advertisement are basically displayed to increase the awareness about brands that are introduced in the market or to demonstrate the differences between the products available and the competing products in order to increase their sell.Advertisement helps to get insight of the product. Now a days billboards are preferred to display advertisements in malls. Billboards are effective way of advertising as they are more attractive and noticeable. However it is not possible for advertisers to know the public response for those ads on the billboards thus creating a bottleneck. Our system helps in getting some chunk of that response from the people who are interested in the displayed ads and crunching that data to give advertisers a true feedback for their ads on the billboard. This system includes real-time detection of multiple faces and tracking of those detected faces which will be used to calculate the amount of time person spends in front of the given billboard. It also includes nose and eyes tracking to know whether the viewer's attention is at billboard or outside it. We will be even using calibrated stereo camera for depth estimation that is for knowing at what distance is the viewer from the camera. The timer will be implemented to know time spent by individual in front of the billboard. The data is being analyzed and converted in the form of statistics and visual information such as graphs to provide advertisers valuable feedback for their ads. The statistics which will be viewed after number of faces being detected and time average evaluation for each face which will help the advertiser with the scope of improvement in the advertisement and increase in profit.

Index Term: Billboard, Calibrated stereo camera, Depth estimation, Face detection and tracking, In and Out algorithm, Timer, Viewer’s attention based on head pose estimation.

I. INTRODUCTION. advertisement or announcement, a billboard is usually found Advertising is a form of communication which helps outdoors along roadways, malls etc where number of people people to know about the new products or services in the visit is large in number. Advantage of displaying market and help them to purchase the better ones by giving advertisement through billboards is it is more noticeable and the overview of the product. It plays a vital role in deciding reaches to large number of people as it is large in size and the growth of the company by means of sale, production, displayed in public places. Billboard advertisement can be revenue, popularity etc. Shopping centres, malls offer vast, placed wherever we feel it will have the high impact. This can often untapped opportunities to earn money by advertising be a huge advantage when we have a business we want to products, brands, services etc. As likely said malls are the draw traffic to rightly by some other path, or we have a shop better places to advertise because it does not emphasize only just down the road. on limited products but give user the exposure to numerous other products by making a variety of options available. Advertising is the way for advertiser to get engaged with the consumers on the issues related to product by detailing it. It can help to build trust between the customer and the product. Advertising is being viewed as a technique of mass promotion in which single message can reach to large number of people. Advertising is also being considered a one-way technique of marketing where the customer is not in position to immediately respond to the advertisement. If you want your advertisement to hit on the day a product launches or event is about to happen, this is the only vehicle you control completely. Advertisements are in various forms such as

emails, newspapers, web pages, radio, television, magazines. Fig 1: System Overview Outdoor advertisements are actually the ones which reach to large number of people and can easily catch the attention of Nowadays billboards are the most trending ways to display an the person passing by it. They are displayed outdoor on roads, advertisement in malls because of its visual impact on the malls etc. Outdoor advertisements are in the form of user. Billboards are more appealing and make your hoardings, posters, billboards, electronic displays etc. A flat advertising more noticeable. This motivated us to develop a panel or we can call it as a board that holds some kind of

International Journal of Engineering Science and Computing, March 2016 2775 http://ijesc.org/ system that helps them understand the response of the user to where they spend the maximum time in the store, when and the billboards placed in malls and in turn help them to what they shop and what they look at are all tools that help in understand the needs and interests of the onlooker. Face maximizing the retailer’s targeting efforts and profits, thus, detection of human facial features like the mouth, nose and improving their business. In addition, mostly ads displayed on eyes in a given video we can estimate the time spent by a user screens in and out of stores are not effective in reaching their in viewing the advertisement. For detection of multiple faces target audiences. Retailers can increase their profits if they can we will be using Viola-Jones algorithm [1]. We will be automatically target their customers’ with the most suited adapting a multi-step process in order to achieve the goal. advertisement.” TruMedia offered their customers a solution Once the face is being detected we will use KLT algorithm to analyse customer profile and behaviour in key retail areas. [2][3] for tracking the face. First the camera detects the multiple The solution is a completely innovative and futuristic solution faces and then captures the feature points. Further this features that provides answers to the retailer’s need to understand and points are being tracked. We will be detecting nose, eye and target his end audiences. The solution combines people mouth. We will estimate the geometric centre and get the counting with demographic measurement functionalities. As a centre coordinates. By using this will track nose, eye and result, the retailer is provided with information about his mouth centres to estimate head pose. We will be using customers like the age group and gender, and what they saw in calibrated stereo cameras for depth estimation in which we the store and for how long they looked. In addition, what is would be determining the viewer from the billboard is how the total number of customers entering the store along with far. After this we will be using our own developed algorithm their distribution throughout the day for each store. known as In and out algorithm which we are using to figure III. Related work out whether the person is looking inside the billboard for reading or looking outside the billboard. Further we will be In the last few years, there are many algorithms also using timer which helps us by giving time for how long developed for face detection and tracking, for face detection the person was looking at the billboard content. The timer will and tracking. The existing face detection methods can be be on as soon as the face is being tracked as goes off when our categorized as feature based approach and image based algorithms stop tracing face. With this information and the approach. Feature based approach requires some beforehand timers available, we get the data which is analysed and then data about faces. It makes use of facial features which the statistical reports are given to the advertisers. This includes colour, shape and component feature. Whereas, statistics will increase the scope of improvement. A good image based approach categorizes face without face without advertisement helps one to boost their profit because the good any prior knowledge about face. It feeds facial features into advertisement will draw in many new customers in their system through training. The breakthrough in face detection business. algorithms happened through Viola-Jones. It was a boon for face detection or face recognition systems. "Robust Real-Time II. Literature Survey Face Detection"[1], the paper by the inventors itself Paul Viola and Michael J. Jones have given detailed explanation of Face detection is a process to determine whether there how Viola-Jones face detection algorithm. Given as input is any face present in an image or video. Face detection is not faces and non faces it is trained to detect faces. There are three an easy process as many external and internal factors affect key contribution [1], integral image which allows the features the detection. Once the face is detected it opens the gate for a used by Viola-Jones detector to be computed quickly. Cascade wide range of information say for example one can determine which uses different stages for identifying features quickly how long a person was looking at the camera, their emotions and classify the face as face or non-face. Adaboost removes and even their gender. Some of the recent works on the redundant features or irrelevant features. application of face detection have been greatly accomplished by a Portugal company Oemkiosks. They presented the idea of Drawback of Viola-Jones algorithm was it is Real Catcher the Automated Audience Measurement Solution. computationally expensive. So the accurate and Real Catcher which is based on futuristic image analysis computationally less expensive technique is KLT (Kanade- techniques and uses a standard webcam, IP camera, or video Lucas-Tomasi) face tracking algorithm. "Good Features Track analysis systems like Kinect to perform an analysis on the "[2], explains about KLT tracker. KLT face tracking facial images of people watching displays. The system will algorithm is feature based tracking algorithm. Some more probe the stream of images generated by the camera and tracking methods [4][5][6][7][8][9], are explained in this enables it to measure the effectiveness of digital advertising, papers. Gaze tracking in many of the systems for various counting the number of people passing near the media, how reasons which depends on the system's purpose and many people are actually looking at the media and provides requirement. One of the application of gaze tracking is shown their average stay time, attention time and demographics and in the "Review Paper on Mouse Pointer Movement Using Eye other indicators as viewers gender and age group. Similar Tracking System and Voice Recognition "[9], in which they solutions are presented by companies like “Quividi”, will be using eye movement by tracking eye. Or this system “TruMedia” and many more. the input is person’s eye using which it will operate the screen They all have the same vision of determining the for performing various activities. This system is implemented user’s attention and providing them what they need. TruMedia for disable people or paralytic people. Previous years of states the need of market as “Retailers want to understand who research in algorithms on head pose estimation [10][11], helps their customers are in order to be ready with a targeted us to explore various technique of head pose estimation. products. Knowing the demographics of the user’s as well as "Feature based head pose estimation from images"[10],

International Journal of Engineering Science and Computing, March 2016 2776 http://ijesc.org/ introduced a feature based head pose estimation which is used he/she is not shopaholic, mall is the one thing which will get to know user’s focus of attention. "Method for measuring person closer to his needs and benefits. Now a day billboards stereo camera depth based on stereoscopic vision"[12], gives are widely used for displaying advertisement in malls because corresponding difference between stereo camera depth of its appearance they are more noticed by people. The resolution and human depth resolution. advertiser has no idea about how effective was to display that We will be combining advantages of some of the advertisement at that location, how many people visited that above mentioned algorithms and techniques for ad and much more. For displaying the advertisement the implementation of our system. Our approach first detects face advertiser should know what are the area in the mall where the using Viola-Jones algorithm, then this detected faces which visitors visit the most, the time at which the advertisement is are given to tracker which will track the face. We will be viewed more etc. Every advertiser is curious to get some form using KLT tracker for tracking face. Further it will also track of feedback to know how much people were interested in the eyes, nose and mouth. We will be doing depth estimation to ad. As long as there is increase in products sale which the only know viewer's distance from specific object which is billboard feedback to know how impressive and productive it was by in our case. For this we will be using calibrated stereo displaying the advertisement. Our system presents the cameras. Depth estimation using stereo camera to detect approach which will helps the advertiser to get better in their humans in video taken with a calibrated stereo camera and advertisement and increase in their profit. evaluate their distances from the camera. Then we will be implementing IN And OUT algorithm which is proposed by Acknowledgement the author of this paper, which will help us to determine A project like this takes lot of efforts and dedication for its whether viewer's attention is within the billboard area or completion. As it is often a case, this project owes its outside it. Finally we have implemented a timer which will existence and certainly its quality to number of people, whose start when face is tracked and stop when face is lost and name does not appear on the paper. We would like to thank tracker stops tracking that face. Hence, we will be using the our professor and guide Prof. Rinku Shah for her guidance best methods or algorithms mentioned above which in turn and motivation. Errors and confusions are our responsibility, will prove asset to our system. but the quality of the project is to their credit and we can only thank them. We are highly thankful and feel obliged to staff IV. Market Analysis And Benefits members for nice Co-Operation and valuable suggestions in This project is a great tool for companies who needs our project work. We would even like to thank our head of understand the reaction of viewers on the ads posted by them computer department Prof. Sachin Deshpande for his support at various locations inside a mall. It also helps them to and encouragement in completion of our project. maximise their profits by collecting data such as the average time spend by a viewer reading the ads. This system is References inspired by the ideas and case study conducted by “Quividi” [1] Paul Viola and Michael J. Jones, "Robust Real-Time Face ,“ TruMedia” and ‘verilook surveillance’. The benefits offered Detection" International Journal of Computer Vision 57(2), by the project are: 137–154, 2004 c 2004 Kluwer Academic Publishers. Manufactured in The Netherlands. · The companies will be able to customize its offering to the shoppers, which will trigger an increase in sales [2] Jianbo Shi and Carlo Tomasi, "Good Features to Track" of the products advertised on the screens on an annual IEEE Conference on Computer Vision and Pattern basis. Recognition (CVPR94) Seattle, June 1994 · The effect of Special promotions ads can be examined and compared with regular ads for the same [3] Hannes Fassold, Jakub Rosner, Peter Schallauer and product. Werner Bailer, "Real time KLT Feature Point Tracking for · People tend to pay more attention to ads at a specific High Definition Video " JOANNEUM RESEARCH, Institute time of the day, which can be calculated therefore, of Information Systems, Steyrergasse 17, 8010 Graz, Austria . special promotions should take place mostly at those hours. [4] Fu Jie Huang and Tsuhan Chen ,“Tracking of Multiple Faces for Human-Computer Interfaces and Virtual V. CONCLUSION Environments”, Electrical and Computer Engineering Our paper is a literature survey paper of our system. Carnegie Mellon University Pittsburgh, PA 15213 This paper gives brief idea about how we will be jhuangfu,tsuhan}@cmu.edu implementing our system. We have added literature survey, benefits and related work which includes study of other [5] M. Castrilloń *, O. Deńiz, C. Guerra, M. Hernańdez, systems which are quite similar to our system and algorithms, “Real-time detection of multiple faces at different resolutions techniques which even we will be using for our system. We in video streams”, Instituto Universitario de Sistemas have introduced a system which will be for the advertisers for Inteligentes y Aplicaciones Nume´ricas en Ingenierıá making their advertisement successful. In this paper we are (IUSIANI), Universidad de Las Palmas de Gran Canaria, Las representing a system which will be boon to the advertiser. As Palmas de Gran Canaria 35017, Spain Received 10 June 2005; we are aware how advertisements are something that helps to accepted 15 November 2006 Available online 22 January get the customer closer to the product by giving details of it. 2007, J. Vis. Commun. Image R. 18 (2007) 130–140 Even if person is not passionate about shopping or even if

International Journal of Engineering Science and Computing, March 2016 2777 http://ijesc.org/ [6] H.R. Chennamma, “A SURVEY ON EYE-GAZE TRACKING TECHNIQUES”, H.R. Chennamma et.al / Indian Journal of Computer Science and Engineering (IJCSE).

[7] Kyung-Nam Kim* and R. S. Ramakrishna* , “Vision- Based Eye-Gaze Tracking for Human Computer Interface “ ,*Department of Information and Communications, Kwangju Institute of Science and Technology, Kwangju, 500-712, Korea(ROK) ,0-7803-5731-O/99/$10.00 01999 IEEE

[9] Prajakta Tangade , Shital Musale , Gauri Pasalkar , “A Review Paper on Mouse Pointer Movement Using Eye Tracking System and Voice Recognition”,International Journal of Emerging Engineering Research and Technology Volume 2, Issue 8, November 2014, PP 135-138 ISSN 2349- 4395 (Print) & ISSN 2349-4409 (Online)

[10] Teodora Vatahska, Maren Bennewitz, and Sven Behnke , “Feature-based Head Pose Estimation from Images”, University of Freiburg Computer Science Institute D-79110 Freiburg, Germany.

[11] Gregory P. Meyer, Shalini Gupta, Iuri Frosi, Dikpal Reddy, Jan Kautz, “Robust Model-based 3D Head Pose Estimation”.

[12] Mikko Kytö, Mikko Nuutinen, Pirkko Oittinen, “Method for measuring stereo camera depth accuracy based on stereoscopic vision “, *mikko.kyto @tkk.fi; phone +358947023348; fax 1 222 555-876; http://media.tkk.fi/en/

International Journal of Engineering Science and Computing, March 2016 2778 http://ijesc.org/