A Web Infrastructure for Certifying Multimedia News Content for Fake News Defense
Edward L. Amoruso Stephen P. Johnson Raghu Avula Cliff C. Zou Department of Computer Science Department of Computer Science Department of Computer Science Department of Computer Science University of Central Florida University of Central Florida University of Central Florida University of Central Florida Orlando, FL 32816 US Orlando, FL 32816 US Orlando, FL 32816 US Orlando, FL 32816 US [email protected] [email protected] [email protected] [email protected]
Abstract—In dealing with altered visual multimedia content, Research has shown that visual content can affect public also referred to as fake news, we present a ready-to-deploy extension opinion, leading to various levels of confusion [5]. One example of the current public key infrastructure (PKI), to provide an of altered visual multimedia was seen when someone published endorsement and integrity check platform for newsworthy visual a fake photo of Mitt Romney, shown in figure 1, presidential multimedia content. PKI, which is primarily used for Web domain candidate, spelling the word “Money” instead of his last name authentication, can directly be utilized with any visual multimedia “Romney”. Figure 1(b) was posted on Facebook adding bias file. Unlike many other fake news researches that focus on technical and negative opinion with the following caption, multimedia data processing and verification, we enable various news "The Romney family misspells their own name in what might organizations to use our developed program to certify/endorse a be the greatest Freudian slip in US history" [4]. This post also multimedia news content when they believe this news piece is received over three thousand comments and was shared to over truthiness and newsworthy. Our program digitally signs the multimedia news content with the news organization’s private key, ninety thousand users just on Facebook alone. A study and the endorsed news content can be posted not only by the performed in 2018 conveyed that Facebook was considered the endorser, but also by any other websites. By installing a web browser primary platform for social media. It concluded that 68% of U.S extension developed by us, an end user can easily verify whether a adults used Facebook, with three-quarters accessing the site multimedia news content has been endorsed and by which daily [7]. According to a survey by Pew Research Center organization. During verification, our browser extension will present performed in December of 2016, about 64% of the 1,002 U.S. to the end user a floating logo next to the image or video. This logo, adult citizens interviewed said fabricated news stories cause a in the shape of a shield, will show whether the image has been great deal of confusion about the basic facts of current issues [5]. endorsed, by which news organization, and a few more pieces of essential text information of the news multimedia content. The proposed system can be easily integrated to other closed-web system such as social media networks and easily applied to other non-visual multimedia files. Keywords—Fake News, Information Certification /Endorsement, Newsworthy Visual Multimedia Content.
I. INTRODUCTION (a) Real Image (b) Altered image This paper was motivated by the difficulty associated with identifying fake news, particularly the misleading visual Figure 1. Example showing a real image (a) and fake image (b) of Mitt Romney multimedia content that deludes users on social networks. The posing for a picture prior to a presidential rally. use of images and videos on the internet have been a driving This paper will ultimately demonstrate a system with which force in forging opinions. With the plethora of new software the end-user will be able to identify whether visual multimedia tools available to the common end-user, it has become effortless is validated by a news organizations website. The contributions to produce fake images and misrepresenting as valid content. of this paper will be to provide: These fake images can look plausible enough making them more dangerous with social media’s ability to spread them across the • A new paradigm on fighting against fake news by providing globe before they can be debunked by fact-checking sites. It is a communication channel between news organizations and estimated that over three billion images are shared every day on end users, instead of directly verifying the truthiness of a social media and three hundred hours of video per minute are news piece. uploaded to YouTube [2]. With such statistics and aggressive • competition of news organizations trying to publish their story An easy to implement infrastructure for news organizations first, journalists are forced to spend less time validating the to endorse/certify the truthiness of a news piece, and provenance of images and videos. enables an end user to easily verify the endorsement of a news piece by which news organization via our browser visual multimedia content. Other emphasis in our search was extension. placed on techniques that utilized the CA’s (Certificate Authority) private and public keys for use in visual multimedia • The endorsing/certification of a news content by a news organization has no expiration date and puts the news verification process. organization’s reputation on the line, which will prevent The closely related work is presented in “A method for irresponsive endorsement or usage of our proposed verifying integrity & authenticating digital media,” written by platform. Martin Harran, William Farrelly, and Kevin Curran use a similar component to footprint an image for validation. Harran, The rest of this paper is organized as follows. Section II Farrelly, and Curran only focus on embedding the metadata covers an overview of other related works and how our approach within the image file. They state, “Inserting the certificate into differs. In Section III, we introduce our proposed approach and the file is the only apparent reliable method and is the approach cover the design of both the server and client methodology. used by leading companies such as Microsoft and Adobe for Section IV we discuss the implementation of both the server file digital signing of Office and PDF documents” [1]. To achieve authentication and endorsement, and the client browser extension. Section V will go over both the limitations and future image verification, Harran, Farrelly, and Curran impose the use work of our paper. Finally, Section VI provides our conclusion. of an accepted standard called XMP (Extensible Metadata Platform) to add metadata to various formats of image files, especially JPEG’s. XMP essentially provides a roadmap for metadata placement in several types of image files [3]. II. RELATED WORK C. How Our Approache Differs from Other Approaches A. Fake News Defense in Political and Social Science In our approach, the image or video file is kept in its original Fake news with altered visual multimedia has become a format. In doing so, we can maintain the multimedia visual common tactic in confusing and deceiving readers on the Internet for either political or personal gain. In scenarios of content integrity. This also plays an important role as you will deception, the user’s interpretation can cause a reaction of panic see later in our design. We take advantage of XMP (Extensible and further proliferation of the fake visual multimedia content. Metadata Platform) without the modification of any multimedia More dangerously, the users may experience fear or ire leading visual files, eliminating third party software altogether, by to, for example, the selling of stocks or even creating riots [9]. solely using the XMP sidecar file for establishing a digital Other motivations for creating fake visual multimedia content is footprint. The digital footprint stores both the visual to lure users to a specific website, also referred to as multimedia file’s hash and digital certificate for use in the “clickbait’s”, for advertisement revenue gain [8]. In response, validation process by our web browser extension. This there have been several approaches proposed by different approach eliminates the embedding of metadata within the entities to help detect fake visual multimedia content. Social image file, removing overhead and complexity, for establishing media companies such as Google, Facebook, and Twitter have visual multimedia validation. attempted to solve this problem by denying individuals, associated with the publishing of misleading information, from III. OUR PROPOSED APPROACH acquiring revenue from clicks and increased traffic [8]. Under A. Objective and Design the assumption that credible users provide credible tweets, users on Twitter spread fake news are credible users who are unable The proposed approach is one that maintains the integrity of to verify the news and spread it unintentionally [11]. These the image or video file without any modification while at the companies have contributed little in solving the spread of fake same time providing a hash and digital signature for validation visual multimedia content [8]. Other research has proposed and verification. The hash and digital signature, also referred to solutions that can detect and filter out web sites containing false as the footprint, is embedded into the XMP file located together and misleading information. These approaches typically require with the image file. The HTML page will contain a metatag a custom tool to be downloaded and installed by the end user. which points to the sidecar file for the particular image. An One technical paper, “Detecting Fake News in Social Media example of the shield’s placement and colored border around a Networks”, mentions such a tool in identifying and blocking verified/failed multimedia content can be seen in Figure 2. fake news. According to the authors of this paper, once the tool Our design can be used on any forms of multimedia news is enabled, it uses various techniques such as syntactic content (image, video, etc). For explanation purpose, in this characteristics to detect misinformation [8]. Other proposed paper we will mainly use image as the example for our design works involve image analysis techniques on visual multimedia description. content to determine their validity. These solutions introduce complexity that could easily deter news agencies from utilizing Figure 3 demonstrates the browser extension’s process flow them. as the web page is loaded by the client’s web browser. Our browser extension will first search for the metatag identified B. Related Technological Approaches in Fake News Defense with “x-media-cert”, example shown in Table 3, and then During our research of proposed works that could detect and proceed by reading a sidecar file representing the image. On warn the end user of fake multimedia content, we were unable completion, finally presenting the client a green or red shield and find any that used our approach. We searched for detection border around the image. If the “x-media-cert” metatag is not techniques that did not involve the modification of the original found, the extension will ignore the image. You will notice in Figure 2, if the metadata or sidecar file are missing, the browser Our designed platform mainly contains two parts. One is the extension will ignore the image making it transparent to the end program utilized by news organizations to certify a news user’s web browser. multimedia content; the other is the browser extension program that can be installed and used by end users to verify the certified news content on webpages.
B. Extensible Metadata Platform (XMP) Using already developed standards provides leverage for seamless integration with various platforms and technologies. In an effort to reduce development time and increase innovative efficiency, XMP (Extensible Metadata Platform) was (a) Extension failed verification. (b) Extension passed verification effectively applied to help store metadata in XML (Extensible Figure 2. Extension notification for each image. A floating shield is displayed Markup Language) format. in the top left corner with a colored border around the image after the browser extension performs validation. When the extension detects that an image has XMP (Extensible Metadata Platform) is a data model that been modified, a red symbol and border are displayed with the image as shown provides a unique name and value to broadly represent data in (a). A green shield and boarder are displayed when the extension successfully types, especially those used in digital file formats. It was validates the image as shown in (b). created by Adobe Systems as an ISO standard for use in the interchanging of standardized and custom metadata for digital documents and data sets. XMP’s design provides the roadmap for embedding metadata information into an image, video and document file formats. Metadata describes the properties of a resource, for example, in a jpeg image, the content will include actual binary data while the metadata contains properties such as author, creation date, location, and other information about the image. In scenarios where the image file can’t be manipulated, an XMP sidecar file is implemented instead. XMP sidecar files are not allowed to contain any binary data, but instead use various encoding methods (e.g. base64) to allow the inserting of metadata information. C. Digital Signature using Public Key Infrastructure (PKI)
, Authenticator’s public key (+) and - + − private key ( ) 𝐾𝐾𝐴𝐴 𝐾𝐾𝐴𝐴 , Certificate Authority’s public key and Figure 3. Flow-chart of the browser extension installed by end users. + − private key used in our certification In summary, the design goals of our proposed platform are: 𝐾𝐾𝐶𝐶𝐶𝐶 𝐾𝐾𝐶𝐶𝐶𝐶 procedure • We do not provide news authentication or verification by ( ) Digital certificate for the authenticator’s our platform itself, but instead provide a platform that − + web server facilitate news organizations to certify/endorse a 𝐾𝐾𝐶𝐶𝐶𝐶 𝐾𝐾𝐴𝐴 multimedia news content they believe are truthiness and Metadata The essential information about an image newsworthy. added by the authenticator, including date, geolocation, brief description of the • Our designed platform is transparent to end users who are image story not using our system and does not affect their browsing experience. The added certification information by news image File containing any graphical file format organization is ignored by the end user’s web browser if supported by the webserver (e.g. gif, the browser extension is not installed or disabled. jpeg, jpg, etc.) • Multimedia news content can be easily forwarded or m Digital content of an image and its reposted by other web publishers and still maintain the Metadata authentication information. H(m) Hash value of data content m, e.g., • End user-side component is realized by a browser SHA256 digest of the content m extension so that it can be easily installed by users. ( ( )) Digital signature of the data content m − Table 1. Main notations used in this paper. 𝐾𝐾𝐴𝐴 𝐻𝐻 𝑚𝑚 The digital signature, also referred to as the signed message as the snippet shown in Table 2, is then updated by adding digest, is used to digitally sign a message (referred as m). The additional tags in the markup language with the reference of the XPM file should contain both the digital signature and the hash associated XMP sidecar file as shown in Table 3. values. We want to emphasize that the digital signature is not the hash value H(m), it is the H(m) encrypted by the sender’s img img-fluid" alt="Wreckage" x-media- (from the sender’s digital certificate), decrypt the digital cert="https://cda6908.s3.amazonaws.com/wreckage.xmp"> signature by using the sender’s public key, then compare the results (hash value) with the hash value computed by itself to Table 3. Modified HTML Page image attributes. Note the highlighted code see if they match or not. points the XMP sidecar file that is used by the browser extension to validate the image. D. Server Side Design The server-side program refers to our developed program E. Client Side Design utilized by news organizations to endorse/certify a multimedia news content. The server’s implementation is typically The client side is unique in that it only requires a browser performed by the news agency’s web development team. These extension to accomplish its verification of the visual developers will require some working knowledge of X.509 multimedia content. The browser extension which is written in certificates and how to compute the SHA256 digest. Once the Java Script can be downloaded from the web browsers news agency has verified the visual multimedia content it is up extension store. to them to create the metadata and XMP sidecar file and include In the following, we provide the client-side browser it with the image file. Prior to generating the digital signature, extension code operation procedure: the news agency must have acquired an RSA private/public key pair for the authenticator’s website. Below we break down each 1) Load web page and all its images and associated XMP step involved in preparing the server side of our platform. sidecar files if exist. Processing an image and generating the associated XMP file by 2) For Each image that has an associated XMP file: the authenticator: a) Extract the digital certificate of the authenticator from 1) Authenticator’s personnel inputs Metadata for the XMP file, i.e., ( ). image (date of image, geolocation of the image content, − + b) Process this digital𝐶𝐶𝐶𝐶 𝐴𝐴 certificate to exact: (1). brief description text of the image story) 𝐾𝐾 𝐾𝐾 Authenticator’s name; (2). Authenticator’s public key, 2) Concatenate the image content with the metadata i.e., . together (for example, as a temporary file) + c) Extract𝐾𝐾𝐴𝐴 the digital signature contained in the XMP file, 3) Create SHA256 digest, i.e., ( + ). i.e., ( ( + )). − 4) Create the digital signature𝐻𝐻 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 we need,𝑖𝑖𝑖𝑖𝑖𝑖 𝑖𝑖𝑖𝑖i.e., d) Use 𝐾𝐾𝐴𝐴 to𝐻𝐻 decrypt𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 the digital𝑖𝑖𝑖𝑖𝑖𝑖 𝑖𝑖signature𝑖𝑖 to recover and ( ( + )). obtain + the digest provided by the sender, i.e., 𝐴𝐴 − ( 𝐾𝐾 + ). 5) 𝐾𝐾Create𝐴𝐴 𝐻𝐻 the𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 image’s associated𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 XMP file, containing: e) Extract the Metadata contained in the XMP file, ( ( + )) ( ) 𝐻𝐻 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 Metadata + + concatenate it with the image file. − − + 6) Modify the𝐾𝐾𝐴𝐴 webpage𝐻𝐻 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 snippet 𝑖𝑖containing𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 the𝐾𝐾𝐶𝐶 𝐶𝐶image𝐾𝐾𝐴𝐴 f) Compute the SHA256 digest by the plugin itself using reference to add the reference to the associated XMP file the same hash function, i.e., ( + (see example in Table 3). ). ′ 𝐻𝐻 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 g) 𝑖𝑖Compare𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑖 the extracted digest provided by the sender, 𝐻𝐻match,𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 the image𝑖𝑖𝑖𝑖𝑖𝑖 and′ 𝑖𝑖 𝑖𝑖its metadata are verified and continue next; otherwise,𝐻𝐻 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 printout error𝑖𝑖𝑖𝑖𝑖𝑖 message𝑖𝑖𝑖𝑖 on the Table 2. Non-modified HTML Page. This is a typical image tag found on a image (such as adding a red-colored border around the news organization’s website. image in our implementation) and finish this image operation. Both the visual multimedia and XMP sidecar files must be place in the same directory on the webserver. These files have h) Display information showing the image is certified the same name except for their file extensions which identify (such as adding a green-colored border around the them as either the visual multimedia file (e.g. filename.jpg) or image in our implementation). When the user wants to the XMP sidecar file (e.g. filename.xmp). Finally, the HTML know more about the certification information, a page containing the reference to the visual multimedia file, such simple mouse click will display essential metadata information of the image in the browser on top of the creatorValue, headlineValue, and descriptionValue. Once the image. The information includes: Authenticator’s information is collected, our script then creates the XMP sidecar Name; Image taken date; Geolocation of the image; file (shown in Table 4) based on those inputs. The file created and brief text description of the image story. Figure 4 will have the image file name plus an additional extension shows an example of the certification information if an “.xmp” added (e.g., imagefilename.jpeg.xmp). In the case end user clicks the green shield icon on a certified where our shell script is not supported by the organization’s image content. environment (e.g., Microsoft Windows), a manual approach can be taken to create the XMP sidecar file. This method requires an application called OpenSSL that is available on many platforms including Microsoft Windows [12]. OpenSSL can be used to obtain the Base64, digest, and signature values for our file authentication and endorsement. The steps needed to create our XMP sidecar file are specifically outlined below: 1. Create a new file with the same name as the image file and add the “.xmp” extension to it (e.g., imagefilename.jpeg.xmp). 2. Copy the contents of the XMP template format shown in Table 4 into this file (e.g., imagefilename.jpeg.xmp) 3. Replace the following metadata fields with corresponding values provided by the news agency: dateTimeValue, cityValue, regionValue, countryValue, creatorValue, headlineValue, and descriptionValue. 4. Use the information provided by the news agency and the Base64(image) value of the image to calculate digestValue for the xmp file. The order of values shown below must be followed for the plugin to work correctly. H(metadata + Base64(image)) = H( (dateTimeValue + cityValue + regionValue + countryValue + creatorValue + Figure 4. Display of certification information on an example certified image. headlineValue + descriptionValue) + Base64 (image) ) = By clicking with the mouse on the green shield added by the browser extension, digestValue. This can be calculated by using the following a pop-up window provides the following essential metadata of the image: command: Endorsing organization’s name, Date & Time, Geolocation, Photographer, and brief text description. openssl dgst -sha256 tempMetaImageFile.txt > digestValue 5. Calculate the signature of digestValue by performing K-( IV. IMPLEMENTATION H(meta + image) ) = signatureValue. This can be calculated Our implementation is broken up into two parts. The first by using the following command: part, referred as ‘server-side implementation’, involves the setup openssl dgst -sha256 -sign private.key -out signatureValue and staging of file authentication and endorsement, which digestValue. should be used by news organizations who want to certify some newsworthy multimedia news contents. The second part in our 6. Before replacing the signatureValue with the value implementation is the browser extension, which will be installed calculated from K-( H(meta + image) ), we must convert it by end users on their browsers to process the visual multimedia to Base64 for our xmp file. This is simply done by making content and display the certification/endorsement information. signatureValue = Base64(signatureValue). A. Server File Authentication and Endorsement 7. Finally, replace the certificateValue with the public key that was provided by the Certificate. The server-side implementation is platform independent, allowing it to be hosted on various web server technologies such This process is repeated for each image on the webpage. Only as Microsoft Internet Information Services (IIS), Apache, those images that require validation should have their Nginx, and any others that support the HTTPS protocol. It is corresponding XMP sidecar file, all other miscellaneous important to understand that we only require two things for the multimedia content such as background and thumbnails can be server-side to work, the image file and XMP sidecar file, the rest skipped. Once completed, there is nothing else that needs to be is left to our browser’s plugin. Preparing the server file done by the news organization. authentication and endorsement can be accomplished manually or automatically by using our shell script program. The shell script program is written in BASH (Bourne Again SHell) and is
The proposed fake news defense system gives big and well- known news organization the capability to endorse or certify real news. However, it does not help smaller organization or individual to make their voice heard. If a news content is VI. CONCLUSION [2] Lago, F., Phan, Q., & Boato, G. (2019). Visual and textual analysis for image trustworthiness assessment within online news. Security and Our paper has demonstrated that by utilizing existing digital Communication Networks, 2019, 1-14. doi:10.1155/2019/9236910 certificate infrastructure and digital signature technique, we can [3] XMP, “Adding Intelligence to Media” [online] provide a reliable and easy-to-deploy system to pass the https://www.adobe.com/products/xmp.html endorsement of a news organization on newsworthy multimedia [4] TECHPRESIDENT, “How a Romney Gaffe that Wasn’t Went Viral on content to end consumers. We established this by using a sidecar the Web” [online] http://techpresident.com/news/22741/how-ap-photo- file, that is easily created with our scripts, to contain the digital became-viral-romney-meme signature of the news content. In conjunction with our browser [5] Pew Research Center, “Many Americans Believe Fake News Is Sowing Confusion” [online] https://www.journalism.org/2016/12/15/many- extension, the sidecar file is used to validate the image as they americans-believe-fake-news-is-sowing-confusion/ are loaded by Internet browser. The user is then quickly alerted [6] Jones - Jang, Mo & Kim, Joon. (2018). Third person effects of fake news: to an altered or fake image with an easy to identify marker on Fake news regulation and media literacy interventions. Computers in the image. If the user does not have, or has disabled the browser Human Behavior. 80. 10.1016/j.chb.2017.11.034.Sample extension, the sidecar file is completely ignored by the user’s [7] Pew Research Center, “Social Media Use in 2018” [online] Internet browser. This design provides transparency to the end https://www.pewinternet.org/2018/03/01/social-media-use-in-2018/ user and allows for a voluntary and graduate adaptation and [8] Monther Aldwairi, Ali Alwahedi (2018). Detecting Fake News in Social deployment. Media Networks. Procedia Computer Science 141 (2018) 215-222. [9] S. B. Parikh, V. Patil, R. Makawana and P. K. Atrey, "Towards Impact Scoring of Fake News," 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, USA, 2019, ACKNOWLEDGMENT pp. 529-533. We would like to thank all the reviewers for their review and [10] Smith, J., Leavitt, A., Jackson, G., 2018. Designing new ways to give context to news stories. [online] https://medium.com/facebook- comments. This work is supported by the National Science design/designing-new-ways-to-give-context-to-news-stories- Foundation under grant DGE-1915780. f6c13604f450 [11] Z. Jin, J. Cao, Y. Jiang and Y. Zhang, "News Credibility Evaluation on REFERENCES Microblog with a Hierarchical Propagation Model," 2014 IEEE International Conference on Data Mining, Shenzhen, 2014, pp. 230-239. [1] Harran, M., Farrelly, W., & Curran, K. (2018). A method for verifying integrity & authenticating digital media. Applied Computing and Informatics, 14(2), 145-158. doi:10.1016/j.aci.2017.05.006