Porting C/C++ applications to the web PROJECT P1 Fundamental Research Group Mentors Nagesh Karmali Rajesh Kushalkar Interns Ajit Kumar Akash Soni Sarthak Mishra Shikhar Suman Yasasvi V Peruvemba List of Contents Project Description Simavr Understanding the Problem Overall Problems Emscripten References Vim.wasm Tetris PNGCrush.js Project Description Lots of educational tools like electronic simulators, emulators, IDEs, etc. are written mostly as a native C/C++ applications. This project is focussed on making these applications run on the web as a service in the cloud. It is focused on porting these desktop applications into their JavaScript counterparts. Understanding the problem 1. Converting the underlying logic written in C/C++ into it’s analogous Javascript file 2. Making sure minimum performance loss occurs during porting. Emscripten 1. Emscripten is an Open Source LLVM to JavaScript compiler. 2. Compile any other code that can be translated into LLVM bitcode into JavaScript. https://emscripten.org/docs/introducing_emscrip ten/about_emscripten.html Installing Emscripten # Get the emsdk repo git clone https://github.com/emscripten-core/emsdk.git # Enter that directory cd emsdk # Fetch the latest version of the emsdk (not needed the first time you clone) git pull # Download and install the latest SDK tools. ./emsdk install latest # Make the "latest" SDK "active" for the current user. (writes ~/.emscripten file) ./emsdk activate latest # Activate PATH and other environment variables in the current terminal source ./emsdk_env.sh Porting of Vim Github Repository https://github.com/rhysd/vim.wasm Source files of Vim that are slightly modified can be found here - https://github.com/rhysd/vim.wasm/compare/a9604e61451707b38fdcb088fbfaeea 2b922fef6...f375d042138c824e3e149e0994b791248f2ecf41#files The Vim Editor is completely written in C. Hence, we can use Emscripten, the LLVM to Web Assembly compiler, to convert the C/C++ code to JavaScript which can later be hosted on the web. The entire Vim codebase is of around 2,00,000-3,00,000 lines of code. Making it insanely difficult for us to understand it, in turn causing a major roadblock in calling the right functions according to input in the generated js file. Errors Faced 1. Web Assembly is not recognised by Ubuntu as a valid MIME (Multipurpose Internet Mail Extensions). So the line ‘application/wasm wasm’ Had to be added to the list of MIME types in /etc/mime.types 2. A function called “Pointer_stringify” was being used by the Ported program, but it has been removed from Emscripten. Hence, we had to use another function called “UTF8toString” which resolved this particular issue. 3. In the same string of the Issue raised with the creator, It was found that they also faced the same problems with the emterpret() function, with Asynchronous operations occurring in the “Vim.js” file. It required a much greater and in-depth look into the functioning of emterpret() to solve this issue. Port of the Tetris game Making appropriate Understanding Adding the web changes in C++ Porting of the game the c++ source assets(HTML,CSS) source code to make to Web using code it compatible with emscipten. emscipten. 1. Understanding the code 1. Codebase consists of c++ class files(.cpp) and header files(.h). That represents objects within the context of the game. 2. The various classes include the Piece class ,Board class and the Game class. 3. The constants.h stores all the global variable used in the project so as to increase the readability of the code. 4. The Piece class contains logic to define pieces in terms of shape,color,current orientation .It contains functions to move display the block(move),to rotate the block(rotate)and the isblock() function to determine which kind of piece is present, its location and orientation. 5. The board class instances of pieces class and contains functions to display pieces which have reached the bottom of the board,detect collision between the pieces and to the unite function to clear any full row from the board,moving all pieces above it down by one and to increase the score by one. 6. The game class contains functions to manage the whole game,including generating new random pieces and to enable one to move pieces around the board with key presses. 7. The main.cpp acts as the entry point to the game and runs the game loop until SDL_quit is triggered. 2. Adding the web 1. The game requires us to add basic frontend to display the game in the browser. 2. The HTML files creates a canvas on which the game can be loaded and also includes a div tag under which the current score will be displayed using the inner.hmtl attribute . 3. The css file is optional but is required to display the current score in exactly the same font as would have been displayed by SDL2_ttf(true type font). 3. Making changes in C++ files 1. Although SDL_ttf is supported by emscipten and can be ported but since its function is only to display the score so for reducing the unnecessary complexity of the program we remove the use of SDL_ttf to display score in the DOM. 2. Since the display score is not part of the c++ file so we remove the extra space allotted in the screen height global variable. 3. In the Board class we change the display_score() function so that the score will be displayed by DOM.here emscripten_run_script action finds the<span> element on the DOM and sets the inner html to the current score. 4. Now since the SDL_ttf is not required we update all the functions and header files using the SDL_ttf library. 5. Emscipten provides a emscipten_set_main_loop function that accepts a em_callback_func looping function.however the gameloop() can’t be passed to em_callback_func since it will fail to build.so we remove the game class itself and move the logic to main.cpp and reorder in such a way that all the functions are declared before they are used. 4. Porting of the game to the web using emscripten. 1. To port the game to the web first navigate to the project directory. 2. In the command line type-emcc --bind src/board.cpp src/piece.cpp src/main.cpp -std=c++14 -O3 -s WASM=1 -s USE MODULARIZE=1 -s USE_SDL -o public/index.js 3. To run the game type emrun --browser firefox(chrome from google chrome) --no_emrun_detect public/index.html PNGCrush PNGCrush is a free and open-source command-line utility for optimizing PNG image files. It reduces the size of the file losslessly – that is, the resulting "crushed" image will have the same quality as the source image. It is mainly written in C programming language. ● Open Source Command-line .png image optimizer ● Reduces the size of .png images based on various compression methods Porting Process for PNG Crush Get Source File Changes were made in Binary code file was generated PNGCRUSH makefile of PNG Crush through emscripten which was further used to generate the js file PNG Crush v1.8.11 used . Instead of compiling with gcc, it emcc pngcrush.bc -o pngcrush.js was compiled with emscripten compiler emcc. Source code : https://pmt.sourceforge.io/pngc rush/ Problems Faced 1. The original PNG Crush runs through the terminal and accepts two parameters <INPUT.PNG> and <OUTPUT.PNG> with optional flags 2. Reading / Writing files Giving files to files as input to C++ is EASY :) but there is no direct method to give file inputs to JavaScript :( Sol : Emscripten has a rich File System API which is not included to js file by default but can be included as FS in module object during compile process. FS.createDataFile(“/”, “file_name.png”,byteArray, true, true); FS.deleteFile("/file_name.png"); 3. Communicating with Module object of the generated js File Emscripten provides a global Module object that is used to interact with the ported c++ code. 4. Allowing Users to download the compressed file Sol : This was achieved through creation of Blob object and filsaver.js which has the implementation of saveAs() functionality in javaScript GitHub Link : https://github.com/eligrey/FileSaver.js/ Implementing a Getting image as input Compressing the image with minimal HTML page and sending it to Module[‘run’] generated js file The web page allows the user to This was achieved by using the Created files were given to Module[‘run’] in upload PNG image and get the inbuilt File Reader API of the js. generated js file, this populates the output.png with compressed file. The image was read in an compressed image. The compressed image is now base64 encoding and converted available to download. to byteArrayBuffer via b64.js.. The files input.png and output.png were created using Emscripten FS api SimAvr 1. A lean and mean Atmel AVR simulator for linux.(or any platform which uses avr-gcc) 2. The simulator takes input as ELF file(s)(.hex) and generates corresponding waves using gtkwave(tool, as waveform viewer). 3. It outputs signals like interrupts to be dumped into the file which is then used by gtkwave to plot corres. Waves. 4. It serves, thus, much as a Debugger for avr codes and also emulating peripherals to test before using on actual Hardware. ● Compiled and run on the machine and it runs fine. ● Code contains .c and .h files in two folders. ● Tried to compile using Emscripten; successful in generating .bc file of whole project but improper generation of.js, became the turning point. Raised Issues Repo: https://github.com/emscripten-core/emscripten/issues/8799 Overall Problems till now ● Communication with Emscripten port ● Lack of support for some part of openGL and SDL libraries ● Any program that needs to be ported, must be very scrutinously picked apart for it’s various functionalities, i.e it is pivotal to know each and every functionality of the given program that we try to port..