Countries in Figures
Total Page:16
File Type:pdf, Size:1020Kb
Countries In Figures Introduction Countries In Figures is a web application designed to disseminate select data on the UNECE member countries through bars and graphs. The term web application refers to the Countries In Figures web application. What tools were used to develop the web application? 1. Visual Studio 2015, C# 2. ASP .NET MVC, Razor Pages 3. HTML 2.0, JSON, D3 Javascript library 4. Mobile-friendly approach based on responsive design ideas. 5. Python 3.6, PyOdbc 6. wkhtmltopdf What is the data model? How does the web application work? The web application is ASP .Net MVC application and separates the presentation and logic layers. In addition to the MVC classes, there are classes like, Indicators and TimeSeriesValues, used to store data. Here are some directories for the HTML-related code, /css css files /data data files /js Javascript files /js/data.js - links data files to the Javacript data variables /js/default_json.js - fills in the data variables with default data if data file is empty /js/ecegraphs_common.js - Javascript file making D3 graphs for both viewable and printable pages /js/ecegraphs_print.js - Javascript setting sizes and position for graphs on the printable page /js/ecegraphs_screen.js - Javascript setting sizes and position for graphs on the viewable page /js/general.en.json - country info file in English /js/lang.en.js - language file in English for graph elements /js/update_info.json - update info file with update Id and access code How are SQL tables updated? Data tables are updated in the TSSL job named countries_in_figures. Click here to download the TSSL code for the countries_in_figures TSSL job. The resulting data tables (SQL tables matching the result TSSL cubes) are used by the Python code to populate the data files. Some TSSL cubes are cleaned from data if data is incomplete for a chosen TSSL variable, i.e. there is no data for ALL values in the value set describing the TSSL variable in question. Example, the Internet Use cube is cleaned for the variable Sex_ID thus making sure that only data available for both sexes will be published! What are the Application Management SQL Tables? There are three application management SQL tables, CountriesInFiguresUpdates, stores update Id and access code to make PDF files. Here are the table fields, UpdateId - update Id to version data update files AccessCode - an integer that is used to grant access to the printable web page to the Python code only that creates PDF printable pages. CountriesInFiguresIndicators, stores indicators list, input TSSL cubes, output data file format etc. Here are the table fields, Item_ID - indicator item Id, may point to the secondary cubes in the CountriesInFiguresSecondaryCubes table. VariablePrefix - indicator/variable code, should be the same as the indicator code in the data.js file when applicable Description - indicator description ElementType - may get the following values - T, stands for text, data is saved in the JSON file for single-value text elements - G, stands for graph, data is saved in the Javascript file for graphs. Cube - input TSSL cube UpdateType - - S, single value text element - SJ, single value text element joined with another single value text element - T, simple time series - TJ, simple time series joined with another simple time series - M, multiple value data set used in complex graphs TimeSpan - time span allowed for dissemination Decimals - number of decimals in the result data set UnitInDecimals - number of decimals in the axe to show PivotedVariable - TSSL variable name to pivot to show in a complex graph, like stacked bar etc. SortedVariable - TSSL variable which value set is used to sort text elements in the graph DifferentPeriods - whether or not different time periods are authorized. It is used in complex graphs or joint indicators only. CountriesInFiguresSecondaryCubes, stores list of secondary cubes that share the same time periods with the primary cubes described in the CountriesInFiguresIndicators. Here are the table fields, Item_ID - code of the primary TSSL cube in the CountriesInFiguresIndicators table. Cube - name of the secondary TSSL cube. This field is used to make sure that the time periods in the secondary cubes and the primary cue are shared, i.e. same. How to update the version of the web application? Go to Web.config file in the root folder of the web application, find WebApplicationVersion key and change its value to the next version. Increment the version to force external users' browsers to update the CSS, Javascript files etc. How to deploy the web application? Please follow the list below to compile and deploy the web application, 1. Make sure that the version is incremented as this will force update of the cached CSS and Javascript files in the users' browsers. 2. Create the folder at C:\inetpub\wwwroot\w3.unece.org\CountriesInFigures 3. Start configuring publishing the project. Use File System as the publishing method. 4. Publish the output of the Visual Studio project of the web application in the above folder. 5. Remove update_info.json and general.XX.json (e.g. general.en.json) files in the /js folder in order to avoid replacing the same file on the system. 6. Copy and replace files from the above folder to the respective folder on your web server. How is web statistics logged? All the web statistics is saved to a file. The file path and name are set by the WebStatsLogFilePath key, in the Web.config file in the root folder of the web application. Make sure that your log file is outside your project folder to avoid it is rewritten during the project update How to change language files? So far two languages are supported by the system, English and Russian. The suffix "en" or "ru" are added to the language files for the respective languages. There exist two language file groups, 1. Graph files. Their format is lang.XX.js, where XX is the language code. So far there are two files, lang.en.js and lang.ru.js. The files contain translation of the graph elements. 2. Country files. Their format is general.XX.json, where XX is the language code. So far there are two files, general.en.json and general.ru.json. Th e files contains translation of country specific information, like country title, currency, capital etc, How is data versioned? Data update version is specified in two places, (1) SQL Server database, (2) the web application update file. 1. SQL Server database. There exist the CountriesInFiguresUpdates table in the PCAxis database that contains the update Id. This Id is updated at the end of the PC-Axis Update and thus authorizes release of fresh data. If the Id is different from the one specified in the update file described below, then the Python code creates new files with fresh data. 2. Web application update file. There exist the update_info.json file with the current update Id. In addition, there is the AccessCode that is generated automatically during the last update and protects the printable web page from viewing it by the external user. Only the Python code "knows" the access code and thus is able to generate the PDF file for printing and viewing it. How are data files used in the web application? The update Id defines the version of the data files. The Id is an integer number which is then used as a prefix in the data file names. There exist three types of data files created by the Python code. Below is their description and file name conventions. 1. Javascript files, contains data for graphs. File name convention is a_XXXXXXX_data_graph_YYY.js. In the file name convention XXXXXXX is the update Id, YYY is a three-digit country code. E.g. a_0000045_data_graph_703.js. 2. JSON files, contains data for single value indicators. File name convention is a_XXXXXXX_data_text_YYY.json. In the file name convention XXXXXXX is the update Id, YYY is a three-digit country code. E.g. a_0000045_data_text_703.json. 3. PDF files, contain data for printing. File name convention is a_XXXXXXX_print_YYY.pdf. In the file name convention XXXXXXX is the update Id, YYY is a three-digit country code. E.g. a_0000045_pdf_703.pdf. How to install Python environment? 1. Install Python 3.6 in the custom-defined folder at C:\Python36-32 2. Install PyOdbc. 3. Install wkhtmltopdf to C:\Program Files (x86) 4. 4. Make a folder for Python code at C:\CnF_Update 5. Make sure that create_files.bat points at the right Python folder 6. Make sure that connection parameters and path to wkhtmltopdf in create_files.by are set correctly. How to run Python code? Run create_files.bat to create files with fresh data. What does the Python code do? The Python code does the following, 1. Reads the CountriesInFiguresUpdates table in the SQL Server and reads the update Id 2. Reads the access code in the update_info.json. 3. Uses the update Id to prefix data files. 4. Uses the access code to access the printable web page and create PDF files for printing. 5. Loops through the country codes in the specified value pool and creates Javascript, JSON, PDF files. 6. Sets the new version by modifying the update Id in the update_info.json. How to set up the Python code in the Task Scheduler? If you want to use Windows Task Scheduler, do NOT set the Program/script field in the Task Scheduler to the path for create_files.bat in. If done so, Task Scheduler will not be able to run the script. Instead, set the Program/script field to Python like, C:\Python36-32\python.exe. Then set the Add argument (optional) field to the path for bat file like, C: \CnF_Update\create_files.py.