WebGrab+Plus (WG++) Document V2.1 Advanced XMLTV EPG Grabber A program created by: Jan van Straaten & Francis de Paemeleere Website: www.webgrabplus.com (Document revision 09/08/2017, reflects WebGrab+Plus version 2.1.1) What’s in this document: For everyone new to this program: Read page 7, 8 and 9 (upto chapter 4.2) and APPENDIX A and B The rest of this document is for everyone willing to develop a SiteIni file or simply wants to know more than the basics. 1 Document author: Jan van Straaten ([email protected]) Document V2.1 revision 09/08/2017, reflects WebGrab+Plus version 2.1.1 New in this issue: w.r.t docoment V2.0.1: • New Chapter 5.1.5 Adding xmltv attributes • Changed Chapter 4.5.5 XMLTV attributes • Changed Chapter 1.3 Xmltv, Single - versus multiple - value xmltv elements • New site dependent setting langoverrule in 4.3 General Site dependent data and APPENDIX D • Correction of allowpageoverflow in allowlastdayoverflow in 4.3 General Site dependent data and APPENDIX D • New string converter style=roman2arabicnumeral see 4.6.4.6.3 Cleanup as string converter • New SiteIni element index_urlshow_preload in chapter and Appendix E w.r.t docoment V2.0: • Text of chapter1.2 How to run, files and folders adapted to reflect the V2.0 installer • Corrected the incorrect link to the configuration files on our website in 3. Configuration files • Changed chapter4.4.1.2 argument preload added the latest functionality of preload • New chapter 4.4.3 url_preload to introduce the latest functionality of preload • A few small changes to correct text errors Please report any omissions, errors or suggestions to [email protected] 2 Table of Content: New in this issue: ............................................................................................................... 2 1.Introduction ................................................................................................................... 7 1.1 What it does, features .............................................................................................................................................................. 7 1.2 How to run, files and folders ..................................................................................................................................................... 7 1.3 Xmltv, Single - versus multiple - value xmltv elements ............................................................................................................. 7 2.The grabbing, show update process and update modes .................................................. 8 2.1 The show update process ......................................................................................................................................................... 8 2.2 The update modes .................................................................................................................................................................... 8 2.3 The Grabbed Site pages: index-page, detail-page and sub-detail-page ................................................................................... 8 3. Configuration files ......................................................................................................... 9 3.1 WebGrab++.config.xml ............................................................................................................................................................. 9 3.2 MDB.config.xml ......................................................................................................................................................................... 9 3.3 REX.config.xml ........................................................................................................................................................................... 9 4. SiteIni file ...................................................................................................................... 9 4.1 SiteIni file parts ......................................................................................................................................................................... 9 4.2 The SiteIni file basics ............................................................................................................................................................... 10 4.2.1 scrubstrings ...................................................................................................................................................................... 10 4.2.1.1 The 'separator strings' method ................................................................................................................................. 10 4.2.1.2 The 'regular expression' method .............................................................................................................................. 11 4.2.2 Element names ................................................................................................................................................................ 11 4.2.3 Action specifiers ............................................................................................................................................................... 11 4.2.4 Types ................................................................................................................................................................................ 11 4.2.4.1 type url ...................................................................................................................................................................... 11 4.2.4.2 types single and multi ............................................................................................................................................... 12 4.2.4.3 type regex ................................................................................................................................................................. 12 4.2.5 Arguments ........................................................................................................................................................................ 13 4.2.5.1 Argument includeblock and excludeblock ................................................................................................................ 13 4.2.5.2 Argument separator .................................................................................................................................................. 13 4.2.5.3 Argument max........................................................................................................................................................... 14 4.2.5.4 Arguments include and exclude : ............................................................................................................................. 14 4.2.5.5 Argument debug ....................................................................................................................................................... 15 4.2.5.6 Dedicated Arguments ............................................................................................................................................... 15 4.2.6 String matching / wildcards ............................................................................................................................................. 16 4.2.7 TimeZones ........................................................................................................................................................................ 16 4.3 General Site dependent data .................................................................................................................................................. 16 4.4 Url builder ............................................................................................................................................................................... 18 3 4.4.1 General URL settings ........................................................................................................................................................ 18 4.4.1.1 HTTP Headers, method GET, POST, POST-BACK and SOAP ....................................................................................... 18 4.4.1.2 argument preload ..................................................................................................................................................... 19 4.4.2 url_index .......................................................................................................................................................................... 19 4.4.2.1 urldate format ........................................................................................................................................................... 20 4.4.2.2 subpage format ......................................................................................................................................................... 20 4.4.2.3 Full examples of the url_index specification ............................................................................................................. 21 4.4.3 preload Url’s ..................................................................................................................................................................... 22 4.4.3.1 preload url for url_index ........................................................................................................................................... 22 4.4.3.2 preload url for index_urlshow