KOFAX Communication Server Knowledgebase article ID: 00005239 FoIP Trace Guidelines Version 1.00.00 Knowledgebase article ID: 00005239 Question / Problem The customer has problems with receiving or sending faxes over a Fax over IP (FoIP) network and you need to gather information to narrow down the problem and pass on traces to another support member for further analysis. Identifying the Problem The most important step to solve a problem is to identify where the problem may be located. FoIP problems are most likely located inside the environment, either because the call routing is not correctly set up, T.38 support is not activated or there are problems with the fax communication itself. The difficulty with call routing problems is that inbound calls are not traceable on the KCS-FoIP machine. This is because the calls never reach said machine. Only outbound calls are traceable and even they may result in unclear error messages or no errors at all, as routing problems can lead to various outcomes, e.g. call being routed to an unreachable destination, call being routed to the wrong destination etc. Problems with T.38 support are relatively easy to identify but not so easy to locate. Typically, the Wireshark or TWS traces give a clear error message which is caused by a rejected request to switch to T.38 mode. Fax communication problems are unique problems, most of the time, and rarely have the same cause as similar problems. These problems are difficult to trace and locate and a good portion of these are caused by software bugs either in the FoIP software or in the environment. Preparation You need a text editor with advanced search capabilities. It should be able to list all lines matching a specific search string (Windows Notepad can only navigate to the next search result). If possible, the text editor should accept also 'regular expression' search terms. Additionally the text editor should be able to handle multiple files and search also in multiple Files (if possible also using regular expressions). UltraEdit is a good commercial product with advanced search capabilities.(http://www.ultraedit.com) PSPad (http://www.pspad.com/en) and Notepad++ (http://notepad-plus-plus.org/) are good free alternatives. Collecting the traces There are basically 2 traces you can collect on the KCS-FoIP machine: TWS traces and Wireshark traces. TWS TWS traces are the traces from the KCS-FoIP software itself. The name (TWS) derives from the underlying platform on which the KCS-FoIP runs. TWS traces by default are located in the directory: “C:\TOPCALL\FoIP\00\trace\” and the files itself are named in this format: “tws_xxx.trc” (xxx is used for file numbering). In a new installation you should have up to 10 files at any given time and each has a default maximum size of about 2 MB. 1.Trace Levels The default trace settings generate a dump of T.38 messages if the call fails. This means that many problems can be analysed with the default settings. If you decide to increase the traces levels you should check the other FoIP Trace Guidelines Version 1.00.00 © 2012 Kofax, Inc. Page 2 of 16 Knowledgebase article ID: 00005239 trace file settings as well to prevent that traces are overwritten. Extra note: high trace settings may cause additional problems due to real-time constraints of FoIP. Trace settings are configured as follows: Start the configuration tool "Configure FoIP" from the "Kofax Communication Server" start menu group. Internet Explorer opens showing different sections, expand the section "Advanced". Here you can configure the TraceLevel, the MessageTraceSize (see below), the "Size of Trace file" (2000 or higher is recommended) and the "Number of Trace files" (10 or more is recommended). After changing any configuration values, KCS Foip must be restarted using KCS Monitor. Recommended Trace settings: TraceLevel MessageTraceSize Used for basic trace information (that is often sufficient) without negatively 10 1 impacting performance. This is the default configuration. Provides lot of information. More suitable when duplicating error situations 40 10000 in controlled environment. Attention: Please do not increase the TraceLevel on your own initiative. Only increase it if you are instructed by a Kofax Support Technician as unnecessary high trace settings can make the complete trace useless. 2.Get an overview about all calls Use a text editor and then filter the traces for "bound Call:" to get a list of all inbound and outbound calls. Here is an example output: … "03:UFI"} Inbound Call: CallerId=@55205, TSI=+4318635321 EC= (), Peer=172.20.148.52:30006 … "02:UFI"} Inbound Call: CallerId=@55205, TSI= EC=XL(I:HUMAN;55;21121;4-305), Peer=172.20.148.52:30010 … "03:UFI"} Inbound Call: CallerId=@55205, TSI=+4318635321 EC=ID(I:PROT;70;21120;4-305), Peer=172.20.148.52:30014 … "02:UFI"} Outbound Call: Number=I142 -> 142, CallerId=, CSI=, EC=JX(I:UNKN;45;12603;9-103), Peer=:0 … "02:UFI"} Outbound Call: Number=142 -> 0142, CallerId=, CSI=+4318635321, EC= (), Peer=172.20.148.52:30020 … "02:UFI"} Outbound Call: Number=142 -> 0142, CallerId=, CSI=+4318631, EC=XO(I:PROT;65;21313;15-305), Peer=172.20.148.52:3024 … "02:UFI"} Outbound Call: Number=142 -> 0142, CallerId=, CSI=, EC=ID(I:PROT;60;21322;4-305), Peer=172.20.148.52:30028 … "02:UFI"} Outbound Call: Number=142 -> 0142, CallerId=, CSI=+4318635321, EC= (), Peer=172.20.148.52:30032 In FoIP versions since KCS 9.1.1, the value "EC=" contains the 2-digit TCOSS error code as well as the extended FoIP error information as described in the FoIP manual. In older versions, only the FoIP error is present. “EC= ()” means success. Only the first three values in the brackets are important for error analysis. Example of an error: EC=XL(I:HUMAN;55;21121;4-305) or EC= I:HUMAN;55;21121;4-305 As described above, the value before the brackets defines the 2-digit TCOSS error code. Values in the bracket are separated with semicolons. The first value (in this example "HUMAN") defines the error category – the list can be found in the FoIP Technical Manual, chapter "7.2.1 - Error Categories". The second value (in this example: "55") defines the connection level, i.e. in what state the call was aborted. – FoIP Trace Guidelines Version 1.00.00 © 2012 Kofax, Inc. Page 3 of 16 Knowledgebase article ID: 00005239 definitions for this can be found in chapter "7.2.3 - Connection Level". The third value (in this example: "21121") defines the error code – the list can be found in chapter "7.2.2 - Error Codes". 3.Debug info’s of failed calls All failed calls generate additional debug info as shown in the example below. This debug info can be found directly for every "bound call:". In fact, the "bound call:" statement is the last entry in this debug info so you can always find it directly above. "03:UFI"} <DisconnectInd> "03:UFI"} <CommResult> "03:UFI"} <ErrorCategory>PROTOCOL</ErrorCategory> "03:UFI"} <ConnectionLevel>70</ConnectionLevel> "03:UFI"} <ErrorCode>21120</ErrorCode> "03:UFI"} <Diagnostics>Line DCN during reception (Rx_EPg_InPartialPage->2:DisconnectReq)</Diagnostics> "03:UFI"} </CommResult> "03:UFI"} <StopReason>OK</StopReason> "03:UFI"} <Debug> "03:UFI"} <ConnectAudio time="328">Remote media address=172.20.148.52:30012, LocalPort=10004, RTP_PT=8</ConnectAudio> "03:UFI"} <Connect_T38 time="546">Remote media address=172.20.148.52:30014, LocalPort=10004, T.38 Version=0, AllowV34=1</Connect_T38> "03:UFI"} <Tx time="749">ced/IND_CED</Tx> "03:UFI"} <Tx time="3963">no_sig/IND_NO_SIGNAL</Tx> "03:UFI"} <Tx time="4041">preamble/IND_V21_PREAMBLE</Tx> "03:UFI"} <Tx time="5055">T30Csi/DATA_V21 Data[23]=ff c0 02 04 04 04 04 04 04 8c 8c 8c 8c 8c 8c 04 8c 8c 8c 04 8c 8c d4 </Tx> "03:UFI"} <Tx time="5055">T30Dis/DATA_V21 Data[7]=ff c8 01 00 77 1f 22 </Tx> "03:UFI"} <Tx time="6069">hdlc_sig_end/DATA_V21</Tx> "03:UFI"} <Rx time="6662">preamble/IND_V21_PREAMBLE</Rx> "03:UFI"} <Rx time="8346">T30Optional/DATA_V21 Data[23]=ff c0 c2 8c 4c cc ac cc 6c 1c 8c cc 2c d4 04 04 04 04 04 04 04 04 04 </Rx> "03:UFI"} <Rx time="8658">T30Dcs/DATA_V21 Data[7]=ff c8 c1 00 44 1f 22 </Rx> "03:UFI"} <Rx time="8658">hdlc_sig_end/DATA_V21</Rx> "03:UFI"} <Rx time="8658">no_sig/IND_NO_SIGNAL</Rx> "03:UFI"} <Rx time="8768">training/IND_V17_14400_LONG_TRAINING</Rx> "03:UFI"} <Rx time="10452">non_ecm_data/DATA_V17_14400 Data[25]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="10468">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="10484">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="10499">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... ... "03:UFI"} <Rx time="11903">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="11919">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="11950">non_ecm_data/DATA_V17_14400 Data[32]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ... "03:UFI"} <Rx time="11966">non_ecm_data_sig_end/DATA_V17_14400 Data[16]=00 00 00 00 00 00 00 00 00 00 00 00 00 00 </Rx> "03:UFI"} <Rx time="11966">no_sig/IND_NO_SIGNAL</Rx> "03:UFI"} <Tx time="12168">preamble/IND_V21_PREAMBLE</Tx> "03:UFI"} <Tx time="13182">T30Cfr/DATA_V21 Data[3]=ff c8 21 </Tx> "03:UFI"} <Tx time="13370">hdlc_sig_end/DATA_V21</Tx> "03:UFI"} <Rx time="14243">training/IND_V17_14400_SHORT_TRAINING</Rx> "03:UFI"} <Rx time="14680">T30Fcd/DATA_V17_14400 Data[260]=ff c0 60 00 ff 96 df 73 b1 56 66 c8 86 78 52 9d 99 b3 4c ..