Document: Application Note for Witness Systems
Total Page：16
File Type：pdf, Size：1020Kb
Document: Application Note for Witness Systems Product: Witness Systems IP Recording Suite Call Manager V 3.3 CallManager Revision: 2
Date: October 15, 2003
Author: Maxine Dye
Application Note for Witness Systems Contact Store IP
Table of Contents
1 General Information...... 3 2 Main Contact Information...... 3 3 Witness Contacts...... 3 4 Witness Support Contact Information...... 4 5 Witness Support Detail...... 4 5.1 General Information...... 4 5.2 Severity Description and Response Times...... 5 5.3 Support Level Definitions...... 5 5.4 Call Flow Description...... 6 6 Witness Support Escalation Procedures...... 7 7 Witness Product Information...... 8 8 Witness Product Overview...... 8 9 Witness Product Features...... 8 10 Witness Product Performance (as tested)...... 8 11 Witness Product Security Considerations...... 10 12 Packaging, Bundling, Upgrading...... 10 13 Overview of Integrated Solution (as tested)...... 10 14 Network Designs/Topologies (as tested)...... 11 14.1 Sample network designs/topologies...... 11 14.1.1 Simple Topologies...... 11 14.1.2 Single Site, Single Gateway and E1/T1 Trunk...... 12 14.1.3 Single Site, Dual E1/T1 Trunks...... 12 14.1.4 Hub and Satellite Sites...... 13 14.1.5 More Complex Scenarios...... 14 15 Witness Product Requirements...... 15 16 Witness Product Installation Procedures (as tested)...... 16 17 Witness Product Isolation Procedures...... 16 18 General Witness Product Considerations...... 16 18.1 Limitations of this release...... 16 18.2 If you alter your server...... 17 18.3 If you alter your network...... 17 18.4 If you alter your IP telephony system...... 18 19 Troubleshooting...... 19 19.1 Debugging and Diagnostics...... 19 19.2 Error Messages...... 20 19.3 Tools / Aids...... 22 19.4 List of Commands...... 24 20 Redundancy and Recovery...... 25 20.1 Redundancy...... 25 20.2 Recovery...... 25 20.3 Dual Spanning...... 25 21 Interoperability Test Results...... 25 22 Cisco Product Information (as tested)...... 25 23 Cisco Product Interfaces (as tested)...... 26 24 Cisco Product Requirements (as tested)...... 26 25 CallManager Device Table...... 26 Appendix A...... 27 Glossary of Terms:
API Application Program Interface A set of routines used by an application program to direct the performance of procedures by the computer's operating system. AVVID Architecture for Voice, Video and Integrated Data.
CRM Customer Relationship Management CTI The practice of using a computer to control one or more telephone and communications functions. IVR Interactive Voice Response A computer that operates through the telephone system, in which input commands and data are transmitted to the computer as spoken words and numbers, or tones and dial pulses generated by a telephone instrument; and output instructions and data are received from the computer as prerecorded or synthesized speech. IIS Internet Information Server Microsoft's brand of Web server software, utilizing HTTP (Hypertext Transfer Protocol) to deliver World Wide Web documents. It incorporates various functions for security, allows for CGI (Common Gateway Interfaces) programs, and also provides for Gopher and FTP (File Transfer Protocol) services. IP Internet Protocol LAN Local Area Network PSTN Public Switched Telephone Network Public telephone system. RAID Redundant Array of Independent Disks A data storage method in which data, along with information used for error correction, such as parity bits or Hamming codes, is distributed among two or more hard disk drives in order to improve performance and reliability. The hard disk array is governed by array management software and a disk controller, which handles the error correction. RTP Real-Time Protocol An Internet-standard network transport protocol used in delivering real- time data, including audio and video. SCCP Skinny Client Control Protocol A Cisco protocol used for communication between a gateway and Cisco CallManager. SPAN Switch Port Analyser.
1 VLAN Virtual Local Area Network A local area network consisting of groups of hosts that are on physically different segments but that communicate as though they were on the same wire. VoIP Voice over Internet Protocol The use of the Internet Protocol (IP) for transmitting voice communications. VoIP delivers digitised audio in packet form and can be used for transmitting over intranets, extranets, and the Internet.
2 1 General Information
Witness Systems provides a business-driven multimedia recording, performance analysis and e-learning management solution that enables companies to: Enhance the quality of their customer interactions Achieve greater customer retention and loyalty Develop staff Generate additional revenue
2 Main Contact Information
Main Location Additional Location Additional Location Address Corporate Headquarters European Headquarters Asia Pacific Headquarters 300 Colonial Center Kings Court, Kingston Room 1809-1818, 18F Parkway Road Shui On Centre Roswell, GA 30076 Leatherhead 6-8 Harbour Road USA Surrey, KT22 7SL Wan Chai Hong Kong Main 1-800-494-8637 +44 (0) 1372 869 000 +852 3198 6000 Telephone Main Fax 7707541888 +44 (0) 1372 869 000 +852 3198 6100 General E-mail [email protected]
3 Witness Contacts EMEA Name Phone Email USA 1 800 494 8637 [email protected] Leatherhead 0800 496 1111 [email protected] Asia Pacific: [email protected] ANZ (within 1 300 367 394 [email protected] Australia) (elsewhere) 61 2 9232 5833 [email protected] RoAP (rest of +852 8103 0104 [email protected] APAC)
3 4 Witness Support Contact Information
Team Main Location (USA) Europe – UK Asia Pacific Phone 1 800 494 8637 0800 093 1835 (out of 1 300 367 394 (Aust) hours – for customers 61 2 9232 5833 (ANZ) with out of hours +852 8103 0104 (rest contracts) of Asia Pacific) 0800 496 1111 (7:00 – During business hours 19:00) Out of hours pager service available Fax 770 754 1888 01372 869 005 N/A E-mail [email protected] [email protected] N/A Hours Phone support is 24 N/A hours a day from 3:00pm Sunday – 8:00 pm Friday (excluding holidays); Off hours (emergency based support) is 8:00 pm Friday to 3:00 Sunday.
The IPTelephony category requires 24x7 support contact. List all applicable after hours numbers.
5 Witness Support Detail
5.1 General Information Witness Systems will respond to customer telephone requests for assistance in the order in which calls are received. For Standard Support telephone requests, Witness Systems’ intent is to answer its customer phone inquiries directly. If all Witness Systems CSR are working with other customers at the time of a request, the customer calling for assistance can expect to receive a return call from a Witness Systems CSR within 2 hours of the original call to the company. A call directed to the Hotline number outside standard support hours for critical issues will receive immediate attention and a return phone call within 30 minutes.
In addition, Witness Systems also offer on-line support where customers can log in using unique log in identification details to report and monitor progress of issue resolution.
4 5.2 Severity Description and Response Times
Priority/SeverityDescription Response Time Critical Current WITNESS® performance prohibits the Immediate assignment of service customer from conducting a critical business resources to research and correct the function, thereby creating an “emergency” situation problem as soon as possible. With progress updates every 4 hours High Current WITNESS® performance has significant 5 business days. impact on a business function, but the customer's With progress updates every day. business is still functioning. Medium Current WITNESS® performance has a moderate Workaround in 15 business days and / impact on a business function, but the customer’s or Next Service Pack business is still functioning. With progress update every 5 business days. Low Current WITNESS® performance has a minor Incorporated into next release activities. impact on the customer’s business operations. With What’s New in release documentation. 5.3 Support Level Definitions
Level Description Bronze 09:00 – 17:00 Monday to Friday inclusive but excluding all Bank and Public Holidays. Response time = 4 hours; On Site Support = 8 hours. Silver 09:00 - 17:00 Monday to Friday inclusive but excluding all Bank and Public Holidays. Response time = 2 hours; On Site Support = 4 hours. Gold 07:00 - 21:00 Monday to Friday inclusive but excluding all Bank and Public Holidays. Response time = 2 hours; On Site Support = 4 hours. Platinum 00:00 – 24:00 Monday to Sunday inclusive of all Bank and Public Holidays Betw 07:00 –21:00 5 days Response time = 2 hours; On Site Support = 4 hours. Betw 21:00 – 07:00 and 24 hr weekends Response time = 2 hours; On Site Support = 6 hours
5 5.4 Call Flow Description
Once 'New Incident' is accessed, the incident must Initial Response be worked. It cannot be Queue/Pool Web placed back into the 'New Incident' queue.
Customer Communication Inbound incident logged by call required to remove from handler/Tier 1/Initial Response and Initial Response/Pool Web placed in 'Initial Response and Resolution Pool'
Initial Response and Resolution (Tier 1)
Ready for Closure Closed
Field Engineering Adtech Queue Queue
Technical Analysis Incidents routed Technical Analysis by Manager to (Tier 2) Individuals
Technical Analysis Team
Engineering Support (Tier 3)
6 6 Witness Support Escalation Procedures
Elapsed Time Priority 1 Priority 2 Priority 3 Priority 4 Urgent High Medium Low 1-Hour CIC Manager CIC Manager CIC Manager CIC Manager CIC Director/ES CIC Manager CIC Manager 4-Hour CIC Manager Manager CIC Director/ES CIC Director/ES CIC Manager 24-Hours SVP Global services Manager Manager SVP Global CIC Director/ES CIC Director/ES 48-Hours SVP services Manager Manager SVP Global CIC Director/ES CIC Director/ES 72-Hours SVP services Manager Manager CIC Director/ES 96-Hours SVP SVP SVP Global services Manager
For USA Escalation Contacts Title NamePhone Number Customer Interaction Centre N/A 1 800 494 8637 Customer Interaction Centre ManagerN/A 0770 7548644 Director Customer Interaction Centre N/A 678 254 5363 +44 (0) 1372 869 401 SVP Global Services N/A - or - +44 (0) 7767 791 599
For EMEA Escalation Contacts Title NamePhone Number +44 (0) 1372 869518 Customer Interaction Centre ManagerN/A - or - +44 (0) 7879 607 123 Engineering Support Manger N/A +44 (0) 7767 791 581 Director EMEA CIC N/A +44 (0) 7770 422 788 +44 (0) 1372 869 401 SVP Global Services N/A - or - +44 (0) 7767 791 599 +44 (0) 1372 869 445 SVP EMEA N/A - or - +44 (0) 7968 486 882
7 For Asia Pacific Escalation Contacts Title Name Phone Number ANZ ProfSManager Terry Halvey +61 2 8907 0336 RoAP ProfS ManagerRisk Israel +852 3198 6226 ProfS Director Ian Millward +61 2 8907 0330 SVP Nigel Hewett +852 3198 6000 President Nick Discombe+44 1372 869 468
7 Witness Product Information
8 Witness Product Overview The Witness Systems Contact Store IP architecture provides 3 levels of products to suit the different needs of Cisco customers to record and live monitor their IP telephony conversations. These products are differentiated by functionality and the number of simultaneous recordings they can handle. They are aimed at meeting different business requirements of different user/customer groups. The following table lists the products and related customer groups.
PRODUCT CUSTOMER eQuality ContactStore CEO’s, VPs – record an important call only. Exec eQuality ContactStore As above, aimed at small call centre and bank level. IP Micro eQuality ContactStore As above, with CTI and CRM integration with multi-site IP support and application integration.
eQuality eWare and eQuality Unify are elements to the product suite that are embedded in the product that manage the database and external CTI interfaces respectively. They are not exposed to the customer unless a large enterprise solution is implemented.
9 Witness Product Features See table below
10 Witness Product Performance (as tested) The Witness Systems’ recording suite of software combines scalable platform architecture to provide functionality to customers to suit their needs and budget. The aim of the suite of software is to allow a customer an entry level solution that can be upgraded without the need to replace any element of previously purchased software. The following table shows the different performance capabilities of each level of the suite.
8 Product Capabilities Contact Store Exec Base System has server licence and bundled 2 channels of on- demand recording – 'Contact Store Exec' extension licenses. The system has a maximum capacity of 30 Extensions. Bundled is a single concurrent Eyretel Insight Viewer client license, which cannot be expanded. Multiple user accounts for the single license is possible. No live monitoring is available; this requires an upgrade to Eyretel Observer. No full-time recording is available; this requires an upgrade to Contact Store IP Micro Contact Store IP Micro Capabilities as Contact Store Exec, but also including: Base System has server licence, database application and bundled 1 Eyretel Insight Viewer client license. Support for 1-30 channels of full time recording or Contact Store Exec Multiple Contact Store IP Micro recorders cannot not exist in the same network, upgrade to Contact Store IP is needed Single Eyretel Insight Viewer client license is expandable to 5 concurrent licenses Any of the recorded channels can be live monitored with the purchase of an Observer client license. The maximum live monitoring of conversations with Observer is 5. Contact Store IP As Contact Store IP Micro, but also including: Base System has server licence, database application and bundled 5 concurrent Viewer client licenses. Support for SQL Server 2000 Enterprise Database Support for Multiple Recorders forming an enterprise Recording network Support for 1-60 channels of full time recording per server Support for Replay Studio cal analysis solutions Support for QualityCall Call Evaluation suite Support for Performance Manager Support for 3600 Busy Hour Call Attempts (BHCA) Each recorded channel can be configured as a full-time recording or Contact Store Exec channel Any of the recorded channels can be live monitored with the purchase of an Eyretel Observer client license. The maximum live monitoring of conversations with Observer is 5. Minimum machine specification is 2.4 Ghz pentium 4 processor, 1 Gb RAM; 250 Gb HD. Consolidation time of 60 calls a minute. Recording size of 8 Kb per second.
9 11 Witness Product Security Considerations Witness Systems’ recorder system utilizes Windows 2000 Server and as such there is the scope for an organization to undertake their ‘usual lockdown’ procedure for servers. The scope and the level to which an organization secures their server(s) is varied and at the discretion of that organization.
The main general area for securing the system lies in secure access to replay calls.
The users associated with particular extensions are set in the Administrator console and the usernames themselves are derived from one of 2 scenarios: a) The Windows 2000 Local User accounts of the IP Recorder or b) The Domain/Active Directory users of the Domain the system is hosted on.
Whether the system uses local users or domain users is normally decided by the size of the system and organization. With 1 or 2 users needing to replay in an entire system it is preferable to use standalone users to simplify the install, whereas more than 5 users would mandate integration to existing domain users.
Achieving the different levels of security is purely an Internet Information Server setting.
12 Packaging, Bundling, Upgrading The 3 levels of product are each licensed for specific capabilities and also maximum number of concurrent recordings achievable.
Additionally the product is capable of being licensed on a timed-trial basis, i.e. 5-days, where at the end of the trial the customer has the option of buying a full license key to continue the use of the system in a non-destructive upgrade.
13 Overview of Integrated Solution (as tested) At the highest level, the Witness Systems’ recording system consists of three conceptual sub-systems: The Capture subsystem, which captures information in real time. The Storage subsystem, which compresses and retains these recordings for use later. The Exploitation subsystem, which lets you exploit the recordings in your business. These are shown in Figure 2-1 along with the major data flows through the system. In many cases, these three components are built into a single PC often just referred to as a ‘recorder’ but in larger systems, the three components may be split across your entire enterprise.
10 CAPTURE EXPLOIT
STORE
Figure 2-1 Witness Systems Generic Recording System
The most effective way to ensure normal operation of each of these subcomponents is to test the ‘exploit’ section, as for this to work, the other two must by definition be working.
To do this, simply record a call and attempt to replay that call and check the audio quality.
Contact Store IP performs replay via a web-browser based application (Viewer) that can be used from Internet Explorer V5.5 (Media Player 7.x).
14 Network Designs/Topologies (as tested) 14.1 Sample network designs/topologies
If the Cisco CallManager topology and recording requirements fall within any of the standard scenarios described in Section14.2.1, configuration is straightforward and the options are detailed here.
If the required system does not match one of these simple configurations, you will need to review the information described below to ensure that an appropriate topology and system sizing is determined. This is generally isolated to larger telephony sites where the permutations and combinations are too varied for the scope of this document.
Contact Store IP is licensed according to the maximum number of concurrent extension recordings that the recorder will support so, in all cases, you must determine how many concurrent extension recordings are required.
14.1.1Simple Topologies The following commonly met scenarios are easily met by the configurations and topologies described in the following paragraphs:
11 14.1.2Single Site, Single Gateway and E1/T1 Trunk
Figure 1-1 Single Site, Single Gateway and E1/T1 Trunk The system matches this category if: Only calls originating or terminating in the PSTN are recorded. There is a single Gateway with a single T1 or E1 trunk over which all such calls are routed. The CallManager is connected to the same Catalyst switch as the Gateway. IP phones may be connected to this or other switches via routers. The Contact Store IP recorder is connected to a port on the same Catalyst switch as the Gateway and CallManager. All desks to be recorded use IP phones as opposed to SoftPhones.
If the system meets these criteria: Configure the port to which the recorder is connected to SPAN both the CallManager’s port and the Gateway’s port. Select a PC of specification at least equal to the ‘Minimum recommended’ entry in Witness Systems’ IP Pre-requisites document.
14.1.3Single Site, Dual E1/T1 Trunks
Figure 1-2 Single Site, Dual E1/T1 Trunks
12 If the required system meets all the criteria of paragraph 14.1.2, except that there are 2 E1 or T1 trunks connecting the Gateway to the PSTN, adopt the same configuration as above but ensure the PC specification is at least equal to the ‘Preferred’ entry in IP Pre- requisites document.
14.1.4Hub and Satellite Sites
Figure 1-3 Hub and Satellite Sites The system matches this category if: Only calls originating or terminating in the PSTN are recorded. A single Router links the Satellite site to the central hub site. IP phones are to be recorded at one or more satellite sites. The Contact Store IP recorder is connected to a port on the same Catalyst switch as the Router to the Satellite site. All desks to be recorded use IP phones as opposed to SoftPhones. The link to the remote site is 8Mbps or less. No more than 60 concurrent calls will be carried over the link between central and satellite site.
If your system meets these criteria: Configure the port to which the recorder is connected, to SPAN the port through which all traffic to the satellite site travels. Select a PC of specification at least equal to the Preferred entry in the Witness Systems IP pre-requisites document. Size the disk storage accordingly.
13 Where there are multiple small satellite sites, a single recorder may be set to SPAN multiple sites provided the total bandwidth does not exceed that stated above.
In sizing this topology you need to determine the maximum concurrent number of external calls that can be placed from the remote site. This is often determined by the bandwidth of the link to that site and is typically very much lower than the total number of IP phones at that site unless the site is being operated as a Call Centre, in which case up to 100% load must be catered for.
If a single router is used both to route traffic to the remote site and as a PSTN gateway, then calls made from that site that go via the gateway will not be visible to the central switch. In such cases, you must reconfigure your network to ensure that all calls are transmitted via the port being SPANned.
14.1.5More Complex Scenarios If the topology and requirements do not meet one of the more common scenarios described, you will need to determine the system sizing. You should determine the information detailed in the paragraph below:
Recording First determine whether you need to record: Internal calls – within the CallManager system. External PSTN calls – between one party on the CallManager system and a distant party on the far side of a gateway. External IP call – between one party on the CallManager system and a distant party on the far side of a firewall and ISP up-link.
For each of the above call types determine which transmission types (G.711 or G.729A) are to be used and in what proportion. Moving over to G.729A entirely will require much less processor power on the recorder.
You should also determine how many concurrent calls of each of the above types will be occurring and, of these, how many must be recorded.
Call Type % % Max concurrent Max concurrent G.711 G.729A Calls Recordings Internal External PSTN External IP
14 15 Witness Product Requirements To install eQuality Contact Store IP you will need the following: A working Cisco CallManager system (Version 3.3) with at least one pair of IP phones. A non-switching Ethernet hub or a switched hub or router that supports Switch Port Analyzer (SPAN). Ethernet drop lead for the computer that is to be used as a recorder. A high speed computer with Windows 2000 Server installed. A single processor is required, due to limitations of Windows 2000, whereby the packet driver cannot run with more than one processor. Windows 2000 Service Pack 3. Java Development Kit (version 1.2) installed on the pc where you are installing eQuality ContactStore IP. If eQuality Viewer is to be installed, you will need to install Microsoft Message Queue and IIS. A PCI Ethernet Network Interface Card (NIC). Windows 2000 drivers for the Ethernet Card if not pre-installed. SQL Server 2000 Installation kit. eQuality Contact Store Installation CD. Witness Systems licence code for the required channel count. At least 6 gigabyte (Gbyte) free disk space for installation of the Calls folder. You must be logged in as a local system administrator on the computer on which you are installing this software.
15 LAN configuration For VoIP recording to work, it is essential that the recorder can intercept the call control packets passing between the IP phones and the call managers, and the voice streams passing directly between the two parties engaged in each call. For a Cisco implementation, you are recommended to use the SPAN algorithm to set up a port on the Ethernet switch to receive copies of all the packets needed. Refer to the System Administration Guide, for guidance on topologies. Refer to the Ethernet switch documentation to set this up.
Platform Installation To prepare the computer ready for the installation of the Witness Systems components, the following steps are required: Disable all power-saving features. These could interfere with the recorder’s ability to start recording at any time. Install correct Operating System Windows 2000 Server must be installed and Service Pack 3 must be applied. Windows 2000 upgrades are available as an option on the installation CD. Install and set up Internet Information Server (IIS) Ensure that IIS has been installed as part of your Windows 2000 Server installation and that all necessary IP configuration has been undertaken to allow the PC to act as a web-server. Install Message Queuing Service These services can be installed by the Message Queuing Service Windows Component Install Ethernet Card and Driver Install the Ethernet card, checking for resource conflicts. Prove connectivity over TCP/IP to the outside world. Ensure the PC has a valid IP address (can be statically assigned or via DHCP).
16 Witness Product Installation Procedures (as tested) Please refer to eQuality ContactStore IP Installation Guide document
17 Witness Product Isolation Procedures
18 General Witness Product Considerations 18.1 Limitations of this release Please read the following limitations before submitting problem reports or enhancement requests. Confirm that the PC you are using is capable of sustaining recording under peak load conditions. Guidelines are given earlier but PCs can vary considerably in performance and Witness Systems recommend that you confirm loading figures on your system.
16 Liability of Witness Systems for failure to record any calls is limited under the terms of supply. A limited range of PC configurations have been tested to date. Configurations recommended at this time will include significant contingency - which may prove over cautious in the medium-term. A limited range of compression formats are supported. This list will grow and if an as yet unsupported format is to be received or is required as the long term storage format, please contact Witness Systems for the latest position. Configurations above 90 concurrent recordings, that is, requiring multiple recorders or separate database/disk servers must be designed and verified by Witness Systems. It should be noted that only calls between extensions marked for recording will be recorded. Therefore, if a call is transferred from a recording extension to one which is not recorded, recording will cease. For example, when A calls B, who forwards to C, if B only is marked for recording, then the forwarded call (from A to C) is not recorded. If Executive Record is enabled, even though there is no active call, the phone displays the message: Call is being recorded You can clear this message by selecting the Exit button. If you have the options Record all calls and Executive Record enabled, please note that Record all calls will override Executive Record. For example, a call between a marked extension and an executive extension will be recorded. If Executive Record is enabled and a call is being recorded, if another incoming call arrives, causing an interrupt, then the recording is stopped and Executive Record has to be manually enabled again when the original call is retrieved.
18.2 If you alter your server If you make any changes to the setup of your server you should take a ghost image of it to allow it to be reconstructed quickly in the event of major hardware failure.
18.3 If you alter your network Contact Store IP can only record calls whose voice packets are presented to the port(s) connecting it to the network.
The recorder also has a finite capacity, both for filtering required calls out of the incoming packet stream and for the number of calls that it can record concurrently. Any changes you make to your network topology can potentially impact the recorder’s performance and you should follow these guidelines whenever you change your network.
Changes to IP addresses If you change the IP addresses of any components of the system, you should perform a full commissioning test to confirm correct recording of all required calls.
17 Changes to network topology If you change the topology of your network you should check all SPAN port settings and then perform a full commissioning test to confirm correct recording of all required calls.
18.4 If you alter your IP telephony system
Adding new telephones If you add additional telephones you should check SPAN port settings and the appropriateness of existing tap points, adding additional taps as required.
You should then confirm that calls to and from these new phones are recorded correctly. Where fall-back IP-PBXs or clusters are used, you should force the new phones to use each server in turn and confirm that calls are still recorded. It is possible, for example, to have a backup server that is not correctly configured in the required VLAN. Such errors can go unnoticed for long periods until the time when the main server fails and operation switch to the standby server.
You should also confirm that the total loading on the server is still within its capability.
Adding additional gateway capacity Follow the same processes as for additional telephones: Check SPAN settings Confirm recording of calls via these ports Confirm recording under failover scenarios Confirm load is acceptable
In this case, traffic tends to increase gradually after additional capacity is installed, hence you should pay special attention to loading over the following few weeks until it stabilises.
Configuring your hard disk Your hard disk must be configured to have a minimum space of 30,000. You use the System Administration tool to configure this.
P R O C E D U R E 1 To do this, from the System Administration tool, expand the tree, locate and click on the ContactStore Storage Processing node, then in the right pane, check the setting against the Minimum space to maintain entry. 2 Ensure the value is set to 30,000.
There is no deletion setting. When your disk becomes full, it will automatically delete.
18 19 Troubleshooting
19.1 Debugging and Diagnostics Should you experience problems with your Contact Store IP recorder, there is a wide range of diagnostic tools available to help you determine the cause of and ultimately resolve the problem. In summary you have at your disposal:
Several components of the Windows 2000 system management tool set are of use. Most can be accessed via the Microsoft Management Console (MMC) snap-in provided. These include: Windows Task Manager. To see if the appropriate tasks are running. (The following tools have been included in the default console settings of the Administrator tool for your convenience). Windows Service Manager. To allow you to stop and start services manually. Windows Performance Monitor. To observe a wide range of performance and diagnostic counters – this can directly accessed from within the Administrator console under the system monitor branch. Windows Event Log Viewer. To observe any service start/stop/error messages generated by the Contact Store services.
In addition, each Contact Store service outputs more detailed diagnostic information into log files accessible via the Administrator console. You may also choose to run each service from a command-line prompt, giving more immediate, real-time visibility of any major events occurring. Witness Systems’ CTI Studio can be used to observe and modify the behaviour of the Unify service as it communicates with the other components in the system. You can also use all of the SQL Server administration tools to examine the contents of the database as recordings progress.
19 19.2 Error Messages The following are a list of known error messages and issues. eQuality Contact Store service(s) fail Check the Application Event log, see example below, for error messages using the Windows Event Viewer or the MMC snap-in provided.
Shut the service down and run as console application with the –c command-line option. Also check for the log files Capture.log, Compressor.log, Consolidator.log and DiskManager.log in
Solution: Only solution found to date has been to reset the Network node name to match the underlying name.
Calls observed but no audio
Solution: Check the network topology carefully. If using a dumb, non-switching repeater type hub, beware of mixing 10 and 100Mbps ports as this tends to stop RTP packets from being received reliably. If in doubt, plug the recorder PC into the back of an IP phone from which calls are being made. If the recorder is functioning correctly, it will record calls to/from that phone.
Service functions but no performance counters visible
Solution: Confirm that the service is running under the user account EYR_Contact7k. This account should have administrator privileges.
20 You may also find that the SNMP service was running when the installation program tried to install the performance monitor variables. The current installation kit tests for and warns if this service is running but as it relies on the user to stop the service, it may not succeed in installing the performance monitor variables. The only option here is to remove eQuality ContactStore IP and reinstall with SNMP service stopped. The performance counter can also equal zero if you are viewing them through terminal services.
Configuration under Tools/Options fails When setting the eWare details a dialog box stating ‘Error in setting the Eware2 Directory’ is displayed.
Solution: P R O C E D U R E 1 Double check the password entered for User EWCODE in SQL Server. 2 Create the directory C:\Program Files\Eyretel\Eware2\Log Files and try again. 3 Examine the log file produced in this directory.
Cannot search for calls
Solution: Confirm that the databases ‘EwareCalls’ and ’EwareConfig’ are present within the SQL Server.
If call details are visible within the SQL server’s EwareCalls Database and call files are appearing in the correct call path, but cannot replay calls successfully in Vision, try the following:
Search Manager setup Confirm the registry key: HKEY_LOCAL_MACHINE\SOFTWARE\EYRETEL\Search Manager\Config\Path is set to the node name of the PC running the SQL server as the example shown below. Do NOT include ‘\\’ at the start.
Ignore other settings shown above.
21 Missed call, audio out of sync, missing call party identification information
Cause: These types of issues can be caused by the use of anti-virus software. Use of any anti- virus software will cause server performance issues, in particular packet loss.
Solution: We recommend if you need to run anti-virus software, that you use Norton AntiVirus version 7.5 or greater and make the following changes: Run with File System Realtime Protection scanning disabled. Schedule scanning to occur outside of recording hours and avoid full system scans. Exclude .DAT, .TMP, .XML and .WAV files from scans. Exclude the following paths C:\Calls C:\Program Files\Witness Systems\ContactStore\PersistQ C:\Program Files\Witness Systems\ContactStore\ArchiverTemp C:\Program Files\Witness Systems\Unify\PersistQ These 4 paths assume the default installation path, if you set custom paths during the install, you will need to exclude these.
19.3 Tools / Aids
Performance logs and alerts This branch gives you access to the standard Windows Performance logs and events utility. You can use these branches to set up regular monitoring and logging and to alert you to system events and exceptions such as lack of disk space, overload etc.
Counter logs You can use this branch to log periodic samples of performance monitor variables.
Trace logs You can use this branch to log system events such as process creations and deletions.
Alerts Using the Administrator’s System Monitor node, you can observe, via Microsoft’s Performance Monitor, the behaviour of the Contact Store IP system. A default view is provided which shows all the variables that are available. To alert you to incorrect operation or system problems, however, the system uses a range of system alerts set through this branch of the Administrator. A number of triggers are also defined which indicate abnormal performance. You may wish to customise these to suit your needs and/or alter the actions that will be taken
22 should the trigger fire. This can include generating SNMP traps, pager messages, emails etc. Refer to the help function on Microsoft Management Console for instructions on how to achieve these. Most properties can be set by selecting one of the alerts in the right- hand pane and right-clicking it. Select Properties to adjust the threshold and/or actions to be taken when the alert is triggered. For example, you can select to have alerts pushed to a different PC from the server if you wish to track problems from your desktop PC.
A L A R M M E S S A G E P R O B L E M A C T I O N
Failed to Compress Compressor unable to Check .wav files are being produced, compress audio files disk is not full.
Failed to Consolidate Consolidator unable to Check .xml files are being produced; upload call details into check database is running and database. accessible
Disk Space Low Disk Manager cannot Check for other applications using free disk space excessive disk space.
Failed to Delete Disk Manager unable to Reboot to clear any erroneous file locks. delete files Check for read-only files in recording path.
Excessive Memory Buffer Overload causing Increase CPU power or memory or Usage memory buffer to decrease traffic loading. backlog.
Losing Audio Packets More than a few packets Check network loss rates, check for per second being lost overload.
Excessive Packet Rate Too many packets per Reduce packet load through better second being received SPAN settings.
Concurrent Usage Nearing Within 10% of allowed Increase server capacity to full Contact Limit concurrent loading. Store IP.
Archive Media Full DVD-RAM disk is full Change disk
Archive Drive Empty No disk in DVD-RAM Insert and format clean disk in DVD- drive RAM drive.
23 Services This branch gives you access to the standard Windows Service control utility. You can use this to start, stop and observe the status of the various services running on the computer. Of particular interest are the following services: Contact Store IPCapture Contact Store Compressor Contact Store Consolidator Contact Store Archiver Contact Store Disk Manager Unify All of these should be running permanently. On installation, all of these services are set to restart automatically on failure. You may wish to have the system alert you should these services fail. Select the service required, right-click and select Properties on the shortcut menu to change these recovery options.
19.4 List of Commands Instead of running MediaStore.exe as a service, it can be run from the command line. This is particularly helpful when first configuring the system as it gives immediate feedback of calls being recorded in the command window, When run from the command line, IPCapture.exe supports the following options: -c run as console application rather than a service -i install as a service (will fail if already installed). -r de-install as a service -f accept input from a test file rather than the network When run with the -c option, each service will display text indicating when the service is fully initialised and ready. Each of the services will then display the following service specific information when operating correctly:
ContactStore Shows… When… Service Conso Processi Unify instructs it to lidator ng upload call details 24 ContactStore Shows… When… Service me> Compr DONE .WAV file essor compressed and original deleted Disk Entering Files in the Manag X:\yyy\zz specified directory er z\aa\bb are next candidates for deletion 20 Redundancy and Recovery 20.1 Redundancy Utilizing standard Hardware platforms the resilience of the server itself can be dictated by the customer and any Cost/Benefit analysis they derive. Witness Systems strongly recommends that a minimum of RAID1 configured mirrored hard disks be deployed. However for further redundancy it is possible to implement a distributed architecture to the system. Following is a summary of some of the elements that can be added for redundancy: 1. Multiple storage locations – replicate the audio and XML file created for each call to a secondary device – i.e. Off site, SAN etc. 2. Utilize a centralized SQL Server Database. 3. Replicate & confirm backup of the Call details in the SQL 2000 Database – remember that the underlying data for each call is still stored as an XML file and hence can be recovered by Witness Systems’ Professional Service in the event of a total failure. 4. Connect to multiple switches each with the same SPAN algorithm allowing for redundant feeds from the switches (note this would require the use of min. Adaptec DualPort Ethernet card). 20.2 Recovery Witness Systems’ IP Recording suite of products is totally passive to the AVVID environment. Hence a failure of the system and subsequent automatic or manual recovery has NO impact on the AVVID system. 20.3 Dual Spanning To achieve maximum resilience and redundancy in the system, you can use double spanning. 21 Interoperability Test Results 25 22 Cisco Product Information (as tested) Cisco CallManager 3.3. Cisco 7960 phones. 23 Cisco Product Interfaces (as tested) Witness Systems’ Contact Store IP does not interface directly with to the Cisco Product Interfaces. It passively decodes the SCCP messages. 24 Cisco Product Requirements (as tested) The minimum requirements for the system are those needed for a basic Cisco CallManager demo system, providing basic internal call functionality: a working Cisco CallManager system V2.4 or higher at least one pair of IP phones, 3 phones provide the ability to transfer calls etc. 7960, 7940 & 7910 phones are supported Currently Cisco SoftPhone 1.1 is NOT supported. Although Softphone 1.2 is supported. a non-switching Ethernet hub or a switched hub that supports the “RMON SPAN” algorithm. Optionally the Visual Switch Manager web administration tool for configuring Cisco switches 2 Ethernet drop leads for the PC to be used as a recorder – one for SPAN connection of the Voice VLAN and one for Data LAN connection to service Client replay requests. 25 CallManager Device Table NO CallManager Devices need to be configured. 26 APPENDIX A Patch Files Changed Description Number Filename Version CSIP0009 This is a rollup patch for See readme.txt IP please read attached readme. readme(CSIP0009).TXT CVWR0016 EyrInfQueuedCache.dll 7.1.0.134 Replay can be very slow when a UNC calls path is used instead of accessing the Calls share/folder directly for an IP capture platform. This patch does not affect HTTP retrieval. If HTTP interfaces are used they will not be improved CVWR0017 player.htc The top (or near If a user attempts to replay multiple actionflow.htc top) line of the calls quickly and repeatedly the Contact replaymanager.htc player.htc, Viewer itemsetview.htc actionflow.htc, browser application can get into a state replaymanager.htc, in which it does not replay subsequent and calls. itemsetview.htc Sometimes it recovers but the user can files will read as be in the situation where clicking a call below: does CSIP0012 IPCapture.exe This patch improves the performance of 3.2.7.151 the system, and adds extra diagnostic RTPHandler.dll variables. 1.0.0.28 NGAComponent 7.1.0.3 Unify.spt 7.1.0.12 CSIP0014 Observer.exe 7.2.0.26 This patch fixes a number of issues in observer: - When an extension was monitored it would be highlighted in green in the calls view, but when that call was terminated and a new call started on the same extension the call would be monitored but the call would not be highlighted in green. - The stop button would not work and you would continue to hear audio even after pressing stop! - On a machine that has a full eventlog Observer would crash with an error message. 27