Testing The JCOP Framework

Paul Burkimsher October 2010 What is the JCOP Framework? • You should know that by now… ;-) • It’s been around a while but is of course being maintained. – The problem with maintenance is you can fix one bug and introduce another. – Same for upgrading…

• => Need for Regression testing – Does the original bit still work like it did yesterday?

2 Regression testing • Is boring, repeating the same tests every day – so we automate it. • We have taken a very top-down approach. We pretend to be a user clicking on buttons, manipulating the DEN, declaring devices…

• We write Python programs which fire up PVSS panels, press buttons, type in field values…

3 4 How do we do it? • Use a GUI testing tool called “Squish” from a German company. • We chose Squish because ETM chose Squish for their own GUI testing…

5 Squish is powerful • I agree with the salesman  : • It is flexible – You can run Squish on Windows and test • Fw components also running on Windows or • Fw components running remotely on – It supports several programming languages. I chose to use Python (I’m not a Java, or Tcl fan) • It is easy to use – You can record mouse movements and keystrokes to get your initial scripts

6 Powerful, but • Squish itself is being developed intensively – Not ever so stable (presumably they test it :-P ) • PVSS is changing underneath. ETM try very hard to keep the look and feel the same, but their implementation underneath changes  – If the computer clicks on qt_go button and its name changes to qt_start, you’re screwed. • is changing underneath (and PVSS is built on top of Qt) • … a lot of running fast to just stay still 

7 How does it work? • I defined a sequence of Test Suites (Python programs): – suite_createCleanPut (“ProjectUnderTest”) – suite_installFwInstallationTool – suite_installFwComponents – suite_exerciseAnalogDigitalComponent – suite_exercise CaenComponent • tst_createCrateSy127; tst_createCrateSy403 • tst_createCrateSy527; tst_createCrateSy2525 – suite_exerciseWienerComponent – suite_exerciseTrendEditorNavigator

8 How does it work – II ?

N AUT Background (Application Python Script e or t Under Test) w uses Qt Python Script o in IDE r Special qt.dll k get interpreted Squish Runner Squish server by (interpreter - client)

9 • How do you know its worked?

10 • It gets to the end without breaking?

• You can put assertions in the Python code

• You can inspect the output

11 What is the output of the DEN?

– DPs and hence .dpl files

– We compare the produced .dpl file with an “expected” .dpl file.

– NB You have to remove the timestamps and Dp Ids from the .dpls first.

12 How do we communicate results?

• Produce log files at every stage.

• Squish produces an XML summary of the assertion results (and provides an html translator)

• We create an email with this info and send it to all the JCOP authors whose components are being tested

13 Nightly email

14 Email (cont)

15 Further work • More tests • More software version combinations (PVSS 3.9 etc) • Fully automate the test across to Linux • Code coverage in the scripts

16 Questions ?

17 18