------------
Back to the mid 90's when I first automated a test case (TC) object at Ericsson using an in-house tool called Autosis, I didn't really like it. It took me lots of time to write and tried to sync the test object with the AXE due to timeout issue, i.e. frustration. It should take me about 30% of time spent there to execute the test object manually.
I recalled another QA staff automated a similar test object that resulted in a book for that test case document. How could I read or review an automate TC document, which were usually hard to understand or not for human readings (?). It was really scary.
Most of automated TC is involving sending data and capture results to compare with expected data, i.e. send data - catch results - compare data - report TC status. There were no logics behind those like a typical application program. I preferred to write application programs or test tools used by QA staff than writing automate TC. Because I wrote [applications] something useful to sell to customers.
* It seems to me that many companies are going for test automation as a fashion or to show off to their customers.
[1] The goals to automate test objects should be
- Making regression tests faster without costs or with minimum cost
- No resource or minimum resource involved
- Maintenance costs of those tools/apps or automated TC low. If you had to update major portions of a TC automated object at every new features release, that TC object should not be automated.
- Improve the quality or make sure that old/existing functionality worked as before
[2] Tools or methods used in test automation
- Writing programs to debug/verify existing functionality of our applications
- The automate program must be easy to understand and updated in case we have to open and update it for changes in the applications/systems.
- For UI application: I looked at some tools on the Internet, we had to know the position of user’s click. If your app was dynamically changed for Menu, Drop-down menu, button and screen added/resized, new functionality added or removed on the screen --> The above things may change the position of clicks to activate a feature/function, i.e. automated TC must be updated frequently --> not a candidate for automation.
- Synchronizing you application and the tool used: This un-sync thing would cause major problem to sync both app and tool – trial and error to sync them would be very time consuming and no-warranty that it’d work by the next time you run the automated programs.
[3] Automate TC [unit test] using J-Unit or similar tools:
- Your app should be well designed/architected to group classes/modules of the same/similar functionality together in a block.
- One block of functionality will communicate with other blocks via a well-defined interface, i.e. parameters passing.
- Now you can use the tool to send test data/parameters to a block and capture response data and verify/compare expected results. If you are testing a block, of course, you have to plug or assign some values at the other end for simulation of inter-communication between blocks, i.e. focus on functionality of 1 block at a time.
- With many blocks grouped together, you had tested much functionality at the same time with real codes, i.e. not assigned [forecast or expected] values at one end.
--> We don’t want to write/maintain many automated (tool) programs if we had to assign many resources updating test object frequently.
Notes: I recalled that a senior technical staff of Ericsson said "automation is like writing a program to debug another program." If the automated/debug program was faulty, then so was our system. If the automated programs were hard to read or understand, then testers would break their heads to review and update.
* I preferred "end-to-end" or system integration test using user interface like a customer to verify data. Of course we would need to use some test tools in some applications instead of UI.
Back to the mid 90's when I first automated a test case (TC) object at Ericsson using an in-house tool called Autosis, I didn't really like it. It took me lots of time to write and tried to sync the test object with the AXE due to timeout issue, i.e. frustration. It should take me about 30% of time spent there to execute the test object manually.
I recalled another QA staff automated a similar test object that resulted in a book for that test case document. How could I read or review an automate TC document, which were usually hard to understand or not for human readings (?). It was really scary.
Most of automated TC is involving sending data and capture results to compare with expected data, i.e. send data - catch results - compare data - report TC status. There were no logics behind those like a typical application program. I preferred to write application programs or test tools used by QA staff than writing automate TC. Because I wrote [applications] something useful to sell to customers.
* It seems to me that many companies are going for test automation as a fashion or to show off to their customers.
[1] The goals to automate test objects should be
- Making regression tests faster without costs or with minimum cost
- No resource or minimum resource involved
- Maintenance costs of those tools/apps or automated TC low. If you had to update major portions of a TC automated object at every new features release, that TC object should not be automated.
- Improve the quality or make sure that old/existing functionality worked as before
[2] Tools or methods used in test automation
- Writing programs to debug/verify existing functionality of our applications
- The automate program must be easy to understand and updated in case we have to open and update it for changes in the applications/systems.
- For UI application: I looked at some tools on the Internet, we had to know the position of user’s click. If your app was dynamically changed for Menu, Drop-down menu, button and screen added/resized, new functionality added or removed on the screen --> The above things may change the position of clicks to activate a feature/function, i.e. automated TC must be updated frequently --> not a candidate for automation.
- Synchronizing you application and the tool used: This un-sync thing would cause major problem to sync both app and tool – trial and error to sync them would be very time consuming and no-warranty that it’d work by the next time you run the automated programs.
[3] Automate TC [unit test] using J-Unit or similar tools:
- Your app should be well designed/architected to group classes/modules of the same/similar functionality together in a block.
- One block of functionality will communicate with other blocks via a well-defined interface, i.e. parameters passing.
- Now you can use the tool to send test data/parameters to a block and capture response data and verify/compare expected results. If you are testing a block, of course, you have to plug or assign some values at the other end for simulation of inter-communication between blocks, i.e. focus on functionality of 1 block at a time.
- With many blocks grouped together, you had tested much functionality at the same time with real codes, i.e. not assigned [forecast or expected] values at one end.
--> We don’t want to write/maintain many automated (tool) programs if we had to assign many resources updating test object frequently.
Notes: I recalled that a senior technical staff of Ericsson said "automation is like writing a program to debug another program." If the automated/debug program was faulty, then so was our system. If the automated programs were hard to read or understand, then testers would break their heads to review and update.
* I preferred "end-to-end" or system integration test using user interface like a customer to verify data. Of course we would need to use some test tools in some applications instead of UI.
No comments:
Post a Comment