Automated quality control tends to lower the amount of human operator input needed to draw conclusions on the reproduction of known quality glitches.
That doesn’t mean manual testing isn’t involved. In order to do automated decision making, through training is needed. And this can only be done by providing a plethora of examples, so that the Automated Quality Control (AQC) system can learn.
In the case of jv16 PowerTools, the AQC learns from ‘Backups’ samples. This is a folder written by default in the product’s installation path.
It contains various logs that help us determine the product’s use case scenarios. This is useful when attempting to reproduce reported issues or to improve performance.
We hope you understand that we need to take the necessary time at the drawing board, where we still spend time figuring out how all this hard work can translate into something better and more meaningful for the PC power user universe.
Remaining true to our vision of Project jv16 PowerTools’ future in connection with the Windows operating system is essential.
Glad to let you know that important progress is being made! The Automated Quality Control tool in this example shows the quality rating scores of previous and current jv16 PowerTools versions.
The quality rating is calculated by the AQC from the generated debug log files of each jv16 PowerTools installation we have tested. The higher the quality rating, the less issues are found.
The “Fatal AQC failures” indicate any kind of error or mistake found from the logs. For example, if the system detects that jv16 PowerTools did not correctly detect an installed software from the testing environment, that would be counted as one “AQC failure”, etc.
Or, if the Initial Setup had taken too long to run, the jv16 PowerTools became unresponsive during the run or if Clean and SpeedUp My computer had taken too long to complete its scan, etc.
There are over 100 different test points which are tested automatically by the AQC system.
Since the release of the previous jv16 version last month, we have done 353 manual installations of jv16 PowerTools to different testing environments to confirm the software works as expected.
With the help of users just like you who decided to purchase a license and help us to keep the lights on and the development of jv16 PowerTools going on, since the release of previous jv16 PowerTools version last month, we have bought 11 computers for testing.
These computers have a varied range of hardware, starting from Intel Atom based Windows 10 tablets and going up to a gaming laptop with dedicated NVidia Geforce GPU.
|Test setup name||CPU||RAM||GPU||Storage||Windows||Language||Runs Vms|
|Acer||Intel Core i7 3632QM||16 GB||Intel HD Graphics 4000||128 SSD + 1 TB||Win 10 Pro 64b||Eng||Yes|
|Chuwi||Intel Celeron N3350||4 GB||Intel HD Graphics||64 SSD||Win 10 Home 64b||Eng||No|
|HP||Intel Core i5 7200U||4 GB||ATI AMD Radeon R5||500 GB||Win 10 Pro 64b||Eng||No|
|Lenovo||AMD K15||4 GB||ATI AMD Radeon R3||1 TB||Win 10 Home 64b||Eng||No|
|LenovoAtom||Intel Atom x5||4 GB||Intel HD||64 SSD||Win 10 Home 64b||Eng||No|
|LenovoGold||AMD Ryzen 5||8 GB||ATI AMD Radeon Vega 8||256 SSD||Win 10 Home 64b||Eng||Yes|
|LenovoPentium||Intel Pentium||3 GB||Intel HD||300 GB||Win 7 Ultimate 32-bit SP1||Thai||No|
|SamsungAtom||Intel Atom||1 GB||Intel Graphics||500 GB||Win 7 Ultimate 32-bit SP1||Thai||No|
|Toshiba||AMD Turion II P520||4 GB||ATI Mobility Radeon HD 4500||500 GB||Win 7 Ultimate 64-bit SP1||Thai||No|
|HP Green||Intel Core i5-8300H||8 GB||NVIDIA GeForce GTX 1050||1 TB||Win 10 Home 64b||Thai||Yes|
|Coolermaster PC||Intel Core i7 4771||16 GB||Intel Graphics||128 SSD + 1 TB||Win 10 Pro 64b||Eng||Yes|
On top of testing jv16 PowerTools on every ‘real’ machine described in our testing inventory, a wide range of Oracle VirtualBox based virtual machines have been deployed.
Virtual machines and used in our test lab in an attempt to obtain even more ‘Backups’ data as a result of specific bug / use case scenario reproduction attempts. Such as the tests we have done with our users as part of a private test build on the forum.
Thanks to this testing data, we have been able to produce the following results:
The above screenshot shows the quality comparison view.
It compares how many issues and errors the AQC finds from each PowerTools version. This is a handy tool for comparing quality between versions, and for example to confirm that newer version contains less quality issues than the last version so every version is better than previous.
The Quality Score number is calculated by dividing the number of AQC detected issues with the number of test runs. So that number takes into account that if we have tested some version more than other versions, it is expected that more issues are also found.
The Quality Score is a balanced number between 0 and 10 which indicates quality, ten being the highest possible score meaning zero issues found.
By comparison of previous version Automated Quality Control results, version 184.108.40.2062 has been a public version release candidate in which we have been confident enough that it represents a clear improvement from the most recent previous versions.
This method of quality control is based on test data. If you wish to help us in this direction and you notice something worthy of our attention in what concerns the development of jv16 PowerTools then please submit a ticket.
We will be glad to compare with our own results!
This instance of the Automated Quality Control shows a comparison data between different jv16 PowerTools versions how long the Initial Setup took to ran (in seconds) and how long the Clean and SpeedUp My Computer scan took to run (in seconds).
This is a handy tool to see a new version is not greatly slower than the previous version.
It also shows the speed change in per cent. The comparison also shows the average number of registry errors found by the CSMC.
This metric is used to ensure there is no major increase or decrease in the number of errors found between versions, as that could indicate a problem. This is of course only one such metric, the normal AQC scan analyzes the results of each Clean and SpeedUp My Computer and lists any issues it finds as well.
The way that this tool has been designed helps us to go through a large amount of testing data, without having to manually parse each log. We are currently monitoring progress from version to version and believe in a gradual improvement process.
In this example the Automatic Quality Control system shows the base view of the tool. In this view, we are seeing the report of AQC per one jv16 PowerTools version. The AQC basically lists all things it finds from the PT installation’s Backups data log files which indicate an error state or a crash of some sort.
Typically these error states are not critical errors, for example, case of some software is installed to a computer but jv16 PowerTools’ Software Uninstaller does not detect the software installation. In this case, this is clearly an error, and detected by AQC, but not a critical issue.
That is the only way we will be able to continue creating memorable software that ensures you – our ineffable users – are satisfied and make us proud as developers.
Project jv16 PowerTools continues to evolve and we remain committed to delivering a high-quality user-based experience to Windows operating system fans, while exploring new and exciting opportunities to integrate two new features: common browsers history cleaning and the ability to clean cookies selectively, with the help of a white-listing feature.
Macecraft is in no rush to release Project jv16 PowerTools NMV until we’re satisfied it provides a rock-solid user experience and visuals to match.