ICYMI: Does Microsoft Actually Listen to Its Unpaid Windows 10 Testing Community?

This piece was originally posted earlier in October. 

During the weekend, we reported a data loss problem that some of the early adopters of Windows 10 October 2018 Update have been experiencing. The initial data loss issue that pushed Microsoft to put the release process to a pause was data deletion during the installation process. This latest issue has to do with the built-in zip tool that is apparently missing the “Do you want to replace these files” dialog, which appears during conflict situations when copying from an archive to a folder that already has a file with an identical name in it.

microsoft-xbox-controller-01-headerRelated Microsoft Reportedly Making an Xbox Controller for Phones – xCloud is Coming

While Microsoft is yet to respond to our queries on this particular issue, it appears that the problem was already reported by the company’s Insider community without warranting any action from the Windows maker – exactly what had happened with the data deletion issue, as well.

The continuous stream of faults and bugs brings focus to Microsoft’s Windows 10 development and testing process

Since the hasty release of the October 2018 Update earlier this month when Microsoft decided to bypass the Release Preview ring and opened the floodgates for everyone who wanted to install the latest version of the OS, many have questioned the company’s strategy of issuing and developing Windows 10 feature updates.

In a detailed piece on the Windows 10 development strategy, ArsTechnica wrote that the “fundamental flaw” with the development process is destabilizing the “codebase by integrating inadequately tested features, and then hoping to fix up all the problems later.” The publication added that this system of adding new features first and then fixing them all at the end “wasn’t good when Windows was released every three years, and it’s not good when it’s released every six months.”

windows-phone-internalsRelated Windows Phone Internal Source Code Has Been Published on GitHub

In the last few weeks, many have questioned the frequency of updates being sent to the operating system. However, even if the company does release fewer updates but doesn’t have a strong internal testing process that isn’t primarily run by volunteers who can’t even test these builds on their daily drivers, the frequency of these updates won’t matter. They would still be buggy and we would still be warning against installing a new update fresh off the boat out of compatibility concerns, if not due to problems like data loss issues that didn’t have to do with any external drivers.

As the debate on Microsoft’s approach to developing and testing Windows 10 continues, the company announced a new feature in its Feedback Hub to enable testers to assign a severity rating instead of relying on the “upvote” system to get Microsoft’s attention. However, out of the millions of Windows users that the company says volunteer for the Insider program, how many would be able to assign the correct severity rating to an issue and understand its possible effects on other system components and performance.

Even if everyone did assign the right severity, it still begs the question – does Microsoft actually listen to and trust its unpaid testers?

In a report, ComputerWorld suggests that up to 1,500 Insiders may have had their files deleted during testing the latest version of Windows 10, without triggering any reaction from the Windows maker. The publication estimates this number based on Microsoft’s own figure of “one one-hundredth of one percent of version 1809 installs” having been affected by the data loss issue.

While this figure may not be accurate, it does discount the value of unpaid testers. If a few hundreds of Insiders experienced the same issue and the Windows maker couldn’t detect and fix this issue before the public release, can it really expect a little severity tag to completely change the entire testing process?

The prospect of being part of the early development of a new version of Windows 10 is exciting for a Windows enthusiast. This program and the launch of Windows as a Service has helped Microsoft get into the modern era of consistently updated operating systems and introduce millions of its users to exciting new features. The Insider program is undoubtedly valuable for the company to see what features attract the most attention, to involve the community, and even to start working on new ideas being proposed by this community. However, it certainly shouldn’t be used to replace an entire testing infrastructure on which relies the stability of the world’s (soon to be) most-used desktop operating system.

With all that said, Microsoft isn’t the only company dealing with these software stability issues. Apple had a notoriously sloppy few months last year until the company decided to focus on performance over new features. Considering the sheer number of devices produced by different manufacturers that run Windows, the scale is obviously different.

However, until the approach to detecting and fixing these issues isn’t changed and Microsoft doesn’t start rigorously testing the new updates throughout their development process, these stability and compatibility problems will continue to push consumers to simply lose trust over the update system. If that happens, Windows 10 could potentially get into the bigger mess of fragmentation that Android deals with – something that Microsoft would want to avoid at all cost.