Following the official release date, pricing and specs announcement of the Intel i9 9900K CPU, Intel shared several game benchmarks where the processor crushed all competition, including the AMD Ryzen 7 2700X. Some of these benchmarks were published by PCGamesN, but renown YouTube channel Hardware Unboxed quickly pointed out in a video titled Intel’s New Low: Commissioning Misleading Core i9-9900K Benchmarks that the AMD Ryzen 7 2700X CPU had been handicapped by suboptimal memory timings used in the test.
Following these reports, we reached out to Principled Technologies (which conducted the benchmark commissioned by Intel) asking for a comment. We just got a reply from Bill Catchings, co-founder of Principled Technologies, and we’ve posted it below as is.
Related AMD Ryzen Threadripper 2970WX 24 Core and Ryzen Threadripper 2920X 12 Core HEDT Processors Launching on 29th October
Today, we have seen several reports that suggested Principled Technologies (PT) published misleading information in our recent study comparing Intel’s gaming processors to AMD’s. We apologize for our delay in responding, but it’s been a busy day, and we wanted to be as thorough as possible in addressing inquiries concerning our testing. We’ll address specific questions and share more detail on our methodology in a moment, but we first must respond directly to attempts to call our integrity into question.
For almost 16 years, we have tested products for our clients because they trust our integrity. We have worked not just for any one company but for dozens of the leading technology firms, including rivals such as Intel and AMD, Microsoft and Google, Dell and HP, and many others.
Those clients trust PT in part because our integrity and our technical knowledge are beyond reproach. We work hard to be the best in both of those areas. We chose our company name to emphasize our commitments to both technology and our principles. (And, accusers saying we are only in this for the money obviously haven’t read our book, Limit Your Greed!)
Before going further, we thus must categorically deny any dishonesty in our work on this project for Intel or in any of our other projects.
Now that we’ve gotten that off our chests, let’s address the specific questions from recent videos as well as subsequent posts and tweets.
An overview of the project will provide useful context.
Our overall goal – and Intel’s specific request for this project – was to create as level a playing field as possible for comparing the AMD and Intel processors as the majority of the gaming market would likely use them. To do that, we built and configured 16 systems for this comparative testing; we had two of each processor/motherboard configuration. We matched all components where possible, the only variances being the motherboards, CPUs, and CPU coolers. (Full details are in our interimreport.)
In an effort to be very transparent, we published our interim summary report on Oct 8, 2018. We will continue to be transparent and responsive to any questions.
We have received a number of inquiries regarding the testing methodology we used and the potential for bias in favor of Intel. We are providing additional information to be as transparent as possible and to help allay these concerns.
The following list summarizes many of the inquiries we have received and our responses. (We are continuing to work on addressing additional inquiries.)
Use of “Game Mode” on the AMD Ryzen™ 7 2700X: Some inquiries we have received concern the use of the Ryzen utility and the number of active cores in the AMD-based systems. Based on AMD’s recommendations and our initial testing on the Threadripper processors, we found installing the AMD Ryzen Master utility and enabling the Game Mode increased most results. For consistency purposes, we did that for all AMD systems across Threadripper™ and Ryzen™. We are now doing additional testing with the AMD systems in Creator Mode. We will update the report with the new results.
Cooler choice: We chose Noctua for the CPU coolers, due to having almost identical systems in the NH-U14S (Intel) and NH-U14S TR4-SP3 (AMD), which allowed us to maintain a comparable thermal profile. Because we were not performing any overclocking on any configuration, and because AMD has said it was a good cooler, we stuck with the stock AMD Ryzen™ 7 2700X Wraith Prism cooler.
Memory speeds: To have complete parity across all systems, and to allow the Intel® Core™ i9 X-series and AMD Ryzen™ Threadripper™ to fully utilize memory bandwidth, we used 4 16GB DDR4 DIMMs on all configurations. We took the following memory configuration steps:
Intel:
MSI Z390-A Pro motherboard(i9-9900K)
Load Optimized BIOSdefaults
Enabled: Extreme Memory Profile(X.M.P.)
DRAM Frequency set toDDR4-2666
Asus Prime X299-Deluxe motherboard (i9-9900X,i9-9980XE)
Load Optimized BIOSdefaults
Enabled: Extreme Memory Profile(X.M.P.)
Disabled ASUS MultiCore Enhancement to use stock intel multicore settings
DRAM Frequency set toDDR4-2666
Installed Intel® Turbo Boost Maxdriver/utility
o Asus Prime Z370-A (i7-8086K,i7-8700K)
Load Optimized BIOSdefaults
Enabled: Extreme Memory Profile(X.M.P.)
Disabled ASUS MultiCore Enhancement to use stock intel multicore settings
DRAM Frequency set toDDR4-2666
Power saving Performance mode, set toPerformance
AMD:
Asus Prime X399-A (Threadripper™ 2990WX, Threadripper™2950X)
Load Optimized BIOSdefaults
Verify that D.O.C.P is selected for AMD-equivalent memory settings toXMP
Performance Enhancer, set toDefault
Disabled overclockingenhancement
DRAM frequency set toDDR4-2933
Set Core Performance Boost toAuto
Set performance bias toNone
Installed Ryzen Masterutility
Asus Prime X470 Pro (Ryzen™ 72700X)
Load Optimized BIOSdefaults
Verify that D.O.C.P is selected for AMD-equivalent memory settings toXMP
DRAM frequency set toDDR4-2933
Set performance bias toNone
Installed Ryzen Masterutility
Resolution settings: One goal of this study was to test the CPUs and their graphics subsystems, not the GPUs, so we ran the tests at the most common gaming resolution (62.06%), 1920×1080, according to the Steam Hardware Survey: https://store.steampowered.com/hwsurvey/Steam-Hardware-Software- Survey-Welcome-to-Steam. This allowed us to minimize any GPU-based bottlenecks on the rendering pipeline.
Quality settings: We configured all games to use the “High” or equivalent preset, versus “Ultra” or other presets, also to emphasize CPU over GPU performance. In the case where there were only three presets, we chose the top preset.
Clarification of various installation questions: We installed all games using Steam or the Microsoft Store, and fully updated with the latest patches.
Motherboards: Re a Twitter comment from Cyber Cat @0xCats, “Hey @AMD Apparently according to @PrincipledTech @Intel is able to run Ryzen™ Threadripper™ Chips on Z370 and Z390”: Thanks for catching that copy/paste error in our configuration info. We made an error there. The correct processor/motherboard/BIOS version specs for the AMD procs we tested are the following: Ryzen™ 7 2700X/ASUS PRIME X470-PRO/4024 and Threadripper™ 2950X 2990WX/ASUS PRIME X399-A/0807. We apologize for the error and will post a revised version (with changes noted) soon.
Because our goal is always to do the right thing and get the answers right, we are currently doing additional testing. We will share that data and will certainly call out if something is significantly different from what we’ve already published.
We are confident in our test methodology and results. We welcome questions and we are doing our best to respond to questions from our interim report, but doing so takes time. We will add responses if other issues come up.
Thanks for listening.
Here’s our Keith discussing the original report made by Hardware Unboxed.
