• TV
  • MOVIES
  • MUSIC
  • SHOWBIZ
  • SOAPS
  • GAMING
  • TECH
  • FORUMS
  • Follow
    • Follow
    • facebook
    • twitter
    • google+
    • instagram
    • youtube
Hearst Corporation
  • TV
  • MOVIES
  • MUSIC
  • SHOWBIZ
  • SOAPS
  • GAMING
  • TECH
  • FORUMS
Forums
  • Register
  • Login
  • Forums
  • Gadgets
  • Mobile Phones
Samsung caught doping the benchmarks… again (Galaxy Note 3)
<<
<
2 of 3
>>
>
jonner101
02-10-2013
Originally Posted by ACU:
“I would suspect all companies that ship a device to the people that do the benchmarks, ship a benchmark optimised device. If you performed the exact same task on each phone, then I would say yes. Maybe not by 20% though.



The benchmarks are not there to mislead, they are a guide. Its a dick waving contest between manufacturers, and they are all at it. Samsung arent the only one that do it. As I said earlier, only a naive person would think a benchmark reflects real life useage - thus there is no attempt to mislead.

I agree that they should also provide a benchmark for daily use...i.e. what the user will actually experience. Provide both figures if you want, nothing wrong with that.”

The benchmarks are hard coded into the OS. Sorry but how devious and underhand can you get
Stiggles
02-10-2013
Originally Posted by jonner101:
“The benchmarks are hard coded into the OS. Sorry but how devious and underhand can you get”

Eh?

No they aren't! The code simply allows the device to perform at it's absolute maximum while the test is being run. It's not adding anything to the benchmark!
jonner101
02-10-2013
Originally Posted by Stiggles:
“Eh?

No they aren't! The code simply allows the device to perform at it's absolute maximum while the test is being run. It's not adding anything to the benchmark!”

The relevant info is here they are hard coded when the app was rebuilt with a different package name the performance dropped by 20% for the exact same app.


Quote:
“The file we ended up with is called "DVFSHelper.java," and it contains a hard-coded list of every package that is affected by the special CPU boosting mode. According to this file, the function is used exclusively for benchmarks, and it seems to cover all the popular ones. There's Geekbench, Quadrant, Antutu, Linpack, GFXBench, and even some of Samsung's own benchmarks. The two functions applied to this list seem to be "PACKAGES_FOR_BOOST_ALL_ADJUSTMENT," which is no doubt the CPU booster, and "PACKAGES_FOR_LCD_FRAME_RATE_ADJUSTMENT," which makes it sound like the phone is also altering the display frame rate.

The inclusion of GFXBench is surprising given that it shows no unusual idling behavior in System Monitor. Between the inclusion of that and the suspicious "frame rate adjustment" string, it's clear that Samsung is doing something to the GPU as well, though those clock speeds are more difficult to access than the CPU speeds (a method used by AnandTech on the international S 4 no longer works on the Note 3).

The "DVFS" in "DVFSHelper" stands for "Dynamic frequency scaling," also known as CPU throttling, which has many legitimate uses to manage both heat and power draw. This file contains a few special settings for the camera, Gallery, and some other packed-in apps, but nothing like what is in the above section. Benchmarking apps are the only type of app that is systematically called out and boosted.

To see how some other benchmarks are affected, we made "stealth" versions of those, too—the exact same app, just with a different package name. These results back up the Geekbench findings: we're seeing artificial benchmark increases across the board of about 20 percent; Linpack showed a boosted variance of about 50 percent.”

IvanIV
02-10-2013
Samsung switch off processor's power save mode when a programme with a certain package name is running. By "coincidence" they are all benchmark programmes. Testers repackaged a benchmark programme using a name not on the Samsung list and they got different (worse) results, because power save mode kicked in this time.
Philip Wales
02-10-2013
In my world it's called "not playing by the book", while nothing wrong with showing the capabilities of a phone, they should make it perfectly clear that these figures are not attainable in "real world settings" and then publish the actual "real world" figures.
flagpole
02-10-2013
Originally Posted by jonner101:
“This is very devious of Samsung if they have actually hard coded their OS build to look for these benchmarks and basically allow the processor to be over-clocked when running them.

It would be funny if it were actually possible to fry the chip if you kept running one of these benchmarks. Someone should try it and sue their ass off.”

seems like a lot of trouble to go to for a replacement of an already working phone.
Dark 1
02-10-2013
Originally Posted by Stiggles:
“The thread title is incredibly misleading!!

All this does, as has been explained already, is allow the absolute maximum performance of the device to be used for a few mins at a time to show what in theory it's capable of.

Nothing wrong is being done here. I dare say many other companies do this as well.”

So it's just coincidental that the only apps that can tap into this "maximum performance" are known benchmarking apps which serve no purpose other than to provide numbers which can be compared to competitors products?

And it's just coincidental that only Samsung has been caught doing this? Twice!

No, I think the title is appropriate. It's a cynical tactic to mislead consumers into believing their phones are significantly more powerful than those of their direct competitors.

"They are without honour" - Worf
Philip Wales
02-10-2013
^^^ agree it's bloody dodgy and unfair, I suppose to use the car thing again would be for BMW to claim there 3 Series can top 180mph, then actually say well the M3 can.
Dark 1
02-10-2013
Originally Posted by Philip Wales:
“In my world it's called "not playing by the book", while nothing wrong with showing the capabilities of a phone, they should make it perfectly clear that these figures are not attainable in "real world settings" and then publish the actual "real world" figures.”

But Samsung aren't generally providing the benchmarks. That's left to reviewers who won't know that their results have been 'doped'... until now.
Philip Wales
02-10-2013
^^true, but Samsung must run these benchmarks too, what's to stop them handing over a phone with a totally mad processor, get the benchmarks, then change the processor for the full production model, which in a way is what they are doing.
Dark 1
02-10-2013
Originally Posted by flagpole:
“seems like a lot of trouble to go to for a replacement of an already working phone.”

More likely for someone (an American probably) to buy one just so they can have a punt at suing Samsung, claiming they only did so purely based on the benchmarking scores.
flagpole
02-10-2013
Originally Posted by Dark 1:
“More likely for someone (an American probably) to buy one just so they can have a punt at suing Samsung, claiming they only did so purely based on the benchmarking scores.”

they still wouldn't be able to 'sue their ass off'
Philip Wales
02-10-2013
^^^ no but the Americans do love a "class action" as seen with Microsoft and the Xbox 360
ACU
02-10-2013
Originally Posted by Dark 1:
“So it's just coincidental that the only apps that can tap into this "maximum performance" are known benchmarking apps which serve no purpose other than to provide numbers which can be compared to competitors products?

And it's just coincidental that only Samsung has been caught doing this? Twice!

No, I think the title is appropriate. It's a cynical tactic to mislead consumers into believing their phones are significantly more powerful than those of their direct competitors.

"They are without honour" - Worf”

Whats the point in running at full pelt when you are reading tweets? If Samsung made their phones that ran a full pelt all the time, the battery would last around 5 hours and be dead. What good is that to anyone? It seems what Samsung have done, is make the choice at what apps run at full pelt, which is a sensible way of doing things.

Not sure why you think its a Samsung issue, like I said earlier all companies do it, including Apple. Its nothing new. Funny seeing people grab at straws to have a pop at Samsung. The words glass, houses and stones come to mind.

Good luck finding a company with honour. Care to let me know of one? I cant think of one, that makes smartphones.
Philip Wales
02-10-2013
^^ Do Apple fudge benchmarks, I've not seen a 70 page thread on that yet, there must be one if you say they do, every other Apple Thread gets the treatment.
alanwarwic
02-10-2013
You know what the outcome of this seems to be?

It is that the hyped up Snapdragons are more than a bit crap.
We already had Samsung limiting Exynos to allow the Snapdragon and Exynos S4s to work at the same speed.

This amplifies everything even further. Or behold, is it a case of the exact same?
jabbamk1
02-10-2013
Originally Posted by alanwarwic:
“You know what the outcome of this seems to be?

It is that the hyped up Snapdragons are more than a bit crap.
We already had Samsung limiting Exynos to allow the Snapdragon and Exynos S4s to work at the same speed.

This amplifies everything even further. Or behold, is it a case of the exact same?”

What...?

Qualcomm are not crap.
IvanIV
02-10-2013
Samsung want to have their processor and eat it. First with big fanfares they announce a mad 8-core processor, then it turns out that a four is a new eight. Then they say they use special multicore processor that can switch to a power save mode and save battery. There's only a small problem, benchmarks are worse, because waking up the processor from power save mode is not for free. No problem, turn it off for benchmarking. This is not so harmless as some state here. There's enough people who take benchmarks into consideration when choosing their phones. Phones nowadays are mostly very good and one has to decide based on something, why not the benchmarks, especially if Samsung phones are way ahead of the competition. IMO it's quite dodgy.
alanwarwic
02-10-2013
Originally Posted by Philip Wales:
“....I've not seen a 70 page thread on that yet, there must be one if you say they do, every other Apple Thread gets the treatment.”

I found one on "5S show it wipes floor with S4 and HTC One" but simply cannot find the Safari one.

I was thinking of that time when Safari benchmarks were recording an early finish time, thus increasing reported speed.
paulbrock
02-10-2013
if mobile benchmark tests actually did model real world user usage and then samsung came along and spoiled the test then it'd be right to call foul.

But they don't, they never have. they tend to throw excessive amounts of calculations/polygons at the processors, far more than would ever be encountered in normal daily use, and count how long it takes. Given its not real world cases, there's nothing wrong with manufacturers *optimising* their devices for the benchmarks, because that's really all that's happening. There's no secret overclocking going on, or "set the engnes to 110% captain" or "if benchmark then unlock secret superfast mode".
alanwarwic
02-10-2013
Originally Posted by jabbamk1:
“What...?Qualcomm are not crap.”

But wasn't Samsung shuffling benchmarks to make the two S4's look identical speed.

I'm sort of baffled as to why this is not some of the same.
We have the cheating benchmark mantra and have moved away from reasoning.
jonner101
02-10-2013
Can't believe some people think it's ok to hack your OS so it over clocks when all the well known benchmark apps are run.

Has this been done by other companies, well probably but that doesn't make it OK for Samsung to do it.

Clear fraud on Samsung's behalf.
alanwarwic
02-10-2013
I thought it was actual default underlocking which was prevented.

What is the overlock?
sancheeez
02-10-2013
Don't care. I'll be having one regardless.
alanwarwic
02-10-2013
http://en.m.wikipedia.org/wiki/Note_3
Oh look, just like the S4 there is both the Exynos and the Snapdragon chipsets.

Like I said, they are making the Snapdragon look less crap.
Yes, I overplay the word 'crap' but that counteracts the hype of those last generation ARM CPUs.
<<
<
2 of 3
>>
>
VIEW DESKTOP SITE TOP

JOIN US HERE

  • Facebook
  • Twitter

Hearst Corporation

Hearst Corporation

DIGITAL SPY, PART OF THE HEARST UK ENTERTAINMENT NETWORK

© 2015 Hearst Magazines UK is the trading name of the National Magazine Company Ltd, 72 Broadwick Street, London, W1F 9EP. Registered in England 112955. All rights reserved.

  • Terms & Conditions
  • Privacy Policy
  • Cookie Policy
  • Complaints
  • Site Map