Options
Samsung caught doping the benchmarks… again (Galaxy Note 3)
Dark 1
Posts: 4,088
Forum Member
✭✭✭
I see Samsung's been caught 'doping' it's phones to inflate benchmark scores again.
Their latest Galaxy Note 3 has been found to have that magical speed boost which only kicks in when it detects any benchmarking apps being run. This thereby artificially inflating its benchmark scores around 20% higher than any 'normal' app could attain.
Guess they didn't learn their lesson after being caught with their trousers down doing a similar stunt with the Galaxy S4.
No doubt some will argue benchmarks don't mean anything so this is a none issue. Others however may argue this is a clear attempt to fool unsuspecting consumers who believe bigger numbers = better, which are the actions of a company devoid of integrity. Choose your side.
Read all about it:
arstechnica.com/gadgets/2013/10/galaxy-note-3s-benchmarking-adjustments-inflate-scores-by-up-to-20%
Their latest Galaxy Note 3 has been found to have that magical speed boost which only kicks in when it detects any benchmarking apps being run. This thereby artificially inflating its benchmark scores around 20% higher than any 'normal' app could attain.
Guess they didn't learn their lesson after being caught with their trousers down doing a similar stunt with the Galaxy S4.
No doubt some will argue benchmarks don't mean anything so this is a none issue. Others however may argue this is a clear attempt to fool unsuspecting consumers who believe bigger numbers = better, which are the actions of a company devoid of integrity. Choose your side.
Read all about it:
arstechnica.com/gadgets/2013/10/galaxy-note-3s-benchmarking-adjustments-inflate-scores-by-up-to-20%
0
Comments
Samsung more than likely limited the speed in daily usage to improve overall battery life, which is bound to take a hit if the device is allowed to hit maximum power to cope with heavy workloads.
It's not unlike car manufacturers limiting their cars to a certain speed to maintain optimum safety and usability, but these can be taken off when said car is in a race to allow the full power of the vehicle. The same applies here.
Oooo, that's a rather tenuous analogy I would say.
This speed boost only kicks in for benchmarking. This artificial "maximum power" is unattainable in any sort of real world usage. I see no other reason for it to exist other than to mislead consumers.
Hmmm… wouldn't be surprised if this isn't a class action lawsuit in waiting.
It's deliberately unattainable. These processors on board demand a lot of power, and there's only so big a battery Samsung can put in without making the device too big or bulky for today's standards.
I'm not sure what people want them to do. Benchmarks are designed to show the performance, not day to day usage. Just as a drag race is supposed to prove who is the fastest, not how the car will perform when driving to the supermarket or whatever.
Notice that Samsung don't appear to put these kinds of restrictions on their lower tier devices because they don't have as much processing power, so the speed to battery life ratio is more acceptable.
Some variants of the S4 have as much as 8 processing cores on board, is it that hard to grasp that there may be massive strains on battery life when this processor is at its maximum capability?
Plus with all the issues of these devices heating up to dangerous levels, and without space for on board fans/cooling methods, I'm not surprised Samsung have simply elected to cap the speed of the device in normal usage.
Especially when the Note 3 is actually the best benchmarking phone even without the boosts.
Benchmarks a good way to see how powerful a device is compared to others. But it's real time performance that matters.
Agreed which is why for years iPhones were the phone to get because they were fluid in all things.
If it does not happen in other situations it is a special mode. A modified benchmark with a changed name should be demanding enough to trigger the same behaviour. It does not, so it's likely that a speed is capped for any non-benchmarking apps.
that depends. Benchmark apps are pretty artificial situations, requiring jumps up from 0% cpu to 100% cpu for a minute or two. Is it a speed that would be reached over 10 mins if say an app had a consistently high CPU requirement?
Models shouldn't be penalised for using power-saving CPU modes in daily use, but also having more powerful modes available, even if the higher-powered modes take a little time to 'switch on'.
I agree with what you're saying.
But the article seems to make out that no other app uses the CPU as much as benchmarking apps.
The article could be wrong though and that certain high power profile could be accessed by other apps or the OS.
I suspect this ready to go idling mode is reserved for the selected few benchmark apps and once a work is done a processor is shutting down the cores and switching to low power mode. Although it would make sense to stay with all cores switched on for a while, before switching them off if no work becomes available. It is probably so, but the idling is too long in benchmarks to stay ready without an intervention. So they turned off this one level of regulation of power consumption to get better results.
Thus I think the title of this thread is very misleading.
All this does, as has been explained already, is allow the absolute maximum performance of the device to be used for a few mins at a time to show what in theory it's capable of.
Nothing wrong is being done here. I dare say many other companies do this as well.
You would think this sort of action was even a concern to the companies that produce benchmarks. After all, what are the benchmarks really telling you if the companies continue to do this.
It doesn't reflect the performance of the phone as that performance is never available to the user.
The benchmarks are becoming less credible each time this sort of story comes out.
If a person is naive to think that they will be able to achieve the benchmark results then that is there problem. I think most people know that the benchmark, is what the phone can achieve, but its not something they will ever achieve in daily use. It is meant to be used as guide/comparison with other phones. A bit like the manufacturers quoted mpg for cars, nobody every gets the quoted mpg when driving. However people use it as a guide. Nobody is up in arms, saying the car manufacturers are pulling a fast one, or are caught 'doping the mpg'.
If you compare benchmarks of two phones, one is 20% ahead of the other, would you not expect that that phone would perform better in real situations, too?
Samsung doing this is basically to allow those who brag about power or benchmarks to be misled. As the actual user experience will not convert to having the fastest or most powerful phone day to day. It is misleading and a deliberate attempt to.
If the phone is being restricted to handle daily life or to curb battery usage that is what it should report during the tests. Most cpu's can be over clocked for a short period but it doesn't mean it should be sold as this powerful device if it never will be other than for a marketing tool.
It would be funny if it were actually possible to fry the chip if you kept running one of these benchmarks. Someone should try it and sue their ass off.
And yet imagine the furore if Apple were doing this.
I would suspect all companies that ship a device to the people that do the benchmarks, ship a benchmark optimised device. If you performed the exact same task on each phone, then I would say yes. Maybe not by 20% though.
The benchmarks are not there to mislead, they are a guide. Its a dick waving contest between manufacturers, and they are all at it. Samsung arent the only one that do it. As I said earlier, only a naive person would think a benchmark reflects real life useage - thus there is no attempt to mislead.
I agree that they should also provide a benchmark for daily use...i.e. what the user will actually experience. Provide both figures if you want, nothing wrong with that.
What a lot of tosh.
You can chip most cars these days to increase the bhp, with the prospect that the engine may run at lower mpg and be under higher stress with more emissions.
The performance figures quoted from a production car ( admittedly from ideal conditions ) are from the production vehicle and they would be sued if they chipped the car to give enhanced performance data. This is equivalent to what Samsung are doing
The benchmarks are hard coded into the OS. Sorry but how devious and underhand can you get