[Dark 1]

I see Samsung's been caught 'doping' it's phones to inflate benchmark scores again.

Their latest Galaxy Note 3 has been found to have that magical speed boost which only kicks in when it detects any benchmarking apps being run. This thereby artificially inflating its benchmark scores around 20% higher than any 'normal' app could attain.

Guess they didn't learn their lesson after being caught with their trousers down doing a similar stunt with the Galaxy S4.

No doubt some will argue benchmarks don't mean anything so this is a none issue. Others however may argue this is a clear attempt to fool unsuspecting consumers who believe bigger numbers = better, which are the actions of a company devoid of integrity. Choose your side.

Read all about it:

arstechnica.com/gadgets/2013/10/galaxy-note-3s-benchmarking-adjustments-inflate-scores-by-up-to-20%

[Zack06]

Or maybe it's designed to boost the numbers for benchmarks to show the full capability of the device.

Samsung more than likely limited the speed in daily usage to improve overall battery life, which is bound to take a hit if the device is allowed to hit maximum power to cope with heavy workloads.

It's not unlike car manufacturers limiting their cars to a certain speed to maintain optimum safety and usability, but these can be taken off when said car is in a race to allow the full power of the vehicle. The same applies here.

[IvanIV]

It does not show a full capacity, because it's never used except for the benchmarks. The analogy with car is wrong, because you cannot remove the restrictions, only Samsung can. It's solely purpose is to manipulate those who compare the benchmarks to compare the phones' performance. The results are misleading and meaningless.

[Dark 1]

Quote:

Originally Posted by Zack06

Or maybe it's designed to boost the numbers for benchmarks to show the full capability of the device.

Samsung more than likely limited the speed in daily usage to improve overall battery life, which is bound to take a hit if the device is allowed to hit maximum power to cope with heavy workloads.

It's not unlike car manufacturers limiting their cars to a certain speed to maintain optimum safety and usability, but these can be taken off when said car is in a race to allow the full power of the vehicle. The same applies here.

Oooo, that's a rather tenuous analogy I would say.

This speed boost only kicks in for benchmarking. This artificial "maximum power" is unattainable in any sort of real world usage. I see no other reason for it to exist other than to mislead consumers.

Hmmm… wouldn't be surprised if this isn't a class action lawsuit in waiting.

[Zack06]

Quote:

Originally Posted by Dark 1

Oooo, that's a rather tenuous analogy I would say.

This speed boost only kicks in for benchmarking. This artificial "maximum power" is unattainable in any sort of real world usage. I see no other reason for it to exist other than to mislead consumers.

Hmmm… wouldn't be surprised if this isn't a class action lawsuit in waiting.

It's deliberately unattainable. These processors on board demand a lot of power, and there's only so big a battery Samsung can put in without making the device too big or bulky for today's standards.

I'm not sure what people want them to do. Benchmarks are designed to show the performance, not day to day usage. Just as a drag race is supposed to prove who is the fastest, not how the car will perform when driving to the supermarket or whatever.

Notice that Samsung don't appear to put these kinds of restrictions on their lower tier devices because they don't have as much processing power, so the speed to battery life ratio is more acceptable.

Some variants of the S4 have as much as 8 processing cores on board, is it that hard to grasp that there may be massive strains on battery life when this processor is at its maximum capability?

Plus with all the issues of these devices heating up to dangerous levels, and without space for on board fans/cooling methods, I'm not surprised Samsung have simply elected to cap the speed of the device in normal usage.

[IvanIV]

Benchmarks should help to extrapolate how a device will perform in real life. They make you believe their devices perform better, but it isn't so. Values how fast a device can perform before it goes up in flames are meaningless.

[jabbamk1]

It sucks when companies do this.

Especially when the Note 3 is actually the best benchmarking phone even without the boosts.

Benchmarks a good way to see how powerful a device is compared to others. But it's real time performance that matters.

[enapace]

Quote:

Originally Posted by jabbamk1

It sucks when companies do this.

Especially when the Note 3 is actually the best benchmarking phone even without the boosts.

Benchmarks a good way to see how powerful a device is compared to others. But it's real time performance that matters.

Agreed which is why for years iPhones were the phone to get because they were fluid in all things.

[paulbrock]

it doesn't sound like it unlocks a special mode, it merely makes all the CPUs ready for a high use scenario. More of a running start than a high-power use exclusively for benchmarks.

[IvanIV]

Quote:

Originally Posted by paulbrock

it doesn't sound like it unlocks a special mode, it merely makes all the CPUs ready for a high use scenario. More of a running start than a high-power use exclusively for benchmarks.

If it does not happen in other situations it is a special mode. A modified benchmark with a changed name should be demanding enough to trigger the same behaviour. It does not, so it's likely that a speed is capped for any non-benchmarking apps.

[paulbrock]

Quote:

Originally Posted by IvanIV

If it does not happen in other situations it is a special mode. A modified benchmark with a changed name should be demanding enough to trigger the same behaviour. It does not, so it's likely that a speed is capped for any non-benchmarking apps.

that depends. Benchmark apps are pretty artificial situations, requiring jumps up from 0% cpu to 100% cpu for a minute or two. Is it a speed that would be reached over 10 mins if say an app had a consistently high CPU requirement?

Models shouldn't be penalised for using power-saving CPU modes in daily use, but also having more powerful modes available, even if the higher-powered modes take a little time to 'switch on'.

[jabbamk1]

Quote:

Originally Posted by paulbrock

that depends. Benchmark apps are pretty artificial situations, requiring jumps up from 0% cpu to 100% cpu for a minute or two. Is it a speed that would be reached over 10 mins if say an app had a consistently high CPU requirement?

Models shouldn't be penalised for using power-saving CPU modes in daily use, but also having more powerful modes available, even if the higher-powered modes take a little time to 'switch on'.

I agree with what you're saying.

But the article seems to make out that no other app uses the CPU as much as benchmarking apps.

The article could be wrong though and that certain high power profile could be accessed by other apps or the OS.

[IvanIV]

Quote:

Originally Posted by paulbrock

that depends. Benchmark apps are pretty artificial situations, requiring jumps up from 0% cpu to 100% cpu for a minute or two. Is it a speed that would be reached over 10 mins if say an app had a consistently high CPU requirement?

Models shouldn't be penalised for using power-saving CPU modes in daily use, but also having more powerful modes available, even if the higher-powered modes take a little time to 'switch on'.

I suspect this ready to go idling mode is reserved for the selected few benchmark apps and once a work is done a processor is shutting down the cores and switching to low power mode. Although it would make sense to stay with all cores switched on for a while, before switching them off if no work becomes available. It is probably so, but the idling is too long in benchmarks to stay ready without an intervention. So they turned off this one level of regulation of power consumption to get better results.

[ACU]

I really dont see the issue here. A benchmark is about how fast the phone can run. It has nothing to do with real life use what so ever. In that sense Samsung have done nothing wrong. The internals of the phone will allow it to run as fast as the benchmark states.

Thus I think the title of this thread is very misleading.

[Stiggles]

The thread title is incredibly misleading!!

All this does, as has been explained already, is allow the absolute maximum performance of the device to be used for a few mins at a time to show what in theory it's capable of.

Nothing wrong is being done here. I dare say many other companies do this as well.

[kidspud]

Quote:

Originally Posted by ACU

I really dont see the issue here. A benchmark is about how fast the phone can run. It has nothing to do with real life use what so ever. In that sense Samsung have done nothing wrong. The internals of the phone will allow it to run as fast as the benchmark states.

Thus I think the title of this thread is very misleading.

You would think this sort of action was even a concern to the companies that produce benchmarks. After all, what are the benchmarks really telling you if the companies continue to do this.

It doesn't reflect the performance of the phone as that performance is never available to the user.

The benchmarks are becoming less credible each time this sort of story comes out.

[ACU]

Quote:

Originally Posted by kidspud

You would think this sort of action was even a concern to the companies that produce benchmarks. After all, what are the benchmarks really telling you if the companies continue to do this.

It doesn't reflect the performance of the phone as that performance is never available to the user.

The benchmarks are becoming less credible each time this sort of story comes out.

If a person is naive to think that they will be able to achieve the benchmark results then that is there problem. I think most people know that the benchmark, is what the phone can achieve, but its not something they will ever achieve in daily use. It is meant to be used as guide/comparison with other phones. A bit like the manufacturers quoted mpg for cars, nobody every gets the quoted mpg when driving. However people use it as a guide. Nobody is up in arms, saying the car manufacturers are pulling a fast one, or are caught 'doping the mpg'.

[IvanIV]

If you take benchmarks as meaningless numbers fine. But why make them and compare them? A manufacturer of the processor could give you the same maxed out numbers per processor, done. In this scenario they measure how a processor and an OS are coping with bursts of 0 and 100% performance. Could it have something to do with entering and leaving power save mode? I'd expect that after a few such bursts of (in)activity an OS would decide not to power off the cores in anticipation of a future activity and switch back to normal if the bursts do not occur for a certain time. Sort of an adaptive power management. With Samsung's modification you can measure f-k all, because all cores are always ready and they got lovely numbers, better than anybody else, because they are completely unrealistic.

[IvanIV]

Quote:

Originally Posted by ACU

If a person is naive to think that they will be able to achieve the benchmark results then that is there problem. I think most people know that the benchmark, is what the phone can achieve, but its not something they will ever achieve in daily use. It is meant to be used as guide/comparison with other phones. A bit like the manufacturers quoted mpg for cars, nobody every gets the quoted mpg when driving. However people use it as a guide. Nobody is up in arms, saying the car manufacturers are pulling a fast one, or are caught 'doping the mpg'.

If you compare benchmarks of two phones, one is 20% ahead of the other, would you not expect that that phone would perform better in real situations, too?

[The Lord Lucan]

Some people (rightly or wrongly) choose a phone on benchmark tests alone, some press articles use it as a comparison to show how much faster or powerful a model is and people buy because of those conclusions.

Samsung doing this is basically to allow those who brag about power or benchmarks to be misled. As the actual user experience will not convert to having the fastest or most powerful phone day to day. It is misleading and a deliberate attempt to.

If the phone is being restricted to handle daily life or to curb battery usage that is what it should report during the tests. Most cpu's can be over clocked for a short period but it doesn't mean it should be sold as this powerful device if it never will be other than for a marketing tool.

[jonner101]

This is very devious of Samsung if they have actually hard coded their OS build to look for these benchmarks and basically allow the processor to be over-clocked when running them.

It would be funny if it were actually possible to fry the chip if you kept running one of these benchmarks. Someone should try it and sue their ass off.

[calico_pie]

Quote:

Originally Posted by Zack06

Or maybe it's designed to boost the numbers for benchmarks to show the full capability of the device.

Samsung more than likely limited the speed in daily usage to improve overall battery life, which is bound to take a hit if the device is allowed to hit maximum power to cope with heavy workloads.

It's not unlike car manufacturers limiting their cars to a certain speed to maintain optimum safety and usability, but these can be taken off when said car is in a race to allow the full power of the vehicle. The same applies here.

And yet imagine the furore if Apple were doing this.

[alanwarwic]

The LG one kicks into low power mode, the Samsung one does not.

[ACU]

Quote:

Originally Posted by IvanIV

If you compare benchmarks of two phones, one is 20% ahead of the other, would you not expect that that phone would perform better in real situations, too?

I would suspect all companies that ship a device to the people that do the benchmarks, ship a benchmark optimised device. If you performed the exact same task on each phone, then I would say yes. Maybe not by 20% though.

Quote:

Originally Posted by The Lord Lucan

Some people (rightly or wrongly) choose a phone on benchmark tests alone, some press articles use it as a comparison to show how much faster or powerful a model is and people buy because of those conclusions.

Samsung doing this is basically to allow those who brag about power or benchmarks to be misled. As the actual user experience will not convert to having the fastest or most powerful phone day to day. It is misleading and a deliberate attempt to.

If the phone is being restricted to handle daily life or to curb battery usage that is what it should report during the tests. Most cpu's can be over clocked for a short period but it doesn't mean it should be sold as this powerful device if it never will be other than for a marketing tool.

The benchmarks are not there to mislead, they are a guide. Its a dick waving contest between manufacturers, and they are all at it. Samsung arent the only one that do it. As I said earlier, only a naive person would think a benchmark reflects real life useage - thus there is no attempt to mislead.

I agree that they should also provide a benchmark for daily use...i.e. what the user will actually experience. Provide both figures if you want, nothing wrong with that.

[jonner101]

Quote:

Originally Posted by Zack06

Or maybe it's designed to boost the numbers for benchmarks to show the full capability of the device.

Samsung more than likely limited the speed in daily usage to improve overall battery life, which is bound to take a hit if the device is allowed to hit maximum power to cope with heavy workloads.

It's not unlike car manufacturers limiting their cars to a certain speed to maintain optimum safety and usability, but these can be taken off when said car is in a race to allow the full power of the vehicle. The same applies here.

What a lot of tosh.

You can chip most cars these days to increase the bhp, with the prospect that the engine may run at lower mpg and be under higher stress with more emissions.

The performance figures quoted from a production car ( admittedly from ideal conditions ) are from the production vehicle and they would be sued if they chipped the car to give enhanced performance data. This is equivalent to what Samsung are doing