Samsung caught doping the benchmarks… again (Galaxy Note 3)

paulbrock · 02-10-2013, 20:05

Quote:

Originally Posted by alanwarwic

I thought it was actual default underlocking which was prevented.

Correct.

kidspud · 02-10-2013, 20:54

They are all at it.

Well, not quite

http://www.anandtech.com/show/7384/s...oid-benchmarks

Dark 1 · 03-10-2013, 00:16

Quote:

Originally Posted by kidspud

They are all at it.

Well, not quite

http://www.anandtech.com/show/7384/s...oid-benchmarks

So they are it would seem. Tsk! For those who can't be bothered to read the article, credit goes to Motorola, Google (Nexus) and Apple for seemingly not playing this game.

Notice they haven't tested any Windows phones yet though.

alanwarwic · 03-10-2013, 07:22

It is all interesting and I wonder if it part stems from throttling that has long been used to extend battery life.
Intel chipsets used extensive throttling with different speedstep scenarios for each different chip.
Intel would also classify a fast CP and slower CPU as having the same TDP wattage(TDP banding), when in fact the cheaper slower one in fact cooked the heatsink less.

Google gets no credit. Stock Android would not be very stock if it differentiated like the others.

IvanIV · 03-10-2013, 07:36

Quote:

Originally Posted by Dark 1

Notice they haven't tested any Windows phones yet though.

Are there any benchmarks for WP? A lack of apps is not all that bad after all

IvanIV · 03-10-2013, 07:44

Quote:

Originally Posted by alanwarwic

It is all interesting and I wonder if it part stems from throttling that has long been used to extend battery life.

It's a normal behaviour of the processor that cores are shut down when the processor is idling for a while. It does not make sense to have them always ready. Samsung just realised it makes the benchmark results worse so whenever a benchmark app is running they disable this behaviour => faster, more energy needed.

alanwarwic · 03-10-2013, 10:38

http://highonandroid.com/android-sma...one-is-better/

"The bottom line is that these two processors are about even"

See what I mean about rigging. They obviously buy 3rd party chipsets so as not to be on the backfoot with Apple chipset contracts.
It would be interesting to see how the benchmarks non rigged compare. I'm still guessing that they do not want to see either version seen as a 2nd class product, so they rig it, a bit like we had those bigger signal bars on Apple phones.

ACU · 03-10-2013, 10:53

Quote:

Originally Posted by kidspud

They are all at it.

Well, not quite

http://www.anandtech.com/show/7384/s...oid-benchmarks

Fair enough. It looks like I was wrong about them all being at it.

Dark 1 · 03-10-2013, 21:57

Apparently the accusations are false. Samsung has said in a statement, "The Galaxy Note 3 maximises its CPU/GPU frequencies when running features that demand substantial performance (and is) not an attempt to exaggerate particular benchmarking results."

They didn't mention why this 'boost' only happens when known benchmarking apps are being run though. But they did go on to say, "We remain committed to providing our customers with the best possible user experience."

So that's okay then.

crave.cnet.co.uk/mobiles/samsung-defends-galaxy-note-3-benchmarking-shenanigans-50012403

Dark 1 · 27-11-2013, 23:46

Bump.

The sorry saga continues…

Samsung and HTC face Android 3D graphics test chart removal shame - Benchmark monitor sends alleged test cheats to the back of the class.

www.theregister.co.uk/2013/11/26/samsung_and_htc_face_android_3d_graphics_test_delisting_shame

kidspud · 28-11-2013, 06:55

Quote:

Originally Posted by Dark 1

Bump.

The sorry saga continues…

Samsung and HTC face Android 3D graphics test chart removal shame - Benchmark monitor sends alleged test cheats to the back of the class.

www.theregister.co.uk/2013/11/26/samsung_and_htc_face_android_3d_graphics_test_delisting_shame

I think we are just use to it now. That's why processor performance is never reflected in actual use.

IvanIV · 28-11-2013, 07:20

They just all need to cheat the same way.

Stuart_h · 28-11-2013, 07:48

Quote:

Originally Posted by IvanIV

They just all need to cheat the same way.

Or we all need to start ignoring benchmarks that are never very useful these days anyway ..... Modern phones are so quick that the differences for most tasks are fractions of seconds....

kidspud · 28-11-2013, 08:04

Quote:

Originally Posted by Stuart_h

Or we all need to start ignoring benchmarks that are never very useful these days anyway ..... Modern phones are so quick that the differences for most tasks are fractions of seconds....

^This. In a strange way the cheating is good news for those in the benchmark business because it makes it look like the results are still important to phone suppliers.

BeethovensPiano · 28-11-2013, 08:17

Shady Samsung lol

Stuart_h · 28-11-2013, 08:24

Quote:

Originally Posted by kidspud

^This. In a strange way the cheating is good news for those in the benchmark business because it makes it look like the results are still important to phone suppliers.

Yep. I'm certainly not defending the companies actions. There has always been debate about how useful they are. Things like this just prove that point further.

alanwarwic · 28-11-2013, 08:40

Surely, all they have to do is issue an update.
That is what has usually always happened in the past, be it for Apple, Nvidia etc.

Saying that, how come they do not all publish real tests that do real things?

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

02-10-2013, 20:05	#51
paulbrock Forum Member Join Date: Apr 2002 Location: Wapping, London Posts: 16,222	Quote: Originally Posted by alanwarwic I thought it was actual default underlocking which was prevented. Correct.

02-10-2013, 20:54	#52
kidspud Forum Member Join Date: May 2010 Posts: 11,501	They are all at it. Well, not quite http://www.anandtech.com/show/7384/s...oid-benchmarks

03-10-2013, 00:16	#53
Dark 1 Forum Member Join Date: Nov 2002 Location: Nottingham Posts: 3,975	Quote: Originally Posted by kidspud They are all at it. Well, not quite http://www.anandtech.com/show/7384/s...oid-benchmarks So they are it would seem. Tsk! For those who can't be bothered to read the article, credit goes to Motorola, Google (Nexus) and Apple for seemingly not playing this game. Notice they haven't tested any Windows phones yet though.

03-10-2013, 07:22	#54
alanwarwic Forum Member Join Date: Oct 2003 Location: the wild world web Posts: 28,132	It is all interesting and I wonder if it part stems from throttling that has long been used to extend battery life. Intel chipsets used extensive throttling with different speedstep scenarios for each different chip. Intel would also classify a fast CP and slower CPU as having the same TDP wattage(TDP banding), when in fact the cheaper slower one in fact cooked the heatsink less. Google gets no credit. Stock Android would not be very stock if it differentiated like the others.

03-10-2013, 07:36	#55
IvanIV Forum Member Join Date: May 2006 Posts: 25,199	Quote: Originally Posted by Dark 1 Notice they haven't tested any Windows phones yet though. Are there any benchmarks for WP? A lack of apps is not all that bad after all

03-10-2013, 07:44	#56
IvanIV Forum Member Join Date: May 2006 Posts: 25,199	Quote: Originally Posted by alanwarwic It is all interesting and I wonder if it part stems from throttling that has long been used to extend battery life. It's a normal behaviour of the processor that cores are shut down when the processor is idling for a while. It does not make sense to have them always ready. Samsung just realised it makes the benchmark results worse so whenever a benchmark app is running they disable this behaviour => faster, more energy needed.

03-10-2013, 10:38	#57
alanwarwic Forum Member Join Date: Oct 2003 Location: the wild world web Posts: 28,132	http://highonandroid.com/android-sma...one-is-better/ "The bottom line is that these two processors are about even" See what I mean about rigging. They obviously buy 3rd party chipsets so as not to be on the backfoot with Apple chipset contracts. It would be interesting to see how the benchmarks non rigged compare. I'm still guessing that they do not want to see either version seen as a 2nd class product, so they rig it, a bit like we had those bigger signal bars on Apple phones.

03-10-2013, 10:53	#58
ACU Forum Member Join Date: Aug 2009 Posts: 7,916	Quote: Originally Posted by kidspud They are all at it. Well, not quite http://www.anandtech.com/show/7384/s...oid-benchmarks Fair enough. It looks like I was wrong about them all being at it.

03-10-2013, 21:57	#59
Dark 1 Forum Member Join Date: Nov 2002 Location: Nottingham Posts: 3,975	Apparently the accusations are false. Samsung has said in a statement, "The Galaxy Note 3 maximises its CPU/GPU frequencies when running features that demand substantial performance (and is) not an attempt to exaggerate particular benchmarking results." They didn't mention why this 'boost' only happens when known benchmarking apps are being run though. But they did go on to say, "We remain committed to providing our customers with the best possible user experience." So that's okay then. crave.cnet.co.uk/mobiles/samsung-defends-galaxy-note-3-benchmarking-shenanigans-50012403

27-11-2013, 23:46	#60
Dark 1 Forum Member Join Date: Nov 2002 Location: Nottingham Posts: 3,975	Bump. The sorry saga continues… Samsung and HTC face Android 3D graphics test chart removal shame - Benchmark monitor sends alleged test cheats to the back of the class. www.theregister.co.uk/2013/11/26/samsung_and_htc_face_android_3d_graphics_test_delisting_shame

28-11-2013, 08:17	#65
BeethovensPiano Inactive Member Join Date: Jun 2006 Location: ♫ At The Keyboard ♫ Posts: 11,556	Shady Samsung lol

28-11-2013, 06:55	#61
kidspud Forum Member Join Date: May 2010 Posts: 11,501	Quote: Originally Posted by Dark 1 Bump. The sorry saga continues… Samsung and HTC face Android 3D graphics test chart removal shame - Benchmark monitor sends alleged test cheats to the back of the class. www.theregister.co.uk/2013/11/26/samsung_and_htc_face_android_3d_graphics_test_delisting_shame I think we are just use to it now. That's why processor performance is never reflected in actual use.

28-11-2013, 07:20	#62
IvanIV Forum Member Join Date: May 2006 Posts: 25,199	They just all need to cheat the same way.

28-11-2013, 07:48	#63
Stuart_h Forum Member Join Date: Jul 2005 Posts: 3,474	Quote: Originally Posted by IvanIV They just all need to cheat the same way. Or we all need to start ignoring benchmarks that are never very useful these days anyway ..... Modern phones are so quick that the differences for most tasks are fractions of seconds....

28-11-2013, 08:04	#64
kidspud Forum Member Join Date: May 2010 Posts: 11,501	Quote: Originally Posted by Stuart_h Or we all need to start ignoring benchmarks that are never very useful these days anyway ..... Modern phones are so quick that the differences for most tasks are fractions of seconds.... ^This. In a strange way the cheating is good news for those in the benchmark business because it makes it look like the results are still important to phone suppliers.

28-11-2013, 08:24	#66
Stuart_h Forum Member Join Date: Jul 2005 Posts: 3,474	Quote: Originally Posted by kidspud ^This. In a strange way the cheating is good news for those in the benchmark business because it makes it look like the results are still important to phone suppliers. Yep. I'm certainly not defending the companies actions. There has always been debate about how useful they are. Things like this just prove that point further.

28-11-2013, 08:40	#67
alanwarwic Forum Member Join Date: Oct 2003 Location: the wild world web Posts: 28,132	Surely, all they have to do is issue an update. That is what has usually always happened in the past, be it for Apple, Nvidia etc. Saying that, how come they do not all publish real tests that do real things?

DS Forums

Samsung caught doping the benchmarks… again (Galaxy Note 3)