|
||||||||
Samsung caught doping the benchmarks… again (Galaxy Note 3) |
![]() |
|
|
Thread Tools | Search this Thread |
|
|
#26 |
|
Forum Member
Join Date: Apr 2008
Posts: 3,153
|
Quote:
I would suspect all companies that ship a device to the people that do the benchmarks, ship a benchmark optimised device. If you performed the exact same task on each phone, then I would say yes. Maybe not by 20% though.
The benchmarks are not there to mislead, they are a guide. Its a dick waving contest between manufacturers, and they are all at it. Samsung arent the only one that do it. As I said earlier, only a naive person would think a benchmark reflects real life useage - thus there is no attempt to mislead. I agree that they should also provide a benchmark for daily use...i.e. what the user will actually experience. Provide both figures if you want, nothing wrong with that. |
|
|
|
|
Please sign in or register to remove this advertisement.
|
|
|
#27 |
|
Forum Member
Join Date: Jan 2011
Location: Dundee, Scotland
Posts: 9,292
|
Quote:
The benchmarks are hard coded into the OS. Sorry but how devious and underhand can you get
No they aren't! The code simply allows the device to perform at it's absolute maximum while the test is being run. It's not adding anything to the benchmark! |
|
|
|
|
|
#28 |
|
Forum Member
Join Date: Apr 2008
Posts: 3,153
|
Quote:
Eh?
No they aren't! The code simply allows the device to perform at it's absolute maximum while the test is being run. It's not adding anything to the benchmark! Quote:
The file we ended up with is called "DVFSHelper.java," and it contains a hard-coded list of every package that is affected by the special CPU boosting mode. According to this file, the function is used exclusively for benchmarks, and it seems to cover all the popular ones. There's Geekbench, Quadrant, Antutu, Linpack, GFXBench, and even some of Samsung's own benchmarks. The two functions applied to this list seem to be "PACKAGES_FOR_BOOST_ALL_ADJUSTMENT," which is no doubt the CPU booster, and "PACKAGES_FOR_LCD_FRAME_RATE_ADJUSTMENT," which makes it sound like the phone is also altering the display frame rate.
The inclusion of GFXBench is surprising given that it shows no unusual idling behavior in System Monitor. Between the inclusion of that and the suspicious "frame rate adjustment" string, it's clear that Samsung is doing something to the GPU as well, though those clock speeds are more difficult to access than the CPU speeds (a method used by AnandTech on the international S 4 no longer works on the Note 3). The "DVFS" in "DVFSHelper" stands for "Dynamic frequency scaling," also known as CPU throttling, which has many legitimate uses to manage both heat and power draw. This file contains a few special settings for the camera, Gallery, and some other packed-in apps, but nothing like what is in the above section. Benchmarking apps are the only type of app that is systematically called out and boosted. To see how some other benchmarks are affected, we made "stealth" versions of those, too—the exact same app, just with a different package name. These results back up the Geekbench findings: we're seeing artificial benchmark increases across the board of about 20 percent; Linpack showed a boosted variance of about 50 percent. |
|
|
|
|
|
#29 |
|
Forum Member
Join Date: May 2006
Posts: 25,199
|
Samsung switch off processor's power save mode when a programme with a certain package name is running. By "coincidence" they are all benchmark programmes. Testers repackaged a benchmark programme using a name not on the Samsung list and they got different (worse) results, because power save mode kicked in this time.
|
|
|
|
|
|
#30 |
|
Forum Member
Join Date: Feb 2013
Location: South Wales
Posts: 5,866
|
In my world it's called "not playing by the book", while nothing wrong with showing the capabilities of a phone, they should make it perfectly clear that these figures are not attainable in "real world settings" and then publish the actual "real world" figures.
|
|
|
|
|
|
#31 |
|
Inactive Member
Join Date: Jan 2003
Posts: 43,524
|
Quote:
This is very devious of Samsung if they have actually hard coded their OS build to look for these benchmarks and basically allow the processor to be over-clocked when running them.
It would be funny if it were actually possible to fry the chip if you kept running one of these benchmarks. Someone should try it and sue their ass off. |
|
|
|
|
|
#32 |
|
Forum Member
Join Date: Nov 2002
Location: Nottingham
Posts: 3,975
|
Quote:
The thread title is incredibly misleading!!
All this does, as has been explained already, is allow the absolute maximum performance of the device to be used for a few mins at a time to show what in theory it's capable of. Nothing wrong is being done here. I dare say many other companies do this as well. And it's just coincidental that only Samsung has been caught doing this? Twice! No, I think the title is appropriate. It's a cynical tactic to mislead consumers into believing their phones are significantly more powerful than those of their direct competitors. "They are without honour" - Worf |
|
|
|
|
|
#33 |
|
Forum Member
Join Date: Feb 2013
Location: South Wales
Posts: 5,866
|
^^^ agree it's bloody dodgy and unfair, I suppose to use the car thing again would be for BMW to claim there 3 Series can top 180mph, then actually say well the M3 can.
|
|
|
|
|
|
#34 |
|
Forum Member
Join Date: Nov 2002
Location: Nottingham
Posts: 3,975
|
Quote:
In my world it's called "not playing by the book", while nothing wrong with showing the capabilities of a phone, they should make it perfectly clear that these figures are not attainable in "real world settings" and then publish the actual "real world" figures.
|
|
|
|
|
|
#35 |
|
Forum Member
Join Date: Feb 2013
Location: South Wales
Posts: 5,866
|
^^true, but Samsung must run these benchmarks too, what's to stop them handing over a phone with a totally mad processor, get the benchmarks, then change the processor for the full production model, which in a way is what they are doing.
|
|
|
|
|
|
#36 |
|
Forum Member
Join Date: Nov 2002
Location: Nottingham
Posts: 3,975
|
Quote:
seems like a lot of trouble to go to for a replacement of an already working phone.
|
|
|
|
|
|
#37 |
|
Inactive Member
Join Date: Jan 2003
Posts: 43,524
|
Quote:
More likely for someone (an American probably) to buy one just so they can have a punt at suing Samsung, claiming they only did so purely based on the benchmarking scores.
|
|
|
|
|
|
#38 |
|
Forum Member
Join Date: Feb 2013
Location: South Wales
Posts: 5,866
|
^^^ no but the Americans do love a "class action" as seen with Microsoft and the Xbox 360
|
|
|
|
|
|
#39 |
|
Forum Member
Join Date: Aug 2009
Posts: 7,916
|
Quote:
So it's just coincidental that the only apps that can tap into this "maximum performance" are known benchmarking apps which serve no purpose other than to provide numbers which can be compared to competitors products?
And it's just coincidental that only Samsung has been caught doing this? Twice! No, I think the title is appropriate. It's a cynical tactic to mislead consumers into believing their phones are significantly more powerful than those of their direct competitors. "They are without honour" - Worf Not sure why you think its a Samsung issue, like I said earlier all companies do it, including Apple. Its nothing new. Funny seeing people grab at straws to have a pop at Samsung. The words glass, houses and stones come to mind. Good luck finding a company with honour. Care to let me know of one? I cant think of one, that makes smartphones. |
|
|
|
|
|
#40 |
|
Forum Member
Join Date: Feb 2013
Location: South Wales
Posts: 5,866
|
^^ Do Apple fudge benchmarks, I've not seen a 70 page thread on that yet, there must be one if you say they do, every other Apple Thread gets the treatment.
|
|
|
|
|
|
#41 |
|
Forum Member
Join Date: Oct 2003
Location: the wild world web
Posts: 28,132
|
You know what the outcome of this seems to be?
It is that the hyped up Snapdragons are more than a bit crap. We already had Samsung limiting Exynos to allow the Snapdragon and Exynos S4s to work at the same speed. This amplifies everything even further. Or behold, is it a case of the exact same? |
|
|
|
|
|
#42 |
|
Forum Member
Join Date: Jan 2012
Location: London, UK
Posts: 8,759
|
Quote:
You know what the outcome of this seems to be?
It is that the hyped up Snapdragons are more than a bit crap. We already had Samsung limiting Exynos to allow the Snapdragon and Exynos S4s to work at the same speed. This amplifies everything even further. Or behold, is it a case of the exact same? Qualcomm are not crap. |
|
|
|
|
|
#43 |
|
Forum Member
Join Date: May 2006
Posts: 25,199
|
Samsung want to have their processor and eat it. First with big fanfares they announce a mad 8-core processor, then it turns out that a four is a new eight. Then they say they use special multicore processor that can switch to a power save mode and save battery. There's only a small problem, benchmarks are worse, because waking up the processor from power save mode is not for free. No problem, turn it off for benchmarking. This is not so harmless as some state here. There's enough people who take benchmarks into consideration when choosing their phones. Phones nowadays are mostly very good and one has to decide based on something, why not the benchmarks, especially if Samsung phones are way ahead of the competition. IMO it's quite dodgy.
|
|
|
|
|
|
#44 |
|
Forum Member
Join Date: Oct 2003
Location: the wild world web
Posts: 28,132
|
Quote:
....I've not seen a 70 page thread on that yet, there must be one if you say they do, every other Apple Thread gets the treatment.
I was thinking of that time when Safari benchmarks were recording an early finish time, thus increasing reported speed. |
|
|
|
|
|
#45 |
|
Forum Member
Join Date: Apr 2002
Location: Wapping, London
Posts: 16,222
|
if mobile benchmark tests actually did model real world user usage and then samsung came along and spoiled the test then it'd be right to call foul.
But they don't, they never have. they tend to throw excessive amounts of calculations/polygons at the processors, far more than would ever be encountered in normal daily use, and count how long it takes. Given its not real world cases, there's nothing wrong with manufacturers *optimising* their devices for the benchmarks, because that's really all that's happening. There's no secret overclocking going on, or "set the engnes to 110% captain" or "if benchmark then unlock secret superfast mode". |
|
|
|
|
#46 |
|
Forum Member
Join Date: Oct 2003
Location: the wild world web
Posts: 28,132
|
Quote:
What...?Qualcomm are not crap.
I'm sort of baffled as to why this is not some of the same. We have the cheating benchmark mantra and have moved away from reasoning. |
|
|
|
|
|
#47 |
|
Forum Member
Join Date: Apr 2008
Posts: 3,153
|
Can't believe some people think it's ok to hack your OS so it over clocks when all the well known benchmark apps are run.
Has this been done by other companies, well probably but that doesn't make it OK for Samsung to do it. Clear fraud on Samsung's behalf. |
|
|
|
|
|
#48 |
|
Forum Member
Join Date: Oct 2003
Location: the wild world web
Posts: 28,132
|
I thought it was actual default underlocking which was prevented.
What is the overlock? |
|
|
|
|
|
#49 |
|
Forum Member
Join Date: Feb 2005
Posts: 6,762
|
Don't care. I'll be having one regardless.
|
|
|
|
|
|
#50 |
|
Forum Member
Join Date: Oct 2003
Location: the wild world web
Posts: 28,132
|
http://en.m.wikipedia.org/wiki/Note_3
Oh look, just like the S4 there is both the Exynos and the Snapdragon chipsets. Like I said, they are making the Snapdragon look less crap. Yes, I overplay the word 'crap' but that counteracts the hype of those last generation ARM CPUs. |
|
|
|
![]() |
|
All times are GMT. The time now is 13:10.




