Galaxy S4: Samsung caught doping in benchmarks

xflarexflare Posts: 1,768
Forum Member
✭✭✭
http://www.anandtech.com/show/7187/looking-at-cpugpu-benchmark-optimizations-galaxy-s-4
In looking at Samsung's chip, AnandTech was able to verify what a poster had reported on graphics website Beyond3D: Samsung was specifically enabling full speed GPU performance only when running specific benchmarks, and then reverting to slower 480MHz speeds when being used to play games or run other apps.

oops
«1

Comments

  • kidspudkidspud Posts: 18,341
    Forum Member
    ✭✭
    xflare wrote: »

    I've just been reading this article. Benchmarks have always meant very little, but Samsung obviously thought they were important enough to go out their way to manipulate them.
  • whoever,heywhoever,hey Posts: 30,992
    Forum Member
    ✭✭✭
    I'm a little confused by this. Was the GPU not just overclocking on the fly under demand?
  • Mr. CoolMr. Cool Posts: 1,551
    Forum Member
    ✭✭✭
    I doubt they've sold many phones to the average user based on benchmarks.

    I'll admit they are underhand but they aren't the first company to do it. AMD/Nvidia has too.

    They might even open up the optimisations to other apps.
  • kidspudkidspud Posts: 18,341
    Forum Member
    ✭✭
    I'm a little confused by this. Was the GPU not just overclocking on the fly under demand?

    It claims there was specific code to check the benchmark software running and up the GPS clock speed based on that.
  • whoever,heywhoever,hey Posts: 30,992
    Forum Member
    ✭✭✭
    Actually its damn dodgy having actually read the article.
  • BeethovensPianoBeethovensPiano Posts: 11,689
    Forum Member
    ✭✭
    Engadget have picked up the story now

    http://www.engadget.com/2013/07/30/samsung-benchmarks/
  • [Deleted User][Deleted User] Posts: 468
    Forum Member
    Explains why there phones stutters, pathetic.
  • The Lord LucanThe Lord Lucan Posts: 5,054
    Forum Member
    Would be interesting to see what the battery, heat and reliability is like when they keep repeating the test over and over...

    Can't believe i was lambasted for saying it stuttered so i handed i handed it back. Samsung are showing their true colours with this & the paid student reviews.
  • GormondGormond Posts: 15,838
    Forum Member
    ✭✭
    Actually its damn dodgy having actually read the article.

    Just read it myself and I have to agree, I get the feeling had this been Apple the outcry would be far greater :rolleyes:
  • StigglesStiggles Posts: 9,618
    Forum Member
    Would be interesting to see what the battery, heat and reliability is like when they keep repeating the test over and over...

    Can't believe i was lambasted for saying it stuttered so i handed i handed it back. Samsung are showing their true colours with this & the paid student reviews.

    Like no other company lies or pays people to review them!!

    You weren't lambasted, people disagreed with you. All the S4's i have used after the last update have had no stutters.

    It is however absolutely pointless and stupid of Samsung if true.
  • slick1twoslick1two Posts: 2,877
    Forum Member
    ✭✭✭
    Waiting for Swordman and step666 to defend this. I'm sure they will find a good defense for Sammy.
  • alan1302alan1302 Posts: 6,336
    Forum Member
    Gormond wrote: »
    Just read it myself and I have to agree, I get the feeling had this been Apple the outcry would be far greater :rolleyes:

    I'm sure it would do, but Samsung and Android are the current flavour of the month.
  • alanwarwicalanwarwic Posts: 28,396
    Forum Member
    ✭✭✭
    Actually its damn dodgy having actually read the article.
    Considering in the later benchmark the power govern means the low power Cortex A7s running at 500MHz kick in, there certainly is a standard governor that will lowers benchmarks.

    You could go to an extreme and argue that Samsung has allowed the benchmark to work.
  • swordmanswordman Posts: 6,679
    Forum Member
    slick1two wrote: »
    Waiting for Swordman and step666 to defend this. I'm sure they will find a good defense for Sammy.

    Have you been waiting, not read this properly yet but I will give you my initial take as you have been waiting so patiently.

    Don't see it, I can't imagine samsung or any phone manufacturer doctoring a phone to run a benchmark quicker just doesn't seem realistic or likely to me.

    There is such a small market that would notice or even care about such things it would be more effort than its worth. The days of Nvidia and ATI doing this are completely different.

    However I may be wrong but would be surprised.
  • GormondGormond Posts: 15,838
    Forum Member
    ✭✭
    swordman wrote: »
    Have you been waiting, not read this properly yet but I will give you my initial take as you have been waiting so patiently.

    Don't see it, I can't imagine samsung or any phone manufacturer doctoring a phone to run a benchmark quicker just doesn't seem realistic or likely to me.

    There is such a small market that would notice or even care about such things it would be more effort than its worth. The days of Nvidia and ATI doing this are completely different.

    However I may be wrong but would be surprised.

    Read the article, it's pretty evident this is exactly what they did. Anandtech don't do thing by half, they are very through.
  • swordmanswordman Posts: 6,679
    Forum Member
    Just read it and I don't think its pretty evident at all I am not disputing what they have concluded but as I said I find it very difficult to believe samsung have gone out of their way to tweak the drivers of a phone to doctor a benchmark.

    Not because they are so morally forthright I just don't believe there is any gain in doing so it means nothing to 99.9% of people even us sad gits on here.
  • Zack06Zack06 Posts: 28,304
    Forum Member
    ✭✭✭
    Even though I dislike some of Samsung's practises, I can't really see how this is such a big deal. :confused:

    All they have done is increase the GPU to its maximum potential to show off the full capability of the device. After all that is what a benchmark is for right? These benchmarks are hardly indicative of real world/real time performance, so I don't think Samsung can be vilified for wanting to show off the best that their devices can do.

    I don't think its cheating at all, after all, they only increased it to the true potential of the device. What Samsung have been doing is not dissimilar to what various car companies do. Many high performance vehicles are speed limited to stop overheating or to increase fuel consumption (battery life in a mobile phone context), but for a drag race, which is essentially what a benchmark is, those speed limiters are often taken off so that the full power of the car can be displayed, something they do on Top Gear and the like all the time.

    Samsung have done something similar. In the real world, games and devices work together to provide a balance between battery life and performance. But a benchmark has never been about real world performance, so I don't think Samsung is really at fault at all. It's a media story blown way out of proportion.
  • alanwarwicalanwarwic Posts: 28,396
    Forum Member
    ✭✭✭
    It is an interesting one.
    Should the power governor kick in for those benchmarks too?

    We just recently had those Intel benchmarks shown actually 20% false versus ARM quite suddenly making ARM faster than Atom.(a near non news story for some unknown reason)

    And I quite recall a certain OS in the past that naturally tricked browser benchmarks to 'complete early'.
  • alan1302alan1302 Posts: 6,336
    Forum Member
    Zack06 wrote: »
    Even though I dislike some of Samsung's practises, I can't really see how this is such a big deal. :confused:

    Because Samsung have only allowed the full power to come when using certain benchmarks and without the user knowing. It's a cheat and designed to look good in reviews.

    I can't understand why you think it's ok.
  • swordmanswordman Posts: 6,679
    Forum Member
    Samsung statement

    "Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.
    The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results."


    If right and the gpu can hit 533 under certain circumstances in normal operation then this is not fixing anything. Indeed aligns with what I have said earlier that I cannot see this being worth samsung doing deliberately.
  • GormondGormond Posts: 15,838
    Forum Member
    ✭✭
    alan1302 wrote: »
    Because Samsung have only allowed the full power to come when using certain benchmarks and without the user knowing. It's a cheat and designed to look good in reviews.

    I can't understand why you think it's ok.

    It's okay because it's Samsung that's doing it, if this were Apple these same people would be outraged...

    Samsung doing this damages the entire market as it falsely makes their phone look better than the competitors from companies like Sony, LG and HTC.
  • StigglesStiggles Posts: 9,618
    Forum Member
    Gormond wrote: »
    It's okay because it's Samsung that's doing it, if this were Apple these same people would be outraged...

    Samsung doing this damages the entire market as it falsely makes their phone look better than the competitors from companies like Sony, LG and HTC.

    I'm not sure they did anything wrong.

    If their statement is accurate, then it's perfectly normal to do it
  • swordmanswordman Posts: 6,679
    Forum Member
    Gormond wrote: »
    It's okay because it's Samsung that's doing it, if this were Apple these same people would be outraged...

    Samsung doing this damages the entire market as it falsely makes their phone look better than the competitors from companies like Sony, LG and HTC.

    So they have come out with a deliberate lie to cover this up really, is what what you think?
  • GormondGormond Posts: 15,838
    Forum Member
    ✭✭
    swordman wrote: »
    Samsung statement

    "Under ordinary conditions, the GALAXY S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode. Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player, and certain benchmarking apps, which also demand substantial performance.
    The maximum GPU frequencies for the GALAXY S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results."


    If right and the gpu can hit 533 under certain circumstances in normal operation then this is not fixing anything.

    So by over clocking when a certain benchmark tool is detected is perfectly fine because it isn't running for a while? But a game running for extended periods is not because it would cause all sorts of issues?

    Samsung know the purpose of these benchmarks and have set it up so they look better than real world usage which put the competitors at a disadvantage.
  • alanwarwicalanwarwic Posts: 28,396
    Forum Member
    ✭✭✭
    swordman wrote: »
    Samsung statement
    Anand read like there was a white list. The Samsung statement thus sounds badly worded if so.

    But assuming that benchmark did some standard video tests that would have run at full speed normally, was the benchmark correct in a few aspects?
Sign In or Register to comment.