This is NOT as simple as it sounds.
IF you take audio from the scart socket to the amp / speakers, then the volume is at a fixed level, and you will need to adjust the volume using the amp's controls.
At least if you use the headphone out socket, the level from that will vary as you turn the tv sound up and down, so you will still be able to do everything from the one remote control.
Unless your tv is one of those that puts up an extremely stupid little symbol on the screen to tell you that you have headphones plugged in.
you really would have thought now, with tv speakers being so carp, they would at least give you a pair of speaker sockets, and a switch to turn off the internal speakers.