[tichtich]

As someone with an interest in languages and how they develop, I was amused to notice today how the word "buffering" has in recent years acquired a colloquial sense which is different from the technical sense.

In computing terminology a "buffer" is a queue of data being held until it's needed (or possible the memory location in which such a queue is held). So "to buffer" data is to store it in a such a queue.

In recent years we've all become familiar with video streaming, and particularly with streaming over the internet. Streaming programs use a buffer to queue video data, so they can display an uninterrupted video even when the incoming data arrives erratically. When data arrives over the internet faster than needed, it's buffered (queued) until it's needed. This provides a reserve of data that can be drawn on during periods when new data is arriving too slowly.

When the buffer is empty, there is no new video to display and the playback must be paused until sufficient new data has arrived. Streaming programs wait until they've buffered enough data for a few seconds or so of video before continuing. If they showed each frame as it was received, you would see a jerky slow-motion video when the data is coming in slower than needed. Better to pause for a while and then show a continuous video. During these periods of pausing many programs display the word "buffering".

This has caused people to associate the word "buffering" with pausing, and for many people now "buffering" apparently means pausing (of a video). I only noticed this for the first time today, when I read the following sentences on a web page:

- "...the film will keep pausing while the connection waits for the next bit of data to be sent, known as 'buffering'."
- "Most services recommend a connection speed of around 2MB to prevent buffering..."
- "...the signal could be intermittent, causing the film to buffer."

In this new sense, buffering is a bad thing, something to be prevented. But in the technical sense it's a good thing. We want data to be buffered, as buffering prevents pausing, as long as the buffer doesn't empty. It was this switch from buffering being good to being bad which made the new sense stick out to me like a sore thumb.

Presumably when programmers programmed streamers to say "buffering" while paused, what they had in mind was something like this: "Don't worry. I haven't crashed. I'm replenishing my empty buffer."

Any thoughts? Do you think this will influence the way you read/hear the word "buffering" in future?

[The Phazer]

It’s worse, because people don’t understand the difference between streaming delivery.

I really don’t have an issue with people describing a YouTube video as buffering. It’s a perfectly valid example of the word, and has been in long term use.

But people describe services such as iPlayer and 4OD as “buffering”. These services don’t load anything into a buffer. They send discrete packets of tiny sections of the video, which is played, then thrown away, then the next packet is sent. You can pause an iPlayer programme 10 minutes in, but that doesn’t mean it will continue to download the rest of the programme when you do. All you are doing is throwing the roulette wheel again and hoping your ISP’s link to the CDN is a bit less congested when you unpause it.

Such services really do just “pause”, for content protection reasons. But you do see people who pause iPlayer videos for ten minutes, thinking it will make a difference because it does on YouTube.

[technologist]

All digital services buffer - even your DTT or DSAT ...
the issue is how big in the buffer
(and where it is in the system).

Through the air broadcasting tends to have smaller buffer than a good CDN delivered
which needs less buffering than over the wild west Internet.
and of course a download is just the largest buffer you can have for an asset!.

[tichtich]

Quote:

Originally Posted by The Phazer

Such services really do just “pause”, for content protection reasons. But you do see people who pause iPlayer videos for ten minutes, thinking it will make a difference because it does on YouTube.

Thanks. I didn't know that. But they might (I don't know) still queue a small amount of data during normal running, to guard against momentary breaks in receiving data.

[tichtich]

Quote:

Originally Posted by technologist

All digital services buffer - even your DTT or DSAT ...
the issue is how big in the buffer
(and where it is in the system).

Through the air broadcasting tends to have smaller buffer than a good CDN delivered
which needs less buffering than over the wild west Internet.
and of course a download is just the largest buffer you can have for an asset!.

Good point. But surely a buffer is a container or containing structure, and not the data itself. So strictly it's your hard disk (or a folder) that's the buffer when you download a movie. And the downloading software isn't itself buffering movies, though it's used in the process. Buffering refers to what's going on at a higher level. It's you who's buffering movies, when you build up a stock of one or more movies with the intention of watching them later.

I appreciate (having just looked it up) that "buffer" can be used in a broad sense, to mean any data store between an input and an output. But for present purposes we might want to restrict ourselves to talking about buffering in the sense of queuing data, to guard against differences in the input and output data rates. Perhaps digital TVs (over the air) don't buffer in that sense, since the input and output are so synchronized.

[alcockell]

Isn't it more the case that a memory structure is set up as a FIFO queue, and it's when this is filling up to a sweet spot - then the "buffering" message appears. Ideally you want a nice big bolus of data to keep the buffers (FIFO queues) full...

[d'@ve]

Quote:

Originally Posted by The Phazer

Such services really do just “pause”, for content protection reasons. But you do see people who pause iPlayer videos for ten minutes, thinking it will make a difference because it does on YouTube.

As tichtich mentioned, iPlayer still does buffer a chunk or two even on my fibre connection when you hit pause, it has to in order to do an immediate restart when you hit play again. On HD content, it continues buffering chunks for up to one minute when I first pause, but only a few seconds on subsequent pauses.

Perhaps the slower your connection, the more chunks iPlayer will buffer, or perhaps it will call them up more frequently so on a very slow connection that should still help playback to run smoothly.

My chunks arrive every 9 or 10 seconds at up to 10Mbps on average for most SD stuff, higher bitrates and more frequently for HD.

[technologist]

Quote:

Originally Posted by tichtich

Perhaps digital TVs (over the air) don't buffer in that sense, since the input and output are so synchronized.

But the DTT DSAT TV do needs a buffer as the rate at which the packets are sent is not constant - thus there is always the need to have some data ready to be processed - and that is a buffer!
If you think about it an MPEG video stream has variable amounts of data for each frame - as I frames are more verbose that P or B frames .... (and adaptive streaming can be though as being like that as well!)

I could go on more about muxing of all the other components and then the whole area of stat muxing which provides another ripple on the instant bit rate - ...
But the transport medium - though the air- has little variation - unlike transmission through the Internet ...
and of course packetisation itself is a buffering process.

[tichtich]

Quote:

Originally Posted by technologist

But the DTT DSAT TV do needs a buffer as the rate at which the packets are sent is not constant - thus there is always the need to have some data ready to be processed - and that is a buffer!

Good point. I didn't realise that the DTT stream was split into arbitrary packets, rather like internet packets. I thought they might transmit a whole GOP as a single packet. But on reflection that was implausible.

[Glawster2002]

From a network perspective there is buffering throughout the traffic path from the point of origin to the point of delivery, so just because your PC, or whatever you are using to view the content, is buffering it could just as easily be due to congestion elsewhere in the network and nothing to do with your broadband connection, a point always conveniently overlooked by the broadband providers when they promise their latest giga-, mega-, unbelievably fast-, rapidly running out of hyperbole descriptions- fast broadband "never" buffers.

[spiney2]

all "on demand" internet connections use buffering since its part of the tcp protocols. the alternative is ip mutlicasting, which guarantees delivery but wastes packet switching capacity (some of the network capacity must be premanently set aside).

http://en.wikipedia.org/wiki/TCP_tuning

http://en.wikipedia.org/wiki/IP_multicast

multicasting DOES make sense on intranets where large business concerns have their own private tv channels. its a bit daft for home use since radio waves are a common resource.

[alcockell]

Quote:

Originally Posted by Glawster2002

From a network perspective there is buffering throughout the traffic path from the point of origin to the point of delivery, so just because your PC, or whatever you are using to view the content, is buffering it could just as easily be due to congestion elsewhere in the network and nothing to do with your broadband connection, a point always conveniently overlooked by the broadband providers when they promise their latest giga-, mega-, unbelievably fast-, rapidly running out of hyperbole descriptions- fast broadband "never" buffers.

Which in turn is why the Youview consortium and others negotiate with ISPs to get direct peering in place, and often their own CDN edge servers into ISP datacentres... so the data can be delivered from somewhere close to the user.

[alexw4444]

I think most people understand that a buffer is something to protect them, and therefore it's a good thing.

That being said, I don't know who these "most people" are.

[d'@ve]

Quote:

Originally Posted by alexw4444

I think most people understand that a buffer is something to protect them, and therefore it's a good thing.

...Except when you are watching "live" football in HD (even on TV) and hear cheering for a goal before the move has even started, someone in the next room or next door having heard it 5 or more seconds earlier on medium wave radio!

[LION8TIGER]

Quote:

Originally Posted by d'@ve

...Except when you are watching "live" football in HD (even on TV) and hear cheering for a goal before the move has even started, someone in the next room or next door having heard it 5 or more seconds earlier on medium wave radio!

For me the difference between the HD on Sky in the living room and the non HD non Sky in the bedroom is one or two seconds. Before we got Sky and watched football using Sopcast on a laptop hooked up to the TV, we had that problem you describe .... a very loud fellow Arsenal fan living opposite. You were talking something like 30-60 seconds before the goal you'd know all about it .... just the Arsenal goal that is.

[technologist]

Quote:

Originally Posted by spiney2

all "on demand" internet connections use buffering since its part of the tcp protocols. the alternative is ip mutlicasting, which guarantees delivery but wastes packet switching capacity (some of the network capacity must be premanently set aside).

http://en.wikipedia.org/wiki/TCP_tuning

http://en.wikipedia.org/wiki/IP_multicast

multicasting DOES make sense on intranets where large business concerns have their own private tv channels. its a bit daft for home use since radio waves are a common resource.

But multi casting ALSO uses buffering - as the latency in the network is not constant. Strictly speaking there is also buffering on a deterministic transport like PDH or SDH so it is not surprising that a none deterministic IP or Ethernet where there packets are of different size , randomly contended etc needing more / large buffering as well.
Buffers are a fact of life as soon as you have any thing which is sampled and then digital ....

Just a quick note of protocols
http is under TCP/IP a protocol which hand shakes - and actually streaming by http is becoming quite efficient.
UDP just sends the packet into the network without any hand shaking
Multicast is UDP which propagates within the network so that one stream leaving the server becomes a stream to all who subscribe by it ...

At least with streaming http you have a measure of network latency - you have no idea on UDP etc..
Audio and video over IP is regularly used by the broadcasters - all Audio in the BBC nations and Local radio is over IP as is all video between Welsh sites and also in Scotland. -all over the standard business WAN using UDP without any error correction (as that adds buffering/latency)