Digital Spy

Search Digital Spy
 

DS Forums

 
 
 

Freesat SI tables/Service Discovery


Reply
Thread Tools Search this Thread
Old 14-01-2009, 12:58
essele
Forum Member
 
Join Date: Jan 2009
Posts: 3

Hi,

I'm relatively new to the world of DVB, but have been working on a set-top-box type application for a while now, the initial work was done using DVB-T and Freeview and I've just starting looking at DVB-S and incorporating Freesat support. This has raised a whole heap of questions -- I know the Freesat internals aren't public, but it seems from reading these forums that a number of people are looking it it, so I thought
perhaps this might stimulate some interesting discussion and hopefully some progress...

I'll start with the Service Discovery issue ... so my approach for DVB-T is to read the PAT, all the PMT's referenced in the PAT and then populate a per-service structure based on SDT entries. This seems to work fine for DVB-T and "standard" DVB-S, but since Freesat uses different PID's for NIT/BAT/SDT it's a little more complex.

It looks to me like the Freesat PID's are described in the PMT's for each service on the transponder (whether a Freesat service or not), I can't re-verify this at the moment, but I seem to remember them being described in the "Five" PMT. They show up as a set of type 5 streams with a BBC Private Descriptor type 0xd1 which is a 1 byte descriptor that seems to be a reference to which table is 'tunnelled'.

My guess is 7=NIT, 3/4=SDT/BAT, 1=EIT, 2=EIT p/f, 5/6=TDT/TOT, these seem to be somewhat confirmed by looking at the stream information in the EIT entries, but I'm not clear on how you match the EIT info to the PMT info with the standard EIT (they don't seem to use tags properly!)

Once you get these PID's and start looking at the Freesat SDT and EIT all the tags work properly and it's easy to correctly identify the streams.

So my fundamental question is this... do we really need to lookup a service in the standard PMT to get the Freesat PID info? Wouldn't it have been easier for them to add some kind of private descriptor to the NIT or standard SDT rather than burden every service with this? Or can we assume they are fixed? This all seems a bit of a kludge to me!

I see lots of references to different PID's all over these newsgroups, but so far for every transponder I've looked at they seem to be constant at NIT=3840,SDT/BAT=3841,EIT=3842,EITp/f=3843,TDT/TOT=3844. Also the EITp/f is transmitted with a different table id (0xd1 -- any coincidence there?)

Other private descriptors...

On the NIT we have 0xd2, this looks like a transport_stream_id (for 2315), original (or current) network_id, an unknown byte, then a length (7) followed by incrementing numbers (currently 1,2,3,4,5,6,7) -- is this a reference to the SI tables???

On the SDT we have 0xd9, this seems to be a language followed by a simple string representation of the channel name ... any ideas what this is for? Seems like a duplication of service_name to me?

On the BAT we have 0xd3, this seems to include the LCN for each service referenced in the BAT ... need more on this, but so far:
service_id (16 bits)
unknown (16 bits)
size_of_next_bit (8 bits)
unknown (4 bits)
LCN (12 bits)
unknown (fills "next_bit", sometimes just ff ff, other times mixed)

On the EIT we have 0xda, has a language in there, but I've not really looked at this one yet.

Also, I can see multiple bouquets which seem to be the regional variations (plus HD), but any idea how you determin which one is which and how you apply postcode rules to them? Surely these aren't hardcoded in the receiver?

This is all from the BBC HD transponder, but from my quick scanning around looks to be the same on all the others as well.

Anyway, sorry this is such a long post, it's a partial braindump of recent investigations ... hopefully we can discover some more through wider discussion.

Lee
essele is offline   Reply With Quote
Please sign in or register to remove this advertisement.
Old 15-01-2009, 11:14
essele
Forum Member
 
Join Date: Jan 2009
Posts: 3
Ok, so answering some of my own questions now ... I've looked a bit more through the BAT and this I have a better handle on some of the descriptors.

0xd3 actually looks like a mapping of LCN to region for a given service, with 0xffff seeming to be the wildcard region, not sure what region 0x0000 represents, more digging needed here.

0xd4 seems to be a straightforward region listing for a given BAT section, each region is given a number which is then used in the references above.

0xd8 seems to be a kind of category list (News & Sports, Entertainment etc), but I haven't looked too much at this yet.

0xd7 seems to have a size (16bits), a reference to 0x090b (which is a transport stream), the original network id (0x003b), then what looks like two set of 16bit numbers -- 07 dd 00 02 1a 5f and 07 fd 00 02 18 9d. 1a5f and 189d are service id's for BBC Radio 1 and BBC 1 London. I wonder whether this is some kind of default starting channel??

0xd5 i'm really not sure about yet ...

Regards,

Lee.
essele is offline   Reply With Quote
Old 15-01-2009, 14:43
jzee
Forum Member
 
Join Date: Jan 2007
Posts: 15,161
Ok, so answering some of my own questions now ... I've looked a bit more through the BAT and this I have a better handle on some of the descriptors.

Regards,

Lee.
Lee, I have to admit I don't have much of a clue what you are talking about but are you trying to decode the Freesat EPG? If so, DJBlu, mitsubishi and others have worked on that here.
jzee is offline   Reply With Quote
Old 15-01-2009, 14:58
germanycalling
Forum Member
 
Join Date: Aug 2008
Location: Singapore
Posts: 709
0xd3 actually looks like a mapping of LCN to region for a given service, with 0xffff seeming to be the wildcard region, not sure what region 0x0000 represents, more digging needed here.



0xd7 seems to have a size (16bits), a reference to 0x090b (which is a transport stream), the original network id (0x003b), then what looks like two set of 16bit numbers -- 07 dd 00 02 1a 5f and 07 fd 00 02 18 9d. 1a5f and 189d are service id's for BBC Radio 1 and BBC 1 London. I wonder whether this is some kind of default starting channel??


Regards,

Lee.
Welcome to the Forum, Lee! Just like jzee your posts are way beyond my knowledge. However part of what you said sort of rang a bell. Part of what you are looking at could be indexing a Postcode. For example I did not know until I read it here that I will only be able to get BBC Radio London if I use a London Postcode.
germanycalling is offline   Reply With Quote
Old 15-01-2009, 16:41
jack plug
Forum Member
 
Join Date: May 2006
Posts: 60
Ok, so answering some of my own questions now ...
Have pity Lee. It took me three hours to partly follow your first post.

As far as I can gather, you are saying this:

1) The Freesat service table pids are not fixed, so a mechanism is required by which they may be found.

2) You have discovered such a mechanism.

The mechanism:

3) The Freesat streams (pids) that carry the Freesat service tables are associated in the PMTs of Freesat services with a D1 descriptor which contains a number or numbers identifying the particular table(s) carried.

So, a box, knowing nothing but the meaning of the D1 values, can scan the PMTs for D1 and the associated pid carries the corresponding table(s)

So my fundamental question is this... do we really need to lookup a service in the standard PMT to get the Freesat PID info? Wouldn't it have been easier for them to add some kind of private descriptor to the NIT or standard SDT rather than burden every service with this? Or can we assume they are fixed? This all seems a bit of a kludge to me!
My guess is Freesat don't have the power to mess with the standard tables.

I see lots of references to different PID's all over these newsgroups, but so far for every transponder I've looked at they seem to be constant at NIT=3840,SDT/BAT=3841,EIT=3842,EITp/f=3843,TDT/TOT=3844. Also the EITp/f is transmitted with a different table id (0xd1 -- any coincidence there?)
Every transponder but the Home transponder.

On the NIT we have 0xd2, this looks like a transport_stream_id (for 2315), original (or current) network_id, an unknown byte, then a length (7) followed by incrementing numbers (currently 1,2,3,4,5,6,7) -- is this a reference to the SI tables???
I had a look at D2 in the (Freesat) NIT
Code:
09 0B : 00 3B : 00 : 29  04 : 07 : 01 02 03 04 05 06 07
 2315 :    59 :  ? :  10500 :
 TSID :  ONID :  ? :    SID :
Clearly the address of FreesatHome.

Is this how a Freesat box finds Home?

From TSReader:
Code:
Program Number: 10500 FreesatHome
PMT
3001 Descriptor: 0xd1 07    : NIT
3002 Descriptor: 0xd1 03 04 : SDT/BAT
3003 Descriptor: 0xd1 01    : EIT
3004 Descriptor: 0xd1 02    : ???
3005 Descriptor: 0xd1 05 06 : TDT/TOT
On the SDT we have 0xd9, this seems to be a language followed by a simple string representation of the channel name ... any ideas what this is for? Seems like a duplication of service_name to me?
These are short names for the services. As I don't have access to a Freesat box I'm unable to see where, or if, they are used.

On the BAT we have 0xd3, this seems to include the LCN for each service referenced in the BAT ... need more on this, but so far:
service_id (16 bits)
unknown (16 bits)
size_of_next_bit (8 bits)
unknown (4 bits)
LCN (12 bits)
unknown (fills "next_bit", sometimes just ff ff, other times mixed)
D3 seems roughly analogous to B1 on that other network - which shall not be named.

The unknown after service_id is possibly the equivalent of epg_service_id on B1.

The top bit seems always set. If the lower 15 bits are interpreted as a decimal number, the number groups the services as follows:

Code:
00000-00499 Freesat services
00500-00999 BBC services
01000-01499 ITV services
01500-01999 Channel 4 services
02000-02999 Other services beginning with A
03000-03999 Other services beginning with B
04000-04999 Other services beginning with C
...........................................
...........................................
27000-27999 Other services beginning with Z
28000-28999 Other services beginning with a digit
That's about all I have at the moment.
jack plug is offline   Reply With Quote
Old 15-01-2009, 17:29
essele
Forum Member
 
Join Date: Jan 2009
Posts: 3
Thanks for the response all, especially Jack!

So if the Freesat Home stream has a different set of PID's then it's pretty clear that we need to look for the identifiers each time we tune to a new stream, just in case they are different.

I'd read something else about "Freesat Home", and how that may potentially be where the postcode lists come from ... I'm away from a live stream until later this evening, but I think some analysis is needed on that stream.

I'm also very intrigued that TSReader seems to understand most of the meaning of the 0xd1 descriptor ... I'm on Linux and OSX so haven't played with this yet (dvbsnoop doesn't know anything about 0xd1.)

I can confirm that a 0xd1 type of 0x02 is the EITp/f table with a different table number ... I'm using the information now and it's entirely consistent with a normal EITp/f (apart from the encoding of strings.)

There are still a few other descriptors to look at, but this is really helpful.

Many thanks,

Lee.
essele is offline   Reply With Quote
Old 15-01-2009, 18:14
jack plug
Forum Member
 
Join Date: May 2006
Posts: 60
I'm also very intrigued that TSReader seems to understand most of the meaning of the 0xd1 descriptor ... I'm on Linux and OSX so haven't played with this yet (dvbsnoop doesn't know anything about 0xd1.)
A slight misunderstanding there Lee, my fault.

TSreader just lists the D1 bytes, it does not understand their meaning. The bits after the colons are my comments to indicate what those pids carry.
jack plug is offline   Reply With Quote
Old 10-03-2009, 00:00
ngolian
Forum Member
 
Join Date: Apr 2004
Posts: 17
I'm writing a PVR application too, so essele and I will probably end up rivalling each other . At the moment I'm writing the code to scan for channels. The information in the first post in this thread has been enormously helpful to me, but there are still difficulties.

Using the above method to find SDTs and NITs I can only find ones with table ids 0x46 and 0x41 ie NITs for "other networks" and SDTs for other transponders which list all the services except the ones on the current transponder. Makes things a bit awkward, but not unworkable. I couldn't find any other SDTs or NITs with dvbsnoop in pidscan mode, looks like they don't exist unless they have non-standard table ids.

Has anyone managed to get any more information about how to find LCNs? I've tried parsing the 0xd3 descriptors in the BAT, but there seem to be lots missing with no discernible pattern, eg some BBC regions have LCNs, but others are missing, including all BBC 2s, 3 and 4. E4 and Film 4 have LCNs, but not Channel 4. And I have several ITV1 variants with the same LCN. I've uploaded the result of one of my scans to http://www.realh.ukfsn.org/channels.conf.txt. It's in VDR format so each LCN, where known, is on a line beginning with before the channel info.

Speaking of Channel 4, it and +1 appear several times on the same 2 transponders with different PIDs, is that normal?

Another little niggle is that part-time services don't seem to be in the PMT while off air, ie CBeebies and CBBC don't appear in the evenings. I think Freeview used to do that too, but with my recent test scans the off-air channels do appear in Freeview's PMTs. To allow for when they don't I suppose I'll have to get my PVR to look up the PIDs on the fly when switching channels.
ngolian is offline   Reply With Quote
Old 10-03-2009, 00:09
McBread
Forum Member
 
Join Date: Oct 2005
Posts: 304
Everything you need to know is probably already figured out in the MythTV source.
http://svn.mythtv.org/trac/attachmen...sat_epg.2.diff
McBread is offline   Reply With Quote
Old 10-03-2009, 00:16
ngolian
Forum Member
 
Join Date: Apr 2004
Posts: 17
Everything you need to know is probably already figured out in the MythTV source.
http://svn.mythtv.org/trac/attachmen...sat_epg.2.diff
I tried mythtv a while ago, it doesn't seem to have any support for LCNs in Freeview (they were easy, by reading the code for the linuxtv-dvb scan utility) or Freesat, which is the main thing I'm stuck on.

I haven't got round to fetching EITs yet
ngolian is offline   Reply With Quote
Old 11-03-2009, 19:26
ngolian
Forum Member
 
Join Date: Apr 2004
Posts: 17
I've tried parsing the 0xd3 descriptors in the BAT, but there seem to be lots missing...
I've solved this problem by realising that each descriptor may carry info for more than one service
ngolian is offline   Reply With Quote
 
Reply



Thread Tools Search this Thread
Search this Thread:

Advanced Search

 
Forum Jump


All times are GMT +1. The time now is 08:19.