DS Forums

 
 

Help converting ts files from Humax PVR


Reply
Thread Tools Search this Thread
Old 08-01-2009, 13:30
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
Here's the vb code with remarks added to explain how it works. You can paste this into vb as the ' character is interpreted as a remark.

Public Sub trimTS()

' Dimension array variable junk to hold 3 bytes

Dim junk(3) As Byte

' Dim array variable a to hold 187 bytes

DIM a(187) As Byte

' Dim Single Byte Constant as 47Hex

Const sync_byte As Byte = &H47

' note this declares variable fni and fno out as a variant will work but Dim fni as string, fno as string is more efficient

Dim fni, fno

' Explicitely define source file path and name

fni = "H:\" + "Test from Nick.ts"

' Explicitely define destination file path and name


fno = "H:\" + "fixedNick.ts"

' open source file to read byte by byte as file 1

Open fni For Binary Access Read As #1

' open destination file to write byte by byte as file 2

Open fno For Binary Access Write As #2

' read first 3 bytes from source file

Get #1, , junk

' read next 187 bytes from source file

Get #1, , a

' continue until first byte in 187 byte block is not 47Hex

While a(0) = sync_byte
' write 187 byte block to output file
Put #2, , a
' read next 3 bytes from source
Get #1, , junk
' read next 187 bytes from source
Get #1, , a
' loop back to while statement
Wend
' close both source and destination files
Close
End Sub

If you stay in the vb editor with the cursor in the sub () line and press f8 you can single step through the code as it runs. Hovering the cursor over a variable name will reveal it's current value. If you open an immediate window you can execute a vb command eg Print fno
grahamlthompson is offline   Reply With Quote
Please sign in or register to remove this advertisement.
Old 08-01-2009, 14:15
mccg
Forum Member
 
Join Date: Mar 2003
Location: Portchester, Fareham, Hants
Posts: 114
Looks like m2tstots
(as documented here: http://soft4share.com/m2ts_to_ts.html)
does the same thing - It removes the 4 byte header on each packet.
I cannot find a binary via google now
I have it on my home machine - I will try and upload it somewhere soon.

I used this on an ITV SD ts file, and was then able to use MPG-VCR from womble to remove the ads without re-encoding.
mccg is offline   Reply With Quote
Old 08-01-2009, 18:34
tv-Addict
Forum Member
 
Join Date: Dec 2008
Posts: 148
I have it on my home machine - I will try and upload it somewhere soon.
This is the version of m2tstots I have...
http://www.mikegregory.net/m2tstots.zip
tv-Addict is offline   Reply With Quote
Old 08-01-2009, 18:39
mwardy
Forum Member
 
Join Date: May 2008
Posts: 1,919
Here's the vb code with remarks added to explain how it works. You can paste this into vb as the ' character is interpreted as a remark.
Thanks again! This was all very helpful, especially being able to see the values in the arrays as you step through the module. It does confirm what I was thinking, though, which is that it only works if the first 47 it comes across is the sync byte (there are definitely other 47s in there). I assume this means that somehow there is a guarantee that this will always be true? Or is this a false assumption which is the cause of son_t's failed results? (These qs aren't necessarily directed at you!)

So now the next question (again, one for the panel): what's the structure for h.264 transport streams? I guess there a similar sync byte?
mwardy is offline   Reply With Quote
Old 08-01-2009, 18:41
mwardy
Forum Member
 
Join Date: May 2008
Posts: 1,919
I
The VB script does work on the HD recording - because it works up to the point when copy encryption/protection kicks in... the 35Mb file produce does play. Anyway the result is, that the VB script (stripping of the junk bytes) does not allow Enc BBC HD recordings to be played... (it will only rip upto the enc point...)
Verrrry interesting....
mwardy is offline   Reply With Quote
Old 08-01-2009, 18:54
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
Thanks again! This was all very helpful, especially being able to see the values in the arrays as you step through the module. It does confirm what I was thinking, though, which is that it only works if the first 47 it comes across is the sync byte (there are definitely other 47s in there). I assume this means that somehow there is a guarantee that this will always be true? Or is this a false assumption which is the cause of son_t's failed results?

So now the next question (not directed at you necessarily!): what's the structure for h.264 transport streams? I guess there a similar sync byte?
That's not quite true the code reads 147 bytes at once and only checks the first for 47Hex, an embedded 47Hex will be ignored. It stops when the first byte in the next block of 147 after discarding the junk bytes is not 47Hex. It therefore relies on a fixed block of data of a specific size unlike say a comma delimited text data file where you just keep reading until you find a comma (or whatever character is used as a delimiter). In this case its 3 bytes discarded, 147 bytes of data with a leading byte of 47H repeated ad nauseum.

Although I have a fair degree of VBA knowledge it does not, I am afraid extend to H264.mpeg4 data stream codeing.

Give me the file structure and I can extract the data OK but the key is how the file is built
grahamlthompson is offline   Reply With Quote
Old 08-01-2009, 18:58
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
Verrrry interesting....
It's just a guess but I would say that the file structure is probably the same 3 bytes (junk) one byte 47H (sync) 146 bytes of data but scrambled (encrypted) by whatever algorithm used by Humax
grahamlthompson is offline   Reply With Quote
Old 08-01-2009, 19:18
mwardy
Forum Member
 
Join Date: May 2008
Posts: 1,919
That's not quite true the code reads 147 bytes at once and only checks the first for 47Hex, an embedded 47Hex will be ignored.
Sorry, I did understand that but I wasn't very clear in expressing it. Another way of putting it I suppose would be that it expects the file to begin with the junk, so that the first 47H it finds is guaranteed to be a sync byte. I guess this is easy to arrange.

Give me the file structure and I can extract the data OK but the key is how the file is built
Indeed...
mwardy is offline   Reply With Quote
Old 09-01-2009, 09:16
jack plug
Forum Member
 
Join Date: May 2006
Posts: 60
... And oh yeah, when processing the Talk To Her recording, the VB scripted bombed out with 'Bad record number' error...
It would. VBA has a file-size limitation of 2GB

what's the structure for h.264 transport streams? I guess there a similar sync byte?
The h264 stuff is carried in the same mpeg2 TS packets. All TS packets start with that same sync byte.

Well spotted. I compared the output of m2tstots with the de-junked "test from Nick.ts". They are identical apart from at the very end, where there is an extra "packet" of garbage. That should not cause a problem though. Hopefully m2tstots will handle big files.
jack plug is offline   Reply With Quote
Old 09-01-2009, 10:06
mwardy
Forum Member
 
Join Date: May 2008
Posts: 1,919
The h264 stuff is carried in the same mpeg2 TS packets. All TS packets start with that same sync byte.
Thanks for that.

I wonder what the enc BBC HD file does to stop the module in its tracks (post 69)???

That link produces a 1K file which won't unzip.

Well spotted. I compared the output of m2tstots with the de-junked "test from Nick.ts". They are identical apart from at the very end, where there is an extra "packet" of garbage. That should not cause a problem though. Hopefully m2tstots will handle big files.
But I've found 'tsmuxer' (first hit in google) which seems to have a similar function--using its defaults it also produces a file which is about 200K longer than the output of this module. It looks like it could do a lot more as well....I wonder what it would do with the HD file? (hint, hint )
mwardy is offline   Reply With Quote
Old 09-01-2009, 10:07
son_t
Forum Member
 
Join Date: Jan 2006
Location: Fife
Posts: 4,038
It would. VBA has a file-size limitation of 2GB
I see. I wish people would stop writing code in Mickey-mouse languages and stop using Mickey-mouse filesystems!

Well spotted. I compared the output of m2tstots with the de-junked "test from Nick.ts". They are identical apart from at the very end, where there is an extra "packet" of garbage. That should not cause a problem though. Hopefully m2tstots will handle big files.
I ran m2tstots on my Wallace and Gromit Enc HD recording... it processed all of it, but played even less than your VB code!

(I've now downloaded VLC media player on the Vista machine and it plays back recordings off the HDR just fine - now that the right (K-Lite) codec pack has been installed. The stuttering playback of Talk To Her is not present in VLC as it is in MPC.)

EDIT: Re: Playback of Enc HD recordings off one HDR - will these playback on another?
If you have an HDR and are interested in downloading (FTP) an Enc HD recording to test on your box, please PM me. I have various recordings of different sizes ranging from 120Mb to the Wallace and Gromit at just under 4GB! BTW: this test is to see if encryption is tied to the box the HD programmes are recorded on...
son_t is offline   Reply With Quote
Old 09-01-2009, 10:31
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
It would. VBA has a file-size limitation of 2GB



The h264 stuff is carried in the same mpeg2 TS packets. All TS packets start with that same sync byte.



Well spotted. I compared the output of m2tstots with the de-junked "test from Nick.ts". They are identical apart from at the very end, where there is an extra "packet" of garbage. That should not cause a problem though. Hopefully m2tstots will handle big files.
The vb code will fail with a read past end of file error if there are not at least 190bytes of something after the last read ts packet. Not sure if this data is of any significance. It should be possible to modify the original code to process the file in less than 2Gb chunks writing a seperate file for each chunk. You could then be able to open a command window from within vba and use sendkeys to concatenate the seperate files into one big one on a ntfs partition.
grahamlthompson is offline   Reply With Quote
Old 09-01-2009, 11:08
mwardy
Forum Member
 
Join Date: May 2008
Posts: 1,919
The vb code will fail with a read past end of file error if there are not at least 190bytes of something after the last read ts packet. Not sure if this data is of any significance.
In practical terms, using tsmuxer on the uploaded file produces an identical result to the vb code except that it has one extra P frame at the end. Perhaps this is because it handles the end of file condition more smoothly, but at any rate it's not a problem.

Update: m2tstots downloaded successfully (thanks tv-Addict), and I can see the extra 180 or so bytes of garbage (a repeated 4 byte sequence) jack plug mentions. This is different from the output from tsmuxer which as I say manages to produce a whole extra frame. No significant difference between these different solutions then (except maybe file length limits).
mwardy is offline   Reply With Quote
Old 13-01-2009, 21:38
Portabletv2
Forum Member
 
Join Date: Nov 2008
Posts: 167
Can somebody please point me in the right direction? To save disc space on my Humax, I've been pulling off the ts files onto my PC. They play fine on the PC using VLC Player, but I want to stream them back to my TV using my media streamer: unfortunately it only recognises avi, mpg or wma file extensions, so the ts files won't play. The trial version of VideoReDo Suite that I downloaded does a super job at converting these ts files to mpg, but it's very expensive and I only want to use this conversion part of the suite. I've been looking for three days on the web to find something simpler & cheaper, and I must have tried a dozen or so converters/remuxers, none of which recognise the ts files. Any ideas, please?
Sorry it doesn't help with your problem. But,...how do you get files from a PVR to PC in the first place ?
Portabletv2 is offline   Reply With Quote
Old 14-01-2009, 09:26
son_t
Forum Member
 
Join Date: Jan 2006
Location: Fife
Posts: 4,038
The HDR has a 'copy' facilities allowing you to transfer SD recordings to an external USB hard drive. From my FAQ: http://foxsat-hdr.wikispaces.com/The+HDR+FAQ

Q. How do I transfer recordings to my external hard disk or USB stick?
A. Connect the external drive or USB stick to one of the USB ports. If it is a valid format, no messages should coming up telling you to format it, but that a USB device has been connected. When in the Media list, pressing the Opt+ button will bring up a plate with left and right panes. The right shows the USB device, and the left shows the internal drive. Move the cursor to the right pane and select the item marked sdaX*. Pressing OK will take you into the device and the HDR will show you a list of recordings already there. If there are none, it will be empty. Back to the left hand pane, select the recording (or rmultiple recordings, using the Edit Mode, press the Yellow button and press OK to tick items) you wish to transfer. If the HDR recognise it can transfer the recording(s) to the destination on the right pane, it will show the Green button 'Copy' function. Press Green to transfer the recording(s). [*X being a number, 0 , 1, or 2]
son_t is offline   Reply With Quote
Old 14-01-2009, 10:05
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
grahamlthompson, thank you! It works a treat.

I've also discovered a sort of proof that the junk stripper is doing its job in that the original won't play in videoredo plus (which .ts files default to on my machine rather that videoredo TV suite) but the cleaned up file will.

That primer didn't get as far as dialogue boxes anyway!
There is an update for Video Redo that can read .ts files directly. I and another sent them samples and within 24hrs they had a beta that worked which I now believe is a full production version.
grahamlthompson is offline   Reply With Quote
Old 14-01-2009, 10:08
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
We can't record ITV HD in non-freesat mode as the red interactive button does not work! Although, has anyone tried to record the ITV HD channel 10510 in non-freesat mode to see what that captures?

You can't receive ITV HD in non freesat mode as it's not a proper channel and you can't input the necessary parameters to make the data stream into a receivable channel. This is wierd the following post is a saved edit somehow it became a seperate posting. Any ideas how ?
grahamlthompson is offline   Reply With Quote
Old 14-01-2009, 10:09
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
We can't record ITV HD in non-freesat mode as the red interactive button does not work! Although, has anyone tried to record the ITV HD channel 10510 in non-freesat mode to see what that captures?

You can't receive ITV HD in non freesat mode as it's not a proper channel and you can't input the necessary parameters to make the data stream into a receivable channel. At least I can't unless someone out there knows how.
grahamlthompson is offline   Reply With Quote
Old 14-01-2009, 12:10
son_t
Forum Member
 
Join Date: Jan 2006
Location: Fife
Posts: 4,038
You can't receive ITV HD in non freesat mode as it's not a proper channel and you can't input the necessary parameters to make the data stream into a receivable channel. At least I can't unless someone out there knows how.
ITV HD appears as a channel labelled '10510' in non-freesat mode... It is obfuscated in the H.222 codec, which is why the box can not display picture and sound in non-freesat mode...
son_t is offline   Reply With Quote
Old 14-01-2009, 12:29
grahamlthompson
Forum Member
 
Join Date: May 2005
Location: Redditch Worcs
Posts: 17,289
ITV HD appears as a channel labelled '10510' in non-freesat mode... It is obfuscated in the H.222 codec, which is why the box can not display picture and sound in non-freesat mode...
If there's no video and audio does that mean it still might be possible to record and actually have video and audio on the recording ?
grahamlthompson is offline   Reply With Quote
Old 14-01-2009, 12:44
son_t
Forum Member
 
Join Date: Jan 2006
Location: Fife
Posts: 4,038
No, the HDR is not able to record a channel it can't decode...
son_t is offline   Reply With Quote
Old 14-01-2009, 15:27
Nick123
Forum Member
 
Join Date: Apr 2003
Location: London postcode
Posts: 347
Sorry it doesn't help with your problem. But,...how do you get files from a PVR to PC in the first place ?
Firstly, well done to getting to page 4 of this thread. Although I started it I have no idea after the end of page 1 what is going on. Anyway, it looks like our resident experts are working hard to try and fathom out the intracies of the HD recordings. We will all benefit in the end!
Back to basics, I don't know about 'a' PVR but the Humax Freesat one is simple:
1. Plug in a USB key or FAT-formatted external disc drive into either the front or rear USB slot
2. On the Freesat box, go to Media and press the 'OPT+' button to get the File Manager on the screen.
3. Navigate to the right window and choose a folder on your external device where you want to save the recording.
4. Go to the left side and click OK on what you want to copy.
5. Choose 'copy' from the menu that appears.
There will be three files copied across: all three are needed if you want to play back the recording directly from your external device on the Foxsat box. If you want to now transfer the recording to a computer, all you need is to copy over the ".ts" file to your PC. If you have VLC player installed it will automatically play the recording. Otherwise, you can convert it to an mpg file using VLC player as described earlier in this thread. There are also some other recoders available on the web (some are free) that will convert it into avi, wmv or whatever format you want. My favourite is MediaCoder.
Good luck!
Nick123 is offline   Reply With Quote
Old 15-01-2009, 09:04
IanL
Forum Member
 
Join Date: Apr 2004
Location: Jersey (Fremont and Rennes)
Posts: 243
..... If you want to now transfer the recording to a computer, all you need is to copy over the ".ts" file to your PC. If you have VLC player installed it will automatically play the recording. Otherwise, you can convert it to an mpg file using VLC player as described earlier in this thread. There are also some other recoders available on the web (some are free) that will convert it into avi, wmv or whatever format you want. My favourite is MediaCoder.
Good luck!
Nick,

Are you referring to SD files, or do those converters work for you with HD files?
IanL is offline   Reply With Quote
Old 15-01-2009, 13:53
mccg
Forum Member
 
Join Date: Mar 2003
Location: Portchester, Fareham, Hants
Posts: 114
1. you cannot transfer HD files to a USB device
(It is possible to extract them by removing the HDD, but you will find that...see 2)
2. HD files are encrypted. No decoder has been found.
(An exception is that some HD recording made in non-freesat mode are not encrypted)
mccg is offline   Reply With Quote
Old 15-01-2009, 14:13
son_t
Forum Member
 
Join Date: Jan 2006
Location: Fife
Posts: 4,038
1. you cannot transfer HD files to a USB device
(It is possible to extract them by removing the HDD, but you will find that...see 2)
But you can if you record them unencrypted in non-freesat mode (see 2). You can transfer HD recordings (encrypted or otherwise) FROM a USB drive to the internal.
2. HD files are encrypted. No decoder has been found.
(An exception is that some HD recording made in non-freesat mode are not encrypted)
son_t is offline   Reply With Quote
 
Reply




 
Forum Jump


All times are GMT. The time now is 08:57.