After various email communications over the past few weeks, I thought that it would be easier to discuss a few points here.
Before I start though, may I apologise for my absence, (It’s been a few months since my last post). I have had one or two issues to deal with but the light at the end of the tunnel is getting brighter….
OK, where was I? …..264 files Many appliance type DVR’s have a common method of DVR data export. The internal software separates the cameras from their continuous stream that is housed within the Hard Disk Drive, and splits the requested data into segments. They are not placed into any standard container but given a generic .264 file extension. These .264 files then are then written to a USB device or optical disk allocated by the user for export use. Quite often though, there is no player or associated metadata included.
The first challenge is playback.
You may be lucky and during the export process, a player gets included. This is rarely the case though and the majority of players have to be sourced from the brands website. This would be great, if the brand actually had a website or download area. If your .264 files have been produced from a generic black box, with little information, it can be rather frustrating.
There are many different flavours of player. Some require installation and some don’t. If you have a player that needs installing then always install in a clean virtual environment first. Many include various different codecs and filters that have the ability to cause havoc. If, after assesment, you find that you can cleanly install within a host, then you may find it plays a little better. This especially the case with large, higher than SD resolutions.
There are a number of ways to establish immediate playback without going hunting for a player. The first being FFplay. With FFmpeg housed in your system (See here for help), you simply type ffplay into your command prompt and then drag your .264 file in. Press enter!
You may find (50% of the time), that it doesn’t play. This is due to the initial bytes not being understood by the FFmpeg decoders. In these cases, the usual solution is to force the input as H264. This is done by typing ffplay -f h264 and then drag your file in.
Another option for immediate playback is to chose a specific decoder in software such as VLC. When VLC is open, navigate its menu structure to Tools > Preferences and then set show all in bottom left.
Although this is handy, because you have now set VLC to specifically use the H264 demuxer, it will only open H264 video files.
If you have an image analysis task to perform on the file, and have access to Amped FIVE, then you can conduct the entire review and analysis process from within that software. Remember that many .264 files are just raw streams so you may need to chose what framework reads the file and then you may need to process the file in some way in order for it to be reviewed. If processing is required, then your new file will open up in a new chain.
We now have successful playback, but that’s only the beginning of the story!
Most .264 files are hard encoded at the time of capture with a date and time overlay. In many ways this really helps. I admit that every now and again, something is captured behind the text data but thankfully those occasions are few and far between. Having this timing does help when we start to process the files for analysis. Many .264 files can be processed individually into the .avi container using a tool called AVIgenerator. There are many different versions and some of the older ones will not run in Windows 7 or above.
There is another big warning here. Don’t go installing these into your working machine – use a virtual!
Version 1.8 added these new files to my Operating System and made over 500 registry changes!
C:\Documents and Settings\Administrator\Desktop\AVIGenerator.lnk
C:\Documents and Settings\Administrator\Local Settings\Temp\Perflib_Perfdata_7a0.dat
C:\Documents and Settings\Administrator\Start Menu\Programs\AVIGenerator\AVIGenerator.lnk
C:\Documents and Settings\Administrator\Start Menu\Programs\AVIGenerator\Uninstall.lnk
C:\Program Files\AVIGenerator\AVIGenerator.exe C:\Program Files\AVIGenerator\reg.bat
C:\Program Files\AVIGenerator\reg.vbs C:\Program Files\AVIGenerator\uninst.exe
On the plus side, whilst you are installing into your Virtual OS, you can run Cameyo and virtualise the software. After installation, and after Cameyo has done its stuff, you can copy the new ‘app’ over to your host PC and use it without the worry of all the codecs and filters screwing up your machine!
Most of these generators, are basic wrappers. They scan the stream and then write a header and an index. It’s then all contained within the AVI format. So, all is good? No…. Take a look at the player example again using this file… Now take a look at the rewrapped .avi being decoded through Windows Media Player. The player, and then the new .avi, both display the video in its sampled size. In this case, 704 x 240. It is clear that the original recording has been made in field mode. Just from observing the imagery, they are obviously ‘squished’. You may also find that your dedicated .264 player exports an image (jpg or bmp) in its sampled size as well, rather than its true Display Aspect Ratio. Another warning here…. If you find that you have a player or AVI generator that does rescale – what interpolation method is it using? There are also a few that rescale to a standard size, regardless of the original, sampled size. So, before we start to process our files, we have to understand how they are made up and how they should be displayed. By analyzing the .264 stream and outputting a full frame analysis xml table from FFprobe, it’s possible to identify all the important numbers.
The dimensions for each sampled frame are shown and, as per our initial assessment, they are 704 x 240. When dealing with streams in containers, the sampled width and height are always worth checking. It’s possible that the stream can have one dimension but the container can force it to display in another. Sounds bizarre but as you will see – it can also make our lives a little easier.
Making the decision, and actually performing a rescale, all comes down to what you need to do. You may have hundreds of .264 files spanning a number of cameras from a large time period. If viewing was your initial task, then it may be worthwhile batch wrapping each one using ffmpeg and a simple .bat file. The key here would be to avoid transcoding, but what do we do about the field based sampling? Obviously, we can deal with that easily when it comes to analysis either by using dedicated software such as FIVE, or utilising other rescale methods in FFmpeg or even in Photoshop. Scaling and transcoding many files is going to take time though… and disk space! For viewing however, we can place the raw stream into a container and give the container an aspect ‘flag’. No transcoding required! This is completed in FFmpeg by using -aspect 4:3
Our raw .264 streams are now contained inside a file that when played, will display at a 4:3 aspect ratio. Before I move on – how do I know to display it at that ratio? 704×240 at field level is line doubled to a 704×480 frame. 704×480 is an analogue dimension. Most analogue dimensions should be viewed at a 4:3 Aspect Ratio. It’s worth stating that any correction of aspect ratio for analysis purposes must be completed by a proper scene and equipment assessment. For basic viewing though, the Aspect flag can make life a lot easier. So without transcoding, our video goes from this…
to this… I have not changed the video, I have simply placed a marker within the file header. This marker informs whatever software is being to decode it, to display the footage at a 4:3 Aspect ratio.
So, what about frame rate? In the example file being used here, I can see that we have a GOP of 19 (using the Frame Analysis xml). This is, coincidently, in the middle of the range of FPS that I have observed through scrubbing through the video. By using the time overlay (told you it was handy!) I can count the FPS. In this case it was highly variable with anywhere from 16 to 21 new images per second. It is likely that the DVR is set to record at 19 FPS but it encodes at a variable frame rate.
A concern though is the frame count difference between the avi generated by the proprietary software and then an avi generated through an up to date FFmpeg. In order to remove any issues caused by the H264 encoding, I ran tests into an uncompressed format. I also transcoded the avi produced by the generator. The generator file produced 17066 images. The FFmpeg file produced 17077 images. From Analyzing and comparing these against the frame xml, it’s possible to see that there are missing coded pictures but the GOP remains at 19.
See the bottom of this post for details on how to use a small VB script to identify all the missing coded pictures.
The missing coded pictures all appear in pieces of footage that have a low FPS when the time overlay is observed. Could these frames have been dropped at the original recording stage, or were they dropped during the demuxing and export stage? As for the 11 missing frames from the AVI generated file, it’s a little hard to assess across 17K plus images. My hunch is that the generator has dropped a few when trying to standardise the frame rate for the avi container.
With the time overlay, I can identify a period of 900 seconds. 17077 Frames / by 900 seconds =18.9 FPS.
I’ll have to start ‘wrapping’ this up!
.264 files can be a real easy file to deal with, but they come with hidden traps. If you start grabbing images, or processing the file incorrectly, your interpretation of that footage could be very wrong. I have shared a folder called 264 in my BOX (Widget on the right). It include a player, 3 generators and an example batch file for bulk wrapping into the mp4 container. Just tweak the text to change things like frame rate etc. Also read the text for where to locate it!
There are so many variables when dealing with these types of file that trying to put everything in order has proved quite tricky. In conclusion:
- There are players dedicated for playback
- Other playback methods are possible by utilising FFmpeg based software
- Files can be analyzed and image analysis completed within FIVE with no external processing required
- Files can be ‘contained’ into an avi using the proprietary AVIgenerator program
- Files may need processing in order to display correctly
- Timing must be reviewed and tested.
I hope that some of the issues discussed will help you when dealing with .264 files!