Canon Dust Delete and Mapping for Video Files

Canon has a Dust Delete Data process in its recent EOS DSLRs.  Essentially, this process records where the sensor dust is located on your camera’s image sensor and appends that information to each image file.  This file appendage is then interpreted by Canon’s Digital Photo Professional software and applied to the imported images in order to automatically remove dust spots from your images.  Canon says this is for RAW and JPEG images only – not H.264 video files.

The recent surge in video recording capabilities for DSLRs like the Canon 5D Mark II, Rebel T1i and Canon 7D have made it apparent that similar technology is needed for video files.  As far as I can determine, this technology is not available in-camera yet.  From my limited knowledge of video post-processing, it is possible to remove dust spots, hot pixels and the like from video files, but it is not an ideal solution and must be done in post-processing.  However, that is something that Canon aims to change.

A recent Canon patent application filed with the US Patent and Trademark Office reveals that Canon is working on such a solution.  USPTO Patent Publication No. US 2009/0213249 A1 states that the invention “has as its object to perform dust removal processing in moving image recording without interrupting moving image recording.” (See paragraph 0032.)

Canon Dust Delete for Video Files

In the new patent, Canon sets out the background of the dust mapping  process noting, “In still image shooting, even when the camera internally executes dust removal, the dust removal processing takes a long time, and when the buffer memory in the camera becomes full, shooting can be temporarily interrupted.”  (See paragraph 0031.)

The problem, of course, is that when you are shooting video, you cannot allow the dust removal processing to hog up the buffer.  That is the goal of the patent – to perform the dust removal process, while allowing the buffer to remain unhampered and continue to capture your 1080p HD footage.

Summary of the Canon Dust Delete Process for Video Files

1. Sensor captures the image

2. On-board memory stores dust mapping info

3. Buffer stores the image signal from sensor

4. Correction unit processes and corrects the image signal from the buffer and outputs the corrected signal

5. Control unit monitors the buffer based on the free area remaining and adjusts the appropriate amount of corrections to make

Here’s a brief quote from the patent that relays Canon’s characterization of the process.

After the start of moving image recoding, the image data stored in the frame memory buffer is sent to the dust removal circuit and undergoes dust removal processing. The dust removal processing is performed similarly to that for a still image except that it is executed for each frame using dust correction data (dust information) stored in the nonvolatile memory, instead of dust correction data recorded on the recording medium in association with image data. The dust-removed image data is encoded by the compression/decompression circuit, and temporarily saved in an encoded moving image data buffer in the memory.

. . . .

During moving image recording (live view), image data is written in the frame memory buffer at a predetermined frame rate, and thus encoding processing needs to be done to prevent the frame memory buffer from becoming full. However, the number of dust images to be removed by the dust removal circuit changes depending on the number and size of dust images included in dust information, and optical information acquired from the lens unit. For this reason, the dust removal time during moving image recording is not constant.

Moving image compression encoding processing executed after dust removal processing requires a different processing time depending on the spatial frequency of an object image and the magnitude of motion. Even the difference in processing time also influences the free area of the buffer memory.

(See paragraphs 0161-65.)

The big takeaway from the patent seems to be buffer management more so than dust mapping.  Canon already has the dust mapping technology in its bag; however, coming up with a way to implement dust mapping into each and every frame of a video file, whether it be 24fps or 60fps, is a clear challenge.  Canon’s patent does not completely solve this problem.  Rather, the new process attempts to manage the problem.

If the buffer is in danger of filling up, and therefore interfering with your video recording, the CPU will bypass the dust mapping process, encode the data normally and get it out of the buffer as soon as possible. Additionally, the process calls for two threshold levels.  After the baseline threshold for buffer usage is met, the CPU will perform a bit of a triage on the worst dust spots (taking large dust spots and those near the center of the frame as priorities) until the second threshold for buffer usage is met.  After that point is reached on buffer usage, the dust removal process ceases entirely until the buffer clears.

It’s not a perfect solution; however, it may mean that indie filmmakers can now justify purchasing those SanDisk Extreme Pro CF cards.

Remember, this is just a patent application at this point.  We have seen plenty of patent applications that looked cool sit on the sidelines and, to date, never come to fruition (e.g., Canon Iris Registration Biological Metadata).  This is pretty solid tech though.  I can definitely see the value in pushing this out to the marketplace sooner rather than later.