It is currently Tue Jan 22, 2019 12:54 am




Post new topic Reply to topic  [ 15 posts ] 
 New de-flickering filter for Avisynth 
Author Message
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post New de-flickering filter for Avisynth
Hello Everybody

Recently I've finished new de-flickering filter for Avisynth. I found there is a few on the market but none of them satisfied me completely. So I've decided to create new one.

The reason of creating this filter was simple - the existing flicker-removing filters sometimes do not work for Time Lapse photography well. Most of them are designed to remove flicker for old movies projected and then recorded from the projection screen at a different frame rate. During my short investigation I found that open source filters use "average frame luminosity" to compare the frames. The "average frame luminosity" is only one value between 0 and 255. Using only one value to evaluate frame luminosity seems for me like using "average temperature in a hospital including morgue" and trying to compare it to another hospital. Will it work well? I do not think so.

This filter is created especially for Time Lapse photography needs, so might be someone will found it useful.

http://www.zhitenev.com/avisynth/TimeLapseDF/

---
Denis


Tue May 25, 2010 7:17 pm
Profile
User avatar

Joined: Wed Apr 21, 2010 11:26 am
Posts: 235
Location: Germany, Munich
Post Re: New de-flickering filter for Avisynth
Welcome Dennis,

great first post and even better work!

Believe it or not, but I was just about to ask over doom9.org for an appropiate filter or script for timelapse deflickering.
I have not had time to play around with it, but here are some observations I made from watching the demos and reading the description:
Quicktime sucks! Please do yourself a favor and use x264 to encode your demo files. It totally undermines your good work. If you have questions about it, feel free to drop me a PM - I am not an expert, but I should be able to give you some help with it. This file seems to be corrupt, at least I cannot open it.
Does your filter work globally or is it able to work locally too? Examples for necessary local deflickering would be shadows from moving clouds on the ground, or flickering from artificial light sources in night timelapses. Perhaps some kind of kind of fading (in and out) would be suitable for local deflickering?!
Your filter generally seems to promote banding (could be Quicktime too...), is there anything you can do about it? It should be possible to compensate it with Gradfun2db, or a script like Gradfun2dbmod, but of course avoiding banding is always better than removing it afterwards.

One more thing:
Please compile a 64bit.dll of your filter, I totally switched to 64bit Avisynth, to process my timelapse files.

Best Regards
David

_________________
“Smooth is how we do it” (Ricardo Tubbs, Miami Vice)

Vimeo <<->> flickr


Tue May 25, 2010 9:29 pm
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
David:

Quote:
Does your filter work globally or is it able to work locally too?

I quite not understand that do you mean working locally. At the moment the filter changes a whole frame. How do you imagine 'local' work? how can you define the working area for it?

Quote:
Please compile a 64bit.dll of your filter, I totally switched to 64bit Avisynth, to process my timelapse files.

I've compiled x64 version and put it onto website. However I can't test it by myself - still on 32bit. It would be really great if you (or someone else) test it and let me know the result.

Quote:
Quicktime sucks!

Also, I've uploaded H.264-encoded video as well, the links are updated.

Quote:
Your filter generally seems to promote banding (could be Quicktime too...), is there anything you can do about it?

Yep, the filter is definitely promotes sort of 'banding' after 5-6 seconds of processing. Unfortunately even my huge programming experience does not help with video processing specifics. But, I'm working on it. :-)
I gonna ask guys at Doom9 what I can do with it, but will be able to post to theirs forum only 5 days after the registration.

---
Denis


Wed May 26, 2010 6:51 pm
Profile
User avatar

Joined: Wed Apr 21, 2010 11:26 am
Posts: 235
Location: Germany, Munich
Post Re: New de-flickering filter for Avisynth
Hi Denis,

thanks for the 64bit.dll.

Is there a specific reason why the filter only supports YUY2? Basically every important filter in Avisynth works in YV12 (only), which is basically the same, but the chroma planes (which are not affected at all if I understood it right) only have half the resolution. The necessary conversion from RGB to YUY2 to YV12 costs procesing time and can affect quality (though probably both effects are negligible). If its not too much of a hassle support for YV12 would be preferred.

I have toyed around with it a little bit, and I have not encountered any problems so far.

I found a faulty line in your documentation (errors bolded):
TimeLapseDF(mode="AVG",gradient=true,area="10,10,500,200") would be the working command line.

askden wrote:
I quite not understand that do you mean working locally. At the moment the filter changes a whole frame. How do you imagine 'local' work? how can you define the working area for it?
With locally I mean, that the filter only changes certain parts of the frame, not the luminosity of the whole frame.
Example:
Take fast moving clouds in the top of the frame. If shot in M mode, they are not flickering, but on the ground you have dancing shadows where the sun comes through that flicker a lot. In this case changing only the area where the dancing shadows are would be better instead of changing the luminosity of the whole frame.
Another example would be lights in a nightshot. The luminosity of the sky (usually) will not be affected by single artificial lights like houses, street lights, etc.), but the light soorces itself will flicker a lot if switched on/off.
Do you know what I mean?
From reading your documentation again, the command "area" seems to do that, is that correct?

Have you had a look at the Avisynth standard style of presenting filters and their functions? They may not be pretty, but once you got used to it, they are easy to read for Avisynth users.
Examples: 1, 2

Quote:
Also, I've uploaded H.264-encoded video as well, the links are updated.
Quicktime is h.264 too, it is just a very crappy implementation of it.

Best Regards
David

_________________
“Smooth is how we do it” (Ricardo Tubbs, Miami Vice)

Vimeo <<->> flickr


Thu May 27, 2010 12:43 am
Profile
User avatar

Joined: Mon Nov 16, 2009 4:36 pm
Posts: 387
Location: Australia
Post Re: New de-flickering filter for Avisynth
Joachim Buambeki wrote:
Quicktime is h.264 too, it is just a very crappy implementation of it.

David, as an aside what implementation would you recommend as the best? I'm struggling with banding myself at the moment and have tried the Quicktime (Apple) H.264, and MainConcept H.264 in Premiere Pro (+ same in Quicktime Pro) but haven't managed to remove it...even after tweaking all the relevent options and racking up the Bitrate. My source footage is coming out of After Effects in lossless animation format in pristine form. After H.264 I'm seeing banding in the sky (moonset scene).

_________________
vimeo


Thu May 27, 2010 6:03 am
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
David:

Joachim Buambeki wrote:
Is there a specific reason why the filter only supports YUY2?

I've just uploaded new version which supports YV12 as well. I've tested it a little bit -- seems working :-)
Joachim Buambeki wrote:
I found a faulty line in your documentation (errors bolded):

Thanks, fixed
Joachim Buambeki wrote:
With locally I mean, that the filter only changes certain parts of the frame, not the luminosity of the whole frame.
Example:
Take fast moving clouds in the top of the frame. If shot in M mode, they are not flickering, but on the ground you have dancing shadows where the sun comes through that flicker a lot. In this case changing only the area where the dancing shadows are would be better instead of changing the luminosity of the whole frame.
Another example would be lights in a nightshot. The luminosity of the sky (usually) will not be affected by single artificial lights like houses, street lights, etc.), but the light soorces itself will flicker a lot if switched on/off.
Do you know what I mean?

Yes, I understand that.
Sure it is possible to implement in theory, but filter should know which areas to meter/change. What if the area's shape is not rectangular? And what to do with the area borders. The "corrected" area will definitely stand out against a background. Well, it might be a sort of mask, with gradient borders, or something like this. There are many questions about "partial" correction, unfortunately.

Joachim Buambeki wrote:
From reading your documentation again, the command "area" seems to do that, is that correct?

No, it is the luminosity metering area only. Sometimes it is better to meter only piece of frame (e.g. without moving parts) to get more correct luminosity diff.

Thanks for your questions anyway :-)

---
Denis

_________________
---
Denis


Thu May 27, 2010 7:47 am
Profile
User avatar

Joined: Wed Apr 21, 2010 11:26 am
Posts: 235
Location: Germany, Munich
Post Re: New de-flickering filter for Avisynth
colinmlegg wrote:
David, as an aside what implementation would you recommend as the best?
I sent you a PM. :-)

askden wrote:
I've just uploaded new version which supports YV12 as well. I've tested it a little bit -- seems working :-)
Excellent, thanks!

Quote:
Yes, I understand that.
Sure it is possible to implement in theory, but filter should know which areas to meter/change. What if the area's shape is not rectangular? And what to do with the area borders. The "corrected" area will definitely stand out against a background. Well, it might be a sort of mask, with gradient borders, or something like this. There are many questions about "partial" correction, unfortunately.
I might have some ideas about this, but I don't have a clue how to translate that into a working algorithm or even code. I have basic scripting skills in Avisynth, maybe I can figure out something. :? If there is interest for your filter, you will get answers from the real wizards over at doom9 anyway.Sadly often good ideas do not get the response they deserve, because there is not enough interest for a topic that is very specific.

Quote:
Thanks for your questions anyway :-)
No problem, I am full of ideas and questions about everything. :lol:


I will have a look into the banding issue tomorrow, perhaps my observations can help you to eliminate the problem.

Best Regards
David

_________________
“Smooth is how we do it” (Ricardo Tubbs, Miami Vice)

Vimeo <<->> flickr


Thu May 27, 2010 9:21 am
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
Joachim Buambeki wrote:
I will have a look into the banding issue tomorrow, perhaps my observations can help you to eliminate the problem.


David, please update your DLL - I put a new version with small issue fixed. It seems the result now is more correct.

---
Denis


Fri May 28, 2010 3:00 am
Profile
User avatar

Joined: Wed Apr 21, 2010 11:26 am
Posts: 235
Location: Germany, Munich
Post Re: New de-flickering filter for Avisynth
Hi Denis,

I think I found a a good example for the banding issue. It seems the get worse the darker the video becomes.
Every picture was filtered using the 32bit version of TimeLapseDF because making previews is much easier with AVSPmod (an Avisynth preview editor that supports only 32bit at the moment).

This is a raw frame from my video posted here. The raw frames do not contain a severe amount of flicker, but since the banding is so severe I thought it is still a good example (especially since the filter should not alter the picture at all if there is no flicker). Please note that I additionally denoised the video and used deflicker() before I uploaded it to Vimeo., so the frames cannot be compared absolutely.
Every picture was resized from 3k with:
Code:
Spline36Resize(1280,720)



Code:
NO filter:
Image

Code:
TimeLapseDF(mode="AVG", gradient=false, info=true, showarea=true)
Image

Code:
TimeLapseDF(mode="AVG", gradient=true, info=true, showarea=true)
Image

Code:
TimeLapseDF(mode="CDF", gradient=false, info=true, showarea=true)
Image

Code:
TimeLapseDF(mode="CDF", gradient=true, info=true, showarea=true)
Image


I tried only reading the luma from the sky with area(), but the results were not better nor worse - just different.

Keep up your good work, it is really appreciated!


Best Regards
David

PS: Please include the version in the filename, it helps keeping track.

_________________
“Smooth is how we do it” (Ricardo Tubbs, Miami Vice)

Vimeo <<->> flickr


Fri May 28, 2010 8:15 am
Profile
User avatar

Joined: Wed Apr 21, 2010 11:26 am
Posts: 235
Location: Germany, Munich
Post Re: New de-flickering filter for Avisynth
Hi Denis,

I have another proposition.
What do you think of averaging for the chroma channels?
Take a sunset for example:
It will not only contain luminosity flicker from shooting in AV mode, it will also have frames that have a different colour temperature than the surrounding ones. When averaging these frames, one could stabilize the colour fluctuations present in the clip.
From my basic understanding the (probably slightly modified) "AVG" mode would be suited best for this task. A problem would be moving objects. I do not have another idea than using a motion mask for it. Even that should be problematic, since timelapse footage often is not fluid, because it was not shot with a 180 degree shutter speed.

Best Regards
David

_________________
“Smooth is how we do it” (Ricardo Tubbs, Miami Vice)

Vimeo <<->> flickr


Sat May 29, 2010 7:57 am
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
David,

I have an idea how to eliminate the banding issue, will see if I would be able to put it into code :-)

---
Denis


Sat May 29, 2010 8:56 pm
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
David,

I've just put new version to my website - it is working a way better than previous one. With my samples it is working just perfect. No more 'pixelization' or other color degradation.
Joachim Buambeki wrote:
Sadly often good ideas do not get the response they deserve, because there is not enough interest for a topic that is very specific.

Will see how it will go anyway. I've created it firstly for myself, but like to share the results with others.

Joachim Buambeki wrote:
PS: Please include the version in the filename, it helps keeping track.

Usually I keep updated DLL version. You can see the DLL version in "Windows Explorer", Right-click_on_the_file>Properties>Version.

_________________
---
Denis


Sat Jun 05, 2010 4:36 am
Profile

Joined: Sun Aug 22, 2010 12:20 pm
Posts: 2
Post Re: New de-flickering filter for Avisynth
i have tried your filter out

the gradient luminosity is a good idea, but it is a linear function from beginning to end, and it messes the colours up if the scene does not also have a linear brightness change
for example, a sunrise or a sunset where the brightness increases quickly near the beginning, or decreases quickly near the end

is there a way for it to do a sort of weighted average over a number of frames, so that it follows the scene's natural brightness changes?

also, i am using the x64 version... the MFR mode seems to be extremely slow, i was testing with CDF just now and it was giving me 12fps, with MFR it was 0.24fps ... 50 times slower, and my cpu is not being used alot either


Sun Aug 22, 2010 12:24 pm
Profile
User avatar

Joined: Tue May 25, 2010 7:01 pm
Posts: 8
Post Re: New de-flickering filter for Avisynth
carmatic wrote:
the gradient luminosity is a good idea, but it is a linear function from beginning to end, and it messes the colours up if the scene does not also have a linear brightness change
for example, a sunrise or a sunset where the brightness increases quickly near the beginning, or decreases quickly near the end

is there a way for it to do a sort of weighted average over a number of frames, so that it follows the scene's natural brightness changes?


Actually the MFR mode was designed to resolve this linear gradient problem.

It is very strange to heard about such a slow speed it working. Yes, it has a slow start as it works in two-pass mode. After the initial pause, on my quite old laptop it gives me 2-4 frames per sec in 32 bit mode.

_________________
---
Denis


Mon Aug 30, 2010 1:35 pm
Profile

Joined: Sun Aug 22, 2010 12:20 pm
Posts: 2
Post Re: New de-flickering filter for Avisynth
the speed i mentioned is after the pause, i could see in windows task manager that the cpu is not being loaded at all, and i could see in resource monitor that the disks have almost no activity either


Tue Aug 31, 2010 5:44 pm
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 15 posts ] 


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB © phpBB Group.
Designed by Vjacheslav Trushkin for Free Forums/DivisionCore. pozycjonowanie