Someone stood by the ancient stone Monument as I approached in this a i Generated World things seem to just Change and shift at every moment I must Be quick as the oceans Roar hit me I Knew it was time to ask my vital Question Are you a Transformer because you look Optimus fine And moving on swiftly from the terrible Jokes to the AI video creation things Let’s have a look at some more videos There okay I’ve got some more videos on The screen so you can see the sort of Things that this deforum extension for The automatic 1111 web UE creates it’s a Script that makes 2D and 3D animations From stable diffusion it’s got lots of Features in there essentially you can Zoom in and out move up and down left And right and spin round and round lots Of different features in there to play With lots of different ways to create Videos I’ll be using the automatic 1111 Where Beauty which I assume you have up And running there are the installation Instructions just six steps run through Those and you will have your web UE There and ready to go a little note for The collab users if you’ve used the Deforum collab before it’s basically Exactly the same as that if you want to Run that collab locally so you have all The features I’ve got an entirely
Different video for that but in this one I’m going to be going through installing That extension the basic features on Those tabs some saving and loading Resuming from previous animations in a Slightly different way because I found That resume didn’t work some examples of What the defaults look like some videos There just so you can see what it does How to easily figure out what any of These options do because there are a lot Of options and if you can make it all The way through to the end of some bonus Tips including a little bit of live Video jockey action let’s dive straight Into this then the installation when You’re running your web UE if you have The listen or the share options enabled That can cause problems with the Extensions at the moment that may change In the future but if you have either of Those options for now just take them out That’s probably the easiest way to get To your extensions Tab and then you can Go to the available sub tab you just Click load from and you get this big List here the forums script is the one You want to install there’s loads of Others feel free to install those but I’m just focusing on this The Forum Script so go over and click the install Button I’ve already got it installed so It shows up on my installed tab here it May also get updated at some point in
The future so you can click check for Updates that will check for update Hopefully like mine yours will say Latest if it doesn’t then you can click Apply and restart UI and that will Update for you and restart the UI all Really nice and easy to get installed And running if you now go to the deforum Tab you will see this info and Links at The top we’ll just open that quickly but There is a lot of extra information in There I’ll shrink that back down again I’m going to go through these reasonably Quickly as you should be familiar with These from the text to image tab they’re All over there as well override settings Obviously overrides your settings from a Custom file pop your custom file in There you’ve got the width and the Height all these familiar options Restore faces high-risk fix which Doesn’t have a pop-up do note that your Sampler is there and then you’ve got the Seed for the sampler there so you’ll Probably be playing with those a little Bit Enable extras you’ve got the extras down There again same as from the original Tab steps to go along with your sampler Ddim ETA which you might have seen in Settings already and that applies when You’re using the ddim sampler and those Last three I completely ignore save Settings and save samples I also leave
Those ticked and one thing that you Probably will want to change each time Is this batch name that is basically the Name of the directory where you are Going to save your images relative to Where you’ve got this installed when it Does generate it it tells you in the Information at the bottom exactly where It’s saved anyway file name format time String index and prompt I leave it like That because that’s a fine bar name Format see Behavior you’ve got a number Of options in here you’ve got it fixed Random and schedule now they all do Something ever so slightly different so Itta is the seed will increment by one For each subsequent frame of the Animation fixed is obviously a fixed Seed random it’s a random seed and Schedule you can set your own schedule That covers that tab lots of things in There let’s go to keyframes now this is Where you’ve got your animation mode 2D 3D and video input the video input is Actually on the init tab where you Specify your video is a little bit later But those are your three main modes There Max frames number of frames in Your video border you’ve got replicate Or wrap wrap is a bit like in a video Game where when it Scrolls off the Bottom it’ll appear again back on the Top replicate is more like a sort of Smear now you’ve got all these other
Settings here like 2D and 3D settings Where you’ve got something that looks a Little bit strange and then you’ve got Something which looks a little bit Complicated and there’s a lot of them They go all the way down here until you Hit coherence These are all fairly simple and remember You can pop over any of these things I’m Hovering over angle and it tells me this Is a 2d operator to rotate canvas Clockwise anti-clockwise in degrees per Frame so if you’re not sure what any of These settings do you can just hover Over and it will tell you really handy Now these all have the same sort of Format Pretty easy to understand it’s the frame Number and then what the value should be Set to at that frame number in Brackets So you’ve got frame number colon and Then a value in Brackets and as you can See these are all the same even if some Of them do have maths and other things In there not to worry it’s it’s all Fairly simple and we’ll be zeroing this Out later getting rid of a few of the Complex looking things and that’s kind Of how we start the template now the Next thing we’ve got down here is Diffusion Cadence if you’re used to Disco diffusion Turbo This is the turbo it essentially just Skips some steps it diffuses a frame and
Then it’ll guess the movement which will Be nice and smooth and then it will Diffuse the next frame after that Depending on what you’ve got your Diffusion Cadence set to All of them work quite well I quite like Four two is also quite good one has a Lot of changes but that may be the sort Of video that you’re looking for Um eight is quite nice it’s really Difficult you’ll have to play with the Cadence just to see how smooth you want It and how quickly you want to generate The video and what the whole thing looks Like but four is a good starting value I Think uh 3D depth War being I tend to Leave that exactly the same I barely Ever play with that at all feel free to Change that yourself prompts much like Keyframes can also be scheduled there We’ve got 0 60 80 and 100 it changes Whoops changes the uh The Prompt as you Go along it’s got a beautiful durian a Coconut a banana and a Scenic Countryside as it says up here it Supports weights and negative prompts as Well after minus minus neg for the most Part that’s just going to be a Scenic Countryside with a weight of one and Then all that other complicated stuff is Just an example Let’s have a look at the next bit we’ve Got here which is the init tab this is Your initialization image in it if
You’re going to use in it you can tick In it and you’ll also need a strength in There of some value as well strength Zero is it doesn’t use the image at all And the strength one is it totally uses It that’s the strong as you can get the Image itself is specified there in the In its image path there it’s got a URL So it’ll download an image from the Internet but of course that could be a Local directory as well along with that You can use a mask if you take the use Mask box same thing mask file is where You put that and there are some other Options for that as well video input if You selected the video type this is Where you’ve got your video path there Video init path and videos also have Mask options as well so you can use mask Option and then use that resume from Time stream you can tick from that and Then you can carry on from a particular Time string that’s the time string in Your settings file which is also the First name of any of your file names of Your frames in there personally I found That that hasn’t worked I’ve tried to do A few things I’m probably doing it Completely wrong what I do to resume Instead is I just take the last frame of The video and I put that in as the inner Image We’ll take that out And there I’m using that and then I tick
The inner image and then put that Strength up quite High then it Carries On from that frame and that’s that’s Like a sort of resume I know it’s not Quite right if anyone knows how to Properly resume or if it really is Bugged then that would be interesting to Know video output sort of lives on its Own it’s got a different settings file As well for the most part I like to do 24 FPS and ffmpeg if you haven’t got Ffmpeg installed already on Linux you Can just apps install ffmpeg if you’re On Windows and you’re running in a Condor environment you can condra Install ffmbeg or you can install ffmpeg Any way you fancy if you’ve installed it A strange way and it’s not in the path You can just put the direct path to Ffmpeg in that location use manual Settings there you can tick and that Will let you split out the image path And the MP4 path so you can specify Where those go by default they just go In the same folder over here you see You’ve got the settings there and I’ve Also got the MP And all the frames I’m fine with them All being in the same directory I just Don’t bother with that okay so that’s a Quick summary of all those settings but What do they all do what do all these Defaults and things look like well let’s Have a look shall we let’s have a look
So I’ve got a number of different videos Here they’ve got this same sort of file Name format I’m using the default prompt Here and this is 2D although the last One is 3D I’m using a fixed seed for the First three and then it’s a random and Iter again the seed is 666 in that case And I’m changing the Cadence a little Bit in the first one I’ve got Cadence One the next one I’m using the full Steps in this stable diffusion here In settings there’s an option down here With image to image do exactly the Number of steps the slider specifies if You’ve got that ticked then that will Probably mess up because it’s the same As having a strength of zero all the Time I’ll show you what that looks like It may be something that you want and Then there I’m increasing the Cadence to Four so you can see the effect of the Smoothness here I’m using the ITA seed Option then the random seed option and Then finally 3D mode so let’s have a Look at these First one here this is that fixed seed And as you can see it’s perhaps not Quite what you’re expecting because it’s Not like the introduction videos at all It’s all gone a bit funny that is the Effect of the fixed seed there now if we Do exactly the same thing but with a Full number of steps you can see it’s Like a sort of flickery animation to
Start with and it holds it together Fairly well but then breaks out into That funny shape again as well let’s Have a look at the third one and this is With the Cadence so I’m smoothing that Out and we get the nice smooth animation There so it seems to sort of Bounce in a Little bit nicer and we get some bananas But then it’s it does start to lose it Still towards the end how do we stop it From losing it well that’s where the Itta seed comes in here we have it with The inter seed as you can see it’s it’s Changing quite a bit but it’s a nice Smooth animation it’s kind of keeping it Together but getting the Coconuts we’re Getting the bananas we’re getting the Durian fruit all that sort of thing it’s Looking nice that’s quite a good quite a Good set of settings here we’ve got the Random C the difference there between The random and the itta how much Difference does that make a fair amount Of difference as you can see it’s Changing slightly more dramatically as It zooms in certainly more noticeable on Those mountains we’re getting a lot more Mountain change going on there using the Random seed and if we have a look at This final one which is the 3D mode and In the 3D mode you can see it’s zoom in Quite fast there that’s the zoom of 10 Zooming in a lot lots of change it still Looks alright but it is zooming in quite
Fast that’s what happens if you run that You know you barely change anything Between fixed seed and the Cadence you Get all those different sorts of videos Okay how do you figure out what anything Does in there it’s it’s still all a bit Confusing I mean you understand that you Can go over a thing and you can hover Over it and you’ve got the pop-up and You can read it and it says 2D operator To rotate canvas clockwise Anti-clockwise but you still don’t Really understand what it means okay Then this is for you what we’re going to Do here here is we’re going to do a Couple of things first of all we’re Going to make it a little bit easier and Then once we’ve made it a little bit Easier we’re going to save those Settings and keep it like that as a base To test from so we have a look at the Run tab to start with most of these Defaults are absolutely fine I’ve got my Test name there I’ve got my seed Behavior is as itta that all looks good Don’t want to change very much in there Keyframes I’m going to keep it on 3D Zoom This looks weird I’m going to change This I’m going to change that to one Because that’s what the pop-up says is Static What else have we got in here This I don’t tend to use but you could
Zero that out as well if you want noise That tends to go along with the strength So those two are quite important The noise schedule If it’s very high will add a lot of Noise to it so it’ll get very very Grainy I tend to turn that down to zero If I’m increasing the strength quite a Lot the higher the strength the more the Image won’t change so with a really low Strength like 0.1 the image will change A lot with a really high strength the Image won’t change very much but then You could get that same issue as you saw With the fixed Seed where it starts to Go all a bit weird and blurry for the Most part 0.6 is a good one and 0.01 is Reasonable noise schedule for that Contrast I don’t tend to change guidance Scale identity to change those I don’t Tend to change diffusion between two and Four for my default I’m going to leave That on three for my default And 3D depth warping again I don’t Change prompts I get rid of those Prompts and I just put something in a Lot more simple because then it looks Easier let’s just pop that in there we Go now I’ve got some default prompts in It absolutely fine because that changes Per Thing video output I’ve got all of Those set to exactly how I want now I’ve Got everything set up how I want if you Want to set any other particular things
Now is the time to do it go to any other Tabs if you’ve got any particular Preferences then set those because we’re Going to save these that you can use as A base Here we are we’ve got this text box down Here I save that as nerdy rodent deforum Settings and let’s save those settings Those are saved the same with the video Settings I don’t bother changing it on The video settings because those are I Always want those to be the same for Everything and now I have those settings Saved you can load those in again and it Will set you back to this spot where You’ve got a nice less confusing prompt And all your keyframes are zero oh Forgot to zero out that translation Translation there as well let’s save my Settings again you can do this as you go Along as well sometimes when you’re Working on an animation you may want to Play with it a little bit and it’s like Oh well what happens if I do change this Angle here and this angle here you can Save your settings and if it doesn’t Work you can just load your settings and Often you’ll be changing many things at Once and you won’t have an easy undo Because you’ve changed so much stuff you Can save as you go along as well it’s Quite handy right so there we go now We’ve got a nice empty thing to go from Now
You can test now you can test Everything’s saved you can revert quite Easily if you want to test a particular Thing for example there I’ve got 3D Settings translation Zed so 3D settings Translation Z I just changed that thing To 0.5 Then I’ll know what that thing does Because I haven’t changed anything else I’ve just changed that one thing let’s Have a look at that let’s have a look at That Although in this case I have actually Changed one more thing I’ve changed the Prompt Uh because I want to have a prompt in There my prompt by default is a little Blank one that says please fill in the Prompt I have a portrait painting of a Woman that changes into a portrait Painting of a cute rodent at frame two Let’s have a look at this With the zoom 0.5 in 3D mode Okay interesting it’s zooming in quite Nicely And then turns into a rodent brilliant Although that’s quite interesting isn’t It because I had Portrait of acute rodent in it too and Yet it didn’t turn into a cute rodent Until quite a way through so that’s Something to watch out for is that it Will take a little bit of time to Diffuse those things especially if
You’ve got a very high Cadence because It will skip all those diffusion steps Now you have an understanding of how to Test any setting you can go back you can Reload your settings that’ll set it back To its zero State you can change another Thing so then you could change angle or You could change Zoom or you could Change translation X change any setting You like and you will see exactly what That one setting does all by itself now How do we make this even more awesome we Make it more awesome with maths don’t we Yes oh no okay all right here’s a little Bit on the maths it is rather simple by Using the Desmos graphing calculator so Here we’ve got an example from one of Those things I think that’s the zoom That it does by default that’s the zoom That it does by default so here we can Visualize what that Zoom actually is and How those values change you can probably Just about see that it’s a sine wave so It starts off at 1.02 because there it Is 1.02 plus the sign and then it goes Up and then it goes down and then it Goes up and down and up and down it’s a Little bit easier to see if you make This bigger There we go so as you can see the values Go up and down a t equals zero you’re Going to have that value and then at T Equals five you’re going to have that Value
Etc so that is how you get smoothly Changing value so you don’t have to Enter each keyframe one at a time you’ve Got one two three four that’s T is x in This x is T they are both exactly the Same thing let’s have a look at the Basic one basic one is just starting off Like that as you can see it goes up and Down if you want to keep it purely above The line then you can do it to the power 2 for example and that will keep it Above you can do it to the power of Three and it will look Wiggly there’s All sorts of things you can play with in Here you can times them together as well Say I want to times that by a half of X And we’ll we’ll get let’s say 1.02 Plus Oops Yeah man Last half of X and then we get you know A very very wiggly line that’s not it Doesn’t quite look like a sine wave at All it’s all sorts of strange things if You’re not sure what any of the Functions do that doesn’t matter there’s A show keypad down there you can click Functions it will give you a load of Extra functions a lot of them work in Python many of them don’t but if you Stick to sin and cos then you’re usually Going to be fine so you could change This to cause as well There you go and that’s that’s how it Does it if you want to make it bigger
You can multiply it as well so I could Say uh say five times there and then That makes it bigger you can shift it Along just change these numbers those Are your frames that’s T that’s how you Visualize how your values are going to Change over time using that graphing Calculator I’ve got another little Example here in case none of that makes Any sense say you want something to Happen at four seconds in your video and Your video is running at 24 frames a Second that means 96 frames is when you want that thing to Happen you may want a sine wave for Example And you may want it to go up and down on That 96 seconds We’ve got here if we do this one here Pop this in here Now I have to change pi to the actual Pi Symbol in here Where are we Pi there we go Okay now we’ve got pi divided by 96 Because we want we want it to happen on The 96 but times it by x x is T in Python as you can see there we go then We get over here to 96 it’s back down to Zero there we have the whole first half Of the wave happening at 96 and that’s All relative to your video because That’s when you want that thing to Happen you want it to do a thing until You get to 96 and you want 96 to be zero
Again there you go that’s that’s how you Work it out if you want different things That have nice smooth waves that happen In particular times it’s quite easy as Mentioned in Desmos there it’s X and Into Forum then you have to put T there Also note that Pi it you can’t just type Pi in here I had a go uh if you put 3.14159 that will do as Pi in deforum You could set just copy that in there in Here angle there you go that will be the Angle that will change it will start off At zero and then by frame 96 four Seconds later it will be back down to Zero again that’s how to maths by Looking at it it’s nice to see it’s nice To see the the visualization there and Also to click on it and see how the Numbers go so you can see exactly how Your animation is going to happen that Is more than enough maths let’s have a Look at this next test here This is my awesome test now in this one I used some maths so it’s got some maths In there so it’s a smooth up and down Curve I’ve also used an ancestral Sampler as well with a very low strength Let’s have a look at that one there he Goes it’s a little bit jittery because Of such a low strength it’s 0.25 on that One as you can see it it sort of works For that kind of Animation because it Looks like the rodent is sniffing There’s a little prompt change in there
He starts off in the woods and then Becomes cyberpunk so that’s how you can Smoothly go up smoothly go down using Those waves now some bonus tips for Those of you that made it to this point Those really smooth nice coherent Consistent videos that you’ve seen Uh you’re probably not going to get that With this uh expect things to change Over time many factors will influence Your final outcome have a look at all The different things try all the Different Samplers for example in here You’ve got an absolute boatload of Samplers and they are all ever so Slightly different some faster than Others some slower you get different Outputs with the same seed as well lots Of things to try in there do go with the Flow with your artistic Vision because Um it’s going to happen it’s going to Change things for you you’re going to Start even if you start out with an Initial image it’s going to change it to Something which you probably didn’t Expect things which have lots of Movement in a very good like in my Example video at the beginning I’ve got The waves crashing there other things I Like anything where you’ve got movement So crowds pictures changing in a gallery Which was you know you wouldn’t Necessarily have movement but it’s all Right if the pictures change things like
That have a have a think about also Sorts of stuff he can do now the really Really weird bit and fun bit let’s have A look at this the live video jockey Now I’m using the in painting mask and Uh the in painting checkpoint here if You haven’t got this little scroller at The Top If you go over the settings and Then in the Quick Settings list here I’ve added in paint Mask wait that’s how I got this little slider up here and That slider Works in real time so if you Click generate and you’re generating a Video and it’s happy going through you Can actually change the slider as you go Along you can drop it down to zero if You want wait for it to diffuse you can Drop it we can put it up again and you Can get some really strange effects Doing that there’s a little example Video happening now so there you go you Can do live in painting conditioning While you’re generating if you want to Sit there and play with a slider it’s Quite weird and of course one final Little tip don’t be afraid afraid to try All of the things you’ve got that Methodology for testing things test Every single thing I’ve also got a Couple more videos here here if you’re Interested in clicking on any of those This beginners one for pyty that goes Through a lot of the Motions because That’s what this was built on and
There’s another video for you to click On there if you’re interested in running The collab locally