Prompt Below
Are you interested in running the stable Diffusion version of dream booth that Train multiple Concepts in your own PC In under seven minutes of course you are So let’s dive straight in obviously There is a Google collab for this as Well so you don’t have to do it locally But I will be focusing on the local Edition you’re going to need Microsoft Windows or Linux of course with an Nvidia GPU unfortunately AMD stuff it Probably isn’t going to work for this You’re going to need at least eight gig Of vram for Linux and at least 10 gig of Vram for Microsoft Windows you’ll need Anaconda as well to manage your virtual Python environments just click download And then double click the exe or run the Shell script and that will get you Anaconda installed I do have a link for Microsoft Windows beginners that is just Appearing up there somewhere that you Can click on to get Anaconda installed You’re also going to need a compiler now On Microsoft Windows this is Microsoft Visual Studio I happen to use Visual Studio 2019 build tools because it was Already on there from something else but 2022 seems to be the latest Edition so You can download the free Community Edition and then select desktop Development with C plus plus and that Should get you going on Linux you just Run apt install build essential and that
Will get you you’ll see compile you’re Also going to need nvcc so there it is You go to developer.nvidia.com Cuda Downloads and pick the operating system Architecture version and install it up You’ll be fancy there I’m using the Windows local XE I download that and Then I double click on the XE and follow The on screen prompts there we go so now We’ve got all the prerequisites Installed you’re going to need some Images now ideally you’re going to need At least five images per subject a Subject being an object or a person or a Style or a pose whatever it is that you Want to train now ideally 512 by 512 but It doesn’t matter what size they are Because it will resize them Automatically you’re using a non-square Aspect ratio you may want to use the Minus minus Center crop option you’re Also going to need a diffusers stable Diffusion model not one of the original Checkpoint files This can be in a local directory or it Can just download it from hugging face The hugging face CLI which we will do Later and yes indeed the new Runway ml In painting model can be fine-tuned so Let’s create that new Anaconda Environment you do of course have Anaconda installed and your anaconda Prompt open I have already in created Mine so I don’t need to run condo create
But you will if you don’t have one Already and then I just run conda Activate diffusers or whatever name You’ve picked for it you’re then going To install pytorch and git Now I’m using pytorch 1.12.1 here I know 1.13 has just come out but I haven’t Done any testing with that so um yeah Have a go and let me know see if it Works with 113. uh condo install git if You don’t have git already most people Do have git on Linux you can just apt Install git and that will get you get Forever that you can use everywhere Again Triton is a Linux only thing still Um you may be able to get it working on Windows but good good luck with that so Pip install minus U minus minus pre Triton if you are on Linux xformers this Now works on Microsoft Windows this is The bit that didn’t work before so you Can download this uh if you’re using Linux you can just counter install Xformers So that is nice and easy if you’re on Microsoft Windows like I am here you’re Going to have to download it I Downloaded all the GitHub stuff into a GitHub directory so git clone that then Once you’ve get cloned it CD into that Directory and then update it yes lots of Commands to copy and paste again all Down in the description click the little Arrow then more and you will see all
Those instructions there and a link to The paste bin document as well so it Updated the sub module then pip install Minus r requirements.txt and then we’re Also going to install Funk torch ninja And bits and bytes now there’s another File that you’ll need to edit inside the Exformers directory there so xformus Underscore init underscore.pi and by Default you will have is Funk torch Available false but we’ve just installed It so it’s true there we go Brilliant Now there’s a number of environment Variables you need to set on Microsoft Windows I didn’t need to set these on Linux so if you’re on Linux you can just Ignore them so I set Force Cuda equal to One I also set torch Cuda Arc equals 8.6 If you’re not sure which architecture You’ve got you can go over to the Cuda Page on Wikipedia and that will show you There I’m using a 3090 which is 8.6 so Find your card and that will give you The magic numbers I’m also going to set two divisible Devices to zero because I only have one GPU there’s just a quick Echo just to Show me that everything is all right There we go yes the environment variable Did take it is set to 8.6 then you can Do pip install minus E Dot and that will Install exformers for you now that will Take about 10 minutes to build obviously
I’ve built it already so it’ll be all Nice and quick here if you want to Rebuild it then you can do python Setup.pi clean and also python setup.pi Develop so that will that will clean out The build directory basically it deletes The build directory and recompiles it Now you will get a little note there That is detected Cuda version 11.8 Because that’s the one we installed up Here earlier uh but minor uh version Mismatches don’t really matter and it Will still compile I know we installed 11.8 and 11.6 and it all seems a bit Confusing but don’t worry about it so Next we’re on to the uh Shivan diffusers So we git clone this again as well so I Go back to my GitHub directory git clone Diffusers there we go and then I can CD Into diffusers and then examples and Dream booth And there I am just going to do the PIP Install exactly like it says on this Page here so if you scroll down here You’ve got installing the dependency so I’m basically just running those two Commands apart from I’m running clip Install minus r requirements.txt without The u because High torch 1.13 has just Come out So I’m just making sure that I’m Installing all the versions that I know About so if you’ve got a local model it Doesn’t matter if you don’t need to do
Hugging face CLI if you haven’t got a Local model you will need to do hugging Face CLI to download and there you just Go to that it will give you a little Link and you go over to hugging face CLI It’s got your access token so when it’s Asking you for your token you click copy There that will copy your token then you Go over to Anaconda you press the right Mouse button that will paste it in then Press return it won’t appear on your Screen but don’t worry and that will log You in Once it’s logged you in it’ll give you This option here as well if you want to Save your credentials which is probably Worth doing now if you’re on Linux you Can pip install deep speed at this point But of course Microsoft deep speed is Not available on Microsoft Windows so I Won’t be using it you are next ready to Run accelerate config now I’m going to Basically use all the default options on This so zero zero do I want to use the CPU only no now if you were using deep Speed and you want to try and you know Offload everything a bit like it says in Here so when you scroll down here you’ve Got training on an 8 gig GPU you would Select yes to this bit but I’m I’m not Because it doesn’t work on Windows so no Still want to use fp16 yes I do fb16 Brilliant so we’ve done the accelerate Config and now we are pretty much ready
To do it we’ve done our hugging face CLI We’ve logged in now we’ve got all these Different scripts here so what I’ve done Is I’ve basically taken launch.sha This script here And I’ve changed this into a batch file So there’s the backshot as you can see It’s exactly the same pretty much uh Apart from it’s been windows defied so It’s got Echoes and carrots and all Sorts of things in there so it’s Essentially that script so this is what You’re going to be running it’s a very Long command I’ve also added in this num CPU threads because that’s for my CPU You may not have eight six maybe a Better number or four entirely depends On your CPU it will tell you the first Time you run it and it’ll say hey uh I’ve I’ve run num CPU threads per Processor and I’ve set it to this so you Can just put that before your train Dreambooth command when you’re doing Accelerate launch Okay so now the first option here we’ve Got is the pre-trained model so this is Where it’s going to download the stable Diffusion version 1.5 a brand new shiny One from runwayml if you’ve got a model Downloaded already then obviously you Put your Directory and wherever your diffusers Model is already I haven’t got one well I have but it’s
In the cache so I just leave it like That the entire time it also uses the New vae as well you can pass any other Vae there so that’s using the stability AI the new vae there output directory That’s where you’re going to save your Models with prior preservation yes Please Seed now changing the seed can actually Give you a variety of different outcomes So you know if you’ve trained one model And you’re like a bit I don’t know then Try changing the C that might give you a Completely different result uh trained Text encoder there’s also a new Option It does use a little bit more vram Though so if you’re if you’re struggling With the with memory take out the train Text encoder mixed Precision FB 16 nice And fast now this has a much lower Learning rate when you’re training the Text encoder it’s now one E minor six if You’re not training the text encoder Then do put that back to five E minor Six Okay uh everything else is pretty Standard so number of class images 50 Sample batch size that’s four by default But if you’ve got a very low vrm GPU you May want to put that down to one Max training steps uh 800 so that’s when It will finish save interval now if you Want to save lots of checkpoints along The way so say for example I wanted to
Save one halfway through training then You can set save interval to half the Number of steps now I only want one Checkpoint at the end because reasons so I’m going to set my save interval the Same as Max train steps now also if you Want to automatically convert your thing Into a checkpoint then just add the the Scripts convert diffusers so if we go Back here See in here Scripts Uh convert diffusers to original stable So you just want to run that basically And it’s got a couple of options It’s got the model path so that that’ll Be your output directory there so there And then checkpoint path that’s where You want to save it to so you could put That you could set that to your stable Diffusion uh models directory and it’ll Just output straight into there if you Want to or maybe you’ve got a central Model repository where you put them all But yeah but checkpoint path is the Output model path is the input basically And then we’ve got one final thing here Which is this new bit here Concepts list What is that well this is where you Train them multiple or single Concepts So Here is an example of a Concepts list It’s got them in here as well so Examples dream Booth so that is the Default concept list it’s just got one
Thing in it but you can build these up Because it’s just a Json file so that’s What it is if it’s one face so instance Prompt so this is the face that I’m I’m Trying to train here so Instance is basically what you’re Training class is from the class Preservation uh so that’s Sort of what it’s overwriting uh Training data that’s the there’s the Images that you created earlier all These five twelve five by five twelve Ones that you’ve got saved and that They’re all nice and ready to go yeah And then you’ve got the class directory So It’s probably best to make your own Class images and put them in there so if You’ve got a cartoon face go into Automatic 11 11 for example generate a Car a cartoon face put it in your Classes directory that’ll probably be a Whole lot better than the ones that they Generate but yeah anyway so let’s have Just have a quick look at those so Training for example there we’ve got Bell face so there I’ve got some Examples from there and then big bunny Got some examples from there And then elephant’s dream got some Examples from there for example so They’re just they’re just all square Pictures and they’re as an example of Some William Morris pictures so I’ve got
All those things so how do I do them all At once there we go like that so you Just you just keep adding them how many Things do you want to add you can add Stars you can add faces you can add Poses whatever you want you just keep Putting them in there so it’s exactly The same as the single version but You’ve got loads of them so don’t forget To put a comma at the end there comma at The end there and your very last one Just skip the comma because Json so this Is exactly the same as one but we’ve got Multiples so instance prompt there you Go that’s what I want to create now here I’ve I’ve put a little arrow at the Beginning just because that’s you know It makes it a really obvious token that That is the thing that I’m that I’m Training uh it doesn’t have to be that You know you can have it like that or Whatever you fancy whatever you fancy Class prompt is a style so it’s it’s not The William so the William Morris style As opposed to just a style training William Morris pictures so there’s There’s all the training directories Class I haven’t actually generated them Yet but I’m creating style samples so That will just make a style in the style Samples directory and then rinse and Repeat so there Intel face which is a 3D Face training it’s the syntel face back Over there Training Center face there
I’ve got the sintel face so I’ve got 16 Images yeah okay so you’ve got the You’ve got the gist of things there so That’s how you train multiple Concepts Basically just keep copying and pasting These so if I wanted to add another one For example I’d put comma on the end There oops Paste that in there and then I just Start start typing whatever whatever the New thing is but I’m just going to keep Three in there for now so there you go That’s lots and lots and lots and lots And lots and lots of options uh now There are a few more options in there as Well so you can obviously uh Run with minus minus help and that will Show you all the options so if we don’t Do this try and dream boot Oops but then Train Dreamboat minus minus help and This will give you a whole screen full Of options so do check all of those you Know if you want to scale the learning Rate or if you want to change the log Interval or if you don’t want to Cache The latents and all that sort of stuff So there there are a wide variety of Different options that you can use okay There we go so that will take about Seven minutes by default on the uh on That one on the 800 and the 800. now Another thing when you’re training Multiple Concepts is that you probably
Will want to increase the steps so here We’ve got the uh save the chain steps There 800 the the multiple Concepts I Did that at 1200 so I I say 800 for the First one and then I basically added 200 For every subsequent thing in there Whether that’s a good number or not I Don’t know but it came out all right it Came out okay so let’s have a quick look At this shall we yes all right let’s Switch over to automatic 1111 and we’ll Have a look at what these models Actually produced So let’s try these things individually To start with we’ll start off with the William Morris style let’s see what that Looks like there you go that’s that’s Not too bad that’s pretty good that does Indeed look like the William Morris Style let’s try the syntel face as well Just throw some negative prompts in here Just for fun too there we go let’s look At the syntel face yeah That’s not bad That’s pretty good I do like that since All of this seems to be made up I don’t Think That that was ever an outfit that they Wore okay let’s have a look at the Elephant’s dream face as well completely Different face oh awesome awesome that’s Come out very well I like that okay so There you go that’s it’s got all three Of them in there
Even with multiple noses even with Multiple noses and can you style them Can you style them let’s find out shall We let’s find out okay so let’s have uh The elements dream face in an anime Style can we get an anime style elephant Stream face yeah we can just throw some Random seeds in there as well so it’s Got another guy in there but that that Is definitely him in an anime style That’s pretty cool that is pretty cool I Do like that all right can we do the Same with the syntel face can we have an Anime syntel face will it be completely Different yes it is there we go we’ve Got the trademark hair That’s that’s pretty good that is Definitely an anime version of that okay So oh there’s a real version there’s an Anime cosplay Center That’s pretty good all right so how About can we use the William Morris Style on different things so can we have William Morris mug there we go got a William Morris mug that’s absolutely Fine can we have a vars can we have a Bars with that on as well can we get all Sorts of different things going yes we Can we can put that style on anything we Fancy can we put that style on something Else we trained like the elephant’s Dream face sort of social faces don’t Normally have styles but you can mix Them in together so there you go that’s
Three things all at once trained really Really quickly on your own PC looks Fantastic but don’t forget to click on One of these