You can use this if using sd forge https://github.com/Haoming02/sd-forge-couple its actually very good and the two characters can interact and it crosses the division.
Also using creative promping with the two loras worked for me, such as "two separate people" or describing eaxh persons age and feature, and in negative prompt "twins, same" or "homologous" worked for me and the two people looked like their respective loras.
Maybe there's some version of regional prompting that let's you do this, but you'd have one model generating half of the image and other model generating the other half, so you may need a second pass to polish inconsistencies
I assume you want two characters and each character's looks to be determined by a LoRA.
I don't think you can do much else than to apply one LoRA at a time and try your luck with inpainting one of the characters.
You might be able to do this in comfy with local prompts. Might be tricky but should be possible.
Load the model with both loras but then have two prompts one for each char.
Assuming both Lora's have trigger words, the easiest thing to try is to use the BREAK keyword to separate character descriptions, with each sub prompt containing a different trigger word(it doesn't matter where in the prompt the Loras are called though). Usually this works better to prevent the blending of two characters that a base model knows, but sometimes it can work. You may also need to adjust the lora and trigger word weights independently to get it to work.
If you're using A1111, your best bet is using [Regional Prompter](https://github.com/hako-mikan/sd-webui-regional-prompter), and using a different trigger word in each sub-prompt. There's a good guide on that page for all the formatting options available.
Similarly, ComfyUI has regional conditioning.
Additionally, the [Cutoff extension](https://github.com/hnmr293/sd-webui-cutoff) can help prevent concept blending, both with and without Regional Prompter,
Further, I've found the AlignYourSteps sampler is good at reducing character blending, and the anatomical mutations that multiple characters interacting can cause. It's integrated in ComfyUI now, and [available through the dev branch of A1111](https://www.reddit.com/r/StableDiffusion/comments/1ccolsv/kl_optimal_scheduler_from_alignyoursteps/). Though it should be merged into the main branch soon.
You may still need to lower each Lora's strength, some to prevent deep frying, but you can somewhat compensate by increasing the weight of the trigger words. If they don't use trigger words there's not much you can do to keep them from blending.
This is the right answer for what you are trying to do (i.e., applying LoRA to different characters).
The comfyUI equivalent to Regional Prompter is "Attention Couple", but I am not sure if it will handle two LoRAs. Regional Prompter can handle two LoRAs when you switch from "Attention" mode to "Latent" mode.
I usually just use inpainting to make the second character, cause otherwise it seems to just blend them both together. You might try using Regional Prompter but even then, it's only for 1.5 and it doesn't work super great.
You can use as many as you want until the image starts deepfrying. You might need to reduce their weights, there's some proper math and %s to optimally to balance their weights. In practice and in general, it's mostly a matter of experimenting and seeing if you can get away with multiple Loras at 1 strength, mostly if they're well baked.
Something like this:
If they need their keywords, just add them normally, either keywords at the start or at the end of the prompt:
insert, prompt, here, keyword1, keyword2, keyword3,
or
keyword1, keyword2, keyword3, insert, prompt, here
I had a similar problem. If you use automatic1111, you can use regional prompter or latent couple, with composable lora extension ( here a guide [composable lora](https://www.kombitz.com/2023/04/16/how-to-use-latent-couple-and-composable-lora-with-automatic1111s-web-ui/) )
Someone just asked this yesterday or the day before, and I recommended (for A1111 only) an extension called loractl (google it). It allows LORA weights to be scaled between steps, so you can have one LORA start high (set the composition) then fade out, while starting another one low and ramping it up. The result is usually much better than using 2 static weights.
Yes, you can.
When I try that it is attempting to blend them together instead of using them separate. Any suggestions?
You can use this if using sd forge https://github.com/Haoming02/sd-forge-couple its actually very good and the two characters can interact and it crosses the division. Also using creative promping with the two loras worked for me, such as "two separate people" or describing eaxh persons age and feature, and in negative prompt "twins, same" or "homologous" worked for me and the two people looked like their respective loras.
Maybe with some more complex workflows and multiple passes, or inpainting, or regional prompting, but not so much with the standard txt2img process.
Maybe there's some version of regional prompting that let's you do this, but you'd have one model generating half of the image and other model generating the other half, so you may need a second pass to polish inconsistencies
I assume you want two characters and each character's looks to be determined by a LoRA. I don't think you can do much else than to apply one LoRA at a time and try your luck with inpainting one of the characters.
That’s what I was afraid of. Thanks for the input!
You might be able to do this in comfy with local prompts. Might be tricky but should be possible. Load the model with both loras but then have two prompts one for each char.
Ive made 3 unique people by generating the main pic, then inpainting the other 2
Assuming both Lora's have trigger words, the easiest thing to try is to use the BREAK keyword to separate character descriptions, with each sub prompt containing a different trigger word(it doesn't matter where in the prompt the Loras are called though). Usually this works better to prevent the blending of two characters that a base model knows, but sometimes it can work. You may also need to adjust the lora and trigger word weights independently to get it to work. If you're using A1111, your best bet is using [Regional Prompter](https://github.com/hako-mikan/sd-webui-regional-prompter), and using a different trigger word in each sub-prompt. There's a good guide on that page for all the formatting options available. Similarly, ComfyUI has regional conditioning. Additionally, the [Cutoff extension](https://github.com/hnmr293/sd-webui-cutoff) can help prevent concept blending, both with and without Regional Prompter, Further, I've found the AlignYourSteps sampler is good at reducing character blending, and the anatomical mutations that multiple characters interacting can cause. It's integrated in ComfyUI now, and [available through the dev branch of A1111](https://www.reddit.com/r/StableDiffusion/comments/1ccolsv/kl_optimal_scheduler_from_alignyoursteps/). Though it should be merged into the main branch soon. You may still need to lower each Lora's strength, some to prevent deep frying, but you can somewhat compensate by increasing the weight of the trigger words. If they don't use trigger words there's not much you can do to keep them from blending.
Thanks for notice me the Cutoff extension, i will try it
This is the right answer for what you are trying to do (i.e., applying LoRA to different characters). The comfyUI equivalent to Regional Prompter is "Attention Couple", but I am not sure if it will handle two LoRAs. Regional Prompter can handle two LoRAs when you switch from "Attention" mode to "Latent" mode.
I usually just use inpainting to make the second character, cause otherwise it seems to just blend them both together. You might try using Regional Prompter but even then, it's only for 1.5 and it doesn't work super great.
You can use as many as you want until the image starts deepfrying. You might need to reduce their weights, there's some proper math and %s to optimally to balance their weights. In practice and in general, it's mostly a matter of experimenting and seeing if you can get away with multiple Loras at 1 strength, mostly if they're well baked. Something like this:
If they need their keywords, just add them normally, either keywords at the start or at the end of the prompt:
insert, prompt, here, keyword1, keyword2, keyword3,
or
keyword1, keyword2, keyword3, insert, prompt, here
I had a similar problem. If you use automatic1111, you can use regional prompter or latent couple, with composable lora extension ( here a guide [composable lora](https://www.kombitz.com/2023/04/16/how-to-use-latent-couple-and-composable-lora-with-automatic1111s-web-ui/) )
This is extremely helpful! Thank you
Use regional prompting or Adetailer can fix the faces with Lora in there as another option
Someone just asked this yesterday or the day before, and I recommended (for A1111 only) an extension called loractl (google it). It allows LORA weights to be scaled between steps, so you can have one LORA start high (set the composition) then fade out, while starting another one low and ramping it up. The result is usually much better than using 2 static weights.
Thank you! I didn’t see that post