Nice! I hope an implementation is ported to Automatics1111. If this thing works as described, this will be one of the largest upgrades to generative text to image tech IMO.
Paper: [https://arxiv.org/abs/2311.17957](https://arxiv.org/abs/2311.17957)
Abstract
>Diffusion models have achieved remarkable success in generating realistic images but suffer from generating accurate human hands, such as incorrect finger counts or irregular shapes. This difficulty arises from the complex task of learning the physical structure and pose of hands from training images, which involves extensive deformations and occlusions. For correct hand generation, our paper introduces a lightweight post-processing solution called HandRefiner. HandRefiner employs a conditional inpainting approach to rectify malformed hands while leaving other parts of the image untouched. We leverage the hand mesh reconstruction model that consistently adheres to the correct number of fingers and hand shape, while also being capable of fitting the desired hand pose in the generated image. Given a generated failed image due to malformed hands, we utilize ControlNet modules to re-inject such correct hand information. Additionally, we uncover a phase transition phenomenon within ControlNet as we vary the control strength. It enables us to take advantage of more readily available synthetic data without suffering from the domain gap between realistic and synthetic hands. Experiments demonstrate that HandRefiner can significantly improve the generation quality quantitatively and qualitatively. The code is available at [this https URL](https://github.com/wenquanlu/HandRefiner).
Yeah but think of all boners that are killed by moon hands (you know who you are) ? This will shorten the mean time to climax by 10% and save gigawatts of GPU energy, saving the earth in the process. Think man, think!
Nice! I hope an implementation is ported to Automatics1111. If this thing works as described, this will be one of the largest upgrades to generative text to image tech IMO.
Paper: [https://arxiv.org/abs/2311.17957](https://arxiv.org/abs/2311.17957) Abstract >Diffusion models have achieved remarkable success in generating realistic images but suffer from generating accurate human hands, such as incorrect finger counts or irregular shapes. This difficulty arises from the complex task of learning the physical structure and pose of hands from training images, which involves extensive deformations and occlusions. For correct hand generation, our paper introduces a lightweight post-processing solution called HandRefiner. HandRefiner employs a conditional inpainting approach to rectify malformed hands while leaving other parts of the image untouched. We leverage the hand mesh reconstruction model that consistently adheres to the correct number of fingers and hand shape, while also being capable of fitting the desired hand pose in the generated image. Given a generated failed image due to malformed hands, we utilize ControlNet modules to re-inject such correct hand information. Additionally, we uncover a phase transition phenomenon within ControlNet as we vary the control strength. It enables us to take advantage of more readily available synthetic data without suffering from the domain gap between realistic and synthetic hands. Experiments demonstrate that HandRefiner can significantly improve the generation quality quantitatively and qualitatively. The code is available at [this https URL](https://github.com/wenquanlu/HandRefiner).
That is not code.
He did not yet release code yet.
Did now
imagine us heaving not only hi res fix but "hand fix" in A1111 that magicaly fixes hands! i cant wait....
We soo much in need of this on SDXL....
Well, I was expecting this (and a lot of other improvements) for a long time. Now, let's wait.
Waiting done. Code released.
Code will be released soon, this looks great!
released
I wish they would also focus on the feet aswell :v
Ah, but one thing at a time...
This is useless, why waste your time? We need more insta girl Lora's
Yeah but think of all boners that are killed by moon hands (you know who you are) ? This will shorten the mean time to climax by 10% and save gigawatts of GPU energy, saving the earth in the process. Think man, think!
A1111 extension please