Inbar Huberman-Spiegelglas

Technion
Edit Your Own Image: A Breakthrough Innovation in Image Editing
Inbar huberman

Bio

Inbar is a postdoctoral researcher, working with Prof. Tomer Michaeli. She completed her PhD in Computer Science at the Hebrew University in 2022, under the supervision of Prof. Raanan Fattal. She received her M.Sc. in Computer Science from the Hebrew University in 2014. Her research interests include image generation, object recognition, image manipulations, and perception.

Abstract

Denoising diffusion probabilistic models (DDPMs) employ a sequence of white Gaussian noise samples to generate an image. In analogy with GANs, those noise maps could be considered as the latent code associated with the generated image. However, this native noise space does not possess a convenient structure and is thus challenging to work with in editing tasks.

Here, we propose an alternative latent noise space for DDPM that enables a wide range of editing operations via simple means. We present an inversion method for extracting these edit-friendly noise maps for any given image (real or synthetically generated). As opposed to the native DDPM noise space, the edit-friendly noise maps do not have a standard normal distribution and are not statistically independent across timesteps. However, they allow perfect reconstruction of any desired image, and simple transformations on them translate into meaningful manipulations of the output image (e.g. shifting, color edits).

Moreover, in text-conditional models, fixing those noise maps while changing the text prompt modifies semantics while retaining structure. We illustrate how this property enables text-based editing of real images via the diverse DDPM sampling scheme (in contrast to the popular non-diverse DDIM inversion). We also show how it can be used within existing diffusion-based editing methods to improve their quality and diversity.

Please see the project page: https://github.com/inbarhub/DDPM_inversion

Agenda

8:45 Reception
9:30 Opening remarks by WiDS TLV ambassadors Noah Eyal Altman, Or Basson, and Nitzan Gado
9:45 Dr. Aya Soffer, IBM: "Putting Generative AI to Work: What Have We Learned So Far?"
10:15 Prof. Reut Tsarfaty, Bar-llan University: "Will Hebrew Speakers Be Able to Use Generative AI in Their Native Tongue?"
10:45 Break
11:00 Lightning talks
12:20 Lunch & poster session
13:20 Roundtable session & poster session
14:05 Roundtable closing
14:20 Break
14:30 Dr. Orna Amir & Hila Kantor, Google: "A User-Centric Framework for Quantifying Notification Harm"
14:50 Naomi Ken Korem, Lightricks: "Mastering the Art of Generative Models: Training and Controlling Text-to-Video Models"
15:10 Dr. Yael Mathov, Intuit: "Surviving the AI-pocalypse: Your Guide to LLM Security"
15:30 Closing remarks
15:40 The end

WiDS TLV important update

Dear WiDS TLV attendees,

In light of recent developments, we regret to inform you that the WIDS TLV 2024 event scheduled for tomorrow has been postponed to June 3rd, 2024. We apologize for this last-minute change and look forward to seeing all of you on June 3rd.