Video Editing

We only show a few samples here. For more visualization, click here

Single Input

Multiple Input

Source Video

Reference Image

Edited Video

Instruction: "[Source Video] edit the video following the first frame [Reference Image]"

Source Video

Reference Image

Edited Video

Instruction: "[Source Video] Insert a paper boat in the water [Reference Image] A graceful white swan glides silently across the still surface of a clear lake...Beside it, an orange paper boat drifts lightly."

Video Generation

Image Editing & Generation

Before

After

Instruction: "Change the beach in the picture to a mountain landscape."

Before

After

Instruction: "Make the person raise her right arm."

Before

After

Instruction: "Transfer the image into a faceted low-poly 3D render style"

Instruction: "A photo of four chairs."

Instruction: "A photo of a blue elephant."

Instruction: "A photo of a stop sign above a fork."

Instruction: "A photo of a green suitcase and a blue boat."

Methodology

EditVerse is built on two core principles: enabling powerful in-context learning and ensuring maximum flexibility.

(1) Unified Representation for In-Context Learning: We represent all modalities (text, images, and videos) as a unified one-dimensional token sequence. This sequence is fed into the model built on a full self-attention architecture. This approach allows the model to jointly process and align different modalities, leading to enhanced text comprehension and improved editing quality. Most importantly, it facilitates natural knowledge transfer from the data-rich image domain to the data-scarce video domain.

We design a unified framework for image and video editing and generation, which processes text and vision inputs into a unified sequence. The right part of the figure shows our position embedding design. This framework leverages full self-attention to facilitate robust in-context learning and effective knowledge transfer among modalities.

(2) Flexibility via Interleaved Design: Inspired by Multimodal Large Language Models (MLLMs), we use an interleaved design for text, image, and video inputs. This allows EditVerse to process inputs and outputs of arbitrary resolution, duration, and sequential position. To help the model distinguish between these different dimensions, we introduce a novel four-dimensional Rotary Position Embedding (RoPE) that incorporates sequential, temporal, height, and width information.

Quantitative Comparison

Quantitative comparison on EditVerseBench. For open-source research models, we compare two training-free methods (TokenFlow and STDF), one first-frame propagation method (Señorita-2M), and two instruction-guided video editing method (InsV2V and Lucy Edit). Best results are highlighted in bold. We also provide the results of a commercial model, Runway Aleph. While EditVerse lags Runway Aleph in generation quality due to base model differences, our proposed method EditVerse surpasses it in editing faithfulness (via VLM evaluation on editing quality), which is better aligned with human judgment that is further validated by user studies (see paper).

Citation

If you find our work useful for your research, please consider citing our paper:

@article{ju2025editverse,
  title   = {EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning},
  author  = {Xuan Ju and Tianyu Wang and Yuqian Zhou and He Zhang and Qing Liu and Nanxuan Zhao and Zhifei Zhang and Yijun Li and Yuanhao Cai and Shaoteng Liu and Daniil Pakhomov and Zhe Lin and Soo Ye Kim and Qiang Xu},
  journal = {arXiv preprint arXiv:2509.20360},
  year    = {2025},
  url     = {https://arxiv.org/abs/2509.20360}
}