Heng Yu

News
  • Sep 2024: 4Real paper is accepted by NeurIPS 2024, see you in Vancouver!
  • Feb 2024: CoGS paper is accepted by CVPR 2024, see you in Seattle!
  • Feb 2023: DyLiN paper is accepted by CVPR 2023, see you in Vancouver!
  • Feb 2023: SubZero abstract is accepted by ISMRM 2023 as a power pitch, see you in Toronto!
  • Dec 2022: CoNFies paper has been nominated as a best paper candidate!
  • Nov 2022: Won gold medal at the 8th China International College Students' 'Internet+' Innovation and Entrepreneurship Competition!
  • Sep 2022: CoNFies paper is accepted by FG 2023, see you in Hawaii!
  • Feb 2022: One paper is accepted by Magnetic Resonance in Medicine!
  • Aug 2021: Start my graduate study at CMU RI!
  • Mar 2021: One paper is accepted by Nature Communications!
  • Feb 2021: eRAKI abstract is accepted by ISMRM 2021 as an oral!
  • Apr 2020: MixModule paper is accepted by ISBI 2020!
  • Nov 2019: One paper is accepted by Annals of Surgery!
Research

I have rich experiences in computer vision, MRI reconstruction and medical image analysis. I would like to explore the possibility of AI technology and its applications (e.g. 3D Scene Understanding , Healthcare, etc).

Service

Reviewer: CVPR, ICCV, ECCV, NeurIPS, SIGGRAPH, MICCAI, ISBI, Computer Graphics Forum, ISMRM

Selected Publications

* refers to co-first author. Please refer to my google scholar for more details.

clean-usnob 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
Heng Yu*, Chaoyang Wang*, Peiye Zhuang, Willi Menapace, Aliaksandr Siarohin, Junli Cao, László A. Jeni, Sergey Tulyakov, Hsin-Ying Lee,
NeurIPS 2024
paper / project page

We propose 4Real, the first photorealistic text-to-4D scene generation pipeline.

clean-usnob CoGS: Controllable Gaussian Splatting
Heng Yu, Joel Julin, Zoltan Adam Milacski, Koichiro Niinuma, László A. Jeni,
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024
paper / project page / code

We present CoGS, a method for Controllable Gaussian Splatting, that enables the direct manipulation of scene elements, offering real-time control of dynamic scenes without the prerequisite of pre-computing control signals.

clean-usnob DyLiN: Making Light Field Networks Dynamic
Heng Yu, Joel Julin, Zoltan Adam Milacski, Koichiro Niinuma, László A. Jeni,
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
paper / project page / code / CMU RI News

We propose propose the Dynamic Light Field Network (DyLiN) method that can handle non-rigid deformations, including topological changes, which outperformed state-of-the art methods in terms of visual fidelity and compute complexity.

clean-usnob Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image
Heng Yu, Zoltan Adam Milacski, László A. Jeni,
CVPR workshop, 2023
paper

We propose a general adversarial learning framework for solving Unsupervised 2D to Explicit 3D Style Transfer.

clean-usnob CoNFies: Controllable Neural Face Avatars
Heng Yu, Koichiro Niinuma László A. Jeni,
International Conference on Automatic Face and Gesture Recognition (FG), 2023 - Best Paper Award Finalist
paper / project page / code

We propose a fully-automatic controllable neural representation for face self-portraits.

clean-usnob SubZero: Subspace Zero-Shot MRI Reconstruction
Heng Yu, Yamin Arefeen, Berkin Bilgic
Proceedings of the 31th Annual Meeting of ISMRM, 2023 - Power Pitch
paper / code

We propose a parallel network framework and introduce an attention mechanism to improve subspace-based zero-shot self-supervised learning and enable higher acceleration factors.

clean-usnob Scan-specific artifact reduction in k-space (SPARK) neural networks synergize with physics-based reconstruction to accelerate MRI
Yamin Arefeen, Onur Beker, Jaejin Cho, Heng Yu, Elfar Adalsteinsson, Berkin Bilgic
Magnetic Resonance in Medicine (MRM), 2022
paper / code

We develop a scan-specific model that estimates and corrects k-space errors made when reconstructing accelerated MRI data.

clean-usnob eRAKI: Fast Robust Artificial neural networks for K-space Interpolation (RAKI) with Coil Combination and Joint Reconstruction
Heng Yu, Zijing Dong, Yamin Arefeen, Congyu Liao, Kawin Setsompop, Berkin Bilgic
Proceedings of the 29th Annual Meeting of ISMRM, 2021 - Oral Presentation
paper / code

We accelerate RAKI by more than 200 times by directly learning a coil-combined target.

clean-usnob Predicting treatment response from longitudinal images using multi-task deep learning
Cheng Jin*, Heng Yu*, Jia Ke*, Peirong Ding*, Yongju Yi, Xiaofeng Jiang, Xin Duan, Jinghua Tang, Daniel T. Chang, Xiaojian Wu, Feng Gao, Ruijiang Li
Nature Communications, 2021
paper / code

We present a multi-task deep learning approach that allows simultaneous tumor segmentation and response prediction of pathologic complete response after neoadjuvant chemoradiotherapy.

clean-usnob MixModule: Mixed CNN Kernel Module for Medical Image Segmentation
Heng Yu, Xue Feng, Ziwen Wang, Hao Sun
IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020
paper / code

We use mixed kernels to improve the performance of existing medical image segmentation networks.

clean-usnob Development and validation of a deep learning CT signature to predict survival and chemotherapy benefit in gastric cancer: a multicenter, retrospective study.
Yuming Jiang*, Cheng Jin*, Heng Yu*, Jia Wu*, Chuanli Chen, Qingyu Yuan, Weicai Huang, Yanfeng Hu, Yikai Xu, Zhiwei Zhou, George A. Fisher Jr., Guoxin Li, Ruijiang Li
Annals of surgery, 2020
paper / code

We propose a novel deep neural network (S-net) to construct a CT signature for predicting disease-free survival (DFS) and overall survival.

clean-usnob Deep learning analysis of the primary tumour and the prediction of lymph node metastases in gastric cancer.
C Jin*, Y Jiang*, H Yu*, W Wang, B Li, C Chen, Q Yuan, Y Hu, Y Xu, Z Zhou, G Li, R Li
British Journal of Surgery, 2020
paper / code

We develop a deep learning system for predicting lymph node metastasis in multiple nodal stations based on preoperative CT images in patients with gastric cancer.

clean-usnob SmartPartNet: Part-Informed Person Detection for Body-Worn Smartphones.
Heng Yu, Eshed Ohn-Bar, Donghyun Yoo, Kris Kitani
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2018
paper

We develop an image-based person detection algorithm for wearable computing using commodity smartphones.

clean-usnob Left atrial appendage segmentation using fully convolutional neural networks and modified three-dimensional conditional random fields.
Cheng Jin, Jianjiang Feng , Lei Wang, Heng Yu, Jiang Liu, Jiwen Lu, Jie Zhou
IEEE Journal of Biomedical and Health Informatics (JBHI), 2018
paper

We propose a robust method for automatic left atrial appendage segmentation on computed tomographic angiography data using fully convolutional neural networks with 3D conditional random fields.

clean-usnob Detection of Substances in the Left Atrial Appendage by Spatiotemporal Motion Analysis Based on 4D-CT.
Cheng Jin, Heng Yu, Jianjiang Feng , Lei Wang, Jiwen Lu, Jie Zhou
MICCAI workshop, 2017 - Oral Presentation
paper

we present a new approach for the detection of substances in the left atrial appendage by spatiotemporal motion analysis and make a detailed judgment and analysis of spatial distribution and classification of most objects in the left atrial appendage.

clean-usnob Left atrial appendage neck modeling for closure surgery.
Cheng Jin, Heng Yu, Jianjiang Feng , Lei Wang, Jiwen Lu, Jie Zhou
MICCAI workshop, 2017
paper

We propose a robust method for automatic left atrial appendage segmentation on computed tomographic angiography data using fully convolutional neural networks with 3D conditional random fields.