Research Projects
Real-Time 3D Surgical Scene Reconstruction Based on Endoscopic Data


Programme(s) to which this project applies:
☑ MPhil/PhD ☒MRes[Med] ☒ URIS

Project Overview:

With the increasing adoption of minimally invasive surgery (MIS), endoscopic imaging has become an essential tool for real-time intraoperative visualization. However, conventional 2D endoscopic views provide limited depth perception, making it challenging for surgeons to precisely navigate complex anatomical structures. Real-time 3D reconstruction from endoscopic data has the potential to enhance spatial awareness, improve surgical precision, and enable AI-assisted navigation.

This project aims to develop an AI-driven real-time 3D reconstruction framework that processes endoscopic video streams to generate accurate, dynamic, and depth-aware surgical environments. By integrating computer vision techniques, deep learning models, and high-performance hardware optimization, we seek to transform traditional 2D endoscopic imaging into fully interactive 3D surgical reconstructions.


Key Objectives

  1. Develop a real-time 3D reconstruction pipeline that converts 2D endoscopic video streams into depth-aware 3D models.
  2. Implement AI-based depth estimation and scene understanding to enhance visualization accuracy in complex surgical environments.
  3. Design an interactive real-time rendering system, enabling surgeons to navigate and manipulate reconstructed 3D surgical views.
  4. Optimize hardware-accelerated processing to ensure high-speed performance and seamless integration with endoscopic imaging systems.

Why This Project?

  1. Bridging the Gap Between 2D and 3D Surgery – Transforming conventional 2D endoscopic imaging into a real-time 3D surgical navigation tool enhances intraoperative depth perception.
  2. Hands-On Access to Advanced Endoscopic Imaging Systems – Participants will work with cutting-edge surgical imaging hardware, gaining hands-on experience in real-time data acquisition and processing.
  3. Interdisciplinary Learning Experience – This project integrates computer vision, AI, and medical imaging, offering exposure to both theoretical research and practical implementation.
  4. Shaping the Future of AI-Assisted Surgery – Real-time 3D reconstruction plays a key role in next-generation intelligent surgical systems, paving the way for robot-assisted surgery and augmented reality guidance.

Who Should Join?

  • Students interested in computer vision, AI-driven medical imaging, and surgical robotics.
  • Individuals who want to work with real-world endoscopic imaging hardware and develop cutting-edge real-time 3D reconstruction techniques.
  • Those eager to explore the intersection of AI, deep learning, and surgical innovation.

Join us in advancing AI-assisted surgical visualization, where real-time 3D reconstruction revolutionizes the way surgeons perceive and interact with the surgical field!

 

Dr N Meng, Department of Orthopaedics and Traumatology

Dr. Nan Meng is currently a Research Assistant Professor at HKU. His primary research interests include light field imaging, medical image analysis, digital health, and intelligent healthcare systems.

To date, he has published over 30 research papers, with his work appearing in top-tier international conferences and journals in computer science, such as IEEE T-PAMI, IEEE TIP, IEEE TCSVT, and AAAI, as well as in renowned medical journals including The Lancet and JAMA journals (eClinicalMedicine and JAMA Network Open).

Dr. Meng is currently leading three research projects and co-leading two additional projects, all funded by the Hong Kong Research Grants Council (RGC) and the National Natural Science Foundation of China (NSFC).

He serves as a Review Editor for MDPI Bioengineering and Frontiers in Surgery and has been an active reviewer for prestigious journals and conferences, including IEEE TIP, TCSVT, TNNLS, TCI, TMM, TMI, and The Lancet eClinicalMedicine. Additionally, he has been a regular reviewer for major international conferences such as IEEE EMBC, ISBI, and JBHI.

Biography
HKU Scholars Hub
Lab
ORCID
nanmeng@hku.hk

For more information or to express interest for this project, please email the supervisor or the specified contact point in the project description.  Interested candidates are advised to enclose with your email:

  1. your CV,
  2. a brief description of your research interest and experience, and
  3. two reference letters (not required for HKUMed UG students seeking MRes[Med]/URIS projects).

Information on the research programme, funding support and admission documentations could be referenced online at the Research Postgraduate Admissions website. General admission enquiries should be directed to rpgmed@hku.hk.

HKUMed MBBS students interested in the Master of Research in Medicine (MRes[Med]) programme may visit the programme website for more information.  

HKUMed UG students interested in the Undergraduate Research Internship Scheme (URIS) may visit the scheme’s website for more information. 

Research @ Department of Orthopaedics and Traumatology