Research Projects
Next-Generation Surgical Vision Navigation: Markerless and Precise Instrument Detection and Tracking Based on Multi-View Stereo Collaborative Imaging


Programme(s) to which this project applies:
☑ MPhil/PhD ☒MRes[Med] ☒ URIS

Project Overview

In modern surgical environments, precision and accuracy are critical, especially in minimally invasive procedures where direct visibility is often limited. Traditional surgical navigation systems rely heavily on preoperative imaging and intraoperative tracking markers, which can be cumbersome and introduce additional complexities.

This project aims to develop a next-generation, markerless surgical vision navigation system by integrating multi-view stereo-assisted imaging with AI-driven real-time object recognition and detection. The system will leverage synchronized multi-camera data capture, cutting-edge deep learning models, and high-performance hardware acceleration to enhance real-time surgical guidance.

Unlike conventional purely software-driven research, this project bridges the gap between hardware and software, offering participants hands-on experience with state-of-the-art prototype devices while contributing to software development and system optimization.

Key Objectives

  1. Develop a synchronized multi-camera system for real-time surgical data acquisition, ensuring high-resolution imaging and precise spatial awareness.
  2. Design and train AI-driven object recognition and detection models to accurately identify surgical tools and anatomical structures from multi-view perspectives.
  3. Create an interactive and user-friendly GUI for real-time visualization, integrating efficient algorithm acceleration for seamless operation.
  4. Optimize hardware-software integration, enhancing system efficiency through advanced processing techniques and real-time performance tuning.

Why This Project?

  1. Hands-on Access to Cutting-Edge Hardware – Participants will work directly with next-generation multi-camera prototype systems, gaining valuable experience in hardware setup, calibration, and real-time data collection.
  2. Practical AI & Computer Vision Application – Instead of focusing solely on theoretical AI research, students will have the chance to apply deep learning and computer vision algorithms to a real-world medical challenge.
  3. Interdisciplinary Learning Opportunity – This project sits at the intersection of AI, computer vision, hardware engineering, and medical technology, making it ideal for students interested in both software and hardware development.
  4. Contributing to the Future of Smart Surgery – Participants will be part of a groundbreaking initiative to develop markerless, AI-assisted surgical navigation, helping shape the next generation of intelligent surgical systems.

Who Should Join?

  • Students passionate about AI, computer vision, and medical robotics.
  • Those who want hands-on experience in both software and hardware development.
  • Individuals eager to work with real-world surgical prototype devices and push the boundaries of intelligent healthcare technology.

Join us in building the next frontier of AI-powered surgical vision navigation and gain first-hand experience with cutting-edge medical technology!

Dr N Meng, Department of Orthopaedics and Traumatology

Dr. Nan Meng is currently a Research Assistant Professor at HKU. His primary research interests include light field imaging, medical image analysis, digital health, and intelligent healthcare systems.

To date, he has published over 30 research papers, with his work appearing in top-tier international conferences and journals in computer science, such as IEEE T-PAMI, IEEE TIP, IEEE TCSVT, and AAAI, as well as in renowned medical journals including The Lancet and JAMA journals (eClinicalMedicine and JAMA Network Open).

Dr. Meng is currently leading three research projects and co-leading two additional projects, all funded by the Hong Kong Research Grants Council (RGC) and the National Natural Science Foundation of China (NSFC).

He serves as a Review Editor for MDPI Bioengineering and Frontiers in Surgery and has been an active reviewer for prestigious journals and conferences, including IEEE TIP, TCSVT, TNNLS, TCI, TMM, TMI, and The Lancet eClinicalMedicine. Additionally, he has been a regular reviewer for major international conferences such as IEEE EMBC, ISBI, and JBHI.

Biography
HKU Scholars Hub
Lab
ORCID
nanmeng@hku.hk

For more information or to express interest for this project, please email the supervisor or the specified contact point in the project description.  Interested candidates are advised to enclose with your email:

  1. your CV,
  2. a brief description of your research interest and experience, and
  3. two reference letters (not required for HKUMed UG students seeking MRes[Med]/URIS projects).

Information on the research programme, funding support and admission documentations could be referenced online at the Research Postgraduate Admissions website. General admission enquiries should be directed to rpgmed@hku.hk.

HKUMed MBBS students interested in the Master of Research in Medicine (MRes[Med]) programme may visit the programme website for more information.  

HKUMed UG students interested in the Undergraduate Research Internship Scheme (URIS) may visit the scheme’s website for more information. 

Research @ Department of Orthopaedics and Traumatology