About

Seong Jong Yoo

Research Interests

I am a third year of Ph.D student in Computer Science department at the University of Maryland. I am currently working in the Perception and Robotic Group(PRG) supervised by Prof. Yiannis Aloimonos and Dr. Cornelia Fermüller. Previously, I studied control theory and machine learning during my master in mechanical engineering department. After graduation, I worked at AI research center, KIST. My main research was the intersection between cultural heritage and computer vision.

My current research interests include the areas below.

Computer Vision
Multimodal Perception
Digital Humans
Consciousness

News

I gave a talk titled “Exploring Musical Behavior: From Violin Performance Pose Estimation to Controllable Dance Generation” at KIST
I participated NeuroAI and Beyond workshop as scribe
Interviewed by The Washington Post was published
Our paper SfS++ is preprinted at arXiv
I gave a lighting talk about VioPose at NYC Computer Vision Day
One paper accepted to ARO
I received a Moon-Jung Chung Scholarship from KOCSEA
One paper accepted to WACV 2025

Research

[Google scholar] [Github] [CV]

An illustration of 10 real pots, simultaneously reassembled from 142 unclustered sherds using the proposed SfS++ method, achieving 87 % accuracy.

Structure-from-Sherds++: Robust Incremental 3D Reassembly of Axially Symmetric Pots from Unordered and Mixed Fragment Collections

Seong Jong Yoo^*, Sisung Liu^*, Muhammad Zeeshan Arshad, Jinhyeok Kim, Young Min Kim, Yiannis Aloimonos, Cornelia Fermüller, Kyungdong Joo, Jinwook Kim^† and Je Hyeong Hong^†
(* denotes an equal contribution, † denotes co-corresponding author)
Under review
[ Paper ] [ Code ] [ Project Page ]

VioPose evaluation on VioDat. Green is ground truth and blue is blue is prediction.

VioPose: Violin Performance 4D Pose Estimation by
Hierarchical Audiovisual Inference

Seong Jong Yoo^*†, Snehesh Shrestha^*, Irina Muresanu, Cornelia Fermüller
(* denotes an equal contribution, † denotes corresponding author)
2025 – Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
[ Paper ] [ Code ] [ Video ] [ Project Page ]

AcTExplore: Active Tactile Exploration on Unknown Objects

Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
2024 – International Conference on Robotics and Automation (ICRA)
[ Paper ] [ Code ] [ Video ] [ Project Page ]

Reconstruction result of 5 different pots with 80 fragments

Structure-from-Sherds: Incremental 3D Reassembly of Axially Symmetric Pots from Unordered and Mixed Fragment Collections

Je Hyeong Hong^*†, Seong Jong Yoo^*, Muhammad Zeeshan Arshad, Young Min Kim, Jinwook Kim^†
(* denotes an equal contribution, † denotes co-corresponding author)
2021 – IEEE/CVF International Conference on Computer Vision (ICCV)
[ Paper ] [ Code ] [ Video ]

Fig. 4 Block diagram of the data-driven approach for self-sensing

Data-Driven Self-Sensing Technique for Active Magnetic Bearing

Seong Jong Yoo, Sinyoung Kim, Kwang-Hwi Cho, Hyeong-Joon Ahn
2021 – International Journal of Precision Engineering and Manufacturing(IJPEM)
[ Paper ]

Fig. 6 Restoration result of Pottery 1 (left), 2 (middle), 3 (right)

3D Pots Configuration System by Optimizing over Geometric Constraints

Jae Eun Kim, Muhammad Zeeshan Arshad, Seong Jong Yoo, Je Hyeong Hong, Jinwook Kim, Young Min Kim
2021 – 25th International Conference on Pattern Recognition(ICPR)
[ Paper ]

Fig. 6 Block diagram of the disturbance observer

Software-Based Integral Product Architecture for Modular Motion Control System of a RFC Linear Motor Motion Stage : Model-Based DOB for Residual Vibration Suppression

Seong Jong Yoo, Hyeong-Joon Ahn
2020 – International Journal of Precision Engineering and Manufacturing(IJPEM)
[ Paper ]

Patent

Jinwook Kim, Kyung-Ryoul Mun, Seong Jong Yoo, Seong Geum Yoo, DaWoon Jung, “System and method for time synchronization of multiple cameras using optical time code”, South Korea, issued Aug 23, 2021, doi : 10.8080/1020200017342.
Jinwook Kim, Je Hyeong Hong, Seong Jong Yoo, Muhammad Arshad Zeeshan, “Virtual Matching Method and System of Porcelain Fragments Using Lip Information and Base Information”, South Korea, issued Mar 12, 2021, doi : 10.8080/1020190155353.
Jinwook Kim, Je Hyeong Hong, Seong Jong Yoo, Muhammad Arshad Zeeshan, “Device for extracting of Shape Features of Porcelain Fragments Based on Rotational Axis Information and Virtual Matching System of Porcelain Fragments using the Same”, South Korea, issued Feb 10, 2021, doi : 10.8080/1020190155354.

Research Project

Music Education for All Through Personalized AI and Digital Humanities
(2023 – present)
Development of the 4th industrial revolution technology based performance improvement and the player support system for the national bobsleigh and skeleton team
(2019 – 2022), Ministry of Culture, Sports and Tourism, South Korea
Development of a Simulation Solution for Restoration of Fragmented Pottery
(2019 – 2020), Ministry of Culture, Sports and Tourism, South Korea
Development of Disturbance Observer Based Control Algorithm for Motion Stage
(2018 – 2019), INNO6 Co
Development of Magnetic Levitated OHT Control System
(2017 – 2018), SEMES Co