The goal of the SynthNet project was to develop generative neural approaches to create synthetic image data needed to build and optimize a visual search index. We generated all images from real-world, industry-grade 3D CAD data (SolidWorks) and used them for object identification. For each component, a set of views and depth data (RGB+D) of an object is automatically generated under different lighting and material conditions by respecting the object's metadata using the SynthNet Render Pipeline. We also used neural image augmentation to add backgrounds or distractors and performed style transfer to mimic real photos with our synthetic data. As a result, the Topex-Printer Dataset↗ has been released for public usage and marks the first CAD-to-real multi-domain industrial image dataset, comprising 102 machine parts.

We used the Topex-Printer Dataset to develop a practical approach to Transfer Learning and Unsupervised Domain Adaptation (UDA) in industrial object classification. To be more precise, this means to use no labeled photos but only labeled synthetic images to train a classifier for real photos. For this challenge I developed the SynthNet Transfer Learning Framework, which allows rapid reproducible training configurations including models, hyperparameters, dataloaders, domain transfer techniques, feature fusion techniques, gradual unfreezing, and more. As a result, we developed a two-stage Transfer Learning UDA method, achieving state-of-the-art results on the VisDA-2017 benchmark dataset (93.47%). Our findings were presented at the ECML PKDD 2023 in Turin and published by Springer Nature. Check out the paper website↗.

Finally, our trained models were used as a feature extractor in the SynthNet Retrieval Pipeline to build a visual search index with our synthetic data using Facebook AI Similarity Search (FAISS) and query it with real photos to measure performance metrics (Acc, mAP, NDCG, MRR).

The goal of the BewARe project was the development of a technically supported movement and mobility training for seniors with hypertension based on an intelligent augmented reality system. Various exercises from the training components of endurance, strength, mobility, and coordination were provided, and individual stress norms were taken into account.

I started working on the project as a working student in the context of my master thesis↗ ,"Human Motion Analysis Using 3-D Pose Estimation Input", where I developed algorithms to cut motions from workouts into singular exercise motions and rate the execution quality of each individual motion sequence. After finishing my thesis, I continued working on the project as a research assistant and mainly worked on the artificial intelligence and sensor data components but also contributed on the other parts of the system. Together with my colleague, we developed an assistance system to process sequential heart rate and depth camera data and analyze it within the backend system. Our main task was counting, identifying, and rating the execution quality of motion sequences in near-real-time. To be able to prepare the data for further analysis, we developed the performance-oriented motion analysis library MANA, which provides tested functions to transform, manipulate, and visualize motion sequences from different sources. The prepared motion data was used to perform intelligent analysis steps using different approaches.

One technique was to create so-called motion images from pose estimation data by mapping time and joint positions to colors in the Motion Feature Extraction (MOFEX) system. This allowed us to utilize large pre-trained image classification CNNs for transfer learning and extract meaningful image features, which represent motions. To take into account people who can only perform movements to a limited extent or who are restricted by physical characteristics, we have decided to pursue a distance-based approach for exercise rating. The idea was that patients perform and record exercises once under professional supervision to create their individual gold standard. Later, when patients perform the exercises at home and without supervision, they get near real-time feedback based on their individual body properties.

I started working on the APERTOS project as part of my bachelor thesis (german)↗ 'Design and development of a generic single-page application for the realization of open data platforms', at Fraunhofer FOKUS↗. and continued working on the project until 2019. APERTOS is a core application for the realization of interfaces of open data platforms in the form of a single-page application. The framework made it possible to rapidly create individual, working open data portal prototypes in a few minutes. This enabled us to share styled, working prototypes with potential clients even during the first meeting and significantly contribute to the approval of project funds. Clients include the European Data Portal, WindNODE, International Data Spaces, and the Leistungszentrum Digitale Vernetzung.

To ensure flexible application possibilities, interfaces were designed to be customizable without having to modify any source code of the core application itself. This includes changes to the design of the user interface as well as adjustments to the content of selected components and individually definable interfaces for obtaining the data to be displayed.