Perception Engineer

About the role

Join human-computer-lab to develop the core awareness systems for our character robots, starting with LeLamp. This role focuses on building the computer vision and machine learning capabilities that allow our robots to understand people and their surroundings, enabling natural, real-time interaction. You will contribute to creating emotionally intuitive and human-centered technology.

Key facts

What you'll do

• Design, train, and implement computer vision and perception models for tasks like person detection, tracking, pose estimation, gesture recognition, emotion recognition, and scene understanding.
• Create data collection, labeling, and evaluation pipelines to enhance model performance and dependability.
• Develop perception systems that function consistently across various lighting, environments, and user behaviors.
• Optimize models for real-time inference on edge hardware, balancing latency, accuracy, and power usage.
• Work with multiple sensor inputs, including RGB cameras, depth sensors, IMUs, and audio, to build multi-modal perception systems.
• Build tools for managing datasets, evaluating models, debugging, and monitoring system performance.
• Support the integration of perception systems with behavior, controls, and firmware components.
• Research and apply advanced architectures and techniques to improve the robot's understanding and interaction abilities.
• Systematically debug perception failures and implement data-driven improvements.
• Help define and evolve the platform's perception architecture as the robot's capabilities expand.

Requirements

• A degree in Computer Engineering, Computer Science, Robotics, Electrical Engineering, or a related field, with at least 3 years of practical experience.
• Proficiency in Python and comfort with C++ when necessary.
• Experience training and deploying machine learning models using frameworks such as PyTorch, TensorFlow, or ONNX.
• Background in computer vision problems like object detection, tracking, pose estimation, segmentation, gesture recognition, or activity understanding.
• Experience deploying models to edge devices and optimizing their performance for real-world applications.
• Understanding of the challenges involved in collecting, labeling, and maintaining high-quality datasets.
• Comfort working in environments where architectural decisions are still being made and your input helps shape the direction.
• Ability to document clearly and resolve issues effectively.

Nice to have

• A strong interest in building perception systems that perform reliably outside of controlled lab settings.
• Strong intuition for debugging machine learning systems, distinguishing between data, model, and deployment issues.
• Enjoyment in transforming research into practical, shippable systems.
• Critical thinking about tradeoffs among accuracy, latency, robustness, power consumption, and cost.
• Ability to adapt quickly through iteration cycles without needing perfect information to progress.
• Willingness to take end-to-end ownership of systems, from data collection and training through deployment and monitoring.
• Ability to collaborate effectively in small teams where software, hardware, controls, and AI disciplines interact constantly.
• An appreciation for how perception enables robot awareness, responsiveness, and the feeling of being alive.

Skills & tools

• Python
• C++
• PyTorch
• TensorFlow
• ONNX
• Computer Vision
• Machine Learning
• Edge device deployment
• RGB cameras
• Depth sensors
• IMUs
• Audio inputs

Practical notes

This is a hybrid role. We encourage all qualified applicants to apply, even if they do not meet every single requirement.