AI/ML Senior Engineer | University of Tokyo Startup #238

Company Overview

Founded in 2019 primarily by members of the Matsuo Laboratory at the University of Tokyo, they are an AI startup targeting the manufacturing industry. Guided by our mission to “fill manufacturing jobs with excitement,” they develop and deliver AI software for work analysis. Adopted by leading Japanese automakers such as Nissan and Suzuki, their software has grown into a global product utilized at over 10,000 sites across 40+ countries.

Background

Their core technology, Video Recognition AI, analyzes vast amounts of operational footage daily across manufacturing sites worldwide. In this role, you will be responsible for developing and improving Vision-Language Models (VLMs).

Specifically, you will drive model evolution in the following directions:

• Improving model accuracy and execution speed.
• Searching and leveraging field knowledge using Video RAG (Retrieval-Augmented Generation for video).
• Executing large-scale models on edge devices.
• Enabling real-time inference on manufacturing assembly lines.

With many research and implementation challenges to tackle, you will join as a core member driving these developments.

 

Job Responsibilities

• R&D and optimization of Vision-Language Models (VLMs) to analyze operation video in manufacturing environments.
• Designing, training, and evaluating Computer Vision models (action recognition, pose estimation, object detection, etc.).
• Enhancing robustness across diverse field environments (varying lighting, camera angles, and task types).
• Optimizing and deploying large-scale models on edge devices.
• Building real-time inference pipelines for manufacturing lines.
• Model lightweighting through quantization, distillation, and pruning.

 

Development Environment

• AI / ML: Python, PyTorch, TensorFlow, Hugging Face Transformers, DeepSpeed
• Edge Inference: TensorRT, ONNX Runtime, LiteRT (formerly TFLite)
• Infrastructure: AWS, GCP, On-premise GPU Servers, Terraform (IaC)
• Frontend: TypeScript, Vue, Nuxt.js
• Backend: Go, GraphQL, Python (FastAPI)
• Database: MySQL
• Tools & Workflow: GitHub, GitHub Actions (CI/CD), VSCode, Figma
• AI Coding Assistants: GitHub Copilot, Claude Code, Codex
• Communication & Docs: Slack, Notion

Engineering Team Structure

• Algorithm Development: 7
• Web Application Development: 8
• Infrastructure: 1

 

Why join them?

• High Autonomy: Work in a lean, elite team on a newly launched service with significant decision-making authority.
• Flat Culture: Open environment where team members can proactively voice ideas regardless of title or rank.
• High Growth Phase: Experience both rapid market expansion (with existing adoption by major enterprises) and organizational scaling as a core team member.
• Direct Field Feedback: Opportunities to visit factories to see how your product is used and valued, directly influencing future feature updates.
• Top-tier AI Environment: Learn high-level AI technology alongside a team composed predominantly of University of Tokyo alumni.
• Strong Leadership: Experienced management and business teams provide a stable environment for growth.
• Product-Driven Collaboration: Seamless communication between engineering and business sides to resolve customer needs and drive product growth efficiently.
• Vibrant Team: Engineering team average age is 28.

 

Required Qualifications

At least 1 year of practical experience or equivalent research achievements in one or more of the following:

• Video and image recognition (object detection, action recognition, segmentation, etc.)
• VLMs / Multimodal models (training and application of Vision-Language models)
• LLMs (fine-tuning, RAG, prompt engineering, etc.)
• Software development using Python (1+ years)
• Hands-on experience with Deep Learning frameworks (e.g., PyTorch)

Japanese proficiency equivalent to JLPT N3 or higher(No qualifications required)

 

Preferred Qualifications

• R&D experience in Video Understanding / Action Recognition.
• Experience in training and applying Vision-Language Models.
• Post-training experience with LLMs / VLMs (SFT, instruction tuning, RLHF, DPO, or other reinforcement learning-based alignment techniques).
• Experience designing and building RAG systems for video and image data.
• Experience with edge inference optimization (TensorRT, ONNX Runtime, quantization, distillation, etc.).
• Experience designing and implementing real-time video processing pipelines.
• Domain knowledge or expertise in the manufacturing industry.
• Track record of publishing research papers or presenting at international conferences.
• English communication skills (opportunities to utilize as we expand globally).

 

Ideal Candidate Profile

• Proactively keeps up with the latest AI trends and applies them to the product.
• Driven by a desire to contribute to society through technology.
• Capable of acting autonomously to formulate hypotheses, test, and iterate rapidly in high-uncertainty environments.

 

Conditions & Benefits

• Salary: JPY 7,000,000 – 12,000,000 per annum (commensurate with experience and skills; includes 1 month’s bonus subject to company performance)

• Probationary Period: 3 months

• Benefits: Full social insurance (Health, Employment, Pension, Worker’s Compensation)

• Working Hours: Flextime system (1-hour break; standard 8 working hours per day)

• Holidays: Fully 2-day weekend policy (Sat/Sun), National Holidays, Year-end/New Year, Summer, and Golden Week holidays

• Smoking Policy: Fully non-smoking indoors

 

Selection Process

Document Screening → 2–3 Rounds of Interviews

• Required Documents: Resume / Curriculum Vitae (CV)

• Primary interview language: Japanese (English is OK for complex/technical matters.)

 

Note: Applicants must currently reside in Japan.