← All case studies

Tangible Play / Osmo · September 2019 – February 2023

Computer Vision Product Systems

Real-time CV systems for education products under device and usability constraints.

Role: Senior Research Engineer / CV lead

Computer visionReal-time MLJava deploymentEducation products

Executive summary

Led CV systems that improved worksheet recognition accuracy and supported interactive learning workflows.

93% → 98% worksheet CV accuracy80% IoU shaded-region detection20% engagement improvement99% manual tagging effort reduction

Problem and constraints

Education products required reliable computer vision under varied lighting, paper positions, device constraints, and real-time interaction expectations.

  • Device performance
  • Real-world lighting
  • Low-latency interaction
  • High recognition accuracy
  • Product feedback loops

Architecture

01Image/video input
02Detection and segmentation
03Post-processing
04Real-time inference
05Java deployment
06Product feedback

Decision Theater

Decision fork

Offline accuracy vs product-fit model

A model that wins offline but misses latency constraints fails the product.

Larger model

Pros
  • Higher offline ceiling
Cons
  • Latency/device risk

Production-fit model

Pros
  • Better real-time usability
Cons
  • Requires careful tradeoffs

Chosen: Production-fit CV system. Interactive learning depends on reliable user experience, not leaderboard-only accuracy.

Evaluation and reliability

  • Tracked worksheet recognition accuracy improvement and real-time shaded-region IoU.

Observability and debugging

  • Product usage and engagement outcomes guided CV system iteration.

Reflection

This work proves ML product depth outside the LLM trend cycle: real users, device constraints, deployment, and feedback loops.

This case study uses sanitized architecture and representative examples. It excludes confidential prompts, customer data, proprietary datasets, private implementation details, and internal traces.