TrackOSC

TrackOSC turns your iPhone into a wireless motion-tracking camera. Point it at a person and it streams live tracking data – body poses in 2D and 3D, hand poses, face landmarks, recognised text, animals, people, barcodes, outlines, the horizon and rectangles – over your local network as OSC (Open Sound Control) messages, ready for TouchDesigner, Max/MSP, Processing, openFrameworks, Pure Data, SuperCollider, Unity, or anything else that speaks OSC.

All processing happens on-device with Apple's Vision framework. No video ever leaves your phone – only numbers.

WHAT IT TRACKS
• 2D Body: a 17-joint skeleton in pixels
• 3D Body: a 17-joint skeleton in metres – real depth from a single camera, with each joint's pixel position alongside
• Hand: 21 joints per hand, up to two hands
• Face: the face box, head rotation and jawline contour, and Face Landmarks: the full 76-point constellation – eyes, pupils, brows, nose, lips, jaw
• Text: recognised text with its bounding box
• Animal: cat and dog boxes, and Animal Pose: full 25-joint cat and dog skeletons
• Human: whole-body boxes that keep counting people at distances where pose estimation gives up
• Barcode: QR codes and 1D/2D barcodes with position, corners, symbology and payload
• Contours: every edge outline in the frame as closed polylines
• Horizon: the horizon's angle and a line across the frame
• Rectangles: screens, paper, doors – a box plus four corners

FEATURES
• Live tracking overlays on the camera view, colour-coded per detector
• One chip per detector – run just what you need for the best frame rate; 3D Body runs in its own lane so it never slows the others
• Front or back camera, with a natural selfie mirror (wire data is never mirrored)
• Portrait and landscape, with an orientation lock for phones mounted on tripods and rigs
• Automatic discovery of receivers on your network via Bonjour – or type any IP and port
• Hide-video mode: tracking overlay only, on black – for stage and installation use
• Free, notarised macOS companions – a Receiver with 2D and 3D views, a Recorder that records and replays the stream, a Speaker that reads it aloud, and a Router that turns it into MIDI, Shortcuts, key presses and HTTP – plus open-source receiver examples for Processing, Python, p5.js, TouchDesigner, Max/MSP, Pure Data, openFrameworks and SuperCollider, all from the project page

COMPATIBILITY
TrackOSC speaks exactly the same OSC wire format as VisionOSC and PoseOSC, so existing receivers and patches work unchanged. The newer detectors arrive as additional messages that older receivers simply ignore. Coordinates are in pixels, origin top-left, with the frame dimensions in every message.

PRIVACY
No accounts, no analytics, no ads. Camera frames are analysed on the device and discarded; tracking data is sent only to the destination you choose.

TrackOSC is open source: https://github.com/JGL/TrackOSC