Day 5: The Homecoming Board ✈️❄️

[blackgirlbytes]

Day 5: The Homecoming Board ✈️❄️

Welcome Back, AI Engineer!

The Winter Festival is the talk of the town thanks to your amazing work! Now for the grand finale: The Homecoming Board - a magical display at the festival entrance showing real flights arriving, families reuniting, travelers coming home for the holidays.

The twist? It's freezing outside. Gloves. Mittens. Touchscreens don't work. ❄️🧤

The solution? Pure gesture magic. Wave to navigate. Point to select. No touching required - just hand movements in the air.

---

🎯 Your Mission

Build a gesture-controlled flight tracker showing real airplane arrivals, controlled entirely by hand movements.

You'll create:

• Real flight data from actual airports
• 2+ hand gesture controls (swipe, pinch, wave, point)
• Winter/holiday themed interface
• Webcam-powered hand tracking
• Magical, touchless interaction

Why this matters: You're using pre-trained AI models (MediaPipe) for hand tracking - that's computer vision AI running 30-60 times per second in your browser! This is modern AI engineering: combining ML models with real-world data to create magical experiences.

---

🛠️ What You Need

Tools:

• goose (Desktop or CLI - your pick!)
• Developer extension (definitely need this one)
• MediaPipe - Google's computer vision library (goose will help you set it up)
• Flight API - Try AviationStack (free tier) or OpenSky Network (totally free)

Pro tip: Ask goose "Help me set up MediaPipe for hand tracking" and "What's a good free flight tracking API?" - let it do the research!

---

NEED LLM CREDITS?

Sign up at goose-credits.dev to get free credits for Claude Sonnet 4.5 via OpenRouter!

Use the ACCESS CODE: ADVENTDAY5

✅ Requirements

Must Have:

• ✈️ Real flight arrival data from at least one airport
• 👋 At least 2 different gesture types working
• 📸 Webcam hand tracking with MediaPipe
• 🎄 Winter/holiday themed styling
• 💫 Visual feedback when gestures are detected
• ℹ️ Flight info displayed (number, origin, ETA, airline)

Technical:

• Works in real-time
• Uses MediaPipe for hand tracking
• Integrates with a real flight data API

---

💡 Tips From the Trenches

We learned some things building this challenge - here's what'll save you headaches:

Gesture Selection Matters:

• Start simple! Pick 2 gestures that are very different from each other
• ✅ Great combo: Closed fist + Open palm (totally distinct hand shapes)
• ⚠️ Tricky combos: Pointing finger can be misread as a fist, peace sign vs pointing, etc.
• Similar gestures = confused detection = frustration

Hand Tracking Gotchas:

• If your skeleton overlay appears on the opposite side of your hand, you need to mirror both the video AND the landmarks together
• Lower the debounce time (200-400ms) for snappier response
• MediaPipe's detection confidence can be tuned - start with 0.7

Flight API Tips:

• OpenSky's /states/all endpoint (live aircraft positions) is more reliable than /flights/arrival
• You'll likely need a CORS proxy for browser requests (try corsproxy.io)
• The free tier has rate limits - cache your data!

🎁 Bonus Challenges (Pick Your Level!)

🌟 Beginner Bonus:

• Add a 3rd gesture type
• Show departures too
• Add sound effects for gestures
• Visual hand skeleton overlay

🌟🌟 Intermediate Bonus:

• Support multiple airports (switch with gestures)
• Two-handed gestures
• Voice feedback ("Flight from NYC arriving in 20 minutes")
• Smooth gesture animations

🌟🌟🌟 Advanced Bonus:

• Build a gesture training mode (custom gestures!)
• Add pose detection (full body control)
• Deploy it so others can use it
• AR elements overlaid on video feed

🌟🌟🌟🌟 Ultimate Challenge:

• Multi-user support (multiple people controlling at once!)
• ML that learns your gesture preferences
• Mobile app version
• Analytics dashboard

---

📤 Share Your Work

📮 Required: Submit your work to complete Day 5!

Post your solution in the Advent of AI Discussion under Day 5.
Accepted formats:

• Video/GIF of you controlling it with gestures - we NEED to see those hands in action! 👋
• Link to your live site
• Link to a blog post about your solution
• Link to a video of you solving the problem
• Link to your repo

Tag us on socials:
Discord • Twitter/X • YouTube • LinkedIn • Bluesky

---

**📖 Resources for AI Engineers

Essential:

• goose Documentation
• MediaPipe Hands Guide
• AviationStack API
• OpenSky Network

Computer Vision:

• MediaPipe Examples - Try it in your browser!
• Hand Landmarks Guide - All 21 points explained

---

✨ You Know You Did It When...

You've got a working webcam app with 2+ gestures controlling real flight data, winter-themed, and you can navigate without touching anything. Bonus points: 3+ gestures, multiple airports, sound effects, or you deployed it live!

[j4ckxyz]

Entry for day 5!

Built it using the OpenSky Network API with Claude Sonnet 4.5 in Goose CLI. Uses MediaPipe in the browser to detect hand open/close gestures.

Deployed to Vercel with the Vercel MCP tool we learnt about the other day! 👀

Can try here:

https://festive-flight-tracker.vercel.app/

https://bsky.app/profile/j4ck.xyz/post/3m7bhixmp6s2k

[filiphric]

this one was amazing, I really felt like I’m being challenged here’s the submission and here’s the video walkthrough 😊

[nickytonline]

Will edit this as I make progress.

I went with TanStack Start for framework. Here's the folder in the repo https://github.com/nickytonline/advent-of-ai-2025/tree/main/day-5.

So far hand detection working. I'm using Tensorflow.js atm as I ran into issues in browser at least with MediaPipe. Outside the browser, MediaPipe appears to be working.

CleanShot.2025-12-06.at.01.14.25_compressed.mp4

Got MediaPipe working finally and now gesture dection works.

CleanShot.2025-12-06.at.02.04.17_compressed.mp4

I also added the ability to pick your camera if you have more than one.

Multi-hand gesture detection as well as thumbs up detection

CleanShot.2025-12-06.at.10.52.58_compressed.mp4

OK flight list navigation with hand gestures is working

CleanShot.2025-12-06.at.11.29.48_compressed.mp4

Getting fancy. Thumbs up to show the flight info in a dialog and thumbs down to close the dialog.

CleanShot.2025-12-06.at.12.55.17.mp4

[eriperspective]

Cool.

[clovis1122]

Here's mine! Just tweaked my vercel site to show this at https://adventofai.vercel.app/day-5-gesture-tracker.

[eriperspective]

https://ai-media-pie-flight-tracker.vercel.app/

My Winter Flight Tracker. OMG we are here as in the future. lol Let me contain my excitement. So I designed my Winter Flight Tracker. My UI features a winter-themed design with a teal and glow aesthetic, animated snowfall, glowing icons. The gesture-controlled interface displays real flight data from live APIs. This showing actual flights with departure times, airlines, prices, and an interactive map with glowing teal flight paths.

Now for the star of the show MEDIAPIPE! I absolutely adore this. All I thought was this can help million with disabilities. It seems to be an accessibility type technology. It so futuristic and magical. Yes this absolutely is modern AI engineering that combines ML models with real-world data. It created a magical experience for me.

I thought of the movie Minority Report as well when first saw Media Pipe in action. I ran out of token so fast with this because I had to change the hand gestures to peace sign, and pointer, goose designed with fist, palm, and thumb. Also the UI because I love the design aspect. *For the more elaborate projects and to make it to ultimate (which I was able to do some), perhaps we can be allotted more tokens.

MediaPie.Winter.Tracker.mp4

Winter.Flight.Tracker.UI.mp4