ARchatPet

Upload a photo of your pet and it comes back as a 3D model in augmented reality. Say its name and you can talk to it, and it answers in its own voice.

Event
Hack the Valley 9 (as PortaPet)
Built
October 2024
Role
Image-to-3D pipeline and AR
Team
Krishna Cheemalapati, Edward D and me

The idea

It started as a homesick coder's dream: moving away and missing your pet, and wondering if you could bring them along, virtually.

What I built

  • The image-to-3D pipeline: an API route sends the photo to Meshy's image-to-3D model, polls the task, and when it succeeds copies the .glb model into Vercel Blob so it loads from our own storage.
  • A proxy route for fetching models, and the split between server and client code.
  • The AR scene in React Three Fiber and WebXR: placing the pet, physics, and hooking up the voice conversation.
  • Per-pet routes (/scene/[name]), the audio manager, deployment fixes, and later a GSAP and Lenis restyle of the landing page.

Team

Krishna integrated ElevenLabs for the pet's voice, Edward built the first landing page, and another contributor built the audio recording component. The AI replies run on Cloudflare.

Stack

Next.js 14 · React Three Fiber · @react-three/xr · Three.js · Meshy API · Vercel Blob · ElevenLabs · Cloudflare · GSAP