When Alexa can't answer, Claude takes over

Developer rg321 released hey-claude, an open-source home assistant project documented in github.com's report under the MIT license, designed to catch unhandled Amazon Alexa voice requests and execute them with Anthropic's Claude. The system acts as a failover layer across local consumer hardware: whenever an Amazon Echo responds that it cannot fulfill a spoken query, background scripts capture the event from Amazon voice records, parse intent using a Claude Haiku session, and execute automated actions across connected home equipment. The project supports complex multilingual instructions, regional language requests in Hindi and Punjabi, and multi-step hardware automations that standard smart speaker ecosystems reject.

This teardown examines the technical architecture, communication protocols, file configurations, and hardware requirements of the hey-claude codebase. The system targets home automation enthusiasts, self-hosters, and developers seeking to combine cloud conversational models with local local-area network hardware control without replacing existing Amazon Echo hardware. By bridging voice logs to local bash and Node.js execution scripts, the project converts commercial voice hardware into an open-ended control terminal for living room electronics, network video recorders, and media streaming boxes.

Alexa can't answer, Claude takes over: what actually changed

Standard smart speaker installations drop execution immediately when a user command falls outside a vendor's native skills directory or device integrations. When an Amazon Echo speaks an error phrase like "Sorry, I don't know that", the interaction terminates without triggering downstream automation. The hey-claude utility fundamentally alters this workflow by treating smart speaker failures as event triggers rather than hard dead ends. The software does not attempt to replace native Alexa functionality for simple tasks like basic timers or supported smart switches. Instead, it maintains a failure blacklist inside its poller process, allowing successful native operations to pass untouched while isolating genuinely failed voice logs for secondary execution.

Once a failed phrase is isolated, the system executes intent resolution through Claude Haiku rather than static keyword scripts. This enables the assistant to parse compound natural language commands that combine conditional sequences and colloquial phrases. Documented examples include handling festival explanations in Hindi during Navratri, telling requested jokes in Punjabi, powering on an LG television before launching a movie stream on Netflix, and waking a television display to project an active security camera stream in a single continuous instruction. The user continues speaking to an existing Amazon Echo, but the back-end execution path shifts dynamically between Amazon cloud services and a local Claude-driven execution runtime.

Alexa can't answer, Claude takes over: how it works

The hey-claude runtime operates through a two-stage local pipeline split between an Alexa poller and an execution watcher. The primary poller script, core/poller.js, connects to Amazon servers via the alexa-remote2 library, pulling user voice history every 2 seconds to detect failed voice events. When an unhandled command appears, the core/watch.sh shell watcher picks up the new instruction using a 100-millisecond polling interval. The watcher enforces a 60-second deduplication window to ensure Claude's subsequent actions do not trigger recursive event loops. To maintain natural pacing, the software calculates a dynamic response delay of roughly 20 characters per second so Claude waits for Alexa to finish speaking its standard refusal phrase before delivering audible processing beeps and running its tasks.

When executing, the watcher initiates an isolated Claude Haiku session through the command-line interface, avoiding persistent background daemon memory costs. Claude reads instructions from a local CLAUDE.md file containing user preferences, aliases, and device nicknames, along with hardware definitions auto-loaded from devices/*/device.md. The model executes actions via bash commands on the local network. An LG WebOS television receives commands over direct WebSocket connections using the SSAP protocol to manage power, volume, application launches, screenshots, and screen toggles. A CP Plus DVR streams video over RTSP converted via ffmpeg into HLS for playback in the television browser, while a Jio Set Top Box handles DLNA casting and DIAL protocol app launches. Claude replies through Alexa using SSML in Amazon's Matthew voice.

Alexa can't answer, Claude takes over: why it matters

Commercial voice assistants have long struggled with compound multi-device instructions and nuanced regional languages, often requiring rigid invocation phrases or proprietary manufacturer hubs. The hey-claude architecture demonstrates that local scripting combined with fast, lightweight large language models like Claude Haiku can handle end-to-end device orchestrations without complex smart-home platforms. Because each invocation runs as an ephemeral session rather than an expensive persistent reasoning loop, operational expenses remain low while allowing Claude to read and update a stateful CLAUDE.md file across interactions. The model remembers device aliases, user preferences, and custom operational quirks through simple Markdown persistence.

The framework also standardizes device integration through plain directory structures and documentation files rather than monolithic codebases. Adding hardware requires creating a folder inside devices/, adding an execution script like index.js or control.sh, and supplying a device.md file describing available operations. The core/watch.sh process automatically discovers these Markdown instructions and injects them directly into Claude's prompt context during command execution. This approach eliminates the need to register custom skills with Amazon or maintain proprietary cloud webhooks, allowing direct local-network manipulation of televisions, set-top boxes, and video security infrastructure through standard networking protocols.

Alexa can't answer, Claude takes over: who is affected

This project directly affects self-hosters and smart-home tinkerers using Amazon Echo hardware who want extended device control without discarding their existing speaker setups. Users in multilingual households benefit from Claude's ability to interpret vernacular queries and mixed regional phrases that standard voice engines misinterpret or reject. The software specifically supports setups running Node.js 18 or higher alongside an authenticated Claude CLI installation on a machine connected to the same local area network as the target appliances. The open-source repository gives developers an extensible base to bridge proprietary consumer hardware with independent generative AI tooling.

Owners of specific hardware targets documented in the repository gain immediate plug-and-play functionality across local protocols. The codebase contains built-in device definitions for LG WebOS televisions, Jio Set Top Boxes running DIAL and DLNA streaming, and CP Plus digital video recorders exposing RTSP video feeds. In addition to television and media box control, Amazon Echo units themselves are controlled via alexa-remote2 for volume adjustments, text commands, speech synthesis, and cross-room announcements. Homes equipped with these specific devices can deploy the software to execute complex, multi-protocol automations that native manufacturer integrations do not natively bridge.

Alexa can't answer, Claude takes over: what to watch

As the hey-claude repository evolves under its open-source MIT license, several hardware expansions and architectural refinements remain in development. The project roadmap lists planned support for smart lighting integrations, expanding the platform beyond living room audiovisual devices into whole-home electrical controls. Community contributions focus on refining the failure detection logic within core/poller.js, decreasing end-to-end latency between the spoken query and device execution, and expanding device directories for additional television brands and camera hardware. The project maintainer welcomes pull requests aimed at speeding up execution and broadening third-party appliance compatibility.

Users deploying the project must also monitor Amazon authentication dependencies and local network configurations. The setup requires authenticating through core/alexa/auth.js to generate cookie_data.json, a credential store that must be maintained to sustain polling access against Amazon's voice history endpoints. Local device IP addresses, MAC addresses, and network identifiers must be configured manually inside config.json based on the provided config.example.json template. As Amazon updates its voice portal APIs and streaming interfaces, maintaining steady library support through alexa-remote2 will determine how reliably the failover mechanism detects unhandled voice events over long operational periods.