Add Android Docker Support with Computer Server Integration

[synacktraa]

CUA.android.mp4

This PR adds comprehensive Android emulator support to the CUA Computer Server by introducing a Docker image that runs an Android 11 emulator with full programmatic control via HTTP API. The implementation includes 41 working handler methods and a custom wallpaper manager APK to solve Android's native wallpaper setting limitations.

What Changed

New Features

Android Docker Environment (android-docker/)

• Multi-stage Dockerfile - Builds custom wallpaper manager APK and creates runtime image
• Development Dockerfile - Separate build for local development with source code mounting
• Entry script - Manages emulator startup, APK installation, and server initialization
• Comprehensive README - Usage examples, API documentation, and troubleshooting guide

Custom Wallpaper Manager APK (android-docker/wallpaper-manager/)

• Android project source - Complete Gradle project for custom APK helper
• WallpaperManager API integration - Programmatically sets wallpapers without user interaction
• Multi-target support - Set home, lock, or both screen wallpapers
• Automatic permission handling - Copies files to /data/local/tmp for app access

Android Handler Implementation (handlers/android.py)

• 41 working methods across automation, file operations, window management, and desktop control
• ADB command wrapper - All operations use adb shell commands via CommandExecutor
• Async/await pattern - Non-blocking command execution with timeout support
• Comprehensive error handling - Clear error messages for all failure scenarios

Utility Improvements (utils/helpers.py)

• CommandExecutor class - Reusable async command execution with decode option
• Enhanced OS detection - Verifies Android emulator is running via adb devices
• Synchronous implementation - Works at module import time for handler factory

Modified Components

Handler Factory Updates

• Added Android handler imports and instantiation
• Updated OS detection to use new get_current_os() helper
• Proper handler routing for Android environment

Linux Handler Fixes

• Adapted to match base automation handler structure
• Ensures compatibility across all supported platforms

Technical Highlights

Android Handler Methods (41 total)

Automation (19 methods):

• Touch simulation (tap, long press, swipe, drag)
• Text input with special character escaping
• Key events via Android keycodes
• Scrolling with screen size detection
• Screenshot capture (base64 encoded)
• Clipboard operations

File Operations (11 methods):

• File/directory existence checks
• Read/write text and binary files
• Directory listing and manipulation
• ADB push/pull for binary transfers

Window/App Management (9 methods):

• Launch apps by package or activity
• Get current focused activity
• Force stop apps
• Intent-based URL/file opening

Desktop (2 methods):

• Environment detection
• Wallpaper setting via custom APK

Wallpaper Solution Architecture

Android lacks native ADB commands for wallpaper management. Solved this by:

1. Custom APK - Uses Android's WallpaperManager API directly
2. Build-time compilation - APK built during Docker image creation
3. Auto-installation - Installed on container startup via entry.sh
4. Permission workaround - Files copied to /data/local/tmp (FUSE restrictions on /sdcard)
5. Seamless integration - set_wallpaper() handles everything transparently

Usage

Production Build

Installs cua-computer-server from PyPI

cd android-docker
docker build -t cua-android .
docker run -d -p 6080:6080 -p 8000:8000 \
  -e EMULATOR_DEVICE="Samsung Galaxy S10" \
  -e WEB_VNC=true \
  --device /dev/kvm \
  --name android-container \
  cua-android

Development Build

Installs cua-computer-server from local source

cd libs
docker build -f android-docker/dev.Dockerfile -t cua-android:dev .
docker run -d -p 6080:6080 -p 8000:8000 \
  -e EMULATOR_DEVICE="Samsung Galaxy S10" \
  -e WEB_VNC=true \
  --device /dev/kvm \
  --name android-container \
  cua-android:dev

Access Points

• VNC Web UI: http://localhost:6080
• Computer Server API: http://localhost:8000

API Example

# Set wallpaper (automatically handles permissions)
curl -X POST http://localhost:8000/cmd \
  -H "Content-Type: application/json" \
  -d '{"command": "set_wallpaper", "params": {"path": "/sdcard/image.jpg", "target": "home"}}'

Breaking Changes

None - This is a new feature addition that doesn't affect existing functionality.

[vercel]

@synacktraa is attempting to deploy a commit to the Cua Team on Vercel.

A member of the Team first needs to authorize it.