mirror of
https://github.com/claude-code-best/claude-code.git
synced 2026-06-15 12:55:51 +00:00
feat: Windows Computer Use enhancement — PrintWindow, UI Automation, OCR
New Windows-native capabilities: - windowCapture.ts: PrintWindow API for per-window screenshot (works on occluded/background windows) - windowEnum.ts: EnumWindows for precise window enumeration with HWND - uiAutomation.ts: IUIAutomation for UI tree reading, element clicking, text input, and coordinate-based element identification - ocr.ts: Windows.Media.Ocr for screen text recognition (en-US + zh-CN) Updated win32.ts backend to use EnumWindows for listRunning() and added captureWindowTarget() for window-specific screenshots. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
27
DEV-LOG.md
27
DEV-LOG.md
@@ -1,5 +1,32 @@
|
||||
# DEV-LOG
|
||||
|
||||
## Computer Use Windows 增强:窗口绑定截图 + UI Automation + OCR (2026-04-03)
|
||||
|
||||
在三平台基础实现之上,利用 Windows 原生 API 增强 Computer Use 的 Windows 专属能力。
|
||||
|
||||
**新增文件:**
|
||||
|
||||
| 文件 | 行数 | 说明 |
|
||||
|------|------|------|
|
||||
| `src/utils/computerUse/win32/windowCapture.ts` | — | `PrintWindow` 窗口绑定截图,支持被遮挡/后台窗口 |
|
||||
| `src/utils/computerUse/win32/windowEnum.ts` | — | `EnumWindows` 精确窗口枚举(HWND + PID + 标题) |
|
||||
| `src/utils/computerUse/win32/uiAutomation.ts` | — | `IUIAutomation` UI 元素树读取、按钮点击、文本写入、坐标识别 |
|
||||
| `src/utils/computerUse/win32/ocr.ts` | — | `Windows.Media.Ocr` 截图+文字识别(英语+中文) |
|
||||
|
||||
**修改文件:**
|
||||
|
||||
| 文件 | 变更 |
|
||||
|------|------|
|
||||
| `packages/@ant/computer-use-swift/src/backends/win32.ts` | `listRunning` 改用 EnumWindows;新增 `captureWindowTarget` 窗口级截图 |
|
||||
|
||||
**验证结果(Windows x64):**
|
||||
- 窗口枚举:38 个可见窗口 ✅
|
||||
- 窗口截图:VS Code 2575x1415, 444KB ✅(PrintWindow, 即使被遮挡)
|
||||
- UI Automation:坐标元素识别 ✅
|
||||
- OCR:识别 VS Code 界面文字,34 行 ✅
|
||||
|
||||
---
|
||||
|
||||
## Enable Computer Use — macOS + Windows + Linux (2026-04-03)
|
||||
|
||||
恢复 Computer Use 屏幕操控功能。参考项目仅 macOS,本次扩展为三平台支持。
|
||||
|
||||
Reference in New Issue
Block a user