Desktop (Screen Capture)

The Desktop integration lets Phyllax take a screenshot of your screen and send it to a vision-capable AI model for analysis. Useful when you want the assistant to "look at" what's on your screen and tell you what's happening.

Quick Setup

No credentials required. Just enable the integration in Settings.

1. In Phyllax → Integrations → Desktop → click Connect
2. Grant screen-recording permission when your OS asks (macOS only — Windows doesn't gate this)
3. Done

Requires a vision-capable AI provider

Desktop screenshots are sent to your active AI provider for analysis. The provider you pick must support vision input. Phyllax's vision-capable providers:

• Anthropic Claude (all current models)
• OpenAI (GPT-4o and later)
• Gemini (1.5+ flash, pro, 2.0+)
• Ollama (only models with vision — llava, qwen2.5-vl, etc.)

If your active provider doesn't support vision, Desktop screenshots will fail with a "vision not supported" error.

Required Credentials

None.

What Phyllax Monitors

The Desktop integration doesn't run scheduled checks by default — it's an on-demand tool the AI assistant can call when relevant. Example uses:

• "What's open on my screen right now?" → AI calls Desktop tool → describes your windows
• "Help me read this error message" → AI captures the screen → reads the dialog text
• "What does this chart show?" → AI captures → analyzes the chart

You can also create a schedule using Desktop — e.g., a periodic "what's the user doing?" check — but most users use it conversationally.

Notification Actions

Desktop capture results render as plain text in the AI chat or notification cards. No special action chips.

Privacy & Limitations

• Screenshots are taken only when the AI assistant calls the tool. Phyllax doesn't capture continuously.
• The screenshot is sent to your active LLM provider via the standard API. If you're using a cloud provider (Anthropic, OpenAI, etc.), the image goes there.
• For full local-only operation, run Ollama with a vision model.

Troubleshooting

"Screen recording permission denied" (macOS)

Open System Settings → Privacy & Security → Screen Recording → enable Phyllax. Restart the app.

"Vision not supported"

Your active provider can't process images. Switch to a vision-capable provider in Settings or use one for the assistant pref (Settings → Advanced).

Screenshot is captured but the AI says "I can see a blank screen"

The screenshot was empty or all-black. Common cause: macOS dimming the screen during a long screen-recording prompt. Try again with the target window focused.