PET-P

中文 | English

Python RPA toolkit with 80+ processors orchestrating browser automation, AI/LLM (10 providers), databases, SSH, email, and HTTP tasks. Configurable pipelines with cron scheduling and loops. Runs as wxPython GUI, headless service, or Docker container. Built-in MCP Tool Server (Streamable-HTTP) for AI agent integration.

Pipeline  1:n  Execution
Execution 1:n  Task
Task      1:1  Processor

Links: Web Intro | Web App | Changelog

Quick Start

1. Install Python 3.14

Download from python.org. On Windows, check "Add Python to PATH".

2. Install wxPython (GUI only)

The stable release (4.2.x on PyPI) does not support Python 3.14. Download a 4.3.0-alpha .whl from wxpython.org/Phoenix/snapshot-builds matching your platform:

# macOS Apple Silicon
uv pip install wxPython-4.3.0a1XXXX-cp314-cp314-macosx_11_0_arm64.whl

# Windows 64-bit
uv pip install wxPython-4.3.0a1XXXX-cp314-cp314-win_amd64.whl

Or auto-download via PETP (no wxPython needed):

python PETP_background.py --run-execution OOTB_DOWNLOAD_LATEST_WXPYTHON_mac_arm

For Python 3.12/3.13: pip install wxPython

3. Install Dependencies

pip install -U uv
uv pip install -r requirements.txt        # Full (GUI)
# or: uv pip install -r requirements-nogui.txt   # Headless
# or: uv pip install -r requirements-docker.txt  # Docker

uv pip install -r requirements/core.txt -r requirements/ssh-sftp.txt -r requirements/http-client.txt

See requirements/ directory for all available groups: ai-deepseek.txt, ai-gemini.txt, ai-ollama.txt, database.txt, excel-data.txt, mcp.txt, ocr.txt, web-automation.txt, etc.

4. Run

python PETP.py          # GUI
python PETP_background.py   # Headless service (port 8866)

Screenshots

macOS

Windows

MCP Tool Server — integrate with Claude Code, Cursor, and other AI agents:

Features

🤖 AI Execution Generator

Highlight — Generate and modify PETP task flows through natural language conversation with LLM. Supports 10 LLM providers including Hyperspace, Anthropic, DeepSeek, Zhipu, Gemini, Ollama, and more.

Entry Points:

• Create Execution → "AI Generate" template
• Right-click taskGrid → "AI Assist" (modify existing) — pre-selects only the processors used in the current Execution to minimize token usage
• MCP Editor → "AI" button (auto-generate tool description)

Highlights:

• Multi-turn chat — ask questions, generate flows, modify tasks incrementally
• Processor browser — expandable TreeListCtrl with full documentation, search & filter
• Selective context — only checked Processors are sent to LLM (saves tokens)
• Connection caching — first-time validation, then instant reuse across sessions
• 10 LLM providers — minimal config: just set ai_provider in petpconfig.yaml
• 429 rate-limit defense — exponential backoff (2s / 4s / 8s), UI throttle ≥ 3s, per-call token accounting log

Token controls (new):

• ai_max_response_tokens — hard upper bound on model output length (default 8192)
• ai_max_request_tokens — local pre-flight cap on request size (default 60000); over-limit prompts are rejected before hitting the provider

AI-Powered MCP Tool Publishing:

• One-click generation of mcp_desc JSON for exposing Executions as MCP tools
• Auto-extracts input parameters from INITIAL_PARAMS and output keys from result tasks
• Generates AI-agent-friendly descriptions that help LLMs understand when to call the tool
• Smart merge — new fields are added without overwriting existing configuration
• Progress dialog with live status and preview before applying

AI Error Analysis & Auto-Fix:

• On execution failure, AI automatically analyzes the error with full context (failed task, surrounding tasks, traceback)
• Pinpoints root cause and suggests specific fixes
• One-click "Open AI Assist" pre-fills the diagnosis — continue fixing in multi-turn chat

Vision Model Support (Ollama):

• AI_LLM_QANDA accepts image_path parameter for multimodal prompts
• Works with Ollama vision models (gemma4, llava, moondream, etc.)
• Image path supports expressions — dynamically reference files from previous tasks in data_chain

Configuration (only ai_provider required, rest auto-fills from provider defaults):

application:
  ai_provider: zhipu              # or: deepseek, anthropic, hyperspace, gemini, ollama, etc.
  ai_model: ""                    # empty = provider default (e.g. GLM-5)
  ai_api_key: ""                  # empty = read from default env var (e.g. ZHIPU_ACCESS_KEY)
  ai_base_url: ""                 # empty = provider default URL
  ai_max_response_tokens: 8192
  ai_max_request_tokens: 60000

See Configuration Docs for full provider list and details.

Dynamic Function (_fn) & Expression Enhancements

All dynamic function parameters (_fn, _func, _func_body, lambda_*) receive the Processor instance as p, enabling full access to PETP utilities inside custom code:

# In _fn function body — p is the Processor instance
result = p.get_data("my_key")
p.populate_data("output", processed_value)
today = p.str_to_date("2026-05-11")

Available p methods in both expression and _fn contexts:

Edit Complex Value — Handy Tool button:

• Right-click any property → "Edit Complex Value" includes a Handy Tool button
• Automatically detects whether the parameter is an expression or _fn function body
• Expression context: snippets wrapped in {p.xxx()} for f-string evaluation
• Function body context: raw p.xxx() calls plus import statements

Running Modes

# Run one execution and exit
python PETP_background.py --run-execution ENDECODER --no-http

# Run execution with initial data
python PETP_background.py --run-execution MY_EXEC --init-data '{"key":"value"}' --no-http

# Run pipeline and exit
python PETP_background.py --run-pipeline DAILY_REPORT --no-http

# Run pipeline with initial data
python PETP_background.py --run-pipeline MY_PIPELINE --init-data '{"param":"value"}' --no-http

# Start HTTP/MCP service (default port 8866)
python PETP_background.py

# Custom port and auth token
python PETP_background.py --http-port 9090 --http-token my-secret-token

# Headless Selenium (Chrome without visible window)
python PETP_background.py --run-execution BROWSER_TASK --headless --no-http

# Override log level
python PETP_background.py --log-level DEBUG

# GUI-processor policy: skip (ignore) or abort (fail on GUI tasks)
python PETP_background.py --run-execution HAS_GUI_TASK --ui-policy skip --no-http
python PETP_background.py --run-execution HAS_GUI_TASK --ui-policy abort --no-http

# Stop a running background instance
python PETP_background.py --stop

Notes:

• --run-execution and --run-pipeline can be combined — both will run in sequence
• Pipeline reentrant protection: if the same pipeline is already running, a second call returns {"ok": false, "error": "Pipeline already running"}
• Cron-enabled pipelines (cronEnabled: true in YAML) auto-register as scheduled jobs instead of running immediately

Helper scripts for long-running sessions: see scripts/macos/start_petp.sh and scripts/windows/start_petp.ps1.

Portable Runtime

portable/ is a self-contained unit that runs a single Execution headlessly, decoupled from the GUI and HTTP layers. Copy portable/ into any Python project, or cf push it to Cloud Foundry.

from portable.petp_run import run
result = run("MY_EXECUTION", {"key": "value"})   # {"ok", "data", "error", "meta"}

Workflow: author the Execution in the desktop GUI → Ctrl+S → copy the YAML into portable/core/executions/ → python portable/petp_run.py NAME. The engine and YAML format are shared, so GUI-authored Executions run unchanged.

Chrome binaries (chrome-headless-shell + chromedriver, from Chrome for Testing) are placed under portable/webdriver/<system>/ and are not committed to git. On Cloud Foundry, manifest.yml + apt.yml (apt-buildpack) supply the required system .so libraries. See portable/README.md for full details.

MCP & AI Integration

PETP exposes executions as MCP tools via Streamable-HTTP on port 8866.

🔒 Auth is fail-closed. When http_request_token is unset in petpconfig.yaml, every protected endpoint (/petp/*, /mcp) returns 501 Not Configured. Set a token before exposing the server (e.g. via Tailscale Funnel). Send it as Authorization: Bearer <token> on every request.

🔒 Security hardening (Phase 2).

• CMD processor: defaults to shlex.split (no shell). Set shell="yes" only for trusted commands needing pipes/redirects.
• Dynamic _fn / lambda_* parameters: run in a sandbox with __import__, open, eval, exec, compile, getattr, hasattr removed. Whitelisted modules: re, json, datetime, math.
• Encrypted password salt: override the public default by setting env PETP_SALT or writing ~/.petp/secret (POSIX mode 0600). The default salt is logged with a WARNING — cryptocode ciphertext is not actually secret unless you set a custom salt.
• Path traversal guard (opt-in): set PETP_PATH_ALLOW_ROOTS=/path1:/path2 to confine all file IO processors (READ_*, WRITE_*, OPEN_FILE, FILE_DELETE, UNZIP) to a whitelist of root directories. Default off — preserves existing yaml using absolute paths.
• Request size limits: HTTP body capped at 4 MiB (PETP_MAX_BODY_BYTES); JSON-RPC batch arrays capped at 64 items (PETP_MAX_BATCH_ITEMS). Both return 413 / 400 with no body parsing.
• Log redaction (default on): values of sensitive keys (api_key, password, token, authorization, secret, ...) are masked as ***REDACTED*** in process start / [Type] input log lines. Disable with PETP_LOG_REDACT=off for ad-hoc debugging.

Performance (headless/Docker):

• Shared thread pool for concurrent tool calls (no per-request executor overhead)
• Static execution cache — zero filesystem I/O after startup (no stat/mtime checks)
• Processor class pre-loading on server start (eliminates cold-start latency)
• Real-time task-level SSE progress notifications during long-running tools/call
• Cached outputSchema parsing (same tool re-called without re-parsing mcp_desc JSON)

Built-in MCP Tools:

Claude Code / Cursor / any MCP client:

{
  "mcpServers": {
    "petp": {
      "type": "http",
      "url": "http://localhost:8866/mcp"
    }
  }
}

HTTP API (no MCP client needed):

# List tools
curl -H "Authorization: Bearer $TOKEN" http://localhost:8866/petp/tools

# Trigger execution
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"execution","params":{"execution":"MY_EXEC"},"wait_for_result":"true"}'

# Trigger pipeline (sync)
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"pipeline","params":{"pipeline":"MY_PIPELINE"},"wait_for_result":"true"}'

# Trigger pipeline (async — poll with /petp/result)
curl -X POST http://localhost:8866/petp/exec \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action":"pipeline","params":{"pipeline":"MY_PIPELINE"},"wait_for_result":"false"}'

Build & Docker

# Standalone executable
python build/PETP_build.py   # → dist/PETP.app or dist/PETP.exe

# Docker — Background service (headless, port 8866)
./build/script/docker_build_bg.sh              # build + export tar
./build/script/docker_build_bg.sh --run        # build + start container
./build/script/docker_build_bg.sh --no-tar     # build only
./build/script/docker_build_bg.sh --push repo:tag  # push to registry
./build/script/docker_build_bg.sh --dirty      # use working dir (skip git archive)

# Docker — Web App (port 5555)
./build/script/docker_build_webapp.sh          # build + export tar
./build/script/docker_build_webapp.sh --run    # build + start container
./build/script/docker_build_webapp.sh --dirty  # use working dir (skip git archive)

Note: Build scripts default to git archive mode — only git-tracked and staged files are included in the Docker context. Use --dirty to include all working directory files (relies on .dockerignore).

Deploy to NAS

# Deploy BG image (scp + docker load + start container)
./build/script/deploy_bg_to_nas.sh
./build/script/deploy_bg_to_nas.sh --no-start   # only transfer + load
./build/script/deploy_bg_to_nas.sh --keep-tar   # don't delete remote tar

# Deploy Webapp image
./build/script/deploy_webapp_to_nas.sh

# Override NAS connection / port mapping
NAS_HOST=10.0.0.5 NAS_USER=myuser HOST_PORT=9090 ./build/script/deploy_webapp_to_nas.sh

Tailscale Funnel (run on NAS)

# Refresh funnel routes (reset + reconfigure)
sudo ./build/script/tailscale_funnel_refresh.sh

# Check current status only
./build/script/tailscale_funnel_refresh.sh --status

Routes configured:

• / → localhost:8088 (webapp)
• /mcp → localhost:8866 (petp background)

Project Structure

Troubleshooting

Documentation

Acknowledgements

• wxPython & wxGlade
• Selenium & ChromeDriver

Full Changelog →