diff --git a/README.md b/README.md index d3ca3c5..874b9da 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,13 @@ # scalable-sync -Synchronisiert Scalable Capital Broker-Transaktionen nach Wealthfolio. +Synchronisiert Scalable Capital Broker-Transaktionen nach Wealthfolio via MCP. ## Voraussetzungen - Docker + Docker Compose - Wealthfolio mit MCP aktiviert (`WF_MCP_ENABLED=true`) - Scalable Capital: Agentic Investing aktiviert (Profil → Sicherheit) +- `sc`-CLI auf dem Host installiert (siehe unten) ## Dateien @@ -14,44 +15,83 @@ Synchronisiert Scalable Capital Broker-Transaktionen nach Wealthfolio. |---|---| | `Dockerfile` | Container-Image | | `compose.yaml` | Docker Compose Config | -| `.env` | Alle Einstellungen | +| `example.env` | Vorlage für `.env` | | `scalable_sync.py` | Sync-Adapter | | `requirements.txt` | Python-Abhängigkeiten | ## Setup -```bash -# Dateien in ein Verzeichnis deiner Wahl kopieren, z. B.: -# /opt/scalable-sync/ oder ~/docker/scalable-sync/ +### 1. Scalable CLI auf dem Host installieren +**Linux / WSL:** +```bash +curl -fsSL -o /tmp/sc.tar.gz \ + "[https://github.com/ScalableCapital/scalable-cli/releases/download/v1.0.0/sc-v1.0.0-linux-x86_64-gnu.tar.gz](https://github.com/ScalableCapital/scalable-cli/releases/download/v1.0.0/sc-v1.0.0-linux-x86_64-gnu.tar.gz)" + +curl -fsSL -o /tmp/SHA256SUMS \ + "[https://github.com/ScalableCapital/scalable-cli/releases/download/v1.0.0/sc-v1.0.0-SHA256SUMS](https://github.com/ScalableCapital/scalable-cli/releases/download/v1.0.0/sc-v1.0.0-SHA256SUMS)" + +cd /tmp && grep "linux-x86_64-gnu.tar.gz" SHA256SUMS | awk '{print $1 " sc.tar.gz"}' | sha256sum -c - + +tar -xzf sc.tar.gz +sudo mv sc-v1.0.0-linux-x86_64-gnu/sc /usr/local/bin/sc +sudo chmod +x /usr/local/bin/sc +sc --version +``` + +### 2. Login auf dem Host (einmalig) + +WSL/Linux hat keinen Keyring — daher `session_backend = "file"` setzen: + +```bash +mkdir -p ~/.config/scalable-cli +cat > ~/.config/scalable-cli/config.toml <<'EOF' +[auth] +session_backend = "file" +signing_key_backend = "file" +EOF + +sc login --local-read-only +``` + +### 3. .env anlegen + +```bash +cp example.env .env +# WF_MCP_URL, WF_MCP_TOKEN, WF_ACCOUNT_ID, ASSET_MAP, PUID, PGID anpassen +``` + +### 4. Bind-Mount in compose.yaml anpassen + +```yaml +volumes: + - scalable-sync-data:/app/data + - /home//.config/scalable-cli:/tmp/sc-config:ro +``` + +### 5. Testen und starten + +```bash docker compose build -# 1. Einmalig einloggen (Session wird im Volume gespeichert) -docker compose run --rm -it scalable-sync sc login --local-read-only - -# 2. Test +# Session testen docker compose run --rm scalable-sync sc whoami -docker compose run --rm scalable-sync sc broker holdings --json | jq . +docker compose run --rm scalable-sync sc broker holdings -# 3. Dry-Run +# Dry-Run docker compose run --rm scalable-sync python /app/scalable_sync.py --once --dry-run -# 4. Start +# Start docker compose up -d ``` -## .env +## .env Referenz ```bash -# Wealthfolio WF_MCP_URL=http://wealthfolio:8088/mcp WF_MCP_TOKEN=wf_mcp_dein_token_hier WF_ACCOUNT_ID=dein-konto-uuid -# Optional: Scalable Portfolio-ID (bei mehreren Portfolios) -SC_PORTFOLIO_ID= - -# Sync SYNC_INTERVAL_HOURS=6 SC_OVERLAP_DAYS=14 SC_SETTLED_ONLY=true @@ -59,36 +99,34 @@ SC_SETTLED_ONLY=true # ISIN → Wealthfolio-Symbol ASSET_MAP={"IE00B4L5Y983":"EUNL.DE","IE00B3RBWM25":"VWCE.DE"} -# Container-User PUID=1000 PGID=1000 ``` ### WF_ACCOUNT_ID finden -Wealthfolio → Accounts → Konto anklicken → ID aus der URL kopieren (`/accounts/abc123-def456/...`). +Wealthfolio → Accounts → Konto anklicken → ID aus der URL kopieren. ### ASSET_MAP -JSON-Objekt: `"ISIN": "SYMBOL"`. Wealthfolio-Symbol unter Assets suchen. +JSON-Objekt `"ISIN": "Symbol"`. ISINs ermitteln mit: +```bash +sc broker holdings +``` ### SC_PORTFOLIO_ID -Nur bei mehreren Scalable-Portfolios nötig. Ermitteln mit: +Nur bei mehreren Scalable-Portfolios. Ermitteln mit: ```bash -docker compose run --rm scalable-sync sc broker context show --json +docker compose run --rm scalable-sync sc broker context show ``` -## Volumes +## Session erneuern -Docker legt diese Volumes automatisch an — kein manuelles `mkdir` nötig. - -| Volume | Inhalt | Zweck | -|---|---|---| -| `scalable-sync-data` | state.db, audit.jsonl, dead-letter.jsonl | Sync-Status und Logs | -| `scalable-sync-config` | config.toml, session.json | Scalable CLI Session | - -**Wichtig:** Ohne `scalable-sync-config` geht die Session bei `docker compose down` verloren. +```bash +sc login --local-read-only +docker compose restart scalable-sync +``` ## Befehle @@ -97,15 +135,69 @@ Docker legt diese Volumes automatisch an — kein manuelles `mkdir` nötig. | `docker compose run --rm scalable-sync python /app/scalable_sync.py --once --dry-run` | Testlauf ohne Schreiben | | `docker compose run --rm scalable-sync python /app/scalable_sync.py --once` | Einmaliger Sync | | `docker compose logs -f` | Logs anzeigen | -| `docker compose down && docker compose up -d` | Neustart (Session bleibt erhalten) | +| `docker compose down && docker compose up -d` | Neustart | + +## Volumes + +| Volume | Inhalt | Zweck | +|---|---|---| +| `scalable-sync-data` | state.db, audit.jsonl, dead-letter.jsonl | Sync-Status und Logs | + +Die Session liegt auf dem Host unter `~/.config/scalable-cli/` und wird als read-only Bind-Mount eingehängt. ## Troubleshooting | Problem | Lösung | |---|---| -| "Not logged in" | `docker compose run --rm -it scalable-sync sc login --local-read-only` | +| `sc whoami` schlägt fehl | Session abgelaufen — `sc login --local-read-only` wiederholen, dann `docker compose restart` | | Keine Transaktionen | `SC_SETTLED_ONLY` und `SC_OVERLAP_DAYS` prüfen | | Unmapped ISIN | ISIN in `ASSET_MAP` ergänzen | -| Mehrere Portfolios | `SC_PORTFOLIO_ID` setzen | -| Doppelte Activities | Prüfen ob `WF_ACCOUNT_ID` korrekt ist | -| Session nach Neustart weg | `scalable-sync-config` Volume in compose.yaml prüfen | \ No newline at end of file +| Mehrere Portfolios | `SC_PORTFOLIO_ID` in `.env` setzen | +| Doppelte Activities | `WF_ACCOUNT_ID` prüfen | + +## Wealthfolio-Vorbereitung + +### 1. MCP aktivieren + +In der Wealthfolio `compose.yaml`: + +```yaml +environment: + WF_MCP_ENABLED: "true" + WF_MCP_TOKEN: "wf_mcp_dein_geheimer_token" +``` + +Danach Wealthfolio neu starten. + +### 2. Account anlegen + +Wealthfolio → **Accounts** → **Add Account**: + +| Feld | Wert | +|---|---| +| Name | `Scalable Capital` (frei wählbar) | +| Type | `Brokerage` | +| Currency | `EUR` | + +**WF_ACCOUNT_ID ermitteln:** Account anklicken → ID aus der URL kopieren: + +```` +/accounts/550e8400-e29b-41d4-a716-446655440000/ → diese UUID ist deine WF_ACCOUNT_ID +```` + +### 3. Assets anlegen + +Jedes ETF muss in Wealthfolio als Asset existieren. Wealthfolio → **Assets** → Asset suchen oder neu anlegen. + +ISINs deiner gehaltenen ETFs ermitteln: +```bash +sc broker holdings +``` + +Das **Symbol** in Wealthfolio (z. B. `EUNL.DE`) exakt in `ASSET_MAP` übernehmen: + +```bash +ASSET_MAP={"IE00B4L5Y983":"EUNL.DE","IE00B3RBWM25":"VWCE.DE","DE0002635307":"EXSA.DE"} +``` + +> **Wichtig:** Symbol muss exakt übereinstimmen — am besten aus Wealthfolio kopieren, nicht raten. \ No newline at end of file diff --git a/entrypoint.sh b/entrypoint.sh index 1e26588..cde898d 100644 --- a/entrypoint.sh +++ b/entrypoint.sh @@ -3,9 +3,21 @@ set -e chown -R ${PUID:-1000}:${PGID:-1000} /app/data 2>/dev/null || true -if [ "$1" = "python" ] || [ "$1" = "/app/scalable_sync.py" ]; then +# Wenn der erste Parameter "python" ist → shift und ggf. Skript-Pfad auch entfernen +if [ "$1" = "python" ]; then + shift + # Wenn das nächste Argument das Skript selbst ist → auch entfernen + if [ "$1" = "/app/scalable_sync.py" ]; then + shift + fi + exec gosu appuser python /app/scalable_sync.py "$@" +fi + +# Wenn das Skript direkt als erster Parameter kommt +if [ "$1" = "/app/scalable_sync.py" ]; then shift exec gosu appuser python /app/scalable_sync.py "$@" fi +# Sonst: Befehl direkt als appuser ausführen (z. B. sc login) exec gosu appuser "$@" \ No newline at end of file diff --git a/requirements.txt b/requirements.txt index 6f89372..1adbca2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1 +1,2 @@ -requests>=2.32.0 \ No newline at end of file +mcp +httpx \ No newline at end of file diff --git a/scalable_sync.py b/scalable_sync.py index 67cb0fd..5e4a1a0 100644 --- a/scalable_sync.py +++ b/scalable_sync.py @@ -10,14 +10,17 @@ import os import sqlite3 import subprocess import sys -import tempfile import time from dataclasses import dataclass, asdict from datetime import datetime, timedelta, timezone from pathlib import Path from typing import Any, Optional +import asyncio +import httpx + +from mcp import ClientSession +from mcp.client.streamable_http import streamable_http_client -import requests # --------------------------------------------------------------------------- # Config @@ -32,12 +35,8 @@ SC_SETTLED_ONLY = os.getenv("SC_SETTLED_ONLY", "true").lower() == "true" SC_MAPPING_VERSION = int(os.getenv("SC_MAPPING_VERSION", "1")) ASSET_MAP = json.loads(os.getenv("ASSET_MAP", "{}")) -# Scalable CLI Config: tmpfs-basiert, automatisch generiert -SC_CONFIG_DIR = Path(os.getenv("SC_CONFIG_DIR", tempfile.mkdtemp(prefix="sc-config-"))) -SC_CONFIG_DIR.mkdir(parents=True, exist_ok=True) -(SC_CONFIG_DIR / "config.toml").write_text( - '[auth]\nsession_backend = "file"\nsigning_key_backend = "file"\n' -) +# SC_CONFIG_DIR wird vom Entrypoint gesetzt und als read-only Bind-Mount eingehängt +SC_CONFIG_DIR = Path(os.getenv("SC_CONFIG_DIR", "/tmp/sc-config")) os.environ["SC_CONFIG_DIR"] = str(SC_CONFIG_DIR) DATA_DIR = Path("/app/data") @@ -260,38 +259,74 @@ def map_to_wealthfolio(tx: dict) -> list[WealthfolioActivity]: # --------------------------------------------------------------------------- class WealthfolioMCP: - def __init__(self, url, token): + def __init__(self, url: str, token: str): self.url = url - self.session = requests.Session() - self.session.headers.update({ - "Authorization": f"Bearer {token}", - "Content-Type": "application/json", - "Accept": "application/json", - }) + self.token = token - def _rpc(self, method, params=None): - payload = {"jsonrpc": "2.0", "id": int(time.time() * 1000), "method": method} - if params: payload["params"] = params - resp = self.session.post(self.url, json=payload, timeout=30) - resp.raise_for_status() - return resp.json() + def _http_client(self) -> httpx.AsyncClient: + return httpx.AsyncClient( + headers={ + "Authorization": f"Bearer {self.token}", + "Accept": "application/json, text/event-stream", + }, + timeout=30.0, + ) - def ping(self): + async def _connect_and_call(self, tool_name: str, arguments: dict): + async with self._http_client() as http_client: + async with streamable_http_client( + self.url, + http_client=http_client, + ) as (read_stream, write_stream): + async with ClientSession(read_stream, write_stream) as session: + await session.initialize() + return await session.call_tool(tool_name, arguments) + + async def _list_tools(self) -> list[str]: + async with self._http_client() as http_client: + async with streamable_http_client( + self.url, + http_client=http_client, + ) as (read_stream, write_stream): + async with ClientSession(read_stream, write_stream) as session: + await session.initialize() + result = await session.list_tools() + return [tool.name for tool in result.tools] + + def ping(self) -> bool: try: - return self._rpc("ping").get("result") == "pong" - except Exception as e: - log.error("MCP ping failed: %s", e) + tools = asyncio.run(self._list_tools()) + log.info("Available Wealthfolio MCP tools: %s", tools) + return True + except Exception as exc: + log.exception("MCP initialization failed: %s", exc) return False def record_activities(self, activities): - payload = [{ - "accountId": a.account_id, "date": a.date, "symbol": a.symbol, "activityType": a.activity_type, - "quantity": a.quantity, "unitPrice": a.unit_price, "amount": a.amount, - "fee": a.fee, "tax": a.tax, "currency": a.currency, - "source": a.source, "externalId": a.external_id, - } for a in activities] - return self._rpc("tools/call", {"name": "record_activities", "arguments": {"activities": payload}}) + payload = [ + { + "accountId": activity.account_id, + "date": activity.date, + "symbol": activity.symbol, + "activityType": activity.activity_type, + "quantity": activity.quantity, + "unitPrice": activity.unit_price, + "amount": activity.amount, + "fee": activity.fee, + "tax": activity.tax, + "currency": activity.currency, + "source": activity.source, + "externalId": activity.external_id, + } + for activity in activities + ] + return asyncio.run( + self._connect_and_call( + "record_activities", + {"activities": payload}, + ) + ) # --------------------------------------------------------------------------- # Sync @@ -310,9 +345,13 @@ def reconcile(mcp): def sync_once(dry_run=False): init_db() - mcp = WealthfolioMCP(WF_MCP_URL, WF_MCP_TOKEN) - if not mcp.ping(): - raise RuntimeError("Wealthfolio MCP unreachable") + mcp = None + + if not dry_run: + mcp = WealthfolioMCP(WF_MCP_URL, WF_MCP_TOKEN) + + if not mcp.ping(): + raise RuntimeError("Wealthfolio MCP unreachable") from_time = datetime.now(timezone.utc) - timedelta(days=SC_OVERLAP_DAYS) to_time = datetime.now(timezone.utc) @@ -365,7 +404,8 @@ def sync_once(dry_run=False): if not cursor or not items: break - reconcile(mcp) + if not dry_run: + reconcile(mcp) log.info("Sync complete: %s", stats) return stats