Dockerize with Mullvad VPN, caddy ingress, and richer LLM normalization #1

Merged

moritz merged 1 commit from dockerize-vpn-llm into main 2026-05-31 13:39:26 +00:00

Conversation 0 Commits 1 Files changed 7 +710 -77

moritz commented 2026-05-31 13:39:12 +00:00

Owner

Copy link

Containerization:

• Multi-stage Dockerfile: shared app base + scraper target with the Mullvad
  daemon/CLI (systemctl stub, libdbus, MULLVAD_RESOURCE_DIR, state dir).
• docker-compose: scraper (NET_ADMIN + tun, Mullvad rotation) and read-only
  web viewer sharing a SQLite volume; web exposed at scraper.elbnerds.de via
  caddy-docker-proxy (external caddy network + labels).
• docker-entrypoint.sh boots/logs in the Mullvad daemon, connects, enables the
  killswitch, allows LAN, and points VPN DNS at Docker's resolver so the local
  ntfy container stays reachable. TZ=Europe/Berlin on both services.
• DB_PATH is env-overridable so the DB lives on a mounted volume.

Scraper:

• Sync edited ads (title/price/etc.) back into the DB; re-notify on title or
  price change; title now read from

  (covers non-linked layout).

• LLM normalization now classifies category (weekend/day/addon/merch/wanted),
  ticket_count, and price_per_ticket; notifications fire only for weekend
  passes. For ticket offers it fetches the full detail-page description so
  per-ticket pricing isn't fooled by truncated listing snippets.
• Record last successful scrape time in a meta table.

Web UI:

• Render timestamps in Europe/Berlin; category badges + price/ticket column;
  dim non-relevant offers; client-side "weekend only" toggle (persisted).
• Price-history SVG chart with rolling 7-offer average.
• Show last successful scrape time (turns red when stale).

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

Containerization: - Multi-stage Dockerfile: shared app base + scraper target with the Mullvad daemon/CLI (systemctl stub, libdbus, MULLVAD_RESOURCE_DIR, state dir). - docker-compose: scraper (NET_ADMIN + tun, Mullvad rotation) and read-only web viewer sharing a SQLite volume; web exposed at scraper.elbnerds.de via caddy-docker-proxy (external caddy network + labels). - docker-entrypoint.sh boots/logs in the Mullvad daemon, connects, enables the killswitch, allows LAN, and points VPN DNS at Docker's resolver so the local ntfy container stays reachable. TZ=Europe/Berlin on both services. - DB_PATH is env-overridable so the DB lives on a mounted volume. Scraper: - Sync edited ads (title/price/etc.) back into the DB; re-notify on title or price change; title now read from <h2> (covers non-linked <span> layout). - LLM normalization now classifies category (weekend/day/addon/merch/wanted), ticket_count, and price_per_ticket; notifications fire only for weekend passes. For ticket offers it fetches the full detail-page description so per-ticket pricing isn't fooled by truncated listing snippets. - Record last successful scrape time in a meta table. Web UI: - Render timestamps in Europe/Berlin; category badges + price/ticket column; dim non-relevant offers; client-side "weekend only" toggle (persisted). - Price-history SVG chart with rolling 7-offer average. - Show last successful scrape time (turns red when stale). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

moritz added 1 commit 2026-05-31 13:39:12 +00:00

Dockerize with Mullvad VPN, caddy ingress, and richer LLM normalization ... ceae87850d

Containerization:
- Multi-stage Dockerfile: shared app base + scraper target with the Mullvad
  daemon/CLI (systemctl stub, libdbus, MULLVAD_RESOURCE_DIR, state dir).
- docker-compose: scraper (NET_ADMIN + tun, Mullvad rotation) and read-only
  web viewer sharing a SQLite volume; web exposed at scraper.elbnerds.de via
  caddy-docker-proxy (external caddy network + labels).
- docker-entrypoint.sh boots/logs in the Mullvad daemon, connects, enables the
  killswitch, allows LAN, and points VPN DNS at Docker's resolver so the local
  ntfy container stays reachable. TZ=Europe/Berlin on both services.
- DB_PATH is env-overridable so the DB lives on a mounted volume.

Scraper:
- Sync edited ads (title/price/etc.) back into the DB; re-notify on title or
  price change; title now read from <h2> (covers non-linked <span> layout).
- LLM normalization now classifies category (weekend/day/addon/merch/wanted),
  ticket_count, and price_per_ticket; notifications fire only for weekend
  passes. For ticket offers it fetches the full detail-page description so
  per-ticket pricing isn't fooled by truncated listing snippets.
- Record last successful scrape time in a meta table.

Web UI:
- Render timestamps in Europe/Berlin; category badges + price/ticket column;
  dim non-relevant offers; client-side "weekend only" toggle (persisted).
- Price-history SVG chart with rolling 7-offer average.
- Show last successful scrape time (turns red when stale).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

moritz merged commit c9ceaae119 into main 2026-05-31 13:39:26 +00:00

moritz referenced this pull request from a commit 2026-05-31 13:39:27 +00:00

Merge pull request 'Dockerize with Mullvad VPN, caddy ingress, and richer LLM normalization' (#1) from dockerize-vpn-llm into main

moritz deleted branch dockerize-vpn-llm 2026-05-31 13:39:29 +00:00

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

moritz/rip26-scraper!1

No description provided.