Add phase 5: link import, Instagram fallback and share target
- SSRF-safe fetcher: public IPs only, connects to the checked address, redirects re-validated, size/time limits, honest user agent - schema.org/Recipe JSON-LD extraction (graphs, HowToSections, ISO durations, yields, keywords, images) without extra dependencies; AI fallback on plain page text when no structured recipe exists - Instagram: one login-free request to the public embed page; when Instagram withholds the caption, the import page explains how to paste the text or import a screenshot instead - Suggested source photo can be downloaded and attached on save - Text import works without AI (text becomes the description) - Installable PWA with manifest, icons and Android share target - Offline tests for SSRF rules, parsers, Instagram and import views Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Kyle Dammann
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
1bf8ef8b65
commit
60785e2437
@@ -9,7 +9,7 @@ port; TLS, domain and proxy are out of scope.
|
||||
- [x] Phase 2: Recipes CRUD, visibility (private/public), photos, notes, search, favourites
|
||||
- [x] Phase 3: Comments, serving calculator (server-side `?portionen=`), cooking mode (wake lock)
|
||||
- [x] Phase 4: AI (OpenAI): "Mit KI aufbereiten" on the form + photo/screenshot and text import
|
||||
- [ ] Phase 5: Link import (schema.org JSON-LD, Instagram with paste fallback, Android PWA share target)
|
||||
- [x] Phase 5: Link import (schema.org JSON-LD, Instagram best effort + paste/screenshot fallback, PWA share target)
|
||||
- [ ] Phase 6: Backup script, full README
|
||||
|
||||
## Stack
|
||||
@@ -36,6 +36,13 @@ no JS build step: plain forms with small progressive enhancements in `static/js/
|
||||
re-validated by Pydantic (`AIRecipe`), `map_api_error` → German user messages, per-user/global daily
|
||||
limits (0 = off). `_create_completion` is the only network call (stubbed in tests). Drafts go into
|
||||
`RecipeForm(initial=…)` and are never saved directly. `import_views.py`: `/rezepte/import/` (photo, text).
|
||||
- `recipes/fetch.py` – the ONLY way to request user-supplied URLs: resolves once, requires public IPs,
|
||||
connects to the checked IP (no DNS rebinding), manual redirects re-checked, size/time limits, honest UA.
|
||||
`schema_org.py` parses JSON-LD Recipe + page text (stdlib HTMLParser). `instagram.py` makes one honest
|
||||
request to the post's embed page; Instagram mostly answers with an empty shell → UI falls back to
|
||||
paste/screenshot. Do not add UA spoofing, cookies or login-based scraping (owner's requirement).
|
||||
- `core/pwa.py` – manifest (share_target → `/teilen/`) and a no-cache service worker. Icons via
|
||||
`scripts/make_icons.py`.
|
||||
- `templates/`, `static/` – global templates, `app.css` (mobile first, dark mode), `app.js`.
|
||||
- `docker/entrypoint.sh` – migrate → ensure_admin → purge_ratelimits → gunicorn.
|
||||
- `scripts/` – `init-env.sh` (creates .env with secrets), `dev_server.py` (local run without Docker).
|
||||
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
"""Web app manifest and service worker (installable app, Android share target)."""
|
||||
|
||||
from django.contrib.auth.decorators import login_not_required
|
||||
from django.http import HttpResponse, JsonResponse
|
||||
from django.templatetags.static import static
|
||||
from django.urls import reverse
|
||||
from django.views.decorators.cache import cache_control
|
||||
from django.views.decorators.http import require_GET
|
||||
|
||||
# No caching of pages or API responses: private recipes must never sit in a shared cache,
|
||||
# and the app is online-only. The worker exists so browsers offer installation.
|
||||
SERVICE_WORKER = """self.addEventListener("install", () => self.skipWaiting());
|
||||
self.addEventListener("activate", (event) => event.waitUntil(self.clients.claim()));
|
||||
"""
|
||||
|
||||
|
||||
@login_not_required
|
||||
@require_GET
|
||||
@cache_control(max_age=3600)
|
||||
def manifest(request):
|
||||
data = {
|
||||
"name": "Kochbuch",
|
||||
"short_name": "Kochbuch",
|
||||
"lang": "de",
|
||||
"start_url": reverse("recipes:list"),
|
||||
"scope": "/",
|
||||
"display": "standalone",
|
||||
"background_color": "#faf7f2",
|
||||
"theme_color": "#b4532a",
|
||||
"icons": [
|
||||
{"src": static("icons/icon-192.png"), "sizes": "192x192", "type": "image/png"},
|
||||
{"src": static("icons/icon-512.png"), "sizes": "512x512", "type": "image/png"},
|
||||
{"src": static("icons/icon-maskable-512.png"), "sizes": "512x512", "type": "image/png", "purpose": "maskable"},
|
||||
],
|
||||
"share_target": {
|
||||
"action": reverse("recipes:share"),
|
||||
"method": "GET",
|
||||
"params": {"title": "title", "text": "text", "url": "url"},
|
||||
},
|
||||
}
|
||||
return JsonResponse(data, content_type="application/manifest+json", json_dumps_params={"ensure_ascii": False})
|
||||
|
||||
|
||||
@login_not_required
|
||||
@require_GET
|
||||
@cache_control(max_age=0, no_cache=True)
|
||||
def service_worker(request):
|
||||
return HttpResponse(SERVICE_WORKER, content_type="application/javascript")
|
||||
+3
-1
@@ -1,9 +1,11 @@
|
||||
from django.urls import path
|
||||
|
||||
from . import views
|
||||
from . import pwa, views
|
||||
|
||||
app_name = "core"
|
||||
|
||||
urlpatterns = [
|
||||
path("healthz", views.healthz, name="healthz"),
|
||||
path("manifest.webmanifest", pwa.manifest, name="manifest"),
|
||||
path("sw.js", pwa.service_worker, name="service_worker"),
|
||||
]
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Outgoing HTTP requests to user-supplied URLs, hardened against SSRF.
|
||||
|
||||
The host name is resolved once, every resolved address must be a public (global) IP, and
|
||||
the connection is made to that checked IP (TLS still verifies the real host name). That
|
||||
closes DNS-rebinding tricks. Redirects are followed manually and re-checked on every hop.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from dataclasses import dataclass
|
||||
from urllib.parse import urljoin, urlsplit
|
||||
|
||||
import urllib3
|
||||
|
||||
# Honest identification as a bot; sites that refuse it are handled by the paste fallback.
|
||||
USER_AGENT = "Mozilla/5.0 (compatible; Kochbuch-Rezeptimport/1.0; self-hosted recipe manager)"
|
||||
MAX_REDIRECTS = 5
|
||||
DEFAULT_MAX_BYTES = 3 * 1024 * 1024
|
||||
|
||||
|
||||
class FetchError(Exception):
|
||||
def __init__(self, user_message):
|
||||
super().__init__(user_message)
|
||||
self.user_message = user_message
|
||||
|
||||
|
||||
@dataclass
|
||||
class FetchResult:
|
||||
url: str
|
||||
status: int
|
||||
content_type: str
|
||||
body: bytes
|
||||
|
||||
@property
|
||||
def text(self):
|
||||
charset = "utf-8"
|
||||
for part in self.content_type.split(";")[1:]:
|
||||
key, _, value = part.strip().partition("=")
|
||||
if key.lower() == "charset" and value:
|
||||
charset = value.strip("\"'")
|
||||
try:
|
||||
return self.body.decode(charset, errors="replace")
|
||||
except LookupError:
|
||||
return self.body.decode("utf-8", errors="replace")
|
||||
|
||||
|
||||
def _resolve_public(host, port):
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, type=socket.SOCK_STREAM)
|
||||
except (socket.gaierror, UnicodeError):
|
||||
raise FetchError("Die Adresse konnte nicht gefunden werden.") from None
|
||||
addresses = []
|
||||
for info in infos:
|
||||
ip = ipaddress.ip_address(info[4][0].split("%")[0])
|
||||
if isinstance(ip, ipaddress.IPv6Address) and ip.ipv4_mapped:
|
||||
ip = ip.ipv4_mapped
|
||||
if not ip.is_global or ip.is_multicast:
|
||||
raise FetchError("Diese Adresse ist nicht erlaubt (internes Netz).")
|
||||
addresses.append(str(ip))
|
||||
if not addresses:
|
||||
raise FetchError("Die Adresse konnte nicht gefunden werden.")
|
||||
return addresses[0]
|
||||
|
||||
|
||||
def _validate_url(url):
|
||||
parts = urlsplit(url)
|
||||
if parts.scheme not in ("http", "https") or not parts.hostname:
|
||||
raise FetchError("Bitte gib eine gültige http(s)-Adresse ein.")
|
||||
if parts.username or parts.password:
|
||||
raise FetchError("Adressen mit Zugangsdaten werden nicht unterstützt.")
|
||||
port = parts.port or (443 if parts.scheme == "https" else 80)
|
||||
if port not in (80, 443, 8080, 8443):
|
||||
raise FetchError("Dieser Port wird nicht unterstützt.")
|
||||
return parts, port
|
||||
|
||||
|
||||
def _request_once(url, accept, timeout):
|
||||
parts, port = _validate_url(url)
|
||||
host = parts.hostname
|
||||
ip = _resolve_public(host, port)
|
||||
default_port = 443 if parts.scheme == "https" else 80
|
||||
headers = {
|
||||
"Host": host if port == default_port else f"{host}:{port}",
|
||||
"User-Agent": USER_AGENT,
|
||||
"Accept": accept,
|
||||
"Accept-Language": "de-DE,de;q=0.9,en;q=0.7",
|
||||
}
|
||||
if parts.scheme == "https":
|
||||
pool = urllib3.HTTPSConnectionPool(ip, port, server_hostname=host, assert_hostname=host,
|
||||
cert_reqs="CERT_REQUIRED", retries=False, timeout=timeout)
|
||||
else:
|
||||
pool = urllib3.HTTPConnectionPool(ip, port, retries=False, timeout=timeout)
|
||||
path = parts.path or "/"
|
||||
if parts.query:
|
||||
path += "?" + parts.query
|
||||
try:
|
||||
return pool.urlopen("GET", path, headers=headers, redirect=False, preload_content=False)
|
||||
except urllib3.exceptions.TimeoutError:
|
||||
raise FetchError("Die Seite hat nicht rechtzeitig geantwortet.") from None
|
||||
except urllib3.exceptions.SSLError:
|
||||
raise FetchError("Die sichere Verbindung zur Seite ist fehlgeschlagen.") from None
|
||||
except urllib3.exceptions.HTTPError:
|
||||
raise FetchError("Die Seite ist nicht erreichbar.") from None
|
||||
|
||||
|
||||
def fetch(url, accept="text/html,application/xhtml+xml;q=0.9,*/*;q=0.8", max_bytes=DEFAULT_MAX_BYTES, timeout=None):
|
||||
timeout = timeout or urllib3.Timeout(connect=5, read=15)
|
||||
for _ in range(MAX_REDIRECTS + 1):
|
||||
response = _request_once(url, accept, timeout)
|
||||
try:
|
||||
if response.status in (301, 302, 303, 307, 308) and response.headers.get("Location"):
|
||||
url = urljoin(url, response.headers["Location"])
|
||||
continue
|
||||
chunks, size = [], 0
|
||||
for chunk in response.stream(64 * 1024, decode_content=True):
|
||||
size += len(chunk)
|
||||
if size > max_bytes:
|
||||
raise FetchError("Die Seite ist zu groß.")
|
||||
chunks.append(chunk)
|
||||
return FetchResult(url=url, status=response.status,
|
||||
content_type=response.headers.get("Content-Type", ""), body=b"".join(chunks))
|
||||
except urllib3.exceptions.HTTPError:
|
||||
raise FetchError("Die Seite konnte nicht vollständig geladen werden.") from None
|
||||
finally:
|
||||
response.release_conn()
|
||||
raise FetchError("Zu viele Weiterleitungen.")
|
||||
+8
-1
@@ -59,6 +59,9 @@ class RecipeForm(forms.ModelForm):
|
||||
)
|
||||
source_url = forms.URLField(label="Quelle (URL)", required=False, max_length=1000, assume_scheme="https")
|
||||
photos = photo_field()
|
||||
# Link import: photo suggested by the source page, downloaded only when the user keeps the tick.
|
||||
import_image_url = forms.URLField(required=False, max_length=2000, widget=forms.HiddenInput, assume_scheme="https")
|
||||
use_import_image = forms.BooleanField(label="Bild der Quelle als Foto übernehmen", required=False, initial=True)
|
||||
|
||||
class Meta:
|
||||
model = Recipe
|
||||
@@ -74,7 +77,8 @@ class RecipeForm(forms.ModelForm):
|
||||
}
|
||||
|
||||
field_order = ["title", "description", "servings", "servings_label", "prep_minutes", "cook_minutes",
|
||||
"ingredients_text", "steps_text", "tags_text", "photos", "author_notes", "source_url", "visibility"]
|
||||
"ingredients_text", "steps_text", "tags_text", "use_import_image", "photos", "author_notes",
|
||||
"source_url", "visibility"]
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
@@ -85,6 +89,9 @@ class RecipeForm(forms.ModelForm):
|
||||
self.initial.setdefault("ingredients_text", parsing.ingredients_to_text(recipe.ingredients.all()))
|
||||
self.initial.setdefault("steps_text", parsing.steps_to_text([s.text for s in recipe.steps.all()]))
|
||||
self.initial.setdefault("tags_text", ", ".join(t.name for t in recipe.tags.all()))
|
||||
if recipe.pk or not (self.initial.get("import_image_url") or self.data.get("import_image_url")):
|
||||
del self.fields["import_image_url"]
|
||||
del self.fields["use_import_image"]
|
||||
if recipe.pk:
|
||||
remaining = settings.IMAGE_MAX_PER_RECIPE - recipe.images.count()
|
||||
self.fields["photos"].label = "Weitere Fotos"
|
||||
|
||||
+112
-4
@@ -4,15 +4,30 @@ Every path ends in the normal recipe form, pre-filled as an editable draft and p
|
||||
recipes:create. Nothing is saved before the user presses "Speichern".
|
||||
"""
|
||||
|
||||
import re
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from django import forms
|
||||
from django.contrib import messages
|
||||
from django.shortcuts import redirect, render
|
||||
from django.urls import reverse
|
||||
from django.views.decorators.http import require_POST
|
||||
from django.views.decorators.http import require_GET, require_POST
|
||||
|
||||
from . import ai, images
|
||||
from . import ai, images, instagram, schema_org
|
||||
from .fetch import FetchError, fetch
|
||||
from .forms import MultipleFileInput, MultipleImageField, RecipeForm
|
||||
|
||||
_URL_RE = re.compile(r"https?://[^\s<>\"']+")
|
||||
|
||||
|
||||
class LinkImportForm(forms.Form):
|
||||
url = forms.URLField(
|
||||
label="Link",
|
||||
max_length=2000,
|
||||
assume_scheme="https",
|
||||
widget=forms.URLInput(attrs={"placeholder": "https://… (Rezeptseite oder Instagram)", "inputmode": "url"}),
|
||||
)
|
||||
|
||||
|
||||
class PhotoImportForm(forms.Form):
|
||||
photos = MultipleImageField(
|
||||
@@ -48,11 +63,15 @@ def render_draft(request, initial, notice, ai_draft=None, extra_context=None):
|
||||
})
|
||||
|
||||
|
||||
def import_start(request, photo_form=None, text_form=None):
|
||||
def import_start(request, photo_form=None, text_form=None, link_form=None, instagram_failed=False, status=200):
|
||||
if link_form is None:
|
||||
link_form = LinkImportForm(initial={"url": request.GET.get("url", "")})
|
||||
return render(request, "recipes/import.html", {
|
||||
"photo_form": photo_form or PhotoImportForm(),
|
||||
"text_form": text_form or TextImportForm(),
|
||||
})
|
||||
"link_form": link_form,
|
||||
"instagram_failed": instagram_failed,
|
||||
}, status=status)
|
||||
|
||||
|
||||
@require_POST
|
||||
@@ -74,6 +93,11 @@ def import_text(request):
|
||||
form = TextImportForm(request.POST)
|
||||
if not form.is_valid():
|
||||
return import_start(request, text_form=form)
|
||||
if not ai.is_configured():
|
||||
# Without AI the pasted text becomes the description; the user splits it up by hand.
|
||||
initial = {"description": form.cleaned_data["text"], "source_url": form.cleaned_data["source_url"]}
|
||||
return render_draft(request, initial, "Text übernommen (KI nicht eingerichtet – bitte Zutaten und "
|
||||
"Schritte aus der Beschreibung in die passenden Felder übertragen)")
|
||||
try:
|
||||
draft = ai.structure_text(request.user, form.cleaned_data["text"])
|
||||
except ai.AIError as error:
|
||||
@@ -81,3 +105,87 @@ def import_text(request):
|
||||
return import_start(request, text_form=form)
|
||||
initial = {**draft.to_form_initial(), "source_url": form.cleaned_data["source_url"]}
|
||||
return render_draft(request, initial, "Aus dem Text erkannt", ai_draft=draft)
|
||||
|
||||
|
||||
@require_POST
|
||||
def import_link(request):
|
||||
form = LinkImportForm(request.POST)
|
||||
if not form.is_valid():
|
||||
return import_start(request, link_form=form, status=400)
|
||||
url = form.cleaned_data["url"]
|
||||
code = instagram.shortcode(url)
|
||||
if code:
|
||||
return _import_instagram(request, url, code, form)
|
||||
if instagram.is_instagram(url):
|
||||
form.add_error("url", "Bitte den Link zu einem einzelnen Beitrag oder Reel verwenden.")
|
||||
return import_start(request, link_form=form, status=400)
|
||||
return _import_website(request, url, form)
|
||||
|
||||
|
||||
def _instagram_fallback(request, url, form, reason):
|
||||
messages.warning(request, f"{reason} Füge die Beschreibung des Beitrags unten ein – oder lade einen Screenshot hoch.")
|
||||
text_form = TextImportForm(initial={"source_url": url})
|
||||
return import_start(request, link_form=form, text_form=text_form, instagram_failed=True)
|
||||
|
||||
|
||||
def _import_instagram(request, url, code, form):
|
||||
try:
|
||||
caption, image = instagram.fetch_caption(code)
|
||||
except instagram.InstagramUnavailable:
|
||||
return _instagram_fallback(request, url, form, "Instagram lässt den automatischen Abruf gerade nicht zu.")
|
||||
extra = {"import_image_url": image} if image else {}
|
||||
if not ai.is_configured():
|
||||
initial = {"description": caption, "source_url": url, **extra}
|
||||
return render_draft(request, initial, "Beschreibung von Instagram übernommen (KI nicht eingerichtet – "
|
||||
"bitte Zutaten und Schritte selbst aufteilen)")
|
||||
try:
|
||||
draft = ai.structure_text(request.user, caption, hint="Beschreibung (Caption) eines Instagram-Beitrags:")
|
||||
except ai.AIError as error:
|
||||
messages.error(request, error.user_message)
|
||||
return import_start(request, link_form=form, text_form=TextImportForm(initial={"text": caption, "source_url": url}))
|
||||
return render_draft(request, {**draft.to_form_initial(), "source_url": url, **extra}, "Aus Instagram erkannt",
|
||||
ai_draft=draft)
|
||||
|
||||
|
||||
def _import_website(request, url, form):
|
||||
try:
|
||||
result = fetch(url)
|
||||
except FetchError as error:
|
||||
form.add_error("url", error.user_message)
|
||||
return import_start(request, link_form=form, status=400)
|
||||
if result.status >= 400:
|
||||
form.add_error("url", f"Die Seite antwortete mit Fehler {result.status}.")
|
||||
return import_start(request, link_form=form, status=400)
|
||||
if "html" not in result.content_type.lower():
|
||||
form.add_error("url", "Unter dem Link liegt keine Webseite.")
|
||||
return import_start(request, link_form=form, status=400)
|
||||
|
||||
initial, image, page = schema_org.extract(result.text)
|
||||
extra = {"source_url": url, **({"import_image_url": image} if image else {})}
|
||||
if initial:
|
||||
return render_draft(request, {**initial, **extra}, "Von der Webseite übernommen")
|
||||
if not ai.is_configured():
|
||||
form.add_error("url", "Auf der Seite wurde kein strukturiertes Rezept gefunden, und die KI ist nicht "
|
||||
"eingerichtet. Kopiere den Rezepttext und trag ihn von Hand ein.")
|
||||
return import_start(request, link_form=form, status=400)
|
||||
text = page.title.strip() + "\n\n" + page.visible_text()
|
||||
try:
|
||||
draft = ai.structure_text(request.user, text, hint=f"Text der Webseite {url}:")
|
||||
except ai.AIError as error:
|
||||
form.add_error("url", error.user_message)
|
||||
return import_start(request, link_form=form, status=400)
|
||||
return render_draft(request, {**draft.to_form_initial(), **extra}, "Von der Webseite (per KI) erkannt",
|
||||
ai_draft=draft)
|
||||
|
||||
|
||||
@require_GET
|
||||
def share_target(request):
|
||||
"""Web Share Target (Android PWA): 'Teilen → Kochbuch' lands here with title/text/url."""
|
||||
for value in (request.GET.get("url", ""), request.GET.get("text", ""), request.GET.get("title", "")):
|
||||
match = _URL_RE.search(value)
|
||||
if match:
|
||||
return redirect(f"{reverse('recipes:import')}?{urlencode({'url': match.group(0)})}")
|
||||
text = request.GET.get("text", "").strip()
|
||||
if text:
|
||||
return import_start(request, text_form=TextImportForm(initial={"text": text}))
|
||||
return redirect("recipes:import")
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
"""Best-effort retrieval of an Instagram post caption without login.
|
||||
|
||||
Instagram has no login-free API for captions any more (the official oEmbed endpoint needs a
|
||||
Meta app and no longer returns post content). We make a single, honest request to the
|
||||
public embed page of the post and read the caption from it. Instagram frequently serves an
|
||||
empty app shell to servers instead – then the caller falls back to "paste the caption" or
|
||||
"import a screenshot". No login, no cookies, no evasion of bot detection.
|
||||
"""
|
||||
|
||||
import re
|
||||
from html.parser import HTMLParser
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .fetch import FetchError, fetch
|
||||
from .schema_org import parse_page
|
||||
|
||||
HOSTS = {"instagram.com", "www.instagram.com", "m.instagram.com", "instagr.am", "www.instagr.am"}
|
||||
_PATH_RE = re.compile(r"^/(?:[\w.]+/)?(?:p|reel|reels|tv)/([A-Za-z0-9_-]{5,40})(?:/|$)")
|
||||
_OG_CAPTION_RE = re.compile(r':\s*["“](.+)["”]\s*\.?\s*$', re.DOTALL)
|
||||
|
||||
|
||||
class InstagramUnavailable(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def shortcode(url):
|
||||
"""Shortcode of a post/reel URL, or None if the URL is not an Instagram post."""
|
||||
try:
|
||||
parts = urlsplit(url.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
if (parts.hostname or "").lower() not in HOSTS:
|
||||
return None
|
||||
match = _PATH_RE.match(parts.path)
|
||||
return match.group(1) if match else None
|
||||
|
||||
|
||||
def is_instagram(url):
|
||||
return (urlsplit(url.strip()).hostname or "").lower() in HOSTS
|
||||
|
||||
|
||||
class _EmbedCaptionParser(HTMLParser):
|
||||
"""Reads <div class="Caption"> of the embed page, without username and comment links."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(convert_charrefs=True)
|
||||
self.depth = 0 # >0 while inside the caption div
|
||||
self.skip_depth = 0
|
||||
self.parts = []
|
||||
self.image = ""
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
attrs = dict(attrs)
|
||||
classes = (attrs.get("class") or "").split()
|
||||
if tag == "img" and "EmbeddedMediaImage" in classes and not self.image:
|
||||
self.image = attrs.get("src", "")
|
||||
if self.depth:
|
||||
if tag == "br":
|
||||
self.parts.append("\n")
|
||||
return
|
||||
if self.skip_depth or "CaptionUsername" in classes or "CaptionComments" in classes:
|
||||
self.skip_depth += 1
|
||||
if tag == "div":
|
||||
self.depth += 1
|
||||
elif tag == "div" and "Caption" in classes:
|
||||
self.depth = 1
|
||||
|
||||
def handle_endtag(self, tag):
|
||||
if not self.depth:
|
||||
return
|
||||
if self.skip_depth:
|
||||
self.skip_depth -= 1
|
||||
if tag == "div":
|
||||
self.depth -= 1
|
||||
|
||||
def handle_data(self, data):
|
||||
if self.depth and not self.skip_depth:
|
||||
self.parts.append(data)
|
||||
|
||||
def caption(self):
|
||||
text = "".join(self.parts)
|
||||
text = re.sub(r"[ \t]+", " ", text)
|
||||
return re.sub(r"\n{3,}", "\n\n", text).strip()
|
||||
|
||||
|
||||
def parse_embed(page_html):
|
||||
"""Return (caption, image_url) from an embed page; caption may be empty."""
|
||||
parser = _EmbedCaptionParser()
|
||||
try:
|
||||
parser.feed(page_html)
|
||||
except Exception:
|
||||
return "", ""
|
||||
caption = parser.caption()
|
||||
image = parser.image
|
||||
if not caption:
|
||||
meta = parse_page(page_html).meta
|
||||
match = _OG_CAPTION_RE.search(meta.get("og:description", "") or meta.get("og:title", ""))
|
||||
caption = match.group(1).strip() if match else ""
|
||||
image = image or meta.get("og:image", "")
|
||||
return caption, image
|
||||
|
||||
|
||||
def fetch_caption(code):
|
||||
"""Try to read the caption of a post. Raises InstagramUnavailable if Instagram doesn't deliver it."""
|
||||
try:
|
||||
result = fetch(f"https://www.instagram.com/p/{code}/embed/captioned/")
|
||||
except FetchError as error:
|
||||
raise InstagramUnavailable(error.user_message) from None
|
||||
if result.status != 200:
|
||||
raise InstagramUnavailable(f"Instagram antwortete mit Status {result.status}.")
|
||||
caption, image = parse_embed(result.text)
|
||||
if len(caption) < 20:
|
||||
raise InstagramUnavailable("Instagram hat die Beschreibung nicht herausgegeben.")
|
||||
return caption, image
|
||||
@@ -0,0 +1,216 @@
|
||||
"""Extract schema.org/Recipe (JSON-LD) and page metadata from HTML without extra dependencies."""
|
||||
|
||||
import html
|
||||
import json
|
||||
import re
|
||||
from html.parser import HTMLParser
|
||||
|
||||
from . import parsing
|
||||
|
||||
_DURATION_RE = re.compile(r"^P(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:[\d.]+S)?)?$", re.IGNORECASE)
|
||||
_TAG_RE = re.compile(r"<[^>]+>")
|
||||
MAX_TEXT_CHARS = 15_000
|
||||
|
||||
|
||||
class PageParser(HTMLParser):
|
||||
"""Collects JSON-LD blocks, <meta> tags, the <title> and the visible text of a page."""
|
||||
|
||||
SKIP_TEXT = {"script", "style", "noscript", "svg", "nav", "footer", "header", "form", "button", "iframe"}
|
||||
BLOCK = {"p", "div", "li", "br", "h1", "h2", "h3", "h4", "h5", "h6", "tr", "section", "article", "ol", "ul"}
|
||||
|
||||
def __init__(self):
|
||||
super().__init__(convert_charrefs=True)
|
||||
self.json_ld, self.meta, self.title = [], {}, ""
|
||||
self._stack, self._buffer, self._text = [], None, []
|
||||
self._in_title = False
|
||||
|
||||
def handle_starttag(self, tag, attrs):
|
||||
attrs = dict(attrs)
|
||||
if tag == "script" and (attrs.get("type") or "").lower().startswith("application/ld+json"):
|
||||
self._buffer = []
|
||||
if tag == "meta":
|
||||
key = (attrs.get("property") or attrs.get("name") or "").lower()
|
||||
if key and attrs.get("content") and key not in self.meta:
|
||||
self.meta[key] = attrs["content"]
|
||||
if tag == "title":
|
||||
self._in_title = True
|
||||
if tag in self.BLOCK:
|
||||
self._text.append("\n")
|
||||
if tag not in ("br", "meta", "img", "link", "input", "hr"):
|
||||
self._stack.append(tag)
|
||||
|
||||
def handle_endtag(self, tag):
|
||||
if tag == "script" and self._buffer is not None:
|
||||
self.json_ld.append("".join(self._buffer))
|
||||
self._buffer = None
|
||||
if tag == "title":
|
||||
self._in_title = False
|
||||
if tag in self._stack:
|
||||
while self._stack and self._stack.pop() != tag:
|
||||
pass
|
||||
|
||||
def handle_data(self, data):
|
||||
if self._buffer is not None:
|
||||
self._buffer.append(data)
|
||||
return
|
||||
if self._in_title:
|
||||
self.title += data
|
||||
if not any(t in self.SKIP_TEXT for t in self._stack):
|
||||
self._text.append(data)
|
||||
|
||||
def visible_text(self):
|
||||
text = "".join(self._text)
|
||||
text = re.sub(r"[ \t\r\f\v]+", " ", text)
|
||||
text = re.sub(r"\n\s*\n+", "\n", text)
|
||||
return text.strip()[:MAX_TEXT_CHARS]
|
||||
|
||||
|
||||
def parse_page(page_html):
|
||||
parser = PageParser()
|
||||
try:
|
||||
parser.feed(page_html)
|
||||
parser.close()
|
||||
except Exception: # malformed HTML must never break the import
|
||||
pass
|
||||
return parser
|
||||
|
||||
|
||||
def _types(node):
|
||||
value = node.get("@type", [])
|
||||
return {t.lower() for t in (value if isinstance(value, list) else [value]) if isinstance(t, str)}
|
||||
|
||||
|
||||
def find_recipe(data):
|
||||
"""Depth-first search for the first object whose @type includes Recipe."""
|
||||
if isinstance(data, list):
|
||||
for item in data:
|
||||
found = find_recipe(item)
|
||||
if found:
|
||||
return found
|
||||
elif isinstance(data, dict):
|
||||
if "recipe" in _types(data):
|
||||
return data
|
||||
for key in ("@graph", "mainEntity", "mainEntityOfPage", "itemListElement", "item"):
|
||||
if key in data:
|
||||
found = find_recipe(data[key])
|
||||
if found:
|
||||
return found
|
||||
return None
|
||||
|
||||
|
||||
def _clean(text):
|
||||
if not isinstance(text, str):
|
||||
return ""
|
||||
text = html.unescape(_TAG_RE.sub(" ", text))
|
||||
text = re.sub(r"\s+", " ", text).strip()
|
||||
return re.sub(r"\s+([.,;:!?])", r"\1", text)
|
||||
|
||||
|
||||
def _duration_minutes(value):
|
||||
if not isinstance(value, str):
|
||||
return None
|
||||
match = _DURATION_RE.match(value.strip())
|
||||
if not match or not any(match.groups()):
|
||||
return None
|
||||
days, hours, minutes = (int(g) if g else 0 for g in match.groups())
|
||||
total = days * 1440 + hours * 60 + minutes
|
||||
return total if 0 < total <= 7 * 1440 else None
|
||||
|
||||
|
||||
def _yield(value):
|
||||
if isinstance(value, list):
|
||||
value = next((v for v in value if isinstance(v, (str, int))), None)
|
||||
if isinstance(value, (int, float)):
|
||||
return int(value), ""
|
||||
if isinstance(value, str):
|
||||
match = re.search(r"(\d+)", value)
|
||||
if match:
|
||||
label = _clean(value[match.end():]).strip(" ()")
|
||||
return int(match.group(1)), label[:30]
|
||||
return None, ""
|
||||
|
||||
|
||||
def _steps(value):
|
||||
steps = []
|
||||
if isinstance(value, str):
|
||||
return parsing.parse_steps(_clean_multiline(value))
|
||||
if isinstance(value, dict):
|
||||
value = [value]
|
||||
for item in value or []:
|
||||
if isinstance(item, str):
|
||||
steps.append(_clean(item))
|
||||
elif isinstance(item, dict):
|
||||
types = _types(item)
|
||||
if "howtosection" in types:
|
||||
steps.extend(_steps(item.get("itemListElement")))
|
||||
else:
|
||||
steps.append(_clean(item.get("text") or item.get("name") or ""))
|
||||
return [s for s in steps if s]
|
||||
|
||||
|
||||
def _clean_multiline(text):
|
||||
text = re.sub(r"<br\s*/?>|</p>|</li>", "\n", text, flags=re.IGNORECASE)
|
||||
return html.unescape(_TAG_RE.sub("", text))
|
||||
|
||||
|
||||
def _keywords(recipe):
|
||||
words = []
|
||||
for key in ("recipeCategory", "recipeCuisine", "keywords"):
|
||||
value = recipe.get(key)
|
||||
if isinstance(value, str):
|
||||
words.extend(value.split(","))
|
||||
elif isinstance(value, list):
|
||||
words.extend(v for v in value if isinstance(v, str))
|
||||
return parsing.parse_tags(",".join(_clean(w) for w in words if len(w.strip()) <= 30))[:6]
|
||||
|
||||
|
||||
def _image(value):
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, list) and value:
|
||||
return _image(value[0])
|
||||
if isinstance(value, dict):
|
||||
return value.get("url") or value.get("contentUrl") or ""
|
||||
return ""
|
||||
|
||||
|
||||
def recipe_to_initial(recipe):
|
||||
"""Map a schema.org Recipe dict to RecipeForm initial values (+ image URL)."""
|
||||
servings, label = _yield(recipe.get("recipeYield"))
|
||||
ingredients = recipe.get("recipeIngredient") or recipe.get("ingredients") or []
|
||||
if isinstance(ingredients, str):
|
||||
ingredients = [ingredients]
|
||||
prep = _duration_minutes(recipe.get("prepTime"))
|
||||
cook = _duration_minutes(recipe.get("cookTime"))
|
||||
if prep is None and cook is None:
|
||||
prep = _duration_minutes(recipe.get("totalTime"))
|
||||
initial = {
|
||||
"title": _clean(recipe.get("name"))[:200],
|
||||
"description": _clean(recipe.get("description")),
|
||||
"prep_minutes": prep,
|
||||
"cook_minutes": cook,
|
||||
"ingredients_text": "\n".join(line for line in (_clean(i) for i in ingredients if isinstance(i, str)) if line),
|
||||
"steps_text": parsing.steps_to_text(_steps(recipe.get("recipeInstructions"))),
|
||||
"tags_text": ", ".join(_keywords(recipe)),
|
||||
}
|
||||
if servings and 1 <= servings <= 100:
|
||||
initial["servings"] = servings
|
||||
initial["servings_label"] = label if label and len(label) < 30 else "Portionen"
|
||||
return initial, _image(recipe.get("image"))
|
||||
|
||||
|
||||
def extract(page_html):
|
||||
"""Return (initial, image_url, page) – initial is None when the page has no usable Recipe."""
|
||||
page = parse_page(page_html)
|
||||
for block in page.json_ld:
|
||||
try:
|
||||
data = json.loads(block.strip())
|
||||
except json.JSONDecodeError:
|
||||
# Some sites put several objects or trailing commas in one block; skip those.
|
||||
continue
|
||||
recipe = find_recipe(data)
|
||||
if recipe:
|
||||
initial, image = recipe_to_initial(recipe)
|
||||
if initial["ingredients_text"] or initial["steps_text"]:
|
||||
return initial, image or page.meta.get("og:image", ""), page
|
||||
return None, page.meta.get("og:image", ""), page
|
||||
@@ -10,6 +10,8 @@ urlpatterns = [
|
||||
path("rezepte/import/", import_views.import_start, name="import"),
|
||||
path("rezepte/import/foto/", import_views.import_photo, name="import_photo"),
|
||||
path("rezepte/import/text/", import_views.import_text, name="import_text"),
|
||||
path("rezepte/import/link/", import_views.import_link, name="import_link"),
|
||||
path("teilen/", import_views.share_target, name="share"),
|
||||
path("rezepte/<int:pk>/", views.recipe_detail, name="detail"),
|
||||
path("rezepte/<int:pk>/bearbeiten/", views.recipe_edit, name="edit"),
|
||||
path("rezepte/<int:pk>/loeschen/", views.recipe_delete, name="delete"),
|
||||
|
||||
+21
-2
@@ -3,6 +3,7 @@ from datetime import timedelta
|
||||
|
||||
from django.conf import settings
|
||||
from django.contrib import messages
|
||||
from django.core.files.uploadedfile import SimpleUploadedFile
|
||||
from django.core.paginator import Paginator
|
||||
from django.db import transaction
|
||||
from django.db.models import Exists, OuterRef, Prefetch
|
||||
@@ -14,7 +15,7 @@ from django.views.decorators.http import require_GET, require_POST
|
||||
|
||||
from core import ratelimit
|
||||
|
||||
from . import ai, images
|
||||
from . import ai, fetch, images
|
||||
from .forms import CommentForm, PhotoUploadForm, RecipeForm, SearchForm
|
||||
from .models import Comment, Favorite, Recipe, RecipeImage, Tag
|
||||
|
||||
@@ -27,6 +28,19 @@ def _visible(request):
|
||||
return Recipe.objects.visible_to(request.user)
|
||||
|
||||
|
||||
def download_image(request, url):
|
||||
"""Fetch a suggested source photo (SSRF-safe). Returns an uploaded-file object or None."""
|
||||
try:
|
||||
result = fetch.fetch(url, accept="image/*", max_bytes=images.max_upload_bytes())
|
||||
except fetch.FetchError as error:
|
||||
messages.warning(request, f"Das Bild der Quelle konnte nicht geladen werden: {error.user_message}")
|
||||
return None
|
||||
if result.status != 200 or not result.content_type.lower().startswith("image/"):
|
||||
messages.warning(request, "Das Bild der Quelle konnte nicht geladen werden.")
|
||||
return None
|
||||
return SimpleUploadedFile("quelle.jpg", result.body, content_type=result.content_type)
|
||||
|
||||
|
||||
def attach_photos(request, recipe, files):
|
||||
"""Process uploaded files and attach them. Problems are reported as messages, not exceptions."""
|
||||
has_cover = recipe.images.filter(is_cover=True).exists()
|
||||
@@ -124,7 +138,12 @@ def recipe_create(request):
|
||||
form = RecipeForm(request.POST or None, request.FILES or None)
|
||||
if request.method == "POST" and form.is_valid():
|
||||
recipe = form.save(owner=request.user)
|
||||
attach_photos(request, recipe, form.cleaned_data.get("photos") or [])
|
||||
photos = list(form.cleaned_data.get("photos") or [])
|
||||
if form.cleaned_data.get("use_import_image") and form.cleaned_data.get("import_image_url"):
|
||||
downloaded = download_image(request, form.cleaned_data["import_image_url"])
|
||||
if downloaded:
|
||||
photos.insert(0, downloaded)
|
||||
attach_photos(request, recipe, photos[: settings.IMAGE_MAX_PER_RECIPE])
|
||||
messages.success(request, "Rezept gespeichert.")
|
||||
return redirect("recipes:detail", pk=recipe.pk)
|
||||
return _render_form(request, form)
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""Generates the PWA icons in static/icons (run once; output is committed)."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
OUT = Path(__file__).resolve().parent.parent / "static" / "icons"
|
||||
ACCENT = (180, 83, 42)
|
||||
CREAM = (250, 247, 242)
|
||||
|
||||
|
||||
def draw_pot(size, padding):
|
||||
img = Image.new("RGB", (size, size), ACCENT)
|
||||
d = ImageDraw.Draw(img)
|
||||
s = size - 2 * padding
|
||||
x0, y0 = padding, padding
|
||||
# pot body
|
||||
d.rounded_rectangle([x0 + s * 0.18, y0 + s * 0.42, x0 + s * 0.82, y0 + s * 0.82], radius=s * 0.08, fill=CREAM)
|
||||
# rim
|
||||
d.rounded_rectangle([x0 + s * 0.12, y0 + s * 0.36, x0 + s * 0.88, y0 + s * 0.45], radius=s * 0.04, fill=CREAM)
|
||||
# handles
|
||||
d.rounded_rectangle([x0 + s * 0.05, y0 + s * 0.5, x0 + s * 0.2, y0 + s * 0.58], radius=s * 0.03, fill=CREAM)
|
||||
d.rounded_rectangle([x0 + s * 0.8, y0 + s * 0.5, x0 + s * 0.95, y0 + s * 0.58], radius=s * 0.03, fill=CREAM)
|
||||
# steam
|
||||
for i, dx in enumerate((0.35, 0.5, 0.65)):
|
||||
top = y0 + s * (0.12 if i == 1 else 0.17)
|
||||
d.line([x0 + s * dx, top, x0 + s * dx, y0 + s * 0.3], fill=CREAM, width=max(2, int(s * 0.045)))
|
||||
return img
|
||||
|
||||
|
||||
def main():
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
draw_pot(192, 16).save(OUT / "icon-192.png")
|
||||
draw_pot(512, 40).save(OUT / "icon-512.png")
|
||||
draw_pot(512, 110).save(OUT / "icon-maskable-512.png") # content inside the safe zone
|
||||
draw_pot(180, 14).save(OUT / "apple-touch-icon.png")
|
||||
draw_pot(64, 4).save(OUT / "favicon.png")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -280,3 +280,6 @@ h3 { font-size: 1rem; margin: 1rem 0 0.25rem; color: var(--muted); }
|
||||
.preview .ingredients .amount { flex-basis: 5.5rem; }
|
||||
button[disabled] { opacity: 0.6; cursor: progress; }
|
||||
code { font-size: 0.9em; }
|
||||
.help-box { border-left: 4px solid var(--accent); }
|
||||
.help-box ol { padding-left: 1.2rem; margin: 0; }
|
||||
.help-box li { margin-bottom: 0.4rem; }
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 716 B |
Binary file not shown.
|
After Width: | Height: | Size: 316 B |
Binary file not shown.
|
After Width: | Height: | Size: 787 B |
Binary file not shown.
|
After Width: | Height: | Size: 2.4 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 2.2 KiB |
@@ -61,3 +61,8 @@ window.addEventListener("pageshow", () => {
|
||||
b.textContent = b.dataset.label;
|
||||
});
|
||||
});
|
||||
|
||||
// Register the (no-cache) service worker so the app can be installed and used as share target.
|
||||
if ("serviceWorker" in navigator) {
|
||||
window.addEventListener("load", () => navigator.serviceWorker.register("/sw.js").catch(() => {}));
|
||||
}
|
||||
|
||||
@@ -6,6 +6,11 @@
|
||||
<meta name="theme-color" content="#b4532a">
|
||||
<meta name="robots" content="noindex, nofollow">
|
||||
<title>{% block title %}Kochbuch{% endblock %} · Kochbuch</title>
|
||||
<link rel="manifest" href="{% url 'core:manifest' %}">
|
||||
<link rel="icon" type="image/png" href="{% static 'icons/favicon.png' %}">
|
||||
<link rel="apple-touch-icon" href="{% static 'icons/apple-touch-icon.png' %}">
|
||||
<meta name="apple-mobile-web-app-capable" content="yes">
|
||||
<meta name="apple-mobile-web-app-title" content="Kochbuch">
|
||||
<link rel="stylesheet" href="{% static 'css/app.css' %}">
|
||||
<script src="{% static 'js/app.js' %}" defer></script>
|
||||
</head>
|
||||
@@ -17,6 +22,8 @@
|
||||
<summary aria-label="Menü">☰</summary>
|
||||
<nav>
|
||||
<a href="{% url 'recipes:list' %}">Rezepte</a>
|
||||
<a href="{% url 'recipes:list' %}?ansicht=favoriten">Favoriten</a>
|
||||
<a href="{% url 'recipes:import' %}">Importieren</a>
|
||||
<a href="{% url 'accounts:profile' %}">Mein Konto</a>
|
||||
{% if user.is_admin %}
|
||||
<a href="{% url 'accounts:admin_users' %}">Nutzer verwalten</a>
|
||||
|
||||
@@ -4,7 +4,26 @@
|
||||
<h1>Rezept importieren</h1>
|
||||
<p class="muted">Das Ergebnis ist immer ein Entwurf, den du prüfst und erst dann speicherst.</p>
|
||||
|
||||
{% block link_import %}{% endblock %}
|
||||
<section class="card">
|
||||
<h2>🔗 Link</h2>
|
||||
<form method="post" action="{% url 'recipes:import_link' %}" novalidate>
|
||||
{% csrf_token %}
|
||||
{% with form=link_form %}{% include "form_fields.html" %}{% endwith %}
|
||||
<p class="muted small">Rezeptseiten werden direkt ausgelesen. Bei Instagram wird die Beschreibung des Beitrags übernommen{% if ai_enabled %} und per KI aufbereitet{% endif %}.</p>
|
||||
<button type="submit" class="button" data-busy-text="Wird geladen …">Importieren</button>
|
||||
</form>
|
||||
</section>
|
||||
|
||||
{% if instagram_failed %}
|
||||
<section class="card help-box">
|
||||
<h2>So klappt es mit Instagram</h2>
|
||||
<ol>
|
||||
<li>Öffne den Beitrag in der Instagram-App und tippe auf „… mehr“, damit die ganze Beschreibung sichtbar ist.</li>
|
||||
<li>Mach einen <strong>Screenshot</strong> und lade ihn unten bei „Foto oder Screenshot“ hoch{% if not ai_enabled %} (braucht die KI){% endif %}, <em>oder</em></li>
|
||||
<li>öffne den Beitrag im Browser, markiere den Text, kopiere ihn und füge ihn unten bei „Text einfügen“ ein.</li>
|
||||
</ol>
|
||||
</section>
|
||||
{% endif %}
|
||||
|
||||
<section class="card">
|
||||
<h2>📷 Foto oder Screenshot</h2>
|
||||
@@ -22,15 +41,16 @@
|
||||
|
||||
<section class="card">
|
||||
<h2>📝 Text einfügen</h2>
|
||||
{% if ai_enabled %}
|
||||
<form method="post" action="{% url 'recipes:import_text' %}" novalidate>
|
||||
{% csrf_token %}
|
||||
{% with form=text_form %}{% include "form_fields.html" %}{% endwith %}
|
||||
{% if ai_enabled %}
|
||||
<button type="submit" class="button" data-busy-text="KI bereitet auf …">Mit KI aufbereiten</button>
|
||||
{% else %}
|
||||
<p class="muted small">Ohne KI wird der Text als Beschreibung übernommen und du teilst ihn selbst auf.</p>
|
||||
<button type="submit" class="button">Als Entwurf übernehmen</button>
|
||||
{% endif %}
|
||||
</form>
|
||||
{% else %}
|
||||
{% include "recipes/_ai_disabled.html" %}
|
||||
{% endif %}
|
||||
</section>
|
||||
|
||||
<p><a href="{% url 'recipes:create' %}">Oder ganz von Hand eintragen →</a></p>
|
||||
|
||||
+12
-2
@@ -164,9 +164,19 @@ def test_without_key_ai_is_disabled_but_app_works(settings, user_client):
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_without_key_imports_give_clear_message(settings, user_client):
|
||||
def test_without_key_text_import_becomes_description(settings, user_client):
|
||||
settings.OPENAI_API_KEY = ""
|
||||
response = user_client.post(reverse("recipes:import_text"), {"text": "200 g Mehl, 2 Eier, alles verrühren und backen"})
|
||||
text = "200 g Mehl, 2 Eier, alles verrühren und backen"
|
||||
response = user_client.post(reverse("recipes:import_text"), {"text": text})
|
||||
assert "nicht eingerichtet" in response.content.decode()
|
||||
assert response.context["form"].initial["description"] == text
|
||||
assert not Recipe.objects.exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_without_key_photo_import_gives_clear_message(settings, user_client):
|
||||
settings.OPENAI_API_KEY = ""
|
||||
response = user_client.post(reverse("recipes:import_photo"), {"photos": [image_upload()]}, follow=True)
|
||||
assert "nicht eingerichtet" in response.content.decode()
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
import io
|
||||
import json
|
||||
import socket
|
||||
|
||||
import pytest
|
||||
from django.urls import reverse
|
||||
from PIL import Image
|
||||
|
||||
from recipes import ai, fetch, import_views, instagram, schema_org
|
||||
from recipes import views as recipe_views
|
||||
from recipes.fetch import FetchError, FetchResult
|
||||
from recipes.models import Recipe
|
||||
|
||||
from .recipe_helpers import form_data
|
||||
from .test_ai import ai_answer
|
||||
|
||||
# --- SSRF protection ------------------------------------------------------------------------
|
||||
|
||||
|
||||
def fake_dns(monkeypatch, ip):
|
||||
monkeypatch.setattr(socket, "getaddrinfo", lambda *a, **k: [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, 443))])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ip", ["127.0.0.1", "10.1.2.3", "192.168.178.1", "172.17.0.1", "169.254.169.254", "100.64.0.1", "0.0.0.0"])
|
||||
def test_internal_addresses_blocked(monkeypatch, ip):
|
||||
fake_dns(monkeypatch, ip)
|
||||
with pytest.raises(FetchError, match="nicht erlaubt"):
|
||||
fetch.fetch("https://harmless-looking.example/recipe")
|
||||
|
||||
|
||||
def test_ipv6_loopback_and_mapped_blocked(monkeypatch):
|
||||
for ip in ["::1", "::ffff:127.0.0.1", "fd00::1"]:
|
||||
monkeypatch.setattr(socket, "getaddrinfo", lambda *a, ip=ip, **k: [(socket.AF_INET6, socket.SOCK_STREAM, 6, "", (ip, 443, 0, 0))])
|
||||
with pytest.raises(FetchError):
|
||||
fetch.fetch("https://x.example/")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", ["file:///etc/passwd", "ftp://x.example/", "https://user:[email protected]/", "https://x.example:22/", "gopher://x"])
|
||||
def test_bad_urls_rejected(url):
|
||||
with pytest.raises(FetchError):
|
||||
fetch.fetch(url)
|
||||
|
||||
|
||||
def test_redirect_target_is_checked_again(monkeypatch):
|
||||
class Response:
|
||||
status = 302
|
||||
headers = {"Location": "http://internal.example/admin"}
|
||||
|
||||
def release_conn(self):
|
||||
pass
|
||||
|
||||
def fake_request(url, accept, timeout):
|
||||
if "internal" in url:
|
||||
raise FetchError("Diese Adresse ist nicht erlaubt (internes Netz).")
|
||||
return Response()
|
||||
|
||||
monkeypatch.setattr(fetch, "_request_once", fake_request)
|
||||
with pytest.raises(FetchError, match="nicht erlaubt"):
|
||||
fetch.fetch("https://public.example/")
|
||||
|
||||
|
||||
# --- schema.org parsing -----------------------------------------------------------------------
|
||||
|
||||
RECIPE_LD = {
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
{"@type": "WebPage", "name": "Seite"},
|
||||
{
|
||||
"@type": ["Recipe", "NewsArticle"],
|
||||
"name": "Kürbissuppe & Ingwer",
|
||||
"description": "<p>Wärmt von innen.</p>",
|
||||
"recipeYield": ["4", "4 Portionen"],
|
||||
"prepTime": "PT15M",
|
||||
"cookTime": "PT1H5M",
|
||||
"image": [{"@type": "ImageObject", "url": "https://cdn.example/suppe.jpg"}],
|
||||
"keywords": "Suppe, Herbst, vegan",
|
||||
"recipeCategory": "Hauptgericht",
|
||||
"recipeIngredient": ["1 kg Hokkaido", "2 cm Ingwer", "400 ml Kokosmilch"],
|
||||
"recipeInstructions": [
|
||||
{"@type": "HowToSection", "name": "Vorbereitung", "itemListElement": [
|
||||
{"@type": "HowToStep", "text": "Kürbis würfeln."},
|
||||
]},
|
||||
{"@type": "HowToStep", "text": "Alles <b>pürieren</b>."},
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def page_with(ld, extra=""):
|
||||
return f"""<html><head><title>Suppe</title><meta property="og:image" content="https://cdn.example/og.jpg">
|
||||
<script type="application/ld+json">{{ broken json,</script>
|
||||
<script type="application/ld+json">{json.dumps(ld)}</script>{extra}</head><body><p>Text</p></body></html>"""
|
||||
|
||||
|
||||
def test_schema_org_recipe_extracted():
|
||||
initial, image, _ = schema_org.extract(page_with(RECIPE_LD))
|
||||
assert initial["title"] == "Kürbissuppe & Ingwer"
|
||||
assert initial["description"] == "Wärmt von innen."
|
||||
assert initial["servings"] == 4 and initial["prep_minutes"] == 15 and initial["cook_minutes"] == 65
|
||||
assert initial["ingredients_text"].splitlines() == ["1 kg Hokkaido", "2 cm Ingwer", "400 ml Kokosmilch"]
|
||||
assert initial["steps_text"] == "Kürbis würfeln.\n\nAlles pürieren."
|
||||
assert initial["tags_text"] == "hauptgericht, suppe, herbst, vegan"
|
||||
assert image == "https://cdn.example/suppe.jpg"
|
||||
|
||||
|
||||
def test_schema_org_string_instructions_and_yield():
|
||||
ld = {"@type": "Recipe", "name": "X", "recipeYield": "12 Stück", "recipeIngredient": ["Mehl"],
|
||||
"recipeInstructions": "Erst das.<br>Dann das."}
|
||||
initial, image, _ = schema_org.extract(page_with(ld))
|
||||
assert initial["servings"] == 12 and initial["servings_label"] == "Stück"
|
||||
assert initial["steps_text"] == "Erst das.\n\nDann das."
|
||||
assert image == "https://cdn.example/og.jpg" # falls back to og:image
|
||||
|
||||
|
||||
def test_page_without_recipe():
|
||||
initial, image, page = schema_org.extract("<html><head><title>Blog</title></head><body><nav>Menü</nav><p>Hallo Welt</p><script>x()</script></body></html>")
|
||||
assert initial is None
|
||||
assert "Hallo Welt" in page.visible_text() and "Menü" not in page.visible_text() and "x()" not in page.visible_text()
|
||||
|
||||
|
||||
# --- Instagram ---------------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"url, code",
|
||||
[
|
||||
("https://www.instagram.com/p/C1a2b3c4D5e/", "C1a2b3c4D5e"),
|
||||
("https://instagram.com/reel/C1a2b3c4D5e?igsh=abc", "C1a2b3c4D5e"),
|
||||
("https://www.instagram.com/kochprofi/p/C1a2b3c4D5e/", "C1a2b3c4D5e"),
|
||||
("https://www.instagram.com/reels/DaU7f-CjE1c/", "DaU7f-CjE1c"),
|
||||
("https://www.instagram.com/kochprofi/", None),
|
||||
("https://evil.example/p/C1a2b3c4D5e/", None),
|
||||
("https://instagram.com.evil.example/p/C1a2b3c4D5e/", None),
|
||||
],
|
||||
)
|
||||
def test_shortcode(url, code):
|
||||
assert instagram.shortcode(url) == code
|
||||
|
||||
|
||||
EMBED_HTML = """<html><body><div class="Embed">
|
||||
<img class="EmbeddedMediaImage" src="https://scontent.cdninstagram.com/x.jpg">
|
||||
<div class="Caption"><a class="CaptionUsername" href="#">kochprofi</a><br><br>Einfaches Dal 🍛<br><br>
|
||||
Zutaten:<br>200 g rote Linsen<br>1 Dose Kokosmilch<br><br>Linsen kochen, fertig!
|
||||
<div class="CaptionComments"><a href="#">Alle 23 Kommentare ansehen</a></div></div></div></body></html>"""
|
||||
|
||||
|
||||
def test_parse_embed_caption():
|
||||
caption, image = instagram.parse_embed(EMBED_HTML)
|
||||
assert caption.startswith("Einfaches Dal")
|
||||
assert "200 g rote Linsen" in caption and "kochprofi" not in caption and "Kommentare" not in caption
|
||||
assert image == "https://scontent.cdninstagram.com/x.jpg"
|
||||
|
||||
|
||||
def test_parse_og_description_fallback():
|
||||
html = '<meta property="og:description" content="1.234 likes, 12 comments - kochprofi am 1. März 2026: "Dal mit 200 g Linsen und Kokosmilch"">'
|
||||
caption, _ = instagram.parse_embed(html)
|
||||
assert caption == "Dal mit 200 g Linsen und Kokosmilch"
|
||||
|
||||
|
||||
def test_app_shell_means_unavailable(monkeypatch):
|
||||
monkeypatch.setattr(instagram, "fetch", lambda url: FetchResult(url, 200, "text/html", b"<html><title>Instagram</title></html>"))
|
||||
with pytest.raises(instagram.InstagramUnavailable):
|
||||
instagram.fetch_caption("C1a2b3c4D5e")
|
||||
|
||||
|
||||
# --- Views ---------------------------------------------------------------------------------------
|
||||
|
||||
def stub_fetch(monkeypatch, html, status=200, content_type="text/html; charset=utf-8"):
|
||||
calls = []
|
||||
|
||||
def fake(url, **kwargs):
|
||||
calls.append(url)
|
||||
return FetchResult(url, status, content_type, html.encode())
|
||||
|
||||
monkeypatch.setattr(import_views, "fetch", fake)
|
||||
return calls
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_link_import_uses_schema_org_without_ai(monkeypatch, settings, user_client):
|
||||
settings.OPENAI_API_KEY = ""
|
||||
stub_fetch(monkeypatch, page_with(RECIPE_LD))
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://rezepte.example/suppe"})
|
||||
assert response.status_code == 200
|
||||
initial = response.context["form"].initial
|
||||
assert initial["title"] == "Kürbissuppe & Ingwer"
|
||||
assert initial["source_url"] == "https://rezepte.example/suppe"
|
||||
html = response.content.decode()
|
||||
assert 'name="import_image_url"' in html and "Bild der Quelle" in html
|
||||
assert not Recipe.objects.exists()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_link_import_falls_back_to_ai(monkeypatch, settings, user_client):
|
||||
settings.OPENAI_API_KEY = "sk-test"
|
||||
stub_fetch(monkeypatch, "<html><title>Omas Dal</title><body><p>200 g Linsen kochen ...</p></body></html>")
|
||||
seen = []
|
||||
monkeypatch.setattr(ai, "_create_completion", lambda messages: seen.append(messages) or ai_answer())
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://blog.example/dal"})
|
||||
assert response.context["form"].initial["title"] == "Linsen-Dal"
|
||||
assert "200 g Linsen" in seen[0][1]["content"]
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_link_import_without_recipe_and_without_ai(monkeypatch, settings, user_client):
|
||||
settings.OPENAI_API_KEY = ""
|
||||
stub_fetch(monkeypatch, "<html><body>nichts</body></html>")
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://blog.example/x"})
|
||||
assert response.status_code == 400
|
||||
assert "kein strukturiertes Rezept" in response.content.decode()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_link_import_reports_fetch_errors(monkeypatch, user_client):
|
||||
def fail(url, **kwargs):
|
||||
raise FetchError("Die Seite ist nicht erreichbar.")
|
||||
|
||||
monkeypatch.setattr(import_views, "fetch", fail)
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://down.example/"})
|
||||
assert "nicht erreichbar" in response.content.decode()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_instagram_failure_offers_paste_fallback(monkeypatch, user_client):
|
||||
def unavailable(code):
|
||||
raise instagram.InstagramUnavailable("nope")
|
||||
|
||||
monkeypatch.setattr(instagram, "fetch_caption", unavailable)
|
||||
url = "https://www.instagram.com/p/C1a2b3c4D5e/"
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": url}, follow=True)
|
||||
html = response.content.decode()
|
||||
assert "So klappt es mit Instagram" in html
|
||||
assert response.context["text_form"].initial["source_url"] == url
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_instagram_caption_structured_by_ai(monkeypatch, settings, user_client):
|
||||
settings.OPENAI_API_KEY = "sk-test"
|
||||
monkeypatch.setattr(instagram, "fetch_caption", lambda code: ("Dal: 200 g Linsen, Kokosmilch. Kochen!", "https://cdn.example/i.jpg"))
|
||||
seen = []
|
||||
monkeypatch.setattr(ai, "_create_completion", lambda messages: seen.append(messages) or ai_answer())
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://www.instagram.com/reel/C1a2b3c4D5e/"})
|
||||
initial = response.context["form"].initial
|
||||
assert initial["title"] == "Linsen-Dal" and initial["import_image_url"] == "https://cdn.example/i.jpg"
|
||||
assert "Instagram" in seen[0][1]["content"]
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_instagram_profile_link_rejected(user_client):
|
||||
response = user_client.post(reverse("recipes:import_link"), {"url": "https://www.instagram.com/kochprofi/"})
|
||||
assert "einzelnen Beitrag" in response.content.decode()
|
||||
|
||||
|
||||
def jpeg_bytes():
|
||||
buffer = io.BytesIO()
|
||||
Image.new("RGB", (800, 600), (10, 200, 10)).save(buffer, "JPEG")
|
||||
return buffer.getvalue()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_saving_draft_downloads_source_image(monkeypatch, user_client):
|
||||
monkeypatch.setattr(recipe_views.fetch, "fetch",
|
||||
lambda url, **kw: FetchResult(url, 200, "image/jpeg", jpeg_bytes()))
|
||||
user_client.post(reverse("recipes:create"), form_data(import_image_url="https://cdn.example/i.jpg", use_import_image="on"))
|
||||
recipe = Recipe.objects.get()
|
||||
assert recipe.images.count() == 1 and recipe.images.get().is_cover
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_source_image_skipped_when_unticked(monkeypatch, user_client):
|
||||
called = []
|
||||
monkeypatch.setattr(recipe_views.fetch, "fetch", lambda url, **kw: called.append(url))
|
||||
user_client.post(reverse("recipes:create"), form_data(import_image_url="https://cdn.example/i.jpg"))
|
||||
assert not called and Recipe.objects.get().images.count() == 0
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_source_image_to_internal_address_is_refused(monkeypatch, user_client):
|
||||
fake_dns(monkeypatch, "10.0.0.5")
|
||||
response = user_client.post(reverse("recipes:create"),
|
||||
form_data(import_image_url="https://nas.example/secret.jpg", use_import_image="on"), follow=True)
|
||||
assert "nicht erlaubt" in response.content.decode()
|
||||
assert Recipe.objects.get().images.count() == 0
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_source_image_must_be_an_image(monkeypatch, user_client):
|
||||
monkeypatch.setattr(recipe_views.fetch, "fetch", lambda url, **kw: FetchResult(url, 200, "text/html", b"<html>"))
|
||||
user_client.post(reverse("recipes:create"), form_data(import_image_url="https://x.example/", use_import_image="on"))
|
||||
assert Recipe.objects.get().images.count() == 0
|
||||
|
||||
|
||||
# --- PWA / share target ----------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@pytest.mark.parametrize(
|
||||
"params, expected",
|
||||
[
|
||||
({"url": "https://www.instagram.com/p/C1a2b3c4D5e/"}, "https%3A%2F%2Fwww.instagram.com%2Fp%2FC1a2b3c4D5e%2F"),
|
||||
({"text": "Schau mal https://www.instagram.com/reel/XYZ12345/ lecker!"}, "reel%2FXYZ12345%2F"),
|
||||
],
|
||||
)
|
||||
def test_share_target_redirects_to_import(user_client, params, expected):
|
||||
response = user_client.get(reverse("recipes:share"), params)
|
||||
assert response.status_code == 302 and expected in response["Location"]
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_share_target_with_plain_text(user_client):
|
||||
response = user_client.get(reverse("recipes:share"), {"text": "200 g Mehl und 2 Eier verrühren"})
|
||||
assert response.context["text_form"].initial["text"].startswith("200 g Mehl")
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_import_page_prefills_url(user_client):
|
||||
response = user_client.get(reverse("recipes:import"), {"url": "https://x.example/r"})
|
||||
assert response.context["link_form"].initial["url"] == "https://x.example/r"
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
def test_manifest_and_service_worker(client):
|
||||
manifest = client.get("/manifest.webmanifest")
|
||||
assert manifest.status_code == 200 and manifest["Content-Type"] == "application/manifest+json"
|
||||
data = manifest.json()
|
||||
assert data["share_target"]["action"] == reverse("recipes:share")
|
||||
sw = client.get("/sw.js")
|
||||
assert sw.status_code == 200 and "javascript" in sw["Content-Type"]
|
||||
assert "fetch" not in sw.content.decode() # no caching of private pages
|
||||
Reference in New Issue
Block a user