Files
theread.me/photo-upload/README.md
T

18 KiB
Raw Blame History

iPhone → blog, with GitHub Actions

Photos → Share → Publish to blog → optional caption and highlight name → upload.

The authenticated upload service converts photos to progressive JPEG (maximum 1600 px and 300 KiB), applies orientation, converts embedded color profiles to sRGB, and strips public metadata. Originals stay in a private volume. A worker commits optimized images and an _art entry to github.com/mdibaiee/mahdi.blog, on master.

Every push to master triggers .github/workflows/publish.yml. GitHub Actions builds Jekyll and uploads the static output over SSH. nginx switches to the new release only after the transfer completes. Other blog edits use the same workflow. No Jekyll build runs on the server, and Nextcloud is not involved.

GitHub repository configuration

Actions is already enabled; there were no publishing workflows or deployment secrets when inspected. Add these at Settings → Secrets and variables → Actions:

Repository secret Value
DEPLOY_SSH_KEY Private SSH key whose public key is authorized for deployment on theread.me
DEPLOY_KNOWN_HOSTS Verified SSH known_hosts entry for theread.me (include [host]:port if nonstandard)

The workflow has these defaults; override with repository variables if needed:

Variable Default
DEPLOY_HOST theread.me
DEPLOY_USER mahdi
DEPLOY_PORT 22
DEPLOY_PATH /home/mahdi/blog-published

The server login needs write access to DEPLOY_PATH, plus bash, rsync, and flock. Install its public key in that account's ~/.ssh/authorized_keys; use a separate key for this workflow. Verify the SSH host fingerprint independently, not by blindly trusting an unauthenticated scan. The server must accept SSH from GitHub-hosted runners. Deployment itself needs no sudo or nginx reload.

Separately, add the photo publisher's public SSH key at Settings → Deploy keys, with Allow write access enabled for this repository. This key stays on your server and allows photo pushes to trigger Actions; it is not a workflow secret or GITHUB_TOKEN. GitHub is the publishing source of truth. Push ordinary edits to the github remote too; pushing only to git.theread.me does not trigger this workflow.

The workflow uses read-only repository permissions and pinned action commits. It runs only for master, and serialized deployments plus a server-side run-number check prevent an older workflow from replacing a newer deployment. Ruby matches this blog's existing 3.1.2 requirement. The lockfile removes the obsolete jekyll-latex dependency that was already absent from Gemfile, allowing frozen CI installs. Runtime upgrades are separate from this change.

Server setup

The server is Ubuntu 24.04. Before this setup, nginx served /home/mahdi/theread.me/_site. /home/mahdi/update-server/update-server.js existed, was not running, and used a different port from nginx's old /_update route. The new workflow replaces that rebuild hook.

  1. Install Podman and a Compose provider (sudo apt-get install podman podman-compose).
  2. Keep this service in its own checkout, e.g. /home/mahdi/blog-photo-service.
  3. Prepare private credentials and the public output directory:
install -d -m 700 "$HOME/.config/blog-photos"
install -d -m 755 /home/mahdi/blog-published
umask 077
openssl rand -hex 32 > "$HOME/.config/blog-photos/token"
ssh-keygen -t ed25519 -N '' -f "$HOME/.config/blog-photos/git_key" -C blog-photo-publisher

Register git_key.pub as the repository's write-enabled deploy key. Put GitHub's verified SSH host key in ~/.config/blog-photos/known_hosts. Keep the private key readable only by its owner. This key is separate from Actions' server-login key.

cd /home/mahdi/blog-photo-service/photo-upload
cp .env.example .env
# Adjust absolute paths if needed.
podman compose config
systemctl --user enable --now podman.socket
podman compose up -d --build
podman compose logs -f publisher

On SELinux hosts add shared :z labels to bind mounts as appropriate.

Commit and push this change to GitHub, and wait for the first Actions deployment. Then back up nginx's site config and use host-nginx.conf.example: change the HTTPS server's root to /home/mahdi/blog-published/current and add /api/photos. Preserve the existing /, /raw/, TLS, /_xpanel, and unrelated routes. Ensure nginx can traverse parent directories and read the output.

sudo nginx -t
sudo systemctl reload nginx

The API listens on loopback port 8088; HTTPS remains on the existing nginx. Rollback by restoring the original nginx root and reloading. The old checkout and site are untouched. Once verified, retire the old rebuild route.

For reboot persistence, create ~/.config/systemd/user/blog-photos.service:

[Unit]
Description=Blog photo uploads
After=network-online.target podman.socket
Wants=network-online.target
Requires=podman.socket

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/home/mahdi/blog-photo-service/photo-upload
ExecStart=/usr/bin/podman compose up -d
ExecStop=/usr/bin/podman compose stop
TimeoutStartSec=180

[Install]
WantedBy=default.target

Then run systemctl --user daemon-reload, systemctl --user enable --now blog-photos.service, and sudo loginctl enable-linger mahdi.

iPhone shortcut

Create Publish to blog:

  1. Enable Show in Share Sheet, accepting Images.
  2. Ask for Input (Text): “Caption (optional)”.
  3. Add another Ask for Input (Text): “Highlight name (optional)”.
  4. Repeat with Each item in Shortcut Input. Inside the repeat, use Get Details of Images to get Date Taken, then Format Date using ISO 8601. After End Repeat, Combine Text from Repeat Results with New Lines and save it as Photo Dates. This preserves capture dates even when Shortcuts strips them from the exported JPEG data.
  5. Add Ask for Input, type Text, with the prompt Date Taken (optional; YYYY, YYYY-MM-DD, or ISO timestamp). Save its response as Date Taken. This is useful for analog scans; leave it blank for digital photos.
  6. Make Archive from Shortcut Input, format ZIP. Keep this output in a variable named Photo Archive. This packages every selected image into one upload; using an image list directly in a File form field can send only the first image on some Shortcuts versions.
  7. Get Contents of URL: https://theread.me/api/photos.
    • Method: POST.
    • Header: Authorization = Bearer YOUR_TOKEN.
    • Request Body: Form.
    • archive: type File, value Photo Archive. Remove the old photos field.
    • caption: type Text, value the caption response.
    • highlight: type Text, value the highlight-name response. Select each prompt's specific output variable so the caption and highlight stay separate.
    • photo_dates: type Text, value Photo Dates.
    • date_taken: type Text, value Date Taken.
    • Do not set Content-Type; Shortcuts supplies the multipart boundary.
  8. Show the actual response and check its count against the number of selected images. Only show “Uploaded; publishing” when there is no error. Optionally poll the returned status_url with the same header. pushed means “GitHub Actions is building/deploying”; only published means the deployed site's receipt contains this album.

The ZIP route accepts 1–20 images in archive-entry order and keeps them as one upload job. Each photo is stored as its own _art document and renders as its own gallery <li>. The shared caption is repeated on each photo and an optional highlight begins on the first photo. Photos remain contiguous in the selection order. Older batch documents are still split at render time for backward compatibility. Metadata sidecars under __MACOSX/ and .DS_Store are ignored. Ordinary API clients can also send repeated photos parts; send either archive or photos, never both. Select still photos; videos/Live Photo video components aren't supported. The service reads EXIF DateTimeOriginal automatically when it is present. Because iOS Shortcuts can blank that EXIF field while creating the ZIP, photo_dates supplies the same Photos-library metadata explicitly, one ISO 8601 value per image in archive order. date_taken overrides both sources for the whole batch; it accepts a year, calendar date, or ISO timestamp and is intended for analog photos. An upload without any date source is rejected instead of being placed under the upload year. Re-sharing the same images, caption, and highlight with dates replaces the earlier undated upload. Captions are plain text and optional. An optional highlight name (up to 200 characters) adds one link to Highlights, pointing to the first photo in the batch. Leave it blank for an ordinary upload.

Gallery images are progressive JPEGs with a maximum edge of 1600 pixels and a 300 KiB size limit. The uploader tries moderate JPEG quality levels, then reduces dimensions further when necessary. Private originals are retained unchanged.

The token is visible in the shortcut editor; remove it before sharing the shortcut. Rotate it by replacing the token file and recreating the upload container with podman compose up -d --force-recreate upload.

Snippets shortcut

The same service and token also publish to https://theread.me/snippets/. Text stays literal (including line breaks and Persian text); standalone HTTP/HTTPS URL lines become clickable links. Images use the same optimization and metadata removal as photo uploads. Each submission is prepended below the introduction, with a separator from the previous snippet. Existing snippets are preserved.

Create Publish snippet:

  1. Enable Show in Share Sheet, accepting Text, URLs, and Images.
  2. Repeat with Each item in Shortcut Input. Use Get Type on the repeat item. If it is Image, add the item to a variable called Snippet Images. Otherwise use Get Text from Input on the item and add its result to Snippet Text. These variables accumulate the shared items during this run.
  3. Combine Text from Snippet Text, separated by New Lines. For an image-only share, leave the text empty. Don't fetch the contents of shared URLs: send the link itself.
  4. Ask for Input (Text): “Note (optional)”. Keep its output separate from the shared text. You can also run the shortcut without shared input and use a text prompt as Snippet Text to write a new snippet.
  5. Get Contents of URL: https://theread.me/api/snippets, method POST, request body Form:
    • Header Authorization: Bearer YOUR_TOKEN (same token as photos).
    • text: Text, the combined shared text.
    • caption: Text, the optional note.
    • For a single image, photo: File, the shared image. No archive is needed. Use only one of photo, photos, or archive per request.
    • For images, Make Archive (ZIP) from Snippet Images first, and send its output as archive, type File. For text-only requests, omit this field. Use an If on whether there are images to choose between the form with archive and the text-only form.
    • Delete any blank header rows. Do not manually set Content-Type.
  6. Show Result using the actual response. A response with error means the upload failed; don't show a fixed success message. queued means accepted; publishing happens afterwards. The returned status_url can be polled with the same authorization header; published confirms deployment.

For the simplest initial shortcut, accept just Text and URLs and omit the image branch and archive field; add the image branch when that works on your phone.

POST /api/snippets accepts up to 20,000 characters of text, an optional 4,000-character caption, and up to 20 images in a ZIP archive (or repeated photos parts), within the existing 100 MiB request limit. At least text or an image is required. The singular photo field accepts one image and produces the same submission as sending that image in photos. It returns id, count (image count), url, status_url, and status. GET /api/snippets/<id> checks publication, and POST /api/snippets/<id>/retry retries failed Git publication. Identical text, note, and original images in the same order are deduplicated.

Keep <!-- uploaded-snippets --> in snippets.md: it marks the insertion point. The publisher fails safely if that marker is removed. GitHub Actions includes rendered snippet IDs in photo-publication.json, alongside photo IDs.

Markdown blog-post shortcut

Create Publish blog post and enable Show in Share Sheet, accepting Files. Share one UTF-8 .md or .markdown file, up to 1 MiB. In Get Contents of URL:

  • URL: https://theread.me/api/posts; method POST; request body Form.
  • Header: Authorization = Bearer YOUR_TOKEN (no empty header rows).
  • Form field file: type File, value Shortcut Input.
  • Let Shortcuts set Content-Type automatically.

Show the actual response, and poll https://theread.me plus its status_url with the same header. Matching happens in the publisher: 202 only means the file was accepted, not that a new post or update was published. The status response supplies the final url, post_id, and action (created or updated). failed includes an error; published confirms the deployed revision. An older upload can become superseded when a newer revision replaces it.

No front matter is required. The first # Heading becomes the title, or the filename is used if there is no heading. Optional YAML front matter supports title, subtitle, lang, description, categories, tags, toc, math, date, and permalink (a path such as /my-essay/). Markdown formatting is preserved; Liquid template evaluation is disabled for imported post bodies. The original file is retained privately in the service volume.

New posts derive post_id from the title: My New Post! becomes my-new-post. Spaces, underscores, and punctuation become single dashes; Unicode letters are preserved. The default URL is /my-new-post/. Explicit post_id values still override this default. Updates keep the existing ID and URL even if the title changes. If unrelated content has the same title-derived ID, publication fails instead of overwriting it; choose another title or an explicit ID.

Automatic updates by content

The publisher compares the body against existing Markdown posts, ignoring front matter and whitespace changes. It updates a unique match with at least 92% word sequence similarity, at least 200 characters in both texts, and a lead of at least 8 percentage points over the next plausible match. An exact body match of at least 40 characters also qualifies. These scores are a conservative heuristic, not a probability. Renaming the file does not prevent matching; reusing a filename for unrelated writing does not overwrite its previous post.

If several posts match closely, the upload fails without changing Git. If no post meets the threshold, it creates a new post. Large rewrites and very short posts may therefore need an explicit ID. For a known imported post, add the returned post_id to its front matter to target it directly:

---
title: My revised essay
post_id: THE_ID_FROM_THE_UPLOAD_STATUS
---

For an older Git-authored post, first add a unique post_id to that existing post's front matter and use the same ID in the uploaded file. Updates preserve the original publication date and public URL, and remain reversible through Git. Identical file retries are deduplicated and do not roll back a newer revision.

The upload takes the Markdown file only. Relative links to images or attachments in an export folder will not upload those files; use existing public image URLs. Export Markdown from the source app before sharing; proprietary note files, PDFs, ZIP bundles, and rich-text files are not Markdown uploads.

API, failures, and backups

All API routes require the bearer token:

  • POST /api/photos: 1–20 multipart photos, optional caption (4000 characters). Optional highlight (200 characters) names a Highlights link to the batch's first photo. Maximum request 100 MiB; each photo at most 60 megapixels.
  • GET /api/photos/<id>: queued, publishing, pushed, published, or failed.
  • POST /api/photos/<id>/retry: retry a failed Git publication.

Identical original bytes, ordering, caption, and highlight name produce the same ID. Retrying does not duplicate entries or commits. Re-encoding or changing the caption creates a new submission; edit an existing entry through Git. Keep the upload response/status URL.

For Git failures, check podman compose logs publisher, fix access/conflicts, and POST to the status URL plus /retry. No force push is used. If Actions fails, status stays pushed: inspect and rerun the failed workflow on GitHub, rather than uploading again. Deployment only switches nginx after a complete transfer; the old site stays available during builds and failures. The receipt is built from rendered photo anchors so the service does not mistake a successful push for publication.

Back up the private photo-data volume (originals and job state) and GitHub's repository. photo-work is disposable. Never expose private data through nginx or run compose down -v. Originals are retained, so monitor disk usage. After a successful deployment switches current, all older releases are automatically pruned; newer uploads still in progress are preserved. There are no retained server-side rollback copies; rebuild an older Git revision if needed. Interrupted uploads may leave files in photo-data/temporary; clean that folder only while the API is stopped. Public Git images are optimized copies.

Tests

python3 -m venv /tmp/blog-photo-tests
/tmp/blog-photo-tests/bin/pip install -r requirements.txt pytest
/tmp/blog-photo-tests/bin/pytest -q tests

Tests use a temporary local bare Git repository; no test pushes go to GitHub or the live server. The iPhone shortcut and production deployment need an end-to-end check after credentials are configured.