From 07439d09b17541c95dbe684b168bcc7d184e0f01 Mon Sep 17 00:00:00 2001 From: LeRatierBretonnier Date: Wed, 24 Jun 2026 10:40:23 +0200 Subject: [PATCH] docs: update README with auth, locale, externals, Gitea hook, caveats from deploy experience --- README.md | 97 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 94 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index b855c8b..db879c3 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ Py sidecar SVN↔Git bidirectional mirror for Gitea. Uses own mapping DB (not `g - `git` + `git-svn` (for one-time `git svn clone --no-metadata` during init) - `svn` CLI (Subversion client) - Gitea instance (bare repos accessible on disk, not just via HTTP API) +- UTF-8 system locale (SVN needs it for non-ASCII filenames) ### Install system deps @@ -38,8 +39,22 @@ apt install python3 python3-pip git git-svn subversion # RHEL / Rocky / Alma yum install python3 python3-pip git git-svn subversion + +# OpenSUSE +zypper install python311 python311-pip git git-svn subversion glibc-locale ``` +### UTF-8 locale + +SVN requires a UTF-8 locale to handle filenames with accents/spaces: + +```bash +sudo localedef -i en_US -f UTF-8 en_US.UTF-8 +export LC_ALL=en_US.UTF-8 +``` + +Add to `~/.bashrc` or `~/.profile` to make it permanent. + ## Installation ```bash @@ -72,6 +87,8 @@ mirrors: # trunk: trunk # branches: branches # tags: tags + username: # SVN login (omit if anonymous access) + password: # SVN password gitea: owner: myorg @@ -97,9 +114,13 @@ The sidecar accesses Gitea's repos **on disk**, not via HTTP API. Find the path | Gitea binary | `/var/lib/gitea/data/repositories` | | Gitea Docker | `/data/git/repositories` | | `git` user home | `/home/git/repositories` | +| Gitea LXC | `/home/gitea/repo/` | If unset the tool tries common defaults. Set it explicitly to be safe. +**Note:** `repo` should be the repo name **without** `.git` suffix — the tool appends it. +For example: `repo: myproject`, not `repo: myproject.git`. + ## Quick start ### 1. Validate config @@ -129,6 +150,17 @@ python -m svn_mirror init --mirror my-project --config /etc/svn-mirror/config.ym Runs `git svn clone --no-metadata` (one-time bootstrap), then tears down `git-svn` metadata. All ongoing sync is handled by the tool's own engine. +**Auth:** If the SVN server requires login, set `username` and `password` +in the config. The password is piped to `git svn`'s stdin prompt +(`git svn` does not accept `--password` on the command line). + +The tool also caches credentials via `svn info --username X --password Y` +before `git svn clone` so the SVN auth cache is populated. + +**External definitions:** If the SVN repo uses `svn:externals` pointing to +different repositories, the tool passes `--ignore-externals` to all +`svn checkout` / `svn update` calls. The externals are not followed. + ### 4. Sync SVN → Git ```bash @@ -155,6 +187,13 @@ python -m svn_mirror status --mirror my-project --config /etc/svn-mirror/config. python -m svn_mirror daemon --config /etc/svn-mirror/config.yml ``` +**Note:** `daemon` does **not** accept `--mirror` — it processes all enabled +mirrors. Use `screen` (or `tmux`) to run in background: + +```bash +screen -dmS svn-mirror bash -c 'python3.11 -m svn_mirror daemon --config /etc/svn-mirror/config.yml 2>&1 | tee /tmp/daemon.log' +``` + Polls SVN every `sync_interval` seconds. Pushes new Git commits to SVN on the same interval. @@ -202,8 +241,36 @@ Run daemon AND webhook. The daemon handles SVN→Git polling; the webhook handles instant Git→SVN on developer push. ```bash -python -m svn_mirror daemon --config /etc/svn-mirror/config.yml & -python -m svn_mirror webhook --config /etc/svn-mirror/config.yml & +screen -dmS daemon python3.11 -m svn_mirror daemon --config /etc/svn-mirror/config.yml +screen -dmS webhook python3.11 -m svn_mirror webhook --config /etc/svn-mirror/config.yml +``` + +### Mode D: Gitea post-receive hook (filesystem) + +For setups without network access to a webhook receiver, install a +post-receive hook in Gitea's repo that calls sync directly: + +```bash +# Create hook (one-time setup) +tee /home/gitea/repo/inpixal/pixkit.git/hooks/post-receive.d/sync-to-svn << 'HOOK' +#!/bin/bash +cd /home/git/apps/svn-git-server +. .venv/bin/activate +python3.11 -m svn_mirror sync --mirror pixkit --config /etc/svn-mirror/config.yml +HOOK +chmod +x /home/gitea/repo/inpixal/pixkit.git/hooks/post-receive.d/sync-to-svn +``` + +Each `git push` to Gitea then triggers an immediate SVN sync. + +**Note:** Direct filesystem `git push` to Gitea's bare repo is blocked by +Gitea's `pre-receive` hook. To force-push (e.g. initial import to Gitea), +temporarily disable the hook: + +```bash +mv /path/to/repo.git/hooks/pre-receive{,.disabled} +# push... +mv /path/to/repo.git/hooks/pre-receive{.disabled,} ``` ## Reconciliation (fixing divergence) @@ -229,6 +296,19 @@ The reconciliation: - Records the mapping so `sync_git_to_svn` doesn't re-push those commits - Leaves `last_svn_revision` unchanged — `sync` processes pending SVN revisions naturally +**SVN author fix:** After `svn commit` / `svn copy`, the tool sets the correct +author via `svn propset --revprop -r REV svn:author "Real Author"`. +This requires the `pre-revprop-change` hook on the SVN server: + +```bash +# On the SVN server — create this script, make it executable +# /srv/svn/pixkit/hooks/pre-revprop-change +#!/bin/bash +REPOS="$1"; REV="$2"; USER="$3"; PROPNAME="$4"; ACTION="$5" +[ "$PROPNAME" = "svn:author" ] || [ "$PROPNAME" = "svn:log" ] || exit 1 +exit 0 +``` + **Dry-run mode:** ```bash @@ -266,8 +346,19 @@ Shows the assessment report and what would be done, without making any changes. ## Caveats +- **Auth**: SVN `username`/`password` in config is sent to `git svn clone` + pipe and CLI args. Password is stored in plaintext in config file. - **Binary files**: supported (fix in `sync.py` uses raw bytes for `hash-object`) - **Empty directories**: SVN tracks them, Git does not — skipped during sync -- **Large repos**: initial `git svn clone` time depends on SVN history size +- **Large repos**: initial `git svn clone` time depends on SVN history size; + 6+ GiB repos can take several hours over HTTPS - **One trunk only**: reconciliation only handles `refs/heads/master` ↔ SVN trunk. Other branches are not yet supported by the reconcile tool. +- **`git svn` username**: `--username` is supported but `--password` is not + (the tool pipes password to stdin instead). +- **All SVN branches become Git branches**: If the SVN repo uses branches for + release tags, the Git canonical repo will contain hundreds of branches. + Consider filtering via `refs/tags/` in the Gitea push refspec. +- **Gitea filesystem push**: Direct `git push` to Gitea's on-disk bare repo + is blocked by Gitea's `pre-receive` hook. The daemon/sync uses its own + `push_to_gitea()` to work around this.