second commit

README.md

@@ -8,7 +8,8 @@
 > The button flow has not been tested against a live repository yet. Test it once after publishing.
 
 A long-running worker that reads video posts from a Telegram channel, downloads each video, and
-uploads it to Google Drive. The caption's **Title** becomes the file name and its **Topic** picks the
+uploads it to Google Drive. You can process the whole channel (`/sync` + `/run`) or just a range of
+posts with the interactive **`/upload`** command (send a start link and an end link, then `/confirm`). The caption's **Title** becomes the file name and its **Topic** picks the
 Drive folder. Videos are processed one at a time. All progress is stored in a database, so the bot
 resumes after crashes, restarts and redeploys without uploading anything twice.
 
@@ -51,6 +52,11 @@ the temp file → next video.**
 - Bounded retries with exponential backoff, honouring Telegram flood waits and Drive rate limits. Permission errors are not retried.
 - Disk-space checks before each download. A full disk, Drive quota or expired auth halts the queue cleanly instead of burning retries.
 - Admin-only Telegram commands, plus one progress message that is edited in place rather than spammed.
+- **Interactive `/upload` tasks:** pick an inclusive message range by pasting two post links,
+  confirm, and watch a live task progress message. Tasks are stored in the database and survive restarts.
+- **Automatic temp-file cleanup:** each video's local file is deleted right after its upload is
+  verified in Drive *and* recorded in the database. Abandoned temp files are swept at startup,
+  periodically, and whenever disk space runs low.
 - Graceful shutdown on `SIGTERM` (Heroku restarts) at a resumable checkpoint.
 - A single-worker guard (PostgreSQL advisory lock) prevents two dynos from processing the same queue.
 
@@ -216,6 +222,7 @@ The Heroku button only works with **public** repositories.
 |---|---|---|
 | New posts (live) | ✅ | ✅ (bot must be a channel admin) |
 | Existing history | ✅ full history the account can see | ⚠️ best effort: bots can't list history, so message IDs are probed in batches until `BOT_SCAN_MAX_EMPTY` consecutive gaps (or up to `BOT_SCAN_END_ID`) |
+| `/upload` message ranges | ✅ | ✅ messages are fetched by ID, which bots that are channel admins may do |
 | Large files | ✅ up to Telegram's file limit | ✅ uses MTProto, so the HTTP Bot API's 20 MB download limit does **not** apply |
 | Private channel by numeric ID | ✅ | ⚠️ the bot learns the channel's access key from its first live post; the key is then stored in the database |
 | Extra secret needed | `TELEGRAM_SESSION_STRING` | none |
@@ -302,6 +309,11 @@ Set these in the dashboard (*Settings → Reveal Config Vars*) or with
 | `DOWNLOAD_DIR` | no (`/tmp/telegram_drive_bot` in app.json) | temporary files |
 | `UPLOAD_CHUNK_SIZE` | no (`8MB` in app.json) | multiple of 256KB; memory per chunk |
 | `AUTO_START` | no (`false`) | start processing on every boot |
+| `UPLOAD_RESTRICT_TO_SOURCE_CHANNEL` | no (`true`) | `/upload` links must point to `SOURCE_CHANNEL_ID` |
+| `UPLOAD_SETUP_TIMEOUT` | no (`30`) | minutes before an unfinished `/upload` setup expires |
+| `UPLOAD_MAX_RANGE` | no (`5000`) | largest number of message IDs per task |
+| `KEEP_FAILED_DOWNLOADS` | no (`false`) | keep local files of FAILED videos |
+| `TEMP_CLEANUP_INTERVAL` | no (`3600`) | seconds between abandoned-temp-file sweeps |
 
 Step-by-step instructions for obtaining every value are in **[ENV_SETUP.md](ENV_SETUP.md)**. Every other supported variable is listed with an explanation in [`.env.example`](.env.example).
 
@@ -332,8 +344,9 @@ Production state lives in **PostgreSQL**:
 1. Deploy (button or CLI). Watch the logs: `heroku logs --tail --app YOUR_APP_NAME`.
    You should see the database backend, `Google Drive credentials validated`, and `Source channel resolved`.
 2. The bot messages each admin: *Bot online (PostgreSQL); send /sync then /run to start.*
-3. Send `/sync` to discover channel videos, then `/run`. Watch `/progress`.
-4. After `/run`, the choice is stored. After any dyno restart the bot syncs and resumes on its own.
+3. Either send `/upload` to process a message range (see [§13](#13-telegram-commands)), or send
+   `/sync` to discover all channel videos and then `/run`. Watch `/taskstatus` or `/progress`.
+4. Running tasks and the `/run` choice are stored. After any dyno restart the bot resumes on its own.
    `/pause` and `/stop` are also remembered.
 
 ## 13. Telegram commands
@@ -354,8 +367,82 @@ Send these to the bot in a **private chat**. Users not in `ADMIN_USER_IDS` are r
 | `/failed` | Failed items with short error summaries |
 | `/folders` | Folder count and uploads per topic |
 | `/check ID` | Full record for one Telegram message ID |
+| `/upload` | Start an interactive upload task for a message range |
+| `/confirm` | Start the prepared task (only after both links were validated) |
+| `/cancel` | Discard your setup, or cancel your active task at a safe checkpoint. `/cancel ID` targets a specific task. |
+| `/taskstatus` | Your setup step, plus active tasks with range, counters and live progress. `/taskstatus ID` for one task. |
+| `/uploadhistory` | The 10 most recent tasks and their results |
 
-Progress is a single message, edited every `PROGRESS_INTERVAL` seconds:
+`/pause`, `/resume` and `/stop` also apply to upload tasks. `/stop` parks a running task as PAUSED;
+`/resume` continues it.
+
+### Processing a message range with `/upload`
+
+1. Send `/upload`. The bot asks for the **start link**: in Telegram, right-click (or long-press) a
+   post → **Copy Post Link**, e.g. `https://t.me/c/3909337424/14`.
+2. Send the link. The bot asks for the **end link**, e.g. `https://t.me/c/3909337424/20`.
+3. The bot shows *Upload Task Ready* with both message IDs and the channel.
+4. Send `/confirm` to start, or `/cancel` to discard. Nothing is downloaded before `/confirm`.
+
+Link rules:
+- Accepted formats: `t.me/c/<id>/<msg>` (private), `t.me/<username>/<msg>` (public), and
+  `t.me/s/<username>/<msg>`. `www.`, `telegram.me`, a trailing slash and `?single` are fine.
+- Invite links and links without a message ID are rejected with an explanation.
+- The `/c/` number is Telegram's internal channel ID. It is converted to the API ID
+  `-(10¹² + id)` and then **resolved through the bot's Telegram account**. A channel that account
+  can't access is refused; nothing bypasses membership or permissions.
+- Both links must be from the same channel, the end ID must be ≥ the start ID, the range is
+  limited by `UPLOAD_MAX_RANGE`, and both messages must exist.
+- By default the links must point to `SOURCE_CHANNEL_ID`. Set
+  `UPLOAD_RESTRICT_TO_SOURCE_CHANNEL=false` to allow any channel the account (or admin bot) can read.
+- If the end link is invalid, the start link is kept; just send another end link.
+
+How a task runs:
+- **The range is inclusive** (14–20 means 14, 15, …, 20), processed in ascending order, one video at a time.
+- **Text, photos, stickers and deleted messages** are counted as skipped, never as failures.
+- **Videos already uploaded earlier** are counted as duplicates and are not uploaded again.
+  Repeating a finished task therefore uploads nothing new.
+- **Caption metadata** (Title, Topic, …) is used exactly as for whole-channel processing.
+- **Each local file is deleted** only after the Drive upload is verified *and* its COMPLETED state is committed.
+- **Failed videos** are recorded, never marked completed, and can be re-queued with `/retry`.
+  Their local file is deleted (unless `KEEP_FAILED_DOWNLOADS=true`) because it can be downloaded again.
+- **`/cancel` during processing** stops at the next safe checkpoint:
+  - finished uploads stay in Drive;
+  - the remaining videos are marked *cancelled* and their temp files are removed;
+  - a later task covering the same range picks the cancelled videos up again.
+- **Setups are per administrator.** Two admins can prepare tasks at the same time without
+  affecting each other. A second `/upload` never overwrites your unfinished setup. A setup expires
+  after `UPLOAD_SETUP_TIMEOUT` minutes.
+- **Confirmed tasks run one after another** in the order they were confirmed. Running and queued
+  tasks resume automatically after a restart or redeploy.
+
+Task progress is one message, edited in place:
+
+```
+🚀 Upload Task Running
+
+Task: #12
+Range: 14–20
+Current Message: 16
+Current Video: Class 01 Aman sir.mp4
+Topic: Misc• Grammar Practice
+Status: Uploading to Google Drive
+Upload: 45% (540.0 MB / 1.2 GB) · 18.3 MB/s · ETA 36s
+
+Completed: 1
+Skipped: 1
+Failed: 0
+```
+
+When the task finishes, the bot sends a summary covering:
+- the range;
+- messages examined;
+- eligible videos;
+- uploads, duplicates, non-video messages and failures;
+- temp files deleted;
+- the Drive folders used.
+
+For whole-channel processing, progress is a single message, edited every `PROGRESS_INTERVAL` seconds:
 
 ```
 Processing video 12 of 150
@@ -437,6 +524,10 @@ pipeline is intentionally sequential.
   guarantee a specific capacity, and the disk is wiped on every restart (at least daily).
   - The bot checks free space before each download and halts with a message if a video won't fit.
   - Multi-GB videos may not fit on a dyno; run the bot locally for those.
+- **Temporary files:** downloads live only in `DOWNLOAD_DIR`. The bot refuses to use `/`, `/tmp`,
+  your home directory or the project directory itself. Cleanup only removes files named
+  `<channel>_<message>.part|.video` directly inside that directory, never follows symlinks, and
+  re-checks the database first. Files of unfinished items are kept for resuming.
 - **Memory:** one upload chunk (`UPLOAD_CHUNK_SIZE`) is held in RAM at a time. `8MB` is comfortable on 512 MB dynos.
 - **Daily restarts:** in-flight transfers resume from their checkpoint. A partially downloaded file
   on the old dyno is lost and is downloaded again, but the Drive upload session is reused.
@@ -449,7 +540,7 @@ pipeline is intentionally sequential.
 
 ### What has and has not been verified
 
-- **Automated tests (171):** run against SQLite **and** a real local PostgreSQL 16, with mocked Telegram and Google Drive.
+- **Automated tests (259):** run against SQLite **and** a real local PostgreSQL 16, with mocked Telegram and Google Drive.
 - **Not tested:** real Telegram logins and downloads, real Google Drive uploads, and an actual Heroku deployment. Follow the manual end-to-end procedure below before relying on it.
 
 ## 19. Local development
@@ -488,6 +579,15 @@ both backends. Without it, only SQLite is tested. The suite covers:
 - a successful upload followed by a database-write failure;
 - sequential processing, pause/resume and graceful shutdown;
 - admin-only commands;
+- the `/upload` workflow:
+  - link parsing and validation;
+  - per-admin setups, persistence and expiry;
+  - inclusive ascending ranges, and skipping non-video messages;
+  - delete-only-after-verified-upload, and failed uploads never completed;
+  - repeat tasks without duplicates;
+  - cancellation;
+  - crash/restart recovery and pause/resume;
+- temp-file cleanup safety;
 - configuration and Heroku validation, env-var credentials, log redaction;
 - `app.json` / Procfile consistency.
 
@@ -508,3 +608,11 @@ both backends. Without it, only SQLite is tested. The suite covers:
    Restart it. Confirm the upload continues and Drive ends up with exactly one copy.
 5. `/stop`, post a new video, `/run`: only the new video is processed.
 6. Revoke Drive access temporarily and confirm the queue halts with a clear message. Restore access, then `/resume`.
+7. `/upload`: send the link of the first test post, then the last one, then `/confirm`. Check that:
+   - only that range is processed;
+   - a text post inside the range is reported as a skipped non-video;
+   - the summary counts are correct.
+   Run the same range again and confirm it reports duplicates and uploads nothing.
+8. Start another `/upload` task and send `/cancel` while a video is downloading. Check that:
+   - finished uploads remain in Drive;
+   - `downloads/` is empty.