drive/ENV_SETUP.md

How to Obtain Every Environment Variable

This guide explains where each configuration value comes from and how to get it. Work through the sections in order: some steps (the Telegram session string and the Google token) need values from earlier steps.

• Local runs: put the values in a .env file (cp .env.example .env).
• Heroku: set them as Config Vars, either in the dashboard (App → Settings → Reveal Config Vars) or with heroku config:set NAME=value --app YOUR_APP_NAME.

All values below are placeholders. Never commit real values or paste them into chats, issues or screenshots. .env, credentials/ and data/ are already in .gitignore.

---

Quick checklist

#	Variable	Required?	Where it comes from
1	TELEGRAM_API_ID	Always	my.telegram.org
2	TELEGRAM_API_HASH	Always	my.telegram.org
3	TELEGRAM_BOT_TOKEN	Always on Heroku (control commands); required in bot mode	@BotFather
4	ADMIN_USER_IDS	When a bot token is set	@userinfobot
5	SOURCE_CHANNEL_ID	Always	Channel username, link, or numeric ID
6	TELEGRAM_MODE	Optional (default user)	Your choice: user or bot
7	TELEGRAM_SESSION_STRING	TELEGRAM_MODE=user on Heroku	python main.py login --session-string
8	GOOGLE_AUTH_MODE	Optional (default oauth)	Your choice: oauth or service_account
9	GOOGLE_OAUTH_TOKEN_JSON	oauth on Heroku	python main.py drive-auth
10	GOOGLE_SERVICE_ACCOUNT_JSON	service_account on Heroku	Google Cloud Console key
11	GOOGLE_DRIVE_ROOT_FOLDER_ID	Recommended	Drive folder URL, or python main.py drive-init
12	GOOGLE_DRIVE_SCOPE	Optional (default drive.file)	Your choice
13	DRIVE_SHARED_DRIVE_ID	Shared Drive setups only	Shared Drive URL
14	DATABASE_URL	Required on Heroku	Heroku Postgres (set automatically)

Everything else has a sensible default (see Optional settings).

---

Part 1 — Telegram

1–2. TELEGRAM_API_ID and TELEGRAM_API_HASH

These identify your Telegram API application. They are needed in both modes, because the bot talks to Telegram through Telethon (MTProto).

1. Open https://my.telegram.org and log in with your phone number. Telegram sends a code to your Telegram app, not by SMS.
2. Click API development tools.
3. If no app exists yet, fill in the form:
   • App title: anything, for example Drive Organizer;
   • Short name: 5–32 letters or digits, for example driveorganizer;
   • Platform: Desktop;
   • URL and description can stay empty.
4. Click Create application.
5. Copy the two values:
   • App api_id (a number) → TELEGRAM_API_ID
   • App api_hash (32 hex characters) → TELEGRAM_API_HASH

TELEGRAM_API_ID=1234567
TELEGRAM_API_HASH=0123456789abcdef0123456789abcdef

The API hash is a secret. If my.telegram.org shows an "ERROR" when you create the app, try another browser, disable ad blockers or VPNs, and use a short alphanumeric short name.

3. TELEGRAM_BOT_TOKEN

The bot provides the admin commands (/run, /status, …) and progress messages. In bot mode it also reads the channel.

1. In Telegram, open @BotFather and send /newbot.
2. Enter a display name, then a username ending in bot (for example my_drive_organizer_bot).
3. BotFather replies with a token like 123456789:AAH.... Copy it → TELEGRAM_BOT_TOKEN.
4. Open your new bot and press Start. Bots cannot message you until you do, so without this you get no progress messages.

TELEGRAM_BOT_TOKEN=123456789:replace-with-your-bot-token

If the token leaks, send /revoke to @BotFather to get a new one.

4. ADMIN_USER_IDS (and optional NOTIFY_CHAT_IDS)

Only these numeric Telegram user IDs may control the bot.

1. Open @userinfobot and press Start.
2. It replies with your Id (a number such as 111111111). Copy it.
3. For several admins, ask each person to do the same and separate the IDs with commas.

ADMIN_USER_IDS=111111111,222222222

NOTIFY_CHAT_IDS is optional. It lists the chats that receive progress and error messages, and defaults to ADMIN_USER_IDS. Each person listed must have pressed Start in the bot.

5. SOURCE_CHANNEL_ID

Any of these formats is accepted:

Channel type	How to get it	Example value
Public channel	The @username shown in the channel info	@my_channel or https://t.me/my_channel
Private channel: post link	Right-click (or long-press) any post → Copy Post Link	https://t.me/c/1234567890/55
Private channel: numeric ID	Prefix the number from the post link with -100	-1001234567890
Private channel: Telegram Web	Open the channel at https://web.telegram.org/k/; the URL ends with #-100…	-1001234567890

Another way: forward a channel post to @JsonDumpBot and read forward_from_chat.id.

SOURCE_CHANNEL_ID=-1001234567890

The same post links are what you send to the bot's /upload command to pick a message range.

Bot mode with a private channel: the bot must be a channel administrator. If the numeric ID can't be resolved at the first start, post one message in the channel. The bot then learns the channel and remembers it in the database.

6. TELEGRAM_MODE

This is a choice, not a credential.

Value	Use when	Needs
user (default)	You need the channel's existing history, or you are a member but not an admin	TELEGRAM_SESSION_STRING (Heroku) or a local login
bot	You only need new posts, and you can make the bot a channel admin	Bot added as channel administrator

To add the bot as admin: open the channel → Manage channel → Administrators → Add Admin → search for your bot's username → save. No extra rights are needed beyond the default.

7. TELEGRAM_SESSION_STRING (user mode only)

This lets the bot act as your Telegram account without a session file. Heroku's disk is wiped on every restart, so a file wouldn't survive. You create it once, on your own computer.

1. Do the local setup: create the virtualenv and install requirements.
2. Create .env containing at least:

   TELEGRAM_API_ID=1234567
   TELEGRAM_API_HASH=0123456789abcdef0123456789abcdef
   

3. Run:

   python main.py login --session-string
   

4. Enter your phone number in international format (+15551234567), then the login code Telegram sends to your app, then your 2FA password if you have one. None of these are stored.
5. The string is written to data/sessions/session_string.txt with owner-only permissions.
6. Put it on Heroku without printing it:

   heroku config:set TELEGRAM_SESSION_STRING="$(cat data/sessions/session_string.txt)" --app YOUR_APP_NAME
   

   For a local .env, paste the file's contents after TELEGRAM_SESSION_STRING=.
7. Delete the file: rm data/sessions/session_string.txt.

The session string grants full access to your Telegram account. Treat it like a password. To revoke it: Telegram → Settings → Devices → terminate the session. Then generate a new string.

For local-only use you can skip the string and run python main.py login, which creates data/sessions/user.session instead.

---

Part 2 — Google Drive

Pick one authentication method:

Your Drive	Use	Variable to obtain
Personal Google account (@gmail.com) or your own Workspace Drive	OAuth (GOOGLE_AUTH_MODE=oauth)	GOOGLE_OAUTH_TOKEN_JSON
Google Workspace Shared Drive	Service account (GOOGLE_AUTH_MODE=service_account)	GOOGLE_SERVICE_ACCOUNT_JSON

A service account cannot upload into a personal My Drive: it has no storage quota there. Use OAuth for personal Drives.

Common first step: Google Cloud project and Drive API

1. Open https://console.cloud.google.com/ and sign in with the Google account that owns the destination Drive.
2. Use the project selector at the top → New Project → give it a name → Create, then select it.
3. Go to APIs & Services → Library, search for Google Drive API, and click Enable.

9. GOOGLE_OAUTH_TOKEN_JSON (OAuth)

a) Configure the consent screen

1. Go to APIs & Services → OAuth consent screen. In the newer console this is Google Auth Platform → Branding / Audience.
2. User type: External (or Internal if you use Workspace and only your organisation needs access).
3. Fill in the app name, your support email and developer contact email, then save.
4. Under Audience (or Test users), add your own Google account.
5. Publish the app (Publishing status → In production).

   If the app stays in Testing, Google expires its refresh tokens after 7 days and the bot will lose Drive access every week. With the default drive.file scope no Google verification is needed. With the full drive scope you'll see an "unverified app" warning during your own login: click Advanced → Go to (app name).

b) Create the OAuth client

1. Go to APIs & Services → Credentials → Create Credentials → OAuth client ID.
2. Application type: Desktop app. Give it any name → Create.
3. Click Download JSON and save it in the project as credentials/client_secret.json. Create the credentials/ folder if needed; it is gitignored.

c) Choose the scope before authorizing

GOOGLE_DRIVE_SCOPE	Meaning	Choose it if
drive.file (default, least privilege)	The bot only sees files and folders it created	You let the bot create its own root folder (step 11, option B)
drive	Full access to your Drive	You want uploads to go into a folder you created

Set it in .env (for example GOOGLE_DRIVE_SCOPE=drive.file). Use the same value on Heroku.

d) Authorize and get the token

python main.py drive-auth            # opens a browser
python main.py drive-auth --no-browser   # prints a URL to open yourself

Sign in, approve access, and the token is saved to credentials/token.json. It contains the refresh token, client ID and client secret. Then set it on Heroku:

heroku config:set GOOGLE_OAUTH_TOKEN_JSON="$(cat credentials/token.json)" --app YOUR_APP_NAME

If you prefer pasting into the Heroku dashboard, a single-line base64 version also works:

base64 < credentials/token.json | tr -d '\n'

GOOGLE_AUTH_MODE=oauth
GOOGLE_OAUTH_TOKEN_JSON={"token": "...", "refresh_token": "...", "client_id": "...", "client_secret": "...", ...}

If you change GOOGLE_DRIVE_SCOPE later, delete credentials/token.json, run drive-auth again, and update the Config Var. To revoke access: https://myaccount.google.com/permissions.

10. GOOGLE_SERVICE_ACCOUNT_JSON (Shared Drive)

1. In the Cloud Console: IAM & Admin → Service Accounts → Create service account. Give it a name; no project roles are needed. Click Done.
2. Open the service account → Keys → Add key → Create new key → JSON. The file downloads once. Store it safely.
3. Copy the client_email value from the JSON (for example uploader@your-project.iam.gserviceaccount.com).
4. In Google Drive, open the Shared Drive → Manage members → add that email as Content manager.
5. Set the variables:

   heroku config:set GOOGLE_AUTH_MODE=service_account \
     GOOGLE_DRIVE_SCOPE=drive \
     GOOGLE_SERVICE_ACCOUNT_JSON="$(cat path/to/key.json)" --app YOUR_APP_NAME
   

   Use GOOGLE_DRIVE_SCOPE=drive, because the service account must reach a Shared Drive folder it did not create. Locally you can instead set GOOGLE_SERVICE_ACCOUNT_FILE=path/to/key.json.

Your organisation may block service-account key creation (policy iam.disableServiceAccountKeyCreation). In that case use OAuth.

11. GOOGLE_DRIVE_ROOT_FOLDER_ID

This is the folder that will contain all topic folders. If you leave it empty, topic folders are created at the top level of My Drive.

Option A: a folder you created (needs GOOGLE_DRIVE_SCOPE=drive, or a Shared Drive folder)

1. Open the folder in Google Drive in a browser.
2. The URL looks like https://drive.google.com/drive/folders/1AbCdEfGhIjKlMnOpQrStUvWxYz.
3. The part after /folders/ is the ID → GOOGLE_DRIVE_ROOT_FOLDER_ID.

Option B: let the bot create it (works with the default drive.file scope)

1. Run this after drive-auth:

   python main.py drive-init --name "Telegram Videos"
   

2. It prints Set GOOGLE_DRIVE_ROOT_FOLDER_ID=.... Copy that ID.

GOOGLE_DRIVE_ROOT_FOLDER_ID=1AbCdEfGhIjKlMnOpQrStUvWxYz

At startup the bot checks that it can see this folder. If it can't, it stops and explains how to fix the problem.

13. DRIVE_SHARED_DRIVE_ID (Shared Drive only)

Open the Shared Drive itself (its top level). The URL is https://drive.google.com/drive/folders/<ID>, and that <ID> is the Shared Drive ID.

DRIVE_SHARED_DRIVE_ID=0AbCdEfGhIjKlUk9PVA

DRIVE_SHARE_MODE

Leave it as none so uploads stay private. Set anyone_with_link only if every uploaded video should be viewable by anyone who has its link.

---

Part 3 — Database

14. DATABASE_URL

• Heroku Deploy button: the Heroku Postgres add-on is provisioned and DATABASE_URL is set automatically. Nothing to do.
• Manual Heroku setup:

  heroku addons:create heroku-postgresql:essential-0 --app YOUR_APP_NAME
  heroku config:get DATABASE_URL --app YOUR_APP_NAME   # confirm it is set
  

  This is a paid plan; check Heroku's current pricing. Don't copy the URL anywhere else, because Heroku may rotate it.
• Other PostgreSQL providers: use their connection string (postgresql://USER:PASSWORD@HOST:5432/DBNAME). TLS is added automatically for remote hosts.
• Local development: leave it empty to use SQLite at data/bot.db.

On Heroku the bot refuses to start without DATABASE_URL, unless you set ALLOW_EPHEMERAL_DATABASE=true. With that set, all progress is lost on every restart.

---

Optional settings

These are preferences, not credentials. Defaults are shown; full descriptions are in .env.example.

Variable	Default	Purpose
PROCESS_MODE	both	existing (history), new (new posts), or both
START_MESSAGE_ID	1	First message ID to scan
START_DATE	empty	Only messages on or after this ISO date, for example 2024-01-31
INCLUDE_DOCUMENT_VIDEOS	true	Also process videos sent as files
FILTER_TOPICS / FILTER_BATCHES	empty	Comma-separated allow-lists
BATCH_SUBFOLDERS_ENABLED	false	Root/Batch/Topic folder layout
DUPLICATE_POLICY	suffix_number	suffix_number, suffix_message_id, skip, fail
MAX_RETRIES / RETRY_DELAY / RETRY_MAX_DELAY	4 / 30 / 1800	Retry behaviour (seconds)
PROGRESS_INTERVAL	10	Seconds between progress-message edits
DOWNLOAD_DIR	downloads (/tmp/telegram_drive_bot on Heroku)	Temporary video storage
UPLOAD_CHUNK_SIZE	16MB (8MB on Heroku)	Upload chunk size (multiple of 256KB)
MIN_FREE_DISK_SPACE	1GB (512MB on Heroku)	Free-disk reserve
AUTO_START	false	Start processing at every boot
LOG_LEVEL	info	debug, info, warning, error
BOT_SCAN_MAX_EMPTY / BOT_SCAN_END_ID	200 / empty	Bot-mode history probing limits
UPLOAD_RESTRICT_TO_SOURCE_CHANNEL	true	/upload links must point to SOURCE_CHANNEL_ID; false allows any channel the account/bot can access
UPLOAD_SETUP_TIMEOUT	30	Minutes before an unfinished /upload setup expires
UPLOAD_MAX_RANGE	5000	Largest number of message IDs in one /upload task
KEEP_FAILED_DOWNLOADS	false	Keep the local file of a FAILED video instead of deleting it
TEMP_CLEANUP_INTERVAL	3600	Seconds between sweeps that delete abandoned temporary files

---

Verify your configuration

Locally, with a filled .env:

python main.py check-config

This prints Configuration OK. or lists every problem with a fix. On Heroku, run heroku logs --tail --app YOUR_APP_NAME after deploying. A healthy start logs:

• Database: PostgreSQL
• Google Drive credentials validated
• Source channel resolved

The bot then sends you Bot online in Telegram.

Minimal example (Heroku, user mode, OAuth)

TELEGRAM_MODE=user
TELEGRAM_API_ID=1234567
TELEGRAM_API_HASH=0123456789abcdef0123456789abcdef
TELEGRAM_BOT_TOKEN=123456789:replace-with-your-bot-token
TELEGRAM_SESSION_STRING=1BVtsOK...replace-with-your-session-string
ADMIN_USER_IDS=111111111
SOURCE_CHANNEL_ID=-1001234567890
GOOGLE_AUTH_MODE=oauth
GOOGLE_DRIVE_SCOPE=drive.file
GOOGLE_OAUTH_TOKEN_JSON={"refresh_token": "...", "client_id": "...", "client_secret": "...", "token_uri": "https://oauth2.googleapis.com/token"}
GOOGLE_DRIVE_ROOT_FOLDER_ID=1AbCdEfGhIjKlMnOpQrStUvWxYz
# DATABASE_URL is set automatically by Heroku Postgres