krispire/spsays-training
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: update project context

Browse Source
This commit is contained in:
gosangam
2026-05-06 13:23:08 +05:30
parent c8f6a96169
commit 0498e3c0ac
1 changed files with 112 additions and 192 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PROJECT_CONTEXT.md
+112 -192
View File
@@ -2,9 +2,9 @@
## What this project is
A Next.js app that converts legacy RTF files containing Vedic scriptures and teachings by His Divine Grace A.C. Bhaktivedanta Swami Prabhupāda into structured, browsable content with proper Unicode Sanskrit.
A Next.js app presenting the books, letters, essays, lectures, and teachings of His Divine Grace A.C. Bhaktivedanta Swami Prabhupāda in a modern, accessible, and searchable format with proper Unicode Sanskrit (IAST + Devanagari).
The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/Inbeni, RM Devanagari, Balaram, etc.) where ASCII characters map to Devanagari/Bengali glyphs and IAST diacritical marks. The extraction pipeline decodes these into proper Unicode, generates Devanagari from IAST, and stores everything in MongoDB for serving via a Next.js frontend.
Content is sourced from the pre-1980s Vedabase, extracted and indexed into MongoDB, and served via a Next.js frontend. Extraction scripts and raw data live separately in the `extractor/` directory.
---
@@ -24,186 +24,96 @@ The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/I
## Project Structure
The project is split into two directories:
- **`knowledgebase/`** — Production Next.js app (~3.5MB excluding node_modules)
- **`extractor/`** — Extraction scripts, seed scripts, raw data, legacy fonts (~750MB)
```
spsays-training/
knowledgebase/
├── app/
│ ├── page.tsx → Landing page with Logo, links to Books/Essays/Journals/Glossary/Other Documents/RTFs
│ ├── layout.tsx → ThemeProvider + TooltipProvider + PasswordGate wrapper
│ ├── page.tsx → Landing page (Books, Essays, Magazines, Letters, Transcriptions, etc.)
│ ├── layout.tsx → ThemeProvider + TooltipProvider + TextSelectionToolbar + TextHighlighter
│ ├── globals.css → Vintage color palette + reading settings CSS
│ ├── books/
│ ├── about/page.tsx → About page (mission, data source, roadmap)
│ ├── books/ → 37 book routes (verse-based + prose)
│ │ ├── page.tsx → Books index (searchable grid, diacritic-aware search)
│ │ ├── books-grid.tsx → Client component for searchable book grid
│ │ ├── chapter-viewer.tsx → Shared client component for verse-based books (BG/SB/CC/SB1962)
│ │ ├── extracted-verse-count.tsx → (i) tooltip for verse counts
│ │ ├── bg/ → Bhagavad-gītā (18 chapters, 653 verses)
│ │ ├── sb/ → Śrīmad-Bhāgavatam (10 cantos, 8258 verses)
│ │ ├── cc/ → Caitanya-caritāmṛta (3 līlās, 11348 verses)
│ │ ├── sb1962/ → SB First Canto 1962 Edition (19 chapters, 762 verses)
│ │ ├── kb/ → Kṛṣṇa Book (89 chapters, 2 volumes)
│ │ ├── nod/ → The Nectar of Devotion (51 chapters, 4 parts)
│ │ ├── tlc/ → Teachings of Lord Caitanya (32 chapters, 2 parts)
│ │ ├── iso/ → Śrī Īśopaniṣad (18 mantras + invocation)
│ │ ├── noi/ → The Nectar of Instruction (11 texts)
│ │ ├── ej/ → Easy Journey to Other Planets (2 chapters)
│ │ ├── ek/ → Elevation to Kṛṣṇa Consciousness (6 chapters)
│ │ ├── mg/ → Kṛṣṇa Consciousness: The Matchless Gift (7 chapters)
│ │ ├── bbd/ → Beyond Birth and Death (5 chapters)
│ │ ├── owk/ → On the Way to Kṛṣṇa (5 chapters)
│ │ ├── rv/ → Rāja-vidyā: The King of Knowledge (8 chapters)
│ │ ├── poy/ → The Perfection of Yoga (11 chapters)
│ │ ├── pqpa/ → Perfect Questions Perfect Answers (9 chapters)
│ │ ├── ttp/ → Transcendental Teachings of Prahlāda Mahārāja (5 chapters)
│ │ ├── ochkm/ → On Chanting the Hare Kṛṣṇa Mantra (4 chapters)
│ │ ├── hkh/ → The Kṛṣṇa Consciousness Handbook (19 chapters)
│ │ ├── ssr/ → The Science of Self-Realization (8 chapters, 50 sub-chapters)
│ │ ├── ds/ → Dialectic Spiritualism (13 chapters, 36 sub-chapters)
│ │ └── va/ → Vaiśiṣṭyāṣṭakam (8 chapters)
│ ├── magazines/
│ │ ├── page.tsx → Magazines index page (grouped by flag: "Pre 1978")
│ │ └── [slug]/
│ │ ├── page.tsx → Magazine issue listing (grouped by volume)
│ │ └── [issue]/page.tsx → Individual issue reader with sidebar
│ │ ├── chapter-viewer.tsx → Shared client component for verse-based books
│ │ ├── bg/, sb/, cc/, sb1962/, iso/, noi/, bs/, mm/, nbs/ → Verse-based books
│ │ ├── kb/, nod/, tlc/, ej/, ek/, mg/, bbd/, owk/, rv/ → Prose books
│ │ ├── poy/, pqpa/, ttp/, ochkm/, hkh/, lcl/, lob/ → More prose books
│ │ ├── ssr/, ds/ → Hierarchical prose books (sub-chapters)
│ │ └── sva/, gg/, mog/, cbhc/, bk/, pk/, va/ → Other books
│ ├── magazines/ → Magazine browsing (BTG 1944-1960, BTG 1966-1973)
│ ├── letters/
│ │ ├── page.tsx → Letters index with 3-way nav (year/recipient/location)
│ │ ├── letters-nav.tsx → Client component for tab navigation
│ │ ├── letters-browser.tsx → Client component with search + pagination
│ │ ├── [slug]/page.tsx → Individual letter reader with prev/next
│ │ ├── year/[year]/page.tsx → Letters filtered by year
│ │ ├── recipient/[name]/page.tsx → Letters filtered by recipient
│ │ └── location/[name]/page.tsx → Letters filtered by location
│ ├── essays/
│ │ ├── page.tsx → Essays grid page (searchable)
│ │ ├── essays-grid.tsx → Client component for searchable essay grid
│ │ └── [slug]/page.tsx → Individual essay reader
│ ├── journals/
│ │ ├── page.tsx → Journals index page (grid)
│ │ └── [slug]/
│ │ ├── page.tsx → Journal chapter listing (hierarchical: months → days)
│ │ └── [chapter]/page.tsx → Individual journal entry reader
│ ├── other-documents/
│ │ ├── page.tsx → Other documents grid page (searchable)
│ │ ├── other-documents-grid.tsx → Client component for searchable grid
│ │ └── [slug]/
│ │ ├── page.tsx → Document chapter listing
│ │ └── [chapter]/page.tsx → Individual chapter reader
│ ├── glossary/
│ │ ├── page.tsx → Server component wrapper
│ │ └── glossary-client.tsx → Searchable client component (2749 entries)
│ ├── rtfs/
│ │ ├── page.tsx → RTF file listing (gated by ALLOW_RTF env)
│ │ └── [slug]/page.tsx → RTF iframe viewer
│ │ ├── filter-year-browser.tsx → Shared component: sidebar years + month-grouped content
│ │ ├── [slug]/page.tsx → Individual letter reader
│ │ ├── year/[year]/
│ │ │ ├── page.tsx → Letters by year
│ │ │ └── year-month-browser.tsx → Sidebar months + letter list
│ │ ├── recipient/[name]/page.tsx → Letters by recipient (year sidebar + month groups)
│ │ └── location/[name]/page.tsx → Letters by location (year sidebar + month groups)
│ ├── essays/ → 23 essays across 6 groups
│ ├── journals/ → Jaladuta Diary, The Beginning
│ ├── other-documents/ → 12 miscellaneous documents
│ ├── glossary/ → 2,749 searchable Sanskrit terms
│ ├── disciples/ → 4,747 initiated disciples directory
│ │ ├── page.tsx → Disciples browser (letter nav, suffix filter, search, sort, pagination)
│ │ └── disciples-browser.tsx → Client component
│ ├── transcriptions/page.tsx → Coming Soon page (lectures & conversations)
│ └── api/
│ ├── books/
│ │ ├── route.ts → GET all book metas
│ │ └── [book]/
│ │ ├── route.ts → GET single book meta
│ │ └── verses/route.ts → GET verses (?chapter=&verse=&division=&indexOnly=&page=&limit=)
│ ├── glossary/route.ts → GET all glossary entries
│ └── verify-access/route.ts → GET (check if password enabled) + POST (validate key)
│ ├── og/route.tsx → Dynamic OG image generator (Noto Sans, Prabhupāda image)
│ ├── books/ → Book/verse API routes
│ └── glossary/route.ts → Glossary API
├── components/
│ ├── ui/ → shadcn components (button, card, badge, tooltip, input, drawer)
│ ├── page-header.tsx → Shared header with ReadingSettings
│ ├── reading-settings.tsx → Theme/font/size/justify accessibility settings panel
│ ├── selection-toolbar/ → Text selection toolbar sub-components
│ │ ├── shared.tsx → KautilyaIcon, types, hooks (useMediaQuery, useCopyFeedback)
│ │ ├── floating-toolbar.tsx → Desktop floating pill / mobile bottom bar
│ │ ├── share-modal.tsx → Share modal (desktop) / drawer (mobile)
│ │ ├── collections-drawer.tsx → Save to collection drawer
│ │ └── more-popover.tsx → More options popover (desktop) / drawer (mobile)
│ ├── text-selection-toolbar.tsx → Main orchestrator (event listeners, state management)
│ ├── text-highlighter.tsx → Processes incoming ?hl= URLs (scroll + highlight)
│ ├── page-header.tsx → Shared header with ReadingSettings (max-w-5xl)
│ ├── reading-settings.tsx → Theme/font/size/justify accessibility panel
│ ├── mobile-nav.tsx → Left-slide drawer for mobile sidebar navigation
│ ├── logo.tsx → Theme-aware logo component (light/dark variants)
│ ├── password-gate.tsx → Client-side password protection (runtime check)
│ ├── site-footer.tsx → Global footer (About + Home links)
│ ├── logo.tsx → Theme-aware logo component
│ └── theme-provider.tsx → next-themes wrapper
├── lib/
│ ├── db.ts → Prisma client singleton
│ ├── api.ts → Data access layer (getAllBooks, getBookMeta, getChapterVerses, etc.)
│ ├── api.ts → Core data access (books, verses, prose chapters, glossary)
│ ├── constants.ts → Shared constants (BOOK_NAMES, CC_LILAS)
│ ├── og.ts → OG image metadata builders (ogForVerse, ogForLetter, etc.)
│ ├── text-highlight.ts → Text fragment encode/decode, highlight matching
│ ├── source-reference.ts → Source reference from URL (for copy attribution)
│ ├── disciples.ts → Disciples data access (by letter, place, year, search)
│ ├── letters.ts → Letters data access (by year, recipient, location)
│ ├── utils.ts → cn() helper
│ ├── nod.ts → NoD data access (Prisma)
│ ├── tlc.ts → TLC data access (Prisma)
│ ├── iso.ts → Īśopaniṣad data access (Prisma)
│ ├── noi.ts → NoI data access (Prisma)
│ ├── ej.ts → Easy Journey data access (Prisma)
│ ├── ek.ts → Elevation to KC data access (Prisma)
│ ├── mg.ts → Matchless Gift data access (Prisma)
│ ├── bbd.ts → Beyond Birth and Death data access (Prisma)
│ ├── owk.ts → On the Way to Kṛṣṇa data access (Prisma)
│ ├── rv.ts → Rāja-vidyā data access (Prisma)
│ ├── poy.ts → Perfection of Yoga data access (Prisma)
│ ├── pqpa.ts → Perfect Questions data access (Prisma)
│ ├── ttp.ts → Prahlāda Mahārāja data access (Prisma)
│ ├── ochkm.ts → On Chanting data access (Prisma)
│ ├── hkh.ts → KC Handbook data access (Prisma)
│ ├── ssr.ts → Science of Self-Realization data access (Prisma, hierarchical)
│ ├── ds.ts → Dialectic Spiritualism data access (Prisma, hierarchical)
│ ├── te.ts → Two Essays data access (Prisma, uses essays collection)
│ ├── va.ts → Vaiśiṣṭyāṣṭakam data access (Prisma)
│ ├── other-documents.ts → Other Documents data access (Prisma, dedicated collections)
│ ├── journals.ts → Journals data access (Prisma, dedicated collections)
│ ├── magazines.ts → Magazines data access (Prisma, dedicated collections)
│ ├── letters.ts → Letters data access (Prisma, queries by year/recipient/location)
│ └── jsons/books/ → Extracted JSON data (all books)
│ ├── bg/, sb/, cc/, sb-1962/, kb/ → Original verse/prose books
│ ├── nod/, tlc/, iso/, noi/ → Extracted from other.html
│ ├── ej/, ek/, mg/, bbd/, owk/ → Small prose books
│ ├── rv/, poy/, pqpa/, ttp/, ochkm/ → More prose books
│ ├── hkh/, ssr/, ds/ → Large structured books
│ ├── te/ → Two Essays
│ ├── va/ → Vaiśiṣṭyāṣṭakam
│ ├── glossary/index.json → 2,749 glossary entries
│ ├── essays/ → Essay JSON files (pre-1977, cc-essays, standalone, etc.)
│ ├── other-documents/ → Other document JSON files (12 documents)
│ ├── journals/ → Journal JSON files (2 journals, 300 chapters)
│ ├── magazines/ → Magazine JSON files (2 magazines, 59 issues)
│ │ ├── btg/ → Back to Godhead (1944–1960), 29 issues
│ │ └── btg-1978/ → Back to Godhead (1966–1973), 30 issues
│ └── letters/ → Letter JSON files (6,866 letters, grouped by year)
│ ├── index.json → Summary index (stats, top recipients/locations)
│ ├── 1947.json … 1977.json → 28 year-based letter files
│ └── undated.json → 58 undated letters
│ ├── [book].ts → Per-book data access layers (nod, tlc, iso, noi, ej, ek, etc.)
│ ├── other-documents.ts → Other Documents data access
│ ├── journals.ts → Journals data access
│ └── magazines.ts → Magazines data access
├── prisma/
│ └── schema.prisma → Verse, BookMeta, ProseChapter, Essay, GlossaryEntry, OtherDocumentMeta, OtherDocumentChapter, JournalMeta, JournalChapter, MagazineMeta, MagazineIssue, Letter models
├── scripts/
│ ├── bg_extract.py → BG extraction from bg.html
│ ├── sb_extract.py → SB extraction from sb.html
│ ├── cc_extract.py → CC extraction from cc.html
│ ├── sb1962_extract.py → 1962 SB extraction
│ ├── nod_extract.py → NoD extraction from other.html
│ ├── tlc_extract.py → TLC extraction from other.html
│ ├── iso_extract.py → Īśopaniṣad extraction from other.html
│ ├── noi_extract.py → NoI extraction from other.html
│ ├── ej_extract.py → Easy Journey extraction
│ ├── ek_extract.py → Elevation to KC extraction
│ ├── mg_extract.py → Matchless Gift extraction
│ ├── bbd_extract.py → Beyond Birth and Death extraction
│ ├── owk_extract.py → On the Way to Kṛṣṇa extraction
│ ├── rv_extract.py → Rāja-vidyā extraction
│ ├── poy_extract.py → Perfection of Yoga extraction
│ ├── pqpa_extract.py → Perfect Questions extraction
│ ├── ttp_extract.py → Prahlāda Mahārāja extraction
│ ├── ochkm_extract.py → On Chanting extraction
│ ├── hkh_extract.py → KC Handbook extraction
│ ├── ssr_extract.py → Science of Self-Realization extraction (hierarchical sub-chapters)
│ ├── ds_extract.py → Dialectic Spiritualism extraction (hierarchical sub-chapters)
│ ├── te_extract.py → Two Essays extraction
│ ├── glossary_extract.py → Glossary extraction
│ ├── seed-db.ts → Main MongoDB seeder (BG, SB, CC, SB1962, KB, glossary)
│ ├── seed-nod.ts … seed-ds.ts → Individual book seeders
│ ├── seed-te.ts → Two Essays seeder (uses essays collection)
│ ├── seed-va.ts → Vaiśiṣṭyāṣṭakam seeder
│ ├── seed-pre-1977-essays.ts → Pre-1977 essays seeder (13 essays)
│ ├── seed-other-documents.ts → Other documents seeder (dedicated collections)
│ ├── seed-journals.ts → Journals seeder (dedicated collections)
│ ├── jaladuta_diary_extract.py → Jaladuta Diary extraction (83 day entries)
│ ├── the_beginning_extract.py → The Beginning extraction (189 day entries + 20 excerpts)
│ ├── btg_extract.py → BTG (1944–1960) extraction from btg-1960.html
│ ├── btg_1978_extract.py → BTG (1966–1973) extraction from btg-1978.html
│ ├── seed-magazines.ts → Magazines seeder (dedicated collections)
│ ├── letters_extract.py → Letters extraction from letters.html (6,866 letters)
│ ├── seed-letters.ts → Letters seeder (per-document inserts)
│ ├── kb_extract_*.py → KB extraction scripts (intro, vol1, vol2)
│ ├── inject-fonts.py → Inject @font-face rules into converted HTML
│ └── fix-bengali-encoding.py → Fix Bengali font encoding in cc.html
│ └── schema.prisma → All models (Verse, BookMeta, ProseChapter, Essay, Letter,
│ GlossaryEntry, Disciple, Transcription, etc.)
└── public/
├── rtfs/ → Original RTF files (~430MB, 17 files)
├── html/ → HTML converted from RTF (textutil + @font-face injection)
├── fonts/ → Custom TTF fonts (Balaram, ScaGoudy, Indevr, Inbenr, RM Devanagari, etc.)
├── logo_dark.png → Logo for light theme (with "says.org")
├── logo_light.png → Logo for dark theme (subtle script)
└── robots.txt → Disallow all crawlers
├── fonts/Noto_Sans/static/ → NotoSans-Regular.ttf + NotoSans-Bold.ttf (OG images)
├── images/ → OG card image (Prabhupāda)
├── logo_dark.png → Logo for light theme
├── logo_light.png → Logo for dark theme
└── robots.txt
extractor/ → Extraction & seeding tools (not deployed)
├── scripts/ → Python extractors + TypeScript seed scripts
├── jsons/ → Extracted JSON data (books, letters, essays, etc.)
├── html/ → RTF→HTML converted files
├── rtfs/ → Original RTF source files
├── fonts/ → Legacy custom fonts (ScaGoudy, Indevr, Balaram, etc.)
├── app-rtfs/ → RTF viewer pages (archived)
└── .venv/ → Python virtual environment
```
---
@@ -743,12 +653,38 @@ SSR and DS have parent chapters with sub-chapters stored as individual `prose_ch
- **Light mode ("Aged Parchment")**: Warm cream backgrounds, deep sepia text, saffron/ochre primary
- **Dark mode ("Ancient Manuscript")**: Deep warm brown backgrounds, parchment text, golden saffron primary
### Password Protection
- Client-side gate component (`PasswordGate`) wrapping the entire app
- Runtime check via `GET /api/verify-access` (checks if `ACCESS_KEY` env var is set)
- Validation via `POST /api/verify-access` (compares submitted key)
- Stores valid key in `sessionStorage`
- Bypassed when `ACCESS_KEY` is not set
### Text Selection Toolbar
- Global floating toolbar appears on text selection across all reading pages (except RTFs)
- **Desktop**: floating pill above selection; **Mobile**: fixed bottom bar
- Actions: Ask Kautilya (Phase II), Share, Copy (with source reference), More
- **Share**: modal (desktop) / bottom drawer (mobile) — share to feed, copy highlight link
- **More**: popover (desktop) / bottom drawer (mobile) — bookmark, save to collection, copy link, mention
- **Copy** appends source reference: title, chapter/verse, highlight URL
- All toolbar surfaces use dark contrast treatment (bg-black, border-white/10)
- Kautilya lotus icon for the AI companion button
### Text Highlight Links
- Selected text encoded as compact URL fragment: `?hl=prefix§suffix§length`
- Uses `§` delimiter (safe for Sanskrit, English, and all Vedic text)
- Short selections (≤8 words) use full text as prefix
- Incoming `?hl=` URLs: text is found in DOM, wrapped in `<mark>`, scrolled to, highlighted permanently
- `TextHighlighter` component runs on page load via `useSearchParams`
- Server-side `extractMatchedText()` searches paragraphs for OG image quote extraction
### Dynamic OG Images
- `/api/og` route generates 1200×630 PNG images using `next/og` (Satori)
- Left panel: Prabhupāda image with quote mark and branding (from PNG)
- Right panel: title (bold), detail, quote text, "Read more at prabhupadasays.org"
- Font: Noto Sans (Regular + Bold TTF from `public/fonts/Noto_Sans/static/`)
- Full IAST diacritical support (ā, ī, ū, ṛ, ṣ, ṇ, ṭ, ḍ, ś, ñ, ṅ, ḥ, ṁ)
- Every content page has `generateMetadata` with contextual fallback descriptions
- When `?hl=` is present, the actual selected text is extracted from DB content for the OG quote
### Site Footer
- Global footer on every page with © year, About link, Home link
### Consistent Container Width
- All pages use `max-w-5xl` for both header and body containers
---
@@ -756,8 +692,7 @@ SSR and DS have parent chapters with sub-chapters stored as individual `prose_ch
```
DATABASE_URL=mongodb://user:pass@host:port/spsays?authSource=admin
ALLOW_RTF=true # optional — enables RTF viewer routes
ACCESS_KEY="secret" # optional — enables password protection (quote if contains #)
NEXT_PUBLIC_SITE_URL=https://prabhupadasays.org # used for OG image URLs
```
---
@@ -765,31 +700,16 @@ ACCESS_KEY="secret" # optional — enables password protection (quote if co
## How to Run
```bash
# Development
# Development (from knowledgebase/)
npm run dev
# Prisma
npx prisma generate # Regenerate Prisma client after schema changes
# Extraction scripts (Python 3 + beautifulsoup4 + indic-transliteration)
python3 scripts/bg_extract.py # Extract BG
python3 scripts/sb_extract.py # Extract SB
python3 scripts/cc_extract.py # Extract CC
python3 scripts/nod_extract.py # Extract NoD from other.html
python3 scripts/iso_extract.py # Extract Īśopaniṣad from other.html
python3 scripts/ssr_extract.py # Extract SSR from other.html
python3 scripts/ds_extract.py # Extract DS from other.html
# ... (similar for all other books)
# Database seeding
npx tsx scripts/seed-db.ts # Seed BG, SB, CC, SB1962, KB, glossary
npx tsx scripts/seed-nod.ts # Seed NoD
npx tsx scripts/seed-iso.ts # Seed Īśopaniṣad
npx tsx scripts/seed-te.ts # Seed Two Essays
# ... (similar for all other books)
# Build
npx next build
# Extraction & seeding scripts are in extractor/ — see extractor/scripts/
```
---