diff --git a/PROJECT_CONTEXT.md b/PROJECT_CONTEXT.md new file mode 100644 index 00000000..d56d7941 --- /dev/null +++ b/PROJECT_CONTEXT.md @@ -0,0 +1,428 @@ +# Prabhupāda Says — Project Context + +## What this project is + +A Next.js app that converts legacy RTF files containing Vedic scriptures (Bhagavad-gītā, Śrīmad-Bhāgavatam, Caitanya-caritāmṛta, Kṛṣṇa Book, letters, magazines, etc.) into structured, browsable content with proper Unicode Sanskrit. + +The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/Inbeni, RM Devanagari, Balaram, etc.) where ASCII characters map to Devanagari/Bengali glyphs and IAST diacritical marks. + +--- + +## Tech Stack + +- **Next.js 16** with Turbopack, React 19, TypeScript +- **Tailwind CSS v4** with `@tailwindcss/postcss` +- **shadcn/ui** (base-nova style, neutral base color, lucide icons) — components: Button, Card, Badge, Tooltip, Input +- **next-themes** for dark/light mode +- **Prisma v6** with MongoDB (provider `prisma-client-js`) +- **MongoDB** native driver (`mongodb` package) for seeding (Prisma requires replica set for writes) +- **Python 3** with BeautifulSoup4 + `indic-transliteration` for HTML extraction scripts +- **Database**: MongoDB at `DATABASE_URL` env variable with `?authSource=admin` + +--- + +## Project Structure + +``` +spsays-training/ +├── app/ +│ ├── page.tsx → "Prabhupāda Says" landing page (Books, Glossary, RTFs) +│ ├── layout.tsx → ThemeProvider + TooltipProvider wrapper +│ ├── globals.css → shadcn/Tailwind v4 CSS with dark mode +│ ├── books/ +│ │ ├── page.tsx → Books index (BG, SB, CC, SB1962, KB) +│ │ ├── chapter-viewer.tsx → Shared client component for verse-based books +│ │ ├── extracted-verse-count.tsx → (i) tooltip for verse counts +│ │ ├── bg/ → Bhagavad-gītā pages +│ │ │ ├── page.tsx → Chapter list +│ │ │ └── [chapter]/ +│ │ │ ├── page.tsx → Redirects to first verse +│ │ │ └── [verse]/page.tsx → Single verse viewer +│ │ ├── sb/ → Śrīmad-Bhāgavatam pages +│ │ │ ├── page.tsx → Canto list +│ │ │ └── [canto]/ +│ │ │ ├── page.tsx → Chapter list +│ │ │ └── [chapter]/ +│ │ │ ├── page.tsx → Redirects to first verse +│ │ │ └── [verse]/page.tsx → Single verse viewer +│ │ ├── cc/ → Caitanya-caritāmṛta pages (same pattern, [lila]/[chapter]/[verse]) +│ │ ├── sb1962/ → 1962 SB First Canto (same pattern as BG, [chapter]/[verse]) +│ │ └── kb/ → Kṛṣṇa Book (prose, no verses) +│ │ ├── page.tsx → Volume list +│ │ └── [volume]/ +│ │ ├── page.tsx → Chapter list + intro segments +│ │ └── [slug]/page.tsx → Reading page with sidebar +│ ├── glossary/ +│ │ ├── page.tsx → Server component wrapper +│ │ └── glossary-client.tsx → Searchable client component +│ ├── rtfs/ +│ │ ├── page.tsx → RTF file listing (gated by ALLOW_RTF env) +│ │ └── [slug]/page.tsx → RTF iframe viewer (gated by ALLOW_RTF env) +│ └── api/ +│ ├── books/ +│ │ ├── route.ts → GET all book metas +│ │ └── [book]/ +│ │ ├── route.ts → GET single book meta +│ │ └── verses/route.ts → GET verses (?chapter=&verse=&division=&indexOnly=&page=&limit=) +│ └── glossary/route.ts → GET all glossary entries +├── components/ +│ ├── ui/ → shadcn components (button, card, badge, tooltip, input) +│ ├── page-header.tsx → Shared header with ThemeToggle +│ ├── theme-provider.tsx → next-themes wrapper +│ └── theme-toggle.tsx → Sun/moon toggle +├── lib/ +│ ├── db.ts → Prisma client singleton +│ ├── api.ts → Data access layer (getAllBooks, getBookMeta, getChapterVerses, etc.) +│ ├── utils.ts → cn() helper +│ └── jsons/books/ → Extracted JSON data +│ ├── bg/ → BG chapters/verses/*.json + intro.json +│ ├── sb/ → SB cantos/chapters/verses/*.json + intro.json +│ ├── cc/ → CC lilas/chapters/verses/*.json + intro.json +│ ├── sb-1962/ → 1962 SB chapters/verses/*.json + intro.json +│ ├── kb/ → Kṛṣṇa Book +│ │ ├── intro.json → Book metadata + chapter listing (89 chapters) +│ │ ├── vol-1/intro/ → 5 intro segments (flap, dedication, George Harrison, preface, intro) +│ │ ├── vol-1/chapters/ → 54 chapter JSONs +│ │ └── vol-2/chapters/ → 35 chapter JSONs +│ └── glossary/index.json → 2,749 glossary entries +├── prisma/ +│ └── schema.prisma → Verse, BookMeta, ProseChapter, GlossaryEntry models +├── scripts/ +│ ├── bg_extract.py → BG extraction from bg.html +│ ├── sb_extract.py → SB extraction from sb.html +│ ├── cc_extract.py → CC extraction from cc.html +│ ├── sb1962_extract.py → 1962 SB extraction from others-before-1977.html +│ ├── match_sb1962.py → Match 1962 SB transliterations with standard SB IAST +│ ├── glossary_extract.py → Glossary extraction from glossary.html +│ ├── kb_extract_intro.py → KB book-level intro + chapter listing +│ ├── kb_extract_vol1_intro.py → KB Vol 1 front matter segments +│ ├── kb_extract_vol1_chapters.py → KB Vol 1 chapters +│ ├── kb_extract_vol2_chapters.py → KB Vol 2 chapters +│ ├── fix-bengali-encoding.py → Fix Bengali font encoding in cc.html +│ ├── inject-fonts.py → Inject @font-face rules into converted HTML +│ ├── seed-db.ts → Main MongoDB seeder (all books + glossary) +│ └── seed-kb-vol2.ts → KB Vol 2 seeder (uses $push workaround) +└── public/ + ├── rtfs/ → Original RTF files (~430MB, 17 files) + ├── html/ → HTML converted from RTF (textutil + @font-face injection) + └── fonts/ → Custom TTF fonts (Balaram, ScaGoudy, Indevr, Inbenr, etc.) +``` + +--- + +## Database Schema (Prisma + MongoDB) + +```prisma +type Synonym { + word String + meanings String[] +} + +model Verse { + id String @id @default(auto()) @map("_id") @db.ObjectId + book String // "bg" | "sb" | "cc" | "sb1962" + division String? // null for BG/SB1962; canto for SB; lila for CC + chapter Int + verse Int + verseEnd Int? + reference String + bookTitle String + devanagari String + bengali String? // only CC has Bengali + transliteration String + synonyms Synonym[] + translation String + purport String[] + + @@unique([book, division, chapter, verse]) + @@index([book]) + @@index([book, division]) + @@index([book, division, chapter]) + @@map("verses") +} + +model BookMeta { + id String @id @default(auto()) @map("_id") @db.ObjectId + book String @unique // "bg" | "sb" | "cc" | "sb1962" | "kb" + data Json + + @@map("book_meta") +} + +model ProseChapter { + id String @id @default(auto()) @map("_id") @db.ObjectId + book String // "kb" + volume Int + chapter Int? // null for intro segments + title String + type String // "chapter" | "intro-segment" + slug String + order Int + paragraphs String[] + + @@unique([book, volume, slug]) + @@index([book]) + @@index([book, volume]) + @@index([book, volume, type]) + @@map("prose_chapters") +} + +model GlossaryEntry { + id String @id @default(auto()) @map("_id") @db.ObjectId + term String + definition String + definitionHtml String + + @@unique([term]) + @@map("glossary") +} +``` + +--- + +## Books Extracted + +| Book | Code | Type | Verses/Chapters | Division | +|------|------|------|-----------------|----------| +| Bhagavad-gītā As It Is | `bg` | verse | 653 verses, 18 chapters | — | +| Śrīmad-Bhāgavatam | `sb` | verse | 8,260 verses, 213 chapters | 10 cantos | +| Śrī Caitanya-caritāmṛta | `cc` | verse | 11,348 verses, 62 chapters | 3 līlās (adi/madhya/antya) | +| SB First Canto (1962 Edition) | `sb1962` | verse | 762 verses, 19 chapters | — | +| Kṛṣṇa Book | `kb` | prose | 89 chapters (54+35), 2 volumes | 2 volumes | +| Glossary | — | glossary | 2,749 entries | — | + +--- + +## Font Encoding System + +### ScaGoudy → IAST mapping (used in BG, SB, CC, glossary) + +``` +ä→ā Ä→Ā å→ṛ Å→Ṛ ñ→ṣ Ñ→Ṣ ö→ṭ Ö→Ṭ ë→ṇ Ë→Ṇ ò→ḍ Ò→Ḍ +ç→ś Ç→Ś ì→ṅ ï→ñ ü→ū Ü→Ū é→ī É→Ī à→ṁ ù→ḥ è→ṝ û→ṁl ÿ→ḷ +``` + +### Bengali fonts (Inbenr/Inbeni in CC) + +Custom-encoded fonts where ASCII codepoints render as Bengali script. The cc.html had **Apple Symbol encoding corruption** — `textutil` mapped bytes through Symbol encoding instead of PUA. Fixed by `fix-bengali-encoding.py` which remaps 356,396 characters back to PUA range U+F0xx. + +### Devanagari generation + +IAST transliteration → Devanagari via `indic-transliteration` library. Key fixes: +- `join_virama_words()` — joins virāma + consonant AND virāma + vowel (fixed 12,000+ rendering issues in SB) +- `normalize_devanagari_anusvara()` — ङ् → ं before velars, ञ् → ं before palatals + +--- + +## Verse Navigation (verse-based books) + +- **One verse per page** with path-based URLs: `/books/bg/1/5`, `/books/sb/6/6/1`, `/books/cc/adi/1/3` +- **ChapterViewer** (`app/books/chapter-viewer.tsx`): Client component with: + - Sidebar showing ALL verse numbers (from lightweight index query) + - Prev/Next buttons at top and bottom + - Keyboard navigation (← →) + - Client-side verse caching + prefetching adjacent verses + - Click any verse in sidebar → fetches from API and navigates +- **API route** supports: `?verse=N` (single), `?indexOnly=true` (lightweight index), `?page=&limit=` (paginated) +- Server component SSRs the first verse, client handles navigation + +--- + +## KB (Kṛṣṇa Book) — Prose Book + +Different from verse books — chapters are prose paragraphs, no verse structure: +- `ProseChapter` model with `paragraphs: String[]` +- 5 intro segments per volume (flap, dedication, George Harrison intro, preface, introduction) +- Sanskrit verses within prose are grouped into `
...
` blocks with `
` tags +- Reading page has left sidebar with chapter index, prev/next navigation + +--- + +## Data Access Layer (`lib/api.ts`) + +```typescript +// Book meta +getAllBooks(): Promise +getBookMeta(book: string): Promise + +// Verse index (lightweight — for sidebar) +getChapterVerseIndex(book, division, chapter): Promise + +// Single verse +getSingleVerse(book, division, chapter, verse): Promise + +// Paginated verses +getChapterVerses(book, division, chapter, page?, limit?): Promise + +// Prose chapters (for KB) +getProseChapters(book, volume, type?): Promise +getProseChapter(book, volume, slug): Promise + +// Glossary +getAllGlossaryEntries(): Promise +``` + +The data access layer includes `normalizeProse()` which handles mixed storage (some chapters stored with native arrays, some with stringified JSON due to BSON corruption workaround). It also includes `sanitizeHtml()` which strips `