feat: sva extracted and seeded
This commit is contained in:
gosangam
+348
-205
@@ -2,9 +2,9 @@
|
||||
|
||||
## What this project is
|
||||
|
||||
A Next.js app that converts legacy RTF files containing Vedic scriptures (Bhagavad-gītā, Śrīmad-Bhāgavatam, Caitanya-caritāmṛta, Kṛṣṇa Book, letters, magazines, etc.) into structured, browsable content with proper Unicode Sanskrit.
|
||||
A Next.js app that converts legacy RTF files containing Vedic scriptures and teachings by His Divine Grace A.C. Bhaktivedanta Swami Prabhupāda into structured, browsable content with proper Unicode Sanskrit.
|
||||
|
||||
The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/Inbeni, RM Devanagari, Balaram, etc.) where ASCII characters map to Devanagari/Bengali glyphs and IAST diacritical marks.
|
||||
The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/Inbeni, RM Devanagari, Balaram, etc.) where ASCII characters map to Devanagari/Bengali glyphs and IAST diacritical marks. The extraction pipeline decodes these into proper Unicode, generates Devanagari from IAST, and stores everything in MongoDB for serving via a Next.js frontend.
|
||||
|
||||
---
|
||||
|
||||
@@ -12,8 +12,9 @@ The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/I
|
||||
|
||||
- **Next.js 16** with Turbopack, React 19, TypeScript
|
||||
- **Tailwind CSS v4** with `@tailwindcss/postcss`
|
||||
- **shadcn/ui** (base-nova style, neutral base color, lucide icons) — components: Button, Card, Badge, Tooltip, Input
|
||||
- **next-themes** for dark/light mode
|
||||
- **shadcn/ui** (base-nova style, lucide icons) — components: Button, Card, Badge, Tooltip, Input, Drawer
|
||||
- **next-themes** for dark/light/system mode
|
||||
- **vaul** for mobile drawer navigation
|
||||
- **Prisma v6** with MongoDB (provider `prisma-client-js`)
|
||||
- **MongoDB** native driver (`mongodb` package) for seeding (Prisma requires replica set for writes)
|
||||
- **Python 3** with BeautifulSoup4 + `indic-transliteration` for HTML extraction scripts
|
||||
@@ -26,86 +27,130 @@ The original RTF files use **custom font encodings** (Indevr, ScaGoudy, Inbenr/I
|
||||
```
|
||||
spsays-training/
|
||||
├── app/
|
||||
│ ├── page.tsx → "Prabhupāda Says" landing page (Books, Glossary, RTFs)
|
||||
│ ├── layout.tsx → ThemeProvider + TooltipProvider wrapper
|
||||
│ ├── globals.css → shadcn/Tailwind v4 CSS with dark mode
|
||||
│ ├── page.tsx → Landing page with Logo, links to Books/Essays/Glossary/RTFs
|
||||
│ ├── layout.tsx → ThemeProvider + TooltipProvider + PasswordGate wrapper
|
||||
│ ├── globals.css → Vintage color palette + reading settings CSS
|
||||
│ ├── books/
|
||||
│ │ ├── page.tsx → Books index (BG, SB, CC, SB1962, KB)
|
||||
│ │ ├── chapter-viewer.tsx → Shared client component for verse-based books
|
||||
│ │ ├── page.tsx → Books index (searchable grid, diacritic-aware search)
|
||||
│ │ ├── books-grid.tsx → Client component for searchable book grid
|
||||
│ │ ├── chapter-viewer.tsx → Shared client component for verse-based books (BG/SB/CC/SB1962)
|
||||
│ │ ├── extracted-verse-count.tsx → (i) tooltip for verse counts
|
||||
│ │ ├── bg/ → Bhagavad-gītā pages
|
||||
│ │ │ ├── page.tsx → Chapter list
|
||||
│ │ │ └── [chapter]/
|
||||
│ │ │ ├── page.tsx → Redirects to first verse
|
||||
│ │ │ └── [verse]/page.tsx → Single verse viewer
|
||||
│ │ ├── sb/ → Śrīmad-Bhāgavatam pages
|
||||
│ │ │ ├── page.tsx → Canto list
|
||||
│ │ │ └── [canto]/
|
||||
│ │ │ ├── page.tsx → Chapter list
|
||||
│ │ │ └── [chapter]/
|
||||
│ │ │ ├── page.tsx → Redirects to first verse
|
||||
│ │ │ └── [verse]/page.tsx → Single verse viewer
|
||||
│ │ ├── cc/ → Caitanya-caritāmṛta pages (same pattern, [lila]/[chapter]/[verse])
|
||||
│ │ ├── sb1962/ → 1962 SB First Canto (same pattern as BG, [chapter]/[verse])
|
||||
│ │ └── kb/ → Kṛṣṇa Book (prose, no verses)
|
||||
│ │ ├── page.tsx → Volume list
|
||||
│ │ └── [volume]/
|
||||
│ │ ├── page.tsx → Chapter list + intro segments
|
||||
│ │ └── [slug]/page.tsx → Reading page with sidebar
|
||||
│ │ ├── bg/ → Bhagavad-gītā (18 chapters, 653 verses)
|
||||
│ │ ├── sb/ → Śrīmad-Bhāgavatam (10 cantos, 8258 verses)
|
||||
│ │ ├── cc/ → Caitanya-caritāmṛta (3 līlās, 11348 verses)
|
||||
│ │ ├── sb1962/ → SB First Canto 1962 Edition (19 chapters, 762 verses)
|
||||
│ │ ├── kb/ → Kṛṣṇa Book (89 chapters, 2 volumes)
|
||||
│ │ ├── nod/ → The Nectar of Devotion (51 chapters, 4 parts)
|
||||
│ │ ├── tlc/ → Teachings of Lord Caitanya (32 chapters, 2 parts)
|
||||
│ │ ├── iso/ → Śrī Īśopaniṣad (18 mantras + invocation)
|
||||
│ │ ├── noi/ → The Nectar of Instruction (11 texts)
|
||||
│ │ ├── ej/ → Easy Journey to Other Planets (2 chapters)
|
||||
│ │ ├── ek/ → Elevation to Kṛṣṇa Consciousness (6 chapters)
|
||||
│ │ ├── mg/ → Kṛṣṇa Consciousness: The Matchless Gift (7 chapters)
|
||||
│ │ ├── bbd/ → Beyond Birth and Death (5 chapters)
|
||||
│ │ ├── owk/ → On the Way to Kṛṣṇa (5 chapters)
|
||||
│ │ ├── rv/ → Rāja-vidyā: The King of Knowledge (8 chapters)
|
||||
│ │ ├── poy/ → The Perfection of Yoga (11 chapters)
|
||||
│ │ ├── pqpa/ → Perfect Questions Perfect Answers (9 chapters)
|
||||
│ │ ├── ttp/ → Transcendental Teachings of Prahlāda Mahārāja (5 chapters)
|
||||
│ │ ├── ochkm/ → On Chanting the Hare Kṛṣṇa Mantra (4 chapters)
|
||||
│ │ ├── hkh/ → The Kṛṣṇa Consciousness Handbook (19 chapters)
|
||||
│ │ ├── ssr/ → The Science of Self-Realization (8 chapters, 50 sub-chapters)
|
||||
│ │ └── ds/ → Dialectic Spiritualism (13 chapters, 36 sub-chapters)
|
||||
│ ├── essays/
|
||||
│ │ ├── page.tsx → Essays listing page
|
||||
│ │ └── [slug]/page.tsx → Individual essay reader
|
||||
│ ├── glossary/
|
||||
│ │ ├── page.tsx → Server component wrapper
|
||||
│ │ └── glossary-client.tsx → Searchable client component
|
||||
│ │ ├── page.tsx → Server component wrapper
|
||||
│ │ └── glossary-client.tsx → Searchable client component (2749 entries)
|
||||
│ ├── rtfs/
|
||||
│ │ ├── page.tsx → RTF file listing (gated by ALLOW_RTF env)
|
||||
│ │ └── [slug]/page.tsx → RTF iframe viewer (gated by ALLOW_RTF env)
|
||||
│ │ ├── page.tsx → RTF file listing (gated by ALLOW_RTF env)
|
||||
│ │ └── [slug]/page.tsx → RTF iframe viewer
|
||||
│ └── api/
|
||||
│ ├── books/
|
||||
│ │ ├── route.ts → GET all book metas
|
||||
│ │ ├── route.ts → GET all book metas
|
||||
│ │ └── [book]/
|
||||
│ │ ├── route.ts → GET single book meta
|
||||
│ │ ├── route.ts → GET single book meta
|
||||
│ │ └── verses/route.ts → GET verses (?chapter=&verse=&division=&indexOnly=&page=&limit=)
|
||||
│ └── glossary/route.ts → GET all glossary entries
|
||||
│ ├── glossary/route.ts → GET all glossary entries
|
||||
│ └── verify-access/route.ts → GET (check if password enabled) + POST (validate key)
|
||||
├── components/
|
||||
│ ├── ui/ → shadcn components (button, card, badge, tooltip, input)
|
||||
│ ├── page-header.tsx → Shared header with ThemeToggle
|
||||
│ ├── theme-provider.tsx → next-themes wrapper
|
||||
│ └── theme-toggle.tsx → Sun/moon toggle
|
||||
│ ├── ui/ → shadcn components (button, card, badge, tooltip, input, drawer)
|
||||
│ ├── page-header.tsx → Shared header with ReadingSettings
|
||||
│ ├── reading-settings.tsx → Theme/font/size/justify accessibility settings panel
|
||||
│ ├── mobile-nav.tsx → Left-slide drawer for mobile sidebar navigation
|
||||
│ ├── logo.tsx → Theme-aware logo component (light/dark variants)
|
||||
│ ├── password-gate.tsx → Client-side password protection (runtime check)
|
||||
│ └── theme-provider.tsx → next-themes wrapper
|
||||
├── lib/
|
||||
│ ├── db.ts → Prisma client singleton
|
||||
│ ├── api.ts → Data access layer (getAllBooks, getBookMeta, getChapterVerses, etc.)
|
||||
│ ├── utils.ts → cn() helper
|
||||
│ └── jsons/books/ → Extracted JSON data
|
||||
│ ├── bg/ → BG chapters/verses/*.json + intro.json
|
||||
│ ├── sb/ → SB cantos/chapters/verses/*.json + intro.json
|
||||
│ ├── cc/ → CC lilas/chapters/verses/*.json + intro.json
|
||||
│ ├── sb-1962/ → 1962 SB chapters/verses/*.json + intro.json
|
||||
│ ├── kb/ → Kṛṣṇa Book
|
||||
│ │ ├── intro.json → Book metadata + chapter listing (89 chapters)
|
||||
│ │ ├── vol-1/intro/ → 5 intro segments (flap, dedication, George Harrison, preface, intro)
|
||||
│ │ ├── vol-1/chapters/ → 54 chapter JSONs
|
||||
│ │ └── vol-2/chapters/ → 35 chapter JSONs
|
||||
│ └── glossary/index.json → 2,749 glossary entries
|
||||
│ ├── db.ts → Prisma client singleton
|
||||
│ ├── api.ts → Data access layer (getAllBooks, getBookMeta, getChapterVerses, etc.)
|
||||
│ ├── utils.ts → cn() helper
|
||||
│ ├── nod.ts → NoD data access (Prisma)
|
||||
│ ├── tlc.ts → TLC data access (Prisma)
|
||||
│ ├── iso.ts → Īśopaniṣad data access (Prisma)
|
||||
│ ├── noi.ts → NoI data access (Prisma)
|
||||
│ ├── ej.ts → Easy Journey data access (Prisma)
|
||||
│ ├── ek.ts → Elevation to KC data access (Prisma)
|
||||
│ ├── mg.ts → Matchless Gift data access (Prisma)
|
||||
│ ├── bbd.ts → Beyond Birth and Death data access (Prisma)
|
||||
│ ├── owk.ts → On the Way to Kṛṣṇa data access (Prisma)
|
||||
│ ├── rv.ts → Rāja-vidyā data access (Prisma)
|
||||
│ ├── poy.ts → Perfection of Yoga data access (Prisma)
|
||||
│ ├── pqpa.ts → Perfect Questions data access (Prisma)
|
||||
│ ├── ttp.ts → Prahlāda Mahārāja data access (Prisma)
|
||||
│ ├── ochkm.ts → On Chanting data access (Prisma)
|
||||
│ ├── hkh.ts → KC Handbook data access (Prisma)
|
||||
│ ├── ssr.ts → Science of Self-Realization data access (Prisma, hierarchical)
|
||||
│ ├── ds.ts → Dialectic Spiritualism data access (Prisma, hierarchical)
|
||||
│ ├── te.ts → Two Essays data access (Prisma, uses essays collection)
|
||||
│ └── jsons/books/ → Extracted JSON data (all books)
|
||||
│ ├── bg/, sb/, cc/, sb-1962/, kb/ → Original verse/prose books
|
||||
│ ├── nod/, tlc/, iso/, noi/ → Extracted from other.html
|
||||
│ ├── ej/, ek/, mg/, bbd/, owk/ → Small prose books
|
||||
│ ├── rv/, poy/, pqpa/, ttp/, ochkm/ → More prose books
|
||||
│ ├── hkh/, ssr/, ds/ → Large structured books
|
||||
│ ├── te/ → Two Essays
|
||||
│ └── glossary/index.json → 2,749 glossary entries
|
||||
├── prisma/
|
||||
│ └── schema.prisma → Verse, BookMeta, ProseChapter, GlossaryEntry models
|
||||
│ └── schema.prisma → Verse, BookMeta, ProseChapter, Essay, GlossaryEntry models
|
||||
├── scripts/
|
||||
│ ├── bg_extract.py → BG extraction from bg.html
|
||||
│ ├── sb_extract.py → SB extraction from sb.html
|
||||
│ ├── cc_extract.py → CC extraction from cc.html
|
||||
│ ├── sb1962_extract.py → 1962 SB extraction from others-before-1977.html
|
||||
│ ├── match_sb1962.py → Match 1962 SB transliterations with standard SB IAST
|
||||
│ ├── glossary_extract.py → Glossary extraction from glossary.html
|
||||
│ ├── kb_extract_intro.py → KB book-level intro + chapter listing
|
||||
│ ├── kb_extract_vol1_intro.py → KB Vol 1 front matter segments
|
||||
│ ├── kb_extract_vol1_chapters.py → KB Vol 1 chapters
|
||||
│ ├── kb_extract_vol2_chapters.py → KB Vol 2 chapters
|
||||
│ ├── fix-bengali-encoding.py → Fix Bengali font encoding in cc.html
|
||||
│ ├── inject-fonts.py → Inject @font-face rules into converted HTML
|
||||
│ ├── seed-db.ts → Main MongoDB seeder (all books + glossary)
|
||||
│ └── seed-kb-vol2.ts → KB Vol 2 seeder (uses $push workaround)
|
||||
│ ├── bg_extract.py → BG extraction from bg.html
|
||||
│ ├── sb_extract.py → SB extraction from sb.html
|
||||
│ ├── cc_extract.py → CC extraction from cc.html
|
||||
│ ├── sb1962_extract.py → 1962 SB extraction
|
||||
│ ├── nod_extract.py → NoD extraction from other.html
|
||||
│ ├── tlc_extract.py → TLC extraction from other.html
|
||||
│ ├── iso_extract.py → Īśopaniṣad extraction from other.html
|
||||
│ ├── noi_extract.py → NoI extraction from other.html
|
||||
│ ├── ej_extract.py → Easy Journey extraction
|
||||
│ ├── ek_extract.py → Elevation to KC extraction
|
||||
│ ├── mg_extract.py → Matchless Gift extraction
|
||||
│ ├── bbd_extract.py → Beyond Birth and Death extraction
|
||||
│ ├── owk_extract.py → On the Way to Kṛṣṇa extraction
|
||||
│ ├── rv_extract.py → Rāja-vidyā extraction
|
||||
│ ├── poy_extract.py → Perfection of Yoga extraction
|
||||
│ ├── pqpa_extract.py → Perfect Questions extraction
|
||||
│ ├── ttp_extract.py → Prahlāda Mahārāja extraction
|
||||
│ ├── ochkm_extract.py → On Chanting extraction
|
||||
│ ├── hkh_extract.py → KC Handbook extraction
|
||||
│ ├── ssr_extract.py → Science of Self-Realization extraction (hierarchical sub-chapters)
|
||||
│ ├── ds_extract.py → Dialectic Spiritualism extraction (hierarchical sub-chapters)
|
||||
│ ├── te_extract.py → Two Essays extraction
|
||||
│ ├── glossary_extract.py → Glossary extraction
|
||||
│ ├── seed-db.ts → Main MongoDB seeder (BG, SB, CC, SB1962, KB, glossary)
|
||||
│ ├── seed-nod.ts … seed-ds.ts → Individual book seeders
|
||||
│ ├── seed-te.ts → Two Essays seeder (uses essays collection)
|
||||
│ ├── kb_extract_*.py → KB extraction scripts (intro, vol1, vol2)
|
||||
│ ├── inject-fonts.py → Inject @font-face rules into converted HTML
|
||||
│ └── fix-bengali-encoding.py → Fix Bengali font encoding in cc.html
|
||||
└── public/
|
||||
├── rtfs/ → Original RTF files (~430MB, 17 files)
|
||||
├── html/ → HTML converted from RTF (textutil + @font-face injection)
|
||||
└── fonts/ → Custom TTF fonts (Balaram, ScaGoudy, Indevr, Inbenr, etc.)
|
||||
├── rtfs/ → Original RTF files (~430MB, 17 files)
|
||||
├── html/ → HTML converted from RTF (textutil + @font-face injection)
|
||||
├── fonts/ → Custom TTF fonts (Balaram, ScaGoudy, Indevr, Inbenr, RM Devanagari, etc.)
|
||||
├── logo_dark.png → Logo for light theme (with "says.org")
|
||||
├── logo_light.png → Logo for dark theme (subtle script)
|
||||
└── robots.txt → Disallow all crawlers
|
||||
```
|
||||
|
||||
---
|
||||
@@ -120,8 +165,8 @@ type Synonym {
|
||||
|
||||
model Verse {
|
||||
id String @id @default(auto()) @map("_id") @db.ObjectId
|
||||
book String // "bg" | "sb" | "cc" | "sb1962"
|
||||
division String? // null for BG/SB1962; canto for SB; lila for CC
|
||||
book String // "bg" | "sb" | "cc" | "sb1962" | "iso" | "noi"
|
||||
division String? // null for BG/SB1962/ISO/NOI; canto for SB; lila for CC
|
||||
chapter Int
|
||||
verse Int
|
||||
verseEnd Int?
|
||||
@@ -143,7 +188,7 @@ model Verse {
|
||||
|
||||
model BookMeta {
|
||||
id String @id @default(auto()) @map("_id") @db.ObjectId
|
||||
book String @unique // "bg" | "sb" | "cc" | "sb1962" | "kb"
|
||||
book String @unique
|
||||
data Json
|
||||
|
||||
@@map("book_meta")
|
||||
@@ -151,11 +196,11 @@ model BookMeta {
|
||||
|
||||
model ProseChapter {
|
||||
id String @id @default(auto()) @map("_id") @db.ObjectId
|
||||
book String // "kb"
|
||||
volume Int
|
||||
book String
|
||||
volume Int // 0 = no volumes (sentinel for non-volume books)
|
||||
chapter Int? // null for intro segments
|
||||
title String
|
||||
type String // "chapter" | "intro-segment"
|
||||
type String // "chapter" | "intro-segment" | "conclusion"
|
||||
slug String
|
||||
order Int
|
||||
paragraphs String[]
|
||||
@@ -167,6 +212,20 @@ model ProseChapter {
|
||||
@@map("prose_chapters")
|
||||
}
|
||||
|
||||
model Essay {
|
||||
id String @id @default(auto()) @map("_id") @db.ObjectId
|
||||
slug String @unique
|
||||
title String
|
||||
author String
|
||||
year Int?
|
||||
group String? // e.g., "two-essays"
|
||||
order Int
|
||||
paragraphs String[]
|
||||
|
||||
@@index([group])
|
||||
@@map("essays")
|
||||
}
|
||||
|
||||
model GlossaryEntry {
|
||||
id String @id @default(auto()) @map("_id") @db.ObjectId
|
||||
term String
|
||||
@@ -182,20 +241,56 @@ model GlossaryEntry {
|
||||
|
||||
## Books Extracted
|
||||
|
||||
| Book | Code | Type | Verses/Chapters | Division |
|
||||
|------|------|------|-----------------|----------|
|
||||
| Bhagavad-gītā As It Is | `bg` | verse | 653 verses, 18 chapters | — |
|
||||
| Śrīmad-Bhāgavatam | `sb` | verse | 8,260 verses, 213 chapters | 10 cantos |
|
||||
| Śrī Caitanya-caritāmṛta | `cc` | verse | 11,348 verses, 62 chapters | 3 līlās (adi/madhya/antya) |
|
||||
| SB First Canto (1962 Edition) | `sb1962` | verse | 762 verses, 19 chapters | — |
|
||||
| Kṛṣṇa Book | `kb` | prose | 89 chapters (54+35), 2 volumes | 2 volumes |
|
||||
| Glossary | — | glossary | 2,749 entries | — |
|
||||
### Verse-based Books (stored in `verses` collection)
|
||||
|
||||
| Book | Code | Verses | Chapters | Division |
|
||||
|------|------|--------|----------|----------|
|
||||
| Bhagavad-gītā As It Is | `bg` | 653 | 18 | — |
|
||||
| Śrīmad-Bhāgavatam | `sb` | 8,258 | 213 | 10 cantos |
|
||||
| Śrī Caitanya-caritāmṛta | `cc` | 11,348 | 62 | 3 līlās |
|
||||
| SB First Canto (1962 Edition) | `sb1962` | 762 | 19 | — |
|
||||
| Śrī Īśopaniṣad | `iso` | 19 | — | Invocation + 18 mantras |
|
||||
| The Nectar of Instruction | `noi` | 11 | — | 11 texts |
|
||||
|
||||
### Prose Books (stored in `prose_chapters` collection)
|
||||
|
||||
| Book | Code | Chapters | Source |
|
||||
|------|------|----------|--------|
|
||||
| Kṛṣṇa Book | `kb` | 89 (2 volumes) | other.html |
|
||||
| The Nectar of Devotion | `nod` | 51 (4 parts) | other.html |
|
||||
| Teachings of Lord Caitanya | `tlc` | 32 (2 parts) | other.html (1968 ed., plain English) |
|
||||
| Easy Journey to Other Planets | `ej` | 2 | other.html |
|
||||
| Elevation to Kṛṣṇa Consciousness | `ek` | 6 | other.html |
|
||||
| Kṛṣṇa Consciousness: The Matchless Gift | `mg` | 7 | other.html |
|
||||
| Beyond Birth and Death | `bbd` | 5 | other.html |
|
||||
| On the Way to Kṛṣṇa | `owk` | 5 | other.html |
|
||||
| Rāja-vidyā: The King of Knowledge | `rv` | 8 | other.html |
|
||||
| The Perfection of Yoga | `poy` | 11 | other.html |
|
||||
| Perfect Questions Perfect Answers | `pqpa` | 9 | other.html |
|
||||
| Transcendental Teachings of Prahlāda Mahārāja | `ttp` | 5 | other.html |
|
||||
| On Chanting the Hare Kṛṣṇa Mantra | `ochkm` | 4 | other.html |
|
||||
| The Kṛṣṇa Consciousness Handbook | `hkh` | 19 | other.html |
|
||||
| The Science of Self-Realization | `ssr` | 50 sub-chapters (8 parents) | other.html |
|
||||
| Dialectic Spiritualism | `ds` | 36 sub-chapters (13 parents) | other.html |
|
||||
|
||||
### Essays (stored in `essays` collection)
|
||||
|
||||
| Essay | Slug | Group |
|
||||
|-------|------|-------|
|
||||
| Krishna The Reservoir Of Pleasure | `krishna-reservoir-of-pleasure` | `two-essays` |
|
||||
| Who Is Crazy? | `who-is-crazy` | `two-essays` |
|
||||
|
||||
### Other
|
||||
|
||||
| Content | Collection |
|
||||
|---------|-----------|
|
||||
| Glossary (2,749 entries) | `glossary` |
|
||||
|
||||
---
|
||||
|
||||
## Font Encoding System
|
||||
|
||||
### ScaGoudy → IAST mapping (used in BG, SB, CC, glossary)
|
||||
### ScaGoudy → IAST mapping (used in BG, SB, CC, ISO, NOI, NoD, and most 1969+ books)
|
||||
|
||||
```
|
||||
ä→ā Ä→Ā å→ṛ Å→Ṛ ñ→ṣ Ñ→Ṣ ö→ṭ Ö→Ṭ ë→ṇ Ë→Ṇ ò→ḍ Ò→Ḍ
|
||||
@@ -204,81 +299,137 @@ model GlossaryEntry {
|
||||
|
||||
### Bengali fonts (Inbenr/Inbeni in CC)
|
||||
|
||||
Custom-encoded fonts where ASCII codepoints render as Bengali script. The cc.html had **Apple Symbol encoding corruption** — `textutil` mapped bytes through Symbol encoding instead of PUA. Fixed by `fix-bengali-encoding.py` which remaps 356,396 characters back to PUA range U+F0xx.
|
||||
Custom-encoded fonts where ASCII codepoints render as Bengali script. Fixed by `fix-bengali-encoding.py`.
|
||||
|
||||
### RM Devanagari font (used in ISO, NOI)
|
||||
|
||||
Custom-encoded Devanagari font (p51 CSS class). The text appears as garbled ASCII without the font. **Skipped during extraction** — Unicode Devanagari is generated from IAST transliteration instead.
|
||||
|
||||
### Devanagari generation
|
||||
|
||||
IAST transliteration → Devanagari via `indic-transliteration` library. Key fixes:
|
||||
- `join_virama_words()` — joins virāma + consonant AND virāma + vowel (fixed 12,000+ rendering issues in SB)
|
||||
- `join_virama_words()` — joins virāma + consonant AND virāma + vowel
|
||||
- `normalize_devanagari_anusvara()` — ङ् → ं before velars, ञ् → ं before palatals
|
||||
|
||||
---
|
||||
### Books without ScaGoudy encoding
|
||||
|
||||
## Verse Navigation (verse-based books)
|
||||
|
||||
- **One verse per page** with path-based URLs: `/books/bg/1/5`, `/books/sb/6/6/1`, `/books/cc/adi/1/3`
|
||||
- **ChapterViewer** (`app/books/chapter-viewer.tsx`): Client component with:
|
||||
- Sidebar showing ALL verse numbers (from lightweight index query)
|
||||
- Prev/Next buttons at top and bottom
|
||||
- Keyboard navigation (← →)
|
||||
- Client-side verse caching + prefetching adjacent verses
|
||||
- Click any verse in sidebar → fetches from API and navigates
|
||||
- **API route** supports: `?verse=N` (single), `?indexOnly=true` (lightweight index), `?page=&limit=` (paginated)
|
||||
- Server component SSRs the first verse, client handles navigation
|
||||
- **TLC (1968 Edition)** — plain English transliterations ("Krishna" not "Kṛṣṇa")
|
||||
- **Two Essays (1966)** — plain English
|
||||
- All other books from other.html use ScaGoudy encoding
|
||||
|
||||
---
|
||||
|
||||
## KB (Kṛṣṇa Book) — Prose Book
|
||||
## Book Types and URL Patterns
|
||||
|
||||
Different from verse books — chapters are prose paragraphs, no verse structure:
|
||||
- `ProseChapter` model with `paragraphs: String[]`
|
||||
- 5 intro segments per volume (flap, dedication, George Harrison intro, preface, introduction)
|
||||
- Sanskrit verses within prose are grouped into `<div style="text-align:center"><i>...</i></div>` blocks with `<br>` tags
|
||||
- Reading page has left sidebar with chapter index, prev/next navigation
|
||||
### Verse-based books (use ChapterViewer client component)
|
||||
|
||||
| Route Pattern | Books |
|
||||
|---------------|-------|
|
||||
| `/books/bg/[chapter]/[verse]` | BG |
|
||||
| `/books/sb/[canto]/[chapter]/[verse]` | SB |
|
||||
| `/books/cc/[lila]/[chapter]/[verse]` | CC |
|
||||
| `/books/sb1962/[chapter]/[verse]` | SB1962 |
|
||||
| `/books/iso/[mantra]` | Īśopaniṣad (verse + intro segments) |
|
||||
| `/books/noi/[text]` | Nectar of Instruction (verse + intro segments) |
|
||||
|
||||
### Prose books (use per-book data layers + reading pages)
|
||||
|
||||
| Route Pattern | Books |
|
||||
|---------------|-------|
|
||||
| `/books/kb/[volume]/[slug]` | Kṛṣṇa Book |
|
||||
| `/books/{code}/[chapter]` | NoD, TLC, EJ, EK, MG, BBD, OWK, RV, POY, PQPA, TTP, OCHKM, HKH |
|
||||
|
||||
### Hierarchical prose books (sub-chapters as separate pages)
|
||||
|
||||
| Route Pattern | Books |
|
||||
|---------------|-------|
|
||||
| `/books/ssr/[chapter]` | Science of Self-Realization (slugs: "1-1", "1-2", "intro-foreword", etc.) |
|
||||
| `/books/ds/[chapter]` | Dialectic Spiritualism (slugs: "1-1" = Socrates, "1-2" = Plato, etc.) |
|
||||
|
||||
### Essays
|
||||
|
||||
| Route | Content |
|
||||
|-------|---------|
|
||||
| `/essays` | All essays listing |
|
||||
| `/essays/[slug]` | Individual essay reader |
|
||||
|
||||
### Other routes
|
||||
|
||||
| Route | Content |
|
||||
|-------|---------|
|
||||
| `/` | Landing page |
|
||||
| `/books` | Searchable book grid |
|
||||
| `/glossary` | Searchable glossary (2,749 entries) |
|
||||
| `/rtfs` | RTF viewer (gated by ALLOW_RTF=true) |
|
||||
| `/api/verify-access` | Password protection endpoint |
|
||||
|
||||
---
|
||||
|
||||
## Data Access Layer (`lib/api.ts`)
|
||||
## Volume Convention
|
||||
|
||||
```typescript
|
||||
// Book meta
|
||||
getAllBooks(): Promise<BookMeta[]>
|
||||
getBookMeta(book: string): Promise<BookMeta | null>
|
||||
Books without volumes use `volume: 0` as a sentinel value in the `prose_chapters` collection. This satisfies the Prisma schema's `@@unique([book, volume, slug])` constraint without requiring schema changes.
|
||||
|
||||
// Verse index (lightweight — for sidebar)
|
||||
getChapterVerseIndex(book, division, chapter): Promise<VerseIndexEntry[]>
|
||||
|
||||
// Single verse
|
||||
getSingleVerse(book, division, chapter, verse): Promise<VerseRecord | null>
|
||||
|
||||
// Paginated verses
|
||||
getChapterVerses(book, division, chapter, page?, limit?): Promise<PaginatedVerses>
|
||||
|
||||
// Prose chapters (for KB)
|
||||
getProseChapters(book, volume, type?): Promise<ProseChapterRecord[]>
|
||||
getProseChapter(book, volume, slug): Promise<ProseChapterRecord | null>
|
||||
|
||||
// Glossary
|
||||
getAllGlossaryEntries(): Promise<GlossaryRecord[]>
|
||||
```
|
||||
|
||||
The data access layer includes `normalizeProse()` which handles mixed storage (some chapters stored with native arrays, some with stringified JSON due to BSON corruption workaround). It also includes `sanitizeHtml()` which strips `<script>`, `<style>`, `<iframe>`, event handlers from paragraphs.
|
||||
- KB uses `volume: 1` and `volume: 2`
|
||||
- All other prose books use `volume: 0`
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
## Hierarchical Books (SSR, DS)
|
||||
|
||||
1. **MongoDB BSON wire corruption**: The MongoDB server (7.0.31) behind a proxy/tunnel occasionally corrupts large BSON documents during transmission. Workaround: seed using `insertOne` + `$push` for paragraphs one at a time. The `insertMany` and large `insertOne` operations fail non-deterministically with "Invalid UTF-8 string" or "Unrecognized BSON type" errors. Retrying usually works.
|
||||
SSR and DS have parent chapters with sub-chapters stored as individual `prose_chapters` docs:
|
||||
|
||||
2. **SB verse numbering**: The HTML had 5 duplicate `p20` verse reference tags. Fixed by using the `p21` TEXT N header as the authoritative verse number (corrects the wrong `p20` reference).
|
||||
- Sub-chapter slugs: `"1-1"`, `"1-2"`, `"2-1"`, etc.
|
||||
- Parent chapter info stored in `intro.json` metadata:
|
||||
```json
|
||||
{
|
||||
"chapters": [
|
||||
{
|
||||
"number": 1,
|
||||
"title": "Learning the Science of the Soul",
|
||||
"subChapters": [
|
||||
{ "slug": "1-1", "title": "Seeing the Aim of Human Life" },
|
||||
{ "slug": "1-2", "title": "Your Original Consciousness Is Kṛṣṇa Consciousness" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
- Sidebar UI shows parent headings (non-clickable) with sub-chapters indented below
|
||||
- Prev/next navigation skips parent headings
|
||||
|
||||
3. **SB 1962 chapter 15**: Verses 41-45 have different numbering between the 1962 edition and standard SB. The matching script skips these 5 verses (keeps original plain English transliteration).
|
||||
---
|
||||
|
||||
4. **RTF routes gated**: `/rtfs` and `/rtfs/[slug]` only accessible when `ALLOW_RTF=true` env variable is set.
|
||||
## UI Features
|
||||
|
||||
5. **`scripts/` excluded from tsconfig**: Added to `"exclude"` array to prevent TypeScript build errors from seed scripts.
|
||||
### Reading Settings (accessibility)
|
||||
- **Theme**: Light / Dark / System toggle
|
||||
- **Serif Font**: Toggle to switch reading content to Georgia/serif
|
||||
- **Font Size**: Slider (XS → XXL) using CSS `zoom` on `main` and `aside`
|
||||
- **Justify Text**: Toggle for justified text alignment on reading content
|
||||
- Settings persist in `localStorage` under `spsays_reading_prefs`
|
||||
- Applied via CSS classes on `<html>` element
|
||||
|
||||
6. **MongoDB requires no replica set for reads** but requires replica set for Prisma writes. All writes use the native MongoDB driver instead of Prisma.
|
||||
### Mobile Navigation
|
||||
- Left-slide drawer (vaul) triggered by a menu button next to the Previous button
|
||||
- Only visible below `lg` breakpoint
|
||||
- Shows the full sidebar navigation grouped by section
|
||||
- Implemented in all 24+ reader pages via shared `MobileNav` component
|
||||
|
||||
### Search
|
||||
- Books page: diacritic-aware search (typing "Krsna" matches "Kṛṣṇa")
|
||||
- Glossary page: full-text search across terms and definitions
|
||||
- Uses Unicode NFD normalization + manual IAST character stripping
|
||||
|
||||
### Color Palette
|
||||
- **Light mode ("Aged Parchment")**: Warm cream backgrounds, deep sepia text, saffron/ochre primary
|
||||
- **Dark mode ("Ancient Manuscript")**: Deep warm brown backgrounds, parchment text, golden saffron primary
|
||||
|
||||
### Password Protection
|
||||
- Client-side gate component (`PasswordGate`) wrapping the entire app
|
||||
- Runtime check via `GET /api/verify-access` (checks if `ACCESS_KEY` env var is set)
|
||||
- Validation via `POST /api/verify-access` (compares submitted key)
|
||||
- Stores valid key in `sessionStorage`
|
||||
- Bypassed when `ACCESS_KEY` is not set
|
||||
|
||||
---
|
||||
|
||||
@@ -286,7 +437,8 @@ The data access layer includes `normalizeProse()` which handles mixed storage (s
|
||||
|
||||
```
|
||||
DATABASE_URL=mongodb://user:pass@host:port/spsays?authSource=admin
|
||||
ALLOW_RTF=true # optional — enables RTF viewer routes
|
||||
ALLOW_RTF=true # optional — enables RTF viewer routes
|
||||
ACCESS_KEY="secret" # optional — enables password protection (quote if contains #)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -295,67 +447,35 @@ ALLOW_RTF=true # optional — enables RTF viewer routes
|
||||
|
||||
```bash
|
||||
# Development
|
||||
npm run dev # Start dev server
|
||||
npm run dev
|
||||
|
||||
# Prisma
|
||||
npx prisma generate # Regenerate Prisma client after schema changes
|
||||
|
||||
# Extraction scripts (Python 3 + beautifulsoup4 + indic-transliteration)
|
||||
python3 scripts/bg_extract.py # Extract BG → lib/jsons/books/bg/
|
||||
python3 scripts/sb_extract.py # Extract SB → lib/jsons/books/sb/
|
||||
python3 scripts/cc_extract.py # Extract CC → lib/jsons/books/cc/
|
||||
python3 scripts/glossary_extract.py # Extract glossary → lib/jsons/glossary/
|
||||
python3 scripts/sb1962_extract.py # Extract 1962 SB → lib/jsons/books/sb-1962/
|
||||
python3 scripts/match_sb1962.py # Match 1962 SB with standard SB IAST
|
||||
python3 scripts/kb_extract_intro.py # Extract KB intro → lib/jsons/books/kb/intro.json
|
||||
python3 scripts/kb_extract_vol1_intro.py # KB Vol 1 front matter
|
||||
python3 scripts/kb_extract_vol1_chapters.py # KB Vol 1 chapters
|
||||
python3 scripts/kb_extract_vol2_chapters.py # KB Vol 2 chapters
|
||||
python3 scripts/bg_extract.py # Extract BG
|
||||
python3 scripts/sb_extract.py # Extract SB
|
||||
python3 scripts/cc_extract.py # Extract CC
|
||||
python3 scripts/nod_extract.py # Extract NoD from other.html
|
||||
python3 scripts/iso_extract.py # Extract Īśopaniṣad from other.html
|
||||
python3 scripts/ssr_extract.py # Extract SSR from other.html
|
||||
python3 scripts/ds_extract.py # Extract DS from other.html
|
||||
# ... (similar for all other books)
|
||||
|
||||
# Database seeding (TypeScript + tsx)
|
||||
npx tsx scripts/seed-db.ts # Seed all data to MongoDB
|
||||
npx tsx scripts/seed-kb-vol2.ts # Seed KB Vol 2 (separate due to BSON issues)
|
||||
# Database seeding
|
||||
npx tsx scripts/seed-db.ts # Seed BG, SB, CC, SB1962, KB, glossary
|
||||
npx tsx scripts/seed-nod.ts # Seed NoD
|
||||
npx tsx scripts/seed-iso.ts # Seed Īśopaniṣad
|
||||
npx tsx scripts/seed-te.ts # Seed Two Essays
|
||||
# ... (similar for all other books)
|
||||
|
||||
# Build
|
||||
npx next build # Production build
|
||||
npx next build
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## URL Routes
|
||||
|
||||
| Route | Description |
|
||||
|-------|-------------|
|
||||
| `/` | Landing page — Books, Glossary, RTF Viewer cards |
|
||||
| `/books` | All books index |
|
||||
| `/books/bg` | BG chapter list |
|
||||
| `/books/bg/[chapter]` | Redirects to first verse |
|
||||
| `/books/bg/[chapter]/[verse]` | Single verse viewer |
|
||||
| `/books/sb` | SB canto list |
|
||||
| `/books/sb/[canto]` | Chapter list for a canto |
|
||||
| `/books/sb/[canto]/[chapter]` | Redirects to first verse |
|
||||
| `/books/sb/[canto]/[chapter]/[verse]` | Single verse viewer |
|
||||
| `/books/cc` | CC līlā list |
|
||||
| `/books/cc/[lila]` | Chapter list for a līlā |
|
||||
| `/books/cc/[lila]/[chapter]` | Redirects to first verse |
|
||||
| `/books/cc/[lila]/[chapter]/[verse]` | Single verse viewer |
|
||||
| `/books/sb1962` | 1962 SB chapter list |
|
||||
| `/books/sb1962/[chapter]` | Redirects to first verse |
|
||||
| `/books/sb1962/[chapter]/[verse]` | Single verse viewer |
|
||||
| `/books/kb` | KB volume list |
|
||||
| `/books/kb/[volume]` | Chapter list + intro segments |
|
||||
| `/books/kb/[volume]/[slug]` | Prose reading page with sidebar |
|
||||
| `/glossary` | Searchable glossary (2,749 entries) |
|
||||
| `/rtfs` | RTF file listing (requires ALLOW_RTF=true) |
|
||||
| `/rtfs/[slug]` | RTF iframe viewer (requires ALLOW_RTF=true) |
|
||||
| `/api/books` | GET all book metas |
|
||||
| `/api/books/[book]` | GET single book meta |
|
||||
| `/api/books/[book]/verses` | GET verses with query params |
|
||||
| `/api/glossary` | GET all glossary entries |
|
||||
|
||||
---
|
||||
|
||||
## Verse JSON Schema (BG/SB/CC/SB1962)
|
||||
## Verse JSON Schema (BG/SB/CC/SB1962/ISO/NOI)
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -373,39 +493,46 @@ npx next build # Production build
|
||||
{ "word": "namaḥ", "meanings": ["offering my obeisances"] }
|
||||
],
|
||||
"translation": "<b>I offer my obeisances unto Lord Śrī Kṛṣṇa...</b>",
|
||||
"purport": ["Obeisances unto the Personality of Godhead...", "paragraph 2..."]
|
||||
"purport": ["paragraph 1...", "paragraph 2..."]
|
||||
}
|
||||
```
|
||||
|
||||
## Prose Chapter JSON Schema (KB)
|
||||
## Prose Chapter JSON Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"book": "kb",
|
||||
"volume": 1,
|
||||
"book": "ek",
|
||||
"chapter": 1,
|
||||
"title": "Advent of Lord Kṛṣṇa",
|
||||
"title": "Choosing Human and Animal Lives",
|
||||
"type": "chapter",
|
||||
"paragraphs": [
|
||||
"<b>1 /Advent of Lord Kṛṣṇa</b>",
|
||||
"Once the world was overburdened...",
|
||||
"<div style=\"text-align:center\"><i>verse line 1<br>verse line 2</i></div>",
|
||||
"<i>Thus ends the Bhaktivedanta purport of the First Chapter of </i>Kṛṣṇa..."
|
||||
]
|
||||
"paragraphs": ["<b>heading</b>", "body text...", "<div style=\"text-align:center\"><i>verse</i></div>"]
|
||||
}
|
||||
```
|
||||
|
||||
## Intro Segment JSON Schema (KB)
|
||||
## Intro Segment JSON Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"slug": "words-from-apple",
|
||||
"title": "Words from Apple — by George Harrison",
|
||||
"slug": "preface",
|
||||
"title": "Preface",
|
||||
"type": "intro-segment",
|
||||
"book": "kb",
|
||||
"volume": 1,
|
||||
"order": 3,
|
||||
"paragraphs": ["<b>Words From Apple</b>", "Everybody is looking for KṚṢṆA.", "..."]
|
||||
"book": "ek",
|
||||
"order": 2,
|
||||
"paragraphs": ["paragraph 1...", "paragraph 2..."]
|
||||
}
|
||||
```
|
||||
|
||||
## Essay JSON Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"slug": "krishna-reservoir-of-pleasure",
|
||||
"title": "Krishna The Reservoir Of Pleasure",
|
||||
"author": "His Divine Grace A.C. Bhaktivedanta Swami Prabhupāda",
|
||||
"year": 1966,
|
||||
"group": "two-essays",
|
||||
"order": 1,
|
||||
"paragraphs": ["paragraph 1...", "paragraph 2..."]
|
||||
}
|
||||
```
|
||||
|
||||
@@ -414,7 +541,7 @@ npx next build # Production build
|
||||
```json
|
||||
{
|
||||
"title": "Glossary of Vedic Terms",
|
||||
"description": "A comprehensive glossary of Sanskrit and Vedic terms...",
|
||||
"description": "A comprehensive glossary...",
|
||||
"totalEntries": 2749,
|
||||
"letters": ["A", "B", "C", ...],
|
||||
"entries": [
|
||||
@@ -426,3 +553,19 @@ npx next build # Production build
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known Issues
|
||||
|
||||
1. **MongoDB BSON wire corruption**: The MongoDB server occasionally corrupts large BSON documents. Workaround: seed using `insertOne` + `$push` for paragraphs one at a time.
|
||||
|
||||
2. **SB verse numbering**: 5 duplicate verse reference tags in the HTML. Fixed by using the TEXT N header as authoritative.
|
||||
|
||||
3. **RTF routes gated**: `/rtfs` only accessible when `ALLOW_RTF=true`.
|
||||
|
||||
4. **`scripts/` excluded from tsconfig**: Prevents TypeScript build errors from seed scripts.
|
||||
|
||||
5. **CSS zoom for font scaling**: `zoom` property works in Chrome/Safari/Edge but may not work in older Firefox (< 126). A `transform: scale()` fallback is provided via `@supports not (zoom: 1.1)`.
|
||||
|
||||
6. **Password protection requires ACCESS_KEY at runtime**: The `PasswordGate` component checks the server at runtime via `/api/verify-access`. The env var must be available when the server is running, not just at build time.
|
||||
Reference in New Issue
Block a user