Family Finance AI-Assistant Plan

Build plan for a family financial management app combining AI Agent (LangGraph) + HITL + Tools
#aiagent#finance
single

🎯 Project Goals

🎯 Primary Goals (must-have)

  1. Solve real family financial problems
    • Wife & husband can manage 100% of monthly cash flow in/out without spending > 5 minutes/day.
    • Answer questions: "How much left this month?", "How long until debt is paid?", "Is Net Worth increasing or decreasing?", "Manage assets (Gold, Land, Stocks...)".
    • Create shared financial recording habits for both spouses (shared ownership, transparency).
  2. Master modern AI Engineering stack
    • Master LangGraph (state machine, interrupt/resume, checkpointer, multi-agent).
    • Master MCP (build server + expose tools for Claude Desktop).
    • Master HITL pattern β€” one of the most important trends of 2025-2026.
    • Master RAG with vector DB at production scale.
    • Master BE APIs and Database
    • Respect the process, deep thinking & analysis
  3. Build portfolio/CV "for money"
    • Have a real product running with real users (family), real data β€” no longer a demo.
    • Can show live demo in interviews, or invite interviewer to try.
    • Source code open on GitHub (or at least public repo with README + screenshots).

🌱 Secondary Goals (nice-to-have)

  1. Build personal brand on LinkedIn β€” share "build in public" journey to connect with VN AI engineer community (see strategy in final section).
  2. Document the process β€” 1-2 blog posts/ADRs per phase, for future reference or sharing.
  3. Potential productization β€” if app runs well, can open to friends/relatives β†’ validate potential SaaS idea.

πŸ“ Success Metrics (how to know if successful?)

MetricTarget after 3 months
Wife + husband use app regularlyβ‰₯ 5 days/week each person
Transactions recordedβ‰₯ 90% of actual transactions logged
Net Worth trackingContinuous snapshots β‰₯ 60 days
HITL PDF accuracyβ‰₯ 95% items correct on first try (no edits)
LinkedIn postsβ‰₯ 8 posts (1 post/week average)
GitHub starsβ‰₯ 20 (if public) β€” just a secondary indicator
Project mentioned in interviewsβ‰₯ 3 interviews

🚫 Non-goals (clearly NOT doing)

  • NOT building multi-tenant SaaS from the start β€” keep it simple, single household.
  • NOT integrating with banks/Plaid/Banking API β€” too complex & VN doesn't have good infra yet.
  • NOT doing mobile native (React Native/Flutter) β€” PWA is sufficient, saves 50% effort.
  • NOT over-engineering: no K8s, no microservices, no event-sourcing.
  • NOT trying to replace Money Lover / MISA β€” focus on AI + HITL as differentiator.

πŸ’‘ Feature Summary (Feature Matrix)

ModuleDetailed FeaturesAI & Stack RoleBusiness/Finance Value
1. IncomeRecord total income (Salary, bonus, freelance) of wife and husband.NestJS/SQL: Manage income history by month/quarter.Understand total available financial resources.
2. ExpensesCategorize expenses: Living, children, ceremonies.LangGraph: Auto-categorize based on keywords or history.Control "lifestyle inflation", cut excess costs.
3. AssetsGold, Real Estate, stocks, savings, crypto.MCP Tool: Fetch real-time gold/exchange/stock prices.Track current_value and Net Worth growth.
4. LiabilitiesBank loans, credit, installment payments.NestJS: Calculate amortization, support floating rate & prepayment penalty.Control debt pressure, optimize interest.
5. Budget & Cash FlowSet budget by category, forecast cash flow 3-6 months.NestJS: Rule engine + alert when over budget. LangGraph: Forecast patterns.Plan expenses proactively, avoid shortfall.
6. Emergency FundTrack 3-6 month emergency fund.NestJS: Auto-calculate target based on avg monthly expense.Personal finance principle #1 β€” safety before investing.
7. PDF Statement Parsing & HITLUpload bank/Visa PDF statement, parse text β†’ transaction, user reviews before saving.pdfjs-dist: Parse text from PDF. Regex/Mapping: Standardize per bank (VIB, Techcombank, VPBank...). LangGraph (Interrupt): Pause flow to wait for confirm.Ensure ~100% data accuracy, no AI token waste, processing < 1 sec.
8. AI Chat (Multi-view)Q&A in natural language, render table/chart/editable.LangGraph: Tool-calling + structured output. FE: Dynamic renderer.Self-serve "ask AI" instead of manually querying/filtering.
9. AI StrategySuggest debt payoff, restructure investment portfolio.LangGraph (Reasoning): Analyze above items to give "CFO" advice.Optimize cash flow, reduce interest, achieve financial freedom faster.
10. Net Worth DashboardView total Assets - Liabilities over time.NestJS: Daily snapshot. FE: Time-series chart.Measure real "financial health" of the family.

πŸ“± "Non-AI" Product Slice (Manual-first)

Compare with roadmap notes (income/outflow, insight + chart, budget, manual entry, CRUD on UI). Everything below does NOT depend on LangGraph/LLM; ship in Phase 1 (can start right after Phase 0).

FE RequestMeaningCovered in Spec
View Income & ExpensesAggregate by month/time range, filter by categoryPhase 1.2–1.3 (list + KPI + charts)
Insight with chartsKPI, pie/bar/line, rule-based insight (no LLM)Phase 1.3
View / manage budgetTarget spending by category, progress, warning when exceeded (initially UI + API, push after upgrade)Phase 1.3 (progress + basic budget); Phase 3.3 expand forecast & alert
Add manual transactionForm for one-off & recurring income/expensePhase 1.2
Full CRUD on UICreate / Read / Update / Delete for transaction, category (within household), budget linePhase 1.2 + 1.3

Backend for this slice: only REST (JSON) + MySQL; MCP not required for above screens but should be on same server as API so agent/Claude Desktop can reuse domain logic (see architecture section).


πŸ—οΈ System Architecture Diagram

REST + MCP on Same Backend (NestJS)

  • Next.js calls REST/JSON for entire manual flow (auth, household, transactions, budgets, aggregates for chart/insight).
  • MCP runs in same apps/api (NestJS) β€” e.g., MCP module mounts additional transport (stdio for Claude Desktop, or HTTP/SSE if needed) but reuses same service layer / Drizzle with REST, same auth context (API key or household token), same MySQL. Avoid duplicate business logic between "REST handlers" and "MCP tools".
  • LangGraph / agent calls MCP tools (or directly calls shared services) to read/write data like REST β€” single source of truth for domain.

πŸ”„ PDF Statement Processing (HITL PDF Flow)

Edge Cases to Handle

  • Timeout: User doesn't confirm after 24h β†’ save draft in pending_review state, can resume later.
  • Partial confirmation: Statement has many rows, user edits only a few β†’ diff & merge on resume.
  • Unsupported bank format: Unknown PDF format β†’ show warning, allow manual mapping or retry.
  • Duplicate detection: Auto-detect duplicates with existing transactions in DB.
  • Checkpointer cleanup: Cron job deletes checkpoints > 7 days unconfirmed to avoid DB bloat.

πŸ› οΈ Technical Stack for Phase 2 β€” PDF Parsing

ComponentTechnologyNew Role
PDF Parserpdfjs-distExtract text from bank/Visa PDF instead of Claude Vision.
Logic LayerJS/TS Regex & MappingConvert raw PDF text to correct Transaction schema.
AI AgentLangGraphFocus 100% on Reasoning and Advising.
Vector DBweaviate vectorStore "memory" about spending habits so Agent understands context better.

πŸ’¬ AI Chat β€” Multi-view Rendering

Chat doesn't just return text, but renders dynamic view based on tool_call result + structured output.

View typeWhen to useExample question
TextExplanatory answer, advice."What should I cut back on this month?"
TableList transactions, assets."List expenses > 500k in October."
Editable TableNeed user confirm/edit (HITL)."Parse PDF statement", "AI suggests new budget".
Chart (line)Time-series: Net Worth, cash flow."Draw expense chart for last 6 months."
Chart (pie)Breakdown by category."Expense ratio this month?"
KPI CardHighlight important metric."Current Net Worth?", "Emergency fund remaining?"
Action CardSuggestion with CTA button (Confirm/Dismiss)."Suggest paying extra 5tr debt this month β€” Agree?"

Implementation (FE)

  • LangGraph returns { type: "chart" | "table" | ..., payload: {...} } instead of plain text.
  • FE has <MessageRenderer> switch-case to render correct component.
  • Editable table uses AntD Table with editable cell pattern (similar to ListeningPartyDetailForm pattern already in codebase).

πŸ“ˆ Financial Formulas to Implement

Fixed-rate interest

M=Pr(1+r)n(1+r)nβˆ’1M = P \frac{r(1+r)^n}{(1+r)^n - 1}

  • MM: Monthly payment amount.
  • PP: Initial principal balance.
  • rr: Monthly interest rate (Annual rate / 12).
  • nn: Total number of months.

Floating interest rate

In Vietnam, banks typically apply: fixed N months first β†’ floating = base rate + margin (3-4%). Service needs to model:

floatingRate(month) = month <= fixedMonths
  ? fixedRate
  : baseRate(month) + margin

Prepayment penalty

  • Usually 1-3% of prepayment amount in first 1-5 years.
  • ⚠️ If not calculated, AI will advise "pay early" but actual penalty fees may eat up the benefits.

Reducing balance vs flat interest

  • Reducing: Interest calculated on remaining balance β†’ formula MM above.
  • Flat: Interest calculated on original principal β†’ total interest much higher, easy to confuse.

🧠 Vector DB (Weaviate) β€” Use Case

Use Weaviate vector for personal finance Knowledge Base:

  • Embed content of articles/books: Cashflow Quadrant (Kiyosaki), 50/30/20 rule, Pay Yourself First, Snowball vs Avalanche debt method, Emergency Fund 3-6 months, Index Fund Investing…
  • When user asks "Should I pay off debt first or invest?", Chat Node will retrieve top-k related chunks β†’ put in context for AI to give evidence-based advice, not just talk.
  • Schema: knowledge_chunks(id, source, title, content, embedding vector(1536)).

πŸ”’ Security & Privacy

Family finance app β†’ extremely sensitive. Required:

  • Auth: Google OAuth (web/mobile).
  • Multi-user: Role OWNER (wife/husband have same rights) + VIEWER (grandparents can view Net Worth but not edit).
  • Encryption at rest: Balances, receipt images encrypted.
  • PII masking: PDF parsed locally β€” don't send to external AI, improves security.
  • Audit log: Table audit_log(user_id, action, entity, before, after, at) for every transaction change.

πŸš€ Deployment β€” VPS

Deliberately self-host to control financial data (don't rely on cloud provider).

Infrastructure Stack

LayerToolNote
VPSHetzner / Contabo / VN VPS (~5-10 USD/month)2 CPU / 4GB RAM sufficient for family app.
ContainerDocker ComposeSingle file, simpler than K8s.
Reverse ProxyCaddy (auto HTTPS) or Nginx + CertbotCaddy simpler, Let's Encrypt built-in.
DBMySQL + Weaviate VectorRun in container, mount volume.
Backupmysqldump daily + rsync to Backblaze B2Budget ~1 USD/month for 100GB.
MonitoringUptime Kuma + Grafana (optional)Self-host on VPS too.
CI/CDGitHub Actions β†’ SSH deploy β†’ docker compose pull && up -dSimple, sufficient.
Secrets.env on VPS + docker secrets (optional)Don't commit to repo.
Logdocker logs + Loki (optional)Start simple with docker logs.

Expected docker-compose.yml

services:
  web: # Next.js
  api: # NestJS + REST + MCP (in-process / same image)
  agent: # LangGraph (Python or Node)
  db: # MySQL + Weaviate Vector
  caddy: # Reverse proxy + TLS
  backup: # cron container mysqldump

πŸ—οΈ Initial Project Setup (Monorepo)

Use Turborepo + pnpm workspaces for JS/TS services. LangGraph Agent can be separate (Python) or use LangGraphJS to share monorepo.

Proposed Structure

family-finance/
β”œβ”€β”€ apps/
β”‚   β”œβ”€β”€ web/              # Next.js  App Router
β”‚   β”œβ”€β”€ api/              # NestJS  + Drizzle
β”‚   └── agent/            # LangGraphJS (or Python - separate repo)
β”œβ”€β”€ packages/
β”‚   β”œβ”€β”€ ui/               # Shared React components (AntD wrappers)
β”‚   β”œβ”€β”€ types/            # Shared TS types (DTO, zod schemas)
β”‚   β”œβ”€β”€ db/               # Drizzle schema + migrations
β”‚   └── config/           # ESLint, tsconfig, tailwind presets
β”œβ”€β”€ infra/
β”‚   β”œβ”€β”€ docker-compose.yml
β”‚   β”œβ”€β”€ Caddyfile
β”‚   └── backup.sh
β”œβ”€β”€ docs/
β”‚   └── adr/              # Architecture Decision Records
β”œβ”€β”€ .github/workflows/    # CI/CD
β”œβ”€β”€ turbo.json
β”œβ”€β”€ pnpm-workspace.yaml
└── package.json

Initial Setup Checklist

  • pnpm init + install Turborepo, create pnpm-workspace.yaml.
  • Create apps/web with create-next-app (TS, App Router, Tailwind).
  • Create apps/api with nest new (with --skip-install, run in workspace).
  • Create packages/db with Drizzle, connect to local MySQL (Docker).
  • Create packages/types export DTO/zod schemas used by FE-BE.
  • Setup ESLint + Prettier + Husky + lint-staged (in packages/config).
  • Setup .env.example for each app + dotenv-cli load env.
  • Write infra/docker-compose.dev.yml (MySQL + Weaviate Vector + Adminer).
  • Setup GitHub repo, branch protection, CI runs turbo lint test build.
  • Write README.md with pnpm dev to run everything with one command.
  • First ADR: reason for monorepo, reason for LangGraphJS vs Python.

πŸ—ΊοΈ Roadmap 3 Phases (with Checklist)

Principle: Each phase must be shippable, usable for real. Don't build then use. Each task has checkbox [ ] to track progress β€” mark [x] when done.


πŸ”΅ PHASE 0 β€” Project Setup (Week 1)

Goal: Have a runnable local monorepo + deployable to VPS with "Hello World".

  • Init monorepo with above structure (checklist in section above).
  • Setup docker-compose.dev.yml: MySQL + Weaviate Vector + Adminer.
  • apps/web renders simple landing page, calls /api/health from apps/api.
  • apps/api has /health endpoint + connects to MySQL successfully.
  • Setup Drizzle, migrate first user table.
  • Auth magic link (use next-auth or self-built).
  • Deploy to VPS: Caddy + Docker Compose + domain + HTTPS.
  • CI/CD: push main β†’ auto deploy VPS.
  • Write CONTRIBUTING.md + conventional commits.

Definition of Done (DoD): Wife opens https://finance.yourdomain.com, logs in successfully, sees "Welcome".


🟒 PHASE 1 β€” MVP Manual + Visualization (Weeks 2-4)

Goal: Actually usable daily β€” manual income/expense entry, visual expense tracking. No AI needed yet.

1.0 REST API (supporting non-AI FE)

OpenAPI/Swagger should generate from NestJS DTOs; below is proposed contract (can group under /api/v1/...).

GroupMethod & path (suggestion)Purpose
HealthGET /healthPhase 0; probe deploy
Auth / sessionGET /auth/session, POST /auth/logout
HouseholdGET /households/current, PATCH /households/:idDefault single-household
CategoriesGET /categories, POST /categories, PATCH /categories/:id, DELETE /categories/:idCRUD category (household-scoped)
IncomeGET /incomes, POST /incomes, GET /incomes/:id, PATCH /incomes/:id, DELETE /incomes/:idRecurring + one-off
Expense / transactionGET /transactions, POST /transactions, GET /transactions/:id, PATCH /transactions/:id, DELETE /transactions/:idQuery: from, to, categoryId, type, minAmount, maxAmount, page
Aggregates (charts / KPI)GET /reports/summary?month=YYYY-MM, GET /reports/by-category?month=, GET /reports/trend?months=6, GET /reports/calendar-heatmap?month=Insight + chart without AI
BudgetGET /budgets?month=, PUT /budgets (bulk upsert by category), GET /budgets/:categoryId/progress?month=View & edit budget + progress
Export (optional Phase 1)GET /transactions/export?format=csvLight backup

Corresponding MCP tools (same server, Phase 3 advanced β€” can stub early): mirror above read/query (get_transactions, get_monthly_summary, get_budget_status, …) so Claude Desktop / agent doesn't need parallel HTTP.

1.1 Data layer

  • Schema: income, expense, category, user, household.
  • Seed 20 default categories (Food, Living, Children, Ceremonies, Transport...).
  • Migration + seed script runs in CI.

1.2 Basic CRUD + API

  • NestJS REST + OpenAPI per contract in 1.0; service layer shared with MCP later.
  • Income entry form (monthly recurring + one-off).
  • Expense entry form (date, amount, category, note).
  • Transaction list with filters (month, category, amount range).
  • Edit/Delete transaction.
  • Quick-add floating button (mobile-first).

1.3 Visualization Dashboard (Phase 1 highlight)

  • KPI Cards: Total income, total expense, monthly balance, % vs last month.
  • Pie chart: Expense breakdown by category (current month).
  • Bar chart: Income vs Expense last 6 months (side-by-side comparison).
  • Line chart: Daily expense trend within month (cumulative).
  • Heatmap calendar: Which days spent most β€” similar to GitHub contributions.
  • Top 5 expenses: Card listing 5 largest transactions of month.
  • Budget progress bar: For each category, set budget + progress bar changes color when exceeded.
  • Auto-generate insight text (no AI, just rule-based): "This month you spent 15% more than last month."

1.4 UX polish

  • Mobile-first responsive (wife mainly uses phone).
  • Dark mode.
  • PWA manifest (add to home screen).
  • Full Vietnamese localization.
  • Empty states with guidance (no data β†’ "Add your first transaction").

DoD Phase 1: After 2 weeks of real use, wife & husband can answer "How much did we spend this month, where was the most?" just by opening the app.


🟑 PHASE 2 β€” AI Layer (Weeks 5-8)

Goal: Add AI to app β€” users can import all past data, upload bank statements monthly, and ask AI insight in chat window.

Architecture decision (ADR-002):

  • Runtime: LangGraphJS (TypeScript) β€” keep pure JS/TS monorepo, no separate Python service.
  • Chat protocol: FE β†’ LangGraph Agent direct (HTTP + SSE streaming) β€” don't proxy through NestJS to reduce latency and simplify flow.
  • State / History: All thread state and conversation history stored via LangGraph Postgres Checkpointer β€” no separate tables needed.
  • NestJS is still the source of truth for domain data (transactions, budgets…) β€” Agent calls NestJS REST or MCP tools to read/write, doesn't access DB directly.
FE (Next.js)
  │── REST ──────────────────► NestJS API ──► MySQL
  β”‚                                 β–²
  └── SSE / HTTP ──► LangGraph Agent
                          │── tool_call ──► MCP Tools (in NestJS)
                          │── checkpointer β–Ί MySQL (thread state)
                          └── embed/search β–Ί Weaviate Vector

2.1 LangGraph Foundation

Setup entire agent infrastructure before building each flow.

  • Initialize apps/agent with LangGraphJS (@langchain/langgraph).
  • Configure MySQL Checkpointer (MySQLSaver) β€” use same DB instance as NestJS, separate schema langgraph_*.
  • Expose HTTP server in apps/agent (Fastify or Express):
    • POST /threads β€” create new thread, return thread_id.
    • POST /threads/:id/runs β€” send message, stream response via SSE.
    • POST /threads/:id/resume β€” resume after HITL interrupt.
  • Auth: each request includes household_id (JWT from NextAuth) β€” agent validates before executing.
  • Add apps/agent to docker-compose.yml + CI pipeline.
  • ADR-002: record decision on LangGraphJS vs Python, FE-to-Agent direct.

2.2 Task 1 β€” Import Old Data from Excel

Users can import entire financial history from Excel file (Google Sheets export, Money Lover export, or self-created file).

Flow:

User upload Excel β†’ FE parse preview β†’ Editable table mapping β†’ Confirm β†’ Bulk insert DB

FE (without Agent β€” pure REST):

  • "Import from Excel" screen with dropzone upload .xlsx / .csv.
  • Use xlsx (SheetJS) to parse file on client β€” don't send raw file to server.
  • Show preview of first 10 rows, auto-detect columns: date, amount, description, type (income/expense).
  • Column mapping UI: if column names don't match schema β†’ user drag-drop to manually map.
  • Validate each row: missing date / amount β†’ highlight error, allow inline edit.
  • Detect and show duplicate warning (same date + amount + description as existing data).
  • "Confirm import X transactions" button β†’ call POST /transactions/bulk.

Backend (NestJS):

  • POST /transactions/bulk β€” receive transaction array, upsert (idempotent based on hash date+amount+description).
  • Return summary: { inserted: N, skipped: M, errors: [] }.
  • After import completes β†’ trigger embed descriptions into Weaviate vector (for auto-categorize in 2.5).

Edge cases:

  • File > 5MB or > 5,000 rows β†’ chunked import, progress bar.
  • type column unclear (no income/expense) β†’ user selects default for entire file.
  • Non-standard date format (dd/mm/yyyy vs mm/dd/yyyy) β†’ auto-detect + let user confirm.

2.3 Task 2 β€” Upload PDF Statement (TPBank)

User uploads monthly PDF statement from banking app β†’ AI parse β†’ HITL review β†’ save to DB.

Flow (HITL):

Upload PDF β†’ pdfjs-dist extract text β†’ TPBank parser β†’ LangGraph Interrupt
            β†’ FE editable table β†’ User confirm/edit β†’ Resume β†’ Bulk insert + embed

Parser β€” TPBank (priority #1):

  • Research TPBank PDF statement format (export from TPBank Mobile / TPBank iBank).
  • Write apps/agent/src/parsers/tpbank.parser.ts:
    • Regex/pattern match fields: ngΓ y GD (transaction date), mΓ΄ tαΊ£ (description), sα»‘ tiền (amount), loαΊ‘i GD (Tx type: Dr/Cr), sα»‘ dΖ° (balance).
    • Map to Transaction schema: { date, amount, description, type: 'income'|'expense', rawText }.
  • Unit test parser with at least 3 real PDF statement files (mask sensitive numbers).
  • Fallback: if parse < 50% valid rows β†’ error "Format not recognized, please try again or enter manually".

LangGraph Node β€” pdf_import_graph:

  • Node parse_pdf: receive file_buffer β†’ run pdfjs-dist β†’ run TPBank parser β†’ return transactions[].
  • Node interrupt_review: interrupt() β€” return { transactions, thread_id } to FE.
  • Node save_transactions: receive approved data β†’ call POST /transactions/bulk β†’ embed to Weaviate vector.

FE β€” HITL Review UI:

  • Upload dropzone (PDF, max 20MB).
  • Show editable table (AntD) after parsing β€” each row editable: date, amount, description, category, type.
  • Status badge per row: βœ… OK / ⚠️ Needs review / πŸ”΄ Missing data.
  • Bulk actions: "Select all", "Delete error rows", "Set category for multiple rows at once".
  • "Confirm X transactions" button β†’ POST /threads/:id/resume.

Edge cases:

  • Timeout 24h: user doesn't confirm β†’ state pending_review in checkpointer, can resume later.
  • Partial confirm: user unchecks some rows β†’ only import checked rows.
  • PDF scan (image, no text): pdfjs-dist returns blank β†’ clear message "This file is a scanned image, cannot auto-parse".
  • Duplicate: auto-detect vs DB β†’ show warning, default uncheck duplicate rows.
  • Checkpointer cleanup: cron job deletes pending_review threads > 7 days.

Later expansion (Phase 3+): Add parsers for VIB, Techcombank, VPBank, Vietcombank, BIDV per sprint.


2.4 Task 3 β€” MCP Tools + AI Chat (Insight in Chat Window)

User asks financial insight in natural language in chat β€” AI queries real data and returns result as text / table / chart.

MCP Tools (registered in apps/api β€” NestJS):

ToolInputOutputDescription
get_transactions{ from, to, categoryId?, type?, limit }Transaction[]Get transaction list with filter
get_monthly_summary{ month: 'YYYY-MM' }{ income, expense, balance, topCategories[] }Monthly income/expense summary
get_budget_status{ month: 'YYYY-MM' }BudgetProgress[]Budget progress per category
get_trend{ months: number }MonthlyTrend[]Income/expense trend for last N months
get_net_worth_snapshotβ€”{ assets, liabilities, netWorth }Current Net Worth (if Phase 3 data exists)
create_chart_data{ type, data[] }ChartPayloadFormat data for FE renderer
  • Register MCP server in apps/api β€” reuse service layer (no duplicate logic).
  • LangGraph Chat Node uses tool-calling to call MCP tools β€” doesn't access DB directly.
  • All tool calls scoped by household_id from JWT β€” no data leak between households.

LangGraph Node β€” chat_graph:

  • Node chat: receive user message β†’ LLM with tool definitions β†’ tool_call if data needed.
  • Node tool_executor: execute MCP tool call β†’ return result to LLM.
  • LLM format output: { type: 'text'|'table'|'chart_line'|'chart_pie'|'kpi_card'|'action_card', payload: {...} }.
  • Streaming: each token stream to FE via SSE β€” type/payload sent after completion.

FE β€” Chat UI:

  • Chat window with SSE streaming β€” render tokens real-time.
  • <MessageRenderer>: switch-case render correct component by type:
    • text β†’ markdown render.
    • table β†’ AntD Table (read-only).
    • chart_line / chart_pie β†’ Recharts component.
    • kpi_card β†’ Card highlight metric.
    • action_card β†’ Card with Confirm / Dismiss button (prepare for HITL Phase 3).
  • Prompt chips (hard-coded, not generated): 6 default suggestions:
    • "How much did I spend this month?"
    • "Where did I spend the most?"
    • "Compare income/expense for last 3 months"
    • "How much budget left this month?"
    • "Largest transaction this week?"
    • "How much did I save this month?"
  • Thread history: reload conversation by thread_id (from MySQL Checkpointer).
  • thread_id tied to household_id β†’ wife/husband see same chat history.

2.5 Auto-Categorize (supporting 2.3 & 2.4)

When parsing PDF or importing Excel, auto-suggest category based on existing transaction history.

  • After each user confirms category (HITL) β†’ embed { description, category_id } into Weaviate vector.
  • Node categorize: receive new description β†’ cosine search Weaviate vector β†’ get top-3 similar β†’ suggest category with highest confidence.
  • FE shows suggested category with badge AI suggested β€” user can accept or override.
  • Override β†’ write to category_feedback(description_embedding, category_id, source: 'user_override') table β†’ use for retrain/fine-tune later.
  • Cold start (no data): use keyword matching fallback (pre-seeded category_keyword table).

2.6 Knowledge Base (Personal Finance)

RAG for Agent to give "evidence-based" advice β€” cite from finance books/articles instead of just talking.

  • Ingest 5-10 documents: Cashflow Quadrant, 50/30/20 rule, Pay Yourself First, Snowball vs Avalanche, Emergency Fund, Index Fund.
  • Chunk (500 tokens, overlap 50) + embed + store knowledge_chunks(id, source, title, content, embedding vector(1536)).
  • Chat Node: if question relates to financial theory β†’ RAG retrieve top-3 chunks β†’ put in context.
  • Citation: response includes [Source: Cashflow Quadrant - Chapter 2] so user knows what AI is based on.

⚠️ Note: This sub-task can be deferred to Phase 3 if Phase 2 timeline is pressured. Core value of Phase 2 is 2.2 + 2.3 + 2.4.


βœ… Definition of Done β€” Phase 2

CriteriaPass condition
Excel ImportUpload 100-row Excel file β†’ preview correct β†’ map columns β†’ confirm β†’ 100 transactions appear correctly in dashboard. Duplicates detected and skipped.
PDF TPBankUpload real TPBank PDF statement β†’ parse β‰₯ 85% transactions correctly β†’ HITL review β†’ save to DB successfully. Scanned PDF (image) shows clear error message.
Auto-categorizeAfter importing 50+ transactions and user confirms category β†’ new transactions of same type suggested correct category β‰₯ 75% of time.
AI ChatAsk "What did I spend the most on this month?" β†’ returns correct pie chart. Ask "Compare income/expense 3 months" β†’ returns bar chart. Ask off-topic β†’ gracefully declines.
StreamingChat response streams real-time, no blank > 3 seconds.
Thread historyClose tab, reopen β†’ chat history still exists. Wife and husband see same thread.
Auth / isolationCannot call agent with thread_id from other household.
HITL timeoutThread pending_review > 24h still resumable. Thread > 7 days gets cron cleanup.

End-of-phase smoke test: Wife performs herself (without husband support) β€” import TPBank statement for last month + ask "Where did we spend the most this month?" β†’ receives correct answer with chart.


πŸ”΄ PHASE 3 β€” Advanced Finance (Weeks 8-11)

Goal: App becomes real "Family CFO".

3.1 Assets & Liabilities

  • Schema: asset, liability, asset_valuation (value snapshot).
  • CRUD Forms for Assets (Gold, Real Estate, Stocks, Savings).
  • CRUD Forms for Liabilities (Bank Loan, Credit).
  • Amortization schedule UI for each loan.
  • Support floating interest + prepayment penalty.

3.2 MCP Server (in Same NestJS as REST)

  • Register MCP in apps/api: shared services with REST (no separate logic fork per tool).
  • Tool get_gold_price(type) β€” scrape/API SJC.
  • Tool get_exchange_rate(from, to).
  • Tool get_stock_price(ticker) β€” SSI/VNDirect API.
  • Tool calculate_loan_schedule(...).
  • Tool get_transactions(filter) β€” same filter as GET /transactions.
  • Read-only tools for insight: get_monthly_summary, get_budget_status (match REST in 1.0).
  • Expose MCP (stdio and/or HTTP) to connect from Claude Desktop (great demo for CV).

3.3 Budget & Cash Flow

  • UI to set budget by category, by month.
  • Alert (notification/email) when 80% budget exceeded.
  • Cash flow forecast 3-6 months (based on recurring income + avg expense + debt payment schedule).

3.4 Emergency Fund

  • UI tracker for emergency fund.
  • Auto-calculate target = avg_monthly_expense * 6.
  • Progress bar + alert when withdrawing from fund.

3.5 Net Worth Dashboard

  • Cron daily snapshot assets_total - liabilities_total.
  • Time-series chart Net Worth 12 months.
  • Breakdown: % Gold / Real Estate / Cash / Debt.

3.6 Multi-agent Strategy

  • Node Mediator (based on real data + knowledge base to conclude).
  • UI "Ask the CFO" β€” user asks big question, AI debates then concludes.

DoD Phase 3: Ask "Should I pay extra on debt or save for gold?" β†’ Agent debates, gives specific numbers, concludes with action plan.


πŸ“Š Progress Tracking Template

Use GitHub Projects / Linear / Notion β€” or just checkboxes in this file are enough for 2-person team.

PhaseStart dateTarget endActual endStatus
Phase 0 β€” Setup⬜ Not started
Phase 1 β€” MVP + Visualization⬜ Not started
Phase 2 β€” AI Layer⬜ Not started
Phase 3 β€” Advanced Finance⬜ Not started

Weekly ritual: Every Sunday, wife & husband review app together for 15 minutes β†’ note:

  • Which feature is most used?
  • Which pain point is blocking? β†’ prioritize fix next week.
  • Does data match actual wallet?

🌐 Strategy: Build in Public on LinkedIn

Principle: "Build in public" > "Launch & pray". Sharing journey regularly while building will create better signal for recruiter/community than one final launch.

🎯 LinkedIn Goals

  • Build authority in niche AI Engineer (LangGraph + HITL + MCP) β€” this niche is "hot" and not many VN creators yet.
  • Connect with: engineers working on AI Agent, AI startup founders, tech recruiters.
  • Create social proof when interviewing β€” recruiter searches your name β†’ sees clear journey.

πŸ“ Personal Positioning

Don't post generically. Pick one angle and stick to it:

"I'm a frontend/full-stack engineer building AI-assistant for my family β€” sharing everything I learn about LangGraph, HITL, MCP along the way."

This angle hits 3 points:

  • Relatable (everyone has family, everyone wants to manage money).
  • Technical (LangGraph, HITL, MCP are hot keywords for recruiters).
  • Authentic (not a generic tutorial, it's a real journey).

πŸ—“οΈ Content Calendar β€” 1 post/week cadence

Split content by phase, each phase β‰₯ 2-3 posts.

Phase 0 β€” Setup (Week 1-2)

  • Post 1 β€” "Why" (The Story): Why did I build this app? Combine family story + desire to master LangGraph. Hook: "My wife said: 'You code AI all day, when will AI help our family save any money?'. From that question, I decided..."
  • Post 2 β€” "The Stack": Screenshot architecture (can take mermaid diagram from this file). Brief explanation of why LangGraph instead of self-coded orchestration.

Phase 1 β€” MVP + Visualization (Week 3-5)

  • Post 3 β€” "Monorepo setup": Screenshot Turborepo + pnpm structure, lessons learned about splitting packages.
  • Post 4 β€” "The Dashboard": Screenshot dashboard visualization (pie chart, heatmap, KPI cards). Caption: "MVP doesn't need AI. This is version 0 that my wife has been using daily."
  • Post 5 β€” "Rule-based Insight": Share "fake AI" trick with rules β€” why don't need LLM yet but users still feel app is "smart".

Phase 2 β€” AI Layer (Week 6-8)

  • Post 6 β€” "HITL in action" (⭐ flagship post): 30s demo video upload PDF statement β†’ parse < 1s β†’ edit β†’ save. Caption explaining LangGraph interrupt() β€” this post has viral potential.
  • Post 7 β€” "Multi-view Chat": Demo chat asking "What did I spend the most this month?" β†’ returns pie chart. Explain structured output + dynamic renderer pattern.
  • Post 8 β€” "RAG with Weaviate vector": Share how to embed personal finance books + lesson about chunking strategy.

Phase 3 β€” Advanced (Week 9-12)

  • Post 9 β€” "MCP Server": Video demo Claude Desktop querying family DB via MCP. Keyword "MCP" is very hot right now β€” easy viral.
  • Post 10 β€” "Multi-agent debate": Screenshot agent "Wife" vs "Husband" debating whether to pay debt or buy gold. Very interesting & relatable.
  • Post 11 β€” "Net Worth tracking": Chart Net Worth 3 months real family data (blur specific numbers). Very real β†’ very convincing.
  • Post 12 β€” "Lessons learned": Full journey retrospective. Summarize what was learned, mistakes, trade-offs.

🧩 Content Template β€” Standard LinkedIn post

πŸͺ HOOK (1-2 lines): Opening must stop scroll.
   Example: "My wife knows nothing about AI. But last week, she edited
   LangGraph output every day."

πŸ“– STORY (2-3 paragraphs): Context, problem, solution.
   - Real problem.
   - Technical approach (LangGraph interrupt).
   - Result.

πŸ’‘ LESSON (2-3 bullets): What was learned.
   β€’ Insight 1
   β€’ Insight 2
   β€’ Insight 3

πŸ”— CTA: Gentle. Ask open question.
   "Have you tried HITL pattern? Comment your use case."

#️⃣ HASHTAGS: 3-5 tags in right niche.
   #AIEngineering #LangGraph #HumanInTheLoop #BuildInPublic #AIAgent

πŸ“Έ Visual Strategy

LinkedIn rewards image/video over text-only much more:

  • UI screenshots (mask sensitive data) β€” each post should have 1-2 screenshots.
  • Mermaid diagrams in this file β†’ export PNG β†’ post.
  • Short video 15-30s (Loom / QuickTime) for OCR flow, chat flow, MCP demo.
  • Carousel post (PDF slides) for long lessons β€” very high engagement on LinkedIn.
  • Beautiful code snippets (ray.so, carbon.now.sh) for LangGraph snippets.

🎣 Hook Ideas (ready to use, avoid blank when writing)

  • "3 months ago, my wife didn't believe AI could help the family save money. Last week she..."
  • "I tried 5 finance management apps. None kept me > 2 weeks. So I built my own."
  • "interrupt() in LangGraph β€” 1 line of code, changed how I think about AI Agents."
  • "Recruiter asked: 'Do you have any production LangGraph project?'. From that day I built this."
  • "MCP is not just a protocol. It's how Claude Desktop 'talks' to your family database."
thongvmdev_M9VMOt
WRITTEN BY

thongvmdev

Share and grow together