The architecture of News-cookie

Project Overview: News-Cookie

News-Cookie is a web application that summarizes news articles based on user queries. The project was conceived with the goal of providing higher-quality, more reliable data than what standard, out-of-the-box AI responses typically offer.

Architecture of News-cookie

 

Let's see the details

 

Technology Stack: NEXT.JS (Frontend + Backend)

I chose Next.js as the web development framework for this project. The reasons are as follows:

• Cost-Effectiveness and Low Maintenance for Solo Development
  • Since I am running this project as a solo developer, building and managing a separate backend server is highly inefficient in terms of cost, maintenance, and management effort. (e.g., Managing a stack like AWS, Docker, Nginx, and Spring requires separate maintenance and incurs high operational costs.)
• Unified Development Environment
  • Next.js allows me to handle both the frontend and backend within a single framework, streamlining the entire development process.
• Optimized Deployment with Vercel
  • It is perfectly optimized for deployment via Vercel (the creators of Next.js). This setup allows for seamless CI/CD, where any merge into the GitHub main branch automatically triggers a production deployment.

 

SEO & Tracking

• Search Engine Optimization (SEO)
  • I have decided to implement rigorous SEO for both Google and Naver. Since discovering the service is a primary goal, this is a natural requirement. While I am familiar with the unique name "News-Cookie," potential users are not. To ensure the application ranks for universally recognized keywords like "cookie" or "news," a proper SEO setup is absolutely essential from day one.
• Vercel Analytics
  • Tracking and analyzing user traffic is vital. However, introducing a complex logging layer at this early stage poses a significant structural risk for an unvalidated product. For initial traffic analysis, I highly recommend Vercel Analytics. It provides a lightweight, frictionless way to monitor daily visitor numbers and active usage without the overhead of a dedicated logging system, which can be integrated later as the user base grows.

AI-Agent & Core Infrastructure

• Why LangChain?
  • To Explore and Learn: Simply put, it is incredibly popular right now, and I wanted to get hands-on experience with it. In fact, this entire project was conceived as a vehicle to explore the capabilities of LangChain.
  • Delivering Tailored User Experiences: LangChain acts as a robust "harness" or framework wrapped around an AI model, allowing developers to define custom behaviors and guardrails. I designed this service specifically for users who want to consume news rapidly and efficiently. It is also a tool I personally needed to stay updated on the latest AI advancements.
• Tavily AI
  • This API serves as the foundational search harness for the AI. It is specifically engineered to crawl web content in an LLM-friendly format, filtering out noise to deliver only the core substance. Its dedicated news search mode is highly optimized for capturing the latest breaking international news. Furthermore, its free tier of 1,000 queries per month offers an ideal, cost-effective runway for early-stage development.

Authentication & Database Architecture

• Supabase & Google OAuth
  • Because the service operates on a token-based model—where users consume an AI currency called "Cookies"—a secure membership system is required to track and manage individual token balances. To provide a seamless user experience, I will implement Google OAuth ("Sign in with Google") for user authentication, managed through Supabase.
• RDBMS (Relational Database) for Transactional Integrity
  • I chose a Relational Database Management System (RDBMS) specifically to guarantee ACID complianceand transactional integrity. Since the platform involves a payment and token ecosystem, financial actions—such as charging token packages or deducting tokens per query—must be entirely fault-tolerant. If any failure occurs mid-process, the system must completely roll back to its pre-transaction state. Managing these token transactions securely within database transaction blocks is non-negotiable.

 

Database Design & Architecture Optimization

1. Payment History Table: Ensuring Idempotency with UNIQUE Constraints

• In the payment_history table, we face an issue where the payment gateway (Polar service) occasionally sends duplicate webhook notifications for a single successful transaction.
• Left unchecked, this could result in a single user paying once but receiving double the token amount—leading to operational revenue loss. To prevent this duplicate processing (achieve idempotency), the polar_order_id is now enforced with a UNIQUE constraint to reject duplicate incoming calls.

2. Token Usage Logs Table: Optimizing Performance with Bigint over UUID

• As the number of active users grows, the volume of records in the token_usage_logs table will scale rapidly. If a random UUID is used as the Primary Key (PK) here, the database's internal index tree becomes heavily fragmented upon every insertion. To preserve index capacity and query velocity, we explicitly use a sequentially incremented bigint instead of a UUID.

A Deep Dive into Database Engine Mechanics (MySQL vs. PostgreSQL)

• MySQL (Clustered Index Structure): MySQL utilizes a clustered index architecture, meaning the physical order of data on the disk strictly mirrors the order of the Primary Key. Inserting random values like a UUID forces the engine to push its way into random physical disk sectors. This triggers frequent Page Splits, causing severe write amplification and degraded performance. Therefore, an sequential Auto_increment is heavily favored in MySQL.
• PostgreSQL (Heap Table Structure): PostgreSQL uses a heap table architecture, where creating a PK does not physically sort the actual records on the disk. Records are written wherever free space is available, and the PK simply maps to a B-tree index that points to these physical addresses.

While PostgreSQL doesn't suffer from the physical disk reshuffling overhead of MySQL when using UUIDs, the PK Index Tree (B-tree) must still be updated for every single insertion. A random UUID causes random index node splitting, leading to bloated index sizes and poor memory efficiency. In contrast, a bigint sequentially appends new nodes to the far right end of the B-tree, ensuring highly stable and efficient memory utilization. Furthermore, a UUID demands 16 bytes (or up to 36 bytes if stored as text), whereas a bigint requires only 8 bytes. At millions of rows, this difference in index footprint expands exponentially, significantly affecting overall DB query performance.

3. News Snapchat Table: Utilizing JSONB for Semi-Structured Caching & Logs

• The news_snapshots table differs from standard tables as it relies on the JSONB data type. While our transactional tables rely on strict RDBMS architectures for ACID compliance (grouping payment and token usage events), the news_snapshots table serves a completely different purpose. It functions as a logging system and a temporary caching layer to store raw news data scraped in real-time by the AI Agent via external APIs (Tavily).
• The core reasons for persisting user query strings alongside their corresponding AI-scraped news results in this format are as follows:

1. Schema Flexibility & Extensibility: News data formats, structures, and metadata fields vary significantly across different media outlets. Forcing this highly variable data into rigid RDBMS columns makes schema modifications increasingly fragile. By leveraging PostgreSQL's high-performance, binary-optimized NoSQL data type (JSONB), we can ingest changing raw data in its entirety without complex transformation logic.
2. Cost Reduction through Strategic Caching (Future Scope): When users input queries with similar semantic meanings (e.g., "Latest ChatGPT news" vs. "What's new with OpenAI's ChatGPT lately?"), it is highly inefficient to repeatedly trigger expensive external search APIs and LLM invocations. Although caching will not be activated in the initial release to avoid premature optimization before validating product-market fit, this structural layout ensures we can seamlessly implement vector similarity searches over historical snapshots in the future. This will drastically reduce system latency and minimize AI API operational overhead.
3. Prompt Debugging & Hallucination Mitigation: The snapshots act as a crucial analysis log. By tracing exactly what raw data the Agent collected and how the LLM processed and summarized it, we can continuously monitor prompt accuracy, prevent model hallucination, and feed these insights back into our system refinement loop.

Conclusion & Next Steps

This concludes the deep dive into the system architecture for News-Cookie. Laying out a sound, logically justified architecture early on is an indispensable phase of any serious product cycle.

In the upcoming documentation, I plan to map out the application's core application logic and the end-to-end user query flow.