Skip to main content

Report and Ads Amazon

img_864.png

Requirements​

Functional Requirements​

  1. User authentication
  2. Querying and visualization of advertising data
  3. Starting and stopping ad campaigns via Amazon API
  4. Scheduling and executing regular data collection tasks
  5. Storing and processing ad campaign data
  6. Caching frequently requested data for quick access

Non-Functional Requirements​

  1. High system performance
  2. Scalability to handle growing data volumes
  3. Security when working with sensitive data
  4. Reliability and fault tolerance
  5. Monitoring and logging to track system status

Load Estimation​

  • Assume the system serves 1000 active users
  • Each user makes an average of 50 requests per day
  • Data volume generated by each ad campaign: 1 MB/day
  • Number of active ad campaigns: 10,000

Calculations:

  • Total number of requests per day: 1000 * 50 = 50,000
  • Peak load (assuming 20% of requests occur during peak hours): 50,000 * 0.2 / 3600 ≈ 3 requests per second
  • Data volume generated daily: 10,000 * 1 MB = 10 GB
  • Annual data growth: 10 GB * 365 = 3.65 TB

API Design​

  1. Authentication
POST /v1/auth/login
{
"username": "string",
"password": "string"
}
  1. Get Ad Campaign Data
GET /v1/campaigns/{campaignId}/data
  1. Start Ad Campaign
POST /v1/campaigns/{campaignId}/start
  1. Stop Ad Campaign
POST /v1/campaigns/{campaignId}/stop
  1. Get Analytics Data
GET /v1/analytics?startDate=string&endDate=string

Storage Schema​

  1. Users Table

    CREATE TABLE users (
      id INT PRIMARY KEY,
      username VARCHAR(50) UNIQUE NOT NULL,
      password_hash VARCHAR(255) NOT NULL,
      created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
    );

  2. Campaigns Table

    CREATE TABLE campaigns (
      id INT PRIMARY KEY,
      name VARCHAR(100) NOT NULL,
      status ENUM('active', 'paused', 'stopped') NOT NULL,
      start_date DATE,
      end_date DATE
    );

  3. Campaign Data Table

    CREATE TABLE campaign_data (
      id INT PRIMARY KEY,
      campaign_id INT,
      date DATE,
      impressions INT,
      clicks INT,
      spend DECIMAL(10, 2),
      FOREIGN KEY (campaign_id) REFERENCES campaigns(id)
    );

High-level Design​

The system consists of the following components:

  1. User Interface: Where users interact with the system
  2. Auth Service: Handles user authentication
  3. API Service: Processes user requests and interacts with other components
  4. Lambda Script: Executes scheduled tasks and processes data
  5. Event Bridge: Schedules regular tasks
  6. RDS SQL: Stores structured data (user info, campaign data)
  7. S3: Stores large volumes of raw data
  8. ElastiCache: Caches frequently accessed data
  9. CloudWatch: Monitors system performance and logs events
  10. AWS Secret Manager: Securely stores sensitive information
  11. Amazon API: Interacts with Amazon's advertising platform

Detailed Design​

img_864.png

  1. User Flow:

    • User sends a request for data
    • Request passes through Auth service for authentication
    • Authenticated request is processed by the Service component
    • Service component interacts with ElastiCache for fast data retrieval
    • If data is not in cache, it's fetched from RDS SQL
    • Processed data is sent back to the user for visualization

  2. Data Collection Flow:

    • Event Bridge triggers Lambda script on a schedule
    • Lambda script fetches new data from Amazon API
    • New data is processed and stored in RDS SQL
    • Relevant data is also stored in S3 for long-term storage
    • CloudWatch monitors the process and logs events

  3. Campaign Management Flow:

    • User initiates campaign start/stop request
    • Request is processed by the Service component
    • Service component interacts with Amazon API to execute the action
    • Campaign status is updated in RDS SQL
    • CloudWatch logs the action

Evaluation​

  1. Scalability: The system can handle increased load by scaling the Lambda functions and RDS instances.

  2. Performance: ElastiCache improves read performance for frequently accessed data.

  3. Reliability: Use of managed AWS services (Lambda, RDS, S3) ensures high availability and fault tolerance.

  4. Security: AWS Secret Manager securely stores sensitive data. Auth service ensures only authorized access.

  5. Maintainability: Separation of concerns allows for easy updates and maintenance of individual components.

  6. Cost-effectiveness: Use of serverless components (Lambda) and efficient data storage (S3 for raw data, RDS for processed data) optimizes costs.

Distinctive Component​

The integration of AWS services, particularly the use of Lambda for serverless computing and Event Bridge for task scheduling, allows for a highly scalable and cost-effective solution. The system's ability to interact directly with Amazon's advertising API also provides a seamless experience for managing ad campaigns.