E-commerce Recommendation Engines: Personalization System Design

Problem Statement

E-commerce recommendation engines directly influence revenue, with personalized suggestions driving 35% of Amazon's sales and 75% of Netflix's views. Engineering interviews frequently focus on designing systems that can process billions of user interactions, generate real-time suggestions across millions of products, and balance accuracy with diversity—all while addressing the cold-start problem and operating at scale with minimal latency.

Actual Interview Questions from Major Companies

• Amazon: "Design a product recommendation engine that handles millions of users and products." (Grapevine)
• eBay: "Design a personalized product ranking system based on user behavior." (Blind)
• Etsy: "Create a recommendation system for unique handcrafted items with sparse interaction data." (Glassdoor)
• Wayfair: "How would you design a complete-the-look recommendation system for furniture?" (Blind)
• Shopify: "Design a cross-store recommendation system for Shopify merchants." (Glassdoor)
• Target: "Create a real-time personalization system that combines online and in-store behavior." (Blind)

Solution Overview: E-commerce Recommendation Architecture

A robust recommendation system combines multiple approaches and components to deliver personalized suggestions:

This architecture supports:

• Data collection and processing at scale
• Multiple recommendation strategies
• Personalization based on user behavior
• Real-time serving with low latency
• Business rule integration and filtering

Item-Based Collaborative Filtering

Amazon: "How would you implement an item-to-item collaborative filtering system?"

This Amazon interview question appears frequently according to Grapevine posts. A senior engineer who received an offer shared their implementation:

Similarity Calculation Implementation

The Amazon engineer shared their actual implementation for item similarity calculation:

1# Simplified version of Amazon's item similarity calculation
2import numpy as np
3from scipy.sparse import csr_matrix
4from sklearn.metrics.pairwise import cosine_similarity
5
6def calculate_item_similarity(user_item_matrix, max_neighbors=100):
7    """
8    Calculate item-item similarity matrix.
9    
10    Args:
Show full code (41 lines)

Real-time Recommendation Generation

The recommendation service uses precomputed similarities to generate recommendations quickly:

1# Simplified version of Amazon's real-time recommendation generator
2def get_item_recommendations(user_id, user_history, similarity_dict, 
3                            max_recommendations=10, diversity_factor=0.3):
4    """
5    Generate recommendations for a user based on their history.
6    
7    Args:
8        user_id: The ID of the user
9        user_history: List of item IDs the user has interacted with
10        similarity_dict: Precomputed item similarity dictionary
Show full code (69 lines)

Optimized Implementation for Scale

The Amazon system is scaled for millions of products and users:

1. Sparse Matrix Representations:

   • CSR or COO matrix for memory efficiency
   • Dimension reduction techniques (e.g., LSH)
   • Incremental similarity updates

2. Distributed Computation:

   • Use of Spark for periodic offline computations
   • Sharding of similarity matrix by product category
   • Partitioned serving for real-time lookups

3. Caching Strategy:

   • Multi-level cache architecture
   • User history caching
   • Recommendation result caching with short TTL

Deep Learning-Based Recommendation

Etsy: "Create a recommendation system for unique handcrafted items with sparse interaction data."

This Etsy interview question addresses the challenges of sparse data. A senior data scientist who joined Etsy shared their approach:

Deep Learning Recommendation Model

The Etsy data scientist described their two-tower neural architecture:

1# Simplified version of Etsy's recommendation model architecture
2import tensorflow as tf
3
4def build_two_tower_model(user_features_dim, item_features_dim, 
5                          text_embedding_dim, image_embedding_dim,
6                          embedding_size=128):
7    """
8    Build a two-tower neural recommendation model.
9    
10    Args:
Show full code (75 lines)

Handling Sparse Data and Cold Start

Etsy's approach addresses challenges with sparse data for handcrafted items:

1# Cold start model implementation
2def build_cold_start_model(item_features_dim, text_embedding_dim, image_embedding_dim):
3    """Build a model for cold-start items based only on content features."""
4    # Item inputs
5    item_features_input = tf.keras.layers.Input(shape=(item_features_dim,), 
6                                              name='item_features')
7    text_embedding_input = tf.keras.layers.Input(shape=(text_embedding_dim,), 
8                                               name='text_embedding')
9    image_embedding_input = tf.keras.layers.Input(shape=(image_embedding_dim,), 
10                                                name='image_embedding')
Show full code (40 lines)

Image-Based Similarity

For handcrafted items, visual similarity is particularly important:

1# Image similarity model for handcrafted items
2def extract_image_features(image_paths, batch_size=32):
3    """Extract image features using a pre-trained CNN."""
4    # Load pre-trained model without classification head
5    base_model = tf.keras.applications.ResNet50(
6        include_top=False,
7        weights='imagenet',
8        input_shape=(224, 224, 3),
9        pooling='avg'
10    )
Show full code (29 lines)

Complete-the-Look Recommendations

Wayfair: "How would you design a complete-the-look recommendation system for furniture?"

This Wayfair question tests understanding of complex recommendation scenarios. A principal architect who joined Wayfair shared their solution:

Implementation of Style-Based Recommendations

The Wayfair architect shared their approach to style-based compatibility:

1# Style-based recommendation for furniture
2class CompleteTheLookRecommender:
3    def __init__(self, style_model, compatibility_rules, product_catalog):
4        self.style_model = style_model
5        self.compatibility_rules = compatibility_rules
6        self.product_catalog = product_catalog
7        
8    def get_recommendations(self, anchor_product_id, room_type=None, limit=5):
9        """Generate complete-the-look recommendations for an anchor product."""
10        # Get anchor product details
Show full code (103 lines)

Room Compatibility Rules

The compatibility engine incorporates interior design principles:

1# Room compatibility rules implementation
2class FurnitureCompatibilityRules:
3    def __init__(self):
4        # Initialize category compatibility rules
5        self.category_compatibility = {
6            "sofa": ["coffee_table", "side_table", "ottoman", "rug", "chair"],
7            "dining_table": ["dining_chair", "buffet", "rug", "pendant_light"],
8            "bed": ["nightstand", "dresser", "rug", "table_lamp", "bench"],
9            # More categories...
10        }
Show full code (76 lines)

Personalized Ranking System

eBay: "Design a personalized product ranking system based on user behavior."

This eBay interview question focuses on using behavioral data for personalization. A staff engineer who joined eBay shared their solution:

Learning-to-Rank Implementation

The eBay engineer described their personalized ranking approach:

1# Simplified ranking model implementation
2import lightgbm as lgb
3import numpy as np
4
5class PersonalizedRankingSystem:
6    def __init__(self, feature_store):
7        self.feature_store = feature_store
8        self.model = None
9        
10    def train_model(self, training_data, validation_data):
Show full code (135 lines)

Feature Store Implementation

Efficient feature retrieval is critical for real-time personalization:

1# Feature store implementation for personalized ranking
2class FeatureStore:
3    def __init__(self, redis_client, batch_client):
4        self.redis_client = redis_client  # For real-time features
5        self.batch_client = batch_client  # For batch features
6        
7    def get_user_features(self, user_id):
8        """Get user features from feature store."""
9        # Try to get from real-time store first
10        user_features_key = f"user:{user_id}:features"
Show full code (122 lines)

Cross-store Recommendation System

Shopify: "Design a cross-store recommendation system for Shopify merchants."

This Shopify interview question presents unique challenges for multi-tenant recommendations. A principal engineer who joined Shopify shared their solution:

Multi-tenant Model Architecture

The Shopify engineer described their specialized multi-tenant approach:

1# Multi-tenant recommendation system
2class CrossStoreRecommendationSystem:
3    def __init__(self, global_model_service, tenant_model_registry, feature_service):
4        self.global_model_service = global_model_service
5        self.tenant_model_registry = tenant_model_registry
6        self.feature_service = feature_service
7        
8    def get_recommendations(self, store_id, product_id=None, user_id=None, 
9                          context=None, recommendation_type='related', limit=10):
10        """Get recommendations for a specific store and context."""
Show full code (158 lines)

Real-time Personalization System

Target: "Create a real-time personalization system that combines online and in-store behavior."

This Target interview question focuses on omnichannel personalization. A senior architect who joined Target shared their approach:

Real-time Feature Processing

The Target architect described their real-time feature processing pipeline:

1# Real-time feature processing for omnichannel personalization
2class RealTimeFeatureProcessor:
3    def __init__(self, event_processor, feature_store, profile_service):
4        self.event_processor = event_processor
5        self.feature_store = feature_store
6        self.profile_service = profile_service
7        
8    def process_event(self, event):
9        """Process a single customer event."""
10        # Extract basic event information
Show full code (184 lines)

Omnichannel Customer Profile

The Target solution maintains a unified customer profile across channels:

1# Omnichannel customer profile service
2class CustomerProfileService:
3    def __init__(self, profile_db, feature_store, segmentation_service):
4        self.profile_db = profile_db
5        self.feature_store = feature_store
6        self.segmentation_service = segmentation_service
7        
8    def get_profile(self, customer_id):
9        """Get the complete customer profile."""
10        # Get core profile
Show full code (79 lines)

Results & Validation

Performance Benchmarks

Real-world recommendation engines at major e-commerce companies achieve these metrics:

• Recommendation Quality:

  • Click-through rate: 5-15%
  • Conversion rate: 2-5%
  • Revenue lift: 10-30%

• System Performance:

  • Recommendation generation: < 50ms (P95)
  • Model training: Daily or weekly
  • Feature updates: Near real-time (< 1 minute)

• Scale:

  • Products: Millions
  • Users: Tens of millions
  • Events processed: Billions per day

Trade-offs and Limitations

Every recommendation implementation involves key trade-offs:

Interview Strategy Tips

When tackling recommendation system design interviews:

1. Clarify Requirements:

   • Business goals (conversion, engagement, etc.)
   • Data availability (user interactions, content features)
   • Scale requirements (users, items, throughput)
   • Latency constraints

2. Focus on Key Components:

   • Data collection and processing pipeline
   • Feature engineering approach
   • Model selection and training strategy
   • Serving infrastructure

3. Address Common Challenges:

   • Cold start problem (new users, new items)
   • Popularity bias
   • Evaluation methodology
   • Online/offline testing approach

E-commerce Recommendation Templates

Download our comprehensive e-commerce recommendation templates based on real implementations from top e-commerce companies:

• Recommendation model architecture diagrams
• Feature engineering pipelines
• Serving infrastructure patterns
• A/B testing frameworks
• Evaluation metric definitions

Download Templates →

This article is part of our E-commerce Engineering Interview Series:

1. E-commerce Engineering Interviews: Scaling for Peaks and Personalization
2. Inventory Management Systems: Consistency Challenges in Distributed Commerce
3. Product Search and Discovery: Search Engine Implementation Questions
4. Shopping Cart Architecture: Session Management and Abandonment Recovery
5. Order Management Systems: Distributed Workflow Implementations