Read Time:26 Minute, 24 Second
As a developer, choosing the right database can feel like laying the foundation of a skyscraper. If you choose wrong, everything else becomes a costly patch job. For modern applications, MongoDB has emerged not just as an alternative to SQL databases, but as the standard for agility, scalability, and developer happiness.
But knowing the basics of find() and insert() won’t save you at 2 AM when your production query slows to a crawl. This guide is your comprehensive playbook. We will go beyond the hype, dive into real-world use cases, and explore the advanced patterns required to ship production-grade applications.
Part 1: Why MongoDB? (The Developer’s Perspective)
At its core, MongoDB is a NoSQL, Document-Oriented Database. It uses a flexible, JSON-like format called BSON (Binary JSON) to store data.
The “Single View” Advantage
In a relational database, an e-commerce “Order” might live across 5 different tables (Orders, Users, Products, Inventory, Payments). To render a receipt, you perform expensive JOIN operations.
In MongoDB, the order is a document. You can embed the user’s shipping address directly inside the order document, or reference the product ID. This creates a “Single Source of Truth” that maps directly to how your application code actually uses the data .
The Schema-Less Superpower
You can change the data structure without running a risky ALTER TABLE on a billion-row database. If you add a new field, “loyalty_tier,” to your user documents, old documents without that field are simply ignored by queries—or you can set defaults on the fly in your code.
Part 2: Data Modeling for Scale (Production Reality)
The biggest mistake developers make is treating MongoDB like SQL. Normalization is the enemy of performance in a document database. In the SQL world, you remove duplication. In the MongoDB world, you often embrace duplication (denormalization) for speed.
The Golden Rule: Data that is accessed together, should be stored together.
1. Embedded Data Models (Optimal for 90% of cases)
Use this for “Contains” relationships (e.g., an order contains line items).
- Pro: Single round trip to the database.
- Con: Hard to query individually. Document size limit is 16MB.
2. Referenced Data Models (Relational approach)
Use this for “Many-to-Many” or frequently changing data (e.g., a User belongs to many Organizations).
- Pro: Keeps data DRY (Don’t Repeat Yourself).
- Con: Requires multiple queries or
$lookup(aggregation join), which is slower.
The 16MB Trap
The 16MB document limit is real. If you have a “Product Reviews” collection and a popular product gets 10,000 reviews, you will exceed 16MB. In this case, you must reference the reviews in a separate collection rather than embedding them.
Part 3: Mastering Queries & Aggregations
You will rarely use find() alone in production. You need the Aggregation Pipeline. Think of it as the MongoDB equivalent of a Unix pipe (|). You pass the output of one stage to the input of the next.
Real-World Pattern: “Incremental Analytics”
Imagine you run an e-commerce site. Calculating total sales by scanning millions of orders every day is expensive. The smart way is Incremental Analytics.
You run a daily job that looks only at today’s orders, sums them up, and merges the result into a “summary” collection. The UI queries only the summary collection.
Here is the pipeline that does this magic:
$match: Filters only today’s orders.$group: Sums the totals.$merge: Writes the result to a “daily_reports” collection. If a report for today exists, it replaces it; if not, it inserts it
db.orders.aggregate([
{ $match: { orderDate: { $gte: todayStart, $lt: todayEnd } } },
{ $group: { _id: null, dailyTotal: { $sum: "$amount" }, count: { $sum: 1 } } },
{ $merge: { into: "daily_reports", on: "_id", whenMatched: "replace" } }
])Part 4: Indexing Strategy (The Performance Knob)
No index means a collection scan (reading every document). This is fatal for APIs.
The Index Recipe Book
- Single Field: Good for specific lookups (e.g.,
find({user_id: 123})). - Compound Index (The Workhorse): Crucial for sorting or filtering on multiple fields. Order matters. The first field in the index is the most important.
- Query:
find({ status: "active" }).sort({ created_at: -1 }) - Index:
createIndex({ status: 1, created_at: -1 })
- Query:
- Partial Index: Only index documents that meet a specific filter.
- Use Case: You only search for “VIP” users. Don’t waste space indexing “Guest” users.
- Index:
createIndex({ email: 1 }, { partialFilterExpression: { tier: "VIP" } })
The “Covered Query” Holy Grail
If your index contains all the fields your query asks for, MongoDB never loads the actual document. It answers directly from RAM. It is blindingly fast.
Part 5: Real-Time & Modern Use Cases (2026+)
MongoDB isn’t just for storing JSON anymore. It is becoming an Operational Data Platform.
1. Real-Time Analytics (IoT & Fraud Detection)
Traditional databases struggle with high-velocity data. MongoDB handles millions of IoT sensor writes per second.
- How: Use Change Streams. Listen to the database in real-time.
- Example: A banking app. A transaction is inserted. A Node.js service listens for the
insertevent, runs a fraud check (e.g., “Has this user spent $10k in 1 hour?”), and rejects it instantly .
2. AI & Vector Search (The Generative AI Stack)
This is the newest trend. Large Language Models (LLMs) have a short memory (context window). You need to feed them your proprietary data (RAG – Retrieval Augmented Generation).
- MongoDB Atlas Vector Search allows you to store vector embeddings (numeric representations of meaning) right next to your operational data.
- Why it matters: You can search for “comfortable shoes” and find “plush slippers” and “cloud-like sneakers” without matching keywords. It understands meaning.
3. Geospatial Queries
If you are building Uber or Tinder:
// Find users within 5km of a specific point
db.users.find({
location: {
$near: {
$geometry: { type: "Point", coordinates: [ -73.9667, 40.78 ] },
$maxDistance: 5000
}
}
})Create a 2dsphere index on the location field
Part 6: Production Hardening (DevOps Guide)
Writing code is easy. Keeping it alive at 3 AM is hard.
1. Replica Sets (High Availability)
Never run MongoDB on a single server. Use a Replica Set (1 Primary, 2 Secondaries).
- If the Primary dies, an election happens (usually < 30 seconds), and a Secondary becomes Primary.
- Driver Hint: Use
readPreference=secondaryPreferredto offload analytics traffic to the replicas .
2. Sharding (Horizontal Scaling)
When one server isn’t enough, you shard. You split your data across 100 servers.
- The Shard Key: This is critical. A bad shard key (e.g.,
{ country: 1 }) leads to “Jumbo Chunks” (data imbalance). - Best Practice: Use a Hashed Index on a high-cardinality field (like
user_idoruuid) to ensure data is randomly distributed .
3. Backup & Disaster Recovery
- Atlas Backups: Use cloud-native snapshots (Continuous backups allow Point-in-Time Recovery, down to the second).
- mongodump: The standard tool, but it impacts performance. Prefer
mongodump --oplogto capture point-in-time consistency.
Part 7: The Developer’s Daily Checklist
Before you push your code to production, run through this checklist:
- Index Analysis: Run
db.collection.find({...}).explain("executionStats"). Do you see"stage": "COLLSCAN"? If yes, stop and add an index. - Projections: Are you returning
{ name: 1, price: 1 }or just dumping the whole document? Never return fields you don’t need. - Bulk Writes: Are you inserting 10,000 records in a
forloop? UsebulkWrite()to batch them into chunks of 1,000. - Connection Pooling: In serverless functions (AWS Lambda), do not create a new Mongo client on every invocation. Define the client outside the function handler to reuse the connection pool .
- Transaction Length: If you use multi-document transactions (ACID), keep them short. They require optimistic locking and can slow down if they run for seconds
MongoDB is not a niche tool for startups anymore. It is powering the backend of massive enterprises like Adobe, eBay, and Toyota because of its horizontal scaling (Sharding) and flexible schema.
For you, the developer, it eliminates the friction of mapping objects to tables (ORM hell). You store objects, you get objects back.
Start with schema design, master the aggregation pipeline, and never forget your indexes.
MongoDB CRUD
Prerequisites
Install MongoDB Go Driver
go get go.mongodb.org/mongo-driver/mongo
go get go.mongodb.org/mongo-driver/mongo/options
Also install for better development
go get github.com/joho/godotenv
go get github.com/gorilla/mux
Part 1: Production-Ready Connection Setup
The Connection Manager (With Proper Pooling)
// database/mongodb.go
package database
import (
"context"
"fmt"
"log"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
type MongoDB struct {
Client *mongo.Client
Database *mongo.Database
}
var DB *MongoDB
// Connect establishes connection to MongoDB with production settings
func Connect(uri, dbName string) error {
// Critical: Connection pool settings for production
clientOptions := options.Client().
ApplyURI(uri).
SetMaxPoolSize(100). // Max concurrent connections
SetMinPoolSize(10). // Keep 10 connections ready
SetMaxConnIdleTime(60 * time.Second). // Close idle connections
SetConnectTimeout(10 * time.Second). // Fail fast
SetSocketTimeout(45 * time.Second). // Kill slow queries
SetServerSelectionTimeout(5 * time.Second).
SetRetryWrites(true). // Auto-retry on failover
SetRetryReads(true) // Auto-retry reads
// Connect with timeout context
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
client, err := mongo.Connect(ctx, clientOptions)
if err != nil {
return fmt.Errorf("failed to connect to MongoDB: %w", err)
}
// Verify connection
ctx, cancel = context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := client.Ping(ctx, readpref.Primary()); err != nil {
return fmt.Errorf("failed to ping MongoDB: %w", err)
}
DB = &MongoDB{
Client: client,
Database: client.Database(dbName),
}
log.Println("✅ Connected to MongoDB successfully")
return nil
}
// Disconnect gracefully closes the connection
func Disconnect() error {
if DB == nil || DB.Client == nil {
return nil
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
return DB.Client.Disconnect(ctx)
}
// GetCollection returns a collection with proper context handling
func GetCollection(collectionName string) *mongo.Collection {
return DB.Database.Collection(collectionName)
}Main Application Setup
// main.go
package main
import (
"context"
"log"
"os"
"os/signal"
"syscall"
"time"
"github.com/gorilla/mux"
"github.com/joho/godotenv"
"your-app/database"
"your-app/handlers"
)
func main() {
// Load environment variables
if err := godotenv.Load(); err != nil {
log.Println("No .env file found, using system env")
}
// Connect to MongoDB
mongoURI := os.Getenv("MONGODB_URI")
if mongoURI == "" {
mongoURI = "mongodb://localhost:27017"
}
dbName := os.Getenv("DB_NAME")
if dbName == "" {
dbName = "ecommerce"
}
if err := database.Connect(mongoURI, dbName); err != nil {
log.Fatal("Failed to connect to database:", err)
}
defer database.Disconnect()
// Setup router
router := mux.NewRouter()
// Initialize handlers
productHandler := handlers.NewProductHandler()
// Routes
api := router.PathPrefix("/api/v1").Subrouter()
api.HandleFunc("/products", productHandler.CreateProduct).Methods("POST")
api.HandleFunc("/products", productHandler.ListProducts).Methods("GET")
api.HandleFunc("/products/{id}", productHandler.GetProduct).Methods("GET")
api.HandleFunc("/products/{id}", productHandler.UpdateProduct).Methods("PUT")
api.HandleFunc("/products/{id}", productHandler.DeleteProduct).Methods("DELETE")
// Start server
server := &http.Server{
Addr: ":3000",
Handler: router,
ReadTimeout: 15 * time.Second,
WriteTimeout: 15 * time.Second,
IdleTimeout: 60 * time.Second,
}
// Graceful shutdown
go func() {
log.Println("Server starting on :3000")
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatal("Server failed:", err)
}
}()
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
log.Println("Shutting down server...")
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
if err := server.Shutdown(ctx); err != nil {
log.Fatal("Server forced to shutdown:", err)
}
log.Println("Server exited properly")
}Part 2: Data Models (With Validation)
// models/product.go
package models
import (
"time"
"go.mongodb.org/mongo-driver/bson/primitive"
)
// Product represents our main product model
type Product struct {
ID primitive.ObjectID `bson:"_id,omitempty" json:"id"`
Name string `bson:"name" json:"name" validate:"required,min=3,max=100"`
Description string `bson:"description" json:"description"`
SKU string `bson:"sku" json:"sku" validate:"required,unique"`
Price float64 `bson:"price" json:"price" validate:"required,gt=0"`
Category string `bson:"category" json:"category"`
Brand string `bson:"brand" json:"brand"`
Inventory int `bson:"inventory" json:"inventory"`
Status string `bson:"status" json:"status"` // active, inactive, deleted
SalesCount int `bson:"salesCount" json:"salesCount"`
AverageRating float64 `bson:"averageRating" json:"averageRating"`
Tags []string `bson:"tags" json:"tags"`
Variants []ProductVariant `bson:"variants,omitempty" json:"variants,omitempty"`
Reviews []Review `bson:"reviews,omitempty" json:"reviews,omitempty"`
Metadata map[string]interface{} `bson:"metadata,omitempty" json:"metadata,omitempty"`
CreatedAt time.Time `bson:"createdAt" json:"createdAt"`
UpdatedAt time.Time `bson:"updatedAt" json:"updatedAt"`
}
type ProductVariant struct {
Size string `bson:"size" json:"size"`
Color string `bson:"color" json:"color"`
Price float64 `bson:"price" json:"price"`
Inventory int `bson:"inventory" json:"inventory"`
SKU string `bson:"sku" json:"sku"`
}
type Review struct {
UserID primitive.ObjectID `bson:"userId" json:"userId"`
Rating int `bson:"rating" json:"rating" validate:"min=1,max=5"`
Comment string `bson:"comment" json:"comment"`
CreatedAt time.Time `bson:"createdAt" json:"createdAt"`
}
// For API requests/responses
type CreateProductRequest struct {
Name string `json:"name" validate:"required"`
Description string `json:"description"`
SKU string `json:"sku" validate:"required"`
Price float64 `json:"price" validate:"required,gt=0"`
Category string `json:"category"`
Brand string `json:"brand"`
Inventory int `json:"inventory"`
Tags []string `json:"tags"`
Variants []ProductVariant `json:"variants"`
}
type UpdateProductRequest struct {
Name *string `json:"name,omitempty"`
Description *string `json:"description,omitempty"`
Price *float64 `json:"price,omitempty"`
Category *string `json:"category,omitempty"`
Brand *string `json:"brand,omitempty"`
Inventory *int `json:"inventory,omitempty"`
Status *string `json:"status,omitempty"`
Tags []string `json:"tags,omitempty"`
}
type ProductFilter struct {
Category string
Brand string
MinPrice float64
MaxPrice float64
InStock bool
Search string
Limit int64
Offset int64
SortBy string
SortOrder int // 1 for asc, -1 for desc
}Part 3: Complete CRUD Operations
Repository Layer (Database Operations)
// repository/product_repo.go
package repository
import (
"context"
"errors"
"fmt"
"time"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/primitive"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"your-app/database"
"your-app/models"
)
type ProductRepository struct {
collection *mongo.Collection
}
func NewProductRepository() *ProductRepository {
return &ProductRepository{
collection: database.GetCollection("products"),
}
}
// CREATE: Insert a single product
func (r *ProductRepository) Create(ctx context.Context, product *models.Product) error {
product.ID = primitive.NewObjectID()
product.CreatedAt = time.Now()
product.UpdatedAt = time.Now()
product.Status = "active"
product.SalesCount = 0
product.AverageRating = 0
// Add unique index on SKU
_, err := r.collection.InsertOne(ctx, product)
if err != nil {
// Check for duplicate key error
if mongo.IsDuplicateKeyError(err) {
return fmt.Errorf("product with SKU %s already exists", product.SKU)
}
return err
}
return nil
}
// BULK CREATE: Insert multiple products efficiently
func (r *ProductRepository) BulkCreate(ctx context.Context, products []*models.Product) error {
if len(products) == 0 {
return nil
}
// Prepare all documents
docs := make([]interface{}, len(products))
for i, p := range products {
p.ID = primitive.NewObjectID()
p.CreatedAt = time.Now()
p.UpdatedAt = time.Now()
p.Status = "active"
docs[i] = p
}
// Use ordered: false to continue on errors
opts := options.InsertMany().SetOrdered(false)
result, err := r.collection.InsertMany(ctx, docs, opts)
if err != nil {
// Handle partial success
if bulkErr, ok := err.(mongo.BulkWriteException); ok {
return fmt.Errorf("inserted %d documents, errors: %v",
len(result.InsertedIDs), bulkErr.WriteErrors)
}
return err
}
return nil
}
// READ: Find product by ID
func (r *ProductRepository) FindByID(ctx context.Context, id string) (*models.Product, error) {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return nil, errors.New("invalid product ID format")
}
var product models.Product
filter := bson.M{
"_id": objectID,
"status": bson.M{"$ne": "deleted"}, // Soft delete filter
}
err = r.collection.FindOne(ctx, filter).Decode(&product)
if err == mongo.ErrNoDocuments {
return nil, errors.New("product not found")
}
if err != nil {
return nil, err
}
return &product, nil
}
// READ: Complex filtering with pagination
func (r *ProductRepository) FindAll(ctx context.Context, filter models.ProductFilter) ([]*models.Product, int64, error) {
// Build query filter
query := bson.M{"status": bson.M{"$ne": "deleted"}}
// Apply filters
if filter.Category != "" {
query["category"] = filter.Category
}
if filter.Brand != "" {
query["brand"] = filter.Brand
}
if filter.MinPrice > 0 || filter.MaxPrice > 0 {
priceFilter := bson.M{}
if filter.MinPrice > 0 {
priceFilter["$gte"] = filter.MinPrice
}
if filter.MaxPrice > 0 {
priceFilter["$lte"] = filter.MaxPrice
}
query["price"] = priceFilter
}
if filter.InStock {
query["inventory"] = bson.M{"$gt": 0}
}
// Text search (requires text index)
if filter.Search != "" {
query["$text"] = bson.M{"$search": filter.Search}
}
// Setup pagination
limit := filter.Limit
if limit <= 0 {
limit = 20
}
if limit > 100 {
limit = 100
}
offset := filter.Offset
if offset < 0 {
offset = 0
}
// Setup sorting
sort := bson.D{}
switch filter.SortBy {
case "price":
sort = append(sort, bson.E{Key: "price", Value: filter.SortOrder})
case "createdAt":
sort = append(sort, bson.E{Key: "createdAt", Value: filter.SortOrder})
case "salesCount":
sort = append(sort, bson.E{Key: "salesCount", Value: filter.SortOrder})
case "rating":
sort = append(sort, bson.E{Key: "averageRating", Value: filter.SortOrder})
default:
sort = append(sort, bson.E{Key: "createdAt", Value: -1})
}
// Execute count and find in parallel for better performance
var total int64
var products []*models.Product
// Get total count
total, err := r.collection.CountDocuments(ctx, query)
if err != nil {
return nil, 0, err
}
// Get paginated results
opts := options.Find().
SetSort(sort).
SetSkip(offset).
SetLimit(limit)
cursor, err := r.collection.Find(ctx, query, opts)
if err != nil {
return nil, 0, err
}
defer cursor.Close(ctx)
if err = cursor.All(ctx, &products); err != nil {
return nil, 0, err
}
return products, total, nil
}
// UPDATE: Partial update with struct
func (r *ProductRepository) Update(ctx context.Context, id string, updates map[string]interface{}) error {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return errors.New("invalid product ID format")
}
// Add updated timestamp
updates["updatedAt"] = time.Now()
filter := bson.M{
"_id": objectID,
"status": bson.M{"$ne": "deleted"},
}
update := bson.M{
"$set": updates,
}
result, err := r.collection.UpdateOne(ctx, filter, update)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errors.New("product not found")
}
return nil
}
// UPDATE: Increment counter (atomic operation)
func (r *ProductRepository) IncrementSalesCount(ctx context.Context, id string, quantity int) error {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return err
}
filter := bson.M{"_id": objectID}
update := bson.M{
"$inc": bson.M{"salesCount": quantity},
"$set": bson.M{"updatedAt": time.Now()},
}
result, err := r.collection.UpdateOne(ctx, filter, update)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errors.New("product not found")
}
return nil
}
// UPDATE: Add item to array field
func (r *ProductRepository) AddReview(ctx context.Context, productID string, review models.Review) error {
objectID, err := primitive.ObjectIDFromHex(productID)
if err != nil {
return err
}
review.CreatedAt = time.Now()
filter := bson.M{"_id": objectID}
update := bson.M{
"$push": bson.M{"reviews": review},
"$inc": bson.M{"totalReviews": 1},
"$set": bson.M{"updatedAt": time.Now()},
}
_, err = r.collection.UpdateOne(ctx, filter, update)
if err != nil {
return err
}
// Recalculate average rating
return r.recalculateAverageRating(ctx, objectID)
}
func (r *ProductRepository) recalculateAverageRating(ctx context.Context, productID primitive.ObjectID) error {
// Use aggregation to calculate average
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{"_id": productID}}},
{{Key: "$unwind", Value: "$reviews"}},
{{Key: "$group", Value: bson.M{
"_id": "$_id",
"avgRating": bson.M{"$avg": "$reviews.rating"},
}}},
}
cursor, err := r.collection.Aggregate(ctx, pipeline)
if err != nil {
return err
}
defer cursor.Close(ctx)
var result struct {
AvgRating float64 `bson:"avgRating"`
}
if cursor.Next(ctx) {
cursor.Decode(&result)
update := bson.M{
"$set": bson.M{
"averageRating": result.AvgRating,
"updatedAt": time.Now(),
},
}
_, err = r.collection.UpdateOne(ctx, bson.M{"_id": productID}, update)
return err
}
return nil
}
// DELETE: Soft delete (recommended for production)
func (r *ProductRepository) SoftDelete(ctx context.Context, id string) error {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return err
}
filter := bson.M{"_id": objectID}
update := bson.M{
"$set": bson.M{
"status": "deleted",
"deletedAt": time.Now(),
"updatedAt": time.Now(),
},
}
result, err := r.collection.UpdateOne(ctx, filter, update)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errors.New("product not found")
}
return nil
}
// DELETE: Hard delete (use with caution)
func (r *ProductRepository) HardDelete(ctx context.Context, id string) error {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return err
}
filter := bson.M{"_id": objectID}
result, err := r.collection.DeleteOne(ctx, filter)
if err != nil {
return err
}
if result.DeletedCount == 0 {
return errors.New("product not found")
}
return nil
}
// ATOMIC: Find and update with version check (optimistic locking)
func (r *ProductRepository) UpdateWithOptimisticLock(ctx context.Context, id string, version int, updates map[string]interface{}) error {
objectID, err := primitive.ObjectIDFromHex(id)
if err != nil {
return err
}
filter := bson.M{
"_id": objectID,
"version": version,
}
updates["version"] = version + 1
updates["updatedAt"] = time.Now()
update := bson.M{"$set": updates}
result, err := r.collection.UpdateOne(ctx, filter, update)
if err != nil {
return err
}
if result.MatchedCount == 0 {
return errors.New("product was modified by another request, please retry")
}
return nil
}Part 4: Advanced Aggregation Queries
Complex Analytics with Aggregation Pipeline
// repository/analytics_repo.go
package repository
import (
"context"
"time"
"go.mongodb.org/mongo-driver/bson"
"your-app/database"
)
type AnalyticsRepository struct {
collection *mongo.Collection
}
func NewAnalyticsRepository() *AnalyticsRepository {
return &AnalyticsRepository{
collection: database.GetCollection("products"),
}
}
// Get category-wise statistics
func (r *AnalyticsRepository) GetCategoryStats(ctx context.Context) ([]bson.M, error) {
pipeline := mongo.Pipeline{
// Stage 1: Filter active products only
{{Key: "$match", Value: bson.M{
"status": "active",
"price": bson.M{"$gt": 0},
}}},
// Stage 2: Group by category
{{Key: "$group", Value: bson.M{
"_id": "$category",
"productCount": bson.M{"$sum": 1},
"averagePrice": bson.M{"$avg": "$price"},
"totalRevenue": bson.M{"$sum": bson.M{"$multiply": []interface{}{"$price", "$salesCount"}}},
"totalSales": bson.M{"$sum": "$salesCount"},
"minPrice": bson.M{"$min": "$price"},
"maxPrice": bson.M{"$max": "$price"},
}}},
// Stage 3: Sort by product count
{{Key: "$sort", Value: bson.M{"productCount": -1}}},
// Stage 4: Project clean output
{{Key: "$project", Value: bson.M{
"category": "$_id",
"productCount": 1,
"averagePrice": bson.M{"$round": []interface{}{"$averagePrice", 2}},
"totalRevenue": bson.M{"$round": []interface{}{"$totalRevenue", 2}},
"totalSales": 1,
"minPrice": 1,
"maxPrice": 1,
"_id": 0,
}}},
}
cursor, err := r.collection.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
defer cursor.Close(ctx)
var results []bson.M
if err = cursor.All(ctx, &results); err != nil {
return nil, err
}
return results, nil
}
// Get top selling products
func (r *AnalyticsRepository) GetTopSellingProducts(ctx context.Context, limit int) ([]bson.M, error) {
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{"status": "active"}}},
{{Key: "$sort", Value: bson.M{"salesCount": -1}}},
{{Key: "$limit", Value: limit}},
{{Key: "$project", Value: bson.M{
"name": 1,
"price": 1,
"salesCount": 1,
"category": 1,
"revenue": bson.M{"$multiply": []interface{}{"$price", "$salesCount"}},
}}},
}
cursor, err := r.collection.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
defer cursor.Close(ctx)
var results []bson.M
if err = cursor.All(ctx, &results); err != nil {
return nil, err
}
return results, nil
}
// Get inventory report (low stock items)
func (r *AnalyticsRepository) GetLowStockReport(ctx context.Context, threshold int) ([]bson.M, error) {
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{
"status": "active",
"inventory": bson.M{"$lte": threshold},
}}},
{{Key: "$sort", Value: bson.M{"inventory": 1}}},
{{Key: "$project", Value: bson.M{
"name": 1,
"sku": 1,
"inventory": 1,
"category": 1,
"alert": bson.M{
"$cond": bson.M{
"if": bson.M{"$lte": []interface{}{"$inventory", 5}},
"then": "CRITICAL",
"else": "WARNING",
},
},
}}},
}
cursor, err := r.collection.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
defer cursor.Close(ctx)
var results []bson.M
if err = cursor.All(ctx, &results); err != nil {
return nil, err
}
return results, nil
}
// Time-series analysis: Sales trend by month
func (r *AnalyticsRepository) GetMonthlySalesTrend(ctx context.Context, year int) ([]bson.M, error) {
pipeline := mongo.Pipeline{
{{Key: "$match", Value: bson.M{
"status": "active",
"createdAt": bson.M{
"$gte": time.Date(year, 1, 1, 0, 0, 0, 0, time.UTC),
"$lt": time.Date(year+1, 1, 1, 0, 0, 0, 0, time.UTC),
},
}}},
{{Key: "$group", Value: bson.M{
"_id": bson.M{
"month": bson.M{"$month": "$createdAt"},
"year": bson.M{"$year": "$createdAt"},
},
"totalProducts": bson.M{"$sum": 1},
"avgPrice": bson.M{"$avg": "$price"},
"totalInventory": bson.M{"$sum": "$inventory"},
}}},
{{Key: "$sort", Value: bson.M{"_id.month": 1}}},
{{Key: "$project", Value: bson.M{
"month": "$_id.month",
"year": "$_id.year",
"totalProducts": 1,
"avgPrice": bson.M{"$round": []interface{}{"$avgPrice", 2}},
"totalInventory": 1,
"_id": 0,
}}},
}
cursor, err := r.collection.Aggregate(ctx, pipeline)
if err != nil {
return nil, err
}
defer cursor.Close(ctx)
var results []bson.M
if err = cursor.All(ctx, &results); err != nil {
return nil, err
}
return results, nil
}Part 5: HTTP Handlers with Error Handling
// handlers/product_handler.go
package handlers
import (
"encoding/json"
"net/http"
"strconv"
"github.com/gorilla/mux"
"go.mongodb.org/mongo-driver/mongo"
"your-app/models"
"your-app/repository"
)
type ProductHandler struct {
repo *repository.ProductRepository
}
func NewProductHandler() *ProductHandler {
return &ProductHandler{
repo: repository.NewProductRepository(),
}
}
// CreateProduct handles POST /api/v1/products
func (h *ProductHandler) CreateProduct(w http.ResponseWriter, r *http.Request) {
var req models.CreateProductRequest
// Parse request body
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Invalid request body", http.StatusBadRequest)
return
}
// Convert to product model
product := &models.Product{
Name: req.Name,
Description: req.Description,
SKU: req.SKU,
Price: req.Price,
Category: req.Category,
Brand: req.Brand,
Inventory: req.Inventory,
Tags: req.Tags,
Variants: req.Variants,
}
// Create product
if err := h.repo.Create(r.Context(), product); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(product)
}
// ListProducts handles GET /api/v1/products
func (h *ProductHandler) ListProducts(w http.ResponseWriter, r *http.Request) {
// Parse query parameters
query := r.URL.Query()
limit, _ := strconv.ParseInt(query.Get("limit"), 10, 64)
offset, _ := strconv.ParseInt(query.Get("offset"), 10, 64)
minPrice, _ := strconv.ParseFloat(query.Get("minPrice"), 64)
maxPrice, _ := strconv.ParseFloat(query.Get("maxPrice"), 64)
filter := models.ProductFilter{
Category: query.Get("category"),
Brand: query.Get("brand"),
MinPrice: minPrice,
MaxPrice: maxPrice,
InStock: query.Get("inStock") == "true",
Search: query.Get("search"),
Limit: limit,
Offset: offset,
SortBy: query.Get("sortBy"),
SortOrder: 1, // default asc
}
if query.Get("sortOrder") == "desc" {
filter.SortOrder = -1
}
// Get products
products, total, err := h.repo.FindAll(r.Context(), filter)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
response := map[string]interface{}{
"products": products,
"total": total,
"limit": limit,
"offset": offset,
}
json.NewEncoder(w).Encode(response)
}
// GetProduct handles GET /api/v1/products/{id}
func (h *ProductHandler) GetProduct(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id := vars["id"]
product, err := h.repo.FindByID(r.Context(), id)
if err != nil {
http.Error(w, err.Error(), http.StatusNotFound)
return
}
json.NewEncoder(w).Encode(product)
}
// UpdateProduct handles PUT /api/v1/products/{id}
func (h *ProductHandler) UpdateProduct(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id := vars["id"]
var req models.UpdateProductRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
http.Error(w, "Invalid request body", http.StatusBadRequest)
return
}
// Build update map (only non-nil fields)
updates := make(map[string]interface{})
if req.Name != nil {
updates["name"] = *req.Name
}
if req.Description != nil {
updates["description"] = *req.Description
}
if req.Price != nil {
updates["price"] = *req.Price
}
if req.Category != nil {
updates["category"] = *req.Category
}
if req.Brand != nil {
updates["brand"] = *req.Brand
}
if req.Inventory != nil {
updates["inventory"] = *req.Inventory
}
if req.Status != nil {
updates["status"] = *req.Status
}
if req.Tags != nil {
updates["tags"] = req.Tags
}
if err := h.repo.Update(r.Context(), id, updates); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusOK)
json.NewEncoder(w).Encode(map[string]string{"message": "product updated successfully"})
}
// DeleteProduct handles DELETE /api/v1/products/{id}
func (h *ProductHandler) DeleteProduct(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id := vars["id"]
// Use soft delete for production
if err := h.repo.SoftDelete(r.Context(), id); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}Part 6: Index Management (Critical for Production)
// migrations/indexes.go
package migrations
import (
"context"
"log"
"time"
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"your-app/database"
)
func SetupIndexes() error {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
products := database.GetCollection("products")
// 1. Unique index on SKU
_, err := products.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "sku", Value: 1}},
Options: options.Index().SetUnique(true).SetName("idx_unique_sku"),
})
if err != nil {
return err
}
// 2. Compound index for common queries
_, err = products.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{
{Key: "category", Value: 1},
{Key: "price", Value: -1},
{Key: "status", Value: 1},
},
Options: options.Index().SetName("idx_category_price_status"),
})
if err != nil {
return err
}
// 3. Text index for search functionality
_, err = products.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{
{Key: "name", Value: "text"},
{Key: "description", Value: "text"},
{Key: "tags", Value: "text"},
},
Options: options.Index().
SetName("idx_text_search").
SetDefaultLanguage("english").
SetWeights(bson.M{
"name": 10,
"tags": 5,
"description": 1,
}),
})
if err != nil {
return err
}
// 4. Partial index for active products only
_, err = products.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "status", Value: 1}, {Key: "createdAt", Value: -1}},
Options: options.Index().
SetName("idx_active_products").
SetPartialFilterExpression(bson.M{"status": "active"}),
})
if err != nil {
return err
}
// 5. TTL index for soft-deleted items (auto-cleanup after 30 days)
_, err = products.Indexes().CreateOne(ctx, mongo.IndexModel{
Keys: bson.D{{Key: "deletedAt", Value: 1}},
Options: options.Index().
SetName("idx_cleanup_deleted").
SetExpireAfterSeconds(2592000), // 30 days
})
if err != nil {
return err
}
log.Println("✅ All indexes created successfully")
return nil
}Part 7: Testing (Production Quality)
// handlers/product_handler_test.go
package handlers
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/gorilla/mux"
"github.com/stretchr/testify/assert"
"go.mongodb.org/mongo-driver/mongo/integration/mtest"
)
func TestCreateProduct(t *testing.T) {
mt := mtest.New(t, mtest.NewOptions().ClientType(mtest.Mock))
mt.Run("successfully creates product", func(mt *mtest.T) {
handler := NewProductHandler()
requestBody := models.CreateProductRequest{
Name: "Test Product",
SKU: "TEST001",
Price: 99.99,
Category: "Electronics",
Inventory: 100,
}
body, _ := json.Marshal(requestBody)
req := httptest.NewRequest("POST", "/api/v1/products", bytes.NewBuffer(body))
w := httptest.NewRecorder()
handler.CreateProduct(w, req)
assert.Equal(t, http.StatusCreated, w.Code)
})
}
// Integration test with real MongoDB
func TestProductCRUDIntegration(t *testing.T) {
// Skip if no MongoDB running
if testing.Short() {
t.Skip("Skipping integration test")
}
// Setup test database
testDB := setupTestDB(t)
defer cleanupTestDB(testDB)
// Create product
product := &models.Product{
Name: "Integration Test Product",
SKU: "INT001",
Price: 49.99,
Category: "Testing",
Inventory: 50,
}
err := testDB.repo.Create(context.Background(), product)
assert.NoError(t, err)
assert.NotEmpty(t, product.ID)
// Retrieve product
found, err := testDB.repo.FindByID(context.Background(), product.ID.Hex())
assert.NoError(t, err)
assert.Equal(t, product.Name, found.Name)
}Part 8: Environment Configuration
# .env file
MONGODB_URI=mongodb://localhost:27017
DB_NAME=ecommerce_prod
PORT=3000
ENVIRONMENT=production
# Connection pool settings
MAX_POOL_SIZE=100
MIN_POOL_SIZE=10
CONNECTION_TIMEOUT=10
SOCKET_TIMEOUT=45
# For MongoDB Atlas (production)
# MONGODB_URI=mongodb+srv://username:password@cluster.mongodb.net/?retryWrites=true&w=majorityQuick Start Script
!/bin/bash
setup.sh – Complete project setup
Create project structure
mkdir -p mongodb-go-crud/{handlers,models,repository,database,migrations}
cd mongodb-go-crud
Initialize go module
go mod init your-app
Install dependencies
go get go.mongodb.org/mongo-driver/mongo
go get github.com/gorilla/mux
go get github.com/joho/godotenv
go get github.com/stretchr/testify
Set environment
echo “MONGODB_URI=mongodb://localhost:27017” > .env
echo “DB_NAME=testdb” >> .env
Run the application
go run main.go
Or build and run
go build -o app .
./app
Production Checklist
Before deploying to production, ensure you have:
- ✅ Indexes created – Run your migration scripts
- ✅ Connection pooling configured – Set proper pool sizes
- ✅ Retry logic enabled – For failover scenarios
- ✅ Context timeouts set – Prevent hanging operations
- ✅ Soft delete implemented – Never hard delete in production
- ✅ Replica set configured – For high availability
- ✅ Monitoring setup – Use MongoDB Atlas or Prometheus
- ✅ Backup strategy – Regular snapshots
- ✅ Read preference configured – Distribute load to secondaries
- ✅ Write concern set – Balance durability vs performance
The patterns shown here (connection pooling, error handling, indexing, aggregation) are exactly what you’ll use in real-world applications.
Happy
0 %
Sad
0 %
Excited
0 %
Sleepy
0 %
Angry
0 %
Surprise
0 %