← Back to Mission Control

Git LFS Mastery

15 min read

Binary Asset Management

Large File Management Protocol

You are entering advanced binary asset management where you'll architect Git LFS solutions for handling massive files, media assets, and datasets while maintaining repository performance and team productivity.

Mission Briefing

Commander, managing large files in version control is like coordinating cargo operations for deep space missions. Just as spacecraft have specialized cargo bays for different types of equipment—with some items requiring special storage conditions, tracking protocols, and handling procedures—your repositories need sophisticated systems for managing binary assets, media files, machine learning models, and datasets that are too large for standard Git operations.

You'll master Git Large File Storage (LFS) and Binary Asset Management - the specialized systems that enable teams to version control massive files without compromising repository performance. From LFS setup and configuration to migration strategies and enterprise deployment patterns, you'll build expertise in handling terabytes of binary data across distributed development teams.

Large File Management Objectives

  • Master Git LFS setup, configuration, and enterprise deployment strategies
  • Implement efficient workflows for binary assets, media, and large datasets
  • Design migration strategies for existing repositories with large files
  • Optimize LFS performance with caching, bandwidth management, and storage tiers
  • Implement enterprise LFS server solutions and access control
  • Create backup and disaster recovery strategies for LFS repositories
15 minutes
6 Sections
2 Labs
Expert Level

Git LFS Fundamentals

Expert 3 minutes

Git LFS (Large File Storage) is a Git extension that replaces large files with text pointers inside Git, while storing the actual file contents on a remote server. This architecture enables version control of large binary assets without bloating repository size or degrading performance.

🏗️ Git LFS Architecture

Local Repository
source.js 2 KB
video.mp4 Pointer (128 B)
model.pkl Pointer (128 B)
⬇️ Git Operations
Git Server
source.js
LFS Pointers
⬇️ LFS Protocol
LFS Server
video.mp4 500 MB
model.pkl 2.3 GB

📊 LFS vs. Standard Git Comparison

Aspect Standard Git Git LFS
Repository Size Grows with every file version Only stores pointers (~128 bytes)
Clone Performance Downloads entire history Downloads current version only
Network Bandwidth Full file history transfer Selective file download
Diff Support Full text diff capabilities Limited to binary comparison
Merge Conflicts Advanced merge strategies Binary merge conflicts
Storage Requirements High disk usage Efficient server storage

🎯 When to Use Git LFS

Ideal Use Cases

  • Media Assets: Images, videos, audio files (>10MB)
  • Binary Executables: Compiled applications, installers
  • Data Files: Datasets, databases, ML models (>100MB)
  • Design Assets: CAD files, 3D models, textures
  • Game Assets: Sprites, textures, sound effects
  • Documents: Large PDFs, presentations, archives

Avoid LFS For

  • Small Files: Files under 1MB (pointer overhead)
  • Frequently Changed: Files modified in every commit
  • Text Files: Source code, configuration files
  • Mergeable Content: Files requiring line-by-line merging
  • High Availability: Critical files needing local access

📏 Enterprise File Size Guidelines

0 - 1MB
Standard Git
Source code, configs, small assets
1MB - 10MB
Consider LFS
Depends on change frequency and team size
10MB - 100MB
Use LFS
Media files, small datasets, documents
100MB+
Must Use LFS
Large datasets, ML models, video files

LFS Setup & Configuration

Expert 3 minutes

Setting up Git LFS requires installation, configuration, and proper tracking patterns to ensure efficient large file management across your development team and CI/CD systems.

🚀 Installation & Initialization

Step 1: Install Git LFS

Platform-Specific Installation 
# macOS (Homebrew)
brew install git-lfs

# Ubuntu/Debian
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs

# Windows (Chocolatey)
choco install git-lfs

# Windows (Manual - download from GitHub releases)
# https://github.com/git-lfs/git-lfs/releases

# Verify installation
git lfs version

Step 2: Initialize LFS in Repository

Repository LFS Setup 
# Initialize LFS in existing repository
git lfs install

# Initialize LFS globally for all repositories
git lfs install --system

# Initialize LFS in new repository
git init
git lfs install

# Verify LFS is properly configured
git lfs env

Step 3: Configure File Tracking

Track Large Files with LFS 
# Track specific file types
git lfs track "*.zip"
git lfs track "*.mp4"
git lfs track "*.psd"
git lfs track "*.pkl"

# Track files by size (all files over 50MB)
git lfs track --lockable "*.bin"

# Track specific file or directory
git lfs track "assets/videos/*"
git lfs track "models/large-model.h5"

# Commit the .gitattributes file
git add .gitattributes
git commit -m "feat: configure LFS tracking for large assets"

⚙️ Advanced LFS Configuration

LFS Server Configuration

Configure Custom LFS Server 
# Set custom LFS server endpoint
git config lfs.url "https://lfs.company.com/api/lfs"

# Configure authentication
git config lfs.https://lfs.company.com.access basic
git config lfs.https://lfs.company.com.locksverify true

# Set batch transfer settings
git config lfs.batch true
git config lfs.transfer.maxretries 3
git config lfs.transfer.maxretrydelay 10

# Configure concurrent transfers
git config lfs.concurrenttransfers 8

Fetch & Cache Configuration

Optimize LFS Performance 
# Configure LFS fetch behavior
git config lfs.fetchrecentalways true
git config lfs.fetchrecentrefsdays 7
git config lfs.fetchrecentcommitsdays 3

# Set up selective download
git lfs fetch --recent
git lfs fetch origin main

# Configure LFS cache
git config lfs.tlscachesize 128m
git config lfs.activitytimeout 30
git config lfs.dialtimeout 30

File Locking Configuration

Configure File Locking for Binary Assets 
# Track files with locking enabled
git lfs track --lockable "*.psd"
git lfs track --lockable "*.blend"
git lfs track --lockable "*.max"

# Lock a file for exclusive editing
git lfs lock "assets/logo.psd"

# Unlock a file after editing
git lfs unlock "assets/logo.psd"

# List locked files
git lfs locks

# Force unlock (admin only)
git lfs unlock --force "assets/logo.psd"

📄 Enterprise .gitattributes Configuration

Comprehensive .gitattributes for Enterprise Projects 
# Media Files
*.jpg filter=lfs diff=lfs merge=lfs -text
*.jpeg filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
*.bmp filter=lfs diff=lfs merge=lfs -text
*.tiff filter=lfs diff=lfs merge=lfs -text
*.webp filter=lfs diff=lfs merge=lfs -text

# Video Files
*.mp4 filter=lfs diff=lfs merge=lfs -text
*.avi filter=lfs diff=lfs merge=lfs -text
*.mov filter=lfs diff=lfs merge=lfs -text
*.mkv filter=lfs diff=lfs merge=lfs -text
*.webm filter=lfs diff=lfs merge=lfs -text

# Audio Files
*.mp3 filter=lfs diff=lfs merge=lfs -text
*.wav filter=lfs diff=lfs merge=lfs -text
*.flac filter=lfs diff=lfs merge=lfs -text
*.aac filter=lfs diff=lfs merge=lfs -text

# Archives & Executables
*.zip filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.7z filter=lfs diff=lfs merge=lfs -text
*.tar.gz filter=lfs diff=lfs merge=lfs -text
*.exe filter=lfs diff=lfs merge=lfs -text
*.dmg filter=lfs diff=lfs merge=lfs -text
*.app filter=lfs diff=lfs merge=lfs -text

# Design Files (with locking)
*.psd filter=lfs diff=lfs merge=lfs -text lockable
*.ai filter=lfs diff=lfs merge=lfs -text lockable
*.sketch filter=lfs diff=lfs merge=lfs -text lockable
*.fig filter=lfs diff=lfs merge=lfs -text lockable

# 3D & CAD Files
*.blend filter=lfs diff=lfs merge=lfs -text lockable
*.max filter=lfs diff=lfs merge=lfs -text lockable
*.obj filter=lfs diff=lfs merge=lfs -text
*.fbx filter=lfs diff=lfs merge=lfs -text
*.stl filter=lfs diff=lfs merge=lfs -text

# Data & ML Files
*.pkl filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.hdf5 filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.db filter=lfs diff=lfs merge=lfs -text

# Documents
*.pdf filter=lfs diff=lfs merge=lfs -text
*.docx filter=lfs diff=lfs merge=lfs -text
*.pptx filter=lfs diff=lfs merge=lfs -text
*.xlsx filter=lfs diff=lfs merge=lfs -text

# Game Assets
*.unity filter=lfs diff=lfs merge=lfs -text
*.unitypackage filter=lfs diff=lfs merge=lfs -text
*.asset filter=lfs diff=lfs merge=lfs -text

LFS Workflow Operations

Expert 3 minutes

Mastering LFS workflows requires understanding how to efficiently work with large files in daily development activities, from adding new assets to collaborating with team members and managing storage.

🔄 Essential LFS Operations

Adding Files to LFS

Add Large Files to Repository 
# Add large files (ensure tracking is configured first)
git add assets/video.mp4
git add models/neural-network.h5
git add design/logo.psd

# Commit large files (creates LFS pointers)
git commit -m "feat: add marketing video and ML model assets"

# Push to remote (uploads to LFS server)
git push origin main

# Verify files are tracked by LFS
git lfs ls-files
git lfs pointer --file=assets/video.mp4

Cloning LFS Repositories

Clone Repository with LFS Files 
# Clone with all LFS files
git clone https://github.com/company/project.git

# Clone without LFS files (faster for CI)
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/company/project.git

# Clone with selective LFS download
git lfs install --local
git clone https://github.com/company/project.git
git lfs pull --include="*.jpg,*.png" --exclude="videos/*"

Selective File Operations

Manage Large File Downloads Selectively 
# Fetch specific LFS files
git lfs fetch --include="*.psd,*.ai"
git lfs fetch --exclude="*.mp4"

# Download LFS files for specific paths
git lfs pull --include="assets/images/*"
git lfs pull --exclude="assets/videos/*"

# Check LFS file status
git lfs status
git lfs ls-files --size
git lfs fsck

# Prune old LFS files from local cache
git lfs prune --dry-run
git lfs prune --recent --verify-remote

🎯 Advanced LFS Workflows

Team Collaboration

Coordinate large file changes across distributed teams with file locking and selective synchronization.

# Designer workflow - lock file for editing
git lfs lock "designs/main-logo.psd"
# Edit file
git add designs/main-logo.psd
git commit -m "design: update main logo colors"
git push origin feature/logo-update
git lfs unlock "designs/main-logo.psd"

# Developer workflow - selective download
git lfs pull --include="assets/images/*.png"
# Skip unnecessary video assets during development

CI/CD Integration

Optimize CI/CD pipelines by downloading only necessary LFS files for build processes.

# GitHub Actions with selective LFS
- name: Checkout with LFS
  uses: actions/checkout@v3
  with:
    lfs: false

- name: Download required assets
  run: |
    git lfs pull --include="build-assets/*"
    git lfs pull --include="*.dll,*.so"

Branching Strategy

Handle large files across feature branches with proper merge strategies and conflict resolution.

# Feature branch with large files
git checkout -b feature/video-assets
git lfs track "*.mp4"
git add .gitattributes
git commit -m "track: configure video file tracking"

# Add large files to feature branch
git add videos/demo.mp4
git commit -m "feat: add product demo video"

⚡ LFS Performance Optimization

Download Optimization

  • Selective Fetching: Use --include/--exclude patterns to download only needed files
  • Batch Operations: Configure concurrent transfers for faster downloads
  • Local Caching: Leverage LFS cache to avoid re-downloading files
  • Shallow Clones: Combine with Git shallow clones for faster initial setup

Upload Optimization

  • Batch Uploads: Group multiple file uploads in single operations
  • Compression: Use compressed file formats when possible
  • Resume Support: Leverage LFS resume capabilities for large uploads
  • Network Configuration: Optimize timeout and retry settings

LFS Migration Strategies

Expert 3 minutes

Migrating existing repositories to use Git LFS requires careful planning and execution to preserve history while improving repository performance. Different migration strategies suit different scenarios and risk tolerances.

🔄 Migration Approach Selection

Retroactive Migration

Rewrites Git history to move large files to LFS throughout entire repository history.

✅ Advantages
  • Significantly reduces repository size
  • Optimal performance for all operations
  • Clean, consistent LFS usage
⚠️ Risks
  • Rewrites entire Git history
  • Requires team coordination
  • Breaks existing clones

Progressive Migration

Preserves Git history and migrates large files to LFS going forward only.

✅ Advantages
  • Preserves existing Git history
  • Lower risk, gradual transition
  • Existing clones remain functional
⚠️ Limitations
  • Repository size includes historical large files
  • Performance benefits only for future operations
  • Mixed storage approach

🔄 Retroactive Migration Process

Use git-lfs-migrate to rewrite repository history and move large files to LFS.

Step 1: Pre-Migration Assessment

Analyze Repository for Migration Candidates 
# Install git-lfs-migrate (included with recent LFS versions)
git lfs version

# Analyze repository to find large files
git lfs migrate info --everything --above=10MB

# Get detailed breakdown by file type
git lfs migrate info --everything --top=50

# Analyze specific branches
git lfs migrate info --include-ref=main --above=5MB

# Check repository size before migration
du -sh .git

Step 2: Execute Migration

Migrate Large Files to LFS 
# Migrate specific file types across all history
git lfs migrate import --include="*.zip,*.mp4,*.psd" --everything

# Migrate files above certain size
git lfs migrate import --above=50MB --everything

# Migrate with include/exclude patterns
git lfs migrate import \
  --include="*.jpg,*.png,*.gif" \
  --exclude="thumbnails/*" \
  --everything

# Migrate specific paths
git lfs migrate import --include="assets/videos/*" --everything

# Verify migration results
git lfs ls-files
git lfs migrate info --everything

Step 3: Post-Migration Cleanup

Complete Migration and Cleanup 
# Force push rewritten history (DESTRUCTIVE)
git push origin --force --all
git push origin --force --tags

# Clean up local repository
git reflog expire --expire=now --all
git gc --prune=now --aggressive

# Verify repository size reduction
du -sh .git

# Team coordination: Everyone must re-clone
echo "⚠️  ALL TEAM MEMBERS MUST RE-CLONE THE REPOSITORY"
echo "Old clones are incompatible with rewritten history"

Critical Migration Warning

Retroactive migration rewrites Git history and is irreversible. Ensure:

  • Complete team coordination and communication
  • Full repository backup before migration
  • All team members ready to re-clone
  • CI/CD systems updated for new repository state
  • External integrations tested with migrated repository

📈 Progressive Migration Process

Implement LFS for new large files while preserving existing repository history.

Set Up LFS for Future Files 
# Initialize LFS in existing repository
git lfs install

# Configure tracking for new large files
git lfs track "*.zip"
git lfs track "*.mp4" 
git lfs track "*.psd"
git lfs track --lockable "*.ai"

# Commit LFS configuration
git add .gitattributes
git commit -m "feat: configure LFS tracking for new large files"

# Document migration strategy for team
echo "# LFS Migration Strategy" > LFS_MIGRATION.md
echo "- New large files (>10MB) use LFS" >> LFS_MIGRATION.md
echo "- Existing files remain in Git history" >> LFS_MIGRATION.md
echo "- Consider retroactive migration for major version" >> LFS_MIGRATION.md

✨ Migration Best Practices

Backup Strategy

Create complete repository backups and test recovery procedures before migration.

Team Coordination

Plan migration during low-activity periods with full team communication.

Test Migration

Practice migration on repository copy to identify issues and timing.

Monitor Results

Track repository performance and team feedback after migration.

Enterprise LFS Deployment

Expert 2 minutes

Enterprise LFS deployments require robust server infrastructure, security controls, and operational procedures to support large-scale development teams with diverse large file requirements.

🏢 Enterprise LFS Server Solutions

GitHub Enterprise

Hosted Solution: Fully managed LFS with GitHub Enterprise Cloud/Server

Features
  • Integrated with GitHub workflows
  • Advanced access controls
  • Automatic scaling and backups
  • Enterprise security compliance
# GitHub Enterprise LFS configuration
git config lfs.url "https://github.company.com/api/lfs"
git config lfs.batch true

GitLab LFS

Self-Managed: GitLab CE/EE with integrated LFS support

Features
  • Self-hosted control
  • Integrated CI/CD
  • Object storage backends
  • Custom authentication
# GitLab LFS configuration
gitlab_rails['lfs_enabled'] = true
gitlab_rails['lfs_storage_path'] = "/mnt/lfs-storage"

Custom LFS Server

Self-Built: Custom implementation using LFS protocol specification

Popular Solutions
  • git-lfs-server: Reference implementation
  • lfs-server: Go-based server
  • Artifactory: JFrog enterprise solution
  • Nexus: Sonatype repository manager

💾 Enterprise Storage Architecture

Hot Storage (Frequently Accessed)

High-performance storage for active development files

  • Local SSD: Current branch files, recent commits
  • Network SSD: Shared team assets, CI/CD files
  • CDN: Globally distributed for remote teams

Warm Storage (Occasionally Accessed)

Balanced performance and cost for historical versions

  • Network HDD: Previous release assets
  • Object Storage: AWS S3 Standard, Azure Blob Hot
  • Compression: Automated compression for older files

Cold Storage (Rarely Accessed)

Cost-optimized storage for long-term retention

  • Glacier: AWS S3 Glacier, Azure Archive
  • Tape Storage: Enterprise backup systems
  • Compliance: Regulatory retention requirements

🔐 Security & Access Control

Authentication & Authorization

# Configure LFS authentication
git config lfs.https://lfs.company.com.access basic
git config credential.https://lfs.company.com.username "service-account"

# Use token-based authentication
git config credential.helper store
echo "https://token:${LFS_TOKEN}@lfs.company.com" >> ~/.git-credentials

Transport Security

  • TLS 1.2+: Encrypted data transmission
  • Certificate Pinning: Prevent MITM attacks
  • VPN Integration: Secure network access
  • Rate Limiting: DDoS protection

Access Policies

  • Repository Permissions: Team-based access control
  • File-Level Security: Selective file access
  • Audit Logging: Complete access trail
  • Compliance: SOC2, GDPR, HIPAA support

LFS Monitoring & Maintenance

Expert 1 minute

Maintaining optimal LFS performance requires continuous monitoring, proactive maintenance, and efficient troubleshooting procedures for enterprise-scale deployments.

📊 Key LFS Metrics

Server Metrics

  • Storage Usage: Total LFS storage consumption
  • Transfer Bandwidth: Upload/download rates
  • Request Latency: LFS API response times
  • Error Rates: Failed upload/download operations
  • Concurrent Users: Active LFS sessions

User Metrics

  • Clone Performance: Repository clone times
  • Fetch Efficiency: LFS file download performance
  • Cache Hit Rates: Local LFS cache effectiveness
  • Lock Contention: File locking conflicts
  • User Experience: Operation success rates

🔧 Regular Maintenance Tasks

LFS Cleanup

# Clean up local LFS cache
git lfs prune --dry-run
git lfs prune --recent --verify-remote

# Server-side cleanup (admin)
git lfs-server gc --older-than=30d
git lfs-server prune --unreferenced

Storage Analysis

# Analyze LFS usage patterns
git lfs ls-files --size | sort -k2 -h
git lfs migrate info --everything --top=20

# Repository health check
git lfs fsck
git lfs env

Security Audits

  • Review LFS access logs
  • Audit user permissions
  • Check certificate validity
  • Validate backup integrity

🔍 Common LFS Issues & Solutions

Slow LFS Operations

  1. Check network bandwidth and latency
  2. Increase concurrent transfer settings
  3. Configure local LFS cache
  4. Use selective file fetching
  5. Consider CDN deployment

File Lock Conflicts

  1. Identify locked files: git lfs locks
  2. Communicate with lock owner
  3. Force unlock if necessary (admin)
  4. Implement lock timeout policies
  5. Train team on locking workflow

Failed Downloads

  1. Check LFS server connectivity
  2. Verify authentication credentials
  3. Increase timeout settings
  4. Retry with exponential backoff
  5. Check server storage availability

Mission Status: COMPLETE

Exceptional work, Commander! You have successfully mastered Git LFS and large file management strategies for enterprise environments. Your expertise in LFS setup, migration strategies, and enterprise deployment will enable you to efficiently manage massive binary assets while maintaining optimal repository performance.

Your next commander operation will be Security & Compliance, where you'll learn to implement signed commits, GPG keys, and audit trails for enterprise security requirements.

Previous Mission Mission Control Next Mission: Security