Containers vs Virtual Machines

Incus uniquely supports both system containers and full virtual machines through a unified interface. Understanding the differences helps you choose the right instance type for your workload.

Quick Comparison

Feature	Containers	Virtual Machines
Technology	LXC (Linux Containers)	QEMU/KVM
Kernel	Shared with host	Independent guest kernel
Boot Time	1-2 seconds	10-30 seconds
Memory Overhead	~1-5 MB	~100-200 MB minimum
Density	Very high (100s per host)	Moderate (10s per host)
Isolation	Process-level isolation	Hardware-level isolation
Performance	Near-native (~99%)	Near-native (~95-98%)
OS Support	Linux only	Any OS (Windows, BSD, etc.)
Security	Namespace isolation	Hardware isolation
Use Case	Microservices, dev/test	Legacy apps, non-Linux

Visual Comparison

Architecture Differences

Containers: Deep Dive

How Containers Work

System containers in Incus use Linux kernel features to create isolated execution environments that share the host kernel. They provide a full Linux system environment while being extremely lightweight.

                Container Isolation Technologies
                Namespaces: Process, network, mount, IPC, UTS, user isolation
Cgroups: Resource limits and accounting (CPU, memory, I/O, network)
AppArmor/SELinux: Mandatory access control profiles
Seccomp: System call filtering for security
Capabilities: Fine-grained privilege control

            

Container Use Cases

Microservices: Lightweight, fast deployment for service-oriented architectures
Development Environments: Quick spin-up of isolated Linux environments
CI/CD Pipelines: Fast, consistent build environments
Multi-tenant Hosting: High-density hosting with good isolation
Testing: Rapid creation and destruction of test environments
System Administration: Safe testing of system configurations

Container Limitations

Linux only (cannot run Windows, BSD, or other operating systems)
Shares kernel with host (kernel vulnerabilities affect all containers)
Cannot run different kernel versions than host
Less isolation than VMs (shared kernel attack surface)
Privileged containers can be security risks if not carefully managed

Virtual Machines: Deep Dive

How VMs Work

Incus VMs use QEMU with KVM acceleration to provide full hardware virtualization. Each VM has its own kernel, complete OS installation, and emulated or paravirtualized hardware.

                VM Virtualization Technologies
                KVM: Kernel-based Virtual Machine for hardware acceleration
QEMU: Machine emulator and virtualizer
virtio: Paravirtualized I/O drivers for performance
UEFI/BIOS: Firmware support for modern and legacy systems
VirtIO-FS: High-performance filesystem sharing
VFIO: Device passthrough for GPU, USB, PCIe devices

            

VM Use Cases

Non-Linux Workloads: Windows, BSD, macOS, or other operating systems
Kernel Development: Testing custom kernels without rebooting host
Legacy Applications: Running old software requiring specific OS versions
Security-Critical Workloads: Maximum isolation for untrusted code
Compliance Requirements: When regulations require hardware-level isolation
Different Linux Distributions: Running kernels with specific features
GPU Workloads: GPU passthrough for machine learning, gaming, rendering

VM Limitations

Higher overhead (requires full OS installation and dedicated resources)
Slower startup times compared to containers
Lower density (fewer VMs per host due to resource requirements)
More complex snapshots (full disk state vs filesystem-only)
Larger disk space requirements

Performance Comparison

Metric	Containers	VMs	Notes
CPU Performance	99-100% native	95-98% native	KVM provides near-native performance
Memory Performance	100% native	98-99% native	Minimal overhead with KVM
Disk I/O	95-100% native	85-95% native	VirtIO reduces gap significantly
Network I/O	95-100% native	90-95% native	VirtIO-net provides good performance
Boot Time	1-3 seconds	10-30 seconds	Containers skip full OS boot
Memory Overhead	1-5 MB	100-200 MB+	VMs need full OS in memory

Choosing Between Containers and VMs

Decision Framework

Choose Containers when:

Running Linux workloads exclusively
Need maximum density and resource efficiency
Require fast startup and shutdown times
Working with microservices or cloud-native applications
Need disposable, ephemeral environments
Security isolation at process level is sufficient

Choose VMs when:

Running non-Linux operating systems (Windows, BSD, etc.)
Need hardware-level isolation for security
Require a specific kernel version or custom kernel
Working with legacy applications expecting full OS
Need GPU passthrough or device passthrough
Compliance requires complete isolation
Testing kernel modules or system-level software

Incus Unified Management

One of Incus's key strengths is providing a unified interface for both containers and VMs. The same commands, APIs, and workflows apply to both instance types.

Creating Instances

# Create a container (default)
incus launch ubuntu:22.04 my-container

# Create a VM (add --vm flag)
incus launch ubuntu:22.04 my-vm --vm

# Both use the same image format
# Both managed with identical commands
incus list
incus exec my-container -- bash
incus exec my-vm -- bash

Hybrid Deployments

You can run containers and VMs side-by-side on the same Incus host, choosing the best tool for each workload:

Web servers and APIs as lightweight containers
Database servers as VMs for isolation and performance
Windows-based services as VMs
Development environments as containers for rapid iteration

Resource Efficiency Example

Typical Server Capacity

Server: 32 CPU cores, 128 GB RAM, 2 TB SSD

Container Deployment:

100+ lightweight containers (web apps, microservices)
Average: 512 MB RAM, 0.5 CPU per container
Fast deployment, instant scaling
Remaining resources for bursting

VM Deployment:

20-30 VMs (full OS instances)
Average: 4 GB RAM, 2 CPU per VM
Complete isolation, multiple OS types
Better for production databases, Windows apps

Hybrid Deployment (Recommended):

50 containers for stateless applications
10 VMs for databases, Windows services, special requirements
Best of both worlds: efficiency + flexibility

Migration Capabilities

Feature	Containers	VMs
Live Migration	Yes (CRIU)	Yes (KVM)
Stateful Snapshots	Yes (with CRIU)	Yes (memory + disk)
Cold Migration	Fast (rsync)	Slower (full disk)
Downtime	Milliseconds	Seconds

Understanding the trade-offs between containers and VMs allows you to design efficient, secure, and performant infrastructure with Incus. The unified management interface means you can use both technologies seamlessly.

Incus Reference Guide