*Guidelines for provisioning large quantities of devices efficiently*

This document provides guidance for scaling provisioning operations from single devices to high-volume production deployments.

# Overview

The provisioning system supports scalable deployment from individual devices to high-volume manufacturing operations. This guide covers:

- Connection requirements and infrastructure

- Provisioning strategies and workflow optimization

- Performance characteristics and capacity planning

- Hardware requirements for scaling

- Common issues and solutions

Target audience: Manufacturing engineers, production managers, and operations teams planning mass deployment.

# Connection Requirements

## USB Connection (Mandatory)

USB connectivity is **mandatory** for all provisioning operations as it provides:

- **Secure Boot Enablement**: USB connection is the only supported method for configuring secure boot on Raspberry Pi devices

- **System Bootstrapping**: Initial system configuration and image deployment requires USB connectivity

- **Security Isolation**: Point-to-point USB provides a controlled, isolated communication channel for sensitive provisioning operations

Each provisioning head requires a dedicated USB connection to the target Raspberry Pi device.

## Ethernet Connection (Optional)

Ethernet connectivity is **optional** and provides **only one benefit**:

- **Accelerated Image Transfer**: Can supplement USB for faster image transfer during provisioning

**Note**: Ethernet connection does not provide any other provisioning benefits beyond accelerated image transfer.

### Network Requirements for Ethernet

When utilizing Ethernet connectivity:

- **DHCPv4/v6 Support**: Network must provide automatic DHCPv4/v6 address assignment

- **Routing**: Network must be routable to the machine running rpi-sb-provisioner

- **Firewall**: Ensure provisioning service ports are accessible (refer to API endpoints documentation)

- **Bandwidth**: Sufficient bandwidth to support concurrent image transfers during provisioning

# Provisioning Strategies

## Pipelined Provisioning (Recommended)

**Pipelined provisioning** is the most efficient approach for mass operations:

- **Continuous Flow**: Devices move through provisioning stages in sequence

- **Parallel Processing**: Multiple devices at different provisioning stages simultaneously

- **Resource Optimization**: Maximizes utilization of provisioning infrastructure

- **Reduced Idle Time**: Minimizes operator and equipment downtime

### Practical Pipeline Operation

Pipelined provisioning connects devices in sequence while earlier devices complete provisioning automatically. Device preparation time (unboxing, alignment, cable connection) typically matches or exceeds individual device provisioning time, enabling efficient workflow.

**Example Pipeline (2.5-minute provisioning time per device):**

| Time | Operator Action                                      |
|------|------------------------------------------------------|
| 0:00 | Connect Device 1 (provisioning starts automatically) |
| 0:30 | Connect Device 2 (Device 1 continues in background)  |
| 1:00 | Connect Device 3 (Devices 1 & 2 continue)            |
| 1:30 | Connect Device 4 (Devices 1–3 continue)              |
| 2:00 | Connect Device 5 (Devices 1–4 continue)              |
| 2:30 | Remove Device 1 (complete), Connect Device 6         |

Once the pipeline is established, operators maintain steady workflow without waiting for individual device completion. The provisioning duration becomes the throughput bottleneck rather than operator availability.

## Batch Provisioning (Less Efficient)

**Batch provisioning** processes groups of devices simultaneously:

- **Synchronous Processing**: All devices in batch must complete before next batch begins

- **Resource Contention**: Higher peak resource usage during active phases

- **Idle Periods**: Significant downtime between batches

- **Scaling Limitations**: More difficult to scale beyond initial batch size

# Performance Characteristics

## Measured Capacity

Testing with a 2.6GB system image:

| Metric                           | Value                                         |
|----------------------------------|-----------------------------------------------|
| **Provisioning time**            | Approximately 2.5 minutes per device          |
| **Operator capacity**            | Up to 5 provisioning heads per operator       |
| **Throughput (single operator)** | ~650 devices per 8-hour shift (with pipeline) |

## Performance Variables

Performance varies based on:

- **Image size:** Provisioning time increases proportionally with image size

- **Security mode:** `secure-boot` and `fde-only` modes take longer than `naked` mode due to encryption operations

- **Storage type:** NVMe is generally faster than eMMC or SD card

- **Network connectivity:** Ethernet accelerates image transfer but not storage write operations

- **Device preparation:** Physical handling time affects pipeline efficiency

## Throughput Optimization

To maximize provisioning throughput:

1.  **Optimize Image Size**: Minimize system image size while maintaining required functionality

2.  **Parallel Operations**: Utilize multiple provisioning heads per operator

3.  **Storage Performance**: Use high-speed storage (NVMe SSD) for image hosting

4.  **Network Optimization**: Configure low-latency, high-bandwidth network infrastructure

5.  **Process Standardization**: Develop standardized operator procedures and workflows

# Infrastructure Considerations

## Hardware Requirements

### Provisioning Server

- **Platform:** Raspberry Pi hardware (required - system does not run on x86 or other architectures)

- **Recommended model:** Raspberry Pi 5 with maximum available RAM

- **Storage:** NVMe SSD for image hosting (significantly faster than SD card)

- **Power supply:** Official Raspberry Pi 27W USB C power supply

- **Operating system:** Raspberry Pi OS Bookworm or later

### Infrastructure

- **Network:** Gigabit Ethernet minimum for multi-device operations

- **Switching:** Sufficient port capacity for all devices plus provisioning server

- **USB hubs:** Official Raspberry Pi powered USB hubs (recommended for reliability)

- **Power distribution:** Adequate capacity for all connected devices

- **Workspace:** Organized layout with proper cable management and labeling

## USB Scaling Solutions

For scaling USB connectivity in mass provisioning operations, the **official Raspberry Pi powered USB hub** is the recommended solution:

- **Validated Hardware**: Officially tested and certified for Raspberry Pi device connectivity

- **Reliable Power Delivery**: Provides consistent power to multiple connected devices

- **Proven Performance**: Designed to handle the specific power and data requirements of Raspberry Pi provisioning

- **Simplified Deployment**: Reduces complexity compared to generic USB hub solutions

- **Support and Warranty**: Backed by official Raspberry Pi support channels

When expanding provisioning capacity, use multiple official Raspberry Pi powered USB hubs rather than daisy-chaining or using untested hub solutions to ensure reliable operation and minimize connectivity issues.

## Quality Assurance

- **Verification Procedures**: Automated testing of provisioned devices

- **Error Handling**: Robust error detection and recovery procedures

- **Audit Trail**: Complete logging of all provisioning operations

- **Batch Tracking**: Device serial number and configuration tracking

# Troubleshooting Common Scaling Issues

## Performance Bottlenecks

- **USB Bandwidth Saturation**: Connect both USB and Ethernet to target devices and allow Ethernet to perform accelerated image transfer while USB handles secure boot configuration. Note that Ethernet only accelerates image transfer time, not the time required to write data to storage

- **Network Congestion**: Implement QoS policies or network segmentation

- **Storage I/O Limits**: Upgrade to faster storage subsystem. Consider using NVMe SSD for image hosting

- **Memory Constraints**: Consider using a Raspberry Pi device with the largest amount of RAM available

## Operational Challenges

- **Device Detection Issues**: Check USB connections and power delivery

- **Network Connectivity Problems**: Verify DHCP and routing configuration

- **Image Corruption**: Verify image integrity using checksums - rpi-sb-provisioner includes a webUI that records the supplied image checksum for verification

- **Process Synchronization**: Establish clear operator procedures and checkpoints

# Security Considerations

- **Network Isolation**: Use dedicated provisioning networks when possible

- **Access Controls**: Implement appropriate authentication and authorization

- **Audit Logging**: Maintain comprehensive logs of all provisioning activities

- **Physical Security**: Ensure secure physical access to provisioning infrastructure

- **Key Management**: Secure handling of cryptographic keys and certificates

# Monitoring and Metrics

Track key performance indicators:

- **Provisioning Rate**: Devices provisioned per hour/day

- **Error Rate**: Failed provisioning attempts and root causes

- **Resource Utilization**: CPU, memory, network, and storage usage

- **Operator Efficiency**: Time per device and process bottlenecks

- **Quality Metrics**: Post-provisioning verification success rates

# Planning Your Deployment

## Small-Scale Operations (1–50 devices)

- Single provisioning head with manual workflow

- Direct USB connection, Ethernet optional

- Standard workspace with basic cable management

- Estimated setup time: 30-90 minutes

- No special infrastructure required

## Medium-Scale Operations (50–500 devices)

- Multiple provisioning heads (3–7 recommended)

- Pipelined provisioning workflow

- Powered USB hub for multi-device support

- Gigabit Ethernet recommended

- Dedicated workspace with organized layout

- Estimated setup time: 1 day

- Consider manufacturing database for tracking

## Large-Scale Operations (500+ devices)

- Multiple provisioning servers with load distribution

- Fully pipelined operations with dedicated operators

- Network infrastructure with QoS policies

- Automated quality assurance integration

- Manufacturing execution system (MES) integration via API

- Comprehensive audit and compliance tracking

- Estimated setup time: 1–2 weeks

- Requires detailed process documentation

## Key Success Factors

- **Process standardization:** Document and train operators on consistent procedures

- **Quality control:** Implement verification steps at critical points

- **Monitoring:** Track throughput, error rates, and resource utilization

- **Continuous improvement:** Analyze bottlenecks and optimize workflow

- **Infrastructure investment:** Quality equipment reduces downtime and improves reliability

# Summary

Efficient mass provisioning requires:

- **Connection strategy:** USB required, Ethernet optional for acceleration

- **Workflow optimization:** Pipelined provisioning maximizes throughput

- **Infrastructure planning:** Quality hardware and organized workspace

- **Process discipline:** Standardized procedures and quality controls

- **Monitoring:** Track performance and continuously improve

For implementation details:

- Configuration options: See [Configuration Reference](../config_vars.md)

- API integration: See [API Documentation](../api_endpoints.md)

- Device-specific procedures: See [Device Guidance](../device-guidance/)
