exo-explore/exo Installation Guide

⚡

Automated Install (Recommended)

Quick installation instructions for exo-explore/exo. This is the fastest way to complete project installation and setup.

Install via curl

curl -fsSL https://hexmos.com/ipm-install | bash && 
ipm i exo-explore/exo

Install via PowerShell

iwr https://hexmos.com/ipm-install-ps -UseBasicParsing | iex; 
ipm i exo-explore/exo

Install via npx

npx @hexmos/ipm i exo-explore/exo

Prerequisites

Python

language

Version: >=3.12.0

NVIDIA CUDA Toolkit

build_tool

cuDNN Library

library

Docker

system_tool

Git

system_tool

Manual Installation Methods

Source Code Installation

git clone https://github.com/exo-explore/exo.git

cd exo

pip install -e .

Run from Source (macOS)

git clone https://github.com/exo-explore/exo.git

cd exo

cd dashboard && npm install && npm run build && cd ..

uv run exo

Run from Source (Linux)

git clone https://github.com/exo-explore/exo.git

cd exo

cd dashboard && npm install && npm run build && cd ..

uv run exo

macOS App

Download https://assets.exolabs.net/EXO-latest.dmg

Uninstalling the macOS App

Open the app, click the menu bar icon -> Advanced -> Uninstall.

sudo ./app/EXO/uninstall-exo.sh

Enabling RDMA on macOS

Shut down your Mac.

Hold down the power button for 10 seconds until the boot menu appears.

Select 'Options' to enter Recovery mode.

Open Terminal from the Utilities menu.

rdma_ctl enable

Reboot your Mac.

sh tmp/set_rdma_network_config.sh

Nix Quick Start (macOS)

nix run .#exo

macOS Prerequisites

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

brew install uv macmon node

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

rustup toolchain install nightly

Linux Prerequisites (Ubuntu/Debian)

sudo apt update && sudo apt install nodejs npm

curl -LsSf https://astral.sh/uv/install.sh | sh

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

rustup toolchain install nightly

Linux Prerequisites (Homebrew)

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

brew install uv node

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

rustup toolchain install nightly

Configure Nix for macOS

echo "trusted-users = root $(whoami)" | sudo tee -a /etc/nix/nix.conf

echo "experimental-features = nix-command flakes" | sudo tee -a /etc/nix/nix.conf

sudo launchctl kickstart -k system/org.nixos.nix-daemon

Run from Source Configuration Options (Linux)

uv run exo --no-worker

macOS App Installation

Download https://assets.exolabs.net/EXO-latest.dmg

macOS App Configuration

Set the EXO_LIBP2P_NAMESPACE environment variable to your desired namespace.

Enabling RDMA on macOS

Shut down your Mac.

Hold down the power button for 10 seconds until the boot menu appears.

Select 'Options' to enter Recovery mode.

Open Terminal from the Utilities menu.

rdma_ctl enable

Reboot your Mac.

sh tmp/set_rdma_network_config.sh

Environment Variables

export EXO_MODELS_PATH=/mnt/nfs/models:/opt/ai-models && uv run exo

export EXO_OFFLINE=true && uv run exo

export EXO_ENABLE_IMAGE_MODELS=true && uv run exo

export EXO_LIBP2P_NAMESPACE=my-dev-cluster && uv run exo

API - Create a model instance

curl -X POST http://localhost:52415/instance -H 'Content-Type: application/json' -d '{ "instance": {...} }'

API - Send a chat completion

curl -N -X POST http://localhost:52415/v1/chat/completions -H 'Content-Type: application/json' -d '{ "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", "messages": [{"role": "user", "content": "What is Llama 3.2 1B?"}], "stream": true }'

API - Claude Messages API Compatibility

curl -N -X POST http://localhost:52415/v1/messages -H 'Content-Type: application/json' -d '{ "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 1024, "stream": true }'

API - OpenAI Responses API Compatibility

curl -N -X POST http://localhost:52415/v1/responses -H 'Content-Type: application/json' -d '{ "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", "messages": [{"role": "user", "content": "Hello"}], "stream": true }'

API - Ollama API Compatibility

curl -X POST http://localhost:52415/ollama/api/chat -H 'Content-Type: application/json' -d '{ "model": "mlx-community/Llama-3.2-1B-Instruct-4bit", "messages": [{"role": "user", "content": "Hello"}], "stream": false }'

curl http://localhost:52415/ollama/api/tags

API - Custom Model Loading from HuggingFace

curl -X POST http://localhost:52415/models/add -H 'Content-Type: application/json' -d '{ "model_id": "mlx-community/my-custom-model" }'

Benchmarking

uv run bench/exo_bench.py --model Llama-3.2-1B-Instruct-4bit --pp 128,256,512 --tg 128,256

Post Installation Steps

Open your browser and navigate to http://localhost:52415. You should see the ChatGPT-like WebUI.
To test the API, send a request to http://localhost:52415/v1/chat/completions with JSON data in the body. For example, you can use curl to send a request like this: ```bash curl -X POST -H "Content-Type: application/json" -d '{ "model": "llama-3.2-3b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "What is the meaning of exo?" } ] } ]" }' http://localhost:52415/v1/chat/completions ```
For developers, exo also starts a ChatGPT-compatible API endpoint on http://localhost:52415/v1/chat/completions. You can use this to send requests and receive responses from the model.

← Back to tool All Installation Guides