Install Now
How to install karpathy/minbpe. Official installation instructions. Project installation guide. Framework or library installation. Build installation instructions. How to setup and install from GitHub. Installation steps and setup instructions. Official docs and installation instructions GitHub.
karpathy/minbpe Installation Guide
How to install karpathy/minbpe. Official project installation instructions and setup guide.
⚡
Automated Install (Recommended)
Quick installation instructions for karpathy/minbpe. This is the fastest way to complete project installation and setup.
Install via curl
curl -fsSL https://hexmos.com/ipm-install | bash &&
ipm i karpathy/minbpeor
Install via npx
npx @hexmos/ipm i karpathy/minbpePrerequisites
Python
languageVersion: >=3.8
pip
package_managerregex
libraryRequired for the RegexTokenizer splitting patterns.
tiktoken
libraryRequired for parity tests and GPT4Tokenizer data.
Manual Installation Methods
Manual installation instructions. How to install from GitHub source.
Developer Setup
git clone https://github.com/karpathy/minbpe.git
cd minbpe
pip install -r requirements.txt
pip install pytest tiktokenPost Installation Steps
- Verify the installation by running the test suite: `pytest -v .`
- Run the sample training script: `python train.py`
- To use in your own code, import the tokenizer: `from minbpe import BasicTokenizer`