CAMEL: Communicative Agents for “Mind” Exploration of Large Scale Language Model Society

Community | Installation | Documentation | Examples | Paper | Citation | Contributing | CAMEL-AI

Core Concept

As large language models (LLMs) exhibit superior performance in conversational tasks, there's a growing dependency on constant human direction to achieve complex objectives. This project investigates scalable methodologies to foster autonomous cooperation among communicative artificial entities, offering insights into their collective 'cognitive' mechanisms. We introduce a novel architectural paradigm termed role-playing within our communicative agent framework. This system employs inception prompting to guide agents through multi-step task resolution while ensuring adherence to initial human specifications. Role-playing facilitates the large-scale generation of conversational datasets suitable for rigorous analysis of multi-agent system behaviors. Key contributions include the framework itself, a mechanism for scalable study of cooperative AI, and the public release of our associated library to advance research in agent interaction.

Community Engagement

🐫 CAMEL is an open-source library fundamentally created for empirical study of self-directing, communicative artificial entities. We posit that large-scale simulation of these agents yields critical understandings of their capabilities, operational norms, and potential risks. To support this endeavor, we provide implementations for diverse agent types, tasks, prompt structures, model interfaces, and simulated operational contexts.

Join our growing collective via Slack, Discord, or WeChat as we collectively advance the frontier of constructing artificial intelligence societies.

Quick Start Demo

Execute the provided notebook demonstrating a dialogue between two specialized ChatGPT agents—a Python Developer and a Stock Trader—collaborating to engineer a stock market analysis bot.

Software Acquisition

Via Python Package Index (PyPI)

Install the core CAMEL package using: pip install camel-ai

Optional dependencies are available for specific functionalities:

For Hugging Face agent support, execute: pip install camel-ai[huggingface-agent]

To utilize Retrieval-Augmented Generation (RAG) capabilities or agent persistence mechanisms, run: pip install camel-ai[tools]

For a complete installation encompassing all features: pip install camel-ai[all]

From Source Repository

Installation from source using Poetry (Recommended): sh

Verify Python version is 3.9 or higher (pyenv recommended for version management)

Clone the repository

git clone https://github.com/camel-ai/camel.git

Navigate into the project directory

cd camel

Initialize the virtual environment

poetry shell

Install CAMEL and its dependencies (approx. 90s resolution time)

poetry install

Include Hugging Face agent extensions (Optional)

poetry install -E huggingface-agent

Proceed with using CAMEL

Deactivate the environment

exit

Installation from source using Conda and pip: sh

Set up a new conda environment

conda create --name camel python=3.10

Activate the environment

conda activate camel

Clone the repository (specific version tag shown)

git clone -b v0.1.0 https://github.com/camel-ai/camel.git

Navigate into the project directory

cd camel

Install CAMEL in editable mode

pip install -e .

Include Hugging Face agent extensions (Optional)

pip install -e .[huggingface-agent]

Operational Documentation

Detailed API and library documentation is available at: CAMEL package documentation pages

Illustrative Examples

A catalog of role pairings (Assistant vs. User roles) and associated task descriptions can be accessed here.

To execute the core role-playing demonstration:

First, set your OpenAI access credentials in the system environment variables. The method varies based on your operating system and shell:

Bash (Linux, macOS, Git Bash on Windows):

bash

Set OpenAI credentials

export OPENAI_API_KEY= export OPENAI_API_BASE_URL= #(Use this if routing through an OpenAI-compatible proxy)

Windows Command Prompt:

cmd REM Set OpenAI credentials set OPENAI_API_KEY= set OPENAI_API_BASE_URL= #(Use this if routing through an OpenAI-compatible proxy)

Windows PowerShell:

powershell

Set OpenAI credentials

$env:OPENAI_API_KEY="" $env:OPENAI_API_BASE_URL="" #(Use this if routing through an OpenAI-compatible proxy)

Ensure the API key placeholder is replaced with your actual secret key, avoiding extraneous spaces around the assignment operator. Once keys are configured, execute the script:

bash

The role configuration and initial task prompt are defined within role_playing.py

python examples/ai_society/role_playing.py

Note: Environment variable settings are transient and only apply to the current terminal session.

Utilizing Open-Source Model Backends

Integration with self-hosted, open-source models relies on an external inference service configured to expose OpenAI-compatible APIs, specifically supporting the openai.ChatCompletion.create endpoint. We decoupled CAMEL from any specific server implementation, requiring users to deploy their preferred backend.

Below are instructions demonstrating configuration with FastChat serving a LLaMA2 variant (meta-llama/Llama-2-7b-chat-hf):

1. Prior to running CAMEL, initiate the FastChat service components. These steps must run in separate, concurrent processes:

sh

Start the coordination process

python -m fastchat.serve.controller

Initialize the model serving node

python3 -m fastchat.serve.model_worker --model-path meta-llama/Llama-2-7b-chat-hf

Activate the local RESTful interface

python3 -m fastchat.serve.openai_api_server --host localhost --port 8000

1. Confirm the controller logs indicate successful registration of the model worker. The endpoint will then be accessible, typically at http://localhost:8000/v1.

2. Run the demonstration script role_playing_with_open_source_model.py, configuring agents to point to this local endpoint via the OpenSourceConfig:

python system_message = # ...

agent_kwargs = dict( model=model_type, model_config=OpenSourceConfig( model_path="meta-llama/Llama-2-7b-chat-hf", server_url="http://localhost:8000/v1", ), )

agent = ChatAgent( system_message, **agent_kwargs, )

Supported Inference Engines

• Models derived from LLaMA2 (e.g., meta-llama/Llama-2-7b-chat-hf)
• Models derived from Vicuna (e.g., lmsys/vicuna-7b-v1.5)

Data Resources (Via Hugging Face)

We host several curated datasets generated using the framework, categorized by domain:

Dataset	Primary Interaction Format	Instruction-Based Format	Translated Chat Format
AI Society	Chat Archive	Instruction Set	Translated Chat
Code Generation	Chat Archive	Instruction Set	N/A
Mathematics	Chat Archive	N/A	N/A
Physics	Chat Archive	N/A	N/A
Chemistry	Chat Archive	N/A	N/A
Biology	Chat Archive	N/A	N/A

Visualization of Data Structures

Interactive visualizations of the task instructions and generated conversational sequences are provided via Nomic Atlas:

Dataset	Instruction Maps	Task Execution Maps
AI Society	Instructions View	Tasks View
Code	Instructions View	Tasks View
Misalignment	Instructions View	Tasks View

Implemented External Research Concepts

We have integrated modules reflecting promising research methodologies from external sources, encouraging benchmarking and customization. Proper attribution to the original authors is requested if these components are utilized: - Agent structures such as TaskCreationAgent, TaskPrioritizationAgent, and the BabyAGI pattern, originating from Nakajima et al.: Task-Driven Autonomous Agent. [Demonstration]

Project Milestones

• Public release of the AI Society and Code datasets (April 2, 2023)
• Initial public availability of the CAMEL Python library (March 21, 2023)

Citation Information

@inproceedings{li2023camel, title={CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society}, author={Li, Guohao and Hammoud, Hasan Abed Al Kader and Itani, Hani and Khizbullin, Dmitrii and Ghanem, Bernard}, booktitle={Thirty-seventh Conference on Neural Information Processing Systems}, year={2023} }

Acknowledgements

We extend sincere gratitude to Nomic AI for granting extensive access to their data visualization platform, Atlas.

We also thank Haya Hammoud for developing the project's visual identity (logo).

Licensing Terms

CAMEL is strictly intended and licensed for academic and research applications only.

The source code falls under the Apache 2.0 license.

The accompanying datasets are distributed under the CC BY NC 4.0 license, strictly permitting non-commercial use. Users developing models with this data must restrict their model deployment to research activities.

Collaboration on CAMEL 🐫

We warmly welcome external contributions to this open-source project. Detailed instructions for submitting contributions are provided in our contributing guidelines document. Please consult this guide to ensure seamless collaboration. 🤝🚀

Contact Information

For further inquiries, please reach out to camel.ai.team@gmail.com.