Interactive Demo Tasks

• Deploy the InternLM2-Chat-1.8B locally for intelligent conversations

Assignments

Basic Assignment

Generate a 300-word short story using the InternLM2-Chat-1.8B model.

Advanced Assignment

Download the config.json file for InternLM-Chat-7B using the huggingface_hub package.

import os 
from huggingface_hub import hf_hub_download

# Download configuration file
hf_hub_download(repo_id="internlm/internlm2-7b", filename="config.json")

Enviroment Setup

Create a Python virtual environment using conda and install required dependencies:

conda create -n internlm_demo python==3.10 -y
conda activate internlm_demo
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.7 -c pytorch -c nvidia
pip install huggingface-hub==0.17.3
pip install transformers==4.34
pip install psutil==5.9.8
pip install accelerate==0.24.1
pip install streamlit==1.32.2
pip install matplotlib==3.8.3
pip install modelscope==1.9.5
pip install sentencepiece==0.1.99

Deploying InternLM2-Chat-1.8B for Intelligent Conversation

Download the Model

Create a directory to save the model:

mkdir -p ~/internlm_workspace
touch ~/internlm_workspace/download_model.py
cd ~/internlm_workspace

Use modelscope's snapshot_download to download the model. Open the ~/internlm_workspace/download_model.py file and copy the following code:

import os
from modelscope.hub.snapshot_download import snapshot_download

# Create directory for model storage
os.makedirs("~/model_storage", exist_ok=True)

# save_dir is the local directory for storing the model
save_dir = "~/model_storage"

snapshot_download("Shanghai_AI_Laboratory/internlm2-chat-1_8b", 
                  cache_dir=save_dir, 
                  revision='v1.1.0')

Execute the command to downloda the model parameters:

python ~/internlm_workspace/download_model.py

Load Model and Initiate Conversation

Create a demo file:

touch ~/internlm_workspace/chat_interface.py

Open the ~/internlm_workspace/chat_interface.py file and copy the following code:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_path = "~/model_storage/Shanghai_AI_Laboratory/internlm2-chat-1_8b"

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, device_map='cuda:0')
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='cuda:0')
model = model.eval()

initial_prompt = """You are an AI assistant named InternLM.
- InternLM is a conversational language model developed by Shanghai AI Laboratory.
- It is designed to be helpful, honest, and harmless.
- InternLM can understand and communicate fluently in languages such as English and Chinese.
"""

conversation_history = [(initial_prompt, '')]

print("=============Welcome to InternLM chat interface. Type 'quit' to exit.=============")

while True:
    user_input = input("\nUser >>> ")
    user_input = user_input.strip()
    if user_input.lower() == "quit":
        break

    response_length = 0
    for response, _ in model.stream_chat(tokenizer, user_input, conversation_history):
        if response is not None:
            print(response[response_length:], flush=True, end="")
            response_length = len(response)

Run the demo program with the following command:

python ~/internlm_workspace/chat_interface.py

Wait for the model to load (approximately 5GB GPU memory), then enter a prompt like: "Please create a 300-word short story."