Development Tasks

Primary Task	Establish SSH connection with port forwarding and execute `hello_world.py`	10min
Optional Task 1	Execute fundamental Linux commands on the development machine	10min
Optional Task 2	Connect to development machine remotely using VSCode and create a conda environment	10min
Optional Task 3	Create and execute a `test.sh` script	10min

Implementation Guide

Creating Development Machine

Generating SSH Keys Locally

In your terminal, execute the following command to generate an SSH key pair:

ssh-keygen

Configuring Public Key in Development Platform

Copy the generated public key to your development platform. The public key is typically stored at:

~/.ssh/id_rsa.pub

You can view the file contents using the built-in `cat` command:

cat ~/.ssh/id_rsa.pub

Copy this SSH public key to your development platform. This key is shared across all development machines and only needs to be configured once unless you change your local machine.

Connecting to Development Machine via SSH

Navigate to the development machine list and locate the SSH command (port, username, IP).

Use the VSCode Remote-SSH plugin to connect to the development machine. After the initial successful connection, wait for some time as it synchronizes local plugins and configurations to the development machine (~/.vscode directory).

Successfully connected to the remote server!

Port Forwarding

Port forwarding is a networking technique that maps external network ports to internal network ports, enabling communication between external and internal networks. Through port forwarding, services or applications within an internal network can be accessed from an external network, facilitating convenient communication across networks.

Why do we need port forwarding when using development machines? In subsequent courses, we'll deploy model web demos, which may encounter issues with incomplete web UI loading. This happens because when running web demos in the development machine's Web IDE, directly accessing http/https services on the development machine might encounter proxy problems, causing external UI resources to not load completely.

To solve this issue, we need to perform port forwarding for the web demo connection, mapping the external link to our local machine. We then use the local connection to access the service, resolving the proxy problem. Let's practice this now.

ssh -p 37367 root@ssh.intern-ai.org.cn -CNg -L {LOCAL_PORT}:127.0.0.1:{REMOTE_PORT} -o StrictHostKeyChecking=no

Here's an explanation of each part of the command:

• -p 37367: Specifies the SSH connection port as 37367.
• root@ssh.intern-ai.org.cn: Indicates connecting to the host ssh.intern-ai.org.cn as the root user.
• -CNg:
  • -C typically enables compression.
  • -N means no remote command execution, only establishing connections for port forwarding.
  • -g allows remote hosts to connect to locally forwarded ports.
• -L {LOCAL_PORT}:127.0.0.1:{REMOTE_PORT}: Sets up local port forwarding, mapping a specific port on the local machine to the 127.0.0.1 address and specified port on the remote machine.
• -o StrictHostKeyChecking=no: Disables strict host key checking, avoiding prompts or errors due to unknown host keys during the first connection.

When running a web demo, you can use this command for port forwarding. For example:

Create a hello_world.py file with the following content:

import socket
import re
import gradio as gr

# Get hostname
def get_hostname():
    hostname = socket.gethostname()
    match = re.search(r'-(\d+)$', hostname)
    name = match.group(1)
    return name

# Create Gradio interface
with gr.Blocks(gr.themes.Soft()) as demo:
    html_code = f"""
            <p align="center">
            <a href="https://intern-ai.org.cn/home">
                <img src="https://intern-ai.org.cn/assets/headerLogo-4ea34f23.svg" alt="Logo" width="20%" style="border-radius: 5px;">
            </a>
            </p>
            <h1 style="text-align: center;">☁️ Welcome {get_hostname()} user, welcome to the LLM Practical Workshop!</h1>
            <h2 style="text-align: center;">😀 Let's explore machine learning together.</h2>
            <p align="center">
                <a href="https://github.com/Example/Tutorial/blob/main">
                    <img src="https://example.com/assets/logo.jpg" alt="Logo" width="20%" style="border-radius: 5px;">
                </a>
            </p>
            """
    gr.Markdown(html_code)

demo.launch()

Before running the code, install the required dependency package using:

pip install gradio==4.29.0

Then run the hello_world.py in the terminal of the Web IDE.

Essential Linux Commands

Creating Files with touch

The touch command allows you to create files quickly without manual creation. For example, to create a demo.py file:

touch demo.py

Creating Directories with mkdir

Similarly, use the mkdir command to create directories. To create a directory named test:

mkdir test

Navigating Directories with cd

Change to the test directory:

cd test

Displaying Current Directory with pwd

The pwd command shows the current working directory, helping you determine your location in the file system:

pwd

Copying Files and Creating Links with cp and ln

The cp command is frequently used in subsequent courses to copy files or directories to another location. Common usage includes:

• Copy file: cp source_file target_file
• Copy directory: cp -r source_directory target_directory

However, when working with models, this operation consumes significant disk space. Therefore, we typically use the ln command, similar to shortcuts in Windows. Linux has two types of links: hard links and symbolic links. Hard links allow a file to have multiple names, while symbolic links create a special file that points to another file's location. Hard links exist within the same file system, while symbolic links can span different file systems.

Moving and Renaming Files with mv

The mv command is similar to rm but is used to move or rename files and directories. The rm command is used to delete files or directories.

Common mv command parameters:

• -i: Interactive mode, asks before overwriting.
• -f: Force overwrite.
• -u: Only moves if the source is newer than the target.

Usage examples:

• mv file1.txt dir1/: Moves file1.txt to directory dir1.
• mv file1.txt file2.txt: Renames file1.txt to file2.txt.

Common rm command parameters:

• -i: Interactive mode, asks before deletion.
• -f: Force deletion, ignores non-existent files without confirmation.
• -r: Recursively deletes directories and their contents.

Usage examples:

• rm file.txt: Deletes file.txt.
• rm -r dir1/: Recursively deletes directory dir1 and all its contents.

The rmdir command can also be used to delete directories.

Setting Up Conda Environment in VSCode

Check the current conda version on the development machine:

Creating a Virtual Environment

conda create -n ml_env python=3.8

Listing Available Environments

conda env list

Activating and Exiting Virtual Environments

conda activate ml_env  # Enter environment
conda deactivate ml_env  # Exit environment