Understanding and Implementing Prompt Engineering

Prompt engineering, also known as instruction engineering, is the practice of designing and refining input prompts to guide large language models (LLMs) toward generating desired outputs. It involves crafting precise instructions that leverage the model's capabilities to solve specific problems or perform complex tasks. A well-constructed prompt can significantly enhance the accuracy, relevance, and structure of the model's responses.

Core Principles of Effective Prompting

Effective prompt engineering relies on several key principles:

• Clarity and Specificity: Prompts should be unambiguous and clearly define the task. Vague instructions often lead to inconsistent or irrelevant results.
• Role Definition: Assigning a specific role to the AI (e.g., "You are a software engineer," "You are a financial analyst") helps contextualize the task and improves the quality of the response.
• Context Provision: Providing relevant background information or constraints ensures the model understands the scope and boundaries of the task.
• Example Inclusion: Demonstrating desired output formats or providing few-shot examples (in-context learning) can guide the model toward producing structured and accurate results.
• Output Formatting: Specifying the desired output format (e.g., JSON, XML) allows for easier integration with downstream systems and automated processing.

Practical Application: Building an AI-Powered Recruitment Assistant

Let's explore how to build a system that evaluates candidate responses for an AI Algorithm Engineer posision using prompt engineering. The system will assess candidates based on education, experience, project count, and attitude, and determine if they meet the hiring criteria.

Step 1: Define Evaluation Criteria

First, we establish the scoring rubric for each criterion:

A candidate is considered qualified if their total score is 30 or higher.

Step 2: Implementing the Natural Language Understanding (NLU)

We'll use the OpenAI API too process candidate responses and generate structured evaluations. The following Python code demonstrates this process.

# Import necessary libraries
from openai import OpenAI
import os
from dotenv import load_dotenv, find_dotenv

# Load environment variables from .env file
load_dotenv(find_dotenv())
api_key = os.getenv("OPENAI_API_KEY")

# Initialize the OpenAI client
client = OpenAI(api_key=api_key)

def evaluate_candidate(candidate_response: str) -> str:
    """
    Evaluates a candidate's response based on predefined criteria using the OpenAI API.

    Args:
        candidate_response: The text input from the candidate.

    Returns:
        A JSON string containing the evaluation results.
    """
    # Define the instruction for the AI model
    instruction = """
    Your task is to evaluate a candidate's response for an AI Algorithm Engineer position.
    Score the candidate based on the following criteria:
    - Education: PhD (9), Master's (8), Bachelor's (7)
    - Experience: 10+ years (9), 7-9 years (8), 5-6 years (7)
    - Project Count: 10+ projects (9), 7-9 projects (8), 4-6 projects (7)
    - Attitude: Excellent (9), Very Good (8), Good (7)

    Attitude criteria:
    - Excellent: Deep understanding of company business and culture, high enthusiasm, professional demeanor.
    - Very Good: Relevant experience and skills, confidence in performance, interest in products/services.
    - Good: Some company knowledge, willingness to learn, interest in the position, prepared for challenges.

    Calculate the total score. If the total score is 30 or higher, the candidate is qualified; otherwise, they are not.
    Provide a JSON output with the following keys: education, experience, project_count, attitude, total_score, is_qualified, and reason (if not qualified).
    """

    # Define the desired output format
    output_format = """
    {
        "education": "score as integer",
        "experience": "score as integer",
        "project_count": "score as integer",
        "attitude": "score as integer",
        "total_score": "sum of all scores as integer",
        "is_qualified": "true or false",
        "reason": "explanation if not qualified, otherwise null"
    }
    """

    # Construct the prompt
    prompt = f"""
    {instruction}

    {output_format}

    Candidate Response:
    {candidate_response}
    """

    # Call the OpenAI API
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,  # Deterministic output
    )

    # Return the model's response
    return response.choices[0].message.content

# Example usage
candidate_input = """
I hold a Master's degree and have 6 years of experience.
I have completed 8 practical projects.
I am very familiar with your company's business and culture and am eager to join your team.
"""
evaluation_result = evaluate_candidate(candidate_input)
print(evaluation_result)

Step 3: Refining the Prompt with Examples

To improve accuracy, we can include examples of desired outputs within the prompt. This technique, known as few-shot learning, helps the model udnerstand the expected format and content.

def evaluate_candidate_with_examples(candidate_response: str) -> str:
    """
    Evaluates a candidate's response with few-shot examples for improved accuracy.

    Args:
        candidate_response: The text input from the candidate.

    Returns:
        A JSON string containing the evaluation results.
    """
    instruction = """
    Your task is to evaluate a candidate's response for an AI Algorithm Engineer position.
    Score the candidate based on the following criteria:
    - Education: PhD (9), Master's (8), Bachelor's (7)
    - Experience: 10+ years (9), 7-9 years (8), 5-6 years (7)
    - Project Count: 10+ projects (9), 7-9 projects (8), 4-6 projects (7)
    - Attitude: Excellent (9), Very Good (8), Good (7)

    Attitude criteria:
    - Excellent: Deep understanding of company business and culture, high enthusiasm, professional demeanor.
    - Very Good: Relevant experience and skills, confidence in performance, interest in products/services.
    - Good: Some company knowledge, willingness to learn, interest in the position, prepared for challenges.

    Calculate the total score. If the total score is 30 or higher, the candidate is qualified; otherwise, they are not.
    Provide a JSON output with the following keys: education, experience, project_count, attitude, total_score, is_qualified, and reason (if not qualified).

    Examples:
    1. Candidate: "I have a PhD, 12 years of experience, and 15 projects. I know your company well." -> Qualified
    2. Candidate: "I have a Bachelor's, 3 years of experience, and 2 projects. I'm interested." -> Not Qualified
    """

    output_format = """
    {
        "education": "score as integer",
        "experience": "score as integer",
        "project_count": "score as integer",
        "attitude": "score as integer",
        "total_score": "sum of all scores as integer",
        "is_qualified": "true or false",
        "reason": "explanation if not qualified, otherwise null"
    }
    """

    prompt = f"""
    {instruction}

    {output_format}

    Candidate Response:
    {candidate_response}
    """

    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
    )

    return response.choices[0].message.content

# Example usage with few-shot learning
candidate_input_with_examples = """
I have a Bachelor's degree and 2 years of experience at Penguin Inc.
I have worked on 1 project.
I am familiar with your company's business and culture and am very interested in this position.
"""
evaluation_result_examples = evaluate_candidate_with_examples(candidate_input_with_examples)
print(evaluation_result_examples)

Step 4: Handling Multi-Turn Dialogues

For interactive systems, we need to maintain context across multiple turns. This involves tracking the dialogue state (DST) and updating the prompt accordingly.

def handle_dialogue_turn(dialogue_history: list, new_input: str) -> str:
    """
    Handles a single turn in a multi-turn dialogue, maintaining context.

    Args:
        dialogue_history: A list of previous turns in the conversation.
        new_input: The current user input.

    Returns:
        The AI's response for the current turn.
    """
    # Combine dialogue history into a single string
    context = "\n".join(dialogue_history)

    instruction = """
    You are an AI recruitment assistant. Analyze the ongoing conversation to evaluate a candidate.
    Maintain context from previous turns. Score the candidate based on education, experience, project count, and attitude.
    Provide a JSON output with: education, experience, project_count, attitude, total_score, is_qualified, and reason.
    """

    output_format = """
    {
        "education": "score as integer",
        "experience": "score as integer",
        "project_count": "score as integer",
        "attitude": "score as integer",
        "total_score": "sum of all scores as integer",
        "is_qualified": "true or false",
        "reason": "explanation if not qualified, otherwise null"
    }
    """

    prompt = f"""
    {instruction}

    Previous Conversation:
    {context}

    {output_format}

    New Input:
    {new_input}
    """

    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        temperature=0,
    )

    return response.choices[0].message.content

# Example of multi-turn dialogue
dialogue_history = []
turn1 = "What is your highest level of education?"
response1 = "I have a Master's degree."
dialogue_history.append(f"Assistant: {turn1}\nCandidate: {response1}")

turn2 = "How many years of experience do you have?"
response2 = "I have 6 years of experience."
dialogue_history.append(f"Assistant: {turn2}\nCandidate: {response2}")

turn3 = "How many projects have you completed?"
response3 = "I have completed 8 projects."
dialogue_history.append(f"Assistant: {turn3}\nCandidate: {response3}")

turn4 = "Are you familiar with our company's business and culture?"
response4 = "Yes, I am very familiar and very interested in this position."
dialogue_history.append(f"Assistant: {turn4}\nCandidate: {response4}")

final_evaluation = handle_dialogue_turn(dialogue_history, "Thank you for your responses.")
print(final_evaluation)