Prompting for Data Analysis and Research

# CHAPTER 12

Prompting for Data Analysis and Research #

1. Introduction #

2. Learning Objectives #

• Use Prompts for rapid document summarization.

• Engineer prompts to extract specific data from messy text.

• Utilize AI for sentiment analysis and trend identification.

• Understand the limits of AI math in data analysis.

3. Beginner-Friendly Explanation #

4. The Summarization Prompt #

Engineered Summarization Prompt:

Role: Executive Business Assistant.
Task: Summarize the following 10-page Q3 Earnings Call transcript.
Constraints:
- Focus EXCLUSIVELY on mentions of our competitor, "Acme Corp".
- Ignore all financial metrics about our own company.
- Format the summary as 3 bullet points.
Transcript: """ [Paste Transcript] """

5. Data Extraction (Messy to Clean) #

The Extraction Prompt:

Task: Extract all dates, names, and dollar amounts from the messy text below.
Format: Output a Markdown table with the columns: [Name, Date, Amount].
Constraint: If a value is missing, write "N/A".

Text: """
Hi it's John, I paid $50 on Tuesday the 4th. Sarah didn't pay her $100 bill. Mike paid on October 12th but I forgot the amount.
"""

6. Sentiment Analysis and Trend Spotting #

The Trend Prompt:

Task: Read the following 100 customer reviews for our new coffee machine.
Identify the 3 most common complaints. Provide a direct quote from the reviews for each complaint to prove it.
Output: A bulleted list.

7. The Math Limitation (Advanced Data Analysis) #

8. Python Example: Batch Processing Reviews #

import openai
client = openai.OpenAI()

# A list of thousands of messy reviews
reviews = ["The food was cold.", "Loved the ambient lighting!", "Terrible service."]

for review in reviews:
    prompt = f"Analyze the sentiment of this review. Output exactly ONE word: Positive, Negative, or Neutral. Review: '{review}'"
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}]
    )
    print(f"Review: {review} -> Sentiment: {response.choices[0].message.content}")

9. Mini Project #

10. Best Practices #

• Citations Required: When prompting an AI to research a long document, always add the constraint: *"Provide exact quotes from the text to support your claims."* This prevents the AI from hallucinating a summary that sounds good but isn't actually in the document.

11. Common Mistakes #

• Exceeding the Context Window: If you paste a 300-page book into a standard LLM prompt, it will crash or "forget" the middle of the book. For massive research, you must use RAG (Retrieval-Augmented Generation) or an AI tool specifically designed for huge context windows (like Claude 3).

12. Exercises #

1. 1. Explain why you should explicitly instruct the AI to "Ignore" irrelevant information when asking it to summarize a large document.

13. MCQs with Answers #

14. Interview Questions #

• Q: How do you engineer a prompt to extract highly structured JSON data from an unstructured, messy PDF document?

• Q: Discuss the limitations of standard Large Language Models in performing complex arithmetic on datasets, and how modern platforms overcome this limitation.

15. FAQs #

16. Summary #

17. Next Chapter Recommendation #