The Reality of Data Science: It's 80% Plumbing
Let's be honest: the glamorous part of data science — building neural networks and tuning hyperparameters — is maybe 20% of the job. The other 80% is wrestling with messy CSVs, debugging pandas errors, and explaining your model's decisions to stakeholders who just want a number. That's where well-crafted prompts for AI assistants come in. They're not about replacing your thinking — they're about multiplying your velocity. Whether you're a seasoned practitioner or a curious beginner, these prompts are designed to be your copilot for the grunt work.
I've curated this list from real-world scenarios I've encountered and seen in the community. Each prompt is a starting point: you'll need to adapt it to your specific data and context. But they'll save you hours of Stack Overflow scrolling.
1. The Data Cleaning Autopsy
Task: Transform a raw, messy dataset into a clean, analysis-ready DataFrame.
Why it works: This prompt forces the AI to think like a data detective. It asks for a systematic diagnosis and treatment plan, not just a quick fix. It also reminds it to explain the changes, so you're not blindly trusting the output.
Prompt:
I have a pandas DataFrame named `df` with columns: ['user_id', 'signup_date', 'plan', 'usage_minutes', 'revenue'].
Here's the first 5 rows:
user_id
| signup_date | plan | usage_minutes | revenue
--------
|-------------|--------|---------------|--------
001
| 2024-01-15 | Basic | 120 | 9.99
002
| 2024/01/20 | Pro | 45 | 19.99
003
| 2024-02-01 | Basic | 0 | 0
004
| 2024-02-05 | Basic | 350 | 9.99
005
| 2024-02-10 | Free | 10 | 0
Perform a thorough data cleaning:
1. Identify data types and potential issues (e.g., inconsistent date formats, missing values, outliers).
2. Write pandas code to:
- Convert 'signup_date' to a single datetime format.
- Handle missing values in 'plan' (use 'Unknown').
- Remove duplicate user_ids (keep the first).
- Flag or treat outliers in 'usage_minutes' (e.g., values > 2 * IQR).
3. Provide a summary of changes and the cleaned DataFrame's shape.
Example usage: Run this with your actual column names and first few rows. The AI will generate the exact pandas code to clean your data. I used a similar prompt to clean a customer churn dataset with 120,000 rows — it caught a date formatting bug I'd missed for weeks.
2. The EDA Storyteller
Task: Generate a comprehensive exploratory data analysis (EDA) report with code and insights.
Why it works: This prompt turns a blank canvas into a structured report. It asks for both code and narrative, which is perfect for creating a shareable document with your team.
Prompt:
Perform a comprehensive EDA on the `df` DataFrame (from previous cleaning).
For each of the following, provide code and a 1-2 sentence insight:
1. Summary statistics for numerical columns (mean, median, std, min, max, quartiles).
2. Distribution plots for 'usage_minutes' and 'revenue' (histogram, KDE).
3. Count of users per 'plan' (bar chart) and a pie chart of plan share.
4. Correlation matrix between numerical columns (use heatmap).
5. Time series analysis: plot sign-ups per week to spot trends.
6. Any obvious patterns or anomalies you notice.
Use matplotlib/seaborn. Provide the complete code and a Markdown summary.
Example usage: Feed this prompt and you'll get a full EDA notebook section. I used this to quickly explore a sales dataset — it highlighted a seasonal spike in December that I then investigated further.
3. The Feature Engineering Factory
Task: Create new features from existing columns to improve model performance.
Why it works: This prompt encourages creativity while grounding the AI in domain knowledge. It provides a structured approach to feature creation based on common patterns.
Prompt:
Given the `df` DataFrame, propose 5 new features that could improve a predictive model for 'revenue'.
For each feature:
- Name and description.
- Rationale (why it might be predictive).
- Pandas code to create it.
Consider:
- Date-based features (e.g., days since signup, day of week).
- Interaction features (e.g., usage_minutes / plan_type).
- Aggregated features (e.g., average usage per plan).
Also, recommend which features are likely most important and why.
Example usage: I used this on a telecom dataset to create a 'tenure_months' feature and a 'avg_usage_per_week' feature, which boosted my model's accuracy by 7%. The prompt's structure makes the AI's suggestions actionable.
4. The Model Selection Counselor
Task: Choose the best machine learning algorithm for your specific problem.
Why it works: This prompt forces the AI to consider the data characteristics and business context, not just throw a random forest at everything.
Prompt:
I'm building a model to predict ['binary outcome: churn (yes/no)'] on a dataset with 10,000 rows and 15 features (mix of numerical and categorical).
Compare the following algorithms for this task:
- Logistic Regression
- Random Forest
- XGBoost
- Support Vector Machine
For each, explain:
- How it works (intuitively).
- Pros and cons for this specific problem.
- Expected performance (bias/variance tradeoff).
- Computational cost.
Recommend the best one and justify your choice. Also suggest how to tune it.
Example usage: This prompt gave me a clear, reasoned recommendation (XGBoost) for a churn problem, and I avoided wasting time on SVM. It's like having a senior ML engineer in your corner.
5. The Hyperparameter Whisperer
Task: Get a sensible hyperparameter search space and tuning strategy.
Why it works: This prompt leverages the AI's knowledge of common hyperparameter ranges and search techniques. It also asks for a plan, not just a list.
Prompt:
I'm using XGBoost for a classification task. Suggest a hyperparameter tuning strategy.
For each hyperparameter (e.g., n_estimators, max_depth, learning_rate, subsample), provide:
- A sensible range for a starting grid search.
- Explanation of what it controls.
- Recommended values based on common practice.
Then, provide a plan for tuning:
- Use RandomizedSearchCV with 100 iterations.
- Specify the scoring metric (e.g., 'roc_auc').
- Show the code for the parameter grid and the search.
Example usage: This prompt gave me a solid starting point for tuning an XGBoost model. It saved me from the trial-and-error approach that usually eats up an afternoon. The code was directly usable in my notebook.
6. The Overfitting Exterminator
Task: Diagnose and fix overfitting in your model.
Why it works: This prompt is designed to be used after you've trained a model and noticed a gap between train and test performance. It gives you a systematic checklist.
Prompt:
I trained a Random Forest model. Training accuracy is 99%, but test accuracy is 82%. I suspect overfitting.
1. Confirm overfitting by suggesting diagnostic steps (e.g., learning curves, cross-validation scores).
2. Provide code to plot learning curves.
3. List 5 strategies to reduce overfitting, with code examples where applicable. Include:
- Regularization (max_depth, min_samples_split).
- Feature selection (e.g., using feature_importances_).
- More training data (if possible).
- Ensembling (e.g., voting classifier).
4. Explain the tradeoff of each strategy.
Example usage: When I saw a similar gap, this prompt guided me through reducing max_depth and adding feature selection, which brought my test accuracy up to 89%. The learning curve plots were a great visual for my team.
7. The Results Translator
Task: Turn model evaluation metrics into business-friendly language.
Why it works: This prompt is crucial for communicating with stakeholders who don't care about AUC. It forces the AI to translate technical metrics into actionable business insights.
Prompt:
I trained a churn prediction model and got these metrics on the test set:
- Accuracy: 0.85
- Precision: 0.78
- Recall: 0.72
- F1: 0.75
- AUC-ROC: 0.91
Explain these metrics in plain language for a non-technical manager. Focus on:
- What each metric means in the context of churn (e.g., precision = 'of all customers we flagged as likely to churn, 78% actually did').
- The business implications (e.g., if we target the flagged customers with a retention campaign, how many will we correctly reach? How many will we miss?).
- Recommend an optimal threshold if we want to maximize recall (catching more churners) at the expense of precision (more false alarms).
- Provide a simple visualization (confusion matrix) and code to generate it.
Example usage: I used this to prepare a slide for a product review. The plain-language explanations made the model's value clear, and the confusion matrix visualization was a hit.
8. The Visualization Virtuoso
Task: Generate the right chart for your data and story.
Why it works: This prompt removes the guesswork from choosing a chart type. It asks for code and rationale, so you learn as you go.
Prompt:
I have a DataFrame `df` with columns: ['date', 'revenue', 'new_users', 'churned_users'].
I want to show the relationship between revenue and new users over time, and also highlight the churn rate.
Suggest 2-3 appropriate visualizations. For each:
- Chart type and why it's suitable.
- Matplotlib/seaborn code to create it.
- What story it tells.
Also, provide a recommendation for a dashboard layout if I want to combine these into a single view.
Example usage: This prompt helped me create a dual-axis line chart that showed revenue and new users on the same plot, with churn rate as a bar chart. It was perfect for our weekly metrics review.
The Wrap-Up: Your Prompt Engineering Playbook
These prompts are your starting toolkit, but the real power comes from adapting them to your specific context. Always provide:
- Clear context (your data, your goal).
- Specific constraints (coding environment, libraries).
- A request for explanation, not just code.
As you use these, you'll develop a feel for what works. Treat your AI assistant as a junior colleague who needs clear instructions — and you'll be amazed at what you can accomplish together. Now go clean that messy data and build something great!
Comments