Introduction
Machine learning is no longer a niche skill—it's a core competency for data scientists, analysts, and even software engineers. Yet, many practitioners get stuck in the middle: they know the theory, but when they face a real dataset, they waste hours on boilerplate code, debugging pipelines, or choosing the right hyperparameters. That's where prompts come in—not for LLMs, but for your own workflow. A good prompt is a reusable, parameterized template that guides you through a specific ML task: from loading data and cleaning it, to training a model and evaluating it.
In this article, I've curated 30 actionable prompts organized by skill level. They cover the three most popular Python ML libraries—Scikit-learn, XGBoost, and CatBoost—and span the entire lifecycle: preprocessing, feature engineering, training, tuning, and interpretation. Each prompt includes a clear task, the prompt itself (ready to copy-paste into your notebook or script), and a concrete example result. Whether you're a beginner who needs a clean starting point or an expert looking for advanced tuning strategies, you'll find something useful.
Let's dive in.
Basic Prompts: Foundations for Beginners
These prompts assume you have a basic understanding of Python and pandas, but little experience with ML libraries. They focus on getting you from raw CSV to a trained model with minimal friction.
1. Load and Inspect a Dataset
Task: Load a CSV file into a pandas DataFrame and display basic statistics.
Prompt:
import pandas as pd
df = pd.read_csv('your_dataset.csv')
print('Shape:', df.shape)
print('Columns:', df.columns.tolist())
print(df.head())
print(df.describe(include='all'))
print('Missing values:\n', df.isnull().sum())
Example Result:
Shape: (1000, 12)
Columns: ['age', 'income', 'education', ...]
age income education ...
0 34 45000 3 ...
...
age income ...
count 980.0 950.0 ...
mean 42.5 52000.0 ...
Missing values:
age 20
income 50
...
2. Simple Train-Test Split
Task: Split the data into training and test sets (80/20) and check shapes.
Prompt:
from sklearn.model_selection import train_test_split
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
print('Train:', X_train.shape, y_train.shape)
print('Test:', X_test.shape, y_test.shape)
Example Result:
Train: (800, 11) (800,)
Test: (200, 11) (200,)
3. Train a Logistic Regression Baseline
Task: Train a simple logistic regression model and print accuracy.
Prompt:
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print('Accuracy:', accuracy_score(y_test, y_pred))
Example Result:
Accuracy: 0.785
4. Quick XGBoost Classifier with Defaults
Task: Train an XGBoost classifier using default parameters.
Prompt:
import xgboost as xgb
model = xgb.XGBClassifier(use_label_encoder=False, eval_metric='logloss')
model.fit(X_train, y_train)
print('Accuracy:', accuracy_score(y_test, model.predict(X_test)))
Example Result:
Accuracy: 0.825
5. Quick CatBoost Classifier with Defaults
Task: Train a CatBoost classifier without tuning.
Prompt:
from catboost import CatBoostClassifier
model = CatBoostClassifier(verbose=0)
model.fit(X_train, y_train)
print('Accuracy:', accuracy_score(y_test, model.predict(X_test)))
Example Result:
Accuracy: 0.831
6. Handle Missing Values with Simple Imputation
Task: Impute missing numerical values with the median.
Prompt:
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy='median')
X_train_imp = imputer.fit_transform(X_train)
X_test_imp = imputer.transform(X_test)
print('Missing after imputation:', pd.DataFrame(X_train_imp).isnull().sum().sum())
Example Result:
Missing after imputation: 0
7. One-Hot Encode Categorical Features
Task: Convert categorical columns to one-hot encoding.
Prompt:
from sklearn.preprocessing import OneHotEncoder
cat_cols = ['education', 'marital_status']
encoder = OneHotEncoder(drop='first', sparse_output=False)
X_train_enc = encoder.fit_transform(X_train[cat_cols])
X_test_enc = encoder.transform(X_test[cat_cols])
print('Encoded shape:', X_train_enc.shape)
Example Result:
Encoded shape: (800, 5)
8. Standardize Numerical Features
Task: Scale numerical features to zero mean and unit variance.
Prompt:
from sklearn.preprocessing import StandardScaler
num_cols = ['age', 'income']
scaler = StandardScaler()
X_train_num = scaler.fit_transform(X_train[num_cols])
X_test_num = scaler.transform(X_test[num_cols])
print('Mean after scaling:', X_train_num.mean(axis=0))
Example Result:
Mean after scaling: [-1.23e-16 2.45e-16]
9. Build a Pipeline to Chain Preprocessing and Model
Task: Create a Scikit-learn pipeline that imputes, scales, and trains a LogisticRegression.
Prompt:
from sklearn.pipeline import Pipeline
pipe = Pipeline([
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler()),
('clf', LogisticRegression(max_iter=1000))
])
pipe.fit(X_train, y_train)
print('Pipeline accuracy:', accuracy_score(y_test, pipe.predict(X_test)))
Example Result:
Pipeline accuracy: 0.790
10. Evaluate with Confusion Matrix and Classification Report
Task: Print confusion matrix and precision/recall/F1.
Prompt:
from sklearn.metrics import confusion_matrix, classification_report
print('Confusion Matrix:')
print(confusion_matrix(y_test, y_pred))
print('\nClassification Report:')
print(classification_report(y_test, y_pred))
Example Result:
Confusion Matrix:
[[85 15]
[28 72]]
Classification Report:
precision recall f1-score support
0 0.75 0.85 0.80 100
1 0.83 0.72 0.77 100
Advanced Prompts: Intermediate to Pro
These prompts assume you are comfortable with pipelines and want to improve performance through feature engineering, cross-validation, and hyperparameter tuning.
11. Cross-Validate a Model with Stratified K-Fold
Task: Perform 5-fold stratified cross-validation and report mean accuracy.
Prompt:
from sklearn.model_selection import cross_val_score, StratifiedKFold
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_val_score(pipe, X_train, y_train, cv=cv, scoring='accuracy')
print('CV scores:', scores)
print('Mean accuracy:', scores.mean())
Example Result:
CV scores: [0.7875 0.8000 0.8125 0.7750 0.7938]
Mean accuracy: 0.7938
12. Grid Search for Logistic Regression Hyperparameters
Task: Find best C and penalty for LogisticRegression using GridSearchCV.
Prompt:
from sklearn.model_selection import GridSearchCV
param_grid = {
'clf__C': [0.01, 0.1, 1, 10],
'clf__penalty': ['l1', 'l2']
}
grid = GridSearchCV(pipe, param_grid, cv=5, scoring='accuracy')
grid.fit(X_train, y_train)
print('Best params:', grid.best_params_)
print('Best CV score:', grid.best_score_)
print('Test accuracy:', accuracy_score(y_test, grid.predict(X_test)))
Example Result:
Best params: {'clf__C': 1, 'clf__penalty': 'l2'}
Best CV score: 0.7950
Test accuracy: 0.800
13. Randomized Search for XGBoost with Early Stopping
Task: Use RandomizedSearchCV to tune XGBoost with early stopping on a validation set.
Prompt:
from sklearn.model_selection import RandomizedSearchCV
param_dist = {
'n_estimators': [50, 100, 200],
'max_depth': [3, 5, 7],
'learning_rate': [0.01, 0.1, 0.2],
'subsample': [0.8, 1.0]
}
xgb_model = xgb.XGBClassifier(use_label_encoder=False, eval_metric='logloss', early_stopping_rounds=10)
random_search = RandomizedSearchCV(
xgb_model, param_dist, n_iter=10, cv=3, scoring='accuracy', random_state=42
)
random_search.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=0)
print('Best params:', random_search.best_params_)
print('Test accuracy:', accuracy_score(y_test, random_search.predict(X_test)))
Example Result:
Best params: {'subsample': 0.8, 'n_estimators': 200, 'max_depth': 5, 'learning_rate': 0.1}
Test accuracy: 0.845
14. CatBoost with Categorical Feature Handling
Task: Train CatBoost with explicit categorical feature indices.
Prompt:
cat_features = [0, 3] # indices of categorical columns
model = CatBoostClassifier(iterations=500, learning_rate=0.1, depth=6, verbose=0)
model.fit(X_train, y_train, cat_features=cat_features)
print('CatBoost accuracy:', accuracy_score(y_test, model.predict(X_test)))
Example Result:
CatBoost accuracy: 0.852
15. Feature Importance Plot for XGBoost
Task: Plot the top 10 most important features from an XGBoost model.
Prompt:
import matplotlib.pyplot as plt
model = xgb.XGBClassifier().fit(X_train, y_train)
xgb.plot_importance(model, max_num_features=10)
plt.show()
Example Result:
A bar chart showing features like income, age, education ranked by gain.
16. Recursive Feature Elimination (RFE) with Cross-Validation
Task: Select top 5 features using RFECV with a RandomForest.
Prompt:
from sklearn.feature_selection import RFECV
from sklearn.ensemble import RandomForestClassifier
estimator = RandomForestClassifier(n_estimators=100, random_state=42)
selector = RFECV(estimator, step=1, cv=5, scoring='accuracy')
selector.fit(X_train, y_train)
print('Optimal number of features:', selector.n_features_)
print('Selected features:', X_train.columns[selector.support_])
Example Result:
Optimal number of features: 5
Selected features: Index(['income', 'age', 'education', 'marital_status', 'loan_amount'], dtype='object')
17. Polynomial Feature Expansion
Task: Add interaction and polynomial features up to degree 2.
Prompt:
from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=2, interaction_only=False, include_bias=False)
X_train_poly = poly.fit_transform(X_train_num)
print('Original shape:', X_train_num.shape)
print('Polynomial shape:', X_train_poly.shape)
Example Result:
Original shape: (800, 2)
Polynomial shape: (800, 5)
18. Pipeline with ColumnTransformer for Mixed Types
Task: Apply different preprocessing to numerical and categorical columns.
Prompt:
from sklearn.compose import ColumnTransformer
num_cols = ['age', 'income']
cat_cols = ['education', 'marital_status']
preprocessor = ColumnTransformer([
('num', StandardScaler(), num_cols),
('cat', OneHotEncoder(drop='first'), cat_cols)
])
pipe = Pipeline([
('prep', preprocessor),
('clf', LogisticRegression(max_iter=1000))
])
pipe.fit(X_train, y_train)
print('Accuracy:', accuracy_score(y_test, pipe.predict(X_test)))
Example Result:
Accuracy: 0.795
19. Learning Curve Analysis
Task: Plot learning curve to diagnose bias/variance.
Prompt:
from sklearn.model_selection import learning_curve
import numpy as np
train_sizes, train_scores, test_scores = learning_curve(
pipe, X_train, y_train, train_sizes=np.linspace(0.1, 1.0, 5), cv=5, scoring='accuracy'
)
print('Train scores mean:', train_scores.mean(axis=1))
print('Test scores mean:', test_scores.mean(axis=1))
Example Result:
Train scores mean: [0.85 0.86 0.87 0.88 0.89]
Test scores mean: [0.78 0.79 0.80 0.80 0.81]
20. Save and Load a Trained Model
Task: Serialize a model with joblib and reload it.
Prompt:
import joblib
joblib.dump(pipe, 'model.pkl')
loaded_pipe = joblib.load('model.pkl')
print('Loaded model accuracy:', accuracy_score(y_test, loaded_pipe.predict(X_test)))
Example Result:
Loaded model accuracy: 0.795
Expert Prompts: Production-Grade & Cutting-Edge
These prompts are for practitioners who need to squeeze out the last bit of performance, handle large datasets, or deploy models reliably.
21. Custom Weighted Loss Function in XGBoost
Task: Train XGBoost with a custom objective (e.g., weighted logistic loss for imbalanced classes).
Prompt:
import numpy as np
def weighted_logloss(y_true, y_pred):
grad = (y_pred - y_true) * (1 + 2 * y_true)
hess = np.ones_like(y_true) * (1 + 2 * y_true)
return grad, hess
model = xgb.XGBClassifier(objective=weighted_logloss, eval_metric='logloss')
model.fit(X_train, y_train)
print('Accuracy:', accuracy_score(y_test, model.predict(X_test)))
Example Result:
Accuracy: 0.860
(Note: Real performance depends on class weights; this is illustrative.)
22. CatBoost with GPU Training
Task: Train CatBoost on GPU for faster iteration.
Prompt:
model = CatBoostClassifier(iterations=1000, learning_rate=0.05, task_type='GPU', devices='0:1', verbose=0)
model.fit(X_train, y_train)
print('GPU CatBoost accuracy:', accuracy_score(y_test, model.predict(X_test)))
Example Result:
GPU CatBoost accuracy: 0.855
23. XGBoost with Custom Evaluation Metric
Task: Use a custom metric (e.g., F1-score) during training.
Prompt:
from sklearn.metrics import f1_score
def f1_eval(y_pred, dtrain):
y_true = dtrain.get_label()
y_pred_bin = (y_pred > 0.5).astype(int)
return 'f1', f1_score(y_true, y_pred_bin)
model = xgb.train(
{'objective': 'binary:logistic', 'eval_metric': f1_eval},
xgb.DMatrix(X_train, y_train),
num_boost_round=100
)
Example Result:
[0] train-f1:0.723
[1] train-f1:0.741
...
[99] train-f1:0.892
24. Bayesian Hyperparameter Optimization with Optuna
Task: Use Optuna to tune XGBoost hyperparameters efficiently.
Prompt:
import optuna
def objective(trial):
params = {
'n_estimators': trial.suggest_int('n_estimators', 50, 300),
'max_depth': trial.suggest_int('max_depth', 3, 10),
'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.3, log=True),
'subsample': trial.suggest_float('subsample', 0.6, 1.0),
}
model = xgb.XGBClassifier(**params, use_label_encoder=False, eval_metric='logloss')
score = cross_val_score(model, X_train, y_train, cv=3, scoring='accuracy').mean()
return score
study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=20)
print('Best trial:', study.best_trial.params)
Example Result:
Best trial: {'n_estimators': 250, 'max_depth': 7, 'learning_rate': 0.08, 'subsample': 0.85}
Best score: 0.867
25. SHAP Values for Model Interpretability
Task: Compute SHAP values and create a summary plot.
Prompt:
import shap
model = xgb.XGBClassifier().fit(X_train, y_train)
explainer = shap.Explainer(model)
shap_values = explainer(X_test)
shap.summary_plot(shap_values, X_test)
Example Result:
A beeswarm plot showing feature impact on model output.
26. Calibration Curve for Probability Calibration
Task: Plot calibration curve for the model's predicted probabilities.
Prompt:
from sklearn.calibration import calibration_curve
prob_pos = pipe.predict_proba(X_test)[:, 1]
fraction_of_positives, mean_predicted_value = calibration_curve(y_test, prob_pos, n_bins=10)
plt.plot(mean_predicted_value, fraction_of_positives, marker='o')
plt.plot([0, 1], [0, 1], linestyle='--')
plt.show()
Example Result:
A plot comparing predicted probabilities to actual frequencies.
27. Ensemble of Scikit-learn, XGBoost, and CatBoost (Voting Classifier)
Task: Combine three models using soft voting.
Prompt:
from sklearn.ensemble import VotingClassifier
clf1 = LogisticRegression(max_iter=1000)
clf2 = xgb.XGBClassifier(use_label_encoder=False, eval_metric='logloss')
clf3 = CatBoostClassifier(verbose=0)
voting = VotingClassifier(
estimators=[('lr', clf1), ('xgb', clf2), ('cat', clf3)],
voting='soft'
)
voting.fit(X_train, y_train)
print('Ensemble accuracy:', accuracy_score(y_test, voting.predict(X_test)))
Example Result:
Ensemble accuracy: 0.863
28. Stratified Sampling for Imbalanced Data (SMOTE)
Task: Apply SMOTE to oversample the minority class.
Prompt:
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline as ImbPipeline
smote = SMOTE(random_state=42)
imb_pipe = ImbPipeline([
('prep', preprocessor),
('smote', smote),
('clf', LogisticRegression(max_iter=1000))
])
imb_pipe.fit(X_train, y_train)
print('SMOTE accuracy:', accuracy_score(y_test, imb_pipe.predict(X_test)))
Example Result:
SMOTE accuracy: 0.808
29. Time-Series Cross-Validation for Temporal Data
Task: Use TimeSeriesSplit to avoid data leakage.
Prompt:
from sklearn.model_selection import TimeSeriesSplit
tscv = TimeSeriesSplit(n_splits=5)
for train_idx, val_idx in tscv.split(X_train):
X_tr, X_val = X_train.iloc[train_idx], X_train.iloc[val_idx]
y_tr, y_val = y_train.iloc[train_idx], y_train.iloc[val_idx]
model = xgb.XGBClassifier().fit(X_tr, y_tr)
print('Fold accuracy:', accuracy_score(y_val, model.predict(X_val)))
Example Result:
Fold accuracy: 0.81
Fold accuracy: 0.82
Fold accuracy: 0.79
Fold accuracy: 0.83
Fold accuracy: 0.80
30. Automated ML Pipeline with TPOT
Task: Use TPOT to automatically search for the best pipeline.
Prompt:
from tpot import TPOTClassifier
tpot = TPOTClassifier(generations=5, population_size=20, cv=3, random_state=42, verbosity=2)
tpot.fit(X_train, y_train)
print('TPOT test accuracy:', accuracy_score(y_test, tpot.predict(X_test)))
tpot.export('best_pipeline.py')
Example Result:
TPOT test accuracy: 0.875
Exported pipeline to 'best_pipeline.py'.
Conclusion
These 30 prompts cover the full spectrum of ML workflows using Scikit-learn, XGBoost, and CatBoost—from getting your first baseline to building production-ready ensembles with Bayesian tuning and SHAP interpretability. The beauty of these prompts is that they are not just one-off code snippets; they are templates you can adapt to any dataset. Start with the basic prompts for a quick start, then gradually incorporate advanced techniques like column transformers, early stopping, and custom objectives as your needs grow.
Remember, the best model is not the one with the highest accuracy on paper, but the one that generalizes well and can be maintained in production. Use these prompts as your toolbox, and always validate your choices with cross-validation and domain knowledge. Happy modeling!
Comments