Research
Mortality Risk in Heart Failure
Heart failure presents considerable mortality risks, highlighting the need for reliable predictors to support effective patient prognosis. This study explores the predictive power of specific clinical indicators, including ejection fraction, kidney function, smoking status, and follow-up time, for mortality in heart failure patients. Using a balanced training dataset, we conducted feature selection based on validation set accuracy, identifying these four indicators as the most influential subset. Models were subsequently tested on an unseen test set, with Random Forest achieving the highest accuracy at 87%, followed by Support Vector Machine (SVM) at 83.3%, and Logistic Regression at 80%. When all 12 initial indicators were applied, accuracy across all models converged to 83.3%. These findings emphasize the role of targeted feature selection in improving model performance, suggesting that focusing on key predictors can streamline clinical assessments while enhancing prognostic accuracy. This study underscores the potential of refined indica- tor selection to optimize resource allocation and improve patient outcomes in heart failure management.
In [2]:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, roc_auc_score, roc_curve, classification_report, confusion_matrix, ConfusionMatrixDisplay
from itertools import combinations
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.svm import SVC
from sklearn.preprocessing import StandardScaler
from imblearn.over_sampling import SMOTE
# Load CSV Data
data_file = 'drive/MyDrive/1AConferencesJournals/BIBM24/BiomarkerProject/heart+failure+clinical+records/heart_failure_clinical_records_dataset.csv'
data = pd.read_csv(data_file)
# Define feature matrix (X) and target vector (y)
X = data.iloc[:, :-1] # All columns except the last one
y = data.iloc[:, -1] # Last column as the target variable (mortality)
# Split data into training, validation, and test sets
X_train_full, X_test, y_train_full, y_test = train_test_split(X, y, test_size=0.1, random_state=0)
X_train, X_val, y_train, y_val = train_test_split(X_train_full, y_train_full, test_size=0.2, random_state=0) # 60% train, 20% val, 20% test
# Check the distribution of the target variable (assuming 'mortality' is binary: 1 = positive, 0 = negative)
target_counts = y_train_full.value_counts() # Assuming last column is the target (mortality)
# Plot distribution
plt.figure(figsize=(8, 6))
plt.bar(['Survived', 'Deceased'], target_counts, color=['skyblue', 'salmon'])
plt.xlabel('Mortality Status')
plt.ylabel('Number of Samples')
plt.title('Distribution of Mortality in the Dataset before SMOTE')
plt.show()
# Apply SMOTE to handle class imbalance
smote = SMOTE(random_state=0)
X_train, y_train = smote.fit_resample(X_train, y_train)
# Plot distribution
plt.figure(figsize=(8, 6))
plt.bar(['Survived', 'Deceased'], y_train.value_counts(), color=['skyblue', 'salmon'])
plt.xlabel('Mortality Status')
plt.ylabel('Number of Samples')
plt.title('Distribution of Mortality in the Dataset after SMOTE')
plt.show()
# Standardize features for consistency across models
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_val = scaler.transform(X_val)
X_test = scaler.transform(X_test)
# Initialize lists to store results
best_subset = None
best_auc = 0
all_results = []
# Iterate through all 3-feature combinations
for subset in combinations(X.columns, 4):
# Select the current subset of features
X_train_subset = X_train[:, [X.columns.get_loc(feature) for feature in subset]]
X_val_subset = X_val[:, [X.columns.get_loc(feature) for feature in subset]]
# Train a model on this subset
rf_model = RandomForestClassifier(random_state=0)
rf_model.fit(X_train_subset, y_train)
# Predict and calculate AUC on validation set
val_probs = rf_model.predict_proba(X_val_subset)[:, 1]
val_auc = roc_auc_score(y_val, val_probs)
# Store results
all_results.append((subset, val_auc))
if val_auc > best_auc:
best_auc = val_auc
best_subset = subset
# Print all subset performances on validation set
print("Performance of all 4-feature subsets on validation set:")
for subset, auc in all_results:
print(f"Features: {subset}, AUC: {auc:.3f}")
print(f"\nBest Subset: {best_subset}, Best AUC on Validation: {best_auc:.3f}")
# Apply the best subset on the test set
X_train_best = X_train[:, [X.columns.get_loc(feature) for feature in best_subset]]
X_test_best = X_test[:, [X.columns.get_loc(feature) for feature in best_subset]]
# Retrain on the best subset
best_rf_model = RandomForestClassifier(random_state=0)
best_rf_model.fit(X_train_best, y_train)
# Get feature importances
feature_importances = best_rf_model.feature_importances_
print("Features Importance:", feature_importances)
colors = ['skyblue', 'salmon', 'lightgreen', 'gold']
# Plotting the histogram
plt.figure(figsize=(8, 6))
plt.bar(best_subset, feature_importances, color=colors)
plt.xlabel('Features')
plt.ylabel('Importance')
plt.title('Features Importance Weights in Predicting Mortality')
plt.xticks(rotation=45)
plt.show()
# Retrain on the best subset
best_rf_model = RandomForestClassifier(random_state=0)
best_rf_model.fit(X_train_best, y_train)
test_probs = best_rf_model.predict_proba(X_test_best)[:, 1]
test_auc = roc_auc_score(y_test, test_probs)
# Print final performance on test set
print(f"\nFinal Performance on Test Set using Best Subset {best_subset}")
print(f"Test AUC: {test_auc:.3f}")
print("Classification Report on Test Set:")
print(classification_report(y_test, best_rf_model.predict(X_test_best)))
# Confusion Matrix
conf_matrix = confusion_matrix(y_test, best_rf_model.predict(X_test_best))
print("Confusion Matrix:\n", conf_matrix)
# Plot confusion matrix
# Plot confusion matrix with color annotations
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
# Train SVM on the scaled data
svm_model = SVC(probability=True, random_state=0)
svm_model.fit(X_train_best, y_train)
y_test_pred_svm = svm_model.predict(X_test_best)
# Evaluate SVM
svm_accuracy = accuracy_score(y_test, y_test_pred_svm)
print("\nSVM Accuracy:", svm_accuracy)
print("\nSVM Classification Report:")
print(classification_report(y_test, y_test_pred_svm))
# Calculate and print confusion matrix
conf_matrix_svm = confusion_matrix(y_test, y_test_pred_svm)
print("SVM Confusion Matrix:\n", conf_matrix_svm)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_svm, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
#
# Fit the logistic regression model on selected features
logistic_model = LogisticRegression()
logistic_model.fit(X_train_best, y_train)
y_test_pred_log = logistic_model.predict(X_test_best)
# Evaluate SVM
svm_accuracy = accuracy_score(y_test, y_test_pred_log)
print("\nLogistic Regression Accuracy:", svm_accuracy)
print("\nLogistic Regression Classification Report:")
print(classification_report(y_test, y_test_pred_log))
# Calculate and print confusion matrix
conf_matrix_log= confusion_matrix(y_test, y_test_pred_log)
print("Logistic Regression Confusion Matrix:\n", conf_matrix_log)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_log, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
#RF ROC
test_probs = best_rf_model.predict_proba(X_test_best)[:, 1]
fpr_rf, tpr_rf, _ = roc_curve(y_test, test_probs)
roc_auc_rf = roc_auc_score(y_test, test_probs)
# Plot ROC curve for SVM
svm_probs = svm_model.predict_proba(X_test_best)[:, 1]
fpr_svm, tpr_svm, _ = roc_curve(y_test, svm_probs)
roc_auc_svm = roc_auc_score(y_test, svm_probs)
#Logistic regression
# Plot ROC curve for SVM
log_probs = logistic_model.predict_proba(X_test_best)[:, 1]
fpr_log, tpr_log, _ = roc_curve(y_test, log_probs)
roc_auc_log = roc_auc_score(y_test, log_probs)
# Plot both ROC curves
plt.figure(figsize=(10, 6))
plt.plot(fpr_rf, tpr_rf, color='blue', label=f'Random Forest (AUC = {roc_auc_rf:.2f})')
plt.plot(fpr_svm, tpr_svm, color='green', label=f'SVM (AUC = {roc_auc_svm:.2f})')
plt.plot(fpr_log, tpr_log, color='red', label=f'Logistic Regression (AUC = {roc_auc_log:.2f})')
plt.plot([0, 1], [0, 1], color='grey', linestyle='--')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
plt.legend()
plt.show()
print("Without Smoothing nor best features selection")
# Split data into training, validation, and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=0)
# Standardize features for consistency across models
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# Apply the best subset on the test set
X_train_best = X_train
X_test_best = X_test
# Retrain on the best subset
best_rf_model = RandomForestClassifier(random_state=0)
best_rf_model.fit(X_train_best, y_train)
# Print final performance on test set
print(f"\nFinal Performance on Test Set using Best Subset {best_subset}")
print(f"Test AUC: {test_auc:.3f}")
print("Classification Report on Test Set:")
print(classification_report(y_test, best_rf_model.predict(X_test_best)))
# Confusion Matrix
conf_matrix = confusion_matrix(y_test, best_rf_model.predict(X_test_best))
print("Confusion Matrix:\n", conf_matrix)
# Plot confusion matrix
# Plot confusion matrix with color annotations
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
# Train SVM on the scaled data
svm_model = SVC(probability=True, random_state=0)
svm_model.fit(X_train_best, y_train)
y_test_pred_svm = svm_model.predict(X_test_best)
# Evaluate SVM
svm_accuracy = accuracy_score(y_test, y_test_pred_svm)
print("\nSVM Accuracy:", svm_accuracy)
print("\nSVM Classification Report:")
print(classification_report(y_test, y_test_pred_svm))
# Calculate and print confusion matrix
conf_matrix_svm = confusion_matrix(y_test, y_test_pred_svm)
print("SVM Confusion Matrix:\n", conf_matrix_svm)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_svm, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
#
# Fit the logistic regression model on selected features
logistic_model = LogisticRegression()
logistic_model.fit(X_train_best, y_train)
y_test_pred_log = logistic_model.predict(X_test_best)
# Evaluate SVM
svm_accuracy = accuracy_score(y_test, y_test_pred_log)
print("\nLogistic Regression Accuracy:", svm_accuracy)
print("\nLogistic Regression Classification Report:")
print(classification_report(y_test, y_test_pred_log))
# Calculate and print confusion matrix
conf_matrix_log= confusion_matrix(y_test, y_test_pred_log)
print("Logistic Regression Confusion Matrix:\n", conf_matrix_log)
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix_log, annot=True, fmt="d", cmap="Blues", cbar=False, xticklabels=['Alive', 'Death'], yticklabels=['Alive', 'Death'])
plt.xlabel("Predicted Labels")
plt.ylabel("True Labels")
plt.title("Confusion Matrix for Mortality Prediction")
plt.show()
#RF ROC
test_probs = best_rf_model.predict_proba(X_test_best)[:, 1]
fpr_rf, tpr_rf, _ = roc_curve(y_test, test_probs)
roc_auc_rf = roc_auc_score(y_test, test_probs)
# Plot ROC curve for SVM
svm_probs = svm_model.predict_proba(X_test_best)[:, 1]
fpr_svm, tpr_svm, _ = roc_curve(y_test, svm_probs)
roc_auc_svm = roc_auc_score(y_test, svm_probs)
#Logistic regression
# Plot ROC curve for SVM
log_probs = logistic_model.predict_proba(X_test_best)[:, 1]
fpr_log, tpr_log, _ = roc_curve(y_test, log_probs)
roc_auc_log = roc_auc_score(y_test, log_probs)
# Plot both ROC curves
plt.figure(figsize=(10, 6))
plt.plot(fpr_rf, tpr_rf, color='blue', label=f'Random Forest (AUC = {roc_auc_rf:.2f})')
plt.plot(fpr_svm, tpr_svm, color='green', label=f'SVM (AUC = {roc_auc_svm:.2f})')
plt.plot(fpr_log, tpr_log, color='red', label=f'Logistic Regression (AUC = {roc_auc_log:.2f})')
plt.plot([0, 1], [0, 1], color='grey', linestyle='--')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC Curve')
plt.legend()
plt.show()
# Get weights (coefficients)
#feature_weights = model.coef_
#print("Feature weights:", feature_weights)
Performance of all 4-feature subsets on validation set:
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'diabetes'), AUC: 0.450
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'ejection_fraction'), AUC: 0.663
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'high_blood_pressure'), AUC: 0.517
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'platelets'), AUC: 0.552
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'serum_creatinine'), AUC: 0.817
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'serum_sodium'), AUC: 0.509
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'sex'), AUC: 0.503
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'smoking'), AUC: 0.437
Features: ('age', 'anaemia', 'creatinine_phosphokinase', 'time'), AUC: 0.868
Features: ('age', 'anaemia', 'diabetes', 'ejection_fraction'), AUC: 0.735
Features: ('age', 'anaemia', 'diabetes', 'high_blood_pressure'), AUC: 0.479
Features: ('age', 'anaemia', 'diabetes', 'platelets'), AUC: 0.576
Features: ('age', 'anaemia', 'diabetes', 'serum_creatinine'), AUC: 0.770
Features: ('age', 'anaemia', 'diabetes', 'serum_sodium'), AUC: 0.545
Features: ('age', 'anaemia', 'diabetes', 'sex'), AUC: 0.576
Features: ('age', 'anaemia', 'diabetes', 'smoking'), AUC: 0.544
Features: ('age', 'anaemia', 'diabetes', 'time'), AUC: 0.862
Features: ('age', 'anaemia', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.617
Features: ('age', 'anaemia', 'ejection_fraction', 'platelets'), AUC: 0.570
Features: ('age', 'anaemia', 'ejection_fraction', 'serum_creatinine'), AUC: 0.866
Features: ('age', 'anaemia', 'ejection_fraction', 'serum_sodium'), AUC: 0.650
Features: ('age', 'anaemia', 'ejection_fraction', 'sex'), AUC: 0.696
Features: ('age', 'anaemia', 'ejection_fraction', 'smoking'), AUC: 0.656
Features: ('age', 'anaemia', 'ejection_fraction', 'time'), AUC: 0.883
Features: ('age', 'anaemia', 'high_blood_pressure', 'platelets'), AUC: 0.510
Features: ('age', 'anaemia', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.763
Features: ('age', 'anaemia', 'high_blood_pressure', 'serum_sodium'), AUC: 0.487
Features: ('age', 'anaemia', 'high_blood_pressure', 'sex'), AUC: 0.613
Features: ('age', 'anaemia', 'high_blood_pressure', 'smoking'), AUC: 0.554
Features: ('age', 'anaemia', 'high_blood_pressure', 'time'), AUC: 0.846
Features: ('age', 'anaemia', 'platelets', 'serum_creatinine'), AUC: 0.718
Features: ('age', 'anaemia', 'platelets', 'serum_sodium'), AUC: 0.583
Features: ('age', 'anaemia', 'platelets', 'sex'), AUC: 0.613
Features: ('age', 'anaemia', 'platelets', 'smoking'), AUC: 0.501
Features: ('age', 'anaemia', 'platelets', 'time'), AUC: 0.854
Features: ('age', 'anaemia', 'serum_creatinine', 'serum_sodium'), AUC: 0.798
Features: ('age', 'anaemia', 'serum_creatinine', 'sex'), AUC: 0.829
Features: ('age', 'anaemia', 'serum_creatinine', 'smoking'), AUC: 0.692
Features: ('age', 'anaemia', 'serum_creatinine', 'time'), AUC: 0.883
Features: ('age', 'anaemia', 'serum_sodium', 'sex'), AUC: 0.514
Features: ('age', 'anaemia', 'serum_sodium', 'smoking'), AUC: 0.504
Features: ('age', 'anaemia', 'serum_sodium', 'time'), AUC: 0.853
Features: ('age', 'anaemia', 'sex', 'smoking'), AUC: 0.561
Features: ('age', 'anaemia', 'sex', 'time'), AUC: 0.891
Features: ('age', 'anaemia', 'smoking', 'time'), AUC: 0.859
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'ejection_fraction'), AUC: 0.748
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'high_blood_pressure'), AUC: 0.502
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'platelets'), AUC: 0.525
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'serum_creatinine'), AUC: 0.812
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'serum_sodium'), AUC: 0.532
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'sex'), AUC: 0.494
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'smoking'), AUC: 0.538
Features: ('age', 'creatinine_phosphokinase', 'diabetes', 'time'), AUC: 0.833
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.721
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'platelets'), AUC: 0.618
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine'), AUC: 0.829
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'serum_sodium'), AUC: 0.625
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'sex'), AUC: 0.739
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'smoking'), AUC: 0.710
Features: ('age', 'creatinine_phosphokinase', 'ejection_fraction', 'time'), AUC: 0.877
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'platelets'), AUC: 0.529
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.815
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'serum_sodium'), AUC: 0.544
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'sex'), AUC: 0.618
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'smoking'), AUC: 0.512
Features: ('age', 'creatinine_phosphokinase', 'high_blood_pressure', 'time'), AUC: 0.860
Features: ('age', 'creatinine_phosphokinase', 'platelets', 'serum_creatinine'), AUC: 0.785
Features: ('age', 'creatinine_phosphokinase', 'platelets', 'serum_sodium'), AUC: 0.627
Features: ('age', 'creatinine_phosphokinase', 'platelets', 'sex'), AUC: 0.599
Features: ('age', 'creatinine_phosphokinase', 'platelets', 'smoking'), AUC: 0.579
Features: ('age', 'creatinine_phosphokinase', 'platelets', 'time'), AUC: 0.860
Features: ('age', 'creatinine_phosphokinase', 'serum_creatinine', 'serum_sodium'), AUC: 0.803
Features: ('age', 'creatinine_phosphokinase', 'serum_creatinine', 'sex'), AUC: 0.762
Features: ('age', 'creatinine_phosphokinase', 'serum_creatinine', 'smoking'), AUC: 0.763
Features: ('age', 'creatinine_phosphokinase', 'serum_creatinine', 'time'), AUC: 0.897
Features: ('age', 'creatinine_phosphokinase', 'serum_sodium', 'sex'), AUC: 0.498
Features: ('age', 'creatinine_phosphokinase', 'serum_sodium', 'smoking'), AUC: 0.504
Features: ('age', 'creatinine_phosphokinase', 'serum_sodium', 'time'), AUC: 0.817
Features: ('age', 'creatinine_phosphokinase', 'sex', 'smoking'), AUC: 0.527
Features: ('age', 'creatinine_phosphokinase', 'sex', 'time'), AUC: 0.855
Features: ('age', 'creatinine_phosphokinase', 'smoking', 'time'), AUC: 0.850
Features: ('age', 'diabetes', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.726
Features: ('age', 'diabetes', 'ejection_fraction', 'platelets'), AUC: 0.604
Features: ('age', 'diabetes', 'ejection_fraction', 'serum_creatinine'), AUC: 0.822
Features: ('age', 'diabetes', 'ejection_fraction', 'serum_sodium'), AUC: 0.659
Features: ('age', 'diabetes', 'ejection_fraction', 'sex'), AUC: 0.724
Features: ('age', 'diabetes', 'ejection_fraction', 'smoking'), AUC: 0.696
Features: ('age', 'diabetes', 'ejection_fraction', 'time'), AUC: 0.886
Features: ('age', 'diabetes', 'high_blood_pressure', 'platelets'), AUC: 0.559
Features: ('age', 'diabetes', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.758
Features: ('age', 'diabetes', 'high_blood_pressure', 'serum_sodium'), AUC: 0.541
Features: ('age', 'diabetes', 'high_blood_pressure', 'sex'), AUC: 0.601
Features: ('age', 'diabetes', 'high_blood_pressure', 'smoking'), AUC: 0.599
Features: ('age', 'diabetes', 'high_blood_pressure', 'time'), AUC: 0.896
Features: ('age', 'diabetes', 'platelets', 'serum_creatinine'), AUC: 0.725
Features: ('age', 'diabetes', 'platelets', 'serum_sodium'), AUC: 0.558
Features: ('age', 'diabetes', 'platelets', 'sex'), AUC: 0.633
Features: ('age', 'diabetes', 'platelets', 'smoking'), AUC: 0.563
Features: ('age', 'diabetes', 'platelets', 'time'), AUC: 0.868
Features: ('age', 'diabetes', 'serum_creatinine', 'serum_sodium'), AUC: 0.812
Features: ('age', 'diabetes', 'serum_creatinine', 'sex'), AUC: 0.786
Features: ('age', 'diabetes', 'serum_creatinine', 'smoking'), AUC: 0.695
Features: ('age', 'diabetes', 'serum_creatinine', 'time'), AUC: 0.864
Features: ('age', 'diabetes', 'serum_sodium', 'sex'), AUC: 0.529
Features: ('age', 'diabetes', 'serum_sodium', 'smoking'), AUC: 0.627
Features: ('age', 'diabetes', 'serum_sodium', 'time'), AUC: 0.863
Features: ('age', 'diabetes', 'sex', 'smoking'), AUC: 0.546
Features: ('age', 'diabetes', 'sex', 'time'), AUC: 0.894
Features: ('age', 'diabetes', 'smoking', 'time'), AUC: 0.892
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'platelets'), AUC: 0.569
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.830
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'serum_sodium'), AUC: 0.618
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'sex'), AUC: 0.670
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'smoking'), AUC: 0.651
Features: ('age', 'ejection_fraction', 'high_blood_pressure', 'time'), AUC: 0.886
Features: ('age', 'ejection_fraction', 'platelets', 'serum_creatinine'), AUC: 0.795
Features: ('age', 'ejection_fraction', 'platelets', 'serum_sodium'), AUC: 0.609
Features: ('age', 'ejection_fraction', 'platelets', 'sex'), AUC: 0.658
Features: ('age', 'ejection_fraction', 'platelets', 'smoking'), AUC: 0.607
Features: ('age', 'ejection_fraction', 'platelets', 'time'), AUC: 0.882
Features: ('age', 'ejection_fraction', 'serum_creatinine', 'serum_sodium'), AUC: 0.775
Features: ('age', 'ejection_fraction', 'serum_creatinine', 'sex'), AUC: 0.826
Features: ('age', 'ejection_fraction', 'serum_creatinine', 'smoking'), AUC: 0.835
Features: ('age', 'ejection_fraction', 'serum_creatinine', 'time'), AUC: 0.909
Features: ('age', 'ejection_fraction', 'serum_sodium', 'sex'), AUC: 0.637
Features: ('age', 'ejection_fraction', 'serum_sodium', 'smoking'), AUC: 0.641
Features: ('age', 'ejection_fraction', 'serum_sodium', 'time'), AUC: 0.884
Features: ('age', 'ejection_fraction', 'sex', 'smoking'), AUC: 0.706
Features: ('age', 'ejection_fraction', 'sex', 'time'), AUC: 0.906
Features: ('age', 'ejection_fraction', 'smoking', 'time'), AUC: 0.892
Features: ('age', 'high_blood_pressure', 'platelets', 'serum_creatinine'), AUC: 0.726
Features: ('age', 'high_blood_pressure', 'platelets', 'serum_sodium'), AUC: 0.581
Features: ('age', 'high_blood_pressure', 'platelets', 'sex'), AUC: 0.611
Features: ('age', 'high_blood_pressure', 'platelets', 'smoking'), AUC: 0.546
Features: ('age', 'high_blood_pressure', 'platelets', 'time'), AUC: 0.831
Features: ('age', 'high_blood_pressure', 'serum_creatinine', 'serum_sodium'), AUC: 0.779
Features: ('age', 'high_blood_pressure', 'serum_creatinine', 'sex'), AUC: 0.779
Features: ('age', 'high_blood_pressure', 'serum_creatinine', 'smoking'), AUC: 0.738
Features: ('age', 'high_blood_pressure', 'serum_creatinine', 'time'), AUC: 0.872
Features: ('age', 'high_blood_pressure', 'serum_sodium', 'sex'), AUC: 0.524
Features: ('age', 'high_blood_pressure', 'serum_sodium', 'smoking'), AUC: 0.546
Features: ('age', 'high_blood_pressure', 'serum_sodium', 'time'), AUC: 0.849
Features: ('age', 'high_blood_pressure', 'sex', 'smoking'), AUC: 0.569
Features: ('age', 'high_blood_pressure', 'sex', 'time'), AUC: 0.878
Features: ('age', 'high_blood_pressure', 'smoking', 'time'), AUC: 0.859
Features: ('age', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.757
Features: ('age', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.731
Features: ('age', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.683
Features: ('age', 'platelets', 'serum_creatinine', 'time'), AUC: 0.874
Features: ('age', 'platelets', 'serum_sodium', 'sex'), AUC: 0.542
Features: ('age', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.528
Features: ('age', 'platelets', 'serum_sodium', 'time'), AUC: 0.814
Features: ('age', 'platelets', 'sex', 'smoking'), AUC: 0.594
Features: ('age', 'platelets', 'sex', 'time'), AUC: 0.879
Features: ('age', 'platelets', 'smoking', 'time'), AUC: 0.859
Features: ('age', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.837
Features: ('age', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.730
Features: ('age', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.888
Features: ('age', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.740
Features: ('age', 'serum_creatinine', 'sex', 'time'), AUC: 0.887
Features: ('age', 'serum_creatinine', 'smoking', 'time'), AUC: 0.862
Features: ('age', 'serum_sodium', 'sex', 'smoking'), AUC: 0.514
Features: ('age', 'serum_sodium', 'sex', 'time'), AUC: 0.834
Features: ('age', 'serum_sodium', 'smoking', 'time'), AUC: 0.859
Features: ('age', 'sex', 'smoking', 'time'), AUC: 0.896
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'ejection_fraction'), AUC: 0.509
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'high_blood_pressure'), AUC: 0.491
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'platelets'), AUC: 0.525
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'serum_creatinine'), AUC: 0.865
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'serum_sodium'), AUC: 0.529
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'sex'), AUC: 0.654
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'smoking'), AUC: 0.567
Features: ('anaemia', 'creatinine_phosphokinase', 'diabetes', 'time'), AUC: 0.844
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.492
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'platelets'), AUC: 0.495
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine'), AUC: 0.811
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'serum_sodium'), AUC: 0.509
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'sex'), AUC: 0.609
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'smoking'), AUC: 0.456
Features: ('anaemia', 'creatinine_phosphokinase', 'ejection_fraction', 'time'), AUC: 0.854
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'platelets'), AUC: 0.487
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.832
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'serum_sodium'), AUC: 0.516
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'sex'), AUC: 0.586
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'smoking'), AUC: 0.538
Features: ('anaemia', 'creatinine_phosphokinase', 'high_blood_pressure', 'time'), AUC: 0.821
Features: ('anaemia', 'creatinine_phosphokinase', 'platelets', 'serum_creatinine'), AUC: 0.810
Features: ('anaemia', 'creatinine_phosphokinase', 'platelets', 'serum_sodium'), AUC: 0.549
Features: ('anaemia', 'creatinine_phosphokinase', 'platelets', 'sex'), AUC: 0.629
Features: ('anaemia', 'creatinine_phosphokinase', 'platelets', 'smoking'), AUC: 0.515
Features: ('anaemia', 'creatinine_phosphokinase', 'platelets', 'time'), AUC: 0.828
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_creatinine', 'serum_sodium'), AUC: 0.757
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_creatinine', 'sex'), AUC: 0.771
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_creatinine', 'smoking'), AUC: 0.840
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_creatinine', 'time'), AUC: 0.914
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_sodium', 'sex'), AUC: 0.469
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_sodium', 'smoking'), AUC: 0.434
Features: ('anaemia', 'creatinine_phosphokinase', 'serum_sodium', 'time'), AUC: 0.819
Features: ('anaemia', 'creatinine_phosphokinase', 'sex', 'smoking'), AUC: 0.636
Features: ('anaemia', 'creatinine_phosphokinase', 'sex', 'time'), AUC: 0.828
Features: ('anaemia', 'creatinine_phosphokinase', 'smoking', 'time'), AUC: 0.851
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.574
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'platelets'), AUC: 0.539
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'serum_creatinine'), AUC: 0.840
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'serum_sodium'), AUC: 0.586
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'sex'), AUC: 0.635
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'smoking'), AUC: 0.615
Features: ('anaemia', 'diabetes', 'ejection_fraction', 'time'), AUC: 0.868
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'platelets'), AUC: 0.517
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.729
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'serum_sodium'), AUC: 0.539
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'sex'), AUC: 0.505
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'smoking'), AUC: 0.590
Features: ('anaemia', 'diabetes', 'high_blood_pressure', 'time'), AUC: 0.799
Features: ('anaemia', 'diabetes', 'platelets', 'serum_creatinine'), AUC: 0.785
Features: ('anaemia', 'diabetes', 'platelets', 'serum_sodium'), AUC: 0.557
Features: ('anaemia', 'diabetes', 'platelets', 'sex'), AUC: 0.637
Features: ('anaemia', 'diabetes', 'platelets', 'smoking'), AUC: 0.528
Features: ('anaemia', 'diabetes', 'platelets', 'time'), AUC: 0.824
Features: ('anaemia', 'diabetes', 'serum_creatinine', 'serum_sodium'), AUC: 0.808
Features: ('anaemia', 'diabetes', 'serum_creatinine', 'sex'), AUC: 0.728
Features: ('anaemia', 'diabetes', 'serum_creatinine', 'smoking'), AUC: 0.743
Features: ('anaemia', 'diabetes', 'serum_creatinine', 'time'), AUC: 0.892
Features: ('anaemia', 'diabetes', 'serum_sodium', 'sex'), AUC: 0.517
Features: ('anaemia', 'diabetes', 'serum_sodium', 'smoking'), AUC: 0.542
Features: ('anaemia', 'diabetes', 'serum_sodium', 'time'), AUC: 0.826
Features: ('anaemia', 'diabetes', 'sex', 'smoking'), AUC: 0.616
Features: ('anaemia', 'diabetes', 'sex', 'time'), AUC: 0.894
Features: ('anaemia', 'diabetes', 'smoking', 'time'), AUC: 0.863
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'platelets'), AUC: 0.499
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.844
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'serum_sodium'), AUC: 0.522
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'sex'), AUC: 0.605
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'smoking'), AUC: 0.498
Features: ('anaemia', 'ejection_fraction', 'high_blood_pressure', 'time'), AUC: 0.874
Features: ('anaemia', 'ejection_fraction', 'platelets', 'serum_creatinine'), AUC: 0.739
Features: ('anaemia', 'ejection_fraction', 'platelets', 'serum_sodium'), AUC: 0.571
Features: ('anaemia', 'ejection_fraction', 'platelets', 'sex'), AUC: 0.531
Features: ('anaemia', 'ejection_fraction', 'platelets', 'smoking'), AUC: 0.424
Features: ('anaemia', 'ejection_fraction', 'platelets', 'time'), AUC: 0.840
Features: ('anaemia', 'ejection_fraction', 'serum_creatinine', 'serum_sodium'), AUC: 0.733
Features: ('anaemia', 'ejection_fraction', 'serum_creatinine', 'sex'), AUC: 0.837
Features: ('anaemia', 'ejection_fraction', 'serum_creatinine', 'smoking'), AUC: 0.840
Features: ('anaemia', 'ejection_fraction', 'serum_creatinine', 'time'), AUC: 0.917
Features: ('anaemia', 'ejection_fraction', 'serum_sodium', 'sex'), AUC: 0.610
Features: ('anaemia', 'ejection_fraction', 'serum_sodium', 'smoking'), AUC: 0.589
Features: ('anaemia', 'ejection_fraction', 'serum_sodium', 'time'), AUC: 0.877
Features: ('anaemia', 'ejection_fraction', 'sex', 'smoking'), AUC: 0.568
Features: ('anaemia', 'ejection_fraction', 'sex', 'time'), AUC: 0.850
Features: ('anaemia', 'ejection_fraction', 'smoking', 'time'), AUC: 0.870
Features: ('anaemia', 'high_blood_pressure', 'platelets', 'serum_creatinine'), AUC: 0.716
Features: ('anaemia', 'high_blood_pressure', 'platelets', 'serum_sodium'), AUC: 0.568
Features: ('anaemia', 'high_blood_pressure', 'platelets', 'sex'), AUC: 0.571
Features: ('anaemia', 'high_blood_pressure', 'platelets', 'smoking'), AUC: 0.401
Features: ('anaemia', 'high_blood_pressure', 'platelets', 'time'), AUC: 0.843
Features: ('anaemia', 'high_blood_pressure', 'serum_creatinine', 'serum_sodium'), AUC: 0.757
Features: ('anaemia', 'high_blood_pressure', 'serum_creatinine', 'sex'), AUC: 0.748
Features: ('anaemia', 'high_blood_pressure', 'serum_creatinine', 'smoking'), AUC: 0.839
Features: ('anaemia', 'high_blood_pressure', 'serum_creatinine', 'time'), AUC: 0.884
Features: ('anaemia', 'high_blood_pressure', 'serum_sodium', 'sex'), AUC: 0.507
Features: ('anaemia', 'high_blood_pressure', 'serum_sodium', 'smoking'), AUC: 0.546
Features: ('anaemia', 'high_blood_pressure', 'serum_sodium', 'time'), AUC: 0.825
Features: ('anaemia', 'high_blood_pressure', 'sex', 'smoking'), AUC: 0.587
Features: ('anaemia', 'high_blood_pressure', 'sex', 'time'), AUC: 0.833
Features: ('anaemia', 'high_blood_pressure', 'smoking', 'time'), AUC: 0.838
Features: ('anaemia', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.716
Features: ('anaemia', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.822
Features: ('anaemia', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.777
Features: ('anaemia', 'platelets', 'serum_creatinine', 'time'), AUC: 0.893
Features: ('anaemia', 'platelets', 'serum_sodium', 'sex'), AUC: 0.608
Features: ('anaemia', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.504
Features: ('anaemia', 'platelets', 'serum_sodium', 'time'), AUC: 0.825
Features: ('anaemia', 'platelets', 'sex', 'smoking'), AUC: 0.563
Features: ('anaemia', 'platelets', 'sex', 'time'), AUC: 0.898
Features: ('anaemia', 'platelets', 'smoking', 'time'), AUC: 0.808
Features: ('anaemia', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.668
Features: ('anaemia', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.627
Features: ('anaemia', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.883
Features: ('anaemia', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.781
Features: ('anaemia', 'serum_creatinine', 'sex', 'time'), AUC: 0.913
Features: ('anaemia', 'serum_creatinine', 'smoking', 'time'), AUC: 0.905
Features: ('anaemia', 'serum_sodium', 'sex', 'smoking'), AUC: 0.488
Features: ('anaemia', 'serum_sodium', 'sex', 'time'), AUC: 0.790
Features: ('anaemia', 'serum_sodium', 'smoking', 'time'), AUC: 0.806
Features: ('anaemia', 'sex', 'smoking', 'time'), AUC: 0.875
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'high_blood_pressure'), AUC: 0.638
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'platelets'), AUC: 0.601
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'serum_creatinine'), AUC: 0.812
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'serum_sodium'), AUC: 0.581
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'sex'), AUC: 0.700
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'smoking'), AUC: 0.657
Features: ('creatinine_phosphokinase', 'diabetes', 'ejection_fraction', 'time'), AUC: 0.864
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'platelets'), AUC: 0.539
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.810
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'serum_sodium'), AUC: 0.574
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'sex'), AUC: 0.554
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'smoking'), AUC: 0.509
Features: ('creatinine_phosphokinase', 'diabetes', 'high_blood_pressure', 'time'), AUC: 0.843
Features: ('creatinine_phosphokinase', 'diabetes', 'platelets', 'serum_creatinine'), AUC: 0.811
Features: ('creatinine_phosphokinase', 'diabetes', 'platelets', 'serum_sodium'), AUC: 0.566
Features: ('creatinine_phosphokinase', 'diabetes', 'platelets', 'sex'), AUC: 0.601
Features: ('creatinine_phosphokinase', 'diabetes', 'platelets', 'smoking'), AUC: 0.572
Features: ('creatinine_phosphokinase', 'diabetes', 'platelets', 'time'), AUC: 0.803
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_creatinine', 'serum_sodium'), AUC: 0.757
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_creatinine', 'sex'), AUC: 0.781
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_creatinine', 'smoking'), AUC: 0.811
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_creatinine', 'time'), AUC: 0.903
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_sodium', 'sex'), AUC: 0.522
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_sodium', 'smoking'), AUC: 0.580
Features: ('creatinine_phosphokinase', 'diabetes', 'serum_sodium', 'time'), AUC: 0.831
Features: ('creatinine_phosphokinase', 'diabetes', 'sex', 'smoking'), AUC: 0.623
Features: ('creatinine_phosphokinase', 'diabetes', 'sex', 'time'), AUC: 0.816
Features: ('creatinine_phosphokinase', 'diabetes', 'smoking', 'time'), AUC: 0.853
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'platelets'), AUC: 0.562
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.798
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'serum_sodium'), AUC: 0.551
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'sex'), AUC: 0.669
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'smoking'), AUC: 0.669
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'high_blood_pressure', 'time'), AUC: 0.851
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'platelets', 'serum_creatinine'), AUC: 0.783
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'platelets', 'serum_sodium'), AUC: 0.582
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'platelets', 'sex'), AUC: 0.611
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'platelets', 'smoking'), AUC: 0.607
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'platelets', 'time'), AUC: 0.802
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine', 'serum_sodium'), AUC: 0.728
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine', 'sex'), AUC: 0.771
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine', 'smoking'), AUC: 0.798
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_creatinine', 'time'), AUC: 0.912
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_sodium', 'sex'), AUC: 0.601
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_sodium', 'smoking'), AUC: 0.518
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'serum_sodium', 'time'), AUC: 0.847
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'sex', 'smoking'), AUC: 0.695
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'sex', 'time'), AUC: 0.836
Features: ('creatinine_phosphokinase', 'ejection_fraction', 'smoking', 'time'), AUC: 0.875
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'platelets', 'serum_creatinine'), AUC: 0.795
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'platelets', 'serum_sodium'), AUC: 0.538
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'platelets', 'sex'), AUC: 0.613
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'platelets', 'smoking'), AUC: 0.511
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'platelets', 'time'), AUC: 0.825
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine', 'serum_sodium'), AUC: 0.776
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine', 'sex'), AUC: 0.740
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine', 'smoking'), AUC: 0.812
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_creatinine', 'time'), AUC: 0.907
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_sodium', 'sex'), AUC: 0.534
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_sodium', 'smoking'), AUC: 0.521
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'serum_sodium', 'time'), AUC: 0.837
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'sex', 'smoking'), AUC: 0.587
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'sex', 'time'), AUC: 0.836
Features: ('creatinine_phosphokinase', 'high_blood_pressure', 'smoking', 'time'), AUC: 0.842
Features: ('creatinine_phosphokinase', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.742
Features: ('creatinine_phosphokinase', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.765
Features: ('creatinine_phosphokinase', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.792
Features: ('creatinine_phosphokinase', 'platelets', 'serum_creatinine', 'time'), AUC: 0.878
Features: ('creatinine_phosphokinase', 'platelets', 'serum_sodium', 'sex'), AUC: 0.535
Features: ('creatinine_phosphokinase', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.508
Features: ('creatinine_phosphokinase', 'platelets', 'serum_sodium', 'time'), AUC: 0.833
Features: ('creatinine_phosphokinase', 'platelets', 'sex', 'smoking'), AUC: 0.633
Features: ('creatinine_phosphokinase', 'platelets', 'sex', 'time'), AUC: 0.847
Features: ('creatinine_phosphokinase', 'platelets', 'smoking', 'time'), AUC: 0.841
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.717
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.689
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.890
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.775
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'sex', 'time'), AUC: 0.891
Features: ('creatinine_phosphokinase', 'serum_creatinine', 'smoking', 'time'), AUC: 0.905
Features: ('creatinine_phosphokinase', 'serum_sodium', 'sex', 'smoking'), AUC: 0.549
Features: ('creatinine_phosphokinase', 'serum_sodium', 'sex', 'time'), AUC: 0.832
Features: ('creatinine_phosphokinase', 'serum_sodium', 'smoking', 'time'), AUC: 0.839
Features: ('creatinine_phosphokinase', 'sex', 'smoking', 'time'), AUC: 0.830
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'platelets'), AUC: 0.576
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'serum_creatinine'), AUC: 0.755
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'serum_sodium'), AUC: 0.623
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'sex'), AUC: 0.622
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'smoking'), AUC: 0.666
Features: ('diabetes', 'ejection_fraction', 'high_blood_pressure', 'time'), AUC: 0.846
Features: ('diabetes', 'ejection_fraction', 'platelets', 'serum_creatinine'), AUC: 0.740
Features: ('diabetes', 'ejection_fraction', 'platelets', 'serum_sodium'), AUC: 0.533
Features: ('diabetes', 'ejection_fraction', 'platelets', 'sex'), AUC: 0.657
Features: ('diabetes', 'ejection_fraction', 'platelets', 'smoking'), AUC: 0.555
Features: ('diabetes', 'ejection_fraction', 'platelets', 'time'), AUC: 0.825
Features: ('diabetes', 'ejection_fraction', 'serum_creatinine', 'serum_sodium'), AUC: 0.683
Features: ('diabetes', 'ejection_fraction', 'serum_creatinine', 'sex'), AUC: 0.833
Features: ('diabetes', 'ejection_fraction', 'serum_creatinine', 'smoking'), AUC: 0.817
Features: ('diabetes', 'ejection_fraction', 'serum_creatinine', 'time'), AUC: 0.920
Features: ('diabetes', 'ejection_fraction', 'serum_sodium', 'sex'), AUC: 0.632
Features: ('diabetes', 'ejection_fraction', 'serum_sodium', 'smoking'), AUC: 0.630
Features: ('diabetes', 'ejection_fraction', 'serum_sodium', 'time'), AUC: 0.877
Features: ('diabetes', 'ejection_fraction', 'sex', 'smoking'), AUC: 0.737
Features: ('diabetes', 'ejection_fraction', 'sex', 'time'), AUC: 0.866
Features: ('diabetes', 'ejection_fraction', 'smoking', 'time'), AUC: 0.895
Features: ('diabetes', 'high_blood_pressure', 'platelets', 'serum_creatinine'), AUC: 0.759
Features: ('diabetes', 'high_blood_pressure', 'platelets', 'serum_sodium'), AUC: 0.549
Features: ('diabetes', 'high_blood_pressure', 'platelets', 'sex'), AUC: 0.620
Features: ('diabetes', 'high_blood_pressure', 'platelets', 'smoking'), AUC: 0.626
Features: ('diabetes', 'high_blood_pressure', 'platelets', 'time'), AUC: 0.846
Features: ('diabetes', 'high_blood_pressure', 'serum_creatinine', 'serum_sodium'), AUC: 0.730
Features: ('diabetes', 'high_blood_pressure', 'serum_creatinine', 'sex'), AUC: 0.708
Features: ('diabetes', 'high_blood_pressure', 'serum_creatinine', 'smoking'), AUC: 0.772
Features: ('diabetes', 'high_blood_pressure', 'serum_creatinine', 'time'), AUC: 0.881
Features: ('diabetes', 'high_blood_pressure', 'serum_sodium', 'sex'), AUC: 0.595
Features: ('diabetes', 'high_blood_pressure', 'serum_sodium', 'smoking'), AUC: 0.628
Features: ('diabetes', 'high_blood_pressure', 'serum_sodium', 'time'), AUC: 0.880
Features: ('diabetes', 'high_blood_pressure', 'sex', 'smoking'), AUC: 0.638
Features: ('diabetes', 'high_blood_pressure', 'sex', 'time'), AUC: 0.871
Features: ('diabetes', 'high_blood_pressure', 'smoking', 'time'), AUC: 0.875
Features: ('diabetes', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.734
Features: ('diabetes', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.729
Features: ('diabetes', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.768
Features: ('diabetes', 'platelets', 'serum_creatinine', 'time'), AUC: 0.895
Features: ('diabetes', 'platelets', 'serum_sodium', 'sex'), AUC: 0.521
Features: ('diabetes', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.499
Features: ('diabetes', 'platelets', 'serum_sodium', 'time'), AUC: 0.813
Features: ('diabetes', 'platelets', 'sex', 'smoking'), AUC: 0.578
Features: ('diabetes', 'platelets', 'sex', 'time'), AUC: 0.900
Features: ('diabetes', 'platelets', 'smoking', 'time'), AUC: 0.855
Features: ('diabetes', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.747
Features: ('diabetes', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.630
Features: ('diabetes', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.870
Features: ('diabetes', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.772
Features: ('diabetes', 'serum_creatinine', 'sex', 'time'), AUC: 0.880
Features: ('diabetes', 'serum_creatinine', 'smoking', 'time'), AUC: 0.899
Features: ('diabetes', 'serum_sodium', 'sex', 'smoking'), AUC: 0.568
Features: ('diabetes', 'serum_sodium', 'sex', 'time'), AUC: 0.853
Features: ('diabetes', 'serum_sodium', 'smoking', 'time'), AUC: 0.865
Features: ('diabetes', 'sex', 'smoking', 'time'), AUC: 0.882
Features: ('ejection_fraction', 'high_blood_pressure', 'platelets', 'serum_creatinine'), AUC: 0.743
Features: ('ejection_fraction', 'high_blood_pressure', 'platelets', 'serum_sodium'), AUC: 0.539
Features: ('ejection_fraction', 'high_blood_pressure', 'platelets', 'sex'), AUC: 0.516
Features: ('ejection_fraction', 'high_blood_pressure', 'platelets', 'smoking'), AUC: 0.512
Features: ('ejection_fraction', 'high_blood_pressure', 'platelets', 'time'), AUC: 0.826
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_creatinine', 'serum_sodium'), AUC: 0.716
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_creatinine', 'sex'), AUC: 0.791
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_creatinine', 'smoking'), AUC: 0.831
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_creatinine', 'time'), AUC: 0.910
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_sodium', 'sex'), AUC: 0.662
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_sodium', 'smoking'), AUC: 0.632
Features: ('ejection_fraction', 'high_blood_pressure', 'serum_sodium', 'time'), AUC: 0.894
Features: ('ejection_fraction', 'high_blood_pressure', 'sex', 'smoking'), AUC: 0.618
Features: ('ejection_fraction', 'high_blood_pressure', 'sex', 'time'), AUC: 0.844
Features: ('ejection_fraction', 'high_blood_pressure', 'smoking', 'time'), AUC: 0.871
Features: ('ejection_fraction', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.668
Features: ('ejection_fraction', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.762
Features: ('ejection_fraction', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.734
Features: ('ejection_fraction', 'platelets', 'serum_creatinine', 'time'), AUC: 0.905
Features: ('ejection_fraction', 'platelets', 'serum_sodium', 'sex'), AUC: 0.587
Features: ('ejection_fraction', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.546
Features: ('ejection_fraction', 'platelets', 'serum_sodium', 'time'), AUC: 0.862
Features: ('ejection_fraction', 'platelets', 'sex', 'smoking'), AUC: 0.588
Features: ('ejection_fraction', 'platelets', 'sex', 'time'), AUC: 0.859
Features: ('ejection_fraction', 'platelets', 'smoking', 'time'), AUC: 0.846
Features: ('ejection_fraction', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.725
Features: ('ejection_fraction', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.690
Features: ('ejection_fraction', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.908
Features: ('ejection_fraction', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.810
Features: ('ejection_fraction', 'serum_creatinine', 'sex', 'time'), AUC: 0.916
Features: ('ejection_fraction', 'serum_creatinine', 'smoking', 'time'), AUC: 0.933
Features: ('ejection_fraction', 'serum_sodium', 'sex', 'smoking'), AUC: 0.601
Features: ('ejection_fraction', 'serum_sodium', 'sex', 'time'), AUC: 0.875
Features: ('ejection_fraction', 'serum_sodium', 'smoking', 'time'), AUC: 0.889
Features: ('ejection_fraction', 'sex', 'smoking', 'time'), AUC: 0.873
Features: ('high_blood_pressure', 'platelets', 'serum_creatinine', 'serum_sodium'), AUC: 0.673
Features: ('high_blood_pressure', 'platelets', 'serum_creatinine', 'sex'), AUC: 0.705
Features: ('high_blood_pressure', 'platelets', 'serum_creatinine', 'smoking'), AUC: 0.734
Features: ('high_blood_pressure', 'platelets', 'serum_creatinine', 'time'), AUC: 0.877
Features: ('high_blood_pressure', 'platelets', 'serum_sodium', 'sex'), AUC: 0.553
Features: ('high_blood_pressure', 'platelets', 'serum_sodium', 'smoking'), AUC: 0.557
Features: ('high_blood_pressure', 'platelets', 'serum_sodium', 'time'), AUC: 0.823
Features: ('high_blood_pressure', 'platelets', 'sex', 'smoking'), AUC: 0.618
Features: ('high_blood_pressure', 'platelets', 'sex', 'time'), AUC: 0.893
Features: ('high_blood_pressure', 'platelets', 'smoking', 'time'), AUC: 0.840
Features: ('high_blood_pressure', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.688
Features: ('high_blood_pressure', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.710
Features: ('high_blood_pressure', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.890
Features: ('high_blood_pressure', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.776
Features: ('high_blood_pressure', 'serum_creatinine', 'sex', 'time'), AUC: 0.897
Features: ('high_blood_pressure', 'serum_creatinine', 'smoking', 'time'), AUC: 0.910
Features: ('high_blood_pressure', 'serum_sodium', 'sex', 'smoking'), AUC: 0.628
Features: ('high_blood_pressure', 'serum_sodium', 'sex', 'time'), AUC: 0.850
Features: ('high_blood_pressure', 'serum_sodium', 'smoking', 'time'), AUC: 0.871
Features: ('high_blood_pressure', 'sex', 'smoking', 'time'), AUC: 0.896
Features: ('platelets', 'serum_creatinine', 'serum_sodium', 'sex'), AUC: 0.692
Features: ('platelets', 'serum_creatinine', 'serum_sodium', 'smoking'), AUC: 0.688
Features: ('platelets', 'serum_creatinine', 'serum_sodium', 'time'), AUC: 0.865
Features: ('platelets', 'serum_creatinine', 'sex', 'smoking'), AUC: 0.758
Features: ('platelets', 'serum_creatinine', 'sex', 'time'), AUC: 0.898
Features: ('platelets', 'serum_creatinine', 'smoking', 'time'), AUC: 0.882
Features: ('platelets', 'serum_sodium', 'sex', 'smoking'), AUC: 0.505
Features: ('platelets', 'serum_sodium', 'sex', 'time'), AUC: 0.845
Features: ('platelets', 'serum_sodium', 'smoking', 'time'), AUC: 0.817
Features: ('platelets', 'sex', 'smoking', 'time'), AUC: 0.880
Features: ('serum_creatinine', 'serum_sodium', 'sex', 'smoking'), AUC: 0.645
Features: ('serum_creatinine', 'serum_sodium', 'sex', 'time'), AUC: 0.887
Features: ('serum_creatinine', 'serum_sodium', 'smoking', 'time'), AUC: 0.891
Features: ('serum_creatinine', 'sex', 'smoking', 'time'), AUC: 0.906
Features: ('serum_sodium', 'sex', 'smoking', 'time'), AUC: 0.848
Best Subset: ('ejection_fraction', 'serum_creatinine', 'smoking', 'time'), Best AUC on Validation: 0.933
Features Importance: [0.15765345 0.24700171 0.0250969 0.57024794]
Final Performance on Test Set using Best Subset ('ejection_fraction', 'serum_creatinine', 'smoking', 'time')
Test AUC: 0.945
Classification Report on Test Set:
precision recall f1-score support
0 0.90 0.90 0.90 20
1 0.80 0.80 0.80 10
accuracy 0.87 30
macro avg 0.85 0.85 0.85 30
weighted avg 0.87 0.87 0.87 30
Confusion Matrix:
[[18 2]
[ 2 8]]
SVM Accuracy: 0.8333333333333334
SVM Classification Report:
precision recall f1-score support
0 0.89 0.85 0.87 20
1 0.73 0.80 0.76 10
accuracy 0.83 30
macro avg 0.81 0.82 0.82 30
weighted avg 0.84 0.83 0.84 30
SVM Confusion Matrix:
[[17 3]
[ 2 8]]
Logistic Regression Accuracy: 0.8
Logistic Regression Classification Report:
precision recall f1-score support
0 0.82 0.90 0.86 20
1 0.75 0.60 0.67 10
accuracy 0.80 30
macro avg 0.78 0.75 0.76 30
weighted avg 0.80 0.80 0.79 30
Logistic Regression Confusion Matrix:
[[18 2]
[ 4 6]]
Without Smoothing nor best features selection
Final Performance on Test Set using Best Subset ('ejection_fraction', 'serum_creatinine', 'smoking', 'time')
Test AUC: 0.945
Classification Report on Test Set:
precision recall f1-score support
0 0.83 0.95 0.88 20
1 0.86 0.60 0.71 10
accuracy 0.83 30
macro avg 0.84 0.77 0.79 30
weighted avg 0.84 0.83 0.82 30
Confusion Matrix:
[[19 1]
[ 4 6]]
SVM Accuracy: 0.8333333333333334
SVM Classification Report:
precision recall f1-score support
0 0.83 0.95 0.88 20
1 0.86 0.60 0.71 10
accuracy 0.83 30
macro avg 0.84 0.77 0.79 30
weighted avg 0.84 0.83 0.82 30
SVM Confusion Matrix:
[[19 1]
[ 4 6]]
Logistic Regression Accuracy: 0.8333333333333334
Logistic Regression Classification Report:
precision recall f1-score support
0 0.83 0.95 0.88 20
1 0.86 0.60 0.71 10
accuracy 0.83 30
macro avg 0.84 0.77 0.79 30
weighted avg 0.84 0.83 0.82 30
Logistic Regression Confusion Matrix:
[[19 1]
[ 4 6]]
