Selecting dimensionality reduction with Pipeline and GridSearchCV

This example constructs a pipeline that does dimensionality reduction followed by prediction with a support vector classifier. It demonstrates the use of GridSearchCV and Pipeline to optimize over different classes of estimators in a single CV run – unsupervised PCA and NMF dimensionality reductions are compared to univariate feature selection during the grid search.

Additionally, Pipeline can be instantiated with the memory argument to memoize the transformers within the pipeline, avoiding to fit again the same transformers over and over.

Note that the use of memory to enable caching becomes interesting when the fitting of a transformer is costly.

Illustration of Pipeline and GridSearchCV

This section illustrates the use of a Pipeline with GridSearchCV
# Authors: Robert McGibbon, Joel Nothman, Guillaume Lemaitre

from __future__ import print_function, division

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2

print(__doc__)

pipe = Pipeline([
    ('reduce_dim', PCA()),
    ('classify', LinearSVC())
])

N_FEATURES_OPTIONS = [2, 4, 8]
C_OPTIONS = [1, 10, 100, 1000]
param_grid = [
    {
        'reduce_dim': [PCA(iterated_power=7), NMF()],
        'reduce_dim__n_components': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
    {
        'reduce_dim': [SelectKBest(chi2)],
        'reduce_dim__k': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
]
reducer_labels = ['PCA', 'NMF', 'KBest(chi2)']

grid = GridSearchCV(pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

mean_scores = np.array(grid.cv_results_['mean_test_score'])
# scores are in the order of param_grid iteration, which is alphabetical
mean_scores = mean_scores.reshape(len(C_OPTIONS), -1, len(N_FEATURES_OPTIONS))
# select score for best C
mean_scores = mean_scores.max(axis=0)
bar_offsets = (np.arange(len(N_FEATURES_OPTIONS)) *
               (len(reducer_labels) + 1) + .5)

plt.figure()
COLORS = 'bgrcmyk'
for i, (label, reducer_scores) in enumerate(zip(reducer_labels, mean_scores)):
    plt.bar(bar_offsets + i, reducer_scores, label=label, color=COLORS[i])

plt.title("Comparing feature reduction techniques")
plt.xlabel('Reduced number of features')
plt.xticks(bar_offsets + len(reducer_labels) / 2, N_FEATURES_OPTIONS)
plt.ylabel('Digit classification accuracy')
plt.ylim((0, 1))
plt.legend(loc='upper left')
../_images/sphx_glr_plot_compare_reduction_001.png

Caching transformers within a Pipeline

It is sometimes worthwhile storing the state of a specific transformer since it could be used again. Using a pipeline in GridSearchCV triggers such situations. Therefore, we use the argument memory to enable caching.

Warning

Note that this example is, however, only an illustration since for this specific case fitting PCA is not necessarily slower than loading the cache. Hence, use the memory constructor parameter when the fitting of a transformer is costly.

from tempfile import mkdtemp
from shutil import rmtree
from sklearn.externals.joblib import Memory

# Create a temporary folder to store the transformers of the pipeline
cachedir = mkdtemp()
memory = Memory(cachedir=cachedir, verbose=10)
cached_pipe = Pipeline([('reduce_dim', PCA()),
                        ('classify', LinearSVC())],
                       memory=memory)

# This time, a cached pipeline will be used within the grid search
grid = GridSearchCV(cached_pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

# Delete the temporary cache before exiting
rmtree(cachedir)

Out:

________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1cb15da5c1aa4df440f5dc8973b14f01
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/0b46392939726670c9b2d3402d952e0d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/be8f6f267ac8f5c42ae32910e7ca36e0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/62f633cb4350d20ae0990a5bfd5854f0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1fcb19c82b18df040b148bf3e8723dbd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3453dab8e42b96b6dff235006fd23802
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/04bf9d6d28983c0327a1b0ffd1e8f73c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/a7819e42989608d712ceafe0de4a9822
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/8e6c6961f9d7936a86b0ab1f5a108401
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/99ac2dadcf4f3e74c172935304717fb4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/7350e42b07df7a01972e2f480116a959
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/2c5939961ef767850e517e1b7d891955
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3222f7742a3ab6f15f49719b0a556938
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/239fa7703c6447fcf0a0d219c64ca94d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/46d4b8bacacd814450c55b3daef6ecb5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3a8eff04beca36aa3153173121b1d001
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3b0da8c3d3ef13aaa5725954840becf8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/43b676275dfa93c679c751f6ad4757d6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1cb15da5c1aa4df440f5dc8973b14f01
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/0b46392939726670c9b2d3402d952e0d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/be8f6f267ac8f5c42ae32910e7ca36e0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/62f633cb4350d20ae0990a5bfd5854f0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1fcb19c82b18df040b148bf3e8723dbd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3453dab8e42b96b6dff235006fd23802
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/04bf9d6d28983c0327a1b0ffd1e8f73c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/a7819e42989608d712ceafe0de4a9822
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/8e6c6961f9d7936a86b0ab1f5a108401
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/99ac2dadcf4f3e74c172935304717fb4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/7350e42b07df7a01972e2f480116a959
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/2c5939961ef767850e517e1b7d891955
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3222f7742a3ab6f15f49719b0a556938
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/239fa7703c6447fcf0a0d219c64ca94d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/46d4b8bacacd814450c55b3daef6ecb5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3a8eff04beca36aa3153173121b1d001
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3b0da8c3d3ef13aaa5725954840becf8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/43b676275dfa93c679c751f6ad4757d6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1cb15da5c1aa4df440f5dc8973b14f01
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/0b46392939726670c9b2d3402d952e0d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/be8f6f267ac8f5c42ae32910e7ca36e0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/62f633cb4350d20ae0990a5bfd5854f0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1fcb19c82b18df040b148bf3e8723dbd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3453dab8e42b96b6dff235006fd23802
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/04bf9d6d28983c0327a1b0ffd1e8f73c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/a7819e42989608d712ceafe0de4a9822
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/8e6c6961f9d7936a86b0ab1f5a108401
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/99ac2dadcf4f3e74c172935304717fb4
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/7350e42b07df7a01972e2f480116a959
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/2c5939961ef767850e517e1b7d891955
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3222f7742a3ab6f15f49719b0a556938
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/239fa7703c6447fcf0a0d219c64ca94d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/46d4b8bacacd814450c55b3daef6ecb5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3a8eff04beca36aa3153173121b1d001
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3b0da8c3d3ef13aaa5725954840becf8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/43b676275dfa93c679c751f6ad4757d6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x6259488>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/54806d420e5a1bdecc806a3b58b31ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1a59a6fa5a022056929a221c76b80b46
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c7c0b033c5b194aa33e0e08df030e162
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/44aca40f46845a8d86faf52b9588c49a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c2f7e1c959269c1fd429b7ad16c40389
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/9ae618ceb6a5e42f706e3a4c2521e88b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/d923fdaf1f9f51424f208f784802b120
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3600566b723849113b412d5daa3da9a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c05584bd30bfe3ca2acbead7d9ef8ff2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/54806d420e5a1bdecc806a3b58b31ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1a59a6fa5a022056929a221c76b80b46
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c7c0b033c5b194aa33e0e08df030e162
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/44aca40f46845a8d86faf52b9588c49a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c2f7e1c959269c1fd429b7ad16c40389
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/9ae618ceb6a5e42f706e3a4c2521e88b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/d923fdaf1f9f51424f208f784802b120
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3600566b723849113b412d5daa3da9a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c05584bd30bfe3ca2acbead7d9ef8ff2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/54806d420e5a1bdecc806a3b58b31ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/1a59a6fa5a022056929a221c76b80b46
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c7c0b033c5b194aa33e0e08df030e162
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/44aca40f46845a8d86faf52b9588c49a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c2f7e1c959269c1fd429b7ad16c40389
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/9ae618ceb6a5e42f706e3a4c2521e88b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/d923fdaf1f9f51424f208f784802b120
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/3600566b723849113b412d5daa3da9a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmphRwhCn/joblib/sklearn/pipeline/_fit_transform_one/c05584bd30bfe3ca2acbead7d9ef8ff2
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min

The PCA fitting is only computed at the evaluation of the first configuration of the C parameter of the LinearSVC classifier. The other configurations of C will trigger the loading of the cached PCA estimator data, leading to save processing time. Therefore, the use of caching the pipeline using memory is highly beneficial when fitting a transformer is costly.

Total running time of the script: ( 1 minutes 21.920 seconds)

Generated by Sphinx-Gallery