Export Outputs to JSON

In this guide, we will demonstrate how to export the Check’s and the Suite’s output to JSON format.

This enables to use the exported results as a serializeable object that can later be shared or reviewed in other Python scopes, and also enables working with the check’s output (name, parameters, result value and result display) not only in a visual manner (e.g. by parsing the output JSON).

Notebook Structure:

• Load Data and Run Check

• Export Check to JSON

• Display Check from JSON

• Export Suite to JSON

Load Data

from deepchecks.datasets.classification import iris

train_dataset, test_dataset = iris.load_data()
model = iris.load_fitted_model()

Run a Check

from deepchecks.checks import ModelErrorAnalysis

result = ModelErrorAnalysis().add_condition_segments_performance_relative_difference_not_greater_than(\
                                ).run(train_dataset, test_dataset, model)

Observe CheckResult Display and Value

result

Model Error Analysis

Find features that best split the data into segments of high and low model error. Read More...

Conditions Summary

Status	Condition	More Info
!	The performance difference of the detected segments must not be greater than 5%	Found change in Accuracy in features above threshold: {'petal length (cm)': '8%', 'petal width (cm)': '8%'}

Additional Outputs

The following graphs show the distribution of error for top features that are most useful for distinguishing high error samples from low error samples. Top features are calculated using `feature_importances_`.

# and value can be observed
result.value

{'scorer_name': 'Accuracy',
 'feature_segments': {'petal length (cm)': {'segment1': {'score': 0.92,
    'n_samples': 25,
    'frac_samples': 0.6578947368421053},
   'segment2': {'score': 1.0,
    'n_samples': 13,
    'frac_samples': 0.34210526315789475}},
  'petal width (cm)': {'segment1': {'score': 0.92,
    'n_samples': 25,
    'frac_samples': 0.6578947368421053},
   'segment2': {'score': 1.0,
    'n_samples': 13,
    'frac_samples': 0.34210526315789475}}}}

Export a Check’s Output (CheckResult) to a JSON Format

Serialization of the output to a JSON format is possible using the to_json function. This function takes the check outputs and serializes it to a JSON string.

The format of the check’s output json includes all of the info required to reconstruct the check run and it’s output: it’s name, the parameters the check receives, one sentence summary of the check’s purpose, it’s result value and a json of the data relevant for displaying the check’s outputs

See Check JSON Structure

from deepchecks import CheckResult

help(CheckResult.to_json)

Help on function to_json in module deepchecks.base.check:

to_json(self, with_display: bool = True)
    Return check result as json.

    Parameters
    ----------
    with_display : bool
        controls if to serialize display or not

    Returns
    --------
    str
        {'name': .., 'params': .., 'header': ..,
         'summary': .., 'conditions_table': .., 'value', 'display': ..}

# get output JSON
serialized_output = result.to_json()

import json

# note - conditions_table key exists only if there are conditions that were defined on check
json.loads(serialized_output).keys()

dict_keys(['name', 'params', 'header', 'summary', 'conditions_table', 'value', 'display'])

Observe JSON Output

json.loads(serialized_output)

{'name': 'Model Error Analysis',
 'params': {'max_features_to_show': 3,
  'min_feature_contribution': 0.15,
  'min_error_model_score': 0.5,
  'min_segment_size': 0.05,
  'n_samples': 50000,
  'n_display_samples': 5000},
 'header': 'Model Error Analysis',
 'summary': 'Find features that best split the data into segments of high and low model error.',
 'conditions_table': '[{"Status":"<div style=\\"color: orange;text-align: center;font-weight:bold\\">!<\\/div>","Condition":"The performance difference of the detected segments must not be greater than 5%","More Info":"Found change in Accuracy in features above threshold: {\'petal length (cm)\': \'8%\', \'petal width (cm)\': \'8%\'}"}]',
 'value': {'scorer_name': 'Accuracy',
  'feature_segments': {'petal length (cm)': {'segment1': {'score': {'py/reduce': [{'py/function': 'numpy.core.multiarray.scalar'},
       {'py/tuple': [{'py/reduce': [{'py/type': 'numpy.dtype'},
           {'py/tuple': ['f8', False, True]},
           {'py/tuple': [3, '<', None, None, None, -1, -1, 0]}]},
         {'py/b64': 'cT0K16Nw7T8='}]}]},
     'n_samples': 25,
     'frac_samples': 0.6578947368421053},
    'segment2': {'score': {'py/reduce': [{'py/function': 'numpy.core.multiarray.scalar'},
       {'py/tuple': [{'py/id': 7}, {'py/b64': 'AAAAAAAA8D8='}]}]},
     'n_samples': 13,
     'frac_samples': 0.34210526315789475}},
   'petal width (cm)': {'segment1': {'score': {'py/reduce': [{'py/function': 'numpy.core.multiarray.scalar'},
       {'py/tuple': [{'py/id': 7}, {'py/b64': 'cT0K16Nw7T8='}]}]},
     'n_samples': 25,
     'frac_samples': 0.6578947368421053},
    'segment2': {'score': {'py/reduce': [{'py/function': 'numpy.core.multiarray.scalar'},
       {'py/tuple': [{'py/id': 7}, {'py/b64': 'AAAAAAAA8D8='}]}]},
     'n_samples': 13,
     'frac_samples': 0.34210526315789475}}}},
 'display': [{'py/tuple': ['html',
    '<span>\n            The following graphs show the distribution of error for top features that are most useful for distinguishing\n            high error samples from low error samples. Top features are calculated using `feature_importances_`.\n        </span>']},
  {'py/tuple': ['plotly',
    '{"data":[{"hovertemplate":"color=Accuracy: 1, Samples: 13 (34.21%)<br>petal length (cm)=%{x}<br>model error=%{y}<extra></extra>","legendgroup":"Accuracy: 1, Samples: 13 (34.21%)","marker":{"color":"#69b3a2","symbol":"circle"},"mode":"markers","name":"Accuracy: 1, Samples: 13 (34.21%)","orientation":"v","showlegend":true,"x":[1.5,1.6,1.7,1.5,1.5,1.5,1.5,1.9,1.4,1.4,1.5,1.3,1.4],"xaxis":"x","y":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"yaxis":"y","type":"scatter"},{"hovertemplate":"color=Accuracy: 0.92, Samples: 25 (65.79%)<br>petal length (cm)=%{x}<br>model error=%{y}<extra></extra>","legendgroup":"Accuracy: 0.92, Samples: 25 (65.79%)","marker":{"color":"#d74949","symbol":"circle"},"mode":"markers","name":"Accuracy: 0.92, Samples: 25 (65.79%)","orientation":"v","showlegend":true,"x":[4.0,4.1,4.6,4.7,4.1,5.8,5.6,5.4,4.5,4.7,5.0,4.4,4.5,5.6,5.1,3.7,5.6,5.1,4.1,6.0,5.2,6.1,4.9,4.4,4.5],"xaxis":"x","y":[68.53406762210857,66.32935635986169,49.957091686276,45.287598380531506,68.28013306230294,89.54833095987044,76.24853908883159,134.69491832183357,62.72779942768973,44.41105244990409,77.10798645611227,51.496235683455495,43.485400501081465,119.99165575606166,75.45458639466798,66.05516348550101,100.1897669968595,71.01113317605152,65.28903189134176,122.59218665880665,128.1109893252574,142.0193023043133,75.86127936898247,68.36825217004687,44.36200576986826],"yaxis":"y","type":"scatter"}],"layout":{"template":{"data":{"bar":[{"error_x":{"color":"#2a3f5f"},"error_y":{"color":"#2a3f5f"},"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"bar"}],"barpolar":[{"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"barpolar"}],"carpet":[{"aaxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"baxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"type":"carpet"}],"choropleth":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"choropleth"}],"contour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"contour"}],"contourcarpet":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"contourcarpet"}],"heatmap":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmap"}],"heatmapgl":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmapgl"}],"histogram":[{"marker":{"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"histogram"}],"histogram2d":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2d"}],"histogram2dcontour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2dcontour"}],"mesh3d":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"mesh3d"}],"parcoords":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"parcoords"}],"pie":[{"automargin":true,"type":"pie"}],"scatter":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter"}],"scatter3d":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter3d"}],"scattercarpet":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattercarpet"}],"scattergeo":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergeo"}],"scattergl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergl"}],"scattermapbox":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattermapbox"}],"scatterpolar":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolar"}],"scatterpolargl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolargl"}],"scatterternary":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterternary"}],"surface":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"surface"}],"table":[{"cells":{"fill":{"color":"#EBF0F8"},"line":{"color":"white"}},"header":{"fill":{"color":"#C8D4E3"},"line":{"color":"white"}},"type":"table"}]},"layout":{"annotationdefaults":{"arrowcolor":"#2a3f5f","arrowhead":0,"arrowwidth":1},"autotypenumbers":"strict","coloraxis":{"colorbar":{"outlinewidth":0,"ticks":""}},"colorscale":{"diverging":[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]],"sequential":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"sequentialminus":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]},"colorway":["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"],"font":{"color":"#2a3f5f"},"geo":{"bgcolor":"white","lakecolor":"white","landcolor":"#E5ECF6","showlakes":true,"showland":true,"subunitcolor":"white"},"hoverlabel":{"align":"left"},"hovermode":"closest","mapbox":{"style":"light"},"paper_bgcolor":"white","plot_bgcolor":"#E5ECF6","polar":{"angularaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","radialaxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"scene":{"xaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"yaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"zaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"}},"shapedefaults":{"line":{"color":"#2a3f5f"}},"ternary":{"aaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"baxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","caxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"title":{"x":0.05},"xaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2},"yaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2}}},"xaxis":{"anchor":"y","domain":[0.0,1.0],"title":{"text":"petal length (cm)"}},"yaxis":{"anchor":"x","domain":[0.0,1.0],"title":{"text":"model error"}},"legend":{"title":{"text":"color"},"tracegroupgap":0},"title":{"text":"Segmentation of error by petal length (cm)"},"width":1200,"height":400}}']},
  {'py/tuple': ['plotly',
    '{"data":[{"hovertemplate":"color=Accuracy: 1, Samples: 13 (34.21%)<br>petal width (cm)=%{x}<br>model error=%{y}<extra></extra>","legendgroup":"Accuracy: 1, Samples: 13 (34.21%)","marker":{"color":"#69b3a2","symbol":"circle"},"mode":"markers","name":"Accuracy: 1, Samples: 13 (34.21%)","orientation":"v","showlegend":true,"x":[0.3,0.2,0.5,0.2,0.2,0.2,0.4,0.2,0.1,0.2,0.1,0.2,0.3],"xaxis":"x","y":[0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0],"yaxis":"y","type":"scatter"},{"hovertemplate":"color=Accuracy: 0.92, Samples: 25 (65.79%)<br>petal width (cm)=%{x}<br>model error=%{y}<extra></extra>","legendgroup":"Accuracy: 0.92, Samples: 25 (65.79%)","marker":{"color":"#d74949","symbol":"circle"},"mode":"markers","name":"Accuracy: 0.92, Samples: 25 (65.79%)","orientation":"v","showlegend":true,"x":[1.3,1.0,1.5,1.4,1.3,1.6,1.4,2.1,1.3,1.5,2.0,1.4,1.6,2.4,1.9,1.0,1.8,1.8,1.3,2.5,2.0,2.5,2.0,1.2,1.5],"xaxis":"x","y":[68.53406762210857,66.32935635986169,49.957091686276,45.287598380531506,68.28013306230294,89.54833095987044,76.24853908883159,134.69491832183357,62.72779942768973,44.41105244990409,77.10798645611227,51.496235683455495,43.485400501081465,119.99165575606166,75.45458639466798,66.05516348550101,100.1897669968595,71.01113317605152,65.28903189134176,122.59218665880665,128.1109893252574,142.0193023043133,75.86127936898247,68.36825217004687,44.36200576986826],"yaxis":"y","type":"scatter"}],"layout":{"template":{"data":{"bar":[{"error_x":{"color":"#2a3f5f"},"error_y":{"color":"#2a3f5f"},"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"bar"}],"barpolar":[{"marker":{"line":{"color":"#E5ECF6","width":0.5},"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"barpolar"}],"carpet":[{"aaxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"baxis":{"endlinecolor":"#2a3f5f","gridcolor":"white","linecolor":"white","minorgridcolor":"white","startlinecolor":"#2a3f5f"},"type":"carpet"}],"choropleth":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"choropleth"}],"contour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"contour"}],"contourcarpet":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"contourcarpet"}],"heatmap":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmap"}],"heatmapgl":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"heatmapgl"}],"histogram":[{"marker":{"pattern":{"fillmode":"overlay","size":10,"solidity":0.2}},"type":"histogram"}],"histogram2d":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2d"}],"histogram2dcontour":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"histogram2dcontour"}],"mesh3d":[{"colorbar":{"outlinewidth":0,"ticks":""},"type":"mesh3d"}],"parcoords":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"parcoords"}],"pie":[{"automargin":true,"type":"pie"}],"scatter":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter"}],"scatter3d":[{"line":{"colorbar":{"outlinewidth":0,"ticks":""}},"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatter3d"}],"scattercarpet":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattercarpet"}],"scattergeo":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergeo"}],"scattergl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattergl"}],"scattermapbox":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scattermapbox"}],"scatterpolar":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolar"}],"scatterpolargl":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterpolargl"}],"scatterternary":[{"marker":{"colorbar":{"outlinewidth":0,"ticks":""}},"type":"scatterternary"}],"surface":[{"colorbar":{"outlinewidth":0,"ticks":""},"colorscale":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"type":"surface"}],"table":[{"cells":{"fill":{"color":"#EBF0F8"},"line":{"color":"white"}},"header":{"fill":{"color":"#C8D4E3"},"line":{"color":"white"}},"type":"table"}]},"layout":{"annotationdefaults":{"arrowcolor":"#2a3f5f","arrowhead":0,"arrowwidth":1},"autotypenumbers":"strict","coloraxis":{"colorbar":{"outlinewidth":0,"ticks":""}},"colorscale":{"diverging":[[0,"#8e0152"],[0.1,"#c51b7d"],[0.2,"#de77ae"],[0.3,"#f1b6da"],[0.4,"#fde0ef"],[0.5,"#f7f7f7"],[0.6,"#e6f5d0"],[0.7,"#b8e186"],[0.8,"#7fbc41"],[0.9,"#4d9221"],[1,"#276419"]],"sequential":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]],"sequentialminus":[[0.0,"#0d0887"],[0.1111111111111111,"#46039f"],[0.2222222222222222,"#7201a8"],[0.3333333333333333,"#9c179e"],[0.4444444444444444,"#bd3786"],[0.5555555555555556,"#d8576b"],[0.6666666666666666,"#ed7953"],[0.7777777777777778,"#fb9f3a"],[0.8888888888888888,"#fdca26"],[1.0,"#f0f921"]]},"colorway":["#636efa","#EF553B","#00cc96","#ab63fa","#FFA15A","#19d3f3","#FF6692","#B6E880","#FF97FF","#FECB52"],"font":{"color":"#2a3f5f"},"geo":{"bgcolor":"white","lakecolor":"white","landcolor":"#E5ECF6","showlakes":true,"showland":true,"subunitcolor":"white"},"hoverlabel":{"align":"left"},"hovermode":"closest","mapbox":{"style":"light"},"paper_bgcolor":"white","plot_bgcolor":"#E5ECF6","polar":{"angularaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","radialaxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"scene":{"xaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"yaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"},"zaxis":{"backgroundcolor":"#E5ECF6","gridcolor":"white","gridwidth":2,"linecolor":"white","showbackground":true,"ticks":"","zerolinecolor":"white"}},"shapedefaults":{"line":{"color":"#2a3f5f"}},"ternary":{"aaxis":{"gridcolor":"white","linecolor":"white","ticks":""},"baxis":{"gridcolor":"white","linecolor":"white","ticks":""},"bgcolor":"#E5ECF6","caxis":{"gridcolor":"white","linecolor":"white","ticks":""}},"title":{"x":0.05},"xaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2},"yaxis":{"automargin":true,"gridcolor":"white","linecolor":"white","ticks":"","title":{"standoff":15},"zerolinecolor":"white","zerolinewidth":2}}},"xaxis":{"anchor":"y","domain":[0.0,1.0],"title":{"text":"petal width (cm)"}},"yaxis":{"anchor":"x","domain":[0.0,1.0],"title":{"text":"model error"}},"legend":{"title":{"text":"color"},"tracegroupgap":0},"title":{"text":"Segmentation of error by petal width (cm)"},"width":1200,"height":400}}']}]}

Display the CheckResult Output from a JSON

The serialized JSON can be used to reproduce the run in other Python scopes, by using the display_from_json function

CheckResult.display_from_json(serialized_output)

Model Error Analysis

Find features that best split the data into segments of high and low model error.

Conditions Summary

Status	Condition	More Info
!	The performance difference of the detected segments must not be greater than 5%	Found change in Accuracy in features above threshold: {'petal length (cm)': '8%', 'petal width (cm)': '8%'}

Additional Outputs

The following graphs show the distribution of error for top features that are most useful for distinguishing high error samples from low error samples. Top features are calculated using `feature_importances_`.

Exporting a Suite’s Output (SuiteResult) to JSON

Run Suite and Save to JSON

from deepchecks.suites import full_suite

suite = full_suite()

suite_result = suite.run(train_dataset=train_dataset, test_dataset=test_dataset, model=model)
suite_json = suite_result.to_json()

Observe Suite’s JSON Strucutre

from deepchecks import SuiteResult

help(SuiteResult.to_json)

Help on function to_json in module deepchecks.base.suite:

to_json(self, with_display: bool = True)
    Return check result as json.

    Parameters
    ----------
    with_display : bool
        controls if to serialize display of checks or not

    Returns
    -------
    dict
        {'name': .., 'results': ..}

Suite name:

json.loads(suite_json)['name']

'Full Suite'

Results is an array of CheckResult JSON’s, let’s see how many checks ran in suite:

len(json.loads(suite_json)['results'])

52

Lets observe a specific check’s output, which is similar to the Check output’s JSON Structure we saw before for each check

json.loads(json.loads(suite_json)['results'][0])

{'name': 'Model Info',
 'params': {},
 'header': 'Model Info',
 'summary': 'Summarize given model parameters.',
 'value': {'type': 'RandomForestClassifier',
  'params': {'bootstrap': True,
   'ccp_alpha': 0.0,
   'class_weight': None,
   'criterion': 'gini',
   'max_depth': None,
   'max_features': 'auto',
   'max_leaf_nodes': None,
   'max_samples': None,
   'min_impurity_decrease': 0.0,
   'min_samples_leaf': 1,
   'min_samples_split': 2,
   'min_weight_fraction_leaf': 0.0,
   'n_estimators': 100,
   'n_jobs': None,
   'oob_score': False,
   'random_state': None,
   'verbose': 0,
   'warm_start': False}},
 'display': [{'py/tuple': ['html', 'Model Type: RandomForestClassifier']},
  {'py/tuple': ['dataframe',
    '[{"Parameter":"bootstrap","Value":true,"Default":true},{"Parameter":"ccp_alpha","Value":0.0,"Default":0.0},{"Parameter":"class_weight","Value":null,"Default":null},{"Parameter":"criterion","Value":"gini","Default":"gini"},{"Parameter":"max_depth","Value":null,"Default":null},{"Parameter":"max_features","Value":"auto","Default":"auto"},{"Parameter":"max_leaf_nodes","Value":null,"Default":null},{"Parameter":"max_samples","Value":null,"Default":null},{"Parameter":"min_impurity_decrease","Value":0.0,"Default":0.0},{"Parameter":"min_samples_leaf","Value":1,"Default":1},{"Parameter":"min_samples_split","Value":2,"Default":2},{"Parameter":"min_weight_fraction_leaf","Value":0.0,"Default":0.0},{"Parameter":"n_estimators","Value":100,"Default":100},{"Parameter":"n_jobs","Value":null,"Default":null},{"Parameter":"oob_score","Value":false,"Default":false},{"Parameter":"random_state","Value":null,"Default":null},{"Parameter":"verbose","Value":0,"Default":0},{"Parameter":"warm_start","Value":false,"Default":false}]']},
  {'py/tuple': ['html',
    '<p style="font-size:0.7em"><i>Colored rows are parameters with non-default values</i></p>']}]}

previous

Export Suite Output to a HTML Report

next

Export Outputs to Weights & Biases (wandb)