Checking Status

Notice – All publicly available Indico APIs will be deprecated on Jan 1, 2020

Whether you have mutliple Collections or would like to train a particularly large one and can’t wait for it to finish, checking status is key to managing your Collection.

Among other information, checking your Collection’s status returns metrics which help evaluate how useful your Collection is at solving the intended task. For classification problems, this means a mean class accuracy score and per-class precision and recall scores. For regression problems, mean absolute error and mean squared error metrics are returned. By inspecting these scores you can evaluate whether or not adding additonal data to your Collection or changing the Collection domain mayb be useful for improving the quality of your predictions. Please note that classification Collections may not return metrics if some of the classes have less than 10 examples.

Checking the status of your Collection does not count towards your monthly call volume.

Sample Response
{
    'image_tagger': {
        'model_type': 'classification',
        'input_type': 'image',
        'number_of_examples': 4000,
        'status': 'ready',
        'permissions': {
        	'read': [],
        	'write': []
        },
        'metrics': {
        	'class_accuracy': 0.875,
        	'precision': {
        		'cat': 0.9,
        		'dog': 0.875
        	},
        	'recall': {
        		'cat': 0.85,
        		'dog': 1.0
        	}
        },
        'public': False,
        'registered': False,
    },
    'detect_rapper': {
        'model_type': 'classification',
        'input_type': 'text',
        'number_of_examples': 120000,
        'status': 'ready',
        'permissions': {
        	'read': [],
        	'write': []
        },
        'public': False,
        'registered': False,
        ...
    }, ...
}
Sample Response – Annotation
{
  'input_type': 'text', 
  'model_type': 'sequence', 
  'domain': 'standard', 
  'name': 'sequence-labeling-demo', 
  'metrics': {
    'token_precision': {
      'Named Entity': 0.5714285714285714
    }, 
    'token_recall': {
      'Named Entity': 1.0
    },
    'token_counts': {
      'Named Entity': 12, 'None': 14
    }
  }, 
  'warnings': None, 
  'status': 'ready', 
  'number_of_examples': 10, 
  'permissions': {
    'read': [], 
    'write': []
  }, 
  'registered': False, 
  'public': False
}