API

django_ai API is what makes possible to seamessly integrate different Statistical Models and Techniques into your Django project.

It provides an abstraction layer so you can integrate any machine learning or statistical engine to the framework and therefore into your project.

Design Goals

The API is designed to provide the main goals of Interchangeabilty, Persistance across requests and Separation from the User’s request cycle.

Interchangeabilty

Each Statistical Model or Technique implemented by an engine is isolated via the API and provides its functionality through the interface. This allows to interchange any Technique within a System or your code seamessly - provided that they are of the same type (i.e. Classification techniques).

This decoupling (or pattern) has very nice consecuences, such as allowing versioning to improve the models and algorithms independently.

Persistance across requests

The inference, calculations, state, etc. done by the engines must be available and persisted across requests.

Separation from the User’s request-response cycle

Heavy calculations should be taken away from the User’s request cycle, done independently and expose the relevant results in it.

The Application Programming interface

There are three main types of cases that django_ai provides: Systems, General Techniques and Techniques.

Systems use Techniques or General Techniques to make a full implementation of a Statistical Model on a particular task, providing “end-to-end functionality” - like a Spam Filter. They provide an API on their own, besides leveraging the API of Techniques.

General Techniques are those which provide a framework for implementing Techniques - like Bayesian Networks. They are like a System but they do not provide “end-to-end functionality” but building blocks instead.

Techniques are particular implementations of a Statistical Model providing the building block for constructing higher functionality.

Each one should be encapsulated in a Django model which inherits from the appropiate class from django_ai.base.models, providing all the functionality through the public API.

Statistical Model API

This is the base class for all, it defines the basic functionality that must be adhered to comply with the Design Goals of the API - besides each System or Technique implementing particular fields and methods for their functioning.

General Techniques should subclass this Abstract Model and implement the required methods.

class base.models.StatisticalModel(*args, **kwargs)

Metaclass for Learning Techniques.

It defines the common interface so the Techniques can be “plugged” along the framework and the applications.

General

Fields

StatisticalModel.name

Unique Name, meant to be used for retrieveing the object.

The name is meant to be the way the object is retrieved / referenced from other Techniques, Systems or plain code, so, it must be unique.

StatisticalModel.sm_type

Type of the Statistical Model

StatisticalModel.SM_TYPE_CHOICES = ((0, 'General'), (1, 'Classification'), (2, 'Regression'))

Choices for Statistical Model Type

StatisticalModel.metadata

Field for storing metadata (results and information related to internal tasks) of the System or Technique

This field for storing results and information related to internal tasks (pre-processing, visualization, etc.). It is initialized on .save()

Methods

StatisticalModel.rotate_metadata()

Rotates metadata from “current_inference” to “previous_inference” if it is not empty.

Engine-related

The Engine-related fields and methods are those that encapsulates a Statistical Engine and provide a uniform interface for the framework.

A Statistical Engine is the third-party implementation of an algorithm or technique which is being integrated, such as BayesPy or scikit-learn.

Fields

StatisticalModel.engine_object

This is where the main object of the Engine resides.

StatisticalModel.engine_object_timestamp

The timestamp of the Engine Object creation or last update

StatisticalModel.engine_meta_iterations

Number of times to run the Engine inference

In the cases where the Statistical Engine uses random initialization in the algorithm for performing inference, the result may depend on that initial state. If the engine does not provide a solution for this, a way of improving this is to run the Engine inference several times (Meta Iterations) and, given a measure of “improvement”, choose the best result.

The measure of improvement and the selection of the result must be implemented in the perform_inferece method.

StatisticalModel.engine_iterations

Engine Maximum iterations safeguard

The Statistical Engines usually provide a safeguard parameter to set the Maximum Iterations for the case when the convergence of the optimization method or algorithm is not guarranted or to avoid excessive run-time in some setups.

StatisticalModel.is_inferred

If Inference has been performed on the System or Technique

Methods

StatisticalModel.get_engine_object(reconstruct=False, save=True)

Returns the main object provided by the Statistical Engine.

It is responsible for initializing the Engine object if not exists - or is indicated by the “reconstruct” kwarg - and save it to the “engine_object” field.

StatisticalModel.reset_engine_object(save=True)

Resets the Engine-related fields. (engine_object, engine_object_timestamp, metadata and is_inferred).

StatisticalModel.perform_inference(recalculate=False, save=True)

Performs the Inference with the Statistical Engine and updates the Engine Object

StatisticalModel.reset_inference(save=True)

Base inference resetting (defaults to reset_engine_object())

Automation

For simple automation of the System or Technique into a project, three fields are provided: Counter, Counter Threshold and Threshold Actions.

StatisticalModel.counter

Automation: Internal Counter

StatisticalModel.counter_threshold

Automation: Internal Counter Threshold

StatisticalModel.threshold_actions

Automation: Actions to be run when the threshold is met.

The rationale is very simple: increment the counter until a threshold where actions are triggered.

Is up to the user when, where and how the counter is incremented. If the field Counter Threshold is set, when this counter reaches that Threshold, the actions in Threshold Actions will be run on the object’s save() method or the evaluation can be triggered with the following method:

StatisticalModel.parse_and_run_threshold_actions()

Parses and runs the thresholds actions.

IMPORTANT: The user should take care also to avoid triggering Threshold Actions inside of the user’s “navigation” request-response cycle, which may lead to hurt the user experience. For a concrete example, see here.

The allowed keywords for Threshold Actions are set in Model constant ACTIONS_KEYWORDS:

StatisticalModel.ACTIONS_KEYWORDS = [':recalculate']

Allowed Keywords for Threshold actions

which defaults to:

:recalculate

Recalculates (performs again) the inference on the model.

Supervised Learning Technique

This is the Base Class for Supervised Learning Techniques and Systems.

Besides having all the functionality of Statistical Model API, it defines the common interface for Supervised Learning.

class base.models.SupervisedLearningTechnique(*args, **kwargs)

Metaclass for Supervised Learning Techniques.

General

Fields

SupervisedLearningTechnique.sl_type Supervised Learning Type

Supervised Learning Type

SupervisedLearningTechnique.SL_TYPE_CHOICES = ((0, 'Classification'), (1, 'Regression'))

Choices for Supervised Learning Type

Engine-related

Fields

SupervisedLearningTechnique.cv_is_enabled

Enable Cross Validation (k-Folded)

Cross Validation (CV) is model validation technique, its goal is to estimate the performance of a predictive model in an independent dataset.

SupervisedLearningTechnique.cv_folds

Quantity of Folds to be used in Cross Validation

SupervisedLearningTechnique.cv_metric

Metric to be evaluated in Cross Validation

Methods

SupervisedLearningTechnique.perform_cross_validation(data=None, labels=None, update_metadata=False)

Performs Cross Validation with the current state of the model on the available data or in a given set.

Data-related

Fields

SupervisedLearningTechnique.labels_column

Field or Attribute containing the labels of the data

SupervisedLearningTechnique.pretraining

Django Model containing the pre-training dataset in the “app_label.model” format, i.e. “examples.SFPTEnron”

Methods

SupervisedLearningTechnique.get_labels()

Returns a list of labels of the data available for the model.

SupervisedLearningTechnique.get_pretraining_data()

Returns the pre-training data

SupervisedLearningTechnique.get_pretraining_labels()

Returns the pre-training labels

Unsupervised Learning Technique

This is the Base Class for Unsupervised Learning Techniques and Systems.

Besides having all the functionality of Statistical Model API, it defines the common interface for Unsupervised Learning.

class base.models.UnsupervisedLearningTechnique(*args, **kwargs)

Metaclass for Supervised Learning Techniques.