API documentation

Connect

Convenience functions

elastic_connect.connect.create_mappings(model_classes)

Shortcut for _namespaces[‘_default’].create_mappings

Creates index mapping in Elasticsearch for each model passed in. Doesn’t update existing mappings.

Parameters:model_classes – a list of classes for which indices are created Returns:returns the names of indices which were actually created

elastic_connect.connect.connect(conf, index_prefix='')

Establish a connection to elasticsearch using the _default namespace.

Parameters:

• conf – The parameters of the _default namespace
• index_prefix – prefix to be used for all indices using this connection. Default = ‘’

Returns:

instance of the _default Namespace

elastic_connect.connect.delete_indices(indices)

Delete all of the provided indices. Blocks untill they are deleted.

Unlike the create_mappings and other operations, index deletes in elastic_connect don’t perform any index_name prefix magic. All index deletions in elastic_connect are attempted with the name provided ‘as is’.

Parameters:indices – names of indices to be deleted Returns:None

Classes

class elastic_connect.connect.DocTypeConnection(model, es_namespace, index, doc_type, default_args={})

Connection for a specific model to Elasticsearch.

In ES >= 6 each model type needs it’s own index.

deprecated - ES < 6 supports multiple doc_types in a single index.

Parameters:

• model – class of the model for which the connection is created
• es_namespace – es_namespace in which the connection is created
• index – name of the index
• doc_type – name of the doc type - for future compliance should match the index name
• default_args – a dict of default args to pass to the underlying elasticsearch connections

__getattr__(name)

All methods are redirected to the underlying elasticsearch connection. Search and get methods return Result on success, otherwise the JSON from elasticseach is returned.

class elastic_connect.connect.Result(result, model, method, pass_args)

Handles the conversion of Elasticsearch query results to models.

search_after()

Utilizes the Elasticsearch search_after capability to perform some real-time scrolling through the results. Uses the parameters of the search that generated this result to perform another search, with the last models order values as search_after values.

Example:

found = models.find_by(website='www.zive.cz', size=10)
# first 10 results (sorted by _uid in ascending order - the
# default) are returned

found.search_after()
# further 10 results (sorted by _uid) are returned

Returns:further results

BaseModel

exception elastic_connect.base_model.IntegrityError

class elastic_connect.base_model.Model(**kw)

Base class for Models stored in elasticseach.

Handles creation, loading and saving of the models. Also handles simple SQL-like joins, in fact lazy loading dependent child/parent models.

Creates an instance of the model using **kw parameters for setting values of attributes. The values get converted by their respective data_type.from_python method.

_compute_id()

Count or return stored id for this model instance.

Returns:None for unsaved models which should receive a unique id generated by Elasticsearch. Should be overriden and return some computed value for models which handle their uniqe id by themselves - mainly to keep a model parameter unique

classmethod _create(model)

Handles the creation of the model in elasticsearch Models without an id are indexed, thus receiving id from elasticsearch. Models with id are created. This prevents the creation of duplicates.

Parameters:model – the model to be created Returns:the model with the id set

_lazy_load()

Lazy loads model’s joins - child / parent models.

_mapping = {'id': <elastic_connect.data_types.base.Keyword object>{'name': 'id'}}

Dictionary describing the model. property_name: elasticsearch data type or ‘ref’ for reference to other model, defined by a join keys starting with _ are not saved in elasticsearch

classmethod all(size=100, sort=None)

Get all models from Elasticsearch. :param size: max number of hits to return. Default = 100. :param sort: sorting of the result as provided by

prepare_sort(sort)

Returns:returns an instance of elastic_connect.connect.Result

classmethod create(**kw) → elastic_connect.base_model.Model

Create, save and return a model instance based on dictionary. Property id gets set by elasticsearch or computed depending on cls._compute_id()

Parameters:kw – keyword arguments describing the model’s attributes Returns:instance of the model with the id set

delete()

Delete a model from elasticsearch.

Returns:None

classmethod find_by(size=100, sort=None, search_after=None, query=None, **kw)

Search for models in Elasticsearch by attribute values.

Example:

# return model with email="test@test.cz"
model.find_by(email="test@test.cz")

# return model with both email="test@test.cz" and parent=10
model.find_by(email="test@test.cz", parent=10)

# return models with parent 10 sorted by email ascending
model.find_by(parent=10, sort=[{"email":"asc"}])

# return models with email >= "foo@bar.cz" (and _uid > '' as
# per default sort order, every _uid is greated than '')
model.find_by(parent=10,
              sort=[{"email":"asc"}],
              search_after["foo@bar.cz", ''])

# return models with parent 10 and email _anything_@bar.cz
model.find_by(query="parent: 10 AND email: *@bar.cz")

Parameters:

• size – max number of hits to return. Default = 100.
• kw – attributes of the model by which to search
• sort – sorting of the result as provided by prepare_sort(sort)
• search_after – searches for results ‘after’ the value(s) supplied, preferably used with elastic_connect.connect.Result.search_after_values
• query – instead of specifying kw search arguments, you may enter here a wildcard query

Returns:

returns an instance of elastic_connect.connect.Result

classmethod from_dict(**kw)

Create and return an unsaved model instance based on dict.

Parameters:kw – keyword arguments describing the model’s attributes Returns:instance of the model

classmethod from_es(hit)

Create and return an unsaved model instance based on elasticsearch query result.

Parameters:hit – a hit from an elasticsearch query Returns:instance of the model

classmethod get(id)

Get a model by id from elasticsearch.

Parameters:id – id of the model to get Returns:returns an instance of elastic_connect.connect.Result

classmethod get_default_sort()

Returns the default sort order, which is used by find_by() and all() if no other sorting is explicitly provided in their call.

classmethod get_doctype()

Deprecated:

Returns the name of the index this model is stored in, including any prefixes defined globally or in namespace.

In ES >= 6 each model type needs it’s own index.

ES < 6 supports multiple doc_types in a single index.

classmethod get_es_connection()

Initializes or returns an existing DocTypeConnection to elasticsearch for this model.

Returns:DocTypeConnection

classmethod get_es_mapping()

Returns a dict representing the elastic search mapping for this model

Returns:dict

classmethod get_index()

Returns the name of the index this model is stored

in, including any prefixes defined globally or in namespace.

In ES >= 6 each model type needs it’s own index.

ES < 6 supports multiple doc_types in a single index.

classmethod prepare_sort(sort=None, stringify=False)

Prepares sorting for model. Defaults to get_default_sort, {“_uid”: “asc”} is also appended as last resort to all sorts that don’t use _uid. Sorting by _id is not supported by elasticsearch, use _uid (_doc_type + ‘#’ + _id) instead. Important: _uid is not incremental in elasticsearch, it’s here just to get constistent results on the same dataset. :param sort: array of {property: “asc|desc”} values :param stringify: default False: if the result should be

stringified for kw parameter, or left in the json format for body of Elasticsearch query.

Returns:returns the input sort with appended {“_uid”: “asc”}

classmethod refresh()

Refresh the index where this model is stored to make all changes immediately visible to others.

save()

Save a model that has an id, index a model without an id into elasticsearch. Saves unsaved joins recursively. Joined models, which already have an id (and thus are already present in the database) are not re-saved automatically. You must save them yourself if they changed.

Returns:self with dependencies updated

serialize(exclude=['password'], depth=0, to_str=False, flat=False)

Serilaizes the model for storing to elasticsearch.

Joins are flattened from join: model format to join: model.id format. Other attributes are serialized by their respective type.serialize method

Parameters:

• exclude – default=[“password”]
• depth – default=0, the depth up to which models are serialized as jsons, deeper than that models are reduced to their id
• to_str – default=False, pass True if the serialization is for console output purposes
• flat – default=False, unsaved joined models are returned as Models if False, as None if True

Returns:

json representation of the model

DataTypes

Base

class elastic_connect.data_types.base.BaseDataType(name)

class elastic_connect.data_types.base.Boolean(name)

class elastic_connect.data_types.base.Date(name)

class elastic_connect.data_types.base.Integer(name)

class elastic_connect.data_types.base.Keyword(name)

class elastic_connect.data_types.base.Long(name)

class elastic_connect.data_types.base.ScaledFloat(name, scaling_factor)

class elastic_connect.data_types.base.Text(name)

Join

class elastic_connect.data_types.join.Join(name: str, source: str, target: str)

Abstract parent of model joins - dependent child / parent models.

static class_for_name(module_name: str, class_name: str)

Handles import of target model class. Needed to prevent circular dependency.

get_source()

Gets the source model of the join.

get_target()

Gets the target model of the join.

class elastic_connect.data_types.join.LooseJoin(name: str, source: str, target: str)

class elastic_connect.data_types.join.MultiJoin(name: str, source: str, target: str, join_by=None)

1:N model join.

class elastic_connect.data_types.join.MultiJoinLoose(name: str, source: str, target: str, join_by=None, do_lazy_load=False)

Important! Dosen’t preserve order!

class elastic_connect.data_types.join.SingleJoin(name: str, source: str, target: str)

1:1 model join.

class elastic_connect.data_types.join.SingleJoinLoose(name: str, source: str, target: str, do_lazy_load=False)

Namespaces

elastic_connect.namespace._global_prefix

Global index prefix. Used for example to distinguish between index names used in production and in tests.

elastic_connect.namespace._namespaces

A singleton dict containing all registered namespaces indexed by their names.

elastic_connect.namespace.register_namespace(namespace: elastic_connect.namespace.Namespace)

Register a new namespace. Changing a Namespace’s parameters after it was registered may do crazy things, don’t do it.

Parameters:namespace – Namespace instance to be registered Returns:None Raises:NamespaceAlreadyExistsError if a Namespace with the same name already exists

class elastic_connect.namespace.Namespace(name, es_conf, index_prefix=None)

Object describing a namespace of an elasticsearch cluster or a connection to a different elasticsearch cluster. Each namespace may have a different es_conf, thus connecting to a different elasticsearch cluster and/or a different index_prefix thus using a different set of indices on the same cluster.

For example you may use two different namespaces to run two instances of the same application against a single elasticsearch cluster. Due to using different index_prefixes on the _default namespace, each application will preserve it’s own data, i.e. one, with index_prefix="our" using indices our_users and our_data, the other with index_prefix="their" using indices their_users and their_data.

It is also possible to use multiple namespaces in a single application.

Parameters:

• name – name of the namespace, must be unique
• es_conf – the configuration of the namespace i.e. at least {‘host’:…, ‘port’:…}. It is internally passed to the underlaying elasticsearch.Elasticsearch class.
• index_prefix – prefix of the namespace, it should probably be unique on the same cluster for sanity reasons, but no check is enforced

create_mappings(model_classes)

Creates index mapping in elasticsearch for each model passed in. Doesn’t update existing mappings.

Parameters:model_classes – a list of classes for which indices are created Returns:returns the names of indices which were actually created

delete_index(index, timeout=2.0)

Deletes an index from elasticsearch and blocks until it is deleted.

Unlike the create_mappings and other operations, index deletes in elastic_connect don’t perform any index_name prefix magic. All index deletions in elastic_connect are attempted with the name provided ‘as is’.

Parameters:

• index – name of index to be deleted
• timeout – if the index is not deleted after the number of seconds, Exception is raised. If timeout = 0 doesn’t block and returns immediately

Returns:

none

delete_indices(indices)

Deletes multiple indices, blocks until they are deleted.

Unlike the create_mappings and other operations, index deletes in elastic_connect don’t perform any index_name prefix magic. All index deletions in elastic_connect are attempted with the name provided ‘as is’.

Parameters:indices – names of indices to be deleted Returns:None

index_prefix

@property

Returns the calculated index prefix, taking into account any global prefixes as well.

register_model_class(model_class)

Registers a model class in this namespace. By default all model classes are registered in the _default namespace. By registering a model in a namespace it is possible to reuse it to connect to a different Elasticsearch cluster.

Parameters:model_class – The model class to be registered Returns:Returns a new model class with name prefixed with Namespace.name and properly set _es_namespace reference.

wait_for_http_connection(initial_wait=10.0, step=0.1, timeout=30.0, https=False)

Waits for http(s) connection to elasticsearch to be ready

Parameters:

• initial_wait – initially wait in seconds
• step – try each step seconds after initial wait
• timeout – raise NamespaceConnectionError after timeout seconds of trying. This includes the inital wait.
• https – whether to use http or https protocol

Returns:

True

Raises:

NamespaceConnectionError on connection timeout

wait_for_ready(initial_attempt=True, initial_wait=2.0, step=0.1, timeout=30.0, https=False)

Waits for elasticsearch to get ready. First waits for the node to responde over http, then waits for the cluster to turn at least yellow.

Parameters:

• initial_attempt – If True, attempts a http connection right away, even before starting the initial_wait
• initial_wait – initially wait in seconds
• step – try each step seconds after initial wait
• timeout – raise NamespaceConnectionError after timeout seconds of trying. This includes the inital wait.
• https – whether to use http or https protocol

Returns:

returns cluster health info

exception elastic_connect.namespace.NamespaceConnectionError

exception elastic_connect.namespace.NamespaceAlreadyExistsError