Df Types

class dffml.df.types.Definition(name: str, primitive: str, default: ~typing.Any = <dffml.df.types._NO_DEFAULT object>, lock: bool = False, spec: ~typing.Optional[~typing.NamedTuple] = None, subspec: bool = False, validate: ~typing.Optional[~typing.Callable[[~typing.Any], ~typing.Any]] = None)

Examples

>>> from dffml import Definition, Input
>>> from typing import NamedTuple
>>>
>>> class Person(NamedTuple):
...   name: str
...   age: int
>>>
>>> Friend = Definition(name="Friend", primitive="map", spec=Person)
>>> Input(value={"name": "Bob", "age": 42}, definition=Friend)
Input(value=Person(name='Bob', age=42), definition=Friend)
>>>
>>> SignUpQueue = Definition(name="SignUpQueue", primitive="array", spec=Person, subspec=True)
>>> Input(value=[{"name": "Bob", "age": 42}], definition=SignUpQueue)
Input(value=[Person(name='Bob', age=42)], definition=SignUpQueue)
>>>
>>> AddressBook = Definition(name="AddressBook", primitive="map", spec=Person, subspec=True)
>>> Input(value={"bob": {"name": "Bob", "age": 42}}, definition=AddressBook)
Input(value={'bob': Person(name='Bob', age=42)}, definition=AddressBook)

property default

Alias for field number 2

property lock

Alias for field number 3

property name

Alias for field number 0

property primitive

Alias for field number 1

property spec

Alias for field number 4

property subspec

Alias for field number 5

property validate

Alias for field number 6

exception dffml.df.types.DefinitionMissing

Definition missing from linked DataFlow

exception dffml.df.types.FailedToLoadOperation

Raised when an Operation wasn’t found to be registered with the dffml.operation entrypoint.

class dffml.df.types.Forward(book: Dict[str, List[Definitions]] = None)

Keeps a map of operation instance_names to list of definitions of inputs which should be forwarded to the subflow running in that operation.

get_instances_to_forward(definition: Definition) → List[str]

Returns a list of all instances of operation to which definition should be forwarded to.

class dffml.df.types.Input(value: Any, definition: Definition, parents: Optional[List[Input]] = None, origin: Optional[Union[str, Tuple[Operation, str]]] = 'seed', validated: bool = True, *, uid: Optional[str] = '')

All inputs have a unique id. Without it they can’t be tracked for locking purposes.

class dffml.df.types.InputFlow(inputs: Optional[Dict[str, List[Dict[str, Operation]]]] = None, conditions: Optional[List[Dict[str, Operation]]] = None)

Inputs of an operation by their name as used by the operation implementation mapped to a list of locations they can come from. The list contains strings in the format of operation_instance_name.key_in_output_mapping or the literal “seed” which specifies that the value could be seeded to the network.

static get_alternate_definitions(origin: Tuple[Union[List[str], Tuple[str]], str]) → Tuple[Union[List[str], Tuple[str]], str]

Returns the alternate definitions and origin for an entry within an input flow. If there are no alternate defintions then the first element of the returned tuple is an empty list.

Examples

>>> from dffml import InputFlow
>>>
>>> input_flow = InputFlow(
...     inputs={
...         "features": [
...             {"seed": ["Years", "Expertise", "Trust", "Salary"]}
...         ],
...         "token": [
...             "client",
...         ]
...     }
... )
>>>
>>> input_flow.get_alternate_definitions(list(input_flow.inputs["features"][0].items())[0])
(['Years', 'Expertise', 'Trust', 'Salary'], 'seed')
>>>
input_flow.get_alternate_definitions(list(input_flow.inputs["other"][0].items())[0])
([], 'client')

class dffml.df.types.Operation(name: str, inputs: Dict[str, dffml.df.types.Definition] = <factory>, outputs: Dict[str, dffml.df.types.Definition] = <factory>, stage: dffml.df.types.Stage = <Stage.PROCESSING: 'processing'>, conditions: Union[List[dffml.df.types.Definition], NoneType] = <factory>, expand: Union[List[str], NoneType] = <factory>, instance_name: Union[str, NoneType] = None, validator: bool = False, retry: int = 0)

classmethod definitions(*args: Operation)

Create key value mapping of definition names to definitions for all given operations.

classmethod load(loading=None)

Loads all installed loading and returns them as a list. Sources to be loaded should be registered to ENTRYPOINT via setuptools.

class dffml.df.types.Output(name, select, fill, single, ismap)

property fill

Alias for field number 2

property ismap

Alias for field number 4

property name

Alias for field number 0

property select

Alias for field number 1

property single

Alias for field number 3

class dffml.df.types.Parameter(key: str, value: Any, origin: Input, definition: Definition)

exception dffml.df.types.PrimitiveDoesNotMatchValue

Primitive does not match the value type

class dffml.df.types.Stage(value)

An enumeration.