utility

utility

DomoDataflow Utility Actions

General utility actions for data transformation in Magic ETL v2. These correspond to the "Utility" category in the Domo ETL UI sidebar.

DomoDataflow_Action_Constant dataclass

DomoDataflow_Action_Constant(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    fields: list[dict] = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Constant action - adds constant value columns.

Also known as "Add Constants" in the Magic ETL UI.

Attributes:

Name	Type	Description
fields	list[dict]	List of constant field definitions with name, type, and value

Example

constant_action = dataflow.get_action_objects("Constant")[0] for field in constant_action.fields: ... print(f"{field.get('name')}: {field.get('value')}")

column_relationships property

column_relationships: set[ColumnRelationship]

Column relationships for Constant (Add Constants) tile.

Constants produce new columns with no upstream column source. We record them as DIRECT/TRANSFORMATION with to_column=None.

constants property

constants: dict[str, Any]

Get mapping of constant name -> value.

DomoDataflow_Action_ExpressionEvaluator dataclass

DomoDataflow_Action_ExpressionEvaluator(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    expressions: list[dict] = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Expression Evaluator action - creates calculated columns using formulas.

Also known as "Add Formula" or "Beast Mode" in the Magic ETL UI.

Attributes:

Name	Type	Description
expressions	list[dict]	List of expression definitions with name, expression, and type

Example

formula_action = dataflow.get_action_objects("ExpressionEvaluator")[0] for expr in formula_action.expressions: ... print(f"{expr.get('name')}: {expr.get('expression')}")

column_relationships property

column_relationships: set[ColumnRelationship]

Column relationships for ExpressionEvaluator (Add Formula) tile.

Each expression produces a new output column. The expression string may reference multiple input columns. We capture the formula as the expression and mark it as DIRECT/TRANSFORMATION.

formulas property

formulas: dict[str, str]

Get mapping of formula name -> expression.

DomoDataflow_Action_Metadata dataclass

DomoDataflow_Action_Metadata(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    fields: list[dict] = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Metadata action - modifies column metadata (rename, type change).

Also known as "Set Column Type" or "Rename Columns" in the Magic ETL UI.

Attributes:

Name	Type	Description
fields	list[dict]	List of field metadata modifications

Example

metadata_action = dataflow.get_action_objects("Metadata")[0] for field in metadata_action.fields: ... print(f"{field.get('name')}: {field.get('type')}")

column_relationships property

column_relationships: set[ColumnRelationship]

IDENTITY relationships for Metadata (Alter) tile.

Metadata tiles rename columns and change types. Each field maps from the output column name (rename if present, else original name) to the input column name (original name).

Columns NOT listed in fields are implicitly passed through (Metadata is an "Alter" tile — it only modifies what's explicitly listed).

type_changes property

type_changes: dict[str, str]

Get mapping of column -> new type.

DomoDataflow_Action_SQL dataclass

DomoDataflow_Action_SQL(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    statements: list[str] = None,
    sql_dialect: str = None,
)

Bases: DomoDataflow_Action_Base

SQL transform action - executes SQL statements.

Attributes:

Name	Type	Description
statements	list[str]	List of SQL statements to execute
sql_dialect	str	SQL dialect (e.g., "MAGIC")

Example

sql_action = dataflow.get_action_objects("SQL")[0] sql_action.sql = "SELECT * FROM Input LIMIT 10" dataflow = await dataflow.update_action(sql_action.name, sql_action.raw)

sql property writable

sql: str | None

Get or set the first SQL statement (convenience property).

Getting returns the first statement or None. Setting updates both self.statements and self.raw.

extract_tables_and_fields

extract_tables_and_fields() -> dict[str, list[str]]

Parse referenced tables and fields from all SQL statements.

Uses sqlglot to extract table and column references. SELECT * yields no explicit field names; * is excluded.

Returns:

Type	Description
dict[str, list[str]]	{"tables": sorted list of table names, "fields": sorted list of column names (empty for SELECT *)}

Source code in src/crew_dcs/classes/DomoDataflow/action/utility.py

def extract_tables_and_fields(self) -> dict[str, list[str]]:
    """Parse referenced tables and fields from all SQL statements.

    Uses sqlglot to extract table and column references.
    SELECT * yields no explicit field names; * is excluded.

    Returns:
        {"tables": sorted list of table names,
         "fields": sorted list of column names (empty for SELECT *)}
    """
    import sqlglot
    import sqlglot.expressions as sqlexp

    tables: set[str] = set()
    fields: set[str] = set()

    for stmt in self.statements or []:
        try:
            tree = sqlglot.parse_one(stmt, dialect="spark")
        except Exception:  # noqa: BLE001
            continue
        tables.update(t.name for t in tree.find_all(sqlexp.Table) if t.name)
        fields.update(
            c.name for c in tree.find_all(sqlexp.Column) if c.name and c.name != "*"
        )

    return {"tables": sorted(tables), "fields": sorted(fields)}

DomoDataflow_Action_SelectValues dataclass

DomoDataflow_Action_SelectValues(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    fields: list[dict] = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Select Values action - selects/renames/reorders columns.

Also known as "Select Columns" in the Magic ETL UI.

Attributes:

Name	Type	Description
fields	list[dict]	List of field configurations (name, rename, include/exclude)

Example

select_action = dataflow.get_action_objects("SelectValues")[0] for field in select_action.fields: ... print(f"{field.get('name')} -> {field.get('rename', field.get('name'))}")

column_names property

column_names: list[str]

Get list of selected column names.

column_relationships property

column_relationships: set[ColumnRelationship]

IDENTITY relationships for SelectValues (Select Columns) tile.

SelectValues explicitly lists columns to include. Each field maps from the output column name (rename if present, else original name) to the input column name (original name).

Fields with remove=True are excluded from the output.

renamed_columns property

renamed_columns: dict[str, str]

Get mapping of original name -> renamed name for renamed columns.

convert_to_metadata

convert_to_metadata(
    type_overrides: dict[str, str] | None = None,
) -> None

Convert this SelectValues action into a Metadata action in-place.

Metadata (Alter) tiles automatically pass through columns that are not explicitly listed, so only fields that actually rename or change type need to be kept in the explicit list.

Join safety: When the output feeds into a Join or SplitJoin, mismatched types on join keys cause silent wrong results. Use type_overrides to explicitly pin the type of join columns so the assumption either holds or fails hard::

select_action.convert_to_metadata(
    type_overrides={"Page ID": "STRING"}
)

Mutates self.raw["type"] to "Metadata", applies any type_overrides, and filters self.raw["fields"] to only those with a rename or type key. Also updates self.action_type and self.fields on the typed object.

Parameters:

Name	Type	Description	Default
type_overrides	dict[str, str] | None	Mapping of field name → explicit Domo type string (e.g. "STRING", "INT", "DOUBLE"). Applied before the filter so overridden fields are always kept even if they have no rename.	None

Raises:

Type	Description
RuntimeError	If self.raw is None.

Example::

select_action.convert_to_metadata()
select_action.convert_to_metadata(
    type_overrides={"Page ID": "STRING", "User ID": "INT"}
)

Source code in src/crew_dcs/classes/DomoDataflow/action/utility.py

def convert_to_metadata(
    self,
    type_overrides: dict[str, str] | None = None,
) -> None:
    """Convert this SelectValues action into a Metadata action in-place.

    Metadata (Alter) tiles automatically pass through columns that are
    not explicitly listed, so only fields that actually rename or change
    type need to be kept in the explicit list.

    **Join safety**: When the output feeds into a Join or SplitJoin,
    mismatched types on join keys cause silent wrong results.  Use
    *type_overrides* to explicitly pin the type of join columns so the
    assumption either holds or fails hard::

        select_action.convert_to_metadata(
            type_overrides={"Page ID": "STRING"}
        )

    Mutates ``self.raw["type"]`` to ``"Metadata"``, applies any
    *type_overrides*, and filters ``self.raw["fields"]`` to only those
    with a ``rename`` or ``type`` key.  Also updates ``self.action_type``
    and ``self.fields`` on the typed object.

    Args:
        type_overrides: Mapping of field ``name`` → explicit Domo type
            string (e.g. ``"STRING"``, ``"INT"``, ``"DOUBLE"``).
            Applied **before** the filter so overridden fields are always
            kept even if they have no rename.

    Raises:
        RuntimeError: If ``self.raw`` is None.

    Example::

        select_action.convert_to_metadata()
        select_action.convert_to_metadata(
            type_overrides={"Page ID": "STRING", "User ID": "INT"}
        )
    """
    if self.raw is None:
        raise RuntimeError(
            "Cannot convert: action has no raw dict (was it created from_dict?)"
        )

    # Apply explicit type overrides before filtering
    if type_overrides:
        for f in self.raw.get("fields") or []:
            if f.get("name") in type_overrides:
                f["type"] = type_overrides[f["name"]]

    self.raw["type"] = "Metadata"
    self.raw["fields"] = [
        f
        for f in (self.raw.get("fields") or [])
        if f.get("rename") or f.get("type")
    ]
    self.action_type = "Metadata"
    self.fields = self.raw["fields"]

DomoDataflow_Action_SetValueField dataclass

DomoDataflow_Action_SetValueField(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    fields: list[dict] = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Set Value Field action - sets or updates field values.

Attributes:

Name	Type	Description
fields	list[dict]	List of field value settings

Example

setvalue_action = dataflow.get_action_objects("SetValueField")[0] for field in setvalue_action.fields: ... print(f"{field.get('name')}: {field.get('value')}")

DomoDataflow_Action_ValueMapper dataclass

DomoDataflow_Action_ValueMapper(
    id: str,
    action_type: str = None,
    tile_type: str | None = None,
    name: str = None,
    depends_on: list[str] = None,
    disabled: bool = False,
    gui: dict = None,
    settings: dict = None,
    raw: dict = None,
    parent_actions: list[DomoDataflow_Action_Base] = None,
    field_to_use: str = None,
    target_field: str = None,
    target_type: str = None,
    mappings: list[dict] = None,
    unmapped_behavior: str = None,
    input: str = None,
    tables: list[dict] = None,
)

Bases: DomoDataflow_Action_Base

Value Mapper action - maps values from one set to another.

Also known as "Map Values" in the Magic ETL UI.

Attributes:

Name	Type	Description
field_to_use	str	Source column to map from
target_field	str	Target column name
mappings	list[dict]	List of value mappings
unmapped_behavior	str	What to do with unmapped values

Example

mapper_action = dataflow.get_action_objects("ValueMapper")[0] print(f"Mapping {mapper_action.field_to_use} -> {mapper_action.target_field}")

column_relationships property

column_relationships: set[ColumnRelationship]

Column relationships for ValueMapper (Map Values) tile.

Maps values from source column to target column.

value_map property

value_map: dict[Any, Any]

Get mapping of source value -> target value.