dpmcore.services.ast_generator

Generate engine-ready validation scripts from DPM-XL expressions. The resulting enriched_ast is keyed by the resolved module URI and carries the AST, resolved release, per-validation severity, operands, tables, preconditions, and dependency information.

The same script generation is exposed through the generate-script CLI command and the /api/v1/scripts REST endpoint.

The service also exports a module’s calculations set – calculations_for_module() and its companion calculations_datapoints() – which is the operations linked to the module version through OperationOutput, rather than the validations scoped to it. That export is exposed through the export-calculations CLI command.

ASTGeneratorService

class dpmcore.services.ast_generator.ASTGeneratorService(session=None)

Bases: object

Generate engine-ready validation scripts from DPM-XL expressions.

Parameters:

session (Optional[Session]) – An open SQLAlchemy session (required for script).

__init__(session=None)

Build the service, optionally bound to a SQLAlchemy session.

Parameters:

session (Session | None)

Return type:

None

script(expressions, module_code, module_version, preconditions=None, severity=None, severities=None, release=None)

Generate an engine-ready validations script.

Parameters:

• expressions (List[Tuple[str, str]]) – [(expression, validation_code), ...].

• module_code (str) – Code of the primary module (e.g. "COREP_Con").

• module_version (str) – Version of the primary module (e.g. "2.0.1").

• preconditions (Optional[List[Union[Tuple[str, List[str]], Dict[str, Any]]]]) – Optional list of precondition specs. Each entry is either a tuple (precondition_expression, [validation_codes]) or a dict with keys expression and affected_operations (optional code and version_id are also accepted). A precondition can guard many validation codes; a validation may have no precondition. Codes that end up in failed_operations are stripped from the emitted affected_operations, and a precondition left gating nothing is dropped entirely.

• severity (Optional[str]) – Optional global default severity tag ("error", "warning", "info"). Defaults to "warning".

• severities (Optional[Dict[str, str]]) – Optional per-validation override map {validation_code: severity}. Resolution per validation is severities.get(code, severity_global). Unknown codes (not present in expressions) raise ValueError.

• release (Optional[str]) – Optional release code. When omitted, resolved to the latest release whose window contains the requested ModuleVersion. The resolved release is surfaced in the dpm_release block and used for every downstream DB filter.

Return type:

Dict[str, Any]

Returns:

A dict with keys success (bool), enriched_ast (the namespaced dict, or None on failure), error (str or None), and failed_operations (a {validation_code: error_message} map of expressions skipped due to semantic errors, or because their precondition cannot be evaluated by the engine). The namespaced dict mirrors the shape pydpm’s generate_validations_script produces.

script_from_db(expressions, operation_vids, mv, release_row, preconditions=None, severity=None, severities=None, from_submission_dates=None)

Generate a DB-discovered module version’s validations script.

For each operation, its persisted OperationNode tree is tried first (db_ast.build_ast_from_db()) — falling back to re-parsing its expression text only when that isn’t possible (an unsupported construct, or the reshaper for it not being implemented yet — see dpmcore#364). Shares _process_operation()/_assemble_script() with script(); the only thing that differs between the two is how each operation’s ast_dict/parameters/ts/ root_operator_id are obtained.

Parameters:

• expressions (List[Tuple[str, str]]) – same as script().

• operation_vids (Dict[str, int]) – {code: OperationVID}, from _discover_module_validations() — which operations have a persisted AST to try first.

• mv (Any) – already resolved by the caller (script_for_module()), so this doesn’t re-resolve it.

• release_row (Any) – already resolved by the caller (script_for_module()), so this doesn’t re-resolve it.

• preconditions (Optional[List[Union[Tuple[str, List[str]], Dict[str, Any]]]]) – same as script().

• severity (Optional[str]) – same as script().

• severities (Optional[Dict[str, str]]) – same as script().

• from_submission_dates (Optional[Dict[str, str]]) – {code: FromSubmissionDate}, from _discover_module_validations() — the real per-operation submission date. A code missing here (the column was null) falls back to _build_operation_entry()’s own default, same as script()’s caller-supplied expressions.

Return type:

Dict[str, Any]

Returns:

Same shape as script(). Unlike script(), a per-operation scope-calculation error does not fail the whole result: it is recorded in failed_operations and that operation is left out, the same as a semantic error or an ungateable precondition. script() still aborts on a scope error (#122) — this method discovers every active validation for the module itself, so one bad operation failing the whole discovered set would be a worse surprise here than for a caller-supplied expressions list.

script_for_module(module_code, module_version, release=None)

Auto-discover a module version’s active validations and script them.

No expressions, preconditions or severities need to be supplied — they are looked up from OperationScope / OperationScopeComposition / OperationVersion for this module version, then handed to script_from_db().

Parameters:

• module_code (str) – Code of the module (e.g. "COREP_Con").

• module_version (str) – Version of the module (e.g. "2.0.1").

• release (Optional[str]) – Optional release code; resolved the same way as in script().

Return type:

Dict[str, Any]

Returns:

Same shape as script().

calculations_for_module(module_code, reference_date, publication_date=None)

Export a module version’s calculations set.

Where script_for_module() scripts a module’s validations, this exports its calculations: the operations linked to the module version through OperationOutput, parsed as one script, reordered so a calculation follows the ones it consumes, and serialized with every operand resolved to its datapoints.

The companion datapoint map is calculations_datapoints(); the two are computed by the same pass, so a caller that wants both should use calculations_export() and read the two fields off it.

Parameters:

• module_code (str) – Code of the module (e.g. "KRI").

• reference_date (str) – Reference date, YYYY-MM-DD. The module version whose reference-date window contains it is the one exported; exactly one must.

• publication_date (Optional[str]) – Publication date stamped into the dpm_release block; defaults to today.

Return type:

Dict[str, Any]

Returns:

The export, keyed by the module’s EBA taxonomy URI.

Raises:

• ValueError – If the service has no database session.

• DpmCoreError – If the module version, its calculations or the OperationOutput table cannot be resolved.

calculations_datapoints(module_code, reference_date, publication_date=None)

Map every datapoint a module’s calculations touch to its cell.

The companion of calculations_for_module(), with the same arguments and the same failure modes.

Parameters:

• module_code (str) – Code of the module (e.g. "KRI").

• reference_date (str) – Reference date, YYYY-MM-DD.

• publication_date (Optional[str]) – Publication date stamped into the dpm_release block; defaults to today.

Return type:

Dict[str, Dict[str, Any]]

Returns:

{variable_id: {"table", "row", "column", "sheet"}}.

Raises:

• ValueError – If the service has no database session.

• DpmCoreError – If the module version, its calculations or the OperationOutput table cannot be resolved.

calculations_export(module_code, reference_date, publication_date=None)

Export a module’s calculations and datapoint map in one pass.

Parameters:

• module_code (str) – Code of the module (e.g. "KRI").

• reference_date (str) – Reference date, YYYY-MM-DD.

• publication_date (Optional[str]) – Publication date stamped into the dpm_release block; defaults to today.

Return type:

CalculationsExport

Returns:

Both halves of the export.

Raises:

• ValueError – If the service has no database session.

• DpmCoreError – If the module version, its calculations or the OperationOutput table cannot be resolved.

list_module_versions(module_code=None, release=None)

Enumerate (module_code, version_number) pairs to sweep.

Used by the CLI’s --all-modules/--all-versions sweep to discover targets for script_for_module().

Parameters:

• module_code (Optional[str]) – Restrict to this module’s versions. None enumerates every module in the database.

• release (Optional[str]) – When given, only ModuleVersion rows whose window contains this release (via filter_by_release()) are returned.

Return type:

List[Tuple[str, str]]

Returns:

(module_code, version_number) pairs, ordered by (code, start_release_id), excluding ghost/phantom ModuleVersion rows (from_reference_date == to_reference_date, both non-null — the same test _walk_ghost_chain() uses). With release, a module whose only covering versions are ghosts is represented by its ghost fallback instead of being dropped (#372, see _ghost_fallback_versions()).

Raises:

ValueError – If a database session is missing, or release doesn’t match any Release.code.

Calculations export

class dpmcore.services.calculations_export.CalculationsExport(calculations, datapoints)

A module version’s exported calculations and its datapoint map.

Parameters:

• calculations (Dict[str, Any])

• datapoints (Dict[str, Dict[str, Any]])

calculations

The export, keyed by the module’s EBA taxonomy URI.

datapoints

{variable_id: {table, row, column, sheet}} for every datapoint the calculations touch – the companion file the deployment reads alongside the export.

class dpmcore.services.calculations_export.CalculationsExporter(session)

Exports a module version’s calculations set.

Parameters:

session (Session) – An open SQLAlchemy session.

__init__(session)

Bind the exporter to session.

Parameters:

session (Session)

Return type:

None

export(module_code, reference_date, publication_date=None)

Export module_code’s calculations as of reference_date.

Parameters:

• module_code (str) – Module code (e.g. "KRI").

• reference_date (str) – Reference date, YYYY-MM-DD, selecting the module version.

• publication_date (Optional[str]) – Publication date stamped into the dpm_release block; defaults to today.

Return type:

CalculationsExport

Returns:

The export and its datapoint map.

Raises:

• ConfigurationError – If the database has no OperationOutput table.

• NotFound – If the module version, or its calculations, cannot be resolved.

• Invalid – If the parsed script does not have one statement per calculation.

• SemanticError – 6-4 if the calculations form a cycle, or 6-1 if two of them assign the same output.