Skip to content

Python: [Bug]: WorkflowState fallback evaluator ignores And/Or precedence and evaluates chained - and / right-to-left #10162

Description

Observed Behavior

When PowerFx is unavailable, WorkflowState.eval() falls back to the Python implementation in WorkflowState._eval_simple().

The fallback evaluator produces incorrect results for expressions involving logical operator precedence, arithmetic associativity, and operators appearing inside string literals.

The following examples demonstrate the incorrect behavior:

1. Incorrect logical operator precedence

s.eval("=Local.a Or Local.b And Local.c")

Actual result: False
Expected result: True

And should have higher precedence than Or, so the expression should be evaluated as Local.a Or (Local.b And Local.c).

2. Incorrect subtraction associativity

s.eval("=10 - 2 - 3")

Actual result: 11.0
Expected result: 5.0

Subtraction should be evaluated left-to-right: (10 - 2) - 3.

3. Incorrect division associativity

s.eval("=100 / 10 / 2")

Actual result: 20.0
Expected result: 5.0

Division should also be evaluated left-to-right: (100 / 10) / 2.

4. Operators inside string literals

s.eval('=Local.text = "a And b"')

Actual result: False
Expected result: True, when Local.text contains "a And b".

The underlying problem is that _eval_simple() uses string splitting to identify operators without accounting for operator precedence, associativity, parentheses, or string-literal boundaries.

As a result, expressions evaluated through the Python fallback can produce results that differ from their expected PowerFx semantics.

Expected Behavior

WorkflowState.eval() should produce consistent and predictable results when evaluating expressions through the Python fallback.

Specifically:

  1. Logical operator precedence: And should bind more tightly than Or, ensuring that logical expressions are evaluated in the correct order.

  2. Arithmetic associativity: Operators with the same precedence should be evaluated left-to-right. For example:

    • 10 - 2 - 3 should evaluate to 5.
    • 100 / 10 / 2 should evaluate to 5.
  3. String literal handling: Operators appearing inside quoted string literals should not be interpreted as expression operators.

  4. Parenthesized expressions: Explicit grouping should be respected, allowing parentheses to override the default operator precedence.

  5. Backward compatibility: Existing variable resolution, custom-function dispatch, comparison behavior, and other supported expression functionality should continue working as expected.

The Python fallback should produce results consistent with the corresponding PowerFx expressions for the supported expression subset.

Steps to Reproduce

  1. Create a WorkflowState instance.

  2. Configure the state with the following variables:

    • Local.a = True
    • Local.b = False
    • Local.c = False
    • Local.text = "a And b"
  3. Ensure that PowerFx is unavailable so that the Python fallback evaluator is used.

  4. Evaluate the following expressions:

    s.eval("=Local.a Or Local.b And Local.c")
    s.eval("=10 - 2 - 3")
    s.eval("=100 / 10 / 2")
    s.eval('=Local.text = "a And b"')
  5. Observe that the fallback evaluator returns incorrect results for these expressions.

The fallback can also be forced deterministically in tests by setting _powerfx_engine to None.

Minimal Reproduction

from agent_framework_declarative import WorkflowState

s = WorkflowState()

s.set("Local.a", True)
s.set("Local.b", False)
s.set("Local.c", False)
s.set("Local.text", "a And b")

# Incorrect logical operator precedence
print(s.eval("=Local.a Or Local.b And Local.c"))
# Actual: False
# Expected: True

# Incorrect subtraction associativity
print(s.eval("=10 - 2 - 3"))
# Actual: 11.0
# Expected: 5.0

# Incorrect division associativity
print(s.eval("=100 / 10 / 2"))
# Actual: 20.0
# Expected: 5.0

# Operator inside a string literal
print(s.eval('=Local.text = "a And b"'))
# Actual: False
# Expected: True

Error Messages and Stack Traces

N/A

No exception or stack trace is produced. The issue concerns incorrect expression evaluation rather than a runtime failure.

Package Versions

  • Package: agent-framework-declarative
  • MAF Core Version: 1.25.0
  • Installation: Source checkout of microsoft/agent-framework (main branch).
  • PowerFx: Unavailable when reproducing the fallback behavior.

Python Version

Python 3.13.15

Operating System

Windows

Regression

Yes. The behavior is reproducible in the current implementation, but the specific version in which it was introduced has not yet been established.

Additional Context

Root Cause

The issue originates in WorkflowState._eval_simple() in:

python/packages/declarative/agent_framework_declarative/_workflows/_state.py

The current implementation uses string operations such as formula.split(" And ", 1) to identify binary operators.

This introduces several problems:

  • Incorrect logical precedence: And is processed before Or in the evaluator's splitting order, effectively giving And lower precedence than Or.
  • Incorrect associativity: Splitting at the first occurrence and recursively evaluating the right-hand side causes chained subtraction and division to be evaluated right-to-left.
  • String literal handling: Operators inside quoted strings are incorrectly treated as expression operators.
  • Parenthesized expressions: The evaluator does not consistently recognize parentheses as explicit grouping boundaries.

Scope

This issue affects the standalone WorkflowState Python fallback evaluator.

DeclarativeWorkflowState, which is used by WorkflowFactory, follows a different evaluation path and is not affected by the absence of PowerFx in the same way.

Reproduction and Testing

Regression tests can force the Python fallback by setting _powerfx_engine to None, independently of whether PowerFx is installed in the environment.

The regression tests cover:

  • Logical operator precedence (And, Or, and ||).
  • Negation precedence.
  • Parenthesized expressions.
  • Left-associative subtraction and division.
  • Mixed arithmetic operators and their precedence.
  • Operators appearing inside string literals.

These tests demonstrate the incorrect behavior in the existing implementation and verify the expected results after the fix.

Suggested Area for Investigation

WorkflowState._eval_simple() and its binary operator parsing logic.

Acknowledgements

  • I searched existing issues and did not find a duplicate.
  • I personally verified this behavior and the reproduction details are authentic.
  • I will wait for explicit maintainer agreement before starting implementation of a non-trivial change.

Activity

  1. added
    pythonUsage: [Issues, PRs], Target: Python
    triageUsage: [Issues], Target: All issues that still need to be triaged
    on Nov 7, 2026
  2. added
    declarativeUsage: [Issues, PRs], Target: declarative agents and workflows
    reproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflow
    on Nov 7, 2026
  3. github-actions commented on Nov 7, 2026

    @github-actions
    Contributor

    Reproduced on main (924a6e5) by forcing the fallback (agent_framework_declarative._workflows._state._powerfx_engine = None). Results: Local.a Or Local.b And Local.c → False, 10 - 2 - 3 → 11.0, 100 / 10 / 2 → 20.0, Local.text = "a And b" → False, matching the report. With PowerFx active, the same expressions give the expected values. The cause is in WorkflowState._eval_simple, which splits on the first And / Or / - / / occurrence.

    Support boundary: unclear. I found no maintained spec or ADR defining the Python fallback's semantics, so a maintainer needs to decide the scope. Evidence is in the Actions run; routed to the declarative package owner. Priority is left to maintainers.

    Generated by Issue Routing for #10162 · copilot · auto · ~25 AIC · ⌖ ~1 AIC · ⊞ 5.2K · ◷

  4. eavanvalkenburg commented on Nov 9, 2026

    @eavanvalkenburg
    Member

    We need to take a broader design look before implementing this. Fixing these examples starts to open a Pandora’s box of Power Fx compatibility: precedence, associativity, grouping, quoting, coercion, functions, and error behavior. The fallback should be explicitly best effort when Power Fx is unavailable, not a full replacement. We need to define the supported subset and failure or compatibility contract here before another PR.

  5. eavanvalkenburg commented on Nov 9, 2026

    @eavanvalkenburg
    Member

    /plan Follow the existing maintainer decision: define an explicitly best-effort Python fallback, not a second Power Fx implementation. Independently reproduced on current main: mixed And/Or precedence, chained subtraction/division, and operator text inside quoted strings produce incorrect results. First identify the supported subset from maintained docstrings/tests and real standalone WorkflowState callers, distinguishing DeclarativeWorkflowState and WorkflowFactory. Compare minimal bounded corrections with explicitly rejecting or delegating unsupported expressions; describe compatibility and failure behavior, including the fallback after an ordinary Power Fx evaluation error. Do not silently broaden syntax, coercion, functions, grouping, or error semantics. Propose a small maintainable contract and focused regression tests, preserve state-budget protections, and estimate production scope only after the contract is agreed. PR #10163 is already closed pending this design; do not reopen it or start implementation.

  6. github-actions commented on Nov 9, 2026

    @github-actions
    Contributor

    Summary

    The Python-only WorkflowState.eval() fallback (_eval_simple) splits formulas on the first " And ", " Or ", " - ", " / " it finds. That gives wrong answers for mixed And/Or, chained - and /, and operator text inside quotes. Per the maintainer decision, the goal is an explicitly best-effort fallback, not a second Power Fx.

    What I confirmed in the code (read only; nothing run)

    • Who uses the fallback. Only the public, standalone WorkflowState (_state.py eval / _eval_simple). It falls back when powerfx isn't importable and after any ordinary (non-budget) Power Fx exception, which is only logged as a warning. _PowerFxStateLimitError is re-raised and never reaches the fallback.
    • DeclarativeWorkflowState / WorkflowFactory are unaffected. Its eval has no _eval_simple. It raises RuntimeError if Power Fx is missing, and re-raises other errors (except "isn't recognized" → None). I found no production caller of the standalone WorkflowState.eval in the repo; the hosting package's WorkflowState is an unrelated class. Real users are therefore external code (unknown).
    • Documented subset (_eval_simple docstring and tests/test_workflow_state.py): Local./System./Workflow.Inputs. references; literals (string, int/float, true/false); And/Or/||/Not()/!; < > <= >= <> =; + - * / (None→0, + falls back to string concat, divide-by-zero→None); CUSTOM_FUNCTIONS (UserMessage, MessageText, ...) via whole_call_args. The docstring does not promise precedence, grouping, quoting or Power Fx parity. Existing tests only use single-operator expressions, so nothing pins the buggy behavior.
    • The current fallback already silently diverges in other ways (unparseable text is returned as the formula string; function errors return the formula; + and numeric coercion are lenient). That is the "Pandora's box" risk.

    Recommendation

    Code and docs change, kept small. The reported results are real defects even for a best-effort evaluator, because they return a confidently wrong value rather than failing. I'd compare three options:

    Option Pros Cons
    A. Patch the splitting (e.g. split on the last -//, And before Or, skip quotes) Tiny diff Leaves it ad hoc; parentheses and nested cases still break; invites more patches
    B. Small tokenizer and precedence parser for exactly the documented operators Correct for the documented subset; one place for quote handling More code than A; risk of creeping into a Power Fx clone
    C. Reject anything outside a tiny supported shape (raise ValueError) and document "install Power Fx for anything else" Smallest, most honest Behavior change for callers who rely on today's lenient output

    Suggested direction: B, bounded, with C-style rejection only for forms we know we can't evaluate safely (e.g. mixed forms the parser cannot classify). A is the cheapest but cannot honor quote safety cleanly.

    Proposed contract (for agreement; not implemented)

    1. Supported: the subset listed above, nothing new. Operator tokens are recognized only outside "..."/'...'.
    2. Precedence/associativity: Not/! > * / > + - > comparisons > And > Or/||; same-level operators left-to-right. Parentheses: open question below.
    3. Unsupported: functions beyond CUSTOM_FUNCTIONS/Not, other syntax. Today these return the raw text. Keep that existing return behavior for compatibility, but log a warning that the fallback is best-effort. Alternatively raise. Needs your call.
    4. Existing leniencies stay as-is (None→0, + string concat, divide by zero → None). No new coercion or functions.
    5. Failure: the budget error continues to propagate before any fallback. After an ordinary Power Fx error the fallback still runs (current behavior), but the warning should say so explicitly, and the docstring should state that results may differ from Power Fx. Note that this means a real Power Fx syntax error can be masked by a plausible fallback value.
    6. Docs: update the eval/_eval_simple docstrings (and the package AGENTS.md line on WorkflowState) to say "best-effort subset; DeclarativeWorkflowState does not use it".

    Focused regression tests (all force _powerfx_engine = None)

    • The four reported cases, plus Local.a And Local.b Or Local.c, 10 - 2 - 3 - 1, 100 / 10 / 2, 2 + 3 * 4.
    • Operator text inside both quote styles, including = and + inside strings.
    • Existing single-operator tests unchanged, as the compatibility guard.
    • Ordinary Power Fx exception → fallback still used (mocked engine); _PowerFxStateLimitError → still raised with no fallback.
    • Unsupported expression behaves per decision 3.
    • No test asserting full Power Fx parity.

    Questions before sizing the work

    1. Parentheses: the fallback currently has no grouping, and the issue lists it as expected. Should grouping be in the contract (small parser addition), or explicitly unsupported?
    2. Unsupported input: keep returning the raw formula text with a warning (compatible), or raise ValueError (fail loudly, a behavior change)?
    3. After an ordinary Power Fx error: keep falling back (compatible, may hide a real error), or only fall back when Power Fx is unavailable?

    Once these are agreed I can estimate the production scope. Nothing has been changed, and PR #10163 stays closed.

    Generated by Issue Planning for #10162 · copilot · auto · ~25 AIC · ⌖ ~1 AIC · ⊞ 4.8K · ◷

  7. PratikWayase commented on Nov 11, 2026

    @PratikWayase
    Contributor

    Jose Alvarez (@jpalvarezl) I propose replacing the current .split() parsing with a tokenizer and precedence-climbing parser to properly handle operator precedence, strings, and comments. Local validation confirms this fixes all four reported issues, adds parenthesis support, and passes 50/52 existing assertions. Before implementing, I'd appreciate your feedback on raising ValueError for unsupported syntax, including basic parentheses, and keeping &/&& unsupported for now.

  8. Namraa310806 commented on Nov 12, 2026

    @Namraa310806
    ContributorAuthor

    Thanks Eduard van Valkenburg (@eavanvalkenburg) for the investigation and plan. I reported and reproduced this, and I have a tested prototype in the now-closed #10163. I agree with the design-first approach and won't open a PR until the contract is agreed. The fallback should stay best-effort and not become a second Power Fx.

    Proposed approach

    A bounded correction of the existing evaluator rather than a general-purpose parser. Keep _eval_simple's structure and replace the str.split calls with one helper that finds the last top-level operator:

    • Operators are recognized only outside quoted strings (both quote styles, doubled-quote escapes). This is required to fix the reported "a And b" case, not an attempt to support new syntax.
    • Operators inside nested (), [], {} are skipped.
    • Using the last occurrence makes chains left-associative.

    Precedence for the existing operators, lowest to highest: Or/||, And, comparisons, + -, * /, !/Not(), then literals, references and calls.

    Proposed contract

    • Supported syntax: only the currently documented subset. No new operators, functions, comments or coercion rules.
    • Operator spacing: the existing requirement for spaces around binary operators stays, and gets documented.
    • Grouping: basic parenthesized grouping, if you agree it belongs in the initial contract. The issue lists it as expected, and the same scan supports it at little cost. Otherwise I'd document it as unsupported.
    • Unsupported or unparseable input: keep returning the original formula, plus a best-effort warning. This avoids a breaking change for existing callers. Raising ValueError would be a separate, explicit compatibility decision.
    • After an ordinary Power Fx error: keeping the fallback is compatible with today's behavior, but it can mask a genuine Power Fx error. I'd keep it, with a warning that says Power Fx failed and the result may differ. _PowerFxStateLimitError keeps propagating with no fallback.
    • Existing leniencies (None→0, + string concat, divide-by-zero→None), variable resolution and custom-function dispatch are unchanged.
    • Docs: update the eval/_eval_simple docstrings and the AGENTS.md line to say "best-effort subset; DeclarativeWorkflowState is unaffected."

    Regression tests (all force _powerfx_engine = None)

    • The four reported cases.
    • Mixed precedence and chained arithmetic (10 - 2 - 3 - 1, 2 + 3 * 4, a And b Or c).
    • Operator text inside both quote styles.
    • The agreed grouping behavior.
    • Existing single-operator tests unchanged, as the compatibility guard.
    • Fallback with a warning after an ordinary Power Fx error (mocked engine).
    • _PowerFxStateLimitError still propagates.
    • No assertions of full Power Fx parity.

    I understand the concern about gradually expanding the fallback's semantics. My aim is to correct the documented behavior while keeping the boundaries explicit.

    pratik wayase (@PratikWayase), thanks for the parser proposal. It covers the same reported cases, and I'm happy to go with whichever approach the maintainers prefer. I'd appreciate feedback on grouping, on raw-formula versus ValueError for unsupported input, and on keeping the fallback after ordinary Power Fx errors. Once the contract is settled, I'm glad to implement it and size the scope then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

declarativeUsage: [Issues, PRs], Target: declarative agents and workflowspythonUsage: [Issues, PRs], Target: PythonreproducedUsage: [Issues], Target: all issues that can be reproduced by the triage workflowtriageUsage: [Issues], Target: All issues that still need to be triaged

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions