diff --git a/README.md b/README.md index ad88b08..51d1e6b 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ the selected simulator by a foreign language interface the package builds itself called through VHPIDIRECT (NVC, GHDL) or the FLI (Questa/ModelSim), or a VHPI application built with the simulator's own compiler driver (Riviera-PRO/Active-HDL). Values cross the interface with their VHDL types: `integer`, `real`, `string`, `boolean`, `std_ulogic`, `unsigned`/`signed`, the vector -types, and `integer_array_t` as a NumPy array. +types, `integer_array_t` as a NumPy array, and VUnit's `dict_t` as a Python `dict`. ## Installation @@ -78,7 +78,7 @@ check(call_integer_vector("gain", arg(integer_vector'(1, 2))) = integer_vector'( ## Requirements -- VUnit 5.0.0.dev12 or later (`vunit_hdl` on PyPI), which pip installs with the package. +- VUnit 5.0.0.dev15 or later (`vunit_hdl` on PyPI), which pip installs with the package. - VHDL-2008 or later. - CPython 3.10 or later, standard (GIL) build, with a shared `libpython` (`--enable-shared`), which is what distribution Pythons, `actions/setup-python`, `uv`, `pyenv` and conda-forge provide by default. diff --git a/docs/other_simulators.rst b/docs/other_simulators.rst index e8ac486..ffefe7a 100644 --- a/docs/other_simulators.rst +++ b/docs/other_simulators.rst @@ -16,8 +16,8 @@ simulator installation change, so a run script needs nothing beyond This application differs from the Python bridge in a few ways: only the default session exists, the operations implemented by the bridge -(``integer_array_t`` values, the ``boolean``, ``std_ulogic``, vector and -``integer_array_t`` results, ``exec_file``) report that they require NVC, GHDL +(``integer_array_t`` values, the ``boolean``, ``std_ulogic``, vector, +``integer_array_t`` and ``dict_t`` results, ``exec_file``) report that they require NVC, GHDL or Questa, a Python error stops the simulation with the message printed by the application rather than through the logger of the session, and ``real`` values outside the single precision float range are rejected. diff --git a/docs/running_code.rst b/docs/running_code.rst index ed3b1de..69ae7b5 100644 --- a/docs/running_code.rst +++ b/docs/running_code.rst @@ -105,6 +105,7 @@ bridge, ``eval`` has more result types: * ``eval_std_ulogic_vector``, returning an unconstrained ``std_ulogic_vector``. * ``eval_integer_array``, returning ``integer_array_t``, see :ref:`python_bridge:integer_array`. +* ``eval_dict``, returning ``dict_t``, see :ref:`python_bridge:dict`. ``eval_boolean`` and ``eval_std_ulogic`` are aliased ``eval`` like the other result types. ``eval_std_ulogic_vector`` and ``eval_integer_array`` are not, @@ -164,6 +165,7 @@ VHDL Python ``unsigned`` ``int``, passed as ``arg_unsigned``/``kwarg_unsigned`` ``signed`` ``int``, passed as ``arg_signed``/``kwarg_signed`` ``integer_array_t`` NumPy array, not on Riviera-PRO/Active-HDL +``dict_t`` ``dict`` with ``str`` keys, see :ref:`python_bridge:dict` ========================== ======================================================== An ``unsigned`` or ``signed`` value of any width becomes an exact Python @@ -201,7 +203,7 @@ passed as a number or as the string of its characters: On NVC, GHDL and Questa, ``call`` returns the same types as ``eval``: ``call_boolean``, ``call_std_ulogic``, ``call_std_ulogic_vector``, -``call_integer_array``, ``call_string``, ``call_real_vector`` and +``call_integer_array``, ``call_dict``, ``call_string``, ``call_real_vector`` and ``call_integer_vector_ptr``, plus the procedures ``call_std_ulogic_vector``, ``call_signed`` and ``call_unsigned`` taking the result as an ``out`` parameter. All of the functions but ``call_std_ulogic_vector`` are aliased diff --git a/docs/types.rst b/docs/types.rst index 1bd7d0e..eed5196 100644 --- a/docs/types.rst +++ b/docs/types.rst @@ -16,6 +16,8 @@ Type mapping * ``signed``/``unsigned`` ↔ ``int`` (procedure results only, Python bridge). * ``integer_array_t`` ↔ ``numpy.ndarray`` (Python bridge), see :ref:`python_bridge:integer_array`. +* ``dict_t`` ↔ ``dict``, see + :ref:`python_bridge:dict`. The types marked as needing the Python bridge are available on NVC, GHDL and Questa, but not on Riviera-PRO/Active-HDL. @@ -24,6 +26,47 @@ Results are strict: a value that does not fit the VHDL type, or is of the wrong Python type, is an error. Values are never silently truncated or wrapped. +.. _python_bridge:dict: + +dict_t and dict +--------------- + +A ``dict_t`` from VUnit's ``dict_pkg`` (string keys, values of mixed types) is +a Python ``dict``. As an argument, ``arg(dict)`` and ``kwarg("name", dict)`` +write the ``dict`` as the Python literal ``{"key": value, ...}``. The values +are ``integer``, ``real``, ``string``, ``boolean``, ``std_ulogic``, +``integer_vector``, ``real_vector``, +``integer_vector_ptr_t``, ``integer_array_t`` and nested ``dict_t``. Any +other value type is reported like any other argument that cannot be +converted. An ``integer_array_t`` value becomes a NumPy array as described in +:ref:`python_bridge:integer_array`. The keys are +visited in the order of ``get_key``, which VUnit does not specify. + +.. code-block:: vhdl + + constant cfg : dict_t := new_dict; + set_integer(cfg, "taps", 8); + set_real(cfg, "gain", 0.5); + call("model.configure", arg(cfg)); + +``eval_dict`` and ``call_dict`` (aliased ``eval`` and ``call``) convert a +Python ``dict`` with ``str`` keys to a new ``dict_t`` that the caller owns and +deallocates. The values are converted strictly: ``int`` to ``integer``, +``float`` to ``real``, ``str`` to ``string``, ``bool`` to ``boolean``, a +``list`` of ``int`` to ``integer_vector_ptr_t``, a NumPy array of integers to +``integer_array_t`` and a ``dict`` to a nested ``dict_t``, the last three +stored with ``set_integer_vector_ptr_t_ref``, ``set_integer_array_t_ref`` and +``set_dict_t_ref``. The bit width and signedness of an ``integer_array_t`` +follow the dtype of the array like for ``eval_integer_array``, see +:ref:`python_bridge:integer_array`. ``None``, an ``int`` outside the VHDL integer range, a key +that is not a ``str`` or any other value is an error, and ``eval_dict`` then +returns an empty ``dict_t`` after reporting it. + +.. code-block:: vhdl + + constant result : dict_t := eval("{'name': 'fir', 'taps': [1, 2, 1], 'gain': 0.5}"); + check_equal(get_string(result, "name"), "fir"); + .. _python_bridge:integer_array: integer_array_t and NumPy diff --git a/pyproject.toml b/pyproject.toml index 1b1d581..ceb9dd8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -24,7 +24,7 @@ classifiers = [ dependencies = [ "numpy", # The first VUnit with packages and simulator hooks, VUnit/vunit#1221 - "vunit_hdl>=5.0.0.dev12", + "vunit_hdl>=5.0.0.dev15", ] [tool.setuptools] diff --git a/release_notes/news.d/11.feature.md b/release_notes/news.d/11.feature.md new file mode 100644 index 0000000..2402771 --- /dev/null +++ b/release_notes/news.d/11.feature.md @@ -0,0 +1 @@ +VUnit's `dict_t` is supported in both directions: `eval_dict` and `call_dict`, aliased `eval` and `call`, return a Python dict as a new `dict_t`, and `arg(dict_t)` and `kwarg(kw, dict_t)` pass a `dict_t` to Python as a dict. Values are integers, reals, strings, booleans, integer vectors, `integer_array_t` (NumPy arrays) and nested dicts. It needs `get_key` and `get_value_type` of `dict_pkg`, so VUnit 5.0.0.dev15 or later is required. diff --git a/src/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd b/src/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd index ab9260d..83d2ec5 100644 --- a/src/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd +++ b/src/vunit_python_bridge/hdl/src/python_ffi_pkg_bridge.vhd @@ -123,6 +123,7 @@ package python_ffi_pkg is constant p_kind_integer_array : integer := 8; constant p_kind_integer_vector : integer := 9; constant p_kind_real_vector : integer := 10; + constant p_kind_dict : integer := 11; -- Names of the operations, used in the error messages impure function p_exec_operation(session : python_session_t := default_session) return string; diff --git a/src/vunit_python_bridge/hdl/src/python_pkg.vhd b/src/vunit_python_bridge/hdl/src/python_pkg.vhd index 449e4b1..5707c8d 100644 --- a/src/vunit_python_bridge/hdl/src/python_pkg.vhd +++ b/src/vunit_python_bridge/hdl/src/python_pkg.vhd @@ -14,6 +14,9 @@ use vunit_lib.run_pkg.all; use vunit_lib.runner_pkg.all; use vunit_lib.integer_vector_ptr_pkg.all; use vunit_lib.string_ops.all; +use vunit_lib.dict_pkg.all; +use vunit_lib.dict_2008p_pkg.all; +use vunit_lib.data_types_private_pkg.all; use std.textio.all; @@ -82,6 +85,12 @@ package python_pkg is -- -- H and L are read as 1 and 0. Any other metavalue is an error. -- + -- A dict_t value becomes a Python dict, {"key": value, ...}, whose values + -- are written the same way: integer, real, string, boolean, std_ulogic, + -- integer_vector, real_vector, integer_vector_ptr_t, integer_array_t and + -- dict_t values. Any other value type is an error. An integer_array_t value + -- is transferred as described below. + -- -- An integer_array_t value is transferred to Python by the Python bridge, -- which is only available for NVC, GHDL and Questa, and is referred to by -- the expression, which means that it can be used in several calls. @@ -97,6 +106,8 @@ package python_pkg is impure function kwarg_signed(kw : string; value : signed) return arg_t; impure function arg(value : integer_array_t) return arg_t; impure function kwarg(kw : string; value : integer_array_t) return arg_t; + impure function arg(value : dict_t) return arg_t; + impure function kwarg(kw : string; value : dict_t) return arg_t; -- The Python expression calling identifier with the given arguments, for -- example to embed a call in a larger exec or eval string: @@ -168,6 +179,11 @@ package python_pkg is ----------------------------------------------------------------------------- -- Results of eval: boolean, std_ulogic, vectors and arrays ----------------------------------------------------------------------------- + -- The result of eval_dict and call_dict is a new dict_t that the caller owns + -- and deallocates. Its values are integer, real, string, boolean, + -- integer_vector_ptr_t (a list of int), integer_array_t (a NumPy array, with + -- the word size of an eval_integer_array result) and dict_t (a nested dict). + -- -- std_ulogic_vector and integer_array_t results are only available under -- their explicit names, not as eval overloads, which keeps -- check_equal(eval("17"), 17) and length(eval("[1, 2]")) unambiguous. @@ -191,6 +207,11 @@ package python_pkg is expr : string; session : python_session_t := default_session ) return integer_array_t; + impure function eval_dict( + expr : string; session : python_session_t := default_session + ) return dict_t; + alias eval is eval_dict[string, python_session_t return dict_t]; + procedure eval_std_ulogic_vector( expr : string; result : out std_ulogic_vector; session : python_session_t := default_session ); @@ -251,6 +272,13 @@ package python_pkg is alias call is call_integer_array[ string, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, python_session_t return integer_array_t]; + impure function call_dict( + identifier : string; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg; + session : python_session_t := default_session + ) return dict_t; + alias call is call_dict[ + string, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, arg_t, python_session_t return dict_t]; + procedure call_std_ulogic_vector( identifier : string; result : out std_ulogic_vector; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg; @@ -622,6 +650,45 @@ package body python_pkg is return "__vunit__.staged(" & integer'image(staged_id) & ")"; end; + impure function p_arg_value(value : dict_t; operation : string) return string; + + -- The Python source text of the value stored for a key of a dict + impure function p_dict_item(value : dict_t; key, operation : string) return string is + begin + case get_value_type(value, key) is + when vhdl_integer => return to_string(get_integer(value, key)); + when vhdl_real => return to_string(get_real(value, key), "%.16e"); + when vhdl_string => return p_quoted(get_string(value, key)); + when vhdl_boolean => return arg(get_boolean(value, key)).value; + when ieee_std_ulogic => return p_arg_value(get_std_ulogic(value, key), operation); + when vhdl_integer_vector => return to_py_list_str(get_integer_vector(value, key)); + when vhdl_real_vector => return to_py_list_str(get_real_vector(value, key)); + when vunit_integer_vector_ptr_t => return p_arg_value(get_integer_vector_ptr_t_ref(value, key)); + when vunit_integer_array_t => return p_arg_value(get_integer_array_t_ref(value, key), operation); + when vunit_dict_t => return p_arg_value(get_dict_t_ref(value, key), operation); + when others => + return p_failed_value( + operation & " cannot convert the " & to_string(get_value_type(value, key)) & + " value of the dict key """ & key & """" + ); + end case; + end; + + impure function p_arg_value(value : dict_t; operation : string) return string is + variable result : line; + begin + swrite(result, "{"); + for idx in 0 to num_keys(value) - 1 loop + if idx > 0 then + swrite(result, ", "); + end if; + swrite(result, p_quoted(get_key(value, idx)) & ": " & p_dict_item(value, get_key(value, idx), operation)); + end loop; + swrite(result, "}"); + + return result.all; + end; + function arg(value : real_vector) return arg_t is begin return (p_positional_arg, p_arg_value(value)); @@ -682,6 +749,16 @@ package body python_pkg is return (kw, p_arg_value(value, "kwarg")); end; + impure function arg(value : dict_t) return arg_t is + begin + return (p_positional_arg, p_arg_value(value, "arg")); + end; + + impure function kwarg(kw : string; value : dict_t) return arg_t is + begin + return (kw, p_arg_value(value, "kwarg")); + end; + ----------------------------------------------------------------------------- -- Argument groups ----------------------------------------------------------------------------- @@ -896,6 +973,51 @@ package body python_pkg is return result; end; + -- The dict_t of a dict result. Python keeps the converted entries of the dict + -- under the handle, see dict_kinds, dict_key and dict_item of runtime.py, and + -- VHDL fetches them one at a time with the eval of the type of each value. + impure function p_to_dict(handle : integer; session : python_session_t) return dict_t; + + procedure p_add_dict_item(dict : dict_t; handle, index, kind : integer; session : python_session_t) is + constant entry : string := "(" & integer'image(handle) & ", " & integer'image(index) & ")"; + constant key : string := eval_string("__vunit__.dict_key" & entry, session); + constant item : string := "__vunit__.dict_item" & entry; + variable ptr : integer_vector_ptr_t; + variable arr : integer_array_t; + variable nested : dict_t; + begin + case kind is + when p_kind_integer => set_integer(dict, key, eval_integer(item, session)); + when p_kind_real => set_real(dict, key, eval_real(item, session)); + when p_kind_boolean => set_boolean(dict, key, eval_boolean(item, session)); + when p_kind_string => set_string(dict, key, eval_string(item, session)); + when p_kind_integer_vector => + ptr := eval_integer_vector_ptr(item, session); + set_integer_vector_ptr_t_ref(dict, key, ptr); + when p_kind_integer_array => + arr := eval_integer_array(item, session); + set_integer_array_t_ref(dict, key, arr); + when others => + -- A nested dict, whose result is its handle + if p_eval(item, p_kind_dict, -1, p_eval_operation(item, session), session) then + nested := p_to_dict(p_result_integer, session); + set_dict_t_ref(dict, key, nested); + end if; + end case; + end; + + impure function p_to_dict(handle : integer; session : python_session_t) return dict_t is + constant kinds : integer_vector := eval_integer_vector( + "__vunit__.dict_kinds(" & integer'image(handle) & ")", session + ); + constant result : dict_t := new_dict; + begin + for idx in kinds'range loop + p_add_dict_item(result, handle, idx - kinds'low, kinds(idx), session); + end loop; + return result; + end; + ----------------------------------------------------------------------------- -- exec ----------------------------------------------------------------------------- @@ -1004,6 +1126,16 @@ package body python_pkg is return null_integer_array; end; + impure function eval_dict( + expr : string; session : python_session_t := default_session + ) return dict_t is + begin + if p_eval(expr, p_kind_dict, -1, p_eval_operation(expr, session), session) then + return p_to_dict(p_result_integer, session); + end if; + return new_dict; + end; + procedure eval_std_ulogic_vector( expr : string; result : out std_ulogic_vector; session : python_session_t := default_session ) is @@ -1104,6 +1236,16 @@ package body python_pkg is ); end; + impure function call_dict( + identifier : string; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg; + session : python_session_t := default_session + ) return dict_t is + begin + return eval_dict( + to_call_str(identifier, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10), session + ); + end; + procedure call_std_ulogic_vector( identifier : string; result : out std_ulogic_vector; arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9, arg10 : arg_t := null_arg; diff --git a/src/vunit_python_bridge/hdl/src/python_pkg_vhpi.vhd b/src/vunit_python_bridge/hdl/src/python_pkg_vhpi.vhd index 0512c4a..496df46 100644 --- a/src/vunit_python_bridge/hdl/src/python_pkg_vhpi.vhd +++ b/src/vunit_python_bridge/hdl/src/python_pkg_vhpi.vhd @@ -105,6 +105,7 @@ package python_ffi_pkg is constant p_kind_integer_array : integer := 8; constant p_kind_integer_vector : integer := 9; constant p_kind_real_vector : integer := 10; + constant p_kind_dict : integer := 11; -- Names of the operations, used in the error messages impure function p_exec_file_operation( diff --git a/src/vunit_python_bridge/hdl/tools/generate_python_pkg.py b/src/vunit_python_bridge/hdl/tools/generate_python_pkg.py index 8dc3252..253d7d8 100644 --- a/src/vunit_python_bridge/hdl/tools/generate_python_pkg.py +++ b/src/vunit_python_bridge/hdl/tools/generate_python_pkg.py @@ -58,6 +58,7 @@ dict(vhdl="unsigned", impure=True, fails=True, suffix="_unsigned"), dict(vhdl="signed", impure=True, fails=True, suffix="_signed"), dict(vhdl="integer_array_t", impure=True, fails=True, suffix=""), + dict(vhdl="dict_t", impure=True, fails=True, suffix=""), ] # Result types of eval and call. @@ -156,6 +157,15 @@ # the integer_array_t and integer_vector_ptr_t results. Use eval_integer_array. alias_eval=False, ), + dict( + name="dict", + vhdl="dict_t", + kind="p_kind_dict", + value="p_to_dict(p_result_integer, session)", + default="new_dict", + hand_written_eval=False, + hand_written_call=False, + ), ] # Result types whose width is given by the actual of the result parameter diff --git a/src/vunit_python_bridge/hdl/tools/python_pkg.vhd.in b/src/vunit_python_bridge/hdl/tools/python_pkg.vhd.in index 27372e6..c7a0225 100644 --- a/src/vunit_python_bridge/hdl/tools/python_pkg.vhd.in +++ b/src/vunit_python_bridge/hdl/tools/python_pkg.vhd.in @@ -14,6 +14,9 @@ use vunit_lib.run_pkg.all; use vunit_lib.runner_pkg.all; use vunit_lib.integer_vector_ptr_pkg.all; use vunit_lib.string_ops.all; +use vunit_lib.dict_pkg.all; +use vunit_lib.dict_2008p_pkg.all; +use vunit_lib.data_types_private_pkg.all; use std.textio.all; @@ -82,6 +85,12 @@ package python_pkg is -- -- H and L are read as 1 and 0. Any other metavalue is an error. -- + -- A dict_t value becomes a Python dict, {"key": value, ...}, whose values + -- are written the same way: integer, real, string, boolean, std_ulogic, + -- integer_vector, real_vector, integer_vector_ptr_t, integer_array_t and + -- dict_t values. Any other value type is an error. An integer_array_t value + -- is transferred as described below. + -- -- An integer_array_t value is transferred to Python by the Python bridge, -- which is only available for NVC, GHDL and Questa, and is referred to by -- the expression, which means that it can be used in several calls. @@ -157,6 +166,11 @@ $arg_declarations ----------------------------------------------------------------------------- -- Results of eval: boolean, std_ulogic, vectors and arrays ----------------------------------------------------------------------------- + -- The result of eval_dict and call_dict is a new dict_t that the caller owns + -- and deallocates. Its values are integer, real, string, boolean, + -- integer_vector_ptr_t (a list of int), integer_array_t (a NumPy array, with + -- the word size of an eval_integer_array result) and dict_t (a nested dict). + -- -- std_ulogic_vector and integer_array_t results are only available under -- their explicit names, not as eval overloads, which keeps -- check_equal(eval("17"), 17) and length(eval("[1, 2]")) unambiguous. @@ -524,6 +538,45 @@ package body python_pkg is return "__vunit__.staged(" & integer'image(staged_id) & ")"; end; + impure function p_arg_value(value : dict_t; operation : string) return string; + + -- The Python source text of the value stored for a key of a dict + impure function p_dict_item(value : dict_t; key, operation : string) return string is + begin + case get_value_type(value, key) is + when vhdl_integer => return to_string(get_integer(value, key)); + when vhdl_real => return to_string(get_real(value, key), "%.16e"); + when vhdl_string => return p_quoted(get_string(value, key)); + when vhdl_boolean => return arg(get_boolean(value, key)).value; + when ieee_std_ulogic => return p_arg_value(get_std_ulogic(value, key), operation); + when vhdl_integer_vector => return to_py_list_str(get_integer_vector(value, key)); + when vhdl_real_vector => return to_py_list_str(get_real_vector(value, key)); + when vunit_integer_vector_ptr_t => return p_arg_value(get_integer_vector_ptr_t_ref(value, key)); + when vunit_integer_array_t => return p_arg_value(get_integer_array_t_ref(value, key), operation); + when vunit_dict_t => return p_arg_value(get_dict_t_ref(value, key), operation); + when others => + return p_failed_value( + operation & " cannot convert the " & to_string(get_value_type(value, key)) & + " value of the dict key """ & key & """" + ); + end case; + end; + + impure function p_arg_value(value : dict_t; operation : string) return string is + variable result : line; + begin + swrite(result, "{"); + for idx in 0 to num_keys(value) - 1 loop + if idx > 0 then + swrite(result, ", "); + end if; + swrite(result, p_quoted(get_key(value, idx)) & ": " & p_dict_item(value, get_key(value, idx), operation)); + end loop; + swrite(result, "}"); + + return result.all; + end; + $arg_functions ----------------------------------------------------------------------------- @@ -740,6 +793,51 @@ $arg_functions return result; end; + -- The dict_t of a dict result. Python keeps the converted entries of the dict + -- under the handle, see dict_kinds, dict_key and dict_item of runtime.py, and + -- VHDL fetches them one at a time with the eval of the type of each value. + impure function p_to_dict(handle : integer; session : python_session_t) return dict_t; + + procedure p_add_dict_item(dict : dict_t; handle, index, kind : integer; session : python_session_t) is + constant entry : string := "(" & integer'image(handle) & ", " & integer'image(index) & ")"; + constant key : string := eval_string("__vunit__.dict_key" & entry, session); + constant item : string := "__vunit__.dict_item" & entry; + variable ptr : integer_vector_ptr_t; + variable arr : integer_array_t; + variable nested : dict_t; + begin + case kind is + when p_kind_integer => set_integer(dict, key, eval_integer(item, session)); + when p_kind_real => set_real(dict, key, eval_real(item, session)); + when p_kind_boolean => set_boolean(dict, key, eval_boolean(item, session)); + when p_kind_string => set_string(dict, key, eval_string(item, session)); + when p_kind_integer_vector => + ptr := eval_integer_vector_ptr(item, session); + set_integer_vector_ptr_t_ref(dict, key, ptr); + when p_kind_integer_array => + arr := eval_integer_array(item, session); + set_integer_array_t_ref(dict, key, arr); + when others => + -- A nested dict, whose result is its handle + if p_eval(item, p_kind_dict, -1, p_eval_operation(item, session), session) then + nested := p_to_dict(p_result_integer, session); + set_dict_t_ref(dict, key, nested); + end if; + end case; + end; + + impure function p_to_dict(handle : integer; session : python_session_t) return dict_t is + constant kinds : integer_vector := eval_integer_vector( + "__vunit__.dict_kinds(" & integer'image(handle) & ")", session + ); + constant result : dict_t := new_dict; + begin + for idx in kinds'range loop + p_add_dict_item(result, handle, idx - kinds'low, kinds(idx), session); + end loop; + return result; + end; + ----------------------------------------------------------------------------- -- exec ----------------------------------------------------------------------------- diff --git a/src/vunit_python_bridge/runtime.py b/src/vunit_python_bridge/runtime.py index 1231889..5df7743 100644 --- a/src/vunit_python_bridge/runtime.py +++ b/src/vunit_python_bridge/runtime.py @@ -38,6 +38,7 @@ KIND_INTEGER_ARRAY = 8 KIND_INTEGER_VECTOR = 9 KIND_REAL_VECTOR = 10 +KIND_DICT = 11 VHDL_TYPE_NAMES = { KIND_INTEGER: "integer", @@ -51,6 +52,7 @@ KIND_INTEGER_ARRAY: "integer_array_t", KIND_INTEGER_VECTOR: "integer_vector", KIND_REAL_VECTOR: "real_vector", + KIND_DICT: "dict_t", } STD_ULOGIC_VALUES = frozenset("UX01ZWLH-") @@ -69,6 +71,19 @@ _NO_VALUE = object() +class _Converted: # pylint: disable=too-few-public-methods + """ + A dict value converted together with its dict, returned by dict_item and + passed on as it is by convert_result. + """ + + __slots__ = ("kind", "result") + + def __init__(self, kind, result): + self.kind = kind + self.result = result + + def _type_name(value): return type(value).__qualname__ @@ -125,6 +140,24 @@ def staged(self, staged_id): """ return self._runtime.staged(staged_id) + def dict_kinds(self, handle): + """ + The result kinds of the entries of a dict result, see Runtime.dict_kinds. + """ + return self._runtime.dict_kinds(handle) + + def dict_key(self, handle, index): + """ + The key of an entry of a dict result. + """ + return self._runtime.dict_key(handle, index) + + def dict_item(self, handle, index): + """ + The converted value of an entry of a dict result. + """ + return self._runtime.dict_item(handle, index) + @staticmethod def error(message): """ @@ -166,6 +199,9 @@ def __init__(self, run_script_dir, prefix): # releases them early, so that one VHDL constant can be used in # several calls. self._staged = {} + # The entries of the last dict result and its nested dicts, indexed by + # handle: (key, kind, converted value). VHDL fetches them one at a time. + self._dicts = [] self._check_environment(prefix) @@ -415,6 +451,27 @@ def make_array( self._array_meta[id(array)] = (array, bit_width, bool(is_signed)) return array + def dict_kinds(self, handle): + """ + The result kinds of the entries of a dict result, used from VHDL as + __vunit__.dict_kinds(handle). + """ + return [kind for _, kind, _ in self._dicts[handle]] + + def dict_key(self, handle, index): + """ + The key of an entry of a dict result, __vunit__.dict_key(handle, index). + """ + return self._dicts[handle][index][0] + + def dict_item(self, handle, index): + """ + The value of an entry of a dict result, __vunit__.dict_item(handle, + index), already converted to its result kind. + """ + _, kind, result = self._dicts[handle][index] + return _Converted(kind, result) + # ------------------------------------------------------------------ # Results # ------------------------------------------------------------------ @@ -432,9 +489,15 @@ def convert_result(self, kind, width): self._array_meta = {} if value is _NO_VALUE: raise RuntimeError("Internal error: no Python result available") + if isinstance(value, _Converted): + if value.kind != kind: + raise RuntimeError(f"Internal error: a {VHDL_TYPE_NAMES[value.kind]} dict value read as {kind}") + return value.result if kind == KIND_INTEGER_ARRAY: return self._array_result(value, array_meta) + if kind == KIND_DICT: + return self._dict_result(value, array_meta) if kind not in VHDL_TYPE_NAMES: raise RuntimeError(f"Internal error: unknown result kind {kind}") converter = "bits" if kind in (KIND_SIGNED, KIND_UNSIGNED) else VHDL_TYPE_NAMES[kind] @@ -580,6 +643,67 @@ def _real_vector_result(self, kind, value, _width): data = struct.pack(f"={len(reals)}d", *reals) return (0, 0.0, data, (len(reals),)) + def _dict_result(self, value, array_meta): + """ + Convert a dict and keep its entries for VHDL to fetch, see dict_kinds. + The result is the handle of the dict. + """ + self._dicts = [] + return (self._convert_dict(value, array_meta), 0.0, b"", ()) + + def _convert_dict(self, value, array_meta): + """ + Convert the entries of a dict, and of its nested dicts, and return its handle. + """ + if not isinstance(value, dict): + raise _type_error(KIND_DICT, value, "a dict") + entries = [] + for key, item in value.items(): + if not isinstance(key, str): + raise TypeError(f"Cannot convert the dict key {key!r:.100}, a {_type_name(key)}; expected str") + entries.append((key,) + self._dict_value(key, item, array_meta)) + self._dicts.append(entries) + return len(self._dicts) - 1 + + def _dict_value(self, key, item, array_meta): + """ + The result kind and the converted value of a dict entry. + """ + if _is_bool(item): + return KIND_BOOLEAN, (int(bool(item)), 0.0, b"", ()) + if _is_integer(item): + return KIND_INTEGER, (self._dict_integer(key, item), 0.0, b"", ()) + if _is_float(item): + if not math.isfinite(item): + raise ValueError(f"{item} of dict key {key!r:.100} cannot be represented as VHDL real") + return KIND_REAL, (0, float(item), b"", ()) + if isinstance(item, str): + return KIND_STRING, self._string_result(KIND_STRING, item, -1) + if isinstance(item, dict): + return KIND_DICT, (self._convert_dict(item, array_meta), 0.0, b"", ()) + if isinstance(item, numpy.ndarray): + try: + return KIND_INTEGER_ARRAY, self._array_result(item, array_meta) + except (TypeError, ValueError, OverflowError) as error: + raise type(error)(f"dict key {key!r:.100}: {error}") from None + if isinstance(item, list) and all(_is_integer(element) for element in item): + integers = [self._dict_integer(key, element) for element in item] + return KIND_INTEGER_VECTOR, (0, 0.0, struct.pack(f"={len(integers)}i", *integers), (len(integers),)) + raise TypeError( + f"Cannot convert the value of dict key {key!r:.100}, a {_type_name(item)} ({item!r:.100}); " + "expected int, float, str, bool, a list of int, a NumPy array of integers or a dict" + ) + + @staticmethod + def _dict_integer(key, value): + value = int(value) + if not INTEGER_LOW <= value <= INTEGER_HIGH: + raise OverflowError( + f"{value} of dict key {key!r:.100} is outside the range of VHDL integer " + f"({INTEGER_LOW} to {INTEGER_HIGH})" + ) + return value + _DTYPE_WORD_SIZE = { # dtype.str without byte order: (bit_width, is_signed) "b1": (1, False), @@ -594,6 +718,14 @@ def _array_result(self, value, array_meta): """ Convert a NumPy array (or nested int sequence) to integer_array_t data. """ + storage, shape = self._array_storage(value, array_meta) + return (0, 0.0, storage.tobytes(), (storage.size,) + shape) + + def _array_storage(self, value, array_meta): + """ + The int32 values of an integer_array_t in its storage order, and its + (width, height, depth, bit_width, is_signed). + """ original = value value = self._integer_ndarray(value) bit_width, is_signed = self._word_size(original, value, array_meta) @@ -613,8 +745,8 @@ def _array_result(self, value, array_meta): if value.ndim > 1: value = value.swapaxes(0, 1) - data = numpy.ascontiguousarray(value, dtype=numpy.int32).tobytes() - return (0, 0.0, data, (value.size, width, height, depth, bit_width, int(is_signed))) + storage = numpy.ascontiguousarray(value, dtype=numpy.int32).reshape(-1) + return storage, (width, height, depth, bit_width, int(is_signed)) @staticmethod def _integer_ndarray(value): diff --git a/tests/tb_python_pkg_bridge.vhd b/tests/tb_python_pkg_bridge.vhd index 2842016..5d2b7af 100644 --- a/tests/tb_python_pkg_bridge.vhd +++ b/tests/tb_python_pkg_bridge.vhd @@ -16,6 +16,7 @@ library vunit_lib; context vunit_lib.vunit_context; library python_bridge; context python_bridge.python_context; +use vunit_lib.dict_2008p_pkg.all; entity tb_python_pkg_bridge is generic (runner_cfg : string); @@ -34,6 +35,9 @@ architecture tb of tb_python_pkg_bridge is constant std_ulogic_arg_error : string := "arg cannot convert 'X'; expected '0', '1', 'L' or 'H'"; constant std_ulogic_arg_call : string := "describe(__vunit__.error(""arg cannot convert 'X'; expected '0', '1', 'L' or 'H'""))"; + constant dict_arg_error : string := "arg cannot convert the character value of the dict key ""c"""; + constant dict_arg_call : string := + "describe({""c"": __vunit__.error(""arg cannot convert the character value of the dict key \""c\"""")})"; constant unsigned_arg_error : string := "kwarg_unsigned cannot convert ""1010X010""; the value has metavalues"; constant unsigned_arg_call : string := "describe(v=__vunit__.error(""kwarg_unsigned cannot convert \""1010X010\""; the value has metavalues""))"; @@ -59,6 +63,7 @@ begin variable arr, arr_b, result : integer_array_t; variable ptr : integer_vector_ptr_t; + variable dict, inner : dict_t; variable slv4 : std_ulogic_vector(3 downto 0); variable slv9_desc : std_ulogic_vector(8 downto 0) := "UX01ZWLH-"; variable slv9_asc : std_ulogic_vector(0 to 8) := "UX01ZWLH-"; @@ -1100,6 +1105,259 @@ begin unmock(default_logger); unmock(golden_logger); + --------------------------------------------------------------------- + -- dict_t + --------------------------------------------------------------------- + elsif run("Test eval of dict with every value type") then + dict := eval_dict("{'i': -7, 'r': 0.5, 's': 'h\u00e9llo', 'b': True, 'l': [1, 2, 3], 'e': [], 'n': {'x': 1}}"); + check_equal(num_keys(dict), 7); + check_equal(get_integer(dict, "i"), -7); + check_equal(get_real(dict, "r"), 0.5); + check_equal(get_string(dict, "s"), "h" & character'val(195) & character'val(169) & "llo"); + check_equal(get_boolean(dict, "b"), true); + ptr := get_integer_vector_ptr_t_ref(dict, "l"); + check_equal(length(ptr), 3); + check_equal(get(ptr, 2), 3); + check_equal(length(get_integer_vector_ptr_t_ref(dict, "e")), 0); + inner := get_dict_t_ref(dict, "n"); + check_equal(get_integer(inner, "x"), 1); + + -- The eval and call aliases select the overload from the result type + dict := eval("{'a': 1}"); + check_equal(get_integer(dict, "a"), 1); + exec("def make(n):" + " return {'n': n, 'sq': {'v': n * n}}"); + dict := call("make", arg(5)); + check_equal(get_integer(dict, "n"), 5); + check_equal(get_integer(get_dict_t_ref(dict, "sq"), "v"), 25); + dict := call_dict("make", arg(3)); + check_equal(num_keys(dict), 2); + + dict := eval_dict("{}"); + check_equal(num_keys(dict), 0); + + elsif run("Test that a returned dict can be deallocated") then + for i in 1 to 3 loop + dict := eval_dict("{'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 6}"); + check_equal(get_integer(dict, "f"), 6); + deallocate(dict); + end loop; + + elsif run("Test that eval of dict gives exact reals") then + dict := eval_dict("{'a': 0.1, 'b': -2.5e-100, 'c': 1e300, 'e': 0.0, 'f': 1.7976931348623157e308}"); + check_equal(get_real(dict, "a"), 0.1, max_diff => 0.0); + check_equal(get_real(dict, "b"), -2.5e-100, max_diff => 0.0); + check_equal(get_real(dict, "c"), 1.0e300, max_diff => 0.0); + check_equal(get_real(dict, "e"), 0.0, max_diff => 0.0); + check_equal(get_real(dict, "f"), 1.7976931348623157e308, max_diff => 0.0); + + elsif run("Test that eval of dict is strict about keys and values") then + mock(default_logger, failure); + dict := eval_dict("[1]"); + check_log( + default_logger, + "eval(""[1]"") failed:" & LF & + "TypeError: Cannot convert Python list ([1]) to VHDL dict_t; expected a dict", + failure + ); + check_equal(num_keys(dict), 0); + + dict := eval_dict("{1: 2}"); + check_log( + default_logger, + "eval(""{1: 2}"") failed:" & LF & + "TypeError: Cannot convert the dict key 1, a int; expected str", + failure + ); + + dict := eval_dict("{'a': None}"); + check_log( + default_logger, + "eval(""{'a': None}"") failed:" & LF & + "TypeError: Cannot convert the value of dict key 'a', a NoneType (None); " & + "expected int, float, str, bool, a list of int, a NumPy array of integers or a dict", + failure + ); + + dict := eval_dict("{'a': 2**31}"); + check_log( + default_logger, + "eval(""{'a': 2**31}"") failed:" & LF & + "OverflowError: 2147483648 of dict key 'a' is outside the range of VHDL integer " & + "(-2147483648 to 2147483647)", + failure + ); + + dict := eval_dict("{'a': {'b': [1, 'x']}}"); + check_log( + default_logger, + "eval(""{'a': {'b': [1, 'x']}}"") failed:" & LF & + "TypeError: Cannot convert the value of dict key 'b', a list ([1, 'x']); " & + "expected int, float, str, bool, a list of int, a NumPy array of integers or a dict", + failure + ); + + exec("import numpy as np"); + dict := eval_dict("{'a': np.array([2**31])}"); + check_log( + default_logger, + "eval(""{'a': np.array([2**31])}"") failed:" & LF & + "OverflowError: dict key 'a': The array values (2147483648 to 2147483648) do not fit in " & + "the integer_array_t word size (32 bit signed, -2147483648 to 2147483647)", + failure + ); + + dict := eval_dict("{'a': np.array([0.5])}"); + check_log( + default_logger, + "eval(""{'a': np.array([0.5])}"") failed:" & LF & + "TypeError: dict key 'a': Cannot convert a NumPy array of dtype float64 to a VHDL integer_array_t; " & + "expected an integer or boolean dtype", + failure + ); + + dict := call_dict("dict", kwarg("a", 1.5)); + check_equal(get_real(dict, "a"), 1.5); + check_no_log; + unmock(default_logger); + + elsif run("Test eval of dict with NumPy arrays") then + exec("import numpy as np"); + dict := eval_dict( + "{'u8': np.array([[1, 2, 3], [4, 5, 255]], dtype=np.uint8), 'b': np.array([True, False]), " & + "'i64': np.array([-5, 7]), 'e': np.array([], dtype=np.int16)}" + ); + result := get_integer_array_t_ref(dict, "u8"); + check_equal(width(result), 2); + check_equal(height(result), 3); + check_equal(bit_width(result), 8); + check_false(is_signed(result)); + check_equal(get(result, 0, 1), 2); + check_equal(get(result, 1, 2), 255); + result := get_integer_array_t_ref(dict, "b"); + check_equal(length(result), 2); + check_equal(bit_width(result), 1); + check_false(is_signed(result)); + check_equal(get(result, 0), 1); + check_equal(get(result, 1), 0); + result := get_integer_array_t_ref(dict, "i64"); + check_equal(bit_width(result), 32); + check_true(is_signed(result)); + check_equal(get(result, 0), -5); + check_equal(get(result, 1), 7); + result := get_integer_array_t_ref(dict, "e"); + check_equal(length(result), 0); + check_equal(bit_width(result), 16); + + elsif run("Test integer_array_t values of a dict_t argument") then + exec( + "def describe_arrays(d, **kwargs):" + + " return ' '.join(f'{k}:{v.shape}:{v.sum()}' for k, v in list(d.items()) + list(kwargs['k'].items()))" + ); + arr := new_2d(width => 3, height => 2, bit_width => 16, is_signed => false); + for y in 0 to 1 loop + for x in 0 to 2 loop + set(arr, x, y, y * 10 + x); + end loop; + end loop; + dict := new_dict; + set_integer_array_t_ref(dict, "a", arr); + inner := new_dict; + arr_b := new_1d(length => 2, bit_width => 8, is_signed => false); + set(arr_b, 0, 4); + set(arr_b, 1, 5); + set_integer_array_t_ref(inner, "b", arr_b); + check_equal(call_string("describe_arrays", arg(dict), kwarg("k", inner)), "a:(3, 2):36 b:(2,):9"); + + elsif run("Test integer_array_t values of a dict_t round trip through Python") then + exec("def identity(**kwargs):" + " return kwargs['d']"); + arr := new_1d(length => 3, bit_width => 12, is_signed => true); + set(arr, 0, -2048); + set(arr, 1, 0); + set(arr, 2, 2047); + dict := new_dict; + set_integer_array_t_ref(dict, "a", arr); + inner := call_dict("identity", kwarg("d", dict)); + result := get_integer_array_t_ref(inner, "a"); + check_equal(bit_width(result), 12); + check_true(is_signed(result)); + check_equal(get(result, 0), -2048); + check_equal(get(result, 2), 2047); + + elsif run("Test arg and kwarg of dict_t values") then + define_describe; + dict := new_dict; + set_integer(dict, "i", 5); + check_equal(call_string("describe", arg(dict)), "{'i': 5}"); + check_equal(call_string("describe", kwarg("d", dict)), "d={'i': 5}"); + check_equal(call_string("describe", arg(new_dict)), "{}"); + + dict := new_dict; + set_real(dict, "r", 0.5); + set_string(dict, "s", "a ""q"""); + set_boolean(dict, "b", false); + set_std_ulogic(dict, "u", 'H'); + set_integer_vector(dict, "iv", integer_vector'(1, 2)); + set_real_vector(dict, "rv", real_vector'(1.5, 2.5)); + inner := new_dict; + set_integer(inner, "n", 1); + set_dict_t_ref(dict, "d", inner); + exec("def show(d):" + " return repr(sorted(d.items()))"); + check_equal( + call_string("show", arg(dict)), + "[('b', False), ('d', {'n': 1}), ('iv', [1, 2]), ('r', 0.5), ('rv', [1.5, 2.5]), ('s', 'a ""q""'), ('u', True)]" + ); + + elsif run("Test dict_t round trip through Python") then + exec("def identity(**kwargs):" + " return kwargs['d']"); + dict := new_dict; + inner := new_dict; + set_integer(inner, "x", -3); + set_dict_t_ref(dict, "inner", inner); + ptr := new_integer_vector_ptr(2); + set(ptr, 0, 4); + set(ptr, 1, 5); + set_integer_vector_ptr_t_ref(dict, "ints", ptr); + set_integer(dict, "i", 1); + set_real(dict, "r", 0.1); + set_string(dict, "s", "a ""quoted"" \ string"); + set_boolean(dict, "b", true); + set_std_ulogic(dict, "u", '1'); + set_integer_vector(dict, "iv", integer_vector'(7, 8)); + inner := call_dict("identity", kwarg("d", dict)); + check_equal(num_keys(inner), 8); + check_equal(get(get_integer_vector_ptr_t_ref(inner, "iv"), 0), 7); + check_equal(get_integer(inner, "i"), 1); + check_equal(get_real(inner, "r"), 0.1); + check_equal(get_string(inner, "s"), "a ""quoted"" \ string"); + check_equal(get_boolean(inner, "b"), true); + check_equal(get_integer(get_dict_t_ref(inner, "inner"), "x"), -3); + check_equal(get(get_integer_vector_ptr_t_ref(inner, "ints"), 1), 5); + + elsif run("Test dict_t values that cannot be converted to Python") then + define_describe; + define_error_helper; + dict := new_dict; + set_character(dict, "c", 'x'); + mock(python_logger, failure); + mock(default_logger, failure); + check_equal(call_string("describe", arg(dict)), ""); + check_log(python_logger, dict_arg_error, failure); + exec("failing_source = r" & py_quotes & dict_arg_call & py_quotes); + check_log( + default_logger, + "eval(""" & dict_arg_call & """) failed:" & LF & + eval_string("expected_error(failing_source, '', True)"), + failure + ); + unmock(python_logger); + unmock(default_logger); + + elsif run("Test a dict_t keyword argument in a group") then + define_describe; + dict := new_dict; + set_integer(dict, "k", 1); + check_equal(call_string("describe", arg(1), arg(2) & kwarg("d", dict) & kwarg("e", 3)), "1, 2, d={'k': 1}, e=3"); + end if; end loop;