int output value error upon callback to iRODS rule-language rule with remote block

[tsmeele]

Server: iRODS 4.3.2

Preconditions:
We execute a Python rule "myrule" via irule, and the rule does a callback to an iRods rule language rule "intfunction".
The rule intfunction returns an integer value calculated within a remote block.

Expected:
No errors in server log, integer value returned from intfunction to myrule.

Actual:
The integer value is misinterpreted, and the server log shows an exception "no to_python (by value) converter found for C++ type: int"

Issue can be reproduced using below iRODS rule intfunction:

intfunction(*output) {
   remote("irods","null") {
      *output = 42;
   }
}

Example client rule myrule (executed via irule and Python rule engine) that shows issue:

def main(rule_args, callback, rei):
    output = ""
    ret = callback.intfunction(output)
    callback.writeLine("stdout", "Answer is ={}=".format(ret))

input null
output ruleExecOut

iRule output: (note the asterisk as output instead of value 42)

irods@irods:/etc/irods$ irule -r irods_rule_engine_plugin-python-instance -F myrule.r
Answer is ={'code': 0, 'status': True, 'arguments': ['*']}=
irods@irods:/etc/irods$

RodsLog entry in syslog:

Jul 12 18:07:22 irods irodsServer[9422]: {"log_category":"rule_engine","log_level":"error","log_message":"caught python exception","python_exception":"TypeError: No to_python (by-value) converter found for C++ type: int\n","request_api_name":"EXEC_MY_RULE_AN","request_api_number":625,"request_api_version":"d","request_client_user":"rods","request_host":"127.0.0.1","request_proxy_user":"rods","request_release_version":"rods4.3.2","rule_engine_plugin":"python","server_host":"irods","server_pid":9422,"server_timestamp":"2024-07-12T18:07:22.652Z","server_type":"agent","server_zone":"tempZone"}

[korydraughn]

I believe only strings can be passed across REP boundaries. We recently added documentation which demonstrates how to pass values between the NREP and PREP. See https://github.com/irods/irods_rule_engine_plugin_python?tab=readme-ov-file#how-do-i-pass-values-back-to-the-caller-across-rule-engine-plugin-boundaries.

What happens if intfunction sets *output to "42"?

[trel]

I'm very happy that both the example AND the documentation use 42.

[tsmeele]

Well spotted Terrell! Now what was the question? This number seemed most appropriate for Python.

[tsmeele]

It turns out that the Python exception in the server log only shows up when the remote block is used. Nevertheless the exception is a good indication of what actually seems to happen.
Without the remote block, the interpretation issue remains, and is influenced by the type of the variable passed by caller as shown by the following snippets:

Updated intfunction:

intfunction(*output1, *output2, *output3, *output4) {
   *output1 = 42;
   *output2 = 42;
   *output3 = "42";
   *output4 = "42";
}

Updated myrule and irule output:

irods@irods:/etc/irods$ cat myrule.r 
#!/usr/bin/irule -r irods_rule_engine_plugin-python-instance -F

def main(rule_args, callback, rei):
    output1 = 10
    output2 = '11'
    output3 = 12
    output4 = '13'
    ret = callback.intfunction(output1, output2, output3, output4)
    callback.writeLine("stdout", "Answer is ={}=".format(ret))

input null
output ruleExecOut
irods@irods:/etc/irods$ irule -r irods_rule_engine_plugin-python-instance -F myrule.r
Answer is ={'code': 0, 'status': True, 'arguments': [10, '*', 12, '42']}=
irods@irods:/etc/irods$

Where caller variable is an int, the variable is not updated with output from intfunction. Where caller variable is string, its content is updated by intfunction yet int values returned by intfunction get mangled (probably interpreted as a char).