Add support for progressive call results

Great to finally see this in the light David (and all you others of course)! It looks great.
We're currently using Crossbar at work, but treating it as a "black box" and not making use of its application component hosting features (and many other features), and using another router which is more slim is thus of interest to us.
One thing we need though from the Advanced Profile is progressive call result (a stable part), as we're using this to deliver frames from an image detector. I don't know how high up on your agenda this is, but I'm filing this issue to keep track.

Thank you for your support and interest. Progressive call results is a key feature and is fairly high on my list. I suspect that it is at most a fews days of coding and testing to add to bonefish. I should have some spare time over the next few weeks to make it happen.

\o/

Looks like I will have time on Saturday to start looking into this.

We're going to hold you to that, Dave! My recommendation is underpromise & overdeliver :)

I got this work started on Sat. I should be able to wrap it up next weekend.

Dave, if this requires more work still, can you push your (presumably unfinished) code to your branch so potentially interested parties could take it from there?

Sure thing. At some point tomorrow I should be able to push what I have.
On Thu, Jan 7, 2016 at 5:35 PM, Jakob Petsovits<EMAIL_ADDRESS>wrote:

Dave, if this requires more work still, can you push your (presumably
unfinished) code to your branch so potentially interested parties could
take it from there?
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-169828978.

Awesome, thanks.

Still working on it. Just need a bit of time over the weekend to finish
things up.
On Fri, Jan 8, 2016 at 12:50 PM, Pramukta Kumar<EMAIL_ADDRESS>wrote:

Awesome, thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170072943.

@pramukta @estan I have pushed a patch to https://github.com/davidchappelle/bonefish.git for progressive call result support. Once you clone that repo just checkout the feature/progressive_call_results branch.
I haven't had a chance to do much in terms of testing as I am on the road now until 11:30pm. I have a corresponding autobahn-cpp commit in progress as well and should have that in tomorrow. If someone wants to test things out with autobahn-python in the meantime that would be great.

This is embarrassing, but I'm actually having trouble checking out the branch. It's not visible in git branch -r after a git clone --recursive:
[estan@pyret bonefish]$ git remote -v
origin  https://github.com/tplgy/bonefish.git (fetch)
origin  https://github.com/tplgy/bonefish.git (push)
[estan@pyret bonefish]$ git fetch origin
[estan@pyret bonefish]$ git branch -r
  origin/HEAD -> origin/master
  origin/master
[estan@pyret bonefish]$

Anyone know what the problem might be? The branch is clearly visible in GitHub UI.

@estan, probably because it's in Dave's own fork :) you could add his repot as a remote.

@jrogers facepalm. Thanks :)

@davidchappelle, I may be wrong, but shouldn't
https://github.com/davidchappelle/bonefish/blob/master/src/bonefish/messages/wamp_result_message.hpp#L166
be ==, not != ? I hit that assert while testing. Right now having another problem I'm trying to debug.

Yes you are correct.
On Saturday, January 9, 2016, Elvis Stansvik<EMAIL_ADDRESS>wrote:

@davidchappelle https://github.com/davidchappelle, I may be wrong, but
shouldn't
https://github.com/davidchappelle/bonefish/blob/master/src/bonefish/messages/wamp_result_message.hpp#L166
be ==, not != ? I hit that assert while testing. Right now having another
problem I'm trying to debug.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170280999.

Alright, if I change that to ==, I instead hit this assert in rapidjson when the router tries to serialize the RESULT (I think):
[estan@pyret bonefish-build]$ gdb ./daemon/bonefish
GNU gdb (GDB) 7.10.1
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./daemon/bonefish...done.
(gdb) run  --realm "crossbardemo" --websocket-port 8080
Starting program: /home/estan/Nedladdat/bonefish-build/daemon/bonefish --realm "crossbardemo" --websocket-port 8080
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
bonefish: /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:328: void rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator>::Prefix(rapidjson::Type) [with OutputStream = bonefish::{anonymous}::omemstream; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]: Assertion `type == kStringType' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff6db35f8 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff6db35f8 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff6db4a7a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff6dac417 in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff6dac4c2 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00000000007ba6fa in rapidjson::Writer<bonefish::(anonymous namespace)::omemstream, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator>::Prefix (this=0x7fffffffbb10,
    type=rapidjson::kNullType) at /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:328
#5  0x00000000007bade1 in rapidjson::Writer<bonefish::(anonymous namespace)::omemstream, rapidjson::UTF8<char>, rapidjson::UTF8<char>, rapidjson::CrtAllocator>::Null (this=0x7fffffffbb10)
    at /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:108
#6  0x00000000007bac49 in json_msgpack::write_json<rapidjson::Writer<bonefish::(anonymous namespace)::omemstream>, json_msgpack::no_custom_string_conversion> (writer=..., o=...)
    at /home/estan/Nedladdat/bonefish/third-party/json-msgpack/include/json_msgpack/json_msgpack_sax.hpp:338
#7  0x00000000007ba2f8 in json_msgpack::write_json<rapidjson::Writer<bonefish::(anonymous namespace)::omemstream>, bonefish::(anonymous namespace)::wamp_bin_string_conversion> (writer=...,
    o=...) at /home/estan/Nedladdat/bonefish/third-party/json-msgpack/include/json_msgpack/json_msgpack_sax.hpp:327
#8  0x00000000007b9e03 in bonefish::json_serializer::serialize (this=0xb9e5d0, message=...) at /home/estan/Nedladdat/bonefish/src/bonefish/serialization/json_serializer.cpp:214
#9  0x000000000082ff1b in bonefish::websocket_transport::send_message(bonefish::wamp_message&&) (this=0xba6e30,
    message=<unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x38bafc, DIE 0x3cd406>)
    at /home/estan/Nedladdat/bonefish/src/bonefish/websocket/websocket_transport.cpp:40
#10 0x000000000084eee9 in bonefish::wamp_dealer::process_yield_message (this=0xb9fa50, session_id=..., yield_message=0xbaeb90)
    at /home/estan/Nedladdat/bonefish/src/bonefish/dealer/wamp_dealer.cpp:478
#11 0x00000000007b10e2 in bonefish::wamp_router_impl::process_yield_message (this=0xb9ef40, session_id=..., yield_message=0xbaeb90)
    at /home/estan/Nedladdat/bonefish/src/bonefish/router/wamp_router_impl.cpp:273
#12 0x00000000007a94b2 in bonefish::wamp_router::process_yield_message (this=0xb9e500, session_id=..., yield_message=0xbaeb90)
    at /home/estan/Nedladdat/bonefish/src/bonefish/router/wamp_router.cpp:138
#13 0x00000000008452b5 in bonefish::wamp_message_processor::process_message(std::unique_ptr<bonefish::wamp_message, std::default_delete<bonefish::wamp_message> > const&, std::unique_ptr<bonefish::wamp_transport, std::default_delete<bonefish::wamp_transport> >&&, bonefish::wamp_connection_base*) (this=0xba0918, message=std::unique_ptr<bonefish::wamp_message> containing 0xbaeb90,
    transport=<unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x459219, DIE 0x495841>, connection=0xba0ef0)
    at /home/estan/Nedladdat/bonefish/src/bonefish/common/wamp_message_processor.cpp:165
#14 0x00000000007d0fd4 in bonefish::websocket_server_impl::on_message (this=0xba08d0, handle=std::weak_ptr (count 6, weak 8) 0xba0ef0, buffer=std::shared_ptr (count 3, weak 0) 0xbae530)
    at /home/estan/Nedladdat/bonefish/src/bonefish/websocket/websocket_server_impl.cpp:218
#15 0x00000000007cfd46 in bonefish::websocket_server_impl::<lambda(websocketpp::connection_hdl, websocketpp::endpoint<websocketpp::connection<bonefish::websocket_config>, bonefish::websocket_config>::message_ptr)>::operator()(websocketpp::connection_hdl, websocketpp::endpoint<websocketpp::connection<bonefish::websocket_config>, bonefish::websocket_config>::message_ptr) const (
    __closure=0xba6330, handle=std::weak_ptr (count 6, weak 8) 0xba0ef0, message=std::shared_ptr (count 3, weak 0) 0xbae530)
    at /home/estan/Nedladdat/bonefish/src/bonefish/websocket/websocket_server_impl.cpp:108
#16 0x00000000007d1dc0 in std::_Function_handler<void(std::weak_ptr<void>, std::shared_ptr<websocketpp::message_buffer::message<websocketpp::message_buffer::alloc::con_msg_manager> >), bonefish::websocket_server_impl::start(const boost::asio::ip::address&, uint16_t)::<lambda(websocketpp::connection_hdl, websocketpp::endpoint<websocketpp::connection<bonefish::websocket_config>, bonefish::websocket_config>::message_ptr)> >::_M_invoke(const std::_Any_data &, <unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x295a52, DIE 0x330949>, <unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x295a52, DIE 0x33094e>) (__functor=...,
    __args#0=<unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x295a52, DIE 0x330949>,
    __args#1=<unknown type in /home/estan/Nedladdat/bonefish-build/daemon/bonefish, CU 0x295a52, DIE 0x33094e>) at /usr/include/c++/5.3.0/functional:1871
#17 0x0000000000806485 in std::function<void (std::weak_ptr<void>, std::shared_ptr<websocketpp::message_buffer::message<websocketpp::message_buffer::alloc::con_msg_manager> >)>::operator()(std::weak_ptr<void>, std::shared_ptr<websocketpp::message_buffer::message<websocketpp::message_buffer::alloc::con_msg_manager> >) const (this=0xba1a90, __args#0=std::weak_ptr (empty) 0x0,
    __args#1=std::shared_ptr (empty) 0x0) at /usr/include/c++/5.3.0/functional:2271
---Type <return> to continue, or q <return> to quit---

Haven't quite figured out what happens.
I'm testing right now with the examples/twisted/wamp/rpc/progress from AutobahnPython.

The debug output from bonefish when running frontend.py and getting this crash is:
[2016-01-09 22:22:13] [connect] WebSocket Connection <IP_ADDRESS>:47436 v13 "AutobahnPython/0.11.0" / 101
[wamp_message_processor.cpp:46][process_message] processing message: hello
[wamp_router_impl.cpp:108][attach_session] attaching session: session [2257228500326156,crossbardemo,none]
[wamp_dealer.cpp:65][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_broker.cpp:51][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_router_impl.cpp:164][process_hello_message] session [2257228500326156,crossbardemo,none], hello [crossbardemo, {"roles":{"subscriber":{"features":{"publisher_identification":true, "pattern_based_subscription":true, "subscription_revocation":true}}, "publisher":{"features":{"publisher_identification":true, "publisher_exclusion":true, "subscriber_blackwhite_listing":true}}, "caller":{"features":{"caller_identification":true, "progressive_call_results":true}}, "callee":{"features":{"progressive_call_results":true, "pattern_based_registration":true, "registration_revocation":true, "shared_registration":true, "caller_identification":true}}}}]
[wamp_router_impl.cpp:192][process_hello_message] session [2257228500326156,crossbardemo,open], welcome [2257228500326156, {"roles":{"dealer":{"features":{"call_timeout":true, "progressive_call_results":true}}, "broker":{}}}]
[websocket_transport.cpp:39][send_message] sending message: welcome
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 120 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 78

[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (120) [1] [2,2257228500326156,{"roles":{"dealer":{"features":{"call_timeout":true,"progressive_call_results":true}},"broker":{}}}]

[wamp_message_processor.cpp:46][process_message] processing message: call
[wamp_dealer.cpp:166][process_call_message] session [2257228500326156,crossbardemo,open], call [1, {"receive_progress":true}, com.myapp.longop, [3], nil]
[wamp_dealer.cpp:219][process_call_message] session [2257228500326156,crossbardemo,open], invocation [1, 1, {"receive_progress":true}, [3], nil]
[websocket_transport.cpp:39][send_message] sending message: invocation
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 38 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 26

[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (38) [1] [68,1,1,{"receive_progress":true},[3]]

[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [0], nil]
[wamp_dealer.cpp:477][process_yield_message] session [7090515283965860,crossbardemo,open], result [1, {nil:nil}, [0], nil]
[websocket_transport.cpp:39][send_message] sending message: result
bonefish: /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:328: void rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator>::Prefix(rapidjson::Type) [with OutputStream = bonefish::{anonymous}::omemstream; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]: Assertion `type == kStringType' failed.
Aborted (core dumped)
[estan@pyret bonefish-build]$

The result object doesn't look right.
On Saturday, January 9, 2016, Elvis Stansvik<EMAIL_ADDRESS>wrote:

The debug output from bonefish when running frontend.py and getting this
crash is:
[2016-01-09 22:22:13] [connect] WebSocket Connection <IP_ADDRESS>:47436 v13 "AutobahnPython/0.11.0" / 101
[wamp_message_processor.cpp:46][process_message] processing message: hello
[wamp_router_impl.cpp:108][attach_session] attaching session: session [2257228500326156,crossbardemo,none]
[wamp_dealer.cpp:65][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_broker.cpp:51][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_router_impl.cpp:164][process_hello_message] session [2257228500326156,crossbardemo,none], hello [crossbardemo, {"roles":{"subscriber":{"features":{"publisher_identification":true, "pattern_based_subscription":true, "subscription_revocation":true}}, "publisher":{"features":{"publisher_identification":true, "publisher_exclusion":true, "subscriber_blackwhite_listing":true}}, "caller":{"features":{"caller_identification":true, "progressive_call_results":true}}, "callee":{"features":{"progressive_call_results":true, "pattern_based_registration":true, "registration_revocation":true, "shared_registration":true, "caller_identification":true}}}}]
[wamp_router_impl.cpp:192][process_hello_message] session [2257228500326156,crossbardemo,open], welcome [2257228500326156, {"roles":{"dealer":{"features":{"call_timeout":true, "progressive_call_results":true}}, "broker":{}}}]
[websocket_transport.cpp:39][send_message] sending message: welcome
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 120 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 78
[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (120) [1] [2,2257228500326156,{"roles":{"dealer":{"features":{"call_timeout":true,"progressive_call_results":true}},"broker":{}}}]
[wamp_message_processor.cpp:46][process_message] processing message: call
[wamp_dealer.cpp:166][process_call_message] session [2257228500326156,crossbardemo,open], call [1, {"receive_progress":true}, com.myapp.longop, [3], nil]
[wamp_dealer.cpp:219][process_call_message] session [2257228500326156,crossbardemo,open], invocation [1, 1, {"receive_progress":true}, [3], nil]
[websocket_transport.cpp:39][send_message] sending message: invocation
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 38 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 26
[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (38) [1] [68,1,1,{"receive_progress":true},[3]]
[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [0], nil]
[wamp_dealer.cpp:477][process_yield_message] session [7090515283965860,crossbardemo,open], result [1, {nil:nil}, [0], nil]
[websocket_transport.cpp:39][send_message] sending message: result
bonefish: /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:328: void rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator>::Prefix(rapidjson::Type) [with OutputStream = bonefish::{anonymous}::omemstream; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]: Assertion `type == kStringType' failed.
Aborted (core dumped)
[estan@pyret bonefish-build]$
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170283429.

Looks like the result details are invalid. Maybe not getting set correctly.
On Saturday, January 9, 2016, Elvis Stansvik<EMAIL_ADDRESS>wrote:

The debug output from bonefish when running frontend.py and getting this
crash is:
[2016-01-09 22:22:13] [connect] WebSocket Connection <IP_ADDRESS>:47436 v13 "AutobahnPython/0.11.0" / 101
[wamp_message_processor.cpp:46][process_message] processing message: hello
[wamp_router_impl.cpp:108][attach_session] attaching session: session [2257228500326156,crossbardemo,none]
[wamp_dealer.cpp:65][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_broker.cpp:51][attach_session] attach session: session [2257228500326156,crossbardemo,none]
[wamp_router_impl.cpp:164][process_hello_message] session [2257228500326156,crossbardemo,none], hello [crossbardemo, {"roles":{"subscriber":{"features":{"publisher_identification":true, "pattern_based_subscription":true, "subscription_revocation":true}}, "publisher":{"features":{"publisher_identification":true, "publisher_exclusion":true, "subscriber_blackwhite_listing":true}}, "caller":{"features":{"caller_identification":true, "progressive_call_results":true}}, "callee":{"features":{"progressive_call_results":true, "pattern_based_registration":true, "registration_revocation":true, "shared_registration":true, "caller_identification":true}}}}]
[wamp_router_impl.cpp:192][process_hello_message] session [2257228500326156,crossbardemo,open], welcome [2257228500326156, {"roles":{"dealer":{"features":{"call_timeout":true, "progressive_call_results":true}}, "broker":{}}}]
[websocket_transport.cpp:39][send_message] sending message: welcome
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 120 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 78
[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (120) [1] [2,2257228500326156,{"roles":{"dealer":{"features":{"call_timeout":true,"progressive_call_results":true}},"broker":{}}}]
[wamp_message_processor.cpp:46][process_message] processing message: call
[wamp_dealer.cpp:166][process_call_message] session [2257228500326156,crossbardemo,open], call [1, {"receive_progress":true}, com.myapp.longop, [3], nil]
[wamp_dealer.cpp:219][process_call_message] session [2257228500326156,crossbardemo,open], invocation [1, 1, {"receive_progress":true}, [3], nil]
[websocket_transport.cpp:39][send_message] sending message: invocation
[2016-01-09 22:22:13] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 38 payload bytes
[2016-01-09 22:22:13] [frame_header] Header Bytes:
[0] (2) 81 26
[2016-01-09 22:22:13] [frame_payload] Payload Bytes:
[0] (38) [1] [68,1,1,{"receive_progress":true},[3]]
[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [0], nil]
[wamp_dealer.cpp:477][process_yield_message] session [7090515283965860,crossbardemo,open], result [1, {nil:nil}, [0], nil]
[websocket_transport.cpp:39][send_message] sending message: result
bonefish: /home/estan/Nedladdat/bonefish/third-party/rapidjson/include/rapidjson/writer.h:328: void rapidjson::Writer<OutputStream, SourceEncoding, TargetEncoding, StackAllocator>::Prefix(rapidjson::Type) [with OutputStream = bonefish::{anonymous}::omemstream; SourceEncoding = rapidjson::UTF8<>; TargetEncoding = rapidjson::UTF8<>; StackAllocator = rapidjson::CrtAllocator]: Assertion `type == kStringType' failed.
Aborted (core dumped)
[estan@pyret bonefish-build]$
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170283429.

I haven't used msgpack before, but this:
 result_message->set_details(result_details.marshal(yield_message->get_zone()));

Shouldn't that be result_message->get_zone()?

If I change that, it seems to get a little further in processing the YIELDs:
[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [0], nil]
[wamp_dealer.cpp:477][process_yield_message] session [7090515283965860,crossbardemo,open], result [1, {"progress":true}, [0], nil]
[websocket_transport.cpp:39][send_message] sending message: result
[2016-01-09 22:34:33] [frame_header] Dispatching write containing 1 message(s) containing 2 header bytes and 28 payload bytes
[2016-01-09 22:34:33] [frame_header] Header Bytes:
[0] (2) 81 1C

[2016-01-09 22:34:33] [frame_payload] Payload Bytes:
[0] (28) [1] [50,1,{"progress":true},[0]]

[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [1], nil]
[wamp_dealer.cpp:428][process_yield_message] unable to find invocation ... timed out or session closed
[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {"progress":true}, [2], nil]
[wamp_dealer.cpp:428][process_yield_message] unable to find invocation ... timed out or session closed
[wamp_message_processor.cpp:46][process_message] processing message: yield
[wamp_dealer.cpp:424][process_yield_message] session [7090515283965860,crossbardemo,open], yield [1, {}, [3], nil]
[wamp_dealer.cpp:428][process_yield_message] unable to find invocation ... timed out or session closed

Yep good catch. Apologies for the bugs. Time crunch this morning getting
out the door.
On Saturday, January 9, 2016, Elvis Stansvik<EMAIL_ADDRESS>wrote:

I haven't used msgpack before, but this:
result_message->set_details(result_details.marshal(yield_message->get_zone()));
Shouldn't that be result_message->get_zone()?
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170283937.

That is, it seems it's able to send the first RESULT to the caller:
(test_env)[estan@pyret progress]$ python frontend.py
2016-01-09T22:34:33+0100 Starting factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x7faf44b55710>
2016-01-09T22:34:33+0100 session attached
2016-01-09T22:34:33+0100 Progress: 0

But after that, the invocation seems gone in the router.

Heh, no problem :)

I guess it's the uncondiational erasure of the invocation at the end of wamp_dealer::process_yield_message that's the culprit. I think it will need some more logic to keep the invocation alive for the duration of the call?

Anyway, I gotto go to bed, but this looks promising. Thanks!

I just had to quickly test, and with
diff --git a/src/bonefish/dealer/wamp_dealer.cpp b/src/bonefish/dealer/wamp_dealer.cpp
index ecd67c6..1a179a0 100644
--- a/src/bonefish/dealer/wamp_dealer.cpp
+++ b/src/bonefish/dealer/wamp_dealer.cpp
@@ -466,7 +466,7 @@ void wamp_dealer::process_yield_message(const wamp_session_id& session_id,
     std::unique_ptr<wamp_result_message> result_message(
             new wamp_result_message(std::move(yield_message->release_zone())));
     result_message->set_request_id(dealer_invocation->get_request_id());
-    result_message->set_details(result_details.marshal(yield_message->get_zone()));
+    result_message->set_details(result_details.marshal(result_message->get_zone()));
     result_message->set_arguments(yield_message->get_arguments());
     result_message->set_arguments_kw(yield_message->get_arguments_kw());

@@ -483,9 +483,11 @@ void wamp_dealer::process_yield_message(const wamp_session_id& session_id,
     // will detach the session. When this happens the pending invocations
     // be cleaned up. So we don't use an iterator here to erase the pending
     // invocation because it may have just been invalidated above.
-    m_pending_callee_invocations[session_id].erase(request_id);
-    m_pending_caller_invocations[session->get_session_id()].erase(request_id);
-    m_pending_invocations.erase(request_id);
+    if (!yield_options.get_option_or("progress", false)) {
+        m_pending_callee_invocations[session_id].erase(request_id);
+        m_pending_caller_invocations[session->get_session_id()].erase(request_id);
+        m_pending_invocations.erase(request_id);
+    }
 }

 void wamp_dealer::send_error(const std::unique_ptr<wamp_transport>& transport,
diff --git a/src/bonefish/messages/wamp_result_message.hpp b/src/bonefish/messages/wamp_result_message.hpp
index 3ec1af1..802a591 100644
--- a/src/bonefish/messages/wamp_result_message.hpp
+++ b/src/bonefish/messages/wamp_result_message.hpp
@@ -163,7 +163,7 @@ inline void wamp_result_message::set_request_id(const wamp_request_id& request_i

 inline void wamp_result_message::set_details(const msgpack::object& details)
 {
-    if (details.type != msgpack::type::MAP) {
+    if (details.type == msgpack::type::MAP) {
         m_details = msgpack::object(details, get_zone());
     } else {
         throw std::invalid_argument("invalid details");

it seems to work:
(test_env)[estan@pyret progress]$ python frontend.py
2016-01-09T23:13:32+0100 Starting factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x7f0667c3b710>
2016-01-09T23:13:32+0100 session attached
2016-01-09T23:13:32+0100 Progress: 0
2016-01-09T23:13:33+0100 Progress: 1
2016-01-09T23:13:34+0100 Progress: 2
2016-01-09T23:13:35+0100 Final: 3
2016-01-09T23:13:35+0100 wamp.error.goodbye_and_out: None
2016-01-09T23:13:35+0100 disconnected
2016-01-09T23:13:35+0100 Stopping factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x7f0667c3b710>
2016-01-09T23:13:35+0100 Main loop terminated.
(test_env)[estan@pyret progress]$

But I have no idea if the if (!yield_options.get_option_or("progress", false)) I added is a too liberal/crude fix :P

@davidchappelle, I'm a little unsure on how important the order of things in process_yield_message is, but those .erase that are done in the middle of the method, is it important that those are kept there, before the sending of the result message? They will be removed anyway at the end of the function.
If it's not important that they are removed there and then, then I think the function could be simplified a little with:
diff --git a/src/bonefish/dealer/wamp_dealer.cpp b/src/bonefish/dealer/wamp_dealer.cpp
index ecd67c6..c86a9e7 100644
--- a/src/bonefish/dealer/wamp_dealer.cpp
+++ b/src/bonefish/dealer/wamp_dealer.cpp
@@ -435,11 +435,12 @@ void wamp_dealer::process_yield_message(const wamp_session_id& session_id,
     // We can't have a pending invocation without a pending caller/callee
     // as they are tracked in a synchronized manner. So to have one without
     // the other is considered to be an error.
-    auto pending_callee_invocations_itr =
-            m_pending_callee_invocations.find(session_itr->second->get_session_id());
-    if (pending_callee_invocations_itr == m_pending_callee_invocations.end()) {
+    if (!m_pending_callee_invocations.count(session_itr->second->get_session_id())) {
         throw std::logic_error("dealer pending callee invocations out of sync");
     }
+    if (!m_pending_caller_invocations.count(session->get_session_id())) {
+        throw std::logic_error("dealer pending caller invocations out of sync");
+    }

     wamp_yield_options yield_options;
     yield_options.unmarshal(yield_message->get_options());
@@ -452,21 +453,12 @@ void wamp_dealer::process_yield_message(const wamp_session_id& session_id,
     wamp_result_details result_details;
     if (yield_options.get_option_or("progress", false)) {
         result_details.set_detail("progress", true);
-    } else {
-        pending_callee_invocations_itr->second.erase(request_id);
     }

-    auto pending_caller_invocations_itr =
-            m_pending_caller_invocations.find(session->get_session_id());
-    if (pending_caller_invocations_itr == m_pending_caller_invocations.end()) {
-        throw std::logic_error("dealer pending caller invocations out of sync");
-    }
-    pending_caller_invocations_itr->second.erase(request_id);
-
     std::unique_ptr<wamp_result_message> result_message(
             new wamp_result_message(std::move(yield_message->release_zone())));
     result_message->set_request_id(dealer_invocation->get_request_id());
-    result_message->set_details(result_details.marshal(yield_message->get_zone()));
+    result_message->set_details(result_details.marshal(result_message->get_zone()));
     result_message->set_arguments(yield_message->get_arguments());
     result_message->set_arguments_kw(yield_message->get_arguments_kw());

@@ -483,9 +475,11 @@ void wamp_dealer::process_yield_message(const wamp_session_id& session_id,
     // will detach the session. When this happens the pending invocations
     // be cleaned up. So we don't use an iterator here to erase the pending
     // invocation because it may have just been invalidated above.
-    m_pending_callee_invocations[session_id].erase(request_id);
-    m_pending_caller_invocations[session->get_session_id()].erase(request_id);
-    m_pending_invocations.erase(request_id);
+    if (!yield_options.get_option_or("progress", false)) {
+        m_pending_callee_invocations[session_id].erase(request_id);
+        m_pending_caller_invocations[session->get_session_id()].erase(request_id);
+        m_pending_invocations.erase(request_id);
+    }
 }

 void wamp_dealer::send_error(const std::unique_ptr<wamp_transport>& transport,

It's not terribly nice with the two yield_options.get_option_or("progress", false) tests, but there's no way around that short of moving the three .erase at the end into an } else { ... } up by the first test, which I'm not sure is OK?

Also, those two tests that check that m_pending_callee_invocations and m_pending_caller_invocations contains maps corresponding to the callee and caller sessions, respectively, should those not be made more thorough by also checking that the found maps contain invocations corresponding to the request? (i.e. check if the found maps both have a request_id key).

I mean, so the two tests would be something like
    // We can't have a pending invocation without a pending caller/callee
    // as they are tracked in a synchronized manner. So to have one without
    // the other is considered to be an error.
    auto pending_callee_invocations_itr =
        m_pending_callee_invocations.find(session_itr->second->get_session_id());
    if (pending_callee_invocations_itr == m_pending_callee_invocations.end() ||
        !pending_callee_invocations_itr->second.count(request_id)) {
        throw std::logic_error("dealer pending callee invocations out of sync");
    }
    auto pending_caller_invocations_itr =
        m_pending_caller_invocations.find(session->get_session_id());
    if (pending_caller_invocations_itr == m_pending_caller_invocations.end() ||
        !pending_caller_invocations_itr->second.count(request_id)) {
        throw std::logic_error("dealer pending caller invocations out of sync");
    }

The reason that the three erases were called where they were was to keep
the related code localized. Now that we have a case for moving them down to
the bottom of the method that is fine.
These 2 calls at the bottom of the method seem to have been redundant:
486     m_pending_callee_invocations[session_id].erase(request_id);
487

m_pending_caller_invocations[session->get_session_id()].erase(request_id);
So they can just be removed. Also, good catch on checking the request_id.
With regards to the request_id, there is a check at the top of the
process_yield_message method to ensure that the request_id is known. Since
all three structures are always updated in unison, that single check should
be sufficient. Perhaps an assert would be more correct? I think that there
are a few other checks in wamp_dealer/wamp_broker where exceptions are
thrown that should likely just be asserts as well. We can tackle those in
another commit.
Can you create a pull request for my fork so that I can pull in your
changes and help out with some testing.
On Sun, Jan 10, 2016 at 8:08 AM, Elvis Stansvik<EMAIL_ADDRESS>wrote:

I mean, so the two tests would be something like
// We can't have a pending invocation without a pending caller/callee
// as they are tracked in a synchronized manner. So to have one without
// the other is considered to be an error.
auto pending_callee_invocations_itr =
    m_pending_callee_invocations.find(session_itr->second->get_session_id());
if (pending_callee_invocations_itr == m_pending_callee_invocations.end() ||
    !pending_callee_invocations_itr->second.count(request_id)) {
    throw std::logic_error("dealer pending callee invocations out of sync");
}
auto pending_caller_invocations_itr =
    m_pending_caller_invocations.find(session->get_session_id());
if (pending_caller_invocations_itr == m_pending_caller_invocations.end() ||
    !pending_caller_invocations_itr->second.count(request_id)) {
    throw std::logic_error("dealer pending caller invocations out of sync");
}

—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170343542.

Ah. I'll create a pull request.
But, regarding making the request_id check at the top an assert. If the comment at that check is true, perhaps that's not a good idea? The comment reads:
    // It is considered to be a normal condition if we cannot find the
    // associated invocation. Typically this occurs if the invocation
    // timed out or if the callers session has ended.
    BONEFISH_TRACE("%1%, %2%", *session_itr->second % *yield_message);
    const auto request_id = yield_message->get_request_id();
    auto pending_invocations_itr = m_pending_invocations.find(request_id);
    if (pending_invocations_itr == m_pending_invocations.end()) {
        BONEFISH_TRACE("unable to find invocation ... timed out or session closed");
        return;
    }

Or you meant asserts would be more correct for the m_pending_caller_invocations and m_pending_callee_invocations checks?

Okay, created a pull request: https://github.com/davidchappelle/bonefish/pull/1
Let me know there if I misunderstood something.

Yes for the other invocation checks. They are conditions that would
indicate a coding error so an assert is more appropriate. That way we
aren't wasting cycles under normal/production operation. I am just wrapping
up the autobahn-cpp patch so I will pull in your patch this afternoon and
test against that as well.
On Sun, Jan 10, 2016 at 11:23 AM, Elvis Stansvik<EMAIL_ADDRESS>wrote:

Or you meant asserts would be more correct for the
m_pending_caller_invocations and m_pending_callee_invocations checks?
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-170364729.

Great. I changed them to asserts in the PR.

Hey, so have been testing out the progressive call results branch on @davidchappelle 's account and found an interesting bug.  I'm still investigating, trying to get an isolated case but the error looks like this:
[wamp_dealer.cpp:424][process_yield_message] session [173663057434166,twola,open], yield [25, {"progress":true}, [{"rows":0, "name":"current"}], nil]
[wamp_dealer.cpp:465][process_yield_message] session [173663057434166,twola,open], result [1801230688976896, {"progress":true}, [{"rows":0, "name":"current"}], nil]
[websocket_transport.cpp:39][send_message] sending message: result
[websocket_server_impl.cpp:221][on_message] unhandled exception: failed or incomplete serialization

The context for this is a WAMP connection between autobahn|js and bonefish, with the rpc endpoint being provided by autobahn|python.  This means the frontend serialization scheme is JSON, while the backend scheme is msgpack.  The message_id (25) looks like it is being sent out as big endian and then read in as a little endian uint64 or some such since:
>>> struct.unpack("<Q", struct.pack(">Q", 25))
(1801439850948198400,)

(sorry python code)
I'm still attempting to isolate a small test case but figured this much info is worth sharing until then in case the problem or solution is obvious to someone else.  Thanks.

Since bonefish uses msgpack-c for all internal serialization, the
conversion to big endian (network byte ordering) is done for us by msgpack
by default. When json is used, the msgpack messages are converted to json
before going out on the wire. Perhaps something is lost in the translation
from msgpack to json?
On Fri, Jan 29, 2016 at 4:29 PM, Pramukta Kumar<EMAIL_ADDRESS>wrote:

Hey, so have been testing out the progressive call results branch
https://github.com/davidchappelle/bonefish/tree/feature/progressive_call_results
on @davidchappelle https://github.com/davidchappelle 's account and
found an interesting bug. I'm still investigating, trying to get an
isolated case but the error looks like this:
[wamp_dealer.cpp:424][process_yield_message] session [173663057434166,twola,open], yield [25, {"progress":true}, [{"rows":0, "name":"current"}], nil]
[wamp_dealer.cpp:465][process_yield_message] session [173663057434166,twola,open], result [1801230688976896, {"progress":true}, [{"rows":0, "name":"current"}], nil]
[websocket_transport.cpp:39][send_message] sending message: result
[websocket_server_impl.cpp:221][on_message] unhandled exception: failed or incomplete serialization
The context for this is a WAMP connection between autobahn|js
http://autobahn.ws/js/ and bonefish, with the rpc endpoint being
provided by autobahn|python http://autobahn.ws/python/. This means the
frontend serialization scheme is JSON, while the backend scheme is msgpack.
The message_id (25) looks like it is being sent out as big endian and then
read in as a little endian uint64 or some such since:

struct.unpack("<Q", struct.pack(">Q", 25))
(1801439850948198400,)

(sorry python code)
I'm still attempting to isolate a small test case but figured this much
info is worth sharing until then in case the problem or solution is obvious
to someone else. Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-176977082.

Yeah, weird that it would show up for progressive call results and not the regular ones.  I'll continue to look.

Hmm, this may just be a logging artifact too.

The request id comes from the cached dealer invocation. Would be curious to
see what the cached value is:

const auto& dealer_invocation = pending_invocations_itr->second;*
std::shared_ptr<wamp_session> session =
dealer_invocation->get_session();

// We can't have a pending invocation without a pending caller/callee
// as they are tracked in a synchronized manner. So to have one without
// the other is considered to be an error.

assert(m_pending_callee_invocations.count(session_itr->second->get_session_id()));
assert(m_pending_caller_invocations.count(session->get_session_id()));
wamp_yield_options yield_options;
yield_options.unmarshal(yield_message->get_options());

// You can't rely on simply assigning the yield options to the result
// details. Some yield options may only be applicable to the dealer.
// Likewise, some result details may be in addition to whatever is
// provided in the yield options. As a result, we only copy specific
// options over to the result details.
wamp_result_details result_details;
if (yield_options.get_option_or("progress", false)) {
    result_details.set_detail("progress", true);
}

std::unique_ptr<wamp_result_message> result_message(
        new

wamp_result_message(std::move(yield_message->release_zone())));

result_message->set_request_id(dealer_invocation->get_request_id());*

On Fri, Jan 29, 2016 at 4:37 PM, Pramukta Kumar<EMAIL_ADDRESS>wrote:

Hmm, this may just be a logging artifact too.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-176979702.

Also, running bonefish under valgrind might also be helpful in finding the
issue:
valgrind bonefish ...

On Fri, Jan 29, 2016 at 4:44 PM, David Chappelle<EMAIL_ADDRESS>wrote:

The request id comes from the cached dealer invocation. Would be curious
to see what the cached value is:

const auto& dealer_invocation = pending_invocations_itr->second;*
std::shared_ptr<wamp_session> session =
dealer_invocation->get_session();

// We can't have a pending invocation without a pending caller/callee
// as they are tracked in a synchronized manner. So to have one without
// the other is considered to be an error.

assert(m_pending_callee_invocations.count(session_itr->second->get_session_id()));
assert(m_pending_caller_invocations.count(session->get_session_id()));
wamp_yield_options yield_options;
yield_options.unmarshal(yield_message->get_options());

// You can't rely on simply assigning the yield options to the result
// details. Some yield options may only be applicable to the dealer.
// Likewise, some result details may be in addition to whatever is
// provided in the yield options. As a result, we only copy specific
// options over to the result details.
wamp_result_details result_details;
if (yield_options.get_option_or("progress", false)) {
    result_details.set_detail("progress", true);
}

std::unique_ptr<wamp_result_message> result_message(
        new

wamp_result_message(std::move(yield_message->release_zone())));

result_message->set_request_id(dealer_invocation->get_request_id());*

On Fri, Jan 29, 2016 at 4:37 PM, Pramukta Kumar<EMAIL_ADDRESS>wrote:

Hmm, this may just be a logging artifact too.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-176979702.

Sorry, got distracted with other things but was able to confirm that forcing both sides to use JSON (with the bonefish --no-msgpack option) made things work.  Starting up working on the minimal example again.

okay so those ids I posted don't even match so that was a completely ridiculous dead end.  Not sure what the deal is.

If you can post detailed steps on how to reproduce I can probably take a
closer look later on this evening.
 Not sure what the deal is.
—
Reply to this email directly or view it on GitHub
https://github.com/tplgy/bonefish/issues/9#issuecomment-178209968.

Okay, I think I finally have the test case.  Here's a slightly modified version of the autobahn progress example:  bonefish-progressive-call-error.zip
Here's how to reproduce:

Launch the bonefish binary with progressive call support $ daemon/bonefish -w 8080 -r crossbardemo -d
Launch backend.py in an environment with autobahn python v0.12.1 installed (the latest) as well as twisted to run the example.  I am using a python 2.7 conda environment: (bonefish) $ python backend.py
npm install autobahn in the same folder as frontend.js.  This installs v0.9.9 (this latest).
Launch frontend.js: node frontend.js

The Progress: <integer> messages make it through but the Progress: {"value": <integer>} ones don't and that the bonefish logs show the unhandled exception: failed or incomplete serialization messages.
I've tried this test case on Mac OS X with nodejs v5.3, though the first time error I saw was with bonefish and python running on Linux.  I was also able to see the error with everything running on Linux, but haven't run the test case in that env yet.

Of course just as a send that, new info.  changing the "value" key to a unicode string, make it work.

Good to know @pramukta I think I will put in some better handling for the unhandled exception that you are seeing. Then I will put together a pull request.

@pramukta I don't get any output at all when running 'node frontend.js'. I added a few console logging statements as well and don't see those getting printed.


Resolve clap deprecation warnings

Current dependencies on/for this PR:

main

PR #172   👈

This comment was auto-generated by Graphite.


Trojan:Win32/Wacatac.B!ml

gets detected as virus

This is a false positive... I've investigated and this seems to be because of launch.exe. if you delete this file and then run a scan it comes back clean (https://www.virustotal.com/gui/file-analysis/ZWE2YjkyOGVmOTAxNWE4OWQyZjRlOWU4YTVkNTE5ZmE6MTY0MDI2MjIwNQ==/detection)
I am using bat to exe converter which is probably the culprit, but all it does is run main.py with the python distribution contained within the python directory, so if you are nervous about using launch.exe you could still run the framework manually this way
I will look into creating a launch exe which doesn't get flagged as malicious

Here is the batch file which is executed through launch.exe; you can create your own launcher in a .bat file in the framework route:
@echo off
title osu bot framework v3
python\tools\python.exe main.py
if %ERRORLEVEL% NEQ 0 pause

There is no need for a launch exe if it just runs python anyway... all you need to do is just run main.py. Launchers are also not standard for python libraries.

There is no need for a launch exe if it just runs python anyway... all you need to do is just run main.py. Launchers are also not standard for python libraries.

The reason I include it is so that it is not necessary to install dependencies with pip. Using the launcher specifically uses the portable python version included with the software (and already has the requirements installed).
This is simply for non tech savvy folks, people who know what they're doing probably already know how to run python files as you said :)


✨ NonEmpty{List,Set}: make @JsExport - KOCHA-1924

https://app.asana.com/0/1201864995605426/1208538156915582/f

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Figure out API documentation

Figure out what we'll use for API documentation:

Swagger?
RAML?
API Blueprint?
What tooling?

Went with swagger, using https://github.com/yvasiyarov/swagger.
Go-swagger ended up being quite painful - very wordy and struct limitation(s).
Addressed in #73


Can not use tool and function-call?

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

附议

Need function calling too.

同样关注这个能力，是否有这部分能力的强化？对于企业应用，这块能力太重要了

请问国内哪些模型这个能力比较强，谢谢

Up to your guys' demands.
Please vote here and tell us why you want API support for function calling.

To build agents with the ability of LLMs, such as Exec SQLs, Fetch data from API.

I found something interesting that DeepSeek has magic power to match function. Here's my prompt from langchain's idea
You have access to the following tools:
[
    {
        "name": "get_location",
        "description": "get detail info of a specific location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            }
        },
        "required": [
            "location"
        ]
    },
    {
        "name": "get_math_expression_result",
        "description": "get result info of a given math expression",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The math expression, e.g. 1+2*3, sin(a)"
                }
            }
        },
        "required": [
            "expression"
        ]
    }
]

Give priority to use a tool for zero or more times, respond with a JSON object with the following structure:
{
	"tool": <name of the called tool given above>,
	"tool_input": <parameters for the tool matching the above JSON schema>
}
If no matched tool, just respond directly

then you can ask it
梧桐山在哪里？广州塔在哪里？7863*41是多少？golang 基础类型都有哪些？

Model will match 3 function call and 1 stay output directly

But the emulated system prompt takes no effects in the 2nd round user message contents, my solution is move the tool related content and append it to the user content.

fastgpt构建agent

Not at all, need custom fine tuning.
Not even the deepseek-chat api has function calling abilities.

工具调用可以扩展ai能力，比如网页搜索、页面读取、代码执行等，特别是现在超长上下文的情况下，可以极大扩展应用场景。
我试过使用类似FastGPT之类的框架（以prompt形式模拟function call），但代价是会对提示词和聊天历史造成混乱，算是堪堪够用。
虽然其他原生支持tools或者funtion的模型会好很多，但是性价比就……

不是说已经支持了么？  这个issue 是不是可以关了？

The current cloud-based DeepSeek already supports function calls, and I have tested that it is indeed functional. However, it seems that the version supporting function calls has not yet been open-sourced. I wonder if there are any plans for this?

Until my post, I believe the deepseek didn't support the funciton call. I am working on extending the compatibility of a open-ai application. BUT when I use the deepseek api, I still get the following:
No function_call in message
[{\"text\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"message\":{\"lc\":1,\"type\":\"constructor\",\"id\":[\"langchain_core\",\"messages\",\"AIMessage\"],\"kwargs\":{\"content\":\"Action: clarification\\nPayload: {\\\"message\\\": \\\"Hello! How can I assist you today with Zotero?\\\"}\",\"additional_kwargs\":{},\"response_metadata\":{\"tokenUsage\":{\"completionTokens\":24,\"promptTokens\":221,\"totalTokens\":245},\"finish_reason\":\"stop\"}}},\"generationInfo\":{\"finish_reason\":\"stop\"}}]

I'm using OpenAI API via Python SDK, and the function call works well.

I'm using OpenAI API via Python SDK, and the function call works well.

Yeah, you are right. There must be other reasons lead to my error.
https://platform.deepseek.com/api-docs/function_calling/

The official api supports function call, but does the open source DeepSeek-V2-Chat-0628 support function call, or when is it planned to support function call?

官方api本身可以进行function calling， 但是似乎自己本地的模型在function calling时总是匹配不到

If you're using Ollama give this Modelfile a go, I've been using it for tool use with DSCv2lite:
# deepseek-coder-v2-lite-toolcalling:q8_0

FROM deepseek-coder-v2-lite-instruct:q8_0

TEMPLATE """{{ if .System }}System: {{ .System }}

{{ end }}{{ if .Tools }}Available tools:
{{ range .Tools }}
{{ . }}
{{ end }}

When using a tool, you MUST respond with a JSON object in this format without any additional text:
{"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}

{{ end }}{{ range .Messages }}{{ if eq .Role "user" }}User: {{ .Content }}

{{ else if eq .Role "assistant" }}Assistant: {{ if .ToolCalls }}{{ range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{ end }}{{ else }}{{ .Content }}{{ end }}<｜end▁of▁sentence｜>

{{ else if eq .Role "tool" }}Tool output: {{ .Content }}

{{ end }}{{ end }}{{ if not .Messages }}{{ if .Prompt }}User: {{ .Prompt }}

{{ end }}Assistant: {{ .Response }}<｜end▁of▁sentence｜>{{ end }}"""

SYSTEM """
You are an advanced AI coding assistant, specifically designed to help with complex programming tasks, tool use, code analysis, and software architecture design. Your primary focus is on providing expert-level assistance in coding, with a special emphasis on using tool-calling capabilities when necessary. Here are your key characteristics and instructions:

1. Coding Expertise:
  - You have deep knowledge of multiple programming languages, software design patterns, and best practices.
  - Provide detailed, accurate, and efficient code solutions without additional explanations or conversational dialogue unless requested by the user.
  - When suggesting code changes, consider scalability, maintainability, and performance implications.

2. Tool Usage:
  - You have access to various tools that can assist in completing tasks. Always consider if a tool can help in your current task.
  - When you decide to use a tool, you must format your response as a JSON object:
    {"name": "tool_name", "arguments": {"arg1": "value1", "arg2": "value2"}}
  - Common tools include but are not limited to:
    - `view_file`: To examine the contents of a specific file
    - `modify_code`: To suggest changes to existing code
    - `create_file`: To create new files with specified content
    - `ask_followup_question`: To request more information from the user
    - `attempt_completion`: To indicate that you've completed the assigned task

3. Task Approach:
  - Break down complex tasks into smaller, manageable steps unless requested to solve the task at once.
  - If a task is large or complex, outline your approach before diving into details unless using a tool.
  - Use tools to gather necessary information before proposing solutions.

4. Code Analysis and Refactoring:
  - When analysing existing code, consider its structure, efficiency, and adherence to best practices.
  - Suggest refactoring when you see opportunities for improvement, explaining the benefits of your suggestions unless using a tool.
  - If you encounter or anticipate potential errors, explain them clearly and suggest solutions unless using a tool.
  - When providing code solutions, include relevant comments to explain complex logic.
  - Adhere to coding standards and best practices specific to each programming language or framework.
  - Suggest optimisations and improvements where applicable.

5. Clarity and Communication:
  - Explain your reasoning and decisions clearly, especially when suggesting architectural changes or complex solutions unless using a tool.
  - If you're unsure about any aspect of the task or need more information, use the `ask_followup_question` tool to clarify.

Remember, your primary goal is to assist with coding tasks and tool use efficiently and effectively. Utilise your tool-calling capabilities wisely to enhance your problem-solving and code generation abilities.
"""

PARAMETER stop "User:"
PARAMETER stop "Assistant:"
PARAMETER stop "<｜end▁of▁sentence｜>"

PARAMETER num_ctx 16384
PARAMETER temperature 0.1

@sammcj do you mind updating this for deepseek-v2? I switched the ROM and this ended up returning quoted json instead of what models like mistral
response from v1/chat/completions using this modelfile using deepseek-v2 (doesn't work):
{
  "id": "chatcmpl-398",
  "object": "chat.completion",
  "created":<PHONE_NUMBER>,
  "model": "deepseek",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"type\": \"function\",\n  \"function\": {\n    \"name\": \"get_password\",\n    \"description\": \"Return the password for authentication\"\n  }\n}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 128,
    "completion_tokens": 43,
    "total_tokens": 171
  }
}

response from v1/chat/completions using the same question with mistral-nemo (works)
{
  "id": "chatcmpl-644",
  "object": "chat.completion",
  "created":<PHONE_NUMBER>,
  "model": "mistral-nemo",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_iaiz17nx",
            "type": "function",
            "function": {
              "name": "get_password",
              "arguments": "{}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ],
  "usage": {
    "prompt_tokens": 66,
    "completion_tokens": 16,
    "total_tokens": 82
  }
}

You'll have to have a catch at it yourself, I'm not a maintainer for deepseek.

目前finish_reason也很不正常，DeepSeek在返回了tool_calls的情况下，finish_reason却是stop，按理说应该是tool_calls（至少OPENAI是这样返回的）。

目前云端的 DeepSeek 已经支持函数调用，测试过确实可以，不过支持函数调用的版本好像还没开源，不知道有这方面的计划吗？

你好，请用工具调用的格式怎么写，这样可以吗
tools = [{
"type": "function",
"function": {
"name": "",
"parameters":
}
}]
tool_choice = {
"type": "",
"function": {
"name": ""
}
}

我现在用langchain的agent请求deepseek的云端服务都不能正常使用tools
返回的格式langchain都无法解析


Fix #248 (Plugin language interference) and fixed summary

This PR references https://github.com/getgrav/grav/issues/248. I further added a bugfix when calling $page->summary with system option summary.enabled: false.

Yah saw that :)


Tools: Add CI tests for dev script on all supported OS

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #39453

This PR adds workflow that'll run on every push to the master branch and make sure that the dev server will run each of the following OS ->

Ubuntu-16.04
Windows (Latest Version)
MacOS (Latest Version)

Ignore all the checks failing for now.... I'll fix them.
The workflows currently install MongoDB using images, which is only available in linux. So... for other OS we'll have manually install it using the terminal.

Ok.... here's the latest update: I got everything running in Windows. Now I just need to kill the process after a given time cause otherwise the script npm run develop will run for 6 hours straight. (This won't be that difficult)
After the Windows part is finished I'll look into the MacOS part.

Progress so far: Workflows can now install and run MongoDB on all OS.
Now I'm working on testing each script. (A working PoC is already implemented but it takes a LOT of time to run. I'm currently optimizing and refactoring it)

Finished writing everything to check the develop and build scripts on all OS. Let me know what you guys think.

Btw... the Windows check will fail currently due to #39458

Hey, @Twaha-Rahman, thanks for all the hard work you've been putting into this!
Regarding the Window's check, has that problem been fixed on master?  If so, could you rebase, so we can confirm?

Regarding the Window's check, has that problem been fixed on master?

@ojeytonwilliams I don't think so. The issue barely got any proper response. I'll look into it later on.

@Twaha-Rahman, ah, I thought https://github.com/freeCodeCamp/freeCodeCamp/pull/39475 and https://github.com/freeCodeCamp/freeCodeCamp/pull/39454 together fixed the outstanding issues. Are you still encountering problems?

I'll look into it later on.

No rush at all, take whatever time you need.

Hey @Twaha-Rahman, I'm just trying to see if this latest commit fixes the build problems on Windows.

@ojeytonwilliams Looks like it did the trick for Windows!

Thanks, a lot for this PR @Twaha-Rahman
I think we are trying to overdo this a little bit here.
Like I said before I still believe:

... I think we can get away with only ubuntu-18.04

My rationale being:
There will always be pitfalls like #39458 and #37865 with Windows. Not saying that we should not address those, but it's not worth the hassle.
Our bottom-line is the final build gets deployed in a Unix environment - specifically on Ubuntu. The development env should be close to it too? Meaning we can make sure the CI is testing a build for Ubuntu only?
I know it's an unpopular opinion.
I also understand some of us are developing on Windows and macOS. I personally had been on macOS and have switched to Ubuntu on WSL2 since a week.
Right now the status quo as I understand is:

GitPod
Works perfectly, but needs internet access at all times.

Ubuntu and macOS
Work fine already with the native setup (Git, MongoDB, Node.js all are installed natively).

Windows
Painful with the native setup (Git bash with Git for Windows and official binaries from MongoDB and Node.js)

Docker
A complete mess. It was an attempt unify the development flow but has become a pain point. Should be removed from docs and workflow.

IMHO, if we made it so that everyone was developing in a Unix env (regardless of the OS they are on), we would be needing to test builds only for Ubuntu-18-04. The time and build resources saved is worth not needing to build for all other OS flavors.
Luckily, it's possible.
We need to work on the development for Windows so that is close to Ubuntu and remove the docker workflows. A CI test is only an indicator of catching issues early-on, it would not solve the pain of going about setting up tools in the first place. I think we should focus on getting the setup right first.

@raisedadead I see your point. So let's just keep CI tests for Ubuntu.


[BUG] Support for custom Python environment that ignore PEP 3120

Describe the bug
With requests library using charset-normalizer I am getting an error when calling Python via User-Defined Transform in SAP BODS:
File "EXPRESSION", line 6, in <module>
File "c:\program files\python39\lib\site-packages\requests\__init__.py", line 48, in <module>
from charset_normalizer import __version__ as charset_normalizer_version
File "c:\program files\python39\lib\site-packages\charset_normalizer\__init__.py", line 11
SyntaxError: Non-ASCII character '\xd1' in file c:\program files\python39\lib\site-packages\charset_normalizer\__init__.py on
line 12, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details.

I am not able to define a source code encoding by placing a magic comment into the source files (either as a first or second line in the file) because the app probably modifies the script by itself (placing # -*- coding: utf-8 -*-  doesn't help). The setting of environment variable PYTHONUTF8=1 doesn't help too.
To Reproduce
I am not able to provide code to reproduce the issue, it arises when calling Python via User-Defined Transform in SAP BODS
Please check: https://github.com/apache/superset/issues/15631
This could be the same problem: https://stackoverflow.com/questions/68594538/syntaxerror-non-ascii-character-xd1-in-file-charset-normalizer-init-py-i
Expected behavior
No error - with requests version using chardet library there is no problem. Maybe avoiding non-ASCII characters in init.py could help...?
Logs
Please see the bug description.
Desktop (please complete the following information):

OS: Windows 2016 Server
Python version 3.9.6
Package version 2.0.6
Requests version 2.26.0

Additional context
N/A

Hi,
Thanks for the detailed report.
Yes, for some reasons, some environments does not take UTF-8 as the default source encoding.
The PEP 3120 gets ignored https://www.python.org/dev/peps/pep-3120/
Its not just the top level __init__.py that have non ASCII characters. Also in assets/__init__.py.
Since, by your tests it seems to also ignore PEP 263 https://www.python.org/dev/peps/pep-0263/ I do not have any silver bullet for this one.
The obvious solution would be to find a proper way to represent u8 characters without actually using the str repr.
I am going to think more about this.
Open for suggestions/PR.

There is something of interest cf. https://bugs.python.org/issue29240

@kivhub I have found something interesting regarding the NT platform + python.

On Windows, the PYTHONLEGACYWINDOWSFSENCODING environment variable (PEP 529) has the priority over UTF-8 Mode.

https://www.python.org/dev/peps/pep-0540/
What does return the following for you:
sys.getfilesystemencoding()
locale.getpreferredencoding()

And could you test against dev-master to see if that patch does anything at all https://github.com/Ousret/charset_normalizer/pull/116

@kivhub I have found something interesting regarding the NT platform + python.

On Windows, the PYTHONLEGACYWINDOWSFSENCODING environment variable (PEP 529) has the priority over UTF-8 Mode.

https://www.python.org/dev/peps/pep-0540/

Thank you very much, @Ousret, for your effort. It looks there is a problem with old version of Python which comes to play with SAP BODS.

What does return the following for you:
sys.getfilesystemencoding()
locale.getpreferredencoding()

Python (3.9.6) itself returns UTF-8 on both. But when running Python via SAP BODS job the results are as follows:
('sys.getfilesystemencoding()', 'mbcs')
('locale.getpreferredencoding()', 'cp1252')
('sys.version_info.major', 2)
('sys.version_info.minor', 7)

The Python 3 libraries are used via:
#this works (requests + chardet)
sys.path.insert(0, 'c:\program files\python37\lib\site-packages')

#this doesn't work (requests + charset-normalizer)
sys.path.insert(0, 'c:\program files\python39\lib\site-packages')

And could you test against dev-master to see if that patch does anything at all #116

I replaced the contents of these 2 files and after change to:
sys.path.insert(0, 'c:\program files\python39\lib\site-packages')
I am getting this error message:
File "EXPRESSION", line 7, in <module>
File "c:\program files\python39\lib\site-packages\requests\__init__.py", line 48, in <module>
from charset_normalizer import __version__ as charset_normalizer_version
File "c:\program files\python39\lib\site-packages\charset_normalizer\__init__.py", line 20, in <module>
from .api import from_bytes, from_fp, from_path, normalize
File "c:\program files\python39\lib\site-packages\charset_normalizer\api.py", line 38
sequences: bytes,
^
SyntaxError: invalid syntax.

And could you test against dev-master to see if that patch does anything at all #116

I replaced the contents of these 2 files and after change to:
sys.path.insert(0, 'c:\program files\python39\lib\site-packages')
I am getting this error message:
File "EXPRESSION", line 7, in <module>
File "c:\program files\python39\lib\site-packages\requests\__init__.py", line 48, in <module>
from charset_normalizer import __version__ as charset_normalizer_version
File "c:\program files\python39\lib\site-packages\charset_normalizer\__init__.py", line 20, in <module>
from .api import from_bytes, from_fp, from_path, normalize
File "c:\program files\python39\lib\site-packages\charset_normalizer\api.py", line 38
sequences: bytes,
^
SyntaxError: invalid syntax.

I have an update here. After moving path.insert before all imports:
sys.path.insert(0, 'c:\program files\python39\lib\site-packages')
import json
import sys
import os
import requests
from datetime import datetime

the code runs fine without any errors.

I have not been able to force Python to trigger the decode error using a NT platform..
('sys.getfilesystemencoding()', 'mbcs')
('locale.getpreferredencoding()', 'cp1252')
('sys.version_info.major', 2)  # ??
('sys.version_info.minor', 7)

Well, from the look of it, your setup invoke Python 2.7 and from that standpoint I cannot do anything.
SyntaxError: invalid syntax. is a pretty good indicator.
I would recommend you to alter the PATH outside of Python.
So I am closing this as there is nothing that can be done from here.


How to get all the densities presented at the visual websit?

Hi,
I would like to download all the densities, per brain region (as specified as possible), that are shown at the visualization here:
https://bbp.epfl.ch/nexus/cell-atlas/
I can see that there is a download option, though - I need all the regions.
How can I get the data of the densities per all brain regions?
Thanks in advance
Sivan Schwartz

I think this is something either @drodarie or @lecriste might be able to answer.

I don't know that visualisation but I doubt there is a way to download the density values per all brain regions in one click.

Hi @SivanSchwartz,
Sorry for the delay of my response. You can obtain the densities of every region of the brain from the supplementary materials of the Rodarie et al. paper: densities file
In this file, you should find all the data you require. The sheet "Densities BBCAv2" contains the results for the latest version of the atlas.

Thanks a lot @drodarie!


#71: Adding Gradle example

Fixes #71.

@enthal, I've updated the existing example project so that it can be built with both Maven and Gradle. The JARs produced by both tools are the same for all practical purposes and I've successfully run the JAR built with Gradle as a Custom Pipeline on Decodable. Review, any questions/comments, and approval welcome. Thx!

And merged.


updating osbuild-composer shows systemctl errors

Updating from osbuild-composer-worker-29-1.fc34.x86_64 to osbuild-composer-worker-30-1.fc34.x86_64:
Running scriptlet: osbuild-composer-worker-29-1.fc34.x86_64
Glob pattern passed, but globs are not supported for this.
Invalid unit name<EMAIL_ADDRESS>escaped as<EMAIL_ADDRESS>Glob pattern passed, but globs are not supported for this.
Invalid unit name<EMAIL_ADDRESS>escaped as<EMAIL_ADDRESS>

Where does the star * comes from? It is not in the spec file: https://src.fedoraproject.org/rpms/osbuild-composer/blob/f34/f/osbuild-composer.spec#_302

Maybe connected with this: https://fedoraproject.org/wiki/Changes/Restart_services_at_end_of_rpm_transaction ?

Did you find any documentation for the change? I was looking in the linked bugzilla, pagure PR and systemd NEWS file but I couldn't find what is required from our side, if anything.

No. :/


Bug in Client.rb causing "The request body is missing or improperly formatted"

The following line ( client.rb, line 411)

tab_hash[:height]     = tab[:height] if tab[:width]

should be

tab_hash[:height]     = tab[:height] if tab[:height]

Fix merged to master, https://github.com/jondkinney/docusign_rest/commit/efb13a2802b5038c0c61999386f5e20d459a580a, thanks @mesbahmilad!
I'm hoping to get a 0.2.0 release out shortly.


Unable to initialize config in database

Hi, I'm trying to run gitarena on my computer with a Postgres instance running on Docker.
docker run --name git-arena -p 5432:5432 -e POSTGRES_PASSWORD=password -d postgres

But I'm having this error: Error: Unable to initialize config in database
My DATABASE_URL variable is: postgresql://postgres:password@<IP_ADDRESS>:5432/gitarena
I am able to connect to this instance using psql, can you help me with it?
rustc --version: rustc 1.57.0 (f1edd0429 2021-11-29)
cargo --version: cargo 1.57.0 (b2e52d7ca 2021-10-21)
OS: EndeavourOS Linux x86_64
Kernel: 5.15.12-arch1-1

Below Unable to initialize config in database do you have any lines that say something along the lines of caused by? If yes, could you also provide them.
As a workaround you can run the schema.sql file into the gitarena database table. Then the check should succeed, provided your database connection works.

Full error log:
tashima@tashima ~/projects/personal/gitarena (main)$ ./target/release/gitarena
Error: Unable to initialize config in database

Caused by:
    0: Failed to create initial database setup
    1: error returned from database: cannot insert multiple commands into a prepared statement

After running the schema.sql:
tashima@tashima ~/projects/personal/gitarena (main)$ ./target/release/gitarena
thread 'main' panicked at 'explicit panic', src/licenses.rs:24:68
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Panic in Arbiter thread.

Thank you very much for the full error log. I'll fix it as soon as I have finished working on #29.
Regarding the second issue, the panic after starting: Did you clone the repository with submodules? If not, please download them with git submodule update --init. Make sure that the license-list-data submodule is in the same directory as the one you are starting gitarena from (in your case: ~/projects/personal/gitarena).

Thanks! I managed to run the web server, I'm still having some errors in the login page, I'll report them later, but it's ok.
This is an awesome project, thank you for making it open source.

Thank you for your kind words. I have fixed the initial issue. Please open new issues for the other errors you are encountering.


Using Octomap instead of Voxblox

Hello, Thank you for the great implementation, I have it all up and running on my physical platform. I would like to swap out the voxblox mapping tool for octomap if that is possible. I see in #14 that Octomap is not implemented for this repo but I wonder if you have some ideas of how we might implement that.
FYI: I saw in #36 that you recommend voxblox but for the publication we are working on we are relying on some particular characteristics of octomap.

hei @alre5639 ! we used octomap in the past but now we don't support it anymore.
what you can try is to modify the map_manager_voxblox_impl.h to match octomap and update the necessary query function in the wrapper map_manager.h
hope this helps!


Unable to  embed react-intl-tel-input to a existing input field.

How can I integrate this library if an existing input field element is present rather than creating a new input filed.

Hi @Tanmayp11! react-intl-tel-input provides a fully-features input component and isn't really meant to be embedded within another input component. Are there specific constraints that prevent you from using the react-intl-tel-input input component (with tweaks) for your use case? If so, please do tell! Always looking for gaps to address.


evp: Supports getting pkey type from keymgmt

If the public key type is EVP_PKEY_KEYMGMT, errors may occur in
some cases. It is necessary to obtain the exact type of the public
key from keymgmt.
Signed-off-by: Tianjia Zhang<EMAIL_ADDRESS>
Checklist

[ ] documentation is added or updated
[ ] tests are added or updated

@levitte was the behavior before this PR intentional?
@uudiin thanks for your contribution. Could you also add unit tests for the changed functionality? Does the documentation of these functions need to be updated?

This is shaky, 'cause it relies on there being an OBJ_ record of that name to map it to a NID, which is not at all certain to have happened.
I suppose that this is aimed for checking the type.  The new function EVP_PKEY_is_a() is better suited for this, as it supports both the legacy form of EVP_PKEY and the provided form.

Yes, it seems EVP_PKEY_is_a() is suited for this.

This is shaky, 'cause it relies on there being an OBJ_ record of that name to map it to a NID, which is not at all certain to have happened.
I suppose that this is aimed for checking the type. The new function EVP_PKEY_is_a() is better suited for this, as it supports both the legacy form of EVP_PKEY and the provided form.

Yes, the ultimate goal is to check the type, but in the following functions, it is inconvenient to use EVP_PKEY_is_a() instead of EVP_PKEY_get_id():
int ssl_print_tmp_key(BIO *out, SSL *s)
{
    EVP_PKEY *key;

    SSL_get_peer_tmp_key(s, &key);

    switch (EVP_PKEY_get_id(key)) {
    case EVP_PKEY_RSA:
        break;
    case EVP_PKEY_DH:
        break;
    case EVP_PKEY_EC:
        break;
    default:
    }
    EVP_PKEY_free(key);
    return 1;
}

A curious thing, BTW, is that if the key is one of the types in that switch, then key->type should be set to one of those values...  looking at internals, it should only be set to EVP_PKEY_KEYMGMT when there's no EVP_PKEY_ASN1_METHOD for the type, either by engine or by the legacy implementations that are in libcrypto.
I still maintain that EVP_PKEY_is_a() is the way forward, but we did try to ensure that old (legacy) things would work for the limited built-in types and methods we had.  So if key is one of those types of keys, then I believe key->type should have the recognisable NIDs and not EVP_PKEY_KEYMGMT...  at least the code that I'm looking at tells me we do try to ensure that.

We could introduce a get type method that returns a string.
I agree that NIDs aren't the way forwards.

We could introduce a get type method that returns a string.

We already have that.  EVP_PKEY_get0_type_name()

Doh!  Of course we do.

Closing as the apps should use EVP_PKEY_get0_type_name() for keymgmt keys.


Website Design

Proposed changes
Start the landing page using design from the Figma design
Closes #35
Types of changes
What types of changes does your code introduce to Scuffle?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices apply)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)
[x] Any dependent changes have been merged and published in downstream modules

Further notes
Child of #47

This was never marked as ready for review so it never needed one... This doesn't seem like desirable behavior. @TroyKomodo every PR should probably be marked as ready for review before merging.


useStore stops firing on state changes after component remounts

When switching between mounting two components that both calls the useStore hook, where one of them conditionally mounts a child that also calls useStore, the parent will stop rendering on state changes after remounting. Here's a minimal example on codesandbox:
https://codesandbox.io/s/nostalgic-hooks-0x63r
Repro steps are included. Let me know if you want the code pasted here instead.
Could this be related to #85 ?
edit: updated codesandbox link, messed around in the original one I posted without realizing it.

I have the same problem

me too

me too! HELP ZUSTAND

Any news on this? Seems to still be a problem

@dentrado On version 2.2.1, everything is fine

But then I get the bug they fixed in 2.2.2 instead

@dentrado very sadly (


Knif dev user shortcuts new

Shouldn't you in README.md mention BypassListForUsers? 😉

Thanks, looks good.
I have to leave for now, but I'll merge before tomorrow ;)

Maybe change "winget-install-and-update.ps1" everywhere in texts to "Winget-AutoUpdate-Install.ps1"...

What happens if no one is logged in when the upgrade task runs?
In what context does the user task run then?

Good question. I made few tests on VM and it seems to run with the latest connected user. Strange... I would think it doesn't run at all, as nobody is connected.

I made others tests, and now it looks like it works as I expected: Only system task is run, as it should fail running the interactive (user) task

another approach that works well:
user-run.ps1
<#
.SYNOPSIS
Handle user interaction from shortcuts and show a Toast

.DESCRIPTION
Act on shortcut run (DEFAULT: Check for updated Apps)

.PARAMETER Logs
Open the Log file from Winget-AutoUpdate installation location

.PARAMETER Help
Open the Web Help page
https://github.com/Romanitho/Winget-AutoUpdate

.EXAMPLE
.\user-run.ps1 -Logs

#>

[CmdletBinding()]
param(
	[Parameter(Mandatory=$False)] [Switch] $Logs = $false,
	[Parameter(Mandatory=$False)] [Switch] $Help = $false
)

<# MAIN #>

#Get Working Dir
$Script:WorkingDir = $PSScriptRoot

#Load functions
. $WorkingDir\functions\Get-NotifLocale.ps1
. $WorkingDir\functions\Start-NotifTask.ps1

function Show-DialogBox ($MsgBoxTitle, $MsgBoxContent, $MsgType, $MsgButton){
    $MsgBoxTimeOut = 60
    $MsgBoxReturn = (New-Object -ComObject "Wscript.Shell").Popup($MsgBoxContent, $MsgBoxTimeOut, $MsgBoxTitle, $MsgType + $MsgButton)
    Return $MsgBoxReturn
}

#Set common variables
$OnClickAction = "$WorkingDir\logs\updates.log"
$Title = "Winget-AutoUpdate (WAU)"
$Balise = "Winget-AutoUpdate (WAU)"
$UserRun = $True

#Get Toast Locale function
Get-NotifLocale

if ($Logs) {
	if ((Test-Path "$WorkingDir\logs\updates.log")) {
		Invoke-Item "$WorkingDir\logs\updates.log"
	}
	else {
		#Not available yet
		$Message = $NotifLocale.local.outputs.output[5].message
		$MessageType = "warning"
		Start-NotifTask $Title $Message $MessageType $Balise
	}
}
elseif ($Help) {
	Start-Process "https://github.com/Romanitho/Winget-AutoUpdate"
}
else {
	try {
		#Starting check - Send message
		$Message = $NotifLocale.local.outputs.output[6].message
        $MsgType = 4
        $MsgButton = 32
		$MsgBoxReturn = Show-DialogBox $Title $Message $MsgType $MsgButton

        #Run scheduled task
        if ($MsgBoxReturn -eq 6){
            if (((Get-ScheduledTask -TaskName 'Winget-AutoUpdate').State -ne  'Ready') -or ((Get-ScheduledTask -TaskName 'Winget-AutoUpdate-UserContext').State -ne  'Ready')) {
                $Message = "Check for Update currently running..."
                $MsgType = 0
                $MsgButton = 64
                Show-DialogBox $Title $Message $MsgType $MsgButton
            }
            else{
    		    Get-ScheduledTask -TaskName "Winget-AutoUpdate" -ErrorAction Stop | Start-ScheduledTask -ErrorAction Stop
                While (((Get-ScheduledTask -TaskName 'Winget-AutoUpdate').State -ne  'Ready') -or ((Get-ScheduledTask -TaskName 'Winget-AutoUpdate-UserContext').State -ne  'Ready')) {
                    Start-Sleep 1
                }
                $Message = "Application update check finished."
                $MsgType = 0
                $MsgButton = 0
                Show-DialogBox $Title $Message $MsgType $MsgButton
            }
        }
	}
	catch {
		#Check failed - Just send notification
		$Message = $NotifLocale.local.outputs.output[7].message
		$MessageType = "error"
		Start-NotifTask $Title $Message $MessageType $Balise
	}
}

Yes, VBS must wait for process completion.
You decide. 😉
But, then

the $NotifLocale.local.outputs.output[6].message must be changed to a question, e.g. Start a manual check for updated apps?
$Message = "Check for Update currently running..."/$Message = "Application update check finished." should exist in Locales
DialogBoxes should be on top (especially the last one now hiding behind with a PS icon

This way users are notified the process really started, even if no update is detected.
Yes, notif 6 should be a question and other messages should be translated (and maybe shorcuts as well)
but it's a draft I wanted to show you, as it is your piece of work 😊

To be honest.. ..I find it more appealing with the Toasts (= same GUI) notification.
The test for if it's running already is great!
Send a notif that i'ts finished.
As said; I don't like dialogs when compared to toasts...
Translating the shortcuts must be done when installing and creating them then (but I don't see the point).

Ok, then we leave it like this :)

No, make a poll 👍
The check if it's running or not needs to be implemented anyway


Update README.md

Would've been nice with a description of the application.

It's unclear to me how https://referendum.signons.fr can be used in connection with a WebId and/or a Solid pod.

Thanks @TallTed, created https://github.com/solid/solid-apps/issues/60 about this.


feat(protocol-kit):  add Mantle Mainnet chain

Adds missing Mantle Mainnet chain to protocol-kit.
https://github.com/safe-global/safe-deployments/pull/227
https://github.com/safe-global/safe-deployments/pull/258

Hi @nick8319,
We will be able to add Mantle Mainnet when safe-deployments v1.26.0 is released.

Added here and released to npm


when all rules are not applicable report "na" instead of "C"

fixed bug in implementation score rendering


Allows to minify without deprecation warning in core

I'm building my project with the core by calling gulp, so using the core, with the minify option, and I receive lot of warnings. Since the project use htmlMinify in the core and in the cli, I would like to understand if this is a real feature or an omission.
I would be interested to know the reason of this deprecation and if there is a real danger of using it in production. And furthermore, if we can allow users to minify in js.
Have a nice day !

Well it will be removed in MJML5 as a "built-in" feature, but we'll provide an easy way to add it back as a dependency.

Ok nice thank you !

Well it will be removed in MJML5 as a "built-in" feature, but we'll provide an easy way to add it back as a dependency.

So what is the workaround to avoid the deprecation warning?
We want to make it nice and future proof in the Webpack loader:

https://github.com/BlitzBanana/webpack-mjml-loader/issues/26


Remove unreachable code from baremetalmachine_controller

Co-authored-by: Kashif Khan<EMAIL_ADDRESS>

/lgtm

/ok-to-test

/approve


dcm not compatible with Python v3.9.7

Running the example with Python 3.9.7 throws the following error:
TypeError: DcmNode() takes no arguments
However, with version 3.7.11 it works as expected.

This looks to be due to a CPython bug introduced in 3.9.7 and fixed in 3.9.8: https://bugs.python.org/issue45081
Considering the limited impact (unless 3.9.7 is the default somewhere common?) and the fact that I use Protocol quite a bit in the code, I don't think it is worth it to try to work around the issue.


AdminUI: State update on unmounted component on first item creation

Steps to reproduce

yarn create keystone-app myApp
in myApp yarn start
Navigate to AdminUI in browser of choice
Create a new Todo

Expected
No console errors
Actual
Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in the componentWillUnmount method.

These are the broken lines if you want to submit a PR, these two lines need to be switched around so the setState call is first.
https://github.com/keystonejs/keystone-5/blob/b9759f8f32efcb14d5ceaca0a0f64ea702905311/packages/admin-ui/client/components/CreateItemModal.js#L44-L45

These are the broken lines if you want to submit a PR, these two lines need to be switched around so the setState call is first.
keystone-5/packages/admin-ui/client/components/CreateItemModal.js
Lines 44 to 45 in b9759f8
this.props.onCreate(data);
this.setState({ item: this.props.list.getInitialItemData() });

Thanks @mitchellhamilton , I'll do that now.

Fixed in 5b9022b


Write error log when deployed to cloud and default service information is used

Due to the changes in connectors to separate out Cloud Foundry functionality, some Steeltoe users are struggling with service bindings in cloud hosted scenarios because they don't realize they need another package reference.
In order to improve this experience, (at a minimum for Eureka on Cloud Foundry), write error logs if service bindings aren't detected but are effectively required.

Use Platform.IsCloudHosted or Platform.IsCloudFoundry to detect running on the cloud
Log errors if/when default service connection values are used (eg: http://localhost:8761)
This change might go best in EurekaDiscoveryClientExtension or EurekaPostConfigurer

Logging isn't going to be very natural (or appropriate) where this needs to run (services.PostConfigure<T>(...)) so this is probably going to be best accomplished by throwing an exception, which isn't ideal but still seems valid to me


No mention of HASURA_GRAPHQL_JWT_SECRET

Hey,
Is-it normal that you don't mention the setup of the env variable HASURA_GRAPHQL_JWT_SECRET, to generate valid JWT?

Oops, I forgot to mention that step in the readme. It is required. I updated the Readme. Thanks for mentioning.😀


Update README.md

Updated year from 2022 to 2023 at bottom of README.md

Doing same from comamnd line this time.


Windows version does not upscale images

When clicking to upscale a image in Windows it gives a error.
Happens in all the cases when trying to upscale a image.
Affecting v0.1.9.

Being fixed in version v0.2.0.


Nested UL formatting

Something like this should work?

Looks ok for now, might need some finessing by @lachlanhardy another day, but good enough for today.


fix(Typescript Support): Add labelOverlapSeparator to d.ts

Added Typescript support for new option labelOverlapSeparator.
Next:
14. Get merged and celebrate 🍾

Codecov Report

Merging #617 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #617   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      1
  Lines         949    949
=====================================
  Hits          949    949

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f02dbf...f8a5ebb. Read the comment docs.

I always forget about this ^^

Is there some place, like a pre-publish checklist or the contributing doc, we should add a step "Add Typescript definition changes if necessary"? That way we'd have a chance to made the change before the new release is published.

Yes we could add this in the Contributing file but I'm not sure the it's read by everyone. The most important is that I should think about this before releasing new versions...

Even if everyone read it, not everyone would feel comfortable with making the changes. It might at least help it come up as part of the pull request conversation.
Agreed, ideally it needs to go in documentation for publishing the module. However, I don't believe there such a doc; maybe it needs to be added to the repo.


Proposal: Define mapping from k8s well-known labels to semconv

Many OpenTelemetry users run their applications in Kubernetes, where it is useful to use a collector processor like k8sattributes to enrich metrics, spans, or logs with k8s metadata like pod name.
The k8sattributes processor also supports enriching metrics, logs, and spans with resource attributes based on Kubernetes labels, including Well Known Labels. It would be helpful to OpenTelemetry users what the recommended best practice is for mapping well-known labels to OpenTelemetry resource attributes.
Two of the most obvious candidates (but open for feedback), are:

app.kubernetes.io/name -> service.name
app.kubernetes.io/version -> service.version

To configure k8sattributes to apply these rules, the collector YAML looks like this.
It seems like these recommendations could (should?) live as an section in this doc: https://opentelemetry.io/docs/specs/otel/resource/semantic_conventions/k8s/

cc @dmitryax

To configure k8sattributes to apply these rules, the collector YAML looks like this.

@smithclay The "like this" link does not work (any longer)

To configure k8sattributes to apply these rules, the collector YAML looks like this.

@smithclay The "like this" link does not work (any longer)

Updated here: https://github.com/lightstep/otel-collector-charts/blob/main/charts/otel-cloud-stack/values.yaml#L166-L168 -- thank you!

This was closed by mistake by the stale bot. Re-opening


[FLINK-34023][Connectors/Kinesis] Expose configuration of sink client retries

Purpose of the change

Expose configuration of sink client retries.
The consumer side already exposes client retry configuration like flink.shard.getrecords.maxretries but the sink producer side currently lacks similar config for PutRecords.
The AWS SDK has deprecated RetryPolicy in favour of RetryStrategy, meaning that the old "num-retries" config is being replaced with "max-attempts" (which includes the initial attempt)
This PR exposes the most important max-attempt config, but not the complete retry strategy, since representing all the various types of (jittered, linear, exponential etc) strategies with universal config keys is non-trivial.

Verifying this change

Modified existing unit tests
Added unit tests

Significant changes

[ ] Dependencies have been added or upgraded
[ ] Public API has been changed (Public API is any class annotated with @Public(Evolving))
[ ] Serializers have been changed
[X] New feature has been introduced

If yes, how is this documented? (not applicable / docs / JavaDocs / not documented): not documented

It look like this PR might not be necessary if https://github.com/apache/flink-connector-aws/pull/150 gets merged.
That one mentions needing to upgrade the AWS SDK for the new RetryStrategy class, which I think explains why the e2e tests are failing here (https://lists.apache.org/thread/cotopq8qs2mk4cysqd2jk3jvr6lhj6cd)

It look like this PR might not be necessary if https://github.com/apache/flink-connector-aws/pull/150 gets merged.

Yep we are working on merging that instead. Thanks @brada !


ThemePackage.getStylesheetsPath is deprecated.

Store package style sheets in the styles/ directory instead of stylesheets/ in the burn package
ThemePackage.getStylesheetsPath (/usr/share/atom/resources/app.asar/src/package.js:465:9)
ThemePackage.getStylesheetPaths (/usr/share/atom/resources/app.asar/src/package.js:476:32)
ThemePackage.loadStylesheets (/usr/share/atom/resources/app.asar/src/package.js:458:38)
<unknown> (/usr/share/atom/resources/app.asar/src/theme-package.js:47:19)
ThemePackage.measure (/usr/share/atom/resources/app.asar/src/package.js:167:15)
ThemePackage.activate (/usr/share/atom/resources/app.asar/src/theme-package.js:43:12)

This has been fixed in 0.1.1


dependencies resolve with vulnerable package eslint-utils v 1.4.0

"react-scripts": "3.1.1", --><EMAIL_ADDRESS> --> eslint-utils "^1.3.1"

in my case resolved with eslint-utils-1.4.0
below screenshot of warning on gitHub

see yarn.lock

I was able to fix by running yearn upgrade react-scripts.
My version of react-scripts didn't change (still "3.1.1"), but<EMAIL_ADDRESS>updated its dependency of eslint-utils to 1.4.2.
Only run this if you are conscious of how new react-script version will migrate for you.
For new installs this problem should no longer exist.


Add in Flash Attention?

Looks like a great project! I'm a big fan of language models being written in TorchSharp.
I just released an extension library wrapping Flash Attention (see here), which perhaps can be a good addition to the project, when running on cuda?
See the original flash attention repository here: https://github.com/Dao-AILab/flash-attention

The purpose of this project was to provide a follow along project in C# for a demo that Andrej Karpathy wrote in python. That demo didn't use flash attention so I will have so say no to adding it to the main branch but I have no problem with you creating a separate branch that uses it.