New Plugins API

@jhwgh1968   there has been an open issue regarding the fact that creating an Engine is expensive: https://github.com/jonathandturner/rhai/issues/142
It looks like registering all those core functions (such as basic arithmetic) each time during an Engine creation is causing it to be really slow.
The solution, obviously, is to run it only once (since they never change - all scripts will need them).  One solution is to use lazy_static to make this package of functions a global constant, but the downside is that an additional crate must be pulled in: lazy_static.
An alternate solution is to really start breaking down the built-in's into separate packages that are immutable, so the packages themselves can be created only once, stored around, and then passed to all new Engine created.
Something of that sort:
let core_pkg = Library::CoreLib::create();

let mut engine = Engine::new_raw(&[ core_pkg ]);    // Pass in a list of packages to register
       :
let mut engine = Engine::new_raw(&[ core_pkg ]);
let mut engine = Engine::new_raw(&[ core_pkg ]);
       :
   create many engines

The version of plugins I am currently working on would solve this problem. It will allow an engine to delegate the lookups until the function is called.
It also will give more convenient Rust syntax for writing a package:
#[rhai::plugin]
mod MyPlugin {
    pub fn export_to_rhai(...) { /* do stuff* / }
    // more pub fns
}

Once defined, all that is needed will be something like
let mut engine = Engine::new();
rhai::register_package!(engine, MyModule);

If you are wondering how it works, it uses procedural macros so that the module actually looks like this:
mod MyModule {
    pub fn export_to_rhai(i: INT, f: FLOAT, t: MyStruct) { /* ... * }
    pub fn call<'e>(
        engine: &'e Engine,
        fn_name: &str,
        args: Iterator<Item=Dynamic + 'e>
) -> Result<(), rhai::EvalAltResult> {
        switch fn_name {
            "export_to_rhai" => Ok(export_to_rhai(args.next()?.as_ref().cast::<INT>()?,
                  args.next()?.as_ref().cast::<Float>()?,
                  args.next()?.as_ref().cast::<MyStruct>()?))
            },
            _ => Err(rhai::EvalAltResult::RuntimeError("cannot find function '{}' in 'MyModule'", fn_name),
        }
    }
}

This means that registering a module results in registering one "lookup function", which uses static code to generate calls.
This should be much more efficient, and more convenient for users.

Wow, this is something beyond my expectations. I think we can replace the entire packages implementation with this.
One question: when you register a plugin module, I suppose you register all the functions declared within the same module, right?  I don't suppose you can "pick-n-choose"?
You'll also need a way to handle functions that have a first argument that is &mut for update functions.  All other arguments must be passed by value.
Also, I'm thinking how this can handle generic functions with multiple data types.  I suppose you must have some form of name mangling then...  For example, a simple add function can be registered for the whole range of integer data types.

Wow, this is something beyond my expectations. I think we can replace the entire packages implementation with this.

I think we can replace most of it, but there might be some edge cases. See below.

One question: when you register a plugin module, I suppose you register all the functions declared within the same module, right? I don't suppose you can "pick-n-choose"?

In the current prototype, all pub items are exported to Rhai.
It would certainly be possible to add additional attributes to the functions themselves, like #[rhai::export_fn(name = "name_in_chai")] or #[rhai::export_fn(ignore)]. But I'm focusing on the main cases for now.

You'll also need a way to handle functions that have a first argument that is &mut for update functions. All other arguments must be passed by value.

I would probably have a call_mut_reciever as a separately generated table. After all, what's a little awkwardness if the compiler is doing all the work, right?

Also, I'm thinking how this can handle generic functions with multiple data types. I suppose you must have some form of name mangling then...

Generics are something I am not sure if I can support or not.
The syntax wouldn't be difficult to detect and add, but Rust's type inference may be the limitation due to my reliance on the cast operation for Dynamic.
Let's take your example with plus on numeric types. The broadest way to implement that would be to implement a trait RhaiAddable for all ints, floats, etc, and then:
#[rhai::plugin]
mod numeric_operations {
    #[rhai::reg_op("+")]
    pub fn plus<T: RhaiAddable>(a: T, b: T) -> Result<T, rhai::EvalAltResult> {
        /* achieve this with a trait call or special casing... */
    }
}

As currently written, the auto-generated code would create a call to cast::<T> -- that is, the generic type passed in, rather than any specific type.
Even if T is bounded eventually -- i.e. plus allows clear type inference -- I suspect Rust's type inference would sometimes get confused. There might still have to be some alternative mechanism for generic functions in this case.
Also, please note that my use of reg_op in that example was just to explain it. I have no idea how to approach that, so it might be another special case. :smile:

I have opened a PR for the foundational API I need for this work. Hopefully it will be quick to review and merge.

If you can modify your PR to merge into the plugins branch I'll do it right away.
As to generic functions, I think that is the main difference between Rhai and Rust... in this case, Rhai is more like JS.  Not only can arguments be of different types, there can also be different number of arguments.
In many Rust std lib cases, multiple functions with different names can be mapped into the same function name in Rhai.  I think we need to make the Rust API more "Rhai-centric" by leveraging overloading instead of a simple one-to-one mapping.
As for binding multiple generic versions, maybe the plugin can provide a generic version that an outside macro can simply loop over.  For example:
mod MyModule {
    #[rhai(name = "export_to_rhai")]
    #[rhai::reg(i8, f32)]
    #[rhai::reg(i32, f64)]
    pub fn export_to_rhai_3<T: Add, U, V>(i: T, f: U, t: MyStruct) -> String { /* ... * }

    #[rhai(name = "export_to_rhai")]
    #[rhai::reg(char, INT)]
    pub fn export_to_rhai_2<T, U, V>(i: T, f: U) -> bool { /* ... * }

    pub fn call<'e>(
        engine: &'e Engine,
        fn_name: &str,
        args_hash: u64,
        args: Iterator<Item=Dynamic + 'e>
) -> Result<(), rhai::EvalAltResult> {
        switch (fn_name, args.len()) {
            ("export_to_rhai", 3) => {
                  /* Somehow use a hash to map argument types */
                  match args_hash {
                      1234567 => Ok(export_to_rhai_3(args.next()?.as_ref().cast::<i8>()?,
                          args.next()?.as_ref().cast::<f32>()?,
                          args.next()?.as_ref().cast::<MyStruct>()?)),
                      98765 => Ok(export_to_rhai_3(args.next()?.as_ref().cast::<i32>()?,
                          args.next()?.as_ref().cast::<f64>()?,
                          args.next()?.as_ref().cast::<MyStruct>()?)),
                       _ => Err(......   argument types not matched .......),
                  }
           }

           ("export_to_rhai", 2) => ..... ,

            _ => Err(rhai::EvalAltResult::RuntimeError("cannot find function '{}' in 'MyModule'", fn_name),
        }
    }
}

If you can modify your PR to merge into the plugins branch I'll do it right away.

I opened a new PR against the plugins branch.
Please make sure to keep that feature branch updated with all your master changes. I'm relying on you to avoid big merge conflicts later.

In many Rust std lib cases, multiple functions with different names can be mapped into the same function name in Rhai. I think we need to make the Rust API more "Rhai-centric" by leveraging overloading instead of a simple one-to-one mapping.

I generally plan to handle overloading and multiple types with the flexibility in that call function, without using Rust generics. Sorry if I got confused by focusing on those.
To go back to a previous example of mine when I was thinking about varadic functions:
let forty_two = sum(40, 2);
let forty_two_again = sum(35, 5, 1, 1);
let forty_two_point_zero = sum(34.0, 6, 0.5, 0.5);

This is the kind of overloading you mean, right?
My plan for that was to make a function attribute that indicated a "varadic" function. If called, the iterator should be passed directly, and let the function itself handle it:
mod MathModule {
    #[rhai::export_fn(name = "sum", varadic)]
    pub fn add_anything(args: impl Iterator<Item=Dynamic + 'e) -> Result<Dynamic, EvalAltResult> {
        let total: FLOAT = 0.0;
        let return_float = false;
        for (n, i) in args.enumerate() {
            if let Some(i32_box) = i.down_cast::<i32>() {
                total += *i32_box as FLOAT;
            } else if let Some(f32_box) = i.down_cast::<f32>() {
                total += *f32_box;
                return_float = true;
            } else if let Some(i64_box) = i.down_cast::<i64>() {
                /* etc ... */
            } else {
                return Err(EvalAltResult::RuntimeError("argument {} is not a float or integer", n));
            }
        }
        Ok(Box::new(if return_float { total } else { total as INT }))
    }
}

I also hope this problem will be somewhat mitigated by broader use of this plugin style. It will hopefully encourage Rhai to develop more impl From<T> for Dynamic impl blocks for different Ts, to allow automatic casting to do more work.

Please make sure to keep that feature branch updated with all your master changes. I'm relying on you to avoid big merge conflicts later.

OK, I'll make sure I keep it up-to-date.
Your idea of keeping all the argument resolution inside the function implementation itself is a good solution to the problem of overloading.  However, this puts the burden on the plugin author, who must then use a mass number of .is::<T>() or downcast::<T> calls to detect the correct argument types before dispatching to the correct implementation.  This kind of dynamic dispatching is going to make calling each function slower (although I don't have benchmarks to prove that).
One way we can avoid this is to pass in &[Dynamic] instead of an iterator.  This way, at least the author knows the number of arguments, and their types (he can run Dynamic.type_id() on each on to get the TypeId.  He can then hash these TypeId's to quickly find the correct version of the function to dispatch to, and coerce the correct parameter types.

In a perfect world, we should provide macro facilities to do this kind of dispatching so the author doesn't have to worry about it!

Your idea of keeping all the argument resolution inside the function implementation itself is a good solution to the problem of overloading. However, this puts the burden on the plugin author, who must then use a mass number of .is::<T>() or downcast::<T> calls to detect the correct argument types before dispatching to the correct implementation.

I personally don't think this is a significant burden. As shown in my example, I originally came up with this design for algorithms which "fold types together" naturally in the course of their processing. It would be difficult to make sum any shorter using generics if you allowed such type mixing.
That said, I see your concern for functions which are "do the same thing, just accept different types." It may be possible to expand the procedural macro to support this, but my first idea is to expand the types on the Rust side.
For example, take add. It is a binary operation (not varadic, like my sum), and works on ints or floats in any combination.
I would write this definition:
mod CoreMath {
    pub fn add(x: Either<INT, FLOAT>, y: Either<INT, FLOAT>) -> Dynamic {
        match (x, y) {
            (Either::Left(i), Either::Left(j) => Dynamic::from(i + j),
            (Either::Right(i), Either::Right(j) => Dynamic::from(i + j),
            (Either::Left(i), Either::Right(j) => Dynamic::from(i as FLOAT + j),
            (Either::Right(i), Either::Left(j) => Dynamic::from(i + j as FLOAT),
        }
    }
}

Rhai would just need to ensure Dynamic could downcast from an INT or a FLOAT to that Either. (This might be the case already. It is difficult for me to follow the low-level type system APIs you are using.)
This would work with the procedural macro I already have in mind.

This kind of dynamic dispatching is going to make calling each function slower (although I don't have benchmarks to prove that).

My understanding is limited, but the impression I get is that the match shown above and the casting done in sum will be of the same cost after codegen.
Both of them will end up looking like a switch statement in C: choose where to jump based on whether this long integer's value (the type ID or pair of type IDs) is A, B, C, D, or anything else.
Accessing the type ID during the cast shouldn't be slow, either. The functional arguments will surely be in the CPU cache -- after the first conditional check, at the latest.

One way we can avoid this is to pass in &[Dynamic] instead of an iterator. This way, at least the author knows the number of arguments, and their types (he can run Dynamic.type_id() on each on to get the TypeId. He can then hash these TypeId's to quickly find the correct version of the function to dispatch to, and coerce the correct parameter types.

I find this direct use of TypeId unintuitive. I would rather write code which has Rust doing it for me in things like if and match statements.
The reason I picked a type of impl Iterator was because it wasn't clear to me whether you wanted to gather up all the arguments into one place before calling anything. Depending on the size of your Rhai stack, that might be a little slow compared to just writing an iterator to skip to the ones needed.
If you want callers able to know how many arguments they have -- for example, to support "wrong number of arguments" checks early on -- then I would happily change it to an iterator wrapper which implements ExactSizeIterator.

In a perfect world, we should provide macro facilities to do this kind of dispatching so the author doesn't have to worry about it!

My goal is to have procedural macros do as much work as I can. However, I am still figuring things out. A number of your ideas I agree with in general, but I hesitate to make many promises right now.
As things get beyond my first, simplest cases -- currently, all functions accept types passed by value, and return Dynamic -- I will more deeply consider some of your other ideas, and understand what is necessary and possible.

Also, a separate question: name spaces.
Currently my Plugin definition requires providing a static name. My original idea was to use this as a namespace. Perhaps something like:
import "crypto";
let encrypted_message = crypto::aes("key", "message");

The use of crypto::`` in this code would be the signal to Rhai to find the plugin with the name of crypto, and then invoke MyCryptoPlugin::call` (the function generated by the procedural macro) without any regular function lookup steps.
Has any previous work on modules or packages nailed down a syntax I could hook into?

Has any previous work on modules or packages nailed down a syntax I could hook into?

Not really, so you can "modularize" the plugins.  However, I'd say let's keep the namespacing/modules syntax in sync so we don't get into a conflict later on.  At the least we'd need to support user-defined namespaces/modules which can be imported in the same manner.
And are you going to support importing just one function inside a module?

(This might be the case already. It is difficult for me to follow the low-level type system APIs you are using.)

Unfortunately the particular design used in Rhai depends heavily on dyn Any trait objects, meaning that their types are erased.  Just about the only thing you can do is to use is::<T>() or the TypeId to check if it is of a particular type - you need to know the type you're checking in advance.
For example, you won't even know if a dyn Any type is Clone or Copy etc.  This means that any use of a type outside of the built-in's in Dynamic (e.g. u16) will be converted into a heap-allocated, boxed trait object of dyn Any with all type info erased except its TypeId.
This is the way Rhai handles custom objects that it doesn't know about.

mod CoreMath {
    pub fn add(x: Either<INT, FLOAT>, y: Either<INT, FLOAT>) -> Dynamic {
        match (x, y) {
            (Either::Left(i), Either::Left(j) => Dynamic::from(i + j),
            (Either::Right(i), Either::Right(j) => Dynamic::from(i + j),
            (Either::Left(i), Either::Right(j) => Dynamic::from(i as FLOAT + j),
            (Either::Right(i), Either::Left(j) => Dynamic::from(i + j as FLOAT),
        }
    }
}

Usually this is enough, if we're only supporting INT and FLOAT without regards to other integer types.  So far, most of the use for generics is actually for the wide list of integer types, and things like print, to_string that have the same name working on different types.
to_string is actually a good example.  Remember, Rhai functions can be called method-style, meaning you should be able to do:
import aes256_block from "crypto";
let encrypted_message = "key".aes256_block("message");

Meaning that a function call may not always be possible to be qualified with a namespace.  Of course, you can force everybody to use namespaced calls instead and disallow method-style, but that would be very un-Rhai...
For a function like to_string, naturally you have tons of implementations, one for each type.
  He/she would obviously want to implement to_string for those types, and be able to just call it postfix-style.  Therefore, you STILL need to have a way to resolve the same function name purely based on the parameter types just to handle this kind of overloading.
In your system, everything must be pre-baked-in. That is, you'll have one to_string function with a massive switch statement inside that detects all the types it can handle.  If the user adds a new type in a new plugin, the stock to_string implementation won't know about it.

The plugins branch is caught up with the latest changes - nothing touched in the function libraries other than minor bug fixes.

a function call may not always be possible to be qualified with a namespace. Of course, you can force everybody to use namespaced calls instead and disallow method-style, but that would be very un-Rhai...

I am currently prototyping the simplest answer to that question: a module can only define receiver methods on types that it exports. When a type is imported by Rhai, all receiver methods come with it.
This is sufficient for my use case, which is about writing types and functions in Rust, and easily exporting them to Rhai. But I recognize that this will not suffice cases where users want to extend the standard library, or do a more dynamic use module::trait the way that Rust can.
In other words, my current prototype allows this Rhai code:
import "crypto";
let cipherstate = crypto::Aes256::CBC::new("my key", 0 /* the IV */);
let block_one = cipherstate.encrypt("message 1"); /* returns as a string, or perhaps a JS like ArrayBuffer type */

But not something like this, which is what you seem to be describing:
let s = "abc";
let r = s.reverse(); // ERROR: no such method "reverse" on type "string"
import "string_utils" with traits;
let r = s.reverse(); // OK, because the plugin defined "reverse" on &mut String

While it would be easy to write a separate attribute to tie into such syntax, there is the problem of conflicts.
Currently, Rhai plugin loads are presumed infallible. I wasn't going to write them this way at first, but everything they do to the Engine seems to be infallible. It's not clear what happens if two modules try to add a reverse method, for example, to the built-in string.
If there were two calls to register_receiver_fn, with the same name and signature, would the second one panic? Or would it just overwrite the first?

In addition, let me go back to to_string for a moment.
The function to_string has an implementation for a lot of different types. As you said, what if a user wanted to print their own type with to_string?
My off-the-cuff answer would be: there is a to_string in every plugin where a type is defined, and it "derives" from the parent:
module Crypto {
    pub struct CryptoState { /* ... */ }
    impl CryptoState {
        #[rhai::plugin::derive(to_string)

        /* other explicit methods here */
    }
}

The derive macro would have the engine look up a function with that name which took the receiver of Dynamic, and call it on the object instead. Code on that is TBD.

If there were two calls to register_receiver_fn, with the same name and signature, would the second one panic? Or would it just overwrite the first?

I believe the second one overwrites the first right now...

Regarding to_string, currently string handling in Rhai is a bit ad hoc...
For a new type, you need to define to_string, print (which lets it to print), debug (letting it debug print), as well as the +(string, type) and +(type, string) operators to do string concatenation.
Not only that, you'd also want to override Array.push to let it be added into an Array, plus maybe a few other combinations to work with other standard types.
Not the best design, I admit, but that's how it is originally structured.  Any ideas on your side to really streamline/automate this would be appreciated!

import "string_utils" with traits;
let r = s.reverse(); // OK, because the plugin defined "reverse" on &mut String

Can we just do:
import "string_utils";

if reverse is defined there?

Can we just do:
import "string_utils";

if reverse is defined there?

The only reason I wrote that was thinking about Rust's #[macro_export].
They wanted you to opt-in to procedural macros, because the macros would show up in the global namespace. That might cause conflicts you didn't want.
In this case, the conflicts would be between modules, if "traits" were always imported:
import "string_utils"; // contains a reverse method on &mut String
import "string_cesar_cipher"; // also contains a reverse method on &mut String. Oh no!

The with traits is outside of the namespaces proposal, though, since I haven't planned that far ahead.

Good point here, about potential name conflicts.  In that case,, even if we force the user to with traits, there is always a chance that the user wants one trait from one module and another trait from another module, which happens to have a conflicting function.
We'd probably need to do something similar to JS:
import reverse from "string_utils";

s.reverse();

or in case of conflicts:
import reverse as reverse1 from "string.utils";
import reverse as reverse2 from "my_string_utils";

s.reverse1();
s1.reverse2();

And if we have this, mind as well open this up to all other imports:
import "crypto";           // crypto::encrypt(...);    import all under namespace
import * from "crypto";     // encrypt(...);   import all under global
import encrypt as en from "crypto";    // en(...);
import { ??? as ???, ???, ??? as ??? } from "????";    // multiple imports

@jhwgh1968   I've kept the plugins branch up-to-date with the modules work.
Now you can program your macros to the modules.  Except there is no way to do the with traits thing right now to dynamically add methods to existing types...

The with traits idea is something I have not spent much thought on, I'm afraid. I just got time to review your modules implementation, and that is what I am currently focused on.
My design instinct tells me:

Split trait functions into a separate FunctionsLib in every Module.
Make get_fn and friends take an extra boolean that means "include traits in the search."
Add a boolean to the Stmt::Import enum in the AST, to indicate whether with traits was provided (which is itself only two tokens, or perhaps even one).
Have the engine set the get_fn boolean based on the Stmt::Import flag used on the corresponding module.

Does that help?

As for the modules implementation, it took me a while to get my head around what I need to do for plugins, but I think I can make progress.
In particular, I hope to commit a basic plugins implementation soon, which can create new Modules and attach them to a StaticModuleResolver.
However, I'm only committing it because I am a big fan of the old saying, "do it first, do it well second." If I were to start on the procedural macro from that PR, it would fail to achieve some of the benefits, and might make some of the features we discussed above more difficult to implement.
Everything I wrote up there relied on my low-overhead lookup strategy: resolve the namespace to a plugin ahead of time, but defer resolving namespace members until  a script accesses them. That is what the macro would generate.
Your current modules implementation is still incompatible with that approach. A plugin can now enable the StaticModuleResolver, and then set_fn and set_var for all of its contents; but that will box everything up into Dynamic and Arc<FnDef>.
I think my approach would require writing a PluginModuleResolver, which could perform the namespace lookups in a different way; and that would then conflict with a significant amount of your code.
In the process of writing this post, I now realize I need to separate these two related ideas in my head:

Plugins based on procedural macros help Rhai users with the syntax of adding Rust items into the Rhai namespace.
Plugins based on procedural macros help Rhai users add Rust items into the Rhai namespace with less overhead than was previously possible.

How would you rate those, @schungx? Is one worthwhile without the other? I remember you expressed praise for the procedural macro idea, but it might have just been item 1 you were thinking of.

do it first, do it well second

Wise words indeed!  That's how I define "hacking".

I think my approach would require writing a PluginModuleResolver, which could perform the namespace lookups in a different way; and that would then conflict with a significant amount of your code.

I don't think a custom module resolver can help in this case, because it seems like you'd want to lazy-load functions and variables, defer until the first time they're used.  Correct?  That means that, when the module is attached to an Engine, it doesn't contain anything.
However, currently in the interest of speed, I hash up all the functions and variables in the entire tree once a module is loaded.  Doing this has the great advantage of making module-qualified function calls and variable access as fast as normal.  However, this also means that you'd need to have all of them ready in the very beginning.  Sort of a catch-22...

but defer resolving namespace members until a script accesses them

If this is lazy-loading, I think that's a wonderful idea.  Let's not lose it just because of the current architecture.  Let's think of a way to have our cake and eat it.

How would you rate those

Well, I must admin I was thinking of #1.  Haven't really thought about #2 as being possible at all, until you raise it up now.  If it is at all possible, then #2 is clearly the better solution.  That would make loading up Engine's extremely fast because none of the built-in functions need to be loaded until they are needed!

If this is lazy-loading, I think that's a wonderful idea. Let's not lose it just because of the current architecture. Let's think of a way to have our cake and eat it.

I found a way. It's in my current PR. I call them "lazy modules", in order to separate them from the modules you previously wrote.

However, I would really like to know more about what you're delaying. Which workload is put off until needed? Because I think you'll need a large lookup table just to know the names of all the possible functions to match for, and their parameter types

The trait that lazy modules implement is:
pub trait LazyModuleDispatcher {
    fn call<'e>(&self,
            fn_name: &str,
            args: Box<dyn Iterator<Item=Dynamic> + 'e>
    ) -> Option<Result<Dynamic, EvalAltResult>>;
}

This trait method is directly called by the Engine from the FnCall operation. It is expected to do the lookup an the invocation all at once. That is what is being deferred.
With my #[plugin] procedural macro, the macro will generate a module containing a struct implementing that trait, with all the module's contents tied into it. It would be a large table for something like the Rhai standard library, yes. But the key is: it is in code, rather than a data structure.
The current implementation (shown in my test case) is not be the most optimal, I agree. But since it is arbitrary code, it can simply be written to be more efficient once over time.
If you can come up with a template in the procedural macro to do the name and argument hashing on the first call of every module? Great. If Rust's const fn feature becomes more powerful, and the procedural macro can insert a pre-caculated a table? Even better.
Aside from being simpler for me to understasd, having a code-based lookup -- call this function, don't worry about what it does -- allows better flexibility and options for optimization in the future.
That is the promise I see, and why I have been pushing on this idea so hard.

But the key is: it is in code, rather than a data structure

A large match statement or a large chain if if ... else if ... matching names and parameter types is probably gonna be slower than a hash lookup.

If you can come up with a template in the procedural macro to do the name and argument hashing on the first call of every module

There is already such a function.  It is crate::calc_fn_hash.   It is used everywhere to hash functions, modules, and/or parameter types.  I usually pre-calculate these hashes into the call site.
There is already code in Modules to index the entire modules chain once it is loaded.  The entire modules path plus the function name plus the number arguments are hashed at once, and cached at the module root.  This way lookups are extremely fast - they don't have to search deep into the modules structure.
The hash is calculated as two separate steps: the first one with placeholders in types, and the second hash just the argument types.  The final hash is the XOR of these two hashes, but this is an implementation detail.
When the time comes do a function call with real parameter types, a pre-calculated hash with placeholder types is XOR'ed with a hash of the real argument types to form the final hash.  That has is used to find the function in the modules tree.
If we simply separate the "lazy-loaded" functions from normal ones, we can have a separate function Module::find_lazy_function.  We can call that first, and you can return a trait object.  Or hell, we simply modify FnAny to be a struct that implements that trait and have all module functions return a trait object.  Then use the call API to make the call.
The more I think about this, the more this is beautiful.    We should change FnAny which currently is a type def only holding a boxed function pointer.  We can make it struct FnAny that implements Callable with a call API.  In this way, we unify all the functions in the whole system in one single blow!

pub trait LazyModuleDispatcher {
fn call<'e>(&self,
fn_name: &str,
args: Box<dyn Iterator<Item=Dynamic> + 'e>
) -> Option<Result<Dynamic, EvalAltResult>>;
}

Don't forget, not all functions take all value parameters.  Some mutate the object (e.g. String::trim), so you need to have a version with &mut parameters.
You can make args: Box<dyn Iterator<Item=&'a mut Dynamic> + 'e> to have the iterator return &mut Dynamic instead of Dynamic values, and then deferencing the value parameters with mem::take (yes, you can consume the value parameters).  This is how register_fn does it.

@jhwgh1968 I have been pondering a question, so maybe you can help give some thoughts.
It is regarding modules, plugins and "packages".
Right now, they have very similar behavior, with only minor differences:

Type
Namespace
Access
Lazy-loaded

Module
loaded into imported modules namespace tree
mod1::mod2::mod3::func()
No

Plugin
loaded into imported modules namespace tree
mod1::mod2::mod3::func()
Yes

Package
loaded into global namespace
func()
No

Is there any way to unify all these concepts?

You don't need to pass an iterator because all the places where functions are called, the arguments are already collected in a slice.

I wanted to preserve the iterator trait on the theory a non-slice implementation might be more efficient eventually. However, your last round of changes convinces me that is over-thinking things. I'll make it a slice.
I mentioned elsewhere that NativeCallable might help with return types, but the lookup part is what I am focused on at the moment with the Dispatcher trait. I will integrate with it later, but it does not help me at the moment.

Is there any way to unify all these concepts?

My hope was plugins would unify them, but that may be more difficult than I thought. Allow me provide some high-level thoughts of my own.
The more I learn about the dynamic casting system, the more I think: wow, this is not how I would have done this at all. To be clear, that is intended to be a neutral statement. If I were to try and do it over, I might end up making some of the same decisions, or losing benefits from making others.
It is simply something I have not fully wrapped my head around, and have to re-discover the details of whenever I put this down and pick this up. To explain, let me tell you what background informs my expectations.
I know something about the way the Python 2.x interpreter the Perl 5 interpreter are written. Both of them are fairly simple, but have extensibility as a key tenant. From the inside out, they consist of the following, with fairly clean boundaries in between each piece:

An opaque, flexible type to represent a value in the target language. (Equivalent to Rhai's Dynamic.)
A bunch of utility functions for casting these values between the target language and the source language.
An "interpreter core" which does syntax analysis and execution of a given program in the target language.
A "plugin system" for code in the source language, which code in the target language sees no differently, but the interpreter knows how to call with an escape hatch.
A number of source language plugins, which corresponds to "basic operations" on various types, and glue code to the host OS. It is the backbone of the standard library.
A number of target language modules, which fills out the standard library. Some things are fully native, but those that are not wrap and simplify those plugins.

(Note: all statements below about Rhai are my original impressions, and things may have improved somewhat thanks to our efforts.)
Rhai also consists of these pieces, but it's item 4 that felt furthest from my expectations. Both in the amount of code needed, and the amount of dynamic stuff on the heap required.
In the case of Python and Perl, the source language is C. The code in item 4 can rely on a lot on the C language and compiler to do a lot of work, using type definitions and pointers in clever ways. All functions called in this way are statically defined, since C doesn't have a concept of closures, so it's just a bit of ABI trickery that makes up the special opcode.
The C functions called must them follow an API specification and a template: call this to get you args, call these functions to cast things, do your work, and return a result this way, or throw an exception that way.
That is what I expected Rhai to be, more or less. What to call something natively written in Rust? Just make your Rust function according to a particular template -- even if it's an ugly one -- point Rhai at it, and it will "just work."
Rust, of course, means the rules are different. Weak typing and clever casting are things it does not like to do, as they are sources of UB. However, even despite that, it seems this layered approach is not how Rhai is designed.
Rhai seems to be much more like PHP, where any native code is less like a "module", and more like an extension of the engine's behavior. register_fn did not take a function pointer, for example, but a boxed closure, which it put into the global namespace from a language point of view.
That high-level approach is what I am trying to change at a fundamental level. That is the change I think unlocks the real potential of Rhai to become a small but real language, rather than lightweight way to run simple scripts.
That is why I am thinking about native and "lazy" calls -- things akin to what the Python and Perl interpreters do, rather than things baked into the engine before the script runs. That is the keystone to building out a good user experience with Rust's language tools, like procedural macros to handle all the template generation.
What is not quite clear to me is: what parts of the way Rhai currently works should continue, even in this new paradigm?

Is the hashed lookup of names an implementation detail, or is that part of the API for the template?
Are the different access behaviors between the table items actually features, or could they all be reduced to one kind of thing?
What value is there in extending the engine versus writing a plugin? What APIs should be on which boundaries?

Those are my questions at the moment, and why I hesitate to go too far out on a limb. Because even if you like the idea -- and you seem to be warming to it more and more, as we discuss different aspects -- the amount of rework involved is tremendous.

Rhai seems to be much more like PHP, where any native code is less like a "module", and more like an extension of the engine's behavior. register_fn did not take a function pointer, for example, but a boxed closure, which it put into the global namespace from a language point of view.

I suppose the main difference is, since Rust has no GC, and zero-cost abstractions forced the design of closures to be a simple type.  So all function points must be Boxed closures - since a closure is a type to keep the closed over environment as there is no GC to remove them later on.

That is why I am thinking about native and "lazy" calls -- things akin to what the Python and Perl interpreters do, rather than things baked into the engine before the script runs. That is the keystone to building out a good user experience with Rust's language tools, like procedural macros to handle all the template generation.

My feeling is that we'll be fighting Rust's language design here.  Rust has very little capabilities to do things lazy-bound, since everything tends to be compiled in, instantiated and inlined.
The only way to make it lazy is probably to 1) compile a plugin as a C shared library (cdylib) written to a C API, 2) dynamically load the shared lib from Rhai, 3) call the function.  That would make Rhai able to dynamically load separate plugin files (similar to how DLL's are used in the Windows world).  My own project has a mechanism like this and it works well, although I haven't used it to dynamically load plugins into Rhai...  it is used in my project to load device drivers.

Is the hashed lookup of names an implementation detail, or is that part of the API for the template?

I'd say it is an implementation detail and not absolutely necessary.  However, with the namespace of functions getting large, we'll need an efficient way to lookup a particular function short of a huge match statement that compares strings one by one.  Not to mention functions buried deep under a modules tree (when your plugins work is done, I'd suppose we'll have an explosion of the number of modules).
So the hashing is simply an optimization mechanism because it can be pre-computed during parse time.

Are the different access behaviors between the table items actually features, or could they all be reduced to one kind of thing?

I think they can all be reduced to the same codebase.  I already junked the packages code and re-implemented packages on top of modules.  Now there are only modules.  In the future, when you're done, I anticipate moving modules to your plugins.

What value is there in extending the engine versus writing a plugin? What APIs should be on which boundaries?

Extend the engine when you know the feature will always be there.  It'll be compiled in, inlined and optimized by Rust.  The result is one single executable file.
A plugin, if it resides in the same main app program and compiled in, is really no different from an engine extension.
A plugin, if residing in an external file (e.g. a cdylib, .dll or .so shared library), can be hot-loaded and can be discovered during runtime.  Now that's a completely different level of capabilities.  I believe that's what you're thinking of for your plugins system, right?
The entire language library can simply be provided as a huge collection of .so or .dll files and dynamically (lazy) loaded when used.

My suggestion is this:
First, there must be a way to load a plugin as a module.  You can leverage the existing modules resolution mechanism, maybe introducing a PluginsModuleResolver.
import "std/core/string" as str;   // load std/core/string.so from file-system

The Engine then keeps a cache of hot-loaded plugin's.  If it is already loaded, it won't load it again.
If it is not yet loaded, it first loads the C shared library file from disk, then use a standard entry-point to discover all the functions/constants inside this plugin.  Something that takes a call-back closure from Rust and, with repeated calls, each time returning info of one single function or constant.  Return NULL when done.
The Engine then takes this information and then cache them all for fast lookups.
Obviously, for each function, the plugin should return some form of "handle" (which doesn't have to be a hash), maybe simply an offset to an internal array or something.  A function call can simply use this "handle" instead of having to pass the function name + parameter types + ABI all over again.
Alternative implementation
The same plugin-loading mechanism as above.  But, after loading the module, do nothing.
When a call to a module function is made, the engine calls a C API entry-point on that plugin library, passing it the function name, parameter types, and actual parameters, and the ABI type (e.g. the first parameter may be a reference pointer).
The C API then dispatches the function internally based on its own mechanism that the scripting engine doesn't need to know.
This method is much simpler and encapsulated - you can completely change the internal mechanism of the plugins without affect the engine.  Development of plugins and the engine can proceed separately.  However, the downside is always having to pass a lot of information on every single module function call, which will eventually impact performance.
I think the handle method is better for performance, and that's how most C-style programs are done.  One additional benefit for the handle method is ability to polymorph the object behind the handle.  In the alternative method, there is a lot more information exposed and they can only realistically map to a function call.

What value is there in extending the engine versus writing a plugin? What APIs should be on which boundaries?

Extend the engine when you know the feature will always be there. It'll be compiled in, inlined and optimized by Rust. The result is one single executable file.
A plugin, if it resides in the same main app program and compiled in, is really no different from an engine extension.
A plugin, if residing in an external file (e.g. a cdylib, .dll or .so shared library), can be hot-loaded and can be discovered during runtime. Now that's a completely different level of capabilities. I believe that's what you're thinking of for your plugins system, right?

I actually wasn't planning on going that far. Rust does not have a stable ABI yet, so I don't think the return-on-investment is there. As you say, would either: (a) have to make sure plugins are compiled with the same version of the Rust compiler, or (b) marshal a lot to C and un-marshal it on the other side.
In retrospect, perhaps "Plugin" was an imprecise word. Many of the extra "plugins" I listed in Python and Perl are either statically linked, or tightly dynamically linked (not lazily loaded on demand) into their "engine". They are separate code because they are not required by the core itself, even though they are part of the language as a whole.
In Perl, for example, the interpreter core is called "miniperl". You can compile it by itself, and you will get a dumb REPL that is less than half the size of full Perl. However, that interpreter is basically unusable for anything most people want to do Perl. Among other limitations, it can't read a script from anywhere but stdin, it can't use a number of "obvious" system calls, and it has an extremely trivial pattern matching engine which cannot handle even most POSIX regexes.
When you perform a normal build of Perl, you build libraries that are statically linked to "miniperl" in order to create the interpreter you actually use. These are all deeply linked it to the core, and the final Perl executable cannot function without them. One example is PCRE, the Perl Compatible Regex library, which has hooks to override the pattern matching in the core. While PCRE is sometimes dynamically linked because it is a library in its own right, it is still eagerly loaded and required.
That is more like what I was envisioning with plugins in Rhai. They are Rust code that "create the environment" for the script, either fleshing it out into a full interpreter with access to the system, or providing APIs suitable for making it useful and extensible when embedded.
The latter case, in particular, is why I have been focused on how easy it is (or isn't) to expose complex types or a suite of functions to Rhai, with thoughts of procedural macros.

Perhaps, as you say above "extensions" would be a better term.
I suppose I did not use it, because that makes me think of Firefox -- where extensions prior to FF 52 could be deeply invasive, and break everything without regard to API boundaries.
That is not what I was thinking. The engine would have an API that these extensions could hook into, and limit what they could do to the language as a whole.

What you're describing is similar to what an assembly is in .NET world.  It is an essential part of the running app, without it the app won't run.  However, they can be compiled separately and then dynamically linked.  The ABI is stable, so plugins really are simply the same assemblies, but not required for the app to run.
In Rust, it is more difficult to do what you want because Rust has no reflection API.  There is no way to "include" a crate/rlib in a build, and then have the app automatically discover that it is there.  The app must already know that it is there, perhaps through some entry in a lookup table, or a configuration file that is read during build to specify the entry-point function name sto call.  In PERL I assume the build stage will have macros feeding into the C source that provides linkages to compiled-in extensions.
To do your extensions, it might actually be better to compile them into dynlib's, which are Rust ABI crates that can be dynamically loaded and linked.  Then you only need to scan a directory to find all the files to load.
And if you do that, mind as well use standard C ABI.  It really is very simple to expose a C-standard ABI from Rust, as well as to consume one.  I think this route offers to most flexibility, as you can then easily port code written in C to a extension for Rhai.

To try and summarize what happened since I last posted to this thread, I have gotten a basic start on plugins merged into a feature branch.
This vision of plugins focuses on my use case: pieces of Rust code that can modify the engine or expose modules. They are statically linked, defined by a user of Rhai who is embedding Rhai's engine in another project.
Based on what is here, I will spend some time focusing more on the porcelain: procedural macros similar to what I referenced above.

Also, as @schungx has noted, there would be some benefits to folding a number of other features (such as packages) into plugins that are or are not loaded. Since I expect those will look somewhat different than user code, I expect that work can be done in parallel, and re-written as I create porcelain pieces that are helpful.

Looking forward to the proc macros!

Oh, by the way, @jhwgh1968 , I'll soon push a commit that's transparently convert between &str and Immutable.  That's because users will get sick of having to write wrapper closures simply to turn their String parameters into ImmutableString otherwise Rhai won't find them.
So in your proc macros, you might want to take advantage of this.  You simply call set_fn with ImmutableString (if the parameter is &str) and Rhai does the rest.

The commit is in, and it is merged into plugins.  You should now handle parameters of &str in your macros.

@jhwgh1968   a couple of things for you to consider when you implement your proc macros:

Now &str maps to ImmutableString as a convenience, there is a function by_value in fn_register.rs that simply converts from a &mut Dynamic into whatever parameter type required.  It takes care of &str as well (i.e. it'll convert an ImmutableString into &str).  You can simply use this function to downcast function parameters from Dynamic and it takes care of &str automatically.

Rhai supports two calling ABI's: all-owned parameters, and first &mut reference (and the rest owned values).  However, for functions generated by your proc macros, you are not under such restriction.  In fact, you're free to use any parameter type (including & and &mut) in any parameter position, as long as your proc macro do the correct conversion from Dynamic.  This will make plugins extremely versatile, much more flexible than register_fn.  However, notice that all parameters other than the first one are cloned anyway, so you don't really save much by using a reference instead of consuming the value.

@jhwgh1968  just a tip up.  To the CallableFunction variant I added a parameter, a reference to the Engine.  This is to enable writing functions that rely on some Engine setting such as checking the maximum allowed size of an array.
It should not affect your PluginFunction variant.

Just so you know, I have been seeing these comments. I've been pretty busy, but I will make sure they are integrated before I do my next PR.

Great! Take your time!  No pressure!

After making some progress, I was preparing to open an early PR. However, I just tried to rebase onto the plugins branch, and according to the diff, some of the recent merges from master have completely undone all my prior work.
Could you fix that, please?

completely undone all my prior work.

Huh?  I thought I had been careful about this...
Can you let me know what is missing?
Sorry!

OK I can see it now.  I'll fix.
Ended up I wiped out almost all the changes...

@jhwgh1968   it should now be fixed.

Thanks, @schungx! I'll have a PR open within the next day or two, looking for some early feedback.

An update: version 0.1 of the procedural macros crate is now merged to the plugins branch! :tada:
My current to do list:

[ ] Improve macro hygiene in case the user renames the rhai crate -- #23
[ ] Investigate for supporting generic types on functions marked with #[export_fn].
[ ] Implement impls and self parameters on types

@jhwgh1968 There are some breaking changes. Version 0.18.1 has merged in real closures, so the Dynamic API is changed a bit.
downcast_ref ==> read_lock
downcast_mut ==> write_lock
Just replace these calls and it should work...
I've merged in the latest master into plugins. It doesn't build because of the macros referring to downcast_ref which has been made private.

I made the fix.
I see you made the fix overnight, but it doesn't seem quite right compared to what I got working. I'll open a PR for my version.

@jhwgh1968   I've looked through what's needed to replace the current packages implementation with plugins, and there are a few:

Somehow need generics instantiation.  Maybe if I write a macro to drive the plugin macros, not sure if it'll work.  I'll test.  Most of the standard library are basically registering functions on different data types, especially different numeric types.

Does the macro automatically detect that a function is returning Result<Dynamic, Box<EvalAltResult>>?  or Result<T, Box<EvalAltResult>>? Some standard library functions return errors.

To expand the standard library, probably will need a macro that automatically creates custom type wrappers and getters, setters, indexers, methods etc.

OK, it seems normal macros work inside procedural macros, so I can just put a macro to expand one function into many.
macro_rules! expand_display {
    ($($x:ty),*) => {
        $(
        pub fn display_inner(value: $x) -> String {
            display(value)
        }
        )*
    };
}

    fn display_inner<T: std::fmt::Display>(value: T) -> String {
        value.to_string()
    }

However, the problem is, I need to create unique names for all those functions to satisfy Rust.
Or, can your procedure macros actually handle functions with the same name?  If that's the case, I can just declare functions under the same name, and you can procedurally generate a unique function name for each.

Somehow need generics instantiation.  Maybe if I write a macro to drive the plugin macros, not sure if it'll work.  I'll test.  Most of the standard library are basically registering functions on different data types, especially different numeric types.

I did try generics as a brief experiment, and they are unfortunately quite ugly.
When I create calls to downcast_mut and downcast_clone, I always specify <T> in the cast. With generics, I don't know what type T actually is. So I looked into a rule that is easy: "if I would put in a generic T in my code, put in _ instead and rely on type inference."
That does sort of work, but it depends on how the type is used in the function's signature (or more importantly, is not used, such as in the return type only). There are quite a few cases where, given the surrounding context and the flexibility of Dynamic::from the Rust compiler cannot infer the type.
I am hoping item number three on my list -- self impls -- would substitute for many of the uses of generics I've seen in Rhai.
That is, if you wrote this:
#[rhai::export_impl]
impl DisplayInner for FLOAT {
    fn display_inner(&self) { /* ... */ }
}

My (future) macro would generate the same code as if you had written this:
#[rhai::export_fn]
pub fn display_inner(f: &mut FLOAT) { /* ... */ }

But that still can't handle generics.

As mentioned before, need a way to set the function name for overloading.

This is definitely required for the above to work, and is something I can add very soon.

Does the macro automatically detect that a function is returning Result<Dynamic, Box<EvalAltResult>>?  or Result<T, Box<EvalAltResult>>? Some standard library functions return errors.

I had in mind adding another attribute to the macro to say: don't cast to Dynamic::from, return the result as-is. Implementing that will address this. I'll put that second on my list.

Or, can your procedure macros actually handle functions with the same name? If that's the case, I can just declare functions under the same name, and you can procedurally generate a unique function name for each.

The way you should think of my macro is that it creates a Rust module named the same thing as the function is currently named. Two modules with the same name would cause a collision.
That is why I will work on allowing renames ASAP.

The way you should think of my macro is that it creates a Rust module named the same thing as the function is currently named. Two modules with the same name would cause a collision.

In that case, should it be possible for you to generate unique module names then?  Perhaps a serial number, or a hash like what I did with anonymous functions.
I suppose abc123::func_entry() really shouldn't matter what the module/name is, as long as it is registered like engine.register_fn("do_work", abc123::func_entry).  The user only sees the Rhai function name.

I have a PR for renaming support in #25.

In that case, should it be possible for you to generate unique module names then? Perhaps a serial number, or a hash like what I did with anonymous functions.

I am quite nervous about doing that. It is a lot of work, and feels like I am bumping into a complex problem that Rust itself has solved already.
Just supporting renaming to avoid within-module collision is a lightweight solution to work around common cases and aid procedural generation. If that is not enough, then you can always do what my functional test cases do: create sub-modules for everything, and path::it::all::the::way::down when naming the function in the register_fn or exported_module macro calls.

I suppose abc123::func_entry() really shouldn't matter what the module/name is, as long as it is registered like engine.register_fn("do_work", abc123::func_entry). The user only sees the Rhai function name.

Correct.

My view towards generics is: let the user do it. This is the way Rhai is done right now. The engine only recognizes concrete types.

Yes, but my problem is subtler: some generics are possible with register_fn (through type inference) that are not possible with my macro as it is now.
Let me give an example:
struct MagicStream<Read: R> {
    inner: R
}

impl<R: BufRead> Read for MagicStream<R> { /* ... */ }
impl<R: Read> Read for MagicStream<R> { /* ... */ }

impl<R: BufRead> Clone for MagicStream<R> { /* ... */ }
impl<R: Read> Clone for MagicStream<R> { /* ... */ }

IIUC (and I may not, I'm a little fuzzy) you could manipulate this type in Rhai if you hide a concrete version in a Dynamic first. That is, you could do something like this:
engine.register_fn("new_magic_stream",
    |path| Dynamic::from(MagicStream<File> {
        inner: File::open(path).unwrap()
    }
);
engine.register_fn("read_from", |stream: &mut SpecialReader<_>| stream.read());

See that underscore? That's the part I can't replicate. If you tried to write that with my macros:
#[rhai::export_fn]
pub fn new_magic_file_stream() -> MagicReader<File> { /* Ok */ }
#[rhai::export_fn]
pub fn read_from<R: Read>(stream: &mut SpecialStream<R>) -> String {
    /* BOOM: cannot resolve type 'R' */
}

And that boom, by the way, is not one I can guide the user to. It will look like the macro blew up, rather than the truth: "you, the user, made a very subtle mistake to confuse Rust's type inference, and I can't explain it to you."
Whenever you have talked about use of generics in Rhai, based on the code I have seen that is basically the case I feel I need to support. And right now, that support is ugly.
I am considering creating a new undocumented attribute, generics_maybe or something, to add partial support if you are willing to play with it until it does what you want. But I don't think it should ever be a user facing feature, due to the error paths and pitfalls.

engine.register_fn("read_from", |stream: &mut MagicReader<_>| stream.read());

See that underscore? That's the part I can't replicate. If you tried to write that with my macros:

I don't think it works even now.  Rhai requires concrete types, so it'll complain that the underscore must be given a type.
I just tested it and it generates an error, at least on stable.

Specifying the generic parameter works.

Interesting. I could have sworn I had a closure in a (non-Rhai) project be that smart recently by matching the method signature on monomorphization of a bound... but I suppose it's also possible that Rhai is doing more stuff with the context, so that could break connections.
Since I am 99% sure that MagicReader<File> works already, I'll not worry about generics parsing until you actually manage to create a confusing error. And even then, based on your commentary earlier, I'll just make it a more intelligible error pointing to user code.

Rhai's register_fn has some severe restrictions, being that the closure must be 'static, meaning that it cannot be anything more than function pointer.  Thus, there is nothing to infer the type from.
Your example probably passes a closure capturing some type from outside that is used to infer the generic type parameter.

@schungx, you posted a table of how I could optimize my code a little better, for example avoiding casts depending on how things are called things. Could you please paste that here? I can't find where you put it...

Here it goes:
| Argument type                  | Access (`n` = argument position)      | Result                                                     |
| ------------------------------ | ------------------------------------- | ---------------------------------------------------------- |
| [Primary type][standard types] | `args[n].clone().cast::<T>()`         | Copy of value.                                             |
| Custom type                    | `args[n].read_lock::<T>().unwrap()`   | Immutable reference to value.                              |
| Custom type (consumed)         | `std::mem::take(args[n]).cast::<T>()` | The _consumed_ value.<br/>The original value becomes `()`. |
| `this` object                  | `args[0].write_lock::<T>().unwrap()`  | Mutable reference to value.                                |

It is actually in https://schungx.github.io/rhai/rust/register-raw.html

@schungx, I'm looking at the table you pasted, and trying to figure out how I can improve my code. Particularly, I'm looking at rows 1 and 3, in light of my procedural macro's limitations.
As I have explained elsewhere, I cannot know anything about types the user provides at macro generate time. All I can do is paste the user's types into certain contexts, which will then cause type inference to infer traits on them at compile time. The result will either be code that type checks, or throws a type checking error if the user did not follow the rules.
This is why I use helper methods on Dynamic to do the actual conversion logic. Any trait bounds they have are simply "passed up" to the user, and are expected to always unwrap() successfully if they compile. (The signature registered in Rhai will ensure the args always match the user function's argument types.)
The two helpers I currently use are Dynamic::downcast_clone<T> if the function wants to own an input (i.e. treating everything as a "primary type" in the table) and Dynamic::write_lock<T> for inputs of &mut T (as your final row suggests).
So it seems to me, in order to make any optimization of moves, I would need to write a downcast_to_owned that would do two different checks:

If the type implements Default, then take() it.
If the type implements Clone, then clone() it.
Otherwise, return None. You can't get an owned T from a &mut T on this type.

While I could write code for this that "works", I think it would be a setback in usability. Now, the user could make a type error and it wouldn't show up until runtime. That feels very "un-Rust" to me.
What are your thoughts on this?

Well, all types used in Rhai needs to be Clone anyway, so at least you can clone.
However, all arguments passed to functions are already clones, and they are never used afterwards, so you can simply take them.  It doesn't have to be Default because you're taking a Dynamic which already has a default.  Just take, then cast to the type.  The type is guaranteed to be correct.  The argument is guaranteed to be already cloned, so you do not need to call downcast_clone.  That'd be double cloning for no purpose.
For primary types, it might be a little faster to get a reference and then clone it again because cloning a primary type that is Copy is very cheap.  It'd be cheaper than take-ing the Dynamic and then casting it.  However, if you don't know the type, or if you cannot detect that the type is Copy, then don't bother.  Just take it.  The args[n].clone().cast::<T>() route is an optimization for Copy types only.  It is not necessary.
So std::mem::take(args[n]).cast::<T>() should be all you need.
Except for one exception: The &mut in the first argument.  If there is a &mut, you are supposed to mutate that argument, not taking it.  Therefore, you simply need to use write_lock on that argument.  Or read_lock if you don't really intend to mutate that argument (like a len method), but only want to avoid cloning.
So, in summary:

All value arguments: args[n].clone().cast::<T>() on all

&mut, then all value arguments: write_lock/read_lock on first, args[n].clone().cast::<T>() on rest

So std::mem::take(args[n]).cast::<T>() should be all you need.

Oh, right! The T: Default bound is not on the type, but on Dynamic itself. Got it.
Implemented easily in #31.

Now I am looking into the no_std testing hole mentioned in #31. I think I have an approach, but there is a quirk with the RegisterFn trait that is blocking me.
If I create a new project and put this into src/main.rs:
use rhai::{FLOAT, RegisterFn};

fn distance_function(
        x1: FLOAT,
        y1: FLOAT,
        x2: FLOAT,
        y2: FLOAT,
    ) -> Result<rhai::Dynamic, Box<rhai::EvalAltResult>> {
        Ok(rhai::Dynamic::from(
            ((y2 - y1).abs().powf(2.0) + (x2 - x1).abs().powf(2.0)).sqrt(),
        ))
}

fn main() {
    let mut engine = rhai::Engine::new();
    engine.register_fn("euclidean_distance", distance_function);

    assert_eq!(
        engine.eval::<FLOAT>(
            r#"let x = euclidean_distance(0.0, 1.0, 0.0, 42.0)); x"#
        ).unwrap(),
        41.0
    );
}

Then I get this error:
rror[E0277]: the trait bound `rhai::Engine: rhai::RegisterFn<fn(f64, f64, f64, f64) -> std::result::Result<rhai::Dynamic, std::boxed::Box<rhai::EvalAltResult>> {distance_function}, _, _>` is not satisfied
  --> src/main.rs:16:12
   |
16 |     engine.register_fn("euclidean_distance", distance_function);
   |            ^^^^^^^^^^^ the trait `rhai::RegisterFn<fn(f64, f64, f64, f64) -> std::result::Result<rhai::Dynamic, std::boxed::Box<rhai::EvalAltResult>> {distance_function}, _, _>` is not implemented for `rhai::Engine`
   |
   = help: the following implementations were found:
             <rhai::Engine as rhai::RegisterFn<FN, (), RET>>
             <rhai::Engine as rhai::RegisterFn<FN, (A, B, C, D, E, F, G, H, J, K, L, M, N, P, Q, R, S, T, U, V), RET>>
             <rhai::Engine as rhai::RegisterFn<FN, (B, C, D, E, F, G, H, J, K, L, M, N, P, Q, R, S, T, U, V), RET>>
             <rhai::Engine as rhai::RegisterFn<FN, (C, D, E, F, G, H, J, K, L, M, N, P, Q, R, S, T, U, V), RET>>
           and 37 others

error: aborting due to previous error

Your book mentions a get_any_value example and implies (without a demo) I should be able to register this function.
Am I doing something wrong, or is there a bug?

Are you compiling with no_std?
In that case you should import alloc::boxed::Box?

Ah, that explains it. It will mean extra work, but I can work with that.
Basically, I am working on making the register_exported_fn! accept either a Module or an Engine instance as its first parameter. Aside from improving consistency and making things more flexible (at the expense of more ugly generated code humans won't have to look at anyway), it will let me write a functional test that does not require importing Module, and thus, works with no_std.

Making it seamless with Module or Engine both will be wonderful!
I think normally people will want to do it with Engine for one-off cases, and Module if they want to make something that can be loaded multiple times.

Also, don't forget to remove the checking of number of arguments.  The wrong arg count error is not needed.

OK, @jhwgh1968 , I've done some experimentation and move some functions from the BasicMathPackage into plugins.  See the commits I uploaded.
Note: There is also a reference to rhai::FnAccess that is preventing set_exproted_fn! from being used in internal code.
What I found:

Need a way to register functions that return Result.  A lot of math code may return overflow errors.

Need a way to generate functions of the same name (diff parameters) in the same module via export_module!.  #[name = XXX] will do just fine.

Need a way to generate functions of operators which are not valid operator names.  #[name = XXX] will do just fine.

Note: There is also a reference to rhai::FnAccess that is preventing set_exproted_fn! from being used in internal code.

It is "required" to import rhai::plugin::* at scope where the macro is invoked. I just fixed one test file that was missing it in #33.

Need a way to register functions that return Result.  A lot of math code may return overflow errors.

If you are importing the functions to an engine directly, that is the return_raw attribute.
I am currently working on a way to tag functions within modules with such extra parameters.

Need a way to generate functions of the same name (diff parameters) in the same module via export_module!.  #[name = XXX] will do just fine.

Once I add the ability to put parameters onto functions within modules, I think the existing name flag will be enough.

Need a way to generate functions of operators which are not valid operator names.  #[name = XXX] will do just fine.

Could you give an example? I'm not quite sure what you mean by this.

Is there a way to have sub-modules inside one single #[export_module]?

Currently, there isn't. It has always been on my list, but I'm currently work on functions-within-modules at the moment. I think that will cause refactors that make sub-modules easier. (Right now, they are really ugly.)

Could you give an example? I'm not quite sure what you mean by this.

For example, I may want to register a function inside a module as "+" to provide an operator.  This is not a valid function name.
Sorry, by valid operator names I mean valid function names.

It is "required" to import rhai::plugin::* at scope where the macro is invoked. I just fixed one test file that was missing it in #33.

However, I can't import the rhai namespace.  It'll be crate::plugin in internal code in Rhai.  There is no rhai module.

For example, I may want to register a function inside a module as "+" to provide an operator. This is not a valid function name.

I see. I'll see what I can do.

However, I can't import the rhai namespace. It'll be crate::plugin in internal code in Rhai. There is no rhai module.

That should be fine.

Changing rhai::FnAccess to just FnAccess and I can just use crate::parser::FnAccess to satisfy.
BTW, I'll upload some commits, most of them syncing with master.  I've also made the time package mostly using plugins, and in the process I have the following additional request:
Add attributes to register a function as a getter or setter.  make_getter and make_setter can be imported from engine.rs.  This way, I can use plugin syntax to register getters/setters.
There will be additional checkings involved.  For example, a getter has only one &mut parameter, while a setter only has two parameters.

@jhwgh1968   I've rewritten a lot of the packages in plugins.  You may look at them.
You can see that sometimes things get a bit clumpsy, especially with regards to not having attributes on module functions, and the inability to name an operator.

Yeah, I would expect that would be a limitation. Hopefully it will be easy to revise when I get that implemented.
By the way, your commits broke my codegen tests for compiler error messages, since they cause the Rhai package to emit warnings when compiled. I will fix this round of issues in my next PR, and fix CI to let you catch this easier.

Yeah, I would expect that would be a limitation. Hopefully it will be easy to revise when I get that implemented.

From my actual usage, an entire module is the best.  Function-by-function works fine and is flexible, but there is a lot of boilerplate.  You first need to generate the functions, then import them one by one.

By the way, your commits broke my codegen tests for compiler error messages, since they cause the Rhai package to emit warnings when compiled. I will fix this round of issues in my next PR, and fix CI to let you catch this easier.

Ah.  Sorry.  Feel free to fix those warnings.  I've tried to make sure I generate none, but obviously I missed some.

Playing with it some more, some suggestions:

It would be very useful to have a module macro that doesn't create a new module, but adds the functions into an existing module.  For example exported_module_add!(m, some_module).  Right now, I have to create a number of modules just to throw away after merging them.  With this macro, it will be very easy to concat functions.

It would be very useful to treat pub use some_other_mod::*; in a module.  some_other_mod is another #[export_module] module.  This way, modules can be merged together, and just imported once.

The way to do both of these is probably to make rhai_module_generate take a &mut Module parameter.  pub use sub modules can simply call this recursively.
exported_module! can first create a module and pass to this function:
let m = { let mut mm = Module::new(); some_module::rhai_module_generate(&mut mm); mm };

It would be very useful to have a module macro that doesn't create a new module, but adds the functions into an existing module.  For example exported_module_add!(m, some_module).  Right now, I have to create a number of modules just to throw away after merging them.  With this macro, it will be very easy to concat functions.

I was originally planning to handle this by better supporting recursively scoped sub-modules. That is, if you did:
#[export_module]
pub mod foo {
    pub mod bar {
        pub const MAGIC: INT = 42 as INT;
    }
}

Then by default you would be able to access foo::bar::MAGIC from a script. This doesn't work today because sub-modules have to be special cased within the generate function created by the macro, and I have simply not written that code yet.
All that said, I think that would not be a difficult thing to implement if you still need it for other reasons.
Your second point, however, I'm hoping to work around:

It would be very useful to treat pub use some_other_mod::*; in a module.  some_other_mod is another #[export_module] module.  This way, modules can be merged together, and just imported once. [...] merging a module on the declaration level has a benefit: I can hide sub-modules under feature gates, and all this logic stays in the same place.  Otherwise, the feature gates are everywhere.

Suffice it to say, I am a little nervous about creating a way to cross-reference due to the error cases. And while changing macros to use &mut m seems like a simple thing, this would create a bigger token snarl that you might expect. It also might require Rust version 1.46 (which hasn't been released yet), due to a quirk in the way Expr tokens work (which I discovered while writing my macro_unroll test example).
If sub-modules were possible, then I believe that feature gates would work on them without modification. Would that plus the first macro above be a sufficient alternative?

Also, as a general comment since it has now scrolled off the page: this to-do list I am keeping up-to-date. If you find it useful, perhaps clean up the original issue text, and link to it?

Today, I was going to work on supporting nested submodules, which would require significant refactoring. After that, I would do the handling of names that are not valid Rust idents.
When I just looked at the plugins branch and your PR comments, however, it seems you really are eager to get that fixed. Alright, I can do that today instead. I have fetched the branch in its current state, and will use your work as a starting point.
However, if you want me to open a PR for it, please either fix the CI or delete the commits that broke it.

When I just looked at the plugins branch and your PR comments, however, it seems you really are eager to get that fixed. Alright, I can do that today instead. I have fetched the branch in its current state, and will use your work as a starting point.

Yes, I would pick fixing the naming as the highest priority, because that's preventing a large amount of work from being moved to plugins.  Sub-modules is a nice to have, but can be solved simply by splitting into separate modules.
However, I can say making feature gates #[cfg(...)] work inside modules is a higher priority than sub-modules, because mostly sub-modules are used to conditionally include functions based on feature gates.  If feature gates work inside modules, then we don't really need sub-modules.

However, if you want me to open a PR for it, please either fix the CI or delete the commits that broke it.

My commits are just an experiment to see if the idea works and whether it simplifies writing libraries.  I wasn't even sure if it is implemented correctly.  Feel free to overwrite it with something that is more consistent.  Nevertheless, I'll fix the CI tests so at least it is a singe unit.
I wonder whether I can just open a "PR" for my own repo instead of always having to commit directly into it so I won't muck things up...  maybe the right thing to do is to open up a new branch?

OK, I've tried fixing the CI, but I hit on a problem with ui_tests.
Some of the outputs don't match simply by having different spans pointing to the error.  For example:
EXPECTED:
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
error: this type in this position passes from Rhai by value
  --> $DIR/second_shared_ref.rs:12:41
   |
12 | pub fn test_fn(input: Clonable, factor: &bool) -> bool {
   |                                         ^^^^^

error[E0425]: cannot find function `test_fn` in this scope
  --> $DIR/second_shared_ref.rs:23:8
   |
23 |     if test_fn(n, &true) {
   |        ^^^^^^^ not found in this scope
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈

ACTUAL OUTPUT:
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
error: this type in this position passes from Rhai by value
  --> $DIR/second_shared_ref.rs:12:41
   |
12 | pub fn test_fn(input: Clonable, factor: &bool) -> bool {
   |                                         ^

error[E0425]: cannot find function `test_fn` in this scope
  --> $DIR/second_shared_ref.rs:23:8
   |
23 |     if test_fn(n, &true) {
   |        ^^^^^^^ not found in this scope
┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈

From what I can see, the errors are identical except for the position of the arrows.  I am not sure if this is a difference with the compilers we are using.
I hesitate to fix them just to find out that it is failing for you...

I wonder whether I can just open a "PR" for my own repo instead of always having to commit directly into it so I won't muck things up... maybe the right thing to do is to open up a new branch?

I think PRs against your own repo are the best way to go. It would not only run CI, but let me review it.
I will update you on my progress below, which will include comments that I would have put in a code review.

From what I can see, the errors are identical except for the position of the arrows. I am not sure if this is a difference with the compilers we are using.

Yes, that is another small difference in macro behavior between the current stable and nightly. I was going to warn you, but I couldn't find the time to mention it. I guess this is it! :smile:
I'm not 100% sure what causes it, but I wrote my tests to enforce the more "readable" behavior of nightly, where the entire parameter type is highlighted. That is the token span I'm really interested in making sure is preserved, when I am coding away.
However, this error does not affect CI. I specifically set up the Codegen Build job to only use nightly so that it would match the expected output.
I was planning to add a "trybuild" feature at some point, which would be on by default, and you could disable if you were going to do codegen library development on stable. I guess that day is here.

Now, an update on the change in name behavior.
It turns out, your code didn't need any changes. When I open my PR, you will find your commit cherry-picked as-is.
The main issue I had with your change was that the tests were not updated for the behavior change you made. At a minimum:

One of the tests in test_functions is now invalid, since #[export_fn(name = ...)] now does nothing
A new test should be written in test_modules, which replaces that test, by putting everything inside a module to check the renames

I have already fixed this in my soon-to-be PR. But then, I hit a second issue. This demonstrates why macro development is so test heavy.
When I cherry-picked your second commit which updated Rhai to use #[rhai_fn(name = ...)] in the new way, it compiled... but some very strange test failures popped up. As I messed around with features, or tried to comment out specific things to see what other tests did, this snowballed more and more!
All of them were along the same lines: a function called in a script didn't do what it was supposed to, didn't return what it was supposed to, or didn't return the type it was supposed to.
Based on that, I thought: "Hmm... perhaps when @schungx did those renames, he created some name conflicts in his modules by accident, and functions overwrote each other. This tells me that I should implement rename collision detection, and throw an error on that."
So that's what I am working on now.

I think PRs against your own repo are the best way to go. It would not only run CI, but let me review it.

Do you know how to do that? I searched a bit and didn't come up with anything.  Seems like I need to create a branch...

Do you know how to do that? I searched a bit and didn't come up with anything. Seems like I need to create a branch...

Oh! Yeah, sorry, I misread your question.
My suggestion is, create a plugins_dev branch, and then open pull requests against plugins from that. If you go to Pull Requests and click New Pull Request, you can manually select plugins as the destination and plugins_dev as the source of the PR.

However, keep my changes to the packages files. They are already written to take advantage of the new naming arrangement as an experiment. You can use them as a first test to make sure that everything works as planned.

I am planning to do that. However, that is what caused my issues.
... Actually, I just realized I don't have the very last commits that made the CI green. One of them made changes to the string package. I wonder if that will fix the issues I'm seeing?
Even if it is, though, I still think it suggests rename duplicate detection is a good idea. I will open my PR once I am done with that.

... Actually, I just realized I don't have the very last commits that made the CI green. One of them made changes to the string package. I wonder if that will fix the issues I'm seeing?

Yes I fixed a bug in the Strings package.  It wasn't due to naming conflict, but I put in the wrong operator.

My suggestion is, create a plugins_dev branch, and then open pull requests against plugins from that. If you go to Pull Requests and click New Pull Request, you can manually select plugins as the destination and plugins_dev as the source of the PR.

plugins_dev created.

FYI... I pushed some commits with a bug fix for method-call detection of plugins.
https://github.com/schungx/rhai/commit/9e7516ba279cc7252f7588552369a74a7df6105e
It was not recognizing method calls from plugins correctly.

plugins_dev created.

I was thinking you would actually open a PR. That is also so I could review it, and point out things like: you should probably write a test for X case.

FYI... I pushed some commits with a bug fix for method-call detection of plugins.

Thanks for catching that. I never would have!

I have opened a PR with duplicate detection, and some test fixes.

I was thinking you would actually open a PR. That is also so I could review it, and point out things like: you should probably write a test for X case.

Yes, I plan to open PR's from plugins_dev to plugins but it was a simple bug fix so I just pushed it in.  Next time I'll open a PR instead.
I tried creating a PR to your fork, but it generates a godzillion commits dating all the way back to history, so I'm not sure if that's a good idea...

I tried creating a PR to your fork, but it generates a godzillion commits dating all the way back to history, so I'm not sure if that's a good idea...

To clarify, I really meant: open a pull request to merge from schungx/plugins_dev to schungx/plugins, and @ me to review them.
You are correct, my repo branches are out of date, because I don't keep my copy in very good shape except for rebasing purposes.

I plan to open PR's from plugins_dev to plugins but it was a simple bug fix so I just pushed it in. Next time I'll open a PR instead.

In fact, I would ask you to open a PR and let me review  any macro change, even if it is small or fixing a bug. Even if I don't have much to say, I would still like a review for two things:

To suggest test updates. While we are both big fans of "hack first, polish second," I think macros need a higher bar. Like the Rust compiler itself, the number of uses and interactions is so large, every "hack" also needs to have tests to guard against future regressions that can be subtle but serious.

To understand what the change affects. As I noted in the last PR, I wrote a rule narrowly because I hadn't figured out all the effects of changing the way the name attribute worked. A review is encouragement and focus for me to sit down and do that. Flipping through a group of commits when I'm thinking about something else is a different mindset.

At the moment, both of these are currently missing for the getter and setter generators you implemented. I don't actually know what that code does at the moment, and I don't know if I will accidentally break it in some subtle way as I continue forward.
I have it on my personal list to back-fill, after I get submodules (including submodule attributes) finished. I would like that list to not get any longer in the mean time.

I updated the current state of the to-do list, and it got me thinking about the eventual release.
Since we are making good progress, my understanding is that this will ship for Rhai 0.19.0. However, like the Rust 2018 edition and some other features in the Rust compiler, this is a big change.
Even if the main parts of Rhai use it, I imagine end users will find new and clever ways to break the macros. They will do things that are wrong that "experts" (you and I) would not do, and cause it to panic! and not tell them why.
While I plan to try a bunch of experiments for negative testing, I know it won't catch even half of the problems. Every time something is called "fool proof", the universe sends a bigger fool to disprove it. :smile:
Given that, how would you suggest we release this? I can think of several ideas:

Just release 0.19.0, and expect to make a lot of point releases before 0.20.0 (which every single macro fix would require, due to cargo dependencies). My only worry is, I don't know how many bugs will show up, or how easy they'll be to fix. If you can fix or help triage them, then maybe this will work.

A "soft release" for a public beta. Cut and tag version 0.19.0, but don't push it to crates.io for a while. Keep the macro documentation in the Rhai book separate. Encourage people to try the new version and report bugs. For my own personal use case of Rhai, and several other people who had early interest in things like game development, this would work just fine.

Release version 0.19.0, but don't document the macro parts at all. Maybe try to feature gate some parts until 0.20.0. I don't think we could feature gate all of it because of the crate split between rhai and rhai_codegen, so this might be more confusing than helpful.

What do you think, @schungx?

I would suggest release 0.19.0 with macros, but before that merge it into the master repo so people can have some head warning to flush out bugs.  But the real bugs do tend to show up after we release to crates.io, so I guess we have to go through that step anyway.
Right now the macros are restricted to implementing the standard library (and some portions are still not ported to macros yet - I'm waiting for support of feature gates in modules).  As long as all the tests pass, that shouldn't affect stuff much.
The section in the documentation on how to create custom packages need to be scrapped and rewritten.  It should be a new complete section on macros and how to use them.  I can start on that, if you don't think the public API is going to change much.  As it stands, it is already extremely useful and quite clean.

I think as it stand right now, plugins is probably ready to merge into schungx:master.
What do you think?

I was thinking you might be able to clean things up more after I merge submodules and #[rhai_mod(...)] attribute support. But if you can make progress on other things on master based on what is already here, then you'll get no push back from me!

The section in the documentation on how to create custom packages need to be scrapped and rewritten. It should be a new complete section on macros and how to use them. I can start on that, if you don't think the public API is going to change much. As it stands, it is already extremely useful and quite clean.

I am confident the API I currently have will expand not much, and what does will be backward compatible. Getting started on that might not be a bad idea.

I was thinking you might be able to clean things up more after I merge submodules and #[rhai_mod(...)] attribute support. But if you can make progress on other things on master based on what is already here, then you'll get no push back from me!

Well, sub-modules are not as useful without feature gates.  The main attraction of sub-modules is to put them under feature gates to include/exclude entire sections of functions at once.

EDIT: I'm presuming "your master", not "the official master" where anyone else could use it. I think it is currently for "internal use only" at the moment.

Yes, I mean schungx : master

BTW, I have run some benchmarks on plugins.  Seems like we are looking at roughly 10% speed regression.
We might want to start fine-tuning the macros to generate code that has less regression.


Add ability to skip check membership on CI

This is a temporary workaround for external PR builds.

Coverage remained the same at 59.856% when pulling 47214f69b595b9cc7b6ab5505deb4be0f658007b on mdharamadas1:skip-member-check into 685140cda8dd67f02451c818ea1df61384c433b0 on vmware:master.


rbac has to be turned off, or the istio-ingressgateway-operator role has to be manually modified

This should work with rbac on without any extra role config.

@Deadleg okay so, the issue that @jhobbs filed should be resolved by now, I think your issue might be specific to EKS as we don't actively test against EKS. If you have the option of deploying on some other kubernetes like AKS, Charmed Kubernetes or Microk8s you will probably get better results.
Eitherway a new issue for this bug on EKS should probably be opened.

Currently with RBAC the following manual step is required:
kubectl patch role -n kubeflow istio-ingressgateway-operator -p '{"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"name":"istio-ingressgateway-operator"},"rules":[{"apiGroups":["*"],"resources":["*"],"verbs":["*"]}]}'
This is documented in our official documentation


Bitbucket Cloud fixes

Addresses some of the bugs in https://github.com/reviewdog/reviewdog/issues/1285

Limit length of Annotation Summaries to 450 characters. The annotation model for Bitbucket server appears to be different.
Delete report before recreating to purge old annotations (to avoid the 1000 annotation limit). This is in line with the Bitbucket Server implementation.
Does NOT show bitbucket error strings to user.
Does NOT limit the number of annotations to 1000 or warn.

[x] Updated Unreleased section in CHANGELOG or it's not notable changes.

Any updates on this PR?

Same question :-)


python311Packages.cyclonedx-python-lib: 6.4.0 -> 6.4.1

Diff: https://github.com/CycloneDX/cyclonedx-python-lib/compare/refs/tags/v6.4.0...v6.4.1
Changelog: https://github.com/CycloneDX/cyclonedx-python-lib/releases/tag/v6.4.1
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 285060 run on x86_64-linux 1

8 packages built:

checkov
checkov.dist
pip-audit
pip-audit.dist
python311Packages.cyclonedx-python-lib
python311Packages.cyclonedx-python-lib.dist
python312Packages.cyclonedx-python-lib
python312Packages.cyclonedx-python-lib.dist


Error when processing image parameter

Trying to use the image config param, I'm receiving the error "VipsJpeg: Premature end of JPEG file"
Do you have any example image file that works well?
Thanks!

@achopijocoder please share your config and the file.

@achopijocoder please share your config and the file.

This is my config for og_image

og_image:
output_dir: "assets/images/og"
image: "/assets/images/test.jpeg"
canvas:
background_color: "#FFFFFF"
header:
font_family: "Montserrat, ExtraBold"
color: "#333333"
content:
font_family: "Montserrat, Regular"
color: "#333333"
domain: "test.com"
border_bottom:
width: 50
fill:
- "#005073"
- "#107dac"
- "#189ad3"
- "#1ebbd7"
- "#71c7ec"

This is the result:

It starts rendering the image but then stops and the area is gray


Remove SipHash, add SHA-3

Plan of action, to be implemented by me, soon...

Don't put sha3 in this crate (yet), but do remove SipHash (no urgency to re-release).
Put the necessary sha3 stuff in routing for now, and depend on that from safe_vault.
Later: assess hash function usage and consider using SHA2/3 everywhere with multihash.

@michaelsproul So i guess this is no longer needed - could be submit a PR then to get rid of sip-hasher thingy from utilities ? You can close this one if you want or make this the PR with the changes to just remove sip hasher.

I've updated this PR to just remove SipHash, the only remaining mention of sha3 is the branch name.


Refactor parseSignatureFromCommitLine

Replace #28849. Thanks to @yp05327 for the looking into the problem.
Fix #28840
The old behavior of newSignatureFromCommitline is not right. The new parseSignatureFromCommitLine:

never fails
only accept one format (if there is any other, it could be easily added)

And add some tests.

The output format is based on the git command, so we can control it.
And we can add other formats easily if we need.

The output format is based on the git command, so we can control it. And we can add other formats easily if we need.

Let's only put this change in 1.22. For the backport, I will still add the legacy format support, to avoid any breaking 😄

I was unable to create a backport for 1.21. @wxiaoguang, please send one manually. :tea:
go run ./contrib/backport 29054
...  // fix git conflicts if any
go run ./contrib/backport --continue


Create toggles for tracking the Sun

This PR implements #196. Right now it's not hooked up to anything that can actually tell whether or not it is currently tracking (like after using the mouse to move) but will turn on and off tracking and styles are affected to reflect that. It shows both a button and a toggle, we can choose which we want.
Some todo's

[x] Perhaps @Carifio24 could either expose the variable needed to determine if it is currently tracking either here or in another PR.
[ ] Pick whether it will use the toggle or the button. (Don't touch other parts of the UI in this PR or else it will cause merge conflicts)

I fixed the merge conflicts on this, but it looks like the zoom level in the sunscope mode is being ignored. @johnarban or @Carifio24, might one of you be able to take a look at this, please?
SunScope on Main looks like this:

SunScope on this branch looks like this:

Both branches specify this.sunPlace.set_zoomLevel(20);

Done!

@johnarban, please review the UI updates I added for the sun tracker toggle and merge if it looks ok to you. Thanks!


Master -> Live

Docs Build status updates of commit e25dd53:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/community.md
:white_check_mark:Succeeded
View

mixed-reality-docs/images/BiographyImages/ClementeGiorio_270x270.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 8dc167a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded
View

mixed-reality-docs/community.md
:white_check_mark:Succeeded
View

mixed-reality-docs/images/BiographyImages/ClementeGiorio_270x270.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 1da340e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/TOC.md
:white_check_mark:Succeeded
View

mixed-reality-docs/community.md
:white_check_mark:Succeeded
View

mixed-reality-docs/images/BiographyImages/ClementeGiorio_270x270.png
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Create missing BusinessService in CSV import

When there is an unknown BusinessService in Application CSV import, it will be created automatically.

There will be an upcoming PR covering this together with createEntities option and TagTypes&Tags creation.

Change introduced in this PR is part of https://github.com/konveyor/tackle2-hub/pull/123