Incorrect layout with large bitfield

Follow up to https://github.com/rust-lang-nursery/rust-bindgen/pull/1001
Input C/C++ Header
struct {
  unsigned : 632;
} a;

Bindgen Invocation
$ bindgen input.h

Actual Results
Panic when running the layout tests: our generated struct ends up with the wrong size.
Expected Results
We generate a struct with the correct layout, and it passes its layout tests.

Here is where we allocate logical bitfields into physical units:
https://github.com/rust-lang-nursery/rust-bindgen/blob/master/src/ir/comp.rs#L432-L617
If we discover the root cause of the issue, and it doesn't appear to be a simple fix, we should investigate falling back to making the whole struct opaque. Perhaps extending impl IsOpaque for Comp with something like:
self.fields().iter().any(|f| match *f {
    Field::Bitfields(ref unit) => unit.bitfields().iter().any(|b| b.width() > 64),
    _ => false,
})

which would make any bitfield that is bigger than a u64 trigger the whole struct being opaque. I think that is the right check to do here, but I'm not sure. Maybe we should be checking the bitfield unit's layout instead.
cc @aeleos

@highfive: assign me

Hey @aeleos! Thanks for your interest in working on this issue. It's now assigned to you!

@fitzgen so when the Item info for the bitfield is being printed out, it outputs this.
ir: ItemId(1) = Item {
    id: ItemId(
        1
    ),
    local_id: Cell {
        value: Some(
            1
        )
    },
    next_child_local_id: Cell {
        value: 1
    },
    canonical_name_cache: RefCell {
        value: None
    }, "correct" behavior for rust
    comment: None,
    annotations: Annotations {
        opaque: false,
        hide: false,
        use_instead_of: None,
        disallow_copy: false,
        private_fields: None,
        accessor_kind: None,
        constify_enum_variant: false
    },
    parent_id: ItemId(
        0
    ),
    kind: Type(
        Type {
            name: None,
            layout: Some(
                Layout {
                    size: 80,
                    align: 8,
                    packed: false
                }
            ),
            kind: Comp(
                CompInfo {
                    kind: Struct,
                    fields: AfterComputingBitfieldUnits(
                        [
                            Bitfields(
                                BitfieldUnit {
                                    nth: 1,
                                    layout: Layout {
                                        size: 128,
                                        align: 128,
                                        packed: false
                                    },
                                    bitfields: []
                                }
                            )
                        ]
                    ),
                    template_params: [],
                    methods: [],
                    constructors: [],
                    destructor: None,
                    base_members: [],
                    inner_types: [],
                    inner_vars: [],
                    has_own_virtual_method: false,
                    has_destructor: false,
                    has_nonempty_base: false,
                    has_non_type_template_params: false,
                    packed: false,
                    found_unknown_attr: false,
                    is_forward_declaration: false
                }
            ),
            is_const: false
        }
    )
}

So there are 2 different layouts being generated, one with size 80, align 8 and the other with size 128, align 128. Do you know where each of them are generated and what each of them means?
Another question I have is that when the header in question is compiled with clang++ it gives this warning
tests/headers/bitfield_large_overflow.hpp:2:12: warning: width of anonymous bit-field (632 bits) exceeds width of
      its type; value will be truncated to 32 bits [-Wbitfield-width]
  unsigned : 632;
           ^

If clang is truncating the value to 32 bits, does that mean the correct behavior would be to treat it as a 32 bit bitfield?

So there are 2 different layouts being generated, one with size 80, align 8 and the other with size 128, align 128. Do you know where each of them are generated and what each of them means?

The former is the type's layout we get directly from libclang, the latter is the layout we compute for the bitfield allocation unit. I linked where the latter was computed in an earlier comment. The former is the result of calling clang::Type::{size, align} from src/clang.rs.

If clang is truncating the value to 32 bits, does that mean the correct behavior would be to treat it as a 32 bit bitfield?

It sounds like we should be performing that same truncation inside the bitfield method in impl FieldMethod for FieldData. We should also warn!("..."), with a similar message that clang has.
I expect this is the root cause and performing the truncation is the correct fix.
Does gcc have the same behavior?

Here is the warning from gcc
tests/headers/bitfield_large_overflow.hpp:2:14: warning: width of ‘<anonymous struct>::<anonymous>’ exceeds its type
   unsigned : 632;
              ^
tests/headers/bitfield_large_overflow.hpp:3:3: warning: anonymous type with no linkage used to declare variable ‘<anonymous struct> a’ with linkage
 } a;
   ^

It doesn't explicitly say that it is truncating, so I am not 100% sure, but I think that is what it is doing.
Another interesting behavior is that this only actually works with c++. Both gcc and clang fail to compile this code if the file is a .h, rather than just throwing an warning.
test.h:2:3: error: width of ‘<anonymous>’ exceeds its type
   unsigned : 632;
   ^

Looks like g++ agrees with clang++ on a size of 80:
$ cat big-bitfield.cpp
struct HasBigBitfield {
    unsigned : 632;
};

int f(HasBigBitfield*);

int main() {
    HasBigBitfield b;
    return f(&b);
}
$ g++ -g -c big-bitfield.cpp -o big-bitfield.o
big-bitfield.cpp:2:16: warning: width of ‘HasBigBitfield::<anonymous>’ exceeds its type
     unsigned : 632;
                ^~~
$ dwarfdump big-bitfield.o
...
< 1><0x0000002d>    DW_TAG_structure_type
                      DW_AT_name                  HasBigBitfield
                      DW_AT_byte_size             0x00000050
                      DW_AT_decl_file             0x00000001 /home/fitzgen/scratch/big-bitfield.cpp
                      DW_AT_decl_line             0x00000001
...
$

Another interesting behavior is that this only actually works with c++. Both gcc and clang fail to compile this code if the file is a .h, rather than just throwing an warning.

I think we can ignore this because libclang will fail to parse + type check the C headers before we even get our hands on it.

@fitzgen Another interesting thing I just found, the actual type assigned to the anonymous bitfield doesn't change the size of the struct.
$ cat big-bitfield.cpp
struct HasBigBitfield {
    unsigned char: 632;
};

int f(HasBigBitfield*);

int main() {
    HasBigBitfield b;
    return f(&b);
}
$ clang++ -g -c big-bitfield.cpp -o big-bitfield.o
big-bitfield.cpp:2:19: warning: width of anonymous bit-field (632 bits) exceeds width of its type; value will be
      truncated to 8 bits [-Wbitfield-width]
    unsigned char : 632;
                  ^
1 warning generated.
$ dwarfdump big-bitfield.o
...
< 1><0x0000002a>    DW_TAG_structure_type
                      DW_AT_name                  "HasBigBitfield"
                      DW_AT_byte_size             0x00000050
                      DW_AT_decl_file             0x00000001 /home/oliver/rust-bindgen/big-bitfield.cpp
                      DW_AT_decl_line             0x00000001
...
$

g++ also has the same behavior. Considering that it specifically says that the value is being truncated, I find it odd that the memory is being allocated to fit the entire bitfield.
I tried a few different combinations of bitfield sizes are the memory size pattern seems to be 0x40, 0x48, 0x50, 0x58, etc. Is it possible to just check whatever memory size libclang outputs and use that as the memory size? But does that mean we are just allocating extra memory for no reason?

The number one priority is to end up with the same physical representation as the C code. If we don't, then passing these structs across FFI boundaries will lead to memory unsafety.
That certainly is odd behavior. I think the best thing to do is to treat the whole struct as opaque if any bitfields are wider than its type. So we would add something like this to impl IsOpaque for Comp:
self.field().iter().any(|f| match *f {
    Field::DataMember(_) => false,
    Field::Bitfields(ref unit) => unit.bitfields().iter().any(|bf| {
        bf.width() / 8 > ctx.resolve_type(bf.ty()).layout().size()
    })
})

(Probably with a comment explaining why we're checking bitfield widths there and linking back to this issue)

@fitzgen sorry about getting back to you so late, I have been busy the past few days. Do you mean add it to impl IsOpaque for CompInfo? I can't find anything other than that.

Yep!
$ git grep "impl IsOpaque"
src/ir/comp.rs:1455:impl IsOpaque for CompInfo {
# ...

@fitzgen so I tried to implement something, but there are a few issues with it. This is my new is_opaque function
    fn is_opaque(&self, ctx: &BindgenContext, _: &()) -> bool {
        //Early return to avoid extra computation
        if self.has_non_type_template_params {
            return self.has_non_type_template_params
        }

        let has_large_bitfield = self.fields().iter().any(|f| match *f {
            Field::DataMember(_) => {
                false
            },
            Field::Bitfields(ref unit) => {
                    println!("Bitfield Unit: {:?}", unit);
                    unit.bitfields().iter().any(|bf| {
                    let bitfield_layout = ctx.resolve_type(bf.ty())
                        .layout(ctx)
                        .expect("Bitfield without layout? Gah!");
                    println!("Bitfield width: {}", bf.width());
                    bf.width() / 8 > bitfield_layout.size as u32
                })
            }
        });

        println!("Does have large bitfield: {}", has_large_bitfield);

        self.has_non_type_template_params || has_large_bitfield
    }

Right now when it runs with the failing test case, I get this as the output
Bitfield Unit: BitfieldUnit { nth: 1, layout: Layout { size: 128, align: 128, packed: false }, bitfields: [] }
Does have large bitfield: false

I am not sure how the BitfieldUnit struct is supposed to work, but there is nothing inside the bitfields array if in this case. Since we can't see how large the bitfield is in any way other than the layout according to this, what do you think is the best thing to to do?
I did also notice that this line is output
ERROR:bindgen::codegen::struct_layout: Calculated wrong layout for _bindgen_ty_1, too more 40 bytes
It seems that there is at least one point where the issues with the size and layout are found, is it possible to just copy that functionality and if it has been miscalculated then we make the struct opaque?

This is my new is_opaque function

That looks like what I'd expect, although
        if self.has_non_type_template_params {
            return self.has_non_type_template_params
        }

should be
        if self.has_non_type_template_params {
            return true;
        }

and then we don't need to check self.has_non_type_template_params again at the end.

I am not sure how the BitfieldUnit struct is supposed to work,

Pretty much what it says on the tin:
/// A contiguous set of logical bitfields that live within the same physical
/// allocation unit. See 9.2.4 [class.bit] in the C++ standard and [section
/// 2.4.II.1 in the Itanium C++
/// ABI](http://itanium-cxx-abi.github.io/cxx-abi/abi.html#class-types).

Logical bitfields get allocated into physical units. The units end up as a field in the resulting struct, and the bitfields become getters/setters that are masking out everything but their own relevant bits from the unit.

but there is nothing inside the bitfields array if in this case.

And I think this is because we eagerly filter out anonymous bitfields when computing which bitfields end up in which units. We're going to have to stop doing that filtering to make these changes work.
https://github.com/rust-lang-nursery/rust-bindgen/blob/master/src/ir/comp.rs#L583-L589
Sorry I didn't remember this earlier.
Removing the eager filtering, removing the Bitfield::name(&self) -> &str method in favor for of the FieldMethods::name(&self) -> Option<&str> method and updating callers (mostly in codegen) to deal with and ignore anonymous bitfileds could be its own little PR. After that was done, it would make circling back to this issue's changes easier.

I did also notice that this line is output
ERROR:bindgen::codegen::struct_layout: Calculated wrong layout for _bindgen_ty_1, too more 40 bytes
It seems that there is at least one point where the issues with the size and layout are found, is it possible to just copy that functionality and if it has been miscalculated then we make the struct opaque?

That would be desirable, but is a bit orthogonal to this issue, IMO. It also depends on https://github.com/rust-lang-nursery/rust-bindgen/issues/959. If you want to help out, it would be very appreciated :) But I don't think it needs to block this issue.

Does that help clear things up? Anything not make sense? Feel like you know where to go from here?
Cheers! 😸

@fitzgen I think I get whats happening, here is what I am thinking are the next steps

Remove the Bitfield::name(&self) -> &str method and replace all calls with the FieldMethods version
Remove the if statement around checking if a bitfield is anonymous before adding it into the array
Check to make sure these changes fix the new is_opaque function
Submit PR for the above changes referencing this issue
Submit PR for this issue with new is_opaque function and some tests to make sure everything is working

That seems like everything does that sound right to you?

That sounds 💯 !

@fitzgen so the first issue I have run into is with this line
let param_name = bitfield_getter_name(ctx, parent, bf.name());
What is the best way to handle an anonymous bitfield name here? Is there a function that a name for anonymous fields? Or can it just be set to something random if its going removed when the struct is made opaque later?

I code generation, we can simply skip bitfields that don't have names. The allocation units are always emitted, so skipping the bitfield won't cause the struct layout to go wrong, unlike normal data members.


Investigate using MAST for Plutus Core

MAST (Merkelized Abstract Syntax Trees) might save us a lot of space, since it allows us to prune "unused" parts of the program before sending it.
We should investigate how big the savings are.

Broad implementation plan:

Merkelized version of PLC AST: with hashes at each node.
Prunable, merkelized version of PLC AST: nodes can be ommitted and only the hash provided, evaluation, typechecking etc fail if they try to access a missing node.
Pruning: evaluate a (already merkelized?) AST, track which nodes are untouched, prune them.
Compare vs other approaches:

Try pruning just terms, compare to not pruning, maybe compare to compression.
Try pruning type nodes, compare to type erasure.
Try pruning name nodes, compare to name stripping.

Seems like there's some programming niceties to be done if we want to do this in a nice composable way, but for an experiment we could probably just alter the PLC AST so it's always merkelized.

MAST references:

https://blockstream.com/simplicity.pdf
http://www.mit.edu/~jlrubin/public/pdfs/858report.pdf

MPJ thoughts:

Need to make sure that the hash of the non-merkelized PLC AST is the same as of the merkelized one.

Means we probably can't keep doing our current hashing strategy, which is to just hash the serialized bytes (I think?).

Hashes take space, need to be careful we don't make things worse!

For the prunable merkelized AST we probably want to only keep the hashes for nodes that are pruned - the others can be recomputed.
We could also just not prune nodes that are < hash size (when serialized...).

Maybe we should benchmark evaluation speed too? I don't expect it to make much difference, but it would be nice to check.

As an experiment I tried adding a 32-byte SHA256 hash to every node in the ASTs for the sample contracts, including types and names.  That increased the total size by a factor of 15-17, taking all of the contracts well over 1GB (and some to almost 3GB).  I don't think we want to do that.
However, I'm not sure why you would want to do that: either you trust the AST, in which case you don't need the hashes, or you don't trust it, in which case you'll have to recompute them all anyway in order to check that the overall hash is the one you're expecting.
My inclination is to introduce a new term Prune <hash of pruned subtree>.  I think that would allow you to compute the hash of the complete tree by computing hashes for the parts that you're actually going to run and collecting the hashes of the missing parts from Prune nodes.  During execution the evaluator should fail if it ever runs into a prune.
A possible downside to the Merklisation idea is that it may save space but it would probably require a great deal of on-chain hash computation, which could be expensive.  We may have to do some measurements to see how increased computation costs compare with reduced storage/transmission costs.
I think we probably want to discard types and textual names for on-chain evaluation: the figures at https://github.com/input-output-hk/plutus/blob/master/docs/plutus-core/Erasure-Merklisation.md show that this gives significant space savings and I'm not sure that there's any convincing reason to keep them, since you can very easily compare an erased on-chain AST with a full off-chain one to make sure that the former was obtained from the latter.  Maybe a problem is that we'd then have two versions of each validator, and somewhere it would have to be specified exactly what was going on.

My inclination is to introduce a new term Prune . I think that would allow you to compute the hash of the complete tree by computing hashes for the parts that you're actually going to run and collecting the hashes of the missing parts from Prune nodes. During execution the evaluator should fail if it ever runs into a prune.

Yes, exactly. We indeed don't need to have hashes for things we can recompute!

I think we probably want to discard types and textual names for on-chain evaluation

I think erasure is still in the air. Manuel still wants to try the version where we merkelize away the types (although I think this will make things worse). Ideally I'd like to get a picture of all of these operations as they interact with each other! (Although that's a combinatorially increasing amount of work...)

The Marlowe folks pointed out to me that we could also Merkelize Data, which potentially saves us a bunch of space if it gets big. Slightly unclear how to wire this up, though.

See https://github.com/input-output-hk/plutus/tree/master/docs/plutus-core/Merklisation for the results of some experiments.

I think I'll close this now.  Conclusion: simplifying and compressing the AST is helpful, Merklisation isn't.
It might be worth almalgamating the results into a single PDF now that they're complete.


Redirect the old Maven coordinate to the new one

Hi. Maven has a relocation mechanism which can be used for repositories: https://maven.apache.org/guides/mini/guide-relocation.html . The problem is that Gradle, as far as I know, does not support such a mechanism, so I am not sure it can actually be used automatically for Gradle plugins. It also requires a tombstone release to the original destination

Originally posted by @oleg-nenashev in https://github.com/renovatebot/renovate/discussions/30676#discussioncomment-10320920

But what, I just realized that Shadow plugin was only published to Gradle Plugin Portal, so we can't relocate it from MavenCentral.
The only thing we can do from our side might be https://github.com/gradle/plugin-portal-requests/issues/233.

See https://github.com/gradle/plugin-portal-requests/issues/233#issuecomment-2295907778.


Allow bulk deletion in Epinio from CLI

Issue:
Currently, if we have several applications running on Epinio and want to delete them all, we have to do it one by one through the CLI. This does not happen in the UI where bulk deletion is allowed.
Request:

Allow an user to perform bulk deletion for Applications, Services and Configurations and any other type of actions that may contain a number of items subject to be deleted if no collisions exist.

Other possible considerations:

If a collision or any other issue prevented the deletion of a particular app, service or config, this one could be skipped and continue deleting instead of stopping the process
It would be nice if a description of what has been deleted and unbinded in the process could be displayed; like currently happens when an user deletes an app binded to a service:

➜  epinio git:(main) ✗ epinio app delete testapp1

🚢  Deleting application...
Name: testapp1
Namespace: workspace
..........................................🚢
|       UNBOUND CONFIGURATIONS        |
|-------------------------------------|
| x57adfcee73692eb703d7cbd17e99-mysql |

✔  Application deleted.

Edit:
Converted this ticker into an epic and divided it into these tasks:

Configuration batch deletion: https://github.com/epinio/epinio/issues/1740
Applications batch deletion: https://github.com/epinio/epinio/issues/1742
Service batch deletion: https://github.com/epinio/epinio/issues/1743

Converted this issue into epic as commented offline and split into 3 tasks:

Configuration batch deletion: https://github.com/epinio/epinio/issues/1740
Applications batch deletion: https://github.com/epinio/epinio/issues/1742
Service batch deletion: https://github.com/epinio/epinio/issues/1743

Feel free to split further if needed

When we have the batch deletion endpoints we can implement the "delete all" command, using the batch deletion.


Feature/ng update 9

Story
Exploring an upgrade of Angular to version 9 for use with another project. Direct Message me for information.
Changes

Ran through the angular upgrade checklist
Modified a few lines in the README that did not work when the steps were followed
(Not done) I think the version should be updated if this request is approved

Tests
None to date. I would like to run this through the demo tools and will direct message you regarding that.

Hello Jeremy,
Before looking at your PR, I resolve old PR (the one created by git Bot and relative to security).
And now, I see that your PR cannot be approved due to conflicts (probably libs from previous PRs that shall be sync'ed
Could you have a look ?
Thanks
Nicolas


Support binding Actions to Commands programmatically

I have a ViewModel that has an ICommand property and would like to initialize the binding of that command property to a method using Actions in the ViewModel.
Something like:
public class Page : Screen, INavigationPage
{
    public ICommand Command => (ICommand)(new Action((PublishMyEvent)));
    . . .
    public void PublishMyEvent() => eventAggregator.Publish(new MyEvent());
}

Why not use actions?

I mean yeah, I'm referring to the Actions from this framework. It's just that the examples are done in xaml. Does the framework provide a way to do it in code?

I'm binding the Command property above in a style for an ItemsControl inside a CustomControl.
 . . .
<Style TargetType="{x:Type ItemsControl}">
    <Setter Property="ItemTemplate">
        <Setter.Value>
            <DataTemplate>
                <local:Item Command="{Binding Command}" Icon="{Binding Icon}" />
            </DataTemplate>
        </Setter.Value>
    </Setter>
</Style>
. . .

Since the {s:Action MethodName} doesn't work in Styles, I'm finding a way if Stylet's action can work in code-behind. Something similar to my first comment.

Ah right. You'll have to write your own ICommand implementation. They're pretty straightforward.
On 24 April 2020 03:25:06 BST, DaaWaan<EMAIL_ADDRESS>wrote:

I'm binding the Command property above in a style for an ItemsControl
inside a CustomControl.
. . .

. . .
Since the {s:Action MethodName} doesn't work in Styles, I'm finding a
way if Stylet's action can work in code-behind. Something similar to my
first comment.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
https://github.com/canton7/Stylet/issues/124#issuecomment-618766081


Screen wiggling when adding breakpoint

Select some code so the sparkle button shows up
Press F9 to add a breakpoint
The screen shifts as the breakpoint appears then replaces the sparkle icon a moment later

@aeschli this issue is in relation to the discussion we were having yesterday about the sparkle icon obstructing the way for the breakpoints.
I can fix this issue by disabling the addition of breakpoints when a sparkle icon is not visible in the gutter, this would help us avoid the flickering behavior. I am not sure if this is the best decision here though?

Let's wait until we know if we want the sparkle or if we are going to remove it.

The sparkle in the gutter has been entirely removed from the code base, therefore this issue can be closed.


build-ibmcloud-win2012r2-x64-2: The paging file is too small for this operation to complete.

 Program will exit.
09:41:06  Unhandled exception
09:41:06  Type=Segmentation error vmState=0x00020011
09:41:06  Windows_ExceptionCode=c0000005 J9Generic_Signal=00000004 ExceptionAddress=00007FFDAC9AA423 ContextFlags=0010005f
09:41:06  Handler1=00007FFDAD4FE560 Handler2=00007FFDB53AAAF0 InaccessibleReadAddress=0000000000000030
09:41:06  Unhandled exception
09:41:06  Type=Segmentation error vmState=0x00020011

C: only has 7Gb free, possibly not enough disk space if page file is there

Similar failures seen on test-ibmcloud-win2012r2-x64-1 - https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.openjdk_x86-64_windows/63/consoleFull

This might help resolve: https://github.com/eclipse/openj9/issues/10650

As per the OpenJ9 link, I've changed it from the system automatically setting the page file size, to a custom size, initial size is 3072MB and maximum size is 8192MB - Though this is still on the C:\ drive.
For good measure, I've cleared the machine a bit, by removing the old versions of OpenSSL(I kept OpenSSL1.1.1j and OpenSSL1.1.1k). I also removed a stray workspace folder from the C:/ drive, that hadn't been used since 2020 (besides, we use the E:/ drive for builds on that machine, so I assume it was just a leftover test of the system). That machine has 11.5GB free now, so should be able to accommodate the max page file size.

Grinder: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/529/console

Seems to have worked! Closing issue


Morphing Cube Animation

Description
I have created a morphing cube animation.
issue no.  #566
Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] I have made this from my own
[x] I have taken help from some online resources and mentioned in PR
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code

SCREEN-SHOTS

DEPLOYMENT LINK
https://deploy-preview-590--amazing-css-effects.netlify.app/morphing cube/

@YatreeLadani copied.


Add transparent multicall middleware

Motivation
Concurrent eth_calls are expensive and require many network round trips. Large codebases are difficult to refactor to leverage Multicall contract abstraction.
Fixes https://github.com/gakonst/ethers-rs/issues/2508
Solution
Introduces MulticallMiddleware that maintains a queue of call requests and callbacks. Leverages existing Multicall contract to batch many calls into a single RPC request. Uses tokio mpsc and oneshot pattern for async queueing.
PR Checklist

[x] Added Tests
[x] Added Documentation
[x] Breaking changes

hopefully more in scope than my last middleware contribution https://github.com/gakonst/ethers-rs/pull/2398

design wise, I'm not sure we need a new service for this if we instead put the MultiCall handling into a shared state via some async Mutex, we could probably make it work like that, but I guess a service task also works fine

@mattsse I agree ideally users dont need to think about another service
I'm not actually sure how to achieve this because I don't have great knowledge of the available async primitives but if you give me some guidance I can try refactoring

I think one perf issue here is that executing multicalls is now sequential and no longer concurrent which could be an issue

I'm not sure how we can avoid this and achieve the transparent functionality? I assume anyone who wants to use the middleware would like to eliminate the concurrency on the RPC

for posterity, @mattsse and I discussed offline and decided to punt on some parking_lot::Mutex approach for a followup PR

This looks much more ergonomic. Very neat idea.
I don't like that you have to register all the abi ahead of time though. I don't always know the abis at start but I'd still like to have the calls batched still.
And how does calling for blocks other than latest work? I see some possibly related comments about system transactions but need to read more of the code to tell what those are.

This looks much more ergonomic. Very neat idea.

Thanks! Seems related to your work on https://github.com/llamanodes/web3-proxy

I don't like that you have to register all the abi ahead of time though. I don't always know the abis at start but I'd still like to have the calls batched still. Also, having to register and check the abis like this has a potential for hash collisions.

Knowing the ABIs is necessary to leverage multicall because you need to decode and slice the return data. Collisions are not handled but a more robust implementation could register ABIs for specific to addresses, rather than globally. This registration could be automated whenever a contract instance is connected to a provider with this middleware.

And how does calling for blocks other than latest work? I see some possibly related comments about system transactions but need to read more of the code to tell what those are.

This isnt supported but you could implement this by having separate queues per block number

I did some experimenting today combining some of these ideas with some non-transparent multicall code that I wrote years ago. So far it only handles call and not the other helpers, but it doesn't need abis and it supports setting the block.
setting the block i think is rather crucial. Otherwise the transparent nature of the middleware could lead to very surprising results.
I also want to automatically add state overrides for the contract if it's used on a block that predates the contract being deployed.

Here's a gist of what I did: https://gist.github.com/BlinkyStitt/837ffdbee2892c8857d192a5f4604b9c


How to fix unclose SSLSocket warning on replicate.run()

When I close pyton server (using ctrl+c), replicate.run() returns following warning message on console
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=404, family=2, type=1, proto=0, laddr=('<IP_ADDRESS>', 4488), raddr=('<IP_ADDRESS>', 443)>
I have reviewed replicate API for any available function to close SSLSocket but didn't find any function. Here is sample code:
    input = {
                "image": image_object,
                "prompt": "describe this image"
            }

    output = replicate.run(
        REPLICATE_API_URL,
        input = input
    )

    return_data = ("".join(output))
    return return_data

Any suggestion how can I fix this warning?

Hi, @adnanahmad84. That's expected behavior, or at least I don't think that you should be concerned by that warning. You may be able to silence that warning by wrapping your client usage with the closing context manager.
from contextlib import closing
with closing(replicate.default_client) as client:
     replicate.run(...)


cmd/compile/internal/gc: TestDeps failing

$ go test cmd/compile/internal/gc
--- FAIL: TestDeps (1.24s)
    dep_test.go:23: undesired dependency on "go/build"
    dep_test.go:23: undesired dependency on "go/token"
FAIL
FAIL	cmd/compile/internal/gc	40.383s

I noticed this on the current release-branch.go1.13 branch, but it might be a problem on master too.
/cc @andybons
I don't know how serious of a problem this is, we should find who has more context on this test. It's new to 1.13.

Test was added in CL 172997 to fix issue #31563. /cc @josharian @bradfitz
It seems like a very minor optimization, so not a release blocker.
Additionally, this might be a false positive. The test runs exec.Command("go", ...), which I suspect picked up Go 1.12.9 in my PATH instead of the release-branch.go1.13 version I was running tests on.

It was a false positive. The test needs to be improved.
I can reproduce on master branch:
$ ./make.bash
[...]
$ ../bin/go version
go version devel +3c56143442 Tue Sep 3 17:10:37 2019 +0000 darwin/amd64
$ ../bin/go test cmd/compile/internal/gc
--- FAIL: TestDeps (0.16s)
    dep_test.go:23: undesired dependency on "go/build"
    dep_test.go:23: undesired dependency on "go/token"
FAIL
FAIL	cmd/compile/internal/gc	9.685s
FAIL
$ go version
go version go1.12.9 darwin/amd64
$ PATH="$(pwd)/../bin:$PATH" go version
go version devel +3c56143442 Tue Sep 3 17:10:37 2019 +0000 darwin/amd64
$ PATH="$(pwd)/../bin:$PATH" go test cmd/compile/internal/gc
ok  	cmd/compile/internal/gc	7.000s

/cc @hajimehoshi FYI.


Make bundling work with custom output location

It seems currently bundling can not work properly for the generated Prisma Client when it is generated to a custom output location. Problem: No package.json generated in custom output path because of if in code, means no information for Bundler to know what file(s) to include.
Next steps:

Investigate if webpack can pickup package.json files in custom locations (out of node_modules)

(cc @Frankitch @VincentThomas06)

Hi @millsp  @VincentThomas06, I opened an issue a couple of weeks ago (#12296) which has a similar problem but browser side. This issue can be solved by adding a package.json file. Knowing that I have already tried to add a package.json file to solve this current issue but I did not succeed, I hope I did something wrong...

I have a project with multiple Prisma schemas, there is a default one and then one that outputs to ./node_modules/@internal/.
We are constantly getting this error, happens around 50% of the time. Re-running our script to run all of the code generators fixes it. It seems like some sort of race condition.
ERROR in<EMAIL_ADDRESS>27652:17-40
Module not found: Error: Can't resolve '_http_common' in '/Users/jribakoff/groundwater/node_modules/@internal/prisma/client/runtime'

Investigate if webpack can pickup package.json files in custom locations (out of node_modules)

I would like to respectfully question if ya'll pinned down the exact issue, because I am not outputting anything outside of node_modules, furthermore the fact it works sometimes and not others does not seem to jive with your understanding of the issue

@joshribakoff-sm what versions of prisma and @prisma/client are you using? _http_common issue was solved in 3.12  and as of 4.1.0 I can't see this import in prisma runtime code. If you are using older version, please, update. If you see this issue on a current version I'd kindly ask you to submit a reproduction.

Just a little comment to say we have also encountered this issue where we want to generate our client to a known location rather then the automatic prisma behaviour. This is driven by us needing to use two prisma clients in our project for different databases.
By outputting to a folder outside node_modules in our next.js app it caused a Can't resolve 'util/types' error in the build as has been mentioned in these issues.
For now I have resolved using this workaround and published to two known locations within node_modules that we define but I know in future this won't play nice with mono repo's and pnpm etc 😔

We generate a package.json now and it is correctly picked up by bundlers.
If you still have a related problem, please open a new issue or discussion. Thanks!


Preview not working

I hope im just missing somthing here, but the image preview in your image saver does not appear to be working.

It's because comfy doesn't support saving outside of the output directory.
Now for some reason this custom node still manages to save the file. Default SaveImage will refuse:

Afaict there isn't anything I can do about it on the extension side.
If you save within the output dir, you should see the preview.


OrthographicFrustum deprecated properties are removed.

fixes #5109

Thanks @ateshuseyin!  We don't want to merge this until the 1.33 release, so we'll wait until after the 1.32 release goes out on Monday.
@bagnell does this look good to you?

Can you also remove all references to the _useDeprecated property as well as the required deprecationWarning? Otherwise, this looks good.

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a temporary error. The following address(es) deferred:
<EMAIL_ADDRESS>Domain biomassiv.es has exceeded the max emails per hour (106/100 (106%)) allowed.  Message will be reattempted later
------- This is a copy of the message, including all the headers. ------
Received: from github-smtp2-ext3.iad.github.net ([<IP_ADDRESS>]:43854 helo=github-smtp2b-ext-cp1-prd.iad.github.net)
by chi-server32.websitehostserver.net with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.88)
(envelope-from<EMAIL_ADDRESS>id 1cy1L2-000BId-D8
for<EMAIL_ADDRESS>Tue, 11 Apr 2017 14:20:28 -0500
Date: Tue, 11 Apr 2017 12:19:49 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com;
s=pf2014; t=1491938389;
bh=CBxneXDKCbQet/dyGjXuqsHeq7Z3Gf0S/d+bHd+2G7E=;
h=From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID:
List-Archive:List-Post:List-Unsubscribe:From;
b=MhVOwJj4q32D4ZcFdvQKYlyI7hW36fBSM0dlenMICHAILhF/+ElWnmoHk/t8qooFq
sboNhshPaSIlxfgWFyqqnf73EUYrA45IlVMIm85u6rdzdwKJDaB27uqGy6y7urOetX
2pqRg6F9Fr+QARnRDxsuGDsG4/8SzmIByx6NVQCM=
From: Ed Mackey<EMAIL_ADDRESS>Reply-To: AnalyticalGraphicsInc/cesium<EMAIL_ADDRESS>To: AnalyticalGraphicsInc/cesium<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Message-ID<EMAIL_ADDRESS>In-Reply-To<EMAIL_ADDRESS>References<EMAIL_ADDRESS>Subject: Re: [AnalyticalGraphicsInc/cesium] OrthographicFrustum deprecated
properties are removed. (#5170)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_58ed2c55365d_7b963feac4cf9c2c5193d";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: emackey
X-GitHub-Recipient: biomassives
X-GitHub-Reason: subscribed
List-ID: AnalyticalGraphicsInc/cesium
<cesium.AnalyticalGraphicsInc.github.com>
List-Archive: https://github.com/AnalyticalGraphicsInc/cesium
List-Post<EMAIL_ADDRESS>List-Unsubscribe<EMAIL_ADDRESS>https://github.com/notifications/unsubscribe/AELU4j_DtAu7AqjgH2llZh_1P_3s2HYQks5ru9JVgaJpZM4MvctC
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address<EMAIL_ADDRESS>----==_mimepart_58ed2c55365d_7b963feac4cf9c2c5193d
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
emackey commented on this pull request.

@@ -32,6 +32,7 @@ Change Log

Fixed issues with imagerySplitPosition and the international date line in 2D mode. #5151
Fixed an issue with TileBoundingBox that caused the terrain to disappear in certain places 4032
QuadtreePrimitive now uses frameState.afterRender to fire tileLoadProgressEvent #3450
+* Removed left, right, bottom and top properties from OrthographicFrustum.

Move this up out of 1.32 changes.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/AnalyticalGraphicsInc/cesium/pull/5170#pullrequestreview-32202568
----==_mimepart_58ed2c55365d_7b963feac4cf9c2c5193d
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
@emackey commented on this pull request.

In CHANGES.md:
> @@ -32,6 +32,7 @@ Change Log
 * Fixed issues with imagerySplitPosition and the international date line in 2D mode. [#5151](https://github.com/AnalyticalGraphicsInc/cesium/pull/5151)
 * Fixed an issue with `TileBoundingBox` that caused the terrain to disappear in certain places [4032](https://github.com/AnalyticalGraphicsInc/cesium/issues/4032)
 * `QuadtreePrimitive` now uses `frameState.afterRender` to fire `tileLoadProgressEvent` [#3450](https://github.com/AnalyticalGraphicsInc/cesium/issues/3450)
+* Removed left, right, bottom and top properties from `OrthographicFrustum`.

Move this up out of 1.32 changes.
—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.

----==_mimepart_58ed2c55365d_7b963feac4cf9c2c5193d--

Thanks for updating CHANGES.md, @ateshuseyin.  Please comment on a PR after updating so the reviewers get notified.
@emackey please merge when ready.

@emackey is this good to go?  If so, I can just reserve the conflict in CHANGES and merge.  Thanks.

No, sorry, there are test failures here, on the removed properties.

@emackey there are test failures and I cannot find the right solution. Could you help me?

@ateshuseyin Check out the Testing Guide if you haven't run the tests yourself yet.
The source code for the tests is in the Specs folder, and its structure mostly matches the main Source folder.  Running the tests should indicate the filenames of the broken tests.  I think they're broken just because they're testing for the presence of deprecated values that you've removed from this branch.  So in that case, it's correct to remove the offending tests, as long as they're not testing anything else.

@ateshuseyin Oops, my previous comment was incorrect (so I deleted it).
Check out the code that @bagnell pointed out.  You removed a whole if block, one that should evaluate to true now.  If you restore the body of that block, the tests should pass.

@emackey and @bagnell thank you for your help. I reverted if block w/o if(!this._useDeprecated).

Thanks @ateshuseyin.
@bagnell tests pass now.  Please merge when ready.

Thanks again @ateshuseyin!


Intel Aero RTF: no baro0, no mag0

I flashed the intel aero with PX4 1.7.3 (from my laptop) QGC is not able to detect the barometer. When I revert to PX4 1.6.5 (from the preloaded binary on the aero), QGC is able to successfully detect the barometer.
Anyone experiencing similar problems? Or have any suggested solutions?

Thanks for the report! I've tagged the Intel lead for the platform and I'm sure she will investigate this quickly.

@jdiament95 I flashed 1.7.3 version and I don't see any issue. I checked device nodes as well inside nuttx shell, there is a baro node .

@padmashree9110 can you report how you go about flashing the firmware?
I'm flashing the firmware from my Mac laptop to the aero over the aero's access point wifi. First, I had to modify Firmware/Tools/aero_upload.sh with the proper macOS syntax for the local path names. Then I ran make aerofc-v1_default upload and successfully flashed the firmware to the aero. Finally, I confirm that I'm running the proper version because aero-get-version.py returns: Firmware: 1.7.3

If aero-get-version is showing 1.7.3 it will not be issue with flashing. Can you please share the screenshot of QGC with 1.7.3 and 1.6.5.

for PX4 Firmware Version: 1.7.3dev, I get the error message "PREFLIGHT FAIL: NO BARO SENSOR 0." When I do "ls /dev" in a nuttx shell, I can confirm that there is NO baro0 listed.
I don't have any of these problems for PX4 Firmware Version: 1.6.5dev

@anitha-suresh Any updates on this?

@LorenzMeier as @padmashree9110  mentioned this is issue is not observed here
@jdiament95 when you say 1.7.3dev, is it release or are you using the development branch based on 1.7.3?
Can you please share the output of ver git in nsh shell

Right, I see that there 'ls /dev in Nuttx shell' shows baro0 and also QGC doesn't show up any error. The PX4 FW version on which we verified is 1.7.3

nsh> ver git
FW git-hash: 47e5382e7efa68bc3f34fd767a1a76193610112c
FW version: 1.7.3 0 (17236736)
OS: NuttX
OS version: Release 7.22.0 (118882559)
OS git-hash: b18053574bf41712cef93e31bf178518f451a350


Update to latest tendermint-rs and adapt relayer to new light client

Close: #243
Close: #90
Depends on https://github.com/informalsystems/tendermint-rs/pull/583
Description

For contributor use:

[ ] Unit tests written
[ ] Added test to CI if applicable
[ ] Updated CHANGELOG_PENDING.md
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Updated relevant documentation (docs/) and code comments
[ ] Re-reviewed Files changed in the Github PR explorer

Hi @romac, I've checked out your branch romac/light-client-cleanup and when I try cargo build I'm getting an error it can't find the tendermint-rs/tendermint/Cargo.toml file.
~/dev/github.com/informalsystems/ibc-rs$ cargo build
error: failed to read `/home/andy/dev/github.com/informalsystems/tendermint-rs/tendermint/Cargo.toml`

Caused by:
  No such file or directory (os error 2)

On this machine I don't have tendermint-rs in the location it's trying to find it. I believe this is because in Cargo.toml the path is specified like this tendermint = { version = "0.16.0", path = "../../tendermint-rs/tendermint" } but should we have a repo dependency like this ? It assumes users would need to git clone tendermint-rs.
Should we just have the version ?

My bad, sorry about that! Will update it to use the tendermint-rs version from the branch in informalsystems/tendermint-rs#583.

@andynog https://github.com/informalsystems/ibc-rs/pull/255/commits/4fecf8865a2f01c32fa6fe18da922317177cedd6

Just to clarify, the git dependency is needed because this PR depends on these unreleased changes in tendermint-rs: https://github.com/informalsystems/tendermint-rs/pull/583

Hi @romac, it's building fine now. Thanks !

Codecov Report

Merging #255 into master will increase coverage by 21.8%.
The diff coverage is 53.1%.

@@            Coverage Diff            @@
##           master    #255      +/-   ##
=========================================
+ Coverage    13.6%   35.5%   +21.8%
=========================================
  Files          69     111      +42
  Lines        3752    7229    +3477
  Branches     1374    2528    +1154
=========================================
+ Hits          513    2567    +2054
- Misses       2618    4266    +1648
+ Partials      621     396     -225

Impacted Files
Coverage Δ

modules/src/events.rs
0.0% <0.0%> (ø)

modules/src/ics02_client/events.rs
0.0% <ø> (ø)

modules/src/ics02_client/raw.rs
0.0% <0.0%> (ø)

modules/src/ics03_connection/context.rs
0.0% <0.0%> (ø)

modules/src/ics03_connection/error.rs
16.6% <0.0%> (-16.7%)
:arrow_down:

modules/src/ics04_channel/error.rs
75.0% <ø> (+50.0%)
:arrow_up:

modules/src/ics04_channel/packet.rs
0.0% <0.0%> (ø)

modules/src/ics07_tendermint/client_def.rs
0.0% <0.0%> (ø)

modules/src/ics07_tendermint/error.rs
75.0% <ø> (+75.0%)
:arrow_up:

modules/src/ics26_routing/error.rs
0.0% <0.0%> (ø)

... and 199 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2438ede...2b7f3c9. Read the comment docs.

The relayer start command currently fails with a relayer-cli fatal error: hash mismatch.
This is due to a bug in tendermint-rs that we just surfaced today.
As such, I don't think it should block the merge of this PR.

We still need to wait for https://github.com/informalsystems/tendermint-rs/pull/583 to be merged first.

So I've updated the chains (chain_A and chain_B) to run with the latest stargate-3 version. Now there's another error.
(base) andy@ubuntu:~/dev/github.com/informalsystems/ibc-rs$  cargo run --bin relayer -- -c ./relayer/tests/config/fixtures/relayer_conf_example.toml start --reset
    Finished dev [unoptimized + debuginfo] target(s) in 0.15s
     Running `target/debug/relayer -c ./relayer/tests/config/fixtures/relayer_conf_example.toml start --reset`
Sep 30 14:30:01.285  INFO relayer_cli::commands::start: spawning light client chain.id=chain_A
     Relayer spawning light client for chain chain_A
Sep 30 14:30:01.416  INFO relayer_cli::commands::start: resetting client to trust options state chain.id=chain_A
error: relayer-cli fatal error: I/O error: Parse error. Invalid JSON: missing field `app` at line 9 column 9 (code: -32700)

Not sure which JSON it's referring too, maybe it's a genesis.json for chain_A and somewhere there's a validation for that. Maybe this error is coming from other dependencies.


hostmanager stopped working when I moved my Vagrant Home directory to an external drive

When I do vagrant up I now get the following output when I've entered my password:
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory

Sorry, my fault entirely. There was a space ' ' in my $VAGRANT_HOME variable which was causing the usage error with cp. I resolved it by renaming my drive :)


Feature Improvements

Suggestions

[x] Command error handling for those thrown in execution
[ ] Support for message edits when processing commands
[ ] Possible DSL for error handlers
[ ] Allow embeds to have defaults using plugs (Possibly?)
[ ] DSL for building markdown in descriptions + optional escaping

Could you elaborate some of these points a bit more? I'll go over some of these myself with the immediate remarks I have, but please expand on all of them.

Support for message edits when processing commands

This is a tricky one to implement,  MessageUpdateEvents don't contain the 'full' message object, only the changed values. We can't guarantee these to be present in cache either.
The way I see it there's two routes to implement this:

Either we cripple the current CommandEvent and provide a common interface. Effectively reducing a command to a channel behavior and text.
Or we provide a separate DSL for edited messages.

None of those are particularly appealing to me. Perhaps you could clarify the need of this feature, and see if we can't find an alternative approach.

Allow embeds to have defaults using plugs (Possibly?)

This has been asked a few times already in Kord itself. I'm of the opinion myself that defaults can be perfectly simulated by users via extension functions. Users can implement their own, for example, respondSuccess and respondError extension functions without losing clarity or brevity. You're free to debate this point further but I'm almost certainly convinced that extension functions created and maintained by users are a far better solution to the problem of duplicated code.

DSL for building markdown in descriptions + optional escaping

A good feature to request IMO. Something I personally wanted to add to Kord in some way or form myself.
A working version wouldn't be hard to implement per se. The scope of supported markdown is up for debate, it is probably most reasonable to support only Discord's subset of Markdown.
Regarding the optional escaping. I had though this was mostly solved with the allowedMentions DSL in the MessageCreateBuilder.

Support for message edits when processing commands

Main use for this which I've found quite helpful in other bots is for correcting small errors whilst processing commands.
For example, in a math command with a complex equation, if you've mistyped a single number then editing the message is far more appealing to the user than copy pasting the whole thing.
It could possibly be done through manually listening for the message event and then calling the command handler with the message object rather than the event, however I haven't looked into it.

Allow embeds to have defaults using plugs (Possibly?)

Yep that makes a lot of sense. I'll remoe that from the list.

DSL for building markdown in descriptions + optional escaping

Yep, only Discord's subset of markdown should be needed. Also with the escaping I think that'd be a good idea to include, but as false by default in the methods. Main reason is that, for example, certain names with underscores don't italicise a join message for example. Up to you of course.

Support for message edits when processing commands

I recently stumbled across this and needed this for some personal projects, I was working on this since yesterday.
Almost done, but currently the caching for data between invocation of edited command and last ran command (as an example to track the message we've sent on the first/prior run) is not properly handled due to Type inference failed by lambda return type. So, I'd PR for this feature as soon as this lib synchronize with Kord 0.7.x and the mentioned issue is fixed.


"This expression is not callable" type error when trying to use this package from ESM module

I've been using memoize-one from CJS for many years, but when I try to convert the calling code to ESM, I now get this error:
src/lib.ts:73:21 - error TS2349: This expression is not callable.
  Type 'typeof import(".../common/temp/node_modules/.pnpm/memoize-one@6.0.0/node_modules/memoize-one/dist/memoize-one")' has no call signatures.

I'm pretty sure my import is correct:
import memoizeOne from 'memoize-one';

I'm not getting this error with any other dependency, so it seems specific to memoize-one. My package still builds correctly, which means it's maybe just an error with the types. I'm using the latest version (6.0.0).

Switching to Vite/Vitest (away from webpack/Jest) fixed this issue for me.

I'm seeing the same with typechecking with "moduleResolution": "Node16". You can workaround it like so:
import _memoizeOne from 'memoize-one';

const memoizeOne = _memoizeOne as typeof _memoizeOne.default;


Create a reflection on the teamwork

What issues occurred?
How did we resolve them?
What do we plan to change moving forward?

Issues discussed in our "instructor check-in" were fixed by each individual.


feature: bumping terraformls to 0.19.0

Thank you for the contribution! Before I can merge this I'll need you to update this file too:
https://github.com/lspcontainers/dockerfiles/blob/main/.github/workflows/build-image.yml#L58
The file you updated is (slightly misleading) but used to building images locally for dev purposes. The Github Action is what actually has the version for the image pushed to the registry.

Thank you for the contribution! Before I can merge this I'll need you to update this file too:
https://github.com/lspcontainers/dockerfiles/blob/main/.github/workflows/build-image.yml#L58
The file you updated is (slightly misleading) but used to building images locally for dev purposes. The Github Action is what actually has the version for the image pushed to the registry.

updated the  pull request


Need of more user friendly info about chunking

There are lot of samples without usefull info fo novices in WebPack.
The information is short and not understandable for novices
There are plenty of questions out of the board...(
Please do not use abstract modules for samples - use real cases fo review

Just want to point out the documentation is freely editable here https://github.com/webpack/docs/wiki/contents if you wanted to help out with this effort. Thanks!

This issue was moved to webpack/webpack.io#98

This issue was moved to webpack/webpack.io#127


MTB: panic: runtime error: invalid memory address or nil pointer dereference

While running test packages, in case of unqualified tenant it leads to the panic error in the logs which doesn't make any sense actually. It should be handled with a meaningful error handler function and a log. https://github.com/kubernetes-sigs/multi-tenancy/blob/f08fc4cef21e8b0195cb9eda22c63804b2f76f84/benchmarks/e2e/tests/block_cluster_resources/block_cluster_resources.go#L52
/assign @phoenixking25

Hello @phoenixking25, still working on this?

@SomtochiAma yep

Closing this issue, as whole code-base changed. Not facing this issue now.


Alternative socket address via --daemon-addr or NB_DAEMON_ADDR not always respected

Describe the problem
When running the netbird ui on an alternative socket (netbird-ui --daemon-addr=unix:///var/run/netbird-home/sock), clicking on "Network Routes" fails because it still tries to connect to the default socket on /var/run/netbird/sock.

Weirdly, a similar issue is occurring on the CLI tool and environment variables:
export NB_DAEMON_ADDR=${NB_DAEMON_ADDR-'unix:///var/run/netbird-home/sock'}
export NB_SERVICE=${NB_SERVICE-'netbird-home'}
exec netbird  "$@"

Executing this wrapper (let's call it netbird-home) will work just fine when using e.g. netbird-home up, and the daemon address is correctly used. But executing netbird-home routes list tries to connect to /var/run/netbird/sock and results in:
Error: failed to connect to daemon error: context deadline exceeded
If the daemon is not running please run:
netbird service install
netbird service start

Executing netbird-home routes --daemon-addr=unix:///var/run/netbird-home/sock list instead works, so somehow the environment variable seems to not be respected for routes.
To Reproduce

Change the socket address
Set the NB_DAEMON_ADDR environment variable to the new socket and make sure the old one doesn't exist
Execute netbird up (works)
Execute netbird routes list (doesn't work)

Expected behavior
The correct socket should always be used.
Are you using NetBird Cloud?
Self-hosted
NetBird version
0.27.4
NetBird status -d output:
If applicable, add the `netbird status -d' command output.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

Hi @oddlama, the the routes command is not reading the env variables. This is a bug. We will fix it.


Use of option_list in Commands triggers deprecation warning in Django 1.9

Running runserver_plus.py and most other django-extensions commands produces the following deprecation warning:
11:26:59 web.1  | WARNING  [py.warnings:265] {snip}/lib/python2.7/site-packages/django/core/management/base.py:265:
RemovedInDjango110Warning: OptionParser usage for Django management commands is deprecated, use ArgumentParser instead

The docs advise using add_arguments in its place:

You should now override the add_arguments() method to add custom arguments accepted by your
command. See the example above.

This will be breaking in Django 1.10.

Duplicate of #788?


Error upgrading from 1.3.0 -> 1.4.0 via ghost-cli (v1.0.3)

Details below:
✖ Running database migrations
An error occurred.
Message: 'Command failed: /bin/sh -c sudo -E -u ghost /usr/local/share/.config/yarn/global/node_modules/ghost-cli/node_modules/.bin/knex-migrator-migrate --init --mgpath /var/www/ghost/current
[2017-08-02 11:14:16] ERROR
NAME: InternalServerError
CODE: ECONNREFUSED
MESSAGE: The server has encountered an error.
level:normal
InternalServerError: The server has encountered an error.
at new KnexMigrateError (/usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/errors.js:7:26)
at /usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/database.js:58:19
at tryCatcher (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:512:31)
at Promise._settlePromise (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:569:18)
at Promise._settlePromise0 (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:614:10)
at Promise._settlePromises (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:689:18)
at Async._drainQueue (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/async.js:133:16)
at Async._drainQueues (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/async.js:143:10)
at Immediate.Async.drainQueues (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/async.js:17:14)
at runCallback (timers.js:672:20)
at tryOnImmediate (timers.js:645:5)
at processImmediate [as _immediateCallback] (timers.js:617:5)
Error: connect ECONNREFUSED <IP_ADDRESS>:3306
at Object.exports._errnoException (util.js:1018:11)
at exports._exceptionWithHostPort (util.js:1041:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1086:14)
--------------------
at Protocol._enqueue (/usr/local/share/.config/yarn/global/node_modules/mysql/lib/protocol/Protocol.js:141:48)
at Protocol.handshake (/usr/local/share/.config/yarn/global/node_modules/mysql/lib/protocol/Protocol.js:52:41)
at Connection.connect (/usr/local/share/.config/yarn/global/node_modules/mysql/lib/Connection.js:130:18)
at /usr/local/share/.config/yarn/global/node_modules/knex/lib/dialects/mysql/index.js:106:18
at Promise._execute (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/debuggability.js:300:9)
at Promise._resolveFromExecutor (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:483:18)
at new Promise (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/promise.js:79:10)
at Client_MySQL.acquireRawConnection (/usr/local/share/.config/yarn/global/node_modules/knex/lib/dialects/mysql/index.js:104:12)
at Object.create (/usr/local/share/.config/yarn/global/node_modules/knex/lib/client.js:239:16)
at Pool._createResource (/usr/local/share/.config/yarn/global/node_modules/generic-pool/lib/generic-pool.js:354:17)
at Pool._ensureMinimum (/usr/local/share/.config/yarn/global/node_modules/generic-pool/lib/generic-pool.js:408:12)
at new Pool (/usr/local/share/.config/yarn/global/node_modules/generic-pool/lib/generic-pool.js:157:8)
at Client_MySQL.initializePool (/usr/local/share/.config/yarn/global/node_modules/knex/lib/client.js:269:17)
at Client_MySQL.Client (/usr/local/share/.config/yarn/global/node_modules/knex/lib/client.js:116:12)
at new Client_MySQL (/usr/local/share/.config/yarn/global/node_modules/knex/lib/dialects/mysql/index.js:62:20)
at Knex (/usr/local/share/.config/yarn/global/node_modules/knex/lib/index.js:60:34)
at Object.connect (/usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/database.js:24:12)
at Object.createDatabaseIfNotExist (/usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/database.js:41:30)
at KnexMigrator.init (/usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/index.js:73:21)
at KnexMigrator.migrate (/usr/local/share/.config/yarn/global/node_modules/knex-migrator/lib/index.js:160:21)
at /usr/local/share/.config/yarn/global/node_modules/knex-migrator/bin/knex-migrator-migrate:26:29
at tryCatcher (/usr/local/share/.config/yarn/global/node_modules/bluebird/js/release/util.js:16:23)
'
Debug Information:
Node Version: v6.11.1
Ghost-CLI Version: 1.0.3
Environment: production
Command: 'ghost update'

Also, what's the correct method to roll-back to a previous version? Currently I'm just changing the 'current' symlink back to another version (1.3.0 in this case).

ECONNREFUSED <IP_ADDRESS>:3306 means that the connection to your MySQL service failed.
However, this issue occurred with a Ghost CLI command, therefore can you please re-raise this on the Ghost-CLI repo.

https://docs.ghost.org/docs/upgrade#section-what-to-do-when-an-upgrade-fails


Remove unit test for dr brownfield

This test file is removed to resolve warning messages

@SanjalKatiyar , @GowthamShanmugam Please review

/approve
/lgtm


blog: using kafka with node-red

Description

Related Issue(s)

Checklist

[ ] I have read the contribution guidelines
[ ] I have considered the performance impact of these changes
[ ] Suitable unit/system level tests have been added and they pass
[ ] Documentation has been updated

@gdziuba Could you review in my stead? Can't get enough time to read it fully.

@ZJvandeWeg Yes.  Will do.

@Hasmin-AC Can you review this from a SEO perspective?

Would be good to add a line at the end that a part 2 is coming up indicating how Kafka relates to Node-RED/FlowFuse.

I don't think so, let's link them when we merge the second PR through said Pr

Hi @gdziuba, thanks for reviewing the PR, but I noticed that we haven't updated the title of the blog tile. @Yndira-E, could you please update the tile title?


OCM-12119 | fix: Env var regression with create/network

Unable to use env var for template dir (if flag is not desired) before this MR. This MR checks 1) if the env var is set, 2) if the template-dir flag is used
If both are true, it uses the template-dir flag. If the template-dir flag is not set, it will use the env var instead (sanitizing """" to "")
Also fixes un-discovered bug where when using an extra / in env var or template-dir flag, the command would not remove and it would result in a template dirpath which looked like path/to/dir//here/

/cherry-pick release_1.2.47

/test test

/test coverage

/lgtm


images(clustermesh): make GOPS_CONFIG_DIR writable to all users

gops needs to write data (e.g., the PID file) to the file-system. Make sure that the target directory, configured through the GOPS_CONFIG_DIR environment variable exists, and is writable by anyone, to avoid issues when the container is run with a non-root user.
Should fix https://github.com/cilium/cilium/issues/31524
Fix issue causing clustermesh-apiserver/kvstoremesh to not start when run with a non-root user

/test

What about switching the base image from scratch to distroless nonroot? That's what we've done for Hubble Relay and it also solves this GOPS problem without resorting to chmoding.

What about switching the base image from scratch to distroless nonroot? That's what we've done for Hubble Relay and it also solves this GOPS problem without resorting to chmoding.

Sounds reasonable to me, and it would also be consistent with Hubble Relay. The only caveat is that it wouldn't work if a user (or mutating webhook) configured a user ID different from 65532 (unless setting GOPS_CONFIG_DIR=/tmp). Did that ever cause issues for Hubble Relay? I seem to remember that at least Openshift has some automatic logic to enforce certain user IDs, but I don't know the exact details.

/cc @rolinh ^^

From what I remember, it's not causing issues. Well, if it is, we'll have to fix Hubble Relay as well anyways :slightly_smiling_face:

I seem to remember that at least Openshift has some automatic logic to enforce certain user IDs

This is correct. I believe it can be overwritten via the securityContext however.

From what I remember, it's not causing issues. Well, if it is, we'll have to fix Hubble Relay as well anyways 🙂

Fair enough. I'll update this PR to mimic the Hubble Relay approach then, to ensure consistency from that point of view.

Updated, back to reviewable state.
Concerning the backport to v1.15, I'm planning on preparing a slightly reduced version of the first commit using gcr.io/distroless/static-debian11:latest@sha256:046b92c933032a8ca99a66f4c79a68ac029d9a4ababd1a806a82140b3b899fd3 as base image instead (i.e., the root version) to avoid requiring the fsGroup helm change, which looks more risky for a backport.

/test

Rebased to fix conflict.

/test

@squeed Gentle ping :pray:


[doc] curly single quotes in readme

Hello all
It seems in https://github.com/c58/marsdb/blob/master/README.md there are several cases of "curly" quotes (so to say):
const posts = new Collection(‘posts’);

Probably those should be turned into regular single quotes:
const posts = new Collection('posts');

just in case one tries copy-pasting code bits.
HTH

Thanks, @BrainCrumbz ! Fixed.


Missing POD version for Zoom SDK iOS version <IP_ADDRESS>88

zoom-sdk-ios-<IP_ADDRESS>88 is currently available.

Would be great to get an update, as the current version crashes upon a custom disclaimer.

added new versions


Webos tv

Coverage decreased (-3.1%) to 74.382% when pulling 8ea907c16708c4d9e0c8b24fa56179b7f8583d5a on webos-tv into b689545e637312dd6ba10872dbca265eb06c3776 on develop.


Release/new non object events

New well-known events added:

Eventbridge
Documentdb

Currently tested with new, watch and invoke functionalities of Cargo Lambda.

An error raise at test_build_basic_zip_function():

The process cannot access the file because it is being used by another
│ process. (os error 32)

Is there any connection with our collaboration?


Add EGU Blogpost as non-peer-reviewed publication

Closed in https://github.com/santisoler/cv/commit/d974403bfc60f8fe491a1af2f30798bd086bc17c


Can't compile plugin with version Unreal 4.27 (ERROR: HTTPSConnectionPool... )

I am download and install ue4cli and ue4 conan then I started installing the plugin from source on UnrealEngine 4.27 and  met with an error:
C:\Users\Yako>ue4 conan build "gdal-ue4==2.4.0" "mergetiff-ue4==0.0.6"
Using user-specified engine root: C:\Program Files\UnrealEngine
Using default profile for host platform "ue4.27-Win64"
Exporting package recipes from the following source directories:
- C:\Users\Yako\AppData\Roaming\conan-ue4cli\recipes
- C:\Users\Yako

Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\cares-ue4\\1.13.0\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: e88d6c749d3f4398eb0e18d16d3fe155
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\cares-ue4\\1.16.1\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c5d76be28af38b524a7d5c905d0d8326
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\gdal-ue4\\2.4.0\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: f4199bb3df9a84dd427bd99ab142d95d
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\geos-ue4\\3.6.3\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c6a9297c67094bc3469f5c39db5bc748
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\grpc-ue4\\1.16.0\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, __init__.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: da3287bcba9efc9a1c6b1ee0b5fc9492
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\grpc-ue4\\1.30.2\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, __init__.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: f26cd90937712439c2abde80ef6d04de
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\grpc-ue4\\1.42.0\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, __init__.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 514f72ee6fb19bd9c6e5e79b87e0dbe9
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\MediaIPC-ue4\\0.0.2\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 7e9dcecc38135356e588343e28a1bc91
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\mergetiff-ue4\\0.0.6\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c1c6f72cbabb059c2450415ab73c1a54
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\playfab-gsdk-ue4\\0.6.190103\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports_sources: Copied 1 '.txt' file: CMakeLists.txt
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 434c5c7b5cb057e31bfcb4f5601705bf
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\proj-ue4\\4.9.3\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 7c259b2dd130d0a9095bad11e159db2d
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\protobuf-ue4\\3.12.3\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 400570516b73f9834883d70f771161f5
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\protobuf-ue4\\3.19.1\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 0136836e881358d7ca170610776ecc35
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\protobuf-ue4\\3.6.1\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 0a22da5916fec7197a0040769a492a41
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\\Users\\Yako\\AppData\\Roaming\\conan-ue4cli\\recipes\\python-ue4\\3.6.8\\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 4b8d92e06fd5a0a93dcd2aeb6f2bbde4
The following packages will be built:
- gdal-ue4/2.4.0
- mergetiff-ue4/0.0.6

Building package<EMAIL_ADDRESS>['conan', 'install'<EMAIL_ADDRESS>'--profile=ue4.27-Win64', '--build=missing']
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Windows
os_build=Windows
[options]
[build_requires]
[env]
UNREAL_ENGINE_VERSION=4.27
ERROR: HTTPSConnectionPool(host='center.conan.io', port=443): Max retries exceeded with url: /v1/ping (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1129)')))

Unable to connect to conancenter=https://center.conan.io
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.
Traceback (most recent call last):
  File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python39\Scripts\ue4.exe\__main__.py", line 7, in <module>
  File "c:\python39\lib\site-packages\ue4cli\cli.py", line 222, in main
    SUPPORTED_COMMANDS[command]['action'](manager, args)
  File "c:\python39\lib\site-packages\conan_ue4cli\main.py", line 48, in main
    SUBCOMMANDS[subcommand]['function'](manager, args[1:])
  File "c:\python39\lib\site-packages\conan_ue4cli\commands\build.py", line 130, in build
    builder.build(name, version, args.options if args.options is not None else [])
  File "c:\python39\lib\site-packages\conan_ue4cli\common\PackageBuilder.py", line 45, in build
    self._executor.execute(command, cwd=tempDir, check=True)
  File "c:\python39\lib\site-packages\conan_ue4cli\common\CommandExecutor.py", line 19, in execute
    return subprocess.run(command, **kwargs).returncode == 0
  File "c:\python39\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['conan', 'install'<EMAIL_ADDRESS>'--profile=ue4.27-Win64', '--build=missing']' returned non-zero exit status 1.`

I hope you've worked it out by now, and hopefully you've worked out a more secure solution than I did, but the way I got around this was to disable SSL certification on the conancenter remote.
conan remote remove conancenter
conan remote add conancenter https://center.conan.io false

i am sorry, i am complete your command and have same error( What version you build?

I hope you've worked it out by now, and hopefully you've worked out a more secure solution than I did, but the way I got around this was to disable SSL certification on the conancenter remote.
conan remote remove conancenter conan remote add conancenter https://center.conan.io false

Have you solved ？

Have you solved ？

Nope, yet

I can't compile it too,but the error is different
`PS C:\Users\78510> ue4 conan build "gdal-ue4==2.4.0" "mergetiff-ue4==0.0.6"
Using user-specified engine root: E:\UE_4.27
Using default profile for host platform "ue4.27-Win64"
Exporting package recipes from the following source directories:

C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes
C:\Users\78510

Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\cares-ue4\1.13.0\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: e88d6c749d3f4398eb0e18d16d3fe155
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\cares-ue4\1.16.1\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c5d76be28af38b524a7d5c905d0d8326
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\gdal-ue4\2.4.0\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: f4199bb3df9a84dd427bd99ab142d95d
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\geos-ue4\3.6.3\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c6a9297c67094bc3469f5c39db5bc748
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\grpc-ue4\1.16.0\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, init.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: da3287bcba9efc9a1c6b1ee0b5fc9492
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\grpc-ue4\1.30.2\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, init.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: f26cd90937712439c2abde80ef6d04de
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\grpc-ue4\1.42.0\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports: Copied 3 '.py' files: conanfile.py, grpc_helper.py, init.py
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 514f72ee6fb19bd9c6e5e79b87e0dbe9
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\MediaIPC-ue4\0.0.2\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 7e9dcecc38135356e588343e28a1bc91
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\mergetiff-ue4\0.0.6\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: c1c6f72cbabb059c2450415ab73c1a54
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\playfab-gsdk-ue4\0.6.190103\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>exports_sources: Copied 1 '.txt' file: CMakeLists.txt
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 434c5c7b5cb057e31bfcb4f5601705bf
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\proj-ue4\4.9.3\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 7c259b2dd130d0a9095bad11e159db2d
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\protobuf-ue4\3.12.3\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 400570516b73f9834883d70f771161f5
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\protobuf-ue4\3.19.1\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 0136836e881358d7ca170610776ecc35
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\protobuf-ue4\3.6.1\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 0a22da5916fec7197a0040769a492a41
Exporting recipe for package<EMAIL_ADDRESS>['conan', 'export', 'C:\Users\78510\AppData\Roaming\conan-ue4cli\recipes\python-ue4\3.6.8\conanfile.py'<EMAIL_ADDRESS>Exporting package recipe
<EMAIL_ADDRESS>The stored package has not changed
<EMAIL_ADDRESS>Exported revision: 4b8d92e06fd5a0a93dcd2aeb6f2bbde4
The following packages will be built:

gdal-ue4/2.4.0
mergetiff-ue4/0.0.6

Building package<EMAIL_ADDRESS>['conan', 'install'<EMAIL_ADDRESS>'--profile=ue4.27-Win64', '--build=missing']
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Windows
os_build=Windows
[options]
[build_requires]
[env]
UNREAL_ENGINE_VERSION=4.27
<EMAIL_ADDRESS>Not found in local cache, looking in remotes...
<EMAIL_ADDRESS>Trying with 'conancenter'...
ERROR: Failed requirement<EMAIL_ADDRESS>from<EMAIL_ADDRESS>ERROR: Unable to find<EMAIL_ADDRESS>in remotes
Traceback (most recent call last):
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\Scripts\ue4.exe_main.py", line 7, in
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\site-packages\ue4cli\cli.py", line 222, in main
SUPPORTED_COMMANDS[command]['action'](manager, args)
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\site-packages\conan_ue4cli\main.py", line 48, in main
SUBCOMMANDS[subcommand]['function'](manager, args[1:])
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\site-packages\conan_ue4cli\commands\build.py", line 130, in build
builder.build(name, version, args.options if args.options is not None else [])
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\site-packages\conan_ue4cli\common\PackageBuilder.py", line 45, in build
self._executor.execute(command, cwd=tempDir, check=True)
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\site-packages\conan_ue4cli\common\CommandExecutor.py", line 19, in execute
return subprocess.run(command, **kwargs).returncode == 0
File "C:\Users\78510\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['conan', 'install'<EMAIL_ADDRESS>'--profile=ue4.27-Win64', '--build=missing']' returned non-zero exit status 1.`

This has probably something to do with https://conan.bintray.com getting no more new packages after July 1st 2021: Conan Blogpost here.
With these conan settings:
$ conan remote list
conancenter: https://center.conan.io [Verify SSL: True]
conan-center: https://conan.bintray.com [Verify SSL: True, Disabled: True]

I can find the libcurl module
$ conan search<EMAIL_ADDRESS>Existing packages for recipe<EMAIL_ADDRESS>
    Package_ID: 4a8aa5a92eca740ec3100060d2a1f829bb028521
        [settings]
            arch: x86_64
            build_type: Release
            compiler: clang
            compiler.libcxx: libc++
            compiler.version: 11
            os: Linux
        [requires]
            ue4lib/ue4@adamrehn/profile:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
        Outdated from recipe: False

hogsy9 already briefly mentioned how to change the conan repository:

I hope you've worked it out by now, and hopefully you've worked out a more secure solution than I did, but the way I got around this was to disable SSL certification on the conancenter remote.
conan remote remove conancenter conan remote add conancenter https://center.conan.io false

@iwer Thanks for your reply,I had try these conan setting:
$ conan remote list
conancenter: https://center.conan.io [Verify SSL: True]
conan-center: https://conan.bintray.com [Verify SSL: True, Disabled: True]

But I still got:

conan search<EMAIL_ADDRESS>ERROR: Recipe not found<EMAIL_ADDRESS>ERROR: Failed requirement<EMAIL_ADDRESS>from<EMAIL_ADDRESS>ERROR: Unable to find<EMAIL_ADDRESS>in remotes

and then I also had try these conan setting:
$ conan remote list
conancenter: https://center.conan.io [Verify SSL: True]
conan-center: https://conan.bintray.com [Verify SSL: True]

then I got a different problem:
<EMAIL_ADDRESS>Not found in local cache, looking in remotes...
<EMAIL_ADDRESS>Trying with 'conancenter'...
<EMAIL_ADDRESS>Trying with 'conan-center'...
<EMAIL_ADDRESS>WARN: Remote https://conan.bintray.com is deprecated and will be shut down soon.
<EMAIL_ADDRESS>WARN: Please use the new 'conancenter' default remote.
<EMAIL_ADDRESS>WARN: Add it to your remotes with: conan remote add -i 0 conancenter https://center.conan.io
ERROR: Failed requirement<EMAIL_ADDRESS>from<EMAIL_ADDRESS>ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/ping (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:997)')))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

I also try to disable SSL certification ,but still got the same problem.

Wait just remembered: libcurl gets wrapped in a conan package from UE4 thridparty libraries...
Did you by chance forget to generate the wrapper packages?
$ ue4 conan setroot <Ue3 Root dirtectory>
$ ue4 conan generate
...
Generating and installing wrapper package for libWebSockets...
Generating and installing wrapper package for libcurl...
Generating and installing wrapper package for libstrophe...
...

@iwer Thanks for your great help again,I'm using an installed build of ue,and i don't check the generate information carefully,after i build the source code, the problem has been solved, thank you so much


Foot L/R and Ball L/R bones have odd rotation when imported into UE 5.1

Hello, I'm noticing a small glitch, not sure if this is technically an issue with this addon or if it's more to do with UE5.1
The issue is that when I import a baked Manny anim from Blender (using UE-to-Rigify), the foot and ball bones end up with odd rotational key frames at the beginning and end of the animation sequence.
To reproduce:
-Use Manny template to setup Ue-to-Rigify
-Animate the Rig (the foot/ball deformation happens no matter what the animation is)
-Bake the animation back to the UE based Rig
-Import the animation to the SK_Mannequin skeleton in UE 5.1
-The Foot R/L bones will have overdriven X rot at beginning and end of sequence, and Ball R/L bones will have Z rot issues.
The Skeleton will not show any odd rotations after baking in Blender, this only appears after importing to UE5.1
Have tested with Blender 3.4 and 3.5

can you confirm that the importing animations have been assigned the proper model in the "retarget source" property in asset details panel? (step 5 in the "adding retarget source" section of the linked guide). Retarget Source option tells the engine, which skeletal mesh was the animation created on, because by default I think Quinn is used. I might be wrong in this assumption, I haven't used been messing around Manny/Quinn animation assets a while.

What you're describing is unrelated to my question (explaination below). Have you tried the option I mentioned? The attached image does not display, there might have been some error with replying to github thread via email.
Long explaination below:

The manual Retargetting Unreal's offering (The "Retarget to another skeleton" option when you right click on animation or skeletal asset) is to bind Animation asset to another Skeleton (where bone hierarchy might differ). Binding the Animation Asset happens at import, where Unreal asks to select a Skeleton. It's done at asset creation time. It's okay that you haven't done it to assets in question. I suspected as much.

However, there's another retargetting system that Unreal is running, where Animation asset is adjusted to each Skeletal Mesh sharing the same skeleton (Which means, that the bone hierarchy is the same, only transformations differ). This is automatic process that happens at runtime, and asset creator might not even know it's happening. This needs to happen, because different Meshes might share the same bone hierarchy, but they might be offset, relative to the skeleton asset. This is necessary step for Unreal to share the same skeleton for multiple characters, even if their proportions are different. In order for unreal to figure out which Mesh's Ref Pose should be used when calculating animated pose, the "Retarget Source" needs to be set to the Mesh the Animation was created on.

Those two systems are separate, and only share common name (which is unfortunate, because it creates confusion)
What I suspect is happening in your case is that the Skeleton asset has different Ref Pose (bone transforms) than the Meshes you're trying to display your Animation on. And because Unreal doesn't know which Mesh the Animation asset was created on, it defaults to Skeletal Ref Pose. This creates improper transformations for bones, because Unreal stores bone transformations in animations in relative space.
So I'm asking about the Mesh Retargetting system (latter), while your answer touches the Skeletal Retargetting (former).

Perhaps I'm using this tool in the wrong way, or making some assumptions
about its operation? I hear you on all those points, and I think maybe I
need to do a better job at communicating the issue I'm having.
This issue appears to be specifically with how the IK bones (and weapon_R)
are being baked using UE2Rigify. Those bones specifically look as though
there is a random noise to rotation channels, like they are overdriven, or
baked from the wrong source?
To better explain my situation-
I'm animating in blender, and importing into UE5.1, using the same exact
SKM/skeleton asset. The distortions are not only happening at runtime, you
can see them in the animation sequence window (with the correct retarget
source asset)
To test in order to isolate the issue i did the following-
-I added keyframes for loc/rot/scale with no change (still figure) to the
rigified Manny in Blender, and baked that action using UEtoRigify.
I then exported that FBX animation asset and imported into UE5.1,
selecting the same skeleton that SKM_Manny is using (and what I'm using in
Blender)
After importing, if I preview that animation by opening the sequence
window, the hands, feet and weapon bones have a weird wiggling animation
(all other bones are static as would be expected)
If do the same step in blender, without using the plugin the issues doesn't
appear--
Adding keyframes with no change to the SKM manny (no rigify process),
exporting and importing to Unreal to the same skeleton using the same
method, the weird wiggling hands/feet/weapon bone keyframes do not appear.
If it helps, I had put in a issue recently, where you helped me out, adding
a "bake every bone" feature to this addon. I think that there is something
in that feature update that is causing this weird random rotation? Every
other bone works fine using your tool, except for the IK bones that are
baked using that "bake every bone" feature.
Hope this makes the issue more clear? I tried attaching a photo in case
that works.
I appreciate the help on this, the tool has been so crucial for my
workflow!
On Thu, May 25, 2023 at 1:28 PM Grzegorz Muszyński @.***>
wrote:

What you're describing is unrelated to my question (explaination below). Have
you tried the option I mentioned? The attached image does not display,
there might have been some error with replying to github thread via email.
Long explaination below:

The manual Retargetting Unreal's offering (The "Retarget to another
skeleton" option when you right click on animation or skeletal asset) is to
bind Animation asset to another Skeleton (where bone hierarchy might
differ). Binding the Animation Asset happens at import, where Unreal asks
to select a Skeleton. It's done at asset creation time. It's okay
that you haven't done it to assets in question. I suspected as much.
2.
However, there's another retargetting system that Unreal is running,
where Animation asset is adjusted to each Skeletal Mesh sharing the same
skeleton (Which means, that the bone hierarchy is the same, only
transformations differ). This is automatic process that happens at
runtime, and asset creator might not even know it's happening. This
needs to happen, because different Meshes might share the same bone
hierarchy, but they might be offset, relative to the skeleton asset.
This is necessary step for Unreal to share the same skeleton for multiple
characters, even if their proportions are different. In order for unreal to
figure out which Mesh's Ref Pose should be used when calculating animated
pose, the "Retarget Source" needs to be set to the Mesh the Animation was
created on.
Those two systems are separate, and only share common name (which is
unfortunate, because it creates confusion)
What I suspect is happening in your case is that the Skeleton asset has
different Ref Pose (bone transforms) than the Meshes you're trying to
display your Animation on. And because Unreal doesn't know which Mesh the
Animation asset was created on, it defaults to Skeletal Ref Pose. This
creates improper transformations for bones, because Unreal stores bone
transformations in animations in relative space.
So I'm asking about the Mesh Retargetting system (latter), while your
answer touches the Skeletal Retargetting (former).
—
Reply to this email directly, view it on GitHub
https://github.com/gzmuszynski/UE-to-Rigify/issues/3#issuecomment-1563262943,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7WH3DVG4CMGYPTFUXGJDOLXH6JDXANCNFSM6AAAAAAYNUVHX4
.
You are receiving this because you authored the thread.Message ID:
@.***>

weird wiggling animation
Do you mean random jitter when playing animation? At work we found that certain rotations of the spine will export with microjitters. My assumption was that because there is a Quaternion to Euler conversion (two common way of representing rotations) somewhere down the Rigify pipeline, Send-to-Unreal or UE FBX importer will introduce gimbal locks that manifests as seemingly random jitter. Unfortunately that's something I haven't found solutions for. My workaround for this is to export the animation from unreal back to blender, fix the bones manually without using rigify and reimport. This doesn't get rid of the jitter per se, but minimizes it to acceptable range.

I got your file. And yes, that looks exactly like gimbal lock curves would. Because blender takes any arbitrary rotation, this doesn't pose an issue until it's either baked, exported or imported. Then it tries it's best to recreate the curve in euler (yaw, pitch, roll) rotation, [0, 360] range. But because the original animation had either negative angles, or greater than 360, it automatically wraps around. That causes angles to go from something like 1 deg to 359 deg, and that's interpreted as a 360 degree rotation, instead of treating it like a 2 degree turn (shortest possible path in a ROM of that axis).
Unfortunately, that's an issue with either Epic's blender tools, Blender as software, or Unreal's FBX importer. Nothing I, as a template provider, can do about. As it is now, I couldn't track down when in the pipeline the error is introduced.

It is very helpful to understand the issue more clearly, totally understand
that this is out of your control!
Regardless this gives me a good perspective on next steps to get things
working better on my end.
Thanks again for your time to take a look Grzegorz!
Will be sure to mark as solved
Best
Milton
On Fri, May 26, 2023 at 11:49 AM Grzegorz Muszyński <
@.***> wrote:

I got your file. And yes, that looks exactly like gimbal lock curves
would. Because blender takes any arbitrary rotation, this doesn't pose an
issue until it's either baked, exported or imported. Then it tries it's
best to recreate the curve in euler (yaw, pitch, roll) rotation, [0, 360]
range. But because the original animation had either negative angles, or
greater than 360, it automatically wraps around. That causes angles to go
from something like 1 deg to 359 deg, and that's interpreted as a 360
degree rotation, instead of treating it like a 2 degree turn (shortest
possible path in a ROM of that axis).
Unfortunately, that's an issue with either Epic's blender tools, Blender
as software, or Unreal's FBX importer. Nothing I, as a template provider,
can do about. As it is now, I couldn't track down when in the pipeline the
error is introduced.
—
Reply to this email directly, view it on GitHub
https://github.com/gzmuszynski/UE-to-Rigify/issues/3#issuecomment-1564591651,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7WH3DSF2F5GSLUNRSZ77VDXIDGHBANCNFSM6AAAAAAYNUVHX4
.
You are receiving this because you authored the thread.Message ID:
@.***>