Why it is so difficult to install caffe2???

It takes me 2 days to install caffe2.
I was followed the instruction 100 persent and still stucking by the third-party dependence such as gflags or glog etc.
I don't understand why the installation of caffe2 is setting so complicated and inconvenient, especially compare with TensorFlow.
It's really frustrated and makes people stay away from caffe2.
By the way, it also taken me a lot of time to fix the error of caffe installation because of the mismatch of  matplot.
It seems that caffe always like to torture the users by some trivial third-party libraries.

My mileage somehow varied. Despite a small hiccup, the Caffe2 (+ Detectron) installation was completed smoothly by referencing to the installation guide. Still, with the recent merging of Caffe2 source into Pytorch, the documentation might need additional update.
Currently working configuration is Ubuntu 16.04, CUDA Toolkit 9.1, cuDNN 7.1, and Python 2.7. I have summarized the installation steps in this article as additional reference for others who face problems during installation.
The author installed with CentOS 7, which was different with the configuration that I have tested. I usually had good experience with CentOS but can't further confirm or rebuke until I proceed with the installation in that environment.

Bump! I use TensorFlow and MXnet extensively and both frameworks can be installed on a vanilla Ubuntu install with one command.

We only support Anaconda packages at the moment. If you do not wish to use Anaconda, then you must build Caffe2 from source.

This is extremely annoying. Among all the competing choices for package management, not everyone wishes to use Anaconda. I strongly suggest it be made as simple as the following:
sudo apt-add-repository ppa:facebook/caffe2
sudo apt-get update
sudo apt-get install caffe2-cu90

OR
sudo pip3 install caffe2-cu90

Making it low-friction and as universal as above would definitely speed-up adoption.

Any news about this? It's already been quite a while since the issue was opened.

We are trying to provide a better install experience, and it really has been getting steadily better. Unfortunately we are still just spread a little thing. We are always welcoming PRs to help us out.
If you're having troubles with building from source, read through https://caffe2.ai/docs/faq.html to make sure you understand which libraries you need and how to set up your Python installation correctly.

This is an impossible task on my journey to DensPose
I can't install Caffe2 for not normal ammount of time..
Tried though cmd line and trhough cloning through VS2017
Installed all the dependencies..
What a m I missing ? I can't understand the log files?
Seems to be unsuccessfulbuild" because "AlwaysCreate" was specified. is the first problem ..
From there it's just all down the hill..
SAVE ME!PLEASE!
Added log files.
InitializeBuildStatus:
Creating "cmTC_f4003.dir\Debug\cmTC_f4003.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
CMakeOutput.log
CMakeError.log

Also what is glog: need to build from source?
Is it this one? -https://godoc.org/github.com/golang/glog?
Seems to just import..
Or this - https://github.com/google/glog ?

Also done
git submodule update --init --recursive
python setup.py install
Getting
Building wheel torch-1.0.0a0+0fd176f
The support for PyTorch with Python 2.7 on Windows is very experimental.
Please set the flag FORCE_PY27_BUILD to 1 to continue build.
Where should I change it?

Please set the flag FORCE_PY27_BUILD to 1 to continue build

https://github.com/pytorch/pytorch/blob/master/README.md
Seems that we have to run
set FORCE_PY27_BUILD=1
and bunch of other commands.. Trying

Nightmare..

Also done
git submodule update --init --recursive
python setup.py install
Getting
Building wheel torch-1.0.0a0+0fd176f
The support for PyTorch with Python 2.7 on Windows is very experimental.
Please set the flag FORCE_PY27_BUILD to 1 to continue build.
Where should I change it?

That  -submodule update  - didn't help
Still problem with cpuinfo

@Kaidanov looks like you are building on Windows? That is generally more difficult than Ubuntu or other Linux versions. I'd recommend trying Linux if you can. Also, FYI that DensePose will probably require a high-end GPU (GTX 1080 or higher) to run.

So how can I built the right machine for it?
I have credits with AWS and Google clouds..
Is there predefined machine I could use?

I have 8 gb of GPU
It's not enough?

For cloud options (AWS and Azure) please take a look at https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=cloud

After clean clone I am able to run a build of cmake of caffe to in vs2017
Getting those:
You changed the size of TensorImpl on 64-bit arch.See Note [TensorImpl size constraints] on how to proceed.
Severity	Code	Description	Project	File	Line	Suppression State
Error	C2338	You changed the size of TensorImpl on 64-bit arch.See Note [TensorImpl size constraints] on how to proceed.	C:\WINDOWS\system32\CMakeLists.txt	D:\dev\ML\pytorch\aten\src\ATen\core\TensorImpl.h	1448
And that's the note:
// Note [TensorImpl size constraints]
// ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Changed the size of TensorImpl?  If the size went down, good for
// you!  Adjust the documentation below and the expected size.
// Did it go up?  Read on...
//
// Struct size matters.  In some production systems at Facebook, we have
// 400M live tensors during a training run.  Do the math: every 64-bit
// word you add to Tensor is an extra 3.2 gigabytes in RAM.
//
// If you are a Facebook employee, you can check if the run in question
// has tipped you over the point using the command here:
// https://fburl.com/q5enpv98
//
// For reference, we OOMed at 160 bytes (20 words) per TensorImpl.
// This is not counting overhead from strides out-of-line allocation, and
// StorageImpl space.  We're currently comfortably under this number;
// let's keep it that way.  (One currently approved pending size
// increase is inlining sizes and strides as small vectors, to reduce
// dynamic allocations.)
//
// Our memory usage on 32-bit systems is suboptimal, but we're not checking
// for it at the moment (to help avoid rage inducing cycles when the
// 32-bit number is wrong).
//
// Current breakdown:
//
//    vtable pointer
//    strong refcount           TODO: pack these into one word
//    weak refcount
//    storage pointer
//    sizes vector (start)
//    sizes vector (end)
//    sizes vector (reserved)   TODO: get rid of me
//    strides pointer
//    storage offset
//    numel
//    data type pointer
//    miscellaneous bitfield
//
static_assert(sizeof(void*) != sizeof(int64_t) || // if 64-bit...
sizeof(TensorImpl) == sizeof(int64_t) * 12,
"You changed the size of TensorImpl on 64-bit arch."
"See Note [TensorImpl size constraints] on how to proceed.");
Can I somehow go around it?

If you're trying to get Detectron working, maybe try the changes in https://github.com/pytorch/pytorch/pull/13550#issuecomment-435913149 . I just remembered that we do not support Detectron on Windows right now, but if you want to try and get it working that PR is your best chance.


Bug: Type Parameters are not imported

While deriving TS for a struct, whenever we get to a field, we currently add that field to Dependencies.
However, we additionally need to add all generic parameters of that field to Dependencies as well.

I've implemented that fix here

Nice! I spent the last 30 minutes trying to get this to work before realizing there was the TyoeList::push method lol
fn generate_generics_fn(generics: &Generics) -> TokenStream {
    let generic_types = generics
        .type_params()
        .map(|ty| ty.ident.clone())
        .collect::<Vec<_>>();

    if generic_types.is_empty() {
        return quote!{
            fn generics() -> impl ts_rs::typelist::TypeList {}
        };
    }

    let generic_types = generic_types
        .iter()
        .fold(
            quote!(),
            |acc, cur| quote!{
                (std::marker::PhantomData<#cur>, #acc)
            }
        );

    quote!{
        fn generics() -> impl ts_rs::typelist::TypeList {
            #generic_types
        }
    }
}

Alright, CI passes. (Meaning that our test coverage still isn't ideal ^^).
I haven't really touched the impls for Result, Option, etc., since they are, as you nicely outlined in the discussion, wonky.

Great! At least our underlying problem with generics is fixed! Now we just have to decide on how to even handle the impls I mentioned in the discussion

I think we have a new problem...
#[derive(TS)]
#[ts(export)]
struct Test {}

#[derive(TS)]
#[ts(export)]
enum MyOption<T> {
    Some(T),
    None
}

#[derive(TS)]
#[ts(export)]
struct Outer {
    a: MyOption<Test>
}
```

`Outer.ts` now generates correctly, but `MyOption.ts`
```ts
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { Test } from "./Test";

export type MyOption<T> = { "Some": T } | "None";
```

It now imports `Test` for no reason

I think we have a new problem...
#[derive(TS)]
#[ts(export)]
struct Test {}

#[derive(TS)]
#[ts(export)]
enum MyOption<T> {
    Some(T),
    None
}

#[derive(TS)]
#[ts(export)]
struct Outer {
    a: MyOption<Test>
}

Outer.ts now generates correctly, but MyOption.ts
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { Test } from "./Test";

export type MyOption<T> = { "Some": T } | "None";

It now imports Test for no reason

Ah :D
I think we had a similar issue before. Could you check if that issue is actually new?
As far as I can tell, this is caused by both Outer and MyOption wanting to write MyOption.ts. Here, it seems like Outer won, and got to export MyOption<Test>, which depends on Test.
Gotta look into that a bit deeper. Definetely something we want to fix (I even enabled errors for unused imports in the CI), but at least it's not as bad as a missing import.

It hink I found out why. The expansion of Outer's #[derive(TS)] gives this:
    #[allow(clippy::unused_unit)]
    fn dependency_types() -> impl ts_rs::typelist::TypeList
    where
        Self: 'static,
    {
        {
            use ts_rs::typelist::TypeList;
            ().push::<MyOption<Test>>() // This should be MyOption<ts_rs::Dummy>
                .extend(<MyOption<Test> as ts_rs::TS>::generics())
        }
    }

I have copied the expansion and replaced the type manually, which fixed MyOption.ts

As far as I can tell, this is caused by both Outer and MyOption wanting to write MyOption.ts. Here, it seems like Outer won, and got to export MyOption, which depends on Test.

Yup, seems that way to me as well

I think we had a similar issue before. Could you check if that issue is actually new?

Sure!

It hink I found out why. The expansion of Outer's #[derive(TS)] gives this:
#[allow(clippy::unused_unit)]
fn dependency_types() -> impl ts_rs::typelist::TypeList
where
    Self: 'static,
{
    {
        use ts_rs::typelist::TypeList;
        ().push::<MyOption<Test>>() // This should be MyOption<ts_rs::Dummy>
            .extend(<MyOption<Test> as ts_rs::TS>::generics())
    }
}

I have copied the expansion and replaced the type manually, which fixed MyOption.ts

If this really is the solution, I think type aliases are gonna be a problem again lol

If this really is the solution, I think type aliases are gonna be a problem again lol

True! I think we can do it a bit differently though:
Currently, we start at Outer, and recursively go through each dependency and export it. However, once we reach a type, MyOption<Test>, for example, we first generate the imports of MyOption<Test>, which includes Test.
Only then do we call MyOption<Test>::decl(), which converts it into a MyOption<SomeDummyType>.
If we had some way of first doing that conversion, and then doing the imports, we'd be fine

One approach would be to make the conversion an operation independant of TS::decl.
One way of doing this would be with an associated type. This is pretty ugly, especially since the dummy types would need to be in the public scope. With const &'static str, that'd be much much nicer
Without const &'static str, we could re-use the TypeVisitor trait. Then, decl() would look like this:
fn decl() -> String {
    struct Visit(String);
    impl TypeVisitor for Visit {
        fn visit<T: TS>(&mut self) {
            self.0 = T::decl_concrete();
        }
    }

    let mut visit = Visit(String::new());
    Self::with_generic(&mut visit);
    visit.0
}

One approach would be to make the conversion an operation independant of TS::decl.
One way of doing this would be with an associated type. This is pretty ugly, especially since the dummy types would need to be in the public scope. With const &'static str, that'd be much much nicer
Without const &'static str, we could re-use the TypeVisitor trait. Then, decl() would look like this:
fn decl() -> String {
    struct Visit(String);
    impl TypeVisitor for Visit {
        fn visit<T: TS>(&mut self) {
            self.0 = T::decl_concrete();
        }
    }

    let mut visit = Visit(String::new());
    Self::with_generic(&mut visit);
    visit.0
}

Okay... I'm gonna be honest, I have no idea what any of that means lol.

Alternatively, we could also do the import generation inside decl.

What would that look like?

One way of doing this would be with an associated type

So if I got this right, we'd add T::Generic to Dependencies rather than just T?

I think we had a similar issue before. Could you check if that issue is actually new?

I couldn't find this error anywhere. The closest is #168, which brings extra imports to the type itself, but not its dependencies

Just confirmed that the same thing happens on main, so it's not actually related to the changes in this PR

Maybe we should rename TS::generics though... I managed to get myself confused between the associated type and the method with the same name lol

I think this also needs to be done for Range and RangeInclusive?

@NyxCode this is my proposed fix for those types in the discussion. I have tested them against #168, even changing which types were #[ts(inline)]d and it seems to work as expected while keeping current behavior for inline

Awesome! I'll add a test for it!

I can also give the original idea (using the on-the-fly generated dummy types) a try to see how that'd look. Might be more complicated, but we'll see.

Played around with it a bit, but I'm running into T: 'static problems. So maybe that doesn't work after all.

I renamed TS::Generics to TS::WithoutGenerics. Wasn't sure if TS::WithoutGenerics or TS::WithDummyGenerics was a better name, but I felt like TS::WithoutGenerics better conveyed the intention of the associated type.

Once we get const S: &'static str, TS::WithoutGenerics can actually be
struct Generic<A, B>(A, B);
impl TS for Generic<A, B> {
    type WithoutGenerics = Generic<Dummy<"A">, Dummy<"B">>;
    // ...
}

Then, we can re-write TS::decl to use that associated type, instead of generating new ones.

Hopefully this feature won't be in nightly forever like the ! type lol. It would make these generics a lot simpler

Hey @NyxCode! Is anything else you think needs doing with this PR?

Also one other thing I'd like to ask. Currently in this PR the only types where TS::transparent() returns true are Range<I> and RangeInclusive<I>. Do you think we should refactor these like what we did to Result and Option? This would allow us to remove TS::transparent, as it would always be false

Hey @escritorio-gustavo! Hope you're doing well!

I'm doing great, thanks! Hope you are too

Do you think it makes sense to include that in this PR?

I'm not really sure. One could argue it's an effect of the changes to Result, Option, Vec and HashMap. But there's also the problem of making this PR too big

One thing I'd like to improve is the testing for dependencies. I think a while back you mentioned that we could, for example, just slap #[ts(export)] on every type in the test suite and run tsc over it

Yup! Some #[ts(export)] and maybe #[ts(export_to = "...", rename = "...")] to avoid naming collisions should allow tsc to check everything in the tests directory.

Yup! Some #[ts(export)] and maybe #[ts(export_to = "...", rename = "...")] to avoid naming collisions should allow tsc to check everything in the tests directory.

Minor update. I've already started doing this. I'm about halfway through. Should be able to commit the updated tests tomorrow

These issues may force us to implement https://github.com/Aleph-Alpha/ts-rs/issues/134

Hm. I think the idea here was to just come up with a very convoluted type to see if it generates correct TS.
Neither Record<K, V> nor { [key: K]: V } would support this (Afaik, they are identical). Same goes for serde_json, which just panics for a type like that.
So I'm not sure this is actually a bug. Maybe just changing the test to use a different type would be good enough.

Neither Record<K, V> nor { [key: K]: V } would support this (Afaik, they are identical

You're right, the error just changes to be about index signatures being required to extend string | number | symbol or be a template literal type

Okay, now that the updated tests are passing current CI, I found three tests that fail to generate correct imports:

generics.rs -> types Parent and GenericParent missing an import to type P0 (looks like some kind of depth limit to the imports)
lifetimes.rs -> type A missing an import to type B
self_referential.rs -> type HasT missing an import to type T

So I'm not sure this is actually a bug. Maybe just changing the test to use a different type would be good enough.

Agreed, however I don't think it's possible to fix GenericContainer unless we remove HashMap as we'd need A extends string | number | symbol to be able to use it in a Record as the key

Okay, calling std::any::type_name_of_val(&Parent::dependency_types) gives the following TypeList:
(
      (
          (
              (
                  (
                      (
                          (),
                          (PhantomData::<T1<T0<u32>>>,)
                      ),
                      (
                          (),
                          (PhantomData::<T0<u32>>,)
                      )
                  ),
                  (PhantomData::<T1<P1<T0<P0<u32>>>>>,)
              ),
              (
                  (),
                  (PhantomData::<P1<T0<P0<u32>>>>,)
              )
          ),
          (PhantomData::<T1<P1<()>>>,)
      ),
      (
          (),
          (PhantomData::<P1<()>>,)
      )
  )

If I'm getting this right, the problem seems to be that PhantomData::<P0<...>> never shows up here. Is that right?

The macro expansion's dependency_types returns the following:
().push::<T1<T0<u32>>>()
    .extend(<T1<T0<u32>> as ts_rs::TS>::generics())
    .push::<T1<P1<T0<P0<u32>>>>>()
    .extend(<T1<P1<T0<P0<u32>>>> as ts_rs::TS>::generics())
    .push::<T1<P1<()>>>()
    .extend(<T1<P1<()>> as ts_rs::TS>::generics())

Very interesting!
I simplified it down to
struct Parent {
    a: T1<T0<P0<u32>>>,
}

Parent depends on [T1<...>] + T1<T0<P0<u32>>>::generic_types().
T1<T0<P0<u32>>>::generic_types() yields just [T0], so Parent::dependency_types() is [T1<...>, T0<...>].
Seems like I completely overlooked that generics are "transitive". As far as I can tell, SomeType<SomeGeneric>::generic_types() should not return [SomeGeneric] + SomeGeneric::generic_types(), but it only returns [SomeGeneric] currently.
I'll try implementing that fix to see what happens if we do that 😆

I added .extend(#ident::generics()), which seems to have worked
fn generate_generics_fn(&self) -> TokenStream {
    let generics = self
        .generics
        .type_params()
        .map(|TypeParam { ident, .. }| quote![.push::<#ident>().extend(<#ident as ts_rs::TS>::generics())]);
    quote! {
        #[allow(clippy::unused_unit)]
        fn generics() -> impl ts_rs::typelist::TypeList
        where
            Self: 'static,
        {
            use ts_rs::typelist::TypeList;
            ()#(#generics)*
        }
    }
}

@escritorio-gustavo Ah, you were faster than me 😆! Nice work! Super cool that we caught this by running tsc over everything (though it's insanely slow on my machine, wtf)

New CI finally passing 🎉

I just realized, we will have to document all of this in CHANGELOG.md lol

Okay, CI now fails, but it looks like it's just because TS hates self referential types

Mine too!

Also, notice that the push triggered test failed, but the PR triggered on succeeded

Ah! Many types in self_referential.rs are renamed to T, so one would overwrite the other. In some cases one type might win over the other.

Also, according to https://github.com/microsoft/TypeScript/issues/14174, it is supposed to work. Maybe the CI version of tsc is outdated?

Now we actually see the real error since the files dont get overwritten! Seems like internally tagged enums don't like self-references.

Looks like it doesn't like self references with type intesection

Not sure we can do anything about that.

Agreed. Self references are only allowed as property values

Oh, look at that! Someone left a comment about this xD

// NOTE: The generated type is actually not valid TS here, since the indirections rust enforces for recursive types
//       gets lost during the translation to TypeScript (e.g "Box" => "T").

If that was me, then I dont remember ^^
I'll remove the #[ts(export)] then. I think it makes sense keeping the test itself though.

This either never worked (as in the TS generated was always invalid), or it is a regression caused by enum flattening support

Oh, look at that! Someone left a comment about this xD

// NOTE: The generated type is actually not valid TS here, since the indirections rust enforces for recursive types
//       gets lost during the translation to TypeScript (e.g "Box" => "T").

Well that literally explains a lot xD

If that was me, then I dont remember ^^

Git blame says it was you 😆

This either never worked (as in the TS generated was always invalid), or it is a regression caused by enum flattening support

Okay, I did some thinking. This never worked in the first place, because it cant!
For one, the internally-tagged representation doesn't allow newtype/tuple variants. Serde just panics at runtime for them.
That restriction makes sense, since having an internal tag requires the variant to be an object.
"Internally tagged" means "put the tag inside the variant". But if the variant is not an object to put the tag in, this cant work.
Sensible types, like this one here work, since they don't use the type intersection:
#[derive(TS)]
#[ts(tag = "tag")]
enum X {
    A { inner: Box<Self> },
    B { inner: i32 }
}

export type X = { "tag": "A", inner: X, } | { "tag": "B", inner: number, };

That makes a lot of sense! Especially given that one of the variants was trying to do
{ "tag": "E" } & Array<I>

which is just nonsense

For one, the internally-tagged representation doesn't allow newtype/tuple variants. Serde just panics at runtime for them.
That restriction makes sense, since having an internal tag requires the variant to be an object.

Added a note on the wiki about that

I think it's because impl_shadow doesn't have TS::generics

Yep! Seems like I just skipped that (since it didnt break any tests 😆)

That's odd, in my computer TS complains that CustomKey(null) doesn't extend string | number | symbol, but CI still passes

I'll be out for a about 2 hours now, so I'll be unable to respond. See you later

That's odd, in my computer TS complains that CustomKey(null) doesn't extend string | number | symbol, but CI still passes

Interesting! Seems like that's only caught with --strict, while Record<{x: number}, Something> also fails without --strict.
I've enabled --strict for everything in CI, and fixed CustomKey. Everything works(TM) again!

I think it'd be alright if we merge this. It's getting annoying to scroll through this huge PR.
Is there anything you think still needs to be done here?
Before the next release, I'd like to

see if we can remove TS::transparent()
Maybe make the recusive exporting of dependencies we added in #221 configurable?

While I can't think of a use-case where this would be undesirable, I think it's the change most likely to break existing setups.

Are there any other features you can think of we might want to make configurable for that reason?

Thank you so much for your work, it has been invaluable! I'm super stoked to see where this library is going.
Stuff just works(tm) now, especially inlining, other enum representations and generics. And now we've also increased the test suite by 160 .ts files. Just awesome stuff 😆

I think it'd be alright if we merge this. It's getting annoying to scroll through this huge PR.

Yup, this PR is way too big already

Is there anything you think still needs to be done here?

While adding #[ts(export)] to everything, I noticed we don't have a #[ts(...)] equivalent to #[serde(tag = "...")] for structs, so I'd like to add that in a new PR.

see if we can remove TS::transparent()

Definetly something I'd like to see!

Maybe make the recusive exporting of dependencies we added in #221 configurable?

Cool! Would that be a feature flag or a new attribute?

I think it's the change most likely to break existing setups.

Are there any other features you can think of we might want to make configurable for that reason?

I don't see much of a problem, as long as the next release is 8.0.0 to indicate there are breaking changes.

Thank you so much for your work, it has been invaluable! I'm super stoked to see where this library is going.

Thanks! It's awesome to be able to help out with a project that I use a lot!


config URI normalization by default

Hi,
It seems that HTTP gem do URI normalization by default and I can't find the option to change this behavior. I have an application that won't work if HTTP gem do URI normalization. For example, HTTP.get(url) will decode %2B to +
Please let me know if we can control the URI normalization or not. Thank you!

I'd prefer to be able to configure this and shut it off as well. Normalization has been the cause of some past issues, e.g. #206 #246 #259 #325

@tarcieri I agree that we should allow enable/disable normalization as well as I think it will make sense to allow use different backends for URI (Addressable::URI / ::URI), so that those who don't want to use addressable (there was an issue for that) would be able to avoid it.

@ixti I am working on a PR right now, and I am approaching it in the following way:

I am adding an option normaliz_uri to the request object, which will be true as default to keep the library backward compatible
method normalize_uri will be renamed to construct_url, something like:

 # @return [HTTP::URI] URI with all componentes but query being normalized.
 def construct_uri(uri, normalize_uri)
      uri = HTTP::URI.parse uri

      return uri unless normalize_uri

      HTTP::URI.new(
        :scheme     => uri.normalized_scheme,
        :authority  => uri.normalized_authority,
        :path       => uri.normalized_path,
        :query      => uri.query,
        :fragment   => uri.normalized_fragment
      )
 end

Do you think it's a viable solution without hurting current users?

What if we would do this via feature instead? Like this:
HTTP.use(:uri_normalizer => :itself.to_proc).get(...)

By default normalizer will be what we have now.

I am adding a feature uri_normalizer and I am finishing up the pr right now.
One more query, the way I am adding the feature is that the new feature will handle normalizing the URI, and normalize_uri will move out of request. Following the conventions in the library, I am handling it in wrap_request method inside the feature.
This will break the users calling HTTP::Request.new directly, the way around that is to keep the normalize URI inside the Request module and add an option normalize_uri to the Request module. Which will be passed as false for the case when the user send something like:
HTTP.use(:uri_normalizer => {normalize_uri=>false}).get(...)

Does it make sense? I personally favor the first way, handling normalizing the URI inside the feature. But that will be a breaking change.

I'm not sure I understand.

You need to extend HTTP::Request to accept URI normalizer:

module HTTP
  class Request
    # Default uri normalizer
    def self.normalize_uri(uri); end

    def initialize(opts)
      # ...
      @uri_normalizer = opts.fetch(:uri_normalizer) { self.class.method(:normalize_uri) }
      @uri = @uri_normalizer.call(opts.fetch(:uri))
    end

    # ...
  end
end

Something like this

You will need to create a feature, that will allow to use custom normalizer, which will be passed to Reques.new from within HTTP::Client#build_request.

We have released v4.1.0 version that allows you to disable URI normalization:
normalizer = ->(uri) { ::HTTP::URI.parse(uri) }
HTTP.use(:uri_normalizer => { :normalize => normalizer }).get(...)

Notice though that some of the internals depend on HTTP::URI API, so ensure that returned object of your normalizer is HTTP::URI.


crate msa_alignment subworkflow

merging with failing tests due to renaming.
missing: generate nf-test snapshot of the subworkflow


[stable/coredns] Prevent port template failures if the optional use_tcp parameter is not set

Is this a new chart
No
What this PR does / why we need it:
The parameter use_tcp is described as "Optionally enable tcp for this service as well" but omitting this value results in a template that fails to render.  This change just defaults that value to false if it is not set allowing the template to render successfully and making this "optional" parameter actually optional.
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[X] DCO signed
[X] Chart Version bumped
[X] Variables are documented in the README.md
[X] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @shashidharatd

/ok-to-test

hrm... not sure what caused that test failure... the logs say:
Error: forwarding ports: error upgrading connection: unexpected EOF
but on my local system, everything works fine...
/retest

Looks like it was some kind of transient failure in the test system itself perhaps... a retest and everything went through just fine.  The first run (the one that failed) seems to have never even gotten to the step of deploying the helm chart, so I doubt it was caused by this change.

/retest

/lgtm


Elemental Ammunition not automated with rule element

Foundry 12.328
PF2e 6.0.4

Also, there should be multiple formulas in the compendium, not just one generic one. :)

#6641

Duplicate of #6641


add examples

browserify
myth
bunch of files
with static file server that does content negotiation right

Nah screw this, this stuff should be in a separate module instead.


Search page - search bar full styling support

Requirements

[ ] Finish up styling the search bar

[ ] change karat color to primary
[ ] fix font size and coloring
[ ] fix placeholders

[ ] research using vue-select vs using native <select> element

[ ] Look into safari styling not applying
[ ] is it worth the overhead?
[ ] does it support multi-select?

Dependencies

None (Add any issue dependencies here)

I have some questions about the ticket I was assigned. There are a lot of designs on figma so I do not know exactly which one I am supposed to follow for the TopNav/SearchBar part. Which should I follow?

<select> Choices
Make our own

Pros

Can be customizable to exactly what we want
Possibly might not have trouble to adapt to our data
More control over performance cost

Cons

Time consuming
Needs a lot of testing due to not being used by thousands of other developers

Use Vue Multi-Select
Link to Vue Multi-Select Page

Pros

Tested and proven to work
Easy to add to our project
Solid documentation

Cons

Might be tricky to adapt to our search query depending how we are handling it now
Could have extra performance costs that we cannot control

My Research and Thoughts
Through my research I think it will be better to use the built in component. For an open source project it will be easier for developers who come in and out of the project to read the documentation and adjust the files when needed. I do not think that it will be too much if any extra performance cost. It also handles typing search and multi-select which is wonderful for a built in component.
Styling mismatch on Safari and Windows

The select itself needs to use webkit to get rid of the default styling so it then takes our styling based on my research. So we need to change the original CSS file. See MDN here
I think this is worth it for the scroll bar like I was talking about before as they appear differently as well as the select field to make the styling consistent across browsers.

This is a rather old issue and as there is an issue for a redesign epic, I think we can close this for now and implement a different solution later. Also in one of the PRs there is a multiselect utility function built in.


Access to container has stopped working.

Describe the bug
Doesn't seem to work anymore for me, cannot connect via http://<IP_ADDRESS>:5500/
Version
In the top right area: 0.1.2
To Reproduce
Steps to reproduce the behavior:
Using desktop docker
docker pull chuckmacdev/adrfinder
docker run -d --restart always -p "<IP_ADDRESS>:5500:5500" -v datastore-volume:/datastore --name adrfinder chuckmacdev/adrfinder
http://<IP_ADDRESS>:5500/
results in
<IP_ADDRESS> didn’t send any data.
ERR_EMPTY_RESPONSE
Docker logs shows
(1) wsgi starting up on http://<IP_ADDRESS>:5500

Request failed: 404 Client Error: Not Found for url: https://www.disneyworld.co.uk/dining/
(1) accepted ('<IP_ADDRESS>', 54274)
<IP_ADDRESS> - - [05/Jul/2022 12:29:02] "GET / HTTP/1.1" 200 0 2.409906
wsgi exiting
(1) wsgi exited, is_accepting=True

Additional context
installed ping and curl within the container to test and could ping out to internet OK but curl to https://www.disneyworld.co.uk/dining/ resulted in refused connection.

Disney has changed their auth schema for retrieving restaurants.  Working on updating it.

Thanks, pesky Disney!
Cheers
Giles
From: ChuckMac @.>
Sent: 05 July 2022 13:48
To: ChuckMac/ADRFinder @.>
Cc: gkeable @.>; Author @.>
Subject: Re: [ChuckMac/ADRFinder] Access to container has stopped working. (Issue #7)
Disney has changed their auth schema for retrieving restaurants. Working on updating it.

Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FChuckMac%2FADRFinder%2Fissues%2F7%23issuecomment-1175019909&data=05|01||bbcc243317a4481a0ff508da5e848df1|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637926220644299678|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=WvHjLCQSodQoQRjLonCJTKJnq9%2Bf1QbK%2Fx1EsocaGXA%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEJR5X5DSAEMZB6IUVILOGTVSQVG5ANCNFSM52WCRIVQ&data=05|01||bbcc243317a4481a0ff508da5e848df1|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637926220644299678|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=kRqEtMYR2jZ1u7T4j5FEh805%2FxnUJfoWPW0yMoE%2F5MY%3D&reserved=0.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

(and didn't mean to close this one)


Unsupported scheme when using package: uri

Running the simple test below in dartium gives  Unsupported scheme error. Not an expert but it seems the : is unwillingly encoded?
test("broken http scheme", () async {
    var loadInfoText = await new Resource('package:test.txt').readAsString();
    expect(loadInfoText, equals('text'));
  });

for example:
Unsupported operation: Unsupported scheme: http%3A//localhost:43551/mcJCoZomCMCstSKrssXwPq90z9HKv%252B50/test/scanner/packages/test.txt

package:resource/src/io_html.dart 50:3          readAsString.<async>
===== asynchronous gap ===========================
dart:async                                      _Completer.completeError
package:resource/src/io_html.dart 53:1          readAsString.<async>
===== asynchronous gap ===========================
dart:async                                      Future.Future.microtask
package:resource/src/io_html.dart               readAsString
package:resource/src/resource_loader.dart 69:7  DefaultLoader.readAsString
package:resource/src/resource.dart 74:20        Resource.readAsString.<async>
===== asynchronous gap ===========================
dart:async                                      _asyncThenWrapperHelper
package:resource/src/resource.dart              Resource.readAsString
res_test.dart 8:63                              main.<fn>.<async>
===== asynchronous gap ===========================
dart:async                                      Future.Future.microtask
res_test.dart                                   main.<fn>

While debugging, it seems Isolate.resolvePackageUri(uri) is returning this. It looks like a bug in dartium to me.

Any movement on this issue?

See: https://github.com/flutter/flutter/issues/14815


update to juttle 0.7.0

Bump the package.json to the new adapter API.
Add a check for nested fields in the query parsing.
Update the changelog for the pending release.

@mstemm

lgtm.

I think a more appropriate place for the check would be Linter. It is scans the filter AST before everything else with an explicit purpose to validate it and there are already similar checks there.

I moved the check to the linter as suggested by @dmajda. Will merge and publish now.


Merge paddle classification notebook into paddle conversion notebook

The 103 and 214 notebooks are very similar. It would be beneficial to merge them into one -> 103 with all features shown.

@OpenVINO-dev-contest do you think you can merge them?

214 was created for 2022.1 launch, it's a bit out-of-data now, but the features in it can be spited and reused into other notebook topics. The features include:
Preprocessing API
Directly Loading a PaddlePaddle Model
Auto-Device Plugin
AsyncInferQueue Python API
Performance Hints
LATENCY Mode
THROUGHPUT Mode
My idea is merge paddle directly loading into 103, and move Preprocessing API to another one.

That sounds good to me. @eaidova, any of your thoughts?

@sbalandi working on preprocessing api tutorial, probably it is better to sync with her
I added direct paddle loading in 002 (like all other model formats), but it is also good to have it in 103 too

If preprocessing api and  paddle directly loading are already included, then i think we can remove 214.

@OpenVINO-dev-contest could you create a PR to remove 214, then?


✨[Feature Polish] Litecoin Card Add password criteria

Goal
Make sure the user knows the password criteria for registration
Criteria

1 Capital letter, 1 number, 8 min chars, accepts special chars

Definition of Done

Shows the criteria in the registration
Add a failure note in the verification/register

Fixed in v3.4.0


Update language with url params resulted mixing page locale

Actual Behavior

Using lang parameter depending on configuration set-up causing mixed page locale selection.
Expected Behavior

Given language by using lang parameter should reflect completely on current page locale without depending on your environment setup.
Steps to Reproduce the Bug
Strange behaviour exist if follow steps to set-up environment and execute;

Set-up different initial locales between;

LOCALE_ID which is used to set default locale at configuration.module.ts:47 (Able to control explicitly by using providers: [{provide: LOCALE_ID, useValue: 'fr-FR'}])
First available of locale from locales at environment.prod.ts or environment.ts (Current setup using en_US as a first available locale)

Assume we have fr_FR as default locale from LOCALE_ID and en_US as first locale from environment.prod.ts or environment.ts
Run application, npm start
Try to navigate http://localhost:4200/home;lang=fr_FR
See mixing over selected locale and page content.

Environment Details
Desktop:

Chrome 80.0.3987.163
PWA 0.18

Possible Root Cause

Application causing repeating language  switch within short period of time, which triggers known ngx-translate issue ngx-translate/core#1030,
As a result application state  switched correct language, loaded translation may differs,
Adding  defer times while switching language like using debounceTime or  removing possible repeated language switch with same language code two times or more by using  translateService.use or translateService.setDefaultLang which is leading translation loads in wrong order.

Hi @anumang,
sorry for the late replay. I've been very busy working on #215. The new view renderer ivy will bring some amazing new features to the PWA 😁.
Using LOCALE_ID from Angular is something, that we do not support. And yes, our documentation is not quite clear on that; I'll work on that...
Currently we are basically only using ngx-translate for localization. However to use pipes on templates for formatting dates, numbers and prices, we have to build on top of the Angular localization framework. That's why we've built wrappers around the native pipes to instruct the Angular pipes on-the-fly with the current locale, that is managed in our NgRx state.
Up until Angular 9 the localization of Angular was always a build-time setting. This means if you would like to run the same code for different locales, you would have to deploy multiple instances of it. Lastly with our support for multiple channels, this was not an option for us and we decided to use ngx-translate. For the Angular part of our application the base locale should be ignored and setting it to different values has therefor no or even strange effects.
So I'm sorry to say, but this is a classic won't fix.
Some time after the Angular 9 upgrade we might think about dropping ngx-translate in favor for the native runtime support.
If I may ask, what is the original problem you would have liked to solve with this approach?

Hi @dhhyi ,
Seems my issue miss understood because issue is not related with usage or even existence of LOCALE_ID. In our code, we never updated or used LOCALE_ID other than existing usage on ISH PWA code at version 0.18.
I just used it in the description to show how to re-produce the same issue with minimum change on your clean code at version 0.18 so that you can inspect further details. Just try to make it simple.
For our case, we have different browser locale setting (as LOCALE_ID) and different first placed locale (as first available locale or initial selected locale) at locales array in environment files.
To create a combination of this issue, just we need to provide;

LOCALE_ID or window.navigator.locale should have different value than given first locale in locales list
Run application with express server,
Try to navigate http://localhost:4200/home;lang=${LOCALE_ID}

We applied temporary fix in our environment depending on ngx-translate/core#1030 issue which I believed the root cause of this issue. So I also mentioned at last section.

So, let me get this right: you want something like #104? (setting language based on browser settings)
This is something that's still in our backlog 😅.

:)
@dhhyi, the issue causing to show translations with wrong locale while application state has running with correct locale and ending up translations & product details are shown in two different locale. So it should not be evaluated duplication or part of a feature.
Just preventing possible race condition  between locale changes solve easily ngx-translate/core#1030 issue which is the root cause.
So just by using debounceTime between locale updates we can easily prevent trouble ngx-translate/core#1030 issue;
https://github.com/intershop/intershop-pwa/blob/41d9d2ef2fa0a3945fca17de668215428f45874f/src/app/core/store/locale/locale.effects.ts#L23-L29
Instead of this, we can use with debounceTime with safety duration;
  @Effect({ dispatch: false })
  setLocale$ = this.store.pipe(
    select(getCurrentLocale),
    mapToProperty('lang'),
    whenTruthy(),
    debounceTime(250),
    tap(lang => this.translateService.use(lang))
  );

Yes, NOW I get it. Sorry it took me so long, must be one of my bad days. 😅
Debouncing actually works. Just re-scheduling with delay zero does also work for me. Can you please double-check? Thank you for pointing me the right way.
Do you mind if I add you as an author to that commit? #karma 😉

@dhhyi, thank you for understanding and resolving the issue :tada:.
At the end, We just made a temporary solution but still I am happy to be part of it :) So it's not necessity but as you wish if you count me as an author for this fix.
My long term expectations,
Translation library  should  bring stable fix for their issue if they decide to wont-fix for their race condition issue and suggest developers to update locales only once, you may need to re-evaluate ISH locales architecture / logic from scratch or may switch another possible solutions which you mentioned with possible changes after angular 9 upgrade.


Can't open with keyboard shortcuts

Hi, I'm running Atom 1.9.8 on x86_64 linux
I've been struggling with this for at least 10 mins. It came up initially a few times but seemed to be conflicting with the native package:
I've restarted atom each time and also done a atom refresh each time (ctrl alt R)
I've tried the defaults,
alt+g alt+d
Interpreted in every possible way

(meaning: hold alt, press g, release both, hold alt press d)
(hold alt press g, keep holding alt, press d)
(hold alt, press g and d and hold all 3 at the same time)

I've also tried setting it as custom keys
'ctrl-h':  'git-diff-details:toggle-git-diff-details';
'alt-d':  'git-diff-details:toggle-git-diff-details';
'ctrl-alt-d':  'git-diff-details:toggle-git-diff-details';
'ctrl-alt-shift-d':  'git-diff-details:toggle-git-diff-details';
'alt-shift-d':  'git-diff-details:toggle-git-diff-details';
I've got custom keys that I've set for other things, so custom keys do work for other things for me. I've also double checked that I'm not using any of the keys tested as my desktop's global shortcuts.
I've tried disabling the core packages: git-diff open-on-github.
Basically can't use it :/
Any ideas?

@lope please note you have to place your cursor on an area that actually has diffs. See #60

@samu thank you very much for your kind and helpful response and sharing this package.
I see that it does work, although it does nothing for the lines that are added (the green plus instead of the yellow circle), which of course is understandable and reasonable now that I'm familiar with it's interface, know what it does and what to expect from it.
One piece of constructive criticism and suggestion I can make is that the UX is a little bit difficult for people who are unfamiliar with this package as to what exactly it does, and the requirement of the cursor being over a modified line. It's extremely frustrating pressing hotkeys and experiencing absolutely nothing happening. It feels like the hotkeys aren't working or the package is not working. To receive absolutely no response by default, in my opinion is a huge UX problem.
I suggest that if there's a nop action (do nothing because the user does not have the cursor on a modified line), that you pop up a msgbox telling them they need to be on a modified line. It will save a lot of time, frustration and github issues/comments.
However personally I'm quite glad it didn't work for me, because it facilitated me trying split-diff which is more in-line with my own preferences/workflow, so personally I will use that now.
Thanks once again for sharing this package and responding in such a helpful and timely manner!

@lope thanks for the response. I'll actually keep this issue open as i think the idea of displaying a message box when the user is outside of a diff is great.

@samu thanks sounds good, after a moment's reflection in Atom I can actually see a use case for using git-diff-details and split-diff packages.
git-diff-details might be better when coding with a few vertical splits and there's no space to fit split-diff's extra pane. Then split-diff can provide the detailed view when needed.

I've added documentation for this to the readme and will leave it at that for now.


Running Test Discovery with Pytest flapping

Environment data
VS Code version: 1.44.0
Extension version (available under the Extensions sidebar): 2020.5.78807
OS and version: MACOS Mojave 10.14.6
Python version (& distribution if applicable, e.g. Anaconda): 3.6.10
Pytest version: 4.0.1 ( can't upgrade due to dependency )
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pyenv
Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): Default
Value of the python.languageServer setting: Default
Expected behaviour
Running discovery with pytest find all tests.
Actual behaviour
Running discovery flaps, sometimes the discovery fails and sometimes succeeds,
the error the keep repeating is a follows:
Test Discovery failed:
 < remove trackback >
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "<my path> /.pyenv/virtualenv/lib/python3.6/site-packages/_pytest/main.py", line 181, in wrap_session
INTERNALERROR>     config._do_configure()
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 629, in _do_configure
INTERNALERROR>     self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/hooks.py", line 308, in call_historic
INTERNALERROR>     res = self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File <my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/_pytest/cacheprovider.py", line 313, in pytest_configure
INTERNALERROR>     config.cache = Cache.for_config(config)
INTERNALERROR>   File "<my path>/.pyenv/virtualenv/lib/python3.6/site-packages/_pytest/cacheprovider.py", line 47, in for_config
INTERNALERROR>     cachedir.mkdir()
INTERNALERROR>   File "<folder> /.pyenv/versions/3.6.10/lib/python3.6/pathlib.py", line 1248, in mkdir
INTERNALERROR>     self._accessor.mkdir(self, mode)
INTERNALERROR>   File "<folder>/.pyenv/versions/3.6.10/lib/python3.6/pathlib.py", line 387, in wrapped
INTERNALERROR>     return strfunc(str(pathobj), *args)
INTERNALERROR> FileExistsError: [Errno 17] File exists: '<my path>/.pytest_cache'

Traceback (most recent call last):
  File "<folder>/.vscode/extensions/ms-python.python-2020.5.78807/pythonFiles/testing_tools/run_adapter.py", line 22, in <module>
    main(tool, cmd, subargs, toolargs)
  File "<folder>/.vscode/extensions/ms-python.python-2020.5.78807/pythonFiles/testing_tools/adapter/__main__.py", line 100, in main
    parents, result = run(toolargs, **subargs)
  File "<folder>/.vscode/extensions/ms-python.python-2020.5.78807/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 44, in discover
    raise Exception("pytest discovery failed (exit code {})".format(ec))
Exception: pytest discovery failed (exit code 3)

Steps to reproduce:
Create two folders ( folder1, folder2 ) and put in each one script with pytest cases, then set settings.json to:
{
    "python.pythonPath": "<env path >/.pyenv/virtualenv/bin/python3.6",
    "git.ignoreLimitWarning": true,
    "python.testing.pytestArgs": [
        "${workspaceFolder}/folder1",
       "${workspaceFolder}/folder2"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "terminal.integrated.env.osx": {
        "PYTHONPATH": "${env:PYTHONPATH}:${workspaceFolder}/"
    }
}

Is seems a mutual exclusion issue when multiple threads are trying to update '/.pytest_cache' since when I change the number of folders in "python.testing.pytestArgs" in settings.json the problem stops and if there is more then a single folder waiting some time between clicking on discovery helps to reduce the number of discoveries I need to run until succeeding.

Hello @erlichsefi, thank you for reaching out.
I could not reproduce your issue with the following folder structure:

Is seems a mutual exclusion issue when multiple threads are trying to update '/.pytest_cache'

This sounds more like an issue with pytest, since the extension is only a thin wrapper around pytest test discovery.
What is the output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python) when you run test discovery?

Thank for the quick response, the out is:
> ~<path to root >/<folderInRoot>/.pyenv/virtualenv/bin/python3.6 ~/.vscode/extensions/ms-python.python-2020.5.78807/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~<path to root > -s --cache-clear ~/<path to root >/<folderInRoot>/<folder1>
cwd: ~<path to root >
~<path to root > /<folderInRoot>/.pyenv/virtualenv/bin/python3.6 ~/.vscode/extensions/ms-python.python-2020.5.78807/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~<path to root > -s --cache-clear ~<path to root >/<folderInRoot>/<folder1>
cwd: ~<path to root >

Can you try not adding these folders to python.testing.pytestArgs?

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on then we will be happy to re-open this issue to pick up where we left off.


Jaeger Collector cannot connect to Kerbrized Kafka with SASL SSL

Describe the bug
We are using kerbrized Kafka with SASL_SSL, Jaeger Collector cannot connect to Kerbrized Kafka
To Reproduce
docker run --net=abcd --name jaeger-collector  -e SPAN_STORAGE_TYPE=kafka -p 14267:14267 -p 14268:14268 -p 9411:9411 -p 14250:14250 hboyina/jaegercollector:latest --kafka.producer.topic=jaeger-spans --kafka.producer.brokers=**:9093 --kafka.producer.authentication=kerberos --kafka.producer.kerberos.config-file=/etc/krb5.conf --kafka.producer.kerberos.keytab-file=/root/hboyina.keytab --kafka.producer.kerberos.use-keytab=true --kafka.producer.kerberos.username=hboyina --kafka.producer.kerberos.realm=.COM
Expected behavior
Jaeger Collector should connect with Kafka , if the Kafka has SASL_SSL Authentication
Screenshots
If applicable, add screenshots to help explain your problem.
Version (please complete the following information):

OS: [e.g. Linux] Rhel
Jaeger version: 1.22
Deployment: Docker

What troubleshooting steps did you try?
Try to follow https://www.jaegertracing.io/docs/latest/troubleshooting/ and describe how far you were able to progress and/or which steps did not work.
Additional context
Add any other context about the problem here.

I have a related issue...
C:\Users\L0282162\Downloads>docker run -d --name jaeger-collector-quay -e SPAN_STORAGE_TYPE=kafka -e KAFKA_PRODUCER_BROKERS=pcdp-k01.bancogalicia.com.ar:9092,pcdp-k02.bancogalicia.com.ar:9092,pcdp-k03.bancogalicia.com.ar:9092,pcdp-k04.bancogalicia.com.ar:9092 -e KAFKA_PRODUCER_TOPIC=jaeger-spans -e KAFKA_PRODUCER_AUTHENTICATION=kerberos -e KAFKA_PRODUCER_KERBEROS_REALM=BGCMZ.BANCOGALICIA.COM.AR -e<EMAIL_ADDRESS>-e KAFKA_PRODUCER_KERBEROS_USE_KEYTAB=true -e LOG_LEVEL=debug -p 16686:16686 -p 14268:14268 -p 14250:14250 -v C:\Users\L0282162\Downloads\udoop22m.keytab:/etc/security/kafka.keytab quay.io/jaegertracing/jaeger-collector
97f5c6819456ac0d4825284ded7f15d7f75862dfc52a9f5a0d4b76623f7dc1fc
C:\Users\L0282162\Downloads>docker ps -a
CONTAINER ID        IMAGE                                                                                COMMAND                  CREATED              STATUS                          PORTS               NAMES
97f5c6819456        quay.io/jaegertracing/jaeger-collector                                                 "/go/bin/collector-l…"   About a minute ago   Exited (1) About a minute ago                       jaeger-collector-quay
C:\Users\L0282162\Downloads>docker logs jaeger-collector-quay
2021/12/17 14:11:53 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
{"level":"info","ts":1639750313.6556678,"caller":"flags/service.go:117","msg":"Mounting metrics handler on admin server","route":"/metrics"}
{"level":"info","ts":1639750313.6558056,"caller":"flags/service.go:123","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
{"level":"info","ts":1639750313.65626,"caller":"flags/admin.go:104","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","ts":1639750313.6563847,"caller":"flags/admin.go:115","msg":"Starting admin HTTP server","http-addr":":14269"}
{"level":"info","ts":1639750313.6564634,"caller":"flags/admin.go:96","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"}
{"level":"info","ts":1639750313.6597028,"caller":"kafka/factory.go:69","msg":"Kafka factory","producer builder":{"Brokers":["pcdp-k01.bancogalicia.com.ar:9092","pcdp-k02.bancogalicia.com.ar:9092","pcdp-k03.bancogalicia.com.ar:9092","pcdp-k04.bancogalicia.com.ar:9092"],"RequiredAcks":1,"Compression":0,"CompressionLevel":0,"ProtocolVersion":"","BatchLinger":0,"BatchSize":0,"BatchMinMessages":0,"BatchMaxMessages":0,"Authentication":"kerberos","Kerberos":{"ServiceName":"kafka","Realm":"BGCMZ.BANCOGALICIA.COM.AR","UseKeyTab":true,"Username":"udoop22m@BGCMZ.BANCOGALICIA.COM.AR","ConfigPath":"/etc/krb5.conf","KeyTabPath":"/etc/security/kafka.keytab"},"TLS":{"Enabled":false,"CAPath":"","CertPath":"","KeyPath":"","ServerName":"","ClientCAPath":"","SkipHostVerify":false},"PlainText":{"Username":"","Mechanism":"PLAIN"}},"topic":"jaeger-spans"}
{"level":"fatal","ts":1639750314.6189425,"caller":"./main.go:80","msg":"Failed to init storage factory","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main.func1\n\t./main.go:80\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.3.0/command.go:974\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:902\nmain.main\n\t./main.go:147\nruntime.main\n\truntime/proc.go:255"}
The cluster is working with other app.
Any clue?
Thanks in advance.

I have a related issue...
C:\Users\L0282162\Downloads>docker run -d --name jaeger-collector-quay -e SPAN_STORAGE_TYPE=kafka -e KAFKA_PRODUCER_BROKERS=pcdp-k01.bancogalicia.com.ar:9092,pcdp-k02.bancogalicia.com.ar:9092,pcdp-k03.bancogalicia.com.ar:9092,pcdp-k04.bancogalicia.com.ar:9092 -e KAFKA_PRODUCER_TOPIC=jaeger-spans -e KAFKA_PRODUCER_AUTHENTICATION=kerberos -e KAFKA_PRODUCER_KERBEROS_REALM=BGCMZ.BANCOGALICIA.COM.AR -e<EMAIL_ADDRESS>-e KAFKA_PRODUCER_KERBEROS_USE_KEYTAB=true -e LOG_LEVEL=debug -p 16686:16686 -p 14268:14268 -p 14250:14250 -v C:\Users\L0282162\Downloads\udoop22m.keytab:/etc/security/kafka.keytab quay.io/jaegertracing/jaeger-collector 97f5c6819456ac0d4825284ded7f15d7f75862dfc52a9f5a0d4b76623f7dc1fc
C:\Users\L0282162\Downloads>docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 97f5c6819456 quay.io/jaegertracing/jaeger-collector "/go/bin/collector-l…" About a minute ago Exited (1) About a minute ago jaeger-collector-quay
C:\Users\L0282162\Downloads>docker logs jaeger-collector-quay 2021/12/17 14:11:53 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined {"level":"info","ts":1639750313.6556678,"caller":"flags/service.go:117","msg":"Mounting metrics handler on admin server","route":"/metrics"} {"level":"info","ts":1639750313.6558056,"caller":"flags/service.go:123","msg":"Mounting expvar handler on admin server","route":"/debug/vars"} {"level":"info","ts":1639750313.65626,"caller":"flags/admin.go:104","msg":"Mounting health check on admin server","route":"/"} {"level":"info","ts":1639750313.6563847,"caller":"flags/admin.go:115","msg":"Starting admin HTTP server","http-addr":":14269"} {"level":"info","ts":1639750313.6564634,"caller":"flags/admin.go:96","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"} {"level":"info","ts":1639750313.6597028,"caller":"kafka/factory.go:69","msg":"Kafka factory","producer builder":{"Brokers":["pcdp-k01.bancogalicia.com.ar:9092","pcdp-k02.bancogalicia.com.ar:9092","pcdp-k03.bancogalicia.com.ar:9092","pcdp-k04.bancogalicia.com.ar:9092"],"RequiredAcks":1,"Compression":0,"CompressionLevel":0,"ProtocolVersion":"","BatchLinger":0,"BatchSize":0,"BatchMinMessages":0,"BatchMaxMessages":0,"Authentication":"kerberos","Kerberos":{"ServiceName":"kafka","Realm":"BGCMZ.BANCOGALICIA.COM.AR","UseKeyTab":true,"Username":"udoop22m@BGCMZ.BANCOGALICIA.COM.AR","ConfigPath":"/etc/krb5.conf","KeyTabPath":"/etc/security/kafka.keytab"},"TLS":{"Enabled":false,"CAPath":"","CertPath":"","KeyPath":"","ServerName":"","ClientCAPath":"","SkipHostVerify":false},"PlainText":{"Username":"","Mechanism":"PLAIN"}},"topic":"jaeger-spans"} {"level":"fatal","ts":1639750314.6189425,"caller":"./main.go:80","msg":"Failed to init storage factory","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)","stacktrace":"main.main.func1\n\t./main.go:80\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.3.0/command.go:974\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:902\nmain.main\n\t./main.go:147\nruntime.main\n\truntime/proc.go:255"}
The cluster is working with other app. Any clue?
Thanks in advance.

Please stop disclosing/leaking internal server information

This should be fixed in v2 Ref : https://github.com/open-telemetry/opentelemetry-collector/issues/1579


Set Ruby version for netlify.com.

I'm setting it to 2.2.3 because that's also the local version I have
installed and which works with our Gemfile.lock file.
This should fix the current netlify.com error:

activesupport-5.1.4 requires ruby version >= 2.2.2, which is incompatible with the current version, ruby 2.1.2p95

I also fixed our build command for NetLify.
@derekperkins building works now :)
The next problem is that the font for the symbols on the homepage somehow seems to be missing.

console errors re: the font:

deploy-preview-3223--vitess.netlify.com/:9 Mixed Content: The page at 'https://deploy-preview-3223--vitess.netlify.com/' was loaded over HTTPS, but requested an insecure stylesheet 'http://fonts.googleapis.com/css?family=Roboto:400,100,100italic,300,300italic,400italic,500,500italic,700,700italic,900,900italic'. This request has been blocked; the content must be served over HTTPS.
deploy-preview-3223--vitess.netlify.com/:1 Access to Font at 'https://d33wubrfki0l68.cloudfront.net/css/5635f80f2754296bfdde164aa444f990bee865be/fonts/fontawesome-webfont.woff2?v=4.4.0' from origin 'https://deploy-preview-3223--vitess.netlify.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://deploy-preview-3223--vitess.netlify.com' is therefore not allowed access. The response had HTTP status code 404.
deploy-preview-3223--vitess.netlify.com/:1 Access to Font at 'https://d33wubrfki0l68.cloudfront.net/css/5635f80f2754296bfdde164aa444f990bee865be/fonts/fontawesome-webfont.woff?v=4.4.0' from origin 'https://deploy-preview-3223--vitess.netlify.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://deploy-preview-3223--vitess.netlify.com' is therefore not allowed access. The response had HTTP status code 404.
deploy-preview-3223--vitess.netlify.com/:1 Access to Font at 'https://d33wubrfki0l68.cloudfront.net/css/5635f80f2754296bfdde164aa444f990bee865be/fonts/fontawesome-webfont.ttf?v=4.4.0' from origin 'https://deploy-preview-3223--vitess.netlify.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://deploy-preview-3223--vitess.netlify.com' is therefore not allowed access. The response had HTTP status code 404.
deploy-preview-3223--vitess.netlify.com/:1 Mixed Content: The page at 'https://deploy-preview-3223--vitess.netlify.com/' was loaded over HTTPS, but requested an insecure stylesheet 'http://fonts.googleapis.com/css?family=Roboto:400,100,100italic,300,300italic,400italic,500,500italic,700,700italic,900,900italic'. This request has been blocked; the content must be served over HTTPS.

Awesome. @tannerlinsley and I are planning on looking through this more tomorrow to get it fully functional.

LGTM
Green builds!


Error mangling in an error message for typed holes

Program: https://code.world/#PQi2vxqOWd0Tle1fcO6Y-9Q
Comment:
This error message has something mangled in it.
Message:
program.hs:47:23: warning:
    • Found hole: _ :: (Number, Number, Number, Number) -> Truth
    • In the expression: _ (ox, oy, x, y)
      In the argument(s) of any, namely
        ([_ (ox, oy, x, y) |
             (ox, _) <- first (obs, 10) |  oy <- [0, 4 .. ]])
    • Valid hole fits include
        collidesObstacle :: (Number, Number, Number, Number) -> Truth
          (bound at program.hs:51:1)
        undefined :: forall a.

Truth)
          (defined in the standard library
           (and originally defined in the standard library))

Current simplest reproduction: https://code.world/#PQ9vOLk3xExhvQsFKyza3jA
Here's the original GHC error message:
Test.hs:1:8: error:
    • Found hole: _ :: t -> t1
      Where: ‘t’, ‘t1’ are rigid type variables bound by
               the inferred type of f :: t -> t1
               at Test.hs:1:1-11
    • In the expression: _
      In the expression: _ (x)
      In an equation for ‘f’: f (x) = _ (x)
    • Relevant bindings include
        x :: t (bound at Test.hs:1:3)
        f :: t -> t1 (bound at Test.hs:1:1)
      Valid hole fits include
        f :: forall t1 t2. t1 -> t2
          with f @t @t1
          (defined at Test.hs:1:1)
        undefined :: forall a.
                     base-<IP_ADDRESS>:GHC.Stack.Types.HasCallStack =>
                     a
          with undefined @(t -> t1)
          (imported from ‘Prelude’ at Test.hs:1:1
           (and originally defined in ‘codeworld-base-<IP_ADDRESS>:Internal.Prelude’))

Here's the mangled message:
program.hs:1:8: warning:
    • Found hole: _ :: t -> t1
      Where: t, t1 are rigid type variables bound by
               the inferred type of f :: t -> t1
               at program.hs:1:1-11
    • Valid hole fits include
        f :: forall t1 t2. t1 -> t2
          with f @t @t1
          (defined at program.hs:1:1)
        undefined :: forall a.

t1)
          (defined in the standard library
           (and originally defined in the standard library))```

The GHC error message is correctly clean of "in the expression ..." context, and relevant bindings, since we don't want these.  But there are three problems with the "Valid hole fits" section:

We don't want the line with f @t @t1, since visible type application is overly technical
The HasCallStack constraint from undefined is removed, but over-aggressively, killing part of the following expression and the following line.
(and originally defined in ...) is unnecessary; imports are not used enough to bother.

Worst mangling is fixed in commit dd12d472903f32fa4f7c5666c44d57cc8222f608
What remains is just to strip out the unnecessary parts, of which there are actually three:

Where: t, t1 are rigid type variables...
with f @t @t1 and with undefined @(t -> t1)
(and originally defined in the standard library)


Drag drop with readonly fields fix

Hi,
If you drag and drop from a readonly field (or a normal field that goes through readonly fields), the writable cells will end up with "undefined" as their value.
For an example see https://jsfiddle.net/brettyj/n09pektc/2 and drag and drop the readonly "Jazz" into the Available column, the Available column will show undefined. After this commit it would instead show "Jazz".
I've created a pull request which fixes this issue https://github.com/paulhodel/jexcel/pull/810

Thanks, I will merge you PR soon.


Unable to use all nodes/threads setup for Databricks Compute

Hi,
Thansk for creating this library.
I am trying to spin up Dask cluster on Azure Databricks.

I've followed the instructions and managed to setup a dask cluster. However, i realised that the compute resources are not maximized. There is only 1 worker instead of 2 (based on the minimum number of workers I configured). Any idea how to fix this ? I have tried to use client.cluster.scale(n=2) but got an error returned.

Second issue is that i realised that URL for the dask dashboard is not accessible and is different if you are using Azure Databricks. The typical URL of an Azure databricks notebook is as followed:

https://adb-1486644750497365.5.azuredatabricks.net/?o=1486644750497365#notebook/1676261431997434/command/1676261431997440

Thanks and any advice is much appreciated.

I would expect two Dask workers to be created. You can't use the scaling functionality in Dask because we are leveraging Databricks to start the processes. Can you take a look at the logs from the init scripts to see why the second worker is failing to start?
Yep we should definitely be handling this URL differently. I don't know enough about how Azure Databricks handles proxying to know what the new URL should be, but I'm sure we can find out. We also need a way to detect if the user is on Azure Databricks, maybe there is an environment variable that would tell us?

Further, you've raised an issue with two separate questions. Could I ask you to open a second issue and split them out so we can handle them separately?

Hi @jacobtomlinson ,
Thanks for the advice. Turned out there was some Azure quota issue and wasn't really because of any issue with the library. I tried it on a different workspace without quota limitation and the issue went away. Apologies as I should have checked this in advance.
Closing this issue. Will create a separate issue for the azure databricks URL.


Can't deploy wheel via PyPI because of git dependencies

Uploading a wheel to PyPI results in:
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Invalid value for requires_dist. Error: Can't have direct dependency: 'container-ontology @ https://github.com/rpgoldman/container-ontology/tarball/main'

I'm not sure if there is an alternative way to configure the setup.py script that will allow git repos as dependencies, or if we should just move the container contology to a git submodule.

For now, let's start with it as a git submodule in order to enable a working release.
Ultimately, however, I think we need to publish the container ontology and obtain it either as a package or from an ontology server.

Do you know what the container-ontology dependency is?  I can get the container ontology moved and ready for pypi next week. Wondering if it would be just as easy to make it not be a dependency until that happens.
It’s an ontology, anyway, so I think you would just be getting URIs, not code. I’ll take a peek.

I just checked, and while the container-ontology’s URI is referenced, I don’t see any code dependency. So probably we could just drop the dependency for now.

Plan is to deploy with the dependency cut for now and the test that uses the container-ontology marked as known failing. When @rpgoldman deploys container ontology on pypi, we will then recover the test.


tree + site plot in csubst site

The visualized sites should be the same as shown in the protein structure, and may be organized as: convergent sites, blank, divergent sites.

Hi Kenji, thank you for developing csubst. Can you show us how to plot this figure by csubst?

This is not a part of CSUBST yet, but I plan to include an R visualization script using ggtree in the future. FYI, this is a prototype script that generated the above plot.
csubst_visualization.r.zip

Thank you so much! This tool is really useful and super cool. I believe it will be very popular and highly influential in the future.

Thank you!

This feature may be better implemented when we make the transition from ete3 to ete4.
https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>8621v1


Swift 6 support to avoid conflicts in packages that depend on higher version of swift syntax

Added extra package.swift to support swift 6. Mainly conflicts due to swift-syntax versions I tried to avoid in a similar way as done in https://github.com/pointfreeco/swift-issue-reporting
Some additional minor cleanup

links to dependencies for apple related swift packages moved to swiftlang
fixed some warnings on imports
removed features that are enabled in swift 6

@doozMen Thank you for this PR!
I've had a local Swift 6 branch already lying around for some time now, which I've just updated and put into #129. So I'm gonna close this in favor of #129, but still thanks for the nudge!
Note, though, that I'll probably wait for the official Swift 6 release before merging #129 (along with updating our other OSS frameworks for Swift 6).

The idea of this different swift packages I think is that you can release with a new version of swift and keep the old release hanging. Like the repo I mention in the PR.
I use your project in a server project so I do not rely on others to update the swift version but cannot use it in my swift 6 only project as I got the swift-syntax error. Maybe this is a package manager issue but maybe I mis understood how to use the different package.swift files?

Never the less I missed your open pr and thanks for the quick response! I looked into #129 and that is looking good. Looking forward to the release then and will keep for now working on my small fork.
Again Thanks!

The idea of this different swift packages I think is that you can release with a new version of swift and keep the old release hanging.

That's correct. This is also true with what I've done in #129.

I use your project in a server project so I do not rely on others to update the swift version but cannot use it in my swift 6 only project as I got the swift-syntax error. Maybe this is a package manager issue but maybe I mis understood how to use the different package.swift files?

This is a more general problem with Swift Syntax. I've seen a few discussions around this problem (e.g. this one in the forums).
We're currently sticking to pinning the Swift Syntax version in the package manifest to the Swift version that is used. This is the simplest approach for now - and is "guaranteed" to work.


[1.1.1-bp][ec] match built-in curves on EC_GROUP_new_from_ecparameters

This is a backport of #9808 for 1.1.1-stable.
See the description in #9808 for a full report.
This commit includes a partial backport of #8555 (commit 8402cd5)
for which the main author is @slontis
Checklist

[x] documentation is added or updated
[x] tests are added or updated

@slontis upon squashing I plan to add the Co-authored-by: Shane Lontis<EMAIL_ADDRESS>tag to credit you, unless you have any objection.

@slontis upon squashing I plan to add the Co-authored-by: Shane Lontis<EMAIL_ADDRESS>tag to credit you, unless you have any objection.

I dont mind either way - You made a significant contribution to the original work..

Merging now.
Editorial note: I will edit the commit title as [ec] Match built-in curves on EC_GROUP_new_from_ecparameters to match the title of the commit on master.

Merged with 9a43a73380 .
Thanks all for the quick feedback!


bazel_0, bazel_0_26, bazel_0_29, bazel_1: remove

Description of changes

These are very old versions of bazel that haven't been maintained by upstream for a while, since bazel 4 is the first LTS version: https://blog.bazel.build/2020/11/10/long-term-support-release.html.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

cc @NixOS/bazel

ping @NixOS/bazel


Multithread SpiceyPy

Has anyone managed to multithread SpiceyPy (or Spice)? I had a quick go with the threading module but it failed miserably.  Just wondering if anyone has had any success?

As far as I know, it is not possible to multithread Spice. I remember this from some discussion on the Spice user list a few months ago. As SpiceyPy is just a wrapper for Spice, it shares the same flaw.

Thanks Andrew - I'm just going to have to run multiple processes and pickle the results.  It's fine just not very python-esq

well until Spice is refactored to support multiple threads or I see a proof of concept for having SpiceyPy somehow fix that, I am considering this issue as closed. I will close this issue formally in the next few days.


Move the infiniteMasonry widget into its own module

To be able to re-use this display in other projects, I will need to move the infiniteMasonry logic into its own module.
It is using a custom connector on top of the infiniteHits base widget. The two main issues when moving it to its own module is that it has two points of contact with other modules:

It expects to be able to reach instantsearch.connectors.connectInfiniteHits. We should make instantsearch an argument, so we can inject it as a dependency.
It expects hits to have been transformed as for the regular hit and infiniteHits display. We should update the code in norska-frontend so it also applies the custom transform to our custom widget. A solution would be to add a .shouldTransformItems: true property on our custom widget, that will be then checked by norska-frontend. Doing so will allow us to not needing to pass the transforms to the infiniteMasonry and replicate this logic there.

I actually moved it into its own norska theme: norska-theme-search-infinite


Add weight annotations to Timestamp, Sudo, Democracy, Treasury, et al.

Begins adding weight annotations to the Timestamp, Sudo, Democracy and Treasury modules. Still needs some work done to refine the annotations and make more accurate. Also needs to merge with @Kianenigma and finish the rest of the modules in Substrate / Polkadot repos
related: #2431

Why is the weight always closed? I don't think that this is valid docs syntax.

This is intended so that this can easily be scraped out (requested by w3f). And this is not a rustdoc syntax hence it should be # <weight> @lsaether so that it is not included in the generated docs.

Why is the weight always closed? I don't think that this is valid docs syntax.

This is intended so that this can easily be scraped out (requested by w3f). And this is not a rustdoc syntax hence it should be # <weight> @lsaether so that it is not included in the generated docs.

Ty, I assumed it, but was not sure.
Said this, could we not just use a markdown parser that gives us everything that belongs to the Weights heading?

Requesting re-review of formatting @joepetrowski

this is a lot of commenting. the only comments that actually matter are those for the public dispatchable functions (though auditing may be helped with the rest of the function).
rustdoc should not be used for any of the non-public-dispatchable functions as it only serves to clutter the documentation.

this is a lot of commenting. the only comments that actually matter are those for the public dispatchable functions (though auditing may be helped with the rest of the function).
rustdoc should not be used for any of the non-public-dispatchable functions as it only serves to clutter the documentation.

Will change all the private fns to //.
Last note: https://github.com/paritytech/substrate/pull/2694/files#diff-b5a024b4f4ae96b0bfc0681acae56709R567
You already have a note about this being bad: Whoever bonds, we create two storage items for them and never remove them unless if the account is dust. A nice optimization could be to remove if the bonded value is less than dust, not the balance of the account.

@paritytech/ci CI needs to be retried.  @lsaether can you push a commit?  It can even be empty ― it just needs to trigger the CI bot.

@paritytech/ci CI needs to be retried. @lsaether can you push a commit? It can even be empty ― it just needs to trigger the CI bot.

done


Inability to process x86 assemblies

Neither NSwag Studio nor command line utility can open x86 assemblies. Is this by design or can be somehow fixed?
Command-line utility stacktrace:
PS C:\Workspace\Projects\***> ..\packages\NSwag.MSBuild.11.3.3\build\NSwag.exe run nswag.json
NSwag command line tool for .NET 4.6+, toolchain v<IP_ADDRESS> (NJsonSchema v<IP_ADDRESS>) (x64)
Visit http://NSwag.org for more information.
NSwag bin directory: C:\Workspace\Projects\***\packages\NSwag.MSBuild.11.3.3\build

Executing file 'nswag.json'...
System.BadImageFormatException: Could not load file or assembly 'file:///C:\Workspace\Projects\***\***\bin\Release\***.exe' or one of its dependencies. An attempt was made to load a program with an incorrect format.
File name: 'file:///C:\Workspace\Projects\***\***\bin\Release\***.exe'

Server stack trace:
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.WebApiAssemblyLoader.<>c.<GetControllerTypesAsync>b__2_0(String path)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.WebApiAssemblyLoader.<GetControllerTypesAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.WebApiAssemblyLoader.<GenerateForControllersAsync>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.WebApiAssemblyLoader.GenerateForControllers(IEnumerable`1 controllerClassNames, String settingsData)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.WebApiAssemblyLoader.GenerateForControllers(IEnumerable`1 controllerClassNames, String settingsData)
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.SwaggerGeneration.WebApi.WebApiAssemblyToSwaggerGenerator.<GenerateForControllersAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.WebApiToSwaggerCommandBase.<<RunAsync>b__77_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.WebApiToSwaggerCommandBase.<RunAsync>d__77.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.WebApiToSwaggerCommandBase.<RunAsync>d__76.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.NSwagDocumentBase.<GenerateDocumentAsync>d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.NSwagDocumentBase.<ExecuteAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.Document.ExecuteDocumentCommandBase.<ExecuteDocumentAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NSwag.Commands.Document.ExecuteDocumentCommandBase.<RunAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NConsole.CommandLineProcessor.<ProcessSingleAsync>d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NConsole.CommandLineProcessor.<ProcessAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NConsole.CommandLineProcessor.Process(String[] args, Object input)
   at NSwag.Commands.NSwagCommandProcessor.Process(String[] args)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

NSwag Studio screenshot:

It seems that a x64 process (NSwag by default) cannot load x86 DLLs - only Any CPU and x64. If you know a way to solve this problem please share it here? Or is it possible to start a .NET process as x86?

Correct. Unfortunately I don't think there's a clean and easy way except creating separate binaries targeting x86. E.g. NSwag.x86.exe. Do you think that's possible?
As an option, if you are against creating separate build configurations for x86, there's a CorFlags utility that modifies built AnyCPU executables to run as x86. So, it can be integrated into the build process of NSwag.

NSwagStudio and NSwagConsole are already built with AnyCPU but with "Prefer 32 bit" disabled.

You'll need to create separate binaries to be able to load x86-only or x64-only binaries. See https://stackoverflow.com/a/5229805/247702
@akamyshanov I assume you have your reasons for targetting x86 instead of AnyCPU?

How about using something like that in the NPM package if a specific flag is set?
http://coolsoft.altervista.org/en/blog/2012/01/run-anycpu-net-applications-x86-mode

@stijnherreman Exactly :) x86-only dependencies using 32-bit native DLLs.
My current workaround is to build AnyCPU (and ignore the produced warnings) just to feed it to NSwag.
@RSuter Also a valid workaround, but having a separate x86 NSwag binary seems cleaner to me. Maybe I'm not the only one with such a problem.

And how can you choose which version of the .exe to run if both binaries are available?

MSBuild => Choose between $(NSwagExe) and $(NSwagExe_x86)
NPM => Add --x86 option
NSwagStudio => ??

What do you think:
https://github.com/RSuter/NSwag/pull/876
?

@RSuter Wow, that was fast! 👍
Cannot say anything about the NPM part but the console and MSBuild parts seem ok to me.
BTW, why create a new project instead of adding a new x86 configuration to the existing one? In order not to build the solution twice?

How can we do that? I thought only multiple target frameworks are supported?

@RSuter
https://github.com/akamyshanov/NSwag/commit/9fc4e89be9a0b567a81159d9b399789cb4190469
However, you would need to build the solution twice in order to get both build results, I thought that's why you created a new project in the first place.

Hmm, if i need to build it twice i think its better as is because it is much harder to integrate in the automated builds...

I agree. Building the whole solution twice just for one project is not the best option from the automated build point of view. Even though it's less "code" to maintain.
Thank you for the quick reaction, will wait for the next release with these changes included.

Released with v11.3.5


When using the code on PHP8.1, the following notices will show up:

On  spatie/schema-org:^3.4
Deprecated function : Return type of Spatie\SchemaOrg\BaseType::offsetSet($offset, $value) should either be compatible with ArrayAccess::offsetSet(mixed $offset, mixed $value): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice dans include() (/project/vendor/spatie/schema-org/src/BaseType.php ligne 12)
As I could not re-open issue #178 , lets try a new one.

Seeing the same Deprecated notice on our website as well.

@pguillard  Upgrading spatie/schema-org to 3.11 fixed our error message.
Seems to be fixed on #174, please confirm.


Add sharable filters functionality

What about a button appears at the bottom of filters when filters are applied? A button that says:
"Copy URL for this filter"
Clicking that could copy a URL like the following:
https://turkiyereliefdao.com?cryptocurrencies=Polkadot
When such a URL is loaded, the query parameters must be deleted and the state better is persisted (#35) on page refreshes.
Another button next to it could say:
"Reset filter"
Advantage:

Relevant subsets of options could be shared with networks, resulting in better targeting and conversion.

Disadvantages:
*Harder to implement than most of the other issues i.e. opportunity cost.
[1] Need to create persistent state management.  Even more work. Could be delegated to another issue. The disadvantage of this

Closing the issue since it's resolved in #71.


Train ML Model with Execution in SAP AI Core (Postman)

Tutorials: https://developers.sap.com/tutorials/ai-core-aiapi-postman-train.html
I believe there is a bug in step 2.
I've been following the tutorial and noticed that in Tutorial No. 3 we define a pathPrefix (tutorial) for the object store. Now, in step 2 of this tutorial we are defining the URL as ai://default/data, when it should be ai://tutorial/data. If I leave it as default, it is not possible to train the model.

Please ignore this issue. I believe there was some confusion on my end when going through the tutorial and dealing with all the Ids :-).