some incompatible instantiation of T when checking polymorphic react components

I'm not sure that this is the right way to do this, but I'm not sure how to do it otherwise. I'm trying to type a component that takes another component as a prop and then expects to receive that component's required props as its own.
I've tried a couple of things and I can't get anything to work properly. For some
/* @flow */
import React from 'react'

type Props = {
  foo: string
}

type Props2 = {
  bar: string
}

type FieldProps<T> = {
  component: ReactClass<T>
} & T;

class Input extends React.Component {
  props: Props
}

class Input2 extends React.Component {
  props: Props2
}

class Field<T> extends React.Component {
  props: FieldProps<T>
}

const x = <Field component={Input} foo="hi" />
const y = <Field component={Input2} bar="hi" />

https://flowtype.org/try/#0PQKgBAAgZgNg9gdzCYAoAlgWwA5wE4AuYASgKYCGAxkVHnJmAOR4XWOqoECe2pYACnWwBnMAF4wAb1RgwUOHABcYYQTzoAdgHNUAXw7deAocIBM4qTLAAjcnmWr12vQZ58AYulIwAJoLgiADwAKgB8FtKylPS4GqQaBMpkVAQAwjDkwsIhoXpgAGRgwQDcVqiUGVlgAJIa2ACuRKQAHgTxPqLJ1AB0qTFwcQmWstgmyv4iLuWVorUNBOYtbRodJKwEvf2DRJFgowHC4yamUxWZop7ePjlgS+2d65s4A-E7Vvsiype+E9lhpwNVGBmhZAt8fGBos9tmJJHNGro5AoxAAiAAW6BRYGAuWiGiBXFB4MhW1esPhC0RtjwqIxWJxHCAA
For some reason this one appears to work for a single Field but if you do more than one with different components in the same example it breaks:
/* @flow */
import React from 'react'

type Props = {
  foo: string
}

type Props2 = {
  bar: string
}

type FieldProps<T> = {
  component: ReactClass<T>
} & T;

class Input extends React.Component {
  props: Props
}

class Input2 extends React.Component {
  props: Props2
}

class Field extends React.Component {
  props: FieldProps<*>
}

const x = <Field component={Input} foo="hi" />;
const y = <Field component={Input2} bar="hi" />;

https://flowtype.org/try/#0PQKgBAAgZgNg9gdzCYAoAlgWwA5wE4AuYASgKYCGAxkVHnJmAOR4XWOqoECe2pYACnWwBnMAF4wAb1RgwUOHABcYYQTzoAdgHNUAXw7deAocIBM4qTLAAjcnmWr12vQZ58AYulIwAJoLgiADwAKgB8FtKylPS4GqQaBMpkVAQAwjDkwsIhoXpgAGRgwQDcVqiUGVlgAJIa2ACuRKQAHgTxPqLJ1AB0qTFwcQmWstgmyv4iLuWVorUNBOYtbRodJKwEvf2DRJFgowHC4yamUxWZop7ePmBL7Z3rmzgD8TtW+yLKl74T2SC5+uUBqowM0LIEvtdok9tmJJHNGro5AoxAAiAAW6BRYGAoVK0Q0wK4YIhYChsResPhC0RtjwqIxWJxpQ4QA

I should also note that ultimately component can be a string as well, in which case it should get no additional props. I'm not sure how to type that either, but one step at  a time.

// @flow
import React from 'react'

type FunctionComponent
 = (props: P) => ?React$Element<any>;
type ClassComponent<D, P, S> = Class<React$Component<D, P, S>>;

type FieldProps
 = {
  component: ClassComponent<void, P, void> | FunctionComponent

} & P;

class Field
 extends React.Component<void, FieldProps
, void> {}

type Props = {
  foo: string
};
type Props2 = {
  bar: string
};

class Input extends React.Component {
  props: Props
}

class Input2 extends React.Component {
  props: Props2
}

const x = <Field component={Input} foo="hi" />;
const y = <Field component={Input2} bar="hi" />;

p.s.
Do not use ReactClass, it throws away type checking https://github.com/facebook/flow/issues/2159

Awesome, thank you @gcanti. I was able to add a | string to component's prop type to cover everything.
Was my attempt highlighting a bug or just a confusing error message?

AFAIK the problem is that you are "shadowing" the base class type parameter
class A<P1> {
  props: P1
}

class B<P2> extends A {
  props: P2
}

const x: B<{ foo: string }> = new B()

src/index.js:7
  7: class B<P2> extends A {
             ^^ P2. This type is incompatible with
  7: class B<P2> extends A {
           ^ some incompatible instantiation of `P2`

P2 might be incompatible with P1 because P2 is defined while P1 is inferred.
This works
class A
 {
  props: P
}

class B
 extends A
 {}

const x: B<{ foo: string }> = new B()

because you are explicitly saying that P2 is P1

As a rule I always use the React.Component<DefaultProps, Props, State> form in order to add typings

I had to switch to React$Component in my type def, React.Component silently let everything through.

@aaronjensen can this be closed now?

Yes. Thank you again @gcanti.   do you have an idea why React.Component did not work for me?

Weird. Is it in a libdef or in normal code? Can you post an example?

It was in a libdef. Try this one, but change line 80 to use React.Component https://gist.github.com/aaronjensen/54643af48aedb2f7eba424d9434ca99f

In a libdef is kind of expected, AFAIK there's no official way to import types from another library, hence the "global types" workaround (React$Element, React$Component, etc..),
I noticed that React.Component is also used in the reduxForm definition, is it working there?

Oh, that's obnoxious. Is this stuff documented somewhere? How do you know so much 😄
You're right that React.Component didn't work in that spot either. It was totally wrong any way, I really wanted a ReactClass or the alternative you proposed.

hmm, I don't know how to type reduxForm. I need it to take a component then return a component that requires the same props as the component that it took minus a few.
Basically I want something like this:
  declare type FormComponentProps = {
    handleSubmit: Function,
  }

  declare type FormComponent =
    FunctionComponent<$Diff
>;

  declare export function reduxForm<T>(config: FormConfig):
   (component: T) => FormComponent<*, T>

But that doesn't work, I'm guessing because P gets inferred down to Object effectively

What about?
type FunctionComponent
 = (props: P) => ?React$Element<any>;
type ClassComponent<D, P, S> = Class<React$Component<D, P, S>>;
type Component
 = ClassComponent<void, P, any> | FunctionComponent
;

type FormComponentProps = {
  handleSubmit: Function
};

declare function reduxForm
(config: FormConfig):
 (component: Component
) => FunctionComponent<$Diff
>

//
// tests
//

type Props = {
  name: string,
  handleSubmit: Function
};

const C1 = (props: Props) =>
{props.name}

class C2 extends React.Component<void, Props, void> {}

const RFC1 = reduxForm()(C1)
const RFC2 = reduxForm()(C2)

// $ExpectError
;<RFC1 />
// $ExpectError
;<RFC1 name={1} />
;<RFC1 name="Giulio" />

// ;<RFC2 /> // <= error
// ;<RFC2 name={1} /> // <= error
;<RFC2 name="Giulio" />

@gcanti that works better, but why don't two class ones work?

Not sure I'm follow, you mean why I wrote this?
// ;<RFC2 /> // <= error
// ;<RFC2 name={1} /> // <= error

Yes, sort of. If I uncomment those they do not fail, which I would expect/hope them to.
I'm also running into this: https://github.com/facebook/flow/issues/2541
So if I import a component from another module, pass it through a HoC, then import that in another module it loses its prop checking.

Now I have a version which works with class components but not with function components, kind of sad
libdef
  declare export type FunctionComponent
 = (props: P) => ?React$Element<any>;
  declare type ClassComponent<D, P, S> = Class<React$Component<D, P, S>>;
  declare type Component
 = ClassComponent<void, P, any> | FunctionComponent
;

  declare type FormComponentProps = {
    handleSubmit: Function
  };

  declare export function reduxForm
(config: FormConfig): (C: Component
) => FunctionComponent<$Diff
>;

a.js
// @flow
import React from 'react'

type Props = {
  name: string,
  handleSubmit: Function
};

export const C1 = (props: Props) =>
{props.name}

export class C2 extends React.Component<void, Props, void> {}

b.js
// @flow
import { reduxForm } from 'redux-form'
import { C1, C2 } from './a'

export const RFC1 = reduxForm({})(C1)
export const RFC2 = reduxForm({})(C2)

index.js
// @flow

import React from 'react'

import { RFC1, RFC2 } from './b'

;<RFC1 /> // <= no error :(
;<RFC1 name={1} /> // <= no error :(
;<RFC1 name="Giulio" />

// $ExpectError
;<RFC2 />
// $ExpectError
;<RFC2 name={1} />
;<RFC2 name="Giulio" />

very frustrating 😖

@aaronjensen Yeah, make sure type inference works properly in some advanced use cases already is hard, but when it involves React components is (currently) almost impossible. Often the culprit is that we have 2 kinds of components: functional and classes.

Hey @gcanti, following this pattern I'm trying to strongly type an HOC, but cannot seem to quite get it right, any thoughts on this example?
I can't seem to spot my error.

@rosskevin I'm running into a very similar issue as you, here's my example and getting a similar error. Were you able to figure your issue out?

@PeteJodo - no- didn't find a solution, ended up refactoring to a different way.


Syntax error on mysql install script

I get the following error when I try to install DB using install-rds.sql:
ERROR 1064 (42000) at line 648 in file: './data/name_data.sql': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Insert into name_data (name_type,name) values ('MALE_FIRST','Agustin')' at line 2
Operation failed with exitcode 1
I am using MySQL workbench to run the script and my database is
db.m4.large RDS instance with MySQL engine version 5.7.22. Please look into this issue. Thanks in advance.

Missing semicolon after previous insert statement. Checked in change.


GEOMESA-1038 Updating license headers

Headers can be verified by running 'mvn license:check'
Headers can be added/updated by running 'mvn license:format'
Travis build now runs the license check goal

looks good...merging

ed35f21623a9e5f17843ffeda0ebba270586b9cc


Class object and dataset can be used ?

Hello ,
Amplifier can be used? to create Numerous data tables from data objects like class objects etc.
In my case, one of my window applications used to perform data calculations is taking time to fill calculated variable results in data tables. I want to run this part on GPU so that performance can be achieved.
may I proceed with this using Amplifier ? Please guide @deepakkumar1984 .

Can you give me an example project which you want to transform. At this point I don't know the answer, I need to evaluate based on your requirement


Width issue on mobile

Hi,
In some episodes, the width of the episode page is exceeding the iPhone screen and forces you to scroll vertically to see all content. This only happens in some episodes.

I’m using an iPhone 12.
regards,

I'm guessing there is something like the tags pushing this wide since it's not happening on all pages.


fix(ci): simplify and fix multi-arch image publishing process

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug
/kind cleanup
/area CI

What this PR does / why we need it:
This is a refactor and fix of the container image publishing process. There are two major changes:

Images with arch-specific tags are pushed to Docker Hub (e.g. :x86_64-0.35.0). This is done because in order to create a multiarch (index, manifest.list) manifest easily we need manifests with the referenced digests already published. It is probably possible to do it without (buildx can do something similar) but it may involve a local registry or some more complicated process. I think this is fine for now.
Instead of building images multiple times and then publishing all of them, this one only builds one version (docker.io/falcosecurity/falco) and then adds any additional tags or registries (ECR, latest tags...). This simplifies things especially when the next PR will be published, which is related to signatures with cosign.

Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Note that the arch specific tags are not pushed to ECR. We can change this behavior
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: LucaGuerra
Once this PR has been reviewed and has the lgtm label, please assign fededp for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: ade5420f7b4d07d0e8bdd63c1f4dd8b1d28c2b56

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Refs https://github.com/falcosecurity/falco/pull/2501


Activation scripts should be POSIX compliant

Solution to issue cannot be found in the documentation.

[X] I checked the documentation.

Issue
I'm attempting to use tox-conda to create and activate conda environments, and it reports error messages like:
sh: 6: /home/omad/miniconda3/envs/odc-core/etc/conda/activate.d/geotiff-activate.sh: [[: not found

This could be fixed within tox-conda by using /bin/bash instead of /bin/sh, however, I think it actually points to an issue with the activation scripts in this repo.
The Deep dive: conda init and activate — conda 4.13.0 documentation shows that activate/deactivate scripts are sourced by the executing shell, not run as independent scripts. This means the #! leading line is being ignored (and as such is misleading), and the activation scripts only work with /bin/bash.
They should be rewritten to be POSIX compliant.
Installed packages
not relevant

Environment info
active environment : odc-core
    active env location : /home/omad/miniconda3/envs/odc-core
            shell level : 1
       user config file : /home/omad/.condarc
 populated config files : /home/omad/.condarc
          conda version : 23.5.2
    conda-build version : not installed
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=6.2.6=0
                          __unix=0=0
       base environment : /home/omad/miniconda3  (writable)
      conda av data dir : /home/omad/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/omad/miniconda3/pkgs
                          /home/omad/.conda/pkgs
       envs directories : /home/omad/miniconda3/envs
                          /home/omad/.conda/envs
               platform : linux-64
             user-agent : conda/23.5.2 requests/2.31.0 CPython/3.10.12 Linux/6.2.6-76060206-generic pop/22.04 glibc/2.35
                UID:GID : 1000:1000
             netrc file : /home/omad/.netrc
           offline mode : False

Happy to receive a PR for this :smile:  I know we had a lot of trouble with libxml2 doing this, I think https://www.shellcheck.net/ helped in the end.


Endless loop when updating a record after creation in batch mode

OrientDB Version: 3.0.22
Java Version: 1.8
OS: Linux
Expected behavior
The record should be updated just once.
Actual behavior
There is an endless loop. The record is updated multiple times.
Steps to reproduce
Use the following SQL script in batch mode :
let a = CREATE VERTEX V SET test = 0; UPDATE $a SET test = $a.@rid;
Hope this helps.
Cheers.

Same problem


Specify files

Also emit types.  Note that this isn't useful yet since the types aren't
exported within main (dist/index)

$ tar tfz malleatus-nyx-v0.2.0.tgz

package
package/CHANGELOG.md
package/LICENSE
package/RELEASE.md
package/dist
package/package.json
package/dist/commands
package/dist/index.d.ts
package/dist/index.js
package/dist/index.test.d.ts
package/dist/index.test.js
package/dist/commands/report-failure.d.ts
package/dist/commands/report-failure.js
package/dist/commands/report-failure.test.d.ts
package/dist/commands/report-failure.test.js

11:18:18 ❯ tar tfz malleatus-nyx-0.2.0.tgz  | sort
package/CHANGELOG.md
package/LICENSE
package/RELEASE.md
package/dist/commands/report-failure.d.ts
package/dist/commands/report-failure.js
package/dist/index.d.ts
package/dist/index.js
package/package.json

Note that this requires npm pack.  The exclusion pattern in files doesn't seem to work with yarn pack.

11:46:48 ❯ tar tfz malleatus-nyx-0.2.0.tgz | sort
package/CHANGELOG.md
package/LICENSE
package/dist/commands/report-failure.d.ts
package/dist/commands/report-failure.js
package/dist/index.d.ts
package/dist/index.js
package/package.json


Problem with apt-get update before installing

Hi, I'm using Elementary OS (Loki) and for the past few days, I've been struggling with installing Nuvola Player. I'm following this tutorial http://www.edivaldobrito.com.br/como-instalar-o-nuvola-player-no-ubuntu-debian-e-derivados/ it's in english, but I think you can get the hang of it.
This is my terminal log when I sudo apt-get update:
sudo apt-get update
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Hit:2 http://ppa.launchpad.net/atareao/atareao/ubuntu xenial InRelease
Hit:3 http://repo.steampowered.com/steam precise InRelease
Hit:4 http://packages.elementary.io/appcenter xenial InRelease
Hit:5 http://repository.spotify.com stable InRelease
Hit:6 http://ppa.launchpad.net/elementary-os/stable/ubuntu xenial InRelease
Hit:7 http://ppa.launchpad.net/elementary-os/os-patches/ubuntu xenial InRelease
Ign:8 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ InRelease
Hit:9 http://ppa.launchpad.net/team-xbmc/ppa/ubuntu xenial InRelease
Ign:10 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Release
Ign:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
Hit:12 http://ppa.launchpad.net/wfg/0ad/ubuntu xenial InRelease
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en
Ign:15 https://tiliado.eu/nuvolaplayer/repository/deb loki InRelease
Ign:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
Ign:16 https://tiliado.eu/nuvolaplayer/repository/deb loki Release
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:17 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable amd64 Packages
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en
Ign:18 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable i386 Packages
Ign:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
Ign:19 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable all Packages
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en
Ign:20 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable Translation-en_US
Ign:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
Ign:21 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable Translation-en
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:22 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable amd64 DEP-11 Metadata
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en
Ign:23 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable DEP-11 64x64 Icons
Hit:24 http://br.archive.ubuntu.com/ubuntu xenial InRelease
Hit:25 http://br.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:26 http://br.archive.ubuntu.com/ubuntu xenial-backports InRelease
Ign:17 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable amd64 Packages
Ign:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
Ign:18 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable i386 Packages
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:19 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable all Packages
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en
Ign:20 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable Translation-en_US
Err:11 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Packages
404  Not Found [IP: <IP_ADDRESS> 80]
Ign:21 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable Translation-en
Ign:13 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Translation-en_US
Ign:14 http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./  Done
W: Target Packages (non-free/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (non-free/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (non-free/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Translations (non-free/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Translations (non-free/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target DEP-11 (non-free/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target DEP-11-icons (non-free/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Translations (stable/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target DEP-11-icons (stable/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Packages (stable/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Translations (stable/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target DEP-11-icons (stable/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: The repository 'http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1 ./ Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: The repository 'https://tiliado.eu/nuvolaplayer/repository/deb loki Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch http://download.opensuse.org/repositories/home:/strycore/xUbuntu_0.4.1/./Packages  404  Not Found [IP: <IP_ADDRESS> 80]
E: Failed to fetch https://tiliado.eu/nuvolaplayer/repository/deb/dists/loki/stable/binary-amd64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
W: Target Packages (non-free/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (non-free/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (non-free/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Translations (non-free/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Translations (non-free/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target DEP-11 (non-free/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target DEP-11-icons (non-free/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/spotify.list:1 and /etc/apt/sources.list.d/spotify.list:2
W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Translations (stable/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target DEP-11-icons (stable/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:2
W: Target Packages (stable/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Packages (stable/binary-i386/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Packages (stable/binary-all/Packages) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Translations (stable/i18n/Translation-en_US) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target Translations (stable/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target DEP-11 (stable/dep11/Components-amd64.yml) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
W: Target DEP-11-icons (stable/dep11/icons-64x64.tar) is configured multiple times in /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:1 and /etc/apt/sources.list.d/tiliado-nuvolaplayer.list:3
So, when I use sudo apt-get install nuvolaplayer3, I obviously come up empty. What am I doing wrong?
Thanks

Hello @cmguigamf. The instructions are for obsolete Nuvola 3.0.9, which will reach the end of life on December 31st, 2017 and its repository will be deleted. Nuvola 4.8 is the latest available release and it officially supports elementaryOS Loki.

Err:17 https://tiliado.eu/nuvolaplayer/repository/deb loki/stable amd64 Packages
404 Not Found

As you can see, the Nuvola 3 repository does not provide packages for Loki.

Run sudo rm -f /etc/apt/sources.list.d/tiliado-nuvolaplayer.list.
Follow installation instructions for Nuvola 4.


Implement BST

We need to implement the BST using ruby.

Assign this task to me

@shaneahmad
Assigned.

Thanku

please assign this issue to me

@snehasish-pathak
Assigned.


Please support istio with auto inject

When istio is set to auto inject the executors fail to talk to the driver and jobs never finish.  Please add first-class support for istio.

@AceHack It's a little different in Istio. Istio don't support headless kubernetes svc by default. @liyinan926 Should we expose how to create the driver svc ?

Istio does support headless services as of 1.4
https://github.com/istio/istio/issues/7495
Istio does not work with job like things in Kubernetes
https://github.com/istio/istio/issues/6324
The problem is spark would need to use something like envoy pre-flight to properly wait for envoy to come up and then shut down pre-flight when the spark application ends.
https://github.com/monzo/envoy-preflight
We are using envoy-preflight for all other job in Kubernetes.

One work around that we tested is configuring the application to run outside the istio service mesh.

Running outside the service mesh is not really a valid work around we want all the observability a service mesh brings with our spark jobs.

Working with the upstream community, we have added a configuration workaround for Zookeeper along with Casssandra, Elasticsearch, Redis and Apache NiFi.  I am sure there are additional applications that are not compatible with sidecars.  Please inform the community if you know of any.
https://www.cncf.io/blog/2020/10/26/service-mesh-is-still-hard/
We have yet to really dig into enhancements in Spark 3, but we may need to accept that we are apart of the "applications that are not compatible with sidecars."

hi, what is exactly incompatible? we added an istio waiter to the image and executor starts. But then fails with:
20/11/19 07:55:58 INFO TransportClientFactory: Successfully created connection to spark.spark.svc/10.X.X.X:7078 after 99 ms (0 ms spent in bootstraps) 20/11/19 07:55:58 ERROR TransportResponseHandler: Still have 1 requests outstanding when connection from  spark.spark.svc/10.X.X.X:7078 is closed Exception in thread "main" java.lang.reflect.UndeclaredThrowableException 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1713) 	at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:64) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:281) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala) Caused by: org.apache.spark.SparkException: Exception thrown in awaitResult: 	at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:226) 	at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75) 	at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:101) 	at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:201) 	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:65) 	at org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:64) 	at java.security.AccessController.doPrivileged(Native Method) 	at javax.security.auth.Subject.doAs(Subject.java:422) 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) 	... 4 more Caused by: java.io.IOException: Connection from  spark.spark.svc/10.X.X.X:7078 closed 	at org.apache.spark.network.client.TransportResponseHandler.channelInactive(TransportResponseHandler.java:146) 	at org.apache.spark.network.server.TransportChannelHandler.channelInactive(TransportChannelHandler.java:108) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at io.netty.handler.timeout.IdleStateHandler.channelInactive(IdleStateHandler.java:277) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:75) 	at org.apache.spark.network.util.TransportFrameDecoder.channelInactive(TransportFrameDecoder.java:182) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:224) 	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1354) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:245) 	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:231) 	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:917) 	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:822) 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) 	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) 	at java.lang.Thread.run(Thread.java:748)
What can it be?

Its discussed here: https://github.com/istio/istio/issues/27900. So may be Istio has a fix in upstream but may have not released it yet.

Its discussed here: istio/istio#27900. So may be Istio has a fix in upstream but may have not released it yet.

There's nothing for Istio to fix there - running Spark in vanilla standalone cluster mode thru Istio requires a small amount of Spark master configuration is all - I've tested Spark and Istio in that fashion and it works fine, workers talk to masters thru Istio via headless services.
If on top of that spark-on-k8s-operator has some additional incompatibility with Istio, more information is needed on what the actual error people are experiencing is.

Its discussed here: istio/istio#27900. So may be Istio has a fix in upstream but may have not released it yet.

There's nothing for Istio to fix there - running Spark in vanilla standalone cluster mode thru Istio requires a small amount of Spark master configuration is all - I've tested Spark and Istio in that fashion and it works fine, workers talk to masters thru Istio via headless services.

As @bleggett said, 2 has a number of workarounds. @AceHack already mentioned envoy-preflight.
For 1, In Spark apps created by Spark operator and In general for Spark on k8s, Driver listens on the POD_IP. Istio has been dependent on the primary container to be listening on <IP_ADDRESS>. This part is hardcoded in the Spark code.
https://github.com/apache/spark/blob/3a299aa6480ac22501512cd0310d31a441d7dfdc/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverServiceFeatureStep.scala#L34

istio/istio#27900 is a fix for that, Istio should be able to work when the driver pod listens on POD_IP.
Spark Master and workers, they are a different way of deployment. In a way they are independent of being deployed on Kubernetes or Mesos as they are not using the platform to create workers/executors.

As @bleggett said, 2 has a number of workarounds. @AceHack already mentioned envoy-preflight.
For 1, In Spark apps created by Spark operator and In general for Spark on k8s, Driver listens on the POD_IP. Istio has been dependent on the primary container to be listening on <IP_ADDRESS>. This part is hardcoded in the Spark code.
https://github.com/apache/spark/blob/3a299aa6480ac22501512cd0310d31a441d7dfdc/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/DriverServiceFeatureStep.scala#L34

istio/istio#27900 is a fix for that, Istio should be able to work when the driver pod listens on POD_IP.
Spark Master and workers, they are a different way of deployment. In a way they are independent of being deployed on Kubernetes or Mesos as they are not using the platform to create workers/executors.

@AceHack @puneetloya For the bitnami Spark Helm chart, all you have to do is set the SPARK_MASTER_HOST environment variable to <IP_ADDRESS> - no Istio changes are required.
Example values.yaml I pass to the Bitnami Spark chart:
master:
#   ## Annotations to add to the statefulset
#   ##
#   ##
#   podAnnotations:
#     proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": true }'
#     sidecar.istio.io/rewriteAppHTTPProbers: "false"
    extraEnvVars:
     - name: SPARK_MASTER_HOST
       value: "<IP_ADDRESS>"

Works fine.

@AceHack @puneetloya For the bitnami Spark Helm chart, all you have to do is set the SPARK_MASTER_HOST environment variable to <IP_ADDRESS> - no Istio changes are required.
Example values.yaml I pass to the Bitnami Spark chart:
master:
#   ## Annotations to add to the statefulset
#   ##
#   ##
#   podAnnotations:
#     proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": true }'
#     sidecar.istio.io/rewriteAppHTTPProbers: "false"
    extraEnvVars:
     - name: SPARK_MASTER_HOST
       value: "<IP_ADDRESS>"

Works fine.

@bleggett I think this advice is outdated with v1.10 that changes the default route envoy forward inbound packets to, to be eth0 rather than lo.
Also on this topic, can we make this operator name the ports according to their protocols?
$ istioctl analyze -n flows
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name blockmanager (port: 7079, targetPort: 7079) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name driver-rpc-port (port: 7078, targetPort: 7078) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name spark-ui (port: 4040, targetPort: 4040) doesn't follow the naming convention of Istio port.
Info [IST0118] (Service sparkpi-py-ui-svc.flows) Port name spark-driver-ui-port (port: 4040, targetPort: 4040) doesn't follow the naming convention of Istio port.

@haf Does #1239 address this?

Info [IST0118] (Service sparkpi-py-0ae54679adebbb65-driver-svc.flows) Port name driver-rpc-port (port: 7078, targetPort: 7078) doesn't follow the naming convention of Istio port.

AFAIK, the naming of ports is not controlled by the spark operator. Its a code change required in the Apache Spark distribution(this was valid atleast till Spark 2.4, I am not sure if this was changed in Spark 3). The headless service is created by the driver once it is up, to make itself discoverable to executors.

@jkleckner Yes, but it would probably make sense to change the default name, too.

Asking here, since it's seems to be the thread. Currently trying to make the Spark operator work with the following stack:

Kubernetes v1.20.8-gke.1500
Istio 1.9.6 (Anthos Service Mesh more specifically)
Spark operator 1.1.6

I have enabled the istio autoinjection on the namespace where I create my SparkApplication (I'm using the spark-pi example)
I have customised the docker image I'm using to include scuttle
I have tried to set the mTLS policy of istio to DISABLE and PERMISSIVE via a PeerAuthentication resource

Whatever I do, the executor fails with the following error:
21/08/24 16:43:47 INFO TransportClientFactory: Successfully created connection to spark-pi-bb55307b78fb84d6-driver-svc.spark.svc/<IP_ADDRESS>:7078 after 151 ms (0 ms spent in bootstraps)
21/08/24 16:43:47 ERROR TransportClient: Failed to send RPC RPC<PHONE_NUMBER>669017718 to spark-pi-bb55307b78fb84d6-driver-svc.spark.svc/<IP_ADDRESS>:7078: java.nio.channels.ClosedChannelException
java.nio.channels.ClosedChannelException
	at io.netty.channel.AbstractChannel$AbstractUnsafe.newClosedChannelException(AbstractChannel.java:957)
	at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:865)
...

I'm not sure what I'm doing wrong: from various comments on this thread and others, this should work, no? The issue with the istio master listening on the pod ip and not <IP_ADDRESS> has been fixed in the versions I'm using.
If anyone has an idea, I'd be very thankful!

Hi @MrTrustor, did you resolve your issue? I am running into the same thing, though with slightly older istio so I am setting SPARK_MASTER_HOST

Failed to send RPC RPC xx to xx: java.nio.channels.ClosedChannelException
FWIW, I was able to get executor->driver communication to work by overriding the env var SPARK_DRIVER_BIND_ADDRESS to <IP_ADDRESS> on the driver. The default is hardcoded to PodIP and was not working with our version of istio.

We were facing the same error as well org.apache.spark.shuffle.FetchFailedException: Failed to send RPC RPC<PHONE_NUMBER>843038265 to /xxx: java.nio.channels.ClosedChannelException, and we were able to get communication between driver & executor pod up & running by the following :

Set up the blockManager port by adding the SparkConf spark.blockManager.port: "7080"
Create the following headless service :

apiVersion: v1
kind: Service
metadata:
  name: spark-executor
spec:
  clusterIP: None
  ports:
  - name: blockmanager
    protocol: TCP
    port: 7080
  selector:
    spark-role: executor

Note that it doesn't have to be 7080 port, instead it can be any port number. By default it's assigning a random port.
Credits to @dayyeung


[FLINK-33608][test-utils] UpsertTestDynamicTableSink's keySerializationSchama should extract and serialize the primary key fields from RowData

What is the purpose of the change
[FLINK-33608][test-utils] UpsertTestDynamicTableSink's keySerializationSchama should extract and serialize the primary key fields from RowData
CREATE TABLE UpsertFileSinkTable (
  user_id INT,
  user_name STRING,
  user_count BIGINT,
  PRIMARY KEY (user_id) NOT ENFORCED
) WITH (
  'connector' = 'upsert-files',
  'key.format' = 'json',
  'value.format' = 'json',
  'output-filepath' = '..'
);

INSERT INTO UpsertFileSinkTable
SELECT user_id, user_name, COUNT(*) AS user_count
FROM (VALUES (1, 'Bob'), (22, 'Tom'), (42, 'Kim'), (42, 'Kim'), (42, 'Kim'), (1, 'Bob'))
  AS UserCountTable(user_id, user_name)
GROUP BY user_id, user_name;

Results:

Key
Value

{"user_id":1,"user_name":"Bob","user_count":2}
{"user_id":1,"user_name":"Bob","user_count":2}

{"user_id":22,"user_name":"Tom","user_count":1}
{"user_id":22,"user_name":"Tom","user_count":1}

{"user_id":42,"user_name":"Kim","user_count":3}
{"user_id":42,"user_name":"Kim","user_count":3}

Expected:

Key
Value

{"user_id":1}
{"user_id":1,"user_name":"Bob","user_count":2}

{"user_id":22}
{"user_id":22,"user_name":"Tom","user_count":1}

{"user_id":42}
{"user_id":42,"user_name":"Kim","user_count":3}

Verifying this change
This change is already covered by existing tests UpsertTestDynamicTableSinkITCase.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not documented)

Hi @alpreu, could you help review this PR when you have time?
Thanks a lot.

@ruanhang1993 Would you like to help review this PR？

Hi @alpreu, could you help review this PR when you have time? Thanks a lot.

Hi @Jiabao-Sun, I just saw your PR. On a first scan it looks good to me but I am not a Flink committer. One small nit, there is a typo in keySerializationSchEma in the commit message


rag-chatbot.ipynb: Add correct langchain version

Problem
Describe the purpose of this change. What problem is being solved and why?
Solution
Describe the approach you took. Link to any relevant bugs, issues, docs, or other resources.
Type of Change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Infrastructure change (CI configs, etc)
[ ] Non-code change (docs, etc)
[ ] None of the above: (explain here)

Test Plan
Describe specific steps for validating this change.

there was a bug in 355 right? Will merge your tweak, thanks!


Benchmarks, e.g. Fig2 of Kraskov et al. 2004

Hi Cottrell,
I recently started to explore the existing implementation of mutual information estimation. I have seen that you are also using these algorithms.
Apparently there is a simple implementation of Kraskov2004 eq(8) in sklearn at https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/feature_selection/_mutual_info.py for 2 1-D variables, which I believe can be easily modified to higher dimensional variables.
By testing basically the setup of two correlated normal distributed variables as in Fig.2, it seems to give the correct answer as far as N is large. However, it seems to be much worse in convergence compared to what is said in Fig.2: "Error bars are smaller than the sizes of the symbols" according to them. For sklearn implementation N=100, 1000, the error varies a lot.
I am wondering if you have done this kind of benchmark for your implementation or other implementations? How's the convergence?
Cheers,

I feel like I must have looked at it but have no record of that in my notes ... I had some brief post here https://github.com/scikit-learn/scikit-learn/issues/16188 a while back.
The more I think about it, I feel like I started the process by using sklearn version but then hit "some issue" with it. I think a lot of these fail the invariance tests (invariant to monotonic transformations). I haven't thought about this for a while so the details are not in my head right now.


Change ArraySortHelper to use Comparison

The Array/List.Sort overloads that take a Comparison<T> have worse performance than the ones that take a IComparer<T>. That's because sorting is implemented around IComparer<T> and a Comparison<T> needs to be wrapped in a comparer object to be used.
At the same time, interface calls are slower than delegate calls so the existing implementation doesn't offer the best performance even when the IComparer<T> based overloads are used.
By changing the implementation to use Comparison<T> we avoid interface calls in both cases.
When IComparer<T> overloads are used a Comparison<T> delegate is created from IComparer<T>.Compare, that's an extra object allocation but sorting is faster and we avoid having two separate sorting implementations.
Port of https://github.com/dotnet/coreclr/pull/8504

Hi @mikedn, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

@mikedn I'll take a look at the Generics test failing tomorrow. This is unlikely to be a problem in your change. Shared generics are not completely done yet and this is either uncovering a bug or hitting a missing feature in shared generics.

Presumably this is caused by IntrospectiveSort(keys, index, length, comparer.Compare); where a delegate is bound to an interface method. Otherwise there's little else that's new, methods that used delegate existed before.
And the compilation exception is:
  Unhandled Exception: System.NotImplementedException: CORINFO_HELP_VIRTUAL_FUNC_PTR
     at Internal.JitInterface.CorInfoImpl.getHelperFtn(CorInfoHelpFunc ftnNum, Void*& ppIndirection)
     at Internal.JitInterface.CorInfoImpl._getHelperFtn(IntPtr thisHandle, IntPtr* ppException, CorInfoHelpFunc ftnNum, Void*& ppIndirection)
  --- End of stack trace from previous location where exception was thrown ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodCodeNode methodCodeNodeNeedingCode, MethodIL methodIL)
     at ILCompiler.RyuJitCompilation.ComputeDependencyNodeDependencies(List`1 obj)
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeDependencies(List`1 deferredStaticDependencies)
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes()
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.get_MarkedNodeList()
     at ILCompiler.RyuJitCompilation.CompileInternal(String outputFile)
     at ILCompiler.Compilation.ILCompiler.ICompilation.Compile(String outputFile)
     at ILCompiler.Program.Run(String[] args)
     at ILCompiler.Program.Main(String[] args)

That CORINFO_HELP_VIRTUAL_FUNC_PTR must be the delegate/interface method stuff.

Presumably this is caused by IntrospectiveSort(keys, index, length, comparer.Compare);

Yup, that's exactly what's going on. Since #2102 is not implemented yet, RyuJIT lands in Compiler::impImportLdvirtftn. On CoreRT we expect RyuJIT to ask for a ReadyToRun helper, but it doesn't seem like CoreCLR defines one when there's runtime lookup involved. We probably want to define a new helper here no matter when #2102 gets implemented to cover for the case of standalone ldvirtftn that is not used in delegate creation.
I'll see if I can cook up something or if I need @sandreenko's help on the RyuJIT side.
Sorry for the trouble.

I suppose I can take a look at the JIT side as well if you and @sandreenko have other priorities.

@dotnet-bot test this please

I suppose it's worth asking if such delegate calls have the same perf characteristics they have in the normal CLR. I'm getting the impression that in the current implementation the call will pass through a stub and that more work is needed to avoid this?

You are right - these calls do not have same perf characteristics in CoreRT right now, but we are going to fix that. https://github.com/dotnet/corert/issues/2342 is tracking fixing it. It should not prevent this change from going in.


Questions re the feed format we're using for Bluesky

If you have any questions about the RSS format we're using for Bluesky, I opened this thread for questions and hopefully answers.
Pointers

An example feed, my most recent Bluesky posts.
The thread where we worked on some of the details of the format.
The new element of the source namespace that's used to point to the user's feed list on Bluesky.
An example list, the list of feeds of people I follow on Bluesky. This is dynamic, when I subscribe to a new feed it automatically appears in this list.
A prototype app I put up to read the list and feeds, to get an idea how it works. The source for the app is in this repo.

@johnspurlock -- here's a change that's worth making when you get a chance, no rush.
In an OPML subscription list, change the category attribute on each <outline> to "all,bluesky".
That will make it possible to have a tab on the news page in FeedLand just for Bluesky feeds.
<outline type="rss"<EMAIL_ADDRESS>on Bluesky" xmlUrl="https://rss.firesky.tv?filter=from%3Akimkelly.bsky.social" htmlUrl="https://firesky.tv?filter=from%3Akimkelly.bsky.social" category="all,bluesky"/>

@johnspurlock -- another thing I'd like to see in the RSS feed for each user is their avatar.
Assuming you can get this info via the API.
Mastodon's RSS feeds have this feature. An example.
How they do it, using the RSS 2.0 image element.
    <image>
      <url>https://files.mastodon.social/accounts/avatars/000/040/242/original/de91b33cdcab3ac2.png</url>
      <title>Dave Winer ☕</title>
      <link>https://mastodon.social/@davew</link>
    </image>

alright, back in the saddle
both of these should be doable with minimal effort, which is the best kind of effort

@johnspurlock -- great! another thing we could use is the user's name and description on the OPML outline element for the user. Saves us from having to read the RSS feed to get that info.

Alright, just deployed some changes to the https://firesky.tv generator:

opml: title and description included in outline, if present (some profiles have no defined displayName or description)
opml: category="all,bluesky"
rss: include channel-level image element, if present (some profiles have no defined avatar)

Hope that helps!

@johnspurlock -- first thanks for picking up the ball here.
A concern -- even though the spec calls for text being the text that's displayed for the outline element, it's common for people to use title in its place. But we stick to the spec, of course. :-)
Here's an example to work with --

I omitted the other fields for brevity.
I think that we should shuffle things a little.

text should be what we're now calling title.
description should stay as it is.
the text that's currently in text needs to be given a whole different name, let's call it id and drop the @ and drop the " on Bluesky" bit, it's not helpful (I strip it in my code).

So we end up with something like this:

I updated the reader app to use the new atts in the OPML files. Here's a screen shot.

btw -- the more i think about it, the more i'm sure that the use of "title" as the attribute is a mistake. i probably have the best view of how OPML is used, and this is going to cause problems if it's used (obviously if it's not used, no problem).
i'm pretty sure the changes i asked for above are as easy as changing a couple of strings in the code. it's worth doing. i give you my word on that.

Sounds good to me - those new tweaks are now live!
👉 https://firesky.tv/lists/scripting.com/follows.opml

@johnspurlock -- thanks for making this change. I had just started working on "real" code based on the OPML files, and could clearly foresee problems when we ask other people to do this. This is all part of the process..
I have updated the blueskyreader demo app, and uploaded the new source.

@johnspurlock and anyone who is listening -- I want to tell the story of how I'm using this wonderful new facility in Bluesky in FeedLand. This is production code, not test code. And it's going in really smoothly.
Feed lists for users
FeedLand has the concept of a user feed list. All feed lists are public, as they are in social media apps. I can see your feed list, you can see mine.
There's a checkbox next to every feed, if you're subscribed to it, it's checked.
So if I'm looking at your list and see a feed that's interesting I can subscribe to it right there, by just clicking the checkbox.
There's also a wedge next to every feed, if I click the wedge I see the five most recent items in the feed.
A feed list from the outside world
Now we have a new idea, a feed list that came from outside of FeedLand.
It works the same way as above.
But the feeds do not have to already be in the database.
When you click the checkbox for a feed that isn't in the database, we add it, and subscribe you to it. This is exactly the process of subscribing to a new feed. Something FeedLand does every day.
If you click the wedge next to a feed that we don't have in the database, we read the feed and otherwise everything works exactly the same.
Why this is nice
What's absolutely beautiful about this is the we have a great source of new subscription lists, coming from Bluesky, and feeds from Bluesky. We've completely bridged the world of Bluesky to the world of FeedLand.
What's esp nice about this is that it should work FeedLand's competitors too. They all know how to read OPML subscription lists, it's a standard, and of course they all read RSS 2.0 feeds.
I think this is how it works -- feed readers are envelopes for the social media world. Right now only Bluesky has the necessary interfaces, but it didn't take long for @johnspurlock to implement it, it won't be a lot of work for the others. From here's it's a matter of finishing development, exciting users, and having them request the feature of the platforms. No small task of course.

BTW, the screen shot suggests the title could be shorter and the description should explain.

Title: Daniel Jalkut on Bluesky
Description: Everyone Daniel Jalkut follows on Bluesky, all from firesky.tv.

This way we can flow the title and description right into the UI, pretty sure this is how all feed readers will do it.

I have all this running on a test server locally, next I'll deploy it to blue.feedland.org so we can begin using it immediately.
I want to see if it works! At a technical level it appears to, but I want to see if it's useful in any way.

@johnspurlock and anyone who is listening -- I want to tell the story of how I'm using this wonderful new facility in Bluesky in FeedLand. This is production code, not test code.

I'm not on Bluesky yet, but the idea of browsing outside feed lists in FeedLand sounds exciting. This should also work between instances of FeedLand, so I could use my own instance as my main reader and get suggestions from feed lists feedland.org.
Maybe I could maybe keep my feed list synchronized on both instances to make my feeds easier to find.

Milestone -- blue.feedland.org, a server I deployed just for this project, has the new version of feedlanddatabase that supports viewing social media subscription lists in OPML like the ones we get for Bluesky.
I've subscribed to all the feeds I follow on Bluesky. This is my feed list.
This is my timeline, with no special provisions for Bluesky content. I can see some things aren't working, like interpreting double-linefeeds.
If you want go ahead and create an account on blue.feedland.org, and start subscribing and reading.

That's the normal stuff. Here's the new thing:
<EMAIL_ADDRESS>That opens a page entitled Everyone Daniel Jalkut follows on Bluesky.
If you're logged in, you can subscribe to any of those feeds by clicking the checkbox. Unsub by clicking again.
I chose his feed at random but he's an interesting guy, he makes a popular text editing tool that works with social media apps and blogging tools. It builds on something I created a long time ago, the Metaweblog API, so it's dear to my heart. ;-)
Also I consider many of the people he follows influential and know a few of them. So I've been using his subscription list as a test case as I developed this stuff.
BTW, it went in really easily. The code is well-structured for exactly this kind of adaptation.
Anyway, so the next step in the grand experiment is to make a very simple app that just works with Bluesky stuff, but of course is a front-end for the full FeedLand. I have a developer API I've been working on and this will be the second app I've built for it. In theory it should just snap together. But you never know until it's working. :smile:

Interesting!  Noticed that if you expand the wedge for some of the accounts, it will show no items.
like on<EMAIL_ADDRESS>I tried expanding a few of the names (like "Solana legend") and there were no items - is that because the underlying rss feed doesn't have items? Info sometimes reports "The feed is not in the database."

@johnspurlock -- glad you find it interesting.
this is new territory. first time we've hooked up feedland to a social network (or been able to) thanks to your work!
How it should work...
if you click on a wedge for a feed with no items it should say "This feed has no items" where the items would go.
Anything else is an error, so please report as a bug, with reproducible steps.  ;-)


[Doc-Enhacement] if we create team with teams template using graph api then it will take few minutes to create.

Hello Team,
We see if we create team with teams template using graph api then it will take few minutes to create. (based on channels and apps added in teams template)
Also, we can track team creation status using location endpoint as below.
If its mentioned in official document then it would be helpful for everyone to understand.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 24d884cb-c126-b5dc-ff90-1a31aa61bf5b
Version Independent ID: ed5054d7-d679-7fad-fd5d-9cd393644b9d
Content: Get started with team templates using Microsoft Graph - Microsoft Teams
Content Source: Teams/get-started-with-teams-templates.md
Service: msteams
GitHub Login: @MicrosoftHeidi
Microsoft Alias: heidip

To make it easier for you to submit feedback on articles on learn.microsoft.com, we're transitioning our feedback system from GitHub Issues to a new experience.
As part of the transition, this GitHub Issue will be moved to a private repository. We're moving Issues to another repository so we can continue working on Issues that were open at the time of the transition. When this Issue is moved, you'll no longer be able to access it.
If you want to provide additional information before this Issue is moved, please update this Issue before December 15th, 2023.
With the new experience, you no longer need to sign in to GitHub to enter and submit your feedback. Instead, you can choose directly on each article's page whether the article was helpful. Then you can then choose one or more reasons for your feedback and optionally provide additional context before you select Submit.
Here's what the new experience looks like.
Note: The new experience is being rolled out across learn.microsoft.com in phases. If you don't see the new experience on an article, please check back later.
First, select whether the article was helpful:

Then, choose at least one reason for your feedback and optionally provide additional details about your feedback:

Article was helpful
Article was unhelpful

Finally, select Submit and you're done!


[WIP] Update TS declarations

Importing natural in typescript projects using the provided type declarations won't work until the declaration files themselves can be compiled without errors. These are the minimum changes needed to resolve the remaining errors.

I haven't tested these changes yet (hence draft PR), but early feedback is certainly welcome.

@Hugo-ter-Doest - I am going to keep working my way through the source to add/update TS declarations as I can find time, but I wanted to get your preference on methods and properties in classes that are more for internal use than for consumers of this library.
The classes (defined as functions intended to be called with new) don't discern between public and private methods and properties. So, from an "accuracy" point of view, TS declarations that include every method and property would be most correct; they really are available to developers. On the other hand, some of them are probably only meant to be used internally. Unfortunately, there isn't a conclusive way for me to figure out which are meant to be public and which are meant to be private.
Are you ok with every property and method being included in the TS declarations? If not, then after I've finished updating this PR, would you be willing to go through the updated .d.ts files and add private access modifiers in front of properties and methods that should not be exposed to end users? For example:
export declare class SomeClass {
  type: string
  value: string
  private index: number

  constructor (type: string, value: string)
  getType (): string
  getValue (): string
  private setIndex (type: string, value: string): number
  private getIndex (): number
}

Yes, I am ok with publishing all properties and methods in a first run. Thank you for working on this and improving the TS declarations. My attempt is based on the DefinitelyTyped declarations but these are not complete and I want it to be part of the library instead of being external.
For most parts of the library I can decide which elements should be private and which public. Keep up the good work.

At some time in the future I want to add TypeScript tests as well. Now I only have some simple tests that cannot be run automatically as part of the CI pipeline.

And, if you have time, please check the linter errors. I am applying StandardJS in this library. So it will complain about string quotes (single) and semicolons (which are not allowed).

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.007%) to 87.281%

Totals

Change from base Build<PHONE_NUMBER>:
-0.007%

Covered Lines:
9615

Relevant Lines:
10645

💛  - Coveralls

And, if you have time, please check the linter errors. I am applying StandardJS in this library. So it will complain about string quotes (single) and semicolons (which are not allowed).

Yep, will do. This looks like forbidden use of T[] for non-primitive types. Need to use Array<T>. Easy enough, will address soon.
I plan to keep working on this incrementally, so apologies in advance if you get notifications for every "Pull Request Test Coverage Report for Build ..." It just takes too much time to decipher types to handle in one sitting.

I am merging this now, and checking for members that should become private in a separate PR.


Run conan config install also in Docker when CONAN_CONFIG_URL

Could the conan config install call be programmed into the system cron or whatever windows have? Maybe that is the correct way to go.

Do you have a specific objection to conan being able to run conan config install if configured to do so?
Your suggestion has very different implications and isn't really comparable.


Split uki config and bump package

Provides uki rootfs config

Doesnt touch normal systems, so merging ti directly


Smart cut does not recognize new industries

The new industry types, such as CattleFarm, MeatPackingPlant, Sandhouse, and watertower_drgw are not recognized by smart cut.

What is "smart cut"?

Smart cut is a Pine Valley only tool, as Lake Valley's trees to not work in the same way. Smart cut is used for cutting trees that are too close to industries or tracks.

Oh okay

But why can't you in pine vally cut all trees then use smart plant trees, thats what not working is when you cut all trees the smart plant plant trees does not work. becAUSE being able to see the inclines and and stuff helps lay the track plus you don't have to cut the trees.
then when your done with the track laying you can use smart plant trees to put them all bacvk or is that not how it works

not that smart cut does not work the cut all and plat all also is not working and seems they moved the tree grid a bit so that affects the alltrees.dat ? is the alltrees.dat just a copy of the tree data from the save file and just saved as .dat so the info would not dissepear ?

so the cut all and plant all trees will work until the save file is saved by the game and then the size goes from 5mb (trees cut and save downloaded from studio) to 10mb, same save file just overwritten by the game with cut all trees done to it

i see the problem now, they save the cut trees in vegetation not removedVegetationAssets and its a bit over my head to change that i get bunch of errors

This issue is for adding the new industries to Pine Valley tree tools. These tools operate on the removedVegitationAssets property which is only used with the Pine Valley map.
Support for Lake Valley, which uses the removedVegetationAssets property, is discussed in #41.

Resolved with #118 / #120


new automated testing step

steps:

uses: "bluejay/findCheck"
with:
values:
- value: 1
computationCount: 1
evidences:
- author:
login: "Antoniiosc7"
method: "TEST"

add deleted method for rollbacks


Added saving project settings to plugin tools

This PR fixes issue #582 by saving the project settings after running the "Enable OpenXR" or "Set Physics Layers" plugin tools.

good catch, lgtm


Warnings from xmltok dependency on installation

When installing globally this warning is displayed repeatedly:
../deps/libexpat/lib/xmltok.c:471:1: warning: missing field 'isName2' initializer [-Wmissing-field-initializers]
};
^

I just pushed a PR #30 to switch from xml2json to xml2js. That removes the warnings as well.


Update pass-user-service to create User resources with a first and last name

About
Updates the Shibboleth SP image, mapping the sn and givenName attribute names to Sn and Givenname respectively.  Those are the attribute names expected by the pass-authz ShibAuthUserProvider
Updates the authz and fcrepo images to use the 0.4.5 release of pass-authz.
Testing

Before checking out this PR, start pass-docker from the master branch.
Do not login to PASS normally.
Wait for Fedora to start completely (docker logs -f fcrepo)
Open an incognito browser, and visit https://pass.local/pass-user-service/whoami
When prompted, login as newSubmitter5, password moo.  Prior to logging in, this person does not have a User resource in PASS.  By logging in, the pass-user-service will create a User for newSubmitter5.
Inspect the resulting JSON, and insure that firstName and lastName are null.

Stop docker, remove containers.

Check out this PR
Run docker-compose pull (new fcrepo and authz images should be downloaded)
Start docker (docker-compose up -d)
Follow the same steps above, waiting for fcrepo then directly visit https://pass.local/pass-user-service/whoami without logging into PASS normally.  Login as newSubmitter5, password moo
Inspect the resulting JSON, and insure that firstName and lastName are not null

Followed above steps for existing master, verified expected behavior.
did docker-compose down, verified that containers were gone
then did
git checkout -b emetsger-authz-user-creation master
git pull https://github.com/emetsger/pass-docker.git authz-user-creation
docker-compose pull
docker-compose up
verified JSON has first name = New, lastName = Submitter


The namespace  already contains a definition

please help me fix the following…

The namespace  already contains a definition  error

Deleting the Extend/Digit folder should be the easiest way to fix this if you don't need anything from that folder.

thank you @YoruCathy it seems to go forward


Debian Buster package

Summary
Debian Buster will be out somewhen. I'ld like a CouchDB package for it.
I know most people thinks before release is too early, but me & others have to start testing things as soon as possible to be ready on release day. If the answer is "not yet", please leave this issue open & close is whenever done, so we can track the release of the package.
Desired Behaviour
buster/ in https://apache.bintray.com/couchdb-deb/dists/

[copied from dev list response from Joan Touzet]
CouchDB does not rush to release binaries for Debian/Ubuntu/CentOS/RHEL on major x.0 releases.
It is unlikely we will publish any buster package until after its official promotion to stable. No release date has been set yet for buster, though we're aware it is in freeze status. After release, we will then add buster to our CI matrix and ensure regression tests pass properly before releasing. If history serves, you can expect the package roughly a month after the buster release.
If you need it running on buster sooner, you are always welcome to compile from scratch. Bug reports and pull requests against our CI or packaging repos from issues you find are always welcome.


Information about the battery charge is not maintained in CGuard protocol.

Attach log & screenshot.
log.zip

All code in commit 19b17695867e00861c719292aad7ae48f33a8eeb

Added support for additional information reported by cGuard devices.


Terrible performance

Not sure why but when I run pycraft i can barely move the cursor and the game freezes every second or so.
On windows 10 if that matters.
Any ideas?

Probably providing more information about your hardware and software environment may help. Like specific OS (look for the screen about Windows, copy everything like 64 or 32 bits, Windows SP version, etc), and also your hardware (number of cores, amount of memory, etc). The more information, the more likely that one developer of the project will pick your issue to investigate that.

Besides a slightly buggy startup, it works fine for me (on Mac).
@traverseda is working on #25, so after that's done you might be able to pin down the problem by switching things off.


Update to next version of Go with security fixes

Required for all non-trivial PRs

[x] Rebased/mergable
[ ] Tests pass
[ ] CHANGELOG.md updated

We've been running unit tests successfully with Go 1.9 but I'm not aware of any formal integration tests with Go 1.9. I think at least some of the developers use Go 1.9 on their workstations.
Go 1.9.X and influxdb 1.4.0 is a conversation we need to revisit with the team.


Release is outdated

Hi guys,
I love the library and recently included it in a project. But I noticed that the issue regarding use of for..of is fixed yet still present in the npm version as your last release was Nov 2015. I'm using git instead of npm for now, but to help newcomers, could you please release a new version containing all recent fixes?
Thanks
-Liam

I've published the changes as v0.1.2. Let us know how that went.


Change FMDB embedding so Xcode 13.1 on M1 Monterey Mac finds it

Fixes #2578 for me
This PR is meant to share the changes I made so others can test them out, too. Not actually meant to be merged because I cannot verify that the app works as expected because of other problems.

Must've been my device 🤷


Add methods to enable and disable Location

This is already achievable by Patrol.tap, but is a bit cumbersome and lengthy.
We'd like to have a more high-level API (like we already have for Wi-Fi, Celluar, and Dark mode).
We want to get from this:
import 'package:patrol/patrol.dart';

import 'config.dart';

void main() {
  final patrol = Patrol.forTest();

  patrolTest(
    'toggles location',
    config: patrolConfig,
    ($) async {
      await patrol.openQuickSettings();
      await patrol.tap(const Selector(text: 'Location'));
      await patrol.tap(const Selector(text: 'Location'));
      await patrol.pressBack();
    },
  );
}

to this:
import 'package:patrol/patrol.dart';

import 'config.dart';

void main() {
  final patrol = Patrol.forTest();

  patrolTest(
    'toggles location',
    config: patrolConfig,
    ($) async {
      await patrol.enableLocation();
      await patrol.disableLocation();
    },
  );
}

Workaround mentioned didn't work for me since I don't have anything written on MIUI. I only have an icon, and depending on the device it might even be hidden (since Quick Settings can be edited).
Patrol could maybe open the Location settings screen which would be more easily accessible with text than the Quick Settings ?

@apalala-dev You can open the Settings app yourself. It's not possible to solve this in a generic way since the system apps differ across OEMs and Android versions.
void main() {
  patrolTest(
    'toggles location',
    nativeAutomation: true,
    ($) async {
      await patrol.openApp('com.android.settings');
      await patrol.tap(const Selector(text: 'Location')); // or whatever text your Settings app has
    },
  );
}

It is possible to open the location settings screen programmatically on the Android side (from my tests):
val callGPSSettingIntent = Intent(Settings.ACTION_LOCATION_SOURCE_SETTINGS)
activity!!.startActivity(callGPSSettingIntent)

On iOS, something similar seems to be doable from what I read on stackoverflow but I didn't test it.
If it is possible on both side, it might be a good idea to provide it directly from patrol, similarly to $.native.openQuickSettings().


colab

can you create a colab to test this please

Hi, the colab has been created and the modelscope demo is also available now.


Hours to seconds

Why:

Closes #1156
What's being changed:

@victoriacheng15 can you retest now


Fix (possibly hack) such that internal .md file links anchors are created correctly...

if you are not converting the whole .wiki tree from the root - but from a sub tree.
e.g. path to root of wiki is /.../Abc.wiki
but the supplied export directory is /.../Abc.wiki/Some/Lower/Level
There may be a more elegant way to do this - feel free to make it more so!

good catch - thank you for the PR!


Some clarifications and corrections

Thanks for the feedback! I've incorporated it into a separate update that I was doing, see here

#please-close


Upgrade openxr to 0.17

Hotham fails to build with rust 1.65 with an error in openxr. Updading openxr solves this.

Tested natively on the quest and with the simulator on the simple scene example.


error: Installed distribution pbr 1.0.1 conflicts with requirement pbr>=0.6,!=0.7,<1.0

pbr version conflicts. work around pip install -Iv pbr==0.11.0
First attempt to fix will be made June 2nd.

This bug is reported at https://bugs.launchpad.net/python-novaclient/+bug/1457100
To avoid pbr version conflict issue, install pbr==0.11.0.

requirements.txt updated.
pbr==0.11.0 will be installed prior to python-novaclient.

I think this will not quite work in case we use python setup.py install as it does not have version
What needs to be done?

i do not see a dependency on pbr

sorry wrong repo ;-)


Code: Reworked instance parameters with flexible structure

This is the implementation of the design described in https://github.com/kubernetes-incubator/service-catalog/pull/1075

/retest

@pmorie sorry, but I never had an expression that we intend to keep parameters they way they are at all. A week ago or so you were going to announce that we will break the parameters API soon, and it should have been redesigned completely. This implementation is a direct implementation of a subset of what was described in https://github.com/kubernetes-incubator/service-catalog/pull/1075
On Aug 1 it has been agreed that we implement it according to https://github.com/kubernetes-incubator/service-catalog/pull/1075 (plus blob support, which is documented there already), and stop discussing alternative proposals for now (extensions are still open for discussion).
@arschles can you comment on this?

That said we personally don't need anything apart from secret blob, and if everyone else is happy with just introducing parametersFrom - fine ¯_(ツ)_/¯

@pmorie @duglin ok fine, I will move the details of current proposal to a separate issue, and reduce the current to the minimum we have agreed upon, and which I fully implement. Since then, when we say "agree", we agree on the proposal. After that we shouldn't ask for any alternative changes in the implementation.
Does this work for you?

/retest

overall it looks good, but I'll hold off on the L GTM until the comments are addressed.

@duglin @pmorie PR updated, ready for the final review.

LGTM once all checks are green

already got my LGTM1


unsupported getDefaultProvider network for Polygon Mumbai

Describe the bug
We have now switched to Polygon Mumbai as our default network... but there is no default provider available in ethersjs for Polygon networks. This means that folks that don't have a provider API key for Infura, Alchemy, or Etherscan run into issues when trying to use Polygon.
To Reproduce
To reproduce, simply try to issue a mutating call (such as create or write) using the SDK when connected to Polygon or while simply using the defaults in NodeJS.
Expected behavior
There should at least be a sane default, or a handy error indicating how to fix this.
Additional context
This is due to our recent move to Polygon by default. We should try to have a default Polygon provider that we use internally to avoid this bug/issue.

I believe this was fixed somewhere in the last 7 months.  I'm closing as part of Linear cleanup.  If this still exists, let's reopen from inside Linear.


fix operator aggregated role.

ref: https://github.com/stolostron/backlog/issues/24609
Signed-off-by: morvencao<EMAIL_ADDRESS>

/lgtm
We may need to find a way in the future to ensure the permission is updated in manager can be updated in operator as well. maybe diff 2 files?

we should have script to sync the manager roles/clusteroles to the operator manifests.


chore(deps): update dependencies

Ping @okonet this PR updates all the dependencies, and also makes @dependabot increase the version number in package.json. Currently it only bumps the package-lock.json file, as can be seen here: https://github.com/okonet/lint-staged/pull/1063

It seems execa contains code that doesn't work on Node.js 12...

@okonet for now I downgraded<EMAIL_ADDRESS>because I couldn't get the tests to pass. I guess we would have to change all our tests to native ESM, meaning to use top-level await import(...) and the unstable jest.unstable_mockModule...

Would @sindresorhus have any pointers for us? The problem with<EMAIL_ADDRESS>is that we transpile all our tests with babel-jest for the mock support, and require('node:...') is not supported on Node.js 12 (only 14.18.0 and 16.0.0, as per history).
Is our only way forward to convert all the Jest tests to native ESM as well?


For histogram and other calculations: check for duplicate panoids

We should check for duplicate pano ids
Because some query lat,lngs may result in the same panoid

This is now fixed both in the .json production and the /vis python code


Add docs for composable service configuration

Extends the docs for composable services to add the changes for
configuration.
Signed-off-by: James Slagle<EMAIL_ADDRESS>

/override dataplane-operator-build-deploy-kuttl

/override pull-ci-openstack-k8s-operators-dataplane-operator-main-dataplane-operator-build-deploy-kuttl


v0.8.8_02: More space should be there under size scale bar

SEVERITY: Minor
MODULE: Stage
Platform:

Windows 7 / Chrome 48.0
Windows 7 / Firefox 43.0

STEPS TO REPRODUCE:

Navigate to URL: http://stage-tools.gapminderdev.org/tools/
Tools page is loaded.
Click “Options”
Note that there is less space under size scale bar

ACTUAL RESULTS:
There is less space under size scale bar in options
EXPECTED RESULTS:
More space should be there under size scale bar
SCREENSHOT:

Should be done with a redesign of options that we talked about before.


Kernel sessions manager

Hi,
Could this feature be available in VScode in the near future?

Kernel sessions manager: shows which kernels are running, and users can easily shut down kernels that are unnecessary.

Easy switch kernels: Users can choose which kernel to run on jupyter notebook / interactive window

This might be embedded on here?

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer

This is already partially available. If you click on the Kernel name in the upper right of the notebook, you can pick a different kernel.
The shutdown we could do as a separate command I think. "Shutdown Kernel(s)" or something.

@rchiodo
I should have mentioned that above feature request is related to Ipython kernal manager wihtin a Jupyter server.
JupyterLab kernal manager allows users to browse IPython kernels that are running within the Jupyter server. On the other hand, current VScode allows users to choose different Python versions to launch Jupyter server.

On the other hand, current VScode allows users to choose different Python versions to launch Jupyter server.

This isn't what's actually happening. We install a kernel for any python version you have (if you pick it), or you can pick one of the preinstalled python kernels that are from jupyter itself. So that list is actually a list of kernels.
However (for local sessions), we do not list out already running ones.

However (for local sessions), we do not list out already running ones.
Yeah, my request is related to this. Thanks for the clarification.

Currently, the kernel is created by VSCode with a random token name. It would be better if we can create the kernel with the name of the  notebook file (the same behavior as JupyterLab), which would help us identify each kernel.
Besides, when we click the kernel button on the upper right of the notebook, it would show a list of running kernels but without the kernel name. It would be better if the kernel names are also displayed, which would help us to identify and switch between different kernels.

Vs code will shutdown kernel if i accidentally close notebook tab. This will be helpful if we have session manger decide whenever to shutdown kernel. Session manager for vscode notebook will be great one.

Any movement on this feature? It would be great to be able to see and manage all active kernels.

Sorry not currently planned. I'm going to close this bug in favor of this one though:
https://github.com/microsoft/vscode-python/issues/14087
I think it more clearly describes the reasons it's needed.


[WIP] cloud-deployment: List +Diff installed packages  (SOC-8782)

Simple functionality which can be called when needed to collect(list) installed packages on the cloud nodes and|or to diff of changes(installed packages) before and after a update for example. Works on Ardana and Crowbar.
When called with "list" it collects installed packages, when with "diff", then it run list with "after" state and do the diff with previous state. States can be defined as shown below or default values will be used.
PR includes:

role
playbook
new stage in pipelines templates for ardana & crowbar

Usage:
list-or-diff-installed-packages.yml -e wanted_action=list -e state1=before_change
list-or-diff-installed-packages.yml -e wanted_action=diff -e state1=before_change state2=after_change

Passed Crowbar job "updates-after-deployment" and S:M:12131: https://ci.suse.de/blue/organizations/jenkins/openstack-crowbar/detail/openstack-crowbar/154/pipeline/141

CI run with failed step - package list are collected: crowbar-failed

CI job crowbar After updates: https://ci.suse.de/blue/organizations/jenkins/openstack-crowbar/detail/openstack-crowbar/352/pipeline/

Crowbar job with updates passed: https://ci.suse.de/blue/organizations/jenkins/openstack-crowbar/detail/openstack-crowbar/377/artifacts


Value Color is invisible

The Value Color is invisible.

Environment and versions

RedMatic Version: 5.5.2
Node-Red version: 1.0.4
dashboard version: 2.19.4
ui-artless-gauge version: 0.1.9

Redmatic is not standard solution so can't even comment. Try to rise issue against the redmatic


WKC installation is get failed

Describe the bug
WKC installation is get failed
We are trying to install WKC, WSL. We are facing issues, 2 pods are continuously in loop of kill, start and pending loop(please refer attached screen shot).  while investigation we found that there is a pod "zen-database-core-xx" restart on every few internal and it's having following error message in the log. not sure if there is a dependency wrt the issue. We are unable to install WKC currently using cp-deployer. May we get some help on this?
time="2023/08/25 11:34:39" level=info msg="Attempting to migrating data from couchdb if any..." func=zen-databases-core/pkg/generated/restapi.migrateMetadata file="/go/src/zen-databases-core/pkg/generated/restapi/configure_databases.go:206"
time="2023/08/25 11:34:39" level=error msg="secrets "couchdb-secrets" not found" func=zen-databases-core/pkg/impl/utils.GetSecretInNamespace file="/go/src/zen-databases-core/pkg/impl/utils/couchdb.go:77"
time="2023/08/25 11:34:39" level=error msg="secrets "couchdb-secrets" not found" func=zen-databases-core/pkg/impl/utils.GetSecretInNamespace file="/go/src/zen-databases-core/pkg/impl/utils/couchdb.go:77"
time="2023/08/25 11:34:39" level=info msg="No couchdb info found... nothing to migrate" func=zen-databases-core/pkg/generated/restapi.migrateMetadata file="/go/src/zen-databases-core/pkg/generated/restapi/configure_databases.go:243"
2023/08/25 11:34:39 Serving databases at https://[::]:3025
we have seen this behaviour in both 4.6.6 an 4.7.0. we are seeing the following event forbidden sysctl: "kernel.shmmni" not allowlisted  for the pod c-db2oltp-wkc-db2u-0

Fixed:

Only a single KubeletConfig will be created (multiple KubeletConfigs not supported by MachineConfig)
Dependent on OCP version, the KubeletConfig will include podPidsLimit
Existing obsolete KubeletConfig is deleted if found

Issue fixed.


(bug) npm i cypress-plugin-snapshots - npm error while installing

Cypress v13.11
npm i cypress-plugin-snapshots
npm error code ERESOLVE
npm error ERESOLVE unable to resolve dependency tree
npm error
npm error While resolving<EMAIL_ADDRESS>npm error Found<EMAIL_ADDRESS>npm error node_modules/cypress
npm error   dev<EMAIL_ADDRESS>from the root project
npm error
npm error Could not resolve dependency:
npm error peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm error node_modules/cypress-plugin-snapshots
npm error   cypress-plugin-snapshots@"*" from the root project
npm error
npm error Fix the upstream dependency conflict, or retry
npm error this command with --force or --legacy-peer-deps
npm error to accept an incorrect (and potentially broken) dependency resolution.
npm error
npm error

There was a change 8 months ago to simply allow any Cypress version higher than 4.5.0:
https://github.com/meinaart/cypress-plugin-snapshots/commit/a8bd88380db56905f5df6fe27cb4eddb7b809b9e
There has not been a release to npm, however.

I'm running into this too

Also seeing this issue

soo....this package is not maintained?

soo....this package is not maintained?

Yeah I gave up on this package. I am now using this one, has worked really well: https://github.com/haim-io/cypress-image-diff

There was a change 8 months ago to simply allow any Cypress version higher than 4.5.0: a8bd883
There has not been a release to npm, however.

I published the fix to npm in a fork<EMAIL_ADDRESS>I'm not gonna promise to maintain the fork, but if people wanna use this one instead of cypress-image-diff, the fork should make the npm install error go away.


MP 0 overwritten by other problems

The code for Machine Problem 0 seems to be overwritten with code from other machine problems arbitrarily. Right now, the code for MP0 on my account is "MP Reduction", shortly before it was tiled matrix multiplication. I tried reverting the code (/mp/0/revert) which reports success, but still the Device Query code is no where to be found.
edit: Not just the code, the questions are also from another lab. The description though, is still Device Query.

Same problem here.

same problem..for me is overwritten by convolution


Feature/world storage

Add interfaces and data structures related to the World Storage.

Regeneration after SolARFramework merge


About havengate

This is not actually havengate you just using transmission wow64 address and call it
I hope u correct me if I’m wrong
I believe havensgate you can call any function that is in 64bit layer or even call shellcode that is x64

Thanks for sharing your opinion.
My understanding is that Heaven's Gate means the procedure of transitioning from 32-bit to 64-bit (such as push 0x33; push 0x31323244; retf).
The PoC simply demonstrate that syscall can be called without going through SysWOW64's ntdll.dll using Heaven's Gate.
I'm working more Heaven's Gate related PoCs (such as shellcode loader and injector) and will release later.

Yes we are in same page about haven’s gate but i think you should rename the project to SyscallWOW64 bcuz haven’s gate is misleading
sorry for bad english not my native language


Support asynchronous format functions

hmm, can you provide a use case?

I'm extracting articles from a website including image links. I'm trying to extract the dominant color from each image, so I download the image, process it with node-canvas and then augment my json object with that data.
I don't know if it's a good enough case to do it inside the format function. Currently I've solved it by having an additional loop once the .run function completes.

yah, i think this may make more sense post-crawl or as the crawler streams.
going to close for now, but open to the idea if there's another use case.


Can't get past "Registering account..."

Well, the script worked great for me for a day or two, but now I am stuck at the "Registering account..." step.  I've checked through the code and can't for the life of me figure out why nothing goes past there.  I've let it sit for more than 20 minutes to no avail.
# python /config/letsencrypt/acme_tiny.py --account-key /config/letsencrypt/account.key --csr /config/letsencrypt/domain.csr --acme-dir /var/www/htdocs/.well-known/acme-challenge/ > /config/letsencrypt/signed.crt
Parsing account key...
Parsing CSR...
Registering account...

I had the same problem just now, but fortunately I was solved. 👌
I think it‘s because of IPv6, the IPv6 will link failure
when put the following code in /etc/hosts, the  problem will solve
<IP_ADDRESS> acme-v01.api.letsencrypt.org

you get the ip by ping   command

IPv6 is also possible, just add
2a02:26f0:11c:195::3d5 acme-v01.api.letsencrypt.org

to etc/hosts.
This is an IP of an Akamai machine, so it might change over time. Try pinging from another machine to get the current IP.

If you're experiencing connectivity problems to the Let's Encrypt API please start a community forum thread. We will work with Akamai to investigate the problem.
Hardcoding specific Akamai edge node IPs is not a recommended workaround and won't help any other users who may be experiencing similar routing problems.

@cpu this was indeed meant as a temporary workaround - I am running a IPv6 only machine with some external NAT64 service. For some reason acme-v01.api.letsencrypt.org was not properly resolved when I tried renewing a few hours ago. It worked before without editing /etc/hosts, so I filed this as a temporary issue.
(Permanently) hardcoding edge node IPs is not good idea. I'm with you on that.

This is a routing problem on the network, so there's not much acme-tiny can do about it.


test/file-register: fix heap overflow read on test_basic

When test_basic is called with fail == 1, the allocated array for
it is just 10, but the close_files is always called with 100.
Signed-off-by: Ammar Faizi<EMAIL_ADDRESS>

Can you rebase this on top of master? Seems to be some unrelated history in yours.

 Thanks!


Cherry-pick fixing Xcode 13 beta 3 build issues (#8405)

Propose disregarding the check failures to avoid back-porting clang-format 12.0.1 changes to the release branch


[Vine] Fixed some vines unable to be downloaded (HTTP 403) (closes #17883)

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[x] Searched the bugtracker for similar pull requests
[x] Read adding new extractor tutorial
[x] Read youtube-dl coding conventions and adjusted the code to meet them
[x] Covered the code with tests (note that PRs without tests will be REJECTED)
[x] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Bug fix
[ ] Improvement
[ ] New extractor
[ ] New feature

Description of your pull request and other information
Fixes HTTP 403 on some newer vines. Resolves issue #17883.

This fixed the issue for me 👍🏻

I still get a 403 when downloading this Vine: https://vine.co/v/5pmBeXz3LKd/