chore(scm): use standard repo file structure, issue labels, and templates

User story
As a contributor,
I need a consistent, organized repository structure
In order to complete tasks quickly.
Acceptance criteria

[x] 1. .git-label/all-git-label-packages.json  - use standardized issue labels with git-labelmaker.
[x] 2. .github/CODE_OF_CONDUCT.md - provide Contributor Covenant Code of Conduct.
[x] 3. .github/CONTRIBUTING.md - publish contribution guidelines.
[x] 4. .github/ISSUE_TEMPLATE.md - populate issues with a standard template.
[x] 5. .github/PULL_REQUEST_TEMPLATE.md - populate PRs with a standard template.

🐛  After installing semantic-release-cli, but sure to run npm install to ensure semantic-release -- not semantic-release-cli -- installs to node_modules/.bin/.


updated the requirements to fix documentation and coverage testing

Work on cleaning up the Travis CI issues.

Thank you! Yeah it's been on my todo list to update this for more recent Django for a while... Let me know when you are done pushing here and I will merge.

It may take awhile.  I haven't used docker, so to test it I either need to
keep pushing to git to test the build, or figure out docker.  I'll try to
get it before the new pip version which makes it necessary.
James
On Thu, Aug 20, 2020 at 10:29 AM Saul Shanabrook<EMAIL_ADDRESS>wrote:

Thank you! Yeah it's been on my todo list to update this for more recent
Django for a while... Let me know when you are done pushing here and I will
merge.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saulshanabrook/django-simpleimages/pull/29#issuecomment-677735652,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABCSSKWDYTSWOH52ZUDUINTSBU6NHANCNFSM4GCFTZLQ
.

--
Be happy because life is good


Building project gets stuck on 'colcon build --symlink-install' command

Describe the bug
When building the project, running the command 'colcon build --symlink-install' freezes after it has finished processing the first 3 modules, and it slows down when it tries to process 4 at a time. After that it outputs "c++: fatal error: Killed signal terminated program cc1plus compilation terminated.
gmake[2]: *** [Create3Hmi/CMakeFiles/Create3Hmi.dir/build.make:85: Create3Hmi/CMakeFiles/Create3Hmi.dir/Create3Hmi_autogen/mocs_compilation.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:153 Create3Hmi/CMakeFiles/Create3Hmi.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Failed   <<< irobot_create_ignition_plugins [3min 18s, exited with code 2]"
To Reproduce
Steps to reproduce the behavior:

export IGNITION_VERSION=fortress
colcon build --symlink-install
See error

Expected behavior
The project should finish building and succeed
Additional context
Gazebo 11
Ubuntu 22.04
ROS humble

Hi @redealexander9
The compiler is crashing, likely because the platform where you are compiling is running out of resources.
Do you really need to build this package?
It can be installed via
sudo apt-get update && sudo apt-get install -y \
        ros-humble-irobot-create-msgs \
        ros-humble-irobot-create-ignition-sim \
        ros-humble-irobot-create-gazebo-sim \
        ros-humble-irobot-create-nodes

If you really need to build it, you should limit the number of concurrent jobs
colcon build --executor sequential --symlink-install

Thank you, I didn't know I could install it and not have to build it. That worked


Add utility class to simultaneously visualize multiple robot poses and DIRCON callback.

Related to #103
Visualizes multiple states simultaneously, also wraps into a DIRCON callback function.
Parses a URDF multiple times to achieve this, with the only option of welding one frame to the world. It would probably be better to simple work off of an existing MBP, if that were possible.

Nice.  Related to https://github.com/RobotLocomotion/drake/issues/9728 .

I could also add an option to not show every knot point during TrajectoryOptimization (instead showing every T knot points). Would this be useful?

Will also add transparency as an option, once upstream Drake feature is implemented.
https://github.com/RobotLocomotion/drake/issues/13126

I could also add an option to not show every knot point during TrajectoryOptimization (instead showing every T knot points). Would this be useful?

Yes this feature will be helpful, especially when we cannot change the transparency of the poses between start and end.
I'm imagining after https://github.com/RobotLocomotion/drake/issues/13126 is fixed we can show non-transparent poses every T knot points, and show the rest of the poses with high transparency.

I'm imagining after RobotLocomotion/drake#13126 is fixed we can show non-transparent poses every T knot points, and show the rest of the poses with high transparency.

Actually, maybe showing the rest of the poses with high transparency is not a good idea, since the poses are probably going to overlap a lot and we won't gain any information from most of the poses.

Added the option to display fractional poses, which I think is worthwhile. Note that it shows poses
(1, 1 + T, 1 + 2T, 1 + 3T, ..., N)
always showing the last pose. Take a look and see what you think.

Another use case just occurred to me, I think it will be great to have an API which takes a list of the key frame indices? For example, if we have a multi-mode trajectory, and the key frames we want to see might not necessarily locate at (1, 1 + T, 1 + 2T, 1 + 3T, ..., N)

LGTM! Having the option to specify the number of poses when calling DrawPoses() might be slightly more flexible but I don't see a dire need for that in the foreseeable future.
By the way, I have code in a branch to visualize the trajectory after saving the states at the knot points to a LCM trajectory, so you don't need to rerun the optimization problem just to visualize something. Could be useful later on.


fix: update packages accordingly

Some error in semantic-release configuration is not updating the package versions accordinly and thats really important.

:tada: This PR is included in version @pluto-encrypted/inmemory-v1.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @pluto-encrypted/leveldb-v1.0.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @pluto-encrypted/database-v1.0.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add WartRemover checks

Would it be a good idea to add checks using wartremover?
https://github.com/wartremover/wartremover
I have seen some good checks like

Remove unsafe calls like head, last, get, etc.
Remove vars and mutable state
Detect Product type inferences
Remove the use of Any

and many others.
We could define a full list of desired checks from here: http://www.wartremover.org/doc/warts.html

I think this is generally a good idea, although I don't agree with a few built-in warts or with the suggested ways to remove some warts.

I found wartremover super useful. @EzequielPostan nice list, Recursion wart deserves a mention too. It forces you to think about stack safety in every recursion.

Thanks for the comment @greenhat , I will go back to work on this issue soon.
@ceilican
We can select warts we would like to adopt and ignore the rest. I also don't like some of the suggested ways to remove some warts, but we can apply our own style.
The good thing is that breaking the wart rules won't let you compile, so it enforces good practices.
I will enable some warts in a new branch and check how many issues it detects.

@ceilican
I enabled TraversableOps  in a local branch and the plugin found the following error cases:
sbt:scorex-core> test:compile
···
[error] .../src/main/scala/scorex/core/api/http/CompositeHttpService.scala:21:44: [wartremover:TraversableOps] reduce is disabled - use reduceOption or fold instead
[error]   val compositeRoute = routes.map(_.route).reduce(_ ~ _) ~ corsHandler(swaggerService.route) ~
[error]                                            ^
[error] .../src/main/scala/scorex/core/consensus/BlockChain.scala:39:36: [wartremover:TraversableOps] head is disabled - use headOption instead
[error]   def lastBlock: B = lastBlocks(1).head
[error]                                    ^
[error] .../src/main/scala/scorex/core/consensus/BlockChain.scala:50:29: [wartremover:TraversableOps] head is disabled - use headOption instead
[error]     val modId = openSurface.head._1
[error]                             ^
[error] .../src/main/scala/scorex/core/consensus/BlockChain.scala:51:37: [wartremover:TraversableOps] head is disabled - use headOption instead
[error]     val s = lookForward(openSurface.head._2, size)
[error]                                     ^
[error] .../src/main/scala/scorex/core/network/NodeViewSynchronizer.scala:305:33: [wartremover:TraversableOps] head is disabled - use headOption instead
[error]         val modType = modifiers.head.modifierTypeId
[error]                                 ^
[error] .../src/main/scala/scorex/core/network/message/BasicMessagesRepo.scala:114:7: [wartremover:TraversableOps] reduce is disabled - use reduceOption or fold instead
[error]     }.reduce(_ ++ _)
[error]       ^
[error] .../src/main/scala/scorex/core/utils/utils.scala:59:38: [wartremover:TraversableOps] head is disabled - use headOption instead
[error]   else concatFixLengthBytes(seq, seq.head.length)
[error]                                      ^
[error] 7 errors found

which reveals that some unsafe calls are made and probably should be reviewed.
Would you like me to enable some Warts and correct them as I can? It may be good to create a separate issues for each Wart we decide to enable.
Should I proceed in this path? If yes, which Warts would you suggest to enforce?

Let me correct myself, there are a lot more of unsafe calls. The ones above are just from scorex-core

@ceilican @greenhat
As pointed above, I created a PR to discuss the application of TraversableOps

Thanks, @EzequielPostan ! I left some comments there. My comments clarify what I meant when I wrote above that:

I think this is generally a good idea, although I don't agree with a few built-in warts or with the suggested ways to remove some warts.

In my opinion, we should only avoid using head, tail and other methods that may throw an exception if we are using them in a buggy way (e.g. when the case that may trigger the exception can actually occur and is not being handled in some way). It is better to have exceptions than to have possibly wrong behaviours for those unexpected cases.

I can try to apply another wart.

I think we should have a separate PR for each wart.

I am not sure how should I proceed with that PR, any suggestion is welcome.

I propose that you modify the PR. For each wart occurrence found by the wart remover, do one of the following:

if the wart is a bug (e.g. the code is not handling the case of an empty traversable as it should) and it is clear how to fix the bug, fix the bug.

if the wart is not a bug (e.g. the code doesn't handle the case of an empty traversable, but this case never happens, or the exceptional case is already handled in ways that WartRemover doesn't understand (such as surrounding the code with Try, ensuring, catch...)), leave the code as it is and switch off WartRemover on that piece of code.   Optionally, do one of the following:

add a comment explaining why the wart is not a bug.
refactor the code removing the wart, but preserving the behaviour, if the refactored code without the wart is clearly more concise and easier to understand (for programmers in general) than with the wart.

if you are unsure whether the wart is a bug, switch off WartRemover around that wart and add a  // fixme comment explaining why it might be a bug.

This is already almost what you did.
What do you think of this proposal?

@kushti @greenhat , what do you think about the proposal above?

I agree with the suggestions of creating different PRs for each Wart and also on how to proceed.
As I am still not that familiar with the code base, I guess I will "fix" few of the warts, but I can leave a trace of the problematic cases as you mention.
Other warts may be "easier" to apply.

@ceilican I agree. And it might be a good idea to create ticket(s) for "warts to enable" list.

I created a PR to enable the wart Recursion as it was really simple to apply #194 .
I will go back to the TraversableOps during the weekend.
Is it ok if I try to apply other Warts? Creating a list of Warts we agree on applying would be good as suggested above.

Var and Null tend to be good Warts.
Null is easy to apply (there are only two uses of null in the whole project)
I haven't checked Var (eliminates the use of var) but it may be hard to apply.
I once found a tricky bug thanks to Product.
Any is also a good wart, but haven't checked its application either.
OptionPartial: replace get for fold in type Option
I think PublicInference could be good as I see a lack of type declarations in some places (but not sure if this happens with public members).
Then other possible warts could be:

DefaultArguments: To eliminate default arguments
ToString: Scala creates a toString method automatically for all classes. Since toString is based on the class name, any rename can potentially introduce bugs. This is especially pernicious for case objects. toString should be explicitly overridden wherever used.
FinalCaseClass: Scala’s case classes provide a useful implementation of logicless data types. Extending a case class can break this functionality in surprising ways. This can be avoided by always making them final or sealed.
FinalVal: Value of a final val is inlined and can cause inconsistency during incremental compilation (see https://github.com/sbt/sbt/issues/1543)
ImplicitConversion: Implicit conversions weaken type safety and always can be replaced by explicit conversions.
IsInstanceOf: isInstanceOf violates parametricity. Refactor so that the type is established statically.
JavaConversions: The standard library provides implicits conversions to and from Java types in scala.collection.JavaConversions. This can make code difficult to understand and read about. The explicit conversions provided by scala.collection.JavaConverters should be used instead.
JavaSerializable: java.io.Serializable is a common subtype to many structures, especially those imported from Java. For example, String is a subtype of java.io.Serializable but not scala.Serializable. The Scala compiler loves to infer java.io.Serializable as a common supertype, but that is almost always incorrect. Explicit type arguments should be used instead.
Serializable: Serializable is a type common to many structures. The Scala compiler loves to infer Serializable as a generic type, but that is almost always incorrect. Explicit type arguments should be used instead.
LeakingSealed: Descendants of a sealed type must be final or sealed. Otherwise this type can be extended in another file through its descendant.
MutableDataStructures: The standard library provides mutable collections. Mutation breaks equational reasoning.

As you see, there are many warts to examine. I think that most of this can be applied in a single PR because they may not involve many changes (for example: Null, Product, FinalVal, JavaConversions, JavaSerializable and Serializable could probably be applied in a single PR). Other warts would need special care in separate PRs (Any, Var, MutableDataStructures).
MutableDataStructures could be related to #78
Enabling a Wart also impose a sort of style standard, so I wouldn't like to add warts without a certain level of consensus. Please let me know which ones you prefer me to work on (if any) and which to ignore (if any).
Thank you all for your time to read/review this.

Here are my comments:

Null is easy to apply (there are only two uses of null in the whole project)

Yes, I totally agree with removing null.

I haven't checked Var (eliminates the use of var) but it may be hard to apply.

Partially agree. We definitely should avoid global var. Locally, if var can be removed without compromising code readability, I'm in favour of it.  Occasionally, though, a local use of var allows things to be achieved more easily and more clearly. In such cases, I think we should keep using var instead of masochistically doing some immutable contortions... :-)

I once found a tricky bug thanks to Product.

I think it is a good one.

Any is also a good wart, but haven't checked its application either.

I think it is a good one too.

OptionPartial: replace get for fold in type Option

I partially agree. As with head and tail (discussed above), sometimes it is ok to use get.

I think PublicInference could be good as I see a lack of type declarations in some places (but not sure if this happens with public members).

IntelliJ already warns to declare types of public members. Therefore, at least for parts of the code that I have touched, this won't make a difference. But I think it is a good one.

DefaultArguments: To eliminate default arguments

I completely disagree with this one. Default arguments are a very nice feature.

ToString: Scala creates a toString method automatically for all classes. Since toString is based on the class name, any rename can potentially introduce bugs. This is especially pernicious for case objects. toString should be explicitly overridden wherever used.

I'm against this one. I never had a problem with "toString", and being forced to override "toString" to use it would make the code unnecessarily more verbose.

FinalCaseClass: Scala’s case classes provide a useful implementation of logicless data types. Extending a case class can break this functionality in surprising ways. This can be avoided by always making them final or sealed.

Isn't "case class inheritance" already prohibited in recent versions of Scala anyway?

FinalVal: Value of a final val is inlined and can cause inconsistency during incremental compilation (see https://github.com/sbt/sbt/issues/1543)

Not sure. I think it wouldn't make a difference anyway.

ImplicitConversion: Implicit conversions weaken type safety and always can be replaced by explicit conversions.

I'm against this one.

IsInstanceOf: isInstanceOf violates parametricity. Refactor so that the type is established statically.

This is a good one.

JavaConversions: The standard library provides implicits conversions to and from Java types in scala.collection.JavaConversions. This can make code difficult to understand and read about. The explicit conversions provided by scala.collection.JavaConverters should be used instead.

This seems ok to me.

JavaSerializable: java.io.Serializable is a common subtype to many structures, especially those imported from Java. For example, String is a subtype of java.io.Serializable but not scala.Serializable. The Scala compiler loves to infer java.io.Serializable as a common supertype, but that is almost always incorrect. Explicit type arguments should be used instead.

Seems ok too.

Serializable: Serializable is a type common to many structures. The Scala compiler loves to infer Serializable as a generic type, but that is almost always incorrect. Explicit type arguments should be used instead.

Seems ok.

LeakingSealed: Descendants of a sealed type must be final or sealed. Otherwise this type can be extended in another file through its descendant.

Good.

MutableDataStructures: The standard library provides mutable collections. Mutation breaks equational reasoning.

Theis one would require very extensive refactoring, because Scorex relies a lot on mutable collections.
Therefore, I think we shouldn't even discuss whether this one is good or not. We should leave it aside for now.

I think that most of them can be applied in a single PR because they may not involve many changes (for example: Null, Product, FinalVal, JavaConversions, JavaSerializable and Serializable could probably be applied in a single PR).

I agree.

MutableDataStructures could be related to #78

Yes, it is. And #78 is a hard issue. #78 alone would already require several PRs, one per actor.

Enabling a Wart also imposes a sort of style standard. Hence, I wouldn't like to add warts without a certain level of consensus. Please let me know which ones you prefer me to work on (if any) and which to ignore (if any).

I think the ones for which I said either "good" or "agree" are quite uncontroversial. I think everyone would agree with them. I would start with those.

Good, thanks @ceilican for the comments.
I will experiment with the agreed ones and have a deeper look to the others.
I can treat the replacement of get for fold in a separate PR in a similar way we are doing with List's unsafe methods.
For the ones you stated a partial or total disagreement or are tricky to apply:

Var: I would leave this one for later as it would need a separate PR in any case.
DefaultArguments: I kinda agree on this one
ToString: I never applied this one before, I guess it could be useful in some contexts. I will leave it aside.
FinalCaseClass: I also think this one is currently forbidden by the compiler, but should check. I will leave it aside too.
FinalVal: I also think there are probably no cases for this one in the code base. I will enable it.
ImplicitConversion: I kinda don't like this feature from the language, but won't deny that it is powerful when used with care. I won't apply this Wart.
MutableDataStructures: As mentioned, this will be hard to apply if we decide to do so. I will leave it aside for now as suggested.

I will then apply (in more than one PR):

TraversableOps (currently in progress)
Null, Product, PublicInference, FinalVal, IsInstanceOf, JavaConversions, JavaSerializable, Serializable and `LeakingSealed`` (probable in a single PR)
OptionPartial (in a separate PR)
Any (in a separate PR)
Var (to be defined, but would be in a separate PR)

These are the pending Warts from the above list:

LeakingSealed (I should review the impact of this one)
OptionPartial (in a separate PR)
Any (in a separate PR)
Var (to be defined, but would be in a separate PR)
MutableDataStructures (won't be applied now, to be evaluated in the future)

Just to clean up this issue.

OptionPartial was enabled and an issue was opened to improve the general use of Option and Try constructors (#243).
LeakingSealed warns some cases of traits that extend other traits which are then extended in other files. Example:
sealed trait NodeViewModifier is defined in src/main/scala/scorex/core/NodeViewModifier.scala
In the same file we can see the definition
trait EphemerealNodeViewModifier extends NodeViewModifier
but then we have class Transaction extending EphemerealNodeViewModifier
abstract class Transaction extends EphemerealNodeViewModifier
in the file src/main/scala/scorex/core/transaction/Transaction.scala
The keyword sealed is usually used to let the compiler do an exhaustive search on pattern matchings, I guess we don't pattern match over instances of NodeViewModifier. Would it be okay to remove sealed from this case and similar ones (if any)?
Any, Val and MutableDataStructures will demand a big effort and I may suggest to open separate issues for each of them if we intend to apply them.

Would it be okay to remove sealed from this case and similar ones (if any)?

I think so.

Any, Val and MutableDataStructures will demand a big effort and I may suggest to open separate issues for each of them if we intend to apply them.

I suggest opening issues for Any and Val, but not for MutableDataStrucutures... Scorex's actors rely so much on mutable data structures that it would be impossible (and probable even not desirable) to get rid of them...


[WIP] Add support for warm migration

Resolves #308
WIP:

[x] Add Type step to plan wizard with cold/warm selection
[ ] Wire up cold/warm selection to spec property when creating plan
[x] Add Type column to plans table
[x] Move source/target provider and VMs columns into expandable rows to make more table space
[ ] Display plan-level status information for warm migrations (mockup)
[ ] Implement Cutover button
[ ] Add warm migration variant of plan details table before cutover (different columns -- mockup)
[ ] Add expanded table detailing each incremental copy instead of pipeline steps (mockup)
[ ] ???

New Type step in the plan wizard:

New layout for plans table:

Note to self: should probably address https://github.com/konveyor/forklift-ui/issues/423 at the same time here since I'll be reworking the logic for migration status anyway

Moved one of the tasks above to a separate issue: https://github.com/konveyor/forklift-ui/issues/454

Thanks!


Randomly fired Homekit actions.

I use homekit2mqtt (which uses HAP-NodeJS) to control my self built electric curtains. It works perfect, but sometimes my curtains randomly open. I checked the debug info, and it doesn't say anything strange: it really looks like it really receives an open command from HAP-NodeJS:
hap set Curtain 1 TargetPosition 100

It seems like HAP-NodeJS fires false Homekit actions. Any idea where to look for any clues or solutions?
@hobbyquaker, creator of homekit2mqtt, suggested to check of there were any others using HAP-NodeJS users with the same issue: https://github.com/hobbyquaker/homekit2mqtt/issues/12

So, I probably found the issue. Which is 100% my fault: I was running the application on two different machines at the same time (by mistake). Homekit probably switches between the two different instances from time to time. I shut down one of the two instances, and will check if this solved the issue ... If not, I'll reopen the issue. Thanks!

Yeah, if this still occurs, I'd encourage you try to run the program with DEBUG=* node xxx to get all debug outputs so we can figure out if the command is originated from iOS device or from hap-nodejs internals.

Will do! Thanks!


Fix the auto iptables detection if ip_tables.ko preloaded on RHEL/CentOS 8

Description
The auto iptables detection logic is updated by the Kubernetes KEP 3178 ,  the 'iptables-wrapper'  can decide which iptables backend to use based on which one kubelet used.
So that the issue can be fixed by the new 'iptables-wrapper' logic.  The PR is to update it with the new logic:
from:
https://github.com/kubernetes-sigs/iptables-wrappers/blob/v1/iptables-wrapper-installer.sh#L130
to:
https://github.com/kubernetes-sigs/iptables-wrappers/blob/master/iptables-wrapper-installer.sh#L107
Some additional information:

a blog article about the issue: https://mihail-milev.medium.com/no-pod-to-pod-communication-on-centos-8-kubernetes-with-calico-56d694d2a6f4
The docs of RHEL in kubespray : https://github.com/kubernetes-sigs/kubespray/blob/master/docs/rhel.md is NFT mode

Related issues/PRs
fixes https://github.com/projectcalico/calico/issues/2322
fixes https://github.com/projectcalico/calico/issues/3709
Todos

[x] Tests
[ ] Documentation
[ ] Release note

Release Note

Fix the auto iptables detection if ip_tables.ko preloaded on RHEL/CentOS 8

Reminder for the reviewer

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.

HI @caseydavenport , @lwr2 , @fasaxc , would you please review it :-)
Thank you very much.

HI @caseydavenport, would you please review the PR , thanks.

HI @tomastigera , would you please review the PR , thank you :-)

Hi @caseydavenport @song-jiang , can you please take a look ? Thanks,  I think it's make a sense. 😄

@yankay sure, I'll review it in the next few days.

@yankay sure, I'll review it in the next few days.

Thank you very much

Thank you very much @fasaxc ,
The PR has changed the default from the NTF to the legacy mode.
Would you please review it again :-)

/sem-approve

LGTM. Noticed there was a change to a couple of existing tests due to the removal of the
if legacyLines >= 10

thanks that's OK now that nft is widely adopted

Thank you very much for the review :-)

Hi @fasaxc , ci/semaphoreci/pr is failling, Does this affect the PR merge?

/sem-approve

Thanks@mazdakn @fasaxc @mgleung  @cyclinder  for the PR review :-)


Cannot use original header on Response Modifier Plugin (I can't modify the backend)

Environment info:

KrakenD version: Version: 2.1.3
System info: docker
Hardware specs: Lenovo ThinkPad E15 Gen 2, 32GB ram
Backend technology: Node
Additional environment information:

Describe what are you trying to do:
Hello
I want to use the request.Header on Response Modifier plugin  without changing anything on the backend (I am not allowed to do it)
For example, the API Gateway receives data from the original request on Headers and the Request Modifier Plugin XPTO or the auth/validator treats the information and attaches some data to a new Header ("X-Username" for example)
I want to get the X-Username header on the Response Modifier Plugin alongside the response body from the backend. The response needs to be encrypted using data present on the original request.headers.
Is it possible?
Many thanks!
Your configuration file:
{
  {
  "$schema": "https://www.krakend.io/schema/v3.json",
  "version": 3,
  "host": "http://localhost:8081",
  "port": "8081",

  "name": "KrakenD ",
  "extra_config": {
    "router": {
      "return_error_msg": true
    },
    "security/cors": {
      "allow_methods": ["GET", "POST", "PUT", "PATCH", "DELETE"]
    }
  },
  "output_encoding": "json",
  "timeout": "2000ms",
  "cache_ttl": "300s",
  "host_v2_1": "v2.1",
  "debug_endpoint": true,
  "plugin": {
    "pattern": ".so",
    "folder": "/etc/krakend/plugins/.bin/"
  },

  "endpoints": [
    {
      "@comment": "Auth: generate token",
      "endpoint": "v2.1/auth/token",
      "method": "POST",
      "backend": [
        {
          "url_pattern": "/oidc/token",
          "host": ["URL_HERE"],
          "method": "POST",
          "encoding": "json",
          "allow": ["access_token", "expires_in", "token_type", "scope"]
        }
      ],
      "output_encoding": "json"
    },
    {
      "endpoint": "v2.1/debug",
      "method": "POST",
      "output_encoding": "json",

      "input_headers": ["Authorization", "Content-Type", "X-Username"],

      "extra_config": {
        "auth/validator": {
          "alg": "RS256",
          "jwk_url": "URL_HERE",
          "cache": true,
          "cache_duration": 600,
          "roles_key": "roles",
          "roles": ["ROLE_SERVICE"],
          "operation_debug": true,
          "propagate_claims": [["username", "X-Username"]]
        }
      },

      "backend": [
        {
          "url_pattern": "/__debug/default",
          "method": "POST",
          "encoding": "json",
          "extra_config": {
            "modifier/martian": {
              "header.Copy": {
                "scope": ["request", "response"],
                "from": "X-Username",
                "to": "X-TEST"
              }
            },
            "plugin/req-resp-modifier": {
              "name": [
                "decryption-payload-plugin",
                "get-entity-plugin",
                "encryption-response-plugin"
              ],
              "decryption-payload-plugin": {
                "HOST_ENTITY_SERVICE": "URL_HERE",
                "HOST_ENCRYPTION_SERVICE": "URL_HERE"
              },
              "encryption-response-plugin": {
                "HOST_ENTITY_SERVICE": "URL_HERE",
                "HOST_ENCRYPTION_SERVICE": "URL_HERE"
              }
            }
          }
        }
      ]
    }
  ]
}

}

Commands used:
How did you start the software?
version: "3"
services:
  krakend:
    container_name: krakend
    image: devopsfaith/krakend:watch
    volumes:
      - .:/etc/krakend
    environment:
      PORT: "8081"
      API_VERSION_2_1: "v2.1"
      HOST_ENTITY_SERVICE: "URL_HERE"
      HOST_ENCRYPTION_SERVICE: "URL_HERE"
      FC_ENABLE: "1"
      FC_SETTINGS: "/etc/krakend/settings"
      FC_TEMPLATES: "/etc/krakend/templates"
      FC_PARTIALS: "/etc/krakend/partials"
      FC_OUT: "/tmp/out.json"
    command:
      ["run", "-d", "-c", "/etc/krakend/krakend.tmpl"]
    ports:
      - 8080:8081

Logs:

Additional comments:

I used HTTP Client Plugin, this way I have both the request and the response of the backend.
https://www.krakend.io/docs/extending/http-client-plugins/


WIP Explicitly add --detach to service CLI calls

##NOTE: this is a PR to just test tests
The behavior of service (create/update/scale) was changed in a recent PR
to docker/cli. This commit serves to remedy test failures experienced
when attempting to use service calls.
Should not affect current behavior.
Signed-off-by: Eli Uriegas<EMAIL_ADDRESS>(cherry picked from commit e5b3ebbc649e4b1a10d4cdca342a153c301ec225)
Signed-off-by: Eli Uriegas<EMAIL_ADDRESS>

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "test_detach" git@github.com:seemethere/docker-ce.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842354185440
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

Closed in favor of #260


Calico pod disk io read bytes is very high

The pod calico-node-xxxx disk io read bytes is high 120MB/S  of prometheus metrics a few minutes, Metrics name is container_fs_reads_bytes_total, Then pod readinessProbe healthcheck is failure occur unavaliable trigger alert
Expected Behavior

disk io read bytes is ~5KB/S
Current Behavior

disk io read bytes is ~120MB/S
Possible Solution

auto recovery
Steps to Reproduce (for bugs)

Context

The pod cannot running and ready status a few mintues
Your Environment

Calico version v3.13.4
Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes v1.18.20
Operating System and version:  CentOS Linux 7 5.4.210-1.el7.elrepo.x86_64
Link to your project (optional):

Calico container env list:

env:

name: CALICO_DISABLE_FILE_LOGGING
value: "true"
name: CALICO_IPV4POOL_CIDR
value: xx.xx.xx.0/18
name: CALICO_IPV4POOL_IPIP
value: "off"
name: CLUSTER_TYPE
value: k8s,bgp
name: DATASTORE_TYPE
value: kubernetes
name: FELIX_DEFAULTENDPOINTTOHOSTACTION
value: ACCEPT
name: FELIX_HEALTHENABLED
value: "true"
name: FELIX_IPINIPENABLED
value: "false"
name: FELIX_IPV6SUPPORT
value: "false"
name: FELIX_LOGSEVERITYSCREEN
value: info
name: IP
value: autodetect
name: WAIT_FOR_DATASTORE
value: "true"

Hi @prometheus-tao , I noticed that the version that you're using is pretty old and there have been many performance fixes that have gone in since 3.13. Could I ask you to try a newer version of Calico and see if your issue persists?

@mgleung thanks, I try to deploy upgrade newer version on test cluster.

@sridhartigera Not yet, I have something do work recently, I try to it next week. thanks!

@prometheus-tao any updates on this?


Trying to findout an way to run model.predict like Python

I'm creating a bot framework that loads tflite NLP model on xamarin android and predict. Any help?

Thanks for the feedback but we'll need more info.


Testing plan: Ran build-release-tarball test.

lgtm.


dex-worker registration email regex flag

dex-worker --enable-registration=true --registration-email-regex=*.coreos.com

The use case is restricting open user registration to emails in a specific domain.

Some prior discussion here: https://github.com/coreos/dex/pull/202


Can't import modules installed from pip

Hey,
when I try to run script on a python file, if I have any imports that aren't part of Python's standard lib, then I get an ImportError stating that the module I'm trying to import doesn't exist. Is there any way to get script to read my pythonpath? I'm using python installed via brew install on OS X, not the Apple standard.

Please close this -- I figured out I had /usr/bin/python in my file instead of /usr/local/bin/python.

Had the same problem, adding
#!/usr/local/bin/python

to the top of the file fixed it.
Thank you, @jalavosus.

Any solution if I don't want to change my script? Any way to make this default behavior?


Change file names

I see that you have used file name "index1.html" for user profile page with "index1.css" as its CSS file, and "index2.html" for about us page. So, I think proper names should be used for the files so that they can be properly accessed and linked with each other.
This will improve readability and maintainability of the code.
Assign it to me under hacktoberfest.

Yah @SandeshGC , that is a work that needs to be done, but it's too less for a change and hope you know that minor contributions like this and fixing grammatical errors do not count as contributions this year. If there is anything more that you want to add, then let me know and I'll assign the issue to you.

Okay, do it. No problem.


fix alpn hash

In rare cases, the ALPN value may be 'http/1.1'. The current implementation returns the entire ALPN value, which results in the following issue:
<IP_ADDRESS>:50474 [14/Sep/2024:03:35:58.419] https~ https/ -1/-1/-1/-1/0 -1 0 - - PR-- 25/24/0/0/0 0/0 "GET /.well-known/nodeinfo HTTP/1.1" 0/0000000000000000/0/0/0 example.com/TLSv1.3/TLS_CHACHA20_POLY1305_SHA256 "t13d3613http/1.1_018971650b2c_5f333c0549e8" "t_13_d_36_13_http/1.1_002f,0033,0035,0039,003c,003d,0067,006b,009c,009d,009e,009f,00ff,1301,1302,1303,1304,c009,c00a,c013,c014,c023,c027,c02b,c02c,c02f,c030,c09c,c09d,c09e,c09f,c0ac,c0ad,cca8,cca9,ccaa_000a,000b,000d,0015,0016,0017,002b,002d,0031,0033,3374_0403,05"
This PR addresses the problem by extracting only the first letter and last letter from the ALPN value, as specified in the JA4 spec.

After conducting research, I found that the current python implementations do not align with the latest JA4 specification. Consequently, I made some adjustments to the code.

Sorry for the delay. Must've overlooked the notification(s)..
Thank you - I have not found a list of values that are returned by HAProxy. Thought this might be an edge-case.
Just a thought - the fc/lc handling looks a little over-engineered to me.
If it's just http/1.1 - why not simply match the whole string instead as we already do with the 00 case.

The challenge with ALPN is that it includes numerous possible combinations. For a comprehensive list, refer to the IANA ALPN Protocol IDs. You can also check out the recent discussion regarding the ALPN value of JA4 here.

@mat1010 May you please take a look? Your opinion would be appreciated

I also had a look on the updated JA4 spec regarding the ALPN value cases. The updated spec has defined quite a ruleset how to handle the different potential values.
This PR handles case 1 and 2 afaict. We still have to handle Case 3, right  @nakoo ?
Case 1

The first and last alphanumeric characters of the ALPN (Application-Layer Protocol Negotiation) first value.
List of possible ALPN Values (scroll down): https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml

Case 2

If the first or last byte of the first ALPN is non-alphanumeric (meaning not 0x30-0x39, 0x41-0x5A, or 0x61-0x7A), then we print the first and last characters of the hex representation of the first ALPN instead. For example:

0xAB would be printed as "ab"
0xAB 0xCD would be printed as "ad"
0x30 0xAB would be printed as "3b"
0x30 0x31 0xAB 0xCD would be printed as "3d"
0x30 0xAB 0xCD 0x31 would be printed as "01"

Case 3

If there is no ALPN extension, no ALPN values, or the first ALPN value is empty, then we print "00" as the value in the fingerprint. If the first ALPN value is only a single character, then that character is treated as both the first and last character.

Thanks for bringing this up. I also had a look on the updated JA4 spec regarding the ALPN value cases. The updated spec has defined quite a ruleset how to handle the different potential values.
This PR handles case 1 and 2 afaict. We still have to handle Case 3, right @nakoo ?

I believe Case 3 is already being addressed in the current codebase. If there is anything missing or needs improvement, please let me know.


Adapters

Extract out existing into own repos?

Create adapters for most popular webhooks:

Provider
Description

⬜
IFTTT
Automation

⬜
Zapier
Automation

⬜
GitHub
CI/CD

⬜
GitHub Actions
CI/CD

⬜
GitLab
CI/CD

⬜
Jenkins
CI/CD

⬜
CircleCI
CI/CD

⬜
Travis CI
CI/CD

⬜
Bitbucket
CI/CD

⬜
DataDog
Monitoring

⬜
PagerDuty
Monitoring

⬜
Discord
Communication

⬜
Microsoft Teams
Communication

⬜
Slack
Communication

⬜
Intercom
Communication

⬜
SendGrid
Email

✅
Mailgun
Email

✅
Mandrill
Email

✅
MailerSend
Email

⬜
WordPress
Content Management

⬜
Shopify
Ecommerce

⬜
Stripe
Ecommerce

✅
Hasura Actions
GraphQL

✅
Hasura Events
GraphQL

⬜
Jira
Project Management

⬜
Asana
Project Management

⬜
Netlify
Deployment

⬜
Heroku
Deployment

⬜
Okta
Identity Management

✅
s3
Storage

⬜
AWS SNS
Notification

✅
Parsio
Parsing

Have a request for an adapter? Let us know in the comments.


Update FMS default version to 2021.04.01

The ufs-weather-model is switching to FMS release 2021.04 (2021.04.01 in spack versioning). We need to update the version number in the common packages.yaml file.

Instead of 2021.04, ufs-weather-model will be going straight to 2022.01 in a few weeks: https://github.com/ufs-community/ufs-weather-model/issues/1105


consider adding a jax_numpy_dtype_promotion option

A user observed that they had an unintentional and silent bfloat16 to float32 promotion which was hard to track down. This kind of promotion is intended for jax.numpy operations (including infix operators like *). But it can be undesirable.
In [11]: x = jnp.ones([3], dtype=jnp.float32)
In [12]: x
Out[12]: DeviceArray([1., 1., 1.], dtype=float32)
In [13]: y = jnp.ones([3], dtype=jnp.bfloat16)
In [14]: y
Out[14]: DeviceArray([1, 1, 1], dtype=bfloat16)
In [15]: x * y
Out[15]: DeviceArray([1., 1., 1.], dtype=float32)

This is like the situation with unexpected float64 promotions, for which we have the option enable_x64 (and a default which deviates from NumPy). This is also analogous to the situation with the jax_numpy_rank_promotion option, which is similarly meant to disable a kind of automatic jax.numpy-level promotion in favor of stricter semantics.
One option is to use the lax functions directly, since they are more strict in that they don't allow silent dtype promotion (or rank promotion, so their behavior is decidedly less "lax" than that of analogous jax.numpy operations):
In [3]: x = jnp.ones([3], dtype=jnp.float32)
In [4]: x
Out[4]: DeviceArray([1., 1., 1.], dtype=float32)
In [5]: y = jnp.ones([3], dtype=jnp.bfloat16)
In [6]: y
Out[6]: DeviceArray([1, 1, 1], dtype=bfloat16)
In [7]: lax.mul(x, y)
...
TypeError: mul requires arguments to have the same dtypes, got float32, bfloat16.

But infix operators are convenient. Moreover, unintended promotion might still happen in library code.
Another option is to add a jax_numpy_dtype_promotion option. It would have to be checked in this logic (and perhaps other places), following an analogous pattern to how the jax_numpy_rank_promotion option is checked in the _promote_shapes function. We'd also have to create the option in config.py, analogous to how we create the jax_numpy_rank_promotion option.
It's not clear yet if this is a good idea, but unless someone can spot a knock-down argument for why this won't work, it might be worth trying.

Another way to this might be to make alternative versions of dtypes that don't implement type promotion, e.g., jnp.strict_bfloat16 which works exactly like jnp.bfloat16 but doesn't participate in type promotion. Or you could even put this in the metadata custom NumPy dtypes, e.g., np.dtype(np.float32, metadata={'strict': True}) (NumPy itself doesn't propagate metadata in operations, but JAX could).
The virtue of this sort of approach is that it would be based on local control, rather than being a global flag, which would make it easier to use in combination with external libraries that might use promotion internally.

Done in https://github.com/google/jax/pull/10824


surface.attach(None, 0, 0) doesn't allow for a re-map of the layer surface.

When configuring a layer_surface and committing a surface with no buffer attached, I expect a configuration event to be emitted. Which is true in this case.
    config(&layer_surface, &surface);
    event_queue.blocking_dispatch(&mut Delegate).unwrap();
    surface.commit();

But importantly I can call this piece of code multiple times and receive multiple configuration events. However the issue arises when I attach a buffer In this way.
   config(&layer_surface, &surface);
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

   surface.attach(Some(&buffer), 0, 0);
   surface.commit();
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

This code will result in a configuration event being emitted however I expect to be able to reconfigure the surface later on by attaching a None to the surface and then committing it, in this way.
   surface.attach(None, 0, 0);
   surface.commit();

   config(&layer_surface, &surface);
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

However what happens instead is configuration event isnt being emitted so the program gets stuck in the blocking_dispatch
Replication code.
use std::{
   fs::File,
   io::{Seek, SeekFrom, Write},
   os::fd::AsFd,
   thread,
   time::Duration,
};

use wayland_client::{
   delegate_noop,
   globals::{registry_queue_init, GlobalListContents},
   protocol::{
       wl_buffer::WlBuffer,
       wl_compositor::WlCompositor,
       wl_output::WlOutput,
       wl_registry::WlRegistry,
       wl_shm::{Format, WlShm},
       wl_shm_pool::WlShmPool,
       wl_surface::WlSurface,
   },
   Connection, Dispatch, QueueHandle,
};
use wayland_protocols_wlr::layer_shell::v1::client::{
   zwlr_layer_shell_v1::{Layer, ZwlrLayerShellV1},
   zwlr_layer_surface_v1::{self, Anchor, KeyboardInteractivity, ZwlrLayerSurfaceV1},
};

struct Delegate;
impl Dispatch<WlRegistry, GlobalListContents> for Delegate {
   fn event(
       _: &mut Self,
       _: &WlRegistry,
       _: <WlRegistry as wayland_client::Proxy>::Event,
       _: &GlobalListContents,
       _: &Connection,
       _: &QueueHandle<Self>,
   ) {
   }
}

impl Dispatch<ZwlrLayerSurfaceV1, ()> for Delegate {
   fn event(
       _: &mut Self,
       layer_surface: &ZwlrLayerSurfaceV1,
       event: <ZwlrLayerSurfaceV1 as wayland_client::Proxy>::Event,
       _: &(),
       _: &Connection,
       _: &QueueHandle<Self>,
   ) {
       println!("{:#?}", event);
       if let zwlr_layer_surface_v1::Event::Configure { serial, .. } = event {
           layer_surface.ack_configure(serial);
       }
   }
}

delegate_noop!(Delegate: ignore WlSurface);
delegate_noop!(Delegate: ignore WlOutput);
delegate_noop!(Delegate: ignore WlShm);
delegate_noop!(Delegate: ignore WlShmPool);
delegate_noop!(Delegate: ignore WlBuffer);
delegate_noop!(Delegate: ignore WlCompositor);
delegate_noop!(Delegate: ignore ZwlrLayerShellV1);

fn main() {
   let conn = Connection::connect_to_env().unwrap();
   let (globals, mut event_queue) = registry_queue_init::<Delegate>(&conn).unwrap();
   let qh = event_queue.handle();

   let mut overlay = tempfile::tempfile().unwrap();

   let shm: WlShm = globals.bind(&qh, 1..=1, ()).unwrap();
   let compositor: WlCompositor = globals.bind(&qh, 1..=4, ()).unwrap();
   let layer_shell: ZwlrLayerShellV1 = globals.bind(&qh, 1..=1, ()).unwrap();

   let (width, height) = (1920, 1080);

   let shm_pool = shm.create_pool(overlay.as_fd(), width * height * 4, &qh, ());

   let buffer = shm_pool.create_buffer(0, width, height, width * 4, Format::Abgr8888, &qh, ());
   let surface = compositor.create_surface(&qh, ());
   let layer_surface = layer_shell.get_layer_surface(
       &surface,
       None,
       Layer::Overlay,
       "ScreenshotUtil".to_string(),
       &qh,
       (),
   );

   draw_background([100, 44, 44, 44], &mut overlay);

   // ===
   config(&layer_surface, &surface);
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

   surface.attach(Some(&buffer), 0, 0);
   surface.commit();
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

   thread::sleep(Duration::from_secs(1));
   println!("test1");

   surface.attach(None, 0, 0);
   surface.commit();

   config(&layer_surface, &surface);
   event_queue.blocking_dispatch(&mut Delegate).unwrap();

   thread::sleep(Duration::from_secs(1));
   println!("test2");

   surface.attach(Some(&buffer), 0, 0);
   surface.commit();

   thread::sleep(Duration::from_secs(1));
   // ===

   println!("finish");
   loop {
       event_queue.blocking_dispatch(&mut Delegate).unwrap();
   }
}

fn draw_background(background: [u8; 4], file: &mut File) {
   file.seek(SeekFrom::Start(0)).unwrap();

   let cashed = u32::from_ne_bytes(background);
   let compressed_buf = vec![cashed; 1920 * 1080];
   let uncompressed_buf = bytemuck::cast_slice::<u32, u8>(&compressed_buf[..]);

   file.write_all(uncompressed_buf).unwrap();
}

fn config(layer_surface: &ZwlrLayerSurfaceV1, surface: &WlSurface) {
   layer_surface.set_size(1920, 1080);
   layer_surface.set_anchor(Anchor::Bottom);
   layer_surface.set_margin(0, 0, 0, 0);
   layer_surface.set_keyboard_interactivity(KeyboardInteractivity::None);
   layer_surface.set_exclusive_zone(-1);
   surface.commit();
}

I expect to be able to reconfigure the surface later on by attaching a None to the surface and then committing it

What you are describing here is indeed what the protocol specification says, but this is a behaviour of the Wayland compositor, and is out of the control of this project (wayland-rs).

@elinorbgr Alright understood, Im just not entirely sure of how all of this works under the hood so I can't really figure out if there is problem is with my code, with the compositor (Hyprland in my case), or with the .attach(None) binding.
I opened the issue on hyprland repo but vaxerski said that it seems to work for other people and encouraged me to read trough the source code of some other project that uses rust but it didnt look like they were using wayland-rs in that case so Im wondering if there might be something wrong with .attach(None)
I will try to read trough hyprland code a little more and check how it is interacting with my client code but Im not exactly sure of what the behavior should be right now.

Alright never mind updating everything on my computer fixed the issue so alright I guess thank you.


SumUpResponseException Client error

Hello,
I had this error when trying to connect with this code
$sumup = new \SumUp\SumUp([
'app_id'     => $client_id,
'app_secret' => $client_secret,
'grant_type' => 'password',
'scopes'      => ['payments', 'transactions.history', 'user.app-settings', 'user.profile_readonly'],
'username'   => $sumupEmail,
'password'   => $sumupPwd
]);
Everything was working fine for months and tonight I got an SumUpResponseException and this message on the response :
"Client error"
=> Too many requests, please try again later.))[2] => Array([file] => /homepages/

I noticed that the sumup server regulary have pb on we.
Could you please tell me how or when this issue will be solved ?
Many thanks in advance.

Hi, It looks like your account has been rate limited because of too many requests in a short period. If you are still experiencing the same problem, please open a new issue!


Improve apk install on android

Description
Improve APK install time by taking a hash of the binary that is being installed and writing it to the emulator as an empty file. On next install, check whether the hash exists and if it does then clear user data without performing uninstall/install. Added a new method of resetAppState to decide between clearing user data and reinstallation.
In terms of performance: uninstall/install takes 6-7s, whereas clearing app state takes 400-600ms.
This pull request fixes #2534
Note: this pr is a WIP because of ongoing problems with the unit tests in CryptoUtils.

For API changes:

[X ] I have made the necessary changes in the types index file.

@jonathanmos what's the status here?

Starting a review.


Use formulae such as the binomial theorem and its generalisations to expand an entity

New expander is probably the solution, e. g.
Console.WriteLine("(x + a + b)3".ToEntity().Expand());
>>> b ^ 3 + 3 * a * b ^ 2 + 3 * a ^ 2 * b + a ^ 3 + 3 * x * b ^ 2 + 6 * x * a * b + 3 * x * a ^ 2 + 3 * x ^ 2 * b + 3 * x ^ 2 * a + x ^ 3


Bump protoc-gen-microweb to include latest fixes

Closes #4276

:boom: Acceptance test cs3ApiTests-ocis failed. Further test are cancelled...


Getting 'OverflowError: mktime argument out of range' while importing the database module from latest version of py2neo

Hello All,
Initially I was using a py2neo's version 2021.1.5 within my project and the database module was getting successfully imported in the code.
But, when I have updated the py2neo version to the latest one i.e. 2021.2.0  and then I was trying to import the database module from the py2neo I was getting an OverFlow error mentioning  'OverflowError: mktime argument out of range'.
May I know the reason for the same ? Why the database module was getting successfully imported with the older version of py2neo?
Thank you in advance.

Commenting out
#from interchange import time in data.py and
#Clock.local_offset = Clock(-int(mktime(gmtime(0)))) in time.py resolved the issue for me.

Thanks @sanketjoshi2012,
I have commented out the two lines from data,py and time.py and my code started working without any error.
But may I ask you the reason for doing this change, as we are updating the pre existing files in python and in py2neo.
Thank you in advance.

Hello @psomesh94
Could you give us more details about how to fix this issue?

When I fixed  data.py and time.py, there is another issue. "NameError: name 'database' is not defined"

I have fixed this issue.I use py2neo version 4.2.0, it works well. Thanks!

I have fixed this issue.I use py2neo version 4.2.0, it works well. Thanks!

Thanks, @Jasondou0709,  I have uninstalled my current version of py2neo then by using pip install py2neo==4.2.0 .
Finally worked

Hey all, may i ask for some reproducible code?
Importing the database module works for me with py2neo-2021.2.0
from py2neo import database
database.Graph()
print("Whats the issue here?")

If there is no feedback i assume the issue is obsolete


Quit if user has provided arguments

When the user provides arguments, perform arg-action and then quit(). Arg-actions should only be performed once.
Reason:

When enabling plugins and args are present, restarting nimwc will try to perform the arg-actions again.
When running in production and args are present, a restart/breakdown will try to perform the arg-actions again.

Ohh I got 2 PR in one... PR #db39763 is about the password lenght. The pass length in createAdmin and login is not the same. CreateAdmin = 9, login = 10.

No problem, looks good to me. :+1:
Im Merging it in a few hours, if you dont do it before. :slightly_smiling_face:


main -> saveDeofbfuscatedFile error

at line 23:
const script_obf = (await fetchAkamaiScript(url)); -> const script_obf = (await fetchAkamaiScript(target));

Fixed ! Sorry, forgot to push all the changes


Error fetching Kubernetes resource: '/k8s/clusters/c-rv4fk/apis/networking.k8s.io/v1/ingresses', error: UNKNOWN_ERROR, status code: 404

Hi,
I got this error from fetching ingress info, but i could fetch ingress data when configuring the customResources like below.

The thing is that i try to override the apiVersion like this, it didn't work.

error configruation, should be ingresses: 'v1beta1'. close the issue


optional socket, http connection

In fact, websocket as a feature does not recommend closing.
The /public directory also has a role, not yet implemented, nor recommended changes.


Update release notes for 1.4.1

As the title says

@rhatdan @TomSweeneyRedHat @haircommander @baude PTAL

Anyone want to merge this?

@vrothberg can you give it a LGTM and I'll merge?

/lgtm


Pull request for a lot of changes and step-by-step guide

I've merged some changes with some significant changes.. some commentary..

Try not to remove functionality without replacing or discussing it.
Try to create pull requests for things in logical chunks. Group a bunch of related changes together and create a pull request with a comment tying them all together.
Beware of line-endings, this repository uses unix line endings. When editing in Windows, be sure you use an editor that respects those line endings and doesn't automatically convert the file.
Where possible, reduce duplicate code. Take a look at what I implemented for your "downtwo" etc commands. I turned a few hundred lines into about 15
Beware of including your personal App ID and IP information. If for no other reason than to protect yourself from potential abuse. That stuff doesn't really belong on Github.

Otherwise, thanks for the input! I've added you to the contributors and have merged your changes in!

In terms of changing the folder names....that was more me being lazy. I had it changed on my desktop and once I realized github took them as is, I never went to edit the folder names.
The IP and app ID should be both made up. I just randomly replaced keys on my original so people would see an example within the guide.
Haven't looked at your code change for the downtwo example, but very glad to hear there is a way of making better! As you can see, I certainly am no programmer - I am merely decent at seeing how something works and duplicating/expanding a bit. Creating is a different aspect :)


Removed "sleep 2" now that cromwell uses sync

:+1:

:+1:

ToL: 0-10, how confident are we that sync has really fixed this?

@cjllanwarne Let's find out. Either way even if the sleeps were doing something it wasn't a good thing for us to have done.

I'm at 9 confidence. The definition of the sync command is exactly what we've been looking for. Rebasing and merging.


JSON type

Hello, thanks for this great library 👋
I was wondering if you'd be interested in a documentation for a JSON serializable type. That's the type that you can safely serialize and de-serialize via JSON.parse(JSON.stringify(thing)). Zod mentions it in their documentation.
My suggestion (based on Zod's implementation) would be:
const JsonLiteral = v.union([v.string(), v.number(), v.boolean(), v.null()]);
type JsonLiteral = v.InferOutput<typeof JsonLiteral>;

type Json = JsonLiteral | { [key: string]: Json } | Array<Json>;
const Json = v.lazy(
  (): v.UnionSchema<any, undefined> =>
    v.union([JsonLiteral, v.array(Json), v.record(v.string(), Json)]),
);

Maybe this could even be included in valibot so that people don't need to copy it.

Where would you place to code in the docs?
Here is a JSON schema I wrote a few months ago:
type Json = string | number | boolean | null | { [key: string]: Json } | Json[];
type LiteralIssue = StringIssue | NumberIssue | BooleanIssue | NullIssue;
type JsonIssue = LiteralIssue | ArrayIssue | RecordIssue;
type JsonSchema = BaseSchema<Json, Json, UnionIssue<JsonIssue> | JsonIssue>;
export const jsonSchema = v.lazy(() =>
	v.union([v.string(), v.number(), v.boolean(), v.null_(), v.array(jsonSchema), v.record(v.string(), jsonSchema)])
) as JsonSchema;

I think I'd expect it under Guides > Advanced.
What do you think about implementing it as valibot action?

We could at it to https://valibot.dev/guides/other/. Feel free to create a PR.
I see and DX advantage of adding a json schema. But I have to think about it because it is a very special kind of schema compared to our other schemas that just mimic TS type functionality. Also, we have had ideas in the past to add something like a json action to validate JSON strings. We need to make sure the naming makes sense. I think this is something I would prefer to investigate after our v1 release.

I've created a PR. valibot.dev/guides/other is the perfect spot, right below lazy :)
I understand that shipping a json schema is a special case that you might want to think about. I think it would be nice 😁


Incorrect SignalCount causes missed signals for reset signals

Given the following scenario;
  repeat-until:
  - repeat-until: TEST_DONE
    then:
    - wait-for: READY
    - set-state: RUN.counter ${{= ${{RUN.counter}} + 1}}
    - set-signal: READY 1

  run-test:
  - for-each:
      name: it
      input: ${{tests}}
    then:
    - log: running test ${{it}}
    - signal: READY
    - sleep: 1s
  - signal: TEST_DONE

states:
  counter: 0
  tests: ['test1', 'test2', 'test3']

A user would expect that run-test would iterate through the array of tests, and for each test signal READY. Therefore the RUN.counter state would be 3 at the end of the run.
However, because READY signal is reset with set-signal: READY 1, when the script is initially parsed, the count for the READY signal is 2.  It is counted once for - signal: TEST_DONE and again at line  - set-signal: READY 1.  Therefore, the 1st signal is missed, as the waiters will not fire until the count reaches 0.
This https://github.com/johnaohara/qDup/commit/c5492f8b872686ea4851d340cdcf9e0ff3a6a382
fails with the following error;
10:17:35.862 [main] DEBUG qdup._tmp_qdup16523449844493090464.state - run-1664356654666 starting state:
 counter = 0
 tests = ["test1","test2","test3"]

10:17:36.304 [execute-4] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:script-cmd: repeat-until
10:17:36.304 [execute-3] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:script-cmd: run-test
10:17:36.305 [execute-5] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:repeat-until
10:17:36.306 [execute-6] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:run-test
10:17:36.306 [execute-7] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:repeat-until: TEST_DONE
10:17:36.333 [execute-8] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:for-each: it = test1
10:17:36.336 [execute-10] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:running test test1
10:17:36.337 [execute-11] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:signal: READY
10:17:37.338 [schedule-0] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:sleep: 1s
10:17:37.344 [execute-14] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:for-each: it = test2
10:17:37.346 [execute-15] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:running test test2
10:17:37.347 [execute-16] INFO  qdup._tmp_qdup16523449844493090464 - reached READY
10:17:37.347 [execute-16] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:wait-for: READY
10:17:37.348 [execute-16] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:signal: READY
10:17:37.649 [execute-17] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:set-state: RUN.counter 1
10:17:37.650 [execute-19] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:set-signal: READY 1
10:17:37.651 [execute-21] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:repeat-until: TEST_DONE
10:17:38.349 [schedule-2] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:sleep: 1s
10:17:38.353 [execute-24] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:for-each: it = test3
10:17:38.354 [execute-25] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:running test test3
10:17:38.354 [execute-26] INFO  qdup._tmp_qdup16523449844493090464 - reached READY
10:17:38.355 [execute-26] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:wait-for: READY
10:17:38.355 [execute-26] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:signal: READY
10:17:38.493 [execute-27] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:set-state: RUN.counter 2
10:17:38.494 [execute-29] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:set-signal: READY 1
10:17:38.494 [execute-31] INFO  qdup._tmp_qdup16523449844493090464 - repeat-until:38@localhost:repeat-until: TEST_DONE
10:17:39.355 [schedule-4] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:sleep: 1s
10:17:39.361 [execute-0] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:for-each: it
10:17:39.362 [execute-0] INFO  qdup._tmp_qdup16523449844493090464 - reached TEST_DONE
10:17:39.362 [execute-0] INFO  qdup._tmp_qdup16523449844493090464 - run-test:37@localhost:signal: TEST_DONE
10:17:39.377 [execute-0] DEBUG qdup._tmp_qdup16523449844493090464.state - run-1664356654666 closing state:
 counter = 2
 tests = ["test1","test2","test3"]
 localhost :
   run-test :
     id=20 :
   repeat-until :
     id=18 :

java.lang.AssertionError:
Expected :3
Actual   :2
<Click to see difference>

With the fix in #138 , this works;
  repeat-until:
  - repeat-until: TEST_DONE
    then:
    - wait-for: READY
    - set-state: RUN.counter ${{= ${{RUN.counter}} + 1}}
    - set-signal:
        name: READY
        count: 1
        reset: true

  run-test:
  - for-each:
      name: it
      input: ${{tests}}
    then:
    - log: running test ${{it}}
    - signal: READY
    - sleep: 1s
  - signal: TEST_DONE

states:
  counter: 0
  tests: ['test1', 'test2', 'test3']

I think #139 is the solution we should use. It avoids incrementing the expected signal count with set-signal and does not change the default behavior for set-signal We should probably open another issue about renaming set-signal's reset


How to make dynamic requests?

Description
I load a model with dynamic batch config
dynamic_batching {
preferred_batch_size: [ 2, 4 ]
max_queue_delay_microseconds: 2000
}
And I send a request to http rest api with following header
headers = {
"NV-InferRequest": 'batch_size: 2 input { name: "%s" } output { name: "%s" cls { count: 1 } }' % (input_name, output_name) }
The tensorrt server return following
model_name: "inception_graphdef"
model_version: 1
batch_size: 2
output {
name: "InceptionV3/Predictions/Softmax"
batch_classes {
cls {
idx: 505
value: 0.826383293
label: "COFFEE MUG"
}
}
batch_classes {
cls {
idx: 505
value: 0.826383293
label: "COFFEE MUG"
}
}
}
The tensorrt document said "Dynamic batching is a feature of the inference server that allows inference requests to be combined by the server"
How is it works?
I expected the combined result, but it return all responses without combine
I try to change batch_size in header to 1 or 3, the result is same.
How to make server to combine for each request?

You will always get a response for each request you make. The dynamic batcher combines multiple requests together for inference processing to improve throughput, but it does not actually combine the results. You will still get a response for each request. This presentation talks about the dynamic batcher: https://on-demand-gtc.gputechconf.com/gtcnew/sessionview.php?sessionName=s9438-maximizing+utilization+for+data+center+inference+with+tensorrt+inference+server


Don't reject tags?

So I have a git repo where there used to be some large .bak files.  No current commits should point to them, but when we do a
git tag -a tagname
git push origin tagname
the filesize hook blocks tag commit based upon size.
I could work around this if there were a way to exclude specific files from the size filter (there are only 3), but my attempts to do so have failed.

There is another request open to be able to exclude tags completely (#3), which should solve your problem.
In addition I think to add an option to also to allow to specify an exclude pattern.

Added an option in version 2.1.0 to specify exclude patterns, see #8. Hope to find some time in the next days to exclude tags completely from checks.

Bug fix release 2.1.1 available as 2.1.0 was incomplete

Nice!  Excluding tags would still be a great addition.

I finally had some time and tags are now excluded from checks starting with version 2.2.0, which was just released to the Atlasssian Marketplace.


[New Port Request]

Library name
kirigami
Library description
A QtQuick based components set
Source repository URL
https://github.com/KDE/kirigami.git
Project homepage (if different from the source repository)
https://develop.kde.org/frameworks/kirigami/
Anything else that is useful to know when adding (such as optional features the library may have that should be included)
Kirigami is a set of QtQuick components for building adaptable user interfaces based on QtQuick Controls 2. Kirigami makes it easy to create applications that look and feel great on Plasma Mobile, Desktop Linux, Android, MacOS, and Windows.

Comment to keep issue open. Personally, I would prefer a Qt6-based version when Plasma 6 gets released.

Comment to keep issue open

bump

bump


Improve shadow for mesh layer

Background
Mesh layer has a sampler which blocks the binding of shadow map sampler array. Because of that the shadow map texture sampling always returns [0, 0, 0, 0]. Made the full mesh layer rendered with shadow.

Change List

fix the shadow map sampler issue
new bounding box for identity mode
add shadow to mesh example
unit test

Coverage increased (+0.008%) to 83.091% when pulling c8890d52ebc5894f154f4488d33edeb223d91267 on fix_mesh_shadow into 02c24be128e42dd8c61e629e1e6004c4778d95d6 on master.


Cilium v.1.15 host firewall ignores virtual IPs on the localhost (lo) interface

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
On our Kubernetes clusters we have (kube-vip) virtual IPs on the lo interface. We are running cilium with kube-proxy-replacement and host-firewall, and we have a CCNP to restrict ingress to all of the host IPs. After upgrading cilium from v1.14 to v1.15 the existing CCNP starts allowing all connections to the lo VIP. Connections to other host IPs (e.g. on bond0) are still dropped as expected.
When comparing the cilium ip list output between 1.14 and 1.15, I no longer see the VIP listed as reserved:host in the new cilium version (but it shows up as reserved:world).
Is this expected behavior in cilium v1.15?
Cilium Version
v1.15 (reproduces with both v1.15.2 and v.1.15.6)
Kernel Version
6.1.90-flatcar
Kubernetes Version
v1.29.5
Regression
v1.14.11
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Any update on this? Curious whether this is expected behavior or not.

I don't see a release note in the upgrade guide here, so I'm not sure whether this was intentional:
https://docs.cilium.io/en/v1.15/operations/upgrade/#upgrade-notes

This could be related to https://github.com/cilium/cilium/issues/33234. The fix for that has been merged and will be in the next v1.15 release. The loopback interface is usually ignored unless it's explicitly listed with "devices" option though, so that might also be needed in addition to this. In addition to the "cilium-dbg ip list" command you can also see addresses that Cilium considers as the node's addresses can also be inspected with "cilium-dbg statedb node-addresses".

Thanks, looking forward to trying the next release to see if it fixes this issue.
In our case we do not have enable-runtime-device-detection but from what I can tell this is related to the fix in the second commit.
We've never had to explicitly list lo under devices (and it works fine in 1.14) but maybe that's a change in 1.15. Although I've tried adding lo to the devices list in v1.15.6 and it didn't make a difference.

I tested today with 1.16.0 which includes the above mentioned fix but we're still having the same problem - the lo IP is not recognized as a reserved:host. I also tried adding lo to the devices list but it didn't make a difference (except now it shows lo under the Host Firewall devices).
The lo VIP in our case is 150.75.xx.xx - below is relevant output after deploying v1.16.
Is there anything else we need to configure in order to get this working properly? This stopped working in v1.15.
cilium-dbg status
KVStore:                 Ok   Disabled
Kubernetes:              Ok   1.29 (v1.29.2) [linux/amd64]
Kubernetes APIs:         ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement:    True      [lo    <IP_ADDRESS> 150.75.xx.xx, bond0     140.28.xx.xx <IP_ADDRESS>  (Direct Routing)]
Host firewall:           Enabled   [lo, bond0]
SRv6:                    Disabled
CNI Chaining:            none
CNI Config file:         successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium:                  Ok   1.16.0 (v1.16.0-82999990)
NodeMonitor:             Listening for events on 64 CPUs with 64x4096 of shared memory
Cilium health daemon:    Ok
IPAM:                    IPv4: 33/254 allocated from <IP_ADDRESS>/24,
IPv4 BIG TCP:            Disabled
IPv6 BIG TCP:            Disabled
BandwidthManager:        EDT with BPF [CUBIC] [lo, bond0]
Routing:                 Network: Tunnel [vxlan]   Host: BPF
Attach Mode:             Legacy TC
Device Mode:             veth
Masquerading:            BPF   [lo, bond0]   <IP_ADDRESS>/24 [IPv4: Enabled, IPv6: Disabled]
Controller Status:       208/208 healthy
Proxy Status:            OK, ip <IP_ADDRESS>, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range:   min 256, max 65535
Hubble:                  Disabled
Encryption:              Wireguard       [NodeEncryption: OptedOut, cilium_wg0 (Pubkey: WVgu2813dVKeRy9+k1bQ3WYKvRihmFtMPTUyTzvWxWY=, Port: 51871, Peers: 0)]
Cluster health:          1/1 reachable   (2024-07-24T21:30:15Z)
Modules Health:          Stopped(0) Degraded(0) OK(126)

cilium-dbg statedb node-addresses
# Address                   NodePort   Primary   DeviceName
140.28.xx.xx                false      false     bond0
<IP_ADDRESS>                true       true      bond0
150.75.xx.xx                false      true      *
<IP_ADDRESS>               false      true      cilium_host

The 150.75.xx.xx IP is missing below and consequently, connections to that IP are not being blocked by cilium network policies.
cilium-dbg ip list | grep "reserved:host"
<IP_ADDRESS>/32               reserved:host
140.28.xx.xx/32               reserved:host
<IP_ADDRESS>/32              reserved:host

Indeed lo is excluded when computing the node addresses, while the devices controller is selecting it for use (https://github.com/cilium/cilium/blob/main/pkg/datapath/tables/node_address.go#L443).
I'll prepare a fix.
A workaround for this issue is to assign the device to a non-loopback device, e.g. dummy0.

Thanks a lot @joamaki ! This will allow us to keep up with the latest cilium updates as we're currently stuck on 1.14. I can also look into using a different device (like dummy0) but we have a ton of existing clusters with this setup so it will require some work to change.
From cilium perspective is what we're doing (assigning a public VIP on lo) considered an anti-pattern?

I wouldn't say it's an anti-pattern, it's just slightly unusual, which is why the recent refactoring didn't manage to retain the behavior.

Just tested with v1.15.8 and it appears to resolve this issue! Loopback addresses are now properly handled by the cilium network policies. Thanks a lot for the fix!


How can tinkerbell actions handle disk/device name changes when a host has multiple disks?

When a host has multiple disks, the disk names depends on the order of being detected during system boot.
How to handle this in tinkerbell actions which usually depend on DEVICE/DISK name?
Expected Behaviour

There should be a stable way to define tinkerbell actions.

Current Behaviour

Possible Solution

Steps to Reproduce (for bugs)

Context

Your Environment

Operating System and version (e.g. Linux, Windows, MacOS):

How are you running Tinkerbell? Using Vagrant & VirtualBox, Vagrant & Libvirt, on Packet using Terraform, or give details:

Link to your project or a code example to reproduce issue:

You can construct your Tinkerbell Template object to reference disks listed on the Hardware (see any example at https://github.com/tinkerbell/playground/blob/main/vagrant/template.yaml#L22).

@chrisdoherty4 It may not help much, in hardware.csv, we need to use the device names, which may change, it may help if we can use the device ID in hardware.csv

@ygao-armada I think you're referring to EKS Anywhere's hardware CSV. For assistance with EKS Anywhere you should raise an issue on EKS Anywhere's repository.


no healthy host for HTTP connection pool, ':status', '503'

Bug description
I use istio ingress-gateway enable https in aws EKS and then test my service by locust.
When testing, everything seems good, but when I stop testing, the error occurs.
2019-10-28T09:30:30.543691991Z [2019-10-28 09:30:30.543][22][debug][upstream] [external/envoy/source/common/upstream/cluster_manager_impl.cc:1146] no healthy host for HTTP connection pool
2019-10-28T09:30:30.543787183Z [2019-10-28 09:30:30.543][22][debug][http] [external/envoy/source/common/http/async_client_impl.cc:91] async http request response headers (end_stream=false):
2019-10-28T09:30:30.543794774Z ':status', '503'
2019-10-28T09:30:30.54379803Z 'content-length', '19'
2019-10-28T09:30:30.54380137Z 'content-type', 'text/plain'

Expected behavior
':status', '200'
Steps to reproduce the bug

apply an test service;
bind a https cert to the test service;
use locust to run to http test;
press [Ctrl +C] to stop locust test, then you will see 503 error.

Version (include the output of istioctl version --remote and kubectl version)
client version: 1.3.3
control plane version: 1.3.0
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13+", GitVersion:"v1.13.11-eks-5876d6", GitCommit:"5876d6b7429820450950ade17fe7b4bf5ccada7f", GitTreeState:"clean", BuildDate:"2019-09-24T20:54:25Z", GoVersion:"go1.11.13", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed?
by the repo: https://github.com/LinLshare/istio-test
Environment where bug was observed (cloud vendor, OS, etc)
Amazon Linux

Seems like this may just be locust nit gracefully shutting down and closing the connection before a request is complete?

@howardjohn maybe, but in our production environment, there are several 503 errors one after one every about 15 minutes, so annoying 😣. It's all the client side's wrong, or I should ignore the error ?

and I wonder why client side's wrong results in 503 error. I think 503 should be the server side's error called  503 internal error.

So, 503s are basically the catch-all error for any issues, so just knowing you get 503s is not enough to root cause this. I would take a look at https://karlstoney.com/2019/05/31/istio-503s-ucs-and-tcp-fun-times, it is an excellent blog about debugging a 503 problem (in a previous release, which has now been fixed). We also have some more 503 fixes in 1.3.4 coming out this week.
One immediate thing you can do is turn on access logs https://istio.io/docs/tasks/telemetry/logs/access-log/ and there is a "Response Flag" field that will give more details about the 503 (like DC, UC, etc)

@howardjohn thanks a lot, I will try it

I had a try, it shows DC as following
[2019-10-28T16:02:46.181Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 329 0 0 - "<ip-a>" "python-requests/2.22.0" "6ec21141-cd04-4f3d-9582-4e106b393c7a" "<domain>" "<IP_ADDRESS>:8080" inbound|8080|http|test-app.default.svc.cluster.local - <ip-b>:8080 <ip-a>:0 - default

std::string ResponseFlagUtils::DOWNSTREAM_CONNECTION_TERMINATION = "DC";
Did you look for parameters in locust to see if 503 could be mitigated?
Is the below timestamp close to the locust stop timing? Which component is print the log? ingressgateway or a sidecar proxy behind ingressgateway?

2019-10-28T09:30:30.543691991Z [2019-10-28 09:30:30.543][22][debug][upstream] [external/envoy/source/common/upstream/cluster_manager_impl.cc:1146] no healthy host for HTTP connection pool
2019-10-28T09:30:30.543787183Z [2019-10-28 09:30:30.543][22][debug][http] [external/envoy/source/common/http/async_client_impl.cc:91] async http request response headers (end_stream=false):

The log also shows no healthy host for HTTP connection pool. The upstream service of this ingress/sidecar jitter? Did you shutdown the tested service before shutting down locust?

@lambdai I try to answer your 3 questions,

I use locust to test my service, just because 503s occur in our production env, so I had not tried to look  look for parameters in locust for mitigating.
Yes, and the sidecar proxy print the log. I open the istio-proxy's debug logs following this article https://karlstoney.com/2019/05/31/istio-503s-ucs-and-tcp-fun-times/index.html.
No, I never do anything to the tested service before shutting down locust. I'm confused about the log no healthy host for HTTP connection pool

I am curious how many envoy instances are involved in the test cases.
You mentioned using istio-ingressgateway, so

is it  locust_outside_mesh -> istio-ingressgateway -> service_sidecar_envoy -> service ?
is the service_sidecar_envoy print the DC log?
If both answer are yes, the DC should be referring to istio-ingressgateway and  is either the ip of istio-ingressgateway  or anything between locust and istio-ingressgateway.
Anyway, could you paste the access log of istio-ingressgateway?
What's more, could you extend the time to confirm if you only see the bad request on locust termination?
The sidecar envoy reports DC, which means it probably there is nothing sidecar envoy could do. Is there any logs from ingressgateway.

@lambdai I had tested it again, please look at following.

is it locust_outside_mesh -> istio-ingressgateway -> service_sidecar_envoy -> service ?

YES, I'm sure.

is the service_sidecar_envoy print the DC log?

YES, look at the following log:
locust say bye
^C[2019-11-01 03:24:53,176] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: Traceback (most recent call last):
[2019-11-01 03:24:53,176] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,176] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run
[2019-11-01 03:24:53,176] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,176] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: File "/usr/local/lib/python3.7/site-packages/locust/runners.py", line 114, in start_locust
    locust().run(runner=self)
.......
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: File "/usr/lib64/python3.7/os.py", line 679, in __getitem__
    return self.decodevalue(value)
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: File "/usr/lib64/python3.7/os.py", line 756, in decode
    return value.decode(encoding, 'surrogateescape')
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: KeyboardInterrupt
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: 2019-11-01T03:24:53Z
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr:
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/ERROR/stderr: <Greenlet at 0x7f821150eb90: start_locust(<class 'latte_perf_test.HostListLocust'>)> failed with KeyboardInterrupt
[2019-11-01 03:24:53,178] ip-10-2-17-199.us-west-2.compute.internal/INFO/locust.main: Shutting down (exit code 0), bye.

test-svc service_sidecar_envoy say DC
[ec2-user@ip-10-2-17-199 ~]$ kubectl logs test-svc-75d859c76b-8hzfr -n ecp-cafe -c istio-proxy --tail=20  -f | grep DC
[2019-11-01T03:24:54.128Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 334 0 4 - "<IP_ADDRESS>" "python-requests/2.22.0" "f37ccf8c-6a3d-4a64-9e0b-b8d600ddb697" "test-svc.com" "-" - - <IP_ADDRESS>:8080 <IP_ADDRESS>:0 - -
[2019-11-01T03:24:54.129Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 347 0 6 - "<IP_ADDRESS>" "python-requests/2.22.0" "783dd755-c0e5-4d9f-84a6-7bca0d8b9587" "test-svc.com" "-" - - <IP_ADDRESS>:8080 <IP_ADDRESS>:0 - -
[2019-11-01T03:24:54.132Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 326 0 10 - "<IP_ADDRESS>" "python-requests/2.22.0" "1a866ba2-f6a1-40df-87a0-b66eec8f22a6" "test-svc.com" "<IP_ADDRESS>:8080" inbound|8080|http|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:8080 <IP_ADDRESS>:0 - default
[2019-11-01T03:24:54.135Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 343 0 10 - "<IP_ADDRESS>" "python-requests/2.22.0" "18abc738-2e63-486f-8a6c-26ca54f22f23" "test-svc.com" "-" - - <IP_ADDRESS>:8080 <IP_ADDRESS>:0 - -
[2019-11-01T03:24:54.193Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 338 0 15 - "<IP_ADDRESS>" "python-requests/2.22.0" "5ff5102d-83fb-4736-b5a9-9425c8dea570" "test-svc.com" "-" - - <IP_ADDRESS>:8080 <IP_ADDRESS>:0 - -

istio-ingressgateway say DC
[ec2-user@ip-10-2-17-199 istio-1.3.3]$ kubectl logs istio-ingressgateway-67fbf57b85-m85kq -n istio-system --tail=20  -f | grep DC
[2019-11-01T03:24:54.128Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 347 0 2 - "<IP_ADDRESS>" "python-requests/2.22.0" "783dd755-c0e5-4d9f-84a6-7bca0d8b9587" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:51206 - -
[2019-11-01T03:24:54.127Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 334 0 4 - "<IP_ADDRESS>" "python-requests/2.22.0" "f37ccf8c-6a3d-4a64-9e0b-b8d600ddb697" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:51626 - -
[2019-11-01T03:24:54.131Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 326 0 10 - "<IP_ADDRESS>" "python-requests/2.22.0" "1a866ba2-f6a1-40df-87a0-b66eec8f22a6" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:51684 - -
[2019-11-01T03:24:54.131Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 343 0 14 - "<IP_ADDRESS>" "python-requests/2.22.0" "18abc738-2e63-486f-8a6c-26ca54f22f23" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:51992 - -
[2019-11-01T03:24:54.192Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 338 0 15 - "<IP_ADDRESS>" "python-requests/2.22.0" "5ff5102d-83fb-4736-b5a9-9425c8dea570" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:51696 - -
[2019-11-01T03:24:54.193Z] "POST /host/list/v1 HTTP/1.1" 0 DC "-" "-" 355 0 18 - "<IP_ADDRESS>" "python-requests/2.22.0" "b2a2c68b-7917-4151-8f75-46cd48553be2" "test-svc.com" "<IP_ADDRESS>:8080" outbound|8080|stable|test-svc.test-ns.svc.cluster.local - <IP_ADDRESS>:443 <IP_ADDRESS>:52004 - -

Tip: The public IPs above had been modified for security.

@howardjohn I have tested it with the v1.3.4 istio, but still show the problem. Is there any plan to solve the problem?


MLHR-1687: removed threshold property from block reader

Chandni what is the replacement for this?

Upstream operator is responsible for slowing down if required by the application.

Upstream operator is responsible for slowing down if required by the application.

Sounds good. If it not possible to throttle the input may be we can break within blocks, so end window boundaries within blocks (at record level). I am doing elsewhere in a different operator in another application.


Replace usage of IPv4 address to IPv6

Problem
There are parts of CHIP stack that's using IPv4 address. CHIP has mandated use of IPv6 addresses. The code should be updated to use the mandated addressing scheme.
Proposed Solution
Update the code.

Some of these may be work arounds for various Android and iOS issues during Rendezvous. Before expunging these wholesale, we should pause to analyze the uses to make sure we aren't introducing regressions for previously-solved problems.


REFACTOR-#3326: Get rid of Ray when using OmniSci engine

Signed-off-by: Igoshev, Yaroslav<EMAIL_ADDRESS>
What do these changes do?

[x] commit message follows format outlined here
[x] passes flake8 modin
[x] passes black --check modin
[x] signed commit with git commit -s
[x] Resolves #3326
[ ] tests added and passing
[x] module layout described at docs/developer/architecture.rst is up-to-date

@YarShev resolve conflicts plz

@ienkovich, I will ping you when I introduce the fixes.

@ienkovich , your comments were applied. @anmyachev , the conflicts were resolved.

Please also try to fix "omnisci in the cloud" thing, as if we merge this right now it would be broken

Good catch! Should be fixed.

I wonder if we also have some cloud examples which use "omnisci on ray" now...

Only NYC taxi

I wonder if we also have some cloud examples which use "omnisci on ray" now...

I suggest not to focus on cloud examples. I will fix it separately https://github.com/modin-project/modin/issues/3420

@dchigarev @ienkovich @gshimansky @vnlitvinov ready for review

If now it is necessary to specify MODIN_ENGINE=native to enable Omnisci backend, please make changes to the following dockerfiles:

./examples/docker/census-on-omnisci/census-omnisci.dockerfile
./examples/docker/plasticc-on-omnisci/plasticc-omnisci.dockerfile
./examples/docker/taxi-on-omnisci/nyc-taxi-omnisci.dockerfile

I am sorry for a misleading comment above. I just realized that I was wrong. Currently examples build container images from conda repository. We have modin 0.10.2 there and release 0.10.2 requires MODIN_ENGINE=ray MODIN_BACKEND=omnisci MODIN_EXPERIMENTAL=true to enable omnisci engine. It does not work when MODIN_ENGINE=native and produces as error. This PR should not add MODIN_ENGINE=native to Dockerfiles.

Yeah, I see. Can you create a issue for updating dockerfiles after new modin release is released?

Yeah, I see. Can you create a issue for updating dockerfiles after new modin release is released?

Yes I create #3424. This PR looks good to me now.

I think we have to add some fallback from OmnisciOnRayFactory to OmnisciOnNativeFactory with a warning, that the OmnisciOnRay has been removed, and keep this behavior for a couple of releases until we, or other users who may use Omnisci backend, will change their env vars to appropriate.
For now, setting ENGINE=Ray, BACKEND=Omnisci leads to a completely non-understandable error, which may really confuse users because "everything worked perfectly until they updated modin".
Traceback (most recent call last):
  File "kek.py", line 3, in <module>
    res = pd.DataFrame([1])
  File "/localdisk1/dchigare/repos/modin_bp/modin/pandas/dataframe.py", line 145, in __init__
    distributed_frame = from_non_pandas(data, index, columns, dtype)
  File "/localdisk1/dchigare/repos/modin_bp/modin/pandas/utils.py", line 42, in from_non_pandas
    new_qc = FactoryDispatcher.from_non_pandas(df, index, columns, dtype)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/dispatcher.py", line 163, in from_non_pandas
    return cls.__factory._from_non_pandas(*args, **kwargs)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/factories.py", line 186, in _from_non_pandas
    return cls.io_cls.from_non_pandas(*args, **kwargs)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/dispatcher.py", line 67, in stub
    raise NotImplementedError(
NotImplementedError: Method ExperimentalOmnisciOnRayFactory.from_non_pandas is not implemented

+1

I think we have to add some fallback from OmnisciOnRayFactory to OmnisciOnNativeFactory with a warning, that the OmnisciOnRay has been removed, and keep this behavior for a couple of releases until we, or other users who may use Omnisci backend, will change their env vars to appropriate.
For now, setting ENGINE=Ray, BACKEND=Omnisci leads to a completely non-understandable error, which may really confuse users because "everything worked perfectly until they updated modin".
Traceback (most recent call last):
  File "kek.py", line 3, in <module>
    res = pd.DataFrame([1])
  File "/localdisk1/dchigare/repos/modin_bp/modin/pandas/dataframe.py", line 145, in __init__
    distributed_frame = from_non_pandas(data, index, columns, dtype)
  File "/localdisk1/dchigare/repos/modin_bp/modin/pandas/utils.py", line 42, in from_non_pandas
    new_qc = FactoryDispatcher.from_non_pandas(df, index, columns, dtype)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/dispatcher.py", line 163, in from_non_pandas
    return cls.__factory._from_non_pandas(*args, **kwargs)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/factories.py", line 186, in _from_non_pandas
    return cls.io_cls.from_non_pandas(*args, **kwargs)
  File "/localdisk1/dchigare/repos/modin_bp/modin/data_management/factories/dispatcher.py", line 67, in stub
    raise NotImplementedError(
NotImplementedError: Method ExperimentalOmnisciOnRayFactory.from_non_pandas is not implemented

This approach is too complex for an experimental feature. I think it will be enough to give a message that the user needs to replace the environment variable.

This approach is too complex for an experimental feature. I think it will be enough to give a message that the user needs to replace the environment variable.

What if we just add a warning at around cls.__factory = StubFactory.set_failing_name(factory_name) line to warn the user that the factory isn't there or something like that? This doesn't sound complex and warns the user straight away to look more precisely at what they're doing.

This approach is too complex for an experimental feature. I think it will be enough to give a message that the user needs to replace the environment variable.

What if we just add a warning at around cls.__factory = StubFactory.set_failing_name(factory_name) line to warn the user that the factory isn't there or something like that? This doesn't sound complex and warns the user straight away to look more precisely at what they're doing.

This warning will be more general. In the current version, we have a special case, which allows us to tell the user more explicitly what to do in order to avoid an error.
Although the idea of issuing warnings is appropriate here, it looks like a separate task to me.


Add time elapsed feature in the client

Add time elapsed feature in the client, discord uses
discordPresence.startTimestamp =<PHONE_NUMBER>; <- epoch time stamp
discordPresence.endTimestamp =<PHONE_NUMBER>;
and there should exist something in the js lib for this method
Additionally, we can also add a method to convert epoch time stamp to current time and make other functions as well for the ease of user.

Issue closed, JS lib needed startTimestamp


Receive #ERROR# TypeError: Cannot read property 'columns' of null on run.

I'm trying to set up this action on a private repo, and receive the following log on run:
Run<EMAIL_ADDRESS>This project is configured at the repo level. Repo Owner:[Owner Org], repo name:[Private repo] project number#17
#ERROR# TypeError: Cannot read property 'columns' of null

My workflow YAML looks like this:
on:
  issues:
    types: [labeled]
jobs:
  Move_Labeled_Issue_On_Project_Board:
    runs-on: ubuntu-latest
    steps:
    - name: Move labeled or milestoned issue to a specific project column
      uses<EMAIL_ADDRESS>      with:
        action-token: "${{ secrets.WORKFLOW_ACTIONS }}"
        project-url: "https://github.com/[Owner]/[Repo]/projects/17"
        column-name: "Needs Info"
        label-name:  "Awaiting more info"
        columns-to-ignore: "Done"

The project URL is definitely correct, as are the names of the columns and labels.
This is my first attempt at using actions, so I could be missing something. Any ideas? Thanks in advance!

Your project URL needs to have the actual owner and repo names, something like:
project-url: https://github.com/konradpabjan/move-labeled-or-milestoned-issue/projects/17
If the URL is correct, you should see output that says something like
This project is configured at the repo level. Repo Owner: konradpabjan, repo name: move-labeled-or-milestoned-issue project number#17

Oh - yes, I've just redacted the owner and repo names in the example config and logs here as it's a private repo. The URL in the real config is correct, and yep, I get that output in the logs.
So I don't think that's the issue. Any other ideas?

Had the same issue.
For people finding this old issue in the future:
action-token: "${{ secrets.WORKFLOW_ACTIONS }}"
Remove the quotes here
=> action-token: ${{ secrets.WORKFLOW_ACTIONS }}

@benjamin-hull Did you find a fix for this?
@emre1702 This did not change the error for me.

Dear all,
I have the same error:
#ERROR# TypeError: Cannot read properties of null (reading 'columns')
Any news about it?

I am getting the same error :
#ERROR# TypeError: Cannot read properties of null (reading 'columns')

... could it be because this a new project, and by new I mean the newer version of GitHub Projects, the legacy ones being discontinued?

I am getting the same error :
#ERROR# TypeError: Cannot read properties of null (reading 'columns')

... could it be because this a new project, and by new I mean the newer version of GitHub Projects, the legacy ones being discontinued?

I am also having the same error currently. I have a suspicion you are correct @fredericsimard but I'm hoping you're incorrect lol 😓 😅

This Issue #28 actually has a comment pointing to a better way to achieve similar things that worked for me!

@colinwilliams91 I believe this might fix the issue. I'll try that tomorrow and let you know.


Supabase as a Data Source for Streamlit

Hey Streamlit,
I am one of the maintainers of the python client libraries for Supabase . Supabase is, loosely speaking, an open source firebase. Streamlit is a great product and I was hoping that we could file a PR with a guide on how to use Supabase as a data source for Streamlit.
Wanted to seek your approval before doing this though.
Let us know!
Joel

Hi @J0 -
Supabase looks like a great project, please go ahead and make a tutorial in a similar style to the ones that already exist!
Best,
Randy

Thank you for your help

@randyzwitch have opened a PR, looking forward to your review whenever ready