Deployment fails when using Istio 1.14

Checks

[X] I have searched the existing issues.
[X] This issue is NOT specific to the CLI. (If so, please open an issue on the CLI repo)

deployKF Version
0.1.2
Kubernetes Version
Client Version: v1.24.16
Kustomize Version: v4.5.4
Server Version: v1.24.16

Description
I tried to deploy Kubeflow using deployKF to cluster with separately installed Istio 1.14.6. According to documentation deployKF should be compatible with version 1.14, but is does not seem to be the case. When applying the generated applications to cluster oauth2-proxy fails to start, istiod logs errors (see logs below). I was able to reproduce the issue in k3s/minikube environment.
With istio 1.15+ everything starts successfully. It seems to me that the issue is with deploykf-istio-gateway--ext-authz-disable EnvoyFilter. It uses envoy.extensions.filters.http.lua.v3.LuaPerRoute, which has been introduced in Envoy 1.15.
Relevant Logs
istiod:
2023-10-26T12:55:59.552785Z     warn    ads     ADS:RDS: ACK ERROR deploykf-gateway-d5b485fb6-pqrk5.deploykf-istio-gateway-1 Internal:Didn't find a registered implementation for name: 'envoy.filters.http.lua.set_userid_header'

deploykf-gateway:
2023-10-26T12:42:01.839962Z     warning envoy config    gRPC config for type.googleapis.com/envoy.config.route.v3.RouteConfiguration rejected: Didn't find a registered implementation for name: 'envoy.filters.http.lua.set_userid_header'
2023-10-26T12:42:12.341233Z     warning envoy config    gRPC config for type.googleapis.com/envoy.config.route.v3.RouteConfiguration rejected: Didn't find a registered implementation for name: 'envoy.filters.http.lua.set_userid_header'
2023-10-26T12:55:59.552330Z     warning envoy config    gRPC config for type.googleapis.com/envoy.config.route.v3.RouteConfiguration rejected: Didn't find a registered implementation for name: 'envoy.filters.http.lua.set_userid_header'

oauth2-proxy:
[2023/10/26 13:05:24] [provider.go:55] Performing OIDC Discovery...
[2023/10/26 13:05:24] [main.go:60] ERROR: Failed to initialise OAuth2 Proxy: error intiailising provider: could not create provider data: error building OIDC ProviderVerifier: could not get verifier builder: error while discovery OIDC configuration: failed to discover OIDC configuration: unexpected status "403": ExtAuthz did not set 'x-auth-request-email' header

deployKF Values (Optional)
# Basically default values with some components disabled
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: deploykf-app-of-apps
  namespace: argocd
  labels:
    app.kubernetes.io/name: deploykf-app-of-apps
    app.kubernetes.io/part-of: deploykf
spec:
  project: "default"
  source:
    repoURL: "https://github.com/deployKF/deployKF.git"
    targetRevision: "v0.1.2"
    path: "."

    plugin:
      name: "deploykf"
      parameters:
        - name: "source_version"
          string: "0.1.2"

        - name: "values_files"
          array:
            - "./sample-values.yaml"

        - name: "values"
          string: |
            deploykf_dependencies:
              cert_manager:
                enabled: true
                namespace: cert-manager

                clusterIssuer:
                  enabled: true
                  issuerName: kf-selfsigned-issuer

              istio:
                # Istio is deployed externally
                enabled: false
                namespace: istio-system

              kyverno:
                enabled: true
                namespace: kyverno

            deploykf_core:
              deploykf_auth:
                dex:
                  staticPasswords:
                    - email<EMAIL_ADDRESS>                      password:
                        value: "admin"
                    - email<EMAIL_ADDRESS>                      password:
                        value: "user1"
                    - email<EMAIL_ADDRESS>                      password:
                        value: "user2"
                  clients:
                    oauth2Proxy:
                      clientId: "oauth2-proxy"
                      clientSecret:
                        existingSecret: "generated--dex-oauth2-proxy-client"
                        existingSecretKey: "client_secret"
                        generateSecret: true

              deploykf_istio_gateway:
                namespace: deploykf-istio-gateway
                gatewayService.type: "ClusterIP"

                charts:
                  istioGateway:
                    enabled: true
                    name: "gateway"
                    version: "1.14.6"

                gateway:
                  hostname: deploykf.example.com
                  ports:
                    http: 8080
                    https: 8443
                  tls:
                    enabled: false
                  selectorLabels:
                    app: deploykf-gateway
                    istio: deploykf-gateway

                gatewayDeployment:
                  serviceAccount:
                    name: deploykf-gateway
                    annotations: {}

                gatewayService:
                  name: deploykf-gateway
                  annotations: {}
                  type: ClusterIP  # Do not expose it at least for now

              deploykf_profiles_generator:
                profileDefaults:
                  profileNamePrefix: ""
                  ownerEmail<EMAIL_ADDRESS>
                  memberAccess:
                    role: view
                    notebooksAccess: false

                  plugins: []
                  tools: {}

                users:
                  - id: user-1
                    email<EMAIL_ADDRESS>                  - id: user-2
                    email<EMAIL_ADDRESS>
                groups:
                  - id: team-1
                    users:
                      - user-1
                      - user-2

                profiles:
                  - name: team-1
                    members:
                      - group: team-1
                        access:
                          role: edit
                          notebooksAccess: true

                  - name: team-1-prod
                    members:
                      - group: team-1
                        access:
                          role: view
                          notebooksAccess: false

            deploykf_opt:
              deploykf_minio:
                enabled: false
              deploykf_mysql:
                enabled: false

            kubeflow_dependencies:
              kubeflow_argo_workflows:
                enabled: false

            kubeflow_tools:
              katib:
                enabled: false
              notebooks:
                enabled: true
              pipelines:
                enabled: false

  destination:
    server: "https://kubernetes.default.svc"
    namespace: "argocd"

@mkls6 thanks for raising this, I did a bit of testing and also ran into the same issue when using Istio 1.14.6 with deployKF 0.1.2.
Either way, given how old Istio 1.14 is, I strongly recommend NOT using it, and I will update the website to reflect its incompatibility with deployKF (which will also let me use some of the newer envoy features in future releases of deployKF).
Finally, there is no clear reason why this should not work in Istio 1.14.6, which actually uses Envoy 1.22.7. I suspect this is actually an Istio or Envoy bug that was patched before the release of Envoy 1.23.0 or Istio 1.15.0.
You can check the envoy version of any Istio proxy by using this docker command:
docker run --entrypoint "/usr/local/bin/envoy" docker.io/istio/proxyv2:1.14.6-distroless --version
# /usr/local/bin/envoy  version: a757eae4ff3ad133fc75efa953e6642436316f98/1.22.7-dev/Clean/RELEASE/BoringSSL


allow analysis of magic calls

Is your feature request related to a problem? Please describe.

Remember, that PHP is an interpreted language and allows constructions that can't be statically analyzed. If you write something like SomeClass::$any_function() or new $class_name, NoColor can't do anything about it.

Describe the solution you'd like
The amazing PHPStan by Ondřej Mirtes solves this using an extension structure to dynamically verify 'magic' calls
Describe alternatives you've considered
N/A
Additional context
This issue is more of an FYI than anything else, but I'd be willing to look into making this work if that's something that'd be appreciated.

Hello.
Thanks for the interesting information, today I'll take a look at
how it works in PHPStan, sounds pretty interesting.

In general, some magic methods, IMHO, are already a problem in the architecture, 😄
as they create a bunch of implicitness in the code.
I see no reason to support magic methods that work for unknown methods or fields,
since they are as strange as possible and use them is already a big problem in code quality.
I see the point in supporting the following methods:

__serialize(), __unserialize()
__invoke()
__clone()

These are useful methods and unlike, for example, __toString, it may be necessary to validate.
If you think that some other magic methods also need to be supported, tell me.

If you think that some other magic methods also need to be supported, tell me.

Quite a few popular frameworks rely on __get, __set, __call and __callStatic for syntactic sugar, being able to introspect those would be great 😄

Okay, thanks, then I'll add to the list of magic methods to support.


Allows creating non utf-8 String without unsafe

https://github.com/rpgp/rpgp/blob/66c4f76ee7420c51d4f04e1f44400d4ec2d34ffb/src/util.rs#L189-L191
Playground
This should probably use one of the from_utf8 family of functions ?
pub fn read_string(raw: &[u8]) -> String {
    // this would (violently) prevent invalid utf8 to propagate to other parts of the code
    std::str::from_utf8(raw).unwrap().to_owned()
}

or
pub fn read_string(raw: &[u8]) -> String {
    // Marks this place as unsafe but behave the same as the original cast
    unsafe {
        std::str::from_utf8_unchecked(raw).to_owned()
    }
}

This needs fixing, but I think there were cases where this wasn't valid utf8, so we definitely need to handle that somehow..

The problem here isn't that the creation of non utf-8 Strings is possible (since the method doesn't call unsafe methods itself it would be a serious bug in the standard library), it just doesn't interpret utf-8 in any sensible way but splits the inputs in bytes, converting each one to the UTF-8 code unit corresponding to the byte value. This doesn't create invalid utf-8, but nonsensical output in the case of non ASCII Input
For Example:
🦀 U+1F980: CRAB is in utf-8: f0 9f a6 80 and would be turned by this function into:
ð 00F0 LATIN SMALL LETTER ETH
 009f [control] (Application Program Control, one of the ASCII Control Characters nobody uses)
¦ 00a6 BROKEN BAR
 0080 [control]

This makes the matter better in some regards (no invalid utf-8 strings are created so this isn't undefined behaviour), but worse in others (it is practically impossible to use this functions with Non-ASCII Characters)
If there is interest in fixing this I would love to contribute

@VincentWo yes please, would appreciate a fix for this

Yes, I will try. What do you want to happen in case of invalid utf-8 though?
The RFC 2440 (don't know whether that's the newest, just the first one I found) writes that utf-8 MAY be assumed and everything else MAY be ignored
This would allow three courses of action:

Just replace the invalid utf-8 with "U+FFFD replacement character" using from_utf8_lossy
Return an error on everything that isn't utf-8

The former would allow users to still receive non UTF-8 User ids for example (no idea how common this is in the wild)
The later would be a hard block for anything non UTF-8, but has a greater chance of printing only correct characters.
I would recommend from_utf_lossy, but wanted maintainer input before deciding.
(There is also the theoretical method of supporting multiple encodings, but this would be a longer task and could be done after any of the easier approach has been implemented)

So my current goal is to be able to roundtrip as much existing key material as possible, which means ideally we should attempt to store it as valid utf8, but if it isn't treat it as just bytes, so we can output it again. So probabyl we need some type like this
enum CompatString {
  Utf8(String),
  Other(Vec<u8>),
}

to support this.

How open would you be about adding a dependency? There is the bstr crate supplying a string that is utf-8 by Convention, but also able to contain any bytes. There is a Display Implementation which converts invalid bytes to � and a Debug Implementation which shows hex codes, maintained by BurntSushi this would be probably much more flexible than anything I would come up with :)

bstr looks quite nice. agreed on disabling the unicode feature for code size benefits

Yes thank you, I will start working on it :)

You can see my progress so far, although tests are still missing.
I also stumbled on a question: Currently the User Id of the Signer is a String and rejects invalid utf-8 , same for the preferred keyserver. I feel that it would be the best to have consistency here, which would either mean moving them all to bstr OR to move the User id of the Signer to bstr, but move the PolicyURI to only accept valid utf-8 since a URI is always utf-8.
I would prefer the later, but wanted to check back to make sure

the latter sounds good to me


Is there any way to save a trained model?

I want to save the trained model and make it usable for serving. Is it possible? How?

I believe you can get the model using clf.load_searcher().load_best_model()

I think you can get the model using clf.load_searcher().load_best_model()

Thank @paindefender for the help.
@ankapale
Please add a function of export keras_model() to the ImageClassifier
by clf.load_searcher().load_best_model().produce_keras_model() here.
https://github.com/jhfjhfj1/autokeras/blob/master/autokeras/graph.py#L542
Thanks.
We also need to find a way to export our own model instead of the Keras model.
I will open an issue on that.

Can you look to export using onnx standard?
So models are interoperable and deployable to production environments.
https://onnx.ai

#Replicating the save-file code from commit a3ca627 :
from autokeras import ImageClassifier
clf = ImageClassifier(verbose=True, augment=False)
clf.load_searcher().load_best_model().produce_keras_model().save('test_save.h5')
#This appears successful. Returns null, but a new 7.3MB file 'test_save.h5' is generated from /tmp/autokeras/12.h5 which was originally 7.0MB.
#Using Keras to plot the newly-saved model:
from keras.models import load_model
model = load_model('test_save.h5')
from keras.utils import plot_model
plot_model(model, to_file='model.png')
#The result of this is successful, the architecture is plotted to an image.
#Can also output just the architecture:
from autokeras import ImageClassifier
clf = ImageClassifier(verbose=True, augment=False)
clf.load_searcher().load_best_model().produce_keras_model().to_json()


add test cases for operator/pkg/manifest

Please provide a description for what this PR is for.
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@tanjunchen: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

update-ref-docs-dry-run_istio
3f1ec1e741f42c5aa46fbadb3d8dad3bb2c917d3
link
/test update-ref-docs-dry-run_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


python chapter3

Sorry I have to close this PR for the following three reasons.

You put too much code into one PR, instead, you should create 12 PRs, each PR has one relatively complete code(s). It's very time-consuming and inconvenient to review 12 files in a short time. ( To create 12 PRs at the same time, you need to create 12 branches, each branch contains one example, and create a PR from each branch)

it is not easy to track code differences of ipynb file, so you have to manually convert your .ipynb files to .py, and upload both py and ipynb files. To do this, in your Jupyter Notebook, open the ipynb file and click File->Export Notebook As-> Executable Script

You need keep the folder structure organized, and modify the corresponding part of README.md.

For ipynb files,

delete unnecessary cells( e.g., empty cells in the end)
make sure you Kernel->restart kernel and run all cells
make sure the ipynb and py contains the same codes


Network diagram: Add --cluster-by-tag 'app'

Request from jvehent: https://twitter.com/jvehent/status/1092636918066614272

This might already be implemented from collapse-by-tag, also see #36

I'm closing this issue as the network diagram functionality of CloudMapper is no longer being maintained.  Other features of CloudMapper, such as the audit functionality, is still maintained. If someone is interested in maintaining the network diagram functionality they can look at the issues tagged unmaintained_functionality for all the issues related to this that have been closed as they will unfortunately not be worked on by me.


Screen that the user see when he clicks in the unsubscribe link

When a user receives a notification email, he could unsubscribe clicking in a link at the bottom. That link redirects the user to a screen where he's informed that has been successfully unsubscribed.
I need a design for that.
Thank you!
// @saleiva

Copy is just a placeholder:

Great work!
Just to clarify, just in case, I have to say, my bad, that the unsubscribe is by type of notification. For example shared table, shared viz, data import, maps likes, etc. I think is the same screen but with different text.

Yep, we should keep the same design and just change the text ; )

Where are the resources to layout this? thx

Hi @ethervoid close this issue if you consider. Do you need help for the ui development?

I will send you by email, do you have sketch?

would be great to have one too for subscribed (currently redirected to cartodb.com)

Hi @matallo. I assign it to you because this is one of the "feedback pages" collection. So maybe we need to consider how to make an standard UI Development for all. I will send to you an email with the sketch with all the cases that we should implement at any point.
@mariodelvalle can help us with this

@matallo we also have it :)

+1000
On Fri, Jul 31, 2015 at 12:06 PM, Carlos Tallón<EMAIL_ADDRESS>wrote:

Hi @matallo https://github.com/matallo. I assign it to you because this
is one of the "feedback pages" collection. So maybe we need to consider how
to make an standard UI Development for all. I will send to you an email
with the sketch with all the cases that we should implement at any point.
@mariodelvalle https://github.com/mariodelvalle can help us with this
—
Reply to this email directly or view it on GitHub
https://github.com/CartoDB/cartodb/issues/4640#issuecomment-126632361.

--
Carlos Matallín
UI developer
CartoDB—Map Your World's Data
C/ Eloy Gonzalo, 27, 3, 28010, Madrid, Spain
T: +34911165823
W: www.cartodb.com
Live sessions to learn CartoDB http://cartodb.com/webinars/

Of course :+1:

In the unsubscription could be interesting to give the user the opportunity to give us feedback of why unsubscription to improve service in the future through a link that will send a mail to support. What do you think?

Like what @mariodelvalle proposes!

Were are the resources for the unsubscribe screen to do it?

I moved this to the next milestone

Ok, so I put that on hold

done


Node numbers

Will we get these changed to the letters on the wiki and will they be consistent? (i.e. If you go from node I to Boss node E, will going to Boss node E from another node be see as like Boss node H?)

there is currently no map to create our own map, nor to pattern it with the nodes on the wikis. right now, we can make do with just showing enemy faces on next node before compass spin.

I'm now pretty sure these numbers are for edges.
Just in case we want to change it back to letters and be consistent with wiki maps, I've created a gist note for this and will be updating it.
Please feel free to comment with your knowledge, as I'm still new to the game and can only get access to limited maps.
And it will help to let more people know about this gist :)

ah do we make people comment on what number they got and what letter it is on the wiki?

I think that will be helpful to also create a wiki page for this(or is there an existing one?), let me know the link and I'll also keep an eye on it :smile:

you mean wiki on wikia? xD or wiki on github? :v
I haven't informed the users yet

@dragonjet I think we can start doing the conversion from KanColle server Edge Id to Wikia Node Letter by the current data. For the rest, we still show numbers so we will know that they haven't been done yet.
Can I start working on it now?

sure, something like an \src\data\edges.json?

yes, I will build it from scratch and load in KC3Meta

:+1:


Dispatch APIs down (dev and org)

The Dispatch APIs are down

https://run.api.biosimulations.org/
https://run.api.biosimulations.dev/

The APIs are returning these errors

502 Bad Gateway
504 Gateway Time-out

Maybe related to this error in serving the dispatch api
[Nest] 19815  - 07/13/2021, 3:17:12 PM   ERROR [ExceptionHandler] natsPackage.JSONCodec is not a function
TypeError: natsPackage.JSONCodec is not a function
    at new NatsJSONSerializer (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/microservices/serializers/nats-json.serializer.js:9:38)
    at ClientNats.initializeSerializer (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/microservices/client/client-nats.js:121:136)
    at new ClientNats (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/microservices/client/client-nats.js:18:14)
    at Function.create (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/microservices/client/client-proxy-factory.js:23:24)
    at InstanceWrapper.useFactory [as metatype] (/home/jonrkarr/Documents/Biosimulations/dist/apps/dispatch-api/webpack:/libs/shared/nats-client/src/lib/shared-nats-client.module.ts:16:35)
    at Injector.instantiateClass (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/injector.js:290:55)
    at callback (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/injector.js:43:41)
    at Injector.resolveConstructorParams (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/injector.js:115:24)
    at Injector.loadInstance (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/injector.js:47:9)
    at Injector.loadProvider (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/injector.js:69:9)
    at async Promise.all (index 3)
    at InstanceLoader.createInstancesOfProviders (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/instance-loader.js:44:9)
    at /home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/instance-loader.js:29:13
    at async Promise.all (index 8)
    at InstanceLoader.createInstances (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/instance-loader.js:28:9)
    at InstanceLoader.createInstancesOfDependencies (/home/jonrkarr/Documents/Biosimulations/node_modules/@nestjs/core/injector/instance-loader.js:18:9)

I believe that error should go away with a fresh npm-install. The issue on
the deployment is different. The apps are not able to connect to the
message broker. I updated the broker and the config to see if that
resolves it.
On Tue, Jul 13, 2021 at 3:59 PM Jonathan Karr @.***>
wrote:

Maybe related to this error in serving the dispatch api
[Nest] 19815  - 07/13/2021, 3:17:12 PM   ERROR [ExceptionHandler] natsPackage.JSONCodec is not a function
TypeError: natsPackage.JSONCodec is not a function
at new NatsJSONSerializer @./microservices/serializers/nats-json.serializer.js:9:38)
at ClientNats.initializeSerializer @./microservices/client/client-nats.js:121:136)
at new ClientNats @./microservices/client/client-nats.js:18:14)
at Function.create @./microservices/client/client-proxy-factory.js:23:24)
at InstanceWrapper.useFactory [as metatype] (/home/jonrkarr/Documents/Biosimulations/dist/apps/dispatch-api/webpack:/libs/shared/nats-client/src/lib/shared-nats-client.module.ts:16:35)
at Injector.instantiateClass @./core/injector/injector.js:290:55)
at callback @./core/injector/injector.js:43:41)
at Injector.resolveConstructorParams @./core/injector/injector.js:115:24)
at Injector.loadInstance @./core/injector/injector.js:47:9)
at Injector.loadProvider @./core/injector/injector.js:69:9)
at async Promise.all (index 3)
at InstanceLoader.createInstancesOfProviders @./core/injector/instance-loader.js:44:9)
at @./core/injector/instance-loader.js:29:13
at async Promise.all (index 8)
at InstanceLoader.createInstances @./core/injector/instance-loader.js:28:9)
at InstanceLoader.createInstancesOfDependencies @.***/core/injector/instance-loader.js:18:9)
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/biosimulations/Biosimulations/issues/2688#issuecomment-879361857,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHX4FIHEF4PCIYQCTBSNZ7DTXSLLBANCNFSM5AJ3UJCQ
.

I tried that a couple of times. I think I got past the JSONCodec issue.
Now I see another issue. Not sure if this is related to secrets.
(node:24415) UnhandledPromiseRejectionWarning: NatsError: CONNECTION_REFUSED
    at Function.errorForCode (/home/jonrkarr/Documents/Biosimulations/node_modules/nats/nats-base-client/error.ts:115:12)
    at NodeTransport.<anonymous> (/home/jonrkarr/Documents/Biosimulations/node_modules/nats/src/node_transport.ts:82:21)
    at Generator.throw (<anonymous>)
    at rejected (/home/jonrkarr/Documents/Biosimulations/node_modules/nats/lib/src/node_transport.js:6:65)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

This is the issue that I am running into in the deployment, but I am unable to replicate it locally. Are you running the NATS docker image locally?

No

Wasn't aware I needed to

Resolved in https://github.com/biosimulations/Biosimulations/commit/1851742fdb222a6330a6b2f52b814ee7d3273c5a
Need to run a couple of simulations to confirm all messaging is working correctly.

Wasn't aware I needed to

Two options here. For both, the nats server will be located at NATS://localhost:42222
To run a nats server locally (isolated from the deployments messages):
  docker run  -p 4222:4222 nats

To connect to the nats-server running in the dev deployment (messages sent from local will be received by deployment and vice-versa):
gcloud container clusters get-credentials biosimulations --zone us-central1-c --project biosimulations  && kubectl port-forward --namespace dev $(kubectl get pod --namespace dev --selector="app=nats,nats_cluster=nats-cluster" --output jsonpath='{.items[0].metadata.name}') 4222:4222

Thanks!


In relation to #146 pull request

Had to create a new branch, I've fixed duplicate addition of headers.

Thank you for opening an issue against the Bot Framework Cognitive Services repo. As part of the Bot Framework v4 release, we’ve moved all the Luis and QnA work to platform SDK repo. We will continue to support and offer maintenance updates via this repo.
From now on, https://github.com/Microsoft/botframework-sdk repo will be used as hub, with pointers to all the different SDK languages, tools and samples repos.
As part of this restructuring, we are closing all tickets in this repo.
For defects or feature requests, please create a new issue in the Bot Framework repo found here:
https://github.com/Microsoft/botframework-sdk/issues
For product behavior, how-to, or general understanding questions, please use Stackoverflow.
https://stackoverflow.com/search?q=bot+framework
Thank you.
The Bot Framework Team


Update to reflect shake animation

Updated the small characters page to reflect shake animation update

The credentials to view the Netlify deploy is the following:

username: crabigator
password: googlebegone

Deploy preview for wanikani-knowledgebase processing.
Building with commit 541affc20da55b35deaecf2ae1e29d6670234601
https://app.netlify.com/sites/wanikani-knowledgebase/deploys/602ec83a327e590007492699


Changing package dir to use pip install via a commit sha

does it work in the repo uses this?

Ship-it :)


fix: 150 Sales Order Connection tab getting error

After adding frappe.db.rollback() the error is not showing.
Reference doc :

https://stackoverflow.com/questions/10399727/psqlexception-current-transaction-is-aborted-commands-ignored-until-end-of-tra
https://dba.stackexchange.com/questions/303935/error-current-transaction-is-aborted-commands-ignored-until-end-of-transaction

Covers :
https://github.com/8848digital/erpnext/issues/150
https://github.com/8848digital/erpnext/issues/192
https://github.com/8848digital/erpnext/issues/92

changes may not able to solve this issue entirely and violation of Semgrep rule


change 'doco' to documents and complete the sentence

Reason
Currently the READ.me says "More doco over there" and as a first time traveler to this repo I was caught a bit off guard. After thinking about it doco just meant documentation, so I'm proposing a different sentence.
Implementation
Update the sentence.

Thanks for taking the time


Grammar update

For issue #31 -- I pretty much overhauled grammar.txt, mostly based on the website version. Let me know what you think!

thank you, very nice and sorry for late merge


Test username is showing consistently between pages

Closes: Granola-Team/mina-indexer#1067

So, the test is failing. Is the test wrong, or is the feature not present?

The test is failing because the feature is not yet present. TDD

The feature is present... Retesting....

Usernames show up in the accounts page: https://minasearch.com/addresses/accounts?q-public-key=B62qq3tqfdj19hqaVCozJFM2q9gT2WezQMaJMKD6wxyvK3fMpHiP9va

@trevorbernard test is showing the same


Move Tests to Mocha

WebFundamentals has a bunch of testing somewhere, but it looks like it's all test.js
@petele would you be willing to move this to mocha? Should make things more readable, extensible and maintainable.

I think I looked at it a while ago and can't remember why I decided against it. Transitioning to Mocha isn't a high priority for me, but happy to discuss if you want to take it on.


Period 4 Brown: Genres NATM Feedback

Genres Website:
https://nighthawkcoders.github.io/flocker_frontend/voteforthegoat/genres/

Non CSP Feedback
Drishya: I liked the frontend and how it is interavtive with the user and it provides a discussion board to engage users in conversation. I liked how you implemented the API requests through the frontend by fetching it. Great work!
Non CSP Feedback
Rhea's Mom: I like the colors and design of your website. It is very engaging.
CSP Feedback
Carson: This website is the cutest one I've seen! It is so creative and fun to look at I'm such a huge fan 100/10
Ava Shallon: I love that I can talk about my favorite artists in your site! Such a creative debate idea great job!
Lalita: Great job creating multiple channels. The disks on the main page are so cute and look great.


It isn't work in my atom

[ ] I have reset Atom to defaults prior to submitting report.
[ ] I have not reset Atom to defaults prior to submitting report.

Description

Steps to reproduce

It can't work on last version of atom downloaded from atom website. It gives me just a blank pane.
It's on Ubuntu 20.4

Expected behavior:
Actual behavior:
Reproduces how often:
Versions

> atom --version

> apm --version

OS name and version:
Platformio-ide-terminal version:
Additional Information

@ahmadrezash see and read https://github.com/platformio/platformio-atom-ide-terminal/issues/668#issuecomment-633130699
Also this ticket contains no helpful info so closing.


Inconsistent Saved state of files, after many hours

VSCode Version: Code 1.18.0 (dcee2202709a4f223185514b9275aa4229841aa7, 2017-11-08T12:44:48.751Z)
OS Version: Linux x64 4.4.0-83-generic
Extensions: the listing length exceeds browsers' URL characters limit

Steps to Reproduce:

Clean Install on LinuxMint 18.2 (Full remove of previous versions from deb package and ~/.config.Code).
Open angular project (excluded node_modules for performance).
Worked the whole day without trouble.
Leaved editor open for next day, not suspending CPU.
Several files show as saved but, when trying to close them, received a message about "Save/Cancel/Don't save" before closing
Even after saving the file, the message still appears.
Leaving and reopening the editor (code .) solves the problem, no issues.

Reproduces without extensions: Yes
Addendum:

I know no other task modified those files,
verified against last saved version (git), before and after saving when at old VSCode instance (the one running from yesterday).
So, somehow, VSCODE did not recognize "saved status" for files, neither refresehed that status after saving.
I verified "Save" was working because my "ng serve" was recompiling every time I ordered to save those files.

Now, I remember I saw in Output window (Ctrl+J) lots of error messages mentioning errors and failures, referred to operations or actions I was unable to interpret, but I lost them. Sorry for this.
Thanks for such an amazing tool.
Eduardo J. Martínez
<EMAIL_ADDRESS>

Can you try to reproduce with our nightly insider builds? You can give our preview releases a try from: https://code.visualstudio.com/insiders/


Strange Reshape Symbol

Assume the input data(X) is image, let say 3 channel image, then one image's could be (3,32,32).
If I have 1234 images, then X has shape (1234,3,32,32).
If I apply mx.symbol.Reshape on X, what I hope to reshape is the shape of one image,i.e. shape (3,32,32).
However mx.symbol.Reshape seems to ask to provide the shape of the whole batch.
If I want to flat the image(Just for an example, thought I should use Flatten Symbol), I have to write something like
mx.symbol.Reshape(data=previous_symbol,target = (batch_size,3×32×32))
But batch_size is unknown and undecided.
Should Reshape function be changing one entry instead of the whole batch shape?

according to  Doc . U can set only the batch_size to zero and the Reshape can compute it given the remain 3 dimensions.
I hope this works for you if u are not trying to to resize your image.

@Godricly  Yes, you are right. I didn't notice that.


Escape sequences behave strangely over conpty when VirtualTerminalLevel is set

Environment
Windows build number: Version 10.0.18362.175
Windows Terminal version (if applicable): 0.2.1831.0

Steps to reproduce

Set the HKCU\Console\VirtualTerminalLevel registry entry to 1, to enable support for VT escape sequences by default.

Makes sure the initialCols option in the Windows Terminal profile is set to 120.

Open a cmd shell in the Windows Terminal (this can't be a WSL bash shell - it must be something that would usually not have VT support enabled without the registry setting).

Run the following Python script:
import sys
sys.stdout.write('X'*(120-10))
sys.stdout.write('\033[1;34;47mHello\033[m\n')

Expected behavior
I'd expect this to display 110 X's followed by the word Hello, in blue on white, on the same line. This is what it looks like in a regular console cmd shell.

Actual behavior
What actually happens in the Windows Terminal is the Hello is displayed on the following line.

I don't have a good understanding of how the conpty stuff works, but my theory is that the escape sequences aren't actually being processed initially, and are just written out to the screen as literal characters. As a result the Hello is forced to wrap onto the next line, because there isn't enough space remaining. This screen structure is then passed through conpty somehow, at which point the escape sequences are processed and the text colors changed. But by that stage the wrapping has already taken effect, so the text is in the wrong place.
It's easier to see what's going on if you apply PR #1964 first. In that case, the ESC character gets translated to a ← when it isn't initially interpreted as an escape sequence. The other side of the conpty pipe then doesn't get a second chance to process those values as an escape sequence, so you just see the raw characters being written out.

This may be a good reason to reject PR #1964 for now, since it's clearly making the situation worse, but I think the real issue is in the conpty code, and if that were fixed then the PR might well be safe to merge.

No, this is actually great! We've had a number of reports come through that some vt things "act weird", when it's just because they're printing escape sequences with ENABLE_VIRTUAL_TERMINAL_PROCESSING turned off.
Bit more info in this thread:
https://github.com/microsoft/terminal/issues/1616#issuecomment-507443706
https://github.com/microsoft/terminal/issues/1616#issuecomment-507453315
I had no idea that VirtualTerminalLevel was a thing. It almost certainly explains all of this behavior.
I've always thought that if we got a raw ESC for the buffer that we should store it as an inert rendering character. This is exactly what that does :grin:

FWIW, just to note, it is incorrect that Windows Terminal (or any conpty consumer) have a "second chance" to process the vt sequence. Since conhost is printing out what it thinks is in the screen buffer, double-processing some escape sequences can result in the cursor misaligning, vt sequences wrapping and being split into multiple lines, etc.

We took a fix in 19H1 to make conpty not load user settings, because that could impact how pseudoconsole hosts work in weird and difficult to debug ways. It looks like in doing so, we dropped VirtualTerminalLevel, which impacts how pseudoconsole hosts work in weird and difficult to debug ways.
Thank you.

I guess this is technically a duplicate then. I did wonder if it might have been related to some of the other weird behaviour bugs, but nobody else seemed to be mentioning VirtualTerminalLevel so I wasn't sure. I'm glad if it has helped clarify the issue, though.

We may want to just consider enabling VT processing by default for all conpty hosts.

Okay, so looks like we have options here:

Even though we're not loading the rest of the users's setting in conpty mode, still check for VirtualTerminalLevel

Pro: Terminal and Conhost will behave consistently when VirtualTerminalLevel is enabled
Con: unsure of how hard this might be
Con: Without setting VirtualTerminalLevel, ESC is still going to be rendered via VT/conpty, causing weird and difficult to debug differences between the terminal and conhost

Automatically enable ENABLE_VIRTUAL_TERMINAL_PROCESSING for conpty

pro: ESC will always be processedcaveat by conhost instead of just placed in the buffer to be rendered by conpty.
con: (caveat) What happens when an application wants to disable VT mode in conpty? We'll be right back in this scenario again, where the terminal has the state different from the conpty.
pro: We think that everyone should be doing VT by default, and this makes everyone do VT by default
con: It was not enabled by default for conhost for legacy reasons. Some legacy applications will behave strangely in conpty now.
(I feel like there are more pros here @DHowett-MSFT help me out)

Convert the ESC into some sort of inert character when rendering via VT

Pro: Terminal and Conhost look consistent
pro: console buffer remains unmodified, as we're only converting on write
Does nothing about the existing VirtualTerminalLevel problem - if it's set, then it won't be set in conpty, and the terminal and console will behave differently.

I'm sure I'm missing scenarios here, but I'm throwing this out for feedback. I don't think any of these changes are particularly complicated code changes, but there's not a clear solution to me.
/cc @miniksa

2+3 is the most correct, honestly. There's an entire range of "printable" control pictures, which provides a bunch of sweet iconography like ␛ and ␃.
We could do 2 today and punt 3 for windows 2xHy, honestly.

I mean, technically 1+3 is the most correct, but 2+3 is the most forward-looking...

If we did PR #1964, would that not address option 3 as well as the third con in option 1, at least for the DOS code pages?
As for enabling vt processing by default, I'd be in favour of doing that for conhost itself, which I assume would then make it the default for conpty as well if we went with option 1. If anyone doesn't want vt processing for some reason, they still can choose to disable it via the registry, and have that work consistently for both conhost and conpty.
That said, I don't know what the downsides are. I would have thought there were more legacy applications that needed some form of ANSI support (thus the popularity of alternatives like ANSICON). So I'm curious to know what kind of legacy applications would be broken by having the vt processing enabled.

It's been admittedly like 4 years since the issue came up originally, but when we first introduced VT support, we got some sort of major internal complaint that it was on by default, and it didn't break a tool, but it made is way slower. (since then, we did some optimization to the parser to make that not so much of a problem.) I honestly don't remember all the details, but the decision to change it to off by default was made, much to our dismay. I don't really want to try and fight that battle again.
However, I think that conpty gives us a fair opportunity to say that this is the new world, and as much as we'd love to provide 100% backwards compatibility with it, this is also the new way of doing things. There are other minor differences that running in a conpty session introduces, so I don't think this is terribly bad.
In conclusion, let's do 2+3. #1964 does something like what I was thinking for 3. I was picturing that we do it on the way out of conpty, in the vt renderer, but this seems reasonable. I'd want to compare how that behaves (esp. in regards to API buffer queries) compared to the v1 console, but that seems reasonable.
I'll use this issue to do "Conpty activates VT mode by default". Apps can still request VT off when they're in conpty mode, and if they write an ESC then, it'll still be rendered to the terminal correctly. There will be behavioral differences between conhost and conpty, but if something doesn't work in the terminal, it'll be unchanged for the legacy console, so use that. We sidestep the VirtualTerminalLevel problem, because it's just on by default in conpty.
@miniksa can @ me if this seems unreasonable or has any other arguments.


[ok.ru] Videos are corrupted, wrong mtime

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2023.01.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
EU
Provide a description that is worded well enough to be understood
yt-dlp corrupts files while downloading.
Here is the URL https://ok.ru/video/1269740472936
yt-dlp's result:
Size: 206353706 bytes (196 MiB)
SHA1: c5150e072871df0f9e2934f50bab4aa47ac6fbab
The file is broken. A video player lags, freezes while playing.
The correct file (downloaded with DevTools):
Size: 206343640 bytes (196 MiB)
SHA1: f55c505035f70cc3d45f6c1f76bf7965229f2ea5

Two more examples:
https://ok.ru/video/1240962828904
https://ok.ru/video/1240964074088

This one was downloaded correctly.
https://ok.ru/video/4217990679137
Possibly, because it have no video quality chooser:

While the other have it:

Also all video files have wrong mtime (2014.01.01).
While filename date key is correct (2019.05.19):
[ok.ru] A L E X—2019.05.19—1269740472936—Encounter a young wild white Arctic Fox in Greenland.mp4

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
$ yt https://ok.ru/video/1269740472936 -vU
[debug] Command-line config: ['--write-thumbnail', '--write-description', '-o', '[%(webpage_url_domain)s] %(uploader,creator,playlist_title)s—%(upload_date>%Y.%m.%d)s—%(id)s—%(full
title|)s.%(ext)s', '-o', 'pl_thumbnail:[%(webpage_url_domain)s][thumbnail] %(uploader,creator,playlist_title)s—%(upload_date>%Y.%m.%d)s—%(playlist_id,id)s—%(fulltitle|)s.%(ext)s',
'https://ok.ru/video/1269740472936', '-vU']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.06 [6becd2508]
[debug] Lazy loading extractors is disabled
[debug] Python 3.9.13 (CPython AMD64 64bit) - Windows-10 (OpenSSL 1.1.1n  15 Mar 2022)
[debug] exe versions: ffmpeg 5.0.1-full_build-www.gyan.dev (setts), ffprobe 5.0.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.12.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.1
[debug] Proxy map: {}
[debug] Loaded 1760 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.06, Current version: 2023.01.06
yt-dlp is up to date (2023.01.06)
[Odnoklassniki] Extracting URL: https://ok.ru/video/1269740472936
[Odnoklassniki]<PHONE_NUMBER>936: Downloading desktop webpage
[Odnoklassniki]<PHONE_NUMBER>936: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasa
ud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info]<PHONE_NUMBER>936: Downloading 1 format(s): mpd-7
[info] There's no video description to write
[info] Downloading video thumbnail 0 ...
[info] Writing video thumbnail 0 to: [ok.ru] A L E X—2019.05.19—1269740472936—Encounter a young wild white Arctic Fox in Greenland.jpg
[debug] Invoking http downloader on "https://vd288.mycdn.me/..."
[debug] File locking is not supported. Proceeding without locking
[download] Destination: [ok.ru] A L E X—2019.05.19—1269740472936—Encounter a young wild white Arctic Fox in Greenland.mp4

It correctly download the data from the logged URL, but the file from the selected URL are corrupted.
It should choose the other URL (which site's player uses).

Seems I get it.
You need just to parse a URL with ct=0 instead of ct=4.
There 8 URLs with ct=4 and 8 URLs with ct=0.
I think a video format from ct=0 URL would be the more correct one.

Also the videos are not corrupted, MPC-HC (K-Lite Codec Pack) seems just does not support iso5 (avc1/iso5/dsms/msix/dash) codec.
A ct=0 URLcontains a video with isom (isom/iso2/avc1/mp41)

I've had the same problem in the past extracting manually without the help of yt-dlp (about a year ago), to work around this, I just gave preference to the m3u8 protocol versions instead of https, that is, yt- dlp does not corrupt the files, manually downloading the DASH video format generates an identical hash.

Also the videos are not corrupted, seems MPC-HC (K-Lite Codec Pack) just does not support iso5 (avc1/iso5/dsms/msix/dash) codec correctly. VLC player can play them with no problem.

I can't reproduce this behavior, my MPC-HC played the iso5 brand files normally.

I have never have any issue with MPC-HC, but with ct=4 videos from ok.ru now.
Seems these videos are some special re-endoded for MSE standard.
Here is how it looks: https://uploadir.com/u/uq9uyla3

MediaInfo
General
Complete name               : M:\552983530088_iso5.mp4
Format                      : MPEG-4
Format profile              : Base Media
Codec ID                    : iso5 (avc1/iso5/dsms/msix/dash)
File size                   : 197 MiB
Duration                    : 1 min 7 s
Overall bit rate            : 24.5 Mb/s
Writing application         : Lavf58.20.100

Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile<EMAIL_ADDRESS>Format settings             : CABAC / 4 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 4 frames
Format settings, GOP        : M=1, N=30
Codec ID                    : avc1
Codec ID/Info               : Advanced Video Coding
Duration                    : 1 min 7 s
Bit rate                    : 24.4 Mb/s
Width                       : 3 840 pixels
Height                      : 2 160 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Variable
Frame rate                  : 29.970 (29970/1000) FPS
Minimum frame rate          : 12.529 FPS
Maximum frame rate          : 30.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.098
Stream size                 : 195 MiB (99%)
Writing library             : x264 core 152 r2851 ba24899
Encoding settings           : cabac=1 / ref=4 / deblock=1:2:2 / analyse=0x3:0x113 / me=hex / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=67 / lookahead_threads=16 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=2 / keyint=30 / keyint_min=3 / scenecut=0 / intra_refresh=0 / rc_lookahead=30 / rc=abr / mbtree=1 / bitrate=24414 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=0.91 / aq=1:1.00
Tagged date                 : UTC 2019-05-19 17:30:25
Codec configuration box     : avcC

Audio
ID                          : 2
Format                      : AAC LC
Format/Info                 : Advanced Audio Codec Low Complexity
Codec ID                    : mp4a-40-2
Duration                    : 1 min 7 s
Bit rate mode               : Constant
Bit rate                    : 256 kb/s
Channel(s)                  : 2 channels
Channel layout              : L R
Sampling rate               : 44.1 kHz
Frame rate                  : 43.066 FPS (1024 SPF)
Compression mode            : Lossy
Stream size                 : 2.05 MiB (1%)
Default                     : Yes
Alternate group             : 1
Tagged date                 : UTC 2019-05-19 17:30:25

From c=0 endpoint (this file is fine):

MediaInfo
General
Complete name               : M:\552983530088_isom.mp4
Format                      : MPEG-4
Format profile              : Base Media
Codec ID                    : isom (isom/iso2/avc1/mp41)
File size                   : 197 MiB
Duration                    : 1 min 7 s
Overall bit rate            : 24.5 Mb/s
Writing application         : Lavf58.20.100

Video
ID                          : 1
Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile<EMAIL_ADDRESS>Format settings             : CABAC / 4 Ref Frames
Format settings, CABAC      : Yes
Format settings, Reference  : 4 frames
Format settings, GOP        : M=1, N=30
Codec ID                    : avc1
Codec ID/Info               : Advanced Video Coding
Duration                    : 1 min 7 s
Duration_FirstFrame         : 46 ms
Bit rate                    : 24.4 Mb/s
Width                       : 3 840 pixels
Height                      : 2 160 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 30.000 FPS
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Progressive
Bits/(Pixel*Frame)          : 0.098
Stream size                 : 195 MiB (99%)
Writing library             : x264 core 152 r2851 ba24899
Encoding settings           : cabac=1 / ref=4 / deblock=1:2:2 / analyse=0x3:0x113 / me=hex / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=67 / lookahead_threads=16 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=2 / keyint=30 / keyint_min=3 / scenecut=0 / intra_refresh=0 / rc_lookahead=30 / rc=abr / mbtree=1 / bitrate=24414 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=0.91 / aq=1:1.00
Tagged date                 : UTC 2019-05-19 17:30:25
Codec configuration box     : avcC

Audio
ID                          : 2
Format                      : AAC LC
Format/Info                 : Advanced Audio Codec Low Complexity
Codec ID                    : mp4a-40-2
Duration                    : 1 min 7 s
Duration_LastFrame          : -4 ms
Bit rate mode               : Constant
Bit rate                    : 256 kb/s
Channel(s)                  : 2 channels
Channel layout              : L R
Sampling rate               : 44.1 kHz
Frame rate                  : 43.066 FPS (1024 SPF)
Compression mode            : Lossy
Stream size                 : 2.05 MiB (1%)
Default                     : Yes
Alternate group             : 1
Tagged date                 : UTC 2019-05-19 17:30:25

Maybe this -S "proto:m3u8_native" or -f "b[protocol^=m3u8]" will help in your case, have you tried it?

It sounds like yt-dlp is extracting both sets of formats and the "corrupt" downloads is player's issue. Am I mistaken?

Forgot to check -F.
ultra     mp4 unknown        │                   https │ unknown            unknown
hls-26493 mp4 3840x2160   30 │ ~216.68MiB 26493k m3u8  │ unknown     26493k unknown    0k
mpd-7     mp4 3840x2160   30 │ ~206.61MiB 25262k https │ avc1.640033 25262k mp4a.40.2  0k 44k DASH video, mp4_dash

Yes, it lists ct=0 as ultra, however, I get for it the 400 exception:

Trace
yt https://ok.ru/video/1269740472936 -f ultra -v

ERROR: unable to download video data: HTTP Error 400: Bad Request
Traceback (most recent call last):
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\YoutubeDL.py", line 3236, in process_info
    success, real_download = self.dl(temp_filename, info_dict)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\YoutubeDL.py", line 2959, in dl
    return fd.download(name, new_info, subtitle)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\downloader\common.py", line 444, in download
    ret = self.real_download(filename, info_dict)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\downloader\http.py", line 371, in real_download
    establish_connection()
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\downloader\http.py", line 129, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\site-packages\yt_dlp\YoutubeDL.py", line 3730, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 523, in open
    response = meth(req, response)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 632, in http_response
    response = self.parent.error(
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 561, in error
    return self._call_chain(*args)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 494, in _call_chain
    result = func(*args)
  File "C:\Users\User\AppData\Local\Programs\Python\Python39\lib\urllib\request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

But the download URL from the log works fine when I put it in the browser's address bar.

Also there is the second issue — the wrong file's mtime (2014.01.01)

yt https://ok.ru/video/1269740472936 -f mpd-7

work fine.
All 3 format are provide the same pictures (the screenshots of the same frame are byte equivalent), but different containers.
ultra is the best for players support. In participial, this format has nice iframe (keyframe) navigation with the player. But the 400 error.


Try to fix some La Palma files not being read.

Due to #142,  I had a quick look at the code and the dictionary keys 'camera_settings' is missing from telescope_description
telescope_description.keys() = dict_keys(['camera_organization', 'pixel_settings', 'disabled_pixels', 'camera_software_settings', 'drive_settings', 'pointing_correction'])

So i inspected a bit the code and found that it seems like in this function which is being looped:
https://github.com/cta-observatory/pyeventio/blob/6050c17c7e0feb8fddd38e2d8fc4f64e12eebadb/eventio/simtel/simtelfile.py#L91
there is one extra next(self) here:
https://github.com/cta-observatory/pyeventio/blob/6050c17c7e0feb8fddd38e2d8fc4f64e12eebadb/eventio/simtel/simtelfile.py#L112
which is preventing the 'CameraSettings' class from being copied to telescope_descriptions_types
I've commented this line and now I can read almost all the simtel files, apart from gamma_20deg_180deg_run100___cta-prod3-lapalma3-2147m-LaPalma.simtel.gz  which raises an error:
/home/thomas/Programs/anaconda3/envs/cta-dev/lib/python3.6/site-packages/astropy/units/quantity.py:639:
RuntimeWarning: invalid value encountered in sqrt
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)

I have no idea why this error is raised and if this is a good fix, but now I can read the simtel files again.
Feel free to change this PR...mine is just a temporary personal fix :sweat_smile:

Well spotted, could you just bump the version up in setup.py from 0.16.0 to 0.16.1 ?

RuntimeWarning: invalid value encountered in sqrt

I think that is just some negative value in an array where a sqrt is applied to.


Gsap animation

New animation for the pods that follows the path of the roads.
fixes #9 and #16
I removed the buzz animation for now.
closes #10 and #12

How does it close #10?  We won't do it for now, but it can be in the "after-dekrook" part right?


Sequencer with firebase function - recursion waiting for config sync

When using the firebase functions (not but not gcloud deploy mode) - the sequencer doesn't work.
Looks like a loop of updating config and waiting for the config sync:
22-04-27T21:47:31Z INFO sequencer Updated state has last_config 2022-04-27T21:47:19Z
2022-04-27T21:47:31Z INFO sequencer Updated config with timestamp 2022-04-27T21:47:19Z
2022-04-27T21:47:31Z INFO sequencer finished waiting for device config reset
2022-04-27T21:47:31Z INFO sequencer start waiting for device config reset
2022-04-27T21:47:31Z NOTICE sequencer Received serial no 1234
2022-04-27T21:47:31Z INFO sequencer Updated state has last_config 2022-04-27T21:47:19Z
2022-04-27T21:47:34Z INFO sequencer Updated config with timestamp 2022-04-27T21:47:31Z
2022-04-27T21:47:34Z INFO sequencer finished waiting for device config reset
2022-04-27T21:47:38Z INFO sequencer start waiting for device config sync
2022-04-27T21:47:38Z INFO sequencer Updated state has last_config 2022-04-27T21:47:19Z
2022-04-27T21:47:38Z INFO sequencer Updated state has last_config 2022-04-27T21:47:31Z
2022-04-27T21:47:38Z INFO sequencer Updated config with timestamp 2022-04-27T21:47:34Z
2022-04-27T21:47:38Z INFO sequencer Updated state has last_config 2022-04-27T21:47:34Z
2022-04-27T21:47:41Z INFO sequencer Updated state has last_config 2022-04-27T21:47:33Z
2022-04-27T21:47:41Z INFO sequencer Updated config with timestamp 2022-04-27T21:47:33Z
2022-04-27T21:47:43Z INFO sequencer Updated config with timestamp 2022-04-27T21:47:36Z
2022-04-27T21:47:43Z INFO sequencer Updated state has last_config 2022-04-27T21:47:36Z
2022-04-27T21:48:19Z NOTICE sequencer RESULT fail extra_config timeout waiting for device config sync

Considered obsolete as Firebase support is deprecated.


Helpful tips

[Enter feedback here]
How about letting us know how to tell which version we are using?
I had a partner set it up for me and would like to check which version before I call them.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 09eb3bfb-5bcc-3fb1-b36b-6222fe1f4e51
Version Independent ID: 9772c16b-8364-3ba2-a8b5-2f31a2abef5d
Content: Decommissioning Azure AD Connect V1 - Microsoft Entra ID
Content Source: docs/identity/hybrid/decommission-connect-sync-v1.md
Service: active-directory
Sub-service: hybrid
GitHub Login: @billmath
Microsoft Alias: billmath

@StreichersBC
Thanks for your feedback! We will investigate and update as appropriate.

The following document tells you how to check your version.
https://learn.microsoft.com/en-us/entra/identity/hybrid/verify-sync-tool-version
Thanks!
#please-close


Add a Printable trait that uses a State for printing

This PR also gets rid of the AttachContext trait.
@greenhat This is a breaking change, pretty much invading into all Ops, Types etc. So I'm creating a PR to inform you before I merge it.
Here's an example of how this simplifies printing:

Thanks, I appreciate it! I'll rebase my PR ASAP after it's merged.

Thank you. I'm hoping that most of you need to do will look like this, which can be done with a few attempts of search-and-replace. Please me ping me if you need help or clarifications.


Rework Daemon / GUI

refactor: removed gui feature, fixed gtk startup time
removed switch_on_close cli flag
better separation between config and command
TODO: better no window selected, set active window on gui click
TODO: add config file for options for initial start
TODO: add --init option

new help:
A CLI/GUI that allows switching between windows in Hyprland

Usage: hyprswitch [OPTIONS] <COMMAND>

Commands:
  init    Initialize and run the Daemon
  gui     Stapropagate_version = truerts/Opens the GUI + sends the command to daemon of GUI is already opened
  simple  Switch without using the GUI / Daemon (switches directly)
  close   Close the GUI, executes the command to switch window
  help    Print this message or the help of the given subcommand(s)

Options:
  -d, --dry-run  Print the command that would be executed
  -v...          Increase the verbosity level (max: -vv)
  -h, --help     Print help
  -V, --version  Print version


provider 1.10.0 crashes if EDL's are used

Describe the bug
Plugin Crashes after upgrade to 1.10.0 if config contains EDL's
Expected behavior
plugin should not crash
Current behavior
plugin crashing while running terraform plan
Possible solution
not sure, works on environment without EDL's configured.
Steps to reproduce
set version of plugin to 1.10.0
run terraform plan
Screenshots
Stack trace from the terraform-provider-panos_v1.10.0 plugin:

panic: runtime error: index out of range [2] with length 2

goroutine 82 [running]:
github.com/terraform-providers/terraform-provider-panos/panos.parseEdlId(0xc00015e540, 0x1e, 0x139fb20, 0x0, 0x15, 0x153fc1a, 0xb, 0x139fb20)
github.com/terraform-providers/terraform-provider-panos/panos/edl.go:433 +0xd7
github.com/terraform-providers/terraform-provider-panos/panos.readEdl(0xc000beef50, 0x1530300, 0xc00000a1e0, 0xc000beef50, 0x0)
github.com/terraform-providers/terraform-provider-panos/panos/edl.go:203 +0xb7
github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc0002f9220, 0xc00090a000, 0x1530300, 0xc00000a1e0, 0xc0008f9178, 0x0, 0x0)
github.com/hashicorp/terraform-plugin-sdk@v1.17.2/helper/schema/resource.go:470 +0x12e
github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ReadResource(0xc000131528, 0x17723d0, 0xc000096c00, 0xc000715260, 0xc000131528, 0xc000096c00, 0xc000c52ba0)
github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/helper/plugin/grpc_provider.go:535 +0x3dd
github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ReadResource_Handler(0x14ea840, 0xc000131528, 0x17723d0, 0xc000096c00, 0xc000715200, 0x0, 0x17723d0, 0xc000096c00, 0xc000902000, 0x1a0)
github.com/hashicorp/terraform-plugin-sdk@v1.17.2/internal/tfplugin5/tfplugin5.pb.go:3269 +0x214
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0009921c0, 0x177b938, 0xc00031ad80, 0xc000ab8a00, 0xc0006fa510, 0x1e97af0, 0x0, 0x0, 0x0)
google.golang.org/grpc@v1.32.0/server.go:1194 +0x52b
google.golang.org/grpc.(*Server).handleStream(0xc0009921c0, 0x177b938, 0xc00031ad80, 0xc000ab8a00, 0x0)
google.golang.org/grpc@v1.32.0/server.go:1517 +0xd0c
google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc00055c540, 0xc0009921c0, 0x177b938, 0xc00031ad80, 0xc000ab8a00)
google.golang.org/grpc@v1.32.0/server.go:859 +0xab
created by google.golang.org/grpc.(*Server).serveStreams.func1
google.golang.org/grpc@v1.32.0/server.go:857 +0x1fd

Error: The terraform-provider-panos_v1.10.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Context
trying to update to latest version to take advantage of 10.1.x PanOS features
Your Environment
panos-provider - 1.10.0 ( coming from 1.8.3)
terraform version - 1.0.3

@shinmog
In our case we are experiencing the same error, when checking the EDL ID on 10.1.3 it contains only 1 :
In provider version 1.10.0 it is expecting 2 :
This is seen here in the following file for v1.10.0
https://github.com/PaloAltoNetworks/terraform-provider-panos/blob/2598357ab1327fb726c5c9de9f6660cdd47fdf32/panos/edl.go#L433
In the previous provider v1.8.3
https://github.com/PaloAltoNetworks/terraform-provider-panos/blob/f311e3bea717316599009f5708e865ec77b2358c/panos/resource_edl.go#L117
On PANOS 10.1.3 we had this working using v1.8.3 and terraform state show module.main.panos_edl.whitelists["ul-shared-dev-ip"] displays the ID with the 1 :
# module.main.panos_edl.whitelists["ul-shared-dev-ip"]:
resource "panos_edl" "whitelists" {
    description         = "list of ip egress for shared-dev"
    exceptions          = []
    id                  = "vsys1:whitelist-shared-dev-ip"
    name                = "whitelist-shared-dev-ip"
    repeat              = "every five minutes"
    repeat_day_of_month = 0
    source              = "http://XXXXXXX"
    type                = "ip"
    vsys                = "vsys1"
}

How do we convert this to the new format or fix this? We are putting the EDL onto a fresh instance running PANOS 10.1.5.h1


Send a good bye email

Lets leave with style.
Who ever shuts off the lights, please send an email on the mailing list.
AC:

Send email
Say good bye from System Team
Perhaps request Quotes from all of us

Good Bye to the Tractus-X Community,
The last day of the consortia phase of Catena-X has finally come.
For most of us from the System Team, that means we have to say good-bye. But we want to thank you all for being a part in our community.
It was a pleasure working with you all and supporting you in providing the testing and development infrastructure. For us it was also a really insightful journey and we learned a lot. Providing you with environments, supporting you with new tooling and guiding you on your way to open-source contributions was also new territory for us.
We hope you had fun doing open source and wish you all the best on your continuous efforts.
Sincerely Your System Team & FOSS Team
Sigi,
Sebastian,
Angelika,
Gabor,
Fabian,
Harry,
Carsten,
Tomasz


Publish APK at Github?

Would you consider publishing the .apk here at Github (preferably attaching them to their resp. releases/), so folks w/o Playstore could use the app? I'd then offer adding it to my F-Droid compatible repo and have it automatically updated there whenever a new release happens, so those folks would even get update notifications and could perform updates without much trouble :innocent:

Thanks for your interest in the app! I've published the APK for the latest release, v1.10.0, on the releases page.

Thanks a lot, Ben! As promised, find it here (and here).  If at any time you pick it up again and publish a new .apk (attached to the resp. releases/ tag), it should appear in my repo within ~24h automatically now.

@blinskey as you've picked up development again :partying_face: may I ask for the APKs once more?

Done!

Thanks! WONDERFUL!

If you want to get rid of that DEPENDENCY_INFO_BLOCK, that's easy:
android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains.

Thanks for the info. I'll keep that in mind if I release a new version in the future.


How should the minWidth for the crop tool work?

I have set a minWidth for the crop attribute. But then when I drag the cropper tool to make the width smaller, it shrinks to clearly below the minWidth I set.
Have a look here: https://codesandbox.io/s/react-image-crop-demo-with-classes-forked-qdebp
I have set a mins and maxs for the cropper yet it does not listen to those values and can still be any size.
Is there something I am missing?

reading old issue the minWidth and minHeigth are not props of crop, but should be props of ReactCrop itself

Ah yes true! i have been doing this wrong the whole time. thanks!


fix: use padding in place to icon to calculate maximum date input size

Defect: DE53025
This is a bizarre bug that despite my best efforts I haven't been able to uncover the underlying reason behind it. I'm calling it The Mysterious Case of the Missing Icon Width.
Background
The date input wants to size its width such that the largest possible date can always fit. Since this can vary based on date value and the locale, it renders some invisible "wide" versions of itself and uses those to set its width. Because the date input renders a calendar icon before the value, <d2l-icon> is included in the invisible versions as well.
In case the entire component is initially hidden (resulting in the invisible version having no width), it wires up a ResizeObserver to redo the calculation. It used to always use the ResizeObserver, but that code was simplified in #3419 which caused This Bug.
This Bug
In Very Specific Situations (which of course happen in legacy LMS pages), the width of the icon in that initial calculation is ignored. That causes the input width to be 40px (0.6rem margin + 0.8rem icon + 0.6rem margin) too small:

Why does it ignore the width of the icon? For the life of my, I do not know. The Very Specific Situations involve an inline <fieldset>, a label and a relative
 parent. The following HTML reproduces it:
<fieldset style="display: inline">
  Date

    <d2l-input-date-time label="Date" label-hidden></d2l-input-date-time>

</fieldset>

Any of the following fixes it:

Make the <fieldset> not inline
Remove the label
Make the
 not relatively positioned (or remove it entirely)

Seems like a very primitive Webkit bug, since only Firefox behaves correctly.
Solution
The solution shown here replaces the <d2l-icon> and its margins in the invisible versions with 2rem of padding.
I tried a bunch of other things like an inline SVG or an inline-block  sized the same as the icon and none of them worked.

Since you determined the minimum HTML to reproduce anyways, should we have a visual-diff test of it to catch a possible regression if we ever change the padding size?

Since you determined the minimum HTML to reproduce anyways, should we have a visual-diff test of it to catch a possible regression if we ever change the padding size?

Yeah good call -- added.


Changed LabelMapping to labels in funnel

Based on comments in this ticket in Groove: https://keen-io.groovehq.com/groove_client/mailboxes/4308/filters/51672/tickets/5120277

:+1: thanks!!


MAC address get concatenated in OSDs with vagrant-libvirt 0.0.35

I've tried setting up CEPH using ceph-ansible with vagrant-libvirt. However, I've got errors setting up OSDs about MAC address on the eth2 interface being different from what was expected. I've debugged a bit and found out that those MAC addresses are two MACs each, concatenated (so the error message is correct). The problem is that vagrant-libvirt 0.0.35 uses an XPath expression to query the MAC from libvirt's XML: "/domain/devices/interface[source[@network='#{@network_name}']]/mac/@address" (change came in in commit https://github.com/vagrant-libvirt/vagrant-libvirt/commit/ee7afab369dc7e80128ff35a2ff11f6eab23d368). However, this is how the XML for an OSD looks like:
    <interface type='network'>
      <mac address='52:54:00:c9:8b:13'/>
      <source network='ceph-ansible0' bridge='virbr2'/>
      <target dev='vnet3'/>
      <model type='virtio'/>
      <alias name='net1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:7a:9b:4c'/>
      <source network='ceph-ansible0' bridge='virbr2'/>
      <target dev='vnet4'/>
      <model type='virtio'/>
      <alias name='net2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>

As you can see, both interfaces (eth1 and eth2) have the same source network name, so that the XPath expression actually returns two addresses, which is not caught by vagrant-libvirt (so a bug report is due there as well).
So the question is basically (apart from vagrant-libvirt's parsing): is this (probably default) network naming correct? If not, could this be changed?

I personally don't use vagrant-libvirt and I'm not sure why both interfaces have the same network name. If you can find the appropriate syntax to change the network name then I'm fine putting a change in the Vagrantfile.

Perfect timing as I had just filed https://github.com/vagrant-libvirt/vagrant-libvirt/issues/645 because I noticed the same error where eth2 for OSDs end up with a DHCP address instead of the statically defined one XXX.XXX.XXX.20X. Defining a different network name for each private network interface on the OSDs could probably be a workaround. Otherwise they just fallback to DHCP.


Long polling isn't cleaned up when exiting browsers

So I have this ExpressJS server that I'm using your awesome library to help manage PhantomJS.
This server maintains a pool of browsers for each request to come in and utilitize to visit URLs and extract data about the pages when they load.  Every so often, depending on the amount of times a given browser in the "pool" is used, we kill that browser, remove it from the pool and replace it w/ a new one.
This has all been working swimmingly (pun intended) up until I started throwing thousands of requests at it.  Over time (after around 15K requests or so) I noticed that the CPU would end up just pegging at 100% on the server.  I ended up narrowing it down to the long polling that node-phantom-simple(NPS) is using to perform IPC.
So I went ahead and created a simple pull request that essentially kills the long poll for a given browser if and when it's explicitly exited from: https://github.com/baudehlo/node-phantom-simple/pull/68
I'm sure there's a lot better ways to approach this (utilize phantom.killed?) but I needed something quick and dirty.. we have this running in production and I've now thrown 2X anything that I've thrown at it in the past and the CPU usages always come back down to normal after processing completes.

One of the users of node-horseman, which depends on node-phantom-simple, ran into this issue: https://github.com/johntitus/node-horseman/issues/39
It appears to be based on this same issue, and doesn't require "thousands of requests".  Here's a very simple node-phantom-simple test that recreates it:
var driver = require('node-phantom-simple');

driver.create(function (err, browser) {
	return browser.createPage(function (err, page) {
		return page.open("http://tilomitra.com/repository/screenscrape/ajax.html", function (err,status) {
			console.log("opened site? ", status);
			browser.exit();
		});
	});
});

The fix provided by @RavenHursT fixed the issue, both in the test script provided above and in node-horseman.

Is fix ok? I will release then

Yep, @Kirill89's fix resolves the issue in node-horseman.

Released.
Enjoy :) !


Update peerDependencies to support Grunt 1.0

Update peerDependencies to support Grunt 1.0
Hello,
This is an automated issue request to update the peerDependencies for your Grunt plugin.
We ask you to merge this and publish a new release on npm to get your plugin working in Grunt 1.0!
Read more here: http://gruntjs.com/blog/2016-02-11-grunt-1.0.0-rc1-released#peer-dependencies
Also on Twitter: https://twitter.com/gruntjs/status/700819604155707392
If you have any questions or you no longer have time to maintain this plugin, then please let us know in this thread.
Thanks for creating and working on this plugin!
(P.S. Close this PR if it is a mistake, sorry)

please merge this pull request so that this can be used again

Seems reasonable. You're pretty cool for a 🤖, @grunt-updater2!

Thanks @rtfeldman


[BUG] Unable to open Quest Log.

Module Version: v0.0.0
Describe the bug
I am unable to open quest log - I have tried to disable it and re-enabling it. I've tried uninstalling and re-installing. The quest log opens in other games with the same modules enabled. I'm at a bit of a loss why I specifically can't open it. My players in the game can open it, but I can not.
Expected behavior
I would expect the quest log to open upon clicking the Quest Log button.
Browser:

Google Chrome

Foundry Version: 0.7.7
Game System: DnD5e

Did you ever find a solution for this? I'm experiencing it after the latest update.

Did you ever find a solution for this? I'm experiencing it after the latest update.

Hi,
Had the same problem, module won't open on one game because of a corrupted quest.
I disabled module, then go to your note/journal tab and delete quests and enable module. (the quest files won't show if module is active for me)
Hopefully I knew which quest was bugged but if you do not I'm afraid you ll have to loose some stuff.
Hope it solve yours prob


Hubot is silent locally after initial install.

I ran through the Yeoman generator as suggested here using the Flowdock adapter.
After the initial install I run hubot from the bin directory and get no responses to commands like help.
As you can see in the image below, hubot echos "hubot help" on the first occurrence, then nothing after that.
I don't even know where to begin looking for the problem here.  It seems to make it through the whole process fine and "configCheck" returns "OK".

That doesn't sound right! I'll try to repro...

I completely forgot one (probably important) thing.
As you can see I ran the Yeoman generator under "C:\code\cr-hubot".
Initially "cr-hubot/bin/hubot.cmd" had this line:
npm install && node_modules.bin\hubot.cmd --name "hubot" %*
On running hubot the very first time, this errored out with "The system cannot find the path specified."
I changed it to:
npm install && ..\node_modules.bin\hubot.cmd --name "hubot" %*
To get it to actually run.  Which is when it does the originally reported problem.  It seems like it expects the node_modules to be installed directly under the /bin/ folder.
Other than that I don't believe I made any changes.

I am seeing this as well. Node 4.2.1 + npm 3.3.9
It was a working hubot that I just popped open to write some new scripts for

Haven't had time to look at this :sob:
I wonder if it has something to do with updated versions of node/npm?

I'm not crazy familiar with hubot (yet) but I know NPM3 is a flat structure of node_modules. Could that mess anything up internally?

Also, just updated my package.json to make sure I had latest hubot core stuff and saw this:
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├─┬<EMAIL_ADDRESS>│ ├─┬<EMAIL_ADDRESS>│ │ ├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>│ │ └──<EMAIL_ADDRESS>│ └─┬<EMAIL_ADDRESS>│   └──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
├──<EMAIL_ADDRESS>extraneous
└──<EMAIL_ADDRESS>extraneous

npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.```

NPM3 changed peerdependencies, could that be an issue? (or maybe I corrupted something in my package)

Both of those could easily be causing problems. Hubot does some funky stuff where the core (a module) requires things in the parent's (bot's) node_modules directory (adapter, script modules).

Erased the peer thing because I'm pretty sure I added mongoose to the project, not you guys.

I tried with npm 2.14.10 (to get rid of the flat node_modules structure) and strangely see the same empty command list behavior.

Hi, I have same issue. Cannot perform clean install of my hubot.
npm --version
3.4.1

node --version
v5.0.0

Is there some workaround?

The same here.
After installation via "yo", hubot is not working.
wojtas@ubuntu:~/hubot3$ yo hubot
_____________________________
/                             \
//\              |      Extracting input for    |
////\    _____    |   self-replication process   |
//////\  /\   \                             /
======= |[^/_]|   /----------------------------
|   | |@@|
+===+/  ///     _\
| |\ /// HUBOT/\
|///      /  \
\      /   +---+
__/    |   |
| //|    +===+
//      |xx|
? Owner User<EMAIL_ADDRESS>? Bot name hubot
? Description A simple helpful robot for your Company
? Bot adapter (campfire) shellgot back true
? Bot adapter campfire
create bin/hubot
create bin/hubot.cmd
create Procfile
create README.md
create external-scripts.json
create hubot-scripts.json
create .gitignore
create package.json
create scripts/example.coffee
create .editorconfig
_____________________________
_____              /                             \
\    \             |   Self-replication process   |
|    |    _____    |          complete...         |
|\|   /\   \     Good luck with that.    /
|//+  |[^/_]|   /----------------------------
|   | |@@|__
+===+/  ///     _\
| |\ /// HUBOT/\
|///      /  \
\      /   +---+
__/    |   |
| //|    +===+
//      |xx|
npm ERR! Linux 3.19.0-25-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "hubot" "hubot-scripts" "hubot-diagnostics" "hubot-help" "hubot-heroku-keepalive" "hubot-google-images" "hubot-google-translate" "hubot-pugme" "hubot-maps" "hubot-redis-brain" "hubot-rules" "hubot-shipit" "--save"
npm ERR! node v5.0.0
npm ERR! npm  v3.4.0
npm ERR! code ENOSELF
npm ERR! Refusing to install hubot as a dependency of itself
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /home/wojtas/hubot3/npm-debug.log
npm-debug.log.txt

@wojtas911 you're getting this error because your package.json name is also hubot, you should change it to some other value.

In other words, when yo hubot asks you what you want to name your bot do not choose hubot choose another name.

For anyone who found this via Google and is having the same issue (like I did two hours ago), I figured this out.  Rather than navigating into bin, and then running hubot, you HAVE to run it from the parent folder using

bin\hubot

This way, you don't have to edit the bat file in the bin folder, and hubot starts responding as normal.

Same issue here, tested on separate Linux machines using node stable (6.2.0) as well as LTS (4.4.4). Redis is working fine, getting "Data for hubot brain retrieved from Redis".

@loleg I just want to say, you have made me feel quite stupid. after coming across this thread and seeing your comment, that's exactly what I was doing.
Thank you.

If it makes you feel any better, you're definitely not alone in the
confusion. The docs should be clearer on this point, it could be echoed on
startup, etc.

Thansk very much. Had the same "issue"

Looks like @RyFisher solved this. Let me know if it's still an issue for anyone.


Better token consumption estimation

For our client side rate limit control, we need to estimate the number of tokens that will be used by each request we send to an LLM completions API.
Right now we are naively estimating that the output of the call will use (max_output_tokens // 4).
Instead we should implement a moving average based on the responses we have gotten so far.
This will help with leaving extra throughput on the table as discovered when looking into #223.

Closed as duplicate to #206


Failed to get a signed key from the CA: ... received error response from keybase api: DB error (error 2623)

I have (following the instructions at https://keybase-ssh-ca-bot.readthedocs.io/en/latest/getting_started.html) configured the Keybase SSH CA bot (using Docker), and have configured a second 'target server' with the CA etc.
When I try to connect to  the 'target server' using kssh, I get the message
Failed to get a signed key from the CA: failed to get config: Failed to load config(s): received error response from keybase api: DB error (error 2623)
I can see in my .ssh folder there are the files keybase-signed-key-- and keybase-signed-key--.pub, but if I try to use keybase-signed-key-- or keybase-signed-key--.pub to SSH to the server directly I get Permission denied (publickey).
I am able to ping @mybotname in the relevant channel for SSH access, and get back a pong @mybotname as I would expect.
Looking at other issues (e.g. https://github.com/keybase/bot-sshca/issues/64) it looks like others see some kind of "Keybase SSH Provision chat in my team" - I see nothing like that in the channel at all. However, if I manually post a Signature_Request:{"ssh_public_key":"xxxxxxxx", ,"uuid":"xxxxxxxxx-xxxxx-xxxx-xxxxxx-xxxxxxxxxxx"}, I get back a Signature_Response:{"signed_key":xxxx" in the channel, so it looks like the bot itself is actually working.
(Note, for the UUID I literally used xxxxxxxxx-xxxxx-xxxx-xxxxxx-xxxxxxxxxxx in case that matters.
So it kind of looks like kssh is failing to post the relevant messages via Keybase? I've tried this on two machines with two keybase users and two different OSs and had the same error.
Extra details (copied from a post I made on Reddit)
I also described this in a reddit post: https://www.reddit.com/r/Keybase/comments/gubooe/keybase_ssh_ca_anyone_got_it_working_received/
I've been trying to get the Keybase teams-based SSH CA working (described https://keybase.io/blog/keybase-ssh-ca) with no success.
I've done all the set-up steps, but when I actually try to use kssh to get to the destination machine (the one set up with the CA, not the one with the bot) I always get the error:
Failed to get a signed key from the CA: failed to get config: Failed to load config(s): received error response from keybase api: DB error (error 2623)
I followed the instructions here: https://keybase-ssh-ca-bot.readthedocs.io/en/latest/getting_started.html
So, I have:

A machine running the bot (Set up using the paper key, and using docker, as described) with a specific bot user (I'll call it @mybot)
A destination machine I want to manage SSH permissions on (with the ca.pub file and /etc/ssh/auth_principals/ files containing the team names, and the TrustedUserCAKeys and AuthorizedPrincipalsFile in the sshd_config as per instructions

Note that I added the bot as a normal user in the channel, not by installing it as a bot. I've tried having it installed as a bot, and also as full user and neither worked.
For reference, the instructions don't specify whether it should be installed as a bot or added as a user (or I don't find it clear, anyway):

Then create {TEAM}.ssh.staging, {TEAM}.ssh.production, {TEAM}.ssh.root_everywhere as new Keybase subteams and add the bot to those subteams. Add users to those subteams based off of the permissions you wish to grant different users

Note that I pulled down the repo using HTTPS rather than SSH as I didn't have SSH keys set up on the server - using the url git clone https://github.com/keybase/bot-sshca.git
I have added the bot to the relevant channels, and verified that I can ping it - i.e. if I ping @mybot then I get pong @myuser. There is nothing in the logs on docker that would make me think it isn't behaving correctly.
2020/06/01 01:24:57 - Subscription: Read -> ok [time=21m1.759092887s]
2020/06/01 01:24:58 + Subscription: Read
2020/06/01 01:24:58 - Subscription: Read -> ok [time=4.447664ms]
2020/06/01 01:24:58 + Subscription: Read

I've tried this using both a Linux client and a Mac client trying to use kssh (although in both cases with the same user). Does anyone have any suggestions as to what to try next? (I haven't opened a github issue or pinged dworken as suggested at the end of the troubleshooting guide - though I'd try the community before bugging them there).

Hi!
This error message:

received error response from keybase api: DB error (error 2623)

might suggest that the issue is on our end. Can you send keybase logs (keybase log send command) from the device that you are trying the kssh command which fails with this error?

Done: log ID is cbad4b1b47756e3f11f2dd1c

Looks like a bug in kssh. As a workaround, can try try to create kssh-config.json file similar to this one:
~/.ssh » cat ~/.ssh/kssh-config.json
{
	"default_bot": "test_bot",
	"default_team": "test_team"
}

pointing kssh to specific team and specific bot that would be providing key signature?

When I added the bot and team to the kssh file, it all worked.
I'm going to go through the set-up documentation again, because I'm now thinking that may not have named something correctly in order for kssh to be able to find the right team. (It's not fully clear to me how kssh works out what team it needs to be requesting the CA signing for, so I'm going to try to get my head around that too).

Thank you @wsot for your report! We're working on a fix at https://github.com/keybase/bot-sshca/pull/101.

But then when I as a user attempt to use kssh to access a particular server, how does my local machine know what channel it should be posting in to request the correct access?

To explain this - when no default bot/team is specified, kssh looks for a kssh config entry (https://github.com/keybase/bot-sshca/blob/master/src/kssh/config.go#L13) that your Keybase user knows about by iterating through the KV stores for every team you are a member of. This all-teams search (https://github.com/keybase/bot-sshca/blob/master/src/kssh/requester.go#L224) is bypassed if a default bot/team is specified. (Our WIP fix modifies the all-team search to handle the case where a user is a member of a team, but does not have permissions to read the KV store, which seems to be the cause of the error.)

@wsot I just merged a fix, would you mind trying that again? Thanks for the report!

Works fine with the fix, thanks. (Having some timeout issues but they are unrelated so will chase them up elsewhere).


Generating 2d noise.

The documentation only shows how to generate 3d Simplex Fractals. How do the other noise types work in 2d?

FastNoiseSIMD doesn't have an 2D noise, it's all 3D.

Is there an efficient way of getting a slice? Or should I just do a 3d with 8 depth.

You can just swap axis and make x or y a size 1

Will that make it slow? It says in the documentation that it should be a multiple of 8 for optimal speed.

Only the Z axis needs to be a multiple of 8

Okay thank you!

If I want a 2D noise, would it be better to use FastNoiseSIMD and get a slice, or use "FastNoise" (the non-SIMD-version) instead?

https://github.com/Auburns/FastNoise#performance-comparisons


yubikey-manager: 4.0.3 -> 4.0.5

Motivation for this change
I also decided to grab the source from the github repository at https://github.com/Yubico/yubikey-manager, since this more easily lets users override the derivation to use their own versions or bump before things are moved up in nixpkgs. @benley @Mic92 Is there a reason were were pulling the source from https://developers.yubico.com/ instead? Is that preferred?
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 132177 at f730b8ed run on x86_64-linux 1

4 packages failed to build:

gen-oath-safe
yubikey-manager
yubikey-manager-qt
yubioath-desktop

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

4 packages failed to build:

Somehow I got the sha256 wrong, now fixed.


OpenTelemetry configuration using appsettings.json

How would you set this Sink up using the Serilog settings provider for Microsoft.Extensions.Configuration?
 "WriteTo": [
      { "Name": "OpenTelemetry",  "Args": {
          "Endpoint" : "http://<IP_ADDRESS>:4317", "ResourceAttributes" : ["??"]}
      }
    ],

In general, SO is the place to go for questions like this - the config syntax is orthogonal to the sink so finding config people and/or intersection of these watching this repo is lower (but post a link to it here...)

Hi @cecilphillip 👋
I don't think the JSON configuration syntax has support for dictionary values - I could be wrong, if I am it'll look like:
 "WriteTo": [
      { "Name": "OpenTelemetry",  "Args": {
          "Endpoint" : "http://<IP_ADDRESS>:4317", "ResourceAttributes" : {"service.name": "Example", "other": "Example2"}
      }
    ],

(Would be great if we could support this; Serilog.Settings.Configuration would be the place to look.)
HTH!

@nblumhardt Thanks boss

Raised again in #103; I'll move it to serilog-settings-configuration :-)

@nblumhardt  I'll take some solace in knowing that I wasn't the only one 🙂


User invalid email bug fixed

Earlier, if the user tried to create an account with email 'a', they would get the message User with email 'a' has been created. Even though @aidanmrli wrote code on the backend to reject such requests, it was not reflected on the client side.
Now, the user will get 'invalid email' if they try to create an account with emails like 'a'.

This seems to accept every email that is of type<EMAIL_ADDRESS>where len(a)>=1, len(b)>=1, and len(c)>=2. I think this is okay because we can't confirm whether the email really exists without sending them a confirmation email. This code seems to solve the issue it wanted to solve, so I'm merging it.
@TODO: create Jest tests to test invalid emails.