Attachments and Body Resources

See https://github.com/luckyframework/carbon/pull/88

Does this require an update to Carbon?

yes, since you have to be able to set those attachments in your email classes.

ok, bump Carbon to v0.5.0 and lets re-run the CI.


Tower Defense version of simply.js needing to get merged

Merged request into 1.0.2 release. might break some compatibility.


stacked 'success' method wrapping in fetch?

fetch : function(options) {
  options || (options = {});
  var model = this;
  var success = options.success;
  options.success = function(resp, status, xhr) {
    if (!model.set(model.parse(resp, xhr), options)) return false;
    if (success) success(model, resp);
  };
  options.error = wrapError(options.error, model, options);
  return (this.sync || Backbone.sync).call(this, 'read', this, options);
},

It looks like multiple calls to that method make success to be called many times at once.

No.  Each method is only called once.
Here you can see a reference to the success function that you pass as an option to fetch is being saved to a variable called success.  And that the original success property you provided is being overwritten with a new wrapped function.
  var success = options.success;
  options.success = function(resp, status, xhr) {

options.success is called when the request completes successfully from whatever DOM/AJAX library you're using finishes.
If the success function we originally passed in, truly exists, (meaning you didn't do fetch({ success: null })) it will then call the success function you originally passed with a normalized signature.
    if (success) success(model, resp);

This is an incredibly powerful and smart way of handling a normalized success return with varying different AJAX libraries.  If you have more questions feel free to continue commenting here.

In your implementation you modify options data (success field). If I call fetch method with the same options object several times success method in options will be called several times per each fetch because of increasing nesting. Actually it is not good idea to modify options in methods. They should always contain original data. Here in fetch method you should either restore original success method:
if (success) {
options.success = success;
success(model, resp);
}
or use another design pattern to not modify options at all.

You are absolutely right.  Will investigate what we can do to prevent this.  Perhaps, extend?

Yes, I think extend should work, for example (not tested):
fetch : function(options) {
  options || (options = {});
  var model = this;
  var opts = _.extend({}, options, {
    error: wrapError(options.error, model, options),
    success: function(resp, status, xhr) {
      if (!model.set(model.parse(resp, xhr), options)) return false;
      if(options.success) options.success(model, resp);
  });
  return (this.sync || Backbone.sync).call(this, 'read', this, opts);
}

This would need to happen in more than just fetch though.  It is also prevalent in save.

Yes, it should be fixed there as well.

I still need to add in some tests, facing some difficulty with the way tests are currently constructed.


FinTSDialogInitError

Thank you for creating this software
I tried to use it with
endpoint: https://hbci.gad.de
and added blz, account and password accordingly using credentials that work with  banking4, moneymoney, hibiscus and VRNetWorld.
With this fints2ledger i get:
fints.exceptions.FinTSDialogInitError: Couldn't establish dialog with bank, Authentication data wrong?
So i assume the authentication works differently than I assume. Which methods for authentication are available and how are these configured?

For fints2ledger there is just one authentication method available.
Can you run the following snipped (with your details filled in and no </>) and check if you get an error or a list of accounts?
from fints.client import FinTS3PinTanClient

client = FinTS3PinTanClient(
            "<your banks blz>",
            "<your account number>",
            "<your online password>",
            "https://hbci.gad.de/"
        )

print(client.get_sepa_accounts())

(You may need to pip3 install fints if the import cannot be found.)
Basically fints2ledger uses the pyfints  library under the hood. If you can get a connection with pyfints then it's possible to make it work here too.
pyfints has some good documentation, which should help investigating where the issue comes from.


Update install commands (cli/typescript)

What does this PR do?

Updates the install command for the cli in the getting started guide
Updates the init command with typescript to match @react-native-community/cli

Issues
Closes https://github.com/react-native-community/react-native-template-typescript/issues/80
Sources
react-native-template-typescript/README.md

duplicate of #1185


Vals Sops: Error getting data key: 0 successful groups required, got 0

Operating system
Mac / Linux
Helmfile Version
0.166.0
Helm Version
3.15.2
Bug description
After updating from 0.165.0 which works fine, loading secrets from sops via val doesnt work anymore.
Example helmfile.yaml
environments:
  default:
    values:
      - sopsfile: "ref+sops://{{ requiredEnv "SOPS_FILE" }}?format=yaml"
      - secrets:
          ARTIFACTORY_TOKEN: '{{ .Values.sopsfile  }}/#ARTIFACTORY_TOKEN'
---
releases:
  -
    name: service1
    namespace: service1
    chart: some_repo/service1
    version: {{ requiredEnv "HELM_CHART_VERSION" }}
    createNamespace: true
    missingFileHandler: Info
    values:
      - ./config/default-values.yaml
      - ./config/values.{{ requiredEnv "K8SNAME" }}.yaml.gotmpl

repositories:
  - name: stpzvirtual
    url: https://artifactory.example.com:443/artifactory/some_repo
    username: some_username
    password:  {{ .Environment.Values.secrets.ARTIFACTORY_TOKEN | fetchSecretValue}}

helmDefaults:
  kubeContext: {{ requiredEnv "K8SNAME" }}
  devel: true

Error message you've seen (if any)
in ./helmfile.yaml: error during helmfile.yaml.part.1 parsing: template: stringTemplate:17:66: executing "stringTemplate" at : error calling fetchSecretValue: expand sops:///common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN: Error getting data key: 0 successful groups required, got 0
Steps to reproduce
run helmfile write-values
Working Helmfile Version
0.166.0
Relevant discussion
No response

@dl-mai need more logs by add --debug flag. thanks so much.

Hey we have the same issue and I did just apply the debug flag. There wasn't really more information on the error:
err: release "service" in "helmfile.yaml" failed: failed processing release service: failed to render values files "service.yaml": expand sops://secrets/service-secret.json?format=json: Error getting data key: 0 successful groups required, got 0
in helmfile.yaml: failed processing release service: failed to render values files "service.yaml": expand sops://secrets/service-secret.json?format=json: Error getting data key: 0 successful groups required, got 0

The SOPS file is encrypted by AWS KMS and the problem only occurs with Helmfile 0.166.0.
I omitted the rendered values since they contain secrets from AWS Secretsmanager which can still be decrypted.

Sorry for the delay. Added logs with debug flag:
processing file "helmfile.yaml" in directory "."
changing working directory to "/Users/johndoe/development/my_project/k8s-cluster/my_cluster/my_service"
first-pass rendering starting for "helmfile.yaml.part.0": inherited=&{default  map[] map[]}, overrode=<nil>
first-pass uses: &{default  map[] map[]}
first-pass rendering output of "helmfile.yaml.part.0":
 0: environments:
 1:   default:
 2:     values:
 3:       - sopsfile: "ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml"
 4:       - secrets:
 5:           ARTIFACTORY_TOKEN_STAGE: '<no value>/#ARTIFACTORY_TOKEN_STAGE'

replaced <no value>s to workaround https://github.com/golang/go/issues/24963 to address https://github.com/roboll/helmfile/issues/553:
  strings.Join({
  	... // 81 identical bytes
  	"mai/development/my_project/shared-values/common-secrets.yaml?fo",
  	"rmat=yaml\"\n      - secrets:\n          ARTIFACTORY_TOKEN_STAGE: '",
- 	"<no value>",
  	"/#ARTIFACTORY_TOKEN_STAGE'",
  }, "")

first-pass produced: &{default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
first-pass rendering result of "helmfile.yaml.part.0": {default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
vals:
map[secrets:map[ARTIFACTORY_TOKEN_STAGE:/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.0":
 0: environments:
 1:   default:
 2:     values:
 3:       - sopsfile: "ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml"
 4:       - secrets:
 5:           ARTIFACTORY_TOKEN_STAGE: 'ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE'

merged environment: &{default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
first-pass rendering starting for "helmfile.yaml.part.1": inherited=&{default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}, overrode=<nil>
first-pass uses: &{default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
first-pass rendering input of "helmfile.yaml.part.1":
 0: releases:
 1:   -
 2:     name: my_service
 3:     namespace: my_service
 4:     chart: my_helm_repo/my_service
 5:     version: {{ requiredEnv "HELM_CHART_VERSION" }}
 6:     createNamespace: true
 7:     missingFileHandler: Info
 8:     values:
 9:       - ./config/default-values.yaml
10:       - ./config/values.{{ requiredEnv "K8SNAME" }}.yaml.gotmpl
11:
12: repositories:
13:   - name: my_helm_repo
14:     url: https://bahnhub.tech.rz.db.de:443/artifactory/stpz-helm-virtual
15:     username: techuser-stage-stpz
16:     password:  {{ .Environment.Values.secrets.ARTIFACTORY_TOKEN_STAGE | fetchSecretValue}}
17:
18:
19: helmDefaults:
20:   kubeContext: {{ requiredEnv "K8SNAME" }}
21:   devel: true
22:

template syntax error: template: stringTemplate:17:72: executing "stringTemplate" at <fetchSecretValue>: error calling fetchSecretValue: expand sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE: Error getting data key: 0 successful groups required, got 0
first-pass rendering output of "helmfile.yaml.part.1":
 0: releases:
 1:   -
 2:     name: my_service
 3:     namespace: my_service
 4:     chart: my_helm_repo/my_service
 5:     version: 0.46.7
 6:     createNamespace: true
 7:     missingFileHandler: Info
 8:     values:
 9:       - ./config/default-values.yaml
10:       - ./config/values.my_cluster-tsta.yaml.gotmpl
11:
12: repositories:
13:   - name: my_helm_repo
14:     url: https://bahnhub.tech.rz.db.de:443/artifactory/stpz-helm-virtual
15:     username: techuser-stage-stpz
16:     password:

first-pass produced: &{default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
first-pass rendering result of "helmfile.yaml.part.1": {default  map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml] map[]}
vals:
map[secrets:map[ARTIFACTORY_TOKEN_STAGE:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE] sopsfile:ref+sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml]
defaultVals:[]
second-pass rendering failed, input of "helmfile.yaml.part.1":
 0: releases:
 1:   -
 2:     name: my_service
 3:     namespace: my_service
 4:     chart: my_helm_repo/my_service
 5:     version: {{ requiredEnv "HELM_CHART_VERSION" }}
 6:     createNamespace: true
 7:     missingFileHandler: Info
 8:     values:
 9:       - ./config/default-values.yaml
10:       - ./config/values.{{ requiredEnv "K8SNAME" }}.yaml.gotmpl
11:
12: repositories:
13:   - name: my_helm_repo
14:     url: https://bahnhub.tech.rz.db.de:443/artifactory/stpz-helm-virtual
15:     username: techuser-stage-stpz
16:     password:  {{ .Environment.Values.secrets.ARTIFACTORY_TOKEN_STAGE | fetchSecretValue}}
17:
18:
19: helmDefaults:
20:   kubeContext: {{ requiredEnv "K8SNAME" }}
21:   devel: true
22:

err: error during helmfile.yaml.part.1 parsing: template: stringTemplate:17:72: executing "stringTemplate" at <fetchSecretValue>: error calling fetchSecretValue: expand sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE: Error getting data key: 0 successful groups required, got 0
changing working directory back to "/Users/johndoe/development/my_project/k8s-cluster/my_cluster/my_service"
in ./helmfile.yaml: error during helmfile.yaml.part.1 parsing: template: stringTemplate:17:72: executing "stringTemplate" at <fetchSecretValue>: error calling fetchSecretValue: expand sops:///Users/johndoe/development/my_project/shared-values/common-secrets.yaml?format=yaml/#ARTIFACTORY_TOKEN_STAGE: Error getting data key: 0 successful groups required, got 0

@patrickdomnick I will fix this.

we are using https://github.com/getsops/sops/releases/tag/v3.9.0, maybe we should see the new sops docs.

@patrickdomnick @dl-mai  could you show your sops file content? so I can debug it and find the reason.

I have created a simplified sops file witih gpg and interestingly this worked correctly.  It seems like it is an issue when using aws kms for encryption.
ARTIFACTORY_TOKEN_STAGE: ENC[AES256_GCM,data:<encoded_stuff>,type:str]
sops:
    kms:
        - arn: kms_arn
          role: role_arn
          created_at: "2024-06-17T17:54:57Z"
          enc: <encoded stuff>
          aws_profile: ""
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age: []
    lastmodified: "2024-06-17T18:51:02Z"
    mac: ENC[AES256_GCM,data:encoded_stuff,type:str]
    pgp: []
    unencrypted_suffix: _unencrypted
    version: 3.8.1

@patrickdomnick @dl-mai @yxxhero 👋 That 0 successful groups required, got 0 is coming from sops that I believe we can't control from helmfile.
Do you, by any chance, have `aws_profile" field in the sops file? If so, I saw something fixed sops by removing that field: https://github.com/carlpett/terraform-provider-sops/issues/74#issuecomment-1766662326

Looks like the latest release fixed the issue! Thank again! 👍


[RISCV] Fix program buffer handling

The implicit ebreak after the last program buffer word can only be used when the whole program buffer is used, otherwise we need to add the ebreak command explicitly.
@Disasm This should fix the debugging problems with GDV32, could you please try it again?

It works!!! Thank you :heart:
At least I can request register values and step through the code.

bors retry


Backend Projects Pre-Installed Packages

Regarding some of the backend projects:
On a few occasions, campers have had issues implementing their projects, because they use the versions of the pre-installed packages.
Specifically, this has been an issue with the MongoDB/Mongoose packages. Honestly, I realise this more-so falls on the camper, but I would rather have one of two things:

Remove all/most of the packages from the boilerplates (everything not used by the fcc-testing api)
Introduce a clear instruction for the camper that they will likely need to update the packages used.

I believe this was not as much an apparent issue when we were using Glitch, because it had an excellent UI advising packages to be updated. I realise the projects are already under much TLC, and I dislike adding to the mix, but this is coming up on the forum enough.

Is the problem that people use the package in the boilerplate, but try to use the current api?
If so, then why don't we take the packages out of the boilerplates, but put in advice about what packages they're likely to need?  That way the amount of guidance stays about the same, but the confusion should be reduced.

Is the problem that people use the package in the boilerplate, but try to use the current api?

@ojeytonwilliams The issue arises when campers just fork/clone the boilerplate, and start creating the API without updating the packages already in the boilerplate.  So, MongoDB is already included, as is Mongoose. However, the versions are not up-to-date.
Then, a camper will go to the Mongoose docs, and copy the latest information on how to use the Mongoose API, but some methods could be incompatible.

...put in advice about what packages they're likely to need?

This is one option, but I would not expect this to be needed. The user-stories should allude to the camper that some form of database is needed, etc.
It should be up to the camper to decide which packages they use. Majority (just not the ones involving Helmet) of the tests seem lenient for any package to achieve the desired output.

copy the latest information on how to use the Mongoose API, but some methods could be incompatible.

Cool, that's what I assumed.

It should be up to the camper to decide which packages they use.

Agreed, but the boilerplate was giving guidance by having some pre-installed.  I'm fine with taking that away if you think the user stories give enough of a hint.

I am going to offer my two cents worth here.  For the certification projects, I would like to see as few as possible packages preinstalled by us.  These projects are to meant validate users understand how to build an app based on user stories provided.

Thank you for reporting this issue.
This is a standard message notifying you that the issue has become stale, so we are going to close it.
If you think we're wrong in closing this issue, please request for it to be reopened. Thank you and happy coding.


Fix Markdown formatting for section headers

Need a space after the hash marks for github to format it properly:
https://guides.github.com/features/mastering-markdown/#syntax
"Before" Screenshot

"After" Screenshot

Thanks for catching that! I'll push out an update to the website later tonight


Create CONTRIBUTING.md

Add of CONTRIBUTING.md to help the new ones in Open Source world on how to make a contribution.

Closes #19


Other Downloads

What's included?
'Other Downloads' were missing from the additional content section of the vacancy details page.
I have re-added them so that uploaded files can be accessed by candidates.
This PR makes these downloads more visible on the admin platform
Who should test?
✅ Product owner
✅ Developers
✅ UTG
How to test?

Upload a document (or multiple) under 'other downloads' [admin side]
Check the file appears, named correctly and downloadable on the apply side

Risk - how likely is this to impact other areas?
🟢 No risk - this is a self-contained piece of work
Additional context
Include screen grabs, notes etc.
APPLY >>>

PREVIEW:DEVELOP
can be OFF, DEVELOP or STAGING

@tomlovesgithub  - Can update this ticket please


N official integration

We just discovered your plugin and it is pretty awesome!
Could become an official N plugin (as moving to @nativescript npm scope) if you are interested.
I think we could make it even better though:

splitting the websocket implementation into a "subplugin" to prevent too many native deps. like nativescript-http-websocket or something which would depend on this one. No real consequence on android but on ios it would remove the requirement for SocketRocket
maybe even do the same thing for TrustKit ?
support forcing the okhttp version the same way as done through N . I did that for nativescript-https https://github.com/EddyVerbruggen/nativescript-https/blob/master/src/platforms/android/include.gradle#L2

Let us know what you think about that opportunity

Sounds good to me! I agree that having less dependencies would be better :+1:
Ability to configure the okhttp version sounds good to me, I already provide a solution for that in the docs using resolutionStrategy.force, but implementing it the same way as other plugins sounds like a better plan.
I'm also pretty active on the Slack (name: Jeroen Bobbeldijk), if you want to discuss more :)

Excellent @jerbob92 really appreciate this plugin. We will reach out to you regarding this in the coming weeks as we strengthen core as well as refactor various parts for optimizations.


Keys Initial Live Tests

Initial 20 Test Scenarios for Keys Sync and Async Client each.

Can one of the admins verify this patch?


expense tracker added

Fixes
#581

Description
Created an expense tracker app using react that also supports voice recognition . User can track his/her daily expenses by just entering them in a specific category .
Type of change

[x] New project (non-breaking change which adds functionality)

How Has This Been Tested?
Ran the app and tested each and every feature . Below is the video for the same .
Tech Stack used:
Reactjs ,Context API , Material UI and react's speechly library
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have included a README.md file
[x] My project follows this structure: Domain Folder/ Project Name Folder/ Project files and Readme.md files
[x] I have added demonstration in the form of GIF/video file
[x] I am a GSSoC'22 Participant

Working:-
https://user-images.githubusercontent.com/81703477/170818313-35444831-0974-4b57-90d8-81cc42f49133.mp4

@heet-kumar please review my PR

Add that video from the pr template into the readme file also.

Okay

@manisha-27  done . please check now

@manisha-27  can you increase the level ? It has voice recognition .


Quoine: can't get BTC Balance

The following code returns 0 BTC, even though I have a non-zero balance on the exchange:
AccountService accountService = exchange.getAccountService();
accountService.getAccountInfo().getWallet().getBalance(Currency.BTC)

The same code works fine with fiat currencies.
I'm using the 4.2.1-Snapshot

@JojoSc Try latest develop version, should work.

I'm in the process of fixing this (it's not actually implemented).
Having a little trouble with the auth though, getting a 401 for some reason.

Is this now fixed?

Yep


[BUG] Please fix server/grpc plugin

Can't install server/grpc package

What are you trying to do?

go get github.com/asim/go-micro/plugins/server/grpc/v3

What did you expect to happen?

Install server/grpc package

What happens instead?

go: github.com/asim/go-micro/plugins/server/grpc/v3 upgrade => v3.0.0-20210130181356-ca3dfc4580fa
go get<EMAIL_ADDRESS>requires
<EMAIL_ADDRESS>invalid version: unknown revision<PHONE_NUMBER>00

go.mod of server/grpc is incorrect. See https://github.com/asim/go-micro/blob/master/plugins/server/grpc/go.mod
Not work with:
github.com/asim/go-micro/plugins/broker/memory/v3 v3.0.0-00010101000000-000000000000
github.com/asim/go-micro/plugins/client/grpc/v3 v3.0.0-00010101000000-000000000000
github.com/asim/go-micro/plugins/registry/memory/v3 v3.0.0-00010101000000-000000000000
github.com/asim/go-micro/plugins/transport/grpc/v3 v3.0.0-00010101000000-000000000000

How to reproduce the bug:
go get github.com/asim/go-micro/plugins/server/grpc/v3

Environment:
go version go1.15.6 linux/amd64

Will have a look soon, no time at the moment

Same thing happening with prometheus:
go: finding module for package github.com/asim/go-micro/plugins/wrapper/monitoring/prometheus/v3
go: found github.com/asim/go-micro/plugins/wrapper/monitoring/prometheus/v3 in github.com/asim/go-micro/plugins/wrapper/monitoring/prometheus/v3 v3.0.0-20210130181356-ca3dfc4580fa
go<EMAIL_ADDRESS>requires
<EMAIL_ADDRESS>invalid version: unknown revision<PHONE_NUMBER>00

@asim when i use etcd registry also has a errorm, i find that the github.com/asim/go-micro/plugins/registry/etcd/v3 etcd registry module should change to go.etcd.io/etcd/v3
github.com/coreos/etcd can not work.
github.com/asim/go-micro/plugins/registry/etcd/v3 imports
        github.com/coreos/etcd/clientv3 tested by
        github.com/coreos/etcd/clientv3.test imports
        github.com/coreos/etcd/integration imports
        github.com/coreos/etcd/proxy/grpcproxy imports
        google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.35.0), but does not contain package google.golang.org/grpc/naming

@iGoogle-ink
go.mod
add
replace google.golang.org/grpc => google.golang.org/grpc v1.26.0

@brandon-welsch hi, now you can update go get -u github.com/asim/go-micro/plugins/server/grpc/v3
$ go get -u github.com/asim/go-micro/plugins/server/grpc/v3

go: downloading github.com/asim/go-micro/plugins/server/grpc/v3 v3.0.0-20210205095042-6f666d63c85a
go: github.com/asim/go-micro/plugins/server/grpc/v3 upgrade => v3.0.0-20210205095042-6f666d63c85a
go: downloading github.com/asim/go-micro/v3 v3.0.0-20210120135431-d94936f6c97c
go: downloading google.golang.org/protobuf v1.24.0
go: downloading google.golang.org/grpc v1.27.0
go: downloading golang.org/x/net v0.0.0-20201224014010-6772e930b67b
go: downloading github.com/google/uuid v1.1.5
go: downloading google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d
go: downloading golang.org/x/sys v0.0.0-20201119102817-f84b799fce68
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading github.com/miekg/dns v1.1.27
go: github.com/go-git/go-git/v5 upgrade => v5.2.0
go: github.com/miekg/dns upgrade => v1.1.38
go: golang.org/x/net upgrade => v0.0.0-20210119194325-5f4716e94777
go: golang.org/x/sync upgrade => v0.0.0-20201207232520-09787c993a3a
go: golang.org/x/crypto upgrade => v0.0.0-20201221181555-eec23a3978ad
go: google.golang.org/genproto upgrade => v0.0.0-20210204154452-deb828366460
go: github.com/golang/protobuf upgrade => v1.4.3
go: github.com/cpuguy83/go-md2man/v2 upgrade => v2.0.0
go: golang.org/x/text upgrade => v0.3.5
go: golang.org/x/sys upgrade => v0.0.0-20210124154548-22da62e12c0c
go: github.com/shurcooL/sanitized_anchor_name upgrade => v1.0.0
go: github.com/asim/go-micro/v3 upgrade => v3.5.0
go: github.com/google/uuid upgrade => v1.2.0
go: github.com/pkg/errors upgrade => v0.9.1
go: github.com/russross/blackfriday/v2 upgrade => v2.1.0
go: google.golang.org/grpc upgrade => v1.35.0
go: github.com/xanzy/ssh-agent upgrade => v0.3.0
go: google.golang.org/protobuf upgrade => v1.25.0
go: github.com/fsnotify/fsnotify upgrade => v1.4.9
go: github.com/kevinburke/ssh_config upgrade => v0.0.0-20201106050909-4977a11b4351
go: github.com/sergi/go-diff upgrade => v1.1.0
go: github.com/imdario/mergo upgrade => v0.3.11
go get: upgrading<EMAIL_ADDRESS><EMAIL_ADDRESS>invalid version: go.mod has non-....v1 module path "github.com/fsnotify/fsnotify" at revision v1.4.9


Fixed incorrect link in CONTRIBUTING.md

Original link was pointing to random html spec, should have been to typed array documentation.

Thanks!


Updates to make smart-dispatch Python 3 compatible

I made a few updates to make smart-dispatch compatible with Python 3, namely replacing all print statement calls by the print function, converting a bytestring to a string and using open instead of file to detect if the argument of the option -f is a file. I did not do extensive testing yet but this does not break it with Python 2 or 3 when running a basic test (smart_dispatch.py -qtest launch nvidia-smi).

Coverage decreased (-0.06%) to 94.473% when pulling c8e697dce63701445f30533071d60800f1391177 on jfsantos:master into c94f08e828d6deec453c054fd340e694ec0e0b9a on SMART-Lab:master.

Coverage decreased (-0.06%) to 94.473% when pulling 2d97763551812c90231a03a7cdcc156c4baea65d on jfsantos:master into c94f08e828d6deec453c054fd340e694ec0e0b9a on SMART-Lab:master.

There are just a few issues on Travis, I'm going to check them out soon.


Questions about Docus icon libraries?

Ask your question
I know that Docus uses some kind of Icon library. One that I know is Hero Icons.
Questions:

Which version of HeroIcons does Docus use?
If Docus uses HeroIcons 1.0.6, How we can upgrade it or Can you upgrade it to the latest version?
What other Icon libraries does Docus use?
I want some Docus icon but they are not working. Why?
How we can use other icon libraries?
Will you add more about icons into the docs of Docus?
Can you give more detailed answer please?

What's happening exactly?
icon: heroicons-outline:rocket is not working in _dir.yml
Additional information
https://heroicons.com/

Hey. Docus is using NuxtIcon component, which is using Iconify


examples: use empty metadata service for azure unattend stage

:white_check_mark: Build windows-openstack-imaging-tools 1.0.261 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/387963d3fa by @ader1990)

:white_check_mark: Build windows-openstack-imaging-tools 1.0.262 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/249e3480cd by @ader1990)

:white_check_mark: Build windows-openstack-imaging-tools 1.0.263 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/ff193717aa by @ader1990)


PHP5.6安装出错

阿里云ECS
Ubuntu 12.04.5 LTS (GNU/Linux 3.2.0-67-generic x86_64)
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
sed: can't read /usr/local/php/etc/php.ini: No such file or directory
/bin/cp: cannot stat `sapi/fpm/init.d.php-fpm': No such file or directory
chmod: cannot access `/etc/init.d/php-fpm': No such file or directory
update-rc.d: /etc/init.d/php-fpm: file does not exist
include/php-5.6.sh: line 139: /usr/local/php/etc/php-fpm.conf: No such file or directory
sed: can't read /usr/local/php/etc/php-fpm.conf: No such file or directory
sed: can't read /usr/local/php/etc/php-fpm.conf: No such file or directory
sed: can't read /usr/local/php/etc/php-fpm.conf: No such file or directory
sed: can't read /usr/local/php/etc/php-fpm.conf: No such file or directory
php-fpm: unrecognized service

日志放到百度云，贴出来看看？


Add MapLibre as a Swift Package

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Remove references to Mapbox in the Podfile
Add MapLibre as a Swift Package from https://github.com/maptiler/maplibre-gl-native-distribution
Tested with Xcode Version 12.4 (12D4e)

Figure 1:  Swift Package Dependencies in the Project Navigator View

Figure 2:  Project Amazon Location Service Demo in the Swift Packages View

Figure 3: Add Package to Amazon Location Service Demo in the Swift Packages View

Figure 4: Embedded MapLibre Libraries in the Target View

Mapbox SDK v5.9.0 as a CocoaPod

MapLibre v5.10.0 as a Swift Package

Mapbox SDK v5.9.0 as a CocoaPod

MapLibre v5.10.0 as a Swift Package


Add BDB backend

I'm not sure what "BDB" is. As there has been no discussion on this in quite some time, I'm closing it.
Josh, if you want this to be re-included, feel free to open another issue.


Daily Bonus: Alt Checks

Prevents stardust farming by checking & synchonizing alts before awarding a bonus.

Good idea, helps stop people like the gravewalker from abusing his alts + daily bonus

^ Ikr xD I was thinking this was needed


shouldOverrideUrlLoading not Working in realse

Hello shouldOverrideUrlLoading work on debug mode only , but in release not working
crossPlatform: InAppWebViewOptions(                                // useShouldOverrideUrlLoading: true,
`shouldOverrideUrlLoading:  (InAppWebViewController controller, ShouldOverrideUrlLoadingRequest shouldOverrideUrlLoadingRequest){
print(shouldOverrideUrlLoadingRequest.url);
                         if (shouldOverrideUrlLoadingRequest.url.startsWith("tel:")) {
                           // launch myapp here
                           print('tel=>>>>>>>>>>>>>>');
                          launch(shouldOverrideUrlLoadingRequest.url);
                           return null;
                         }else if(shouldOverrideUrlLoadingRequest.url.startsWith("https://api.whatsapp.com")){
                           launch(shouldOverrideUrlLoadingRequest.url);
                           return null;
                          }else if(shouldOverrideUrlLoadingRequest.url.startsWith("mailto:")){
                           launch(shouldOverrideUrlLoadingRequest.url);
                           return null;
                         }else {
                           return webView.loadUrl(url: shouldOverrideUrlLoadingRequest.url);
                         }

                       },`

i use
flutter_inappwebview:     git:       url: https://github.com/pichillilorenzo/flutter_inappwebview.git       ref: master

在Android插件源码中将useShouldOverrideUrlLoading属性默认设置为true

@wj1234  i use it but same

Can confirm this as well for Android production builds with the latest code from the master branch of this repo. Did you already identify the cause of the problem?


Add Český Rozhlas

Adds scrobbling from all stations of Český Rozhlas (Czech national radio).

Could we add a test case?

I added a test case by editing the radionomy.js file (I did the same thing for the connector itself). But I don't have deep understanding of how the tests here work and if for example the fact that the radio stations sometimes don't play music (there are also news and other shows played on them).

It would not really make any difference since .fp-play matches the play/pause button no matter what current player state is while .is-paused matches the same button in paused state only.
But I agree that it is clearer now.


[FC UI] Initiate transfer in

As an end-user I need to be able to initiate a transfer in by tapping "buy now" in the review purchase UI.
Similar to PFP version of this issue: #1754
AC:

tapping "buy now" in the review purchase UI initiates a transfer with the FC provider (provider should be given as as a prop for that component)
success response from provider results in a success UI

TODO get designs for a failure UI


Run llama.cpp models

I guess it would be easy for you run the ggml llama.cpp compatible models.
In this case, you don't need the GPU and could run the models in memory.
From a simple test I find that llama.cpp is faster on 13B model then web-llm on 7B model on the same system.
Although, running in GPU migt help:
https://github.com/ggerganov/llama.cpp/discussions/915

HI,
For running the web-llm in browser I've tried a NVIDIA T2000 and a build in I9 intel GPU.
The T2000 is fast in the prompt ingestion at ~17 tokens/s and slowest in generation ~0.6 tokens/s.
The build in I9 intel GPU was ~2 tokens/s in both tasks.
The llama.cpp runs on a CPU only and it uses speedup tricks like mmap to store the model in memory. But, for the 13B model the memory use is ~7 GB.

Just out of curiosity, what do you mean by "bringing maximum performance on macbook gpu" is it a limitation of TVM?
Or, you are using some GPU specific tricks?
Any way, there are two parts to my request:

First is about CPU execution and the possibility to run ggml models in browser, similar to the way as it is run by llama.cpp.
The second is a possibility to run the ggml models with GPU in browser. Similar to the topic of the discussion: https://github.com/ggerganov/llama.cpp/discussions/915

For "bringing maximum performance on macbook gpu", you need to know that the same code implementation could have dramatically different performance on different hardwares. So it's impossible to run equally well on all devices. Our choice is to bring best performance for macbook gpu first, and then improve performance on other devices.
For the rest of your question, the answer is sadly "no". The reason is that ggml doesn't have a good enough codegen for webgpu. Even with Triton, it can only codegen for cuda. TVM is designed for deploying on diverse backends, and that's why we choose it as our framework.
But what we can promise is that : we will try to optimize as much as we could so that we could align or outperform llama.cpp on as many devices as possible.


[BUG] deeplake.compute slow if no tensors in output dataset

🐛🐛 Bug Report
I was debugging a memory leak when using the parallel compute functions and while doing that I discovered that if the output dataset does not have any tensor created, the iteration is painfully slow.
In google Colab the iterations reduced from 75-130 images per second to 25-30 images per second. On my desktop computer it reduced from 110 to 1.5 iterations per second!
Colab example found here.
⚗ Current Behavior
Very slow iteration when no tensors in dataset
Input Code
ds = deeplake.load('hub://activeloop/imagenet-train', token=TOKEN)

@deeplake.compute
def calculate_mean(sample_in, sample_out):
  sample_in.images.numpy()
  return None

tmp_ds = deeplake.empty("mem://tmp_ds")
#tmp_ds.create_tensor("dummy", htype="generic")
calculate_mean().eval(ds, tmp_ds, num_workers=8)

Expected behavior/code
Same speed of iteration for empty output datasets.
I would expect this behavior regardless of how much sense the use case makes. My use case is that I want to create a compute function which only has a side effect, not any output to a new dataset. I could avoid specifying the output dataset to make the compute function perform the (non-existing) change in place, but then I cannot use the function on read-only datasets. Is there any other way to do this?
⚙ Environment

Colab, see link above.
Desktop:

Linux daniel-desktop 5.4.0-126-generic #142-Ubuntu SMP Fri Aug 26 12:12:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
python 3.10.0
deeplake 3.1.0

Thank you for reporting the issue @daniel-falk. We'll look into this asap. For transforms that do not need to create new datasets, have you seen the syntax below? Does that satisfy your use-case?
https://docs.activeloop.ai/tutorials/data-processing-using-parallel-computing#transformations-on-existing-datasets

Hi @istranic, thanks! I have seen that syntax, it is almost what I want but the issue is that I cannot apply it to read-only datasets. Since my function only has side-effects, it will not modify the dataset and thus it should work with read-only datasets too. If it is not possible, then it is no big deal. I can either use a writable copy of the dataset or a temporary output dataset.

Hi @daniel-falk Just spoke to our team about this and I think the main issue is our communication of the purpose of deeplake.compute.
We built this feature in order to support parallel updates to datasets, because users implementing this would be tricky, due to the uniqueness of our data format. For example, for the read-only error, we assume the deeplake.compute function will update the dataset, and we don't want to throw an error after hrs of computation, so we check for write access prior to beginning computations.
For use-cases like the one above, we recommend using any of pythons parallel computation frameworks. For reading data, you won't get much of an advantage by using deeplake.compute.
Does this help clarify things?

Hi @istranic ,
I understand how you are thinking.
Firstly, my first attempt was to use multithreading in python. Doing so I got loads of strange exceptions from deeplake, so I assumed that the datasets was not thread safe and that it was the reason for the compute functions. See example here generating the errors.
Secondly, compare this code with a compute function:
import deeplake
import numpy as np

@deeplake.compute
def calculate_mean(sample_in, _, means, skipped_images):
  img = sample_in.images.numpy()
  if len(img.shape) < 3 or img.shape[2] < 3:
    skipped_images.append(sample_in.index.values[0].value)
    return None
  means.append(img.mean(axis=(0,1)))

skipped_images = []
means = []
ds = deeplake.load('hub://activeloop/imagenet-train')
calculate_mean(means, skipped_images).eval(ds, num_workers=16, read_only_ok=True)
final_mean = np.array(means).mean(axis=0)

print()
print(f"Skipped {len(skipped_images)} gray-scale images")
print(f"Mean is {final_mean / 255}")

with this code that uses multithreading (in hindsight, I might have been able to do it nicer with a Queue, but still..):
import deeplake
import numpy as np
from concurrent.futures import ThreadPoolExecutor, as_completed

# Implement a method that calculates the sum of the means for each channel
# on a range of images from the dataset
def calculate_mean(start, stop, th_idx, skipped, errors):
  sum_of_mean_of_channels = np.zeros(3)
  for i, img in enumerate(ds.images[start:stop]):
    try:
      img = img.numpy()
      if len(img.shape) < 3 or img.shape[2] < 3:
        skipped.append(i)
        continue
      sum_of_mean_of_channels += img.mean(axis=(0,1))
    except Exception as e:
      errors.append(dict(th_idx=th_idx, i=start+i, exc=e))
      break
  return sum_of_mean_of_channels

ds = deeplake.load('hub://activeloop/imagenet-train')

num_workers = 16
with ThreadPoolExecutor(max_workers=num_workers) as pool:
  idx = np.linspace(start=0, stop=len(ds), num=num_workers+1, dtype=np.int32)

  errors = []
  skipped = []
  futures = {}
  for i, (start, stop) in enumerate(zip(idx[:-1], idx[1:])):
    print(f"Scheduling range {start}-{stop} to worker {i}")
    futures[pool.submit(calculate_mean, int(start), int(stop), i, skipped, errors)] = i

  total_sum_of_mean_of_channels = np.zeros(3)
  for future in as_completed(futures):
    total_sum_of_mean_of_channels += future.result()

final_mean = total_sum_of_mean_of_channels / (len(ds) - len(skipped))

print()
print(f"Mean is {final_mean / 255}")

print(errors)

First example also have functionality such as progress bar, second does not since its awkward to implement on a parallel computation.

How about an override for the read-only check, like this: https://github.com/activeloopai/deeplake/pull/2019 ?
calculate_mean(means, skipped_images).eval(ds, num_workers=16, read_only_ok=True)


Cf commands fails intermittently dial tcp: i/o timeout

Hi,
My CF CLI commands fail intermittently giving TCP timeout error.
I've gone through below issue, it says a retry is introduced in v6.6.2
https://github.com/cloudfoundry/cli/issues/272
I am using version 6.14.1+dc6adf6-2015-12-22 for cli, how can I verify that retry is being called?
As in CF debug logs I see request is being logged only one time, however tcp: i/o timeout is logged twice two time.

Hi @nitinpadalia
The CLI does not expose any logs that would help you to determine if, and how many retries have occurred. CF trace would look no different.
The tcp i/o timeout message comes from your system. I don't know how the message was logged twice (e.g. right after each other, or with CF TRACE message templating around it?).
Two options to try is to either put a proxy between your CLI and API endpoint that logs the requests going through, or compile a special version of the CLI with extra debugging statements to log retries.
Feel free to reach out if you need any help with that.
Regards,
Dies Koper
CF CLI PM


[Question] react-router-redux syncHistoryWithStore

I'm wondering if there is any way to sync the history with the redux store so to be able to use the nice handy methods push() that react-router-redux provides.

Hi @kfamilonidis
Hmm, I am not sure myself.  I don't know off hand.  Why are you trying to use redux? Why not just call the api's provided by history directly?

Also, I would consider adding this as a stackoverflow question.  I am not sure it's specific as an issue against this project.

Hi @kfamilonidis I am going to close this for now.  Please let me know how you got along.

thanks @ctrlplusb, your comment helped me a lot to figure it out, I passed history from createHistory to BrowserRouter, and everything seemed to worked. I write it down in case someone has the same issue:
const history = createHistory();
const store = configureStore(history,window.APP_STATE);
syncHistoryWithStore(history,store)

  <BrowserRouter history={history}>
    ...
  </BrowserRouter>


Win32: ShellScalingApi.h not available on MinGW targets

@MouriNaruto I am trying to update the CodeBlocks project to the latest version of the Win32 driver, however, I'm getting an error that  ShellScalingApi.h is missing.
According to this issue, it seems that it's either not available or missing some APIs when using GCC to compile. Could you take a look when you have some time, please?

Got it. I will do some changes in recent days.


Allows to create Contact, but it isn't visible to the user utill advance role is given.

So, If Create or Import Contact role is given then advance role should get turn on automatically.

Auto On Contact Advance on Role Page where User trigger Contact Create Role


Devices not discoverable on large network

Hi,
in the last months i read nearly all the issues reported regarding this topic and i couldnt find anything that helped..
Here is my setup:
My network is <IP_ADDRESS>/16
node-red inside Docker with direct Host network running on <IP_ADDRESS>/16

port 80  - description.xml can be reached from every device in the network
Alexa native and Alexa Sonos devices in 192.168.240.X/16

Phillips Hue Bridge in <IP_ADDRESS>/16
the Philipls Hue devices get discovered by Alexa
Nodered Device dont get discovered.
any ideas ?

what is nodered device? What it is useful for? I only have  Philips Hue devices

Another important info, the network 192.168.x.x is defined as class C, not as class B /16 as you pretend it works. NOt all devices will accept to broadcast on /16 if the net should be /24.
On the other side a network A 10.x.x.x could be restricted as B, so I strongly suggest you to use 10.x addresses

it is just not natural, you can't be sure all devices will work properly. For such trubles you can loose days before discovering that didn't work because of that, I prefer avoid such cases.
I've read back your first post, seems your node red can't be discovered by Alexa at all, you probably have problem in Docker, there are lot of topics with the same problem, have a deep read and you will find solutions

as i said, i can see every broadcast message from the ssdp, and i can also get the  xml and api access.
what i dont know: is the xml data correct?
or is there a compatibility issue with with my devices (Alexa Echo gen 1 and SONOS ONE)

what is the ssdp device? You told you have real philips bulbs and one Docker too, where broadcast coming from? Bulbs or Docker? Are you sure your Docker works? Have you tried power out all real Philips devices and let only Docker? Have you deep reset Echo as written in many posts here? Have you read troubleshooter too?

SSDP is the service used by node-red-contrib-amazon-echo to send the broadcast messages
the node-red-contrib-amazon-echo SSDP Service is received by my notebook (wireshark) and my Phone (upnp discover)
the Port 80 XML data is retrievable from within the network
the broadcast is from <IP_ADDRESS> which is my node-red docker container (host network)
another broadcast in the network from my original phillips hue bridge is also present
i factory reset all 8 Alexa capable devices including a smart TV
discovery is not working for node-red-contrib-amazon-echo devices
discovery is working with original phillips hue bridge
node-red-contrib-amazon-echo is using port 80
node-red is running in privileged mode
i have only one network interface on the node-red machine (no Wifi)  (no multiple IPs)

and just jet i tried discovery while having the original hue bridge unpowered - nope doesn't work -.-
I think i have to try out some alternatives -.-

well, now is pretty clear :)
NR is recomended to don't install with root privileges, I suggest you to try reinstall all following their guide.
Source: https://www.philips-hue.com/en-us/support/friends-of-hue#Can_I_add_two_Hue_Bridges_to_Alexa


Notification needed to identify when the server is listening and ready to accept web requests.

We have an app with Kestrel and an embedded web browser control. We would like some way to be notified that the server is accepting requests before we surf to the index.html with the embedded web browser control. Any ideas ?

This is all in the same process, right? Resolve the IApplicationLifetime service and hook ApplicationStarted
https://github.com/aspnet/Hosting/blob/dev/src/Microsoft.AspNetCore.Hosting.Abstractions/IApplicationLifetime.cs

Perfect.. thanks Tratcher


Fixed media picker action

          2 Messages

:book:
iOS: Executed 1 test, with 0 failures (0 unexpected) in 0.005 (0.007) seconds

:book:
tvOS: Executed 1 test, with 0 failures (0 unexpected) in 0.004 (0.012) seconds

  Generated by :no_entry_sign: Danger


GetLocationById: issues

GetLocationById:
id: is missing format: uuid in the schema.
country in schema is just
country:
type: string
"country": "BE", In Availability.md on line 409. 437 and 629
It should be "country": "Belgium" and not the country code when type: string
Should it be changed in the schema to countryCode:?
countryCode:
type: string
minLength: 1
maxLength: 2
description: The (two-letter) value - e.g. "SE", "BE", ... - MUST exist in the ISO 3166-1 standard list.

Is has been fixed with https://github.com/papinet/papiNet-API/commit/040731a7f5e12d78f8ab2e51425155755104431e.


The server files updated not sync to local

The server files updated not sync to local, the sample file name is not sync.
Can you support this feature?  eg: check sum for content ,not file name.

This has been discussed in #12. If you are interested about my proposal, I can reopen this issue.


Is there any introduction about this repository?

Hi, is this a usable toolkit?

It's just a parser and AST builder for now for SQL16 + Presto extensions. If that's what you are looking for - sure!

If still relevant: I have started to work on some most basic documentation. Please see PR #31  and the preview at https://manticore-projects.github.io/sql/index.html


Replicator _flushTimers clearing

Hi,
I spot this issue during the shutting down of an application.
Given this snippet as example:
/*global require,__dirname*/
const path = require('path')
, process = require('process')
, {setTimeout} = require('timers')
, IPFS = require('ipfs')
, OrbitDB = require('orbit-db')
, wtf = require('wtfnode')
, repo = require(path.resolve(__dirname, 'src', 'store'))
, documentStoreName = 'test';

const ipfs = new IPFS({
  repo,
  'init': true,
  'start': true,
  'EXPERIMENTAL': {
    'pubsub': true
  }
})
, orbitdb = new OrbitDB(ipfs);

ipfs.once('ready', () => {
  orbitdb.docstore(documentStoreName, {
    'indexBy': 'name'
  })
  .then(db => {

    process.on('SIGINT', () => {
      return db.close()
        .then(() => ipfs.stop())
        .then(() => setTimeout(() => {

          wtf.dump();
        }, 5000));
    });

    db.events.once('ready', () => console.info({
      'type': 'model-ready'
    }));

    db.load();
  })
  .then(() => console.info('done'));
});

when I try to shutdown this is the situation (I reported also the overall stdout):
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4003/ws/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
done
{ type: 'model-ready' }
^C[WTF Node?] open handles:
- File descriptors: (note: stdio always exists)
  - fd 1 (tty) (stdio)
  - fd 2 (tty) (stdio)
- Intervals:
  - (3000 ~ 3 s) (anonymous) @ /home/dario/git/tmp/orbit-test/node_modules/orbit-db-store/src/Replicator.js:36

Node isn't stopping and if I try again it complains about the fact that ipfs is already shutted down with this:
^C(node:8083) UnhandledPromiseRejectionWarning: Error: Already stopped
    at Function.promisify (/home/dario/git/tmp/orbit-test/node_modules/ipfs/src/core/components/stop.js:13:23)
    at /home/dario/git/tmp/orbit-test/node_modules/promisify-es6/index.js:41:20
    at new Promise (<anonymous>)
    at IPFS.<anonymous> (/home/dario/git/tmp/orbit-test/node_modules/promisify-es6/index.js:35:16)
    at db.close.then (/home/dario/git/tmp/orbit-test/test.js:30:26)
    at <anonymous>
(node:8083) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:8083) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

If I use my branched version the output becames:
$node test.js
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4003/ws/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
Swarm listening on /ip4/<IP_ADDRESS>/tcp/4002/ipfs/QmRvrTpPReqgY4UZyftPSJBcGtNFuCFPGuQSfspno2MiMd
done
{ type: 'model-ready' }
^C[WTF Node?] open handles:
- File descriptors: (note: stdio always exists)
  - fd 1 (tty) (stdio)
  - fd 2 (tty) (stdio)

Hope this will help.
Dario

@wouldgo thank you for reporting this, good catch and much appreciate the bug report! 👍Never seen it before, but makes sense that it can happen as we don't clear the timers.
I would propose to fix this in a slightly different way: instead of using the "private" variable _flushTimer, it'd be better to add a method to Replicator called stop() and call that in store.close() at the beginning of the close-block (eg. https://github.com/orbitdb/orbit-db-store/blob/master/src/Store.js#L152). This way we can encapsulate any "shutdown" functionality inside the replicator and don't create a member variable dependency between store and replicator. Would that make sense?

@haadcode agreed on keeping some sort of encapsulation.

Fantastic @wouldgo! 👍 LGTM.

Thank you again for reporting and fixing this, very helpful! ❤

You're welcome. :dancing_men: Just to know: when this will be available from the registry?

Published it to npm as v2.5.3. Bubbling it up to OrbitDB and will make a new npm release (0.19.9) as soon as CI checks.


Support Chrome for Testing

With the release of Chrome for Testing, the Chrome team has changed how and where they host their executables. The install path for latest chromedriver is no longer supported at https://chromedriver.storage.googleapis.com/LATEST_RELEASE_. The new way to do this is to use the JSON endpoints.
However, the JSON endpoints are massive and will only grow over time (they list all versions since it was ported over [v113]), so it would be better to use the other APIS which follow the LATEST_RELEASE_ format and return a very small amount of information.

Closed via v2 release


Don't use any-promise

Don't think there's a reason to use this nowadays. Even https://github.com/kevinbeaty/any-promise says to use native promises.
Sending this out because I got this error in Mocha:
Error: global leak detected: '@@any-promise/REGISTRATION'

I have no principal objections, but this requires major version bump. Worth to fix pending bug first. Should happen in 1-2 weeks.

Fixed pending issues. Kick me after 2-4 weeks. Need feedback - is request with last changes stable or we should try upgrade to got again (this change is breaking too).

Please fix conflicts and merge

@puzrin I see you dropped any-promise. Also, publish this new version to npm maybe?


NavFilterTestHarness24': Undefined function or variable 'startTime'

Hi
I am getting the error below when trying to run NavFilterTestHarness24.mdl in Matlab 2015a. I have followed the steps in the readme. Any clue why its happening ?
Thank you
Error evaluating expression 'startTime' for 'StartTime' specified in the Configuration Parameters dialog for block diagram 'NavFilterTestHarness24': Undefined function or variable 'startTime'

I am having the same problem as shening. Can anyone help.?
Thanks.

after changing your current directory to models. run this script in matlab
addpath('../scripts/','../TestData')

I am also facing the same problem as( Undefined function or variable 'startTime'.) I am totally confused about this error. Please someone help me to solve this error.

@esridhar159
I think if you do the procedure that i mention in the above comment. this problem will be solved.

change your directory to models. ( as mentioned in the readme)
runaddpath('../scripts/','../TestData')
run RunNavFilterTestHarness24.m (not RunNavFilterTestHarness24.mdl)

@rafaelmendy why did I follow you,there was error  ,like this
Error using load
Unable to read file 'simtest20.mat': no such file or directory.
I don't know where  is simtest20.mat.

@snn-Lenovo-G405s
Hi
You are right. there is no such file "simtest20.mat"
So, you must use other test data that exist in the "TestData" folder.
open LoadNavFilterTestData.m  and comment "load('simtest20.mat')" and uncomment another one.
for example:
load('TestLog1.mat')
%load('TestLog2.mat')
%load('TestLog3.mat')
%load('simtest20.mat')

@rafaelmendy Thank you .The problem is solved


Jarvis compare ecs

import error fix for ecs_compares

You've got conflicts to fix before this can be merged...


Evaluation of Twitter follower

Track the data of DefiChain Twitter-follower and create an evaluation

The size of the csv-file is becoming very fast, very big.
Maybe another file format is needed or a reduced data is stored (not every twitter id each day)

Data with reduced information will be stored. So, on a daily base the following information will be stored

number of followers
number of people followed new the day before
number of people unfollowed the day before


Fixed IndexError when pytypes splits filepaths in code that warns about argument names

Hi again, I've got another one!
(Python 3.5 inside pipenv on Windows)
Note that setting pytypes.warn_argnames = False avoids this.
In a file preview_output.py, I have code that looks something like this:
@typechecked
class Thing:
	@classmethod
	def Foo(_cls, x: int, y: int):
		...
	...

f = Thing.Foo(3, 5)

(I have reasons for _cls to have that leading underscore, but that's beside the point)
But running preview_output.py gives me this error:
Traceback (most recent call last):
  File "preview_output.py", line 40, in <module>
    f = Thing.Foo(3, 5)
  File "c:\users\lubieowoce\documents\pytypes\pytypes\typechecker.py", line 771, in checker_tp
    func0, slf, clsm)
  File "c:\users\lubieowoce\documents\pytypes\pytypes\util.py", line 799, in _warn_argname
    off = _calc_traceback_list_offset(tb)
  File "c:\users\lubieowoce\documents\pytypes\pytypes\util.py", line 780, in _calc_traceback_list_offset
    if tb_list[off][0].split(os.sep)[-2] == 'pytypes':
IndexError: list index out of range

After inspection, I found out tb_list[off][0] is equivalent¹ to tb_list[off].filename.
In my case, tb_list[off].filename is just 'preview_output.py',
so splitting it on os.sep gives ['preview_output.py'] which has no [-2], hence the IndexError.
I'm guessing that on other platforms / in other circumstances traceback() gives absolute filepaths
(that's what inspect.stack() does on my machine)
and no one hit this error yet. Anyway, I just added a length check (and a comment explaining what it's supposed to do) and I think it should work as intended. Tell me what you think!
¹ ...at least in Python 3.5 – were you using [0] for compatibility reasons?

This one is a bit difficult because we currently don't have tests for checking the error output formats (something like doctests would be needed I suppose). That means it is hard to assert that nothing else gets broken. Anyway, this is fixing a crash so I definitely like to accept this. Do you see a chance to provide a test here?

Sure, I'll get on it later today.

Update: It turns out it's not that easy to write tests for it, especially because I don't know what causes traceback to use a
relative path for the user module in some cases (the cause of my original issue, and the motivation for this PR). Also, I couldn't find a nice way to obtain a deep traceback from within pytypes' internals, so I can't test the traceback-trimming.
I came up with a bunch of potential solutions (tldr - option 3 seems the nicest)

Add a dummy function that just returns the current traceback to the main pytypes package. We then have a way to get a "realistic" traceback on which traceback-trimming functions can be tested. Kinda ugly, but should work. (maybe we could avoid cluttering up the main code by injecting the dummy function dynamically at test time?)

 This frees us from depending on the filesystem and gives us more control – we can simulate any scenario we like. But, the other side of the coin is that we have to trust that the mock tracebacks reflect reality.

Emulate the real scenario – disable pytypes' "nice tracebacks" option; make a @typechecked class with a method that'll cause a warning; intercept the warning object like this¹; and extract the traceback object. This one is the most "realistic", but the tracebacks would all originate from the tests folder – we can't test as many scenarios as with option 2. (perhaps combining the two approaches makes sense?)

I'll to make something like that work tomorrow, let me know what you think!

I told you it's hard to test. But don't spend too much effort. I can accept this without a test. However I would have to review much more carefully and assert somewhat manually that original use cases are not broken. I don't know when I'll find time for this.

Sorry for delaying this so long. I wasn't able to work on pytypes for a while.
This issue looks similar to the one fixed in https://github.com/Stewori/pytypes/commit/0a90a4ca52c0dca0609b9fc963f000695de07283 i.e. #51. Actually I should have fixed it here too but overlooked that the same kind of code is repeated there. I think both solutions work - your's is shorter, mine is a bit safer I suppose. It's mainly cosmetical considerations that I think it should be fixed in the same manner both in _calc_traceback_limit and in _calc_traceback_list_offset.
However I am fine with accepting this PR for now but will probably edit the code later to put both occurrences into the same shape (your's or mine, I didn't decide yet).

Another note: You may want to set pytypes.warn_argnames = False.


Orca group

http://orca66hwnpciepupe5626k2ib6dds6zizjwuuashz67usjps2wehz4id.onion

Thank you so much.
Group added : https://www.ransomware.live/group/orca
Parser written and first victim in the database : https://www.ransomware.live/id/RXhjZWxQbGFzdCBUdW5pc2llQG9yY2E=


License

This would close #14 by having a separate license file which is also included in source uploads to pypi.

Thank!


Patch request not updating as expected and missing some updates

Describe the bug
A clear and concise description of what the bug is
Patch request not updating as expected and missing some updates
Exception or Stack Trace
Add the exception log and stack trace if available
NA
To Reproduce
Steps to reproduce the behavior:
we pre-populate data with empty array and
we will do a patch using below for-each batch
We iterate through the DF to iterate minute array
def write_multi_window_cosmos1min(df, epochID):
final_df = df.withColumn('id', \
                F.lit(F.concat(
                        F.lit(F.col('warehouseid')),
                        F.lit('_'),
                        F.year(F.col('window.end')).cast(T.StringType()),
                        F.lit('_'),
                        F.month(F.col('window.end')).cast(T.StringType()),
                        F.lit('_'),
                        F.dayofmonth(F.col('window.end')).cast(T.StringType()),
                        F.lit('_'),
                        F.hour(F.col('window.end')).cast(T.StringType()),
                        F.lit('_1')
                    ))).withColumn('warehouseid', F.lit(F.col('id')))

print(f'1 min window with batch id {epochID}')
final_df.show(truncate=False)

minute_data = final_df.selectExpr('MINUTE(window.start) as minute_key').collect()

for minute in minute_data:

    curr_minute = minute['minute_key']

    cfg["spark.cosmos.write.patch.columnConfigs"] = f"[col(time_array).path(/time_array/{curr_minute}).op(set)]"

    print('1 min window -> ', cfg["spark.cosmos.write.patch.columnConfigs"])

    final_df\
        .selectExpr('cast(warehouseid as string) as warehouseid',
                    'CAST(id AS STRING) as id',
                    'window.end as curr_tsp',
                    'MINUTE(window.end) as minute_key',
                    'if (actualTemperatureCount is NULL, 0, actualTemperatureCount) as actualShipRate',
                    'if (expectedTemperatureCount is NULL, 0, expectedTemperatureCount) as plannedShipRate'
                )\
        .filter(f'minute_key = {curr_minute}')\
        .groupBy('id', 'warehouseid', 'minute_key').agg(F.expr('struct(MAX(CAST(minute_key AS INT)) AS time_key, MAX(actualShipRate) AS actualShipRate, MAX(plannedShipRate) AS plannedShipRate)').alias('time_array')).drop('minute_key')\
        .write\
        .format('cosmos.oltp')\
        .options(**cfg)\
        .mode('append')\
        .save()

Code Snippet
Add the code snippet that causes the issue.
Cosmos config
cfg = {
"spark.cosmos.accountEndpoint" : cosmosEndpoint,
"spark.cosmos.accountKey" : cosmosMasterKey,
"spark.cosmos.database" : cosmosDatabaseName,
"spark.cosmos.container" : cosmosContainerName,
"spark.cosmos.diagnostics":"simple",
"checkpointLocation":"/tmp",
"spark.cosmos.write.strategy": "ItemPatch",
"spark.cosmos.write.bulk.enabled":"true",
"spark.cosmos.diagnostics":"simple",
"checkpointLocation":"/mnt/nrtpoclanding/multi_window_agg/cosmos-test/checkpointLocation",
"spark.cosmos.write.bulk.enabled":"true"
}
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
Data to update the index as expected
Note we don't see any 429 or 500 issue on comsos side

@xinlian12  please take a look when you get chance, thanks!

@SomanathSankaran the issue is not clearly explained, can you expand? Please also see sample here on creating custom function to patch raw json using Spark Connector: https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/quickstart-spark?tabs=scala#raw-json-support-for-spark-connector.

hi @TheovanKraay  I was able to figure out the issue it is due  to spark object which is shared one and it is getting updated when we query if from multiple parallel threads as spark is shared.Meanwhile I will try the json tags


Create Python virtual environment

Not sure if this is correct (maybe needs to have if else conditions to check different paltform because of different commands), please help to review it.

Activating a venv already adds the python exe path to the env var PATH. I'm wondering if the venv needs to be activated for each step...

It looks like that did the trick! Now time to polish it up.

We probably don't want to pollute the users' workspace with our venv, so we could use a temp dir instead of the /users/runner/workspace. Github actions has a runner.temp variable that might be used to do this.
It would be best practice to use the venv on every platform, so no need to use bash if [[ ${{runner.os}} == 'macOS' ]].

It would be best practice to use the venv on every platform, so no need to use bash if [[ ${{runner.os}} == 'macOS' ]].

Windows will failed if not specified https://github.com/cpp-linter/test-cpp-linter-action/actions/runs/3270593867/jobs/5379399304 Maybe I can create a shell script to deal with all platforms.

That's because the activate command is located differently on Windows:
./venv/Scripts/activate

where as on Linux & macOS:
source ./venv/bin/activate

It might be better to have the venv creation/activation in a .sh script file, so we could just call it from any steps that use the venv.

You can use github.action_path to access venv.sh. Optionally, I would recommend using the env var RUNNER_TEMP in the venv.sh file to keep the venv located outside the users workspace.

I see that you're using the ${{ runner.os }} that isn't working in the venv.sh file. There's a RUNNER_OS env var that you can use in venv.sh.

Hmm. The venv executable path doesn't seem to be getting used after the venv.sh script completes. I'm looking at a similar approach that was done in arduino/compile-sketches composite action in which they directly invoke the executable located in the venv that they create with a bash script.

nope 😠 . It starting to seem easier to just manage the venv in actions.yml as we had before.

That will only work on Linux because they use the deadsnake PPA to install a certain build of python.

And remember that the set-outputs command is now deprecated.

We can probably combine the "create venv" step with the "install deps" step.

Now it seems to be having problems with the Windows path delimiters in a bash prompt on Windows:
D:\a\_actions\cpp-linter\cpp-linter-action\latest\venv\Scripts\activate

is executed in bash like escape chars:
D:a_actionscpp-lintercpp-linter-actionlatestvenvScriptsactivate

OMG! Why is this one of those things that is easier said than done?!
Sorry, but I'm going to sleep now. I'll try to pick this up tomorrow (& hopefully try resolve cpp-linter/clang-tools-pip#15 in a more platform independent way).

I've started trying to use actions/setup-python to create a venv that doesn't modify the workspace env, but I think we still need to address the cpp-linter/clang-tools-pip#15 so that cpp-linter can invoke the clang-*-<version> executable (or symlink).
The problem, as I see it, is that the python executable path is not guaranteed to be in the env var PATH. Meaning we have to choose a path that is guaranteed to be in the env var PATH when installing the clang-tools (or symlinks).

closing in favor of #113


Fix LD:json

:tada: This issue has been resolved in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


[Preview3.2]MySQL Lift Bug

When using MySQL as datasource,
executing  prisma2 lift save --name 'init', no error
executing prisma2 lift up --verbose with error:
Specified key was too long; max key length is 767 bytes, which is usually caused by long varchar,  usually more than varchar(255).

After investigating, I believe it's a prisma2 bug,
the generated README file in my migration file states that all my id fields decored with @id @default(cuid() or uuid()) are mapping to 'id' varchar(1000) NOT NULL, which will apparently cause the error I described at the beginning.

My Schema File

This is fixed in alpha. Will go to stable with preview6.


[K4] 4.0 Compatibility

Locale::getPrimaryLanguage(): Passing null to parameter #1 ($locale) of type string is deprecated

Seems to work fine out of the box.

Typographer 2 is in the works and will definitely work with Kirby 4.


Problem in integrating vaadin elements in seed project

Hello guys I am trying to integrate vaadin elements in in my seed project.I went through these links https://github.com/mgechev/angular2-seed/wiki/Add-external-dependency  &  https://vaadin.com/docs/-/part/elements/angular2-polymer/tutorial-index.html  and made changes accordingly but i am getting this error Error: Error: XHR error (404 Not Found) loading http://localhost:5555/node_modules/@vaadin/package.json(…)
Somebody please tell me why am i getting this error and what is the correct way to add polymer dependencies in my seed project.I have made a repo here https://github.com/abhibly/Angular2-Seed-Project demonstrating my issue.Thanks

For support requests you can use gitter or take a look at project's the wiki page in order to get more complete idea on how the it works.
Also duplication of #1106

@TheDonDope Hi...I went through the resources which you gave but couldn't succeed.For integrating vaadin-polymer in angular2 application i used to make the following changes in systemjs.config.js file
`  // map tells the System loader where to look for things
var map = {
'@vaadin':                    'node_modules/@vaadin'
};
// packages tells the System loader how to load when no filename and/or no extension
var packages = {
'@vaadin/angular2-polymer':   { main: 'index.js', defaultExtension: 'js' }
};`
I made the following changes in project.config.ts
   this.SYSTEM_CONFIG_DEV.paths['@vaadin'] =        ${this.APP_BASE}node_modules/@vaadin`;
this.SYSTEM_BUILDER_CONFIG.packages['@vaadin/angular2-polymer'] = {
    main: 'index.js',
    defaultExtension: 'js'
};`

and got this error
Error: Error: XHR error (404 Not Found) loading http://localhost:5555/node_modules/@vaadin/package.json(…)
What changes should i make in project.config.ts to make vaadin-polymer work in my angular-seed?Can you please show me the changes to make my vaadin-polymer work in seed-project

any progress on this ?

I am facing similar issue. Please let me know if you find a solution.

I made the following changes and now it is working for me:
    this.SYSTEM_CONFIG_DEV.paths['@vaadin/angular2-polymer'] =
      `${this.APP_BASE}node_modules/@vaadin/angular2-polymer`

    this.SYSTEM_CONFIG_DEV.packageConfigPaths =
      this.SYSTEM_CONFIG_DEV.packageConfigPaths.concat([
<EMAIL_ADDRESS>      ])

    this.SYSTEM_BUILDER_CONFIG.packages['@vaadin/angular2-polymer'] = {
      main: 'index.js',
      defaultExtension: 'js'
    }


Transform layers

This PR allows for the affine transform associated with the layer(s) to be updated when the coordinate system is changed. It will ignore layers that do not have an sdata object associated with them.
Furthermore, this PR includes a bugfix for the issue of  a KeyError being raised when the visibility of the layer is updated and the layer has no sdata associated with it.

Codecov Report
Patch coverage: 100.00% and project coverage change: +0.23% :tada:

Comparison is base (d674bcd) 74.70% compared to head (b5638d1) 74.93%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #146      +/-   ##
==========================================
+ Coverage   74.70%   74.93%   +0.23%
==========================================
  Files          16       16
  Lines        1605     1620      +15
==========================================
+ Hits         1199     1214      +15
  Misses        406      406

Files Changed
Coverage Δ

src/napari_spatialdata/_sdata_widgets.py
87.12% <100.00%> (+0.53%)
:arrow_up:

src/napari_spatialdata/_viewer.py
66.66% <100.00%> (+4.32%)
:arrow_up:

src/napari_spatialdata/utils/_utils.py
74.37% <100.00%> (+0.16%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Request for clarification: how to refresh local schema when schema changes on server?

Let's say I add a new field to my WordPress GraphQL schema, for example using the ACF plugin to add a new field.
Based on my observations, it appears necessary to exit development mode, run npm run clean to clear the local cache, and restart in dev mode from scratch. Is that correct?

Would also love to know this, i've been having to do the same. It becomes a right pain when you start having a reasonable amount of images that need to be rebuilt due to the cache being cleared.

That's true, it is indeed not really convenient to restart the server every time the field was added/deleted somewhere. By the way, as I know, cleaning cache is not required in such case
Also, I have to restart the server almost every time I change the branch :(

As of<EMAIL_ADDRESS>the schema updates automatically in development. Let me know if you run into any issues with it!


[WIP] Re-write test helper as an AVA macro

This is currently failing xo linting right now, due to this usage of macros + the ava/test-title ESLint rule.
I know that I can assign a function to urlMatcherMacro.title and the lint rule won't nag, but, for this scenario, that's not really needed.
@sindresorhus @jamestalmage Any suggestions? One thing I considered doing was removing the need to pass in the detectFn. Instead, we would use the first string passed in for the dynamic test title + as a key for looking up the detectFn on the pageDetect object. The snag there was that the first string arg passed to a test call gets passed to the title fn, but not to the macro itself. I could make the title/method name the last arg to the macro.

See https://github.com/avajs/eslint-plugin-ava/pull/112
Merged to master, but not published.

Nice. I saw that you had done that work and merged it in, just jumped to the assumption that it was already released. Thanks!

Should be published now

@jamestalmage Thanks! That fixed it

Looks good :)


webmin configuration two-factor authentication

When selecting either two factor option the input textbox doesn't show for API key.

Alright, thanks! I fixed it. It will be working in version 9.0.0. If you sure that the field for the key should also appear for Google Auth, please address this issue to Jamie Cameron, because it doesn't work in the default theme as well.


Fixups for MIPS and callgrind format

Fixed typo in passing arguments to _syscall6 - it is correctly stated in comment that old MIPS ABI use only 4 registers for argument passing, but the code was trying to pass arg5 and arg6 as registers too.
Also, fixed printf misuse in pprof - the printed string is passed as format. It can cause troubles if string contains "%".

Applied. Thanks a lot.


Allow each content section to have a description

This will be useful for the declaration, where in the guidance each page has an introductory paragraph. Being able to have this in the interface might mean we don't have to publish the documentation.
Suggested usage, manifest.yml:
-
  name: Grounds for mandatory exclusion
  description: You must be able to answer all of these questions
  questions:
    - …

:+1:
and :+1: to not publishing guidance.


WIP: Fixed support for linux environment variables

dot (.) is a shell limitation, changing it to underscore.
Stackoverflow reference: https://stackoverflow.com/questions/57110682/travis-ci-environment-variable-key-with-dot?noredirect=1#comment100741182_57110682

Closing as this will not work because the script was tied up on local.properties


Chef-12: backport 6356

backport of #6356

similarly the travis failures are preexisting issues on the chef-12 branch


Invalid characters allowed in mobileNumber Field of /notifications/api/v1/orders Endpoint

Description
The POST /notifications/api/v1/orders endpoint does not properly handle invalid characters in the mobileNumber field. For example, submitting a value like "+47invalidChar" results in an internal server error (500) without providing a meaningful error message to the user.
Steps to Reproduce

Send a POST request to the /notifications/api/v1/orders endpoint with a mobileNumber field containing invalid characters (e.g., "+47invalidChar").
Observe that the API responds with an internal server error (500) instead of rejecting the request with a meaningful validation error message.

Example Request:
{
    "notificationChannel": "Sms",
    "smsTemplate": {
        "senderNumber": "+4746269164",
        "body": "Hei, Du har en ny melding i Altinn."
    },
    "sendersReference": "907c8054-b2cc-4dca-b4df-c95b89860762",
    "recipients": [
        {
            "mobileNumber": "+47invalidChar"
        }
    ]
}

Expected Behavior
The API should validate the mobileNumber field to ensure it adheres to valid international phone number format standards. If invalid characters are submitted, the API should reject the request with a descriptive error message, such as "Invalid mobile number format," preventing further processing.
Suggested Fix
Introduce or improve input validation on the mobileNumber field to ensure it only accepts valid numeric characters following international phone number standards. When validation fails, return a clear and informative error message, rejecting the request before it reaches internal processing.
Acceptance criteria:

[ ] Validation-scheme documented at docs.altinn.studio
[ ] Failed validations give response-code 400 and a message (e.g. "mobileNumber format not adhering to...")

Updating issue to be "adhere to validation standard + document this", to avoid confusion on permitted content. Feiled validations should give 400 and a sensible message.
See: https://altinn.slack.com/archives/C069J71UQCQ/p1727789663181299?thread_ts=1727703986.978639&cid=C069J71UQCQ

Skjønner, siste klarering da: Jeg får 400 bad request på visse telefonnumre og formater, kan du si hvilken standard dere bruker for å validere gyldige numre?
Ut ifra testingen min så virker det til å være E.164

Our initial guess was correct - the 500 error results from the validation package not handling the letters

Manual tests performed and changes approved.


Only show nice menu options when user is logged in

⬆ What the title said

Looks good :neckbeard:

:+1:


Fix/snowflake source quoting

Information Schema queries for Sources
This PR revamps how dbt handles identifier quoting for sources. It addresses https://github.com/fishtown-analytics/dbt/issues/1317, but also addresses another 0.13.0 bug that we don't currently have an issue for. This second bug was way gnarlier than #1317, and the majority of this PR is intended to fix that bug
Source quoting
In 0.13.0, dbt was modified to better support custom databases. As a part of this change, an
information_schema_name macro was added which, given a database literal, returns the database and schema for the relevant information_schema.
This macro uses the project-wide quoting config to determine if databases should be quoted. This is the difference between:
select * from "raw".information_schema.tables

and
select * from raw.information_schema.tables

There is no difference between these two queries on pg/redshift (and we don't use BQ's information schema), so the effects of this macro are localized to Snowflake. The key problem here is that dbt is using project-level quoting configs to determine if it should quote source databases. While these seem similar, they're actually super different. Because dbt uses the quoting config to both 1) build and 2) query relations, it is internally consistent. For source databases however, dbt's quoting config has no bearing at all on whether or not the source database should be quoted! The existing information_schema_name logic would effectively make it impossible to address a lowercase+quoted source database on snowflake if database quoting was turned off for the project. This PR divorces the logic of generating information_schema relations for sources from the quoting config specified in the dbt_project.yml file.
Two approaches
There are two approaches we can take here:

quote the database/schema/identifier exactly as supplied in schema.yml files
do not quote sources automatically, and instead force the user to quote their identifiers twice in schema.yml files (the one implemented here)

The first approach is implemented in the first commit for this PR, and it worked! It also resulted in a pretty unpleasant experience for Snowflake users. This required users to write code like:
version: 2
sources:
  - name: snowplow
    database: RAW
    schema: SNOWPLOW
    tables:
      - name: event
        identifier: EVENT
      - name: web_page
        identifier: WEB_PAGE

Effectively every table name would need to be duplicated in order to provide a sane (lower case) name and correct upper-cased identifier. The 90% case for snowflake sources is that they are uppercase/unquoted, so it didn't feel right to impose this burden on dbt users.
The second approach results in a much nicer UX at the expense of some... not ideal.... logic in dbt. This makes the users with troublesome source data identifiers feel the pain, which is probably how it should be :). In the approach implemented in this PR, dbt inspects the specified identifiers for sources. If the identifiers include quotes, then dbt will strip those quotes and record that the given identifier should be quoted in information_schema queries. In this version, the above schema.yml can be written as:
version: 2
sources:
  - name: snowplow
    database: raw
    tables:
      - name: event
      - name: web_page

Much nicer! If source tables need to be quoted (b/c they include illegal characters, or are lowercased on Snowflake) users can do something like:
version: 2
sources:
  - name: snowplow
    database: '"raw-database"' # example database that needs to be quoted
    tables:
      - name: event

When dbt tries to query the raw-database information schema, it will render SQL like:
select * from "raw-database".information_schema.tables

So, i think this checks all the boxes:

mostly transparent to users
databases that must be quoted are now addressable by sources
the adapter contract hasn't really changed, which is a nice plus

There's some cleanup work to do here, but wanted to get some initial feedback as I've been chasing my tail here for a couple of days now.
TODO:

lots of tests
update the presto catalog query to remove this line of code
improve error messages

The behavior in this PR is implemented by #1338, closing this one accordingly


If no keyboard shortcut is set, display help about how to set it

Fixed in https://github.com/cheshire137/hubnav/commit/1b64a1e224016111c0874c893a20e72c5a400731.


About Page

Lets make a page describing the network monitor similar to https://eosrate.io/about
Monitoring what is happening on the network is essential to improve it and measure the impact of our solutions. The main objective of the monitoring tools is to produce decentralized and reliable information about each node and the activity of the network.

Maybe we can use text from the following link:
https://latamlink.io/docs/monitoring


Simplifying cartoon figures

Resolving issue 533

I think your branch is not up to date @jclagarde
And should have only done this for figures 2 and 3.

I can take it from here. Thanks.


Update fr.json

A new version with wilcard
i hope is good

Yes looks good, thank you for fixing it!


[not for review] Testing mellanox ci

Thank you for your PR,
You can test hw-offload configuration by commenting one of the following:

/test-hw-offload.
/test-all.

Alternatively you can skip the tests by commenting:

/skip-hw-offload.
/skip-all.

These commands can be run by anyone.
Best regards.


Campaign list doesn't show buttons in narrow view

On the responsive view, only the title shows. You can see the bottom border actually finishes halfway through the list item styling, so the space for buttons is there - they're just hidden.

/cc @clarkepaul

Pull request available for fix #5446


test: add test for bare-metal deployer

related: #94

Codecov Report

Merging #125 (51ff550) into develop (eb80cbd) will increase coverage by 6.17%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #125      +/-   ##
===========================================
+ Coverage    22.02%   28.19%   +6.17%
===========================================
  Files            8        8
  Lines          940      940
===========================================
+ Hits           207      265      +58
+ Misses         677      613      -64
- Partials        56       62       +6

see 1 file with indirect coverage changes


Allow specifying target directory for DOCS-BUILDER:BUILD

Currently there's no way to influence where BUILD will write generated docs, and it just assumes it knows where to write them, which is pretty inconvenient and inflexible. It also does the guessing separately for each and every builder, meaning they can pick different output directories, and it's not documented in any way, so there's no way for the user to know, other than to let it write wherever it wants and look at the returned path.

Sadly, it is not always possible to make some documentation generator to use a particular directory for output.
Probably, we could add an optional keyword argument and if it is given, then to move documentation to the given folder.
I think, this might be a good "first issue" to solve! Would you like to make a pull-request?

Yup, I can do that.

@mathrick did you solve this problem for yourself somehow?


UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 19: invalid start byte

Describe the bug
When I attempt to process an EML messages using the html.py  i get this error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 19: invalid start byte
from
File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\unstructured\cleaners\core.py:137, in replace_mime_encodings(text)
130 def replace_mime_encodings(text: str) -> str:
131 """Replaces MIME encodings with their UTF-8 equivalent characters.
132
133 Example
134 -------
135 5 w=E2=80-99s -> 5 w’s
136 """
--> 137 return quopri.decodestring(text.encode()).decode("utf-8")
Im actually calling it from Langchain, but my guess is that its using HTML.py
To Reproduce
You would need an email file. I can provide if required.
Expected behavior
Not throw an exception
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. windows, mac, linux]
Browser [e.g. chrome, safari]
Python version [e.g. 3.8.15]

OS: windows
Python: 3.10
Additional context
My guess is this is related to
TODO(robinson) - There's likely a cleaner was to accomplish this and get all of the
unicode characters instead of just the quotes. Doing this for now since quotes are
an issue that are popping up in the SEC filings tests
One way to fix be to allow other decoder arguments other utf-8
return quopri.decodestring(text.encode()).decode("utf-8")
I believe that
https://stackoverflow.com/questions/46000191/utf-8-codec-cant-decode-byte-0x92-in-position-18-invalid-start-byte
says to  use ('ISO-8859-1')

I cloned the project and made  that change and it worked.

@mbalsam-refind - Opened #453 to allow for passing through an encoding kwarg. I was trying to reproduce the UnicodeDecodeError exception but couldn't find a string that tripped it. Do you have an example of a string that raises the exception so we can add a test?
Thanks for reporting this issue!

Nevermind we're all good, found a good input to test against. We'll push the fix out in the next release.

I think i found the file that does it.  I don't know if i can get a string.   I'll stop digging now.  But if you need it i can find it.
Im using this from LangChain.  How would the author of LangChain add this fix, or could I being a user of LangChain.
FYI, new to python.  :)

The UnstructuredFileLoader (source is here) in langchain accepts optional unstructured kwargs, so you can instantiate it like this:
loader = UnstructuredFileLoader("my-email.eml", encoding="ISO-8859-1")
loader.load()

if you need to change the encoding. There also the email loader but that doesn't accept optional kwargs right now (we can PR that in tomorrow though, it's the same pattern as for the file loader).

Great!  I searched for a code that would convert MSG to EML.  So now I can remove that step.  Thanks!

This bug also happens with the LangChain Sitemap loader, which loads multiple file (mime) types, and there is no way for me to know in advance what encoding that is. Any hope in this case? A better auto detection?

@sglebs - Are you using unstructured code in conjunction with the Sitemap loader? We do have functionality now to detect encoding types. If you're error originals with our partition_email or another one of the unstructured functions, we likely just need to patch that in.
https://github.com/Unstructured-IO/unstructured/blob/6ec177e7c6d8e1b6c4eb88e8ebababa3750e606d/unstructured/file_utils/encoding.py#L52

@MthwRobinson yes, Sitemap loader with unstructured. I can force a specific minimal version of unstructured in my requirements.txt and give it a try. What minimal version should I use? I am running unstructured==0.7.8 . Thanks!

Could you try with 0.7.12? If you're getting errors with the latest version and can let us know what filetypes and/or MIME types are producing the errors, we can take a look. Thanks!

Still happens. Maybe the page is malformed? This is the URL that crashed SitemamLoader: https://digix.com.br/quem-somos/

Thanks, we'll check that out and see if we can reproduce.

Any news on this one? I still get the error on https://digix.com.br/quem-somos/ . Thanks for listening.

Hi @sglebs Which partition function did you try? When I tried partition_html, I didn't get any error.
elements = partition_html(url="https://digix.com.br/quem-somos/")
print("\n\n".join([str(el) for el in elements]))

I just use Langchain Sitemap Loader. I do not call unstructured directly at all.


KAFKA-6630: Speed up the processing of TopicDeletionStopReplicaResponseReceived events on the controller

This patch tries to speed up the inefficient functions identified in Kafka-6630 by grouping partitions in the ControllerContext.partitionReplicaAssignment variable by topics. Hence trying to find all replicas for a topic won't need to go through all the replicas in the cluster.
Passed all tests using "gradle testAll"
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@onurkaraman Can you please take a look at this patch? Thanks.

@onurkaraman @junrao Can you please take a look at this patch? Thanks!

@gitlw, are you planning to address the review feedback from @junrao?

@junrao Can you please take another look? Thanks!


fatal: could not read Password for<EMAIL_ADDRESS>No such device or address

I followed the GitHub Action in README, and wrote this workflow for my repository:
name: Publish New Version

on:
  release:
    types: [ published ]

jobs:
  update_wiki:
    name: Update Wiki
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repository
        uses: actions/checkout@v2
      - name: Generate Documentation
        uses: SwiftDocOrg/swift-doc@master
        with:
          inputs: "./Sources"
          module-name: Interval
      - name: Upload Documentation to Wiki
        uses: SwiftDocOrg/github-wiki-publish-action@v1
        env:
          GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Its latest log shows that it fails to update the documentation to the wiki page:
Initialized empty Git repository in /tmp/ci-XXXXJbcEHJ/.git/
From https://github.com/WowbaggersLiquidLunch/Interval.wiki
 * branch            HEAD       -> FETCH_HEAD
cp: can't stat '/README.md': No such file or directory
On branch master
nothing to commit, working tree clean
fatal: could not read Password for<EMAIL_ADDRESS>No such device or address

My workflow deviates from the README's only in 2 places:

It uses actions/checkout@v2 instead of v1.
It uses the default secrets.GITHUB_TOKEN instead a custom secrets.GH_PERSONAL_ACCESS_TOKEN.

Judging from the error message, I assume it might have something to do with the access token? If this is the case, what permissions should I give to the token for the GitHub action to work properly?

It uses the default secrets.GITHUB_TOKEN instead a custom secrets.GH_PERSONAL_ACCESS_TOKEN.

GH_PERSONAL_ACCESS_TOKEN is required for https://github.com/SwiftDocOrg/github-wiki-publish-action. GITHUB_TOKEN isn't sufficient.