android.permission.INTERNET permission

Documentation says:
https://flutter.dev/docs/deployment/android
" Remove the android.permission.INTERNET permission if your application code does not need Internet access."
I did not remove anything from my
/android/app/src/main/AndroidManifest.xml
but there was no line by default
<uses-permission android:name="android.permission.INTERNET" />
in /android/app/src/main/AndroidManifest.xml
Could you please change the code or documentation, so that they match
( I was wondering why my app in play store could not connect to internet...)
I'm using ubuntu 18.04
Focusing android only
Terveisin, Markus
flutter doctor -v
[✓] Flutter (Channel stable, v1.2.1, on Linux, locale en_US.UTF-8)
    • Flutter version 1.2.1 at /home/mka/bin/flutter
    • Framework revision 8661d8aecd (3 months ago), 2019-02-14 19:19:53 -0800
    • Engine revision 3757390fa4
    • Dart version 2.1.2 (build 2.1.2-dev.0.0 0a7dcf17eb)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /home/mka/Android/Sdk
    • Android NDK location not configured (optional; useful for native profiling
      support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /home/mka/bin/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build
      1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[!] Android Studio (version 3.3)
    • Android Studio at /home/mka/bin/android-studio
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build
      1.8.0_152-release-1248-b01)

[✓] VS Code (version 1.32.3)
    • VS Code at /usr/share/code
    • Flutter extension version 2.25.1

[✓] Connected device (1 available)
    • moto x4 • ZY224J8WM2 • android-arm64 • Android 9 (API 28)

! Doctor found issues in 1 category.

Try looking "/android/app/src/profile/AndroidManifest.xml",
or search in your project for android.permission.INTERNET.

Terve, @paapu88. See https://github.com/flutter/flutter/issues/20789

Thank you @artob ! I think it's safe to close the issue. android.permission.INTERNET is automatically added to debug and profile.

@BondarenkoStas this permission in debug and profile manifest is used for hot reload and sync files.


The public keys generated for the twin by the mnemonic and its seed are different

same as https://github.com/threefoldtech/rmb-rs/issues/181
it's happening here
https://github.com/threefoldtech/tfgrid-sdk-ts/blob/14a9eafa5b722c88d5653acc717a3f97da3c9d36/packages/rmb_direct_client/lib/util.ts#L20

is there a way i can verify this ?

Verification Steps

Create an RMB client using your mnemonic then connect it. (you can check the example script in the RMB direct client)
Check the pk field on your twin on TFChain.
Get the hex seed of your mnemonic.
Use the hex seed to create an RMB client instead of the mnemonic then connect it.
Check the pk field on your twin on TFChain again (it should be the same).

already verified and the pk on the twin doesn't change


Exoscale provisioning

Created node template and entered Exoscale API key and secret. When I try to create a cluster on Exoscale, based on this template, I get following error:

Error setting machine configuration from flags provided: missing an API key (--exoscale-api-key) or API secret key (--exoscale-api-secret-key); Timeout waiting for ssh key

Unsucessfully Tried multiple keypairs & multiple accounts.
Docker-machine provisioning from command line works. Rancher UI fails.

Useful
Info

Versions
Rancher v2.0.0 UI: v2.0.41

Access
local admin

Route
authenticated.cluster.nodes.index

Problem still exists in v2.0.2

Same here with v2.0.6

Oh crickey. We made changes in the 1.6 branch which aren't as of yet been ported to the 2.x world. We'll take care of that soonish.

As the "node provider" still uses docker-machine which is in a bugfix mode for a long time already. It's probably easier to rely on the bring your own nodes cluster setup.

A working cloud-init could be the following one.
#cloud-config

manage_etc_hosts: true

apt_sources:
- source: "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable"
  keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88

package_update: true
package_upgrade: true

packages:
- [ docker-ce, "17.03.2~ce-0~ubuntu-xenial" ]

runcmd:
- [ usermod, -aG, docker, ubuntu ]

NB: we are working to bring back the node provider as well.

(same here)
When will the changes be available in Rancher - is the node driver dynamically loaded from somewhere or do we have to wait for a new Rancher version?
Will it be possible to select a zone/location (here, it selected "CH-DK-2" without my intervention - I'd like to control where my data stays - otherwise I can as well use any infrastructure provider).
How can I use the so-called "cloud-init" given above - if possible with a given zone?
Is terraform currently working? Then I could at least try to use iac to provision my environment,

@Remigius2011 some changes have been merged to the ui, https://github.com/rancher/ui/pull/2132 which enables the zone selection.
As for the terraform provider, it should work. https://github.com/exoscale/terraform-provider-exoscale/tree/master/examples/rke
We are in the process to get the provider moved upstream but things may take some time as hashicorp is problably focused on their 0.12 release. Cheers,

@fretscha do you still have the issue using rancher/rancher:v2.0.7-rc2 ?

Is this supposed to be fixed in 2.0.7?

@greut  The issue of the box setup looks like to be resolved in rancher/rancher:v2.0.7-rc2.
Some firewall rules (sg) are still not complete / missing.

@fretscha I have just retestet with rancher/rancher:v2.0.8. Unfortunately, exoscale provisioning still does not work for me. I think I am not doing anything exotic - just a single node, medium, 50G, ch-dk-2. The log is here (remark1: this is just one iteration - it retries to provision in a loop which provides the same entries except for the time stamps, remark2: I have replaced potentially sensitive data like UIDs, IP addresses etc. with start, remark3: sorry, it looks like the intended formatting is broken by github):

2018/08/27 08:41:18 [INFO] stdout: Creating CA: management-state/node/nodes/wrk-1/certs/ca.pem
2018/08/27 08:41:18 [INFO] stdout: Creating client certificate: management-state/node/nodes/wrk-1/certs/cert.pem
2018/08/27 08:41:19 [INFO] stdout: Running pre-create checks...
2018/08/27 08:41:19 [INFO] stdout: Creating machine...
2018/08/27 08:41:19 [INFO] stdout: (wrk-1) Querying exoscale for the requested parameters...
2018/08/27 08:41:20 [INFO] stdout: (wrk-1) Generate an SSH keypair...
2018/08/27 08:41:20 [INFO] stdout: (wrk-1) Spawn exoscale host...
2018/08/27 08:41:20 [INFO] stdout: (wrk-1) Deploy &egoscale.DeployVirtualMachine{Account:"", AffinityGroupIDs:[]string{}, AffinityGroupNames:[]string(nil), CustomID:"", DeploymentPlanner:"", Details:map[string]string(nil), DiskOfferingID:"", DisplayName:"wrk-1", DisplayVM:(bool)(nil), DomainID:"", Group:"", HostID:"", Hypervisor:"", IP4:(bool)(nil), IP6:(bool)(nil), IP6Address:net.IP(nil), IPAddress:net.IP(nil), IPToNetworkList:[]egoscale.IPToNetwork(nil), Keyboard:"", KeyPair:"docker-machine-wrk-1", Name:"wrk-1", NetworkIDs:[]string(nil), ProjectID:"", RootDiskSize:50, SecurityGroupIDs:[]string{"}, SecurityGroupNames:[]string(nil), ServiceOfferingID:"", Size:0, StartVM:(bool)(nil), TemplateID:"", UserData:"", ZoneID:""}
2018/08/27 08:41:41 [INFO] stdout: (wrk-1) IP Address: ..., SSH User: ******
2018/08/27 08:41:44 [INFO] stdout: Waiting for machine to be running, this may take a few minutes...
2018/08/27 08:41:44 [INFO] stdout: Detecting operating system of created instance...
2018/08/27 08:41:44 [INFO] stdout: Waiting for SSH to be available...
2018/08/27 08:41:45 [INFO] stdout: Detecting the provisioner...
2018/08/27 08:41:45 [INFO] stdout: Provisioning with ubuntu(systemd)...
2018/08/27 08:41:56 [INFO] stdout: Installing Docker...
2018/08/27 08:42:11 [INFO] stdout: Copying certs to the local machine directory...
2018/08/27 08:42:13 [INFO] stdout: Copying certs to the remote machine...
2018/08/27 08:42:14 [INFO] stdout: Setting Docker configuration on the remote daemon...
2018/08/27 08:42:17 [INFO] stdout: Checking connection to Docker...
2018/08/27 08:42:40 [INFO] stdout: The default lines below are for a sh/bash shell, you can specify the shell you're using, with the --shell flag.
2018/08/27 08:42:40 [INFO] stdout:
2018/08/27 08:42:40 [INFO] Generating and uploading node config wrk-1
2018/08/27 08:42:40 [ERROR] NodeController c-/m-** [node-controller] failed with : Error creating machine: Error checking the host: Error checking and/or regenerating the certs: There was an error validating certificates for host "...:": dial tcp ...:**: i/o timeout
2018/08/27 08:42:40 [INFO] NodeController c-/m- completed with dropped err: [node-controller] failed with : Error
2018/08/27 08:42:41 [INFO] NodeController c-/m- completed with dropped err: [node-controller] failed with : Error

@fretscha we will take a look about those missing rules.
@Remigius2011 where do you run rancher from? The nodes have to must be able to talk to it. It's a :chicken: and :egg: problem where you need a public instance to spawn more instances.
I have an instance running rancher 2.0, that I access to via its IP address and http://xip.io/. Then that instance spawned the following cluster.

@greut my rancher runs on a machine that is accessible via an nginx reverse proxy - configured according to rancher recommendations - listening on a public IP having a public DNS name. I am able to provision clusters on digitalocean with this setup. If necessary, I can provide more information over a private channel.

@Remigius2011<EMAIL_ADDRESS>would be the perfect channel for this. Thanks!

@greut Thanks for your fast and accurate support via<EMAIL_ADDRESS>As you mentioned in the answer to my support call, the reason was a pre-existing security group rancher-machine which did not contain the correct rules. After deleting it, I could successfully in create a cluster (just a single node for testing) using the exoscale provider.
One minor remark: in the list of node templates the location is displayed as "Unknown", as follows:

Obviously, it would be nice to see the chosen location.

there is some port change
https://github.com/rancher/ui/pull/2194

Please let us know in a comment if you still can not add a cluster with Exoscale node template , and we'll reopen it
For the the location is displayed, please submit a new issue.
Thanks a lot!

Hi, same problem again with rancher 2.1.1. I was unable to delete the rancher-machine security group in the exoscale gui. Specifying another one to be created by the node driver (I would not know the rules to create one from scratch...) is also not supported.

Meanwhile I was able to regenerate the rancher-machine security group, but this did not fix the problem this time. I am always stuck in a loop while provisioning the cluster.

see also my exoscale Support Ticket #1023093.

As mentioned in my last message before the issue was reopened (and in the support ticket), I managed to regenerate the rancher-machine security group.
The Exoscale node driver works in 2.0.8, but ceased again to work in 2.1.1 (I didn't check 2.1.0 as this was never a stable version).
Please check also whether the code that fixed the problem was cross-merged to the 2.1.x, 2.2.x and main branches (and whatever branches are necessary to make the fix available in future releases).

It looks like it is not really dependent on the rancher version. I have now a new rancher v 2.0.8 cluster running which fails with the same pattern, whereas in the other - also v 2.0.8 - it works.

I'm about to check differences in network configuration etc. - I'll keep you posted on the result.

I can add a cluster with exoscale node template successfully in v2.1.1
I guess if the user who upgrades Rancher from v2.0.X might have the rancher-machine security group issue.
Workaround is
Option 1: Please see https://github.com/rancher/rancher/issues/13291#issuecomment-431868108
Option 2: Delete rancher-machine in exoscale console and create a node template in UI again. It will send out request to create a new rancher-machine group with proper rules
Options 3: Config your own security group in  exoscale console with the rules https://user-images.githubusercontent.com/15922119/44715367-ec993200-aab7-11e8-83f1-5c7dfefabbb9.png and select it as the security group in Rancher UI

@loganhz thanks for your suggestions.
Since yesterday evening (see my comment above) I cannot exclude it is a network problem, as it also occurs on a v2.0.8 rancher that I have installed on the same machine. I have already tried your suggestions 1 and 2. I have also meanwhile checked the following configurations on both rancher nodes without noticing a difference that could lead to the observed behavior, i.e. one node can create clusters and the other fails (as a side note that may make everybody go ahhh! I must tell both rancher nodes run behind an nginx reverse proxy):

/etc/network/interfaces on the rancher nodes
proxy environment variables on nodes (identical) and inside rancher containers (both none)
nginx site configuration (identical except for the host names and the certificates)
host certificates (identical except for the key pair)
rest of nginx config (no asymmetries between the nodes found)
docker command line to start rancher (identical)
firewall configuration (both IP addresses belong to the same alias and are thus treated equal)

Also, as I have discovered this morning, the problem is not restricted to Exoscale, also the creation of a digitalocean cluster fails on the new rancher server (albeit with a different error message).
I will try now to set up a custom cluster and see whether I find a more detailed error message on the log output of the cluster node.

the behavior with custom clusters is at least consistent with node-provider provisioned ones: it succeeds on the 'good' node and fails on the 'bad' one - with:
[INFO] cluster [c-6glbm] provisioning: [healthcheck] Start Healthcheck on service [kube-apiserver] on host [x.x.x.x]

then a little while later:
[ERROR] cluster [c-6glbm] provisioning: [controlPlane] Failed to bring up Control Plane: Failed to verify healthcheck: Failed to check https://localhost:6443/healthz for service...

at least this mostly excludes that it is the same problem as in the beginning of this issue - so I'll stop reporting and continue my experiments silently...

Can you try to clean up the 'bad' node first and re-add it again, please?
You can find the clean up script via https://gist.github.com/superseb/2cf186726807a012af59a027cb41270d

tl;dr - I think you can close this issue (as far as I am concerned).
The problem problem is solved: as I already suspected some time before, it was not the exoscale node driver, but last night discovered a problem in the nginx site configuration of the reverse proxy (aka external load balancer). I am adding the details here - even though it does not really fit into this issue - for other people arriving here via search engine who might have a similar problem.
As I had a running instance, I was assuming the new instance would work with the same nginx configuration, but the running one was stepwise upgraded from v2.0.2 to v2.0.8, so it had a different history than the new one and hence a different state.
In the server sections for http and https ingress traffic, I had - historically - listen statements not only for ipv4, but also for ipv6, e.g.:
  listen [::]:443 ssl http2;

As soon as I have removed them, it worked (and still works).
Also, as suggested here I added the parameter --no-cacerts to the docker run command that starts the rancher server (but I did not retest to verify whether this had any influence - initially it didn't but adding it will not hurt if you have an ssl certificate on the reverse proxy).
The correct configuration is described here and it may make sense to check once in a while for changes and align even working configurations with this example.
Thanks and sorry to everybody who has helped me to chase the phantom and solve this riddle.

Thanks for the update! Close this issue


ipsec: arp caching problem

Available in rancher/net:v0.11.0 (ipsec template version v0.0.5)  and rancher/network-manager:v0.6.0 (network-services template version v0.1.2)

Closing as per the findings from https://github.com/rancher/rancher/issues/8412#issuecomment-291932089.


institution_id -- add official insititute URL?

Hello,
ES-DOC would like to display an official URL for an institute on the further_info_url pages, e.g.   https://furtherinfo.es-doc.org/CMIP6.MOHC.HadGEM3-GC31-HM.abrupt-4xCO2.none.r1i1p1f1 These pages currently use a URL that seems sensible to us, but may not be what the modelling group would want. Note that this institute URL is a general one, and so is distinct from  the so-called  "further further_info_url", which relates to a CMIP6-specific page, which will be collected independently by ES-DOC.
We wondering if a) CMIP already has such general URLs and b) if not, would CMIP like to collect them and store them in this CV?
If the answer to both of these questions is "no", then there is no problem, as ES-DOC will simply  collect the information from the ES-DOC liaison at each institute, but if "yes" to either of them, we can of could harvest the information from the CMIP6 CV.
Many thanks,
David Hassell

Hi David,
We could add it to CMIP6_institution_id.json, but that would require modifying that file from being a simple dictionary.  Also, I would say the url might be somewhat less permanent than the full institution name and address, so the CV's would need to evolve (which is not a good thing).  So, currently I think the answer is "no" to both questions and you should feel free to obtain the information from the modeling groups for es-doc.
Any thoughts, Paul?
Karl

@davidhassell thanks for this, it's on the to-do list and this issue is a dupe of #379

Hi Karl and Paul,
I was just writing back to say "ES-DOC will get these, no need to change on the CMIP6 side" when I saw Paul's reply. I guess I should have known about #379, as it came from es-doc ...
So does that mean the answer is in fact "yes" to both questions (I couldn't see a PR relating to #379)?
David

This is a dupe of #379
@davidhassell we needed to reconfigure a number of things on this side at the same time that the CV update is undertaken, as there was no immediate need I had delayed this, but it seems this is now an ES-DOC requirement (correct?) which will pull it back up the priority list

Hi Paul,
It is an ES-DOC requirement, but only one which we don't need until "later in the year", when we are scheduled to go back to the ES-DOC liaisons to gather Experiment documentation. So - needed, but not desperately urgent.

@davidhassell "later this year" was about the timeline that I was working to implement the change - just so we're aligning calendars, what month/when is "later this year", I was thinking post-June..

@durack1 Yes, most definitely post-June

@davidhassell I will close this issue as a dupe, and we can continue discussions in #379 as that's where we all seem to be coalescing


Add default format 'http-stream' when creating a function.

Fix: #455 Creating a function does not default to "http-stream" format

+1 at this point we can probably say 'use old cli' - am in the process of removing on server side this week

@JadeRedworth I think we can close this and make a PR to remove format from CLI altogether? atm API will just ignore it, so it's fine, but we can get rid of it now.

feel free to re-open / update* however you'd like to do things...


Store a copy of dataset and weight on BaiduYun

Due to well-known reasons, google drive is difficult to access in China. Can you store a copy of the dataset and weight file on BaiduYun (A web drive in China mainland)? It will be helpful for us. Thanks a lot!

I uploaded to BaiduYun. But it limited to share the link. If you still need it, you could send me a email.

从谷歌上下载下来，解压后 只有一个13G+ 的大文件

@zhuohoudeputao, sorry for the late reply. I've updated a BaiduYun link in README.md. Please check it out.
@Johnzdh, thank you for spreading the dataset!
@ghzhangnj 你解压的方式对吗? 可以在命令行里试一下tar -xvf CAPS-MegaDepth-release-light.tar，解压出来应该是一个文件夹，包含图片和标注等等。

谢谢， 可以了


Support bind parameters.

Support bind parameters. Proposal included.

 (8) Bind Parameters

Added the do-not-merge to hold onto this until after we have branched for 3.7 release.


Type FIle (Rack::Multipart::UploadedFile) replace all non-English letters in file name with underscores ( _ )

Hello, I'm notice, that grape File type validation replace all non-English letters with an underscore symbol
I send:
------WebKitFormBoundarybo7OclPI12IMowot Content-Disposition: form-data; name="file"; filename="привет-hello.png" Content-Type: image/png
and on endpoint I have next validation:
params do
  requires :file, type: File, allow_blank: false, desc: 'Image'
end

but in params I see:
{"file"=>   {"filename"=>"\xD0\xBF\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82-hello.png",    "type"=>"image/png",    "name"=>"file",    "tempfile"=>#<File:/tmp/RackMultipart20200710-8-11mcjd3.png>,    "head"=>"Content-Disposition: form-data; name=\"file\"; filename=\"\xD0\xBF\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82-hello.png\"\r\nContent-Type: image/png\r\n"} }
And this "\xD0\xBF\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82-hello.png" it same as  "____________-hello.png"

Looks suspicious. Can you please try turning this into a spec, @alexgonor ?

@dblock I'm not quite sure what you want to see, can you clarify the request, please?

You're saying that Grape file validation is mangling data. I am pretty sure it does not. Write a spec that demonstrates your problem - AFAIK what you're seeing is some kind of encoding issue and maybe Grape is converting unicode characters into some other kind of encoding and that could be a bug.

Ok, this is simple Rails app with grape, you can send file with non-English letters in file name to PUT localhost:3000/ and in params you will be see somthing like this:
Started PUT "/" for ::1 at 2020-07-13 11:14:54 +0300 {"привет-hello.png"=>{"filename"=>"\xD0\xBF\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82-hello.png", "type"=>"image/png", "name"=>"привет-hello.png", "tempfile"=>#<Tempfile:/tmp/RackMultipart20200713-32573-14kbwtr.png>, "head"=>"Content-Disposition: form-data; name=\"\"; filename=\"\xD0\xBF\xD1\x80\xD0\xB8\xD0\xB2\xD0\xB5\xD1\x82-hello.png\"; filename*=\"UTF-8''%D0%BF%D1%80%D0%B8%D0%B2%D0%B5%D1%82-hello.png\"\r\nContent-Type: image/png\r\n"}}

I can take a look when I have a chance. To fix the issue faster:

Check out this project.
Add a spec somewhere in spec/** that reproduces this behavior.
Make a pull request with the failing spec.

@alexgonor I'm having the same issue, have you managed to fix it?

I figured out what was the reason: it is a combination of Grape + Carriervawe. Curious moment that we can explore on this test.
When we use кириллица.png file, then params[:file][:filename] on line 1345 returns кириллица.png
When we use кириллица.xls (real excel file) it returns escaped filename (I suppose using CGI.escape):
"\xD0\xBA\xD0\xB8\xD1\x80\xD0\xB8\xD0\xBB\xD0\xBB\xD0\xB8\xD1\x86\xD0\xB0.xls"

This is the first part. The second part is actually sanitization performed by Carrierwave. It simply replaces all evil characters as (_)
The only question for me, why behavior is different for different files


.scrollTo()/.filter() issue with .material_select()

Hi,
Was investigating on a strange error with .material_select():
Uncaught TypeError: Cannot read property 'top' of undefined
 a.fn.scrollTo                                                  materialize.js:2
 a.ready.a.fn.material_select.a.each.activateOption             materialize.js:3
 (anonymous function)                                           materialize.js:3
 jQuery.event.dispatch                                            jquery.js:4430
 jQuery.event.add.elemData.handle                                 jquery.js:4116

The first one is that there is no check if the element exist at https://github.com/Dogfalo/materialize/blob/a6c4a6acd6e7774ff391afdfeb5f1c60b94a450d/js/dropdown.js#L6 ( Cannot read property 'top' of undefined )
The second one is in https://github.com/Dogfalo/materialize/blob/cd3b02d0661b9a78fbdddd81ad95eed632cb4c0c/js/forms.js#L288
Because the Exception appear ONLY if you do something like this (with new lines/spaces):
        <option ng-repeat="sample in samples" value="{{ sample.id }}">
                {{ sample.name }}
        </option>

The following syntax works well (without new lines/spaces):
        <option ng-repeat="sample in samples" value="{{ sample.id }}">{{ sample.name }}</option>

Thanks.
PS: The Exception is Throwing when you click on a  with new lines/spaces.
v0.95.3

http://jsfiddle.net/z4eo725e/

Yeah, it seems like a bad one :) Maybe submit pull request?
A good thing to do is cache the $(this) call, as it can be expensive.

This one seems to be already fixed. Proof:
http://jsfiddle.net/z4eo725e/4/
I faced this problem today using Meteor package (Materialize v0.97.0)


javascript update

when is the javascript going to be updated ? most of js functions are deprecated in jquery for some time now

Feel free to update it yourself and send in a PR?


EL functions: add randomAlphanumeric()

cherry-picked, thanks a lot!
Note: I've added a commit to improve performance a bit

My code has gone) But your code seems cool! I haven't seen a cfor before ...
Anyway, thanks for the correction)


Option to obtain static IP by creating Elastic IP and NAT Gateway

When ELT process accesses the resource that requires IP whitelisting, Fargate task needs to run from the same IP address every time.
In AWS, the implementation requires Elastic IP and NAT Gateway and they would cost even when the tasks are not running. So make this an optional feature.

The resource group needs to be recreated upon the release of this feature. So, this is going to be a 0.4.0 milestone.


[Bug] Ed25519PrivateKey should not implement default fmt::(Display|Debug)

🐛 Bug
5cec179acf0e5df15edc94f8215fabedef627887 recently introduced vanilla Display|Debug implementations for Ed25519PrivateKey, leading to a printout of those keys in tests, or anywhere  Debug is called on a struct that contains such a key.
Expected Behavior
Ed25519PrivateKey should implement elided printouts using derive macros SilentDisplay and Silent Debug.

This is fixed in #3470.


Patch 4

Category

[ ] Content fix
[x] New article

Related issues

fixes #395

Contents of the Pull Request
wrote the article "why Microsoft 365 adoption projects fail", still need to update some images.

Docs Build status updates of commit 1a5e7d8:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:warning:Warning
View
Details

Community/why-m365-adoption-projects-fail.md

Line 3, Column 10: [Warning-ms-date-invalid] Invalid date format for 'ms.date': '17/01/2020'. Must be a date in format M/D/YYYY, no more than 30 days from today.
Line 214, Column 1: [Warning-hard-coded-locale] Link 'https://docs.microsoft.com/en-us/microsoft-365/community/microsoft365-maturity-model--intro' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 1a5e7d8:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:warning:Warning
View
Details

Community/why-m365-adoption-projects-fail.md

Line 3, Column 10: [Warning-ms-date-invalid] Invalid date format for 'ms.date': '17/01/2020'. Must be a date in format M/D/YYYY, no more than 30 days from today.
Line 214, Column 1: [Warning-hard-coded-locale] Link 'https://docs.microsoft.com/en-us/microsoft-365/community/microsoft365-maturity-model--intro' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

dear @sympmarc, wrote the article, will update tomorrow with pictures. Please feel free to add comment with your thoughts and where I should link to MM4M365.

dear @sympmarc, wrote the article, will update tomorrow with pictures. Please feel free to add comment with your thoughts and where I should link to MM4M365.

Docs Build status updates of commit 0c0c900:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 0c0c900:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit cb25b4d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit cb25b4d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit bc909c2:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit bc909c2:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit f5c8604:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit f5c8604:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 012b279:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

Docs Build status updates of commit 012b279:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

Community/why-m365-adoption-projects-fail.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel

@LuiseFreese - A few questions on this one...
First off, I like what you're talking about here! It meshes well with some of the Maturity Model content. I'm making edits for readability, mainly. Please read through and let me know if I've changed any of your meaning; I've tried not to.

In Volatility: "The world is changing faster than ever before, BUT the world will never change as slowly as it is right now." Not sure what you mean there.
Is there a source for the iceberg model you should reference? If it's your own idea: smashing!
"Servant and shared leadership will result in employees feeling more included in decision-making processes and taking responsibility" Servant? I can't figure out what word that should be.
Maybe define #LearnItAll (or link to something?)
I've added links to Wikipedia for some terms you touch on but probably don't want to define inline: holacracy, gamification. If there are better ones, please replace.

After you've reviewed and made sure I didn't ruin it, let's get it published!

@LuiseFreese - A few questions on this one...
First off, I like what you're talking about here! It meshes well with some of the Maturity Model content. I'm making edits for readability, mainly. Please read through and let me know if I've changed any of your meaning; I've tried not to.

In Volatility: "The world is changing faster than ever before, BUT the world will never change as slowly as it is right now." Not sure what you mean there.
Is there a source for the iceberg model you should reference? If it's your own idea: smashing!
"Servant and shared leadership will result in employees feeling more included in decision-making processes and taking responsibility" Servant? I can't figure out what word that should be.
Maybe define #LearnItAll (or link to something?)
I've added links to Wikipedia for some terms you touch on but probably don't want to define inline: holacracy, gamification. If there are better ones, please replace.

After you've reviewed and made sure I didn't ruin it, let's get it published!

Hi there @sympmarc,
thank you for taking your time with this one - always good to have a second pair of eyes, especially if you need to write not in your first language :-)
Volatility: what I meant: Everything changes extremely fast, faster as ever before, but as the the pace of changes will also increase, it will never be that slow as today (although today is fast). Please feel free to rephrase accordingly.
Iceberg model: its a somehow known model (which you can apply nearly everywhere when you need to investigate the root of a cause more intensely.  I found here, that ONE iceberg model is described in "The Systems Thinking Playbook by Linda Booth Sweeney and
Dennis Meadows"
Servant leadership is explained here in Wikipedia
LearnItAll here is a nice reference about Satya saying this
will now read the article again, pretty sure that you didn't ruin in.

Hi there @sympmarc,
thank you for taking your time with this one - always good to have a second pair of eyes, especially if you need to write not in your first language :-)
Volatility: what I meant: Everything changes extremely fast, faster as ever before, but as the the pace of changes will also increase, it will never be that slow as today (although today is fast). Please feel free to rephrase accordingly.
Iceberg model: its a somehow known model (which you can apply nearly everywhere when you need to investigate the root of a cause more intensely.  I found here, that ONE iceberg model is described in "The Systems Thinking Playbook by Linda Booth Sweeney and
Dennis Meadows"
Servant leadership is explained here in Wikipedia
LearnItAll here is a nice reference about Satya saying this
will now read the article again, pretty sure that you didn't ruin in.

@sympmarc you did NOT ruin it :-)

@sympmarc you did NOT ruin it :-)

Good! I can make those changes above and we'll get it published early this week! Good stuff, truly.

Good! I can make those changes above and we'll get it published early this week! Good stuff, truly.

glad to hear that @sympmarc - you really seam to like it  :-)

glad to hear that @sympmarc - you really seam to like it  :-)

Edits made - ready for publising.

Edits made - ready for publising.


Finish scl-lib integration

Add scl-lib will ensure adequate overall test coverage.
There's a bit more to do here.
Also we don't want deleting of supervisions to only sometimes remove the LN.

removeSupervision is now merged allowing LN or Val removal.

The main missing item is associated with creating new supervision LNs with no Val element.
For others which would be useful, see:
https://github.com/OpenEnergyTools/scl-lib/issues/79
https://github.com/OpenEnergyTools/scl-lib/issues/80
https://github.com/OpenEnergyTools/scl-lib/issues/83
I don't think this is necessary for the first release of this plugin, so I'll de-milestone it.


Cross-site request detected! (500_service_error)

Hello
Thank you for this nice little plugin.
I tried to install it but if I request the heroku endpoint I receive the message
Cross-site request detected!`
If I use the slack slash command I receive
500_service_error
Any chance for an update :)

Hey @honsa, thanks for your interest!
I haven't been maintaining this project for a few years now and am unsure if the steps outlined are still compatible with Slack's current design for slash commands. If you find any of the steps out dated then please feel free to contribute an update.
As for the error you're seeing, per the code, a 500 error is returned if the environment variable for the Slack command token was not set in your Heroku instace. Can you please confirm that you have set it and it matches the token value in Slack?

Thank you @shaykalyan for your reply
I tried but it seems the API Token is not usable anymore.
https://api.slack.com/docs/token-types#legacy_tokens


feat: instance detail monitoring summary #862

Closes #
IaaS Components - Instance detail monitoring summary #862
Changelog
New
Added react component for Instance detail monitoring
Changed

{{change thing}}

Removed

{{removed thing}}

@joshblack Yes as far as I see!

:tada: This PR is included in version 2.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Added mapM_

#49

Oh, the problem seems to be the use of void, which entails a Functor constraint.  Maybe just line the equivalent behavior:
mapM_ action = sink (\a -> action a >> return ())

Oh, sorry, I only built against lts-3.10.

Shouldn't it read "more general than sink"?

I suppose it's more general in the effectful function, but less general in that it specializes the monoid to (). Shrug. I'll let you decide the correct wording for the comment :)

Oh yeah, I forgot that sink is generalized to Monoid.  Actually, maybe make the type take a (a -> m () instead and note that it is a specialized version of sink

Done


URLs with colons in them aren't parsed correctly

Checks

[X] I have checked that this issue has not already been reported.

[X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example
[project]
name = "min-rep"
version = "0.1.0"
channels = []
platforms = ["linux-64", "osx-arm64", "win-64"]
[pypi-dependencies]
py-rattler = { git = "ssh://git@github.com:mamba-org/rattler.git", subdirectory = "py-rattler" }

then run pixi shell and the result is the following:
 test> pixi shell;
  × failed to parse project manifest
   ╭─[pixi.toml:7:14]
 6 │ [pypi-dependencies]
 7 │ py-rattler = { git = "ssh://git@github.com:mamba-org/rattler.git", subdirectory = "py-rattler" }
   ·              ─────────────────────────────────────────┬─────────────────────────────────────────
   ·                                                       ╰── data did not match any variant of untagged enum PyPiRequirement
   ╰────

 test> pixi --version
pixi 0.27.1

Issue description
This looks like a regression, the problematic line is in the documentation here:
https://pixi.sh/latest/reference/project_configuration/#pypi-dependencies
It can be worked around by replacing the colon with a  slash.
Expected behavior
I expect to get a shell.

Thanks for reporting! Let us change the documentation because ssh://git@github.com:mamba-org/rattler.git is indeed not a valid URL.
WDYT @ruben-arts ?

Thanks for the quick response!

Thanks for reporting! Let us change the documentation because ssh://git@github.com:mamba-org/rattler.git is indeed not a valid URL.
WDYT @ruben-arts ?

Sadly the problem is deeper. ssh://git@github.com:mamba-org/rattler.git is a valid url. If you go to gitlab, anything in a gitlab namespace contains a colon. For example, this project: https://gitlab.com/gitlab-org/gitlab-foss. The ssh clone URL is:
git@gitlab.com:gitlab-org/gitlab-foss.git

It think it is a valid for git but its not a valid url because the : should be followed by a port number.
 Strange that gitlab formats their ssh urls. this way.

I think for now its easiest to change the docs but it would be nice if we can properly handle this form down the line.
That would be nice. It was a little baffling to not have the ssh feature not work with a copy-pasted ssh clone url.

It's cool to see change so fast in pixi!


See

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Yep

When I know that she is going to cheat on me

I don't Want her to be with anyone else ever again for the of her life she is mine then I will relax


GH-42104: [C++] Fixes following OTel logging PR

Rationale for this change
Follow-up to https://github.com/apache/arrow/pull/39905, as some issues popped after merging.
What changes are included in this PR?

Fixes tests failing due to the ARROW_LOGGING_BACKEND env var not being defined
Removes example log statements accidentally left in PR

Are these changes tested?
Yes
Are there any user-facing changes?

No

GitHub Issue: #42104

Approved assuming CI passes (though I see some failures; since it's MATLAB etc maybe those are flakes?)

though I see some failures; since it's MATLAB etc maybe those are flakes?

Took a closer look and the failures seem unrelated to me.

The MATLAB failure is https://github.com/apache/arrow/issues/42015 and it'll be fixed by rebasing on main.

@pitrou Can we merge this? I want to fix CI failures on main.


Update to use docker-compose

Also update pip requirements to latest versions.

r+

I'm going to leave this open until I update the docs.

Closing in favor of pull request #88


⚠ Bofrost.de has degraded performance

In fdd1295, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 2399 ms

Resolved: Bofrost.de performance has improved in a703d19.


Improve documentation about disabling features

Improves documentation disabling features, particularly fixing a typo where
no_FEATURENAME: true values should be no-FEATURENAME: true, and adding
which file they should be set in, to avoid confusion with other token
files.

/lgtm

/retest


opt avoid to check delete bitmap while lookup rowkey in some situation to reduce CPU cost

Proposed changes
Issue Number: close #xxx
MoW performs a lookup on the primary key index for each key during the data loading process, and when a key is hit in the index, it continues to check if the key has been marked for deletion. Generally this check is not very costly.
However, in some scenarios, users perform high-frequency real-time update operations on a larger table, and most of the writes are updating existing data. In this scenario, the version of the table grows very fast, and the delete bitmap is also dense because duplicate keys are continuously being written.
In this scenario, this check is very costly

because it means calling the contains method of the roaring bitmap for almost every version of the rowset hit by an imported key to check if it has been marked for deletion
due to the high frequency of imports, there are typically thousands of versions that are not merged to base compaction.
because of the high duplication rate, every key is basically hit in the index
so this means that for almost every imported key, a loop needs to be called up to thousands of times to check if it has been marked for deletion
This overhead becomes very exaggerated when we are doing load jobs of about 100,000+ rows per second for a table

Here's a flame diagram for this scenario

For tables that don't use seq columns, and for non-column update imports, this check can be skipped. Even if a key is already marked for deletion, it's not a problem to mark it for deletion again as if it existed.

run buildall

run buildall


Added ChatBot

Issue No. #410

Description
I have added a Chat Bot for the EduHub Community and here is the Live Video.
It is fun contributing this project and the codebase is maintained 👍🏻
Video -
Screencast from 26-06-23 02:05:23 PM IST.webm
Screenshots

Checklist

[x] My code adheres to the established style guidelines of the project.
[x] I have included comments in areas that may be difficult to understand.
[x] My changes have not introduced any new warnings.
[x] I have conducted a self-review of my code.

Hello @SahityaRoy,
It is ready to Merge 👍🏻

Thanks a lot for your contribution

It's my pleasure

Logic will be same @killer2O .

checked with diffcompare for script.js file both the files are same. @gurjeetsinghvirdee you can check for other files

Why you're stalking me IDK.

@SahityaRoy ,
He is a bot or a spammer please ignore this as you can see he is blocked from GITHUB now, and also it's name changed to Ghost.
Proof :
Uploading Screencast from 02-07-23 05:03:03 PM IST.webm…

@SahityaRoy ,
You can check your chatbot. It's all fine and not same as this spammer @Killer2-O  saying.


Feat/clickable avatars

HI! Please consider adding a label hacktoberfest-accepted if you're merging. Thanks!


Embed camera definition files for Yuneec camera

This is an interim workaround where the camera definition xml files are embedded in the library.
The tool to embed the xml files is cpp_rsc.

Note that this does not work for the iOS build yet because cpp_rsc compiles a binary that runs on the host as part of the build process. We need to work our way around this somehow, e.g. by re-using the cpp_rsc binary from the default build.

So Yuneec MAVLink camera doesn't supply the camera definition file URL - as per the MAVLink Camera API spec?

No, they do supply it but the problem is that the file is not accessible currently through the MFi transport layer (mavlink is transmitted but no http yet), so this is an interim workaround.


add travis-ci and coveralls

I only changed .travis.yml, README.md, gulpfile.js, karma.conf.js, package.json
I do not know why some other changes appear in my repo T_T

You should not add the files under dist/


Fix multicluster kind script for Mac and ease running e2e tests [K8SSAND-903]

What this PR does:
Removes gnu getopt usage in the kind multicluster creation script and adds a few targets in the makefile to ease up setting up a test environment and running e2e tests:

make kind-e2e-test will delete any existing k8ssandra-0/1 kind cluster and recreate them, build the docker image and load it into the kind clusters and run the e2e tests.
make kind-setup will delete and recreate a k8ssandra-0 kind cluster, build the docker image, load it and deploy cass-operator and k8ssandra-operator. It allows to have an environment ready for creating K8ssandraCluster resources.

Which issue(s) this PR fixes:
Fixes #141
Checklist

[x] Changes manually tested
[ ] Automated Tests added/updated
[x] Documentation added/updated
[ ] CHANGELOG.md updated (not required for documentation PRs)
[x] CLA Signed:  DataStax CLA

Testing again.
make single-up
Failing with an error due to something wrong with the kustomize configs (I think). I'd hazard a guess that we have leader-election-role appearing multiple times at different levels in the kustomization.
deployment "cert-manager-webhook" successfully rolled out
/Users/milesgarnsey/projects/k8ssandra-operator/bin/kustomize build scripts/control_plane | kubectl apply -f -
Error: obj '{"apiVersion": "rbac.authorization.k8s.io/v1", "kind": "RoleBinding", "metadata": {
    "name": "cass-operator-leader-election-rolebinding", "namespace": "default"},
  "roleRef": {"apiGroup": "rbac.authorization.k8s.io", "kind": "Role", "name": "leader-election-role"},
  "subjects": [{"kind": "ServiceAccount", "name": "cass-operator-controller-manager",
      "namespace": "default"}]}
' at path 'roleRef/name': multiple matches for rbac.authorization.k8s.io_v1_Role|~X|leader-election-role:
  [rbac.authorization.k8s.io_v1_Role|default|cass-operator-leader-election-role
 rbac.authorization.k8s.io_v1_Role|default|leader-election-role
]
error: no objects passed to apply
make: *** [single-up] Error 1

make multi-up
Same error as above 🤔

With the kustomize changes Alex has merged, this is working smoothly for me. Thanks again Alex - teamwork! 🎉
This PR gets my approval, John has some questions/comments RE deleting clusters which I'll leave to you folks to resolve. I am open minded here.
I am in favour of removing getopt if possible FWIW - less dependancies is always better, especially when they cause cross-platform inconsistencies.


BatchClient - initial addition

Closing in favor of the newer
https://github.com/joseharriaga/openai-in-typespec/pull/67


API "Create an incident" got 404

Using "Create an incident" in https://learn.microsoft.com/en-us/azure/ai-services/content-safety/how-to/custom-categories-rapid?tabs=curl#create-an-incident-object
I always get 404.

With
curl --location --request PATCH 'https://%endpoint%/contentsafety/text/incidents/%text-incident-name%?api-version=2024-02-15-preview '
--header 'Ocp-Apim-Subscription-Key: %your-content-safety-key%'
--header 'Content-Type: application/json'
--data '{
"incidentName": "%text-incident-name%",
"incidentDefinition": "string"
}'
I replaced %endpoint% with my content safety resource(F0) endpoint in east us. (With this endpoint I successfully used for Create or modify a blocklist)
I replaced %text-incident-name% with "incident1".
I replaced %your-content-safety-key% with key of this resource.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 232a8d1a-1d92-d4f0-c350-086156959426
Version Independent ID: 77b08ac3-03ba-bc34-c3ea-7e91a6fd0761
Content: Use the custom categories (rapid) API - Azure AI services
Content Source: articles/ai-services/content-safety/how-to/custom-categories-rapid.md
Service: azure-ai-content-safety
GitHub Login: @PatrickFarley
Microsoft Alias: pafarley

@whw23
Thanks for your feedback! We will investigate and update as appropriate.

@whw23
Thank you for bringing this to our attention.
I've delegated this to content author @eric-urban, who will review it and offer their insightful opinions.

@eric-urban
Please review it and update as appropriate.

@Naveenommi-MSFT I can repro the issue. Please open a product issue for this. I'll also notify the product team myself
#please-close


Handling poor / no internet connection

Hi all,
Not marking this as a bug as potentially just my own misunderstanding of the docs. In brief, I’m struggling gracefully failing if internet connection drops out and an update is triggered.
I have a resource as such:
import request from "superagent";

export default class EntryResource extends Resource {
  readonly UserCode: string | undefined = undefined;

  static async fetch<T extends typeof Resource>(
    this: T,
    method: Method = "get",
    url: string,
    body?: Readonly<object | string>
  ) {
    const req = request[method](url);
    return req
      .then((res) => {
        if (res.body.items) return res.body.items;
        if (res.body.status === "fail") return [];
        else return res.body.codes;
      })
  }

  static update<T extends typeof Resource>(this: T) {
    return {
      ...this.updateShape(),
      options: {
        ...this.getFetchOptions(),
        optimisticUpdate: (params: any, body: any) => ({
          ...body,
          otherKeys: etc,
        }),
      },
    };
  }

  pk() {
    return this.etc;
  }

  static urlRoot = `etc`;
};

And I’m wrapping the various components in suspense and NetworkErrorBoundarys.
<Suspense fallback={<Loading />}>
                <NetworkErrorBoundary fallbackComponent={ErrorPage}>
			<Component /> (which uses the resource above)
                </NetworkErrorBoundary>
 </Suspense>

The resource works fine until there is an internet connection dropout. At this point the NetwrokErrorBoundary seems to be missed, leading to displaying the suspense’s loading spinner infinitely.
The behaviour I’d preferably like would be to cache the update data and try to send it again when there is an internet connection. Failing that, I’d like to stop attempting the update/PUT and display a no internet connection popup, keeping displaying the data before the update call. I’ve had no joy catching the error in the fetch and returning something to trigger an error as this breaks the expected data shape. Overall, I’m unsure if this behaviour (specifically missing the NetworkErrorBoundary) is expected and therefore is or isn’t a bug. Can anyone please advise on this or on my wider aims of handling this specific error?

At Coinbase I think we had a https://resthooks.io/docs/next/api/Manager that handled online offline stuff. @nickcherry think you could share a snippet for this use case?

At Coinbase I think we had a https://resthooks.io/docs/next/api/Manager that handled online offline stuff. @nickcherry think you could share a snippet for this use case?

Thanks for the report  Murray!
If you want your app to presume it updated even when the network is down, you can achieve this using optimistic updates, combined with a fetch that waits til network connection is restored.
An example of online/offline detection can be found here: https://github.com/coinbase/rest-hooks/blob/master/packages/rest-hooks/src/manager/PollingSubscription.ts#L136
For reference, NetworkErrorBoundary catches any error that has a 'status' member. You can use a general error boundary component to catch other such errors. In the built in fetch (non-custom) there is a mechanism to add the status member for cases like CORS or network failures that would not normally result in a status member.

Thanks for the report  Murray!
If you want your app to presume it updated even when the network is down, you can achieve this using optimistic updates, combined with a fetch that waits til network connection is restored.
An example of online/offline detection can be found here: https://github.com/coinbase/rest-hooks/blob/master/packages/rest-hooks/src/manager/PollingSubscription.ts#L136
For reference, NetworkErrorBoundary catches any error that has a 'status' member. You can use a general error boundary component to catch other such errors. In the built in fetch (non-custom) there is a mechanism to add the status member for cases like CORS or network failures that would not normally result in a status member.

@ntucker great, thanks for the quick reply! I'll have a look at those and see what I can get working

@ntucker great, thanks for the quick reply! I'll have a look at those and see what I can get working

test


Deprecation notice of clue/reactphp-buzz

clue/reactphp-buzz
This package has now been migrated over to react/http and only exists for BC reasons.

Fixed.

Expected to close automatically, oh well.


how to exclude node which is not connected into any other node ?

get the information of the node and check if it has inputs and outputs.
const data = editor.getNodeFromId(id)
console.log(data);

But i want to exclude node which is not connected at the time of saving json structure ?

In the export is the same information. You will have to go through the export and remove the empty nodes.


Azure Linux VMs cannot run jfrog-cli as "jenkins" home directory is "adoptopenjdk"

The jenkins home directory for Azure Linux VMs is /home/adoptopenjdk, instead of /home/jenkins, this results in plugin tools like "jfrog-cli" not being found as the jenkins pipeline searches /home/jenkins/tools which does not exist...?

Temporarily disabled the ci.adoptium.net Azure Cloud Linux-build agent, by changing dockerBuild tag to XdockerBuild
This is to allow the installer job to run successfully on the 1 static dockerBuild node.

The problem is not actually the VM, it's due to the Artifactory installer upload pipeline incorrectly combining "Declarative" and "Scripted" Jenkins pipeline syntax: https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile
As stated in the Jfrog-cli doc: https://jfrog.com/help/r/jfrog-integrations-documentation/use-declarative-or-scripted-syntax-with-pipelines
When working with the Jenkins Artifactory plugin, be sure to choose either scripted or declarative. In other words, do not use declarative and scripted steps within a single pipeline. This will not work.

The jfrog-cli Declarative tools definition sets up the pipeline environment for the agent the top-level pipeline runs on, the pipeline then switches to "scripted" mode and switches context using node(), the jfrog-cli will then not necessarily work in that context.
Raised installer issue: https://github.com/adoptium/installer/issues/1042


Recommendation to Remove Redundant jQuery File from Codebase for Improved Optimization

Hey @llaske
I noticed that  "jQuery.min.js" file found in the "lib" directory does not appear to have any impact on the functionality of the following activities.

Tanagram Activity
EbookReader Activity
Abacus Activity

Since it is not being used in these activities, it is adding unnecessary bloat to the code and can be removed to reduce the file size.

Hey @VedantSharma11 , i guess the jquery.min.js is not a bloat to the code , as by removing this file im getting this issue
see for your reference
Before deleting

After

@llaske can you please review my PR

There is an error in ebook reader activity:

it was working fine on my side let me check it again and fix it @llaske

@llaske i'm not able to reproduce the console error on my side can you please tell me exactly where it is happening

Try to refresh your dev branch.

@llaske my dev branch is up to date but still i'm not able to produce it on my side, its working fine on my side

Nice. Thanks.


Migrate table

Summary

What was changed:
Migrate New Terra Table to Framework from feature branch in Terra Core.
Why it was changed:
We want the Terra Table package to live inside Terra Framework as it will be consumed by Data Grid.
Testing

This change was tested using:

[x] WDIO
[x] Jest
[x] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[ ] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[x] UX review
[x] Accessibility review
[x] Functional review

Additional Details

This PR resolves:
UXPLATFORM-9540

Thank you for contributing to Terra.
@cerner/terra

You should have a separate example for the Default Terra Table based on similar component strategies.

@kenk2  It was hard to fully verify this for accessibility and functionally based on how early in the development of Table we are.  Functionally with what was there looked good, my only issue was not being able to scroll the examples via keyboard that as you mentioned that is addressed via UXPLATFORM-9729.
I feel like it's too early to fully perform accessibility test so I went ahead removed those labels. I did try testing it with Jaws. One issue I will call out that I encountered is the values of each cell are being communicated as 1 value rather than 3 values.  For example, the Default Table example the Heart Rate Monitor row has values 68, 66, and 67 the screen reader will read it as 686,667.  If you think this is unexpected at this point and we can explore how best to account for this, whether it's a part of this story or I create a new story specifically to get the AT stuff ironed out.

@chrismichalewicz I think for now given how early everything is, we can create another story to get the screen reader functionality up to par.

@chrismichalewicz I think for now given how early everything is, we can create another story to get the screen reader functionality up to par.

I would not expect the screen reader to be off in the initial release.  We don't have this issue in the DataGrid, right?

@kenk2  @cm9361 I retested this morning with Jaws in Edge and am not able to recreate what I was seeing yesterday.  Each individual cell value is being read individually now as I would expect. This is working the same in Data Grid as well.

@cm9361 I've advised @kenk2 that mentions of API features like Cell Selection and Masking shouldn't appear in the documentation for Terra Table, even if they're needed for components that consume it. That might include making props private.

@cm9361 I've advised @kenk2 that mentions of API features like Cell Selection and Masking shouldn't appear in the documentation for Terra Table, even if they're needed for components that consume it. That might include making props private.

I agree with this statement.  "onCellSelect" should not be part of the "public API".  I am not certain removing masked cells is necessary though.  If a team desires to use it for some reason, it would still be accessible.


Menu items changing names on show time for infra.

Alternative to #2715

removed the def inside def which was a mistake
merged the 2 classes to have less code

Is it better this way? Ping @Fryguy
Or should I create a class for each new menu item with the case copied into each of them? My guess is that we will have more menu items that will change names sooner or later and I would prefer a more generic way to do that (as I have in this PR) to having to write a class or a if-else statement for each such changing menu item.
Also more generic approach can better be used from plugins that would define menu items.

I don't like always doing the case statement at runtime, but I think the overall idea is fine.  The overall idea being: provide each menu item with an object that knows how to look up the name.

@tenderlove : moved the branching

Looks good to me!


wav2vec2 padding issue

🐛 Bug
I'm trying to use the https://github.com/pytorch/fairseq/tree/master/examples/wav2vec model on mobile, as I can trace but not script the model, I tried using padding to allow for different audio inputs. But calling Wav2Vec2Tokenizer with max_length padding results in bad recognition results if max_length gets larger.
To Reproduce
Steps to reproduce the behavior (always include the command you ran):
Below is the code and results - padding a 1.4s wav to 3s or 5s lead to about the same recognition, but to 7s or 9s lead to worse and worse results:
import soundfile as sf
import torch
from transformers import Wav2Vec2ForMaskedLM, Wav2Vec2Tokenizer

tokenizer = Wav2Vec2Tokenizer.from_pretrained("facebook/wav2vec2-base-960h")
model = Wav2Vec2ForMaskedLM.from_pretrained("facebook/wav2vec2-base-960h")
model.eval()
audio_input, _ = sf.read("ionlywishtobealone.wav")
input_values = tokenizer(audio_input, return_tensors="pt").input_values
logits = model(input_values).logits
predicted_ids = torch.argmax(logits, dim=-1)
transcription = tokenizer.batch_decode(predicted_ids)[0]
print(transcription)
print(input_values.shape)

# pad the 1.42s audio to about 3,5,7,9s:
for n in [3,5,7,9]:
  input_values = tokenizer(audio_input, return_tensors="pt", padding="max_length", max_length=n*10000).input_values
  logits = model(input_values).logits
  predicted_ids = torch.argmax(logits, dim=-1)
  transcription = tokenizer.batch_decode(predicted_ids)[0]
  print(input_values.shape)
  print("padding to {}s: {}\n".format(n, transcription))

I ONLY WAS TO BE ALONE
torch.Size([1, 15616])

torch.Size([1, 30000])
padding to 3s: I ONLY RIS TO BE ALONE

torch.Size([1, 50000])
padding to 5s: I ONLY IS TO BE ALONE

torch.Size([1, 70000])
padding to 7s: ONER IS TO BE LON

torch.Size([1, 90000])
padding to 9s: ONERISTO BELAN

Expected behavior
Padding to longer audio should lead to the same results.
Environment

fairseq Version (e.g., 1.0 or master): NA
PyTorch Version (e.g., 1.0) 1.7 & master
OS (e.g., Linux): Mac
How you installed fairseq (pip, source): NA
Build command you used (if compiling from source): NA
Python version: 3.8
CUDA/cuDNN version: NA
GPU models and configuration: NA
Any other relevant information:

Additional context

ionlywishtobealone.wav.zip

Fixed in the update of the script here using torchaudio.


Capture build scans on ge.apache.org to benefit from deep build insights

Description
This PR publishes a build scan for every CI build on Jenkins and GitHub Actions and for every local build from an authenticated Apache committer. The build will not fail if publishing fails.
The build scans of the Apache JMeter project are published to the Gradle Enterprise instance at ge.apache.org, hosted by the Apache Software Foundation and run in partnership between the ASF and Gradle. This Gradle Enterprise instance has all features and extensions enabled and is freely available for use by the Apache JMeter project and all other Apache projects.
Motivation and Context

Currently, the Apache JMeter build is configured to publish build scans to the publicly available scans.gradle.com. This pull request enhances that functionality by instead publishing build scans to ge.apache.org. On this Gradle Enterprise instance, Apache JMeter will have access not only to all of the published build scans but other aggregate data features such as:

Dashboards to view all historical build scans, along with performance trends over time
Build failure analytics for enhanced investigation and diagnosis of build failures
Test failure analytics to better understand trends and causes around slow, failing, and flaky tests

If interested in exploring a fully populated Gradle Enterprise instance, please explore the Maven builds already connected to ge.apache.org, the Spring project’s instance, or any number of other OSS projects for which we sponsor instances of Gradle Enterprise.
Please let me know if there are any questions about the value of Gradle Enterprise or the changes in this pull request and I’d be happy to address them.
How Has This Been Tested?

This change has been verified by publishing build scans to a Gradle Enterprise instance, rather than to scans.gradle.com.
Types of changes

Build improvements

Checklist:

[x] My code follows the code style of this project.
[x] I have updated the documentation accordingly.

Nice. Can we configure remote build cache as well?

Thanks, I've checked authentication works, and the scan publishes.
An important bit is that this build scan exposes the names of non-committed files which might be unexpected.


Don't seed fake Drupal IDs or sources.

What's this PR do?
No longer create fake drupal_id or source values when seeding the database, since this just makes it confusing when those don't match up with real corresponding data in other services.
How should this be reviewed?
If you run the seeder, it shouldn't have errors & shouldn't seed random values for those fields. 💁
Checklist

[ ] Documentation added for changed endpoints.
[ ] Tests added for new features/bug fixes.

For review: @angaither

@DFurnes does this just set drupal_id to null for seeded users?

Yep, it won't save a drupal_id to those documents since we don't include it in the attributes specified on the factory. (MongoDB is weird and treats null and "not having a field" differently, but yeah 😝)

👍
I wonder if we do need that in the seed though, like some people should have real phoenix ids?

Yeah that's definitely useful for testing, but I don't think we can seed that without knowing the corresponding profiles in the associated Drupal instance.

For context, I noticed this when testing the Phoenix export script on my local and having the "real" data from my Phoenix instance conflict with the existing "fake" IDs that Faker had created.


Training Optimization

Tests some new frame sampling implementations and frame optimizations.

Remove old frame shuffling.
Add config that trains the last attention layers instead of all.
Refactor index frame implementation.
Update instructions

Closing as needs more testing.


LANG, OUT OF STOCK

upgraded out of stock information from https://github.com/Paymenter/Paymenter/pull/138

Improved version https://github.com/Paymenter/Paymenter/pull/138

Co-authored-by: Turkidev<EMAIL_ADDRESS>

its<EMAIL_ADDRESS>:-)


Text search with gorm

Hello,
How can I make a text search with go? I try with this code but not thing happen
name := "Any Name"
tx := aar.db.Limit(10)
tx.Where("name @@ to_tsquery(?)", name)

So what I have to do to perform search.

@MuhmdHsn313  refer to https://gorm.cn/docs/


Synchronization issue with Spotify client

Issue 1:

Play song from stopped state
Time bar jumps to the end of the song on the client
Pause and play
Time is now correct

Issue 2:

Let song play
Pause
Time goes forward a couple of seconds
Play
Time goes back a couple of seconds (as was before before pause)

Maybe it gives you some hints: https://github.com/librespot-org/librespot/issues/210

Nice catch, I'll try to use kPlayStatusLoading.

That didn't fix the issue, apart from the buffering one. (updated the main post with a better description)

This also happens when the next song loads. Pausing and unpausing doesn't seem to fix it, only seeking does, and even then we still see the progress bar jump when pausing. In addition restarting the track causes the progress bar to jump to the end.

This fixed itself by using the new endpoint.

The recent commits in remote-3 branch? will give it a go.

Yep, I've just merged it with master.

so i saw. building now.

I had some issues with Git, pull now.

fixed some of the issues, though I noticed that the progress bar doesn't move for the first 15 seconds of any song now for some reason. I note that the bar starts moving a split second before the client logs receiving the 3rd chunk.

Fixed by c35f37fe108d299e65bea1f0b439d807dd0c92c9


please remove empty space in Lite mode.

OS : windows 10
Ver : 28b
description :
please remove empty space beside total balance in Lite mode.
I think that is for Z-balance in native mode, but no need that in lite mode .

Issue is corrected, closing


Allow listing IPs and deleting them from this list

Hey.
I implemented a simple table of IP addresses and a button to delete them.
This is way more convenient than remembering which IP you previously registered and prevents too many IPs from cluttering up the database

Thank for that. I think the new UI update is going to confuse most people. Where there are no IPs in the list, the UI just displays IP	Action with no explanation what that means. I think at the very least, when there are no IPs, it should redirect to /add.

@ab77 You're right. I was thinking about having an add button, but your solution is probably better.

@ab77 Updated the PR accordingly.

I've tested it on a fresh Droplet and I get a HTTP error after logging in. Have you tested it?

@ab77 Yes of course. but let me test it again, just to make sure I didn't miss anything

@ab77  works for me, BUT I had to adapt my Caddyfile to get this running. Guess there was an update and the deprecated proxy_header was removed in favor of header_upstream

What exactly did you adapt? Does the Caddyfile.template need to be updated?

@ab77 yes it needs to be updated.
Instead of proxy_header you have to use header_upstream. That's it.

You need to update build.sh and test it from a fresh install before this can be merged.

@ab77 ??? I tested it with a fresh droplet and it works fine (other than the Caddyfile thing which is unrelated). So what do I need to update in the build.sh script?

I'd prefer to keep the Caddyfile changes separate from this PR. This way it's easier to keep track of the changes

See #241

OK, I've merged your Caddyfile change and tested it, all good. This PR however can't be accepted, because the admin site is still broken after test merging your updates. After logging in (Chrome), the error is:
178.62.x.y sent an invalid response.
ERR_RESPONSE_HEADERS_MULTIPLE_LOCATION```

Please check your code and re-submit.

@ab77 Weird. I installed it on a new droplet and it worked. Will double check..

@ab77 found the issue. I was raising a redirect (copied the code from one of you methods) which caused this.

Thank you, merged.


Added __traits getUnitTestName

Hi I have added a trait for getting the names of Unit Tests
So now it's possible to do this to extract the names from the unittests, if nameless then the name is empty ""
module foo;
unittest {
}

unittest "My named test" {
}

void main() {
    foreach(test; __traits(getUnitTests, foo)) {
        static assert(__traits(getUnitTestName, test) == "My named test");
    }
}

I'll go ahead and make the changes Max talked about.

Here's how I did it:
https://github.com/opendlang/opend/commit/a878f9f91ff8debc6f8f789820561305b25ac450
using the const(char)[] throughout and then pulling the ustring out of the token - since you typed TOK.string_ you can be reasonably confident that part of the union is set, and that part doesn't have quotes, already replaced sequences like backslash n, etc, so it is more reliably for what the string is supposed to actually represent.

Thank you guys 😊


reverting hacktoberfest changes

ref: https://github.com/kubernetes/website/pull/24342
ref: https://github.com/kubernetes/website/pull/24306
Hacktoberfest has changed to opt-in and we can (hopefully) start reverting our memos. (ref: https://hacktoberfest.digitalocean.com/hacktoberfest-update).
/cc @kubernetes/sig-docs-en-owners
/cc @mrbobbytables
/hold
Holding for discussion or if spam increases. Anyone can remove the hold at any time if they want.

:crossed_fingers:
/approve

/hold cancel

@jimangel @sftim I'm part of the spanish translation team and we're wondering if this means that translations on the website can participate on Hacktoberfest.
/cc @raelga


Docs: update themes docs to work and match contemporary package scopes

Thanks for this, I was struggling to figure this out.


TabBar issue on iOS 15

The floating panel clips out the tab bar background then I move it down!

TabBar appearance has refined on iOS15. I confirmed the same issue in Samples app. To fix the clear background, I need to check 'Scroll Edge' of the tab bar in the storyboard or set a value to `UITabBar.scrollEdgeAppearance. As a result, the tab bar show as well as iOS 14 and below.
https://developer.apple.com/videos/play/wwdc2021/10059/
https://stackoverflow.com/questions/68688270/ios-15-uitabbarcontrollers-tabbar-background-color-turns-black

        let appearance = UITabBarController().tabBar.standardAppearance
        appearance.backgroundEffect = nil

        if #available(iOS 15.0, *) {
            tabBarController.tabBar.scrollEdgeAppearance = appearance
            tabBarController.tabBar.standardAppearance = appearance
        }


Update cd-to to 2.6.0

After making all changes to the cask:

[x] brew cask audit --download {{cask_file}} is error-free.
[x] brew cask style --fix {{cask_file}} left no offenses.
[x] The commit message includes the cask’s name and version.

Thank you for the contribution. It was merged directly as 8d160d8054b90db4a95727e649eb98aae04fda56. Your contribution is still credited to you.


Add musical instruments to DemoPlayer and their support

Added new musical instruments
Now can select an instrument in the settings
Attempt to eliminate crackling

Known issues:

Due to the conversion of the audio file, the amount of RAM used increases

The electric guitar is called LightGuitar, and there is no light guitar for a donation yet.

You need to install a new library pip install soundfile and add its version to requirements.txt

The old piano didn't go... Where did you get the audio files for this piano?

@Ai-Vonie I prefer not to include sound files within the codebase, as doing so would significantly increase the repository size, which is something I want to avoid. Initially, I had a collection of sound files for a dozen instruments locally, but I only included the piano sound in the repository for demonstration purposes. The intention is not for users to pull all sound files directly from the codebase. Instead, I want users to download the sounds via network requests, keeping them separate from the code.

Hmm, kk. Maybe tomorrow I'll see what I can do.
Just this week I did some refactoring, logic improvements and adding some functions in lua code for the old farm bot in mmo.


Design confusion about interface assignment

I think the code bellow should case a compilation error. Because type *A didn't implement interface jump. Why does Go allow this assignment expression?  Why do we need it? It makes people confused!!!
By the way, auto referencing and dereferencing are also a bad design. Code should be very clear about our intention!!!
package main

import (
	"fmt"
)

type jump interface {
	modify()
	myPrint()
}

type A struct {
	data int
}

func (m A) myPrint() {
	fmt.Println("data:", m.data)
}

func (m A) modify() {
	m.data += 1
}

func main() {
	var a = A{data: 1}
	var b jump
	b = &a // I think this should case a compilation error
	b.modify()
	b.myPrint()
}

*A exactly implements jump. The method set of *A is always a super set of A.
Please read https://go101.org/article/method.html and https://go101.org/article/interface.html for details.

This is WAI as explained above.
For further questions, please take a look at the Questions wiki page. Thanks.


server_name not logged properly

In our cloud-hosted sentry UI events from our celery server consistently show up with
server_name: celery-8c78c4545-n987w
The server name should be celery0.
caleb@celery0.cloud51:~$ hostname
celery0

caleb@celery0.cloud51:~$ python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.gethostname()
'celery0'

We initialize our sentry config like so:
def init_sentry_sdk(sentry_config):
    sentry_sdk.init(
        dsn=sentry_config['dsn'],
        integrations=[
            sentry_logging_integration,
            DjangoIntegration(),
            CeleryIntegration()
        ],
        send_default_pii=True,
        release=sentry_config['release'],
    )

The SDK determines server_name using the same exact code you're using on the REPL to show a different hostname. I am somewhat sure you're just running the REPL outside of the docker container while your app runs inside.
If you want a different server_name, pass server_name as argument to init.

I am somewhat sure you're just running the REPL outside of the docker container while your app runs inside.

Nope, we are not using docker on that server. It's a ec2 instance that I ssh'd into. Python runs under celery which is launched via systemd

Also not sure if you're a sentry employee but if you are this a link to the sentry incident with the incorrect server name:

https://sentry.io/organizations/15five/issues/1661181013/?project=1385044&query=is%3Aunresolved+server_name%3Acelery*&statsPeriod=14d

My point still stands that we use the same exact code that you presented in your REPL to determine the hostname as part of the SDK. That leads me to believe that your celery worker runs in a very different context than your REPL as the function call here: https://github.com/getsentry/sentry-python/blob/464ca8dda09155fcc43dfbb6fa09cf00313bf5b8/sentry_sdk/client.py#L70 seems to return the "wrong" result.
A good repro case would be to spawn a celery task on those machines to compute and return the hostname. Again I have severe doubts that this would produce something other than the SDK does.

Turns out the error was coming from a kubernetes instance I was unaware of, sorry about that.


crawler start and finish in 1 second

hey, first thank you of your great work,
i have some problem about this repo,
now my machine on aliyun, and status is:
splash: install by docker, run at localhost:8058
squid: install by yum, run at *:3128
and i use pip to install your all requirements
my python version is 3.6.4
python crawler_booter.py --usage crawler

and i see log in logs directory like this:
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 267658),
'log_count/INFO': 4,
'memusage/max': 46108672,
'memusage/startup': 46108672,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 242833)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 269282),
'log_count/INFO': 4,
'memusage/max': 46108672,
'memusage/startup': 46108672,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 250614)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 270711),
'log_count/INFO': 4,
'memusage/max': 46309376,
'memusage/startup': 46309376,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 258190)}
2018-05-22 16:24:19 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 272169),
'log_count/INFO': 4,
'memusage/max': 46559232,
'memusage/startup': 46559232,
'start_time': datetime.datetime(2018, 5, 22, 8, 24, 19, 265645)}
can you tell me how can i run this? thanks

which version did you use? Release version is recommended. The master  is unstable.

@ResolveWang
I use release version v0.10

There are not many useful messages for giving advice.
1.Can you connect to redis or splash?
2.Your can remove all the log code in this project and run the crawler at the front, so that all errors  will expose. Maybe some critical infos are hidden.

ok, i have find the question, i run your code on windows and open debug.
my redis have not password and i comment REDIS_PASSWORD option in settings.py so program raise exception, but i can't find it in logs/log on centos.
thanks your quickly response and great work

i recommend make a default value when setting's option key is empty

Thanks for your advice. I'll do that job in next release


Use StatsD format for metrics and sink to message channel

@jkschneider as discussed, we're going to put this one aside for now and instead especially after making few modification that allowed me bring the message format to what it was before. So the string argument here is backward compatibility.
That said, we will definitely consider bringing statsd publisher for 2.1. I'll raise a separate issue and link to this PR


Mangled tmst and many dupes

I configured a virtual device (modified to output tmst) and connected it to an unasserted synced miner. I configured Packet Purchaser to route Join Requests for my test device to a test endpoint, configured with lorawan-sniffer.
I allowed the device to send two Join Requests before stopping the program. The tmst line is added and printed when the UDP packet is dispatched to Miner.
[2021-08-16T08:33:55Z INFO  virtual_lorawan_device::virtual_device::udp_radio] Connecting to server <IP_ADDRESS>:1680 from <IP_ADDRESS>:0
tmst 908
[2021-08-16T08:34:01Z WARN  virtual_lorawan_device::virtual_device] No Join Accept Received
tmst 6101071

The full Miner logs are attached, but in summary, things appear to go well:
The first packet is transacted
2021-08-16 08:42:58.712 7 [info] <0.1547.0>@miner_lora:handle_udp_packet:366 PUSH_DATA [{<<"rxpk">>,[[{<<"chan">>,0},{<<"codr">>,<<"4/5">>},{<<"data">>,<<"AAAAAQAADCUA7qAAAAH///+bg3NaQWY=">>},{<<"datr">>,<<"SF10BW125">>},{<<"freq">>,904.7},{<<"lsnr">>,5.5},{<<"modu">>,<<"LORA">>},{<<"rfch">>,0},{<<"rssi">>,-112},{<<"size">>,23},{<<"stat">>,1},{<<"tmst">>,274}]]}] from<PHONE_NUMBER>0382856 on 37061
2021-08-16 08:42:58.712 7 [notice] <0.1547.0>@miner_lora:handle_packets:529 Routing {eui,18446742978492932334,10427768277893120}
2021-08-16 08:42:58.713 7 [notice] <0.1521.0>@blockchain_state_channels_client:handle_cast:156 failed to find router for join packet with routing information {eui,18446742978492932334,10427768277893120}:eui_not_matched, trying default routers
2021-08-16 08:42:58.713 7 [info] <0.1521.0>@blockchain_state_channels_client:handle_packet:356 handle_packet #packet_pb{oui=0,type=lorawan,payload=<<0,0,0,1,0,0,12,37,0,238,160,0,0,1,255,255,255,155,131,115,90,65,102>>,timestamp=274,signal_strength=-112,frequency=904.7,datarate=<<"SF10BW125">>,snr=5.5,routing=#routing_information_pb{data={eui,{eui_pb,18446742978492932334,10427768277893120}}},rx2_window=undefined} to ["/p2p/11w77YQLhgUt8HUJrMtntGGr97RyXmot1ofs5Ct2ELTmbFoYsQa"]
2021-08-16 08:43:00.490 7 [info] <0.28527.2>@blockchain_state_channel_handler:handle_data:153 sc_handler client got banner, sc_id: <<142,36,36,142,174,217,229,225,23,167,76,150,9,66,150,4,123,16,156,46,168,37,197,189,231,102,137,177,13,198,222,223>>
2021-08-16 08:43:00.510 7 [info] <0.1521.0>@blockchain_state_channels_client:maybe_send_packets:741 valid banner for "/p2p/11w77YQLhgUt8HUJrMtntGGr97RyXmot1ofs5Ct2ELTmbFoYsQa", sending 1 packets

The second packet is transacted:
2021-08-16 08:43:04.815 7 [info] <0.1547.0>@miner_lora:handle_udp_packet:366 PUSH_DATA [{<<"rxpk">>,[[{<<"chan">>,0},{<<"codr">>,<<"4/5">>},{<<"data">>,<<"AAAAAQAADCUA7qAAAAH////FfKjoTU4=">>},{<<"datr">>,<<"SF10BW125">>},{<<"freq">>,904.1},{<<"lsnr">>,5.5},{<<"modu">>,<<"LORA">>},{<<"rfch">>,0},{<<"rssi">>,-112},{<<"size">>,23},{<<"stat">>,1},{<<"tmst">>,6103072}]]}] from<PHONE_NUMBER>0382856 on 37061
2021-08-16 08:43:04.816 7 [notice] <0.1547.0>@miner_lora:handle_packets:529 Routing {eui,18446742978492932334,10427768277893120}
2021-08-16 08:43:04.816 7 [notice] <0.1521.0>@blockchain_state_channels_client:handle_cast:156 failed to find router for join packet with routing information {eui,18446742978492932334,10427768277893120}:eui_not_matched, trying default routers
2021-08-16 08:43:04.816 7 [info] <0.1521.0>@blockchain_state_channels_client:handle_packet:356 handle_packet #packet_pb{oui=0,type=lorawan,payload=<<0,0,0,1,0,0,12,37,0,238,160,0,0,1,255,255,255,197,124,168,232,77,78>>,timestamp=6103072,signal_strength=-112,frequency=904.1,datarate=<<"SF10BW125">>,snr=5.5,routing=#routing_information_pb{data={eui,{eui_pb,18446742978492932334,10427768277893120}}},rx2_window=undefined} to ["/p2p/11w77YQLhgUt8HUJrMtntGGr97RyXmot1ofs5Ct2ELTmbFoYsQa"]
2021-08-16 08:43:04.819 7 [info] <0.1521.0>@blockchain_state_channels_client:send_offer:665 OfferMsg1: {blockchain_state_channel_offer_v1_pb,{routing_information_pb,{eui,{eui_pb,18446742978492932334,10427768277893120}}},<<252,30,244,157,170,213,181,118,87,171,110,170,255,204,208,251,46,52,247,167,201,94,37,44,132,46,149,107,81,151,75,64>>,23,0,<<0,238,238,0,211,83,133,110,218,196,47,176,217,1,105,251,23,236,205,56,14,61,38,38,162,122,83,153,239,201,252,104,117>>,<<48,68,2,32,47,134,119,235,107,248,247,208,178,104,34,254,232,227,21,82,72,43,144,223,38,46,27,134,62,143,64,199,135,191,93,107,2,32,67,228,47,120,94,67,97,228,142,161,32,49,184,134,196,211,65,78,145,57,217,45,8,243,241,91,178,212,253,8,136,1>>,'US915'}
2021-08-16 08:43:04.819 7 [info] <0.1521.0>@blockchain_state_channel_handler:send_offer:70 sending offer: {blockchain_state_channel_offer_v1_pb,{routing_information_pb,{eui,{eui_pb,18446742978492932334,10427768277893120}}},<<252,30,244,157,170,213,181,118,87,171,110,170,255,204,208,251,46,52,247,167,201,94,37,44,132,46,149,107,81,151,75,64>>,23,0,<<0,238,238,0,211,83,133,110,218,196,47,176,217,1,105,251,23,236,205,56,14,61,38,38,162,122,83,153,239,201,252,104,117>>,<<48,68,2,32,47,134,119,235,107,248,247,208,178,104,34,254,232,227,21,82,72,43,144,223,38,46,27,134,62,143,64,199,135,191,93,107,2,32,67,228,47,120,94,67,97,228,142,161,32,49,184,134,196,211,65,78,145,57,217,45,8,243,241,91,178,212,253,8,136,1>>,'US915'}, pid: <0.28527.2>
2021-08-16 08:43:05.178 7 [info] <0.28527.2>@blockchain_state_channel_handler:handle_data:165 sc_handler client got purchase, sc_id: <<239,212,148,161,200,235,144,90,145,142,235,136,208,238,13,36,54,255,217,161,210,159,165,80,205,163,47,124,192,131,194,251>>

lorawan-sniffer was waiting at the endpoint lorawan-sniffer -- --in-port 1680 --enable-tmst
Here is the complete output:
[2021-08-16 08:43:00.891]  JoinRequest	904.7 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,775,410
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 839B
[2021-08-16 08:43:02.893]  JoinRequest	904.7 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,775,410
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 839B
[2021-08-16 08:43:04.894]  JoinRequest	904.7 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,775,410
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 839B
[2021-08-16 08:43:05.292]  JoinRequest	904.1 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,779,720
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 7CC5
[2021-08-16 08:43:06.895]  JoinRequest	904.7 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,775,410
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 839B
[2021-08-16 08:43:07.293]  JoinRequest	904.1 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,779,720
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 7CC5
[2021-08-16 08:43:08.896]  JoinRequest	904.7 MHz 	SF10BW125	RSSI: -112	LSNR: 5.5
	tmst: 1,310,775,410
	  AppEui: 00250C0000010000 DevEui: FFFFFF010000A0EE DevNonce: 839B

The output here has two issues.
The timestamps are all off
Recall that tmst is expressed in ns μs. Therefore, the virtual device emits the frames: (6101071 - 908)/10**6 = 6.100163 seconds apart which is a JoinRequest, and two expired RX windows followed by an immediate second JoinRequest. Note that these timestamps are parsed properly in the Miner logs. Meanwhile, the reported timestamps to the lorawan-sniffer by packet-purchaser are incredible:<PHONE_NUMBER> -<PHONE_NUMBER>)/10**6 = 0.00431 seconds apart.
The same two packets appear many times
You can determine this by the unique DevNonce where 839B appears 5 times and 7CC5 appears 2 times. A single Miner is being used with a virtual device here, so there is no chance that these are being reported by another Miner.

But I don't think we change TMST at all, other than to keep it bounded within a 32 bit number. Isn't that the LNS' job?

https://github.com/helium/packet-purchaser/blob/811e9a3202753106f8af3301c197554f9ffdd85c/src/state_channels/pp_sc_packet_handler.erl#L83 is the only place that seems to change TMST at all.

Additionally, according to https://www.thethingsnetwork.org/forum/uploads/default/original/1X/4fbda86583605f4aa24dcedaab874ca5a1572825.pdf page 11, the tmst is in microseconds, not nanoseconds.

The duplicate packet issue does look worth investigating.

Do you know if the same packet came in several times from the same udp source port/gwid or if those changed?

Is your lorawan-sniffer sending PULL_ACK messages? I think the packet purchaser is not very tolerant of missing PULL_ACK messages and it will close/reconnect the udp connection (which may trigger resends).

Yeah, not sending a PUSH_ACK message in response to a PUSH_DATA udp packet will trigger a re-send: https://github.com/helium/packet-purchaser/blob/master/src/pp_udp_worker.erl#L128
This is not correct, and I've filed #30 to address that.

What is the explanation for the tmst going from 908 -> 274 -> 1,310,775,410

Yeah, not sending a PUSH_ACK message in response to a PUSH_DATA udp packet will trigger a re-send: https://github.com/helium/packet-purchaser/blob/master/src/pp_udp_worker.erl#L128
This is not correct, and I've filed #30 to address that.

That seems like a good explanation then and makes it less bothersome. LoRaWAN sniffer is just receiving the packets and doing nothing else about the protocol

What is the explanation for the tmst going from 908 -> 274 -> 1,310,775,410

My bad on this. The Miner logs are from a different tracing session, but they did match the output from the virtual device is the point

Fixed by #31


fix links in images.md

this PR fix links in Static Non-Image Resources 'metro links', in version-0.62 it does't exists images.md so i used docusaurus docs to add  images.md in version-0.62 and fix this links.!
when you click this links appears error 404 so i fix that.

The correct way to fix this issue will be correcting the links in the version which is load for 0.62 docs, rather than creating new page.

thanks @Simek !! i found this file in version-0.5


Update code to take advantage of explicit nullability.

Example for enabling it:
https://agirlamonggeeks.com/2019/10/20/how-to-turn-c-8-nullable-reference-types/

WPF itself doesn't really make it feasible to enable nullability for this project at this time.


Allow users to set hdfs_namenode_principal in HDFSHook config

snakebite library just added the support to specify hdfs_namenode_principal for Kerberos auth method, and this PR allows users to pass in this config from HDFSHook

Please review @mistercrunch @bolkedebruin
Thanks!

@garthcn this won't be enough at it is dependent on a snakebite release. Is that new release available yet? If so then increase the requirements in setup.py. If not we have to wait :-)

@bolkedebruin snakebite has a new release https://pypi.python.org/pypi/snakebite/ and I just verified that it includes the PR I mentioned.
And in https://github.com/airbnb/airflow/blob/master/setup.py#L56, it has hdfs = ['snakebite>=2.4.13'] so it should install the latest snakebite :)

But it won't install a new version. 2.7.8 is now required for existing installations. So it does need to be updated.

@bolkedebruin that makes sense. i just bumped the version in setup.py to the lastest for snakebite

@artwr, can you run our local tests against this for a final blessing?! Feel free to merge once you confirm it works.

@mistercrunch @artwr any update on this?

Sorry, I am a little tied up at the moment, but I promise I will get to it asap...

@artwr no worries. thanks for working on this :)

Ping :) @artwr

@garthcn can you rebase? i will merge then

@bolkedebruin just rebased to master and combined all commits.thanks!

Coverage decreased (-0.02%) to 63.508% when pulling 4af24ee2df42bd4917217788f1f0782d031f139f on garthcn:hdfs-principal into ae8062aff483b575941770723c727aa5da1700be on airbnb:master.