False positives using when using pytest-xdist -f (--looponfail)

When doing TDD, after the first mypy error, the plugin start emitting errors on collection.
It will continue normally until you hit an actual mypy fail, then in the follow round (after mypy error is fixed) this warning is emitted forever until pytest is killed and restarted.
Steps to Reproduce (assumes python 3.8 and pipenv available):

mkdir ~/tmp20200412
cd ~/tmp20200412
git clone git@bitbucket.org:y2kbugger/test_driven_development_by_example_tdd_follow_along.git
cd ./test*
git checkout b0a2f56
replicate versions exactly from Pipfile.lock pipenv sync
put venv on PATH pipenv shell
run pytest pytest --mypy -f money.py test_money.py

In another terminal:

cd ~/tmp20200412/test*
cause mypy error echo 'lol: str = 2' >> money.py
undo mypy error git checkout b0a2f56 -- .

###################################### waiting for changes #######################################
### Watching:   /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
# MODIFIED /home/y2k/devel/test_driven_development_by_example_tdd_follow_along/money.py
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
plugins: forked-1.1.3, mypy-0.6.1, xdist-1.31.0, flake8-1.0.4
collected 0 items / 1 error
collected 0 items / 1 error

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/runner.py:244: in from_call
    result = func()
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/runner.py:264: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/main.py:498: in collect
    yield from self._collect(fspath, parts)
../../.local/share/virtualenvs/test_driven_development_by_example_tdd_fol-TPMT2dI5/lib/python3.8/site-packages/_pytest/main.py:537: in _collect
    assert not names, "invalid arg {!r}".format((argpath, names))
E   AssertionError: invalid arg (local('/home/y2k/devel/test_driven_development_by_example_tdd_follow_along'), ['mypy'])
=========================== short test summary info ============================
ERROR  - AssertionError: invalid arg (local('/home/y2k/devel/test_driven_deve...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.22s ===============================
######################################### LOOPONFAILING ##########################################
money.py::mypy
test_money.py::test_multiplication
test_money.py::test_franc_multiplication
test_money.py::test_equality
test_money.py::test_currency
::mypy
###################################### waiting for changes #######################################
### Watching:   /home/y2k/devel/test_driven_development_by_example_tdd_follow_along

Note I kill with keyboard interrupt here:
^C

Then everything goes fine again:
(test_driven_development_by_example_tdd_follow_along) [y2k@15r test_driven_development_by_example_tdd_follow_along]
$ pytest --mypy -f money.py test_money.py
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /home/y2k/devel/test_driven_development_by_example_tdd_follow_along
plugins: forked-1.1.3, mypy-0.6.1, xdist-1.31.0, flake8-1.0.4
collected 7 items
collected 7 items

money.py .                                                               [ 12%]
test_money.py ......                                                     [ 87%]
. .                                                                      [100%]. [100%]
===================================== mypy =====================================

Success: no issues found in 2 source files
============================== 9 passed in 0.34s ===============================
###################################### waiting for changes #######################################
### Watching:   /home/y2k/devel/test_driven_development_by_example_tdd_follow_along

Thanks for the excellent repro steps! 💯
The problem seems to have been introduced in 0.6.0 (#79) 🤔

I did notice that I was passing both the source file and the test file to
pytest but I think I was able to reproduce when only passing the test file.
I will get back to you on that.
On Sun, Apr 12, 2020, 17:17 David Tucker<EMAIL_ADDRESS>wrote:

Thanks for the excellent repro steps! 💯
The problem seems to have been introduced in 0.6.0 (#79
https://github.com/dbader/pytest-mypy/pull/79) 🤔
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dbader/pytest-mypy/issues/85#issuecomment-612677120,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABORVLM2V5ZGUMKNUNLAR63RMIVVFANCNFSM4MGQ6OJA
.

Can confirm error occurs even if ran properly:
pytest -f --mypy ./test_money.py

When a test fails, pytest-xdist holds onto the node ID of the failing test. The MypyStatusItem that is injected in pytest_collection_modifyitems has parent=session which produces a node ID of ::mypy (where mypy is nodeid_name).
Running that node ID explicitly (like xdist does) reproduces this:
pytest --verbose --mypy ::mypy

================================================ test session starts =================================================
platform linux -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /home/dtux/Projects/pytest-mypy/venv/bin/python
cachedir: .pytest_cache
Using --randomly-seed=1597611273
rootdir: /home/dtux/Projects/pytest-mypy
plugins: xdist-2.0.0, forked-1.3.0, randomly-2.1.1, cov-2.5.1, mypy-0.6.3.dev21+gd052c4c.d20200816
collected 0 items / 1 error

======================================================= ERRORS =======================================================
___________________________________________ ERROR collecting test session ____________________________________________
venv/lib/python3.8/site-packages/_pytest/runner.py:294: in from_call
    result = func()  # type: Optional[_T]
venv/lib/python3.8/site-packages/_pytest/runner.py:324: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
venv/lib/python3.8/site-packages/_pytest/main.py:576: in collect
    yield from self._collect(fspath, parts)
venv/lib/python3.8/site-packages/_pytest/main.py:617: in _collect
    assert not names, "invalid arg {!r}".format((argpath, names))
E   AssertionError: invalid arg (local('/home/dtux/Projects/pytest-mypy'), ['mypy'])
============================================== short test summary info ===============================================
ERROR  - AssertionError: invalid arg (local('/home/dtux/Projects/pytest-mypy'), ['mypy'])
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================== 1 error in 0.08s ==================================================

Would you mind cutting a release with this? I would love to have this working.

Fixed in https://pypi.org/project/pytest-mypy/0.8.0/


NSFW hqporner.com

adults only

I think this is legitime :smile_cat:

(.)(.)

Fixed in https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist/commit/fcaa791f48194829bd090e8a15d87bb4de6b0e1d

I'm reopening this and I'll for sure add this to whitelist but @mitchellkrogza have to explain why is this into BaddBoysHosts ? :thinking:


Add outer for Dask Arrays

Fixes https://github.com/dask/dask/issues/3656
Adds an implementation of NumPy's outer for Dask Arrays. Includes tests and API documentation.

[x] Tests added / passed
[x] Passes flake8 dask

Looks good to me.  Merging.  Thanks @jakirkham !


05_00_ARC05_9_2B.txt

Should probably standardize on something 土砂, because it was previously translated as

dirt/earth and sand
sediment
dust

and not all of those are totally the same thing. earth+sand seems most reasonable of those 3 (sediment feels a bit alluvial, dust too insubstantial). Feels a bit weird for sand to come in though, it's not something cities are usually built on. Might suggest just overwriting it with detritus/debris for clarity.

Still need to playtest this myself, but I've been at this file on and off for quite a few hours and am reaching the end of what's productive. Will try and get to that early tomorrow.


ZNDMWG03LM on version 1.4.5_0012 - how to get to 1.4.6_0012?

Helo @AlexxIT,
I bought the Xiaomi Gateway 3 from China and it has the fw version: 1.4.5_0012.
I used the Packet Sender Tool to obtain the token but unfortunately, I am not able to register the Gateway in the HA - the error says "ERROR: No response from the device".
I read your attention note that only firmwares v1.4.6_0012 and v1.4.6_0030 are supported. Since I am on the version 1.4.5_0012 and the Xiaomi Home app suggests me to update to the latest version 1.4.5_0016 (only?), I wanted to ask you how can I get to the version v1.4.6_0012? Or did not you mean v1.4.5_0012 instead?
I really do not want to "try" if I can get to the higher version once I update to 1.4.5_0016 as the Xiaomi Home app suggests. What is your recommendation?
Thank you in advance for your response.
AG

after you update to 1.4.5_0016 it will offer another ota. some updates must be done one by one.

As @rezmus said - you can safely upgrade 1.4.5_0012

Sorry guys for my late response.
I tried that as you helped me and boom... it works very nice now!!!
Thanks you a lot!


[BUG] VM got IO error after host restart

Describe the bug
VM can't restart after a host accidentally restarts. VMI changes frequently between ready and unready and it has VMI was paused, IO error error message. At first, related Volume's robustness is fault and engine is stopped. After around 30 minutes, the volume is attached and engine is running, but VMI still can't get ready. We have to check why VM can't be ready after the LH engine is back.
> kubectl get vmi -w
NAME                          AGE   PHASE     IP             NODENAME           READY
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   False
rke2-3-pool1-47f7bd68-r99dx   44m   Running   <IP_ADDRESS>   harvester-node-2   False
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   False
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   True
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   False
rke2-3-pool1-47f7bd68-r99dx   44m   Running   <IP_ADDRESS>   harvester-node-2   False
rke2-3-pool1-47f7bd68-r99dx   44m   Running   <IP_ADDRESS>   harvester-node-2   True
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   True
rke2-3-pool1-47f7bd68-r99dx   44m   Running                  harvester-node-2   False

To Reproduce
Steps to reproduce the behavior:

Create a 3-node harvester cluster.
Provision a 3-node RKE2 cluster on the harvester and wait for it's ready.
Choose one node to stop kubelet.
a. Find the pid of kubelet process by ps -aux | grep kubelet.
b. Freeze the kubelet process by kill -STOP <kubelet-pid>.
After the node is unavailable. Run shutdown now -r.
After the node is running again, the VM can't be ready.

Expected behavior
The RKE2 VM can be migrated to other hosts and the VM can be ready after a while.
Support bundle
supportbundle_f3b2af65-e81b-4c19-939a-14ff8688c5a1_2023-10-19T03-37-52Z.zip
Environment

Harvester ISO version: v1.2.0
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): KVM

Additional context
Related LH issues:
https://github.com/longhorn/longhorn/issues/6919
https://github.com/longhorn/longhorn/issues/6961
Work around
Restart the failed VM and it can get back and be ready after rescheduling.

Updated reproduce steps and work around. New steps are more reproducible. I tried six times and I can get a unready VM in four times.
From VM scheduling perspective, we can try to restart VM if it matches following conditions:

PVC is bound and related Volume is healthy.
VM pod is running but there is ReadinessGatesNotReady in the condition.
VMI has PausedIOError.

New support bundle. In this bundle, the volume is healthy, but the VMI can't be ready.
supportbundle_aed7c5f9-39d1-4c1d-8fa8-b1a09baceea4_2023-10-20T03-08-34Z.zip

If we check log in VM pod, we can see it keeps running Synced vmi with IO Error.
{"component":"virt-launcher","kind":"","level":"info","msg":"Synced vmi","name":"rke2-2-pool1-926e1b37-fx2bl","namespace":"default","pos":"server.go:190","timestamp":"2023-1
0-20T06:42:25.787009Z","uid":"feeaed37-53a7-4c61-bedd-d56b2f57e61e"}
{"component":"virt-launcher","level":"info","msg":"Domain name event: default_rke2-2-pool1-926e1b37-fx2bl","pos":"client.go:413","timestamp":"2023-10-20T06:42:25.821255Z"}
{"component":"virt-launcher","level":"info","msg":"kubevirt domain status: Paused(3):IOError(5)","pos":"client.go:288","timestamp":"2023-10-20T06:42:25.823399Z"}
{"component":"virt-launcher","level":"info","msg":"DomainLifecycle event 4 with reason 0 received","pos":"client.go:435","timestamp":"2023-10-20T06:42:25.833676Z"}
{"component":"virt-launcher","level":"info","msg":"DomainLifecycle event 3 with reason 2 received","pos":"client.go:435","timestamp":"2023-10-20T06:42:25.852570Z"}
{"component":"virt-launcher","level":"info","msg":"Libvirt event channel is full, dropping event.","pos":"client.go:443","timestamp":"2023-10-20T06:42:25.852663Z"}
{"component":"virt-launcher","kind":"","level":"info","msg":"Domain unpaused.","name":"rke2-2-pool1-926e1b37-fx2bl","namespace":"default","pos":"manager.go:901","timestamp":
"2023-10-20T06:42:25.868221Z","uid":"feeaed37-53a7-4c61-bedd-d56b2f57e61e"}
{"component":"virt-launcher","kind":"","level":"info","msg":"Synced vmi","name":"rke2-2-pool1-926e1b37-fx2bl","namespace":"default","pos":"server.go:190","timestamp":"2023-1
0-20T06:42:25.870412Z","uid":"feeaed37-53a7-4c61-bedd-d56b2f57e61e"}
{"component":"virt-launcher","level":"info","msg":"Domain name event: default_rke2-2-pool1-926e1b37-fx2bl","pos":"client.go:413","timestamp":"2023-10-20T06:42:25.929426Z"}
{"component":"virt-launcher","level":"info","msg":"DomainLifecycle event 4 with reason 0 received","pos":"client.go:435","timestamp":"2023-10-20T06:42:25.939416Z"}
{"component":"virt-launcher","level":"info","msg":"DomainLifecycle event 3 with reason 2 received","pos":"client.go:435","timestamp":"2023-10-20T06:42:25.947074Z"}
{"component":"virt-launcher","level":"info","msg":"kubevirt domain status: Paused(3):IOError(5)","pos":"client.go:288","timestamp":"2023-10-20T06:42:25.947162Z"}
{"component":"virt-launcher","level":"info","msg":"Libvirt event channel is full, dropping event.","pos":"client.go:443","timestamp":"2023-10-20T06:42:25.947174Z"}
{"component":"virt-launcher","kind":"","level":"info","msg":"Domain unpaused.","name":"rke2-2-pool1-926e1b37-fx2bl","namespace":"default","pos":"manager.go:901","timestamp":
"2023-10-20T06:42:25.954738Z","uid":"feeaed37-53a7-4c61-bedd-d56b2f57e61e"}

If we run kg vm rke2-2-pool1-926e1b37-fx2bl -o jsonpath='{.status.printableStatus}' -w, it changes between Paused and Running.
Paused
Running
Running
Paused
Paused

The bug is similar with VM status flipping between Paused and Running.

Previous support bundle is based on harvester v1.2.0. This one is based on v1.2.1-rc1. The behavior is same as previous one.
supportbundle_e6f9e2ea-cdcc-4703-8aff-4157f3646dba_2023-10-24T06-02-42Z.zip

In the recreate I analyzed for https://github.com/longhorn/longhorn/issues/6961, the high level issue was that Longhorn intentionally crashed the volume underneath the VM. The VM experienced an I/O error when it tried to write to the crashed volume. Not sure if the exact lead up to the crash is always the same (need to check all the support bundles for that). Longhorn automatically reattaches the volume (which is why it looks good), but it is not usable without restarting the workload.
We should definitely work towards a fix for this on the Longhorn side, though we may need the virtualization layer to change its behavior a bit as well.
I think it may still be a good idea to implement https://github.com/harvester/harvester/pull/4645. Longhorn automatically restarts workload pods when their corresponding volumes have crashed (if the setting is enabled) to avoid similar situations where a workload is stuck indefinitely. The logic is essentially, since there's no other way to recover, we may as well preemptively take the recovery action. (I know there's a PR comment about being too silent when doing this, which I think is also valid.)

@bk201 @FrankYang0529
Given the decision https://github.com/harvester/harvester/pull/4645#issuecomment-1777299500, would suggest we add this issue and workaround into Harvester doc as well. thanks.

@FrankYang0529 @ejweber
Happend to filter this issue https://github.com/harvester/harvester/issues/1905 in context of https://github.com/harvester/harvester/issues/4659.
In #1905 , the VM re-balancing looked caused by others, but the failure

but the other 3 existing vms are now blinking (paused/running)

seemed to be a bit like here #4633.

In this support bundle, there are 2 running engine and VMI can be back without restart.
supportbundle_e15b5165-f184-45fc-a9ea-02224b33c0de_2023-10-27T02-59-47Z.zip

Given the decision #4645 (comment), would suggest we add this issue and workaround into Harvester doc (https://docs.harvesterhci.io/v1.3/troubleshooting/index with a new chapter like Virtual Machine) as well. thanks.

Sounds good. I will add it.

Wait for LH v1.4.5 and bump it in Harv v1.2.2.

IMO Longhorn v1.4.5 will not be enough.
Longhorn v1.4.5 will mitigate against the I/O errors in this issue by fixing https://github.com/longhorn/longhorn/issues/6961. However, there is an underlying issue in Kubernetes that will keep Longhorn volumes attached to two nodes indefinitely after unplanned reboots. In most cases https://github.com/longhorn/longhorn/issues/6961 will keep the Longhorn engine alive on the new node the VM has migrated to, but we will observe a number of additional issues until the volume is detached from the old node.
This behavior will be entirely fixed upstream by https://github.com/kubernetes/kubernetes/pull/122211 (tracked in https://github.com/longhorn/longhorn/issues/6919), but we will need a new Kubernetes version to make use of it.
The workaround for https://github.com/longhorn/longhorn/issues/6919 is to restart the kubelet after the node has come back online and is logging the following:
E1019 01:11:18.316662     967 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/csi/driver.longhorn.io^pvc-37461bdd-7133-4111-8fb7-c4ddf8dacca5 podName:19e41dfe-8cee-40c2-a39b-37c68b01c9a7 nodeName:}" failed. No retries permitted until 2023-10-19 01:13:20.316609446 +0000 UTC m=+1883.799156551 (durationBeforeRetry 2m2s). Error: UnmapVolume.UnmapBlockVolume failed for volume "pvc-37461bdd-7133-4111-8fb7-c4ddf8dacca5" (UniqueName: "kubernetes.io/csi/driver.longhorn.io^pvc-37461bdd-7133-4111-8fb7-c4ddf8dacca5") pod "19e41dfe-8cee-40c2-a39b-37c68b01c9a7" (UID: "19e41dfe-8cee-40c2-a39b-37c68b01c9a7") : blkUtil.DetachFileDevice failed. globalUnmapPath:, podUID: 19e41dfe-8cee-40c2-a39b-37c68b01c9a7, bindMount: true: failed to unmap device from map path. mapPath is empty

This was not possible before without causing further I/O errors, but https://github.com/longhorn/longhorn/issues/7302 (which will also be fixed in v1.4.5) allows for it.
In summary, Harvester needs at least Longhorn v1.4.5, v1.5.4, or v1.6.0 AND one of the following:

A Kubernetes version with https://github.com/kubernetes/kubernetes/pull/122211, or
A mechanism by which the Kubelet is restarted when it starts logging the above.

cc @phanle1010

@FrankYang0529
Could we double check, with bumped LH, k8s version (>=v1.2.6 release) in Harvester v1.3.0, this issue has been fixed?
cc @bk201 @markhillgit : should we move this issue to v1.3.0 to validate, and add backport v1.2.2 ?
BTW: the fix of this issue, is a very important enhancement of https://github.com/harvester/harvester/issues/3261

Please move to ready for testing if appropriate, I've optimistically moved this into 1.3.0 with 1.2.2 backport per the comments above

Currently, we are still waiting for RKE2 release. Both v1.26.13 and v1.27.10 are pre-released.

https://github.com/rancher/rke2/releases/tag/v1.26.13-rc2%2Brke2r1
https://github.com/rancher/rke2/releases/tag/v1.27.10-rc2%2Brke2r1

RKE2 v1.27.10+rke2r1 has been released
https://github.com/rancher/rke2/releases/tag/v1.27.10%2Brke2r1
If we bump it per #5041, then v1.3.0 will fix this important issue.

To QA:
When validating this issue, please note those issues are caused by same root cause though they have slightly different phenomenon.
#4633
#4033
#3539
#4789

Provide test result on v1.3.0-rc3, close as fixed.
Environment

Harvester

Version: v1.3.0-rc3
Profile: QEMU/KVM, 3 nodes (8C/16G/500G)
ui-source: Auto

Steps

Create a VM with LVM

 Mount LVM and generate some test data :heavy_check_mark:

Restart the node VM on (1st try)

 Freeze kubelet and node is Unavailable, then restart node :heavy_check_mark:

 VM becomes Not Ready :heavy_check_mark:

 Node is back and VM can be Running again after a while :heavy_check_mark:

Restart the node VM on (2nd try)

 Freeze kubelet and node is Unavailable, then restart node :heavy_check_mark:

 VM becomes Not Ready :heavy_check_mark:

 Node is back and VM can be Running again after a while :heavy_check_mark:

Restart the node VM on (3rd try)

 Freeze kubelet and node is Unavailable, then restart node :heavy_check_mark:

 VM becomes Not Ready :heavy_check_mark:

 Node is back and VM can be Running again after a while :heavy_check_mark:

Note

Can be reproduce on v1.2.1

Host is back, but VM still NotReady


"with-vercel-website" template does not save page Hero Type correctly (its value is undefined). Type always defaults to 'lowImpact'.

Describe the Bug
In the config.ts, the Hero type should be saved correctly accordingly to what is set in the admin dashboard. Unfortunately, the value is lost somewhere this road and ends up being undefined to which it defaults to 'lowImpact'. This makes it that the hero image and links do not display on the homepage.
Link to the code that reproduces this issue
https://github.com/payloadcms/payload/tree/main/templates/with-vercel-website
Reproduction Steps
Follow the steps in the README file:
First clone the repo if you have not done so already
cd my-project && cp .env.example .env to copy the example environment variables. You'll need to add the POSTGRES_URL and BLOB_READ_WRITE_TOKEN from your Vercel project to your .env if you want to use Vercel Blob and the Neon database that was created for you.
pnpm install && pnpm dev to install dependencies and start the dev server
open http://localhost:3000 to open the app in your browser
Which area(s) are affected? (Select all that apply)
db-vercel-postgres, area: core
Environment Info
Binaries:
  Node: 22.2.0
  npm: 10.7.0
  Yarn: N/A
  pnpm: 9.14.4
Relevant Packages:
  payload: 3.2.2
  next: 15.0.3
  @payloadcms/email-nodemailer: 3.2.2
  @payloadcms/graphql: 3.2.2
  @payloadcms/live-preview: 3.2.2
  @payloadcms/live-preview-react: 3.2.2
  @payloadcms/next/utilities: 3.2.2
  @payloadcms/payload-cloud: 3.2.2
  @payloadcms/plugin-cloud-storage: 3.2.2
  @payloadcms/plugin-form-builder: 3.2.2
  @payloadcms/plugin-nested-docs: 3.2.2
  @payloadcms/plugin-redirects: 3.2.2
  @payloadcms/plugin-search: 3.2.2
  @payloadcms/plugin-seo: 3.2.2
  @payloadcms/richtext-lexical: 3.2.2
  @payloadcms/storage-vercel-blob: 3.2.2
  @payloadcms/translations: 3.2.2
  @payloadcms/ui/shared: 3.2.2
  react: 19.0.0-rc-65a56d0e-20241020
  react-dom: 19.0.0-rc-65a56d0e-20241020
Operating System:
  Platform: darwin
  Arch: x64
  Version: Darwin Kernel Version 22.6.0: Mon Apr 22 20:54:28 PDT 2024; root:xnu-87<IP_ADDRESS>5.2~1/RELEASE_X86_64
  Available memory (MB): 16384
  Available CPU cores: 8

Fixed it by using the page.tsx from @/app/(frontend)/[slug]/page.tsx from the normal website template. The main difference is in the page variable initialisation and the metadata generation. I think that the homepage stays homeStatic from before the seeding and does not switch to the default home after the seeding.


Startup issue when authentication fails.

Startup fails when authentication fails.
When the program attempts to contact the pdu with incorrect authentication startup fails with an error indicating the incorrect mime type.
[2023-10-16 16:42:52,911] ERROR: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://pdu-rdc-17-red1.evergreen.laika.com/bulk')
[2023-10-16 16:42:52,912] CRITICAL: Exporter shut down due to an error during the setup procedure. Please contact your administrator.

The pdu does correctly throw a 401 and if you are browsing to the above it presents a dialog for logging in which may be the source of mimetype error.
[root@core-vm-tsdb-staging prometheus_raritan_pdu_exporter]# wget -v --user=baduser --password=bogus --no-check-certificate https://pdu-rdc-39-blue1.admin.evergreen.laika.com/bulk
--2023-10-16 11:40:12--  https://pdu-rdc-39-blue1.admin.evergreen.laika.com/bulk
Resolving pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)... <IP_ADDRESS>
Connecting to pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)|<IP_ADDRESS>|:443... connected.
WARNING: cannot verify pdu-rdc-39-blue1.admin.evergreen.laika.com's certificate, issued by ‘/CN=Generic Demo CA’:
  Unable to locally verify the issuer's authority.
    WARNING: certificate common name ‘Demo Certificate’ doesn't match requested host name ‘pdu-rdc-39-blue1.admin.evergreen.laika.com’.
HTTP request sent, awaiting response... 401 Unauthorized
Connecting to pdu-rdc-39-blue1.admin.evergreen.laika.com (pdu-rdc-39-blue1.admin.evergreen.laika.com)|<IP_ADDRESS>|:443... connected.
WARNING: cannot verify pdu-rdc-39-blue1.admin.evergreen.laika.com's certificate, issued by ‘/CN=Generic Demo CA’:
  Unable to locally verify the issuer's authority.
    WARNING: certificate common name ‘Demo Certificate’ doesn't match requested host name ‘pdu-rdc-39-blue1.admin.evergreen.laika.com’.
HTTP request sent, awaiting response... 401 Unauthorized
Authorization failed.

If its possible to catch and warn on this as the unreachable hosts that would be awesome.
Not sure why the 401 doesn't throw an exception.

We are going to see if we can fix this in house if we do we will make a pull request.
This is what our tech lead thought we should work toward.

Fix authentication==wrong mime type
Flag and export pdu state for unreachable devices at startup.
Re query devices when they move from down to up.


Dealing with step function ~1984

Hi there! Thanks for all of the work on this tool - super exciting to have these options in GEE!
I am testing this out for some Arctic applications and have been walking through this script for northern Alaska coastlines. Using the inspector after running Step 7 to build the final collection for all Landsat data, I have a clear change at 1984 as mentioned specifically in the readme to watch out for.

I'm wondering what the troubleshooting options are when this happens? My assumption is that this means the relationship between the old MSS and new TM/OLI sensors isn't properly established (?) but am not sure where to begin to try and resolve this issue.
I ran the script as-is, except that I used the existing options for adding in a dummy image for the MSS collections, as well as added in this functionality for building the final TM/OLI collections at Step 7 for years when no good images existed.
Thanks for your time!

Hi @nkhelder sorry for the slow reply. This is a very side project that occasionally gets sprint attention and to this point basically no support. The JS version that you've run is replaced by a Python version that runs in a Colab notebook. Over the next couple of months there are two conferences where the new version will be described. This repo will be updated with the new workflow. The step function is still a possibility with the newest version, but I've done a couple of things to reduce it. Very cool that you got results out of it for AK (primary testing has been in OR and CA). I'll come back around to this thread when the conferences are over and see if you are still interested in trying again with the new workflow (it's way more automated!).

Hi @jdbcode! No worries, I ended up getting it working and have been using it across the northern AK coastline - very cool! Looking forward to learning about the newest version.. I will be at Geo for Good (if that's one of the conferences you are referring to), so will look at for updates there. Thanks!

Great you'll be at G4G! @annietaylor and I'll be presenting the latest version on Oct 6th 10:15-11:15 (I think). It would be nice to have you there.


TestImage errors out in IJulia

When using TestImage as follows with Julia v0.4.1,
using TestImages
im = TestImages("mandrill")

I get the following error:
LoadError: MethodError: `call` has no method matching call(::Module, ::ASCIIString)
Closest candidates are:
  BoundsError()
  BoundsError(!Matched::Any...)
  DivideError()
  ...
while loading In[1], in expression starting on line 2

Here's my Pkg status output:
julia> Pkg.status()
8 required packages:
 - Color                         0.2.11
 - Gadfly                        0.4.0
 - IJulia                        1.1.8
 - ImageMagick                   0.1.1              master
 - Images                        0.5.0+             master
 - QuartzImageIO                 0.1.0
 - RDatasets                     0.1.2
 - TestImages                    0.1.0
49 additional packages:
 - ArrayViews                    0.6.4
 - BinDeps                       0.3.19
 - Calculus                      0.1.14
 - Codecs                        0.1.5
 - ColorTypes                    0.2.0
 - ColorVectorSpace              0.1.0
 - Colors                        0.6.0
 - Compat                        0.7.8
 - Compose                       0.4.0
 - Conda                         0.1.8
 - Contour                       0.0.8
 - DataArrays                    0.2.20
 - DataFrames                    0.6.10
 - DataStructures                0.3.13
 - Dates                         0.4.4
 - Distances                     0.2.1
 - Distributions                 0.8.7
 - Docile                        0.5.19
 - DualNumbers                   0.1.5
 - FileIO                        0.0.3
 - FixedPointNumbers             0.1.1
 - GZip                          0.2.18
 - Graphics                      0.1.3
 - Grid                          0.4.0
 - Hexagons                      0.0.4
 - Homebrew                      0.2.0
 - ImmutableArrays               0.0.11
 - Iterators                     0.1.9
 - JSON                          0.5.0
 - KernelDensity                 0.1.2
 - Loess                         0.0.6
 - Measures                      0.0.1
 - NaNMath                       0.1.1
 - Nettle                        0.2.0
 - Optim                         0.4.4
 - PDMats                        0.3.6
 - Reexport                      0.0.3
 - SHA                           0.1.2
 - SIUnits                       0.0.6
 - Showoff                       0.0.6
 - SortingAlgorithms             0.0.6
 - StatsBase                     0.7.4
 - StatsFuns                     0.2.0
 - TexExtensions                 0.0.3
 - URIParser                     0.1.1
 - WoodburyMatrices              0.1.2
 - ZMQ                           0.3.1
 - ZipFile                       0.2.5
 - Zlib                          0.1.12

Try img = testimage("mandrill"). Note the lack of capitalization and the lack of an s at the end.
It's also worth pointing out that in julia, im is the imaginary-number constant. It's fine to reassign it as you've done above, but just so you know (in case you want to use it for something in the same session).

Thanks for the pointers. I restarted the kernel and tried the following:
using Images
using TestImages
using ImageMagick
img = testimage("autumn_leaves")

and I got:
 in error at /usr/local/Cellar/julia/0.4.1/lib/julia/sys.dylib
 in error at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:149
 in setimageformat at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:346
 in getblob at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:212
 in writemime_ at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/ImageMagick.jl:250
 in writemime at /Users/tamasnagy/.julia/v0.4/Images/src/writemime.jl:28
 in base64encode at base64.jl:160
 in display_dict at /Users/tamasnagy/.julia/v0.4/IJulia/src/execute_request.jl:32

 in error at /usr/local/Cellar/julia/0.4.1/lib/julia/sys.dylib
 in error at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:149
 in setimageformat at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:346
 in getblob at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/libmagickwand.jl:212
 in writemime_ at /Users/tamasnagy/.julia/v0.4/ImageMagick/src/ImageMagick.jl:250
 in writemime at /Users/tamasnagy/.julia/v0.4/Images/src/writemime.jl:28
 in base64encode at base64.jl:160
 in display_dict at /Users/tamasnagy/.julia/v0.4/IJulia/src/execute_request.jl:32

You may want to try these:
https://github.com/timholy/Images.jl/blob/master/doc/install.md#fixing-broken-installations-on-macs
https://github.com/JuliaLang/Homebrew.jl/wiki/Debugging-Homebrew.jl
Also, try Pkg.rm("ImageMagick"); Pkg.add("QuartzImageIO"). That may be good enough to support reading.

I forgot about this issue, but ran into a problem with this again. Your first link is broken?

I am having the same problem in the jupyter notebook on OS X. QuartzImageIO does nothing, or at least trying to display the image results in a suggestion to use ImageMagick. I got it to run without an error by checking out the latest master for ImageView and ImageMagic and calling the following code. However, nothing is displayed.
using TestImages
using ImageMagick
using ImageView
img = testimage("mandrill")

not sure where to go from here. any pointers?

Ok I fixed it. it was quite involved. The issue is I had image magick already installed with my global homebrew installation and that messed everything up. It seems Homebrew.jl tried to use this global installation but failed to do so. The "debugging homebrew" tip didn't work after this point, because imagemagick was to interweaved it seems. I had to completely delete the folders for Homebrew.jl, and ImageMagick.jl (I also did that for Images.jl and ImageView.jl to be safe), and then deinstall imagemagick from the global homebrew, before running Julia. naturally messing with the Homebrew.jl folder means I had to build a couple of packages before they were usable again (like IJulia). All in all this was quite messy.
Pro tip: do not by the love of god install imagemagick using the global homebrew installation

Is there anyway to do this without breaking the global install of imagemagick? I need it for other things.

Sorry, but I don't know. I took the path of least resistance. I am not even 100% certain I identified down the true cause. All I know is it works now.
One might get away with installing imagemagick using homebrew after the Julia one is set up properly, but I didn't try (yet).

Regarding QuartzImageIO, you might need to be on master (Pkg.checkout("QuartzImageIO")). It hasn't been tagged in a while (CC @rsrock).

Running Pkg.checkout("QuartzImageIO"); Pkg.build("QuartzImageIO") did the trick. I'm now getting images displayed properly in IJulia after running using QuartzImageIO.
The QuartzImageIO README suggests that I shouldn't do this though?

QuartzImageIO will be used as needed if you've said

using FileIO

in your session or module. You should not generally say using QuartzImageIO. See FileIO for further details.

However, using FileIO followed by img = load("test.png") still gives me the following error:
No IO library loaded for writemime image/png with Images.Image{ColorTypes.RGB4{FixedPointNumbers.UFixed{UInt8,8}},2,Array{ColorTypes.RGB4{FixedPointNumbers.UFixed{UInt8,8}},2}}.
            Please consider putting "using ImageMagick" in your script

 in error at /usr/local/Cellar/julia/0.4.3/lib/julia/sys.dylib
 in writemime at /Users/tamasnagy/.julia/v0.4/Images/src/writemime.jl:30
 in base64encode at base64.jl:160
 in display_dict at /Users/tamasnagy/.julia/v0.4/IJulia/src/execute_request.jl:32

which points me to ImageMagick, which has other problems. I feel like this is really confusing for new users.

I feel like this is really confusing for new users.

It certainly is. It's even confusing to the developers :smile:, trying to support the idiosyncrasies of multiple platforms that some of us don't even have access to.
To clarify what's happening, I bet you're using IJulia, and that if you put a semicolon after your load command you will avoid that error. If my guess is correct, it means that the error is being triggered by the attempt to display the image, but that the loading worked just fine.
It seems that QuartzImageIO probably needs the analog of this function. Care to give it a try? (Lacking a mac, I can't test this myself.)

To clarify what's happening, I bet you're using IJulia, and that if you put a semicolon after your load command you will avoid that error. If my guess is correct, it means that the error is being triggered by the attempt to display the image, but that the loading worked just fine.

I can answer this with a yes

It seems that QuartzImageIO probably needs the analog of this function. Care to give it a try?

I may take a shot at this as I currently have to work with Images a lot and just recently bought myself a macbook for travel and presentations (i.e. I'm looking for excuses to use it :) ). It may take a few days/weeks though.

To clarify what's happening, I bet you're using IJulia, and that if you put a semicolon after your load command you will avoid that error.

Can confirm

It seems that QuartzImageIO probably needs the analog of this function. Care to give it a try? (Lacking a mac, I can't test this myself.)

@timholy Doesn't QuartzImageIO already have this function? https://github.com/JuliaIO/QuartzImageIO.jl/blob/master/src/QuartzImageIO.jl#L270-L273

@tlnagy, good point!
OK, this may be a case where you should say using QuartzImageIO before use. If that fixes your problem, we should change the README and also modify the error message in Images.
This code seems like it's not working ideally. @SimonDanisch, I think that's your work from https://github.com/timholy/Images.jl/pull/380. Care to chime in? I'm not sure what the CurrentModule stuff is supposed to achieve. (It also looks like there might be a bug, in that the isdefined call was perhaps intended to be isdefined(CurrentModule, :writemime_). Or perhaps we should get rid of the CurrentModule stuff altogether?)

Huh...yeah there was some problem because FileIO only imports the IO libraries.
I would think, defined(CurrentModule, :writemime_) could solve it?

Pro tip: do not by the love of god install imagemagick using the global homebrew installation

It should work in theory, though! Maybe someone needs to do the same thing I did in https://github.com/JuliaIO/ImageMagick.jl/pull/28 for OSX...

I put in a @show CurrentModule. When I try IJulia, for me CurrentModule is Main. That won't help if the user hadn't already say using ImageMagick or using QuartzImageIO.
In other words, this isn't an OSX-only problem; I doubt it's doing anything useful on Linux, either, and presumably Windows is the same.

Okay... So I guess it wasn't a very good fix for whatever problem I had...
So this means FileIO needs to export writemime_ and every loaded IO module needs to overload it for it's own type? That'd mean IO libraries can overwrite it, but I guess that should be okay, as long as the IO library does the job!

I haven't thought it through carefully. Maybe FileIO could do internal module-wise dispatch on writemime_, similarly to how load and save are handled?

That was our initial idea, wasn't it? But maybe we got only stuck because of a lack of creativity ;)

Thanks to @timholy and @Evizero's hard work the following works in IJulia using the latest masters of FileIO, QuartzImageIO, and Image:
using FileIO, Image, TestImage,
img = testimage("mandrill")

🎉

using the latest masters

I don't think you have to be on master for this, since those changes are already tagged

Thanks, I meant to ask whether this worked with all the tagged versions. I take that as a yes.

I think it might have been due to me having local changes on QuartzImageIO. I ran Pkg.free([Images, FileIO, QuartzImageIO]) and Pkg.update()
Now my versions are:
 - FileIO                        0.0.4
 - Images                        0.4.50
 - QuartzImageIO                 0.0.2

and it works in IJulia.

Great, thanks for checking.


Build is not async

Some internals of the build crate were declared as async and had machinery to run in parallel.  From what I can tell, they never have been async, so the parallelism/join stuff wasn't doing anything - the builds have always been sequential.  The presence of async may be a hangover from during initial development (perhaps the author started out by using tokio::process::Command?).
This PR removes the async modifier and strips out the machinery for joining after parallel builds.
Signed-off-by: itowlson<EMAIL_ADDRESS>

Yes, they print build output as they go, and it would be super confusing to have multiple component build outputs overlapping...!


Fixed small type in default toolbar definition of "image"

There is a small copy & paste error in the name of the image toolbar element.

This has already been fixed in the upcoming version 1.6.0 (the "development" branch), but good catch!


DecodeError when passing Django task to Celery

I'm passing a task from my Django app to a Celery container (through rabbit-mq) but I keep getting the error below.
I've check that the LANG var is set to UTF8 (actually C.UTF8 in this case) but I haven’t found any other solutions when searching online which is why I'm posting here.
If anyone can see what the problem is I'd be most grateful.
[2015-09-17 23:13:33,486: CRITICAL/MainProcess] Can't decode message body: DecodeError(UnicodeDecodeError('ascii', b'\xe7\xc07\xf1\xfff\x14\x8ec\x1c\xc5\xf9\x86\xed\x86\x9a<\xe3M\xa0]x\x80\xd8;\xd5B&+8\xeab', 0, 1, 'ordinal not in range(128)'),) [type:'application/x-python-serialize' encoding:'binary' headers:{}]

body: b'\x80\x02}q\x01(U\x07expiresq\x02NU\x03utcq\x03\x88U\x04argsq\x04cpycoin.key.BIP32Node\nBIP32Node\nq\x05)\x81q\x06}q\x07(U\x06_depthq\x08K\x00U\x14_prefer_uncompressedq\t\x89U\x15_hash160_uncompressedq\nNU\x10_secret_exponentq\x0bNU\r_subkey_cacheq\x0c}U\x08_netcodeq\rU\x03BTCq\x0eU\x13_hash160_compressedq\x0fNU\x0b_chain_codeq\x10U \xe7\xc07\xf1\xfff\x14\x8ec\x1c\xc5\xf9\x86\xed\x86\x9a<\xe3M\xa0]x\x80\xd8;\xd5B&+8\xeabU\x0c_public_pairq\x11\x8a... (928b)
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 55, in _reraise_errors
    yield
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 184, in loads
    return decode(data)
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 345, in unpickle
    return pickle_loads(str_to_bytes(s))
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 64, in pickle_loads
    return load(BytesIO(s))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe7 in position 0: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/kombu/messaging.py", line 590, in _receive_callback
    decoded = None if on_m else message.decode()
  File "/usr/local/lib/python3.4/site-packages/kombu/message.py", line 142, in decode
    self.content_encoding, accept=self.accept)
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 184, in loads
    return decode(data)
  File "/usr/local/lib/python3.4/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 59, in _reraise_errors
    reraise(wrapper, wrapper(exc), sys.exc_info()[2])
  File "/usr/local/lib/python3.4/site-packages/kombu/five.py", line 132, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 55, in _reraise_errors
    yield
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 184, in loads
    return decode(data)
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 345, in unpickle
    return pickle_loads(str_to_bytes(s))
  File "/usr/local/lib/python3.4/site-packages/kombu/serialization.py", line 64, in pickle_loads
    return load(BytesIO(s))
kombu.exceptions.DecodeError: 'ascii' codec can't decode byte 0xe7 in position 0: ordinal not in range(128)

Thanks in advance :)

UPDATE:
Turns out the object I'm serialising has a field named '_chain_code' which is causing the problem, here is an example of one - '\xe7\xc07\xf1\xfff\x14\x8ec\x1c\xc5\xf9\x86\xed\x86\x9a<\xe3M\xa0]x\x80\xd8;\xd5B&+8\xeab'. How can I make celery ignore this or prevent it from trying to serialise it?
Thanks

The answer was to pass the id of the database object I wanted to use to my task and then have the task retrieve the object from the database when needed. The fact that I was passing an object into my task was the problem.

You have no idea my friend, how much this has helped me !!


Client doesn't work if third-party session storage is disabled

In Chrome (55.0.2883.95), for privacy reasons, I set "block third-party cookies and site data".
With that, the code below produces this error:

Uncaught DOMException: Failed to read the 'sessionStorage' property from 'Window': Access is denied for this document.
at x.C (https://ssl.gstatic.com/accounts/o/644096210-idpiframe.js:12:285)
at Object.k.vb (https://ssl.gstatic.com/accounts/o/644096210-idpiframe.js:8:121)
at S.h.start (https://ssl.gstatic.com/accounts/o/644096210-idpiframe.js:50:170)
at Object.Bb [as startIdpIFrame] (https://ssl.gstatic.com/accounts/o/644096210-idpiframe.js:84:713)
at https://accounts.google.com/o/oauth2/iframe:1:413

Code:
    gapi.load('client', function() {
      gapi.client.init({
        clientId: clientId,
        scope: scope
      }).then(function() {
        gapi.auth2.getAuthInstance().signIn().then(function() {
          var user = gapi.auth2.getAuthInstance().currentUser.get();
          processContacts(user.getAuthResponse()['access_token']);
        });
      });

This can be fixed by adding accounts.google.com to the exceptions list, or by allowing third-party storage wholesale, but really, this shouldn't be necessary; the client should degrade gracefully in the case where the browser doesn't allow third-party access to local storage.

Thanks for reporting this issue. The issue is already known to us, but I'm not sure when or whether a fix will be available.
In the meantime, you may be able to work around this using the steps described in http://stackoverflow.com/questions/33200681/google-signin-not-working-in-safari-private-mode or something similar, although I have not tried that myself.

@bsittler, I'm pretty sure that the workaround that you linked to does not help with the current issue.
(The workaround suggests replacing the #setItem() method of Storage.prototype when the browser is in private mode, because the native method throws an error, at least in Safari. The exception that @asadovsky reported above occurs when trying to access sessionStorage, i.e. before sessionStorage.setItem() could be called.)
A different workaround to avoid the "Access is denied" exception was proposed on the Chromium issue tracker (https://bugs.chromium.org/p/chromium/issues/detail?id=357625).
That workaround replaces localStorage and sessionStorage with proxy objects, using Object.defineProperty(window, 'localStorage', {value: /*your polyfill here*/ });.
This second workaround did not work for me -- the exception is still thrown. Could this be because the function that causes the error is running in an iframe, which has a different window object from the main page?
I'd appreciate any further suggestions :)

Correct, this error occurs in an IFRAME on a separate origin, so a polyfill won't work. I was intending to suggest that you could test for working storage and avoid loading the library if the storage does not work.

Ah, gotcha, thanks for the quick response.

@mbloch @asadovsky we released a change that allows you to gracefully degrade under such environment, please have a look at the documentation: https://developers.google.com/identity/sign-in/web/reference#googleauththenoninit-onerror.

@TMSCH do you mean gracefully degrade as it show an error message telling them they need to enable 3rd party cookies?

@mjgallag yes, as you are now able to detect the issue, you can tell them that their browser is unsupported (and list potential reasons). It's very difficult to accurately detect that 3rd party data is blocked from our library, which is why we haven't yet been able to release a proper fix.

@TMSCH got it, thanks for the quick reply!

@mjgallag you're welcome!

We are experiencing this issue as well. Using the idpiframe_initialization_failed  error is really helpful for us to be able to message to our users what is happening. But even if we handle the error (gapi.auth2.getAuthInstance().then(success, err => {/*handle error*/})), the gapi library still throws a javascript error, which is problematic for the following reasons:

What is thrown is an object, not an Error., which means there is no stacktrace associated with it. This is especially problematic for us since we log javascript errors to the server, but we don't have access to the stacktrace.
The object is thrown inside of a setTimeout, which means that a window.onerror handler doesn't even have access the file, line, or column of the code that threw this error. So we can't filter out all errors originating from google.com domains, because that is lost with the setTimeout.

What do you guys think of updating the library code so that it doesn't actually throw the error, but only provide the error through the getAuthInstance().then(...) error handling? Or if not that, then perhaps converting it to be a real Error object instead of a plain object?

Hi @joeldenning!
Thanks for the feedback. We actually continued to throw the error to be consistent with previous behavior, which is probably not useful anymore.
However I'd like to make sure I observe the same error when third-party data is blocked. When I reproduce it, an error is thrown from within the IFrame, not from the client window, and so I also cannot catch it (although it is an instance of Error).
Is it your case? Or is the error thrown from the main window? Also, what is the error log?
Thanks!

Would like to report we get same behaviour as @TMSCH. We cannot catch the error. Can't implement client-side OAuth2 when user doesn't allow 3rd party cookies/storage.

Hi @philovivero ! We recently pushed a changed to not throw the error from within the IFrame. Could you test again and let me know if that works? You should not observe any JS error, but be able to catch it in the promise. Thanks!
However, chrome with 3rd party cookies/storage disabled is still unsupported.

I'm not sure I will be able to repro. I ripped out all that code. For now we've decided to just let customers who uncheck that get failures, and when we look into it again, will likely implement OAuth2 server-side, since we assume at least some percentage of customers will be privacy-minded.

@TMSCH I just checked today and the error still is getting thrown when the "Block third party cookies and site data" checkbox is checked in Chrome. Here are some screenshots:
The code inside of gapi:

The error:

One thing to note is that I am using ravenjs (the js client for Sentry), which patches setTimeout with its own implementation (in order to handle errors inside of setTimeouts better). That's why you see the raven.js code in the stacktrace. For a while, I thought that maybe because I'm patching setTimeout that that may be causing the error to surface when it otherwise would not, but after some investigation I don't think that's the case. In the first screenshot, gapi is still calling setTimeout(() => {throw a}) which is something that causes an error regardless of if setTimeout is patched or not. So I don't think it's related to that, but rather just that gapi is still throwing the error.
Was the change you pushed out recently something that would prevent this from happening? It looks like the iframe is communicating the error up to the main document through a port1 onmessage handler, and then the main document is still throwing the error.

@joeldenning are you also catching the error in the initialization promise? If you don't, it will get thrown again (that's the standard behavior of a promise). I tried not catching it and observed the same behavior than you.
gapi.auth2.init({...}).catch(function(error) {
  // That's where the IFrame initialization error should be handled.
  // If you don't catch the rejection of the promise here, the error will be thrown.
});

@TMSCH Thanks, I got it work! Since gapi promises aren't real promises and don't follow the spec, the code sample you provided didn't work (It seems like you can't call .catch() on gapi promises). But if I catch it by providing two arguments to the .then(), it works.
gapi.auth2.init({...}).then(
  function success() {
  },
  function error(err) {
    // That's where the IFrame initialization error should be handled.
    // If you don't catch the rejection of the promise here, the error will be thrown.
  }
);

Thanks for your help with this and for pushing out the fix where it doesn't throw when the error is caught.

Oh yes, @joeldenning, you're right... Sorry about that. The return value is just a Thenable, there's no catch method. Glad you got it to work!

It would be helpful if this was covered in the docs as a known issue and potential caveat.
We recently ran into this on a project during final testing, then had to scramble a server-side flow instead.  (This seems like the only workaround, other than prompting users to change security settings in their browser?)
Would've been useful to know this upfront - perhaps the could be some kind of introduction that explains pros and cons of different flows?

@bkoski we will add this to the documentation, you're right, it should be mentioned. Server-side flow is indeed the most widely supported mechanism for now, as it doesn't rely on any storage or other characteristics of the browser. Keep in mind that such option in Chrome, block third party data, is not the default and a pretty advanced feature that most users won't have enabled.

@TMSCH Privacy-conscious users enable it, and privacy-conscious tech geeks who set up systems for their less-technically-abled family and friends may also enable it. I never specifically enabled it on my system. I suspect one of the various privacy plugins I installed toggled it for me. I suspect there are many ways it becomes enabled where a non-advanced user will be the one suffering the consequence.
@bkoski I also do not know any other work-around other than implementing it server-side. I actually think it should be strongly recommended to never do client-side auth until the situation with Chrome (and other browsers?) changes.

@philovivero that's true. However, in recent browsers, the "Incognito", "InPrivate" or "Private" modes tend to have supplanted such option. It used to be very prominent in the browser's settings (I remember specifically IE), when now, it is relegated to a subcategory of advanced settings (eg Chrome), if not completely removed (I can't find the option in my version of Firefox).To be noted: in these modes and across browsers, gapi.auth2 works.

A lot of our customers are behind firewalls and this option is preset and grayed out for them.
Interestingly, this old js works fine when 3rd party cookies are disabled:
<script type="text/javascript" src="https://apis.google.com/js/client.js"></script>
<script type="text/javascript" src="https://apis.google.com/js/platform.js"></script>
...
gapi.client.setApiKey(<apiKey>);
...
// On sign in button click:
gapi.auth.authorize({
  client_id: <clientId>,
  scope: window.ActivelyRead.account.scopes,
  immediate: false,
  authuser: "",
}, function (authResult) {
  if (authResult.error) {
    console.log('error', authResult);
  }
  else {
    gapi.client.load('oauth2', 'v2').then(
      function (response) {
        if (response && response.hasOwnProperty('error')) {
          console.log('error', response);
          return;
        }
        else {
          var request = gapi.client.oauth2.userinfo.get();
          request.execute(function (obj) {
            var user = {
              email: obj.email,
              firstName: obj.given_name,
              lastName: obj.family_name,
            };
          });
        }
      }
    );
  }
});

@admehta01 we are implementing a solution for such environment. I'll keep this thread updated.

@TMSCH this is great news!
I think it's important to remember that some users are opted into blocking third-party cookies without their direct action, based on corporate settings pushed to them or defaults set on corporate machines.  I wish we could just ignore this issue as well ("if you want it to work, just change your settings") but once it has been set as a corporate default it is difficult to make the case that it is a user issue.

@bkoski that's a good point :)

I am facing the same issue. :(

Yes, using a library that requires a client to disable privacy settings is a non-starter for us.  I suspect that's why sites like StackOverflow only use Google auth with a Redirect URL and not within a pop-up.  We will do the same.
This information, in my opinion, needs to be far more obvious in the documentation for web sign-in.  I wasted a few hours yesterday because it seemed impossible this library wouldn't work if users are following a recommended privacy practice by disabling third-party cookies.

Hi everyone,
Chrome with third-party data blocked disables cookies in IFrames. Cookies is a required feature to be able to securely authenticate the user. Without them, it is not possible to have a pure client side library that remains secure. This is why we won't support it. Single Sign-On JS libraries generally cannot be supported in such environment (Facebook login doesn't work neither).
If many of your users have this privacy feature enabled, you can implement a server side code flow.
We added documentation regarding this: https://developers.google.com/identity/sign-in/web/troubleshooting.
We apologize for the absence of support, but there is no work around without cookies in IFrame.

@TMSCH Why do you believe this is a technical limitation?  Disqus, for example, works with pop-up authentication and third party cookies blocked.  Facebook works as well, contrary to your post.  (E.g. go to Disqus.com and sign in with Facebook.)
I always have third party cookies disabled, and using this Google library is the first time SSO via pop-up has failed.  I subsequently visited feverup.com, and they must unfortunately be using this library, because SSO silently failed there.
This does not in any way appear to be a technical limitation, unless you believe Facebook and Disqus's implementations are not secure, in which case I'd like very much to hear your reasoning!

@TMSCH Another example that works is here at Meteor's contribution agreement.  "Sign in with GitHub" performs pop-up validation, even with third party cookies disabled.

The Auth V1 API (gapi.auth.authorize) used to work with third-party cookies disabled until recently (as also mentioned in https://github.com/google/google-api-javascript-client/issues/260#issuecomment-338063157).
But on March 3rd I noticed that it is no longer working. That seems like a regression.

For us this is a non-starter.  We're not using the library and won't if it doesn't work with a common, positive privacy practice like disabling third party cookies.
I understand Google has a vested interest in discouraging or killing this practice since Google relies on such cookies for ad revenue.  However, I'd certainly like to believe the best, that the devs here will do the right thing to support users and other developers.

@lukewlms @vladimir-i we reopened the bug as we're working on a potential (partial) solution.

Chrome with third-party data blocked disables cookies in IFrames. Cookies is a required feature to be able to securely authenticate the user. Without them, it is not possible to have a pure client side library that remains secure. This is why we won't support it. Single Sign-On JS libraries generally cannot be supported in such environment (Facebook login doesn't work neither).

So, for what it's worth, here is a quick implementation of a pure-client-side, pure-OAuth2 flow (that works with 3rd-party cookies disabled). If it serves to inspire this library's developers, wonderful; otherwise I can try to rigorize this and provide third-party support.
index.html:
<!DOCTYPE html>
<script src="https://apis.google.com/js/api.js"></script>
<script src="auth.js"></script>
<script>
const CLIENT_ID = '...';
const API_KEY = '...';
const SCOPES = ['https://www.googleapis.com/auth/calendar'];

googleAuthInit(CLIENT_ID, SCOPES).then(token => gapi.load('client:auth', () => gapiLoaded(token)));

function gapiLoaded(access_token) {
  gapi.auth.setToken({access_token});
  gapi.client.init({
      apiKey: API_KEY,
      discoveryDocs: ['https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest'],
  }).then(() => {
    gapi.client.calendar.events.list({
        'calendarId': 'primary',
        'timeMin': '2018-04-24T00:00:00Z',
        'timeMax': '2018-04-26T00:00:00Z'}).then(response => {for (let event of response.result.items) console.log(event)});
  });
}
</script>

auth.js:
function googleAuthInit(client_id, scopes) {
  if (document.location.hash) {
    let params = {};
    for (let piece of document.location.hash.substr(1).split('&')) {
      let [k, v] = piece.split('=', 2);
      params[decodeURIComponent(k)] = v ? decodeURIComponent(v.replace(/[+]/g, ' ')) : '';
    }
    if (params.access_token && params.expires_in) {
      localStorage.setItem('access_token_expires', Date.now() + Number(params.expires_in) * 1000);
      localStorage.setItem('access_token', params.access_token);
      let base = window.location.href;
      let i = base.indexOf('#');
      if (i > -1)
        base = base.substring(0, i);
      window.history.replaceState(null, document.title, base + (params.state ? atob(params.state) : ''));
    }
  }

  function getAuthUri() {
    let base = window.location.href, state = '';
    let i = base.indexOf('#');
    if (i > -1) {
      state = base.substring(i);
      base = base.substring(0, i);
    }
    return 'https://accounts.google.com/o/oauth2/v2/auth' +
        '?client_id=' + encodeURIComponent(client_id) +
        '&redirect_uri=' + encodeURIComponent(base) +
        '&state=' + encodeURIComponent(btoa(state)) +
        '&response_type=' + encodeURIComponent('token') +
        '&scope=' + encodeURIComponent(scopes.join(' ')) +
        '&include_granted_scopes=' + encodeURIComponent('true');
  }

  let access_token_expires_in = Number(localStorage.getItem('access_token_expires')) - Date.now() - 1000;
  if (access_token_expires_in < 0) {
    window.location.replace(getAuthUri());
    return Promise.reject();
  }

  window.setTimeout(() => window.location.replace(getAuthUri()), access_token_expires_in);
  return Promise.resolve(localStorage.getItem('access_token'));
}

Here's a drop-in version of auth.js that should work without changes to the [current] examples. It first tries upstream gapi.auth2.init, but catches idpiframe_initialization_failed and tries conventional OAuth2, then shoves the access token back in via gapi.auth.setToken and forces gapi.auth2.getAuthInstance().isSignedIn.get() to return true. I'd love for upstream gapi.auth2.init to do something similar (is the uncompiled code for at least the auth2 component available?), but if that's not possible I can keep working on this, just let me know!
(function() {

var patchedScriptOnload = false;
var patchedGapiLoad = false;
var patchedGapiAuth2Init = false;

tryPatchGapi();

function tryPatchGapi() {
  if (!window.gapi) {
    if (!patchedScriptOnload) {
      for (let script of document.getElementsByTagName('script')) {
        if (script.src.match(/^https:[/][/]apis[.]google[.]com[/]js[/]/)) {
          patchedScriptOnload = true;
          if (script.onload) {
            console.log('gapi-issue260: Installing onload patcher to', script);
            let scriptOnload = script.onload;
            script.onload = function(e) {
              tryPatchGapi();
              scriptOnload(e);
            };
          } else {
            console.log('gapi-issue260: Installing "load" event patcher to', script);
            script.addEventListener('load', tryPatchGapi);
          }
        }
      }
    }
    return;
  }

  if (!gapi.auth2) {
    if (!patchedGapiLoad) {
      patchedGapiLoad = true;
      console.log('gapi-issue260: Patching gapi.load.');
      let gapiLoad = gapi.load;
      gapi.load = function(targets, callback) {
        console.log('gapi-issue260: Watching gapi.load(', targets, ', ', callback, ').');
        if (callback instanceof Function) {
          gapiLoad(targets, () => {
            tryPatchGapi();
            callback();
          });
        } else {
          var newCallback = {
            callback: () => {
              tryPatchGapi();
              callback.callback();
            },
            onerror: callback.onerror,
          };
          gapiLoad(targets, newCallback);
        }
      };
    }
    return;
  }

  if (patchedGapiAuth2Init)
    return;
  patchedGapiAuth2Init = true;
  console.log('gapi-issue260: Patching gapi.auth2.init.');
  let gapiAuth2Init = gapi.auth2.init;
  gapi.auth2.init = function(params) {
    console.log('gapi-issue260: Watching gapi.auth2.init(', params, ').');
    return new Promise((resolve, reject) => {
      gapiAuth2Init(params).then(
          resolve,
          e => {
            if (e.error == 'idpiframe_initialization_failed') {
              console.log('gapi-issue260: Caught', e, '-- trying workaround.');
              workaround(params).then(access_token => {
                console.log('gapi-issue260: Success! Switching to gapi.auth and continuing.');
                gapi.auth.setToken({access_token});
                gapi.auth2.getAuthInstance().isSignedIn.get = () => true;
                resolve();
              });
            } else {
              reject(e);
            }
          });
    });
  };
}

function workaround(params) {
  if (document.location.hash) {
    let params = {};
    for (let piece of document.location.hash.substr(1).split('&')) {
      let [k, v] = piece.split('=', 2);
      params[decodeURIComponent(k)] = v ? decodeURIComponent(v.replace(/[+]/g, ' ')) : '';
    }
    if (params.access_token && params.expires_in) {
      localStorage.setItem('access_token_expires', Date.now() + Number(params.expires_in) * 1000);
      localStorage.setItem('access_token', params.access_token);
      let base = window.location.href;
      let i = base.indexOf('#');
      if (i > -1)
        base = base.substring(0, i);
      window.history.replaceState(null, document.title, base + (params.state ? atob(params.state) : ''));
    }
  }

  function getAuthUri() {
    let base = window.location.href;
    let state = '';
    let i = base.indexOf('#');
    if (i > -1) {
      state = base.substring(i);
      base = base.substring(0, i);
    }
    return 'https://accounts.google.com/o/oauth2/v2/auth' +
        '?client_id=' + encodeURIComponent(params.client_id) +
        '&redirect_uri=' + encodeURIComponent(base) +
        '&state=' + encodeURIComponent(btoa(state)) +
        '&response_type=token' +
        '&scope=' + encodeURIComponent(params.scope) +
        '&include_granted_scopes=true';
  }

  let access_token_expires_in = Number(localStorage.getItem('access_token_expires')) - Date.now() - 1000;
  if (access_token_expires_in < 0) {
    window.location.replace(getAuthUri());
    return Promise.reject();
  }

  window.setTimeout(() => window.location.replace(getAuthUri()), access_token_expires_in);
  return Promise.resolve(localStorage.getItem('access_token'));
}

})();

Can Anyone help me with this Issue?
I have to find out the issue that why website does not load the page properly after sign in with google api but when i run the project it works fine on my localhost

@UserDac Hi, can you be more specific regarding your issue? Can you describe what happens?

@nmlorg
Thank you for your script. It works for accessing the API like drive.
But I cannot get the user anymore:
gapi.auth2.getAuthInstance().currentUser.get() returns an non initialized user object.
Also gapi.auth.signIn() or signOut` is no longer working.

We have 587 users that experienced this problem, it would be nice to fix this in the library 🛠.

@TMSCH Any news on progress made in the last year?

Not a fix, but at least it won't fail silently.
I have found that a message is posted to the parent window that indicates that cookies are not enabled when the page is first loaded. Only tested in Chrome with Third Party cookies turned off.
method: "fireIdpEvent"
	params:
		error: "Cookies are not enabled in current environment."

This is basic code and assumes if here is an error, that the error is a cookie error which I'm sure is not always the case, and I'm no JS expert...  So modify as needed.
window.addEventListener('message', function(event) {
	if(typeof(event.data) != "undefined") {
		var msg_json = JSON.parse(event.data);
		if(typeof(msg_json.params) != "undefined") {
			if(typeof(msg_json.params.error) != "undefined"){
				// there is an error warn the user
				$('#cookie_warn').html('You must allow third-party cookies in your browser before you can log in');
			}
		}
	}
});

@TMSCH Any progress on this? Would be great to see a fix for this as part of the upcoming security audit.

plase can you answer me on the same issue https://stackoverflow.com/questions/58166870/google-api-sign-in-status-is-changed-after-refresh

Any news or progress related to this issue? Is there at least a confirmed workaround until the issue is not completely fixed? Thanks.

I pasted a workaround with some old js code earlier (not sure if it still works):
https://github.com/google/google-api-javascript-client/issues/260#issuecomment-338063157
My solution was to move everything to the backend:
https://developers.google.com/identity/sign-in/web/backend-auth
No issues with it so far.

See my comment here. May help in your case

This thread is a bit old so wanted to comment with what appears to be the current solution.
Google's documents this as a Known Issue.
Asking users to change their browser security settings is out of the question, so the only real solution is to abandon this JS library and implement server-side Auth 2.0 flow.
Suggestion:   As more browsers move toward disabling 3rd party cookies by default, this client side auth library is going to work for fewer and fewer users. This known issue about 3rd party cookies should be added as a big red warning at the beginning of the installation instructions.

Google Chrome blocks 3rd party cookies by default now

Any updates on this?
Apple seems to blocking third party cookies by default now too: https://webkit.org/blog/10218/full-third-party-cookie-blocking-and-more/. It seems like a big deal to find a workaround that isn't replacing all client-side auth.

It seems like the Firebase team has a workaround: https://github.com/firebase/firebase-js-sdk/issues/1040

Hi everyone! We might have found a very simple workaround: https://github.com/pwa-builder/PWABuilder/issues/3286#issuecomment-1219798658 (confirmed from multiple websites).


Password in plaintext

Is it possible to install a service with a useraccount without having to have the password in the code or the parameter in plaintext?
My constellation:
I build the service in azure and deploy it to a server and install it via an included powershell script.
So the pw stays either in the code or in the powershell which are both not acceptable.

I am closing all issues that haven't seen any activity. You're welcome to submit a PR to resolve the issue or provide more information.
And I don't know of another way, sorry.


Cerebro 0.6.1 snapshot listing timeout over 60 seconds

Hi,
I was using kopf under my 2.x elasticsearch cluster and was listing my snapshot from my s3 directory with no problem. I always have the last year worth of daily snapshot in S3.
Upgrading my cluster to 5.x. I installed Cerebro 0.6.1. Everything work perfectly apart from listing my snapshot. It give an error "Error connecting to the server".
curl -XGET '<IP_ADDRESS>:9200/_snapshot/dp-prod-backup/_all?pretty' i can see that it takes around 1 minutes 10 seconds with my setup.
In Cerebro I see that I get the error at 60 second everytime. Which make be believe that there is a timeout of 1 minutes.
Is there a way to change that?

Nevermind. They say that a night of sleep helps.... In this case a whole weekend. This morning I though to myself, why would there be a timeout of 60 seconds. Then I remembered that I am proxying through apache
I added a "timeout 90" to my config and now all work perfectly fine!
Sorry for the trouble.

haha, thank you for following up on this.
I haven't invested much time on it, but I had already discarded all points where a 60s timeout could be occurring, so I was a bit puzzled and wondering where to look next :)

Thank you for your time!
Have a great day!


Explaining the concept of "ecosystem"

Hello folks -- I mentioned this at today's meeting, and here I'd like to elaborate it maybe a bit more and see what you think:
The word "ecosystem" is ambiguous when used as things like Jupyter ecosystem. I still remember I checked up some dictionaries and webpages for what it exactly means but with no luck, and that was not even a long time ago. I feel that there might be many geoscience people who aren't used to this concept, and maybe it's good to spend a paragraph or two explaining the idea behind this ecosystem jargon on the introduction page?
Many scientists working in the geoscience domain may know what a toolbox (as of Matlab) or a plugin (as in GIS) is. Thus they might be confused with the dependency level when referring to different Jupyter-related tools/packages as part of the Jupyter ecosystem. Not to mention that there is a "Jupyter ecosystem" vs. a "Pangeo ecosystem," as shown in the current tutorial.
My current understanding of an ecosystem (please correct me if I am wrong) refers to a collection of independent packages. Each has a specific purpose, such as parsing data, providing UI, parallel computing, or visualizing results. Those packages need to work together in order to complete a workflow of data analysis. I'd be happy to provide anything to the current tutorial if you think this is useful.

As someone still fairly new to the "ecosystem" beyond notebooks I'd also happily be a guinea pig here!

Closing this issue as we've created some content for this. The follow-up discussion can be together with #33.


Fix paging-related error on aips page

I'm going to try a different approach.

Re-opening: the other approach seems complicated.


feat: Implement fast-deep-equal faster

Tweaks the algo to make it faster.
Below are results of 3x benchmark run:
fast-deep-equal-2 x 127,075 ops/sec ±2.32% (81 runs sampled)
fast-deep-equal x 106,086 ops/sec ±1.07% (87 runs sampled)
nano-equal x 88,697 ops/sec ±1.65% (76 runs sampled)
shallow-equal-fuzzy x 73,826 ops/sec ±1.11% (86 runs sampled)
underscore.isEqual x 53,620 ops/sec ±1.55% (82 runs sampled)
lodash.isEqual x 23,102 ops/sec ±1.24% (86 runs sampled)
deep-equal x 30,485 ops/sec ±1.97% (81 runs sampled)
deep-eql x 15,491 ops/sec ±1.22% (85 runs sampled)
assert.deepStrictEqual x 361 ops/sec ±2.42% (80 runs sampled)
The fastest is fast-deep-equal-2

fast-deep-equal-2 x 135,825 ops/sec ±1.88% (85 runs sampled)
fast-deep-equal x 126,627 ops/sec ±2.73% (85 runs sampled)
nano-equal x 109,734 ops/sec ±1.09% (90 runs sampled)
shallow-equal-fuzzy x 95,265 ops/sec ±0.70% (88 runs sampled)
underscore.isEqual x 67,421 ops/sec ±0.99% (90 runs sampled)
lodash.isEqual x 27,459 ops/sec ±4.50% (79 runs sampled)
deep-equal x 35,484 ops/sec ±1.22% (91 runs sampled)
deep-eql x 19,298 ops/sec ±1.10% (83 runs sampled)
assert.deepStrictEqual x 449 ops/sec ±2.35% (85 runs sampled)
The fastest is fast-deep-equal-2

fast-deep-equal-2 x 141,601 ops/sec ±1.03% (89 runs sampled)
fast-deep-equal x 137,028 ops/sec ±0.88% (90 runs sampled)
nano-equal x 111,036 ops/sec ±0.97% (86 runs sampled)
shallow-equal-fuzzy x 93,458 ops/sec ±1.54% (89 runs sampled)
underscore.isEqual x 68,208 ops/sec ±0.74% (93 runs sampled)
lodash.isEqual x 30,475 ops/sec ±0.86% (90 runs sampled)
deep-equal x 35,877 ops/sec ±0.85% (91 runs sampled)
deep-eql x 19,368 ops/sec ±0.73% (91 runs sampled)
assert.deepStrictEqual x 447 ops/sec ±1.52% (83 runs sampled)
The fastest is fast-deep-equal-2

Coverage remained the same at 100.0% when pulling c55c3d0b712771a67a57450dc4ff0b139063acf8 on streamich:feat/faster-2 into 2758cfb4e6c5d236ac80b546b2bf57a4ab2907b9 on epoberezkin:master.

@streamich cool, thank you. will you make this PR to update the actual code?

Merged it into the actual code.

I have removed in keyword, and ran the benchmark 3 times:
fast-deep-equal-new x 139,648 ops/sec ±1.32% (87 runs sampled)
fast-deep-equal-original x 133,418 ops/sec ±1.36% (90 runs sampled)

fast-deep-equal-new x 140,931 ops/sec ±0.72% (88 runs sampled)
fast-deep-equal-original x 132,433 ops/sec ±1.96% (88 runs sampled)

fast-deep-equal-new x 141,231 ops/sec ±0.63% (88 runs sampled)
fast-deep-equal-original x 135,583 ops/sec ±1.48% (87 runs sampled)

so still faster :)

Moved
if (dateA && dateB) return a.getTime() == b.getTime();

up

@epoberezkin I've mode the key length check before the recursive call, the new version still is faster with current benchmark:
fast-deep-equal x 134,145 ops/sec ±1.15% (89 runs sampled)
fast-deep-equal-original x 130,394 ops/sec ±0.90% (91 runs sampled)

fast-deep-equal x 143,424 ops/sec ±0.77% (91 runs sampled)
fast-deep-equal-original x 137,625 ops/sec ±1.40% (86 runs sampled)

fast-deep-equal x 146,275 ops/sec ±0.66% (91 runs sampled)
fast-deep-equal-original x 139,929 ops/sec ±0.79% (91 runs sampled)


Fix not available navigationState and onNavigate in Modal

ISSUE: https://github.com/aksonov/react-native-router-flux/issues/1379

@aksonov tests fails on __tests__/Example-test.js. It's ok or not?

It is not ok, looks like connected with new react native version... Feel free to fix it or maybe we should fix react native version within package.json...

@aksonov Looks like it is not so ease :) I will not do it inside this PR, but if I'll have a time, I'll create a new one.
I found, that some other opened PRs also fails on that test.

@inferusvv Should be fixed now, are you up to rebase on master?

@charpeni Ok, I'll do it

@charpeni Rebased

Thank you.

after checking out v3.37 on master, this PR seems to have been lost?

Does current master contain it?

no it does not

Must be something weird with my node modules. It says I have 3.37 but it doesnt give me the correct src files


JSON number parsing (v2)

Parse numbers from JSON tokens as both signed and unsigned integers of 32 or 64 bits, as well as a double float point precision.
Changes since v1 (PR #425):

changed sol_util_strtodn() to take a boolean use_locale. If so, the current locale is allowed (as in standard strtod()), otherwise strtod_l() is used with a "C" locale. In the case where strtod_l() is not available but there is locale support we translate the string exchanging '.' and lc->decimal_point, either in-place using the copy (tmpbuf) or creating yet-another copy if lc->decimal_point is more than one byte.
enable test-util.c to be compiled;
fix test_size_mul(), at least on my 32 bits machine, SIZE_MAX was odd and the test was failing since when multiplying by 2 it was not getting the same value;

merged with removed TODOS since @glima's patch with HAVE_LOCALE was merged.

I'll issue another PR tomorrow fixing users of strtod() in the code base.


WIP: Add CI front-end build

[x] Add a new CI job to build css
[x] Update Makefile's bootstrap rule to conditionally install front/back-end dependencies

Coverage remained the same at 100.0% when pulling da1ca751ef43f99fa2d8b432f08e90e9bdfb589e on add-ci-front-end-build into 27a61cb429b6371b36cb8a13215ade58dd316528 on master.

Coverage remained the same at 100.0% when pulling f681d42c04d7d947a4afdd965b761e43f5802c01 on add-ci-front-end-build into 27a61cb429b6371b36cb8a13215ade58dd316528 on master.

Coverage remained the same at 100.0% when pulling 6996e00708c10877feb04cc2f5601ef9d67e459a on add-ci-front-end-build into 722e652dccb2a19e06493e98e6970bcb594f2739 on master.

Coverage remained the same at 100.0% when pulling f500c0b0435042c4c94cee83f8f1c27549aad0d3 on add-ci-front-end-build into 722e652dccb2a19e06493e98e6970bcb594f2739 on master.


Blog and pieces

Hello,
Thanks for your work!
I am trying to build a blog on Apostrophe. I have some issues.
If I install the blog module with npm i apostrophe-blog, stop if I am bad, but I cannot input article content!
What's more, I followed the documentation on pieces: http://apostrophecms.org/docs/tutorials/getting-started/reusable-content-with-pieces.html . And the toolbar for the body is never displayed, neither in the modal dialog, nor in the show.html. It means the user cannot bold, italic and so on the body (case of people module)
Thanks

Hi Chris. Thanks for checking out Apostrophe! For a question like this, I'd recommend checking out existing questions on Stack Overflow. If you can't find an answer in the current list of questions, please post your new question there and tag it with apostrophe-cms. Someone on the team or a fellow Apostrophe expert should follow up on that platform soon.
Here's the link to existing Apostrophe-CMS questions to get you started: https://stackoverflow.com/questions/tagged/apostrophe-cms
Cheers.


add : @angular/pwa error " No matching version found for<EMAIL_ADDRESS>

🐞 Bug report
Command (mark with an x)

[ ] new
[ ] build
[ ] serve
[ ] test
[ ] e2e
[ ] generate
[x] add
[ ] update
[ ] lint
[ ] extract-i18n
[ ] run
[ ] config
[ ] help
[ ] version
[ ] doc

Is this a regression?
Yes, the previous version in which this bug was not present was 11.1
Description
When add @angular/pwa i have this error message

× Package installation failed: npm ERR! code ETARGET
npm ERR! notarget No matching version found for<EMAIL_ADDRESS>npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget
npm ERR! notarget It was specified as a dependency of '@angular/pwa'
npm ERR! notarget

npm ERR!      C:\Users\lmeunier\AppData\Roaming\npm-cache\_logs\2021-04-29T11_19_01_863Z-debug.log
Package install failed, see above.

The package version is published and available via npm<EMAIL_ADDRESS>Please ensure that npm is configured appropriately locally and that any intermediate npm caches and/or proxies are up-to-date.

Closing as per above.


popup css styling injection

Hi there,
The current popup seems not showing the right size. it overflow to the right side of my other cards.
Im curently using the Horizontal (layout-card), provided by HACS frontend layout-card mod.

What i want to approach is:
Full screen pop up with 10% margin of each side.
css shud be something like this..
div.popup {
display: block;
position: fixed;
height: 80%;
width: 80%;
top: 10%;
left: 10%;
z-index: 9999;
background: #000;
}
Question 1.
what is the best approach to inject the CSS to the card? i was trying to using card-mod but im unable to give the css accordingly.
I think it will be very great to have control or having few options, of how the popup behavior in the card settings.
cheers..

We moved on from the popup, and have it in the card itself. Because of the way HA uses popups (dialogs) there wasn't a good way to implement it without bugs. (Not without changing core at least)
Guessing your theme is allowing the card to "spread out".
I don't know if it's possible to inject CSS. The CSS changes dynamically when loading the card view.
You could fork the card and change the source.


Update dependencies: waitress, prometheus-client...

make task trivy-scan happy
Fixes https://github.com/danihodovic/celery-exporter/issues/333

The memory test seems to be failing

I've downgraded pytest-celery to 0.0.0 but that did not help


[Draft] run perf with debugging

Running perf with debug logging turned on.

/azp run python - servicebus - perf

API change check
API changes are not detected in this pull request.


remove addr:postcode ?

From : https://github.com/osmlab/nycbuildings/issues/139
@lxbarth, Is necessary  remove addr:postcode at clean up the special_co address ? we need to know to add in documentation

@Rub21 - the initial import includes the postcode, so yes.

I updated the wiki accordingly: https://github.com/osmlab/nycbuildings/wiki/Clean-up-special-A-B-addresses


Build issues observed when using CONFIG_BUILD_OUTPUT_ADJUST_LMA

Describe the bug
I discovered a heisenbug while adding this new feature to Zephyr: https://github.com/zephyrproject-rtos/zephyr/pull/57229. Some failures with objcopy occur when using CONFIG_BUILD_OUTPUT_ADJUST_LMA, e.g. when compiling some MCUboot tests.
To Reproduce
Use https://github.com/zephyrproject-rtos/zephyr/pull/57229 branch and build:
west build -p -b nrf52840dk_nrf52840 -T tests/boot/test_mcuboot/boot.mcuboot --sysbuild

If the following patch is applied to the branch, the problem disappears:
diff --git a/subsys/pm/policy.c b/subsys/pm/policy.c
index 3b4132c335..31eddf2d28 100644
--- a/subsys/pm/policy.c
+++ b/subsys/pm/policy.c
@@ -122,7 +122,7 @@ static void update_next_event(uint32_t cyc)
                new_next_event_cyc -= UINT32_MAX + 1U;
        }

-       next_event_cyc = new_next_event_cyc;
+       /* next_event_cyc = new_next_event_cyc; */
 }

 #ifdef CONFIG_PM_POLICY_DEFAULT

Expected behavior
Build succeeds
Impact
Blocker
Logs and console output
FAILED: zephyr/zephyr.elf zephyr/zephyr.map zephyr/zephyr.hex zephyr/zephyr.bin zephyr/zephyr.lst zephyr/zephyr.stat zephyr/zephyr.signed.bin zephyr/zephyr.signed.hex /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.map /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.hex /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.bin /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.lst /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.stat /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.signed.bin /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.signed.hex
: && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 zephyr/CMakeFiles/zephyr_final.dir/misc/empty_file.c.obj zephyr/CMakeFiles/zephyr_final.dir/dev_handles.c.obj zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj -o zephyr/zephyr.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -fuse-ld=bfd  -Wl,-T  zephyr/linker.cmd  -Wl,-Map=/home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr_final.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/aarch32/libarch__arm__core__aarch32.a  zephyr/arch/arch/arm/core/aarch32/cortex_m/libarch__arm__core__aarch32__cortex_m.a  zephyr/arch/arch/arm/core/aarch32/mpu/libarch__arm__core__aarch32__mpu.a  zephyr/lib/libc/minimal/liblib__libc__minimal.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/soc/arm/common/cortex_m/libsoc__arm__common__cortex_m.a  zephyr/soc/arm/nordic_nrf/nrf52/libsoc__arm__nordic_nrf__nrf52.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/segger/libmodules__segger.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L"/home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.1.0/thumb/v7e-m/nofp"  -L/home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr  -lgcc  -Wl,--print-memory-usage  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie && cd /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr && /usr/bin/cmake -E copy zephyr_final.map zephyr.map && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --change-section-lma *+483328 zephyr.elf zephyr.elf && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --gap-fill 0xff --output-target=ihex --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.hex && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy --gap-fill 0xff --output-target=binary --remove-section=.comment --remove-section=COMMON --remove-section=.eh_frame zephyr.elf zephyr.bin && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-objdump -d -S zephyr.elf > zephyr.lst && /home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-readelf -e zephyr.elf > zephyr.stat && /home/gmarull/ws/nordic/zephyrproject/.venv/bin/python3.11 -m west sign --quiet --tool imgtool --tool-path /home/gmarull/ws/nordic/zephyrproject/.venv/bin/imgtool --build-dir /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app --bin --sbin /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.signed.bin --hex --shex /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app/zephyr/zephyr.signed.hex -- --key /home/gmarull/ws/nordic/zephyrproject/bootloader/mcuboot/root-rsa-2048.pem
Memory region         Used Size  Region Size  %age Used
           FLASH:       20600 B       472 KB      4.26%
             RAM:        5568 B       256 KB      2.12%
        IDT_LIST:          0 GB         2 KB      0.00%
/home/gmarull/zephyr-sdk-0.15.1/arm-zephyr-eabi/bin/arm-zephyr-eabi-objcopy: zephyr.elf: error in private header data: sorry, cannot handle this file
ninja: build stopped: subcommand failed.
[3/9] Completed 'test_mcuboot'
FAILED: swapped_app-prefix/src/swapped_app-stamp/swapped_app-build /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app-prefix/src/swapped_app-stamp/swapped_app-build
cd /home/gmarull/ws/nordic/zephyrproject/zephyr/build/swapped_app && /usr/bin/cmake --build .
ninja: build stopped: subcommand failed.

Environment (please complete the following information):

OS: Linux
Toolchain: Zephyr SDK 0.15.1
Commit SHA or Version used: https://github.com/zephyrproject-rtos/zephyr/pull/57229

Additional context
N/A

The problem is that the BFD program header rewrite implementation makes a questionable assumption that the LMA of a segment will be aligned to the alignment of the first section in the segment:
		      /* If the first section in a segment does not start at
			 the beginning of the segment, then something is
			 wrong.  */
		      if (align_power (map->p_paddr
				       + (map->includes_filehdr
					  ? iehdr->e_ehsize : 0)
				       + (map->includes_phdrs
					  ? iehdr->e_phnum * iehdr->e_phentsize
					  : 0),
				       output_section->alignment_power * opb)
			  != output_section->lma * opb)
			goto sorry;

This is not a problem when the LMA is equal to the VMA and both are aligned at the section alignment; but, for the sections that have different VMA and LMA, only the VMA is guaranteed to be aligned at the section alignment and the LMA may or may not be aligned at the same boundary.
Now looking at the cases with next_event_cyc = new_next_event_cyc; (bad) and without (good).
Good:
Program Header:
[...]
    LOAD off    0x00005044 vaddr 0x20000000 paddr 0x00010f64 align 2**2
         filesz 0x000000b8 memsz 0x000000b8 flags rw-
[...]
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
[...]
 10 datas         000000b0  20000000  00010f64  00005044  2**2
                  CONTENTS, ALLOC, LOAD, DATA
[...]

Bad:
Program Header:
[...]
    LOAD off    0x00005098 vaddr 0x20000000 paddr 0x00010fb4 align 2**3
         filesz 0x000000c0 memsz 0x000000c0 flags rw-
[...]
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
[...]
10 datas         000000b8  20000000  00010fb4  00005098  2**3
                  CONTENTS, ALLOC, LOAD, DATA
[...]

(the full dump can be found in https://gist.github.com/stephanosio/5946bd6f8f249b5329fb4982f87cd58e)
Note that, for both good and bad, the datas section is the first section in a segment.
For good, the LMA is 0x10f64 which is aligned at 22.
For bad, the LMA is 0x10fb4 which is NOT aligned at 23.
As seen above, the BFD implementation expects the LMA of a segment to be aligned at the alignment of the first section of the segment, and attempts to align the address of the segment LMA to the alignment of the first section before comparing it to the actual LMA of the first section for the purpose of verifying that the first section starts at the beginning of the segment.
When running objcopy --change-section-lma *+483328 ..., which offsets the LMA by 0x76000:
In case of good, since the segment LMA is divisible by 4 (22), this results in 0x86f64 == 0x86f64, which evaluates to true.
In case of bad, since the segment LMA is NOT divisible by 8 (23), this results in 0x86fb8 == 0x86fb4, which evaluates to false.

The dev branch in PR #60400 hits the same bug.

Why do bugs get the stale label? I guess many problems get forgotten and lost just because of this bot action. @carlescufi @stephanosio

Why do bugs get the stale label? I guess many problems get forgotten and lost just because of this bot action. @carlescufi @stephanosio

To remind the stakeholders the existence of the bug and keep the bug count under control by closing any bugs that have no stakeholders -- the idea is that, if there are any stakeholders, they would remove the "stale" label, as seen above.

@stephanosio any plans to get this upstreamed into gcc?

@stephanosio any plans to get this upstreamed into gcc?

I will release an SDK RC with the proposed patch above next week.

Closing this since Zephyr SDK 0.16.5 with the binutils fix for this issue has been mainlined.

@stephanosio @danieldegrasse I believe that https://github.com/zephyrproject-rtos/zephyr/pull/60934 should be reverted ASAP or at least together with a root cause solution to this issue as proposed by @stephanosio . https://github.com/zephyrproject-rtos/zephyr/pull/60934 is a workaround plus it doesn't even fix the alignment issue if the MPU aligment is smaller than the concerned section's VMA alignment. I see failing tests with https://github.com/zephyrproject-rtos/zephyr/pull/60934 applied, with the required section alignment being 23 while the MPU alignment is 22.

Do you agree?
@fgrandel would you like to open an issue to get #60934 reverted? If you can point to a specific bug the commit causes, we might be able to make the change prior to the release

@fgrandel would you like to open an issue to get #60934 reverted? If you can point to a specific bug the commit causes, we might be able to make the change prior to the release

@danieldegrasse Sorry, saw your comment only now, revisiting this issue. I seems to me that the additional Kconfig switch is redundant in the current mainline now that the underlying root cause was fixed. It might cause bugs in the future when the condition described in my original message applies. I might be mistaken, though, as I might misunderstand the intent of the CONFIG_BUILD_ALIGN_LMA switch in the first place.

@fgrandel would you like to open an issue to get #60934 reverted? If you can point to a specific bug the commit causes, we might be able to make the change prior to the release

@danieldegrasse Sorry, saw your comment only now, revisiting this issue. I seems to me that the additional Kconfig switch is redundant in the current mainline now that the underlying root cause was fixed. It might cause bugs in the future when the condition described in my original message applies. I might be mistaken, though, as I might misunderstand the intent of the CONFIG_BUILD_ALIGN_LMA switch in the first place.

No worries, you're understanding the purpose of CONFIG_BUILD_ALIGN_LMA exactly. https://github.com/zephyrproject-rtos/zephyr/pull/72376 should revert the commit adding it, so this workaround will no longer be in tree