fix: use unknown instead of any for additional properties

Status
READY
Description
Fixes https://github.com/anymaniax/orval/issues/1464
The generated type for objects with additionalProperties: true becomes [key: string]: unknown instead of [key: string]: any for better type safety.
Todos

[ ] Tests
[ ] Documentation
[ ] Changelog Entry (unreleased)

Steps to Test or Reproduce
Outline the steps to test or reproduce the PR here.
> git pull --prune
> git checkout <branch>
> grunt jasmine

Shall we change the any on line 209 as well?
https://github.com/anymaniax/orval/blob/82024a4808e57cecf85da2dd2d2befbc033c5da7/packages/core/src/getters/object.ts#L209

Yes definitely, I only concentrated on additionalProperties but it makes sense to fix the issue more broadly.

The original fix was here in 2021: https://github.com/anymaniax/orval/pull/190  But I agree unknown is safer than any

The Typescript checking of my source is invalid once again. If you want to change from any to unknown, please create a new property in the config file to allow other users to decide whether to apply your change.
Thank you!
cc @jmorel @ezequiel

The Typescript checking of my source is invalid once again. If you want to change from any to unknown, please create a new property in the config file to allow other users to decide whether to apply your change. Thank you! cc @jmorel @ezequiel

unknown should've arguably always been the default.
I'm personally not a big fan of making this configurable, simply because any is too dangerous as it will mask potential type errors in your code.
To quickly move forward and upgrade Orval, I suggest adding // @ts-expect-error above the offending lines of code, and resolve them later when time permits.
e.g
// @ts-expect-error todo: fix this
const thing = data.response.thing;

Hi, I've just noticed this change when I tried the latest version. I agree that unknown is much safer than any, however this breaking change is a big pain for my huge codebase. It would be really nice if I can incrementally adopt this change.
In addition to that, is there any way to configure each object type with typescript interface? because I have a lot of 3rd-party dependencies that provide types with interface in the backend, but they all converted into object due to typescript limitation, and then the properties become unknown in the frontend. They are actually not 'unknown' because I can provide types for them. Since editing the auto-generated files is not a good idea, an option to configure them would be ideal.
This is just a small feedback. Thanks for the amazing work

Same here, the change broke any location where we explicitly typed a generic object from the back end.
Instead of using:
const data.someObject as SomeType;

We now need to use:
const data.someObject as unknown as SomeType;

Yes this issue has gone back and forth between any and unknown with both sides arguing why one is better than another.


Entities in combat freeze on 'action' when casting AOE and Quickenings

I am looking into the cause of this, and will fix it as soon as I can.
This issue only seems to happen at 1x speed.

This has been resolved, and will no longer be an issue in the next update
https://gyazo.com/a947ef680f54872f9f902e6b5020382d


Python version

I find that your code is really good. It is exactly what i'm searching for. But the problem is, i'm not really good at translating it into python. I really want to find the python version of gabor filter feature extraction. Is there any suggestion from you? thanks

Hi,
I am not an expert in Python either. Apparently, there is a MATLAB Engine
API for Python that might be useful for you to call the MATLAB function
inside your Python script. Please see the following link:
http://www.mathworks.com/help/matlab/matlab-engine-for-python.html
I hope it helps.
Mohammad
On Sun, Mar 27, 2016 at 11:39 AM, aryopg<EMAIL_ADDRESS>wrote:

I find that your code is really good. It is exactly what i'm searching
for. But the problem is, i'm not really good at translating it into python.
I really want to find the python version of gabor filter feature
extraction. Is there any suggestion from you? thanks
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1

I think i can make the python code already, now i would like to ask what kind of algorithm that i should use if i want to match the gabor feature vector of one image to another?

You can use any classifier such as minimum distance, KNN, SVM, SRC, etc.
On Mon, Mar 28, 2016 at 4:08 AM, aryopg<EMAIL_ADDRESS>wrote:

I think i can make the python code already, now i would like to ask what
kind of algorithm that i should use if i want to match the gabor feature
vector of one image to another?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202294086

Can i use it in a CBIR machine? And is there any example on how to
implement it using feature vectors?
On Mon, Mar 28, 2016 at 9:57 PM, Mohammad Haghighat <
<EMAIL_ADDRESS>wrote:

You can use any classifier such as minimum distance, KNN, SVM, SRC, etc.
On Mon, Mar 28, 2016 at 4:08 AM, aryopg<EMAIL_ADDRESS>wrote:

I think i can make the python code already, now i would like to ask what
kind of algorithm that i should use if i want to match the gabor feature
vector of one image to another?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202294086

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202427215

And one more, when i tried to run my code that was translated from your
code. I got a result of 5*8 orientation, but with not so much differences
between each image. What do you think was wrong in my code? i've translated
it until the feature vector extraction, and got that result
On Tue, Mar 29, 2016 at 5:09 PM, Aryo Pradipta Gema<EMAIL_ADDRESS>wrote:

Can i use it in a CBIR machine? And is there any example on how to
implement it using feature vectors?
On Mon, Mar 28, 2016 at 9:57 PM, Mohammad Haghighat <
<EMAIL_ADDRESS>wrote:

You can use any classifier such as minimum distance, KNN, SVM, SRC, etc.
On Mon, Mar 28, 2016 at 4:08 AM, aryopg<EMAIL_ADDRESS>wrote:

I think i can make the python code already, now i would like to ask what
kind of algorithm that i should use if i want to match the gabor feature
vector of one image to another?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202294086

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202427215

I have no idea!
I am sorry.
On Tue, Mar 29, 2016 at 6:53 AM, aryopg<EMAIL_ADDRESS>wrote:

And one more, when i tried to run my code that was translated from your
code. I got a result of 5*8 orientation, but with not so much differences
between each image. What do you think was wrong in my code? i've translated
it until the feature vector extraction, and got that result
On Tue, Mar 29, 2016 at 5:09 PM, Aryo Pradipta Gema<EMAIL_ADDRESS>wrote:

Can i use it in a CBIR machine? And is there any example on how to
implement it using feature vectors?
On Mon, Mar 28, 2016 at 9:57 PM, Mohammad Haghighat <
<EMAIL_ADDRESS>wrote:

You can use any classifier such as minimum distance, KNN, SVM, SRC, etc.
On Mon, Mar 28, 2016 at 4:08 AM, aryopg<EMAIL_ADDRESS>wrote:

I think i can make the python code already, now i would like to ask
what
kind of algorithm that i should use if i want to match the gabor
feature
vector of one image to another?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202294086

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202427215

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202829793

I believe you can, because they have used it before.
See the two papers attached.
On Tue, Mar 29, 2016 at 6:09 AM, aryopg<EMAIL_ADDRESS>wrote:

Can i use it in a CBIR machine? And is there any example on how to
implement it using feature vectors?
On Mon, Mar 28, 2016 at 9:57 PM, Mohammad Haghighat <
<EMAIL_ADDRESS>wrote:

You can use any classifier such as minimum distance, KNN, SVM, SRC, etc.
On Mon, Mar 28, 2016 at 4:08 AM, aryopg<EMAIL_ADDRESS>wrote:

I think i can make the python code already, now i would like to ask
what
kind of algorithm that i should use if i want to match the gabor
feature
vector of one image to another?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202294086

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202427215

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/mhaghighat/gabor/issues/1#issuecomment-202819335


Update iterator.rs

As you've asked.
It was fetched from delta PR.
I've created it though Github so I'd run tests before/after just in case.
So do you plan to maintain this lib and replace the one in a delta source code with it?

Thanks a lot!

So do you plan to maintain this lib and replace the one in a delta source code with it?

Yes I'm considering that. Do you have any interest in being a co-maintainer? The other option would be to copy the AnsiIterator code into tabled.


[PE-D][Tester C] Edit flaw

Able to have 2 contacts of the same details but cannot add a third with the same

Labels: type.FeatureFlaw severity.Medium
original: Harithhh06/ped#5

Invalid because he doesn't actually have 2 clients with the same details, the second one he's referring to has a different email, which nds in 0.com instead of o.com.


Fix NameError: name 'pilutil' is not defined

See http://stackoverflow.com/questions/9298665

This fix was actually incomplete, apologies for merging that too fast:
======================================================================
ERROR: Failure: AttributeError ('module' object has no attribute 'pilutil')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/nose-1.1.2-py2.6.egg/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/Users/rgommers/Code/scipy/scipy/misc/tests/test_pilutil.py", line 13, in <module>
    import scipy.misc.pilutil as pilutil
AttributeError: 'module' object has no attribute 'pilutil'

----------------------------------------------------------------------

Fixed in commit 12bde30.

Could someone double-check that? Then I'll squash those two and put them in 0.10.x.

I don't quite understand how importing works in this case. It is probably better not to del pilutil and del common in __init__.py because that removes the ability to import pilutil and common from scipy.misc.

For 0.10.x I agree, but for master I prefer to use del, because those are private modules.

The import common line is probably not necessary anyway. Shall we revert that?

For 0.10.x the only change should then be to add import pilutil in the try statement.

I don't understand how from common import * allows to use the common name in this case. I would expect a NameError.

It does actually import the name common. I suspect that's a bug in the import mechanism, because the from X import * mechanism is not supposed to do that.

I'm still trying to find a clear reference on this.

There seems to be a difference between
from X import Y

after which Y is available but X.Y is not, and
from Y import Z

in X.__init__.py. In this case X.Y is available.
I can't find this documented anywhere though, so it may not be very future-proof. Therefore the fix as merged still seems like the best idea to me.

That is also my conclusion. In the case of the original bug report (http://stackoverflow.com/questions/9298665), Y (pilutil) is not available in __init__.py. Maybe it's undefined or implementation dependent?

Sounds fine. Btw, IronPython and PyPy have the same import behavior. I also couldn't find it documented.

We're probably relying on it in a few more places, because the from submodule import * pattern is used almost everywhere. But well, that's for another time.


GM: Activate the FCW lights if a critical notification appears (Immediate Disengage)

Should add that it also triggers when a collision alert happens (same type of chime)


reject ruleset for many Specified path on all domain

this is more like captcha ruleset.
i need reject   ads  ad _stats   and more so i put them on top of persistent and temporary ruleset
and how add more path to them

@heforfree Check this example. Does it answer your question?

@futpib
_stats working example on github i see it blocked xhrs request
but /ad not working, just in case i have to rename to ads and adv to test
in below site i want filter  ads,adv path but not working(ads are in the left of site)
copy link location http://asriran.com/fa/ads/redirect/a/2001
copy image location http://cdn.asriran.com/files/adv/2691_490.gif
i allow image from asriran.com to asriran.com and put myrulset on top but still i can see image
http://www.asriran.com/
and seems such as rule slow down browser

@futpib
so why the second one not working?

thank you i see you updated and now working

@futpib
i updated reject_special_paths.ruleset for myself with some mor path.if you want update it on your
example
https://gist.github.com/heforfree/e9c4ecae8e3bf88217c0


DisplayIO Example Wanted

Add a Basic DisplayIO Based Example
We would like to have a basic displayio example for this library. The example should be written for microcontrollers with a built-in display. At a minimum it should show a Label on the display and update it with live readings from the sensor.
The example should not be overly complex, it's intended to be a good starting point for displayio based projects that utilize this sensor library. Try to keep all visual content as near to the top left corner as possible in order to best fascilitate devices with small built-in display resolutions.
The new example should follow the naming convention examples/libraryname_displayio_simpletest.py with "libraryname" being replaced by the actual name of this library.
You can find an example of a Pull Request that adds this kind of example here: https://github.com/adafruit/Adafruit_CircuitPython_BME680/pull/72
We have a guide that covers the process of contributing with git and github: https://learn.adafruit.com/contribute-to-circuitpython-with-git-and-github
If you're interested in contributing but need additional help, or just want to say hi, feel free to join the Discord server to ask questions: https://adafru.it/discord

Add a Basic DisplayIO Based Example

Issue satisfied with DisplayIO example #19 now merged


Color is using named colors from sass variables instead of hex or rgb.

For example: $tiffany-blue: #0fc4b1;
Gutter Color is rendering a basic blue color when in fact the hex is a shade of green.
This is happening on all my sass variables.

+1 Please specific name color detection only color name such as
http://www.w3schools.com/cssref/css_colornames.asp
or
http://xkcd.com/color/rgb/ (JSON file can download from https://github.com/zeplin/zeplin-palette )

I'll have a fix ready soon.

Done.

https://github.com/ggordan/GutterColor/releases/tag/v0.3.3


Updated constructor params

Resolves #1173
Added ability to pass in host parameter. This is needed for new Logentries accounts which are now on a different domain (rapid7.com).
Note: You also need to pass in $useSSL as false because it seems port 443 is not an option on the new domain, but I left it in for backwards compatibility.

Thanks, merged in 1.x and master.

Note that the correct way to do this is to migrate to the InsightOpsHandler


Hi

Hi how are you?

We are you ftom?

Hi, @alexsandre1985 we are good, thanks for asking! This repository is dedicated to the CLI fal, do you have any questions or suggestions about it?


fix: articles/active-directory/develop/active-directory-signing-key-rollover.md

Escape elements

@nschonni : Thanks for your contribution! The author, @CelesteDG, has been notified to review your proposed change.

Verified editorial correction. Thanks


Code generation should properly case service versions

the code generates the service version as "2022_05_15_preview" (p is small) which got me an error because it should be capital, that's why I modified it.
Originally posted by @ZulaMostafa in https://github.com/Azure/azure-sdk-for-net/pull/28639#discussion_r879816056

Have to use a transform to work around the issue.
/cc @annelo-msft @ShivangiReja

https://github.com/Azure/azure-sdk-for-net/issues?q=is%3Aissue+is%3Aopen+Preview+ServiceVersions+should+use+PascalCase has also been filed for each service.
This may not block GA as GA libraries may not include "preview" versions in their ServiceVersion enum, but it would be good to fix it to be safe.

I appreciate support for GA libraries is top priority, but would still ask we prioritize this because we ship more previews than GAs. I'd rather not have to work around this every time I regenerate against a preview swagger.

We will prioritize this, but it is not required for GA.

Duplicate of #1524


Sync changes in internal moe to public repo

Sync changes in internal moe to public repo. The main functional changes are as follows:

Extend Distributed Grid to enable expert relocation
Expert weight calculation optimization. Avoid calculation on for padding used for unused expert capacity
Simplify interface. Use options dict for MixtureOfExperts and Layers
Refactor use of tutel cumsum op. Use om_ (short for Ort Moe) prefix for custom ops.
Enable DeepSpeed ZeRO optimizations- Set relevant attributes on expert parameters, as per DeepSpeed engine needs.
Add utility to compress all2all tensors using dietgpu library

Improve test coverage

LGTM. Let's see if CI passes


added service pigpiod + improvements related to node-red service

closed it for the moment as it was not my intention to do a pull request in the master repository.

I reopened the pull request.


[SR-4105] Implement is-pattern / is-cast in lib/Syntax

Previous ID
SR-4105

Radar
None

Original Reporter
@bitjammer

Type
Bug

Additional Detail from JIRA

Votes
0

Component/s
Compiler

Labels
Bug, Syntax

Assignee
@bitjammer

Priority
Medium

md5: 1b52417227e9c816b4699d19f7d1f41e

libSyntax was removed in #62145 and succeeded by swift-syntax.


test integration broken on master

Not sure where this is coming from:
---> Making bundle: build-integration-test-binary (in bundles/1.14.0-dev/build-integration-test-binary)
Building test dir: integration-cli
++ cd integration-cli
++ go test -c -o /go/src/github.com/docker/docker/bundles/1.14.0-dev/build-integration-test-binary/../test-integration-cli/test.main -ldflags -w -tags 'autogen netgo static_build sqlite_omit_load_extension apparmor pkcs11 seccomp selinux daemon journald libdm_no_deferred_remove' -installsuffix netgo -i
# testmain
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-234611570/000003.o: unrecognized relocation (0x2a) in section `.text'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

Makefile:140: recipe for target 'test-integration-cli' failed
make: *** [test-integration-cli] Error 1

I'm on Fedora 25, kernel 4.8.14
Also, I had to completely disable OSXCROSS in the Dockerfile, otherwise my system just freeze and I have to reboot

/cc @vdemeester

Hum, I'll have to spawn a fedora 25 to try this out, 'cause I can't reproduce locally.

Could not hit this issue on Fedora 25 locally..
Can you provide more detail to reproduce? @runcom
$ docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 91
Server Version: 1.12.5
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: bridge host null overlay
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Security Options: seccomp selinux
Kernel Version: 4.8.15-300.fc25.x86_64
Operating System: Fedora 25 (Workstation Edition)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 1
Total Memory: 1.954 GiB
Name: localhost.localdomain
ID: ACBV:RCWZ:BVD4:XJ63:LRLC:CLFY:SW2G:NTRX:LFWK:IB5F:62KN:S4VN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 <IP_ADDRESS>/8
Registries: docker.io (secure)

$ docker version
Client:
 Version:         1.12.5
 API version:     1.24
 Package version: docker-common-1.12.5-1.git6009905.fc25.x86_64
 Go version:      go1.7.4
 Git commit:      6009905/1.12.5
 Built:           Fri Dec 16 09:26:10 2016
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.5
 API version:     1.24
 Package version: docker-common-1.12.5-1.git6009905.fc25.x86_64
 Go version:      go1.7.4
 Git commit:      6009905/1.12.5
 Built:           Fri Dec 16 09:26:10 2016
 OS/Arch:         linux/amd64

$ git clone https://github.com/docker/docker.git
$ cd docker
$ git show --summary
commit fb03b56b9b81182ef8561c57e2e7626658dc943b
Merge: e636674 f394b1d
Author: Akihiro Suda<EMAIL_ADDRESS>Date:   Mon Dec 26 10:45:00 2016 +0900

    Merge pull request #29693 from davidxia/fix-typo

    Fix typo in docs

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
$ make test-integration-cli
---> Making bundle: test-integration-cli (in bundles/1.14.0-dev/test-integration-cli)
---> Making bundle: .integration-daemon-start (in bundles/1.14.0-dev/test-integration-cli)
INFO: Waiting for daemon to start...
+++ exec dockerd --debug --host unix:///go/src/github.com/docker/docker/bundles/1.14.0-dev/test-integration-cli/docker.sock --stora
ge-driver overlay2 --pidfile bundles/1.14.0-dev/test-integration-cli/docker.pid --userland-proxy=true
.
---> Making bundle: .integration-daemon-setup (in bundles/1.14.0-dev/test-integration-cli)
---> Making bundle: .detect-daemon-osarch (in bundles/1.14.0-dev/test-integration-cli)
---> Making bundle: .ensure-emptyfs (in bundles/1.14.0-dev/test-integration-cli)
++++ docker load
++++ tar -cC bundles/1.14.0-dev/test-integration-cli/emptyfs .
++ env -i DEST=/go/src/github.com/docker/docker/bundles/1.14.0-dev/test-integration-cli DOCKER_TLS_VERIFY= DOCKER_CERT_PATH= DOCKER
_ENGINE_GOARCH=amd64 DOCKER_GRAPHDRIVER=overlay2 DOCKER_USERLANDPROXY=true DOCKER_HOST=unix:///go/src/github.com/docker/docker/bund
les/1.14.0-dev/test-integration-cli/docker.sock DOCKER_REMAP_ROOT= DOCKER_REMOTE_DAEMON= DOCKERFILE=Dockerfile GOPATH=/go GOTRACEBA
CK=all HOME=/go/src/github.com/docker/docker/bundles/1.14.0-dev/test-integration-cli/fake-HOME PATH=/go/src/github.com/docker/docke
r/bundles/1.14.0-dev/test-integration-cli/../binary-client:/go/src/github.com/docker/docker/bundles/1.14.0-dev/test-integration-cli
/../binary-daemon:/go/src/github.com/docker/docker/bundles/1.14.0-dev/test-integration-cli/../dynbinary-client:/go/src/github.com/d
ocker/docker/bundles/1.14.0-dev/test-integration-cli/../dynbinary-daemon:/go/bin:/usr/local/go/bin:/osxcross/target/bin:/usr/local/
sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin TEMP= TEST_IMAGE_NAMESPACE= /go/src/github.com/docker/docker/bundles/1.14.0-dev/t
est-integration-cli/test.main -check.v -check.timeout=5m -test.timeout=360m
INFO: Testing against a local daemon
PASS: docker_api_test.go:36: DockerSuite.TestAPIClientVersionOldNotSupported    0.011s
PASS: docker_api_network_test.go:238: DockerSuite.TestAPICreateDeletePredefinedNetworks 0.012s
...

You have to test with master, the Docker you have is 1.12.5

@runcom oh master against master :sweat_smile: You didn't say that :angel:

there's 1.14.0-dev in my log :smile: (added version/master tag also), sorry for the confusion

@runcom true, but in bundles/1.14.0-dev/build-integration-test-binary means you're building master (1.14.0-dev) but doesn't tell anything about the underlying docker version 😛

seems like I can run make test-integration-cli again - not sure what fixed it, closing this. I'll open if issues arise again.


FIX: Fixes dot() returning NaN.

Here's a PR to fix angle() sometimes returning NaN with specific long-running vectors; this is probably due to floating-point inaccuracies.
As an example, without this PR, this:
v := vector.Vector{-0.9040721420170615, 0, 0.4273798802338293}
fmt.Println(v.Angle(v))

returns NaN instead of 0 (and similarly NaN instead of pi if the vector were to be tested against its opposite).

Whoops, sorry about that - OK, I force updated my master branch so the current PR now includes a simple dot NaN test.

Excellent - and sorry for the confusing title / description, haha. This issue affects both dot() and angle().

Yeah, i figured that out after seeing the code. Once again thanks for the contribution! :)


question on 'iOS not supported v2"

"Versions 2 and above do not support iOS"
"Version 1 has support for iOS but due to file size problems versions 2 and above currently do not support iOS."
Can you explain a bit about the file size issue? Is it the final release binary? How large is it?
thanks!
Sorry for opening an issue, but didn't know a better way to ask :)

Hi
The necessary binaries are around 100MB which is too big to upload to a free github account.
Please take a look at
https://github.com/mycrewinc/eggnstone_amazon_chime/commits/develop
where maropook et. al. fixed my package.
I am in the process of pulling the changes in:
https://github.com/eggnstone/eggnstone_amazon_chime/commits/Refactor-2022

that's interesting - thanks for the update, I'll have a look :)


⚠ Bofrost.lu has degraded performance

In 82c6d17, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 2308 ms

Resolved: Bofrost.lu performance has improved in eb5989c after 13 minutes.


SMS Backup+ restore some, dups the rest.

Hi everyone,
I'm brand new here so please forgive my [technical] ignorance or carelessness. I've used SMS Backup+ successfully in the past to transfer text from my HTC Evo 4G over to my HTC One M8 HK on Sprint network. My M8 was softbricked recently one day out of the blue while charging. I had to reset it by updating a different rom and flash the update via my laptop. Prior to that, I had no idea how any of that works. Since getting it back up, it has been working fine, but my SMS Backup+ is not able to restore my text messages properly. I have all my text messages backed up to Gmail and I can see "SMS" folder in my Gmail account and all my text, past and current are there, but the SMS app will not restore all of them. I have done this 3 times and it takes a long time to backup each time (about 6 hours or more) since I have over 46K+ messages. After the third time, I was more careful to see what happens and at the end right after it read through all 46K+ messages, it says something like (39,238 dups). I looked up what that could mean and it says that the app assumed that I have that many text on my phone as duplicates already so it didn't restore those, while only the older text messages got restored on my phone. But I do not have those messages on my phone. At the same time, my Google sync is having issue with syncing my [Google] Task app as well, so I am not sure if that is related to what is going on my the SMS Backup. I could really use some guidance on how to fix this.
Here are the software I'm running on:
Andrioid version: 5.0.2
HTC Sense version: 6.0
Software version 4.25.654.18
HTC SDK API Level: 6.55
More: Baseband, Kernel version, etc.
PRI Version: 2.62_003
PRL Version 55030
Thank you.

I am having the same problem.  I am trying to restore my SMS on my new Nexus 6P.  After running several Restores, each time receiving a small batch of messages, the app has stalled.  No messages after June 2014 are being restored.  When I run a Restore now, the app restores 0 messages and counts everything as duplicates.  I did try installing the beta released on Oct 7 per another thread.  Please advise.

I have 52480 items and I see restored up to ~45k restoration. I am trying for the third time. I just installed Cyanogen Mod 13 (Android Marshmallow) on OnePlus One. In both previous runs I have restored to only 4 Month back (Jan 7) SMS. Call logs are not restored at all.

I have the same problem. I backupped about 5900 messages. I even went through them in Thunderbird and deleted some that I am not interested anymore. 5439 remained. When I wanted to restore on my new installation (Android 6.0 coming from 5.1), SMS Backup+ would first download all messages. It would then claim it restored a few hundreds and 5001 were duplicates.
It seems the restored messages were stored chronologically, so only messages up to 2013 were actually restored.
In the settings, I set to restore only 500 messages. Now running Restore again, I got the latest 500 messages restored correctly and no dups.
The Call log however, didn't get restored in either run.

Please try out the beta and open a new issue if it doesn't work https://github.com/jberkel/sms-backup-plus#beta

In the settings, I set to restore only 500 messages. Now running Restore again, I got the latest 500 messages restored correctly and no dups.

Good to hear that worked at least. I tend to just leave old messages in Gmail.

can i comment on this if closed... this will find out...

I have just installed sms backup+ and it appears to only backup a limited number of texts - just a few months worth in 2015! I have reset the limit from 500 (which i noticed in trying to solve this) and the group of contacts to frequent only, and now it responds by starting a backup, and diminishing the number 0/5,000 down to 0/ of nothing to backup... Any ideas?

Hi,
Could anyone advise about how to have all the same messages between the same two people tied together?
I understand there's a 100 message limit, but I'm seeing where many of the messages between me and a single recipient are coming in in counts of far less than 100. Ideally, all messages would be in order and linked up to 100 before another convo thread is started. On my phone it's just 2 people back and forth for years of texts. In Gmail it appears as 100 here, 5 there, 1 here, 3 there, 37 here, 100 there, and on and on randomly despite the same person, subject line, phone numbers, etc.


Make File Organizer

PNG,GIF,JPG,MP4 anything
Write a script to organize files in a directory based on file type. It could automatically create folders like "Images," "Documents," and "Videos" and move files to the appropriate folder.

@sumanth-0 please assign this issue to me.


V1.1/jlm

Sets up a basic working flask API from which to base feature changes, also adds Dockerfile option for containerized deployment.

Nice work! Looks like it returns recommendations — that's exactly what we need for Release Canvas #1! Beautiful.

Nice work! Looks like it returns recommendations — that's exactly what we need for Release Canvas #1! Beautiful.


UI: enhance archive page

on the archive page additional information for each report could be displayed (report type, number of managements, gateways, found rules...)

done


Better optimization around 16,384 byte root directory threshold

If the directory does not fit in only the root, right now the code defaults to creating a root with leaves of a fixed size 4, 8, 16... kB.
It should be smart enough for two cases:

If the total directory is slightly larger than 16,384, the root should have a mix of tile entries and leaf pointers
If the total directory is much larger, it should try to fill as much of the 16,384 as possible.

This is improved to match planetiler now, so going to close.


Bump version to 0.14.4

Coverage increased (+0.1%) to 82.338% when pulling e55a9b77a7517a15c38643f6a595281de916eba2 on geobeau:fix-graphite-metadata-cache into 524a27dd5167632b4dda4c92c9dd2f6a3ed58a6c on criteo:master.


[FEATURE]: Update roadmap

Is there an existing issue for this?

[X] I have searched the existing issues

New feature
Cross off already completed items, add new ones, etc.
Anything else?
No response

Usurped by #68.


router: add response_code to downstream stream_info

Commit Message:
The response_code from the upstream request is not currently available on the stream_info for the downstream request. With this change, we set the downstream's stream_info.response_code_ to that which we receive from the upstream request.
Additional Description:
Risk Level:
Medium - Anyone relying on the response_code to be unset will experience undefined changed behavior.
Testing: unit test -- fails without proposed changes
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

The golang integration tests are failing due to codified assumption that the response code will not be set. The checks were added here, but I cannot determine if we expect the response code not to be set for a particular purpose, or because that was the state of the world at the time of that PR. @doujiang24 do you know which it was by chance?

I guess it is because

%RESPONSE_CODE%
HTTP
HTTP response code. Note that a response code of '0' means that the server never sent the beginning of a response. This generally means that the (downstream) client disconnected.

https://github.com/envoyproxy/envoy/blob/main/docs/root/configuration/observability/access_log/usage.rst
In the onUpstreamHeaders phase, the response may not have been sent to the client.

This is a good point. It makes more sense to me though to have it available, then overwrite it if the downstream disconnects before headers.
Also interesting is that the downstream_response_started_ field is set to true not much later in that same function, so perhaps setting this should occur after route_entry_->finalizeResponseHeaders?
I will try to find an integration test that validates this or create one if none exists

@spacewander this change does not affect the access log when an upstream returns headers, then the downstream disconnects. I added an integration test to validate this, and the test can be validated to have the same result with or without the added code in router.cc

/assign @yanavlasov

@yanavlasov i think this is awaiting further review/feedback
/wait-any

Sorry, I was out last week. LGTM.
@jbohanon please merge main to resolve coverage test error.
/wait

@phlax @yanavlasov Resolved! Passing all of the azure pre-submit checks now

Anything still holding this up from merging? @phlax @yanavlasov


Plans to make tween engine swappable or, more specifically, replace iTween?

Any desire to swap out iTween for either DOTween or LeanTween in the near future? I heard from various sources that iTween has terrible performance (didn't actually benchmark them myself):

iTween iPhone Performance
iTween, DOTween, or LeanTween
LeanTween Speed Comparison to iTween
DOTween - compares near bottom

Has anyone reported fungus not running well on mobile because of this?  I've heard tons of good things about these two. I have been using DOTween and enjoy it very much so far.

yes i was in this topic time ago..
DOTween is great but maybe too complex.. LeanTween is just a script!

What do you mean by DOTween is too complex? And that LeanTween is just a script?

We actually use LeanTween internally in Fungus for stuff like moving portraits on the stage.
iTween is basically terrible, but it's what people were using when I started Fungus. There's now a lot of tutorials and videos out there explaining how to use it, so I don't plan to remove the functionality from Fungus. If somebody wants to contribute new commands that wrap LeanTween I'd be happy to accept a pull request.

@snarlynarwhal i mean: DOTween (that i use everywhere!) is so much powerful, but it creates lots of files to be well setup.. LeanTween is just one file, already embedded in the Fungus core (as @chrisgregan just reminded) so i think it's preferable

@chrisgregan okay I should have time to do that. I

Okay, duh, so when I have time, I can create LeanTween commands that mimic the iTween commands. Fungus strictly uses iTween for commands and LeanTween internally, correct? So theoretically, using these LeanTween commands instead of iTween ones should remove traces of iTween usage?

@snarlynarwhal this was high on my priority list before I got caught up with other work; happy to see someone is taking charge! Looking forward to the PR! :)

It might be a good couple of weeks before I get to this, but it shall get done! To clarify though:
Fungus strictly uses iTween for commands and LeanTween internally, correct? So using these LeanTween commands instead of iTween ones should remove traces of iTween usage within my project?

@snarlynarwhal looking at the references, I see references in CameraManager.cs and Writer.cs; otherwise it's all references to Command's. You're also, for backwards compatibility reasons, going to have to retain the int values from iTween.LoopType and iTween.EaseType and transform them into something consumable by LeanTween, I think it's LeanTweenType.

If this is still desired, I'd propose deprecating all of the old iTween commands entirely rather than trying to match their interface. After a little look into I found that while LeanTween has a 'shake', it is not like iTweens at all. Also means anyone's project that relies on the particular oddness of iTween will not be impacted.

A new set of tweening commands based on Lean Tween might be the best approach, don't worry about matching the old iTween interface. I wouldn't recommend removing or deprecating the existing iTween commands as they work fine for many people and there really is a lot of tutorial material out there based on them. Maybe we could show a small warning in the inspector recommending you to use the new Lean Tween based commands instead?

Sounds good, I'll follow that up probably this weekend.

Upgraded to 2017.2.0f3 today, and unfortunately iTween is looking longer in the tooth than ever. We're getting a mess of warnings out of it:
Assets/Fungus/Thirdparty/iTween/iTween.cs(4139,26): warning CS0618: UnityEngine.GUIText' is obsolete: This component is part of the legacy UI system and will be removed in a future release.'
When the old Unity GUI types are culled, it looks like the bundled version of iTween will kick the bucket. We should probably decide how we want to deal, either by excising the old GUI references, or performing a more aggressive migration (removal of iTween commands in favor of @stevehalliwell's PR?)


Decks Onchain

Context (Problem, Motivation, Solution)
#103 Bring Decks Onchain
#111 Add Deck Validity Status when Constructing Deck
Describe Your Changes
Decks are now saved and modified onchain.
Decks owned by the player are now presented in their collection.
Minor changes to UI but not overhauled (#93)
A couple of functions added in the Inventory.sol contract, namely:
getAllDecks
getDeckNames
getDeckReal
Checklist

[x] I have performed a self-review of my code
[x] I ran make check and fixed resulting issues
[ ] I ran the relevant tests and they all pass
[ ] I wrote tests for my new features, or added regression tests for the bug I fixed

Testing
If you didn't write tests, explain how you made sure the code was correct and working as intended.

Unable to set this to draft.

You don't see "Still in progress? Convert to Draft" in the top right panel?
If you do see it, what happens if you click it?
You have write access to the repo, so you should be able to convert the PR to draft in theory!

I see this still has conflict, so should be rebased on top of master!

It's kind of weird there are commits from Aritra in the middle ... rebase gone wrong?

Aritra commits are GONE.
REBASED on top of master.
All checks PASSING.
Commit history tidied up.


[SPARK-7396][Streaming][Example] Update KafkaWordCountProducer to use new Producer API

Otherwise it will throw exception:
Exception in thread "main" kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
	at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
	at kafka.producer.Producer.send(Producer.scala:77)
	at org.apache.spark.examples.streaming.KafkaWordCountProducer$.main(KafkaWordCount.scala:96)
	at org.apache.spark.examples.streaming.KafkaWordCountProducer.main(KafkaWordCount.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:623)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

Merged build triggered.

Merged build started.

Test build #31965 has started for   PR 5936 at commit 270bbe2.

Test build #31965 timed out     for PR 5936 at commit 270bbe2     after a configured wait of 150m.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31965/
Test FAILed.

CC @koeninger

Jenkins, retest this please

Merged build triggered.

Merged build started.

Test build #31972 has started for   PR 5936 at commit 270bbe2.

Test build #31972 has finished for   PR 5936 at commit 270bbe2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31972/
Test PASSed.

Not that there's necessarily anything wrong with updating the producer api being used...
.. but I don't see how that exception has anything to do with the version of the producer api, or any changes to spark's kafka version.
KafkaWordCountProducer uses no spark code.  It only uses kafka.producer._ (which still exists in the <IP_ADDRESS> artifact), and java.util.properties.
I just ran the old version of KafkaWordCountProducer against a local kafka install with no problems.
@jerryshao how were you getting that exception?

FWIW I ran the new version, and it works as well, I'm just concerned about how that exception was caused.

Hi @koeninger , I just run this example with no additional configuration for the first time, I met such exception when the example is started. But again I run it, there's no such exception, instead I met some of warning logs like:
WARN common.AppInfo$: Can't read Kafka version from MANIFEST.MF. Possible cause: java.lang.NullPointerException

I guess this was introduced by the upgrading of Kafka version, since I didn't meet this before. So I rewrite this example with new Producer API, and this warning log is gone.

Anyway I think it's meaningful to update this example accordingly.

LGTM. Merging this. Thanks!


Update artifact leg in release pipeline

Several artifact legs in VMR / Source-build were updated recently due to OS EOL. We need to reflect this change in the source-build release pipeline

The 8.0 changes were merged so that release also needs updating.

PR - https://dev.azure.com/dnceng/internal/_git/dotnet-release/pullrequest/39280


chore: add RefreshMode parameter

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license
Summary
add RefreshMode parameter
Implementation highlights
(describe how the merge request does for feature changes, share the RFC link if it has)
Test checklist

[ ] add new test cases
[x] all code changes are covered by unit tests
[ ] end-to-end tests

[ ] deploy web console with CloudFront + S3 + API gateway
[ ] deploy web console within VPC
[ ] deploy ingestion server

[ ] with MSK sink
[ ] with KDS sink
[ ] with S3 sink

[ ] deploy data processing
[x] deploy data modeling

[ ] new Redshift Serverless
[ ] provisioned Redshift
[ ] Athena

[ ] deploy with reporting
[ ] streaming ingestion

[ ] with Redshift Serverless
[ ] with provisioned Redshift

Is it a breaking change

[ ] add parameters without default value in stack
[ ] introduce new service permission in stack
[ ] introduce new top level stack module

Miscellaneous

[ ] introduce new symbol link source file(s) to be shared among infra code, web console frontend, and web console backend

SonarQube Quality Gate Result

Result: :white_check_mark: OK
Triggered by @dengmingtong on pull_request

Metric
Status
Value
Error Threshold

Reliability rating
:white_check_mark: OK
1
> 1

Security rating
:white_check_mark: OK
1
> 1

Sqale rating
:white_check_mark: OK
1
> 1

Coverage
:white_check_mark: OK
84.10
< 80

Duplicated lines density
:white_check_mark: OK
5
> 30

Blocker violations
:white_check_mark: OK
0
> 0

Bugs
:white_check_mark: OK
0
> 0

Code smells
:white_check_mark: OK
7
> 40

Critical violations
:white_check_mark: OK
0
> 0

Major violations
:white_check_mark: OK
0
> 0

Vulnerabilities
:white_check_mark: OK
0
> 0

View on SonarQube
updated: 5/11/2024, 11:38:38 (UTC+0)

SonarQube Code Analytics
Quality Gate passed

Additional information
The following metrics might not affect the Quality Gate status but improving them will improve your project code quality.
Issues
 0 Bugs
 0 Vulnerabilities
 0 Code Smells
Coverage and Duplications
No data Coverage
No data Duplication


Missing deck

Hi!
We're trying to play with this deck https://pretendyoure.xyz/zy/metrics/deck/9ZSQH, but once I write /addcardcast 9ZSQH in the chat the server can't find the deck:
Error: Cannot find Cardcast deck with given ID. If you just added this deck to Cardcast, wait a few minutes and try again.

Cardcast was shut down in May; you can't use it anymore.

Cardcast was shut down in May; you can't use it anymore.


[NO-TOPIC] Looking for a nocode job? Sign up if you are not interested

Looking for a no-code job?
We are a no-company always looking for no-code specialists.
Sign up if you are not interested.

I am not signing up as hard as I possibly can right now

I'm not interested enough to find the "not interested" button..

I have no years experience. Is that enough?


varlib.instancer is zeroing out valuerecords

I found my font wasn't working well after static instantiation. So I tried this, which should be a no-op:
fonttools varLib.instancer -o test.ttf my_font.ttf

There are significant differences between the GPOS tables, all of which involve lookups having their valuerecords zeroed out:
...
@@ -391958,7 +410146,7 @@
               <Glyph value="bracketleft.case"/>
             </Coverage>
             <ValueFormat value="4"/>
-            <Value XAdvance="30"/>
+            <Value XAdvance="0"/>
           </SinglePos>
         </ExtensionPos>
         <ExtensionPos index="1" Format="1">
@@ -391976,7 +410164,7 @@
               <Glyph value="bracketright.case"/>
             </Coverage>
             <ValueFormat value="5"/>
-            <Value XPlacement="30" XAdvance="30"/>
+            <Value XPlacement="0" XAdvance="0"/>
           </SinglePos>
         </ExtensionPos>
         <MarkFilteringSet value="0"/>
...

This is obviously wrong.

Ah, this is probably #2555. And oh God I bet I caused it in #2550.

Yes this is #2555, and it was happening even before #2550

Good news, it seems unrelated to #2550.


Fixed picker.js throwing random requirejs script error on development

In Community (development), this script error is rather constant than random:

Putting picker components in requirejs config seems to fix the issue. Console is clean :clap:

This looks good to me :+1:
Just one note for all @lonelyplanet/ui devs: This is going to need updating of requirejs.yml in apps that use Rizzo (just in case you stumble across any issues).
ps: \o/ glad to finally be rid of this. I seem to recall it featuring in TrackJS a lot.


Add documentation for how to use rewrite namespaced maven-plugin properties.

Add documentation for how to use the rewrite namespaced properties added via: https://github.com/openrewrite/rewrite-maven-plugin/pull/226
Note. https://github.com/openrewrite/rewrite-maven-plugin/issues/314 how to run only a specific recipe

Fixed: via 024ca742267f0487604ff1c5ec5465bf65ccc7a5


[Authentication] Technical debt in defaulting code

How to categorize this PR?

/area dev-productivity
/kind technical-debt
What this PR does / why we need it:
This PR cleans up technical debt in our API defaulting code according to https://github.com/gardener/gardener/issues/7312.
This PR tackles the following resources in the API group:

authentication

Which issue(s) this PR fixes:
Part of https://github.com/gardener/gardener/issues/7312
Special notes for your reviewer:
Release note:

NONE

/retest


Add definition for bytesN helpers

Addresses #101

Can you add python typehints for the new function?

@ralexstokes thanks for making this PR. Do you mind fixing the conflicts?

@hwwhww looks like i fixed the conflict.... by pulling in the intervening history. happy to create a new PR that just cherry-picks the relevant commits if you'd like -- the original merge conflict was just w/ the reorg into spec/core documents so it will be easy w/ a cherry-pick

Going to close this PR.
The helper is currently defined on the fly in the spec
We define another set of helpers to be used throughout: bytes1(x): return x.to_bytes(1, 'big'), bytes2(x): return x.to_bytes(2, 'big'), and so on for all integers, particularly 1, 2, 3, 4, 8, 32.


Storage class selection page should be skipped if all PVs are selected to move

In the wizard it is slightly confusing when I approach the storage class selection screen and no rows are displayed in the table. It makes me as a user feel like something is wrong. Would be nice if the UI either displayed a note saying that everything is okay or if the page was skipped.

@eriknelson @dymurray One possible solution I see, is to dynamically generate steps, and move the PV related steps in a sub step.
The outcome could look like this when a PV list is empty or all PVs are set to move action:
or in the opposite case, it will have an additional step for Storage class selection.

This solution for this issue will be delayed, as the UX opinion should be gathered.


{Misc}: update codeowners

Update Codeowner to latest.
History Notes:
(Fill in the following template if multiple notes are needed, otherwise PR title will be used for history note.)
[Component Name 1] (BREAKING CHANGE:) (az command:) make some customer-facing change.
[Component Name 2] (BREAKING CHANGE:) (az command:) make some customer-facing change.

This checklist is used to make sure that common guidelines for a pull request are followed.

[-] The PR title and description has followed the guideline in Submitting Pull Requests.

[-] I adhere to the Command Guidelines.

HI @yonzhan could you pls help to look at this PR? thanks.

add to S166.


Bump app-launcher to v0.2.0

What?
Bump app-launcher from v0.2.0.alpha to v0.2.0
Why?
To use the latest app

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: d-hayashi
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [d-hayashi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Grid halo

clisops version: 0.6.1

Description
Climate model data can be provided on grids that feature a halo, which are duplicated columns at the eastern and/or western bounds of the model grid (eg. MPI-ESM1-2-LR/HR MPIOM). Here is a quote from the NEMO ocean engine document:

Since bicubic interpolation requires the calculation of gradients at eachpoint on the grid, the corresponding arrays are dimensioned with a halo of width one grid point all the way around. When the array of points from the data file is adjacent to an edge of the data grid, the halo is either a copy of the row/column next to it (non-cyclical case), or is a copy of one from the first few columns on the opposite side of the grid (cyclical case).

The cyclical case is sketched here:

The data on the halo grid points might influence the clisops operators:

averaging: the duplicated values might influence global / regional / polygon averages
subset: subset by area/shape might select the duplicate grid points which might or might not be desired
regrid: I experienced problems creating regridding weights (other than bilinear and nearest neighbour) with xESMF when a halo was present

Regarding the halo there are two more issues:

In case of a staggered grid ("On a staggered grid the scalar variables (pressure, density, total enthalpy etc.) are stored in the cell centers of the control volumes, whereas the velocity or momentum variables are located at the cell faces.") one may not simply cut off the halo for variables defined on the cell faces/vertices as for them the halo columns/rows hold actual (not duplicated) information. When dealing with variables defined on the u-points or v-points of a staggered grid, one has to rotate these into latitudinal and longitudinal direction before regridding (and probably also before a spacial average?!).
In case of MPI-ESM1-2-HR MPIOM there are also extra rows added that have to be removed before processing / analysing the data. These extra rows are also part of the halo. Due to the grid layout, some rows almost have a north-south alignment (see the notebook linked in the next post). In that regard it seems for some grids rows/columns are only partially duplicated.

A first way to identify the unique columns of a grid
# 2D latitude and longitude arrays - create an array of (lat,lon) tuples
latlon_halo=np.array(list(zip(ds["latitude"].values.ravel(),ds["longitude"].values.ravel())),
                     dtype=('double,double')).reshape(ds["longitude"].values.shape)

# use numpy.unique to identify unique columns
latlon_no_halo,indices=np.unique(latlon_halo, axis=1, return_index=True)

Looking forward to your comments and suggestions :)

For the model CESM2-FV2 the duplicated grid cells do not result from a halo but from missing values within the lat and lon coordinate variables. I do not think that this is conformal with CF or CMIP requirements, but how would clisops react to that?
Here eg. the lon variable as shown by ncview:

@sol1105 I just saw some code from the Met Office for removing the halo from their own models. I am just noting it here in case it is of use to us. It uses NCO:
ncks -O -dx,1,360 -dy,1,330 filename -o temporary_filename

The missing value problem with data from CESM2-FV2 has to be addressed in a daops/dachar fix as it results from a not set _FillValue attribute. With lon values outside [-360, 360] degrees and lat values outside [-90,90] for these not masked missing values, clisops will raise an exception.
The general issue with the halo has been solved in #243 :
adaptive masking can be used to renormalize contributions from halo cells (duplicated cells) even when not "activating" adaptive masking for renormalization of partly masked cells. Excerpt from core.regrid.regrid:
# Re-normalize at least contributions from duplicated cells, if adaptive masking is deactivated
 if (adaptive_masking_threshold < 0 or adaptive_masking_threshold > 1) and grid_in.contains_duplicated_cells:
        adaptive_masking_threshold = 0.0
        grid_out.ds[data_var] = weights.regridder(
                    grid_in.ds[data_var],
                    skipna=True,
                    na_thres=adaptive_masking_threshold,
                )


feat(v2): redesign mobile UX: inline TOC + doc sidebar in main menu

Note from @slorber : this PR has been marked as a breaking change to draw attention that:

the mobile UX is now significantly different
multiple theme components were updated, and you may likely update your swizzled components accordingly

Motivation
Attempt to resolve #2220.
Closes #4927
Closes #4700
Dirty/draft and apparently improper implementation, very quickly written to demonstrate new navigation UI (TOC) on mobiles (upon request from @markerikson).
The main idea is to render the doc sidebar content on mobiles in the navbar sidebar and also add a new sticky button to display the local l (by document)  TOC. This gives us place for the back to top button.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan

Related PRs
(If this PR adds or changes functionality, please take some time to update the docs at https://github.com/facebook/docusaurus, and link to your PR here.)

Just tried accessing the v2 classic preview on my phone, and this looks excellent!  Thank you!
One bug: I tried selecting an in-page ToC header from the right-hand popup. While the page did scroll to the correct location, the popup didn't close when I clicked the link.

The last commit looks like it fixed the "ToC popup doesn't hide when link tapped" issue, but for some reason the popup is still empty of any link content in Chrome for Android.

That looks pretty cool, thanks for working on this!
Some design feedbacks:

I think the mobile TOC does not show the links with dark mode
The mobile TOC view is too "square". I think something more centered as a large modal, not touching the edges of the screen, could look better (even if it reduces the toc surface).
Should we separate regular navbar items from the sidebar in a more explicit way?

From a technical perspective, will have to take a deeper look.
My initial idea was to avoid using the global data API for this feature because the data is "global", ie it is available on all the pages of the site, and having a long sidebar means it impacts the whole site performance (including the homepage, and we have Docusaurus sites with huge sidebars like this one: https://xsoar.pan.dev/docs/reference/index)
What I would try is to replace the global data API with a "portal" / "slot-fill" library:

you declare a slot in the sidebar, like "sidebar-top" in your implementation
you let the document page fill that slot from another place in the JSX (ie directly in the Doc page JSX)

The problem is that afaik there's no up-to-date maintained lib that supports SSR:

https://github.com/gregberge/react-teleporter does not support SSR
https://github.com/cloudflare/react-gateway supported SSR but is not maintained

Maybe we don't need to support SSR for the mobile sidebar (as we don't display the sidebar before the user click on the hamburger menu, there wouldn't be FOUC), but at the same time it may be important for SEO to render that sidebar in the markup as crawlers are mobile-first? (don't know if this matters because anyway it's hidden before you open the mobile menu)
I would try to use react-teleport instead of the global data API and see if this can solve the use-case in a more efficient way

the popup is still empty of any link content in Chrome for Android.

@markerikson could you please give more details on this (source, screenshots)? I tested in this browser and didn't notice this issue.

@slorber Why is using global data bad in this case? It seems to me that this approach works well, especially since we can remove the permalinkToSidebar field, which you marked as not very useful.
I agree that the current way is probably not well suited for large docs sites, and certain conditions (proper navbar items) are needed to display sidebar contents on mobiles. I have another idea of how to present this menu, but I would like to try to implement it also via using global data, so honesty I don't really like the suggestion with portal.

there seem to be an issue with dark mode style

Yes that's what I reported.
@lex111 go ahead with global data for now, as sidebars data is not huge and I'll try to see if I can refactor/optimize that later.
It's better to have this feature asap even if it's not the ideal implementation IMHO, and we don't document docs global data APIs on purpose currently so we can do breaking changes on it later.

FYI for sites with a very large sidebar, rendering the sidebar on the server for each doc page is expensive (cf https://github.com/facebook/docusaurus/issues/4753, just the sidebar increase a site's build time by 12 minutes!)
We should test this new mobile UX against such a large site and see the potential impacts on build time and runtime perf.
This mobile drawer is not displayed before hydration so we can probably render this sidebar only on the client.

Like this a whole lot as well! What's still open here? Just the darkmode issue?

Sorry for such a long wait! I decided to completely rethink previous UI design, and the obtained result seems to me to be the best user experience:
TOC of page is available as a collapsible panel by pressing button in front of the document heading, and the sidebar contents are displayed as a separate app-like menu. Hope you enjoy this option.
Traditionally, there is internal implementation needs refactoring, and the styles do not fully support dark mode. This will all be fixed after we come to a final solution.

@lex111 this looks great! I really like the inline collapsible TOC - great solution for mobile!

Hmm. I appreciate the effort that went into this, but I'm not convinced it's the right approach.
It looks to me like any time I want to access the in-page TOC, I'd first have to scroll all the way back up to the top of the page to expand that dropdown.  That could easily get annoying on long docs page.
This was actually a great thing about having the floating button. It's accessible no matter how far down you've scrolled on the current page.
Personally I'd still really like to see the approach I first suggested: keep the floating button, and use that to trigger showing the in-page TOC.

It looks to me like any time I want to access the in-page TOC, I'd first have to scroll all the way back up to the top of the page to expand that dropdown. That could easily get annoying on long docs page.

I was thinking about that. But later we will have "back to top" button that will make it easier to go to TOC collapsible panel. So given that, I don't want to clutter the interface with two floating buttons. Make sense for you?

Heh. Having any way to access the page TOC on mobile is an improvement over what we've got now :)  To me, having that floating button bring up a TOC overlay makes the most sense design-wise, but since you're the one implementing this and I'm over here in the peanut gallery, I can't complain too much - I'll take whatever actually gets implemented at this point.

Thank you for understanding ;)
In fact, I researched a lot of docs sites to come up with this new solution that I feel is most appropriate for needs of all users. As always, my concern is for users to focus on the main thing on site -- its content. And the current UI is the best way to do that (imo).

My 2 cents - although I can understand and respect the idea of the floating button allowing access to the TOC no matter where one is currently scrolled on the page, I have to say in my experience I find those floating buttons mostly annoying and rarely use them.
I think a "back to the top" and then having the TOC there, like @lex111 has in their latest update is a perfectly fine compromise.

Just wanted to pop in here to mention, I saw google on their web.dev/learncss course is using the same inline TOC at the top of the page :tada:
https://web.dev/learn/css/

Would love to have a chance to try this out on the Redux docs sites.  Anything left to do on this besides the merge conflicts?

I have just resolved all conflicts. We are now awaiting feedback from @slorber.

Resolved conflicts again, friendly ping for @slorber.

Hi there!
That really looks great @lex111 , good job!
We can figure out later if adding an optional floating button should be added.
I like to access the TOC easily but also agree that floating buttons tend to be annoying.
I think we can decide that in another upcoming PR.
FYI the goal for me to stabilize a bit the current code in a very small beta.3 (fixing some regressions)
And review/merge this next week, to include in beta.4.
You can be an early adopter with the canary release channel (https://docusaurus.io/community/canary) and provide feedback so that we are sure it's good when it's finally released.

This PR is fine regarding the design and others reported positive feedback about it, and we can move on with this UX.
Regarding the implementation, I'm going to do some additional cleanups and refactors to improve from current state.
Mostly for 2 reasons.
FOUC / layout shifts
The current deploy preview has a quite annoying behavior on desktop. This is what we see before React hydration:

Global Data size
Our size bot reports a significant increase of the global data size:
https://github.com/facebook/docusaurus/pull/4273#issuecomment-784177179

This was somehow expected, as we put more data in there.
In the current state, this increase will be proportional to: number of versions * number of docs
We don't have a lot of versions/docs on our site and a 60Kb increase is already quite significant, doubling its size.
Note: this data needs to be fetched by every page of your Docusaurus site before hydration, so it has a significant perf impact on your site. And we have very large sites that will be affected by this: https://xsoar.pan.dev/docs/reference/integrations/abusech-ssl-blacklist-feed

Made small restyling back button so that its background does not cover shadow of navbar brand.

Before
After

So, this PR seems to be ready for a final review before merge!
Please let me know if you see any problem asap.
Layout shift issues should not happen anymore on both mobile/desktop, for all pages
Output assets did not increase significantly, and some were even optimized a bit:

Great, looking forward to seeing this released!

Wow! Thank you for this massive overhaul.
Feedback: on mobile browsers, I think once an item in the main menu is selected, the sidebar should stay there and let us select the next item from its submenu. Say, I am in Docs → Installation, and want to navigate to API → Lifecycle APIs. I hit the menu icon → Back to main menu → API. At this point the sidebar goes away, this behaviour feels bit odd to me (I expected it to let me select next item from its submenu), and it takes to the root of API. Then I need to hit the menu icon again in order to navigate its submenu. It should not disappear until I have selected an item form the submenu or I touched outside the sidebar to hide it. In case I did not select submenu item and touched outside the sidebar to hide, it should take me to the root of selected main menu.

thanks for the feedback @pranabdas. The UX you mention was initially implemented by @lex111  but I had to remove it because it was more complicated to achieve after. I have some ideas to implement this again with our new architecture.
But, does this UX always make sense?
If for example, you have 3 navbar items of type docs inside the navbar, linking to different docs of the same sidebar, don't we want to directly navigate to the linked doc instead of keeping the navbar opened?
For example, https://react-redux.js.org/ has multiple navbar docs to the same sidebar:

It would be weird if pressing on any of those 4 items would lead to the same result: keeping the nav sidebar opened and displaying the docs sidebar (even if a different one would be highlighted).
I think what we could do is introduce a new navbar item with type = docSidebar, and it would link to the first doc of a given sidebar (you have to provide a sidebar name, not a docId), and we could implement such behavior more confidently because it does not make much sense anyway to link twice to the same sidebar from your navbar.
Does it make sense?

@slorber I think the redux website gives double options (same items in navbar and sidebar). Once I pulled the sidebar, I can navigate from there. I don't really need to go Back to main menu in order to get the same sidebar. But this certainly makes sense if one has super long sidebar that would save one from scrolling up/down. You select any item in the main menu, you get the same sidebar but at different vertical scroll positions.
I think if an item has sidebar, the expanded sidebar should stay there to let me choose. If different items on the main menu has separate sidebars, it feels abrupt otherwise. If one touches outside the expanded sidebar to hide it after selecting on the main menu, the behaviour should be as it is now. Just my thoughts. Thank you.

Once I pulled the sidebar, I can navigate from there. I don't really need to go Back to main menu in order to get the same sidebar.

Think about the user being on the homepage and clicking a navbar item. It's not even about going back, but what to expect on a navbar item click. With your UX, to access the Redux tutorial, it requires to press first on the sidebar tutorial element, and then to press on the sidebar overlay to close the sidebar (which is showing the docs sidebar, focusing on the tutorial element). Do we really want the user to press 2 times in this case? @markerikson any thoughts?

I think if an item has sidebar

Technically we currently can't know if a navbar item belongs to a doc inside a sidebar, or an "orphan doc" without sidebar. This is because of the modular nature of Docusaurus: not all the data are available everywhere and we try to keep the "global data" (available on the homepage, blog etc...) very small because it impacts the perfs of your whole site.
Having a new navbar item of type docsSidebar would solve this problem because we could just say "if type === docsSidebar", then show the sidebar secondary panel (docs sidebar) instead of closing it.
We could also have a boolean on navbar items to tell if the mobile sidebar should be closed on click or not.
Both UX makes sense so we should rather not force users of the docs navbar item type to use one UX in particular when they want the other, but make this behavior configurable and explicit.
I already planned to implement the docsSidebar navbar item type, because it make sense when you want to implement some kind of "tab system", like seen on RN site (https://reactnative.dev/ => Docs / Components / API) where each 3 navbar item is linked to a distinct sidebar. Passing the first docId of each sidebar is a weird API to achieve that.
The question is: would this new sidebar item type solve your problem?

Yes, a new docWithSidebar type would give more flexibility. Hopefully it inherits all other properties from the doc type (I am not sure where else it is being used). Thank you.

Thanks @pranabdas , actually tried to see how to make this UX work but it's not easy/possible in current state because our layout/navbar unmount/remount after each navigation and lose its opened state. This requires more complex refactorings to make this UX possible, so we'll figure this out in another PR later

Let's merge this! 🎊
Please test this on your sites with https://docusaurus.io/community/canary

@slorber thank you so much! i found a weird inconsistency when opening/closing menus - sometimes it would go to main menu, sometimes it would go to the TOC menu, and it doesnt seem very predictable:
https://www.loom.com/share/d1ede4c5c04f41e9bd92785dcdac0e89

Thanks for the feedback 😜 and great unusual video bug report.
This looks like a valid bug.
While on a doc with sidebar, you should always end-up opening the mobile
sidebar on the TOC by default, consistently.
Will fix that soon, thanks for reporting.
Le ven. 9 juil. 2021 à 19:59, swyx @.***> a écrit :

@slorber https://github.com/slorber thank you so much! i found a weird
inconsistency when opening/closing menus - sometimes it would go to main
menu, sometimes it would go to the TOC menu, and it doesnt seem very
predictable:
https://www.loom.com/share/d1ede4c5c04f41e9bd92785dcdac0e89
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/facebook/docusaurus/pull/4273#issuecomment-877360318,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFW6PRV3F5RKVJENZKAHT3TW42G5ANCNFSM4YCLN27Q
.

@sw-yx I was wondering why the mobile collapsible TOC does not show up here:

Do you have a PR / preview so that I can inspect it more easily? I can also open a PR to the temporal doc if you want, and we try to get everything working properly here.

ah sorry just saw this! no i didnt have a separate PR opened, i just bumped versions to canary in local dev. thanks for handling it anyway!!!


mac os sierra 10.12.3 install fail

information:

OS?

Mac OS Sierra 10.12.3

Versions.

node v6.9.5
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Repro steps.

sudo npm install -g @angular/cli

The log given by the failure.

`RandyLees-MacBook-Pro:~ randylee$ sudo npm install -g @angular/cli
Password:
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm ERR! unpack Could not read data for<EMAIL_ADDRESS>npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm ERR! Darwin 16.4.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "-g" "@angular/cli"
npm ERR! node v6.9.5
npm ERR! npm  v3.10.10

npm ERR! Invalid name: "@angular/tsc-wrapped"
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/randylee/npm-debug.log`

I am also getting similar error in mac OS Sierra 10.12 and node version is 7.5.0. I am getting following error after running sudo npm install -g @angular/cli :
npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN gentlyRm not removing /Users/Dhrubo/.npm-global/bin/ng as it wasn't installed by /Users/Dhrubo/.npm-global/lib/node_modules/@angular/cli
/Users/Dhrubo/.npm-global/bin/ng -> /Users/Dhrubo/.npm-global/lib/node_modules/@angular/cli/bin/ng
/Users/Dhrubo/.npm-global/lib
└─┬<EMAIL_ADDRESS>  └── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

This is a duplicate of #4530. Try the solution suggested there.

@bboywulong you seem to have some proxy/firewall/npm version issue. You're not getting packages correctly.
@dhrubomoy yours is a dupe as catull said.


[!] Unable to find a specification for Yoga (= 0.42.3.React) depended upon by React/Core

When I use RN 0.42 in my old iOS project, I define a pod file as follows:
project 'RN3.xcodeproj'
target 'RN3' do
pod 'React', :path => '../node_modules/react-native', :subspecs => [
'Core',
'RCTImage',
'RCTNetwork',
'RCTText',
'RCTWebSocket',
]
end
When I exec pod install in command line, there is an error message:
[!] Unable to find a specification for Yoga (= 0.42.3.React) depended upon by React/Core.
I check the React.podspec, I think there is something wrong in this file, is anybody meet the same issue?

Add this line in your Podfile. Have a try.
pod 'Yoga', :path => '../node_modules/react-native/ReactCommon/yoga'

Yoga works now. but anther issue occurs:
platform :ios, '10.2'
project 'RN4.xcodeproj'
target 'RN4' do
pod 'Yoga', :path => '../node_modules/react-native/ReactCommon/yoga'
pod 'React', :path => '../node_modules/react-native', :subspecs => [
'Core',
'RCTImage',
'RCTNetwork',
'RCTText',
'RCTWebSocket',
]
end
In the workspace, #include <folly/dynamic.h>, Xcode says "dynamic.h file not found".

Hey, thanks for reporting this issue!
It looks like your description is missing some necessary information. Can you please add all the details specified in the template? This is necessary for people to be able to understand and reproduce the issue being reported.
I am going to close this, but feel free to open a new issue with the additional information provided. Thanks!

[!] Unable to find a specification for yoga (= 0.51.0.React) depended upon by React/Core
I had a similar problem today and I fixed it by adding to my Podfile:
pod 'yoga', :path => '../node_modules/react-native/ReactCommon/yoga'
'yoga' is in lowercase.


Unable to start span as current

Reproducible example:
object Test extends IOApp {
  override def run(args: List[String]): IO[ExitCode] = {
    val ec = ExecutionContext.fromExecutor(Executors.newSingleThreadExecutor())

    Resource
      .eval(IO(GlobalOpenTelemetry.get))
      .evalMap(OtelJava.forAsync[IO])
      .evalMap(_.tracerProvider.get("test"))
      .use { tracer =>
        for {
          _ <- tracer.spanBuilder("test").build.use { span =>
            println(s"java span context: ${Span.current().getSpanContext}")
            IO(println("otel4s span context: " + span.context))
          }
        } yield ()
      }
      .as(ExitCode.Success)
      .evalOn(ec)
  }
}

Would print following output:
java span context: ImmutableSpanContext{traceId=00000000000000000000000000000000, spanId=0000000000000000, traceFlags=00, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=false}
otel4s span context: WrappedSpanContext(ImmutableSpanContext{traceId=318854a5bd6ac0dd7b0a926f89c97ecb, spanId=925ad3a126cec272, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true})

Current otel4s API does not provide the ability to make the span current and started span is not current. Because of this, I assume, a combination with existing Java instrumentations does not work - because spans that are supposed to be child spans started by auto-instrumentations have no proper parent.
Happy to submit PR. But I am not sure which way is preferable:

Start all spans as current by default?
Provider def makeCurrent: F[Unit] wrapper method.
Inject configuration into TracerProvider to make started span current.

I assume a combination of second and third options is preferable.
Thanks.

@rossabaker Thanks, for the reply. Yeah, I totally agree with both points:

I understand that there are two ecosystems involved here, e.g. context should both to be propagated from Java native OTEL Span/Tracing context to IO's Local Vault and vice versa generally.

Application developer would need to make sure that all ExecutionContext's are wrapped into proxy that supports OTEL context propagation, for instance, though: Context.taskWrapping or similar.

I just wanted to callout problem that as of now otel4s does not leverage of existing Java libs instrumentations, which is a big advantage of OTEL ecosystem. At current implementation, application developer will be forced to implement tracing and metrics middleware for common things like JDBC or Kafka client. From my point of view, it would be really nice to same OTEL experience like in other ecosystems, for instance in Akka or Java: just to plug instrumentation, configure it and around 80% of work it will do for you out of the box.
Happy to hear your thoughts.

Right now at work, we're tracing JDBC via Doobie and Natchez.  The transactor gives us a hook into obtaining a Connection, and we can create decorators around the Connection - Statement - ResultSet triumvirate.  Of course, it would have been much easier to use standard Java instrumentation, and we can only get as far as we get because Doobie had the correct hook.  I think we should absolutely explore a deeper integration like this.  The value in the Java SDK is not just that all the formats and propagation is written, but that it's already integrated with so many things we're already calling from Scala.

We can still expose some utility methods from WrappedSpanContext to ease the interop pain.
E.g. WrapperSpanContext#wrap and WrapperSpanContext#unwrap.
It will not solve the thread local's propagation issue, but at least an end user can extend the existing Java's span (and vice versa).
I agree that we should seamlessly integrate with Java thread-local propagation, but this topic is quite complicated. Especially taking into account the Scala-first implementation of the tracing logic.

Guys, last question. I'm curious about overall instrumentation concept from otel4s perspective. Let's consider example of having Kafka (though fs2-kafka) or Elastic (though elastic4s) clients in codebase. What' the way to go to instrument common cases like those: are there will be set of custom instrumentations for higher level clients (e.g. elastic4s and fs2) or you are planning to leverage existing instrumentation? Or app devs supposed to make tracing middleware by themselves? Thanks.

Throwing this out there, so take it for what it's worth, but we'll soon have a pure-Scala implementation as well, which means otel4s will work with Scala.js too. It'd be interesting to see how propagation works in that context and how best to enable interop with the JS ecosystem.

are there will bte set of custom instrumentations for higher level clients (e.g. elastic4s and fs2) or you are planning to leverage existing instrumentation

I think we'd want to leverage existing instrumentation when we can.

Things that are pure Scala, we're on our own.  For example, [Skunk](Skunk](https://github.com/typelevel/skunk/commit/2a9b4692f48e09fae33ed66dd9780a3921c1fade).
Things where we are invoked by Java, I suspect all we need is existing instrumentation.  For example, http4s-jetty.
Things where we wrap Java, take advantage of the existing instrumentation, but consider spanning it ourselves if the wrapper adds non-negligible timing or metadata.  fs2-kafka would be an example in this category, but I'm not familiar enough with it to know whether the Scala bits merit their own instrumentation.

That third category is where we'd seriously need a smooth handoff between otel4s state and opentelemetry-java state.

I think we can replace the Java SDK's context storage with an implementation that wraps around our IOLocal Vault.
One way is to implement an SPI.  SPIs need to have a no-arg constructor, which means we would need to do unutterable things with respect to IO.
Another way is a wrapper (we call these middlewares in various other Typelevel projects).  That would give us the facility to create one in IO and invoke it before loading the SDK.

@rossabaker Recently I thought about following case: what if we have two fibers running on top of one thread.
For instance second fiber started before first one. Do I understand correctly that in this case second fiber span context would override first five span context, taking into account that Java's OTEL context stored in thread locals.
I assume there are much more corner cases like this can be found, which makes me think and agree with you, that due two runtimes are involved this would be pretty hard, or even feasible at all to have stable Java's OTEL interop.
So, as a conclusion, I can assume that perhaps IO native instrumentation, at least for the most popular libs, is the way to go for app or lib developers. Does it make sense?

Neat.  One tricky thing about ContextStorage is that it while its attach and scope might implement a signature like local, their use in Java is non-local: there's no callback with automatic scope management, but rather the application calls attach and is responsible for closing the scope.  I'm not yet sure how those calls should propagate back to the Scala code.


feat: ow1 take a breather - [merged]

Merges roadhog -> staging

mentioned in commit 86fd1613541da76178e57ffeb97f4d3a27034182

In GitLab by @ghost1 on Nov 14, 2023, 23:19
:tada: This MR is included in version 1.11.0 :tada:
The release is available on GitLab release.
Your semantic-release bot :package: :rocket:

added 3 commits
f838ccd2 - 1 commit from branch staging547625ec - activate ow1 breather when pressing ability 271268a2a - disable trap ability icon when taking breather
Compare with previous version


[HUDI-3350][HUDI-3351] Rebase Record combining semantic into HoodieRecordCombiningEngine

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
Step2 (RFC - 46)：HoodieRecordCombiningEngine and its bridge and spark record
Based on Step1  , Step2 adds the HoodieRecordCombiningEngine API and the HoodieSparkRecord
Brief change log

Rebase Record combining semantic into HoodieRecordCombiningEngine
Create Engine-specific Implementations of HoodieRecord

Jira:
HUDI-3350
HUDI-3351
Verify this pull request
This pull request is already covered by existing tests
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@wulei0302 seems like CI caught some bug
2022-06-27T15:52:42.0821073Z [ERROR] testSchemaEvolutionForTableType{String}[1]  Time elapsed: 14.556 s  <<< ERROR!
2022-06-27T15:52:42.0821677Z org.apache.spark.sql.AnalysisException:
2022-06-27T15:52:42.0822277Z Intersect can only be performed on tables with the same number of columns, but the first table has 4 columns and the second table has 3 columns;;
2022-06-27T15:52:42.0823142Z 'Intersect false
2022-06-27T15:52:42.0823828Z :- LogicalRDD [_row_key#3974, partition#3975, ts#3976L, new_field#3977], false
2022-06-27T15:52:42.0824517Z +- Project [_row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0825448Z    +- Project [_hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0826519Z       +- Project [_hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0827447Z          +- Project [_hoodie_record_key#3984, _hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0828459Z             +- Project [_hoodie_commit_seqno#3983, _hoodie_record_key#3984, _hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0829600Z                +- Relation[_hoodie_commit_time#3982,_hoodie_commit_seqno#3983,_hoodie_record_key#3984,_hoodie_partition_path#3985,_hoodie_file_name#3986,_row_key#3987,partition#3988,ts#3989L] hoodie-parquet

It passes in my local env, I rerun the CI to try

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@hudi-bot run azure

@wulei0302 seems like CI caught some bug
2022-06-27T15:52:42.0821073Z [ERROR] testSchemaEvolutionForTableType{String}[1]  Time elapsed: 14.556 s  <<< ERROR!
2022-06-27T15:52:42.0821677Z org.apache.spark.sql.AnalysisException:
2022-06-27T15:52:42.0822277Z Intersect can only be performed on tables with the same number of columns, but the first table has 4 columns and the second table has 3 columns;;
2022-06-27T15:52:42.0823142Z 'Intersect false
2022-06-27T15:52:42.0823828Z :- LogicalRDD [_row_key#3974, partition#3975, ts#3976L, new_field#3977], false
2022-06-27T15:52:42.0824517Z +- Project [_row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0825448Z    +- Project [_hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0826519Z       +- Project [_hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0827447Z          +- Project [_hoodie_record_key#3984, _hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0828459Z             +- Project [_hoodie_commit_seqno#3983, _hoodie_record_key#3984, _hoodie_partition_path#3985, _hoodie_file_name#3986, _row_key#3987, partition#3988, ts#3989L]
2022-06-27T15:52:42.0829600Z                +- Relation[_hoodie_commit_time#3982,_hoodie_commit_seqno#3983,_hoodie_record_key#3984,_hoodie_partition_path#3985,_hoodie_file_name#3986,_row_key#3987,partition#3988,ts#3989L] hoodie-parquet

hey, shiyan, CI may fail because of this HUDI-4302

@hudi-bot run azure

@hudi-bot run azure

Can we update PR title and description based on the new HoodieMerge design

Made one pass, changes themselves look reasonable. Need to read SparkRecord and its utils closely. My understanding of this PR is

Wraps the payload completely using methods on the HoodieRecord abstraction
All code paths now just call HoodieRecord for merging, the old getInsertValue() is pushed into Record APIs as well.

I would love to understand

What testing has been done? AFAICT the Spark datasource writer is still not working with SparkRecord?
any perf comparisons for the new SparkRecord based writes vs AvroRecord based writes.
My wishful thinking is that we eliminate preCombine as well somehow. This will make the end system very simple for implementation in various engines.

I suggest @danny0405 to look at the changes from the Flink lens once as well.

Yeah, The Testing and SparkRecordWriter/Reader has been completed in Step3
(comparisons)This is in progress on the basis of Step3
Currently, Our guys think that preCombine acts between log files and combineAndGetUpdateValue acts between log and base files. There are some differences in definitions. Maybe from a higher abstraction, it can be merged into one API, but let's discuss this further, and mayby it can be completed in other pr

Can we update PR title and description based on the new HoodieMerge design

Done

@hudi-bot run azure

@wulei0302 @wzx140 will land this as e2e tests passed. Please go through the comments for follow-ups. Some can be tackled in step3. some can be done in later PRs.
In Step3 we should at least get API unification done for spark record, and leave the legacy APIs with avro record for compatibility (mark for deprecation so we can remove it in future versions).