flakey test

https://github.com/pushpad/web-push/blob/20f40d9586b524b8c75e64887683931d6e11b078/spec/web_push/vapid_key_spec.rb#L19
occasionally this is 31 bytes
key = nil
30_000.times do |x|
  key = Webpush::VapidKey.new
  if key.curve.private_key.to_s(2).bytesize < 32
    puts x
    break
  end
end

puts([format("%064x", key.curve.private_key.to_i)].pack('H*').bytes[0...3])
puts '---'
puts(key.curve.private_key.to_s(2).bytes[0...3])

Good catch.
You can also reproduce the issue in the tests with this code:
# vapid_key_spec.rb

it 'returns an encoded private key' do
  30_000.times do
    key = WebPush::VapidKey.new
    puts key.private_key
    expect(Base64.urlsafe_decode64(key.private_key).bytesize).to eq(32)
  end
end

However, after further investigation, it seems that the problem relies in Ruby OpenSSL and not in this gem.
For example this will sometimes fail:
30_000.times do
  key = OpenSSL::PKey::EC.generate('prime256v1')
  expect(key.private_key.to_s(2).bytesize).to eq(32)
end

Now the question is... is it a bug in OpenSSL::PKey::EC or is it normal that the private key is only 31 characters sometimes?

@rossta I see that you are the author of this test, do you have any idea why it fails sometimes (very rare)? What was the meaning of that test on the length?

I seem to recall this as my interpretation of the literature at the time, but I’m not confident this length has to be enforced. It might be worth manually testing the behavior with a shorter key generated from the method shown above.


fix(NOJIRA-123): Set size in pixels by default

When value with no units (string with numbers only) is provided, set size in pixels by default.
Both examples below will assume value in pixels:
window.tf.createWidget(formId, {
  height: "585",  // this PR fixes this case
});

window.tf.createWidget(formId, {
  height: 585,
});

[BOT] Preview available with hash 4e7da6d336c163a5a838635e264374c47db56dcd here.

[BOT] Preview available with hash ec7314ea0b3d8d62700164190d67a50988655180 here.

[BOT] Preview available with hash 61b8c58b433d9cbb5061e6eb2686670d19c64775 here.

:tada: This PR is included in version @typeform/embed-v4.3.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fix debuginfo

Provides the ability to generate debug information for nested functions
Closes #352
We need to support 2 cases:

[ ] @kernel(debug=True) - via decorator, it means propagate flag via function arguments or target context
[ ] NUMBA_DPPY_DEBUGINFO=1 - via env variable, it means in some point use env variable value if the value is not provided in decorator

Fixed, @akharche , @PokhodenkoSA , @AlexanderKalistratov  please check


Concatenate a OIDs

Just draft to implement possibility of https://github.com/rusticata/der-parser/issues/44
const A: Oid = oid!(1.2.3);
const B: Oid = oid_append!(A, 4.5); // <IP_ADDRESS>.5
const C: Oid = oid_append!(A, B);   // <IP_ADDRESS>.<IP_ADDRESS>

But this functionality requires #![feature(const_precise_live_drops)]

How much of a change would it be to change Oid so instead of exposing a single slice of bytes, it exposed an iterator of byte slices instead? This way an Oid could have an optional base_oid field. Serializing and equality comparisons would use the iterator. This would be a breaking change of course, as we'd be removing the byte slice view, but it allows for trivial compile-time and runtime appending. The biggest issue I can see (beyond it being a breaking change) is that ideally you'd want to only allow appending of relative Oids to a base, but this isn't expressed in the type system, so you'd have to define some reasonable behavior for what happens if you append an absolute Oid to another one (the obvious behavior is to treat the absolute one as a relative one instead, which just expands its encoding by one byte, and means oid!(1.2.3) + oid!(4.5.6) doesn't punish me for forgetting the rel keyword).


After update with big extimol i.e. 1.31.1 cloudflare workers stopped working, I think it was because fetch was added to this ApiClientOptions.

          After update with big extimol i.e. 1.31.1 cloudflare workers stopped working, I think it was because fetch was added to this ApiClientOptions.

Error log in Cloidflare workers:
TypeError: Illegal invocation: function called with incorrect this reference.

Originally posted by @BilagoNet in https://github.com/grammyjs/grammY/issues/655#issuecomment-2466660778

@BilagoNet can you update to 1.31.2 and check if this issue was fixed?

@BilagoNet can you update to 1.31.2 and check if this issue was fixed?

@vorant94 can you do the same?

yes, fixed

confirming it on my end, thanks!

Epic, thank you guys!
@all-contributors add @BilagoNet for bug and review
@all-contributors add @BilagoNet for review

@all-contributors add @vorant94 for review, too


TorchCraftAI failed to build

I follow the official website tutorial at this step: python setup.py build
I don't know about this setup. The py file is build, I hope the tutorial can be more detailed, because I am a beginner, thank you

转到目录“TorchCraftAI \ 3rdparty \ pytorch”，然后运行命令“python setup.py build”

In the process of execution, I encountered a compilation error of the source code. I don't know what caused this.

转到目录“TorchCraftAI \ 3rdparty \ pytorch”，然后运行命令“python setup.py build”

Do I need to put the "TorchCraftAI" file and love in the "$Starcarft" directory?

转到目录“TorchCraftAI \ 3rdparty \ pytorch”，然后运行命令“python setup.py build”

@CNGU-wang this is just warning, you can ignore.

@ CNGU-wang这只是警告，你可以忽略它们
.Failed to run' tool sbui1d pytorch_ 1ibs. bat --use-fbgemn --use-mnpack . use-mkldnn -use-qnpack caffe2
But I received this error


The 'Status' column is still visible in the editor even if the column is hidden

Storage Explorer Version: 1.34.0-dev (98)
Build Number: 20240416.1
Branch: main
Platform/OS: Windows 10/Linux Ubuntu 22.04/MacOS Sonoma 14.4(Apple M1 Pro)
Architecture: x64/x64/arm64
How Found: From running test cases
Regression From: Previous release (1.32.1)
Steps to Reproduce

Expand one storage account -> Blob Containers.
Create a blob container -> Upload one blob.
Right click the column header -> Click 'Customize Columns...'.
Hide the 'Status' column -> Click 'OK'.
Check whether the 'Status' column is not visible in the editor.

Expected Experience
The 'Status' column is not visible in the editor.
Actual Experience
The 'Status' column is visible in the editor.

Verified this issue on the build 20240419.1. Fixed.


[WIP] Use openssl 1.1 by default

Tracking hydra jobset: https://hydra.nixos.org/jobset/nixos/openssl-1.1
Most relevant issues:

[ ] all nodejs versions
[ ] erlang < R20
[ ] ruby < 2.4
[x] xar, breaks the world on darwin

We probably will need to create some patches ourselves..

I guess this isn't yet ready to be staged; it seems many packages can't handle the new version yet.

Yes, still wip

Perhaps the package updates are worth merging regardless of default openssl version?

Was planning to finish this in the next few days anyway so not sure if it's worth it.

Retargeting for after 17.03 as too much stuff is breaking and not easily fixable due to openssl being propagated sometimes and stuff (PHP) trying to link to both versions due to dependencies.
I'll merge the version bumps to staging.

BTW, #24574 claims to fix ntp

[Bug 3363] Support for openssl-1.1.0 without compatibility modes

rework of patch set from<EMAIL_ADDRESS><EMAIL_ADDRESS>

Still far from finishing this monster? Perhaps some stuff would easy to get to staging or master directly? (like those various updates)

Considering the amount of failures is already quite low (basically at the noise level) it would seem to me this is ready to go in staging. What do you think @globin?

I'd prefer fixing up a but more, i.e. everything depending on qt 5.6 is broken..

@globin Qt 5.6 seems to be broken on staging as well.
https://nix-cache.s3.amazonaws.com/log/8hya81bnhab2b0mps9amld1ma989hz44-qtbase-5.6.2.drv

Mostly reminder to myself: erlang < R20 needs patching

I wonder, do you still plan to have this in 17.09?

Bumped the milestone to 18.03. We're not expecting this to be ready for 17.09.

xar should be happy again. I even checked the box! 😄

I wonder how much we should really care about old Erlang and Ruby versions...

Bigger breakages I can see ATM (some might only appear with updated master):

[ ] subversion
[ ] openssh – update conflicting with the huge openssl-1.1 patch

Let me take the baton for this evening, then I'll make Hydra re-evaluate with my fixes, and we can hopefully merge soon-ish.

I now see qt5* as significant problems to solve.  They're probably not very good cases to override to different openssl version, due to being a frequent dependency. (I expect collisions if two versions got into one process.)

I didn't notice that the default 5.10 builds out of the box.  I patched 5.9 in d9e691822, and I expect 5.6 can be removed?  What maintainers think of the removal? @ttuegel, @qknight, @periklis.

We might be approaching a point where we could merge and leave the rest to the community – roughly 150 build regressions on x86_64-linux (almost nothing tried on darwin yet).

We might be approaching a point where we could merge and leave the rest to the community.

That's what I thought, too. The current effort is quite comprehensive already, and expecting the handful of contributors to this branch to fix all of Nixpkgs doesn't seem realistic.

@globin: do you remember how the ruby breakage manifests?  All the versions seem to build and respond to --help and --version, and I can't see any comment fixing it (except for switch to 2.4 by default).  Hidden breakages tend to be unpleasant.

The same goes for erlang (<R20?)

@vcunat Can't we throw this PR against a darwin slave to check out the regressions?

@periklis, the Hydra job does build Darwin.

Ok that's what i thought. So then @vcunat means, "nothing tried on Darwin" as "nothing tried on darwin on runtime", right?

No, Hydra's been rather overloaded on Darwin side, so it didn't get to building many of the builds.

ruby: @zimbatm, erlang: @gleber (from .github/CODEOWNERS).
tl;dr: older ruby/erlang versions possibly have problems with openssl-1.1.0.  What do you think?

Why not just get rid of Ruby 2.3? 2.4 and 2.5 have been out for a bit...

@NeQuissimus: see https://github.com/NixOS/nixpkgs/pull/34927

Please do not merge this right now, I want to go through this again first and maybe clean up stuff.

Wrt to ruby at least, the older versions don't build the ssl extension as it "cannot find openssl" which doesn't break the build but the world of ruby libraries trying to import ssl.

this PR is now conflicting, probably because of the ruby changes introduced in #34927

Ruby changes in; Hydra's working on updated binaries, so we can test more easily.  @zimbatm: ruby 2.3 is kept, so the preferred way is to make it use older openssl?  I suppose "we" don't like the current state in this branch: succeeding build with no openssl support.

yes, are we keeping a older openssl around? setting the version for ruby 2.3 would be great.

Of course we are.  A dozen packages keep pinned to 1.0.2.

We have to be very(!) careful with pinning stuff to 1.0.2!! If something is linked to both versions it may fail at runtime, if possible nearly everything should be 1.1. That is one reason why I'm so reluctant in rushing this. This should probably not go into 18.03

For c-like stuff, if you have both in link-closure, the build will fail during linking, as there are symbols resolvable to both versions.

This should probably not go into 18.03.

Actually, this PR should be merged as quickly as possible so that any remaining issues can be discovered and fixed utilizing more eyes and hands that just the 2-3 people who are working on this branch.

I'm certain that if it should get to 18.03, it needs to get to master soon (several days at most).
A similar thing holds for gcc-7, as we branch 18.03 in about two weeks, though there I'm confident much more is catchable during build-time.  With openssl the "dynamic" languages might give more trouble.

Erlang <R20 is failing due to a configure error:
checking for OpenSSL kerberos 5 support... yes
checking for krb5.h in standard locations... not found
configure: WARNING: OpenSSL is configured for kerberos but no krb5.h found

Which disables the Erlang stdlib crypto library, which almost everyone uses:
crypto         : OpenSSL is configured for kerberos but no krb5.h found

The best summary I've found about Erlang and OpenSSL 1.1 is here:
https://www.bountysource.com/issues/48736281-erlang-doesn-t-build-correctly-for-certain-versions-of-openssl-1-1-x
TL;DR: Erlang <R20 does not and probably will not support OpenSSL 1.1. So either we drop older Erlangs or we pin them to previous OpenSSL version.

So ... what do we do about this PR now? Plenty of merge conflicts have popped up over the last 2 months of inactivity and I wonder how to move this effort forward.

It seems like a lot of the changes could be comfortably put into master without bumping the default OpenSSL version. Like adding pkgconfig to svn, making packages that rely strongly on OpenSSL 1.0.2 specify explicitly that in their dependencies, etc.
So perhaps easiest to split it up and put the "no regrets" changes directly into master, and then what's left is the core of changes that rely on OpenSSL 1.1 being the default?

@matthewbauer, huh? Why did you do that?

Sorry unintentional git issue.

Archlinux did this ages ago. I am confident we can do it for 19.03!

I went through all the commits and rebased (or removed) them for the current staging state.
My current version can be found at https://github.com/andir/nixpkgs/tree/openssl-1.1
IIRC there used to be a jobset on hydra. Did I miss it?

IIRC there used to be a jobset on hydra. Did I miss it?

See https://github.com/NixOS/nixpkgs/pull/22357#issue-104256005. :smile: It's at https://hydra.nixos.org/jobset/nixos/openssl-1.1.

@andir, I re-set this PR based on your new version. This means https://hydra.nixos.org/jobset/nixos/openssl-1.1 will be building it now.

@peti thanks ;)

The eval has been pending for about two hours. That is probably the case because that job is disabled? (And probably why I didn't find it in the first place, I guess).

@peti are you able to (re-)enable jobs?

"pending" means that it's queued for evaluation.  Some of our jobsets are rather expensive to evaluate (dozens of minutes).  Now it's building already.

I re-enabled the Hydra job (and it's completed an evaluation already, too).

Rebased on current staging & add an updated version of the qt59 patch

Rebased on current staging, added a few more patches and fixed the qt48 commit that I broke before.

Rebased one more time. Was able to remove a few of the version pinnings.
I started another hydra eval to see if things have improved. Last time I worked on this it was pretty close to be mergeable IMHO.

Moved the jobset to the nixpkgs project in order to be able to compare it to staging:
https://hydra.nixos.org/jobset/nixpkgs/openssl-1.1

Closing this one as it's completely out of date.

@domenkozar, @globin seems to have rebased and started working on this again 10 days ago, hence the commit spam, so I don't think it should be closed. But @globin is probably the best person to know for sure :)

I also don't think this should be closed.
AFAIK we don't have another PR of this kind and this change is very important if we want to avoid having an end-of-life version in 19.09:
https://www.openssl.org/source/:

Note: The latest stable version is the 1.1.1 series. This is also our Long Term Support (LTS) version, supported until 11th September 2023. Our previous LTS version (1.0.2 series) will continue to be supported until 31st December 2019 (security fixes only during the last year of support). The 1.1.0 series is currently only receiving security fixes and will go out of support on 11th September 2019. All users of 1.0.2 and 1.1.0 are encouraged to upgrade to 1.1.1 as soon as possible. The 0.9.8, 1.0.0 and 1.0.1 versions are now out of support and should not be used.

Also: I though the target version of this PR was changed from 1.1.0 to 1.1.1 (or at least that there was a statement in some comment, but I didn't find it anymore :o), but looking at the current version it still seems to be based on 1.1.0 (but I might be wrong here, I only had a quick look with the web interface and this PR is too big for that).

This pull request has been mentioned on Nix community. There might be relevant details there:
https://discourse.nixos.org/t/nixos-19-09-feature-freeze/3707/1

@globin would it be possible to give us a quick status update regarding this PR (also regarding my previous comment: https://github.com/NixOS/nixpkgs/pull/22357#issuecomment-519683341)?
My current interpretation:

If the milestone change wasn't done by accident I'm assuming that this PR won't land in 19.09 anymore -> we'll have to maintain an end-of-life version without official security updates.
The Discourse post and this PR still refer to OpenSSL 1.1 but this isn't an LTS version -> would it be possible to switch to 1.1.1 instead or would that require significantly more effort?

Is this PR still realistic (it is open for ~2.5 years now)? Could we possibly divide it into smaller PRs (e.g. updating the most essential/important packages first)? Would it help if this PR has more visibility (e.g. if we pin it on GitHub, mention it on Discourse, etc. -> maybe the community could help with testing and updating packages)? Or could some other approach work better (e.g. updating the OpenSSL version and pinging all maintainers of affected packages and packages that depend on an older OpenSSL version -> distributing this PR, e.g. like it was done here: https://github.com/NixOS/nixpkgs/pull/56265)?
In any case: Thanks for all of your effort, I really appreciate it!
(I hope this comment doesn't give the wrong impression. I'm glad that we have this PR, but I'm also a bit concerned as we're behind most other Linux distributions and this PR was now delayed multiple times while we desperately need it.)

I wasn't aware they were EOL'ing at the end of this year, I'll fix up the remaining larger issues (IIRC only Erlang) and then merge, then the rest will be fixed with ZHF. I'll also try and cherry-pick the smaller parts to other PRs to get parts in faster.
This is updating to openssl 1.1.1 anyway as openssl 1.1 changed the versioning and 1.1.1 is at least API-compatible to 1.1.0.
Also I'll base this on master as it has its own hydra jobset. (I'm also continuously coordinating with @lheckemann for 19.09)

This is good enough to be merged to master, some minor failures in leaf packages still exist, but they can either be fixed in ZHF or marked as broken if nobody cares

I'll merge this to master and that back to staging/staging-next at around midday GMT if no one beats me to it.

It's in staging-next as well, since 2e6bf42a2.

#67375 for staging (WIP)

This causes a segfault in rtmpsink in gst-plugins-bad, because rtmpdump depends on 1.0 while gst-plugins-bad depends on 1.1. See #67842 for a fix.


Support usage as github action

closes #4

Something similar can be achieved using release-please and release-please-action with github actions
CC @pat-s, I think that was the tool I saw years ago


How to calculate batch_max_steps

Thanks for your effort,
I need to know how i can calculate batch_max_steps for my dataset if the max audio length 6 secs and sampling rate 22050 ?

I have never investigated the effect of batch_max_steps but basically we use around 1 sec for PWG (In the case of 22050 Hz, that is 22050 points, but it must be dividable by hop_size. So the config use 25600 points as default in the case of hop_size=256).


Added serious.py

serious.py contains a main function. It prints a serious string to stdout and returns 0.

Need serious comments in the main function.


[osx_64] libcxx v18.1.8: Failure with ILLEGAL instruction at runtime on dependent project

First observed in ArcticDB, here. Logs for posterity:
The following tests FAILED:
	  1 - Async.SinkBasic (ILLEGAL)
	  2 - Async.DeDupTest (ILLEGAL)
	 23 - Segment.RoundtripTimeseriesDescriptorWriteToBufferV2 (ILLEGAL)
	 27 - SegmentHeader.SerializeUnserializeV1 (ILLEGAL)
	323 - SparseTestStore.Compact (ILLEGAL)
	324 - SparseTestStore.CompactWithStrings (ILLEGAL)
	388 - VersionStoreTest.CompactIncompleteDynamicSchema (ILLEGAL)
	408 - TestEmbedded/SimpleTestSuite.Example/lmdb  # GetParam() = 24-byte object <08-6C 6D-64 62-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00 00-00> (ILLEGAL)
	412 - AllStoragesCommonTests/GenericStorageTest.WriteDuplicateKeyException/(ptr = 0x6000023e8018, value = 8-byte object <90-1F 99-0A 01-00 00-00>) (ILLEGAL)

Also observed on the feedstock with https://github.com/conda-forge/arcticdb-feedstock/pull/249.
Currently solved by using libcxx<18 (see https://github.com/man-group/ArcticDB/pull/1680).
We need to come up with a reproducer for it.

Thanks for the issue. Gah, libcxx 18 is a handful... 😑

We need to come up with a reproducer for it.

If it's a small reproducer we can put it in a test here, if it's larger it would be great if it could be run in the test: section of a feedstock, because then we could add it as a downstreams: here (which doesn't rebuild the feedstock, it just runs the test suite of that other feedstock while installing the local libcxx).

libcxx-18.1.8-hef8daea_0.conda is the problematic build. Should we declare it as broken for now?

libcxx-18.1.8-hef8daea_0.conda is the problematic build. Should we declare it as broken for now?

The package was already downloaded a couple thousand times, and probably shows up in various constraints already. Given that arcticdb is so far (🤞) the only project where problems occurred, I'd prefer not to mark things as broken, unless the blast radius turns out to be bigger than a handful of feedstocks.

So I tried the hypothesis that this had something to do with not shipping an up-to-date libunwind in https://github.com/conda-forge/arcticdb-feedstock/pull/251, but the failure is the same both with the vanilla 18.1.8, as well as the modified version from #163. So from that angle, it looks like the failure comes from somewhere else.
Both fail with:
python/tests/integration/arcticdb/test_arctic.py::test_write_metadata_with_none[real_s3-0] SKIPPED [  1%]
python/tests/integration/arcticdb/test_arctic.py::test_write_metadata_with_none[real_s3-1] SKIPPED [  1%]
Fatal Python error: Illegal instruction

Current thread 0x0000000118f3a600 (most recent call first):
  File "$PREFIX/lib/python3.11/site-packages/arcticdb/version_store/_store.py", line 1926 in compact_incomplete
  File "$PREFIX/lib/python3.11/site-packages/arcticdb/version_store/library.py", line 925 in finalize_staged_data
  File "/Users/runner/miniforge3/conda-bld/arcticdb_1720947707758/test_tmp/python/tests/integration/arcticdb/test_arctic.py", line 229 in test_staged_data
  File "$PREFIX/lib/python3.11/site-packages/_pytest/python.py", line 162 in pytest_pyfunc_call
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "$PREFIX/lib/python3.11/site-packages/_pytest/python.py", line 1632 in runtest
  File "$PREFIX/lib/python3.11/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "$PREFIX/lib/python3.11/site-packages/_pytest/runner.py", line 241 in <lambda>
  File "$PREFIX/lib/python3.11/site-packages/_pytest/runner.py", line 341 in from_call
  File "$PREFIX/lib/python3.11/site-packages/_pytest/runner.py", line 240 in call_and_report
  File "$PREFIX/lib/python3.11/site-packages/_pytest/runner.py", line 135 in runtestprotocol
  File "$PREFIX/lib/python3.11/site-packages/pytest_rerunfailures.py", line 549 in pytest_runtest_protocol
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "$PREFIX/lib/python3.11/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "$PREFIX/lib/python3.11/site-packages/_pytest/main.py", line 339 in _main
  File "$PREFIX/lib/python3.11/site-packages/_pytest/main.py", line 285 in wrap_session
  File "$PREFIX/lib/python3.11/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "$PREFIX/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "$PREFIX/lib/python3.11/site-packages/_pytest/config/__init__.py", line 178 in main
  File "$PREFIX/lib/python3.11/site-packages/_pytest/config/__init__.py", line 206 in console_main
  File "$PREFIX/bin/pytest", line 10 in <module>

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, _brotli, zstandard.backend_c, google.protobuf.pyext._message, msgpack._cmsgpack, yaml._yaml, bson._cbson, pymongo._cmessage, _cffi_backend (total: 61)
/Users/runner/miniforge3/conda-bld/arcticdb_1720947707758/test_tmp/run_test.sh: line 9: 14459 Illegal instruction: 4  ARCTICDB_RAND_SEED=$RANDOM pytest python/tests --verbose --reruns 5 --reruns-delay 1 -k "not Azure and not azurite and not test_lmdb_warnings_when_reopened and not test_force_compact_symbol_list_lock_held_past_ttl and not test_write_non_timestamp_index and not test_write_not_sorted_multi_index_exception and not test_map_size_bad_input and not test_map_size_parsing and not lmdb"
WARNING: Tests failed for arcticdb-4.5.0rc1-py311hbadd9f8_1.conda - moving package to /Users/runner/miniforge3/conda-bld/broken
TESTS FAILED: arcticdb-4.5.0rc1-py311hbadd9f8_1.conda

We also ran into this unexpected Illegal instruction error  for our osx-64 build in https://github.com/TileDB-Inc/tiledb-vcf-feedstock/pull/125#issuecomment-2226735449

Can someone try to attach a debugger and get a better stacktrace than just "illegal instruction"?
In the meantime, I'll try to build a >=MacOS 12.0 version for the libcxx_dev label, to see if the deployment target plays a role somehow.

Also no dice in https://github.com/conda-forge/arcticdb-feedstock/pull/251 with the builds from #164.
@jjerphan @jdblischak, could you check the list of ABI changes in libcxx 18 to see if anything stands out at you?
The first bullet point related to exceptions shouldn't affect arcticdb or our builds here (AFAICT from a code search), but I'm now starting to wonder about one of the oldest patches we're carrying, as it seems to do something related to this.

Does the error happen when libcxx is updated in an existing environment or is it only when arcticdb is built with libcxx=18

Both IIRC.

Does the error happen when libcxx is updated in an existing environment or is it only when arcticdb is built with libcxx=18 ?

I tested this hypothesis in https://github.com/conda-forge/arcticdb-feedstock/pull/251/commits/3b8d221fd27a82c2429d0021e16e30afcd1db384 and the problem only arises when building against libcxx 18. If we build with libcxx 17 and then run with libcxx 18, things work.

Looks like arcticdb links to libc++abi.dylib which is not really necessary or supported I think. With libc++-18 and 17, the linking order is different. Might be the casue.

So, just for debugging purposes (not proposing we do this), I shipped libc++abi.dylib in #166. Makes no difference though... 🤷

To Isuru's point:
   INFO (arcticdb,lib/python3.10/site-packages/arcticdb_ext.cpython-310-darwin.so): Needed DSO /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/usr/lib/libc++abi.dylib found in $SYSROOT

(which is the only occurrence in a green build, e.g. last run on main).

From staring at arcticdb's CMakeLists.txt a bit, I cannot see anything specific for building arcticdb_ext that stands out w.r.t. osx/clang, much less libc++abi. It seems to be a pretty vanilla pybind11 build. However, that recipe is currently limited to pybind <2.11, which means versions >1 year old. Not sure if that has any bearing. 🤷

From the release notes of v2.11.0:

No longer inject -stdlib=libc++, not needed for modern Pythons (macOS 10.9+). https://github.com/pybind/pybind11/pull/4639

So this issue seems to be showing up in polars as well (found while searching for the error): https://github.com/pola-rs/polars/issues/17687
In contrast to arcticdb, I couldn't reproduce this on the feedstock though: https://github.com/conda-forge/polars-feedstock/pull/250

No, that's not related. See the comment above your comment in the upstream issue in polars

Ah OK, didn't know the lts setup for polars. Why isn't this using the new the new archspec capabilities though, instead of merely warning on incompatible CPU instructions...?
In any case, if you have an idea or even just a suspicion what might be causing the libcxx issues, I'm happy to investigate a bit, but for now I'm stumbling in the dark.

The "Illegal Instruction" might be the result of programs jumping to an inappropriate address, hence it might be something related to different convention for stack unwinding, or frame pointers.

From this section of the release notes, some changes to the build system might need to be taken care of:

LIBCXXABI_USE_LLVM_UNWINDER and COMPILER_RT_USE_LLVM_UNWINDER switched defaults from OFF to ON. This means that by default, libc++abi and compiler-rt will link against the LLVM provided libunwind library instead of the system-provided unwinding library. If you are building the LLVM runtimes with the goal of shipping them so that they can interoperate with other system-provided libraries that might be using a different unwinding library (such as libgcc_s), you should pass LIBCXXABI_USE_LLVM_UNWINDER=OFF and COMPILER_RT_USE_LLVM_UNWINDER=OFF to make sure the system-provided unwinding library is used by the LLVM runtimes.

There are also changes regarding LIBUWIND_EXECTOR and LIBCXXABI_EXECUTOR (see the first item of the section), but they should not impact us.

For reference, I also experienced a similar failure in https://github.com/robotology/idyntree/issues/1192, but at the moment I do not have access to a macos machine to test more in depth. Interestingly, in that case only 1 test of ~68 is failing with illegal instruction.

Ditto, I do not have an osx_64 machine to better understand the problem.

The same errors show up in https://github.com/conda-forge/cctools-and-ld64-feedstock/pull/68; the interesting thing is that there we don't have any involvement of libc++abi or libunwind (at least based on the logs).

@jdblischak, is there a way to test the tiledb failure in conda-forge itself? Could you open a PR to the tiledb feedstock that reproduces it? Then I could test out a couple things more easily.

Well, I was hoping libcxx 19.1.0.rc1 would perhaps fare differently, but alas no: https://github.com/conda-forge/cctools-and-ld64-feedstock/pull/72

Some hints from Fedora's distributions: the diff of build scripts for the latest distributed version of LLVM 17 and the latest revision of their version (for LLVM 18.1.8) only introduces -DLIBCXXABI_USE_LLVM_UNWINDER=OFF, as we did previously with -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF:
git diff<PHONE_NUMBER>f948004c108b5273f76e7a40753073..95d5cc4ebcee624dcde61fe31aa38eea556d2bec

diff --git a/libcxx.spec b/libcxx.spec
index ff478d1..23ad663 100644
--- a/libcxx.spec
+++ b/libcxx.spec
@@ -4,8 +4,8 @@
 # https://bugzilla.redhat.com/show_bug.cgi?id=2158587
 %undefine _include_frame_pointers

-%global maj_ver 17
-%global libcxx_version %{maj_ver}.0.6
+%global maj_ver 18
+%global libcxx_version %{maj_ver}.1.8
 #global rc_ver 4
 %global libcxx_srcdir libcxx-%{libcxx_version}%{?rc_ver:rc%{rc_ver}}.src
 %global libcxxabi_srcdir libcxxabi-%{libcxx_version}%{?rc_ver:rc%{rc_ver}}.src
@@ -13,7 +13,7 @@

 Name:		libcxx
 Version:	%{libcxx_version}%{?rc_ver:~rc%{rc_ver}}
-Release:	3%{?dist}
+Release:	2%{?dist}
 Summary:	C++ standard library targeting C++11
 License:	Apache-2.0 WITH LLVM-exception OR MIT OR NCSA
 URL:		http://libcxx.llvm.org/
@@ -133,7 +133,7 @@ mv ../%{libcxx_srcdir} libcxx
 mv ../%{libcxxabi_srcdir} libcxxabi
 mv ../%{libunwind_srcdir} libunwind
 mkdir -p runtimes/cmake/Modules
-mv %{SOURCE8} %{SOURCE9} runtimes/cmake/Modules/
+cp %{SOURCE8} %{SOURCE9} runtimes/cmake/Modules/
 %autopatch -p1

 %py3_shebang_fix libcxx/utils/
@@ -155,6 +155,7 @@ export ASMFLAGS=$CFLAGS
 	-DLIBCXX_INCLUDE_BENCHMARKS=OFF \
 	-DLIBCXX_STATICALLY_LINK_ABI_IN_STATIC_LIBRARY=ON \
 	-DLIBCXX_ENABLE_ABI_LINKER_SCRIPT=ON \
+	-DLIBCXXABI_USE_LLVM_UNWINDER=OFF \
 	-DLLVM_BUILD_DOCS=ON \
 	-DLLVM_ENABLE_SPHINX=ON \
 	-DLIBUNWIND_INCLUDE_DOCS=ON \
@@ -240,6 +241,36 @@ rm %{buildroot}%{_pkgdocdir}/html/.buildinfo
 %doc %{_pkgdocdir}/html

 %changelog
+* Thu Jul 18 2024 Fedora Release Engineering<EMAIL_ADDRESS>- 18.1.8-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_41_Mass_Rebuild
+
+* Thu Jul 11 2024 Jesus Checa Hidalgo<EMAIL_ADDRESS>- 18.1.8-1
+- 18.1.8 Release
+
+* Thu Jun 13 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.7-1
+- 18.1.7 Release
+
+* Tue May 21 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.6-1
+- 18.1.6 Release
+
+* Fri May 03 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.4-1
+- 18.1.4 Release
+
+* Wed Apr 17 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.3-1
+- 18.1.3 Release
+
+* Fri Mar 22 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.2-1
+- 18.1.2 Release
+
+* Wed Mar 13 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.1-1
+- 18.1.1 Release
+
+* Mon Mar 04 2024 Nikita Popov<EMAIL_ADDRESS>- 18.1.0~rc4-2
+- Disable LIBCXXABI_USE_LLVM_UNWINDER (rhbz#2267690)
+
+* Thu Feb 29 2024 Tom Stellard<EMAIL_ADDRESS>- 18.1.0~rc4-1
+- 18.1.0-rc4 Release
+
 * Thu Jan 25 2024 Fedora Release Engineering<EMAIL_ADDRESS>- 17.0.6-3
 - Rebuilt for https://fedoraproject.org/wiki/Fedora_40_Mass_Rebuild

diff --git a/sources b/sources
index d7d78a7..c581215 100644
--- a/sources
+++ b/sources
@@ -1,8 +1,8 @@
-SHA512 (libcxx-17.0.6.src.tar.xz) = 8be288ab14bd34a1946aeadc83c9e11ff68eb0cda707fd6bee711514d0e506112ffc2a40d0317d19f64f05b644e072f1322ed0e8ab238d9640b6b827d42126eb
-SHA512 (libcxx-17.0.6.src.tar.xz.sig) = 01ac59892c16b71b2fc1b2154e5f0af123ece7c1d55c85fa38ff9c7a2410ff0ced336f40b57235c59a480e756c87ea2e6324b056cc20368f86f8f3abde6e1bf4
-SHA512 (libcxxabi-17.0.6.src.tar.xz) = acd2b0b48eee4380625347a955fa3dfafce948aeccd87a2070e09bd91b148ae189aec12b506f3097193a8288095e99496c66ce26974772878622e5892d822cc3
-SHA512 (libcxxabi-17.0.6.src.tar.xz.sig) = f9abb5952bc6a95618eaf53df30dd9ce5dfef7dd11c1abadfc05e8dcd9f270e10f35ed4bee9163dfda0a7e508db8ddc75fdb48e724a88a924b2f150bb1e20fd8
-SHA512 (libunwind-17.0.6.src.tar.xz) = 4f0c1a38796022a476fab06d91c5c5ec76e060e0697941be83ee896a18e548d02605e4431ea9ac09dc36bc28cc5609fac6a200faff703eba2360dfab2dcf5300
-SHA512 (libunwind-17.0.6.src.tar.xz.sig) = 9c0990583855d826652ef1ecf3ecdc1f029592ddeacc0d776400b5fa8d044183193d43631c270857e6b48b501ec4f356d84399529fd9c317f69c3e4a9a377851
+SHA512 (libcxx-18.1.8.src.tar.xz) = b24f8adbc4edd0cde3a28c6ce0ae2d7cd32049c155459c669f30ee6400a7a0e789c968db7f93bb0aa0f972b47b86424b9655af00e99867e242baccece8f323e8
+SHA512 (libcxx-18.1.8.src.tar.xz.sig) = cdaee075a6d24c6c8cd2c5e80c68f30eadb73731d6a59767b7cb54f50bb22f7665707de013f04d5048d08edd807442063ca3154c5ca3c4ae236b90d135811af0
+SHA512 (libcxxabi-18.1.8.src.tar.xz) = 40f8691e86948527cd104b3b7f481757e6f8f4892fbe8632a6f86f35008e0c9d721e5f2d3629c6f0b99e9f150ee9f3d650aa111ea2c5f6df44fec481ff00a1f9
+SHA512 (libcxxabi-18.1.8.src.tar.xz.sig) = 5669aaaef6c3b228de56fa05d2099cfb23191242d9fc973f1a1939d9933dfb12124c0e1d08efddb34e6a546026648e8d9e84b3cb7dd641a688c39d2a9a08c320
+SHA512 (libunwind-18.1.8.src.tar.xz) = d6bf0e462db5d99bcdc1170c3789e84f21d86e35a6e79b75bea0a6f9aad222a25400944bb7fa030bf94e51cccc5e42b7b7002be98c0936f2852ac5688c0c3a84
+SHA512 (libunwind-18.1.8.src.tar.xz.sig) = c20f9e9fea812e94227e0b00d725b24f76e3cfcc7fc5360fa60b162429df847e30ebf5e9191c5e613082fec94ef8a4830f04732045acaf864f08dc6703da5954
 SHA512 (HandleFlags.cmake) = 525fe99751c68b93f28651440d299776ba0ea198989a9150f256ca640187551fc8050bbc27fc75e852fd126d4bcd8590ab5d3d7c224b3a5f7125876fc8e48eb1
 SHA512 (WarningFlags.cmake) = ad0d9eff0ce4ef69a8090061e6260a95effa2006a34099d68f0881ddea91724f9bc17133b00c9a0276a6ae35a4a305a3e164f17350e883152f452762cf8b88c5

It there something with the previous patches which is causing problem?

I am back at work and I have access to a macos machine, and I was able to reproduce the issue on both osx-64 and osx-arm64 with a (definitely non-minimal) reproducer:
˜˜˜
git clone -b debugcxx18 https://github.com/robotology/idyntree
cd idyntree
pixi run test
˜˜˜
this fails with:
˜˜˜
59/59 Test #59: IntegrationTestiCubTorqueEstimation ...........SIGTRAP***Exception:   0.00 sec
˜˜˜

bt is:
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BREAKPOINT (code=1, subcode=0x1002a9d78)
  * frame #0: 0x00000001002a9d78 libidyntree-estimation.dylib`iDynTree::estimateLinkContactWrenchesFromLinkNetExternalWrenches(model=0x000000016fdfd728, unknownWrenches=0x000000016fdfd370, netExtWrenches=0x000000016fdfd060, outputContactWrenches=0x000000016fdfd250) at ExternalWrenchesEstimation.cpp:0 [opt]
    frame #1: 0x0000000100005cfc iCubTorqueEstimationIntegrationTest`extractJointTorquesAndContactForces(berdyHelper=0x000000016fdfd728, jointPos=<unavailable>, estimatedDynamicVariables=0x000000016fdfd5c0, unknownWrenches=0x000000016fdfd370, estimatedJointTorques=<unavailable>, contactWrenches=0x000000016fdfd250) at iCubTorqueEstimationIntegrationTest.cpp:39:5 [opt]
    frame #2: 0x0000000100007380 iCubTorqueEstimationIntegrationTest`compareEstimators(urdfFileName=Summary Unavailable, contactFrames=<unavailable>, imuFrameName=Summary Unavailable, compareEstimatorsOptions=0x000000016fdfdfc0) at iCubTorqueEstimationIntegrationTest.cpp:318:5 [opt]
    frame #3: 0x0000000100009680 iCubTorqueEstimationIntegrationTest`main at iCubTorqueEstimationIntegrationTest.cpp:381:5 [opt]
    frame #4: 0x000000010002108c dyld`start + 520

but I can't find anything useful in it.

Well, I was hopeful after having found https://github.com/Homebrew/homebrew-core/issues/178435 and getting the tip there to pass in #172 without having to do the install-name-tool redirects. However, it still fails with https://github.com/conda-forge/cctools-and-ld64-feedstock/pull/72 😭

is there a way to test the tiledb failure in conda-forge itself? Could you open a PR to the tiledb feedstock that reproduces it? Then I could test out a couple things more easily.

@h-vetinari I'm sorry. It's not clear what you are asking me to do. We have never observed this PR in tiledb-feedstock, so I don't know how to reproduce it there.
We observed the Illegal instruction in TileDB-Inc/tiledb-vcf-feedstock. While it isn't in conda-forge, it is a feedstock created by conda-smithy. Thus you should be able to fork the repository to your account and test it just like any conda-forge feedstock. You can even start from the branch on my fork that I used to reproduce this issue in https://github.com/TileDB-Inc/tiledb-vcf-feedstock/pull/131
Unfortunately I'm leaving for vacation today, so I won't be able to help again until next week

Interesting update on my side. It turns out that in the case of https://github.com/robotology/idyntree/issues/1192 the crash induced by the update to libcxx 18 was actually a problem on my code (fixed in https://github.com/robotology/idyntree/pull/1198), not in libcxx==18.*.
The problematic code was something like:
structFoo
{
    int a;
    int b;
}

std::vector<Foo> vecFoo;
vecFoo.resize(0);

struct Bar
{
    int b;
    int c;
}

std::vector<Bar> vecBar;
vecBar.resize(0);

vecBar[0].b = vecFoo[0].b;

It is clearly an undefined behavior code, but apparently it always worked fine on Linux, macOS and Windows, even under valgrind. I do not know if some kind of sanitizer would have detected this.

OK, this looks to have been the due to the hardening mode we enabled for 18.x, and therefore my fault. I'm very for the troubles this has caused, and grateful for everyone's help in tracking this down (especially the relentless @jjerphan 🙏). For context, this had been discussed in #136, and the hardening documentation during the 18.x cycle did not explain what's there now, namely that the default assertion handler is just __builtin_verbose_trap.
The takeaway for this is that the hardening modes need to be off for production, but it would definitely make sense to provide a LIBCXX_HARDENING_MODE=debug build in a separate label, so that people can opt into testing their builds with hardened libcxx (to find the UB that seems to be present in the affected projects), but avoid this for publishing.

Thank you for all the work on distributing compilers and their implementation of the standard libraries. Now we know the existence of libc++'s hardening mode, and a few projects now know that they can check for UBs and correct them, which is highly beneficial IMO.


Return 410 for services on expired frameworks

When the G-Cloud 5 framework expires, we should return 410 ("Gone") rather than 404s for G5 service pages.
See: https://www.pivotaltracker.com/story/show/107382498
This is related to, but not dependent on: https://github.com/alphagov/digitalmarketplace-buyer-frontend/pull/185

LGTM :+1:

I don't think we should be doing this here. We allow listing services with any framework and status, there's no reason to remove the API ability to return a service based on the framework status.
RIght now buyer frontend checks the service status and returns 404 if it's not published (https://github.com/alphagov/digitalmarketplace-buyer-frontend/blob/master/app/main/views.py#L164), I think we need to either add the framework status check there, or move that logic to the dmutils get_framework method since it will probably be used by supplier frontend too.

This doesn't remove any capability - this endpoint only ever returned live services and 404'd for everything else.  All this change does is change the error code for expired services.

@TheDoubleK you're right, I've missed the .framework_is_live bit. I still don't agree with keeping this in the API – we allow listing expired services, but then don't return individual resources.

Agreed - it doesn't seem right behaviour for the API, but pushing the check to buyer & supplier apps (probably via utils) will be a fair bit more work and doesn't feel like a priority right now... could we add that as a story in the icebox?

@TheDoubleK yeah, let's do that. You're right that we're not changing behaviour here, so there's no reason not to merge this as is. :+1:

I added a story here: https://www.pivotaltracker.com/story/show/107478318


feat(hmac) multiple HMAC algorithms support and enforce_headers option

SUMMARY

Support for HMAC-SHA256, HMAC-SHA384, HMAC-SHA512.
User can enforce which headers must at least be used for
http signature creation using config.enforce_headers.

Full changelog

config.alogorithms and config.enforce_headers added to schema
Logic updated to support multiple algorithms and enforce headers
Tests cases added to test new logic

@p0pr0ck5 makes sense, thanks. Updated code as you advise.

@thibaultcha update PR as you suggested.


LoggableUrl Leaks partial passwords

The Regex for loggableUrl can leak passwords if the password has a @ in it.
Here:  https://github.com/guidesmiths/rascal/blob/2dbe72abce8386eee8762e421ed4b49fbee0c5fc/lib/config/configure.js#L88
This regex matches the first @ found, when it needs to match the last @ found.
For example if the password was "badp@ssword", the log would log: http://user:***@ssword@url instead of the expected http://user:***@url

Thanks @matt1097, will resolve

Fixed in<EMAIL_ADDRESS>

Something strange with the tag. Double checking the fix was published

Yep, all good. Thanks again for reporting


Paused game, seems not to pause audio

Wondering if an interesting difference might be ambient sound, versus state-specific sound, but when I pause, this thing keeps making noise. Not sure if mute is an idea, or a config for mute on pause could or should be added to the engine.
I sort-of hacked something naughty together, which I am sure is wrong
[...document.querySelectorAll('audio')].forEach(async (elem) => await elem.pause())
It seems unpausing automatically schedules all the sounds I would want, so I don't know if this is too blunt an instrument but it seems to work for me.

Nice catch. This is completely my bad.
When Pause is clicked, the code pauses the music but lets sounds run to completion because sounds are generally short and do not loop.
Where did you notice this issue?
Was it Infinite Bunner or Substitute Soccer?
I aim to be extremely faithful to the original Python code in JavaScript ports down to the infinitely looping sounds.

Infinite Bunner and Substitute soccer both did this and if it wasn't a loop, it was a very long soundtrack.
I was under the impression the pause button, was Pause game, not enter pause game-state. It's not that I think all games should be silent if it's a pause menu (gamestate). Perhaps more that the audio from one game-state should not bleed into another.
I'd also not attempted to stay true to pygame implementation with this comment.

If those are the culprits, then I think I fixed it.


Questions about the loss

After consulting you last time, I did some attempts, first I resampled the data, changed the image size to (80,96,112) , and used them for training, loss was normal at the beginning of the training, but at 20,000,30000 times the last three items are all zero. I have also tried to adjust λ1,λ2 and λ3 to be (1000,3,0.1or 100,3,0.001) , but the results was not good either, and even more than 50,000 times the loss becomes 20. Later I tried to use normalized data instead of normalization in the code, but the results were the same. In addition, I also tried using the image_A and image_B you provided, but loss4 is also most of the time 0.Some of the data I sent to you by email.

Another problem is that you are given in the paper the Lmag and Lreg using L2 norm,but Lmag using the L1 norm in your code?

Hi @1164094277,

After consulting you last time, I did some attempts, first I resampled the data, changed the image size to (80,96,112) , and used them for training, loss was normal at the beginning of the training, but at 20,000,30000 times the last three items are all zero. I have also tried to adjust λ1,λ2 and λ3 to be (1000,3,0.1or 100,3,0.001) , but the results was not good either, and even more than 50,000 times the loss becomes 20. Later I tried to use normalized data instead of normalization in the code, but the results were the same. In addition, I also tried using the image_A and image_B you provided, but loss4 is also most of the time 0. Some of the data I sent to you by email.

This is weird. I have managed to train on your data last time. Could you please zip the exact code you are using (with built-in normalization enabled and without the dataset) and send it to my email (cwmokab 'at' connect.ust.hk)? Also, could you specify which files you are included in training (e.g.: orig.nii.gz/ norm.nii.gz/ aligned_orig.nii.gz/ aligned_norm.nii.gz)? Let's break it down into details so that we could figure out where the problem is.

Another problem is that you are given in the paper the Lmag and Lreg using L2 norm,but Lmag using the L1 norm in your code?

You are correct. I have updated the manuscript on arXiv. L_mag should be using L1 norm. Thanks for pointing out the error.

Hi @1164094277 ,
I think you might fall into a common pitfall, which makes the training failed. When you downsampled the data, you must guarantee the background intensity of your input scan is 0 before and after the downsampling. One possible situation is that img_A has background intensity equals to zero before downsampling. However, if you downsample img_A with spline interpolation, e.g.: scipy.ndimage.zoom({{zoom_factor}}, order = default/3), the background intensity will be not equal to zero anymore.
Here is an example of img_A = zoom(img_A, (1, 0.5, 0.5, 0.5)):

The background intensity is shown on the top-right corner, which is equal to -1.21e-17
Here is a correct example of img_A = zoom(img_A, (1, 0.5, 0.5, 0.5), order=0):

Again, the background intensity is shown on the top-right corner, which is equal to 0.
Please let me know whether it solves your problem. If the problem persists, feel free to zip the problematic code and send it to my email. No problem can withstand the assault of sustained thinking.

@cwmok First I unzip aligned_norm.nii.gz in each folder,then using numpy to sample and normalize them. When I found the loss would be 20, I paused the training and checked the data,  I found the data that you provided were in float64 format, so I changed my data format to float64 ( original data type is float32),  loss wouldn't be 20,  but the last three

items is still occur 0.
1.zip

You can download 1. Zip View of my data, preprocessed code, and loss.py  of training in both data formats

@cwmok  First I unzip aligned_norm.nii.gz in each folder,then using numpy to sample and normalize them. When I found the loss would be 20, I paused the training and checked the data, I found the data that you provided were in float64 format, so I changed my data format to float64 ( original data type is float32), loss wouldn't be 20, but the last three items is still occur 0.
You can download 1. Zip View of my data, preprocessed code, and loss.py of training in both data formats.
1.zip

@cwmok I have just seen your latest reply and I will try to resample my data in your way.

@1164094277 I don't see any problem in your preprocessing. To make it stright, here is the SYMNet code
SYMNet_OASIS.zip tailor made for the preprocessed OASIS data from https://github.com/adalca/medical-datasets/blob/master/neurite-oasis.md.
Follow this step and you can reproduce the toy experiment ran in my computer.

Unzip SYMNet_OASIS.zip and download a copy of OASIS dataset from https://github.com/adalca/medical-datasets/blob/master/neurite-oasis.md (and unzip it as well).
Go into SYMNet_OASIS/Data and create a folder "OASIS_Adalca"
Put the unziped dataset (neurite-oasis.v.1.0) inside "OASIS_Adalca"
run the training script. python Train_sym_onepass.py

You will observe that the total loss is dropping toward to -1, and -smo will not equal to 0.

Hopefully, you can get the same training results as mine.

@cwmok My problem is occur after training 20000,30000 times ,and the loss is normal when training hundreds，thousands times. I will try according to your method. Thank you very much for your help!

Hi @1164094277,
Here is the screenshot at the end of the training, using the exact code I give it to you. You can see that the model will not collapse even in 160000 iterations.

@cwmok  Thank you very much. I've already started training the model , the loss looks better than before.  Thank you again for your help!

I remember another question about the Jacobi determinant, and there are codes that have+1, and I want to know which one is right？

This is the code for the paper 《FAIM – A ConvNet Method for Unsupervised 3D Medical Image Registration》（https://github.com/dykuang/Medical-image-registration，loss.py）
I also see a reason for +1（https://itk.org/Doxygen/html/classitk_1_1DisplacementFieldJacobianDeterminantFilter.html）

@1164094277
In short, both implementations are correct. It is all about the addition (or not) of the identity transformation.
In our implementation, you can see that we added the identity transformation (J = y_pred + sample_grid) to the output displacement before calculating the Jacobian determinant. While in the +1 version, instead of adding the identity transformation to the output, they add +1 to the diagonal elements of the Jacobian, which is equivalent.

@cwmok I see, thank you.


Projection Schema could provide stronger validation

Several parts of the projection schema a mutually exclusive. For example, you can either have type=computed or a type=linked. Currently, the schema allows for linked types with computed attributes. It would be nice if the schema provided stronger validation for this case. There might be additional opportunities for stronger validation.

Related to an old issue, https://github.com/bluesky/event-model/issues/76. Validation could be stricter across the board by leveraging jsonschema to a fuller extent and further by enforcing anything that cannot be expressed in jsonschema in Python code in the document composition function.

We have been using https://pydantic-docs.helpmanual.io/ for this. You can express your json schema in terms of Python dataclass-style classes, which can be exported to a json schema, and add custom validators in Python for the more difficult cases that will be picked up when you instantiate the objects.

Thanks, @thomascobb. That seems like a good direction to go in here.


Ghost Editor 0.1.6

no issue

adds Ghost-Editor 0.1.6

Improves markdown parsing speed
Shows toolbar on mouseup now (still freezes on loading unfortunately)
Makes Ghost-Editor full height for easy selection

temporarily pull from a branch that fixes dependency issues
remove unused module-for-acceptance.js file

Coverage remained the same at 81.167% when pulling 281d62915906464b44191d8f7977a21a32108430 on kevinansfield:deps-editor-0-1-6 into d9e54c7c83680c39b202d45dfc0abb43ec9d95f0 on TryGhost:master.

Coverage remained the same at 75.578% when pulling 82654b7350d4c70228cb7da32f549a088c05e411 on kevinansfield:deps-editor-0-1-6 into eaf16ca2a74824a0485288de971d95f0996102bf on TryGhost:master.


@ashfurrow => Add a validation hook for OSS keys

Checks if keys are not wrong.

Cool! Do we want to do this on before_all, or just before a deploy?

ah yeah, you use it for more, I'll leave that to you :P

Classic Orta.

https://en.wikipedia.org/wiki/Delegation_pattern


ARROW-14505: [CI][Docs] Exercise documentation builds more frequently

https://github.com/apache/arrow/runs/4355980806?check_suite_focus=true was filed as ARROW-14896.
@jonkeane @ianmcook Could you take a look at this?

I've got some cleanups / skips related to DuckDB in #11795. I'm ok merging this + #11795 separately, and if the docs builds still fail after that, I'll skip those examples until we can stabilize that integration (there are a few other tickets, and DuckDB is a bit of a moving target cause we sometimes need to wait for their upstream changes before we can test them reliably)

OK. I closed ARROW-14896.


ToString using string buffer instead concatenation

Hello,
Would a String Buffer, String Stream, Binary Stream... be used instead String Concatenation when invoking ToString?
Delphi has TStringBuilder Class.
Best regards.

Hi,
Thanks for your suggestion.
Could you show a simple example simulating McJSON as if it had this feature?
Also, does the current version of Tostring() have any performance issues?
Regards,
Marcelo.

Hi,
For a large JSON Document, using string concatenation would have impact on performance because of string memory allocation.
String Buffering would be implemented as:
procedure TMcJsonItem.InternalToString(Stream:TStringStream;aHuman,aIndent:string;JsonItem:TMcJsonItem);
begin
  ...
  for i := 0 to len do
  begin
    Stream.WriteString(JsonItem...);
    ...
  end;
  ...
end;

function TMcJsonItem.Stringify(aHuman:Boolean=False;const aIndent:string=''):string;
var
  Stream : TStringStream;
begin
  Stream := TStringStream.Create;
  InternalToString(Stream,aHuman,aIndent,Self);
  Result := Stream.DataString;
  Stream.Free;
end;

Best regards.

Hi Hafedh,
Nice suggestion!
I'll try it, just let me finish an update about better Exception handling and descriptions.
Thanks,
Marcelo.

Hi Hafedh,
The last commit uses TStringStream as buffer with ToString().
Thanks,
Marcelo.


Rewrote readme.md

-Added python badge.
-Added star,fork,pull requests badge.
-Categorized the titles.
-Added detailed installation commands.

Thank you! I'll still leave the issue open for possible future modifications :)


Please help with this issue. path.join is not a function

I have the same issue ☹

I ran into the same error! :(

Hi everyone.
U can't call packages in the browser. Use them in node js instead.
I use Sveltekit and put them to a page server ts which uses node js in the background.
import Kuroshiro from 'kuroshiro';
import KuromojiAnalyzer from 'kuroshiro-analyzer-kuromoji';

const kuroshiro = new Kuroshiro();

export const load = async () => {
	await kuroshiro.init(new KuromojiAnalyzer());
	const result = await kuroshiro.convert(
		'感じ取れたら手を繋ごう、重なるのは人生のライン and レミリア最高！',
		{ to: 'hiragana', mode: 'furigana' }
	);

	return { result };
};


fix

What type of PR is this?

[ ] API-change
[ ] BUG
[ ] Improvement
[ ] Documentation
[ ] Feature
[ ] Test and CI
[ ] Code Refactoring

Which issue(s) this PR fixes:
issue #
What this PR does / why we need it:

https://github.com/matrixorigin/ci-test/actions/runs/10349205377

https://github.com/matrixorigin/mo-auto-test/actions/runs/10349653577

https://github.com/matrixorigin/mo-auto-test/actions/runs/10349693964


Support v3 declare (and others)

This change is

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (349fef9) 69.16% compared to head (f5c562a) 69.44%.
Report is 5 commits behind head on main.

Files
Patch %
Lines

crates/blockifier/src/test_utils/declare.rs
97.29%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1200      +/-   ##
==========================================
+ Coverage   69.16%   69.44%   +0.28%
==========================================
  Files          58       58
  Lines        7131     7161      +30
  Branches     7131     7161      +30
==========================================
+ Hits         4932     4973      +41
+ Misses       1784     1773      -11
  Partials      415      415

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


encodeToByteArray() non deterministic?

While ser/de works, the ByteArray returned by encodeToByteArray seems no deterministic.
For example:
@Serializable
data class TaskOptions(
    val runningTimeout: Float? = null
)

fun main() {
    val m = TaskOptions(2F)
    val b1 = Avro.default.encodeToByteArray(TaskOptions.serializer(), m)
    val b2 = Avro.default.encodeToByteArray(TaskOptions.serializer(), m)
    println(b1.contentEquals(b2))
}

will print "false".
Besides being strange, it's an issue for some unit tests

Hey @geomagilles,
the default encoding used by encodeToByteArray is AvroEncodeFormat.Data() which encodes the schema and the written data into the byte array as a so-called Object Container Files.
The structure of a Object Container File contains a random sync marker that can be used to easily split byte streams at these markers in oder to be able to have fast parallel processing. Thus the output from encodeToByteArray will contain random data that differs from call to call.
One could make the random sync marker a property of the AvroEncodeFormat.Data class that can be set if it should be more deterministic. Do you have a use case for this?

Thx for your reply @thake - I've stumbled upon this while writing some unit tests. It was more a surprise to me than a real issue as it was not that difficult to find a workaround. I've raised the issue in case this was the symptom of an underlying problem. If it's not the case, and especially if it comes from the Avro java lib itself, I guess you can mark this as wontfix


Only Returning Half of Group Members

After I run the script, only analytics are provided for 3/6 of the users in my group.
There is no error, please help!

:'(

I am unable to reproduce this issue. Please feel free to reopen if you can provide more specific information.

Let me try it again as it was a long time since I tried it.


Refactor authentication plugin code to support future policy versions

This change allows us to add new policy (version) and the corresponding applier code to handle envoy configuration based on the new policy.
The code is also reorganized under a new security folder in order to make code review and testing easier.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: diemtvu
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [diemtvu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@diemtvu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-local-tests.sh
c115c65602d6e264abcc3ca9c51fa7f2c829d1bf
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@diemtvu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-local-tests.sh
c115c65602d6e264abcc3ca9c51fa7f2c829d1bf
link
/test istio-integ-local-tests

prow/istio-pilot-multicluster-e2e.sh
c115c65602d6e264abcc3ca9c51fa7f2c829d1bf
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@diemtvu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
c115c65602d6e264abcc3ca9c51fa7f2c829d1bf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-local-tests.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test istio-integ-local-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@diemtvu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-pilot-multicluster-e2e.sh
c115c65602d6e264abcc3ca9c51fa7f2c829d1bf
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-local-tests.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test istio-integ-local-tests

prow/e2e-mixer-no_auth.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test e2e-mixer-no_auth

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@diemtvu: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-local-tests.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test istio-integ-local-tests

prow/e2e-mixer-no_auth.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test e2e-mixer-no_auth

prow/istio-pilot-multicluster-e2e.sh
27d8b5feb2a40017fc4b2dc50d2df09329d9443d
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: diemtvu
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [diemtvu]
tests/integration/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: diemtvu, yangminzhu
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [diemtvu]
tests/integration/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/lgtm

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: diemtvu, lei-tang, yangminzhu
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: nmittler
If they are not already assigned, you can assign the PR to them by writing /assign @nmittler in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [diemtvu]
tests/integration/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: diemtvu, lei-tang, nmittler, yangminzhu
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pilot/OWNERS [diemtvu,nmittler]
tests/integration/OWNERS [nmittler]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@diemtvu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-unit-tests.sh
4daa5d7af8381002f1497ca3aad6e9a958ef840e
link
/test istio-unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test istio-unit-tests

@diemtvu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-simpleTests.sh
4daa5d7af8381002f1497ca3aad6e9a958ef840e
link
/test e2e-simpleTests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test e2e-simpleTests

@diemtvu: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
4daa5d7af8381002f1497ca3aad6e9a958ef840e
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Autosave when downloading profiles

Provide us a use case of the feature
How could the user invoke the new function? Which problem would it solve? If new information is obtained, how would it be further processed?
I would like to see instaloader automatically saves an iterator file much like when I manually try to close/stop the process. This is helpful to reduce the number of requests if an error happens or is suddenly closed.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Instaloader automatically saves an iterator file let's say after 50 or 100 posts and updates the progress which can help with large profiles.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Currently, this is sort of implemented. I have to ctrl + c in order to stop the download of posts to have instaloader to save an iterator file.
If the feature request is accepted, would you be willing to submit a pull request?
Yes / No

Yes with help
Additional context
Add any other context about the feature request here.

Are you asking for something different than https://github.com/instaloader/instaloader/pull/732 ?

I believe that #732 has already been implemented. This is different than #732 since it wouldn't require a manual stop to the iterating process to create the .json file.

So what I understand is you are requesting this feature so that it helps in downloading large profiles?

Yes, or while downloading multiple profiles from an args.txt.

Have you tried --latest-stamps feature?

I have read the new --latest-stamps feature but this appears to be more for updating than downloading profiles. There have been numerous times when I am unable to manually stop instaloader (appears to get hung up and refuse to save an iterator file) which results in requiring me to restart the profile download from the beginning. I am trying to reduce duplicate requests for profile post downloads.

Okay so have you tried to download any large profiles yet? And if yes, how do you do it without the feature you are requesting?

Yes, I currently have to Ctrl + C after every 50-100 posts. Which requires me to actively watch the download process. I don't know the exact number of requests Instagram will accept, I generally know after 1000 I should save and stop for another day.
But there has been times when Instaloader does get hung up and can't properly close and save an iterator file. I am unable to consistently reproduce it but I do know once I get an error about "not Instagram while downloading" I am unable to Ctrl + C.

Like described in #998 it would be very complex to proper implement the resume logic if the post on which the resumable iterator gets created differs from the one Instaloader gets aborted on, especially for features like --fast-update.
If I understand it correctly, having the feature you suggested would still require you to manually "Ctrl + C" after 1000 posts. As an alternative, you could use Instaloader as a module and adjust the rate controller to only allow 1000 requests per 24 hours, and you could also easily save the iterator as often as you like.

@Thammus Thanks! That makes more sense to do. I appreciate the response.


External links use a Unicode arrow that doesn't render properly

Describe the bug
Unicode arrow in external links does not render properly in some systems

To Reproduce
Steps to reproduce the behavior:

In the navigation bar, select the RSS feed button
The bug is in the popup that appears, in the button How do I use this?

Expected behavior
The Unicode arrow should render properly like so:

The resolution for this will be to replace the Unicode arrow with an SVG icon.


Upgrade selenium dependencies to support Windows Fall Creator Update

Address part of https://github.com/aspnet/templating/issues/131

@natemcmaster,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Does this fix it on your machine? If so that's great.

Yup, the tests on Windows are working for me now (minus the intermittent npm issues).
Btw, I noticed AppVeyor isn't enabled on this repo. @Eilon can you help us out with that?


Distributive type class?

I found myself wanting distribute for a number of transformers, including transformer stacks. It seems like this is similar to Distributive but for transformers.
Does this makes sense as a type class?
I found myself having to reach for an associated type to implement it because it seems that a bunch of constraints are needed for every transformer in the stack. Would love to know if anyone can think of a better way as it feels like an enormous and inelegant hammer.
class Distributive g where
  type Transformer
    (f :: (* -> *) -> * -> *)
    (g :: (* -> *) -> * -> *)
    (m :: * -> *) :: Constraint

  type Transformer f g m = (
        Monad m
      , Monad (f m)
      , Monad (g m)
      , Monad (f (g m))
      , MonadTrans f
      , MFunctor f
      )

  distribute :: Transformer f g m => g (f m) a -> f (g m) a

instance Distributive MaybeT where
  distribute x =
    lift . MaybeT . pure =<< hoist lift (runMaybeT x)

instance Distributive (ExceptT x) where
  distribute x =
    lift . ExceptT . pure =<< hoist lift (runExceptT x)

instance Monoid w => Distributive (WriterT w) where
  distribute x =
    lift . WriterT . pure =<< hoist lift (runWriterT x)

instance Distributive (ReaderT r) where
  distribute x =
    join . lift . ReaderT $ \r ->
      pure . hoist lift $ runReaderT x r

For larger stacks, just an example:
newtype Example m a =
  Example {
      unExample :: ExceptT String (WriterT [String] (ReaderT Int m)) a
    }

distributeExample :: Transformer t Example m => Example (t m) a -> t (Example m) a
distributeExample =
  hoist Example .
  distribute .
  hoist distribute .
  hoist (hoist distribute) .
  unExample

instance Distributive Example where
  type Transformer t Example m = (
      Transformer t (ReaderT Int) m
    , Transformer t (WriterT [String]) (ReaderT Int m)
    , Transformer t (ExceptT String) (WriterT [String] (ReaderT Int m))
    )

  distribute =
    distributeExample

I'm not sure if this is possibly related to / would help with my other question https://github.com/Gabriel439/Haskell-MMorph-Library/issues/34
I'm not tied to the names of anything here, I just wanted to have a discussion about whether this class makes sense and whether it would be useful to add to mmorph.

Is there a way to simplify the Transformer constraint?  I try to avoid constraints like Monad (f (g m)) in order to simplify inferred types.  If there were a simpler constraint that didn't need to be stored as an associated type constraint then I wouldn't mind adding this to mmorph

Is there a way to simplify the Transformer constraint? I try to avoid constraints like Monad (f (g m)) in order to simplify inferred types.

That's part of the question I suppose, I don't think it's possible, but I may be thinking about it the wrong way.
If you look at the type for distribute in pipes it has the equivalent of the Monad (f (g m)) in the form of Monad (t (Proxy a' a b' b m))):
distribute ::
     Monad m
  => MonadTrans t
  => MFunctor t
  => Monad (t m)
  => Monad (t (Proxy a' a b' b m)))
  => Proxy a' a b' b (t m) r
  -> t (Proxy a' a b' b m) r

I think you can get away without Monad (g m) for many transformers.
Also it seems like as soon as you have a transformer stack you need the associated type constraint to get all the Monad instances you need. It's like there needs to be a way to say "this is a monad and so are all the things it's on top of".


awesome work! Any chance you could add a prop for orientation?

First of all!  Your chessboard is one of the only chessboards that works problem-free on Vue 3 / Vite!
Any chance you could add a prop to see the orientation (black faces up vs white faces up)
And it also would be nice if you could make it a bit easier to modify the board colours. If not through props, then at least show an example of how to do it?
Many thanks!

It warms my heart to hear such amazing feedback!
I made this chessboard for my graduation thesis. Nowadays, I have focused on my thesis delivery so I am sorry for the late reply. After my graduation ( three or four days later), i will focus these issues.
Thank you for your feedback :).

https://github.com/ibrahimdenizz/vue-chessboard/commit/329618ecd1777c382ec4940cb764d841cbba9590 Orientation property named as blackFaceUp is added

Thanks for adding this!
I think you could have stuck to something more simple such as orientation="white" (default) or orientation="black"
Similar to how https://github.com/justinfagnani/chessboard-element implements it (the other modern chessboard implementation that I'm using which doesn't work perfectly with Vue 3 but still works with a bit of workaround.
But I appreciate you adding it even though the blackFaceUp is a bit wordy and verbose for my taste ;-)

Hi, I published a new version https://github.com/ibrahimdenizz/vue-chessboard/commit/b619ed9aea859fac714b19c34a560ff3693594cb . Thanks for the advice. I looked at the library's documentation and it gave me a lot of ideas for future properties. But in this version, I added CSS color variables and removed the scope of CSS. Now, there is an implementation of color in the documentation and I will add CSS classes to the documentation asap. Finally, You are right about the blackFaceUp property that has not been given much thought :). I updated as orientation.

That's wonderful! Thank you for making the customization of colours much easier : )
Looking forward to future updates! Cheers!


Why don't the async methods' return types implement Sync in addition to Send?

I'm using the async_trait crate to define an async method on a trait. I intend to transform the Future it returns into a Stream, wrap the Stream in an Arc and send the Arc over to multiple threads. Minimal reproducible code is as follows:
use async_trait::async_trait;
use futures::stream::{unfold, Stream};
use std::sync::Arc;
use tokio::runtime::Runtime;

#[async_trait]
trait Trait: Send + Sync {
    async fn do_something(&self) -> i32;
}

async fn use_trait<T: Trait>(x: &T) {
    let boxed: Arc<Box<dyn Stream<Item = i32> + Send + Sync>>;
    let lazy_poller = unfold(None, move |state| async move {
        if let Some(value) = state {
            Some((value, Some(value)))
        } else {
            let value = x.do_something().await;
            Some((value, Some(value)))
        }
    });
    boxed = Arc::new(Box::new(lazy_poller));
    let boxed_clone = boxed.clone();
    let rt = Runtime::new().unwrap();
    rt.block_on(async {
        let _moved = boxed_clone;
        // Do something with `_moved.next()`
    });
}

However it compiled with the following error:
error: future cannot be shared between threads safely
  --> src/main.rs:21:22
   |
21 |     boxed = Arc::new(Box::new(lazy_poller));
   |                      ^^^^^^^^^^^^^^^^^^^^^ future created by async block is not `Sync`
   |
   = help: the trait `Sync` is not implemented for `dyn futures::Future<Output = i32> + std::marker::Send`
note: future is not `Sync` as it awaits another future which is not `Sync`
  --> src/main.rs:17:25
   |
17 |             let value = x.do_something().await;
   |                         ^^^^^^^^^^^^^^^^ await occurs here on type `Pin<Box<dyn futures::Future<Output = i32> + std::marker::Send>>`, which is not `Sync`
   = note: required for the cast to the object type `dyn Stream<Item = i32> + Sync + std::marker::Send`

It seems that async_trait desugars the async methods' return types as Pin<Box<dyn Future<...> + Send>>, without specifying Sync. However I thought it would also be fairly common to require Futures to be Sync, in addition to Send.
My questions are:

How should I specify the async method's return type to be Sync, and
Why does async_trait not specify Sync for the return types automatically?

After some searching I found that Futures do not need to be Sync for most of the use cases; and when it may be polled from different tasks, FutureExt::shared is the way to go. Closing

After some searching I found that Futures do not need to be Sync for most of the use cases; and when it may be polled from different tasks, FutureExt::shared is the way to go. Closing


[BUG] Discord Links are invalid

Describe the bug
README Discord links are no longer valid, noticed when clicking on it I just get an expired link.
Raising as it may be the preferred communication place for some people? 😄

Thanks for letting me know! I updated the invite URL in the readme, this one should now work.
It's weird though because when creating the old link I remember specifically setting it to "Not expire", but perhaps I still made a misclick. The new one should not expire anymore, so let's hope it doesn't this time.


Cannot connect to the network truffle

General information
| Description: While I was following the No6 youtube video about truffle. I tried to yarn deploy --network truffle. I get an error message all the time
| Project version: 2.44
| Operating system and version | win10
| Yarn version | 1.22.19
| Truffle version | X.Y.Z |
Describe the bug
While I was following the No6 youtube video about truffle. I tried to yarn deploy --network truffle. I get an error
How to Reproduce
Steps to reproduce the behavior (please provide a reproducer project for complex bugs):

entering yarn deploy --network truffle

Screenshots

did you first start truffle with command: truffle dashboard

Yep, the truffle dashboard works fine, I can connect to my wallet. But once I try to deploy my contract the error comes up.

Hello, Im running into this exact issue and was wondering if a solution was found?


Configure CI to run fuzz tests in distributed mode with MinIO

What type of enhancement is this?
Other
What does the enhancement do?
The PR #3967 enables the fuzz tests to run in distributed mode (with local disk). However, we also need the fuzz tests to run in distributed mode with shared storage(using MinIO).
See also:

https://min.io/docs/minio/kubernetes/upstream/operations/install-deploy-manage/deploy-minio-tenant-helm.html#deploy-tenant-helm
https://github.com/GreptimeTeam/helm-charts/tree/main/charts/greptimedb-cluster

BTW, If you encounter issues with pods failing to start in k8s, please reduce the requested resources(e.g., etcd CPU, memory).
Implementation challenges
No response

can i take it?

can i take it?

Have fun🤩


Website Agent gets stuck now and then

Hi,
From about a month ago, I found my Website Agents get stuck now and then. The job, showing running,  gets stuck at the top of long job queue and cann't be deleted. When this happens, I have to restart Huginn to clear up the stuck job. I tried to find some logs but failed. Last week I updated Huginn to the lastest version but the problem still haunts me. Any ideas?
Thanks.

Discussing the same issue in #1567, so I'd close this one. : )


magicalCamera.getPrivateInformation().getOrientation() is returning null

Hi Fabian,
I need to know what is the orientation of photo. What is the correct way to do it ? Depends on it, i should rotate the image

hI @ardakaplan
Yes in android devices this is an common issue, because for example samsung device have the camera in landscape mode, different to others devices.
Then, the get orientation with ExifInterface, isnt a solution for this issue...
I think that the solution always was measure the photo, and determinate with the result, if is potrait or landscape orientation...
I need to fix it, but in another version


Separation of concerns for the OpenSea creator fees enforcement opt-in, and blacklisting other exchange contracts.

The Current Policy of Registry Use
Hello OpenSea, thank you for sharing the contracts. They are another interesting contribution to the NFT space and another learning opportunity. This issue concerns this part of the README.

Contract owners may implement their own filtering outside of this registry, or they may use this registry to curate their own lists of filtered operators. However, there are certain contracts that are filtered by the default subscription, and must be filtered in order to be eligible for creator fee enforcement on OpenSea.

The example in this repository has an NFT implementing a blacklist of transfers from the default OpenSea registry. https://github.com/ProjectOpenSea/operator-filter-registry/blob/0871c3a9db8d9d07053bce07302eecb4bc3df11c/src/OperatorFilterer.sol#L29-L37
The default OpenSea registry includes the following exchanges.

Name
Address
Network

Blur.io ExecutionDelegate
0x00000000000111AbE46ff893f3B2fdF1F759a8A8
Ethereum Mainnet

LooksRareExchange
0x59728544b08ab483533076417fbbb2fd0b17ce3a
Ethereum Mainnet

X2Y2 ERC721Delegate
0xf849de01b080adc3a814fabe1e2087475cf2e354
Ethereum Mainnet

SudoSwap LSSVMPairRouter
0x2b2e8cda09bba9660dca5cb6233787738ad68329
Ethereum Mainnet

The Problem
As an NFT creator, I receive significant volume on my collections from several of these blacklisted exchanges. While these exchanges support optional creator fees, many of my community members still choose to pay creator fees. Ultimately it is their choice and I would like to respect that.
As an NFT creator, I also appreciate all the support OpenSea can give me. I would like for my NFT to ideally balance both my own expectations of profit from secondary sales and the experiences of my community on their platform of choice.
When my community chooses to use OpenSea as their trading platform, I would like to opt in to enforcing creator fees. I do not, however, want to negatively impact the experience of my community members who choose to use any other exchange and therefore do not want to blacklist the use of those exchanges. In short, I find the present situation as it stands right now ideal and would prefer a way to maintain that situation for my NFTs and my community.
I understand the business rationale behind combining these use cases, but as a creator would prefer if OpenSea gave me the option to opt in to enforced creator fees without also requiring that I blacklist any other exchanges.

I'm happy to contribute a pull request that might accomplish this, though the issue is currently a non-technical policy decision.

Thank you OpenSea for the update today regarding your decision to continue the best-effort collection of creator fees on existing collections. It means that my personal concerns here have been alleviated. I can continue receiving secondary fees from OpenSea without being forced to blacklist any exchanges. This is the ideal situation for the expectations of my current community.
However, I am fortunate to be grandfathered in. I'm still leaving this issue open because per this Tweet you still intend to force all new collections to blacklist exchanges before you will attempt the best-effort collection of creator fees for them. I still believe that you can continue to offer new collections the choice to find themselves in the same situation that I have been grandfathered into.

That's it. Well said.
Time to change the monopoly of OpenSea and move to somewhere else, bc they clearly don't care about anyone except the 2.5% fee they get.
Your 2 months old ticket is still not answered. Laughable. Died GitHub-repo/bitcoin owners are more active than they (especially the quadriga guy lol)

Another policy change today per this tweet. Yet again disappointed that you opened this entire can of worms.


Support SVG Colour Specification

Description
SVG has a specification to support more colour formats, including the CMYK format which is a necessary feature.
You can use a CMYK colour with device icc or user specified icc just like
<circle fill="#CD853F device-cmyk(0.11, 0.48, 0.83, 0.00)"/>

Please consider supporting this feature.
Use Case
CMYK are critical but also necessary for accurate printed colour.

Duplicate of #3974

If anything we should probably close the other issue, as this one is more general


Please make tag for 1.4 version

Please make tag for 1.4 version and push to cocoapods please. I see commit https://github.com/OAuthSwift/OAuthSwift/commit/c86b94c1df595d6529f50aaf4e0ff4422f4d68b7 where version updated in podspec file but without tag. Also .swift-version should be updated to 5 version.

I make a tag 1.4
but I have issue to publish on Cocoapods
$ pod trunk push

[!] Found podspec `OAuthSwift.podspec`
Updating spec repo `master`
Validating podspec
 -> OAuthSwift (1.4.0)
    - NOTE  | xcodebuild:  note: Using new build system
    - NOTE  | xcodebuild:  note: Planning build
    - NOTE  | xcodebuild:  note: Constructing build description
    - NOTE  | xcodebuild:  warning: Capabilities for App may not function correctly because its entitlements use a placeholder team ID. To resolve this, select a development team in the build settings editor. (in target 'App')

[!] An internal server error occurred. Please check for any known status issues at https://twitter.com/CocoaPods and try again later.

same on website https://cocoapods.org/pods/OAuthSwift
I do not know where to ask support

However you can in your podfile set the git url and tag

Did you try to install latest cocoapods? They are released the new version recently.

...also try to run it with --verbose. If nothing help try to write them directly in https://github.com/CocoaPods/CocoaPods/issues/8200

I use the last version, no more information with verbose
reading 141 bytes...
-> "{\"error\":\"An internal server error occurred. Please check for any known status issues at https://twitter.com/CocoaPods and try again later.\"}"
read 141 bytes
Conn keep-alive
[!] An internal server error occurred. Please check for any known status issues at https://twitter.com/CocoaPods and try again later.

I am working also on a breaking change version 2.0 with Result

I make a new version 1.4.1
It seems to be ok


Add installation instructions for Fedora 30+ to readme

I have recently submitted ProcDump to Fedora's repositories (it should land in stable within a few days, see the Fedora Updates System), so it can be now installed easily using the default package manager in Fedora 30 and later.
This commit adds the instructions for installation and removal of the package to the readme.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: mgrabovsky sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@mgrabovsky thank you for your contribution to this but with the release of Procdump 1.1 we were able to complete the release pipeline for RPM packages. Please checkout our install instructions to be able to install the latest version of Procdump.


Support for white background

The trace is basically unreadable when using a bright background for the terminal:

pls add support to an alternative color scheme...

Oops, good call. It's not hard to do, we'll just need to add a configuration file.

This has been resolved in 21a4934.

Thanks. I think envvar would have been simpler, but this works too. 👍


The Card Form Type does not appear documented

The Card Form Type does not appear documented

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 27a43121-413a-fcee-76b1-387255cb0f8e
Version Independent ID: d3be4167-d9c3-f6aa-3cef-c0f1ee6da59a
Content: Type of forms in PowerApps
Content Source: ce/customize/types-forms.md
Service: crm-online
GitHub Login: @Mattp123
Microsoft Alias: matp

@eduardorivas9  Thank you for your feedback. The below topic has been added that describes how to create and use custom card forms. Also, the below Customer Engagement topics have been revised to mention card forms where applicable.
PowerApps
https://docs.microsoft.com/en-us/powerapps/maker/model-driven-apps/create-card-forms
CE
https://docs.microsoft.com/en-us/dynamics365/customer-engagement/customize/create-design-forms
https://docs.microsoft.com/en-us/dynamics365/customer-engagement/customize/assign-form-order
https://docs.microsoft.com/en-us/dynamics365/customer-engagement/customize/control-access-forms


Address apiview comments for CosmosDB

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.CosmosDB


File names save as jpg.jpg

I believe I solved the issue by turning on the I want the readable file name, but now it doesn't show me if I have downloaded the file or not.
Screenshots attached. First one is with Readable fine name option turned on, and in the second one it's turned off.

I also get this. Files always save as .jpg.jpg or .png.png. I'm using default settings, with readable file names turned off.

Just adding to note that this does not seem to happen with .mp4 files, even under the same settings.


Update MAPL (and GEOS) to use Namespace GFE

Recently, @tclune released updates to GFE to move it to use CMake namespace, i.e., use GFTL::gftl instead of gftl in CMake.
Currently this will require changes to many places:

ESMA_env for Baselibs
MAPL
GEOSgcm_GridComp
FvdycoreCubed_GridComp
fvdycore

I'm opening this issue so that @tclune and myself have a place to think about how to handle this. Change all at once? or maybe dribble in changes to CMake and use a -DUSE_GFENAMESPACE=ON to trigger it? I'm doing testing now on my laptop to make sure at least MAPL works.
Steps (official):

Issue new releases of yaFyaml and pFlogger
Issue Baselibs 6.1.0
Decide on namespace transition

Done.


MAINT: include benchmarks and dev files in sdist.

Closes gh-6498.
[ci skip] [skip ci]

It's interesting that Circle CI didn't pick up the [ci skip] tags


Touch Rugby Intro Sessions

Session Name
Dare to TRY
Abstract
Games and activities to introduce touch rugby to the freshmores, to let them learn more about the sport, and to encourage them to join our club.
Student Organisation organising this session
SUTD Rugby Club
Session intent (what to expect)
-We hope that participants can have a good time with their friends while getting a feel of how the game is like
-Participants should come in sports attire + cleats (if they have)
Session details
We expect about 20-30 participants.
Venue: Field
Dates: 22/5 (tbc) and 27/5, 7.30pm - 10pm
(Our trainings are held on Mondays and Wednesdays from 7pm - 10pm)
Speakers/Workshop facilitators
Our coach and club members will be leading the games and guiding the freshmores along.
Logistical details
Drinks - $40
<EMAIL_ADDRESS>

the deadline for budget allocation has passed but we can still have you in the calendar here:
https://discover.sutd.dev/calendar/index.html
Would you still like to have the event?

@pencilleaf also can you confirm 22/5 is go or no go
Tentatively we already have you in the calendar.
https://discover.sutd.dev/events.html#dare-to-try

discussed with touch rug, the will not be doing the session on the 22nd

changes have been noted on website


Multiplayer host does not give feedback on error

regression, I redid the menus today and didn't reimplement the status label

E 0:00:20.629   rpcp: RPC '_receive_my_player_data' on yourself is not allowed by selected mode.
<C++ Error>   Condition "skip_rpc && !(call_local_native || call_local_script)" is true.
<C++ Source>  core/io/multiplayer_api.cpp:706 @ rpcp()
 network.gd:41 @ initialize()
main.gd:42 @ start_game()
main.gd:59 @ host_server()
main.gd:119 @ _on_host_pressed()


Use Closure Compiler with Advanced Mode

Also adds size tracking via filesize.

Awesome work, thanks @kristoferbaxter! 🙌


Add integration tests for ConfigClient

Add test for getCatalogs method
Resolves: OLPEDGE-1230
Signed-off-by: ashevchu<EMAIL_ADDRESS>

Codecov Report

Merging #118 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #118   +/-   ##
======================================
  Coverage    82.8%   82.8%
======================================
  Files          44      44
  Lines         882     882
  Branches      134     134
======================================
  Hits          731     731
  Misses        112     112
  Partials       39      39

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8231b9f...6895a53. Read the comment docs.


Release 2.3.1

Once https://github.com/bblfsh/client-go/pull/44 is merged, release 2.3.1

Done, thanks for the PR.