tvm: 0.8.0 -> 0.9.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/apache/incubator-tvm/releases.
meta.description for tvm is: "An End to End Deep Learning Compiler Stack for CPUs, GPUs and accelerators"
meta.homepage for tvm is: "https://tvm.apache.org/"
meta.changelog for tvm is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.9.0 with grep in /nix/store/7i6m2k6j0srnixnvamin3bfv6zhrm64l-tvm-0.9.0
found 0.9.0 in filename of file in /nix/store/7i6m2k6j0srnixnvamin3bfv6zhrm64l-tvm-0.9.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
tvm

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/7i6m2k6j0srnixnvamin3bfv6zhrm64l-tvm-0.9.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A tvm https://github.com/r-ryantm/nixpkgs/archive/834daa7b5d4be053bcf847017207a31b8d11d1e5.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/7i6m2k6j0srnixnvamin3bfv6zhrm64l-tvm-0.9.0
ls -la /nix/store/7i6m2k6j0srnixnvamin3bfv6zhrm64l-tvm-0.9.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

tvm

Maintainer pings
cc @adelbertc for testing.

Result of nixpkgs-review pr 184105 run on x86_64-linux 1

1 package built:

tvm


Issue 219 - Scale Docker clean up

This implements Docker cleanup for Scale job executions.

Job executions now have a "cluster_id" field that guarantees uniqueness across frameworks.
New node handling in the scheduler tracks a variety of node states. Nodes are put in a state where they must perform an initial cleanup before other tasks can be scheduled on them.
Created a new clean up task that gets scheduled as a Mesos command task. If performing initial clean up on a node, it deletes all non-running Docker containers and all dangling Docker volumes. If cleaning up after specific job executions, it deletes all non-running Docker containers and all Docker volumes for the given job executions.
Implemented a CleanupManager that handles the clean up state for each node. When job executions reach any terminal state, they get sent to the CleanupManager which creates cleanup tasks for the nodes.
Removed Scale Cleanup system job and associated fields from the job execution model.
Added a migration to increase the maximum character limits for the task_update model.

The properties are to signify/allow public read, but not public write. There's a lot of multi-threaded stuff going on, and I wanted clear boundaries to assist developers. I don't think adding the properties is too much of a burden.

There are a lot of manager classes involved in the scheduler that need to be passed to the threads. The number of parameters is too large, so I created a container class to pass in the many managers as one parameter instead of going a global/singleton route. If there is a more Pythonic way to address this, I'm open to suggestions.

Using properties for read-only purposes is a valid use case. Not saying this applies to all of the examples here, but as a general rule I just don't like it when code docs devolve into useless boilerplate such as property "name" - "Gets the name" or whatever.

I would say in general, by using imports based on examples from existing 3rd party projects like say the Django models API, rather than constructor arguments. I haven't studied the bigger picture of all this threading code to say for sure if that would make sense in this context though.


code for linked list

adeed a code which has all function of linked list

Already present, create another

I have added all the function related to linked list.
On Wed, 19 Oct, 2022, 00:45 Arunim Malviya, @.***>
wrote:

Already present, create another
—
Reply to this email directly, view it on GitHub
https://github.com/Arunim313/Hackto.py/pull/84#issuecomment-1282887612,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AX6VPVR5J5MR3LXBVJPZJWLWD3ZOBANCNFSM6AAAAAARIJB3F4
.
You are receiving this because you authored the thread.Message ID:
@.***>


Adds dependency injection libraries for android

Adds dependency injection libraries for android development

@prateekcode I have added DI libraries, kindly check the PR.

@prateekcode can you have look at the PR?

Great @prasad-333 , I'm merging into the master.


add support for contexts that don't need isolation

implement a way for contexts to opt out of context isolation, when they don't change the state of the objects in the context.
        describe("routing", isolation:false) {

            val restaurant = autoClose(
                Restaurant {
                    post("/handlers/reverser", ReverserService())
                }
            )
            it("returns 404 if the route is not found") {
            }
            it("calls handlers with body and returns result") {
            }
        }

without this feature its already possible to declare dependencies outside of the root context, in the object or in the class or outside in the kotlin file. then they will not be recreated. but there is currently no way to close them.

this is already implemented in main


Update batch-jobs README.md

fix relatives links
Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

LGTM! thanks @tzahimizrahi


import-url from bucket with folder names that are prefixes of another folder doesn't work as expected

Bug Report

Description

When using the command import-url to import a folder from a Google cloud storage bucket that is a substring of another folder, it also import that other folder.
Example:
consider the following bucket structure:
my-bucket/data/file1.txt
my-bucket/dataset/file2.txt

Then the command dvc import-url "gs://my-bucket/data" . will also import my-bucket/dataset.
Reproduce

Create a bucket my-bucket with two folders data and dataset
In a dvc project, run dvc import-url "gs://my-bucket/data" .

Expected

Only my-bucket/data should be imported.
Environment information

Output of dvc doctor:
$ dvc doctor

DVC version: 2.35.2 (pip)
Platform: Python 3.10.8 on Linux-4.19.0-22-cloud-amd64-x86_64-with-glibc2.28
Subprojects:
dvc_data = 0.28.0
dvc_objects = 0.14.0
dvc_render = 0.0.14
dvc_task = 0.1.6
dvclive = 1.0.1
scmrepo = 0.1.3
Supports:
gs (gcsfs = 2022.11.0),
http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/sda1
Caches: local
Remotes: gs
Workspace directory: ext4 on /dev/sda1
Repo: dvc, git
Additional Information (if any):

@D34D10CK can you try the latest release? There have been some changes in both dvc and the underlying gcsfs library that may have resoled your issue

Can reproduce in latest main + dvc-gs with and without the trailing /. Only the expected folder is imported (the resulting .dvc file only contains the single intended directory). The issue is that additional objects which match the given prefix are downloaded into the workspace (but they are not included in the resulting .dvc file).

This is fixed in dvc-gs==2.22.1


Harvard Forest 1999

@teixeirak FYI, 1999 at Harvard Forest is going to look a bit weird. This was a pretty tough drought year so DOY25 is happening earlier (interestingly, it seems to be impacting DP tree timing more), likely as a consequence of less growth due to drought. This is essentially what Ref 1 was saying about the DOY variables being dependent on one another. I've tried reducing the SD in the DOY timing cleaning category but it doesn't improve much (though, I think we should reduce this to 2 SD which we can discuss in the data cleaning issue).
We already removed 1998 so removing 1999 as well would leave us with 4 years of data for HF. We have two options:
1.) We can leave it in, explain it, and move on.
2.) Remove the year
I think we would be fine leaving it in, but would like your input.

I would also favor leaving it in.

I think we can close this.


Media availability sync job is broken for jellyfin/emby media servers

Description
The media availability sync job feature in Jellyseerr only supports Plex Media Server as the backend for checking availability. However, when setting up the app with Jellyfin or Emby as the media server backend, the media availability sync job behaves unpredictably. It sometimes removes requests and availability status from media that is available in Jellyfin/Emby and/or Sonarr/Radarr. It does not remove availability status from removed media.
Currently, jellyseerr has the job disabled until it can be reworked to support Jellyfin/Emby media server backends.
Version
Latest
Steps to Reproduce

Remove a media from jellyfin/emby and/or sonarr/radarr
Observe availability status not removed by the media availability sync job
Once job finishes, observe some existing media on both jellyfin/emby and sonarr/radarr to be set to UNKNOWN and request deleted

Screenshots
No response
Logs
No response
Platform
desktop
Device
not relevant
Operating System
not relevant
Browser
not relevant
Additional Context
No response
Code of Conduct

[X] I agree to follow Overseerr's Code of Conduct

:tada: This issue has been resolved in version 1.8.0 :tada:
The release is available on:

v1.8.0
GitHub release

Your semantic-release bot :package::rocket:


added validation while updating profile

What is the change?
Added validation while updating profile, so that if user enters a username which is already taken, then it will show error
Related issue?
close: #569
How was it tested?
Tested locally
Checklist:
Before you create this PR, confirm all the requirements listed below by checking the checkboxes [x]:

[x] Have you followed the Contribution Guidelines while contributing.
[x] Have you checked there aren't other open Pull Requests for the same update/change?
[x] Did you lint your code before making the Pull Request?
[] Have you made corresponding changes to the documentation?
[x] Your submission doesn't break any existing feature.
[x] Have you tested the code before submission?

Screenshots or Video:
Add a screenshot or demo video if appropriate.
https://user-images.githubusercontent.com/68139755/115908992-80e60d80-a488-11eb-8c62-90824192e107.mp4

After the conflicts are resolved can one of you guys test the code @AnanyaNegi @RushanKhan1
I was working with GirlScript to get project name updated for the website and for points table so might not get time rn

Conflicts are resolved, please review


Dynamic lighting missing/broken on some meshes

Godot version:

3.0.6.stable.custom_build (compiled from "https://github.com/godotengine/godot/archive/3.0.6-stable.tar.gz")
OS/device including version:

OpenGL ES 3.0 Renderer: Radeon RX 560 Series (POLARIS11, DRM 3.26.0, 4.18.5-arch1-1-ARCH, LLVM 6.0.1)
(confirmed on other machines)
Issue description:

Lighting does not work for certain meshes.
Adding a UV2 map allows baked lightmaps but not dynamic.
Disabling the normal map OR enabling Triplanar UV2 both seem to fix the problem.

I also noticed the widgets won't draw over the affected objects

Steps to reproduce:
Run attached scene. Meshes are baked into the scene, exported from Maya dae
Minimal reproduction project:

couchbug.zip

A few things to check
-When exporting, make sure the generate tangents checkbox is ticked, I would also make sure the triangulate checkbox is also ticked.
-Normal maps don't work if they are in imported as Srgb, if that is the case then you need to change the color space and re-import them.

Srgb was 'detect' but disabling it did not change the outcome.
Exporting from maya as obj instead of dae seems to fix the problem also. (not sure what settings the artist is using)

I reproduced the problem in Godot 3.0.6.
I also reproduced the problem identically in Godot 3.1 alpha 2 / master.
As noted by the reporter, this seems to be an incompatibility between Baked Light-maps, Normal Maps and the UV2 channel.

There are two issues with the 'LongSofa' meshes:

Their UV2 has not been exported nor generated. The Godot Editor can generate one for you (screenshot below) or your 3D artist can generate it a DCC Tool.

The Tangent and Binormal vertex streams were not imported nor generated, during the COLLADA import in the Godot Editor. This is definitely an issue that I would like to investigate, since it seems to work fine in other cases. @jamie-pate Could you please provide me with the original COLLADA file and/or the Blender file for the 'LongSofa' mesh? Thanks :)

Still reproducible in 3.2.2 rc2
Although the above comment remains unanswered...
Also here's this scene in 23f7f86 heh

Fixed by #44628.


Fix asciidoc typo not showing hyperlink correctly

@marty-johnson59 I got a request from Eric @ Igalia to merge #192, but this isn't my repository to manage to assigning over to you (this does look obvious). I think #190 fixes a subset of #192 and probably? can just be closed after #192 is accepted.

Sounds good.  I'll check it out and merge.  Copying @gpx1000 who's our Vulkan documentation officer.  He may be the right person to own these things moving forward.


Fix sendAcrossNetwork for multi-dimensional Views

This should fix the failing tests in https://github.com/ORNL-CEES/DataTransferKit/pull/574 while not impacting performance for the one-dimensional case.

When using this PR inside DTK, I get
/home/dtk/packages/Search/src/details/ArborX_DetailsUtils.hpp(125): error: no instance of overloaded function "Kokkos::deep_copy" matches the argument list
            argument types are: (const Kokkos::CudaUVMSpace::memory_space, Kokkos::View<int **, Kokkos::LayoutRight, Kokkos::CudaUVMSpace>, const Kokkos::View<int **, DeviceTypeKokkos_Compat_KokkosCudaWrapperNode>)
          detected during:
            instantiation of "auto ArborX::Details::create_layout_right_mirror_view_and_copy(const ExecutionSpace &, const View &, std::enable_if<<expression>, void>::type *) [with View=Kokkos::View<int **, DeviceTypeKokkos_Compat_Kokkos
CudaWrapperNode>, ExecutionSpace=Kokkos::CudaUVMSpace::memory_space]"
/home/dtk/packages/Search/src/details/ArborX_DetailsDistributor.hpp(283): here
            instantiation of "void ArborX::Details::Distributor<DeviceType>::doPostsAndWaits(const ExecutionSpace &, const ExportView &, size_t, const ImportView &) const [with DeviceType=DeviceTypeKokkos_Compat_KokkosCudaWrapperNode, Ex
ecutionSpace=Kokkos::CudaUVMSpace::execution_space, ExportView=Kokkos::View<int **, DeviceTypeKokkos_Compat_KokkosCudaWrapperNode>, ImportView=Kokkos::View<int *, Kokkos::CudaUVMSpace::memory_space, Kokkos::MemoryTraits<1U>>]"
/home/dtk/packages/Search/src/details/ArborX_DetailsDistributedSearchTreeImpl.hpp(215): here
            instantiation of "std::enable_if<Kokkos::is_view<View>::value, void>::type ArborX::Details::DistributedSearchTreeImpl<DeviceType>::sendAcrossNetwork(const ExecutionSpace &, const ArborX::Details::Distributor<DeviceType> &, Vi
ew, View::non_const_type) [with DeviceType=DeviceTypeKokkos_Compat_KokkosCudaWrapperNode, ExecutionSpace=Kokkos::CudaUVMSpace::execution_space, View=Kokkos::View<int **, DeviceTypeKokkos_Compat_KokkosCudaWrapperNode>]"
/home/dtk/packages/Meshfree/test/tstDetailsCommunicationHelpers.cpp(69): here
            instantiation of "void Helper<DeviceType>::checkSendAcrossNetwork(MPI_Comm, const View1 &, const View2 &, const View2 &, __nv_bool &, Teuchos::FancyOStream &) [with DeviceType=DeviceTypeKokkos_Compat_KokkosCudaWrapperNode, Vi
ew1=Kokkos::View<int *, DeviceTypeKokkos_Compat_KokkosCudaWrapperNode>, View2=Kokkos::View<int **, DeviceTypeKokkos_Compat_KokkosCudaWrapperNode>]"
/home/dtk/packages/Meshfree/test/tstDetailsCommunicationHelpers.cpp(126): here
            instantiation of "void DetailsDistributedSearchTreeImpl_send_across_network_UnitTest<DeviceType>::runUnitTestImpl(Teuchos::FancyOStream &, __nv_bool &) const [with DeviceType=DeviceTypeKokkos_Compat_KokkosCudaWrapperNode]"
/home/dtk/packages/Meshfree/test/tstDetailsCommunicationHelpers.cpp(241): here

Thanks for checking! I haven't looked at CUDA yet.

@masterleinad Can you please describe what the problem is, and how you address it?

Sure. The issue is that we need LayoutRight for MPI communication in its current form both for the send buffer and the receive buffer. This requires changing the layout at least when we are using CUDA but also to be careful when applying the permutation.
The idea is to change the layout as late as possible, i.e. after applying the permutation, for the data to send so that we have to deal with the type-punned data as little as possible.
There are not that many actual changes (mostly https://github.com/arborx/ArborX/pull/353/commits/e26474fa63e1106edc5783fc29ec52f820395b67).
It seems that we need another intermediate copy when changing the layout and copying between different memory spaces.

One thing that can likely be improved is to have a better separation between doPostsAndWaits and sendAcrossNetwork and possibly factor applying the permutation in doPostsAndWaits out. I would rather do it in a follow-up, though, to fix DTK.

@dalg24 I addressed your commits and the tests are passing.


create peer when pre-vote received if need

What have you changed? (mandatory)
Create new peer if pre-vote received and there is no such peer. The bad case is showed in the new added test.
What are the type of the changes? (mandatory)
Bug fix.
How has this PR been tested? (mandatory)
make dev.
Does this PR affect documentation (docs/docs-cn) update? (mandatory)
No.

Great Catch

is the test reliable here now? @Hoverbear

or do we still need to remove them?

@siddontang https://github.com/pingcap/tikv/pull/3386 makes them reliable.

ping @overvenus @Hoverbear PTAL thanks.

Please approve it @Hoverbear


Type: fix link in readme

This is straight forward, but makes getting up to speed less of a burden :)

Perfect, thank you!


Maritime Display properties not loading right config

WIthout any other configuration to the maritime display widget, the console logs this error:
SyntaxError: "JSON.parse: unexpected character at line 1 column 1 of the JSON data" at https://marineview.hydro.gov.my:3344/webappbuilder/apps/3/libs/mcs-widgets/DisplaySettings.js?wab_dv=2.12:144
I traced it to line 144 where it is trying to retrieve the this.parametersContent, which ultimately tries to load a bunch of config json data named mcsParametersContent, which does not exist.
How should i properly populate this parameter to get what i assume to be Miscellaneous settings.
Also there are only two tabs , Depth Contour and Micellaneous, whereas the example app, http://nsdemo.esri.com/demo/wab4mcsenc/# has a third, Symbol Size. How do i get that?

Hi @mkmoey13,
Most likely you didn't select any parameters in the "Configure Maritime Display Properties" dialog when you initially add the Maritime Display Property widget to your app. You need to press the query button to load all the available parameters and select one or more parameters.

To get the third tab with symbol size settings, you will have to configure the config.json to include the parameters. Please refer to this section: https://github.com/Esri/maritimechartservice-sample-viewer#new-deployment-options-for-maritime-display-parameters. Older MCS versions do not support the symbol size parameters and few others. Please make sure you meet the version requirement.

Hi Abin,
Ok, i get it now, it's just unfortunate that we can't add new parameters after adding in the widget.


Update instructions to Kubeflow v0.7

Address bugs from Issue #462
Description of changes:
Fixed some bugs and updated instructions to latest Kubeflow version
https://www.kubeflow.org/docs/aws/troubleshooting-aws/
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@dalbhanj do you want to rebase your changes on a clean branch? Even this repo does squash merge, we see unnecessary commits here..

@Jeffwan  & @arun-gupta take a look at newest PR https://github.com/aws-samples/eks-workshop/pull/470. Doesn't have all the other commits and is much cleaner

I'll close this PR in the interest of https://github.com/aws-samples/eks-workshop/pull/470


stable/redis

What this PR does / why we need it:
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[X] Title of the PR starts with chart name (e.g. [stable/chart])

Thanks for the contribution, before continue, you need to sign your existing commits, you can find custom instructions in the “Details” link from the below DCO check, also you should increment the chart version in the Chart.yml according to semver:

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner, and
PATCH version when you make backwards-compatible bug fixes.
Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

In the same way, can you provide more information by filling the template that appears in the description? Also, it would be great if you add a descriptive title


[SR-1895] compilation warning for void result type

Previous ID
SR-1895

Radar
None

Original Reporter
VladimirS (JIRA User)

Type
Bug

Status
Resolved

Resolution
Done

Environment
Swift 3.0

Additional Detail from JIRA

Votes
0

Component/s
Compiler

Labels
Bug, DiagnosticsQoI

Assignee
@ahoppen

Priority
Medium

md5: 256077fcd7fa34c0ecd899dc3c70f0c9

relates to:

SR-1752 Warning about unused result with ternary operator

Issue Description:
let s: String? = "hi"
s.map {print($0)}

we have:
warning: result of call to 'map' is unused
s.map {print($0)}
Isn't it strange that result of call is Void, but we have a warning? Don't
we need to suppress warning in this case?

Do I experience this same issue?
var outputFile: [ Int ] = []
[ 0x01 ].map { outputFile.append( $0 ) }

At least the error message is the same: Warning: result of call to 'map' is unused

Really, you should not use map for this.  Use a for loop; it's easier to read.
var outputFile: [ Int ] = []
[ 0x01 ].map { outputFile.append( $0 ) }

Basically, map should never be used for side-effects like this.


Implement star generation

Description
This PR:

Adds a star generation struct for generating new star
Star type data with sizes, radius etc.
Extends the ResourceHandler to handle StarTypesResource
Generate a new star in each system constructor

Alternate Designs
Not add extend system constructor arguments, instead create a builder.
This would be preferred and will be implemented later on.
Benefits
Adds Star generation
Possible Drawbacks
N/A
Applicable Issues
N/A
Enter any applicable Issues here

Codecov Report

Merging #24 into master will increase coverage by 0.28%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #24      +/-   ##
==========================================
+ Coverage   96.87%   97.16%   +0.28%
==========================================
  Files           4        4
  Lines         128      141      +13
==========================================
+ Hits          124      137      +13
  Misses          4        4

Impacted Files
Coverage Δ

src/astronomicals/mod.rs
100% <ø> (ø)
:arrow_up:

src/resources.rs
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44423ae...94b9378. Read the comment docs.


Chargeback reports fail with error from ActsAsArModel

[----] E, [2016-03-28T18:28:42.781370 #18193:7d9998] ERROR -- : /home/azellner/Projects/manageiq/lib/acts_as_ar_model.rb:156:in `find'
/home/azellner/Projects/manageiq/lib/acts_as_ar_model.rb:161:in `all'
/home/azellner/Projects/manageiq/app/models/rbac.rb:556:in `method_with_scope'
/home/azellner/Projects/manageiq/app/models/rbac.rb:299:in `find_targets_without_rbac'
/home/azellner/Projects/manageiq/app/models/rbac.rb:294:in `find_targets_with_rbac'
/home/azellner/Projects/manageiq/app/models/rbac.rb:515:in `search'
/home/azellner/Projects/manageiq/app/models/miq_report/generator.rb:270:in `_generate_table'
/home/azellner/Projects/manageiq/app/models/miq_report/generator.rb:176:in `block in generate_table'
/home/azellner/Projects/manageiq/app/models/user.rb:257:in `with_user'
/home/azellner/Projects/manageiq/app/models/miq_report/generator.rb:176:in `generate_table'
/home/azellner/Projects/manageiq/app/models/miq_report/generator/async.rb:96:in `_async_generate_table'

@miq-bot add_label chargeback, bug, reporting

@gtanzillo @kbrock - please review

I had additional issues maybe due to an outdated database or bad gemset.
This stopped replicating after I reset everything I in the project could think of.
Sorry if someone wasted time on this.

@zeari I need to find a way to be confident in these two reports. Thank you for raising the concern. It was not a waste of time


Upgrade to OneAuth/MSAL 1.36. Fixes Sign in error: Account Switch Detected

Sign in throws Account Switch Detected error. The latest version of OneAuth fixes this. To unblock customers, update composer dependency to 1.36 and do a patch release whenever possible.
ICM ref; 262551756

This is being tracked at #8708

Closed via #8712


Add input filter rule for wireguard

Description
Without this rule, in an environment with a default drop, Wireguard
traffic will not be accepted. This change adds a static input rule in the
same fashion as done for other encapsulation modes (ipip, vxlan) to
ensure these packets are accepted. No drop rule is needed however as
wireguard does its own validation of incoming messages.
Todos

[X] Tests
[ ] Documentation
[X] Release note

Release Note
Calico will now add an ACCEPT rule for wireguard udp port when enabled.

Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one docs-* label.

docs-pr-required: This change requires a change to the documentation that has not been completed yet.
docs-completed: This change has all necessary documentation completed.
docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

release-note-required: This PR has user-facing changes. Most PRs should have this label.
release-note-not-required: This PR has no user-facing changes.

Other optional labels:

cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi @muff1nman,
First of all, thanks for your contribution!
The wireguard port is added to the failsafes (when enabled) in https://github.com/projectcalico/calico/blob/13bbda0aeec290e720fe43e0085d3cf50e2efa7d/felix/dataplane/driver.go#L186, which ultimately should add an allow rule for the wireguard port and be equivalent to what your changes are doing.
I'm just trying to understand, what was the issue you were seeing with default DROP? I'm thinking the fix may be something else (if wireguard is not working properly with default DROP)...

@coutinhop  Based on our out-of-band conversation I've removed the failsafes logic entirely for wireguard and added any missing rules to the static rules (namely OUTPUT). Feel free to reiterate/summarize our conversation here especially if you feel the latest change doesn't meet your expectations. I'll plan to squash.

@muff1nman thanks for these changes! That was exactly what I had in mind. @caseydavenport @fasaxc could you take a look and see if you can tell anything that could go wrong by switching from the failsafes to these static rules? Thanks!
/sem-approve

/sem-approve


How to install

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/step-kit/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/step-kit/
ERROR: Ignored the following versions that require a different python version: 0.1.0 Requires-Python <4.0,>=3.10; 0.2.0 Requires-Python <4.0,>=3.10; 0.2.1 Requires-Python <4.0,>=3.10; 0.2.2 Requires-Python <4.0,>=3.10
ERROR: Could not find a version that satisfies the requirement step-kit (from versions: none)
ERROR: No matching distribution found for step-kit

I thought this was a network issue. You can try to use a pypi mirror like tsinghua's which works for me. Also, you need to have a python installed with a version at least 3.10.
P.S. I've replied you at wechat with my contact info. You can check it there.


cmd/asm, math: improve performance for math.Floor, math.Ceil, math.Trunc with asm on ppc64x

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
tip
What operating system and processor architecture are you using (go env)?
Ubuntu 16.04 ppc64le
What did you do?
Built and ran the math package benchmark test.
What did you expect to see?
Better performance for floor, ceil, and trunc functions in math package.
What did you see instead?
Room for improvement
The Power instruction set includes frim, frip, and friz which can be used to implement the Go math package's floor, ceil, and trunc functions more efficiently.  These instructions are used in the existing gccgo implementation for these math functions for ppc64x.  These are compatible with power5 so can be used in both ppc64le & ppc64.

frim breaks ppc970 comparability, reopen.

The decision was that ppc64 would support instructions back to power5, not ppc970.  This was documented in the go1.7 release notes.  https://golang.org/doc/go1.7.

@laboger, I updated https://golang.org/wiki/MinimumRequirements

So can this be closed?  Here are other issues where this was discussed https://github.com/golang/go/issues/16004 https://github.com/golang/go/issues/16372 and I thought this was the final decision with respect to this topic.

Yes, looks like it.


general question

when will you finish the GLES 3.0 support on melonDS?
just curious, it's been about half a year since I created the other issue. still love this site <3

I haven't started it, it's low priority. It will be more difficult to port than Beetle PSX.

alright, thanks for the update! love the site and looking forward to the next update, i'm sure it will be great as always :)


fix redirect splats

because I'm too used to typing emojis I added an extra : in there -_-

/lgtm


feat: rename to enviparse

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.65%. Comparing base (5979118) to head (d92ab85).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #11   +/-   ##
=======================================
  Coverage   98.65%   98.65%
=======================================
  Files           4        4
  Lines         149      149
=======================================
  Hits          147      147
  Misses          2        2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Related objects persistance

Related objects of a given entity are not sincronized with the database (this issue is reproduced along Propel\Tests\Generator\Builder\Om\GeneratedObjectRelTest test case).
Get a look at the following code:
$author = new Author();
$author->setName('Oscar Wilde');
$author->save();

$book1 = new Book();
$book1->setTitle('The picture of Dorian Gray');
$book1->setAuthor($author);

$book2 =new Book();
$book2->setTitle('The importance of being Ernest');
$author->addBook($book2);
$author->save();

Everything is ok: 1 author related with 2 books, both  in memory and in db. If I add a book:
$book3 = new Book();
$book3->setTitle('The Duchess of Padua');
$book3->setAuthor($author);
$author->save();

Everything is ok: 1 author related with 3 books, both  in memory and in db.
But if I remove a book:
$author->removeBook($book1);
$author->save();

Now I have: 1 author and 2 books in memory and 1 Author and 3 books in the db.
If I add another book, then I find 4 books in the db.
Possible solutions:

when removing related objects, perform a query to also delete the entity from the database. This piece of code should be written in removeXXX method.
implement a way to mark the entity as 'deleted' and let the persister do the deletion work, via presistDependencies method
implement a preSave hook which cleanup the database before persistence
(suggestions?)

Suggestions are wellcome!

Shouldn't you have still book3 in memory? removeBook should only break the relation, not delete the book3 entity from the database in my opinion.

Ok! Anyway, removeBook should update the database, because if I break the relation, the removed object is no more a dependency and it's not persisted with the Author entity, isn't it?

Yes, it should update the database. It actually is still related, as the db state indicates still that it is related. Once it goes through the change detection it needs to check whether it sets the foreign key from Book to Author accordingly to null.

Yes it should set the foreign key to null. But I thought this works already😱
Sent from my iPhone

On 14. May 2017, at 18:42, Cristiano Cinotti<EMAIL_ADDRESS>wrote:
Ok! Anyway, removeBook should update the database, because if I break the relation, the removed object is no more a dependency and it's not persisted with the Author entity, isn't it?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Found!
The in-memory process works correctly (as @marcj told before) it seems that the modification isn't persisted. Anyway, I misunderstood a couple of test cases :grimacing: .....
I'm closing this issue.
Thank you!


feature/ add sneakable functionality to notification buttons

This PR adds sneakability to the buttons in the notification ui so they can be responded to using the keyboard

@CrossR thanks for the review 👍, yeah the clicking was driving me up the wall 😆


[move-prover] Fix bug for choice operator on type instantiation

Motivation
Resolve https://github.com/diem/diem/issues/9862
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
Add a test case in choice.move

/land


Use portal for rich select

I noticed that rich select components do not work well in containers that have hidden overflows. Is there any plan to use a portal for the rich select?

@jaulz not for this version of the package but is something I'll try to implement in the v3 if I found a way to keep it simple

@alfonsobries exciting 😊  will v3 be for Vue 3 or also for Vue 2?

@jaulz vue 3

Experiencing the same problem. I am trying to make a horizontally scrollable list of filters like on Airbnb


Rust Code "accelerator" - Part 1 - Adding size of global buffers in Uops graph

This is the first patch for adding a Rust Code "accelerator".  As expected the requirements and layout is fairly similar CLANG, however there is a major difference.  The Rust compiler needs to know how big arrays are.
When called from a function like this, the compiler will handle sizing the parameters data0..data3 for the function r_4_32()
fn r_4_32(data0: &mut [f32], data1: &[f32], data2: &[f32], data3: &[f32]) {
    data0[0] = data1[0] * data2[0] + data1[1] * data3[1]
}

pub extern "C" fn net(input0: &[f32; 32], output0: &mut [f32; 4]) {
   let buf1 = [0.0; 64];
   let buf2 = [0.0; 128];
   r_4_32(output0, input0, &buf1, &buf2);
}

However when using FFI to connect to python we need to specify the size like this:
pub extern "C" fn r_4_32(data0: &mut [f32; 4], data1: &[f32; 32], data2: &[f32; 64], data3: &[f32; 128]) {
    data0[0] = data1[0] * data2[0] + data1[1] * data3[1]
}

To enable this sizing feature I made the following changes:

Add size field to args of DEFINE_GLOBAL node in the UOps graph for Rust to size input/output arrays
Adapt dedup key to ignore size, apparently we can get redundant buffers in the graph.
Fixed test

Hmm, so it's not needed now. I'm not going to merge this until it's actually needed by a backend, and I don't think we want a rust backend, what advantage does it have over clang?

I mean... "I" want a rust backend, obviously.
My use case is that I'm building and deploying simple FNNs.  I take a model (ONNX for now) and use tinygrad to render source that I then import to build into my app.  I see a lot places in the embedded world, desktop apps, servers, and even web stuff (webGL/WASM) where using this code export capability to embed AI is an amazing feature unique to tinygrad.
Specifically why do I want a Rust backend?  The app I want to embed tinygrad output into happens to be written in Rust.  I am importing C code into my Rust app today.  But doing that when unnecessary makes for uglier, bad code with unnecessary dependencies.

An RFC level Rust backend patch is ~1 day out.
Would you prefer I move it to extra/ for line count and support purposes? I see extra/backends though that all looks dead.


Address HandleInput

Currently HandleInput is defaulting to true on all containers, even when they are not required to process input. A better default would be false, but we need to discussion the level of automation for handling input on a per-case basis (without the user explicitly specifying true), if any.

Perhaps it would be better to create interfaces for common UI events (Though this also means we would need to create a UI input event system to support it). e.g IBeginDragHandler, IClickHandler etc. This would also unify the UI Logic and allow for bubbling.
I'm no expert by any means but it seems like a much simpler and open solution then the current one. I'm saying this on the assumption that containers are only ever used for Drawables, which it looks like they are.

I don't see how this would simplify the problem we need to tackle here: We need early stopping of the downward recursion for performance reason when we have large collections of drawables that all do not handle input. So with your proposed solution we would still need some wort of automatic detection of whether any children handle input, but unlike with the HandleInput bool this can not be changed at the type-level automatically on-the-fly.


WIP: FI-1555: backend suite options

This branch adds backend support for specifying suite options. Suite options can act to filter which tests get executed and select which validators are used. required_suite_options is just a placeholder name wherever it appears.
Later PRs will add support for suite options to the JSON API and the front end.

Codecov Report

Merging #181 (3c33776) into main (c044e4f) will increase coverage by 0.06%.
The diff coverage is 98.50%.

@@            Coverage Diff             @@
##             main     #181      +/-   ##
==========================================
+ Coverage   93.88%   93.95%   +0.06%
==========================================
  Files         110      111       +1
  Lines        2797     2843      +46
==========================================
+ Hits         2626     2671      +45
- Misses        171      172       +1

Impacted Files
Coverage Δ

lib/inferno/dsl/fhir_validation.rb
96.82% <ø> (ø)

dev_suites/dev_options_suite/options_suite.rb
96.77% <96.77%> (ø)

lib/inferno/dsl/configurable.rb
97.18% <100.00%> (ø)

lib/inferno/dsl/input_output_handling.rb
97.10% <100.00%> (ø)

lib/inferno/dsl/runnable.rb
95.27% <100.00%> (+0.34%)
:arrow_up:

lib/inferno/entities/test.rb
90.47% <100.00%> (+0.11%)
:arrow_up:

lib/inferno/entities/test_group.rb
98.07% <100.00%> (ø)

lib/inferno/entities/test_session.rb
100.00% <100.00%> (ø)

lib/inferno/entities/test_suite.rb
92.45% <100.00%> (+0.61%)
:arrow_up:

lib/inferno/test_runner.rb
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c044e4f...3c33776. Read the comment docs.

class MySuite < InfernoTestSuite
  suite_option :ig_version,
               title: 'IG Version',
               description: 'Which IG Version should be used',
               list_options: [
                 {
                   label: 'v1',
                   value: '1'
                 },
                 {
                   label: 'v2',
                   value: '2'
                 }
               ]

  ... other groups
  group from: :multi_ig_group
end

If we wanted to create separate TestSuites that could appear on the Inferno landing page it would be nice to have separate TestSuites just for those:
class MyIG1Suite < MySuite
  selected_suite_option "1"
end
class MyIG2Suite < MySuite
  selected_suite_option "2"
end

I was trying/hoping to do something like this in https://github.com/inferno-framework/inferno-core/pull/181. I also tried with something with config.option, but that didn't work either because the config isn't available when the class is being defined.


Azure Instances Log Analytics Utilization

Description
This policy captures CPU and Memory utilization metrics from Azure Log Analytics and compares those values to the CPU and Memory utilization parameters which define the thresholds to raise an incident for.
The escalation will tag all identified instances to later be resized by the Downsize Instances policy.
Issues Resolved
N/A
Contribution Check List

[X] All tests pass.
[X] New functionality includes testing.
[X] New functionality has been documented in the README if applicable
[X] New functionality has been documented in CHANGELOG.MD

@srpomeroy  also need to update the repo root README with the new policy


Nested map results in exception

Boon is unable to serialize this:
private Map<Integer, Map<String, FlowPortMap>> portMaps = new HashMap<>();
MESSAGE: fieldName portMaps of class class *.FlowContext had issues for value MAP for field FieldInfo [name=portMaps, type=interface java.util.Map, parentType=class *.FlowContext]
FlowPortMap is a bunch of fields that are primitive (String, Integer)

Code sample?


grunt-run with grunt-watch

I am trying to use osprey-mock-service to mock RAML service in my project. When I serve my application with grunt, I use grunt-run to execute the mock service before the application is shared like this:
//Run
    run: {
      options: {
        wait: false
      },
      commands: {
        exec: 'osprey-mock-service -f RAML\\api.raml -p 8000'
      }
    },
...
...
grunt.task.run([
      'clean:server',
      'wiredep',
      'concurrent:server',
      'autoprefixer:server',
      'run',
      'connect:livereload',
      'watch'
    ]);

I am trying to watch my api.raml file so if it is changed, the osprey-mock-service is re-run automatically like this:
watch: {
      raml:{
          files: ['RAML\\api.raml'],
          tasks: ['run']
      }
}

When I change the file, I get the following error. What am I doing wrong? Is this a bad way to do this?
>> File "RAML\api.raml" changed.
Running "run:commands" (run) task
>> Skipping run:commands since it not itterable. Call it directly or from another task.

I have the same issue - did you work it out?

The problem is with grunt-run. If you've got the gruntfile
module.exports = function(grunt)
{
  grunt.initConfig({
    run: {
      ls: {
        cmd: 'ls'
      }
    }
  });
  grunt.loadNpmTasks('grunt-run');
};

and run grunt run you get the error message. If you run grunt run:ls it works fine.
Looking at grunt-run's source code, it looks as if tasks are skipped if the keyword run appears on the command line, unless the task is explicitly specified or the itterable (note spelling) option is set to true. I'm sure there's a really good reason for this, but I don't know what it might be.
So you would have to add itterable: true to your options for the run config, or change run to run:commands.


Multiple instances all acquire file lock

Describe the bug
If multiple ComboLock or NamedLock instances are created in separate threads, they will all acquire a file lock
To Reproduce
Test case can be found here https://github.com/NeonDaniel/combo-lock/blob/e294c50efc11c07497e564ab4171a62eb623ae7c/tests/test_lock.py#L61-L75
Expected behavior
All instances of ComboLock with the same file should lock using that file
Environment (please complete the following information):

Device type: x64
OS: Linux Mint/Ubuntu
Other versions: fasteners 0.16.3

filelock.FileLock appears to provide the expected functionality based on my testing; this package appears to be a drop-in replacement

Hmm, the testcase doesn't really work, the lock needs to be shared between the threads to work, otherwise each will have their own thread-lock. The way the combo-lock is constructed, the lock needs to be shared between the threads. Like a standard thread lock.
The test case should look something like
    def test_multi_lock_acquire(self):
        results = dict()
        lock = ComboLock(self.lock_file)

        def _test_acquire(idx):
            success = lock.acquire(False)
            results[idx] = success

        for i in range(0, 10):
            Thread(target=_test_acquire, args=(i,), daemon=True).start()

        while not len(results.keys()) == 10:
            time.sleep(0.5)
        self.assertEqual(len([lock for lock in results.values() if lock]), 1)

The file lock would make the thread part of the combolock completely redundant, the idea of the lock is that the IPC lock from fasteners would be faster than the one filelock but I need to measure that to make sure it's still true.

Did some tests and using the filelock straight up is faster. So it's a better tool than combo_lock and should probably be used instead of the combo lock.

I see, that makes sense then.. I thought the plock was essentially equivalent to a file lock, but that would render the thread lock redundant.. Would it make sense to add a file lock to this package? The RAM disk and filename utils are still applicable there

Yes, the processlock was sort of equivalent but it's not blocking for calls from other threads in the same process, hence the thread lock (and the resulting combination of a process lock and a thread lock -> combolock).
Seems like the filelock does both things and even is a bit faster so right now I would recommend to use the filelock straight up.
The combolock may have some extra conveniences and the autonaming provided by the NamedLock, wich are good. Maybe we can do a final release of this as a wrapper around the filelock and then propose upstream changes to filelock?

I think the thread lock is still useful with FileLock to allow one lock instance to be shared between threads (at least, that's how I implemented it in neon-utils).
Filelock will let the same lock acquire multiple times across threads (from https://github.com/tox-dev/py-filelock):
The lock objects are recursive locks, which means that once acquired, they will not block on successive lock requests

So, Filelock isn't quite a drop-in replacement for combo-lock. I think any actual use case can get by with FileLock alone, but adding a thread lock makes it a little easier to manage since you don't need a lock instance per thread (and this would maintain Combolock backwards-compatibility)..

You have a point here


gogdl: d2fa34b -> 89d15f1

Description of changes
Updates the revision of gogdl (still 0.7.3) to a newer revision, fixing hangs and timeout failures. The project seems to bump version occasionally and randomly, as gogdl is mostly just internal to heroic-games-launcher.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Oops, I removed that locally but forgot to include that in the patch. Done now.


type: [Type] doesn't work, causes app to crash

test: {
    type: [Number],
    label: "Test"
}

results in Meteor saying "Your application is crashing. Waiting for file change."
Page says Error: Invalid definition for test.$ field.
When I don't use the square brackets, it works.
What am I doing wrong here? According to Schema Rules https://github.com/aldeed/meteor-simple-schema#schema-rules it is allowed?

OK, this is a duplicate of #284
+1 for documenting this. I would have never suspected my pick() method, let alone knowing to fix it by picking 'test.$', too.


Unable to create windows11 21h2 driver package for microsoft surface

Unable to create windows11 21h2 driver package for microsoft surface. Kindly assist, as the tool is trying to download the driver from wrong website and MDM tool hangs for more than an hour.

Any updates please

Which Surface model are you having trouble with?


change in state for terminating event when using BacksolveAdjoint()

Hi! I am confused on what internal changes to the state are made when using BacksolveAdjoint(). I am solving a Set of 279 (but in general, N) coupled ODEs. My particular implementation has a discrete terminating event. In the scenario in which I'm not using adjoint states, my terminating event function seeks to stop the solver once a minimum threshold on the sum of $N-1$ solutions is achieved.
def condition_fn(state, **kwargs):
    del kwargs
    OMEGA_M_MIN = 1e-4
    Omega_m = jnp.sum(state.y[1:])
    condition = Omega_m < OMEGA_M_MIN
    return lax.cond(condition, lambda _: True, lambda _: False, None)

where the output of state.y is Traced<ShapedArray(float64[279])>with<DynamicJaxprTrace(level=4/0)>, and $279 = N$ as expected. After incorporating BacksolveAdjoint() in my diffeqsolve, given here:
sol = diffeqsolve(equation, solver, t0=t_eval[0], t1=t_eval[-1], dt0=0.01, y0=self.y0,
                        args=(self.Gamma_0,), max_steps=int(1e5), saveat=saveat,
                        stepsize_controller=stepsize_controller, discrete_terminating_event=event
                        , adjoint=BacksolveAdjoint())

the structure of the state is quite different than before. Here's the structure of state.y
Traced<ShapedArray(float64[279])>with<DynamicJaxprTrace(level=4/1)>
Traced<ShapedArray(float64[279])>with<DynamicJaxprTrace(level=4/1)>
(Traced<ShapedArray(float64[279])>with<DynamicJaxprTrace(level=3/1)>, Traced<ShapedArray(float64[279])>with<DynamicJaxprTrace(level=3/1)>, (Traced<ShapedArray(float64[278])>with<DynamicJaxprTrace(level=3/1)>,), WrapTerm(term=ODETerm(vector_field=None), direction=None))

I'm not sure how to properly interpret this, and how to restructure my terminating event function to handle this?

Ah, this looks like a bug. It looks like the terminating event is erroneously being passed to the backward pass. This shouldn't be necessary at all -- termination is a thing we only do on the forward pass; the backward pass just needs to integrate backwards from the time at which we terminated.
So to make this work, it looks like we would need to: (a) not pass the event to the backward pass at all, and (b) record the time at which we terminated, and pass that to the backward pass.
For now though, I'm just going to have this explicitly raise a NotImplementedError. There's almost no use-case for BacksolveAdjoint, after all.

Hi Patrick,
Thanks for the quick response! I am a bit confused -- I thought the motivation behind BackSolveAdjoint was memory efficiency? I only chose to implement it due to some OOM errors while scaling up this code. Do you have any other tips for memory management besides BackSolveAdjoint?

If you're looking to reduce memory usage then RecursiveCheckpointAdjoint(checkpoints=...), with an explicit number of checkpoints (as large as possible) is usually the best option.
BacksolveAdjoint got a lot of press a few years ago, but it's mostly a terrible method, as it gives inaccurate approximations to the gradients. In contrast, checkpointing gives you memory efficiency without sacrificing that.


Failure might be occurred when deploying harbor with tile

Always in the first deployment.

The root cause seems the timeout 10min is too short.
Although the bosh returns the error code, the harbor instance actually is already successfully started. We may need to increase the timeout threshold.
This failure depends on the opsman env If the network is poor, timeout will easily appear.
Issue is found in the local env and local VC. Not found in US env.

logs:
====
[Sun Dec 3 14:14:17 UTC 2017] Starting Harbor 1.2.0 at https://testing.harbor.vmware.com
[Sun Dec 3 14:14:17 UTC 2017] Check if docker daemon is running
[Sun Dec 3 14:14:17 UTC 2017] Docker daemon is running
[Sun Dec 3 14:14:17 UTC 2017] Loading docker images ...

[Sun Dec 3 14:19:49 UTC 2017] Waiting 120 seconds for Harbor Service to be ready ...
[Sun Dec 3 14:21:10 UTC 2017] Starting Harbor 1.2.0 at https://testing.harbor.vmware.com
[Sun Dec 3 14:21:10 UTC 2017] Check if docker daemon is running
[Sun Dec 3 14:21:10 UTC 2017] Docker daemon is running
[Sun Dec 3 14:21:10 UTC 2017] Loading docker images ...
[Sun Dec 3 14:21:46 UTC 2017] Error: Harbor Service failed to start in 120 seconds

Concourse ci log:
Task 608 | 14:08:07 | Preparing deployment: Preparing deployment (00:00:00)
Task 608 | 14:08:07 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 608 | 14:08:07 | Creating missing vms: harbor-app/1839982e-09a8-4ee9-ade4-984bd9d72223 (0) (00:01:34)
Task 608 | 14:09:41 | Updating instance harbor-app: harbor-app/1839982e-09a8-4ee9-ade4-984bd9d72223 (0) (canary) (00:09:40)
L Error: 'harbor-app/1839982e-09a8-4ee9-ade4-984bd9d72223 (0)' is not running after update. Review logs for failed jobs: dockerd, harbor
Task 608 | 14:19:22 | Error: 'harbor-app/1839982e-09a8-4ee9-ade4-984bd9d72223 (0)' is not running after update. Review logs for failed jobs: dockerd, harbor
Task 608 Started  Sun Dec  3 14:08:07 UTC 2017
Task 608 Finished Sun Dec  3 14:19:22 UTC 2017
Task 608 Duration 00:11:15
Task 608 error
Updating deployment:
Expected task '608' to succeed but state is 'error'
Exit code 1
===== 2017-12-03 14:17:05 UTC Finished "bosh2 --no-color --non-interactive --tty --environment=<IP_ADDRESS> --deployment=harbor-tile-82e62d65ebdecc1ca325 deploy /var/tempest/workspaces/default/deployments/harbor-tile-82e62d65ebdecc1ca325.yml"; Duration: 676s; Exit Status: 1
{"type": "step_finished", "id": "bosh.deploying.harbor-tile-82e62d65ebdecc1ca325"}
Exited with 1.
could not execute "apply-changes": installation was unsuccessful

Seems the timeout is always 10 minutes

The update watch time used in deploying harbor with tile seems is still the default values:

update:
canaries: 1
canary_watch_time: 30000-300000
max_errors: 2
max_in_flight: 1
serial: true
update_watch_time: 30000-300000

Add more logs:
harbor-app/168b4d39-0633-4268-984e-3ad52887dca3:/var/vcap/sys/log/harbor$ cat ctl.stdout.log
[Mon Dec 4 04:46:56 UTC 2017] Starting Harbor 1.2.0 at https://testing.harbor.vmware.com
[Mon Dec 4 04:46:56 UTC 2017] Check if docker daemon is running
[Mon Dec 4 04:46:56 UTC 2017] Docker daemon is running
[Mon Dec 4 04:46:56 UTC 2017] Loading docker images ...

[Mon Dec 4 04:52:38 UTC 2017] Waiting 120 seconds for Harbor Service to be ready ...
[Mon Dec 4 04:53:49 UTC 2017] Starting Harbor 1.2.0 at https://testing.harbor.vmware.com
[Mon Dec 4 04:53:49 UTC 2017] Check if docker daemon is running
[Mon Dec 4 04:53:49 UTC 2017] Docker daemon is running
[Mon Dec 4 04:53:49 UTC 2017] Loading docker images ...
[Mon Dec 4 04:54:35 UTC 2017] Error: Harbor Service failed to start in 120 seconds
Loaded image: vmware/nginx-photon:1.11.13
Loaded image: vmware/notary-photon:signer-0.5.1
Loaded image: vmware/clair:v2.0.1-photon
Loaded image: vmware/harbor-ui:v1.3.0-rc1-11-gd13321f
Loaded image: vmware/mariadb-photon:10.2.10
Loaded image: vmware/postgresql:9.6.5-photon
Loaded image: vmware/harbor-log:v1.3.0-rc1-11-gd13321f
Loaded image: vmware/harbor-jobservice:v1.3.0-rc1-11-gd13321f
Loaded image: vmware/harbor-adminserver:v1.3.0-rc1-11-gd13321f
Loaded image: vmware/harbor-db:v1.3.0-rc1-11-gd13321f
Loaded image: vmware/registry:2.6.2-photon
Loaded image: vmware/photon:1.0
Loaded image: vmware/notary-photon:server-0.5.1
Loaded image: vmware/harbor-db-migrator:1.3
[Mon Dec 4 04:57:15 UTC 2017] Launching docker-compose up ...
[Mon Dec 4 04:57:15 UTC 2017] Waiting 120 seconds for Harbor Service to be ready ...
[Mon Dec 4 04:57:34 UTC 2017] Harbor Service started successfully
[Mon Dec 4 04:57:34 UTC 2017] Add cron job for Harbor status check
[Mon Dec 4 04:57:34 UTC 2017] ctl start is successfully done!

From the two ctl.stdout.log files, we can see that the time interval of two "Starting Harbor 1.2.0" log are exactly the same 6 minutes 53 seconds. There are probably some timeout setting which causes the 2nd invoke of harbor clt start while 1st harbor ctl start is still running.

Per investigation,
The error happens when it takes very long to load docker images.
It seems in monit file, there is unnecessary checks:
  if failed port 80 for 3 cycles then restart
  if failed port 443 for 3 cycles then restart

Let's remove it and extend the timeout for ctl start
start program "/var/vcap/jobs/harbor/bin/ctl start" with timeout 600 seconds

extend it to 900 or 1200 seconds.
We believe with this change the "multiple start" problem will be fixed.
However there may be another issue that the canary timeout in bosh release will be overwritten by opsmanager to a hard coded 300 seconds
canary_watch_time: 30000-1200000

let's work with pivotal to see if there's a fix for that.

This issue is resolved by these patches. We're running CI to verify this fix.
Load harbor docker images in harbor job prestart instead of job start.
https://git.eng.vmware.com/harbor/habo/commit/1d29f2875baccd1dce880b0a92d3fb0e28fe94b7
This can solve the canary watch timeout issue #3735, because the time
of prestart execution is not included in the canary_watch_time.
This patch moves docker related utils and bosh env vars to
src/common/utils.sh, and the key code for loading images is
in jobs/harbor/templates/bin/pre-start.erb.
Remove the restart logic in harbor monit spec
https://git.eng.vmware.com/harbor/habo/commit/73e2c7102e5acd64726bf8426d63fbda8f34575a
Wait for dockerd to start after starting the dockerd daemon
https://git.eng.vmware.com/harbor/habo/commit/53598784faf3801c886bdf98517e1f987a701be2
dockerd daemon needs 1 or 2 seconds to start accepting requests,
so add this wait logic, otherwise other jobs' ctl start script which
trying to connect to docker daemon might fail.
Restart harbor job when detecting harbor service failure
https://git.eng.vmware.com/harbor/habo/commit/bd9318e59e48ee72cf16bcafdc03296a8c5cd5e2
Because removing the harbor pid file only leads to harbor job start
rather than job restart, so the abnormal harbor containers can not
be destroyed and recreated.

Verified in CI.


Update project name

Trébol could be implemented with others technologies just as fine as with Angular/Spring Boot, so I'd like to have a specific naming scheme for the repos that make up Trébol's core itself.
This one should be named "springboot-backend", and its maven package should reflect on that as well.

Addressed in #54


Quora icon

What service is this for?
Quora
What are their domain names?
quora.com
Do you have a link to a SVG of their logo?
Not applicable, available as a Font Awesome icon: https://fontawesome.com/icons/quora
Do you hate Quora?
Yes but it should be added anyway

"But vukky. you can add font awesome icons yourself"


Longtail fails updating UE4 version on some machines

On some machines, the following flow results in Longtail errors:

Fetch an UE4 version
Run UE4Editor/UE4Editor-Cmd (this will create some temp files within the UE4 folder structure)
Fetch an updated UE4 version

Step 3 will fail with weird internal errors in Longtail. Re-running the update step 3-4 times will eventually result in a successful fetch.
This might be a problem specific to the Longtail version we are using (0.0.36, over a year old). However, I cannot reproduce this on any of my own machines, so I can't check whether upgrading to a newer Longtail version helps.
Instead, the UpdateUE4 script should delete the entire UE4 folder before doing updates. For the time being. Longtail won't do incremental updates, but it will still do incremental downloads, and it would work well enough for now.

Solved in b44eb6d.


Initial work on allowing extra form fields in image uploader

Pull request checklist:

[ ] CHANGELOG.md was updated, if applicable

Description

Draft since 2022. Closing.


Add support for calculating decibels

Let's make it easy for users to retrieve the power of the microphone's audio signal. This would allow users to visualize a waveform as audio is being streamed to the Speech to Text service.
References:

Measure the power of a Signal
Root-mean-square level
StackOverflow Question
vDSP Reference
Audio Queue Levels

Closing in favor of #405.


There is a missing banner link to coral when requesting a new connector (hashtag1357)

hashtag What happened?
Using the Angular UI I selected new connector and the expected banner to link to the coral ui was missing.
hashtag What did you expect to happen?
The banner should be there to allow users to easily switch over to coral.
hashtag What else do we need to know?
Part of release 2.4.0
The new UI can till be used if the request is made through corals connectors page.

are you working on this? Otherwise I could take that one.

I can take this one unless you want to work on the angular side :) I'll knock it out after the release is finished


Use requirements.txt to lock dependencies

As part of #8592, I updated our dependencies in the azure-cli package to describe their semantic range of support. However, for the sake of actual deployments and test runs, we want to have a locked-down set of dependencies that we use. There are several options for tooling to help with this. However, the first step is to just have something. The simplest way to handle this is to provide requirements.txt based on the different platforms that we're targeting.

#9785 takes care of the first part of this effort, providing the concrete requirements. The second part will be adjusting our installers/test infrastructure to use the appropriate requirements.*.txt file.

add to S164.

RPM, Deb and Docker already use requirements.*.txt file now. MSI and Homebrew still use setup.py for dependencies. For pip, we can use pip install -r requirements.txt.
I will evaluate the work to change MSI and homebrew.

add @haroldrandom to check CI and tests.

azdev setup, CI and packaging except pypi are using requirements.txt to lock dependencies now. This issue can be closed.
Documentation on pip install with requirements.txt or modify the linux install script will be another discussion.


Keyup enter event to change tab

Hi there. Maybe this is a simple question but I'm trying to apply the keyup enter event in order to change to the next tab, but I can't find the way. I was trying to implement it in the template footer but there is no way:
<template slot="footer" slot-scope="props">

           <wizard-button  v-if="props.activeTabIndex > 0 && !props.isLastStep" :style="props.fillButtonStyle">Previous</wizard-button>

          <wizard-button v-if="!props.isLastStep"@keyup.enter="props.nextTab()" class="wizard-footer-right" :style="props.fillButtonStyle">Next</wizard-button>

          <wizard-button v-else @keyup.enter="alert('Done')" class="wizard-footer-right finish-button" :style="props.fillButtonStyle">  {{props.isLastStep ? 'Done' : 'Next'}}</wizard-button>

</template>

thanks in advance.

Hey @richardgvas
If you want to use the enter event, you need a form and use @submit.prevent
https://codesandbox.io/s/ry4q2o96no
Enter event is supported by form and inputs so there are not many alternatives here


Questions about implementations

First of all thank you for creating those videos and articles explaining the details.
It is very useful for reference.
However after reading and looking for a while I still cannot confirm some detail.
1)Is the Acoustic feature in the generator,1d or 2d.
What is it,mel spectogram extracted from natural speech?Or text to mel spectogram(from other framework?)
I noticed there is some notes you used like:Conditioned on 1×F0.
From what i have seen it is like a processed mel spectogram.But i cannot confirm.
2)How to calculate pitch dependent dilated factor.
From the video and paper I see the explainations and the derivations of it.
It is from DCNN.There is a equation for it.just change the definition of d to be a runtime calculate variable
d'=1*ET.How to calculate ET,or pitch dependent dilated factor?
I think you mention it has some properties relates to the wave frequency and periodic,but i cannot visualize it.
Some inherited reference used details.
I know you specifically mentioned most of the changes you made if not all.
However,I am kinda dumb and are uncertain about some details.So just to ask it ahead of time to not fail.
1)The residual block.From the paper diagram is it also like quasi periodic?like adaptive/fixed?
Is it just a no edit copy from P.W.G.?
How does the residual block affect the generator?
2)Is the generated speech and discriminator exactly the same as P.W.G.?Just to confirm.
If it is the same then I will find PWG implementations and study on it.
After all these question you may be curious.Why need to know.Why not just clone repo setup it.
Sadly,I am kinda bored and decided to make it consistent and not dependent on python libraries.
So,I am porting it to java or javacpp and yes sadly i need to implement practically everything other than maybe FFTW or matrix calcs.
Any of your suggestions and time taken for reply is greatly appreciated and I hope you can have a nice day.

1)Is the Acoustic feature in the generator,1d or 2d.
What is it,mel spectogram extracted from natural speech?Or text to mel spectogram(from other framework?)
I noticed there is some notes you used like:Conditioned on 1×F0.
From what i have seen it is like a processed mel spectogram.But i cannot confirm.
The acoustic features are extracted by WORLD (python: https://github.com/JeremyCCHsu/Python-Wrapper-for-World-Vocoder, C: https://github.com/mmorise/World).
WORLD encodes speech into spectral envelope (sp), pitch (F0), and aperiodicity (ap) features.
I further transfer the sp into mel-cepstral by SPTK (python: https://github.com/r9y9/pysptk)
Therefore, I can directly control the pitch using scaled F0 (e.g. 1xF0)

2)How to calculate pitch dependent dilated factor.
From the video and paper I see the explainations and the derivations of it.
It is from DCNN.There is a equation for it.just change the definition of d to be a runtime calculate variable
d'=1*ET.How to calculate ET,or pitch dependent dilated factor?
I think you mention it has some properties relates to the wave frequency and periodic,but i cannot visualize it.
Please refer to equation (7) in "https://arxiv.org/pdf/2007.05663.pdf"
ET = Fs/(F0 x a)
'Fs' is the sampling rate, 'F0' is the pitch of the current frame, and 'a' is a dense factor (constant, I set it 4 in the repo).
For the meaning of 'a', please refer to fig. 4.

1)The residual block.From the paper diagram is it also like quasi periodic?like adaptive/fixed?
Is it just a no edit copy from P.W.G.?
How does the residual block affect the generator?
Yes, the fixed residual block is the same as the residual block in Parallel WaveGAN.
The adaptive residual block only changes the DCNN layer to the PDCNN layer.

2)Is the generated speech and discriminator exactly the same as P.W.G.?Just to confirm.
If it is the same then I will find PWG implementations and study on it.
The discriminator is exactly the same as ParallelWaveGAN, and I only change the generator of Parallel WaveGAN.

If you have any questions, please feel free to ask me.

I tried re implementing it into java.
so i did some basic syntax translation.
However in java the static type is kinda harsh since python does not have hard static type.
The code is all var with hints given by dev.
But generally the output is a tensor.By tensor there are many rank of tensor.
Generally,dimensions.So can you help me deduce the dimensions for a few functions in some of the code?
forward function in FixedResidualBlock or fixed block.The x and s.output
forward in AdaptiveResidualBlock.The x and s.output

these both are output of the conv1x1_skip variable which is also Conv1d1x1(x,y,bias) output.
and yes i do not know the output of it.it seems like a class that create a new conv1d1x1 obj.

ResidualParallelWaveGANDiscriminator forward function,by(b,1,t) can i assume it is 3d?
Stretch2d,again the forward function have a tensor.
Of course there are many others but I think others should be deducible with these initial info.
Any suggestion to quickly get the infos?I suppose one of the way is to full setup edit the code to print info but.
I think making it work will probably take some time.Not sure about the setup needed(need to check).

For the forward function in each class, I have provided the information about the input/output tensor. For example, in the "AdaptiveBlock" class in the "residual_block.py", I have provided the following information
"""Calculate forward propagation.
    Args:
        xC (Tensor): Current input tensor (B, residual_channels, T).
        xP (Tensor): Past input tensor (B, residual_channels, T).
        xF (Tensor): Future input tensor (B, residual_channels, T).
        c (Tensor): Local conditioning auxiliary tensor (B, aux_channels, T).

    Returns:
        Tensor: Output tensor for residual connection (B, residual_channels, T).
        Tensor: Output tensor for skip connection (B, skip_channels, T).

"""
(B, residual_channels, T) denotes a 3-dimensional tensor. B is batch size and T is data length.
For those native functions from PyTorch such as "Conv1d", I may not provide the input/output information.
That information can be found on the PyTorch website. (https://pytorch.org/docs/stable/generated/torch.nn.Conv1d.html)
However, I think it is possible to find some open-source implementation already porting PyTorch into Java.

Ah,I see it thanks.
I can (B, residual_channels, T)as 3d tensor right?
In which the B is a input data stream or array,or list. and T is a tensor or some object?
I am also quite curious on you take of porting.From your view,what would be a ideal library to use as porting.
I knew djl and dl4j both have some pytorch support.
Where djl is specialized in supporting the native c lib to java.
I currently just uses the jblas and apache common fast maths lib these two lib for porting it.
Which is honestly quite stupid,and are considering to utilize wheels invented to cut some porting time.

Yes, (B, residual_channels, T) denotes a 3d tensor.
B->mini batch_size.
T->data length.
Take the training process as an example, according to the config file (https://github.com/bigpon/QPPWG/blob/master/egs/vcc18/conf/vcc18.QPPWGaf_20.yaml), we know that the batch size is 6 and the batch length is 25520.
Therefore, the "B" is 6 and the "T" is 25520 for the training tensor in the training stage.
"xC" will be a 6 x 64 x 25520 tensor.
I never tried to port PyTorch into Java, so I have no experience with using these libraries.
Sorry, I don't have any informative comments on these libraries.

thanks for you infomation and guidance these few days.
I am considering to use DJL to port.
Will come back to you when I find bigger problem.
Or Maybe when I finished porting it.^^


Missing initialization of variable

Library name and version
Azure.Search.Documents
Query/Question
In the code there is a line
ReadOnlyMemory vectorizedResult = VectorSearchEmbeddings.SearchVectorizeDescription;
VectorSearchEmbeddings is not found any where earlier, where is it instantiated ?
Environment
https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/search/Azure.Search.Documents/samples/Sample07_VectorSearch_UsingSemanticHybridQuery.md

Thank you for your feedback.  Tagging and routing to the team member best able to assist.  Please expect and forgive delays during the US holiday season.

Hi @nssidhu, In the sample, we have mentioned that we are using hardcoded embeddings to query vector field. We've referenced VectorSearchEmbeddings.SearchVectorizeDescription simply as a way to demonstrate that you need to replace it with your own embeddings. If you want to generate embeddings for your text data, please refer to the Get Embeddings using Azure.AI.OpenAI section.
I'll also update the samples to make this more clear. Thank you!

Hi @nssidhu, I've improved the samples for better clarity. Instead of using hardcoded embeddings, I am now utilizing OpenAI to get the embeddings. You can find the updated samples here.


[Release fhir 1.0.4] Prepare for release

Purpose

$Subject.

Closing this as https://github.com/wso2-enterprise/open-healthcare/issues/1099 will address this with a major version bump


Dopex Arbitrum: Indexing Errors

https://okgraph.xyz/?q=messari%2Fdopex-arbitrum
A non-deterministic fatal error occured at block 199577827: Failed to transact block operations: internal constraint violated: Batches must go forward. Can't append a batch with block pointer #199577827 (9b90ed286057a8af2caf2a007284d034c92768eade65eaa3dc81d85a62f9914c) to one with block pointer #199577907 (bc4a1de3c7dbb8d70a077df901ab7ce4438066b15be6352eb567647c25becf66)

@dhruv-chauhan A lot of the abritrum subgraphs had a similar indexing error to this, but the next day they seemed to fix themselves. So this may resolve itself, but I figured I would raise it incase it does not.

Most probably an intermittent issue of the graph node, which fixed itself.


OAuth 1.0 - incorrect signature if special characters in query

Problem
Request params are being url-encoded twice during OAuth signature calculation. OAuth specification requires only signle url-encoding: https://oauth.net/core/1.0a/#rfc.section.9.1. As result, Insomnia cannot work with our OAuth-protected resource.
Details
I issue a GET on this url:
https://ehdjo.brndc83d32b6-cad3dc.aqa.int.zone/aps/2/resources?a c
Insomnia generates the following authorization header:
Authorization: OAuth oauth_consumer_key="VcaXEa9a5tGp3vROx6xNLgVnv9LXT4N8", oauth_nonce="UnhVTy", oauth_signature="GET%26https%253A%252F%252Fehdjo.brndc83d32b6-cad3dc.aqa.int.zone%252Faps%252F2%252Fresources%26a%25252520c%253D%2526oauth_consumer_key%253DVcaXEa9a5tGp3vROx6xNLgVnv9LXT4N8%2526oauth_nonce%253DUnhVTy%2526oauth_signature_method%253DPLAINTEXT%2526oauth_timestamp%253D1506018545%2526oauth_version%253D1.0", oauth_signature_method="PLAINTEXT", oauth_timestamp="1506018545", oauth_version="1.0"
Please note I've selected signature method = plaintext for debug purposes.
From this signature we can see that SignatureBaseString contained urldecode("a%25252520c") =  "a%252520c" substring corresponding to request parameters. This means that original request parameters "a%20c" was url-encoded twice:
urlencode(urlencode("a%20c")) = "a%252520c"

Insomnia Version: ___
5.16.6
Operating System: ___
Windows 10 x64

The reason for this is likely not OAuth-related, but instead has to do with Insomnia's preference to automatically encode special characters.
Does it work if you instead use the unencoded value?
https://ehdjo.brndc83d32b6-cad3dc.aqa.int.zone/aps/2/resources?a c


Add some unit tests of groupby

Coverage here is pretty basic but this gets us a bit of test
coverage for basic groupby behaviors.

A test for #103 would be  nice. But anyway, 👍


Likely bug in Request.Validate() - any request with an ID will only validate the ID

Relevant code: https://github.com/dogmatiq/harpy/blob/b8df01b455faf5300cf6ef5ab8334f07e16fe113/request.go#L81-L83

I imagine the way that this slipped through tests was that each test request omitted the ID in order to be as simple an example as possible.

Note to self: also check for len(r.ID) == 0, not just nil.


Endpoint /api/files/bulkRemove returns success even if some files were not deleted because of permissions

The endpoint properly respects permissions and does not delete files that it doesn't have permission to delete, but it returns status:success even if some files were not deleted. It might be better to return a list of files that were deleted a list of files that were not deleted.
https://github.com/clowder-framework/clowder/pull/153/files

@lmarini
I can fix this.
In addition to 1. files deleted 2. files not deleted I might also include 3. file ids not found.


ENOENT error code explaination

Is there detail for the following error:
{ [Error: ENOENT, open 'cert.pem'] errno: 34, code: 'ENOENT', path: 'cert.pem' }

@cthomaschase: where can I set my certificate file location, I am facing same error


Notebooks 2.0 // Frontend // Namespace selector

Provide a React context with namespace selector (namespace dropdown):

Please review those mockups before implementing them.

/assign @ederign

I'm assign this task for myself because I'm working on it's proposal.

/assign


I don't know if it's a fit, but I basically want reactive programming, where an action pushed a value through a stack of coroutines top to bottom.
The basic idea is to make it easy to throw a bunch of asio thingies together  and build a stream from it.

Allocators

This works by matching the coro argument list to the promise, and using std::allocator_arg somewhere in the argument list of a coro type.
template<typename Executor, typename Allocator>
struct promise_frame_base_allocator : promise_frame_base<Executor>
{

    template<typename Class, typename Exec_, typename ... Args>
    void* operator new(std::size_t size, Class&&, Exec_ &&, std::allocator_arg_t, const Allocator & allocator, Args&&...)
    {
        return _new_impl(size, allocator);
    }

    template<typename Exec_, typename ... Args>
    void* operator new(std::size_t size, Exec_ &&, std::allocator_arg_t, const Allocator & allocator, Args&&...)
    {
        return _new_impl(size, allocator);
    }

    template<typename ... Args>
        requires (not std::convertible_to<Args, std::allocator_arg_t> && ...)

    void* operator new(std::size_t size, Args && ...)
    {
        return _new_impl(size, Allocator());
    }

    void operator delete(void * ptr, std::size_t size)
    {
        constexpr auto al = alignof(Allocator);
        std::size_t size_rounded_to_alignment =
            ((size % al) > 0)
            ? (size + (al - (size % al)))
            : size;

        std::size_t allocator_offset = size_rounded_to_alignment;
        void* a = static_cast<char*>(ptr) + allocator_offset;
        Allocator &alloc = std::move(*static_cast<Allocator*>(a));

        Allocator alloc_copy = alloc;
        alloc.~Allocator();
        alloc_copy.deallocate(allocator_offset + sizeof(Allocator));
    }

    using allocator_type = Allocator;
  private:

    static void* _new_impl(std::size_t size, const Allocator & allocator)
    {
        constexpr auto al = alignof(Allocator);
        std::size_t size_rounded_to_alignment =
            ((size % al) > 0)
            ? (size + (al - (size % al)))
            : size;

        std::size_t allocator_offset = size_rounded_to_alignment;
        Allocator alloc = allocator;

        void *ptr = allocator.allocate(allocator_offset + sizeof(Allocator));
        new (static_cast<char*>(ptr) + allocator_offset) Allocator(std::move(alloc));
        return ptr;
    }

};

//this should probably us an allocator_trait instead
template<typename Executor>
struct promise_frame_base_allocator<Executor, std::allocator<void>> : promise_frame_base<Executor>
{
    template<typename ... Args>
    promise_frame_base_allocator(Args && ...args) : promise_frame_base<Executor>(std::forward<Args>(args)...) {}

    auto await_transform(this_coro::allocator_t) noexcept
    {
        return detail::co_await_getter{[this]{return get_allocator();}};
    }

    using allocator_type = std::allocator<char>;
    allocator_type get_allocator() {return std::allocator<char>{};}
};

Additonally one could check if a given input parameter has a get_allocator function and then propagate it like so:
stream<int> foo(stream<int> s); //will use s.get_allocator()

The massive disadvantage of that is that this would propagate the vector allocator:
stream<int> foo(std::vector<int>)_;

Integration into asio.

Basically what I want is a promise type that has two things; wait for next value & cancel operation. Now currently (I talked to Chris, he said he'd be open to adding that option to asio) this isn't possible, but what we could do is something like this:
promise<void> tick(asio::steady_timer & st)
{
     return  st.async_wait(use_promise([&]{st.cancel();});
}

Basically this means that a promise should be able to be able to be used as a callback+cancel.

Cancellation

Cancellation should propagate upwards through a stack of promises/streams:
promise<void> first()
{
   auto tick = [](asio::steady_timer & st) -> promise<void>
  {
     return  st.async_wait(use_promise([&]{st.cancel();});
   };
   asio::steady_timer st;
   co_await tick(st);
}

auto f= first();
f.cancel(); //cancels the timer

Exceptions & Errors

I would like exceptions to be optional, every coro type should have a Error type parameter. If it's a std::exception_ptr it's an exception, otherwise a value. These value should however be propagated:
promise<int, error_code> p
{
    co_await [] () -> promise<int, error_code>
    {
        co_return error_code(42); //cancel the stack of promises
    }();
    //never gets called
}

Streams

So from my perspective a stream is like a promise that can also yield values, so it has three (main) parameter types :

Yield
Return
Error

For an expression of co_await stream I would propose the following type deductions:

Yield
Return
await

T
void
optional

void
U
variant<monostate, U>

T
U
variant<T, U>

asio again

While it makes perfect sense to have co_await stream/promise within a coro, I would also like to have a non-coro interface for a stream. I would model it after asio, i.e. have a async_wait((T, U, Error) =>) and cancel member function.

Great to see your thoughts, thanks for the input.
These are just first impressions -- probably after a round or two of this we should split these into separate issues to make discussion and tracking a bit easier.

Is it only me who finds the std::allocator_arg really ugly? I think it would be neat to have a concept here as well that can be used, but I agree it's important to separate the coroutine stack allocator from one for other types or all sorts of bad things could happen. I have a design for a slab like allocator for use in things like networking and parsing which I'll try to write up at some point.
I was under the impression that ASIO already had some coroutine support, or does it still require something else to provide complete support?
Cancellation is not something I've given much thought to, but your idea looks sound.
(first 4) I don't know much about writing code in an environment where exceptions are not used. Is it enough to just not use them, or does the code have to be more explicit about things? I'd hate to have to add in a ton of conditional compilation support with #ifdef everywhere.
(second 4) There's a technical reason for the difference between a stream and generator -- the generator can be used from a non-coroutine, but it's implementation is restricted to only co_yield. For the stream the coroutine that generates the stream can use co_await, but this means they can also only be used from coroutines as the next value must be co_awaited in turn. Like the generator, the streams here don't co_return values as there's no clear way to get at them (although I think a suitable API is maybe clearer in the stream case, but only maybe...). I think a stream that doesn't co_yield is just a task.
(your 5) This sounds interesting. It would be good to have another way to interface between coroutines and normal functions. I guess a callback like thing could work -- I'd have to think about this

It is incredibly ugly. Implicit contextualization is however less acceptable. We could introduce an allocator concept and the just take that when passed in, instead of the tag.

It does, but it is incredibly fenced off.  But this can be wrapped in a throw_X function that calls terminate with exceptions disabled.

In theory we could make a stream do both, IF the ultimate worker object (e.g. the timer) has a sync function. So instead of calling wait or async_wait the coro picks depending on whether it was called with wait or async_wait. To me this seems like a lot of complexity for no real benefit, since I am only using this on asio, but it would be an option. Overhead shouldn't be too bad.

promise<void> foo() { co_await promisify(timer); };

foo.wait(); // --> timer.wait()
foo.async_wait(); // --> timer.async_wait()
foo.cancel(); //--> timer.cancel()

Then, there wouldn't be a difference between generator & stream.

Well I would just use asio::async_completion for that.

I've been thinking about this, and I was thinking that the exception stuff should be clear enough to start to draw up a bit more of a plan, but the problem with it is that if we want to be able to run tests of a non-exception version (and I'm pretty sure we do) then the test library has to support no exceptions first, which is a bit more difficult. The test library uses exceptions to report test failures so I'll have to think about what a non-exception API might look like, and the problem is the if statements that would be required -- which ironically could be turned into co_await if we were to mix the coroutines in

I don't think I get the problem with the test library, but imo we'd be fine to define the error-type and default it to std::exception_ptr for the time being. I just would like the no-except version to be possible later on.
As far as I played with coros so far, you're most likely gonna put the exception into an exception_ptr anyhow and then rethrow it, so I don't think it would be that different for another type.

Oh another issue: stacking coros, is one I have thought about. boost.asio is doing this by having a "thread" which works like a linked list. co_await will cause the awaitable to be pushed on top of the list and the thread you executed just calls resume on the top one until the stack is empty. That does work, but we could also do another thing: call the next coroutine in await_ready and then suspend if it suspends or just continue without suspension if it completes.

Never mind the last commentary, MSVC doesn't implement that correctly. The correct solution is the return the callee coro_handle from the initial await_suspend and the one of the caller in the await_suspend in the final_suspend.

I've started work on an IO library now, which is at https://github.com/Felspar/io I'm about to land a PR that will make exceptions from IO optional (at the cost of a small amount of extra syntax). I think a similar thing could be done here so that exceptions wouldn't have to be thrown at a co_await etc. and an overload of co_return would allow exceptions to be returned as error values. It's not a great way because examining exceptions isn't very easy without an ability to throw them, but at least their presence can be detected without breaking compatibility with the core language.

The IO library now has optional exceptions and allocators are supported here.


Evaluate Atlassian Clover

Feature Request
Desired behaviour
This might be a replacement for Cobertura which has issues with Java 8
Take a look through https://confluence.atlassian.com/clover/clover-documentation-home-71598318.html and try out the Maven plugin to see if this is a good choice. Also see how to review the results and/or whether it is compatible with coveralls.io

Probably waiting on https://github.com/trautonen/coveralls-maven-plugin/issues/36

jacoco seems to work well


Start of Sidebar Structure

This PR modifies the config file in order to create a sidebar that is more structured. You'll note that it removes the headers being expanded for each selected file.
I also used the toml version of the config file because I think when those get big the json format is super hard to parse for the eye, but I also added the json and yaml format for the config files. All three produce the same result. I'm happy to use any of them, I just wanted to point out that we can use a different one if we want.
This will allow us to compare apples to apples a bit more when looking at GitBook.
Cheers!

This sounds good!!
But I'm a bit too lazy [and not even sure how to do it properly] to compile this branch of a fork of a repo? :laughing:
Could you please save me some trouble and post a screen shot?

@thunderBiscuit Why is the config present in three different formats?
@MaxHillebrand I just merged this PR so that you can take a look at the preliminary doc site.

I just wanted to make sure I wasn't pushing my preferred choice on other people... For now the config.toml version is the one used to build the website, but this way the few contributors can have an early discussion on which format to go forward with.


[c++] Fix dump_model() information for root node

This PR corrects the output of dump_model() and other dump-related functions like trees_to_dataframe(). There are 2 fixes implemented:

The current Tree::Split implementation incorrectly saves the old leaf output value in the internal_value_ array when called on the root node. This in turn makes inspecting the whole training process from python incomplete.

Before:
(Pdb) booster_.trees_to_dataframe()
     tree_index  node_depth node_index left_child right_child parent_index  ... decision_type  missing_direction missing_type     value weight count
0             0           1       0-S0       0-S1        0-S2         None  ...            ==              right         None  0.000000      0   200
1             0           2       0-S1       0-S5        0-S4         0-S0  ...            <=               left         None  0.106573    113   113
2             0           3       0-S5       0-L0        0-L6         0-S1  ...            ==              right         None  0.082122     56    56
3             0           4       0-L0       None        None         0-S5  ...          None               None         None  0.064612     26    26
4             0           4       0-L6       None        None         0-S5  ...          None               None         None  0.097297     30    30

After:
(Pdb) booster_.trees_to_dataframe().head()
   tree_index  node_depth node_index left_child right_child parent_index  ... decision_type  missing_direction missing_type     value weight count
0           0           1       0-S0       0-S1        0-S2         None  ...            ==              right         None  0.081757    200   200
1           0           2       0-S1       0-S5        0-S4         0-S0  ...            <=               left         None  0.106573    113   113
2           0           3       0-S5       0-L0        0-L6         0-S1  ...            ==              right         None  0.082122     56    56
3           0           4       0-L0       None        None         0-S5  ...          None               None         None  0.064612     26    26
4           0           4       0-L6       None        None         0-S5  ...          None               None         None  0.097297     30    30

https://github.com/microsoft/LightGBM/issues/5962

Currently the CI is not passing as #6574 is blocking.

I am open to ideas of ways to test related functionalities.

Tests should now be sufficient for the change.

@jameslamb
Could you take a look at the PR, now that the CI is passing?

Thank you for taking the time to look into the PR and linking a relevant issue.

I'm not sure if this will correctly handle these cases:

custom init_score provided (via Dataset)
boost_from_average=False passed

I think those cases are handled, I remade the test to boost from average.

@neNasko1 could you also look at #5962 and let us know if you think this change would fix the issue @thatlittleboy reported there?

I took the liberty to merge @thatlittleboy's WIP code into mine, additionally fixing the issues that they reported. I will also change the description of the PR to reflect both of the fixes.

I am not the best first reviewer for this. As I said in my comment 2 weeks ago (https://github.com/microsoft/LightGBM/pull/6569#pullrequestreview-2216131312), I'm hoping that @shiyu1994 or @guolinke will have time to help with a review. Thanks for your contribution and sorry for the delayed feedback.

@jameslamb
Just to recap the tests/python_package_test/test_dask.py seems to have previously been a no-op since the 2 models produced with and without an init scores are the same for the classifier case. This however is not related to the current changes. Can you tell me whether I am missing something?

the tests/python_package_test/test_dask.py seems to have previously been a no-op since the 2 models produced with and without an init scores are the same for the classifier case.

I'll investigate this when I can, hopefully in the next few days. In the interim, you can help move this forward by resolving merge conflicts and pulling in the latest changes on master.

@jameslamb can you submit a final review on the change, so that we can merge it?
Sorry for the caused inconvenience!

I will look when I can. I have spent most of my limited open source time in the last few weeks investigating and fixing multiple difficult, time-sensitive CI issues in this project, and there is yet another one that is still not done and a primary focus for me right now (#6651).
If @StrikerRUS has time to re-review the commits and comments you've pushed since his approval, and if he approves, then my review can be dismissed and this can be merged without another review from me. Otherwise, you will have to be patient a bit longer.

/gha run r-valgrind

I will look when I can. I have spent most of my limited open source time in the last few weeks investigating and fixing multiple difficult, time-sensitive CI issues in this project, and there is yet another one that is still not done and a primary focus for me right now (https://github.com/microsoft/LightGBM/pull/6651).

Sorry for making it seem like there is some rush around this. I understand that this project is run mainly by volunteers and I do not want to harass any of the maintainers.

Thanks. I've left some minor suggestions for your consideration, around making the tests stricter and easier to understand.
I've also triggered valgrind checks on this branch, to ensure no new memory-management issues have been introduced by this PR.

Thanks for the suggestions, all of them are reasonable and are now merged. I grouped them into one commit as I wanted to test if everything is okay.
Again, thanks to everyone for the time spent!

@jameslamb

With init_score not provided, the root node's value is a non-0 value, and the test reliably and consistently fail.

Is this related to boost_from_average?

When there is a init_score then boost_from_average does not occur.
After training the first trees(iteration=1) we then add the average to them.

I retract my comment of the test being a no-op previously, however after this fix there is a need for a change in this test.

Thanks, changes look great. For your other questions, let's please stay in the thread you're quoting, so the conversation can all be grouped together. I've responded there: https://github.com/microsoft/LightGBM/pull/6569#discussion_r1792672361

This is looking good, thanks very much for the help!

Thanks for the all the time spent! Glad to see it merged :rocket:


ui: Update dependencies for the React UI

The node dependencies are quite out-of-date for the React UI, let's update to the latest version we can without breaking anything.

I would love to take this up! This issue would involve upgrading all deps to their latest releases and dealing with any code changes that requires, right?

@prmsrswt  can I work on it?

@saswatamcode yep that's exactly what is needed.
@ruddi10 Let's wait for @saswatamcode first and try to not have duplicated effort.

@prmsrswt would love it if you can take a look at this and suggest some changes. I've upgraded the dependencies as much as possible, but some tests fail due to the upgrades and I'm not familiar with the testing architecture of the project yet. Also certain caveats exist due to these changes.

Closing as it got fixed in #3813


Pattern matching

See e.g. https://www.python.org/dev/peps/pep-0635/

This is possible to do using dicts, so I'm not sure this is a high priority.

Binding to variables isn't possible in dicts. But it might be too big of a feature for now.