DB2 ODBC tests are not executed and fail if activated

First of all it looks like to be skipped on TRAVIS=true and in all other cases running.
But it don't run if TRAVIS is not defined. Only if  setTRAVIS=false this 2 tests will be executed.
(at least with CMake 3.11.3 on macOS)
If enabled and configured you will get 4 issues:
15/22 Test #15: soci_odbc_test_db2 .................***Exception: Child aborted  8.13 sec

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
soci_odbc_test_db2 is a Catch v1.9.6 host application.
Run with -? for options

-------------------------------------------------------------------------------
Use and into
  Indicators work correctly more generally
-------------------------------------------------------------------------------
/Users/heikorabe/Github-private/soci/tests/common-tests.h:688
...............................................................................

/Users/heikorabe/Github-private/soci/tests/common-tests.h:694: FAILED:
  CHECK( ind == i_null )
with expansion:
  0 == 1

/Users/heikorabe/Github-private/soci/tests/common-tests.h:699: FAILED:
  CHECK( ind == i_null )
with expansion:
  0 == 1

/Users/heikorabe/Github-private/soci/tests/common-tests.h:705: FAILED:
explicitly with message:
  expected exception not thrown

ASAN:DEADLYSIGNAL
=================================================================
==36092==ERROR: AddressSanitizer: BUS on unknown address 0x608700011ca6 (pc 0x000101505e5c bp 0x7ffeeef3db40 sp 0x7ffeeef3d1c0 T0)
    #0 0x101505e5b in soci::odbc_vector_into_type_backend::post_fetch(bool, soci::indicator*) (libsoci_odbc.4.0.dylib:x86_64+0x46e5b)
    #1 0x1015a3785 in soci::details::vector_into_type::post_fetch(bool, bool) (libsoci_core.4.0.dylib:x86_64+0x34785)
    #2 0x10170f9ad in soci::details::statement_impl::post_fetch(bool, bool) (libsoci_core.4.0.dylib:x86_64+0x1a09ad)
    #3 0x101710486 in soci::details::statement_impl::fetch() (libsoci_core.4.0.dylib:x86_64+0x1a1486)
    #4 0x100f5ba65 in soci::statement::fetch() (soci_odbc_test_db2:x86_64+0x1002a5a65)
    #5 0x100d39aa3 in soci::tests::test_cases::(anonymous namespace)::____C_A_T_C_H____T_E_S_T____16::test() (soci_odbc_test_db2:x86_64+0x100083aa3)
    #6 0x1011a4bc7 in Catch::MethodTestCase<soci::tests::test_cases::(anonymous namespace)::____C_A_T_C_H____T_E_S_T____16>::invoke() const (soci_odbc_test_db2:x86_64+0x1004eebc7)
    #7 0x100ce2d6b in Catch::TestCase::invoke() const (soci_odbc_test_db2:x86_64+0x10002cd6b)
    #8 0x100f03e0d in Catch::RunContext::invokeActiveTestCase() (soci_odbc_test_db2:x86_64+0x10024de0d)
    #9 0x100f0232a in Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) (soci_odbc_test_db2:x86_64+0x10024c32a)
    #10 0x100cc185b in Catch::RunContext::runTest(Catch::TestCase const&) (soci_odbc_test_db2:x86_64+0x10000b85b)
    #11 0x100cbea55 in Catch::runTests(Catch::Ptr<Catch::Config> const&) (soci_odbc_test_db2:x86_64+0x100008a55)
    #12 0x1010d258d in Catch::Session::run() (soci_odbc_test_db2:x86_64+0x10041c58d)
    #13 0x100ea362f in Catch::Session::run(int, char const* const*) (soci_odbc_test_db2:x86_64+0x1001ed62f)
    #14 0x100ea3369 in main (soci_odbc_test_db2:x86_64+0x1001ed369)
    #15 0x7fff68822014 in start (libdyld.dylib:x86_64+0x1014)

==36092==Register values:
rax = 0x0000608700011ca6  rbx = 0x00007ffeeef3d2c0  rcx = 0x0000608700011ca6  rdx = 0x00001c10e0002394
rdi = 0x0000000000000060  rsi = 0x00001c0600006e00  rbp = 0x00007ffeeef3db40  rsp = 0x00007ffeeef3d1c0
 r8 = 0x00001fffddde7a38   r9 = 0x0000000101541f57  r10 = 0x00007ffeeef3d1e0  r11 = 0x00007ffeeef3d1f0
r12 = 0x00007ffeeef3d280  r13 = 0xf8f8f2f2f2f2f801  r14 = 0x00007ffeeef3d200  r15 = 0x00007ffeeef3d240
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: BUS (libsoci_odbc.4.0.dylib:x86_64+0x46e5b) in soci::odbc_vector_into_type_backend::post_fetch(bool, soci::indicator*)
==36092==ABORTING
-------------------------------------------------------------------------------
Repeated and bulk fetch
  std::string
-------------------------------------------------------------------------------
/Users/heikorabe/Github-private/soci/tests/common-tests.h:799
...............................................................................

/Users/heikorabe/Github-private/soci/tests/common-tests.h:799: FAILED:
  {Unknown expression after the reported line}
with expansion:

due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal

===============================================================================
test cases:   4 |   2 passed | 2 failed
assertions: 118 | 114 passed | 4 failed

I would rework this and file a PR next.
I would also update missing ODBC test better enabled by using WITH_ODBC_DB2=ON instead of buggy dependency of TRAVIS.

The DB2 ODBC Driver is the problem. It doesn't work correct in 64bit.
https://bugs.php.net/bug.php?id=54007

A workaround that we successfully used in production for several years includes casting certain 64-bit ODBC values down to 32-bits when comparing for SQL_NULL_DATA. It essentially changes PHP to conform to an outdated ODBC spec that IBM requires. It doesn't break other "real" 64-bit ODBC drivers as long as you aren't pulling back individual values > 4GB.

Indicator size in 64bit should be long (64bit) but IBM uses int (32bit) in 64 bit driver.
So vectors doesn't fit and 64bit long is not SQL_NULL_DATA (-1) !
I will think about it, if it's possible to do that, cause use/into have to be different if DB2 ODBC is used than any other driver!

@hrabe As responded to your post on https://sourceforge.net/p/soci/mailman/message/36343256/

here are differences in ODBC drivers implementations, some features behave differently, some are buggy.

If SOCI can cope with ODBC drivers incompatibilities internally, without affecting the public interface, then I don't mind such fixes.
Otherwise, I'm personally would simply drop support for such broken drivers - with clear notice in release notes, documentation, etc.

Keep in mind, most of us rely heavily on the CI builds - we don't run all RDBMS-s locally, despite there is Vagrant and docker setup. Testing big-arse RDBMS like Oracle and DB2 on the free CI services has proved as maintenance PITA, sometimes. So, extending number of builds for those giants may introduce maintenance burden, which I'd be unable to take care of.


error in WifPrivateKeySerializer::serialize() with a Decred address.

Decred (and maybe others?) uses 2 byte address prefixes rather than 1 byte.
The line that fails is:
Buffer::hex($network->getPrivByte(), 1, $math),
It works upon changing to:
Buffer::hex($network->getPrivByte(), strlen($network->getPrivByte()), $math),
what do you think?  I can submit a pull request if helpful.

A PR would be great! I think we can drop setting the length since we know it's coming from a NetworkInterface already? We did the same to support zcash addresses IIRC.

oh, sorry I see that WifPrivateKeySerializer::serialize() is already changed in master.  I was using v<IP_ADDRESS> because it is latest stable release.  I think the changes in master will work, but I will test.

ok, I verified it works in master, so I am closing this.
any idea when the next tagged version of bitcoin-php might be?


docs: add CII for project to add CII Best Practices Badge

Signed-off-by: Allen Sun<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
The Linux Foundation (LF) Core Infrastructure Initiative (CII) Best Practices badge is a way for Free/Libre and Open Source Software (FLOSS) projects to show that they follow best practices. Projects can voluntarily self-certify, at no cost, by using this web application to explain how they follow each best practice. The CII Best Practices Badge is inspired by the many badges available to projects on GitHub. Consumers of the badge can quickly assess which FLOSS projects are following best practices and as a result are more likely to produce higher-quality secure software.
CNCF tries to add CII for hosted project to guarantee the qualify of it. This PR adds CII badge support for Dragonfly. And we need to make it polished as the time goes on.
Ⅱ. Does this pull request fix one issue?

none
Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
no need.
Ⅳ. Describe how to verify it

Ⅴ. Special notes for reviews
none

Codecov Report

Merging #516 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #516      +/-   ##
==========================================
- Coverage   54.26%   54.23%   -0.03%
==========================================
  Files          82       82
  Lines        4235     4235
==========================================
- Hits         2298     2297       -1
  Misses       1784     1784
- Partials      153      154       +1

Impacted Files
Coverage Δ

supernode/daemon/mgr/scheduler/manager.go
25.4% <0%> (-0.82%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29e1999...55df158. Read the comment docs.

FYI.
We could make it polished based on the details of this reference
https://bestpractices.coreinfrastructure.org/en/projects/2562

I have updated the details and now it passes 67. And we will definitely make it 100%. @Starnop @Frankzhaopku

LGTM.


Births and deaths post-ORU-transition cleanup

Reopening for merging into another branch as the old PR https://github.com/alphagov/smart-answers/pull/1844 is outdated now
This is an attempt to clean up residues of the 5-phase transition to handle most of the registrations via Overseas Registration unit (ORU). Now this has become a default way to handle registrations, but this fact was not well reflected in the code. I believe the series of commits in this PR make the fact clearer and make the code easier to follow.
This is a refactoring, therefore I do not expect there to be any externally visible impact to birth and death registration smartanswers.

Looks good to me.
I didn't completely follow the changes in "Default to the ORU outcome for births registration" or "Default to the ORU outcome for deaths registration abroad" but I'm confident that the regression tests would've told us if something was wrong.


Should we change Url.Action to mention that the 'controller' argument expects a controller route value, instead of a controller's name?

If we want to do routing to a controller class named SomeController, the controller argument that we must pass into UrHelperl.Action is Some, and not SomeController. This is implicitly noticed with the demo code, but never further expanded about. That is not the same for action methods - there, the full name of the action method is used, with a slight caveat that the MVC options allow the consumer to disable Async prefixes being removed from said action methods, as by default, they are trimmed from action method names.
Perhaps also expanding the remarks section to the relevant UrlHelperExtensions.Action method overloads is the more accurate place for this? Or maybe we can just expand both?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8ee6ad2f-1667-8d5c-1f36-9d0511e3720f
Version Independent ID: 7838ccec-c8e8-9baa-2b3e-220b2af5fc58
Content: Routing to controller actions in ASP.NET Core
Content Source: aspnetcore/mvc/controllers/routing.md
Product: aspnet-core
Technology: aspnetcore-mvc
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog manageable. If you believe there is a concern which hasn't been addressed, please file a new issue.

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog manageable. If you believe there is a concern which hasn't been addressed, please file a new issue.

Hello Rick! Shouldn't we make it more clear of what Url.Action expects as parameters? Although I'm not sure where the best place for the addition would be. Probably UrlHelperExtensions.Action?

I'm willing to try and draft something if you guys are open to the idea?

@kikaragyozov If you have a change you'd like to propose, please feel free to submit a PR. Thanks!

I'm willing to try and draft something if you guys are open to the idea?

That would be great, please do.

Sorry, been swarmed with work lately. I'll try to draft something up this weekend! Thanks for the opportunity!

Thanks for your willingness to help with this, @kikaragyozov. Given that a lot of time has passed and we've seen no other members from the community express similar concerns we've decided to close this issue. If you find time to submit a PR with your proposed changes, we'll happily consider it.

Please do it, I spent 2 days in debugging trying to understand why Url.Action is not working. The fact that something is trimmed in these parameters is ridiculous. How people should understand that?


Kubernetes Ingress Resource does not work

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
Following the documentation under https://istio.io/latest/docs/tasks/traffic-management/ingress/kubernetes-ingress/ I'm not able to create an routing using Kubernetes Ingress resource. Maybe there is a missing required configuration which is not documented?
I set up a cluster and Istio as follows:
kind create cluster

Point DNS *.mycluster.local to docker inspect kind-control-plane | jq -r '.[0].NetworkSettings.Networks.kind.IPAddress'
istiooperator.yaml:
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: default
spec:
  meshConfig:
    accessLogFile: "/dev/stdout"
  components:
    ingressGateways:
    - name: istio-ingressgateway
      enabled: true
      k8s:
        overlays:
        - apiVersion: v1
          kind: Service
          name: istio-ingressgateway
          patches:
          - path: spec.type
            value: ClusterIP
        - apiVersion: apps/v1
          kind: Deployment
          name: istio-ingressgateway
          patches:
          - path: kind
            value: DaemonSet
          - path: spec.strategy
          - path: spec.updateStrategy
            value:
              rollingUpdate:
                maxUnavailable: 33%
              type: RollingUpdate
          - path: spec.template.spec.containers.[name:istio-proxy].ports.[containerPort:8080].hostPort
            value: 80
          - path: spec.template.spec.containers.[name:istio-proxy].ports.[containerPort:8443].hostPort
            value: 443
          - path: spec.template.spec.containers.[name:istio-proxy].ports.[containerPort:15021].hostPort
            value: 15021
  values:
    telemetry:
      v2:
        prometheus:
          enabled: false
    pilot:
      autoscaleEnabled: false
    gateways:
      istio-ingressgateway:
        autoscaleEnabled: false

ingressclass.yaml:
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller

Setup Istio:
curl -fSsL https://github.com/istio/istio/releases/download/1.18.0/istioctl-1.18.0-linux-amd64.tar.gz | tar xzvf -
./istioctl install -f istiooperator.yaml
kubectl apply -f ingressclass.yaml

Install Test App:
kubectl create deployment web --image kennethreitz/httpbin
kubectl expose deployment web --port 80
kubectl create ingress web --class=istio --rule="web.mycluster.local/*=web:80"

At this point I don't see listeners. The documentation does not state that a Gateway is required but i guess this is the case?
Then I created a Gateway:
openssl req -x509 -newkey rsa:2048 -keyout tls.key -out tls.crt -sha256 -nodes -subj "/CN=*.mycluster.local" -addext "subjectAltName = DNS:*.mycluster.local"
kubectl create secret -n istio-system tls default-cert --key=tls.key --cert=tls.crt

gateway.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: default-ingress
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 443
      name: default-https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: default-cert
    hosts:
    - "*"
  - port:
      number: 80
      name: default-http
      protocol: HTTP
    tls:
      httpsRedirect: true
    hosts:
    - "*"

kubectl apply -f gateway.yaml

After this I see listeners for 8080 and 8443 but still no route for the ingress:
$ ./istioctl proxy-config route istio-ingressgateway-tk29r.istio-system
NAME                                                     VHOST NAME        DOMAINS     MATCH                  VIRTUAL SERVICE
https.443.default-https.default-ingress.istio-system     blackhole:443     *           /*                     404
http.8080                                                *:80              *           /*                     404
                                                         backend           *           /healthz/ready*
                                                         backend           *           /stats/prometheus*

Hence I get a 404 if I try to reach example app:
curl -k -v https://web.mycluster.local

Version
client version: 1.18.0
control plane version: 1.18.0
data plane version: 1.18.0 (1 proxies)

Client Version: v1.27.1
Kustomize Version: v5.0.1
Server Version: v1.27.3

Additional Information
No response
Affected product area

[ ] Ambient
[X] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Control Plane Revisions

You shouldn't be creating a Gateway AND an Ingress resource. The docs you linked only created an Ingress and IngressClass resource.
Also, I highly recommend using the Kubernetes Gateway API rather than the Ingress API if possible

@keithmattix Thank you for the help. Only creating the IngressClass and the Ingress is what I did first (see setup description above). But this did not work.
I think now I figured out what the problem is. If you run istioctl install before you create the IngressClass istiod does not look for Ingress resources. You have to create the IngressClass before istiod is started. I guess a hint in the documentatin about this would make sense. Also it probably make sense to move the IngressClass part to the top into the main examples, because I guess there are only very little people running Kubernetes clusters in Versions <1.18.
Unfortunately I can't use the Gateway API in this setup.
Do you know if it is possible to change the Gateway/VHost settings for the ingress vhost. I would like to have a single default certificate for all ingress resources.


Delete qr-code-tracking.md

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/qr-code-tracking.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


cmake "configure" step on Windows 10 takes way too long (~7mins)

I've noticed that the "configure" step in our windows cmake build is extremely slow (~7mins)
My observations:

on Windows 7, the configure step takes a reasonable amount of time (much shorter), only on Windows 10 (and newer?) it takes extremely long
on linux and macos the configure step is very quick (~ seconds)

The command we run looks very normal:
cmake -G "Visual Studio 14 2015" -A %ARCHITECTURE% -DgRPC_BUILD_TESTS=OFF -DgRPC_MSVC_STATIC_RUNTIME=ON ../../..
examples where we use it:
https://github.com/grpc/grpc/blob/0605a977c88af6dc4d3ab308dd8b7cbb5aec9ea6/tools/run_tests/helper_scripts/pre_build_csharp.bat#L31
https://github.com/grpc/grpc/blob/0605a977c88af6dc4d3ab308dd8b7cbb5aec9ea6/tools/run_tests/helper_scripts/pre_build_cmake.bat#L27
On stackoverflow I found this:
https://developercommunity.visualstudio.com/content/problem/162361/cmake-configuration-is-unbearably-slow.html
The pre_build_csharp took 434.5secs in my local build
$ python tools/run_tests/run_tests.py -l csharp -c dbg --build_only
2019-11-12 10:37:50,055 START: tools\run_tests\helper_scripts\pre_build_csharp.bat
2019-11-12 10:45:04,562 PASSED: tools\run_tests\helper_scripts\pre_build_csharp.bat [time=434.5sec, retries=0:0]
2019-11-12 10:45:04,565 START: cmake
2019-11-12 10:53:27,068 PASSED: cmake [time=502.5sec, retries=0:0]
2019-11-12 10:53:27,071 START: tools\run_tests\helper_scripts\build_csharp.bat
2019-11-12 10:53:57,391 PASSED: tools\run_tests\helper_scripts\build_csharp.bat [time=30.3sec, retries=0:0]
2019-11-12 10:53:57,394 SUCCESS: All tests passed

@KyleFromKitware  @zackgalbreath   can you take a look? This is most likely some problem with cmake on windows. Suggestions are welcome.

Thanks @jtattermusch. I tried configuring with the Visual Studio generator, and it took about 3 minutes (though not the 7 minutes that it took for you.) It took a little less time with the Ninja generator and a Visual Studio compiler (about 1-2 minutes) and even less with Ninja + MinGW (under a minute.) None of them are blazing fast like on Linux, though.

Visual Studio only: 3:14
Ninja + Visual Studio: 1:34
Ninja + MinGW: 1:02

@KyleFromKitware   random thought: I am consistently seeing higher latencies on my machine (it's a laptop) but I'm also able to reproduce higher latencies on our CI (8core windows VMs). Is there some parameter that influences the duration observed (such as the number of VS versions installed on the machine or so)?
Zack has suggested trying with the latest version of cmake as well, just to see if there are any optimizations we could make use of.

Is there some parameter that influences the duration observed (such as the number of VS versions installed on the machine or so)?

I don't think the number of installed VS versions would matter, as CMake is only using one. However, I spoke to @mathstuf offline, and he mentioned that the Visual Studio generator will inherently take longer to configure because every try_compile() invocation generates four configurations instead of one (this is probably why Ninja runs so much faster.) He recommended trying to cut out as many try_compile()s as we can to avoid unnecessary overhead in the configure step.

Well, VS generators will take 4 times as long to generate. try_compile has a generate step, so that is being paid for there too.
This MR can be used to get configure time performance metrics. It outputs a JSON file that browser debuggers know how to render.

With CMake 3.16.0-rc3:
Visual Studio only: 2:33
Ninja + Visual Studio: 0:56
Ninja + MinGW: 0:45
Definitely a little better, though still not super fast.

try_compile has a generate step, so that is being paid for there too.

Right, so if your configure step has a lot of try_compile()s (which gRPC does) then your configure step will take longer. (I realize that it's technically the generate step that's taking longer within the try_compile(), but from a user point of view, gRPC is taking a long time to configure because of it.)

This MR can be used to get configure time performance metrics. It outputs a JSON file that browser debuggers know how to render.

A quick run with that profiler confirms that we're spending the vast majority of our time in try_compile(), with try_run() being a distant second.
Interestingly, about 75% of the entire configure step is spent in third_party/cares/cares. I bet if we built against an external c-ares, the configure time wouldn't be nearly as long (though the same build on Linux still doesn't have this problem, even with all the try_compile()s.)

Interestingly, about 75% of the entire configure step is spent in third_party/cares/cares

Indeed. I see about 200 or so try_compile calls here: https://github.com/c-ares/c-ares/blob/e982924acee7f7313b4baa4ee5ec000c5e373c30/CMakeLists.txt#L134. Working with upstream to remove the absolutely unnecessary calls first would be best (search to see if the preprocessor define for each is actually used). Next, things like "assume windows.h exists #if _WIN32" logic can be applied to trim down the rest.

Is there any good explanation why on Win 7 the cmake configure step is significantly faster?
2019-11-13 05:05:21,750 START: tools\run_tests\helper_scripts\pre_build_csharp.bat
2019-11-13 05:07:02,828 PASSED: tools\run_tests\helper_scripts\pre_build_csharp.bat [time=101.1sec, retries=0:0; cpu_cost=0.0; estimated=1.0]

Knowing whether it is in the configure or generate phase would be useful. What's the relative time for a reconfigure (which shouldn't have any try_compile runs)?

This is probably still relevant. How is one supposed to interpret a closed "help wanted" issue?
The last activity was asking for numbers.

Yes it is still relevant but we won't be able to spend any time on it in foreseeable future. Help wanted label is there because we'd be happy to accept help if someone knew the solution.

But it being closed sends the wrong message… At least it does to me. I don't search closed issues for things to work on other projects at least.

what is the current state??

Just stumbled on this issue, since this is bugging me as well for quite some time. Right now I am working with CMake 3.23.2 and I just wanted to ask if there is still any activity no this topic. I am not even complaining that everything runs slightly slower on Windows for whatever reason, but I think there is at least some mystery here as well. There always seems a period when nothing seems to happen and most of the time is actually spent there:
// lots of stuff going on here with decent performance and various messages, then
1>-- Configuring done
// nothing seems to happen here but at least a minute is spent until the next message is generated
1>-- Generating done

So can anyone shed some light on what is happening between these last 2 messages?

CMake is taking the abstract target build graph that the CMake described and turning it into the actual instructions for the actual generator. Things that can take a long time here:

complex build graph
complex usage requirements (i.e., deep graph, lots of generator expressions)
large build files (e.g., lots of flags, long command lines)

For Visual Studio builds, you pay the above for each configuration in CMAKE_CONFIGURATION_TYPES. It may pay off to remove MinSizeRel or other unused configurations from the default list if "no one" cares about it. A performance trace would help a lot here.

Dear Ben,
thanks for the hints. I learned quite some new, interesting stuff about CMake and its profiling capabilities from that. However I have to admit, that Windows Defender and the way it is configured on our systems seems to be the guilty one here. Whenever CMake writes back a massive amount of files to the build directory I see the MsMpEng.exe process taking time and as soon as this is done the actual build in Visual Studio starts. So from what I see, CMake has nothing to do with that behaviour. I just wanted to provide this as feedback that might be helpful for others and will try to get the Windows Defender set up differently now.
Thanks again!


RDKVREFPLT-2104: RFC support for AppHibernate functionality in memcr #32Rdk6main feature branch

RDKVREFPLT-2104 - Integrate thunder 4.4 memcr patches to thunder 4.2, RDKShell memcr patches, datamodel & validation
Reason for change: Added RFC support for AppHibernate functionality in memcr.
Test Procedure: Build and verified
Risks: Low

Sanity tests for ['BCM 7216OTT Refboard VA'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/52571/

Sanity tests for ['RPI4 IPSTB'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/52576/


arch-vega: Template MFMA instructions

templated

v_mfma_f64_16x16x4f64

added support for

v_mfma_f32_32x32x2f32
v_mfma_f32_4x4x1_16b_f32
v_mfma_f32_16x16x4f32

formula for gprs needed
formulas for register layouts and lanes used in computation

Hi @mjkpolo,
None of the commits in this pull request contains a Change-ID, which we require for any changes made to gem5.   Thank you.

There are a few style errors which should have been caught by pre-commit. I'd also recommend making a new branch on your repo. I'd like to add some commits to this, but I don't want to put them on top of your develop branch as it would be too easy to accidently git push -f over them

@mjkpolo let me know if you can't figure out how to install pre-commit and the hooks and need help.  Likewise let @abmerop and I know once you've made a separate branch.


[MultiSelect issue] Encountered two children with the same key

I'm facing the same issue described here
If you serve this data array, for example, to the MultiSelect component you'll receive Warning: Encountered two children with the same key, GenericName`. Keys should be unique so that components maintain their identity across updates. Non-unique keys may cause children to be duplicated and/or omitted — the behavior is unsupported and could change in a future version.
const myJ = [
  {Name: "GenericName", Id: 1},
  {Name: "GenericName", Id: 2}
]

<MultiSelect
data={myJ]
labelField="Name"
valueField="Id"
renderSelectedItem={(item, unSelect) => (
          <TouchableOpacity onPress={() => unSelect && unSelect(item)}>
            <View style={styles.selectedStyle}>
              <Text style={styles.textSelectedStyle}>{item.Name}</Text>
              <Image style={{width: 20, height: 20, tintColor: "white"}} source={IMAGES.xCircle}/>
            </View>
          </TouchableOpacity>
        )}
/>

So I believe @TommysG is right. Shouldn't the Id field be used?

Is there still no workaround around this issue?
The fact that you cannot multi select two items with the same despite them having different IDs is quite inconvenient.

Is there still no workaround around this issue? The fact that you cannot multi select two items with the same despite them having different IDs is quite inconvenient.

We ripped off all dropdown elements from our app and replace them with bottom sheets for better UX.
To address this issue, navigate to the 'node_modules' directory, locate the library, and replace the 'key' with 'value' instead of 'label'. Then, patch this version. This is the approach we took until we completely replaced the library. Hope this helps.


haskellPackages: mark some broken packages as not broken

Description of changes
This PR marks the glpk-headers, math-programming, math-programming-glpk, and math-programming-tests packages as not broken.
Existing versions of these packages were indeed broken. I recently released version 0.5.0 of each of these, and they all build on the current release-22.11 branch of nixpkgs.
Note that this PR has not updated the versions of these packages in the hackage-packages.nix file; I suspect that this PR will need to be merged after this file is regenerated in the next few weeks and picks up the newly-released packages above.

Testing done
I created a small Haskell project that relies on the new packages, and nixpkgs branch release-22.11. The resulting project builds. I used the following as shell.nix,
{ sources ? import ./nix/sources.nix
, pkgsF ? import sources.nixpkgs
}:
let
  config = {
    packageOverrides = pkgs: rec {
      haskellPackages = pkgs.haskellPackages.override {
        overrides = self: super: rec {
          test-nixpkgs = self.callCabal2nix "test-nixpkgs" ./. {};
          math-programming = self.callPackage ./math-programming.nix {};
          math-programming-tests = self.callPackage ./math-programming-tests.nix {};
          math-programming-glpk = self.callPackage ./math-programming-glpk.nix {};
          glpk-headers = self.callPackage ./glpk-headers.nix {};
        };
      };
    };
  };

  pkgs = pkgsF { inherit config; };
in
pkgs.haskellPackages.shellFor {
  packages = p: [ pkgs.haskellPackages.test-nixpkgs
                ];
  buildInputs = [ pkgs.cabal-install
                  pkgs.cabal2nix
                  pkgs.haskell-language-server
                ];
  withHoogle = true;
}

where I generated the files glpk-headers.nix, math-programming.nix, math-programming-tests.nix, and math-programming-glpk.nix via
nix-shell --packages cabal2nix --run 'cabal2nix cabal://math-programming-0.5.0' > math-programming.nix
nix-shell --packages cabal2nix --run 'cabal2nix cabal://math-programming-glpk-0.5.0' > math-programming-glpk.nix
nix-shell --packages cabal2nix --run 'cabal2nix cabal://math-programming-tests-0.5.0' > math-programming-tests.nix
nix-shell --packages cabal2nix --run 'cabal2nix cabal://glpk-headers-0.5.0' > glpk-headers.nix

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

Sorry for the massive set of pings on the previous PR; I had started from master, and I swapped the target to haskell-updates late without realizing what that would imply for automation.

I do have the question of "is this the right approach?" If the broken.yaml file is made from scratch each time the Hackage package set is regenerated, then I suspect this PR is unnecessary.

No, PRs like that are very welcome. Hydra does not create the file from scratch. It doesn’t build packages marked as broken at all. So it wouldn’t know which packages to remove. We only add packages to this list semi-manually when packages break and remove them in PRs like this, when someone notices that they work.

I suspect that this PR will need to be merged after this file is regenerated in the next few weeks and picks up the newly-released packages above.

Are we there yet? :-D
Should we change this to draft?

@prsteele I just updated the hackage pin on haskell-updates and opened https://github.com/NixOS/nixpkgs/pull/213089.
I tried compiling glpk-headers, but I got an error about the required version for the hspec dependency.  Could you take a look at that?  (I didn't check the other packages you've unbroken in this PR)

Looking into this. I'll note that I ran into (what seems like) a bug on the current haskell-updates branch. In configuration-common.nix, line 1376, we have
  hls-hlint-plugin = super.hls-hlint-plugin.override {
    # For "ghc-lib" flag see https://github.com/haskell/haskell-language-server/issues/3185#issuecomment-1250264515
    hlint = enableCabalFlag "ghc-lib" super.hlint;
    apply-refact = self.apply-refact_0_11_0_0;
  };

It seems like self.apply-refact_0_11_0_0 doesn't exist.

Okay, I can reproduce the failure you observed. It looks like hspec got a new version, and glpk-headers depends on versions that aren't included any more.
I'll get a 0.5.1 version out for these packages that works with the current haskell-updates branch.

@cdepillabout I've added 0.5.1 versions of each package to Hackage with relaxed bounds on hspec, and I'm able to build each of the packages against this commit. I manually patched hackage-packages.nix, and left that as a standalone commit unless it's better to just regenerate the whole file.

@prsteele Thanks!
I updated the Hackage pin on haskell-updates and cherry-picked d1f0d4c4ac3204e22a5286d51eeab5b41c824320 as c4d460810e6.  You can follow along on https://github.com/NixOS/nixpkgs/pull/213089.


Introduce flag to request all variants

Supplemental PR to #19763, this PR lowers all variants to internal API and adds a flag to request it on ConfigurationInternal.

@bot-gradle test this

OK, I've already triggered the following builds for you:

PullRequestFeedback build


optimize  for default setting in alluxio

Ⅰ. Describe what this PR does
Ⅱ. Does this pull request fix one issue?

Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #172 into master will increase coverage by 2.20%.
The diff coverage is 77.14%.

@@            Coverage Diff             @@
##           master     #172      +/-   ##
==========================================
+ Coverage   15.88%   18.09%   +2.20%
==========================================
  Files          45       46       +1
  Lines        1951     1984      +33
==========================================
+ Hits          310      359      +49
+ Misses       1598     1581      -17
- Partials       43       44       +1

Impacted Files
Coverage Δ

pkg/ddc/alluxio/transform.go
13.33% <40.00%> (+0.75%)
:arrow_up:

pkg/ddc/alluxio/transform_resources.go
74.35% <74.35%> (ø)

pkg/ddc/alluxio/transform_optimization.go
96.20% <88.46%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fed68f1...371396b. Read the comment docs.


[Bug] Graph is finalized after using spleeter in python

Description
I'm using spleeter as a python package in my code, and when using tensorflow as the stft_backend, the default global tensorflow graph will be finalized after spleeter finished the separation, leading to a situation that tensorflow models are unable to be initialized anymore in further process.

Step to reproduce

Installed using pip install spleeter
Run as spleeter python pacakage
Got RuntimeError: Graph is finalized and cannot be modified. error

Version

spleeter - 2.0.1
tensorflow - 2.3.0

Steps to reproduce:
from spleeter.separator import Separator
import tensorflow as tf

sep = Separator("spleeter:2stems")
sep.separate_to_file("example.wav", "/tmp/test")

# Below will fail
model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(10, 10)),
    tf.keras.layers.Dense(10)
])

Output
2020-11-30 23:16:38.229885: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-11-30 23:16:39.902822: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-11-30 23:16:39.906834: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:39.907200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.683GHz coreCount: 28 deviceMemorySize: 10.91GiB deviceMemoryBandwidth: 451.17GiB/s
2020-11-30 23:16:39.907235: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-11-30 23:16:39.909192: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-11-30 23:16:39.910422: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-11-30 23:16:39.910637: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-11-30 23:16:39.911894: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-11-30 23:16:39.912559: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-11-30 23:16:39.915218: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-11-30 23:16:39.915319: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:39.915741: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:39.916050: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-30 23:16:41.953061: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-30 23:16:41.973544: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency:<PHONE_NUMBER> Hz
2020-11-30 23:16:41.973886: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x68f98d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-11-30 23:16:41.973915: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-11-30 23:16:42.031372: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.031785: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4b76b40 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-11-30 23:16:42.031803: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1080 Ti, Compute Capability 6.1
2020-11-30 23:16:42.031954: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.032263: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1080 Ti computeCapability: 6.1
coreClock: 1.683GHz coreCount: 28 deviceMemorySize: 10.91GiB deviceMemoryBandwidth: 451.17GiB/s
2020-11-30 23:16:42.032291: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-11-30 23:16:42.032313: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2020-11-30 23:16:42.032327: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-11-30 23:16:42.032341: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-11-30 23:16:42.032354: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-11-30 23:16:42.032366: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2020-11-30 23:16:42.032379: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-11-30 23:16:42.032428: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.032766: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.033074: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-11-30 23:16:42.033095: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
2020-11-30 23:16:42.333509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-11-30 23:16:42.333558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0
2020-11-30 23:16:42.333566: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N
2020-11-30 23:16:42.333741: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.334136: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-11-30 23:16:42.334456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7822 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-11-30 23:16:43.200089: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-11-30 23:16:43.302117: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.7
2020-11-30 23:16:43.893600: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
INFO:spleeter:File /tmp/test_spleeter/Ava/vocals.wav written succesfully
INFO:spleeter:File /tmp/test_spleeter/Ava/accompaniment.wav written succesfully
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    tf.keras.layers.Dense(10)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 142, in __init__
    self.add(layer)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 457, in _method_wrapper
    result = method(self, *args, **kwargs)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 202, in add
    batch_shape=batch_shape, dtype=dtype, name=layer.name + '_input')
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/input_layer.py", line 311, in Input
    input_layer = InputLayer(**input_layer_config)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/keras/engine/input_layer.py", line 160, in __init__
    ragged=ragged)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 1223, in placeholder
    x = array_ops.placeholder(dtype, shape=shape, name=name)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 3100, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 6809, in placeholder
    "Placeholder", dtype=dtype, shape=shape, name=name)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 744, in _apply_op_helper
    attrs=attr_protos, op_def=op_def)
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3460, in _create_op_internal
    self._check_not_finalized()
  File "/data/omnizart/.venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3050, in _check_not_finalized
    raise RuntimeError("Graph is finalized and cannot be modified.")
RuntimeError: Graph is finalized and cannot be modified.

Environment

OS
Ubuntu-18.04

Installation type
pip

RAM available
128GB

Hardware spec
NVIDIA 1080TI 12GB / Intel Core I7-7700 / CUDA V11.0 / CUDA Driver: 450.80.0

Additional context
Currently my workaround is to set sep._params["stft_backend"] = "librosa" to avoid using the tensorflow backend.
I've checked the code that in librosa approach, new local graph will be initialized, and later separation will all within that graph. The default global graph will thus not being finalized, and all would be fine when initializing new tensorflow models in later process.
I've also tried to initialize the tensorflow model after separation like following:
with tf.Graph().as_default():
    model = tf.keras.models.Sequential(...)

though it succeeded, but when I tried to load some pre-trained models inside the with, it was extremely slow, compared to load the model that is not in the graph mode. It usually takes only a few seconds to load the pre-trained models, but it takes up to 8~10 minutes to load the same model in graph mode.
Spleeter is a great tool and pretty easy to getting start that I've rarely found from other open sources. Hope this issue could help others when they enconter the same problem. It took me a few days getting around the bug, and hope there will be a good solution in the future.
Thanks.

Hi @BreezeWhite
Thank you for reporting this issue.
Indeed the mechanism for separation is a bit different between the librosa and tensorflow backend.
In the tensorflow backend, an infinite tensorflow dataset is created to feed a tensorflow estimator that performs the predictions. The dataset is fed with new data each time the separate (or separate_to_file) method is called. Thus there is an underlying graph for this dataset that is never closed and that may be responsible for the issue you encountered.
Note that on GPU, the tensorflow backend is much faster for performing separation, so it would be a pity to use the librosa backend instead.
I can't see a proper workaround at the moment besides what you suggested which seems to result in speed issues. Rewriting the way spleeter uses tensorflow for performing separation may actually be necessary but I can't see a better way so far. So any help on this topic would be appreciated.

Hi Taufeeque,
Thanks for the edit, this looks much better.
May we use this as my final abstract?
I don't just want to use your edits without consenting with you first.
Regards: Donavin.
On 2020/12/01 18:22, Romain Hennequin wrote:

Hi @BreezeWhite https://github.com/BreezeWhite
Thank you for reporting this issue.
Indeed the mechanism for separation is a bit different between the
|librosa| and |tensorflow| backend.
In the |tensorflow| backend, an infinite tensorflow |dataset| is
created to feed a tensorflow |estimator| that performs the
predictions. The dataset is fed with new data each time the |separate|
(or |separate_to_file|) method is called. Thus there is an underlying
graph for this dataset that is never closed and that may be
responsible for the issue you encountered.
Note that on GPU, the |tensorflow| backend is much faster for
performing separation, so it would be a pity to use the |librosa|
backend instead.
I can't see a proper workaround at the moment besides what you
suggested which seems to result in speed issues. Rewriting the way
spleeter uses tensorflow for performing separation may actually be
necessary but I can't see a better way so far. So any help on this
topic would be appreciated.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/deezer/spleeter/issues/524#issuecomment-736659356,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANZF77Y7BYW4LEGKGYMTA73SSUJ3RANCNFSM4UHX5ZAQ.


k8s admin config context not authorized

Sending the following command to prod k8s produces an error:
$ kubectl get pods,services --all-namespaces -o wide
error: You must be logged in to the server (Unauthorized)

The admin context is currently being used:
$ kubectl config get-contexts
CURRENT   NAME                          CLUSTER      AUTHINFO           NAMESPACE
          gnis                          kubernetes   gnis               gnis
*         kubernetes-admin@kubernetes   kubernetes   kubernetes-admin

The k8s certificates appear to be up-to-date:
$ sudo kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0223 09:28:54.897792    8033 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 18, 2022 23:01 UTC   176d            ca                      no
apiserver                  Aug 18, 2022 22:59 UTC   176d            ca                      no
apiserver-etcd-client      Aug 18, 2022 22:59 UTC   176d            etcd-ca                 no
apiserver-kubelet-client   Aug 18, 2022 22:59 UTC   176d            ca                      no
controller-manager.conf    Aug 18, 2022 23:00 UTC   176d            ca                      no
etcd-healthcheck-client    Aug 18, 2022 22:58 UTC   176d            etcd-ca                 no
etcd-peer                  Aug 18, 2022 22:58 UTC   176d            etcd-ca                 no
etcd-server                Aug 18, 2022 22:58 UTC   176d            etcd-ca                 no
front-proxy-client         Aug 18, 2022 22:59 UTC   176d            front-proxy-ca          no
scheduler.conf             Aug 18, 2022 23:00 UTC   176d            ca                      no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Feb 15, 2030 21:01 UTC   7y              no
etcd-ca                 Feb 15, 2030 21:01 UTC   7y              no
front-proxy-ca          Feb 15, 2030 21:01 UTC   7y              no

The authorization problem was a result of '/etc/kubernetes/admin.confbeing updated. This file contains theclient-certificate-dataand theclient-key-datavalues for thekubernetes-admin` context, which is also in the ~/.kube/config file for admin users.
This file may have been updated when kubeadm was updated via linux system upgrades. Typically this  package has a hold on it so that it is not upgraded during routine OS upgrades.
This was an opportune moment to upgrade k8s, so prod k8s has been upgraded to v1.23.4, which is the most recent.
Note that the new values for the admin kubectl config context will be updated in the NCEAS secure repo.

Hey @gothub, did the config in the secure repo get updated yet?


When creating repo from the UI, give some hint as to how the storage namespace should look like

Also fix the documentation, where the storage namespace appears without the "s3://" prefix:

Hey I would like to tackle this issue for the upcoming hacktoberfest. Do assign this to me, thanks!

Hey @shamikakumar!
Thanks for offering your help with this one. Assigned to you - we are here to help if you have any questions:
https://join.slack.com/t/lakefs/shared_invite/zt-g86mkroy-186GzaxR4xOar1i1Us0bzw
Happy Hacktoberfest!
Yael

Hey I see there's a placeholder for storage-namespace. How should the hint be displayed. Do I add an info button next to the input box ?

Hey I see there's a placeholder for storage-namespace. How should the hint be displayed. Do I add an info button next to the input box ? or do i make it a tootip ?

Can we start with a placeholder on the input field, and see how users feel about that?

@arielshaqed I see that's already been implemented, and the docs are fixed too. Is it possible this issue was not closed after the fix ? would you like me to make any other change here ?

Hi @shamikakumar, you are right. This issue is not up-to-date and I should have described better what I meant when I opened it. Thank you for pointing it out.
Since lakeFS can operate with various backing storage services, the hint should be suitable to the current storage type. For example, when running with Google as the storage service, the hint should be "gs://example-bucket/". I will update the issue to reflect that.
In any case, since our client is currently not aware of the storage type, this issue could be a bit challenging for now, and that's the reason it is not marked as a "good-first-issue". You're welcome to try it anyway and I'll be here to help. Otherwise, we are in the process of creating some more good-first-issues.

Actually, I think it would be better to create a separate issue, since as you mentioned this one has actually been solved. I will link the two issues so that you can access it from here.


No Warning for Overlaps in Pattern Matching

CS-FSSA-022
During pattern-matching a user can define a disjunction of multiple different values as a scrutinee for a
particular arm like the following example:
match v {
1 => …
1 | 3 => …
}

Such a case will not yield any warning however. The root cause is the initialisation of witness_report
at the beginning of usefulness.rs::is_useful_or(). It is initialised to an empty Witnesses
(which still counts as true) instead of NoWitness, as is the case for e.g., is_useful_constructed().
Therefore, is_useful_or() will essentially always return true.

Related to #5097.


have system based tabs as a component for easy loading/interacting (mac/windows/linux)

we should do something not toggle based but more tab based that makes better sense to the user like github's style with: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/checking-for-existing-ssh-keys?platform=mac
it has tabs that auto change the content and autodetects what tab should be loaded --> obv we'll need to figure out a better visual setup since it's not that noticeable right of the bat visually, but would be a better setup than the drop downs for usability
Originally posted by @hanbollar in https://github.com/KittyCAD/PetStore/issues/99#issuecomment-1343334378

This is related and a better UX imo https://github.com/KittyCAD/website/issues/614 would be nice if we could reuse the same react component for both


Add Ahead Guru as company

Add my company to list of supporters, I use terraform heavily and recommended terraform to so many clients.

Added in https://github.com/opentffoundation/manifesto/pull/203


Sypport async mode (aiopg ?)

There's two parts:

async defering tasks (asyncio coroutine)
async executing jobs (asyncio task)
(https://docs.python.org/3/library/asyncio-task.html#awaitables)

This can be done in a single PR or in multiple PRs. I have ideas on how to do it without significantly changing the API, but this will need POC-ing around.

First half is being done in #93 . We'll need the async job execution too to close the ticket.

Splitting the ticket.

Closed by #94


Determine more answers to questionnaire

Update DataKitten and answer a few more questions about where data is
Fixes theodi/shared#853

Coverage increased (+0.07%) to 80.451% when pulling de7b744b6b204acb03fe4ba0fd6271c3927c0805 on pet-the-data-kitten into bf02c80e978ad7871ab62e072848c41f0abfa29b on staging.


Raise BlobNotFoundError for local blobs

depends on #128

Codecov Report

Merging #129 into xbrianh-add-local-blob-get will decrease coverage by 0.31%.
The diff coverage is 73.33%.

@@                      Coverage Diff                       @@
##           xbrianh-add-local-blob-get     #129      +/-   ##
==============================================================
- Coverage                       88.81%   88.49%   -0.32%
==============================================================
  Files                              11       11
  Lines                             751      765      +14
==============================================================
+ Hits                              667      677      +10
- Misses                             84       88       +4

Impacted Files
Coverage Δ

ssds/blobstore/local.py
85.18% <73.33%> (-4.82%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce59b19...e93ee4d. Read the comment docs.


make matchdg features insdie trainer

Codecov Report
Attention: 5 lines in your changes are missing coverage. Please review.

Comparison is base (d35ce75) 95.25% compared to head (9fe2495) 95.16%.

:exclamation: Current head 9fe2495 differs from pull request most recent head 4419661. Consider uploading reports for the commit 4419661 to get more accurate results

Files
Patch %
Lines

domainlab/algos/compos/matchdg_base.py
72.22%
5 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #656      +/-   ##
==========================================
- Coverage   95.25%   95.16%   -0.09%
==========================================
  Files         128      128
  Lines        4822     4824       +2
==========================================
- Hits         4593     4591       -2
- Misses        229      233       +4

Flag
Coverage Δ

unittests
95.16% <92.95%> (-0.09%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Release ITI.DSUBm to Public-Comment

https://github.com/IHE/ITI.DSUBm
http://build.fhir.org/ig/IHE/ITI.DSUBm/branches/master/index.html


Continue world generation beyond the world gen % restriction after queue is filled

Eg: If max gen % is set to 5% and once queue fills up, it continues generating the remaining seeds in queue?
Considerations for this:

Might degrade performance if queue fills up while in game... (Could be made an opt-in config for that).
Would generate seeds the same way as it generated to fill the queue initially (i.e if someone wants to have 3 seeds generating while in-game, we would respect that even for the further loading post queue fill.. this could also fix consideration 1).
Maybe even think about setting lower thread priorities for these types of seeds (could be a config option as well).

That's all I could think of, would love to hear thoughts on this :)

Makes sense as an option, I'm unsure if it should be under Queue Settings or somewhere else.
It makes sense grouped with the Queue Settings but I would also like those to be very simple and as few as possible.
I've marked this as a good-first-issue for new contributors so I don't have to figure out what I would name an option like this :P

Mhm i agree.... Queue Settings makes sense. I'll figure out a good way to do this :D

Coming to justin's issue mention, I could just cross that bridge when I get to it ig. Lemme see tho... I will try to make it as future-proof as possible..

Closed by https://github.com/KingContaria/seedqueue/pull/56


Updated launch.json to netcoreapp2.2

The app itself is an netcoreapp2.2 while launch.json wasn't updated

Thanks!


dev feature to trailhad brach

What does this PR do?
What issues does this PR fix or reference?
#
The PR fulfills these requirements:
[ ] Tests for the proposed changes have been added/updated.
[ ] Code linting and formatting was performed.
Functionality Before
<insert gif and/or summary>
Functionality After
<insert gif and/or summary>

merging


Add algorithms to define the percentage of dry matter in the storage roots

This could be related to the issue https://github.com/lpmorenoc/dssat-csm-os/issues/24 and would require a state variable for the fresh weight with reduced dry matter when there is remobilization of assimilates

(Comments from James):
•	Cultivar is the single most important determinant of dry matter, furthermore there is little GxE interaction. This lack of GxE suggests that most varieties respond similarly to variation in growing conditions. This makes life easy for us, although there may actually be more GxE than we think. (Anecdotal evidence suggests that in some varieties the decrease in starch is more marked than in other at the beginning of the rainy season. However, our proposal will likely support this as those varieties that produce a greater flush of new leaves will also have a greater drop in reserves).
•	As mean temperature increases starch content decreases.
•	After a drought period there is a flush of new leaf growth when it rains. This flush of new growth is supported by translocation of carbohydrate reserves from the roots and the starch content of the roots decreases.

[ ] With the onset of rains and the flush of new growth the total water content of the tubers does not change when the starch reserves are mobilized. That is the size of the tubers or the receptacle does not change.
[ ] When in spillover mode tubers are filling with starch the starch content is the cultivar value modified by a temperature factor.
[ ] When in reverse spillover the total water content of the tubers does not change, and the starch content reflects the starch translocated to the tops. (Note: based on research in rice translocation of starch from stem reserves to grains we suggest a translocation efficiency of 0.7)
[ ] The temperature factor is determined by the air temperature, if that is available from DSSAT at 10 cm depth. If not, it is determined by the air temperature.  A moving average of the temperature for the past ¿20? days will be used to minimize brusque changes.
[ ] When recovering from a trigger event and tubers start filling with starch again, the receptacle size remains constant (ie same total amount of water) and the excess from top needs goes to filling the receptacle. Once the concentration reaches the temperature variety value the concentration of starch is determined by the temperature variety value and the amount of water increases commensurate with the temperature variety starch content.

New variable with fresh weight of the roots:

Now we can generate a % of dry matter variable 63485cb:

As suggested by @GerritHoogenboom, changes in the dry matter content were done as a rate variable instead of a state variable (whole dry weight):

Testing with an experiment in Nigeria (allowing retranslocation):
Storage dry weight:

Percentage of dry matter

Testing with one of the ACAI trials experiments (Ido, 2017 planting)

Testing with the ACAI trial using state instead of rate variables:

Experiment with different temperatures (Keating, Australia) before adding any effect due to temperature (currently no water or nutrients restrictions):

After adding the effect of temperature with low temperatures increasing the dry matter content up to 5% if the air temperature is 13 C. The 5% could be an ecotype coefficient and would use the cardinal temperatures of 13 as "optimum" or maximum dry matter content and 30 as "minimum" (no reduction of dry matter if the mean temperature is above 30):

Mean temperature:

Modifying cardinal temperatures (maximum dry matter: 18 C, minimum dry matter 26 C). f74c829

With effect of cardinal temperatures Nigeria trial:

Adding effect of planting age:


Deriving some common traits + defmt feature and no_std support

[x] In addition to the current Debug derive of IbusPacket and ParsingError it would be nice to have Clone/Copy and defmt::Format (enabled with a features as an optional dep.) to be used in no_std environment.

[x] To support no_std we can use core::error::Error (which is stable since 1.81) so either a MSRV should be added or std::error::Error impl should be enabled only with a feature, e.g. std feature.

[x] core::fmt::Display can be used instead of the std::fmt::Display to support no_std

I will definitely try to migrate the traits provided by std to core, as that is an easy fix. As to why I never did it sooner, I built this for a drone with a flight controller running in a Raspberry Pi 4's userland.
As to the dependencies, I've never actually written a crate with feature flags, although I imagine it is not difficult.
I will begin working on this once my studies become less difficult.

That's great! Good luck with your studies.
I can also open a PR, however, it might be a good opportunity for you to exercise adding features to crates.
Do let me know if you need some assistance.
PS: You can also check e.g. our nmea crate as it adds conditions derives to structures based on feature flags: https://github.com/AeroRust/nmea/

Done.
Core traits added with 4e985e595ce62b808fa86f8ac74a1c762e7ca642
Copy and clone were added with 09bbdc4a7560a3d69bd2bdf25308cc7c061af958
and defmt support was added with 3aafa0b4ec893f4c2b8bdaee0e0125b625cc0b7d
Unfortunately due to not being able to enable features for tests (which is currently being discussed here), I could not verify the functionality of the defmt feature working. Given how simple the derive was, I imagine it works fine.

Yaay! Btw for Marc you can add "rust-version" to Cargo.toml


Start Spotify Implementation

added the spotify library to iOS. still needs to be done to android

playlists and songs dont show up in the menu yet for some reason. however, authentication works and songs from ur library play. controls also kinda work (not if you're on the 'now playing' screen tho)


[event-hubs] update EventHubConsumerClient.close to close subscriptions

Description
Fixes #9083, #8598
This update changed the EventHubConsumerClient.close() method to also stop any active Subscriptions that it spawned via subscribe() calls, and fixes the bug where calling subscription.close() would not immediately cancel any in-flight receiveBatch calls.
Background
The EventHubConsumerClient.subscribe() method returns a Subscription, and starts receiving events. We expect that users can stop receiving events via 2 methods:

Calling close() on the Subscription returned from a subscribe call.
Calling close() on the EventHubConsumerClient that spawned the Subscription.

Currently, only the 1st method works. If the user calls close() on the EventHubConsumerClient, the subscription stays open but continues to encounter errors every 10 seconds because its underlying AMQP connection is closed.
Why doesn't EventHubConsumerClient.close() stop subscriptions?
The EventHubConsumerClient creates an EventProcessor every time subscribe() is called. The EventProcessor is in charge of handling the underlying subscription.
However, once the subscribe() call returns the Subscription to the user, the reference to the EventProcessor is no longer held by the EventHubConsumerClient. Thus when close() is called on the client, it can't close the subscriptions.
How does this update fix that?
This change updates EventHubConsumerClient so that it holds onto any EventProcessor it creates in an internal Set. When the user calls close() on the client, the client calls stop() on each EventProcessor.
To make sure the EventHubConsumerClient doesn't hold onto EventProcessor references beyond when the user is done with them, we remove EventProcessors from the client's internal Set when:

The user calls subscription.close(), which removes the EventProcessor associated with the Subscription.
The user calls close() on the EventHubConsumerClient, which removes all EventProcessor references from the Set.

What about the 1 min delay before the process exits?
There were two issues in the PartitionPump and PumpManager implementations that were causing in-flight receiveBatch calls to not get cancelled when subscription.close() was called.

The PumpManager didn't store a reference to the pump it created in createPump until after starting the pump. This meant that if the user stopped a subscription before processInitialize yielded, the PumpManager didn't know about the pump existed when it went to stop it.
The PartitionPump closed the receiver before calling this._abortController.abort() inside the stop() method. This caused the event listener attached to the abortSignal passed to receiveBatch to be removed before the abort event was emitted.

Both of these code paths have been updated to switch around the order of operations to address these points.
How do we make sure this doesn't break in the future?
I've added a test for the EventHubConsumerClient that creates multiple Subscriptions and ensures they are all closed once close() is called on the client.
Question to the team
One thing I noticed is that right now, calling subscribe after a client has been closed does not throw an error. I think it probably should since the underlying connection has been closed, but that's technically a change in behavior. (I'd argue it's a bug fix though.) If we decide to throw an error on subscribe calls after close() then I can either make that change here, or in a seperate PR.

/azp run js - eventhubs-client - tests

Put PR into draft as I need to wait for https://github.com/Azure/azure-sdk-for-js/pull/9543 to be merged before I can finish writing some of the tests that were requested.

/azp run js - eventhubs-client - tests

@chradek : To your question above about subscribe() after close(): as much as I am saddened by it, I would think that if a call that previously did nothing started throwing exceptions that would qualify as an interface change, no matter how accurate.

/azp run js - eventhubs-client - tests

/azp run js - eventhubs-client - tests

Created https://github.com/Azure/azure-sdk-for-js/issues/9593 to track how we should treat subscribe() calls after close().


Fix security issues: bump alpine to 3.10

clair-scanner --ip=<IP_ADDRESS> registry-sec:latest

2019/07/29 22:03:26 [INFO] ▶ Start clair-scanner
2019/07/29 22:03:27 [INFO] ▶ Server listening on port 9279
2019/07/29 22:03:27 [INFO] ▶ Analyzing 71dfcdef6f6a027f6bffba7af1f1e2b492f442044628d9d0412b8e12d1753a8e
2019/07/29 22:03:27 [INFO] ▶ Analyzing 73459a4b5c3a721fded9b86c1bbb625bfd0377cbaa1210af4250ef8f5b4421bd
2019/07/29 22:03:27 [INFO] ▶ Analyzing 5ad54602b51de7f5954b6019f09915f1f306fc1b8f5f3f9ad8ca9da8f9f9c52b
2019/07/29 22:03:27 [INFO] ▶ Analyzing 8d3aa8b0df87b6ed1192384b5bb3ea24cd7f9befde9bc01aa3faf5c6a64a2334
2019/07/29 22:03:27 [INFO] ▶ Analyzing 3f21afda659b54940055f0dfe8913a9923de3c4cc4ccc30d9f2740b90486a3dc
2019/07/29 22:03:27 [INFO] ▶ Analyzing 2de47790f9637f56324ba24ca369b87ff4dc36078223e582083aa6544735687b
2019/07/29 22:03:27 [WARN] ▶ Image [registry-sec:latest] contains 1 total vulnerabilities
2019/07/29 22:03:27 [ERRO] ▶ Image [registry-sec:latest] contains 1 unapproved vulnerabilities
+------------+---------------------+--------------+-----------------+---------------------------------------------------------------+
| STATUS     | CVE SEVERITY        | PACKAGE NAME | PACKAGE VERSION | CVE DESCRIPTION                                               |
+------------+---------------------+--------------+-----------------+---------------------------------------------------------------+
| Unapproved | High CVE-2018-20843 | expat        | 2.2.5-r0        |                                                               |
|            |                     |              |                 | https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20843 |
+------------+---------------------+--------------+-----------------+---------------------------------------------------------------+

any updates?

@dmcgowan any ideas? From the looks of it I think the apache2-utils package can be removed from the image

@thaJeztah wdyt about this Dockerfile?
https://github.com/docker/distribution/blob/master/Dockerfile#L23-L24

quick test:
diff --git amd64/Dockerfile amd64/Dockerfile
index e966275..7650c59 100644
--- amd64/Dockerfile
+++ amd64/Dockerfile
@@ -3,7 +3,7 @@
 FROM alpine:3.11

 RUN set -ex \
-    && apk add --no-cache ca-certificates apache2-utils
+    && apk add --no-cache ca-certificates

 COPY ./registry /bin/registry
 COPY ./config-example.yml /etc/docker/registry/config.yml
(END)

and docker build/docker run:
docker run registry:latest
time="2020-04-15T21:08:38.3437775Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.11.2 instance.id=2bcabaad-ab55-45f3-af17-ac2f653215b9 service=registry version=v2.7.1
time="2020-04-15T21:08:38.3438738Z" level=info msg="redis not configured" go.version=go1.11.2 instance.id=2bcabaad-ab55-45f3-af17-ac2f653215b9 service=registry version=v2.7.1
time="2020-04-15T21:08:38.3438901Z" level=info msg="Starting upload purge in 20m0s" go.version=go1.11.2 instance.id=2bcabaad-ab55-45f3-af17-ac2f653215b9 service=registry version=v2.7.1
time="2020-04-15T21:08:38.3520526Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.11.2 instance.id=2bcabaad-ab55-45f3-af17-ac2f653215b9 service=registry version=v2.7.1
time="2020-04-15T21:08:38.3523202Z" level=info msg="listening on [::]:5000" go.version=go1.11.2 instance.id=2bcabaad-ab55-45f3-af17-ac2f653215b9 service=registry version=v2.7.1

@andriisoldatenko you mean; use that Dockerfile instead or if it should be removed there as well?
I think for the library images, multi-stage builds is currently not an option (long story; there's a complicated build system used to build a dependency tree, and that doesn't work well with multi-stage builds)
If you meant; can it be removed there as well; feel free to open a PR, but make sure to check it's not used (eg it might be in use in CI / tests in that repo: typing from my phone. so didn't check)

@thaJeztah PTAL

Thanks! I left a comment above; let me know if you need help making those changes. Also, I'm not a maintainer on this repository, so we probably need @dmcgowan to approve as well after that 🤗

done.

@dmcgowan it's just friendly reminder ;)

This PR needs a corresponding update to the docs: https://docs.docker.com/registry/deploying/#native-basic-auth as the docs for how to use basic auth are currently broken. Thanks


Populate the SUMMARY STATS LOCATION column in the published studies download (all studies v1.0.3)

The all studies v1.0.3 table downloaded from the UI (https://www.ebi.ac.uk/gwas/api/search/downloads/studies_new) does not currently include any data for the summary stats location, i.e. the column 'SUMMARY STATS LOCATION', is always null. Studies with summary stats are there, just their locations are missing. This information should be made available here for all studies that have summary statistics.
For comparison, this field does contain the correct value for all studies in the unpublished study table (All unpublished studies v1.0.3)

Please alert Daniel Suveges at Open Targets when completed


qwik-city uses node-fetch when fetch is missing, not undici

Qwik Version
0.12.1
Operating System (or Browser)
any
Node Version (if applicable)
No response
Which component is affected?
Qwik City
Problem
I noticed that Qwik-city adds node-fetch as a global if it's missing, which is fine.
However, in Node 18, global fetch is implemented by undici, which is a little different, mostly around streams: It uses web streams (just like browsers) instead of node streams, and that means that if you get a stream from undici fetch you might have to transform it to a node stream (using stream.Readable.fromWeb).
So, to prevent issues between code running on Node 16 vs Node 18, I propose that qwik-city instead uses undici to set as global fetch.

sounds fair!! could try adding a PR!?

@wmertens do you need help with the PR?

Unless I am mistaken, I think we can close this issue.
The PR #2394 has added undici

nice catch @forresst !
Thanks!


API to persist file mapped font data

Hi,
I'm filing this issue as a API feature request and question. If concluded to be reasonable, I'd be more than happy to do submit an implementation, but I'm unclear about the best API for the use-case:
I'm using fontdb with the memmap feature enabled, on a file system on macOS, Linux and Windows. Initially loading/scanning the system fonts works great with load_system_fonts. Later, when I decide to pick a particular font and fontdb returns the Face ID, the with_face_data API allows accessing the data temporarily.
When rendering text then, I'm using ttf_parser and rustybuzz (indirectly via femtovg) and that works great. But whenever rendering text, there is a chance of encountering uncached glyphs or (more likely) access to the opentype tables, etc. is required for use with rustybuzz. With the current fontdb API that means that either every time a new temporary file mapping needs to be created/destroyed (with_face_data) or the entire font data needs to be duplicated in memory (copied into a Vec<u8>).
I'd like to figure out what the right API would be for fontdb to offer a way to make fontdb persist a file mapping for a given face id, in a way that for example fontdb could return for example an Arc<dyn AsRef<[u8]>> (or basically an Arc with the traits that memmap2::Mmap implements).
I thought of a function like fn shared_face_source(&mut self, id: ID) -> Option<(Arc<dyn AsRef<[u8]>>, u32>), but I think implementation wise that wouldn't fit well with the existing Vec<FaceInfo> or more specifically the public Source enum. While that could in theory hold something like a SharedBinary(Box<dyn AsRef<[u8]>>) variant, it doesn't work in practice because Source also implements Clone.
So I'm curious: Is this a feature that you'd be willing to generally accept?
If yes, can you think of a way to implement this without breaking API compatibility?
If API compatibility is okay to break, what API would be acceptable?
Any input would be greatly appreciated - and as said: I'd be happy to get my hands dirty on an implementation.

I'm not sure I've understood the issue.
You should not load font files into memory. Some fonts, like Apple Emoji, are like 150MB big. You will run out of memory fast. The reason the ability to load fonts into memory exists is for rare cases when you do not have an actual font file (WASM or compressed fonts or something like this).
And we cannot have a persistent mmaping, afaik, because it's unsafe. You can mmap a file. Delete it. Try access it and the app will crash.
So the only solution is to mmap a file on each access. And for something as heavy as shaping or glyphs outlining, the overhead should be neglectable.
You your use case I think we can add a function that will loaded the file into memory, basically swap the Source. Like fn load_into_memory(&mut self, id: ID). So the next time you will need to access this font via with_face_data (for shaping presumably) it would be faster. But again, you will start "leaking" memory eventually. So it would be great to have a function to unload a font. We already have remove_face, but I would prefer a function that simply deallocates a font.
Maybe we can replace File(PathBuf) with File(PathBuf, Option<Vec<u8>>) or something like this.
Breaking changes are fine. But I'm not sure how can we improve this.

You should not load font files into memory.

I agree :)

And we cannot have a persistent mmaping, afaik, because it's unsafe. You can mmap a file. Delete it. Try access it and the app will crash.

I agree that there are ways to make this unsafe. For some applications that's unacceptable, for some applications it might be an acceptable risk (something I would choose to opt-in for fonts - I'm not concerned about the case of somebody truncating ttf files in the file system).
So I'm not looking for load_in_memory but an opt-in API to allow me to accept the risk.

So the only solution is to mmap a file on each access. And for something as heavy as shaping or glyphs outlining, the overhead should be neglectable.

Especially for large fonts that's an allocation of a large amount of virtual address space - repeatedly when rendering text.

Breaking changes are fine.

Here's an idea I just prototyped that maintains compatibility and permits an opt-in:
pub trait SharedFontData: AsRef<[u8]> + std::fmt::Debug {}

#[cfg(feature = "memmap")]
impl SharedFontData for memmap2::Mmap {}

impl SharedFontData for Vec<u8> {}

/// A font source.
///
/// Either a raw binary data or a file path.
///
/// Stores the whole font and not just a single face.
#[derive(Clone, Debug)]
pub enum Source {
    /// A font's raw data. Owned by the database.
    Binary(Vec<u8>),

    /// A font's path.
    #[cfg(feature = "fs")]
    File(PathBuf),

    /// A font's raw data shared between the database and the application.
    /// If the data originates from the file system, then it's a shared mapping.
    SharedBinary(Option<PathBuf>, Arc<dyn SharedFontData>),
}

and then
/// Makes the font data that backs the specified face id shared so that the application can
/// hold a reference to it.
///
/// If the face originates from a file from disk, then the file is mapped from disk. This is unsafe as
/// another process may make changes to the file on disk, which may become visible
/// in this process' mapping and possibly cause crashes.
///
/// If the underlying font provides multiple faces, then all faces are updated to participate in
/// the data sharing. If the face was previously marked for data sharing, then this function will
/// return a clone of the existing reference.
pub unsafe fn make_shared_face_data(
        &mut self,
        id: ID,
    ) -> Option<(Arc<dyn SharedFontData>, u32)>;

/// Transfers ownership of shared font data back to the font database. This is the reverse operation
/// of [`Self::make_shared_face_data`]. If the font data belonging to the specified face is mapped
/// from a file on disk, then that mapping is closed and the data becomes private to the process again.
pub unsafe fn make_face_data_unshared(&mut self, id: ID);

I can submit this with the impl as PR if this looks like an okay direction.

Sorry, but I cannot accept this. This is too unsafe and we cannot hide it behind a safe interface. It violates Rust rules. A file must not be permanently open via mmap. You have to either fork this crate or cache fonts on you side.
And no, we cannot hide it behind a feature flag.
Also, I'm not sure your code will work in the first place. memmap2::Mmap depends on std::fs::File, which will violate borrowing rules, afaik. So you cannot store memmap2::Mmap in the first place.
I understand performance implications, but this is not a solution. On the other hand, opening a file is like 0.01% of text shaping. It's hard to see when it would be a bottleneck.

I've uploaded my prototype to GH at https://github.com/sixtyfpsui/fontdb/tree/simon/shared-data . But I'll respect your choice of course. If you agree then I'll happily create a PR and polish it. If you still disagree about supporting such an API as unsafely marked then that's okay, too.
In any case I hugely admire all the software that you have written, for its stability, it's clean APIs and awesome functionality :)

Hmm... as long as those functions are marked as unsafe - I'm fine. Just add a # Safety section, like this one, to them. Also update the safety section in the library docs, at the top. Just mention that there are exceptions to temporary mapping now.
The real problem now is that I honestly do not understand how your code works. What this code does: Arc::new(data) as Arc<dyn AsRef<[u8]>>? Can you explain it to me? You're storing a trait object to original data somehow? I didn't know that this is even possible. I'm surprised that this doesn't trigger lifetime issues.

Is there something that I perhaps overlooked in the memmap2 crate?

Strange, I thought Mmap cannot outlive File, because File will close a descriptor in the destructor. Will double check later.

Hmm... as long as those functions are marked as unsafe - I'm fine. Just add a # Safety section, like this one, to them. Also update the safety section in the library docs, at the top. Just mention that there are exceptions to temporary mapping now.

Sounds good!

What this code does: Arc::new(data) as Arc<dyn AsRef<[u8]>>? Can you explain it to me? You're storing a trait object to original data somehow? I didn't know that this is even possible. I'm surprised that this doesn't trigger lifetime issues.

Sure :). You're right, this looks like the trait object is stored in the Arc. I think officially this is https://github.com/rust-lang/rfcs/blob/master/text/0982-dst-coercion.md . I think in practice this cast creates a "fat" pointer that firstly refers to the original data (for example the synthetic struct of Arc's refcount and the memmap::Map2 fd, or a Vec) and secondly to the vtable of the dyn AsRef<[u8]> implementation that fits the contained type. So the Arc implementation has a pointer to the data, which is all it needs to maintain the reference count. And the "vtable" allows the compiler to generate code to indirectly call as_ref(), where the implementation knows what's behind the data pointer as well (that's passed along).
An alternative API would be to not erase the type but have something like Arc<memmap2::Map>, but I think exposing memmapi in fontdb's public API is not desirable.

I see.
Yes, &[u8] if preferable.


Add the option to retain the selected text

What would make this extension ever more useful is an option to append the defined html to the selected text - by surrounding the selection with the html tags instead of completely removing it.
This could be an extra setting for the extension, "keepSelection: true"

I agree! Contributions are welcome. :grin:


Mobile Responsive

Mobile Responsive
User Story
AS AN user for the application
I WANT to have a good user experience
SO THAT I can meet my end goals for using the application
Acceptance Criteria
GIVEN the user would like to have a good user experience
WHEN they use this application on their mobile device
THEN the application will have nice aesthetics and will be easy to navigate and interact with

This has been completed.


Fix GETTING_STARTED link

Fixes https://github.com/thoughtbot/factory_bot/issues/1179

Thanks! I'm going to close this since we already have https://github.com/thoughtbot/factory_bot/pull/1175.


Add an Automatic Filename Normalizer (OneDrive Supported Filenames)

Hi,
I use this utility quite a lot and one thing I have noticed is that sometimes there are filenames that have characters like $.|, etc that cause this to give error:400, and when I remove those characters as suggested in Microsoft Graph API Docs, it uploads without an issue.
Another Important Issue
Required Changes for Illegal Characters

Removing White spaces in paths as follows, ./output/ something i like.pdf will not upload due to a while space before the name  something. Where as this will upload ./output/something i like.pdf.
Trailing white spaces also cause issues such as ./output/something i like.pdf  will not upload and hence it needs to be fixed.
Illegal Characters need to be removed, my implementation in python

    """
    Removes Illegal OneDrive Characters
    """

    correct_name = filename.split('/')[2]
    correct_name = correct_name.lstrip()
    # removing the illegal characters
    #  ~ " # % & * : < > ? / \ { | }
    correct_name = correct_name.replace('~', '')
    correct_name = correct_name.replace('"', '')
    correct_name = correct_name.replace('#', '')
    correct_name = correct_name.replace('%', '')
    correct_name = correct_name.replace('&', '')
    correct_name = correct_name.replace('*', '')
    correct_name = correct_name.replace(':', '')
    correct_name = correct_name.replace('<', '')
    correct_name = correct_name.replace('>', '')
    correct_name = correct_name.replace('?', '')
    correct_name = correct_name.replace('/', '')
    correct_name = correct_name.replace('\\', '')
    correct_name = correct_name.replace('{', '')
    correct_name = correct_name.replace('|', '')
    correct_name = correct_name.replace('}', '')

So, I think this should be No. 1 Priority in onedrive-uploader releases from now on.

@virtualzone I believe this Microsoft Support Page should be referenced here for a quick implementation for filename sanitation https://support.microsoft.com/en-us/office/restrictions-and-limitations-in-onedrive-and-sharepoint-64883a5d-228e-48f5-b3d2-eb39e07630fa

Hi @OttomanZ, thanks for this hint! I have implemented file and folder name sanitisation in version 0.8.


drawing: fix build with meson 0.61

Description of changes
https://hydra.nixos.org/build/171050609/nixlog/1
(The build failure is the same as the one in  #166395)
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg build drawing


Cutting down birch trees cause world to crash

Description
Cutting down birch trees cause world to crash with the message Exception in server tick loop
To Reproduce
Cut down birch tree

Versions
Include relevant versions below. If this issue relates to add-ons then include their versions too.

Dynamic Trees:1.16.5-0.10.0-Beta34
Forge:36.2.39
Java: 8
OS: Windows 10

Sorry It seems I dont know how to add my crash report

https://gist.github.com/Jollygreen0502/7c8df573a217d825da4ce5ebeaa4b2db

Cutting down some kind of trees cause crash to me too
crash-2022-09-21_15.54.37-server.txt

This is caused by PMMO. Duplicate of #720.


#1513

Fixed!
Hackaton Eindhoven, together with Bjorn Pirson

@GITS-ONLINE Thanks for contributing.
Can you add a few sentences to the PR description so we can track what changed over time? Thanks!

yes thanks @GITS-ONLINE also if you fix an issue can you add Fixes #IssueNum in your comments so it will reference the two and auto close when the PR is merged.  Thanks again


Fix SIGSEGV on Uring.wait

Fixes https://github.com/ocaml-multicore/ocaml-uring/issues/88

Looks like the behaviour of io_uring_submit_and_wait_timeout changed (see https://github.com/axboe/liburing/issues/684). Previously it returned ETIME for a timeout, but now if it managed to submit something then it returns the number of items submitted instead. Therefore, you can now get success with no CQE.
I've added a test to your branch to check the fix.

Good catch, thanks @edwintorok!


[Question] There may be invalid code in reallocSizers function

“Reallocate the box sizers for the given grid dimensions.”：https://github.com/phosphorjs/phosphor/blob/master/packages/widgets/src/gridlayout.ts#L833

Good catch! Thank you!


Update README.md

Hello,
I just add in the list the config component which is referenced in composer.json
Is this good for you ?
Cheers 😉

thanks, @abdounikarim!


🐛A bug in Caséologue or in me? 🤷🏽♀

Am I again missing something and did something severe? 😱
Or is this a 🐞?

Thanks for checking @hmenager or @albangaignard 🙏🏽🙇🏽♂

None of these @matuskalas:

you didn't break anything
caseologue isn't broken

I think it's a matter of dependency versions, as mentionned in https://stackoverflow.com/questions/78634235/numpy-dtype-size-changed-may-indicate-binary-incompatibility-expected-96-from.
I'll go with the recommended fix, pin reqs for Numpy to numpy==1.26.4.

Should be fixed by d38b999.


Table output

Maybe a table output will be better:

Package
Installed
Latest
Depends

symfony/symfony
2.3.30
2.7.5
~

foo/bar
1.0.4
2.0.5
awesome/package (~1.0)

Issue https://github.com/composer/composer/issues/4487 is needed for this.

Hey Sullivan!
Would it be possible to generate the following bower list like output instead of a table?

I think a renderer base class using adapters could make sense.
You could then implement a TableRenderer (using Symfony\Console Table helper) and a TreeRenderer.
What do you think?

Could be a second option indeed but we have to be sure the output is not confusing by specified that children tree is dependencies.
Do you have some suggestions of library to render this kind of output easily?

Indeed, listing the dependencies could be confusing, especially if you render a really large set of dependencies.
A library, hmm lets see.. this could fit: https://github.com/pbergman/tree-printer
It stays in the context of Symfony/Console.
I haven't worked with it, but it allows to define the "ascii art" forming the tree.
And it should be possible to tweak the settings to render a more compact tree.
https://github.com/pbergman/tree-printer#setformats

Here is a screenshot from VersionEye.com:

I really like this view. An output like bower list can be very confusing. I think this package should show version numbers, not a dependencies tree.


[Refactor][Cherry-pick][Branch-2.4] Fix the wrong log when preCommit failed (#16167)

The preCommit() call the getLastSuccessVersion() wrong. Should call the getLastFailedVersion() instead.
What type of PR is this：

[ ] BugFix
[ ] Feature
[ ] Enhancement
[x] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #16225
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

run starrocks_fe_unittest

run starrocks_fe_unittest