[fix][broker] Fix ResourceGroups loading

Motivation
When org.apache.pulsar.broker.resourcegroup.ResourceGroupConfigListener#loadAllResourceGroups is unfinished, the org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener will be created, this will cause the ResourceGroupNamespaceConfigListener does not find the ResourceGroup.
2023-12-21T09:41:18,749+0000 [ForkJoinPool.commonPool-worker-2] ERROR org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener - Failed to register namespace my-tenant/my-namespace with resource group rg-1
org.apache.pulsar.client.admin.PulsarAdminException: Resource group does not exist: rg-1
        at org.apache.pulsar.broker.resourcegroup.ResourceGroupService.checkResourceGroupExists(ResourceGroupService.java:417) ~[org.apache.pulsar-pulsar-broker-3.1.1.jar:3.1.1]
        at org.apache.pulsar.broker.resourcegroup.ResourceGroupService.registerNameSpace(ResourceGroupService.java:249) ~[org.apache.pulsar-pulsar-broker-3.1.1.jar:3.1.1]
        at org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener.reconcileNamespaceResourceGroup(ResourceGroupNamespaceConfigListener.java:127) ~[org.apache.pulsar-pulsar-broker-3.1.1.jar:3.1.1]
        at org.apache.pulsar.broker.resourcegroup.ResourceGroupNamespaceConfigListener.lambda$updateNamespaceResourceGroup$0(ResourceGroupNamespaceConfigListener.java:69) ~[org.apache.pulsar-pulsar-broker-3.1.1.jar:3.1.1]
        at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?]
        at java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) ~[?:?]
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?]
        at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?]
        at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?]
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?]
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?]

This process is completed asynchronously, and if it fails, an error will not be thrown to prevent the pulsar from working.
Ultimately, the rate limit failed, due to resourceGroup is null, please see org.apache.pulsar.broker.service.AbstractTopic#updateResourceGroupLimiter(org.apache.pulsar.common.policies.data.Policies):
public void updateResourceGroupLimiter(@Nonnull Policies namespacePolicies) {
    // ....
    String rgName = namespacePolicies.resource_group_name;
    if (rgName != null) {
        final ResourceGroup resourceGroup =
                brokerService.getPulsar().getResourceGroupServiceManager().resourceGroupGet(rgName);
        if (resourceGroup != null) {
            this.resourceGroupRateLimitingEnabled = true;
            this.resourceGroupPublishLimiter = resourceGroup.getResourceGroupPublishLimiter();
            log.info("Using resource group {} rate limiter for topic {}", rgName, topic);
        }
    }
    // ....
}

Modifications

Waiting for the ResourceSerivce to start successfully, if the startup fails, pulsar needs to be restarted by the users.

Compared to the failure of pulsar startup, I prefer the rate limiter to work properly.

Make loadAllResourceGroups return the CompletableFuture<Void>, and wait for the return value to be done in the constructor.
Make loadAllNamespaceResourceGroups return the CompletableFuture<Void>, and wait for the return value to be done in the constructor.

Verifying this change
Two tests have been added to verify the fail to load the resource.
Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

/pulsarbot rerun-failure-checks

Codecov Report
Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (b944f10) 73.37% compared to head (8d49c10) 73.39%.
Report is 3 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #21781      +/-   ##
============================================
+ Coverage     73.37%   73.39%   +0.01%
+ Complexity    32772    32690      -82
============================================
  Files          1897     1898       +1
  Lines        140647   140674      +27
  Branches      15489    15492       +3
============================================
+ Hits         103206   103243      +37
+ Misses        29363    29338      -25
- Partials       8078     8093      +15

Flag
Coverage Δ

inttests
24.21% <56.86%> (+0.05%)
:arrow_up:

systests
24.92% <56.86%> (+0.26%)
:arrow_up:

unittests
72.66% <80.39%> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

...ker/resourcegroup/ResourceGroupConfigListener.java
80.26% <81.81%> (+2.79%)
:arrow_up:

...rcegroup/ResourceGroupNamespaceConfigListener.java
70.58% <79.31%> (+2.73%)
:arrow_up:

... and 78 files with indirect coverage changes

Add related issue https://github.com/apache/pulsar/issues/15113

https://github.com/apache/pulsar/pull/15126 has been closed, so this PR needs to be reviewed again.

The comment has been resolved and can be reviewed again.

/pulsarbot rerun-failure-checks


[RFE] Accept third-party CAs to be trusted by all related units

There are two very common scenarios where we need to inject third party CAs on all units at once:

SSLBump proxy: a proxy acts as a MITM and generates certificates using its own CA to sign off for each HTTPS connection
mTLS support: we may need to add extra CAs from other PKIs to be trusted for a mTLS connection

These CAs should be managed the same way as the other CAs (i.e. generating events and alerts if one of the CAs will expire)

This feature request is not clear to me, what exactly is required from this operator to satisfy your needs?

@phvalguima can you please answer my comment above?

Hi @gruyaume, we need 2x config options, e.g. "extra-CAs" and "automatic-update-ca-certificates", where I can add extra CAs to all the units at once that have nothing to do with the CSRs / certificates managed by tls-certificates-operator.
For example:
juju config automatic-update-ca-certificates=True  # Defaults to False, as we may not need it; if set to True, then manage these CAs in the /etc/ssl/certificates and run update-ca-certificates command on that host

juju config tls-certificates-operator extra-cas="----- BEGIN CERTIFICATE -----
....
....
....
----- END CERTIFICATE -----
----- BEGIN CERTIFICATE -----
....
....
....
----- END CERTIFICATE -----
....
----- BEGIN CERTIFICATE -----
....
....
....
----- END CERTIFICATE -----"

Originally, an user would manage certificates for something like SSLBump proxy by injecting those certs using Juju's cloud-init userdata. That is enough to bootstrap and deploy, but cloud-init only takes effect at node boot. Updating the cloud-init also means rebooting the node.
The mTLS is another scenario: I may have to accept multiple PKIs and different CA chains for the mTLS.
Another option is to just run update-ca-certificates manually. However, every new unit, we need to remember that.
My recommendation would be the tls-certificates-operator must:

Receive the CAs from the config
pass it through the validations that we run for every certificate / CA
If the user chooses: manage the certificates with update-ca-certificates command
Make these extra CAs available to any application related to this charm

Example for SSL Bump proxy
In this scenario, we would start by using the cloud-init to get the cluster bootstrap correctly. Here is the step-by-step:
# CREATE THE CLOUD INIT WITH THE SSL BUMP CERTIFICATES
$ juju bootstrap my-cloud my-controller --config cloudinit-userdata=$(cat ./cloud-init.yaml)
$ juju add-model --model-config cloudinit-userdata=$(cat ./cloud-init.yaml)

# ... Deploy all your cool apps
# The logic above will inject the certificates in the cloud-init

$ juju deploy tls-certificates-operator automatic-update-ca-certificates=True extra-cas=".... SAME CAs FROM CLOUD-INIT"
$ juju integrate tls-certificates-operator your-cool-app(s)

# Do the same in the controller model, so we can also manage the certificates in that env

Now, we can use tls-certificates-operator for the entire lifetime of the application. The only extra step in this case is: every time we rotate the SSL Bump CA, then we need to update the tls-certificates operator AND the cloud-init script
Example for mTLS
In this scenario, we do not need to know the extra CAs at deployment time and these CAs only matter for the target-application. That means we do not need tls-certificates-operator to inject the CAs using update-ca-certificates command.
$ juju deploy tls-certificates-operator automatic-update-ca-certificates=False extra-cas="...."
$ juju integrate tls-certificates-operator your-cool-app(s)

I'm not sure how much I agree with this feature request. Your request is along the line of "We want a store of certificates that charms can relate to to get a list of trusted certificates". While I do understand this need, it's not what TLS Certificates Operator is made for at the moment.

@phvalguima I believe this should be regarded more as a generic Juju defect. Juju supports installing CA files via cloud-init and it works fine on VMs, however it (obviously) doesn't work on K8s.

Hi @grobbie the issue is the lifecycle post cloud init as well. We need to keep track, remove older and add newer CAs over time.


[6.x] Fix exception message

For some reason the path instead of the exception message is passed here which seems off. There's an exact same other use case in this class where the message does gets passed correctly. Path is available anyway in the passed-through exception.
Fixes https://github.com/laravel/framework/issues/33824

My only comment is that this is breaking for someone who was using the exception message to determine the path of the file.

Hmm correct but tbvh there's two different usages in the same class. This one makes the most sense. The path is again available just the same in the underlying Flysystem exception.


error in installing pyrealsense2 from source

Required Info

Camera Model
{ D435 }

Firmware Version
Will update soon

Operating System & Version
Ubuntumate

Kernel Version (Linux Only)
will update soon

Platform
PC/Raspberry Pi3 B+

SDK Version
{  2.0 }

Language
{python }

Segment
{Robot }

Issue Description
I was successfully able to install the realsense sdk from source. Verified proper installation by running the realsense viewer.
Right now, I am trying to install Pyrealsense2 from source and got stuck due to errors. The steps are from the following link:
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python
Getting stuck in step 3. Execution of the CMAKE command results in error in line 16 and line 57 of the CmakeList file inside the python folder in wrappers. In order to perform step 3, I executed following command from python folder:
cmake ../ -DBUILD_PYTHON_BINDINGS=bool:true
Line16-->
set_target_properties(pyrealsense2 PROPERTIES VERSION
${REALSENSE_VERSION_STRING} SOVERSION "${REALSENSE_VERSION_MAJOR}.${REALSENSE_VERSION_MINOR}")
error:
Incorrect number of parameters.
Line57-->
if(${BACKEND} STREQUAL RS2_USE_V4L2_BACKEND)
error:
Unknown parameters
It seems that the error is as a result of Cmake not able to get the values of REALSENSE_VERSION_STRING and BACKEND variables.
Enclosing $REALSENSE_VERSION_STRING and $BACKEND in "" resolves the error but I do not think this is the right way forward.
Any help would be great.

@rijo11811  hello,
Building python wrapper from source requires invoking the CMake from the topmost (i.e. Librealsense's root) directory.

@ev-mp hello,
Your input resolved the issue.


iOS 15 ready?

Hey
I have been using your SC long now.
i am currently beta testing.
it looks like some commands have been redesigned by Apple (I.e. „includes“, „starts with“)
Are you thinking about porting it to iOS 15 in the next weeks?
thanks
BTW it looks like the update command is not working properly in iOS 15 PB3. An issue with the update link.

I only have my main device where I test the shortcut, so I plan to update to iOS 15 when the stable version releases and then fix any issues that are out there.


Fix NullPointerException in ApexStateInternals

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[ ] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify.
[ ] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[ ] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

Coverage increased (+0.02%) to 70.857% when pulling 4b4b9720151d60f64fabb9bd2e22c92ce47f2166 on alexandrvl:fix_ApexStateInternals_NullPointerException into 0e429b33ff85eba08da5018c9febd0b99b44f720 on apache:master.

Can someone please review?

R: @tweise

@alexandrvl the error you see is symptom of an invalid dependency artifact in your local .m2:
Please see<EMAIL_ADDRESS>

@tweise Thank you! Can this pull request be closed?


[fluidsynth x86 / x64] build failure

Host Environment

Windows 8.1 x64
MSVC++ 14.25 Visual Studio 2019

To Reproduce
vcpkg install fluidsynth:x86-windows
or
vcpkg install fluidsynth:x64-windows
Failure logs
D:\Code\Repos\vcpkg>vcpkg install fluidsynth:x86-windows
Computing installation plan...
The following packages will be built and installed:
    fluidsynth[core]:x86-windows
Detecting compiler hash for triplet x86-windows...
Starting package 1/1: fluidsynth:x86-windows
Building package fluidsynth[core]:x86-windows...
Could not locate cached archive: C:\Users\Daniel\AppData\Local\vcpkg\archives\5e\5e86e84f5b9760afee78566b4949c31b3fe8cf76.zip
-- Downloading https://github.com/FluidSynth/fluidsynth/archive/37c9ae2bf431a764032f023b3b2c0c0b86b7c272.tar.gz...
-- Extracting source D:/Code/Repos/vcpkg/downloads/FluidSynth-fluidsynth-37c9ae2bf431a764032f023b3b2c0c0b86b7c272.tar.gz
-- Applying patch force-x86-gentables.patch
-- Using source at D:/Code/Repos/vcpkg/buildtrees/fluidsynth/src/0b86b7c272-b21350e92b.clean
-- Configuring x86-windows
-- Building x86-windows-dbg
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:139 (message):
    Command failed: "C:/Program Files/CMake/bin/cmake.exe" --build . --config Debug --target install -- /p:VCPkgLocalAppDataDisabled=true /p:UseIntelMKL=No /m
    Working Directory: D:/Code/Repos/vcpkg/buildtrees/fluidsynth/x86-windows-dbg
    See logs for more information:
      D:\Code\Repos\vcpkg\buildtrees\fluidsynth\install-x86-windows-dbg-out.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_build_cmake.cmake:91 (vcpkg_execute_build_process)
  scripts/cmake/vcpkg_install_cmake.cmake:24 (vcpkg_build_cmake)
  ports/fluidsynth/portfile.cmake:16 (vcpkg_install_cmake)
  scripts/ports.cmake:79 (include)

Error: Building package fluidsynth:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: fluidsynth:x86-windows
  Vcpkg version: 2020.06.15-nohash

Additional context
It's exactly the same situation I described in https://github.com/microsoft/vcpkg/issues/9628 more than 6 months already, just introduced for this port with the latest update. CMake targets the Windows 8.1 SDK for configuration and the build fails because the code is compiled against the latest Win 10 SDK. As I explained in the issue I just linked a temporary "fix" is to add
CMAKE_SYSTEM_VERSION=10.1.19041.1
to the CMake configure options in the resp. portfile, but this is NO proper / valid solution, esp. seeing that now more ports starting to have the same issue. I would actually consider this a tool problem instead of a "port bug". Please check the attached .zip for all files created during the "installation" (x86), thank you.
fluidsynth.zip

Closing this issue since it has been solved.
Please see comments here https://github.com/microsoft/vcpkg/issues/12935#issuecomment-674798858.


Possibility to add an "error" event listener

We are using the interceptor in our application not for mocking, but for intercepting all requests made to (trace/debug) log all requests and responses and some metadata request id, duration, ...
We had the problem that some of the requests fail due to network issues ("socket hang up", "connection refused", ...) that lead to an error instead of an actual response.
The problem is that there is no event that is available to listen for an error like that. Is there any possibility to either implement an "error" listener or are there any workaround to get the error in the interceptor?

Hi, @marcelkottmann. Thanks for raising this.
What would classify for the "error" event? Any request error?
I can imagine a usage like this:
interceptor.on('request:error', ({ request, cause }) => {
  console.log('request failed due to:', cause)
})

Where cause is the original error that resulted in the request's failure.
I think people would want for the request:error event to maintain the request's context so they would be able to retry it. Interceptors currently doesn't support affecting requests at this stage. Request error implies the request was passthrough—it hasn't been handled in the request event. I see this new event as readonly.
How does that compare to your vision of this event and its usage?

Hi @kettanaito ,

What would classify for the "error" event? Any request error?

I would like to be as generic as possible, that means if the underlying request-implementation (e.g. fetch) throws an error (i.e. rejects the promise) this error should be captured and be provided to the listener.

I can imagine a usage like this:
interceptor.on('request:error', ({ request, cause }) => {
  console.log('request failed due to:', cause)
})

Where cause is the original error that resulted in the request's failure.

Thats exactly how I also would imagine this api.

I think people would want for the request:error event to maintain the request's context so they would be able to retry it. Interceptors currently doesn't support affecting requests at this stage. Request error implies the request was passthrough—it hasn't been handled in the request event. I see this new event as readonly.

It feels like I don't fully understand this section, but in case of an error event I wouldn't strive for having retry functionality inside the interceptors listener. Retry logic is the responsibility of the code that originally requests something.

How does that compare to your vision of this event and its usage?

That looks very promising thank you!

I think this should be possible. The request:error event will be dispatched in different situations based on the interceptor:

fetch, the response promise has rejected;
ClientRequest, the error event has been emitted (that should cover socket errors too);
XMLHttpRequest, the error event gets dispatched.

One thing to keep in mind is to separate request errors from any exceptions happening during the request listener phase. Basically, make sure this new feature and the unhandledException don't intersect.


how to add className to calendar itself, not to input. ( just className, not style)

I need to add className to calendar itself, not to input button.
I see some issues created by others here , but I can't find similar issues.

+1


Only part of page (current viewport) rendered to PDF

Which version of PhantomJS are you using? Tip: run phantomjs --version.
2.2.0-development
What steps will reproduce the problem?
I'm trying to render page, that is longer them viewport. But rendered PDF is cropped by viewport. On the old version of phantomjs I have got PDF with full page

My code:
var page = require('webpage').create();
page.open('http://allkharkov.ua', function() {
page.viewportSize = { width: 1920, height: 1080 };
page.render('page.pdf');
phantom.exit();
});
My resulting PDF:
page.pdf

Which operating system are you using?
ubuntu 16.04
Did you use binary PhantomJS or did you compile it from source?
compile

I have tried to solve this problem and added some code to webpage.cpp to function renderPdf before     m_mainFrame->render(&painter):
QSize contentsSize = m_mainFrame->contentsSize();
contentsSize -= QSize(m_scrollPosition.x(), m_scrollPosition.y());
m_customWebPage->setViewportSize(contentsSize);
and rebuilt phantomjs. I have got fullsize pdf. After that I try to change scroll position, but on pdf I get result without scroll.

I tried new build of Vitallium 2.5.0-development, but problem still exists. Only part of PDF is rendered, other part is white

+1 I am having the same problem .. works correctly in 2.1.1 but only rendering the first page


ld: library not found for -lnsync

Hi! I integrated the pod (1.4.1) like this:
pod 'TensorflowPod', :podspec => 'https://raw.githubusercontent.com/rainbean/TensorflowPod/master/TensorflowPod.podspec',
then did pod install. But unfortunately it cannot be compiled.
Xcode 9.2 (high sierra 10.13) during my proj compilation (linking) says:
ld: library not found for -lnsync
clang: error: linker command failed with exit code 1 (use -v to see invocation).
Any ideas how to fix it? Thank you.

Fix in commit #5a5985f9, repack as hotfix at v1.4.1

I updated pod, but now a bunch of errors (hundreds) appears:
Undefined symbols for architecture x86_64:
"std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >::insert(unsigned long, unsigned long, char)", referenced from:
google::protobuf::operator<<(std::__1::basic_ostream<char, std::__1::char_traits >&, google::protobuf::uint128 const&) in libprotobuf.a(int128.o)
"std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >::at(unsigned long) const", referenced from:
google::protobuf::io::Tokenizer::IsIdentifier(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) in libprotobuf.a(tokenizer.o)
...

Reopen the issue due to missing podspec update

@michaelhuman, see if latest commit e70fdc30 helps, local environment/projects work fine. You might need to update pod specs
### close xcode

$ cd PROJECT_FOLDER
$ pod update

# reopen xcode and build

no luck :(. I tried to clean everything I could: pod clean, derrived data clean etc. I created new XCode proj, but got same complitaion errors

@michaelhuman in order to narrow down the root cause, could you reference previous release v1.3.1 instead?
In your project Podfile
pod 'TensorflowPod', :podspec => 'https://raw.githubusercontent.com/rainbean/TensorflowPod/v1.3.1/TensorflowPod.podspec'

damn, I fixed it: I just added .mm file to project and it switched the compiler to the correct (C++?) mode. Now everything is ok while compiling with v.1.4.1!
Thanks for help!

Nice to hear that!
There are more than few undocumented pitfalls on the path to integrate with TensorFlow :)
First tip is to rename objective-c code as C++ (*.mm), another is to create bridge header for Swift project.


tests: add test for bug 6008 SMB_COM_WRITE_ANDX issue - v3

Bug #6008
pre-pr: https://github.com/OISF/suricata-verify/pull/1203

Waiting for https://github.com/OISF/suricata/pull/9047

Rebased in https://github.com/OISF/suricata-verify/pull/1260


Remove usage of unimplemented TargetHealthCon

Why / Balance
#29922
These utility queries were possibly written without the intent to use the behaviour  I implemented in that PR.

Looks like this one superceded.

Looks like this one superceded.
By what?


Question about dataloader and lengths input

Hi,
I have a question about your dataloader for GPO's input "lengths". In fact, the TextEncoder's GPO module need captions' lengths but the dataloader return the "index"??
Is any problems in your code or i make a misunderstood？
https://github.com/woodfrog/vse_infty/blob/c9943b2327a568bb9d1a628bc53f74f21eb97c75/lib/datasets/image_caption.py#L95
https://github.com/woodfrog/vse_infty/blob/c9943b2327a568bb9d1a628bc53f74f21eb97c75/lib/encoders.py#L175

Hi,
The length of the caption is generated for each batch in the collate function here: https://github.com/woodfrog/vse_infty/blob/c9943b2327a568bb9d1a628bc53f74f21eb97c75/lib/datasets/image_caption.py#L277

Oh! I see. Thanks!!!


Error: Output is uniformly zero; network calibration failed.

When trying to generate INT8 Calibration file with the following command -
retinanet export main_resnet50fpn.pth engine.plan --batch 1 --int8 --calibration-table 960x540_retinanet_calib_table_new --calibration-images /data/main_dataset/val_images/ --calibration-batches 500 --size 540 960
It fails with the following error:
633:Conv -> (36, 5, 8)
634:Sigmoid -> (63, 68, 120)
635:Sigmoid -> (63, 34, 60)
636:Sigmoid -> (63, 17, 30)
637:Sigmoid -> (63, 9, 15)
638:Sigmoid -> (63, 5, 8)
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...
Tensor (Unnamed Layer* 281) [PluginV2]_output_0 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 281) [PluginV2]_output_0 copy is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 280) [PluginV2]_output_0 copy is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 282) [PluginV2]_output_2 copy is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 282) [PluginV2]_output_0 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 281) [PluginV2]_output_2 copy is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 282) [PluginV2]_output_0 copy is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 280) [PluginV2]_output_2 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 280) [PluginV2]_output_0 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 281) [PluginV2]_output_2 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 282) [PluginV2]_output_2 is uniformly zero; network calibration failed.
Tensor (Unnamed Layer* 280) [PluginV2]_output_2 copy is uniformly zero; network calibration failed.

Even though it clearly shows network calibration failed, It still generates the Calibration table file. When I plugin the Calibration table during inference, the inference removes the entire content in Calibration table file and replaces it with "1" and resulting in a segfault.
@florischabert  @pkashinkunti Can you please take a look into this and tell me if I am missing something?

@adithya-p can you try out the 19.09 branch?

I tried the 19.09 branch for calibration, When I try plugging the calibration table to export the model using the master branch, It generates the following error:
Building engine...
Building INT8 core model...
Building accelerated plugins...
Applying optimizations and building TRT CUDA engine...
Calibration failure occured with no scaling factors detected. This could be due to no int8 calibrator or insufficient custom scales for network layers. Please see int8 sample to setup calibration correctly.
Segmentation fault (core dumped)

I see that you have used an upgraded version of TensorRT in 19.09, Is that a mandatory requirement during inference also? or Can I stick to the same TRT version(used in the master)?

@adithya-p, I see this error only when the calibration images flag is incorrectly specified. Can you check if the path /data/main_dataset/val_images/ is correct?

@yashnv Correct me if I’m wrong, I already have the CalibrationTable file generated, So When I run the inference since readCache is enabled It will read the calibration table file and skip the calibration process. So images path shouldn't come into the picture, right?

Calibration table is only successfully created after a successful export. You need to export again, with correct images path.

@yashnv I have this CalibrationTable File If I pass this CalibrationTable as a parameter to export engine plan, It should skip performing Calibration(and It shouldn't check) and make use of the file I am passing, right? Let me know If I am missing something here?

@adithya-p if you changed your environment, then you will need to re-calibrate the table for Xavier. Engine plans are device and environment specific, so are the calibration tables.

@yashnv I know that Engine plans are device and environment-specific but didn't know that same applies for calibration tables. Thank you.


Tests and files

Issue #, if available:
Description of changes:

OpenML dataset used in docs/tutorials/course/mlp.md downloaded from S3, not from OpenML
Stuff from this tutorial also added as unit test
Comparison against GPy unit test, which also depends on files in S3 bucket

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The CI always breaks with:
tests/test_check_style.py .                                              [  0%]
tests/unittests/test_bayesopt_searcher.py ..                             [  1%]
tests/unittests/test_checkpoint_resume.py ..                             [  2%]
tests/unittests/test_classification_tricks.py Fatal Python error: Aborted
Thread 0x00007ff32e933700 (most recent call first):
Does anybody know how to get more information? Which of the tests is failing, and where?
All of this runs fine locally on my Mac.

@Innixma @zhanghang1989 Could it be the CI system is running several tests on different CPUs on the same instance?
This is something that gaves us trouble in the past. The test test_bayesopt_searcher.py, which is really just Thibaut's tutorial, is trying to claim all CPUs on the instance for this reason.
I can only say:

This test runs fine when run separately on an instance
Thibaut's tutorial also seems to work as part of the CI, at least it did in the past

@szha As you seem to be most knowledgable about the CI system, could you have a look at this one?
Question: Is the CI system using multiple processes (or even threads) to run several tests on different CPUs on the same instance?
If so, this is something that tends to clash with MXNet, even if things are running on CPU only. If this is possible, it would be great to check what happens if the CI system is forced to run one test at a time, so that we do not run into these issues that we know affect AutoGluon in any case.

Latest update:

test_comparison_gpy goes through
test_bayesopt_search seems to cause the failure, and is this is likely because several tests are run at the same time on one instance, which we know is a major issue with MXNet

I see the following alternatives:

Remove test_bayesopt_search: I am MUCH against that, this would be very risky
Make sure that at least for test_bayesopt_search, but maybe for all tests, the CI system does
not run it (or them) in parallel
Split tests into such which can be run in parallel, and such which cannot. I'd strongly recommend all test which really run an experiment (i.e., scheduler.run(); scheduler.join_jobs()) not to be run in parallel, as this issue will otherwise creep in later

Job PR-523-5 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-523/5/index.html

One more datapoint:
The code in the failing unit test test_bayesopt_searcher.py is pretty much exactly the same code that is in the tutorial in docs/tutorials/course/mlp.md.
And the latter works in the CI system. As you can see, when I removed the test_bayesopt_searcher.py, everything went through, and when I look at the output of
Job PR-523-5, then the tutorial outputs looks all fine. It even ran more trials of HPO, because it has a time limit of 2 mins.
I browsed around, and usually you have to be pretty explicit to tell pytest to run things in parallel (and the command in Jenkinsfile does not do that), so I really do not know what is going on here

I'm not very well versed in how the CI functions, perhaps @zhreshold has more insight?

Latest update:

test_comparison_gpy goes through
test_bayesopt_search seems to cause the failure, and is this is likely because several tests are run at the same time on one instance, which we know is a major issue with MXNet

I see the following alternatives:

Remove test_bayesopt_search: I am MUCH against that, this would be very risky
Make sure that at least for test_bayesopt_search, but maybe for all tests, the CI system does
not run it (or them) in parallel
Split tests into such which can be run in parallel, and such which cannot. I'd strongly recommend all test which really run an experiment (i.e., scheduler.run(); scheduler.join_jobs()) not to be run in parallel, as this issue will otherwise creep in later

If you suspect it's the parallel test who caused the problem, probably you can try pytest -m "serial" to verify? For the ci instance, I don't think there's something we can control for resources that request ALL CPU, if I remember it correctly, the cpu resource manager does not even care about it?

https://stackoverflow.com/questions/56659700/pytest-pytest-benchmark-serial-or-parallel
https://github.com/pytest-dev/pytest-xdist/issues/84
All are saying pytest never runs tests in parallel. To do so, one would have to use pytest-xdist, we are not doing that.
@zhreshold According to docs, -m "serial" just means pytest will only execute tests marked as serial.
Given that pytest by default executes tests sequentially, I am very puzzled about the CI out with the percentages. It just looks like things are happening in parallel.
@zhanghang1989 It would be great if you had a look at this.
Otherwise, the only option I see for now would be to remove this test. We'd then rely on the docs generation to break if anything is wrong. I find this pretty dangerous moving forward.

One guess is that internally in the test, mxnet is run in async mode causing the bug harder to catch.
So if it fails inside of test_bayesopt_searcher.py it will very likely cause the following error
tests/test_check_style.py .                                              [  0%]

tests/unittests/test_bayesopt_searcher.py ...               [  1%].   <- failed here

tests/unittests/test_checkpoint_resume.py ..                             [  2%]

tests/unittests/test_classification_tricks.py Fatal Python error: Aborted

When I search for Fatal Python error: Aborted issue, there's similar problem here and there pointing to memory allocation error, which sounds relavant since the CI instances have 32G memory in total. @mseeger Can you verify if it's the case?
I would also try to synchronize fully for mxnet with mx.nd.waitall() after each test.

You can try either adding waitall between tests (e.g. we did so in mxnet with pytest in conftest.py) or setting the MXNET_ENGINE_TYPE=NaiveEngine in the CI to get unblocked on the asynchronous execution and see which test breaks.

This still fails. Maybe @zhanghang1989 has an idea?
Otherwise, I will remove this test.

Job PR-523-11 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-523/11/index.html

OK, when I remove the unit test, the CI passes now.

OK, when I remove the unit test, the CI passes now.

Could you open an issue relating to this test since it is still unknown why it fails? Also, I will let @zhreshold decide on merging, as I lack context for this PR.

@Innixma Should I wait with raising the issue until this is merged? The test code is part of this PR, but it is marked as pytest.mark.skip, so is not executed. Without the code, it'd be hard to state the issue.
This PR is also modifying the mlp.md tutorial, and adds a unit test for the GP code.

@Innixma Should I wait with raising the issue until this is merged? The test code is part of this PR, but it is marked as pytest.mark.skip, so is not executed. Without the code, it'd be hard to state the issue.
This PR is also modifying the mlp.md tutorial, and adds a unit test for the GP code.

That is fine, it can be created after the PR is merged.

@zhreshold Could we merge this? It does a few things @jwmueller asked for, in particular removes dependence on openml (which can lead to CI failures) and also adds a unit test for our GP code.
The issue with the end-2-end bayesopt unit test remains, but I'd open another issue.


Bug: Tables lose their sorting status when interacting with them

Description of the bug
Each time a rerendering of the data is performed (change filter or for example set a column as favorite) the table lose the information about the sorting status, for example:
external interaction:

internal interaction

I know that's probably a react-table issue but maybe, given that the code is something like
 <Table
        columns={columns}
        data={data}
        customSorting={tableCustomSorting}
        initialSorting={[{ id: "traffic", desc: true }]}
      />

using rerendering, if we were given props like: onSortChanged which returns the sort key, we could pass as new initialSorting key the key just received
Playroom reproduction
No response
Version
0.11.3
What browsers are you seeing the problem on?
Chrome, Safari, Firefox, Edge

Hi @thecreazy, thanks for reporting!
In your example, is customSorting memoized (with useCallback for instance)? This is required by the underlying react-table API, but I've just realized we don't advertise this in the documentation.

@gabro  yes this is the actual implementation of the customSorting
  const tableCustomSorting: TableCustomSorting = useCallback((rows, sortFns) => {
    const otherSortingFns = <T extends typeof rows[number]>(a: T, b: T) => {
      let res = 0;
      for (let i = 0; i < sortFns.length; i++) {
        res = sortFns[i](a, b);
        if (res !== 0) {
          break;
        }
      }

      return res;
    };

    return rows.slice().sort((a, b) => {
      const res = starSorting(a.original.star?.value, b.original.star?.value);
      if (res === 0) return otherSortingFns(a, b);

      return res;
    });
  }, []);

Hi @thecreazy, we found the bug and fixed it. It was a default behavior of react-table that we flipped. It will be released as 0.12.1 soon

thx @gabro


template new naml projects from github

we should be able to template projects from github, so maybe we can release different samples when we cut project releases

ideas for commands:
naml init
naml new webservice myservicename
naml .
naml new pod mypodname
naml new deployment mydeploymentname


global shortcut to hide/show the app

ref - https://github.com/KryDos/todoist-linux/issues/3
It would be great to have global short cut to show or hide the app from/to tray
see the screenshot attached here - https://github.com/KryDos/todoist-linux/issues/3#issuecomment-387346101 to use same keybinding as official client

hey @saikatm
I've just added shortcut for hide and show the app window (CTRL+ALT+T).
It looks useful for me too, so thanks a lot for the idea.
The project's readme file was updated with link to exe file and also you can find EXE on releases page here

Thank you so much!! Keep up the good work. 👍


Mitogen does not support Ansible 2.8 yet

Which version of Ansible are you running? 2.8rc3

Is your version of Ansible patched in any way? no

Are you running with any custom modules, or module_utils loaded? yes

Have you tried the latest master version from Git? yes - this is with latest master

Mention your host and target OS and versions

Host - EL7.6

Mention your host and target Python versions

python 2.7.5
PLAY [webfilter*] **********************************************************************************************************************************************************
[mux  9691] 17:21:18.993407 D mitogen: mitogen.service.Pool(0x7f124f022190, size=32, th='MainThread'): initialized
[mux  9691] 17:21:18.996991 D ansible_mitogen.process: Service pool configured: size=32

TASK [Gathering Facts] *****************************************************************************************************************************************************
task path: /export/home/orion-admin/ansible-boulder/playbooks/e2guardian.yml:2
Sunday 12 May 2019  17:21:19 -0600 (0:00:00.185)       0:00:00.185 ************
ERROR! Unexpected Exception, this is probably a bug: expected string or buffer
the full traceback was:

Traceback (most recent call last):
  File "/bin/ansible-playbook", line 110, in <module>
    exit_code = cli.run()
  File "/usr/lib/python2.7/site-packages/ansible/cli/playbook.py", line 123, in run
    results = pbex.run()
  File "/usr/lib/python2.7/site-packages/ansible/executor/playbook_executor.py", line 170, in run
    result = self._tqm.run(play=play)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 249, in run
    play_return = strategy.run(iterator, play_context)
  File "/export/home/orion-admin/ansible-boulder/mitogen/ansible_mitogen/strategy.py", line 254, in run
    lambda: run(iterator, play_context)
  File "/export/home/orion-admin/ansible-boulder/mitogen/mitogen/core.py", line 561, in _profile_hook
    return func(*args)
  File "/export/home/orion-admin/ansible-boulder/mitogen/ansible_mitogen/strategy.py", line 254, in <lambda>
    lambda: run(iterator, play_context)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/strategy/linear.py", line 306, in run
    self._queue_task(host, task, task_vars, play_context)
  File "/export/home/orion-admin/ansible-boulder/mitogen/ansible_mitogen/strategy.py", line 229, in _queue_task
    class_only=True,
  File "/export/home/orion-admin/ansible-boulder/mitogen/ansible_mitogen/strategy.py", line 98, in wrap_connection_loader__get
    return connection_loader__get(name, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/loader.py", line 557, in get
    path = self.find_plugin(name, collection_list=collection_list)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/loader.py", line 506, in find_plugin
    plugin = self._find_plugin(name, mod_type=mod_type, ignore_deprecated=ignore_deprecated, check_aliases=check_aliases, collection_list=collection_list)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/loader.py", line 395, in _find_plugin
    if (is_collection_ref(name) or collection_list) and not name.startswith('Ansible'):
  File "/usr/lib/python2.7/site-packages/ansible/utils/collection_loader.py", line 298, in is_collection_ref
    return bool(_collection_qualified_re.match(candidate_name))
TypeError: expected string or buffer
[mux  9691] 17:21:19.052346 D mitogen: Waker(Broker(0x7f124f00fa10) rfd=6, wfd=8).on_disconnect()
[mux  9691] 17:21:19.052735 D mitogen: Router(Broker(0x7f124f00fa10)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg.

this happens with the 2.8 release too

For what it's worth, I get the same results with Python 3.7.3 and ansible 2.8.0

Some initial work has started in the issue587 branch, but it's far from complete. Summary of the work so far:

Support for the obsolete config system has been significantly dropped in 2.8, meaning that Connection plugins and (new) Become plugins are responsible for looking up their own configuration values. This is progress, but it puts Mitogen in a tough situation since we want to maintain support for Ansible 2.3/2.4, where the new config system was either non-existent or in a very immature state.
I will not drop support for 2.3 since the extension has 2.3 existing users, and newer Ansibles broke the syntax of working configurations.
It seems the option here is to internally reimplement the new configuration system. That's not a great deal of work, and in fact it's an opportunity to further improve the robustness of connection delegation (by merging the config lookup method for 'direct connect' and 'delegated connection'). I also like this option because upstream's new config system regularly features quite high up in profiler output, and so there is potential to improve on its performance.

Support for become plugins. This sticker here is supporting connection delegation -- it must be possible for an intermediary machine to call back into the become plugin API hosted on the master machine. This again, isn't a huge amount of work, but I've been putting it off to avoid reimplementing the whole thing twice.
There is currently a very large stream-refactor branch that introduces "internally asynchronous" connect. This is a major design change with many benefits, and I'd like to land it first before addressing become plugins, because the become plugin implementation will also need to be asynchronous to match the new implementation.

AFAIK it should be possible to cut a release without become plugin support, and few users will notice the difference, but for now the config system issue is a blocker. It should only take a few days to implement, but I'm short on time for the coming months.

If anyone wants to give issue587 branch a tyre kick - it would be very useful :)
Some internal variables have changed semantic (so CI is still broken), but I overestimated how much compat code has been removed. It's possible this will be a much quicker success than expected

Bare minimum implementation seems to be done now -- please open fresh issues for anything you bump into, but CI is passing so I'm happy :)

This is now on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://www.freelists.org/list/mitogen-announce
Thanks for reporting this!

Thanks for working on this.  I'm running into #590 now, so following that.

Hi @opoplawski, there is now a workaround for #590 on current master. Note CI is currently only set up for 2.8.0 Py2.7/Py3.5 host -> Py2.7 interpreter, so another variant of #590 may be lurking in the -> Py3.5 interpreter direction

Thanks for fixing this - is there any planned release so we can profit from this fix using pip install ?

@varac a 2.8-compatible release is coming real soon. I'm pushing 2 big branches as part of the same release, as they're both related to 2.8 features. That means we'll have one interim featureful release with functional 2.8 support, and all the features in place needed for complete 2.8 support in a subsequent 'mostly fixes only' followup release.

Hey there! Thanks for all of your work on this. Anything released for 2.8 yet? :)

@armenr : https://mitogen.networkgenomics.com/changelog.html#v0-2-8-2019-08-18

Roger that! @makkus - thanks for the ref. All I had to do was RTF to know it. My bad, and thanks for all your excellent work!


Fix for Parcel 2

Seems like Parcel 2 can't detect the dependency when it's URL is produced dynamically.

👍


Pods deployed in istio-injected namespace are not getting started after scheduled shutdown

Hi All,
We are using istio in EKS.
We have 4 worker nodes, we used to shutdown the worker nodes as part of the daily shutdown(edit the autoscaling group min and max to zero) and provision the instance next day morning(edit the autoscaling group).
Once the worker nodes comes up all the pods which has istio-label injected are not getting restarted automatically.
Even all the istio components in istio-system namespace is running fine.
It is happening only to the pods deployed in namespace which has istio-injection=enabled. When I check the logs in replica set, I could see the below error.

Hey @dhineshbabuelango thanks for reporting. This is a somewhat common probably we are tracking in https://github.com/istio/istio/issues/13840. I am going to close this so we can consolidate our efforts there.
I would recommend reading through there for some more info, as well as https://github.com/istio/istio/wiki/Troubleshooting-Istio#sidecar-injection. Thanks!


Multi Tennant Application Causes Timeout To SQL Server When Making Heavy Use Of Redirect URI;s

Hello,
I am running a multi-tenant application, each tenant uses the same client, but has their own redirect URIS's in the client. Once this number goes above around 80 when clicking to view the client details there is an extreme slowdown in the application when trying to look at the details of the client (/Configuration/Client/).
This is occasionally so long the SQL connection times out. I know its a lot of redirects, but it smells like theres a serious performance problem somehwhere in the Entity / SQL that retrieves client details.

Hi, please check this issue: #536
Thanks for reporting this


Re-order Quick actions menu

Just reordered Quick action menus to keep them consistent, lowercasing them. In Datasets the first option is Create map as it's the most relevant.
Acceptance
Make sure Quick actions menu in datasets appears in the following order:

Create map
Edit name & description
Manage tags
Change privacy
Share with colleagues
Duplicate
Lock
Delete

Related Issue
https://github.com/CartoDB/product/issues/282

Everything looks good in Acceptance! ✨


add query string to static assets with cms version

in order to bust cache every time new release is made
reopen from #4872
should fix https://github.com/divio/django-cms/issues/4808

We need to come up with a better solution for managing our static assets then query-params, but this LGTM otherwise.

Coverage increased (+0.02%) to 89.281% when pulling 85e411ba30e1bef7c0cd387df8b8dc47af6e7251 on feature/add-query-param-to-bust-cache-on-static-assets into acce5422caab96dfc18f2110bd1da5a25b4f663a on release/3.2.x.


how to save a history of previous input ?

Thanks to https://github.com/beryx/text-io/issues/17 that I now have the autocomplete setup for my program.
There are times that I would like to reenter the same input that I made in the past.
Is there an elegant way to program the Left and Right arrow keys to scroll backward and forward the input ?
I come up with this in SwingHandler but fall short of what to do next ?
    private static final String KEY_STROKE_LEFT = "pressed LEFT";
    private static final String KEY_STROKE_RIGHT = "pressed RIGHT";
    private String historyInput = "";
    private int historyIndex = -1;
    private String[] history = {};

  public SwingHandler(SwingTextTerminal terminal) {
        this.terminal = terminal;
        terminal.registerHandler(KEY_STROKE_LEFT, t -> {
            if (historyIndex >= 0) {
                historyIndex--;
                String text = (historyIndex < 0) ? historyInput : choices[historyIndex ];
                t.replaceInput(text, false);
            }
            return new ReadHandlerData(ReadInterruptionStrategy.Action.CONTINUE);
        });

You need a way to store and retrieve previously entered values.
In the demo app I implemented the history store using a properties file.
You may want to replace this solution with a more robust persistence mechanism.
I used Ctrl-Shift-Left and Ctrl-Shift-Right instead of just Left and Right, because I didn't want to disable the standard way to move the caret.

That's very awesome ! Using properties file to save the history will do the job well in my case.


Errors: Address containing special characters (#, &)

Case #1: Pound Sign in the Address
Ex. 1000 Mineral Point Ave. #105B, Janesville, WI 53548
System Throws: UltimateWorkflowToolkit.CoreOperations.System.GeoCodeAddress: System.Net.WebException: The remote server returned an error: (401) Unauthorized.
Case #2: Ampersand sign in the Address
Ex. 4th & Walnut Streets, Lebanon, PA 17042
System Throws: UltimateWorkflowToolkit.CoreOperations.System.GeoCodeAddress: System.IndexOutOfRangeException: Index was outside the bounds of the array.

Addresses in <IP_ADDRESS> build


chokidar support - fsevents

Using chokidar inside an gulpfile.js but not in any code called by web.js: devtool web.js throws this error, while node web.js doesn't.

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
https://github.com/paulmillr/chokidar/blob/69a549315cd862f54942c042edfd42edfabadb55/lib/fsevents-handler.js#L22
TypeError: fsevents is not a function
    at createFSEventsInstance (/some-project/node_modules/chokidar/lib/fsevents-handler.js:22:11)
    at setFSEventsListener (/some-project/node_modules/chokidar/lib/fsevents-handler.js:67:16)
    at FSWatcher.FsEventsHandler._watchWithFsEvents (/some-project/node_modules/chokidar/lib/fsevents-handler.js:211:16)
    at FSWatcher.<anonymous> (/some-project/node_modules/chokidar/lib/fsevents-handler.js:342:25)
    at LOOP (fs.js:1623:14)
    at doNTCallback0 (node.js:443:9)
    at process._tickDomainCallback (node.js:413:13)

It looks like this has fixed somewhere between<EMAIL_ADDRESS>and<EMAIL_ADDRESS>Seems to work fine with<EMAIL_ADDRESS>


Broken URLs with trailing / due to .nest()

This is a small reproducer of a problem we are facing in our code, where the use of .nest() leads to routes with a trailing / in the documentation but the URLs are only working without them.
Example
Apply the following patch/diff to the main.rs file in the axum example in the aide Github repository:
[nix-shell:~/code/other/aide]$ git diff
diff --git a/examples/example-axum/src/main.rs b/examples/example-axum/src/main.rs
index ac4f621..84ec2a7 100644
--- a/examples/example-axum/src/main.rs
+++ b/examples/example-axum/src/main.rs
@@ -33,11 +33,11 @@ async fn main() {
     let mut api = OpenApi::default();

     let app = ApiRouter::new()
-        .nest_api_service("/todo", todo_routes(state.clone()))
-        .nest_api_service("/docs", docs_routes(state.clone()))
+        .nest("/todo", todo_routes(state.clone()))
+        .nest("/docs", docs_routes(state.clone()))
         .finish_api_with(&mut api, api_docs)
-        .layer(Extension(Arc::new(api))) // Arc is very important here or you will face massive memory and performance issues
-        .with_state(state);
+        .layer(Extension(Arc::new(api))); // Arc is very important here or you will face massive memory and performance issues
+        //.with_state(state);

     println!("Example docs are accessible at http://<IP_ADDRESS>:3000/docs");

Resulting Documentation
After applying the diff above the app produces the following documentation:

Note that the --url parameters in the curl examples all end with a trailing /.
Broken POST example execution
The screenshot below illustrates that the POST method only works if we remove the trailing /, but if we copy the snippet from the documentation without modification we get a 404 NOT FOUND response.

Issue report
Above you can see that .nest() leads to misleading documentation and broken URLs.
We are currently using a workaround with .merge(), but it makes the code less readable, since we then need to include the base URL in the sub-router. We also figured that using .nest_api_service() like in the original example code enables URLs to work with and without the trailing /, but we don’t want that.
Is this known behavior or as we suspect a bug?

I too am seeing this issue. I'm not even sure it's an aide issue to be honest, as I believe it would happen with axum proper as well. Instantiating separate ApiRouter<()> with their own state (even if all the nested ones would ultimately have the same state) is a fine workaround, but it is more verbose than necessary in many cases.


feat: port/CategoryTheory.Bicategory.Coherence

[x] depends on: #2482

This PR/issue depends on:

leanprover-community/mathlib4#2482
By Dependent Issues (🤖). Happy coding!

This PR/issue depends on:

leanprover-community/mathlib4#2482
By Dependent Issues (🤖). Happy coding!

bors r+


Merge dev into master for v1.2.1 release

CLA overridden: all commits being merged are PRs that have already had the CLA verified.


Eliminate redundant ways of doing the same thing

Currently, knack allows you to register commands with either of these options:
# method 1
self.cli_command(...)

# method 2
with CommandSuperGroup(...) as sg:
   with CommandGroup(...) as g:
     g.command(...)

Argument registration likewise exposes two syntactically very different ways of doing the same thing. Neither method is clearly better--they are both just different.  However, this has resulted in confusion for external partners as to what the difference is and what they should use.  While it doesn't take hours to tell them "they are the same and use whichever you prefer", ideally there would only be one right way to do things, and by this point we should have a better idea what that is (my guess is it would be a third implementation that combines the strengths and mitigates the weaknesses of both existing approaches).
Since this is supposed to be a framework that exposes the simplest way to do the right thing for external partners, we should strive to commit to only a single way of registering commands and arguments, especially give that no implementation can be migrated to the new framework without modification.

Also, note that CLI 2.0 already subclasses CLICommandsLoader, so even if we wanted to preserve the "old ways" we can do that there. There's no need to enshrine that in Knack (on the contrary, there's great reasons NOT to!)


LHN - Messages sent to multiline account in focus mode, in pinned status is not shown in LHN

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to https://staging.new.expensify.com/
Tap profile icon
Tap preferences
Set priority mode to "focus"
Open any long user name multiline "pinned" account<EMAIL_ADDRESS>Send a message
Navigate to LHN
Open android app for same account
Repeat step 2- step 7

Expected Result:
Message sent must be displayed in LHN
Actual Result:
Messages sent to "pinned"multiline account in focus mode, is not shown in LHN in mweb but shown in android
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[x] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.57-2
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/78819774/d098fdee-6bba-4c6e-9e6f-0ec1b117ea69
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

I tested by messaging<EMAIL_ADDRESS>and the message preview does not show up for me on mweb (staging and prod) on android or on the android app. The behavior I'm seeing is the same across both platforms. Closing since I can't reproduce


kld doesn't fail when a module can't be unloaded

Some modules can't be unloaded once loaded.
$ kldload -n accf_http

$ kldunload accf_http
kldunload: can't unload file: Operation not supported

$ kldunload -f accf_http
kldunload: can't unload file: Operation not supported

This has been fixed.


Git Gui crash on Mac OS Sierra

On MacOS Sierra git gui crashes randomlywith this message:
2016-09-22 13:17:36.759 Wish[23615:1501726] *** Terminating app due to uncaught exception 'CALayerInvalidGeometry', reason: 'CALayer position contains NaN: [0 nan]'
*** First throw call stack:
(
        0   CoreFoundation                      0x00007fff7f0407bb __exceptionPreprocess + 171
        1   libobjc.A.dylib                     0x00007fff937ada2a objc_exception_throw + 48
        2   CoreFoundation                      0x00007fff7f0bda65 +[NSException raise:format:] + 197
        3   QuartzCore                          0x00007fff84c09980 _ZN2CA5Layer12set_positionERKNS_4Vec2IdEEb + 152
        4   QuartzCore                          0x00007fff84c09af5 -[CALayer setPosition:] + 44
        5   QuartzCore                          0x00007fff84c0a14b -[CALayer setFrame:] + 644
        6   CoreUI                              0x00007fff8a9b0112 _ZN20CUICoreThemeRenderer26MakeOrUpdateScrollBarLayerEPK13CUIDescriptoraPP7CALayer + 1284
        7   CoreUI                              0x00007fff8a9ac317 _ZN20CUICoreThemeRenderer19CreateOrUpdateLayerEPK13CUIDescriptorPP7CALayer + 1755
        8   CoreUI                              0x00007fff8a92e4d1 _ZN11CUIRenderer19CreateOrUpdateLayerEPK14__CFDictionaryPP7CALayer + 175
        9   CoreUI                              0x00007fff8a931185 CUICreateOrUpdateLayer + 221
        10  AppKit                              0x00007fff7d675623 -[NSCompositeAppearance _callCoreUIWithBlock:options:] + 226
        11  AppKit                              0x00007fff7cd22a9d -[NSAppearance _createOrUpdateLayer:options:] + 76
        12  AppKit                              0x00007fff7cf9b143 -[NSScrollerImp _animateToRolloverState] + 274
        13  AppKit                              0x00007fff7cf5ab79 __49-[NSScrollerImp _installDelayedRolloverAnimation]_block_invoke + 673
        14  AppKit                              0x00007fff7ce21331 -[NSScrollerImp _doWork:] + 15
        15  Foundation                          0x00007fff80a3ec88 __NSFireDelayedPerform + 417
        16  CoreFoundation                      0x00007fff7efc0f44 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 20
        17  CoreFoundation                      0x00007fff7efc0bd3 __CFRunLoopDoTimer + 1075
        18  CoreFoundation                      0x00007fff7efc072a __CFRunLoopDoTimers + 298
        19  CoreFoundation                      0x00007fff7efb82f1 __CFRunLoopRun + 2081
        20  CoreFoundation                      0x00007fff7efb7874 CFRunLoopRunSpecific + 420
        21  HIToolbox                           0x00007fff7e557f6c RunCurrentEventLoopInMode + 240
        22  HIToolbox                           0x00007fff7e557ca9 ReceiveNextEventCommon + 184
        23  HIToolbox                           0x00007fff7e557bd6 _BlockUntilNextEventMatchingListInModeWithFilter + 71
        24  AppKit                              0x00007fff7cc4e5f5 _DPSNextEvent + 1093
        25  AppKit                              0x00007fff7d35e8eb -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1637
        26  Tk                                  0x00000001047cc285 TkGenerateButtonEvent + 494
        27  Tk                                  0x00000001047cc54d Tk_MacOSXSetupTkNotifier + 395
        28  Tcl                                 0x00000001048be5a8 Tcl_DoOneEvent + 237
        29  Tk                                  0x0000000104726f4f Tk_MainLoop + 33
        30  Tk                                  0x0000000104732a5b Tk_MainEx + 1566
        31  Wish                                0x000000010470d55a Wish + 9562
        32  libdyld.dylib                       0x00007fff94089255 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
error: git-gui died of signal 6

This issue tracker is dedicated to the git-scm.com website.
The issue you have raised is related to the git program. If you think there is a bug in the git program, please, send your issue to the git mailing list<EMAIL_ADDRESS>

I'm sorry, thanks for info.

@premekkoch Did you report this to the mailing list? I have this problem too after updating to Sierra, but I can't find anything in their archives.

@bxt I think I fixed it with this:
git config --local --unset gui.geometry

Unfortunately it is not 😔

Not a git issue. Try:
open /System/Library/Frameworks/Tk.framework/Versions/8.5/Resources/Wish.app
Then, attempt to resize the console. This is internal to Apple.

@kjmph I'd assume on https://bugreport.apple.com

If you have Tk v8.4 lying around, it's worth trying to replace the Wish executable within Git with an older Wish version:
cp /System/Library/Frameworks/Tk.framework/Versions/8.4/Resources/Wish.app/Contents/MacOS/Wish /usr/local/Cellar/git/2.10.1/share/git-gui/lib/Git Gui.app/Contents/MacOS/

This seems to have worked around the crashes on my end.
gitk seems to use the wish executable defined in $PATH so you might need to bend that to the 8.4 version.

@sieren, did the look of the git gui changed after replacing the Wish executable to 8.4?

It changed, yeah.

Maybe you applied http://retinizer.mikelpr.com once and that it got overridden is back to normal/"ugly".

Nope, never did that.
It really does look like it got "unretinized" though.

Instead of using the default MacOS tcl-tk, use the latest tcl-tk from homebrew and then link it by force.
brew install tcl-tk
brew link tcl-tk --force

I"m not using this client but we've encountered this call stack on Sierra.  In an NSScrollView the scroll bar is normally hidden until you mouse over it and even then only is shown if there is the potential for scrolling.  We had a NSScroller that was always visible (modernized code) and if no scroll was possible moving the mouse over it resulted in a NaN value being sent to the NSScroller.  The fix was to subclass NSScroller - then the NaN is not sent.

Ran brew install tcl-tk && brew link tcl-tk --force and still got: 2016-10-19 11:12:51.367 Wish[75532:5137867] *** Terminating app due to uncaught exception 'CALayerInvalidGeometry', reason: 'CALayer position contains NaN: [nan 0]'.

As a workaround to me this seem to work (until I update git and then have to rerun it):
## Install newer Tcl/Tk version
brew install tcl-tk
brew link tcl-tk --force

## Your milage may vary
#git_version=$(brew info git --json=v1 | jq -r '.[].linked_keg')
git_version=2.10.1

## Make `git gui` use newer Tcl/Tk version
ln -s --backup=numbered $(which wish) "/usr/local/Cellar/git/$git_version/share/git-gui/lib/Git Gui.app/Contents/MacOS/Wish"

Thank you! I had to do the following, and now gitk works for me:
brew uninstall --force git
brew install homebrew/dupes/tcl-tk
brew link tcl-tk --force
brew install git

Note, I have never had success with Apple bug reports, maybe someone with more clout can open the ticket?

@kjmph interesting, are you saying that brew install git picks up the brew install tcl-tk's path? Iff, that would be even easier.

@djui It seems like it does for gitk but not for git gui, because this one comes with its own bundled copy of Wish.

@bxt is correct, gitk works, git gui still fails. I see why both fixes are required now.

@bxt @kjmph OK. Not optimal, but finally having a temporary fix is great.

so this only fixes gitk..not git gui? Or am I missing something for a workaround that fixes git gui?

https://github.com/git/git-scm.com/issues/853#issuecomment-254768237 "solves" git gui. https://github.com/git/git-scm.com/issues/853#issuecomment-254821097 "solves" gitk.

ahhh, I missed an error running with --backup=numbered . --backup isn't valid for the default mac ln util and the path already existed. Renamed the existing Wish bundled with git-gui and I'm all good now!

After updating to git 2.10.2 I had to repeat the above instructions as well as making the following change.
edit /usr/local/Cellar/git/2.10.2/libexec/git-core/git-gui and set the libdir (line 7) to
    libdir="/usr/local/Cellar/git/2.10.2/share/git-gui/lib"

@garth That the @@HOMEBREW_PREFIX@@ template variable is not replace looks like (i) unrelated to this issue and (ii) like a regression. Wondering what happened? I can't directly see the issue in the Formular, if it wasn't in the bottle.

@djui I wasn't sure if it was related to upgrading an installation that had to be hacked a little to get it working on Sierra. Hopefully these early issues with Sierra will be resolved soon and we can revert to a clean install. Until then we need somewhere to openly share these temporary workarounds.

This has been so frustrating.  I've tried following some of the fixes above but none worked for me.  Is there a consensus for a set of steps that should get me fixed so I can use git gui?  Thanks!

Have you tried @djui's workaround above? Seems to work for me. Maybe double check that you actually run the git gui for which you override the Wish bin (check version numbers etc). Also make sure you have a working Wish somewhere. Replace $(which wish) in the command with the path to the Wish you want to use. You could also get one from e.g. ActiveTcl if you don't use brew.

brew link --force tcl-tk actually fails with the_platinum_searcher installed as bin/pt collides.
This one would fundamentally be on Apple to fix. What do they actually ship their Tcl/Tk for? Is that community not hitting this one?

@Rotonen Not a real help for Tcl problems, but if you want to try out something new, Rust instead of Go, have a look at ripgrep (rg).

Hello everybody,
This exception on Mac OSX Sierra
... 'CALayerInvalidGeometry', reason: 'CALayer position contains NaN: [nan 0]'
can now be solved by upgrading to the lastest Mac OS version available on Applestore : Mac OS 10.12.2.
Best regards Thierry.

... upgrading to the lastest Mac OS version available on Applestore : Mac OS 10.12.2.

Confirmed that this works for me as well. Thanks @Thierry46 for posting your experience here and letting us know to look at 10.12.2.

@Thierry46 I'm running 10.12.2 uninstalled/installed both the .dmg and the homebrew versions and still get this issue. There must be more to this.... :(

@iDVB
http://stackoverflow.com/questions/39833961/git-gui-crashes-on-mac-os-x-sierra/40974770#40974770
This is what I'm currently doing to remedy this in case of a broken TCL/TK installation.
But as stated, this was already fixed Apple-side.

I ended up fixing the issue but I have no idea why it helped. I stumbled on the fix when I noticed that git gui only seemed to crash on one of my repos. I deleted the directory and re-cloned it. FIXED.
Only thing I can think of is that some marbles got lost when I switched the origin remote part way thru and the parent directory name was different.

@bxt I think I fixed it with this:
git config --local --unset gui.geometry

Edit: Nope, never mind

This works for me


Sync repo permissions with new org structure

I have removed HCAPM and SecOps and left the UCSC team and individual collaborators that I know are working on the portal. We may need to create a new group and add some folks back once we start creating new content again but for now I think we have a new minimum of members.


plugin-bundler-webpack: Document more clearly posibility of providing a webpack.config.js

Will be part of docs


Improve documentation

The documentation currently online seems to largely assume familiarity with the PNG spec. Since I have minimal knowledge of the spec, this has given me trouble. I need a fairly low-level interface, since I am trying to port to Rust a program sensitive to whether a PNG is paletted or not.
This PR is intended to improve documentation, including to be more noob-friendly (as much as possible, anyway); I have made a few modifications based off my experience, and noted other possible changes below. If you have ideas, please tell me, and I will add them!

[ ] A code comment mentions that some ColorType/BitDepth combinations are impossible. Maybe this should be pointed out in the docs?
[ ] The C code block in Transformations's doc does not have type for its constants, and it's not clear (to the reader) whether this code comes from this crate, or libpng. Further, what are they intended for?
[ ] Maybe some more examples? This is how I'm using the lib, fwiw. How to use the various fields of Info, for example, I had to determine through trial and error.

The C code block in Transformations's doc does not have type for its constants, and it's not clear (to the reader) whether this code comes from this crate, or libpng. Further, what are they intended for?

Those are the transformations from libpng which aren't yet supported by this crate, but which may be in the future. The syntax is actually for the bitflags macro (from the crate of the same name).

In general I do not think the library should go too far into the png specification. The documentation should be written with the right terminology and in a style that ensures one can quickly find the relevant portions and paragraphs but replicating it in different wording is, in my opinion, not a good approach. For example, the allowed color combinations are due to the specification and while maybe being slightly more obvious to duplicate the table from the specification, it also gives the appearance that the library does something that diverges from the specification or does not support all combinations. That can be highly confusing as well.

Maybe some more examples? This is how I'm using the lib, fwiw. How to use the various fields of Info, for example, I had to determine through trial and error.

Sure, that seems good. Most of the methods lack dedicated examples so short snippets would be appreciated.

Thanks! Feel free to open more PRs with additional improvements, as pointed out in your open questions.

Oh, I thought I'd add more documentation to this PR before merging it. :P
To avoid repeating the spec, and encourage users to read it, a solution may be to pepper the documentation with links to relevant spec paragraphs? Could also be useful as citation-like reference, again to provide guidance to the user. (For example, I thought paletted images couldn't have alpha, until someone pointed out the tRNS chunk to me.)

Yes, link and section numbers would also be helpful.


Export docs into respective files

Is it possible to configure the generator to export the docs into separate files based on the contract names?
Currently it just exports everything into a single file called index.md when running npx hardhat docgen

Yes, you can can try pages: 'items' or pages: 'files' in the config. Result will differ depending on whether you have multiple items in each file.
So in hardhat.config.js/ts you should add:
docgen: {
  pages: 'items',
},

pages: 'files' is exactly what I was looking for and worked like a charm. thanks for the help and quick response ser
on a side note, is there a place I can see all the options for hardhat config? couldn't seem to find it anywhere..

Yeah: here. Link is at the bottom of the readme as well.

ahhh didn't realize that was for hardhat..thank you!
sorry but ran into a minor issue..seems that docgen is generating library contracts (ie OpenZeppelin) with the .sol extension instead of .md. is this normal behavior?

No, I think there was a bug. Just published a fix.


move off of spark-testing-base

also upgrade parent plugin

Coverage increased (+11.0%) to 49.587% when pulling 64febee0fb35223eb3483a89a34051cd75bd8fcf on ryan-williams:ug into 45d6cddf673e479521bac2c58c2f848d10bc5064 on hammerlab:master.


Add Terms of Service consent page

Related Issue or Background Info
Resolves #764

[x] Make this the first page a person sees on the app before going to the form
[ ] Add E2E tests @nullflux @ClydeDroid

Screenshots / Demos

thanks, y'all! 🎉 🎉 🎉


simplify import_image

Signed-off-by: Robert Cerven<EMAIL_ADDRESS>

rn updated:
import_image plugin uses now import_image_tags, which imports image and tags at once, instead of importing tags one by one, and import image after


[404 error] Running GGD recipe: hg38 varpon 20181105

Hi,
I have installed bcbio through the following line, but it get terminated when it comes to the genome data. In the beginning, the error was the connection to Broad institute's server for the downloading of genome file (overloading of users), but this was resolved after a few tries. Thank you for your previous suggestion to retry! The current problem is that the hg38 varpon returns 404 error. I went to the link for the file and it is apparently unavailable https://nc.hartwigmedicalfoundation.nl/index.php/s/a8lgLsUrZI5gndd/download?path=%2FHMF-Pipeline-Resources&files=GermlineHetPon.hg38.bed.gz. Is there an alternative link to retrieve this file?
The error I received is as below.

[wanching@nscc04 wc]$ bcbio_nextgen.py upgrade --data
Upgrading bcbio
Data not installed, no genomes provided with '--genomes' flag
Upgrade completed successfully.
[wanching@nscc04 wc]$ bcbio_nextgen.py upgrade --data --genomes hg38
Upgrading bcbio
Upgrading bcbio-nextgen data files
List of genomes to get (from the config file at '{'genomes': [{'dbkey': 'hg38', 'name': 'Human (hg38) full', 'indexes': ['seq', 'twobit', 'bwa', 'hisat2'], 'annotations': ['ccds', 'capture_regions', 'coverage', 'prioritize', 'dbsnp', 'hapmap_snps', '1000g_omni_snps', 'ACMG56_genes', '1000g_snps', 'mills_indels', '1000g_indels', 'clinvar', 'qsignature', 'genesplicer', 'effects_transcripts', 'varpon', 'vcfanno', 'viral', 'transcripts', 'RADAR', 'rmsk', 'salmon-decoys', 'mirbase'], 'validation': ['giab-NA12878', 'giab-NA24385', 'giab-NA24631', 'platinum-genome-NA12878', 'giab-NA12878-remap', 'giab-NA12878-crossmap', 'dream-syn4-crossmap', 'dream-syn3-crossmap', 'giab-NA12878-NA24385-somatic', 'giab-NA24143', 'giab-NA24149']}], 'genome_indexes': ['rtg'], 'install_liftover': False, 'install_uniref': False}'): Human (hg38) full
Running GGD recipe: hg38 varpon 20181105
--2019-08-28 11:34:07--  https://nc.hartwigmedicalfoundation.nl/index.php/s/a8lgLsUrZI5gndd/download?path=%2FHMF-Pipeline-Resources&files=GermlineHetPon.hg38.bed.gz
Resolving nc.hartwigmedicalfoundation.nl (nc.hartwigmedicalfoundation.nl)... > gsort version 0.0.6
<IP_ADDRESS>
Connecting to nc.hartwigmedicalfoundation.nl (nc.hartwigmedicalfoundation.nl)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-08-28 11:34:08 ERROR 404: Not Found.
gzip: stdin: unexpected end of file
2019/08/28 11:34:08 EOF
Traceback (most recent call last):
File "/home/users/industry/perdana/wanching/scratch/wc/bcbio/bcbio_tooldir/bin/bcbio_nextgen.py", line 221, in
install.upgrade_bcbio(kwargs["args"])
File "/home/users/industry/perdana/wanching/scratch/wc/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 106, in upgrade_bcbio
upgrade_bcbio_data(args, REMOTES)
File "/home/users/industry/perdana/wanching/scratch/wc/bcbio/anaconda/lib/python3.6/site-packages/bcbio/install.py", line 348, in upgrade_bcbio_data
args.cores, ["ggd", "s3", "raw"])
File "/scratch/users/industry/perdana/wanching/wc/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
_prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
File "/scratch/users/industry/perdana/wanching/wc/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
retrieve_fn(env, manager, gid, idx)
File "/scratch/users/industry/perdana/wanching/wc/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 856, in _install_with_ggd
ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
File "/scratch/users/industry/perdana/wanching/wc/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
recipe["recipe"]["full"]["recipe_type"], system_install)
File "/scratch/users/industry/perdana/wanching/wc/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
subprocess.check_output(["bash", run_file])
File "/home/users/industry/perdana/wanching/scratch/wc/bcbio/anaconda/lib/python3.6/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/home/users/industry/perdana/wanching/scratch/wc/bcbio/anaconda/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/scratch/users/industry/perdana/wanching/wc/bcbio/genomes/Hsapiens/hg38/txtmp/ggd-run.sh']' returned non-zero exit status 1.

Rory, you're right, they've moved them to https://nc.hartwigmedicalfoundation.nl/index.php/s/a8lgLsUrZI5gndd?path=%2FHMFTools-Resources for now, but not sure how long that will stay that way.
The GermlineHetPon.hg38.bed.gz file is under Amber fyi.
Good to know that you're hosting in separate bucket.

Slack ;-)

Thanks a lot Rory and Peter! I removed the tmpbcbio-install directory and did bcbio_nextgen.py upgrade --data --genomes hg38. It works fine now.

Great! Thanks for following up.

Hallo everyone.
The same problem is repeated now..
Exactly, when trying to install the h38 genome

https://ftp.ncbi.nih.gov/snp/archive/b154/VCF/GCF_000001405.38.gz
Auflösen des Hostnamens ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)… <IP_ADDRESS>, <IP_ADDRESS>, 2607:f220:41e:250::7, ...
Verbindungsaufbau zu ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|<IP_ADDRESS>|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 404 Not Found
2023-02-09 16:41:02 FEHLER 404: Not Found.

The genome files are no longer there

Hello,
As @alasfar-lina has already mentioned the file in https://ftp.ncbi.nih.gov/snp/archive/b154/VCF/GCF_000001405.38.gz has been removed. Error when trying to install hg38 genome while running cloudbiolinux/ggd-recipes/hg38/dbsnp.yaml


change function for map does not appear to work correctly

Describe the bug
This may not be a bug, but I can't find documentation for the change function / method as it applies to maps. It certainly does not appear to operate in a way that I would expect.
Calling change on an existing, nonempty map appears to operate as if it were called on an empty map.
To Reproduce
This results in a Nothing_ error, when I would expect it to return the map unchanged:
from expression.collections import Map

xs = Map.of(**dict(a=10, b=20))
xs.change("a", lambda x: x)

This results in map [("a", 1)] when I would expect map [("a", 1); ("b", 20)]:
xs.change("a", lambda _: Some(1))

Expected behavior
As indicated above, I would expect that the first block would return the map unchanged, and that the second would only modify the item with the key "a".
Additional context
Add any other context about the problem here.

OS MacOS 13
Expression version 4.2.2
Python version 3.11.1

I'm facing the same issue; a bit more specifically, the map can only change the last item added:
def test_expression_issue():
    m = Map.empty()
    m = m.add("a", 1)
    m = m.add("d", 42)
    print(m) # map [("a", 1); ("d", 42)]
    print(m.get("a")) # 1
    print(m.get("d")) # 42, so both keys are in the map
    m = m.change("d", lambda x: x) #<- works cause d was added last
    print(m) # map [("a", 1); ("d", 42)] Both keys still there
    m = m.change("a", lambda x: x) # fails; if this was the first call to change, it would also fail
    assert m

I noticed that the code in maptree.py lines 315 and 328 uses rebalance but does not return it which seems wrong since that would mean it has side effects (right?)
Adding return rebalance(... makes the tests I wrote pass, and all other tests still pass too. I'll create a PR but hopefully the authors/maintainers can have a close look because I know very little about MapTrees and generally functional programming.


owner(cdc): asynchronously close the ddl sink

What problem does this PR solve?

Issue Number: close #4241
What is changed and how it works?
Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation
Need to update key monitor metrics in both TiCDC document and official document

Release note
Please add a release note.
If you don't think this PR needs a release note then fill it with `None`.

Codecov Report

Merging #4249 (4c54774) into master (08da001) will increase coverage by 0.0582%.
The diff coverage is 52.3390%.

Flag
Coverage Δ

cdc
58.7614% <57.9674%> (+0.1166%)
:arrow_up:

dm
52.2009% <40.5451%> (-0.0537%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #4249        +/-   ##
================================================
+ Coverage   55.1722%   55.2305%   +0.0582%
================================================
  Files           485        491         +6
  Lines         59829      60950      +1121
================================================
+ Hits          33009      33663       +654
- Misses        23484      23876       +392
- Partials       3336       3411        +75

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-all-tests

/run-integration-tests /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558

/run-integration-tests /tidb=pr/30558

/run-integration-tests /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558

/run-kafka-integration-test /tidb=pr/30558

/run-kafka-integration-test /tidb=pr/30558

/run-integration-test /tidb=pr/30558

/run-dm-integration-test /tidb=pr/30558
/run-dm-compatibility-test /tidb=pr/30558

/run-integration-tests /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558

It looks like we don't care if the close is successful, so can we just add a close timeout? That way the impact on the owner and processor is minimal?

It looks like we don't care if the close is successful, so can we just add a close timeout? That way the impact on the owner and processor is minimal?

Such a good idea, I will do this in another new PR. https://github.com/pingcap/tiflow/pull/5186

closed by #5186


Pass list of initially toggled options for multi-select menus

Hey!
I'm looking for something like this:
options = ["apple", "banana", "orange"]
menu = TerminalMenu(options, multi_select=True, selected_options=set([0]))
menu.show()

... which would show:
> [*] apple
  [ ] banana
  [ ] orange

Is it possible to provide a multi-select menu with a list of options that are already set to True?
Thanks for maintaining this amazing project!
Any feedback is appreciated!

Hey @kelvindecosta,
definitely a useful feature request and should not be so much work to implement. I will look into this when I have some free time!

Hey @IngoMeyer441
Thanks for the quick response!
I was playing around and found another interesting approach for simulating multi-select.
fruits = ["apple", "banana", "orange"]
chosen = set()

while True:
    menu = TerminalMenu([f"""[{'*' if f in chosen else ' '}] {f}""" for f in fruits], clear_screen=True, clear_menu_on_exit=False)

    if (choice := menu.show()) is not None:
        fruit = fruits[choice]

        if fruit in chosen:
            chosen.remove(fruit)
        else
            chosen.add(first)

        # do something with fruit
        pass
    else:
         break

This fits my use case but there is one major annoyance.
Whenever menu.show() is called, it re-renders the menu like it should.
However, the cursor resets.
Is there a way I can set the initial cursor position?
Thanks for your time.
Any feedback is appreciated!

You can pass the initial cursor index with the cursor_index constructor parameter.

I implemented a pre-select feature but I'm having trouble committing it because the pre-commit hook keeps crashing with:
    Traceback (most recent call last):
      File "/home/amarold/.cache/pre-commit-zipapp/-eYCHjv49dDHr21Lw7EXR3hR9h4GSmNzRdQVGEnG1aw/python", line 48, in <module>
        exit(main())
      File "/home/amarold/.cache/pre-commit-zipapp/-eYCHjv49dDHr21Lw7EXR3hR9h4GSmNzRdQVGEnG1aw/python", line 32, in main
        runpy.run_module(args.m, run_name='__main__', alter_sys=True)
      File "/usr/lib/python3.8/runpy.py", line 207, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
        _run_code(code, mod_globals, init_globals,
      File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/home/amarold/.cache/pre-commit-zipapp/-eYCHjv49dDHr21Lw7EXR3hR9h4GSmNzRdQVGEnG1aw/wheels/nodeenv-1.6.0-py2.py3-none-any.whl/nodeenv.py", line 48, in <module>
    ModuleNotFoundError: No module named 'pkg_resources'
```

@UnstoppableDrew Great that you want to contribute! :+1:
Hmm, I was not aware that the pre-commit stand alone app has more dependencies than Python. AFAIK, the error message
ModuleNotFoundError: No module named 'pkg_resources'

indicates a missing setuptools installation. Which system do you use? On Ubuntu, for example, you could fix this with
sudo apt install python3-setuptools

The repository also contains a GitHub action, so you can also push to GitHub to let the checks run.

Thanks to @UnstoppableDrew this issue is fixed by PR #30.


Lift keeps moving after you release joystick

When you release the lift joystick, the commands stops changing but the lift keeps trying to catch up with it.  Additionally, when the lift is on the way to a preset and the driver moves the joystick, it should abort the preset move and control from the current position.
Solution:

Keep track of the previous joystick value.
Make a state flag for liftGoingToPreset, and set it true in liftToPreset and liftToValue.
When the joystick changes from positive to (zero or negative), OR from negative to (zero or positive), call stopExtension.
When the joystick is not zero, if liftGoingToPreset is true, make it false and call stopExtension.
After those checks are complete, regardless of the outcome, if joystick is not 0, call controlExtension.

Not sure if this needs the wait-for-coasting feature of the fetcher.  Probably not, since they only use the joystick occasionally.


types for data field can be undefined in suspense mode.

hi. I'm using react query for my data fetcher in react experimental and typescript. but the return type of useQuery hook shows that data could be undefined. but actually not!. if query is fetching or has some problem in concurrent mode, it should throws an error, and boundaries will catch them.
const res = useQuery<Books>(key,fetcher)
//res.data is Books | undefined

@alisajadih Not a maintainer but this has been captured in some other issues - the data will be undefined if you disabled the query for example:
const {data} = useQuery<Books>(key, fetcher, {enabled: false})

This is something specific to react-query, not suspense itself, because you can disable queries (for example if you only want to fetch after a certain condition is met).

additionally, it's very hard to do that on type level just by a flag. That flag can also be set globally in the QueryClientProvider. It's the same as with initialData.
The best we could do is provide a separate useSuspenseQuery, but then we'd also need an infiniteQuery version of it, and a useQueries version of it, and it kinda explodes pretty fast.
It's also just one more character to type (res.data?.map vs res.data.map) to make TypeScript happy :)

For the ones who want a quick fix for this one, you can create a useSuspenseQuery.ts util file in any place, and paste this code:
import {
  QueryFunction,
  QueryKey,
  QueryObserverSuccessResult,
  useQuery,
  UseQueryOptions,
} from 'react-query';

type SuspenseQueryObserverResult<TData = unknown, TError = unknown> = QueryObserverSuccessResult<
  TData,
  TError
>;
type UseSuspenseBaseQueryResult<TData, TError> = SuspenseQueryObserverResult<TData, TError>;

type UseSuspenseQueryResult<TData = unknown, TError = unknown> = UseSuspenseBaseQueryResult<
  TData,
  TError
>;

export function useSuspenseQuery<
  TQueryFnData = unknown,
  TError = unknown,
  TData = TQueryFnData,
  TQueryKey extends QueryKey = QueryKey,
>(
  queryKey: TQueryKey,
  queryFn: QueryFunction<TQueryFnData, TQueryKey>,
  options?: Omit<UseQueryOptions<TQueryFnData, TError, TData, TQueryKey>, 'queryKey' | 'queryFn'>,
): UseSuspenseQueryResult<TData, TError> {
  return useQuery(queryKey, queryFn, options) as unknown as UseSuspenseQueryResult<TData, TError>;
}

It will work the same but with right types.
You can extend to accept other signatures too.

It's also just one more character to type (res.data?.map vs res.data.map) to make TypeScript happy :)

This isn't quite true. There would be a cascading effect of undefined values to guard against.
If it's a list that we are just going to map over we could fallback to an empty list result.data?.map() ?? []. That's not so bad, but if our response data is some kind of object the pain just compounds. My current solution to this problem is to add a top level guard and to throw an exception if data is undefined.

@blazzy this might work for your use-case, but there are many situations where data can still be undefined even if you use  suspense, so we cannot build anything into the library. One example would just be passing enabled: false to useQuery.

Understood. I didn't meant to suggest react-query had to solve this. Just sharing my interim solution as I was too lazy wrap react-query at the moment.


Does this supports RGB buffer for input?

I'm newbie for zig so it may my misunderstanding, but it seems that this encoder implementation interprets the input buffer as RGBA packed pixel images regardless Header.format . If it's true, this implementation can't accept 3-channel images.
Does this supports RGB buffer for input?

This implementation operates not on a byte array, but always on a Color array which is an RGBA value. But if you serialize an image with all alpha bytes as fully opaque, it is serialized as RGB.
Note that this doen't matter at all, as QOI only can encode RGBA images anyways, the file header value is purely informational

This implementation operates not on a byte array, but always on a Color array which is an RGBA value.

OK, so we need to convert a 3-channel packed pixel byte array to a Color array before (or ) encoding, right?

as QOI only can encode RGBA images anyways, the file header value is purely informational

Your point also make sense; the reference implementation always handles 4-channel pixels, but just doesn't touch the alpha when channels == 3.
However, at least the reference can encode the byte array which contains a 3-channel packed pixel image directly according to the channels.
IMO that is an overstatement from this point of view.


Underscores instead of Y/N

So I had an idea: since a lot of fic also uses ____ to denote Y/N, could we implement this as well as Y/N y/n Y/n y/N etc?

I thought about that. But there are some obstacles:
Everyone uses a different number of underscores.
Even if we set it to, like, three, or any repeated occurrence of underscores there's no guarantee the user won't encounter a similar set somewhere else where it isn't a story which would make the extension very annoying.
So the answer is: we can but we definitely shouldn't.

Okay


fp-afterSlideLoad is missing

Hello me again ;)
I found that you missed to call the fp-afterSlideLoad Event
So i added on line to your function afterSlideLoads(v)
$rootScope.$emit('fp-afterSlideLoad', {anchorLink: v.slideAnchor, index: v.sectionIndex, slideAnchor: v.slideAnchor, slideIndex: v.slideIndex})

I will check. Thanks!!

Solved in new release, 1.3.7.
Thanks!


Failed To Pull Backend

Hi, so I've been pushing many changes to my Github today that only affect my front end, and about an hour ago I got home, fixed some router issues and pushed my project. Now when the build process on Amplify runs I get the following error:
2019-11-27T06:55:10.237Z [INFO]: Failed to pull the backend. 2019-11-27T06:55:10.239Z [INFO]: TypeError [ERR_INVALID_ARG_TYPE]: The 'request' argument must be string. Received type undefined                                  at Object.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/lib/lib/attach-backend-steps/a10-queryProvider.js:19:32)                                  at Generator.next (<anonymous>)                                  at fulfilled (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/lib/lib/attach-backend-steps/a10-queryProvider.js:5:58)                                  at process._tickCallback (internal/process/next_tick.js:68:7)                                  at Function.Module.runMain (internal/modules/cjs/loader.js:832:11)                                  at startup (internal/bootstrap/node.js:283:19)
I'm not quite sure whats causing the issue. My first idea was to undo what I did and all I did was change  to  and then back to  and it didn't solve anything. Then I updated Amplify CLI from something like 3.17 to 4.0.0 and pushed my changes everything. And still the same error. At one point I actually passed the build went on to deployment and it just printed (And sorry I don't have the original log)
-Starting Build
-Build Failed
-Exit
No errors, nothing. Just 3 lines.
I've added some screenshots below, if anyone has the time it would be very much appreciated if I could receive some help!

Same here :boom: ! This is insane, I am with a customer in the end of a sprint in order to go live and we cannot work...
Please provide a way to rollback to previous release, or to provide some documentation to fix amplify/cli.

OK so @Rafcin
Go to AWS Amplify Console, then, All Apps > [Your App name] > Build image settings > Edit button > change from latest to 3.17.0:

I was on CLI 3.17 when I had the issue and then upgraded to 4.0.0 and had the same issue. Did this work for you?

Oh actually I see what you mean. Sorry its 4am and i'm still awake

This is horrible. I just got off a call where I had to assure someone that Amplify was the right tool for this project because of other issues we're encountering. I get back to my desk and think I'll show him how much I get done....
I personally think the Amplify team is moving too fast with new features and the console team is not in sync.

Agreed. We can always try to riot 🤣

Ah but as funny as this is for a few moments, I really hope a fix comes. I don't exactly want to deal with going down a version and messing anything up.

I'm sure it will get fixed soon. I assume there's a push to get a lot done for reinvent next week.

@aireater @Rafcin @MBach, This is know issue we have rolled out a fix in 4.2.0. Please update and try again

I can confirm this works on the 4.2 version. Uninstall Amplify as it seems
to take issue with running the update command and then rebuild.
On Thu, Nov 28, 2019, 1:06 AM MBach<EMAIL_ADDRESS>wrote:

Hi @ammarkarachi https://github.com/ammarkarachi
Thank you for the update. I'm sorry but I won't take the risk right now, I
have other fish to fry 🐛
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws-amplify/amplify-cli/issues/2855?email_source=notifications&email_token=ACQCG6ZMZ3V67UOBJPJRMTDQV6C2JA5CNFSM4JSCL4E2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFL5JLQ#issuecomment-559404206,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACQCG66LKLPTGNE2U4L6INLQV6C2JANCNFSM4JSCL4EQ
.

@Rafcin Thanks!, I will be closing this for now, don't hesitate to reopen or submit another issue if you run into other problems


Got compilation error: no property v2_0_4 for type SDLSupport with 0.10.7

Hello,
Tried to switch to a newer version from 0.10.4 and got compilation error.
dlangui-0.10.7/dlangui/src/dlangui/platforms/sdl/sdlapp.d(1613,18): Error: no property v2_0_4 for type SDLSupport
Any idea how to work it around?
running on Ubuntu 20.04
dmd  v2.102.2

it was dependency issue
got it fixed after built with  dub upgrade --force-remove


Rename VectorProtocol.* to VectorProtocol.scaled(by:).

Friend PR: https://github.com/apple/swift/pull/25381

Note: this is an additive change and can be merged before https://github.com/apple/swift/pull/25381.