Better cleanup for combineEvents

Proper clearNode for target returned from combineEvents

Please, solve conflicts.
@doasync please, explain me why you need withRegion here?
What about to add tests for case with region?

@sergeysova we need withRegion here to properly remove all nodes created for the target. For example, each event in combineEvents is linked to the results and the shape inside is updated with newShape on each event. Even if we do clearNode for(target) now, there still will be updates of the results, but target will not be triggered any more,  so they are redundant. withRegion helps to remove the whole branch of the graph related to the target.
Technically it's just a performance/memory optimization and we cannot test these implementation details 🤷♂
P.S. No conficts

@doasync Thank you!


Fix precommit black foxy

Backport of #321

It happens all the time, that I start making foxy PRs on the main branch... sorry for that.

Codecov Report

:exclamation: No coverage uploaded for pull request base (foxy@2d9b011). Click here to learn what that means.
The diff coverage is n/a.

@@          Coverage Diff           @@
##             foxy    #338   +/-   ##
======================================
  Coverage        ?   3.57%
======================================
  Files           ?      13
  Lines           ?     951
  Branches        ?     152
======================================
  Hits            ?      34
  Misses          ?     847
  Partials        ?      70

Flag
Coverage Δ

unittests
3.57% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d9b011...fc96c82. Read the comment docs.


Don't badge the selected, focused conversation as unread on new message

@keybase/react-hackers r?

@chrisnojima Thanks, just switched to reduce, seems to work.  r?


Query about the new update about inference on CPU and GPU.

Description
I am actually interested in the speed up that I can get on CPU and GPU especially for inference.
According to the answer by @yanghaojin
I have tried BMXNet v1 for the same and I get speed up on CPU of about 1.4x - 1.7x on my PC for some models but also a decrease in speed up in some case.
I used : Ubuntu 16.04/64-bit platform on Intel(R) Core™ i5-8250U CPU@ 1.60GHz (supports SSE4.2)
Can you please elaborate about the update of 21st May 2019 wrt speed up ?
The update which is written in the changelog

@kaivu1999 that's a really good discussion you brought up and has been of interest since the release of BMXNetv1.
pointing out the findings of @yanghaojin that for training you skipped the GEMM computation and use the CuDNN implementation for training on GPUs (as per your justification), theoretically for inference (say real-time constraints on robotic platforms) XNOR computations could be speeded up on GPUs as well - did you find the reasons for this ? (eg. well optimized CuDNN code by teams of nVidia?, etc.)
now only considering CPU inference which is the main focus of the whole research (as the ultimate goal is to run DL models preferably in realtime on CPU-only devices):
the results of @kaivu1999 dont coincide with the results of the BMXNet papers and preceeding XNORNet paper (theoretically up to 32x)
clearly there are other steps involved like input binarization or patch2col that add up to the total processing time as part of the convolutional operation
@yanghaojin, @jopyth would be interested as well if you could elaborate a bit on that?

@simonmaurer I think in the XNORNet they mention 32x memory savings and 52x computation speeding.
I would also like to ask specific which specific files should I look for in the repository to understand the optimization part.

@kaivu1999 that's a really good discussion you brought up and has been of interest since the release of BMXNetv1.
pointing out the findings of @yanghaojin that for training you skipped the GEMM computation and use the CuDNN implementation for training on GPUs (as per your justification), theoretically for inference (say real-time constraints on robotic platforms) XNOR computations could be speeded up on GPUs as well - did you find the reasons for this ? (eg. well optimized CuDNN code by teams of nVidia?, etc.)
now only considering CPU inference which is the main focus of the whole research (as the ultimate goal is to run DL models preferably in realtime on CPU-only devices):
the results of @kaivu1999 dont coincide with the results of the BMXNet papers and preceeding XNORNet paper (theoretically up to 32x), maybe we can verify this with BMXNetv2..
clearly there are other steps involved like input binarization or patch2col that add up to the total processing time as part of the convolutional operation
@yanghaojin, @Jopyth would be interested as well if you could elaborate a bit on that?

using CuDNN for training because it is over optimized and much faster than the preliminary implementation of the xnor kernel in cuda, I believe an optimized xnor cuda kernel will indeed improve the speed a lot, but we are not expert for that thus leave this to the community for now.
In our first paper, the evaluation we mainly compared the xnor-gemm function with the traditional DOT operators using CPU. so xnor cpu kernel is indeed much faster than the standard dot engine like CBlas, but as I mentioned here there is also huge optimization potential regarding the whole convolution layer. In BMXNetV1 our evaluation showed that the portion of xnor-gemm computation is too small compared to the overall computations of a single Qconv layer (see this screenshot this result might be outdated but I think the ratio of each part doesn't change much).
According to the 52 times speedup mentioned in the xnor paper. I think this number is also based on the xnor-gemm function, not the whole conv layer. And they only compared to the naive implementation of a dot engine, even without reporting the comparison with CBlas (atlas or others). I have seen their code in Darknet before they removed them years ago (since they initialized a startup company XNOR.ai thus remove the code from darknet).

Thank you very much,
@yanghaojin  for a detailed explanation.
So I would like to ask a few things.

In BMXNet paper, the code example for LeNet and Binary LeNet where a specific order for layers is used (QActivation-QConv/QFC-BatchNorm-Pooling). Is this has to play any role in optimization ?
So is input_binarisation done once and all the layers follow or Do we have to binarize the input layers every time for each layer?
There are these other layers like Pooling and BatchNorm which don't have their accelerated counterparts as I see only (QActivation,QConvolution,QFullyconnected). So do they cause unnecessary data type conversion (if we expect that binarisation happens only once for all consecutive activation layers, which is the whole point right ?)
Is there an effect of batch_size for inference ( Theoretically it should not be) but I get an increase in speedup with an increase in batch_size?

Also I got these numbers for BMXNet v1 considering only inference.

NIN-Cifar10
VGG-11-Cifar10

Where the blue line represents the experiment where I tried to use the accelerated layers as much as possible. I accelerated the last layer (Not recommended for accuracy) and for NIN I had to approximate some of the layers in between so that the input channels are a multiple of 64 as the accelerated seems to support input layers with size in multiples of 64.
Also for a Network with just Convolution and Activation layer pairs. In a network with 8 of such pairs where I accelerated 7 of them in the binary version, I am getting a Speed Up of 6.8x for 256 as batch_size.
Also I would really like to know about the update on similar notes ?

I would also like to ask specific which specific files should I look for in the repository to understand the optimization part?


[FLINK-9384]KafkaAvroTableSource failed to work due to type mismatch

What is the purpose of the change
An exception was thrown when using KafkaAvroTableSource, complaining like:
 "TableSource of type org.apache.flink.streaming.connectors.kafka.Kafka011AvroTableSource returned a DataStream of type GenericType<org.apache.flink.types.Row> that does not match with the type Row(id: Integer, name: String, age: Integer, event: GenericType<java.util.Map>) declared by the TableSource.getReturnType() method. Please validate the implementation of the TableSource."

This pull request fixes the issue by referencing the implementation of KafkaJsonTableSource.
Brief change log

Add a TypeInformation as the second argument to the constructor of AvroRowDeserializationSchema
AvroRowDeserializationSchema overrides the method getProducedType()
KafkaAvroTableSource constructs a TypeInformation converted from AvroRecordClass and pass it to AvroRowDeserializationSchema

Verifying this change
This change is already covered by existing tests, such as KafkaAvroTableSourceTestBase.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): ( no)
The public API, i.e., is any changed class annotated with @Public(Evolving): ( no)
The serializers: (yes)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

Thanks @suez1224 for reviewing. I've committed a new patch as per your suggestions. As for the unit test, the issue was exposed in my integration tests which requires a kafka setup, any advice on how it could be done in unit tests?

Hi @suez1224, I've added some checks in the unit tests(KafkaAvroTableSourceTestBase), which would fail before the fix. Meanwhile, I found an issue in the previous version (specifically,  the produced type info should be inferred from AvroRecordClass not TableSchema), which has been fixed. Would you please take a review?

@twalthr Thanks for the comment. I've commited a new version as per your suggestion. Particularly, I agree that TypeInfomation should always be derived from the 'avroRecrodClass', but the constructor of AvroRowDeserializationSchema only takes argument as type 'Class<? extends SpecificRecord>'. In this version I've changed the argument as 'Class<? extends SpecificRecordBase>' and fixed relevant test cases accordingly. This version of fix has passed relevant unit tests and my local integration tests.

Thank you @tragicjun. LGTM. Merging...

@twalthr sure, just a second, I never did this before, let me try

@twalthr the latest commit could be merged which contains all the changes.


Stack View

Ability to view the stack layout for the current stack and manually add stack variables.

Also, the ability to see the registers and which variables occupy the registers at the point of the selected instruction. Along with #221 , the stack view could change from instruction-to-instruction.

This would also help diagnose some of the current dead store elimination problems fwiw.


[TEST] Update Upgrade Test Case to Support Two-Stage Upgrade for #6647

What's the test to develop? Please describe
The Replace engineImage field in CRDs with image feature was introduced as an enhancement in 1.6.0, The existing 2-stage upgrade test cases are not handled this scenario, as versions 1.4.x and 1.5.x do not support this feature.

PR-1538 was merged. : 2 stage upgrade tests report


Language review: Meet Qt chapter

Language review. Also changing import version in a code snippet to .12 (to reflect LTS version). This allows for the use of TapHandler instead of MouseArea later on.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


GEODE-6370: Clean up LogWrapper between tests

This avoids subsequent tests failing because they are checking for the
presence of LogWrapper.

Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

[ ] Has your PR been rebased against the latest commit within the target branch (typically develop)?

[ ] Is your initial contribution a single, squashed commit?

[ ] Does gradlew build run cleanly?

[ ] Have you written or updated unit tests to verify your changes?

[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to<EMAIL_ADDRESS>

The UpgradeTest failures are WANRollingUpgradeNewSenderProcessOldEvent which seems to be a reoccurrence of GEODE-3967. I recommend having @gesterzhou take a look at that these failures. He marked GEODE-3967 as resolved on Jan 9th.


invalid API use

https://github.com/lamassu/lamassu-scripts/blob/db7eb3869bf96fd5a7acecc66314bd9ce4e29846/bin/ssu-cartridges#L71
this condition will never happen, see many API: http://vitaly-t.github.io/pg-promise/Database.html#.many

Thanks, @vitaly-t. We just moved over to pg-promise from a different library where we had to check for those things. Good catch.
Btw, how did you find this repo?

GitHub search engine shows all new entries against the library.
What was the library, by the way? :)

It was https://github.com/sapienlab/dbh-pg.
And thanks for pg-promise -- great library!

Alright, that's an oldie :)
Cheers!

Fixed in https://github.com/lamassu/lamassu-scripts/commit/4e62b071fe18835fe9aa684a86bfc2904dc9fec7

Just so, manyOrNone has a shorter alias - any ;)

Thanks, that will come in handy. I must have missed that!
Josh Harvey
Co-founder
Lamassu, Inc.
https://lamassu.is
On Fri, Jun 3, 2016 at 10:57 AM, Vitaly Tomilov<EMAIL_ADDRESS>wrote:

Just so, manyOrNone has a shorter alias - any ;)
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/lamassu/lamassu-scripts/issues/42#issuecomment-223515149,
or mute the thread
https://github.com/notifications/unsubscribe/AAASK7oJuBNunfxlYqqYsLx7IBZnZ7ziks5qH95vgaJpZM4IsmVJ
.


Add hugepage Downward API support.

In Kubernetes 1.20, an alpha feature was added to expose the requested
hugepages to the container via the Downward API. Add a new API to check
to see if the hugepage data is passed to the container via the Downward
API, and if so, return the data to the caller.
This is feature is disabled by default in Kubernetes. If enabled when
Kubernetes is deployed via FEATURE_GATES="DownwardAPIHugePages=true",
then Downward API values can be passed to the container.
Signed-off-by: Billy McFall<EMAIL_ADDRESS>

@zshi-redhat PTAL

/lgtm


Cucumber reports does not show step details when we call other feature file

i am using jenkins to generate cucumber html report. till now report were displaying all the steps feature and scenario.
recently I have refactored code to make it more reusable now my code structured ( based on called and caller ) is like this
workflow. feature
Feature: using login
    Scenario: using login cycle
      # login to system
      *def loginResult = call(login.feature@login)
      # clear login details
      *def clearLoginDetails = call(home.feature@clear)
      # logout
      *def logout = call(login.feature@logout)

my called feature looks like
login.feature

Feature: using login

    @login
    Scenario: user login
       Given 'v1/login'
       When method get
        Then status 200

i am using JUnit 5 parallel runner based on tag
cucumber html jenkins pluns setup
stage("generate reports") {
                    sh "yum install git -y"
                    junit allowEmptyResults: true, testResults: '**/target/surefire-reports/TEST-*.xml', fingerprint: true, keepLongStdio: true
                    def author = sh script: "git show --name-only", returnStdout: true
                    cucumber buildStatus: 'STABLE',
                            fileIncludePattern: '**/target/surefire-reports/*.json',
                            trendsLimit: 100,
                            reportTitle: ' test report',
                            sortingMethod: 'ALPHABETICAL',
                            classifications: [
                                    [
                                            key  : 'Commit details',
                                            value: author
                                    ]
                            ]
                }

i dont see stps details in cucumebr html report.
please refer attached screenhot

@patel-5788 cucumber does not have the concept of calling features at all.
please use the karate reports, see example: https://stackoverflow.com/a/61685169/143475
you are welcome to contribute code. closing as wontfix

@patel-5788 cucumber does not have the concept of calling features at all.
please use the karate reports, see example: https://stackoverflow.com/a/61685169/143475
you are welcome to contribute code. closing as wontfix


Fix animations on replace actions

This PR changes how replace action is handled. Previously we always dispatched a replace navigation action with animations disabled (which doesn't really work all the time). After this change, we only dispatch a replace navigation action in case the new route is different from the current route. If the route is the same, we can just update the params of the screen with the new replaced URL.
This fixes issues with animations and double-renders of the replaced screen.

This probably would require that the RNVisitableView supports fetching new page after updating the url prop.

Superseeded by #90


Comma in msbuild commands are not escaped

What You Are Seeing?
MSBuild generated commande line is invalid when property contains a comma.
What is Expected?
Valid generated command line
What version of Cake are you using?
v0.19.5
Are you running on a 32 or 64 bit system?
64 bits
What environment are you running on?  Windows? Linux? Mac?
Mac
Are you running on a CI Server?  If so, which one?
No
How Did You Get This To Happen? (Steps to Reproduce)
I made a simple cake script available there https://gist.github.com/Julien-Mialon/1918cc8c9db470558eaa8e0f9fdfb77b to reproduce the problem
Output Log
Here is the generated command
Executing: /Library/Frameworks/Mono.framework/Versions/Current/Commands/msbuild /v:normal /p:CodesignKey='iPhone Distribution: Company, SA (XXXXXXXXXX)' /target:Build "/Users/Julien/Documents/Source/tmp/cake/test.sln"

Microsoft (R) Build Engine version <IP_ADDRESS> (xplat-2017-02/c2edfeb Thu May 18 13:58:03 EDT 2017)
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1006: Property is not valid.
Switch:  SA (XXXXXXXXXX)

For switch syntax, type "MSBuild /help"

Jenkins has a similar issue and resolve it by replacing every comma by %2c (link to issue for jenkins : https://issues.jenkins-ci.org/browse/JENKINS-5552?focusedCommentId=143769&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-143769)
I would be happy to propose a PR, if the solution to replace comma by %2c is accepted.

@Julien-Mialon your suggestion sounds valid, so a PR sounds reasonable, there's a few more special values which probably also could be covered by the PR

https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild-special-characters
https://docs.microsoft.com/en-us/visualstudio/msbuild/how-to-escape-special-characters-in-msbuild

also looks like uppercase hex is preferred.

@devlead I'm not sure all of specials characters should be escaped because they could be valid values for msbuild properties. The only two characters that really cause issues are comma (,) and semicolon (;)
I'm gonna start working on a PR for this two but we can talk about adding others if really needed.

Sounds reasonable 👍

Fixed by #1793

This has broken the ability to specify multiple constants. Example:
settings.WithProperty("DefineConstants", "A=a;B=b");
The semi-colon should not be escaped in this scenario.
Should a new issue be raised for this, @devlead ?

@georgehemmings doesn't   .WithProperty("DefineConstants", "A=a","B=b"); work? If not then yes please raise an separate issue. Comments on closed issue easy to miss.

@devlead no, it seems only one /p:DefineConstants is allowed, the last one wins.
.WithProperty("DefineConstants", "A=a;B=b".Quote()); last worked in v21.1.
Ok, I'll raise a new issue.


[Abstraction] Configuration-based teleportation of signals

Currently, each IDA that wants to publish events must add a @receiver-decorated function that listens for a signal and calls the event-bus producer (specifying a topic and key-field as well). It would probably be better to instead have openedx-events do this automatically. Perhaps we could have settings like the following:
EVENT_BUS_AUTOSEND = [
  {
    'signal': 'org.openedx.content_authoring.course.catalog_info.changed.v1',
    'topic': 'course-catalog-info-changed',
    'key_field': 'catalog_info.course_key',
  },
  ...
]

IDAs could still hardcode the relationship between signals and events when there's a more complicated relationship, but would not be required to for the simple, common case.
IDAs would have to specify openedx-events as an installed-app or otherwise allow it to perform some initialization at startup (so it could read the config and attach its own signal receivers).
Depending on the outcome of https://github.com/openedx/openedx-events/issues/86 the key-field may not be necessary to specify.

Questions (some of them are likely silly, but it's been a while since my head's really been in this space):

Can we auto-send everything?
Can individual event bus implementations derive the topic from the signal identifier?
If (1) is not possible, but (2) is, can the configuration just be a list of the events themselves, like:

EVENT_BUS_AUTOSEND = [
    STUDENT_REGISTRATION_COMPLETED,
    SESSION_LOGIN_COMPLETED,
    # etc.
]

I'd love to hardcode the topic on the OpenEdxPublicSignal.
We could auto-send everything (iterate over list of OpenEdxPublicSignal, attach receiver to each) but then we'd definitely need to work on https://github.com/openedx/openedx-events/issues/79 first. :-)

I'd love to hardcode the topic on the OpenEdxPublicSignal.

FWIW, my hope would be that the bus implementation could derive a topic from the signal identifier, e.g. org.openedx.content_authoring.course.catalog_info.changed.v1 gets mapped to content_authoring:course:catalog_info or something.

Ah, I see what you mean. Although if we went that route, would we really want the implementations doing it, or the abstraction layer on top of it (something in openedx-events)?

I'm fine with doing that translation at the Kafka driver layer. In the worst case, there's a little extra duplication for different transports, and we can create that separate abstraction later. But I don't think we'll really know what would go in that middle layer until we have two or three of these. For instance, Redis streams doesn't really have partitioning, so it has to overload streams for that. That's a kind of mapping decision that's local to Redis's particular capabilities, and I don't think we need to make an abstraction layer for it.
Honestly, I kind of hope that there's no middle abstraction layer at all–that everything can be done with just the in-process stuff in openedx-events, and then a separate kafka-aware library. I'd like to be able to run Kafka and Redis streams concurrently if necessary (for migration purposes), because neither of them know about the other.

@rgraber I think this is completed as of https://github.com/openedx/edx-platform/pull/33642

There are a couple of places where IDAs still call get_producer directly, but I've notified owners of that code and they can make the switch if they'd like.


Style not loading on pages apart from the login

It seems as though the styles fail to load on screens other than the Login screen. This is on a new project with .Net Core 3.1.3 SDK installed
Console errors:

Home screen:

Please restore client library first. Libs aren't committed to git.

Please check this https://github.com/moemura/AdminLTE.Core/issues/1


Node/python client to golang server: OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

What version of gRPC and what language are you using?
gRPC latest version, golang (server), NodeJS/Python (client)
What operating system (Linux, Windows, …) and version?
Windows 10 x64
What runtime / compiler are you using (e.g. python version or version of gcc)
Python 3.6.5/ Node 8.12.0
What did you do?
I simply copied the "helloworld" example code and used  grpc.credentials.createSsl(<certificate>) on the client (node, or credentials = grpc.ssl_channel_credentials(root_certificates=trusted_certs) in python) and creds, err := credentials.NewServerTLSFromFile(<certificate>, <private_key>) on the golang server to enable SSL.
What did you expect to see?
I expect to see the helloworld example working for every client language
What did you see instead?
If I use the golang client the example works even with SSL with the server and client on different machines, while with other client languages the client always crashes both on the same and different hosts with the message " Handshake failed with fatal error SSL_ERROR_SSL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED".
Anything else we should know about your project / environment?

It sounds like the host you're connecting to does not match the hostname in your certificate. For gRPC Python/Node clients, if the hostnames don't match you will need to specify the grpc.ssl_target_name_override override (this should only be used in tests), as in the Python interop client here. gRPC Go has its own mechanism for this, as seen in the Go interop client.
If this doesn't explain the issue, can you provide a repro case, or at least the code you're using for gRPC Python and Go clients, as well as the Go server code? Thanks

@GuillaumeCisco There was recently an issue fixed in gRPC core involving local trust anchors. Can you check if this resolves your issue? It's too recent to be in a release, but you can grab a daily build from here.

Thank you @gnossen for this tip.
Very surprisingly, installing this new version of grpcio led me to a lot of new errors.
 And very weirdly this error appears on some containers, not all...
More importantly all my python subprocess.Popen commands are hangings forever.
Going bask to version 1.8 fixes the issue... (not for the pure Go resolver which broke my entire config, but it seems to be a known issue for fabric hyperledger: https://jira.hyperledger.org/browse/FAB-13419).

I do not understand why the grpc code sends me a CERTIFICATE_VERIFY_FAILED.
The only thing I have is:
OSError: [Errno fail to send transaction proposal] <_Rendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Connect Failed"
	debug_error_string = "{"created":"@1549533605.346930991","description":"Failed to create subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":2261,"referenced_errors":[{"created":"@1549533605.346895537","description":"Pick Cancelled","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":245,"referenced_errors":[{"created":"@1549533605.346818554","description":"Connect Failed","file":"src/core/ext/filters/client_channel/subchannel.cc","file_line":867,"grpc_status":14,"referenced_errors":[{"created":"@1549533605.346633641","description":"Handshake failed","file":"src/core/lib/security/transport/security_handshaker.cc","file_line":257,"tsi_code":10,"tsi_error":"TSI_PROTOCOL_FAILURE"}]}]}]}"

I tried looking at the c++ code of grpc for understanding what is going on, but without success.
Right now I cannot use grpcio 1.19 and I have no information why the certificate verification fails :/

I've just succeeded making my network works without CERTIFICATE_VERIFY_FAILED
All was about config in hyperledger fabric and fabric-ca
Along this journey I understdood a lot of things about grpc. Especially grpc 1.9 can break a lot of things regarding the pure Go resolver and the cGo resolver.
Maybe my experience will raise information about subprocess.Popen python calls hanging with grpc 1.9 :)

@GuillaumeCisco, can you please open an issue in grpc-go repo regarding the pure Go and cGo resolver issues you are facing. gRPC 1.9 is really old. Please try with latest release and see if you are still facing the same issue.

@GuillaumeCisco Thank you for the tip on the hanging calls to subprocess.Popen. I was able to verify this behavior on master and I was able to verify that it was not present in 1.18. I've filed an issue if you care to follow it.

@GuillaumeCisco As a temporary workaround while the fix for the root cause of the hang is under review, running with the environment variable GRPC_ENABLE_FORK_SUPPORT=0 will disable the core prefork handler that seems to be causing the problem.

@FabioDominio @GuillaumeCisco Can you give it another go with 1.19.0rc1?

@gnossen I can confirm python subprocess.Popen works correctly with 1.19.0rc1


readme: add sinecord game to the list

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Please sign the cla and I will merge


Add rectangular matrix support for FullPivLu

This is an attempt to resolve issue #161.
I'm pretty uncertain about the decisions I made in FullPivLu::solve. The main problem is that, unlike in PartialPivLu, which requires square invertible matrices, FullPivLu::solve can now be given a problem that has no solution or has infinitely many solutions. Thoughts on how I went about dealing with this? The other option I saw was to require the matrix be square and invertible for FullPivLu::solve.

This is a very interesting PR!
I'll do a thorough review later, but I want to have some high level discussion first.
I must admit I hadn't even considered implementing solve for non-square matrices. Perhaps this is a good idea. However, there are some things I don't fully understand. I'll start by explaining what I find the most useful about rectangular LU decomposition. In particular, I can imagine LU decomposition of rectangular is particularly useful when:

You need the actual factors L, U and the permutation matrices, so you unpack the result.
You want to estimate the rank of a rectangular matrix in a potentially much cheaper way than the SVD (which is more accurate).

Let's consider a system Ax = b, where A is m x n and b is a vector of length m. Let's consider first the case m > n. Here we have an overdetermined system, and in the general case there can be no solution to the system. However, in some special cases, namely when b is in the column space of A, the system may still have a solution.
Let now m < n. In this case we have an underdetermined system. Here we have either no solutions or an infinite number of solutions. In any case, we still need to have b lie in the column space of A for a solution to exist.
As far as I know, one typically uses a least-squares approach to these problems. In the case when Ax = b is consistent and has a solution, then the result is obtained with a least-squares procedure. When Ax = b has no solution, one retrieves a solution x which "almost" satisfies the equations in the sense that || Ax - b || is minimal.
If one is only interested in the exact solution and one has no interest in least-squares approximations, it might be possible to use LU decomposition, although I've never seen this used for this purpose (but keep in mind I'm not at all an expert). From a brief glance, I'm intrigued by your technique. Without having carefully analyzed it, it raises some questions:

It seems to me that this could actually work for overdetermined systems with full column rank. However, I don't think it can capture all cases when the system is rank-deficient, does it?
I believe the same is true for undeterdetermined systems - there might exist solutions that the procedure does not see and an error is returned instead. At least I think so - please correct me if I'm wrong.

I'm open to providing  a solve function that actually finds a solution to the system Ax = b for rectangular A, but in order for it to be truly useful I think it needs to satisfy the following criterion:

If a solution to the system Ax = b exists, then solve must return a valid solution to the system. Conversely, if solve returns a solution, it must be a proper solution to the system.

From my understanding by looking at the code, I think the proposed solve function does not quite satisfy the first requirement. In other words, it may fail to give a solution when one or more solutions to the system exist, which is unfortunate, because it cannot be relied on. Am I correct in this assumption? I'd love to hear your thoughts on this, @prismatica!

I totally agree with @andlon that if solve is to be useful, it must be able to find a solution if one exists. I think that the existing implementation will do that even in the rank-deficient case. Given a rank-deficient matrix with rank r, its LU decomposition will have a square full-rank L matrix, and a rectangular rank-deficient U matrix, with the zero vector for the last m - r rows, and non-zero entries along the diagonal for the first r rows. This means that the last m - r  variables are free variables in the x vector, and can take any value in a valid solution. The first r variables are restricted based on the values taken by the free variables.
The approach taken in the PR is to set all the free variables to zero, and solve the equation using just the upper-left r x r portion of the U matrix, called for convenience U'. Since this matrix is full-rank and square, there will be one and only one solution to this restricted equation. If a solution exists, there will be a solution with all zero free variables, and the first r variables taking the values as found by solving the U' x = b solution. Of course, this method can find solutions to U' x = b that are not solutions to U x = b, which is why the next step is to pad the found candidate with the free variables (which are all zero), and then to verify the solution by mapping through the full matrix (using FullPivLu::Mul) and comparing against the input b.
I think that this implementation meets the criteria set out by @andlon. Am I missing something?

@prismatica this makes sense to me though my opinion weighs less heavily than @Andlon and your own on this.

@prismatica: I took a closer look now, and yes, I believe you are right! You only described the underdetermined situation, however (where U is rectangular and L is square). We must also consider the case where the system is overdetermined, in which case L is rectangular and U is square. For clarity, I'll try to outline a generalized procedure below, based on your approach. Please let me know if I've made any mistakes! I think I might also have found a somewhat better way to verify the solution (?).
Problem statement
We wish to find any solution to the problem Ax = b, or return an error if no such solution exists. We consider the decomposition A = PLUQ. Without loss of generality, it is clear that we can reformulate this as LU x' = b', where x' = Q x and b' = P^T b. Hence we need only consider the problem LU x = b, where L is m x p and U is p x n.
The introduction of the size p lets us describe the problem both for the case of m > n (in which case p = n), m = n (in which case p = m = n) and m < n (in which case p = m).
Existence of solutions
We now write
LU x = [ L1 ] U x = [ L1 U x ] = [ b1 ] = b
       [ L2 ]       [ L2 U x ]   [ b2 ]

where L1 is p x p and L2 is (m - p) x p. We note that L1 is an invertible lower triangular matrix, while L2 is just a general matrix.
Because L1 is invertible, we may write Ux = inv(L1) * b1, and we get the requirement
L2 Ux = L2 inv(L1) b1 = b2

Hence, if there exists a solution to the problem Ux = inv(L1) * b1 and the above requirement on b1 and b2 holds, then there exists a solution to the system Ax = b. Note that this solution is in general not unique unless U is invertible.
Solutions to Ux = inv(L1) * b1
We now consider the problem of a possibly rank-deficient upper trapezoidal matrix U. We let r = rank(U) and denote c = inv(L1) * b1. Because of the pivoting scheme, we can write the system in the form
Ux = [ U1   U2 ] [ x1 ] = [ c1 ] = c
     [  0    0 ] [ x2 ]   [ c2 ]

where U1 is an invertible upper triangular matrix of dimensions r x r and U2 is r x (n - r). We now immediately get the following requirement for existence of a solution:
c2 == 0

If this holds, we may find any particular solution by realizing that x2 is a free variable which we may choose arbitrarily. Naturally, x2 = 0 is a sensible choice, in which case we obtain
x1 = inv(U1) * c1.
An algorithm for computing a solution
Since our proof was constructive, we may formulate an algorithm for computing any arbitrary solution to the system Ax = b if it exists, or return an error if it does not.

Compute c2. If c2 != 0, then abort (no solution exists).
Compute c1. If L2 * c1 != b2, then abort (no solution exists).
Solve the system U1 x1 = c1.
Form the solution x = [x1, 0].

In practice one also needs to multiply by the appropriate permutation matrices, of course.
I suggest doing elementwise comparisons in steps 1 and 2 (in fact, we don't have to fully form the product L2 * c1 if we want to save allocation, but this can be a later optimization).
I'm a little worried by the overhead of verifying that the solution is actually valid. It seems to me that depending on matrix dimensions it may represent a considerable cost.
OK, this took me some time. Would greatly appreciate if someone could check my steps! I haven't seen this before, @prismatica, so I really learned something new and interesting! Thanks :-)

Thanks for the clear write-up, @andlon! I might be misunderstanding a few steps in your derivation, so I have a question:
My impression is that, to a certain extent, one can choose the size of the L and U matrices, as long as the product of the two is the size of the original matrix. In the FullPivLu decomposition, I think I've implemented it so that the L matrix will always be square and, since it is a unit lower triangular matrix, also invertible. This is because it can be viewed as the product of some number of elementary matrices (the ones that express the Gaussian elimination that results in the U matrix). Ignoring the permutation matrices for now, this is because one performs elementary row operations in a sequence to the A matrix to get U: (L1 * L2 * ... * Ln) * A = U. Then, L = inv(L1 * L2 * ... * Ln) and since elementary matrices are square and invertible, so is L. The permutation matrices just result in some new, permuted matrix A' to whom the elementary matrices are applied. Is this correct, or have I missed something?
If this can be done, I think the L2 in the derivation is empty, and so we can skip a few of the steps involving it. Is this possible?

Well, let's say that we want to have L a square matrix. Then it has to be m x m, since A is m x n. If m <= n, then L is already square by virtue of the usual decomposition (and U is rectangular). Now, consider the case m > n. Using my previous notation for rectangualar L (i.e. m x p), we see that
[ L1 ] U = [ L1U ] = [ L1  0 ] [ U ]
[ L2 ]     [ L2U ] = [ L2 L3 ] [ 0 ]

for any matrix L3 with dimensions (m - p) x (m - p). If we choose L3 such that it is lower triangular with unit diagonal, we get your proposed L matrix, and so you are indeed correct that it is possible to obtain a decomposition L U where L is square and U rectangular. However, it is not correct to say that L2 is "empty" - it just becomes part of the larger L matrix.
Note that as our "LU matrix" that we produce during decompose (the matrix containing both the L and U factors) is rectangular, our L is in fact rectangular (with more rows than columns), so you cannot simply forward substitute directly (though depending on the implementation it is possible that forward substitute will still return the solution corresponding to L1, I haven't checked).
The approach involving a full square matrix would potentially require a lot more work, as you'd have to solve a system of size m x m, which for m >> n can be very expensive compared to solving a system with L1 of size n x n and verifying the solution by performing a matrix-vector multiply with L2 (which is (m - n) x n).
Does this make sense? Please ask if I haven't been clear!

That does make sense, thanks!
Unfortunately, real life has caught up with me, and I won't be able to give this the time I'd like in the next few weeks. I'll try to get a new version of this that implements your changes up as soon as I can, but if it takes longer than it has in the past, I promise I haven't forgotten about you all :)

@prismatica: no worries! Please take as much time as you need. This PR is also very independent of other ongoing work in the library, so there should be no issues as a result :-)

Alright, sorry, I have another potentially stupid question. You say:
Compute c2. If c2 != 0, then abort (no solution exists).
How does one compute c2? It looks to me like c2 is the product of the zero matrix and what you've called b2. Obviously, the product of the zero matrix with anything is zero, so I must be misunderstanding what your first step is.

This most recent commit doesn't address all of @andlon's concerns. It just changes how unpack works. Instead of allocating a m x m matrix for the L portion and using the internal m x n memory for the U matrix (which doesn't make sense if m >> n), unpack now decides which matrix to store in the repurposed internal memory, and which to allocate. If m >> n, the L matrix will now be m x n, and the U matrix will be n x n, which saves memory overall.
Also, I removed the Mul trait at @AtheMathmo's suggestion.
I'm still trying to understand @Andlon's algorithm, though it's going slowly (my own fault, of course!)
 And if there were, I'd surely be the champion of foolish questions under normal circumstances :-)
Sorry, I was not so explicit with regards to the meaning of some of the quantities I introduced in the above description. We define c = inv(L1) * b1, where b1 corresponds to the first p elements in the b vector. We now define c2 to be the last p - r elements of c (and c1 to be the first r elements). Hence, we must compute c by solving the system L1 * c = b1 and taking the last p - r elements to get c2. (in the above, r is the rank of U)
There is/was actually a typo in the original "algorithm" that I presented: we need to compute L2 * c and check that it's approximately equal to b2, not L2 * c1 as it said at first.

It looks to me like c2 is the product of the zero matrix and what you've called x2.

I understand your confusion. This is not it's definition, but the requirement that c2 needs to fulfill. In other words, c2 is defined as I stated above, but for a solution to exist for the overdetermined system, c2 needs to be precisely zero! Hence we need to check if c2 is actually zero to see if the overdetermined system has a solution. Of course, we have to take into account roundoff errors from finite precision arithmetic, which complicates matters.
I hope this helps! Please don't hesitate to ask more questions. I think we're all learning from this :-)

I've updated the code to what I think is the algorithm listed above, but I think I must be missing something. For example, let's take the case of
Ax = b
A = [1 2 3; 1 2 3]
b = [1; 2]

Then,
L = [1 0; 1 1]
U = [1 2 3; 0 0 0]

Then, since L is already pxp, L1 = L, so c == [1;2].
I think this means that L2 is 0x0.
Since the rank is 1, U1 = [1], so [1] * x = [1;2], and x == [1;2]. However, A * [1;2] != b.
Is there an additional verification step required, such as the explicit map through like in the original implementation, or have I missed something again?

It may be that there are mistakes in the algorithm I proposed, but I think it works for this particular example at least! Let's go through it step by step.

Compute c from L1 * c = b1. As you correctly point out, we have L1 = L and also b1 = b, so we just need to solve L * c = b. However, note that c != b! That means we need to solve the system:

[ 1 0 ] c = [ 1 ]
[ 1 1 ]     [ 2 ]

=> c = [ 1 ]
       [ 1 ]

Since U has rank r = 1, we have that c1 = [ 1 ] and c2 = [ 1 ] != 0. Per step 1 in the above algorithm, this means that the system has no solution and we abort the procedure.
To see that the system actually has no solution, let's write it out with variables. The system Ax = b is the same as:
x1 + 2 * x2 + 3 * x3 = 1
x1 + 2 * x2 + 3 * x3 = 2

If we make x1, x2 and x3 satisfy the first row, then they cannot simultaneously satisfy the second row! Hence the system in question has no solution.
I went through this a bit quickly, so it's very possible I've made mistakes, but I hope it should be correct. I hope this helps. Please don't hesitate to ask more questions!

OK, I finally figured it out. My apologies for all the back-and-forth and delays, and thanks for your help and patience! :)
Please let me know if there's something else I should change.


pencil2d: Add version 0.6.5

Add Pencil2d manifest https://www.pencil2d.org/download/

Hi, I accepted all the requests. It is ok now?
Thanks

Hi, I accepted all the requests. It is ok now?
Thanks

/verify

/verify


How to build job with "predefined parameters" on a job that does not take parameters

In Jenkins you can "Trigger/Call builds on other projects" and you can pass in "predefined parameters" as key/value pairs on jobs that are not parameterized.
The key/value pairs appear as env vars in the triggered job.
I can't seem to figure out how to use the api to do this.
I tried passing in "files" to the invoke function, like this: https://github.com/salimfadhley/jenkinsapi/issues/245
But no love.
I also tried passing in build_params to the invoke function but I get:
jenkinsapi.custom_exceptions.BadParams: This job does not support parameters
Looking at the parameters to the invoke function, it doesn't seem like this is supported:

inspect.getargspec(j.invoke)
ArgSpec(args=['self', 'securitytoken', 'block', 'build_params', 'cause', 'files', 'delay'], varargs=None, keywords=None, defaults=(None, False, None, None, None, 5))

Any idea how this can be done?

Here is a screenshot of what I mean.  The _foo job is NOT a parameterized build, but I can trigger _foo and pass in parameters:

Hey, I dont have a whole load of time for this project any more - but I vaguely remember that we have unit-tests for triggering parametrized builds. If you make a pull request that illustrates a failing test then we can have a go at fixing the library to accommodate your need.

It is not possible to do that using REST API. Perhaps Parametrized Trigger plugin does some magic, but we can't do the same.


change scene without changing the robot

Hi,
I was wondering if without using the environment class, I can run the simulator and, at some point, change the scene without changing the robot, I want to keep the same robot instance.
I used the reload function of the simulator class, and I successfully import the new scene, but when I try to import the old robot it gives an assertion error.
The code (based on scene_interactive_object_rand_example.py):
##########################################
from igibson.robots.turtlebot_robot import Turtlebot
from igibson.simulator import Simulator
from igibson.scenes.igibson_indoor_scene import InteractiveIndoorScene
from igibson.objects.ycb_object import YCBObject
from igibson.utils.utils import parse_config
from igibson.render.mesh_renderer.mesh_renderer_settings import MeshRendererSettings
from igibson.render.profiler import Profiler
import igibson
import os

def main():
    config = parse_config(os.path.join(igibson.example_config_path, 'turtlebot_demo.yaml'))
    settings = MeshRendererSettings(enable_shadow=False, msaa=False)
    sim = Simulator(mode='headless', image_width=256,image_height=256, rendering_settings=settings)

    scene = InteractiveIndoorScene('Rs_int',
                              build_graph=True,
                              pybullet_load_texture=True)
    sim.import_ig_scene(scene)
    turtlebot = Turtlebot(config)
    sim.import_robot(turtlebot)

    for _ in range(10):
        obj = YCBObject('003_cracker_box')
        sim.import_object(obj)
        obj.set_position_orientation(np.random.uniform(
            low=0, high=2, size=3), [0, 0, 0, 1])

    print(sim.renderer.instances)

    for i in range(100):
        with Profiler('Simulator step'):
            turtlebot.apply_action([0.1, 0.1])
            sim.step()
            rgb = sim.renderer.render_robot_cameras(modes=('rgb'))

    print("changing scene")
    sim.reload()
    scene = InteractiveIndoorScene('Merom_0_int', build_graph=True,pybullet_load_texture=True)#Wainscott_0_int
    sim.import_ig_scene(scene)
    sim.import_robot(turtlebot)

    for _ in range(10):
        obj = YCBObject('003_cracker_box')
        sim.import_object(obj)
        obj.set_position_orientation(np.random.uniform(
            low=0, high=2, size=3), [0, 0, 0, 1])

    print(sim.renderer.instances)

    for i in range(100):
        with Profiler('Simulator step'):
            turtlebot.apply_action([0.1, 0.1])
            sim.step()
            rgb = sim.renderer.render_robot_cameras(modes=('rgb'))
        pdb.set_trace()
    sim.disconnect()

if __name__ == '__main__':
    main()

####################
And the error
####################
  File "simulator_interactive_various_scenes.py", line 65, in <module>
    main()
  File "simulator_interactive_various_scenes.py", line 45, in main
    sim.import_robot(turtlebot)
  File "/home/louisette/.local/lib/python3.8/site-packages/igibson/simulator.py", line 267, in wrapped_load_func
    res = load_func(*args, **kwargs)
  File "/home/louisette/.local/lib/python3.8/site-packages/igibson/simulator.py", line 913, in import_robot
    state.initialize(self)
  File "", line 51, in initialize
    assert not self._initialized, "State is already initialized."
AssertionError: State is already initialized.

##########################
I understand that the object was already initialized and therefore it cannot be imported again. But I cannot see another way of doing this.
Any ideas?

When you reload the sim, your original robot gets deleted along with everything else in the simulator. That means not only is the body not where you expect it to be anymore, but it's not even loaded into the physics engine.
Overall, we don't support using the same robot instance across different simulator loads.
But I wonder what you are trying to do this? Perhaps we can find a workaround.

Thanks for replying so fast!
Actually, I just want to change the scene where the robot is.
Say something like there is a village of houses and the same robot goes from one house to the other after cleaning up.
My initial solution was to import the scenes sequentially in the same simulator load, but I did not managed to make it work.
Any suggestion?

You can create another robot with the same config in your new scene. It shouldn't be too important that it's not the same robot but another robot of the same type with the same configuration right? Or is there some sort of state that you want to retain across the scene change? Perhaps there's some way we can copy over that state.

I will try that, but now I will go on vacation. Please keep this issue open, I will come back with some feedback.
Tks!

Hi,
To create another robot works. Overall, this solution of reloading the simulator, loading a new scene and a new robot, takes a lot of time (and it is very verbosed!), but it works. If you have any other ideas, I am open to cleaner solutions.
Anyway, thanks!
You can close the issue.


Require 2FA for creating a new project

We can start requiring 2FA to be enabled when creating a new project, to help ratchet more users into 2FA earlier.

@dstufft  I'm seeing many "Require 2FA to do X", is it sensible to also require the same for new users in general? That way expectations are set right away rather than "You can create an account but immediately after trying to do anything you need 2FA".

https://github.com/pypi/warehouse/issues/13762

Ah I see, thanks for pointing that one out!

Closing now that 2FA is required for all users.


Remove extra MSS field from TCP sockets

Description
This PR removes an extra MSS field (usCurMSS) in the TCP sockets since it was always identical to another field (usInitMSS). Both of these fields are then replaced by a new field (usMSS).
Test Steps

Related Issue

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks for this clean-up. Originally the two fields had a different meaning, and after some changes, they became the same.
Thanks for this PR, it has my approval.


all.sh: make it possible to run a subset of the components

Break up all.sh into components and make it possible to run a subset of the components by listing them on the command line.
To this effect:

Reorganize all.sh into functions.
A very minimal amount of tidying up. I did remove one redundant component.
All components back up config.h systematically.
If you pass component names on the command line, all.sh only executes these components. With --except, all.sh runs all the components except the ones named on the command line. With no non-option argument, all.sh executes all the components. Run all.sh --list-components to list available components.

Apart from some minor differences in the lead-in and clean-up to each component and some minor component reorganization, the result and reporting of all.sh should be the same as before.
Because the review of this PR was delayed and it contains commits that are very tedious to replay and to review, I merged mbedtls-2.16.0 rather than rebasing my work on top of it. The commits before the merge were previously submitted in #2235 which has been reviewed.
Potential follow-ups, which are explicitly out of scope in this PR:

Simplify keep-going mode: if a component fails in the middle, just move on to the next component.
Make the reporting fit the component model better: report the names of failing components, report the number of passing components...
Self-checks to ensure that when someone adds a component, they don't forget to add it to the default list.
Aids to parallelization. The primary goal of this PR is to pave the way for parallelization, not to achieve parallelization.

Run of all.sh (should succeed): https://jenkins-internal.mbed.com/job/mbedtls-release/907/

Note: the release job (which actually tests all.sh) in the CI passed. The PR job (which doesn't test all.sh) only failed in a known-intermittent test, so is as good as a pass.

@Patater Oops, yes!
I was worried about this mistake, and I considered making run_all_components automatically enumerate all the components. I decided against it because there's no way to enumerate functions in sh (although it would have been possible with a dirty hack such as grep <$0) and because a few components are conditional (but not enough to warrant a conditional component declaration mechanism).

I amended the merge commit to fix the missing component invocations.


Swig dependencies

Hi
I use grunt and in one of my project i need this task. But in your package.json, swig is in devDependencies and not dependencies and i don't have swig in my defaults package so your task don't work on my setting.
Can you move swig in package.json under dependencies categories?
Thanks you and sorry for my bad english.

It's not a good idea to specify swig in the dependencies section because I don't want to prescribe any specific version of swig.  The project which depends on this plugin should specify the version of swig to use, not the grunt plugin.
The reason swig is in the devDependencies is because it is needed for unit tests.
I suppose peerDependencies could be used, but I don't really want to take on figuring out what range of swig versions to specify, and keeping that constantly up-to-date.


Add CPU temp file for Raspberry Pi/linux and fix CPU detection for Raspberry Pi devices

Fixes #1960
Description
Add the location for /sys tree CPU temperature file for Raspberry Pi devices.
Tested on a Raspberry Pi 4B (works), and also on a x86_64 machine (no errors), with neofetch --cpu_temp C .

Also fixed CPU detection, since /proc/cpuinfo reports BCM2835 for all Raspberry Pi devices when booted into 64-bit mode as per https://github.com/raspberrypi/linux/issues/3022#issuecomment-580758031 and https://github.com/raspberrypi/linux/issues/3022#issuecomment-505172683 .

The test for a Raspberry Pi is based upon the discussion here: https://forums.raspberrypi.com/viewtopic.php?t=34678

And I just realized the CPU info is wrong for the Raspberry Pi 4B... it should report a BCM2711!

Hi. I have tested it on an RPI 4B with Raspbian 11. When I use neofetch --cpu_temp C it doesn't show the temp next to the CPU. How can I get this to work. Sorry for asking.


Install pkg-config file

These commits make the library easier to install or package in a way that GStreamer and other Linux-centric applications can use. The pkg-config file allows discovering flags and directories; it is supported by many build systems. Using standard dirs and system jansson is expected by distributions such as Debian.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: franciscovg sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

This would be great to get merged - it makes it way easier to pull in libftl on Linux projects.


Is it possible to read_to_end correctly?

So when we read_to_end and the process is still alive we can't finish it as it doesn't reach a EOF.
When it's stopped though we can't read as well because it's gone.
See https://github.com/zhiburt/ptyprocess/blob/4ba741738953f6ff32f3ba0e5996ea417bdd5d25/examples/cat.rs#L16-L25

Notion that if the spawned process isn't actually interactive it works.
https://github.com/zhiburt/ptyprocess/blob/33311044f51cdf2833145bba6e3ece335d08de24/tests/io.rs#L376-L386


Upgrade dependencies

Codecov Report
Base: 69.88% // Head: 69.88% // No change to project coverage :thumbsup:

Coverage data is based on head (fc0ee78) compared to base (2f88883).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #171   +/-   ##
=========================================
  Coverage     69.88%   69.88%
  Complexity      709      709
=========================================
  Files           143      143
  Lines          3377     3377
  Branches        125      125
=========================================
  Hits           2360     2360
  Misses          941      941
  Partials         76       76

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add annotation and handler class

Add a RepositoryMapping annotation and a HandlerMethodArgumentResolver for it that loads the Repository or throws an exception.

Implemented. Waiting for issue #9 before making PR.

Implemented by PR #18.


Enhance the BallBearingElement Class

Hi guys, can you please improve the BallBearingElement class in the docs?
I've read the docs and it only shows the parameters without an explanation on their meaning and a real-life example. Actually I'm trying to create a model to this bearing: https://www.skf.com/br/products/rolling-bearings/ball-bearings/deep-groove-ball-bearings/productid-6201-2Z.
Would you please help me create this model?

I'm facing the same problem in XLRotor. It's documentation doesn't truly explains the parameters necessary to create a model of a ball bearing. Some illustrations would be helpful in this case haha.

Other things that I've noticed. This examples are showing errors:

https://ross-rotordynamics.github.io/ross-website/examples/example_1.html
https://ross-rotordynamics.github.io/ross-website/examples/example_2.html

This code is giving me some errors. Can you explain why?
Code:
N_bearing = 2

n_balls   = 7
d_balls   = 0.006
fs        = 500.0
alpha     = (np.pi * 8) / (6 * n_balls)
bearings  = [
  rs.BallBearingElement(
    n=i*2,
    n_balls=n_balls,
    d_balls=d_balls,
    fs=fs,
    alpha=alpha,
    tag=(f"Ball Bearing {i}")
  )
  for i in range(N_bearing)
]

Error:
ValueError                                Traceback (most recent call last)
<ipython-input-5-7a53838ec39e> in <module>
      8 fs        = 500.0
      9 alpha     = (np.pi * 8) / (6 * n_balls)
---> 10 bearings  = [
     11   rs.BallBearingElement(
     12     n=i*2,

<ipython-input-5-7a53838ec39e> in <listcomp>(.0)
      9 alpha     = (np.pi * 8) / (6 * n_balls)
     10 bearings  = [
---> 11   rs.BallBearingElement(
     12     n=i*2,
     13     n_balls=n_balls,

~/.local/lib/python3.8/site-packages/ross/bearing_seal_element.py in __init__(self, n, n_balls, d_balls, fs, alpha, cxx, cyy, tag, n_link, scale_factor)
   1237         else:
   1238             f = interpolate.interp1d(nb, ratio, kind="quadratic")
-> 1239             kxx = f(n_balls)
   1240
   1241         if cxx is None:

~/.local/lib/python3.8/site-packages/scipy/interpolate/polyint.py in __call__(self, x)
     76         """
     77         x, x_shape = self._prepare_x(x)
---> 78         y = self._evaluate(x)
     79         return self._finish_y(y, x_shape)
     80

~/.local/lib/python3.8/site-packages/scipy/interpolate/interpolate.py in _evaluate(self, x_new)
    675         y_new = self._call(self, x_new)
    676         if not self._extrapolate:
--> 677             below_bounds, above_bounds = self._check_bounds(x_new)
    678             if len(y_new) > 0:
    679                 # Note fill_value must be broadcast up to the proper size

~/.local/lib/python3.8/site-packages/scipy/interpolate/interpolate.py in _check_bounds(self, x_new)
    704         # !! Could provide more information about which values are out of bounds
    705         if self.bounds_error and below_bounds.any():
--> 706             raise ValueError("A value in x_new is below the interpolation "
    707                              "range.")
    708         if self.bounds_error and above_bounds.any():

ValueError: A value in x_new is below the interpolation range.

Hello, @64J0! Thanks for your feedback.
I will work on a better docstring for BallBearingElement, so parameters can be a little bit clearer.
The fact is, several references uses the contact angle between spheres and inner/outer raceways and the loading as input parameters to calculate the bearing stiffness. I understand it's not much practical, but I haven't found a reference that uses the same parameters set you can find on a catalogue.
About the examples, something is odd because it says it can not find matplotlib library.
And about the last error, i will change the interpolation command so it can also extrapolate values.

Regarding some of the error examples, matplotlib has been removed from our requirements after we moved our plots to plotly.
I will try to update this examples to use plotly instead of matplotlib.

Thanks @rodrigomoliveira1, I think you're right, XLRotor's approach use those terms that you mentioned too and I can't figure out how to get them with the datasheet x.x. Some time ago I've found this software called Mesys that can calculate the bearing props but doesn't looked very deep.

Thanks @raphaeltimbo, I'll update my ROSS local version too.

@64J0,
I just created a PR adding the reference we're using to calculate the coefficients. You may check it out to get a closer look on the model.
I have also fixed the interpolation issue by adding an argument that allows extrapolations.

Awesome @rodrigomoliveira1 , thanks. I'll close this issue now.


[#599] Macrobenchmark test module

Contains:

Created new dedicated ui-benchmark-test module
Updated related Architecture and Setup documentation
New benchmark build type
Connected to the app module - the benchmark build variant from ui-benchmark-test module is matched with release build variant from other modules
Related run configuration with custom Gradle task
Basic startup benchmark test included

Two open questions:

Does the adding release build variant to ui test modules have any pitfalls?
Does the enabling debug signing.configs for the app release variant have any pitfalls?

Closes https://github.com/zcash/secant-android-wallet/issues/599.
This code review checklist is intended to serve as a starting point for the author and reviewer, although it may not be appropriate for all types of changes (e.g. fixing a spelling typo in documentation).  For more in-depth discussion of how we think about code review, please see Code Review Guidelines.
Author

[x] Self-review: Did you review your own code in GitHub's web interface? Code often looks different when reviewing the diff in a browser, making it easier to spot potential bugs.
[x] Automated tests: Did you add appropriate automated tests for any code changes?
[ ] Manual tests: Did you update the manual tests as appropriate? While we aim for automated testing of the application, some aspects require manual testing. If you had to manually test something during development of this pull request, write those steps down.
[ ] Code coverage: Did you check the code coverage report for the automated tests?  While we are not looking for perfect coverage, the tool can point out potential cases that have been missed. Code coverage can be generated with: ./gradlew check for Kotlin modules and ./gradlew connectedCheck -PIS_ANDROID_INSTRUMENTATION_TEST_COVERAGE_ENABLED=true for Android modules.
[x] Documentation: Did you update documentation as appropriate? (e.g README.md, etc.)
[ ] Run the app: Did you run the app and try the changes?
[ ] Screenshots: Did you provide before and after UI screenshots in the description of this pull request?  This is only applicable for changes that modify the UI.
[ ] Rebase and squash: Did you pull in the latest changes from the main branch and squash your commits before assigning a reviewer? Having your code up to date and squashed will make it easier for others to review. Use best judgement when squashing commits, as some changes (such as refactoring) might be easier to review as a separate commit.

Reviewer

[ ] Checklist review: Did you go through the code with the Code Review Guidelines checklist?
[ ] Ad hoc review: Did you perform an ad hoc review?  In addition to a first pass using the code review guidelines, do a second pass using your best judgement and experience which may identify additional questions or comments. Research shows that code review is most effective when done in multiple passes, where reviewers look for different things through each pass.
[ ] Automated tests: Did you review the automated tests?
[ ] Manual tests: Did you review the manual tests?
[ ] How is code coverage affected? We encourage you to compare coverage before and after changes and when possible, leaving it in a better place.
[ ] Documentation: Did you review Docs, README.md, and Architecture.md as appropriate?
[ ] Run the app: Did you run the app and try the changes? While the CI server runs the app to look for build failures or crashes, humans running the app are more likely to notice unexpected log messages, UI inconsistencies, or bad output data. Perform this step last, after verifying the code changes are safe to run locally.

Does the Gradle task :ui-benchmark-test:connectedZcashtestnetBenchmarkAndroidTest actually install the release APK? I > had to install it manually to get the Gradle task to run.

It seems to be working just as it is for me. I'm testing with an emulators, because my physical device is api 33, and thus not supported for now. And I don't have to install any additional apk. Tested with:

./gradlew connectedZcashmainnetBenchmarkAndroidTest form cmd
running the same Gradle task form Gralde panel in AS
running AS run configuration ui-benchmark-test:connectedZcashmainnetBenchmarkAndroidTest, which is actually best choice, because it also open the Run panel with test results.

It looks like profiling fails on the Pixel 6 running Android 13, but succeeds on a Samsung Galaxy running Android 12. Should > we add a note to the docs that Android 13 isn't supported yet?

Yes, good idea. Updated.

Does the Gradle task :ui-benchmark-test:connectedZcashtestnetBenchmarkAndroidTest actually install the release APK? I had to install it manually to get the Gradle task to run.

@ccjernigan Update to this question and my previous reply to this question. You've probably seen this error message: Unable to find target package co.electriccoin.zcash, is it installed? It's because you've probably tried to run the new BasicStartupBenchmark#startup test with debug variant of the new benchmark module, thus there wasn't any app with package co.electriccoin.zcash installed, but the debug version of it with a package name suffixed with .debug. Could this be the case?
The debug variant is provided only for a compatibility reason with other modules. I'm trying to explain it in documentation. Should it be more specific, or should we do anything else for it? Thank you.

That was the error message I saw, yes.  I can't remember if it was trying to run the debug version but I seem to recall doing a copy-paste of the Gradle task from the docs when I tried invoking it which should have been the correct task.  It is possible I made an error somewhere though.


debugManageHookOn doesn't turn off

(this is basically a reminder for myself)
Problem Description
I used X.H.ManageDebug to debug an issue with a VirtualBox window going into a tight loop on startup (which I still haven't solved, but it appears to have fixed itself in the meantime, possibly because it's some first-run greeting window). Afterward I noticed that the logHook portion continues to report on every invocation of the logHook instead of shutting itself off after the first run.
Steps to Reproduce

Configure your xmonad.hs with debugManageHookOn
Trigger it with the defined key sequence
Note that it continues to report after the window appears, every time the logHook runs

Configuration File
Please include the smallest full configuration file that reproduces
the problem you are experiencing:
module Main (main) where

import XMonad
import XMonad.Hooks.ManageDebug

main :: IO ()
main = xmonad $ debugManageHookOn "M-S-d" def

Checklist

[x] I've read CONTRIBUTING.md

I tested my configuration

[x] With xmonad version <IP_ADDRESS> (commit 831ca49 if using git)
[x] With xmonad-contrib version <IP_ADDRESS> (commit 923c5a2 if using git)

It's only the logHook that reports repeatedly… but it doesn't start until debugManageHookOn has been triggered, so it's not simply ignoring the flag in XS. Nor does the manageHook part continue to report, so it's clearly turning the XS flag off. This seems odd.

Okay, code answered that. It's two flags instead of one, controlling respectively the logHook and manageHook. So now I have to remember why I did it that way; in particular, it may manage multiple windows before the logHook and can't know which is wanted, so it should log both. I think I simplify this to a single Bool which is enabled in the key event and disabled in the logHook.
…Oh wait: separate is because of the always-ManageDebug case, right.
Oddly, it looks like the code does the right thing (checking and disabling the logHook debugging flag.
It's also the definition of Boolean blindness, so I may rewrite just to make things more obvious. That might even fix the bug.

Also the note above about multiple managed windows means the logHook should disable both flags instead of the manageHook disabling its flag.

For the record, the flags are in order the logHook trigger (always activated by the manageHook) and the key trigger for the triggered manageHook (ignored if the unconditional manageHook is used).

Well, that seems to have fixed it, so I presume I had some confusion about which Bool was which somewhere.


bad behavior when toggling a class

Hello,
There is a bug in the component : when we toggle some classes in className prop, it makes the component not scrollable and scrolled to { top: 0; left: 0 }. I found two hacks to correct :

Change the key prop of the component every time you change a className. Problem: if you have transitions applied to the component, they won't apply. Then the second hack is better
Don't change the styles with className but change with style props, then everything is alright.

I couldn't reproduce the bug in CodePen because I don't know how to add the library to a codepen, but if someone has a clue for me to

My bad, I try to reproduce a simple example starting from create-react-app but the bug is not here anymore... maybe it is coming from my code then. I close the issue, I will re-open if my investigation can reproduce the bug.
import React, { Component } from 'react';
import './App.css';
import PerfectScrollbar from 'react-perfect-scrollbar';
import { withStyles } from '@material-ui/core/styles';
import classNames from 'classnames';
import 'react-perfect-scrollbar/dist/css/styles.css';

const styles = {
  app: {
    border: '3px solid blue',
    width: '100vw',
    height: '100vh',
    display: 'flex',
    flexDirection: 'column',
    // alignItems: 'center',
    boxSizing: 'border-box',
    '& *': {
      boxSizing: 'border-box',
    }
  },
  perfectScrollBar: {
    flex: 1,
    border: '3px solid green',
    overflow: 'hidden',
    transition: 'all 300ms ease-in-out',
    marginLeft: 100,
    marginRight: 100,
  },
  button: {
    margin: 10,
    maxWidth: 250
  },
  withClass: {
    border: '3px solid red',
    marginLeft: 0,
    marginRight: 0,
  },
  insideDiv: {
    width: '200vw',
    height: '200vh',
    border: '3px solid black',
    backgroundRepeat: 'no-repeat',
    backgroundSize: 'cover',
    backgroundPosition: 'center center',
    backgroundImage: 'url(https://picsum.photos/1000/1000)'
  }
}

class App extends Component {
  state = {
    withClass: false
  }
  render() {
    const { classes } = this.props;
    const { withClass } = this.state;
    return (

        <button
          type="button"
          className={classes.button}
          onClick={() => this.setState({ withClass: !withClass })}
        >
          Click to make PerfectScrollBar bug
        </button>
        <PerfectScrollbar
          className={classNames(
            classes.perfectScrollBar,
            withClass && classes.withClass
          )}
          option={{
            handlers: ['click-rail', 'drag-thumb', 'wheel', 'touch'],
            wheelSpeed: 0.8,
            wheelPropagation: false,
          }}
        >

        </PerfectScrollbar>

    );
  }
}

export default withStyles(styles)(App);


incorrect behaviour + memory leak fix

I found that if we call startAnimation more then once when the view has 0 size we will add 2 PreDrawListeners.
And everything was ok except the case when I cancel shimmer animation before the view size change. In this case - animation will start anyway because stopShimmerAnimation will remove only latest PreDrawListener. In this case animation will start anyway.
The second thing - if I navigate back too quick I've noticed that I have a warning about memory leak from LeakCanary. And during the debug found that there can be a case when this PreDrawListener will fire without cancelling and animation will run even if the view is detached.
This PR should fix https://github.com/team-supercharge/ShimmerLayout/issues/75

well, above check shows error not for the PR content but for setup and license acceptance.

hey @veghtomi
can you check this PR?

is this project live? any feedback here?
@veghtomi

hey @szugyi as I can see you are the last guy who committed to this repository, so I'm going to ping you here.
I have a deadline and if no one answer I will have to clone the repo to the project and include source code to fix this issue.

Sorry for the late reply.
This project is pretty much deprecated, as Facebook has improved their implementation of the Shimmering effect quite a lot since this alternative has been published. I would suggest migrating to that library.
https://github.com/facebook/shimmer-android

Thanks for the answer here @szugyi
The bad thing for me now that we have this dependency too https://github.com/ethanhua/Skeleton and this library is based on yours.
For now I seems that your solution works pretty good for now - the only issue is this bug.
I will think about migration but anyway will you have a chance to review, merge and publish this PR or you don't have time for that now?
Regards,
Mikhail

Sorry, but I cannot release new versions. The best option is to make the change you need and use it as source, or release the fix in your own nexus repository.


Check if transform is ok

first mentioned in https://github.com/cogeotiff/rio-cogeo/issues/164
I'm seeing some weird Internal Tile fetching behavior. When working with a Web Mercator aligned file I would expect GDAL to only do 2 GET requests (header + tile) but It seems that we are fetching more than 1 tile.
$ tilebench profile https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif --tile 5-10-9 --add-kernels | jq
{
  "LIST": {
    "count": 0
  },
  "HEAD": {
    "count": 1
  },
  "GET": {
    "count": 3,
    "bytes": 17144,
    "ranges": [
      "0-16383",
      "205900-206279",
      "242386-242765"
    ]
  },
  "WarpKernels": [
    [
      "Src=512,2303,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2431,257x129",
      "Dst=0,128,256x128"
    ]
  ],
  "Timing": 0.4853549003601074
}

☝ we do 3 GET,
[
      "0-16383",  <-- GeoTIFF Header
      "205900-206279",  <--- Tile Data
      "242386-242765".  <--- Tile Data
]

The weird part is when we look at the WarpKernels:
[
    [
      "Src=512,2303,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2431,257x129",
      "Dst=0,128,256x128"
    ]
]

I'm not sure how to read the Kernels but the 257x129 seems weird.
For the particular tile 5-10-9, here are the WarpedVRT info
{
    'add_alpha': True,
    'resampling': <Resampling.nearest: 0>,
    'crs': CRS.from_epsg(3857),
    'transform': Affine(2445.984905125635, 0.0, -7514065.6285459455, 0.0, -2445.984905125635, 8766409.89997027),
    'width': 512,
    'height': 512
}
Window(col_off=0, row_off=0, width=512, height=512)
(3, 256, 256)

The VRT is 512x512 for a 256x256 tile, which is what we expect because the zoom 5 correspond to the first overview.
The input and output size are ok, so maybe we need to check if the transform is aligned with the internal tiles.
https://github.com/cogeotiff/rio-tiler/blob/4bd9be5db98827e40d4222627c3af59e9063bd95/rio_tiler/utils.py#L166-L182

What versions are you using?

@geospatial-jeff version of ?

Probably GDAL?

GDAL 3.2, rasterio 1.2.0

Taking a look at this with aiocogeo, those two particular GET requests are the 130th and 146th tile offset in the IFD.  The first overview is 16x16 tiles (8192 / 512), which means the additional tile is one row down (5/10/10).  You can see the same behavior with other requests (ignore the HEAD requests).
$ tilebench profile https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif --tile 5-10-10 --add-kernels | jq

{
  "LIST": {
    "count": 1
  },
  "HEAD": {
    "count": 8
  },
  "GET": {
    "count": 11,
    "bytes": 17144,
    "ranges": [
      "0-16383",
      "242386-242765",
      "270360-270739"
    ]
  },
  "WarpKernels": [
    [
      "Src=512,2559,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2687,257x129",
      "Dst=0,128,256x128"
    ]
  ],
  "Timing": 4.101164102554321
}

Same story but with the 146th and 162nd offset.  I think this is probably a rounding issue when the VRT transform, not sure if its because how rio-tiler is creating the transform, how GDAL is interpreting/applying it, or because of the file itself.
Given that the origin of the image is top left, and GDAL uses the length of the diagonal to determine file size on warping operations, we would expect to see rounding errors manifest in the bottom/right of the image which maybe explains why we consistently request an additional tile one row below the intended tile.  But we also aren't really defining any warp operations here because the VRT aligns with an internal block so there really isn't any warping to be done.
Maybe it is the file.  I'm pretty confident that aiocogeo can do web-aligned reads in one request if the image is aligned properly:
import asyncio
import json

import mercantile
from aiocogeo.cog import COGReader

async def main():
    async with COGReader("https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif") as cog:
        tile = mercantile.Tile(10, 9, 5)
        bounds = mercantile.xy_bounds(tile)
        await cog.read(bounds=bounds, shape=(256, 256))
        print(json.dumps(cog.requests))

asyncio.run(main())

>>> {"count": 5, "byte_count": 17105, "ranges": [[0, 16384], [242390, 242569], [270364, 270543], [270552, 270731], [242578, 242757]], "header_size": 16385}

FYI I'm getting the same result with a file created using GDAL COG driver
$ tilebench profile https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256px_COG.tif  --tile 5-10-9 --add-kernels | jq
{
  "LIST": {
    "count": 0
  },
  "HEAD": {
    "count": 1
  },
  "GET": {
    "count": 4,
    "bytes": 81920,
    "ranges": [
      "0-16383",
      "16384-49151",
      "376832-393215",
      "425984-442367"
    ]
  },
  "WarpKernels": [
    [
      "Src=512,2304,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2432,257x129",
      "Dst=0,128,256x128"
    ]
  ],
  "Timing": 0.7137670516967773
}

Alright I think I've find something. I totally forget but GDAL GDALWarpOptions has SOURCE_EXTRA (set to 1 by default).

SOURCE_EXTRA: This is a number of extra pixels added around the source window for a given request, and by default it is 1 to take care of rounding error. Setting this larger will increase the amount of data that needs to be read, but can avoid missing source data.

ref: https://gdal.org/api/gdalwarp_cpp.html?highlight=vrt#_CPPv415GDALWarpOptions
BUT
when overwriding source_extra=0 (using vrt_options in .tile()) this has no impact, and looking at the WarpKernels, I'll tend to conclude that SOURCE_EXTRA is already set to 0
# default
[
    [
      "Src=512,2303,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2431,257x129",
      "Dst=0,128,256x128"
    ]
]

# source_extra=0
[
    [
      "Src=512,2303,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2431,257x129",
      "Dst=0,128,256x128"
    ]
]

# source_extra=1
[
    [
      "Src=511,2302,259x131",
      "Dst=0,0,256x128"
    ],
    [
      "Src=511,2430,259x131",
      "Dst=0,128,256x128"
    ]
]

#source_extra=-1
[
    [
      "Src=513,2304,255x127",
      "Dst=0,0,256x128"
    ],
    [
      "Src=513,2432,255x127",
      "Dst=0,128,256x128"
    ]
]

Trying to replicate rio-tiler command with gdalwarp:
$ bash-4.2# gdalwarp -overwrite -q \
>     /vsicurl/https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif \
>     out.tif \
>     -te -7514065.6285459455 7514065.6285459455 -6261721.35712162 8766409.89997027 \
>     -t_srs EPSG:3857 \
>     -ts 256 256 \
>     -r near \
>     -ovr 0 \
>     --config CPL_DEBUG ON --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR

HTTP: libcurl/7.73.0 OpenSSL/1.0.2k-fips zlib/1.2.7 nghttp2/1.42.0
VSICURL: GetFileSize(https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif)=1349486  response_code=200
VSICURL: Downloading 0-16383 (https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif)...
VSICURL: Got response_code=206
GDAL: GDALOpen(/vsicurl/https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif, this=0xf763a0)
GDAL: Using GTiff driver
GDAL: Computing area of interest: -90, -2.67167e-13, 90, 85.0511
...
GDAL: GDALDriver::Create(GTiff,out.tif,256,256,3,Byte,0xf77060)
...
WARP: Selecting overview level 0 for /vsicurl/https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif
...
GDAL: Computing area of interest: -90, -2.67167e-13, 90, 85.0511
VSICURL: Downloading<PHONE_NUMBER>79 (https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif)...
VSICURL: Downloading<PHONE_NUMBER>65 (https://rio-tiler-dev.s3.amazonaws.com/data/eu_webAligned_256pxWEBP.tif)...
...
GDAL: GDALWarpKernel()::GWKNearestNoMasksOrDstDensityOnlyByte() Src=512,2303,257x257 Dst=0,0,256x256
...

Notes:

for an unknown reason I had to force using overview=0 to get the same blocks

Note on the WarpKernels
# rasterio
[
    [
      "Src=512,2303,257x129",
      "Dst=0,0,256x128"
    ],
    [
      "Src=512,2431,257x129",
      "Dst=0,128,256x128"
    ]
]

# gdal
Src=512,2303,257x257 Dst=0,0,256x256

They are both the same, except in rasterio we split the block in 2 (2 x 128).
if we decode the kernels
x = 512
y = 2303 ! <--- it should be 2304 (9x256)
so here we have a difference in the Y coordinates.

Stil trying to get a sense of what is going on: https://gist.github.com/vincentsarago/1bc5b2f8e884594f640a109589a271af
I'm pretty sure it all comes down to precision issues. GDAL and Mercantile don't use the same TMS definition (origin/res are different):
GDAL:        20037508.342789244
Morecantile: 20037508.3427892

And thus the tile bounds and internal block bounds don't align perfectly
# internal block (x=3, y=9, z=1)
COG:                [-7514065.628545967, 7514065.628545967, -6261721.357121639, 8766409.899970295]
rio-COG:            [-7514065.628545957, 7514065.628545931, -6261721.35712163, 8766409.899970258]
# tms tile (x=10, y=9, z=5)
morecantile bouds:  [-7514065.6285459455, 7514065.6285459455, -6261721.35712162, 8766409.89997027]
mercantile bouds:   [-7514065.628545966, 7514065.628545967, -6261721.357121638, 8766409.899970295]

diff between mercantile and GDAL block
[-9.313225746154785e-10, 0.0, -9.313225746154785e-10, 0.0]

diff between morecantile and GDAL block
[-2.1420419216156006e-08, 2.1420419216156006e-08, -1.862645149230957e-08, 2.421438694000244e-08]

diff between morecantile and rio-cogeo block
[-1.1175870895385742e-08, -1.4901161193847656e-08, -9.313225746154785e-09, -1.30385160446167e-08]

as mentioned in #345 the precision errors in the bounds results in errors in the windows which then makes GDAL/Rasterio query more blocks than needed


Fix typo handling deleted nodes on aftermath

Added test that reproduces this use case, using an if_node_exists condition on keep_while that must be exercised

Pull Request Test Coverage Report for Build 543

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.07%) to 87.212%

Totals

Change from base Build 535:
-0.07%

Covered Lines:
1814

Relevant Lines:
2080

💛  - Coveralls


[POLISH][Simplified Collect] Create utils files for each page

Policy.ts is a large file and it'll only get larger and larger. We can keep the code more organized by separating each individual Members, Categories, Tags and Distance Rates into its own utils file, like we did for taxes here, i.e. /actions/Policy/TaxRates.ts, etc

Started on this today, I'll split it up into a few PRs to make review a little easier.
Looking at Category to start, 1 clarifying question: Policy.ts contains category-related functions that are used by the Simplified Collect Category page (like openPolicyCategoriesPage), as well as others used when requesting money (like buildOptimisticPolicyRecentlyUsedCategories). I was going to move them all, but wanted to check that matches what you were thinking @luacmartins?

Yea, that makes sense.

No update - haven't had much time to swing back around to this between some OOO and other commitments the past few days but will prioritize shortly

Have been prioritizing a fast-apis issue and detailed design for the Billing card project. If anyone's interested in knocking this out feel free to take it from me, otherwise I'll work on it once I hit a lull in other work

Discussed with @amyevans, I can take this over! It may still be a bit slow since I'm only in a couple days each week but hopefully will get a good chunk of it going today and Thursday. I like the idea of splitting up the PRs, so I'll aim to get at least one out this week.

not overdue, PRs are in progress. Waiting for merge slush to be less, er, slushy. Since this SHOULDN'T post any problems but a little scary while we're taking a harder look at merges.

Do we think we should include the Owner functions in Member.ts? Kind of like a member...

I think so, they are all members of the policy, but with different access levels.

I think once these are all on prod we should be good to close this, unless you can think of some other files we might want to break this up into? I don't see any nice groupings in the remaining functions really.
Oh and let me know what y'all think about https://github.com/Expensify/App/pull/42305#issuecomment-2136157363 too.

Did the migration solve the issue?

Did the migration solve the issue?

Unfortunately not. It's three functions that are causing the issue:

removePendingFieldsFromCustomUnit defined in Policy.ts, used in Category.ts and DistanceRate.ts
navigateWhenEnableFeature defined in Policy.ts, used in Policy.ts, Tag.ts, Category.ts, and DistanceRate.ts
buildOptimisticPolicyCategories currently defined in BOTH Policy.ts and Category.ts, used in Policy.ts and Category.ts

Ideally we'd keep the first two where they are (in Policy.ts) and move the last one to ONLY Category.ts but that's posing the problem.
The last one makes the most sense to move or duplicate, since the top two are shared in multiple files (though the top two are also much shorter).
So maybe we leave it? Unless there's an additional file we could create with those and perhaps other utility methods or something like that.

Could we move the first 2 to PolicyUtils? For the buildOptimisticPolicyCategories, how does it create a circular dependency? I don't see Policy.ts importing Category.ts and vice-versa, so could we define it in Category.ts and then import Category in Policy.ts?

Can't import category into policy (which it's not doing now) because category is already importing policy.
But moving those two into PolicyUtils should work, I can spin that up.

Can't import category into policy (which it's not doing now) because category is already importing policy.

Ah I missed the Policy import on Category the first time since it's importing specific functions. But the only imports from Policy are the 2 functions we're moving to PolicyUtils, no? So that means we can remove that Policy import in Category

yep! that PR above should solve our problems

No payment needed here but I'll close this on the 24th. Not expecting any regressions (though when ARE we expecting them?)

@dangrous I think we can close this issue since it's deployed to prod and was internal. We'll know if there's a regression haha


Why  generated  subquery  and use DruidOuterQueryRel

how to write this with json context
SELECT "Time", sum("<3"),sum("3~5"),sum("6~15")
from (
    SELECT
    "Time",
     case WHEN "Count1" < 3 then 1 ELSE 0  end as "<3",
     case WHEN "Count1" >= 3 and "Count1"<= 5 then 2  ELSE 0  end as "3~5",
     case WHEN "Count1" >= 6 and "Count1"<= 15 then 3  ELSE 0  end as "6~15"
    from (
          SELECT
            TIME_FLOOR("__time", 'PT5M') AS "Time", "uuid", COUNT(*) AS "Count1"
          FROM "app_time"
          WHERE "__time" >= CURRENT_TIMESTAMP - INTERVAL '1' HOUR AND "__time" <= CURRENT_TIMESTAMP
          GROUP BY 1, 2
          ORDER BY "Time" ASC ))
GROUP BY 1
ORDER BY "Time"

When i use EXPLAIN PLAN FOR to see plan , I saw
DruidOuterQueryRel(query=[{"queryType":"groupBy","dataSource":{"type":"table","name":"__subquery__"}.......

But I don't have this data source，How can I do

Hm, I don't think the nested groupBy is returning the final plan when using EXPLAIN PLAN FOR, "subquery" there is a dummy value.
If you need to run nested queries, you would want to use GroupBy with a Query datasource (https://druid.apache.org/docs/latest/querying/datasource.html) where you specify another GroupBy as the subquery


fix odd input size issue

For some input images, the 'x.shape // 8' creates a different size than the tensor that is created by the network itself.
I believe it also solves the closed issue #13

Hi, have you tested it? Does it impact any performance of the model?


Remove Databox dashboard from Slackbot message

This makes Slackbot's weekly stats message slightly more concise. Since this databox link is always the last link, it gets expanded into a preview. I think it's not very useful since it's only for CesiumJS (it requires payment to aggregate multiple sources I think, plus we don't have as fine grained control over it as Slackbot's stats message).
This will help because I think the next step is making Concierge watch more repo's, so it can be reporting how many PRs we've merged across all repo's.

Thanks for the pull request @OmarShehata!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

:sparkles: :sparkles: :sparkles:I'm the Cesium Concierge and I'm here to test things. :sparkles: :sparkles: :sparkles:

@ggetz this should be good to merge.

Merging this since it's a simple text change.


GitHub Action to lint Python code

These tests will automate the discovery of issues like #17 and #19.
Output https://github.com/cclauss/Games/actions

https://flake8.pycqa.org/en/latest/user/error-codes.html
On the flake8 test selection, this PR does not focus on "style violations" (the majority of flake8 error codes that psf/black can autocorrect).  Instead these tests are focus on runtime safety and correctness:

E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST).  Often these issues are a sign of unused code or code that has not been ported to Python 3.  These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
F63 tests are usually about the confusion between identity and equality in Python.  Use ==/!= to compare str, bytes, and int literals is the classic case.  These are areas where a == b is True but a is b is False (or vice versa).  Python >= 3.8 will raise SyntaxWarnings on these instances.
F7 tests logic errors and syntax errors in type hints
F82 tests are almost always undefined names which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3.  These also would be compile-time errors in a compiled language but in Python a NameError is raised which will halt/crash the script on the user.


UI enhancements for poll creation page

Updating the poll creation UI as per latest hi-fi prototype
1- individual question removal
2- hide/show vote count
3- add containers for questions
4- Button colors

Close as PR merged.


Rename from *-test.ts to *-tests.ts

Rename files from *-test.ts to *-tests.ts + remove trailing spaces for them
Would be nice to have npm test to check for naming conventions (test.ts vs tests.ts) + trailing spaces
Generalizes PR #6413

wow, thanks mate! :+1: