Add public API to get CUDA version and path, CUDNN version and path.

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using): TF 2.2.0rc2
Are you willing to contribute it (Yes/No): no

Describe the feature and the current behavior/state.
Follow-up of the SIG build meeting of the 07/04/2020.
Currently TensorFlow is compiled against a specific CUDA and CUDNN version. When people want to build TensorFlow Addons from source, they have to specify manually the CUDA and CUDNN paths and version to use: https://github.com/tensorflow/addons/blob/master/configure.py#L159
But it should not be needed to ask this information to the user. Because the user have to use the exact CUDA and CUDNN versions that TF was compiled against. Also TF is able to detect where are the CUDA and CUDNN directories on the user filesystem. If we could get this information out of TensorFlow, that would make the building of custom ops easier. We could also do some version checking at runtime and improve the user experience when there is a version mismatch.
Will this change the current api? How?
We would need 4 new functions/attributes (the names are examples or course):

tf.config.get_cuda_version_used_to_compile_tf()
tf.config.get_cudnn_version_used_to_compile_tf()
tf.get_cuda_path()
tf.get_cudnn_path()

Who will benefit with this feature?
People writing custom ops, as it makes the configuration easier for compilation, it also allows checking that people using the TF addons binaries have a compatible TF installed as the cuda and cudnn versions have to match.
Any Other info.
I was asked during the sig-build meeting to tag:
@perfinion @gunan @angerson @yifeif
@seanpmorgan as he manages TF Addons

@alextp for the proposed new APIs.

SGTM but I'd like the cuda_path and cudnn_path ones to also be in tf.config.
Maybe we should just have a tf.config.cuda_information() which returns a namedtuple of the information we want (versions, paths, and whatever else we need in the future)?

I don't have any preference regarding how it's named and under which namespace it's available.

I like the idea of having these under a separate namespace.
The cuda and cudnn paths are read from LD_LIBRARY_PATH.
So it is not known to us at build time. And we rely on OS shared object resolution, we just do dlopen(libcuda... Do you want us to replicate the OS so resolution in TF?

So it is not known to us at build time.

Yes of course, since the path depends on the user system. I'm asking the path found at runtime. Since TF already knows at runtime where is the CUDA and CUDNN path, we'd like to grab that.

We actually do not know the paths. We simply load libcuda.so or libcufft.so.
The logic is here:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/stream_executor/platform/default/dso_loader.cc#L42
The operating system checks LD_LIBRARY_PATH, and loads whatever it can. So it is all on the operating system.

Maybe we can extract the information using dlinfo:
http://man7.org/linux/man-pages/man3/dlinfo.3.html

Thanks @gunan for the explanation! We'll implement that then. We just need the two
tf.config.get_cuda_version_used_to_compile_tf()
tf.config.get_cudnn_version_used_to_compile_tf()
then, or equivalent.

Agreed, let's definitely implement cuda/cudnn version queries.
They have no blockers. Then we can take our time to see if dlinfo tells us the path to the shared objects. If not, we can discuss about the paths again.

TensorFlow also has this mechanism:
from tensorflow.python.platform import build_info

Auto-generated during build time via https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/tools/build_info/gen_build_info.py and tf_py_build_info_genrule in https://osscs.corp.google.com/tensorflow/tensorflow/+/master:tensorflow/tensorflow.bzl. Right now it contains very little, but perhaps we could extend it with the information from https://cs.opensource.google/tensorflow/tensorflow/+/master:third_party/gpus/find_cuda_config.py that's used in configure.py?

+1 to extending build info over new apis
On Wed, Apr 8, 2020, 16:05 Austin Anderson<EMAIL_ADDRESS>wrote:

TensorFlow also has this mechanism:
from tensorflow.python.platform import build_info
Auto-generated during build time via
https://cs.opensource.google/tensorflow/tensorflow/+/master:tensorflow/tools/build_info/gen_build_info.py
and tf_py_build_info_genrule in
https://osscs.corp.google.com/tensorflow/tensorflow/+/master:tensorflow/tensorflow.bzl.
Right now it contains very little, but perhaps we could extend it with the
information from
https://cs.opensource.google/tensorflow/tensorflow/+/master:third_party/gpus/find_cuda_config.py
that's used in configure.py?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/38351#issuecomment-611238429,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAABHRJPOXKLNPNANK6YVSLRLT7LPANCNFSM4MDZ3HQA
.

https://www.tensorflow.org/api_docs/python/tf/sysconfig I've never heard its not public? Its been in the docs for ages and theres no indication its not to be relied on? There isnt much in it so its likely overlooked most of the time but its definitely listed publicly.
I know for sure custom-op uses it to make the build rules so it cant be removed either.

Thanks!

#38964 was reverted due to brittleness -- @r4nt gave some advice on improving it, so I'll work on re-enabling the functionality this week.

It took longer than I thought, but I have an internal change prepared for review that adds the same functionality in a different way.

@angerson It seems that building tf-nightly now requires opt_einsum in host environment:
Mon Jun 8 13:08:20 UTC 2020 : === Building wheel
2020-06-08 13:08:21.249561: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
  File "setup.py", line 47, in <module>
    from tensorflow.python.platform import build_info
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/__init__.py", line 47, in <module>
    from tensorflow.python import keras
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/keras/__init__.py", line 27, in <module>
    from tensorflow.python.keras import models
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/keras/models.py", line 23, in <module>
    from tensorflow.python.keras import backend as K
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/keras/backend.py", line 37, in <module>
    from tensorflow.python.distribute import distribute_coordinator as dc
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/distribute/distribute_coordinator.py", line 33, in <module>
    from tensorflow.python.training import monitored_session
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/training/monitored_session.py", line 45, in <module>
    from tensorflow.python.training.tracking import graph_view
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/training/tracking/graph_view.py", line 27, in <module>
    from tensorflow.python.training import optimizer as optimizer_v1
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/training/optimizer.py", line 36, in <module>
    from tensorflow.python.ops import gradients
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/ops/gradients.py", line 27, in <module>
    from tensorflow.python.ops.gradients_impl import gradients
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/ops/gradients_impl.py", line 30, in <module>
    from tensorflow.python.ops import linalg_grad  # pylint: disable=unused-import
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/ops/linalg_grad.py", line 47, in <module>
    from tensorflow.python.ops.linalg import linalg_impl as _linalg
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/ops/linalg/linalg_impl.py", line 34, in <module>
    from tensorflow.python.ops import special_math_ops
  File "/tmp/tmp.7oZofsqRJ1/tensorflow/python/ops/special_math_ops.py", line 30, in <module>
    import opt_einsum
ImportError: No module named 'opt_einsum'

which seems to break tf-nightly on Linux since your CL landed; not sure if it's going to be fixed in TF or the build environment.

That is an unrelated problem. Could you open a new issue?

@gunan Just found out that it's been fixed by 67099d08529a00f48ee8a444f471067bedcc2f5b. So no need to open a new issue :)

Thanks for the fix, @gunan!
@gabrieldemarmiesse tf.sysconfig.get_build_info() is working in the latest tf-nightly (thanks to follow-up fixes from @chsigg and @gunan). Can you check it out?

Thanks everyone, I'll take a look :)

@angerson this is awesome! Thanks.
>>> import tensorflow as tf
W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:/opt/rh/devtoolset-7/root/usr/lib64/dyninst:/opt/rh/devtoolset-7/root/usr/lib/dyninst:/opt/rh/devtoolset-7/root/usr/lib64:/opt/rh/devtoolset-7/root/usr/lib:/opt/rh/rh-python36/root/usr/lib64:/opt/rh/devtoolset-8/root/usr/lib64:/opt/rh/devtoolset-8/root/usr/lib:/opt/rh/devtoolset-8/root/usr/lib64/dyninst:/opt/rh/devtoolset-8/root/usr/lib/dyninst:/usr/local/lib64:/usr/local/lib
I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
>>>
>>> tf.__version__
'2.4.0-dev20200706'
>>>
>>> tf.sysconfig.get_build_info()
{'cuda_version': '10.1', 'cudnn_version': '7', 'cuda_compute_capabilities': ['sm_35', 'sm_37', 'sm_52', 'sm_60', 'sm_61', 'compute_70'], 'cpu_compiler': '/usr/bin/gcc-5', 'is_rocm_build': False, 'is_cuda_build': True}


Is it possible to limit the light perception range?

I love your module, but it locks token light perception range to infinite which breaks my game. Is there any way to circumvent that or would is be possible to add a "maximum light perception range" setting?

Closing. If it isn't working as I described in my previous comment, let me know.


Feature Selection for Machine Learning with Genetic Algorithm

Descriptions:
Implementing a Genetic Algorithm for feature selection.
Comparing the performance of models trained with all features versus the selected features.
Analyzing the results to determine the effectiveness of the Genetic Algorithm in feature selection.
Steps:

Initialization: Generate an initial population of feature subsets.
Selection: Evaluate the fitness of each subset using a predefined fitness function (e.g., model accuracy).
Crossover: Combine pairs of feature subsets to produce new offspring.
Mutation: Introduce random changes to feature subsets to maintain genetic diversity.
Replacement: Replace less fit subsets with new offspring.
Termination: Stop the algorithm after a set number of generations or if convergence criteria are

@invigorzz313 pls assign it to me I'm Gssoc'24 contributor

@ashis2004 the issue is assigned.


HTTPS example fails on Travis CI with 'python pyOpenSSL module is required'

I think something in the ubuntu1604 container must've changed recently, because up until a couple days ago, everything worked fine:
TASK [Generate an OpenSSL private key.] ****************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "the python pyOpenSSL module is required"}

When testing with Vagrant and the 16.04 base box I maintain, the current playbook works fine, it's only under the Docker container things break.
My best guess is that maybe the system is suddenly defaulting to pip3 instead of pip or something like that, causing the install to fail because things are set up under pip but not pip3?

Could be related to https://github.com/geerlingguy/ansible-role-pip/pull/15

Ah, this task in the Travis CI playbook (not in the main playbook) could've been confusing things:
    - name: Ensure the older Python 2 pip is available.
      package: name=python-pip state=present

Still failing: https://travis-ci.org/geerlingguy/ansible-for-devops/jobs/427246290#L663
Will have to take a look later.

So I'm pretty sure it had to do with some issue with the way Ansible was installed on the Docker container used in those tests. Tests are passing now, after I spent some time revamping all my Docker test environment containers to use a pip-based Ansible installation. Yay!


program: remove unnecessary activation of serde feature of curve25519-dalek

Both solana-program and solana-zk-sdk currently activate the serde feature of curve25519-dalek unnecessarily.
solana-program  currently activates the serde feature of curve25519-dalek unnecessarily.
Fixing this improves compile time by allowing curve25519-dalek to be built earlier in the pipeline

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.6%. Comparing base (af6930d) to head (e56e1a4).
Report is 5 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1327   +/-   ##
=======================================
  Coverage    81.6%    81.6%
=======================================
  Files         867      867
  Lines      368900   368900
=======================================
+ Hits       301052   301058    +6
+ Misses      67848    67842    -6


Support matching on constants in pattern matching

We should support matching on constants directly in pattern matching.
For example:
contract;

use std::{address::Address, chain::auth::{AuthError, Sender, msg_sender}, result::*};

abi MyContract {
    fn test_function() -> bool;
}

const ORACLE_1: b256 = 0x6b63804cfbf9856e68e5b6e7aef238dc8311ec55bec04df774003a2c96e0418e;
const ORACLE_2: b256 = 0x54944e5b8189827e470e5a8bacfc6c3667397dc4e1eef7ef3519d16d6d6c6610;
const ORACLE_3: b256 = 0xe10f526b192593793b7a1559a391445faba82a1d669e3eb2dcd17f9c121b24b1;

impl MyContract for Contract {
    fn test_function() -> bool {
        let sender: Result<Sender, AuthError> = msg_sender();
        let is_one_of_signers = match sender.unwrap() {
            Sender::Address(address) => {
                match address.value {
                    ORACLE_1 => true,
		    ORACLE_2 => true,
                    ORACLE_3 => true,
                    _ => false,
                }
            },
            _ => {
                false
            },
        };
        is_one_of_signers
    }
}

inspired by: https://github.com/FuelLabs/sway/issues/1765

Some notes for the future:
At parse time, constants on the LHS of a match arm (i.e. ORACLE_1 => true) could be parsed to a Scrutinee::Variable { name: "ORACLE_1" }. Each constant will need to be converted into one of the literal Pattern's:
https://github.com/FuelLabs/sway/blob/057e83aee87d17c3812f223f5fc7884cf6d3468a/sway-core/src/semantic_analysis/ast_node/expression/match_expression/analysis/pattern.rs#L101
using this method:
https://github.com/FuelLabs/sway/blob/057e83aee87d17c3812f223f5fc7884cf6d3468a/sway-core/src/semantic_analysis/ast_node/expression/match_expression/analysis/pattern.rs#L125
with a lookup into the namespace to grab the value of the constant:
https://github.com/FuelLabs/sway/blob/e57de1b4b90f6626f69b7f972e9cef28fcceba9a/sway-core/src/semantic_analysis/ast_node/declaration.rs#L439
We will have to do a special check on the value to ensure that it really is a TypedExpression::Literal, but in the case that it is not, we can return a compiler error telling the user that matching on non-literal constants is not yet supported.
Because constants are already in scope in the namespace (as opposed to a case in which you are binding the wildcard case to a new variable), we won't need to create any new variable declarations for them, like we have to do in the aforementioned wildcard case:
https://github.com/FuelLabs/sway/blob/057e83aee87d17c3812f223f5fc7884cf6d3468a/sway-core/src/semantic_analysis/ast_node/expression/match_expression/typed/typed_match_branch.rs#L67
But if, for instance ORACLE_1 isn't in scope, I think the correct thing to do is probably treat it as a wildcard variable, because the alternative is to make decisions based on capitalization, which feels icky. When the collection context goes in #1693, we can add a compiler warning that tells the user that they are using the name of a constant that isn't imported in scope and will be treated like a wildcard variable


Error normalizing log message to internal format Can't convert string from native encoding to 'UTF-8'

Commit the code is fail, commit message has Chinese, how to resolve it?

Can you elaborate on what the issue is? What platform, Code version? What steps do you do? What is the expected result? What is the actual behavior?
Please read our guide about Submitting Bugs and Suggestions.

I have solved it. : )

How did you solve it?


[BUG] Error at startup ComputeConvolution

This is also a known issue, when shutting down, you end up getting exceptions.

I'd rather leave this problem open until it is solved.

I believe that has been fixed.


Fix install for nvm

mean init foo would fail when using NVM because there was no ~/.npm folder.
Fixes #114

I'd like to reassess this.
Can you resubmit on the latest version of the cli? - there are conflicts


Add shrinkWrap to ListView.builder

If nesting scrollables is the problem, you can pass "scroll: false" to the ResponsiveGridList constructor which defaults to true. It builds with rows and columns instead of a ListView.builder if scroll is set to false.
ResponsiveGridList(
     scroll: false,
     ...
)

If nesting scrollables is the problem, you can pass "scroll: false" to the ResponsiveGridList constructor which defaults to true. It builds with rows and columns instead of a ListView.builder if scroll is set to false.
ResponsiveGridList(
     scroll: false,
     ...
)


Manifest V3 Service worker registration failed

Getting error:
"Service worker registration failed" ,
"background": { "service_worker": "background.js" }, is highlighted in the error. When I build for production, the error goes away. I think it has to do with all of the code that this library adds to the background.js file when it is output.
throw new TypeError(bgScriptManifestRequiredMsg.get());
          ^
TypeError: [WER-E2] function(obj) {
obj || (obj = {});
var __t, __p = '';
with (obj) {
__p += 'Background script on manifest is required';

}
return __p
}.

This is from my config file:
new ExtensionReloader({
      manifest: path.resolve(__dirname, 'public/manifest.json'),
      reloadPage: true,
      entries: {
        contentScript: 'injectIcon',
        background: 'background',
      },
})

Any update on this? I am facing the same issue?

The issue happens here: https://github.com/obender/webpack-extension-reloader-v3-manifest/blob/378c0d8bbf3b78d55f00eeac3881b27ede134203/src/utils/manifest.ts#L24
Because background scripts now migrated to service_worker, but I guess the author doesn't support this one anymore.


Columns 70 charactars wide

The commentary says that this will show columns as wide as 90 characters, but the show command shows only 70 characters wide.
df.show(7, 70)

Corrected - thanks


Use OffsetIndex to prune IO with RowSelection

Which issue does this PR close?

Closes #2426.
Rationale for this change

When we have a RowSelection and an OffsetIndex we can reduce IO by fetching only the pages selected.
This also builds on #2464 to remove InMemoryColumnChunk and unify everything to use SerializedPageReader
What changes are included in this PR?

We can represent pre-fetched column chunks as either a "dense" encoding (just Bytes) or a "sparse" encoding which contains only the pages relevant to a given RowSelection.
Also remove InMemoryColumnChunk to help unify the sync and async parquet paths.
Are there any user-facing changes?

@tustvold
Leaving this in draft until #2464 is merged as this includes those changes.

Thank you for this, I'll review first thing tomorrow. I like that you've found a way to allow sharing the page muxing logic in SerializedPageReader 👍

Docs appear to be failing


how to find the noisy label?

is it doable by sorting the target after calculating the elr_reg?
do I have to run it till it converges first?

Yeah, run it till convergence. One way to find it is to use the target in the ELR loss and compare it with the original label (noisy), the labels that are not equal are the possible wrong labels.


Enable the possibility to delete application resources

Is your feature request related to a problem? Please describe.
I'm currently building an extension on base of kubeclarity to automatically update vulnerable resources to a newer version. So after I updated the container images to a newer version, the kubeclarity still tracks the older application resource which isn't used by an application. As a result the application resource overview keeps growing and it's getting more difficult to track the versions currently installed on the kubernetes cluster.
Describe the solution you'd like
A clear and concise description of what you want to happen.
There should be an endpoint, which allows to delete application resources. There might be a rule checking if the application resource is still used by an application, which would result in a 400 bad request error.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
There could also be the possibility, that after a scan, the unused application resources are cleaned up.
Additional context
Add any other context or screenshots about the feature request here.

Hey @fluflo, thanks a lot for the proposal. Indeed a supper useful feature. We can allow the users whether upon an application resource deletion they want to delete its related objects (e.g. un-associate from all applications and delete its related packages and their related vulnerabilities):

Additionally, it possible for your extension to apply a filter "Applications greater than or equal to 1" to disregard the application resources that are detached from an application.
Does the issue affect any other usage of kubeclarity other than your extension (e.g. Dashboard, etc.)?

@fluflo thanks for the clarification. We'll prioritize this to start working on it probably next week.

@fluflo you can find an example of how you can delete an application resource:
https://github.com/openclarity/kubeclarity/pull/434#issuecomment-1566120839
The UI will be updated soon in order to add a delete button for application resources.


About input data

I found that in the MS task, the target column is also within the input range when predicting targets. Will removing the target from the input have a significant impact on the results?

For DLinear and NLinear, they cannot perform MS tasks, since the information in other columns is not used if there is no supervision. The MS for DLinear and NLinear is the same as S, therefore removing the target from the input can have a significant impact.


[BEAM-1036] Support for new State API in streaming mode of FlinkRunner

Sorry, my configuration is a bit wrong again. zhixin.ljs is me.
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[ ] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[ ] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[ ] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

@aljoscha Please take a look.

Changes Unknown when pulling 33942820c3068761c414ba8cb5dc7c604269947a on JingsongLi:master into ** on apache:master**.

Running Flink postcommit here: https://builds.apache.org/view/Beam/job/beam_PostCommit_Java_RunnableOnService_Flink/1497/

I took a look and the code looks good!
@kennknowles The test failed in BeamFnLoggerClientTest, is this a known flaky test right now? (I think it's in Luke's new Fn API proposal package)

The code in question was not green on Travis when it was merged. I haven't tried to track down if it was ever green. You shouldn't block anything on the Fn harness prototype.

@JingsongLi I'll change the commit to refer to your "outside name" (JingsongLi<EMAIL_ADDRESS>right?

@aljoscha Yes.

Merged. Thanks for your work! 👍


Cannot fetch git-lfs data

Running the command 'git lfs fetch' results in the following:
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
Can you please fix this?

I am getting the same error, could you please look into it?
@warcraft12321


[MINOR] Trivial cleanups

These are what I found during working on #22282.

Remove unused value: UnsafeArraySuite#defaultTz
Remove redundant new modifier to the case class, KafkaSourceRDDPartition
Remove unused variables from RDD.scala
Remove trailing space from structured-streaming-kafka-integration.md
Remove redundant parameter from ArrowConvertersSuite: nullable is true by default.
Remove leading empty line: UnsafeRow
Remove trailing empty line: KafkaTestUtils
Remove unthrown exception type: UnsafeMapData
Replace unused declarations: expressions
Remove duplicated default parameter: AnalysisErrorSuite
ObjectExpressionsSuite: remove duplicated parameters, conversions and unused variable

Test build #108145 has started for PR 25251 at commit 600444e.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

add to whitelist

ok to test

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13249/
Test PASSed.

Test build #108147 has started for PR 25251 at commit 600444e.

At the same time, can you check unused imports by using IDE supports?

@maropu I will have a try. Let me see.

Thanks!

Test build #108147 has finished for PR 25251 at commit 600444e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108147/
Test FAILed.

Test build #108145 has finished for PR 25251 at commit 600444e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108145/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13292/
Test PASSed.

Test build #108191 has started for PR 25251 at commit 5cbeaf0.

Test build #108191 has finished for PR 25251 at commit 5cbeaf0.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108191/
Test FAILed.

retest this please.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13301/
Test PASSed.

Test build #108200 has started for PR 25251 at commit 5cbeaf0.

Test build #108200 has finished for PR 25251 at commit 5cbeaf0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108200/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13320/
Test PASSed.

Test build #108220 has started for PR 25251 at commit 942b007.

Test build #108220 has finished for PR 25251 at commit 942b007.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108220/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13366/
Test PASSed.

Test build #108264 has started for PR 25251 at commit 8a97b05.

Test build #108264 has finished for PR 25251 at commit 8a97b05.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108264/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13375/
Test PASSed.

Test build #108275 has started for PR 25251 at commit 2ff50a8.

Test build #108275 has finished for PR 25251 at commit 2ff50a8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108275/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13415/
Test PASSed.

Test build #108318 has started for PR 25251 at commit 32612bd.

Test build #108318 has finished for PR 25251 at commit 32612bd.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108318/
Test FAILed.

retest this please

Test build #108324 has started for PR 25251 at commit 32612bd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/13421/
Test PASSed.

Test build #108324 has finished for PR 25251 at commit 32612bd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/108324/
Test PASSed.

Merged to master.


Add support for extra fields through methods on resource class.

Really small change which adds support for simple extra fields without custom field classes, everything works as classic model attributes.
    class MyModelResource(ModelReource):

        def export_myextra_field(self, obj):

            return obj.related.value

        def save_myextra_field(self, obj, data):
            # optional

            RelatedObject.create(**data)
            RelatedObject1.create(**data)

        def get_myextra_field_name(self):
            # optional

            return "ColumnName"

So you're proposing we add three new API functions, to allow a user to add only a single "extra" field to resource?
I don't see how this provides any substantial benefit over using a field attribute, given the added complexity for such a small use case.
Am I missing something?

That is the point you can simple write import without limitations, which is usefull especially in a complex imports where you have for example Company which has two addresses(copy driven by checkbox), some User entities with CompanyMemberhips and  other entities like Account, ContactInformtion etc.. which could be easily handled by one or more methods depends on what you want which are called in your ordering and all this with diff and transaccions.
I need this features every time when i creating imports/exports stuff and today I decided to make this to the core and I was was surprised how simple is it (new lines ratio).
I respect your opinion about that new API, but without breaking anything and with some lines I proposed usefull extension.

Yes, by only adding, you aren't necessarily breaking anything, but that's not sufficient. Adding complexity to an API requires a clear use case. If you can say "without these API changes, X is impossible" or "Y is difficult because it requires even more complex changes A B C", then you might have a case.
But you are saying the only reason you are adding these 3 new API class is because you don't want to add a field attribute to a class. I still don't understand why adding that field is a problem.
I don't want to close this PR quite yet because I want to give you a chance to make a clear case for your changes.

Yes, i know that, i need make simple import/export without writing 30 fields with same code. I can just write method instead of writing fields. with this lines we can simple do this
from csb.identity.utils import get_customer
from django.contrib.auth.models import User
from import_export import fields, resources, widgets

from .models import (Company, CompanyAddress, CompanySegment,
                     Customer, UserContact)
from oscar.apps.address.models import Country

COMPANY_FIELDS = ('external_id', 'name', 'comp_id', 'vat_id', 'segment',
                  'line1', 'line4', 'postcode',
                  'email', 'first_name', 'last_name', 'phone_number', 'password',
                  'garant_name', 'garant_email', 'garant_phone_number',
                  'garant_password', 'garant_id')

class CompanyResource(resources.ModelResource):

    external_id = fields.Field(
        'external_id',
        column_name='CASE ID')

    name = fields.Field(
        'name',
        column_name='name')

    comp_id = fields.Field('comp_id',
                           column_name='ICO')

    vat_id = fields.Field('vat_id',
                          column_name='DIC')

    def export_garant_name(self, company):
        if company.garant:
            return company.garant.first_name

    def export_garant_phone_number(self, company):

        if company.garant:
            return company.garant.customer.phone_number

    def export_garant_email(self, obj):

        if obj.garant:
            return obj.garant.email

    def save_garant_name(self, obj, data):

        user, user_created = User.objects.get_or_create(**{
            "username": data["garant_email"],
            "defaults": {
                "email": data["garant_email"],
                "first_name": data["garant_name"],
            }
        })

        mobile_queryset = UserContact.objects.filter(
            user=user,
            type="mobile")
        if mobile_queryset.exists():
            mobile_queryset.update(value=data['garant_phone_number'])
        else:
            UserContact.objects.create(
                user=user,
                value=data['garant_phone_number'],
                type="mobile")

        customer = get_customer(user)

        if user_created:
            user.set_password(data['garant_password'])
            user.save()
            customer.external_id = data['garant_id']
            customer.save()
        obj.garant = user
        obj.save()

    def export_garant_id(self, company):
        if company.garant:
            return company.garant.external_id

    def export_segment(self, company):
        if company.segment:
            return company.segment

    def get_line1_name(self):
        return "Street"

    def get_line4_name(self):
        return "City"

    def export_line1(self, company):
        if company.address:
            return company.address.line1

    def export_line4(self, company):
        if company.address:
            return company.address.line4

    def export_postcode(self, company):
        if company.address:
            return company.address.postcode

    def get_admin(self, company):

        try:
            if company and company.admin:
                return company.admin
        except:
            pass

    def export_email(self, company):

        admin = self.get_admin(company)

        if self.get_admin(company):

            return admin.user.email

    def export_first_name(self, company):

        admin = self.get_admin(company)

        if self.get_admin(company):

            return admin.user.first_name

    def export_last_name(self, company):
        admin = self.get_admin(company)

        if self.get_admin(company):

            return admin.user.last_name

    def export_password(self, company):
        pass

    def export_garant_password(self, company):
        pass

    def export_phone_number(self, company):
        admin = self.get_admin(company)

        if self.get_admin(company):

            return admin.phone_number

    def save_segment(self, obj, data):

        if not obj.segment:

            segment, created = CompanySegment.objects.get_or_create(**{
                'name': data['segment'],
                'defaults': {
                    'verbose_name': data['segment']
                }
            })

            obj.segment = segment
            obj.save()

    def save_line1(self, obj, data):

        if not obj.address:
            address = CompanyAddress.objects.create(**{
                "line1": data[self.get_line1_name()],
                "line4": data[self.get_line4_name()],
                "postcode": data["postcode"],
                "country": Country.objects.get(iso_3166_1_a2="CZ")
            })
            obj.address = address

            billing_address = address
            billing_address.id = None
            billing_address.save()

            obj.billing_address = billing_address
            obj.save()
        else:
            obj.address.line1 = data[self.get_line1_name()]
            obj.address.line4 = data[self.get_line4_name()]
            obj.address.postcode = data["postcode"]
            obj.save()

    def save_email(self, obj, data):
        """
        In this method you can save related objects or
        just save a field
        """
        # print self.fields['email'].clean(data)

        user, user_created = User.objects.get_or_create(**{
            "username": data["email"],
            "defaults": {
                "email": data["email"],
                "first_name": data["first_name"],
                "last_name": data["last_name"]
            }
        })

        if user_created:
            user.set_password(data['password'])
            user.save()

            try:
                get_customer(user)
            except:
                pass

            roles = [Customer.ADMIN,
                     Customer.TECHNICAL_CONTACT,
                     Customer.ADMINISTRATIVE_CONTACT]
            for role in roles:
                obj.add_user(role, customer=user.customer)

            mobile_queryset = UserContact.objects.filter(
                user=user,
                type="mobile")
            if mobile_queryset.exists():
                mobile_queryset.update(value=data['phone_number'])
            else:
                UserContact.objects.create(
                    user=user,
                    value=data['phone_number'],
                    type="mobile")

    class Meta:
        model = Company
        fields = COMPANY_FIELDS
        export_order = COMPANY_FIELDS
        skip_unchanged = False
        import_id_fields = ('comp_id',)

This looks like something that could be done without changing Resource class, I imagine Field subclass that take delegate saving/exporting to given functions :
email = MyField(func_save=save_email, func_export=export_email, column_name='Email')

Yes, your imagination is really nice, but I think that with propagation of resource class to the field we can use method generated by column_name and declared directly on resource class which makes your code more clean or we can use both approaches


(hyprland/foot) Image Previews don't work if $TERM is set to foot-extra

If you leave $TERM as just "foot" it causes ncurses applications to misbehave (e.g https://codeberg.org/dnkl/foot/issues/1429) so installing foot-terminfo and setting $TERM to foot-extra is necessary. Problem is yazi doesn't preview images if you do that (At least for me. Can someone else confirm?)
(btw this is not a big deal since I can make a script that literally just runs "$TERM=foot yazi". thought i'd report it just in case still).

Interesting, just found out about foot-extra. Well, I'm not a die-hard fan of foot. 🤣
I've created a simple PR to fix it. Let me know if it works for you. https://github.com/sxyazi/yazi/pull/277

That worked. 👍


Setting orientation="right" for dependetAxis inverts X domain

Execute following code on http://victory.formidable.com/docs/victory-chart
<VictoryChart>
 <VictoryAxis
  scale="time"
  tickValues={[
    new Date(1980, 1, 1),
    new Date(1990, 1, 1),
    new Date(2000, 1, 1),
    new Date(2010, 1, 1),
    new Date(2020, 1, 1)]}
    tickFormat={(x) => x.getFullYear()}/>

  <VictoryAxis
  	dependentAxis
    orientation="right"
    padding={75}
    label="y-axis"
    standalone={false}/>
</VictoryChart>

Actual orientation of X axis:

Expected orientation of X axis:

This issue was moved to FormidableLabs/victory-chart#148


hashfile.tree: add as_trie/from_trie

Related to https://github.com/iterative/dvc/issues/4657.
Using it in https://github.com/iterative/dvc/pull/9389/files to prune subtree and add items to the trie and rebuild tree out of it.

Codecov Report
Patch coverage: 50.00% and project coverage change: -0.04 :warning:

Comparison is base (a2f9ed9) 57.82% compared to head (1c225cc) 57.78%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #345      +/-   ##
==========================================
- Coverage   57.82%   57.78%   -0.04%
==========================================
  Files          51       51
  Lines        3395     3402       +7
  Branches      595      596       +1
==========================================
+ Hits         1963     1966       +3
- Misses       1346     1350       +4
  Partials       86       86

Impacted Files
Coverage Δ

src/dvc_data/hashfile/tree.py
54.66% <50.00%> (-0.38%)
:arrow_down:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


issue in the process of tok

./tokenize-anything.sh < ~/Documents/ZeroShot/sen/AFP_ENG_20030417.0764 > out_file
Invalid range "\x{0CE6}-\x{0BEF}" in transliteration operator at ./support/quote-norm.pl line 149
I met this issue, could you please give me the suggestion to solve it? cheers. @jflanigan

You can try to delete the line 149 in quote-norm.pl file.


Weird issue - DELETE calls fail with Akamai H2 (http/2) enabled

I have 2 Android apps which use Retrofit 2, with the latest version of okhttp. Last week, we enabled http/2 on our API, at the Akamai control panel. That same day, we started getting complaints that users could no longer do certain things in the apps,  and it quickly became apparent that it was only Android apps, and only DELETE calls

Definitely weird. Can you give us more to go on? Perhaps use Charles to record what OkHttp did and how that differs from what it should do?

Sorry, hit the submit button accidentally.  Finishing my thought :
Only DELETE calls are failing.  We tested turning  off H2 support, and all was well again, and turned it back on,  and is failing again.
What can I do to help troubleshoot this issue?

Charles is worth the money and can help you to diagnose HTTP/2 problems.
https://www.charlesproxy.com/
You can also use a tool like WireShark to make DELETE requests in other HTTP/2 clients and show how OkHttp’s requests aren’t as expected.
My guess though is that it’s probably an issue with the server and not the client, and your workaround will be to disable HTTP/2 until they fix their kit.

Calls aren't making it to the server  -  IIS logs allow that. Call never makes it out of Akamai to us. But only deletes. I'll see if I can figure out Wireshark and Charles, but I think there may be a bug in  okhttp causing it.

That should read : IIS logs prove that.

@swankjesse - I've used Charles to compare the request being made by our iOS app (which does work fine making a DELETE call through Akamai) & our Android app (which fails the same call). There were some headers missing, and on a lark, I manually added "content-length: 0" to the DELETE request, and sure enough, it started working.
I can't fine anything in the http/2 spec that says that a DELETE request MUST have a content-length, so I'm not sure if okhttp is wrong or Akamai is.
Thoughts?

I’m curious – how are you specifying the DELETE method? DELETE is special in OkHttp because we permit a request body but don’t require it.
It’s possible we could fix this by changing this line in Http2Codec.writeRequestHeaders():
    boolean permitsRequestBody = HttpMethod.permitsRequestBody(request.method());

to this:
    boolean hasRequestBody = request.body() != null;

@DELETE(PATH)
Call deleteThing(@Query("UserId") String userId, @Query("ThingId") long thingId);
No body, 2 QS params and a couple custom headers ("ApiKey" and "Cache-Control", "public, max-age=9600")

Sorry, I should add that this is using Retrofit2

Same issue here, now that we enabled Akamai with HTTP2.

@hjudez I have fixed it temporarily in my project by adding this to an existing interceptor:
if (request.method().equals("DELETE"))
{
    updatedRequestBuilder.header("content-length", "0");
}

@jkane001 looks like a good fix. Thanks! We've disabled http2 for now but I guess I'll try you suggestion in case back-end enables http2 again.

Probably the same problem as #2892?

Can anyone with this issue confirm whether the latest snapshot fixes the issue (without the interceptor)? Or if you can provide a test URL I'm also happy to test. I'd like to see if #2892 was a possible culprit.

I’m gonna be optimistic here & assume @dave-r12’s fix fixed!

a little bit late but I can confirm it's been fixed.

in which version it is fixed @hjudez @swankjesse ?

The most recent version.
On Thu, Mar 16, 2017, 4:13 AM Eric N<EMAIL_ADDRESS>wrote:

in which version it is fixed @hjudez https://github.com/hjudez
@swankjesse https://github.com/swankjesse ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/square/okhttp/issues/2851#issuecomment-286986158, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEEEZWaUAyjdDChrA2_SmhH_Tn5DerWks5rmO8tgaJpZM4J70On
.

Thanks @JakeWharton , tested and it's working


Global monitor job timeouts (and some small tweaks)

This succeeds #19 and, most notably, adds a deploy-time option to set the timeout. It also has some other small fixes:

When I went to test on k8s, the fsGroup annotation was giving errors so I moved it around.
Upgraded the container to latest Rust/Debian, mostly because the index download times were so slow I thought the DNS was broken (since when are container network issues not DNS??).

I took the liberty of fixing the typos in the commit messages and force-push.


Implement mut arguments

The following shouldn't compile today but it does:
struct A {
    a: u64,
}

impl A {
    fn f(self) {
        self.a = 42;
    }
}

because self is not passed to f as mut.
The change I'm proposing to fix https://github.com/FuelLabs/sway/issues/908 rightfully breaks the above. However, there is no way to un-break it because we can't yet write:
impl A {
    fn f(mut self) {
        self.a = 42;
    }
}

More generally, we want to be able to annotate any argument as mut:
fn bla(mut a: u64, mut b: u64) {
    a = 0;
    b = 0;
}

Edit: the case with arbitrary arguments implies that we support passing arguments by ref, which still has to be property designed and thought of. So, for now, we should just enable mut self.

We need to establish some ground rules around how different types are passed into functions. One proposal is as follows:
struct A {
    a: u64,
}

struct B {
    b1: u64,
    b2: u64,
}

impl A {
    fn f1(self, x: u64, b: B) {
        // Read-only access to self, self.a, x, b.b1, b.b2
    }

    fn f2(mut self, mut x: u64, mut b: B) {
        // Read/write access to self, self.a, x, b.b1, b.b2
    }

    fn f3(ref self, ref x: u64, ref b: B) {
        // Read-only access to self, self.a, x, b.b1, b.b2
    }
   fn f4(ref mut self, ref mut x: u64, ref mut b: B) {
        // Readwrite access to self, self.a, x, b.b1, b.b2
    }
}

fn main() {
    let mut a_data = A { a: 0 };
    let mut x_data = 0;
    let mut b_data = B { b1: 0, b2: 0 };

    a_data.f1(x_data, b_data); // No changes to a_data, x_data, or b_data expected.

    a_data.f2(x_data, b_data); // No changes to a_data, x_data, or b_data expected.

    a_data.f3(x_data, b_data); // No changes to a_data, x_data, or b_data expected.

    a_data.f3(x_data, b_data); // Potential changes to a_data, x_data, or b_data.
}

Note: I don't think there should be any distinction between copy types (such as u64) and non-copy types (such as struct) with regards to the above, even though structs are really just pointers to data with our implementation.

I don't think there should be any distinction between copy types (such as u64) and non-copy types (such as struct) with regards to the above, even though structs are really just pointers to data with our implementation.

Rust does use the Copy trait to show this distinction explicitly. We could do the same?
This all makes sense. I don't think we should necessarily modify how things are passed around, instead just modifying the semantic analysis. Structs shouldn't be copied if they're passed in immutably. In that sense, we could just introduce the mut keyword and disallow mutation by default (unlike now).
The complexity comes in when you want to pass in a single mutable copy type. We copy those into new registers at this point. They're technically never persisted in memory. In fact, a reference to a number like this is itself not currently possible, as types this small exist only in a register.

I don't think we should necessarily modify how things are passed around, instead just modifying the semantic analysis.
Totally agree. All the above makes sense, thanks.

I don't necessarily have a clean solution to the problem of mut single-register types. Is it possible in the IR to signal that a register must not be copied and instead mutated in place? cc @otrho

Is it possible in the IR to signal that a register must not be copied and instead mutated in place?

I have a feeling that the inliner will just do this for us by default... Let me remove the mutability check and see what I get.

We need to find a solution to the problem of mut single-register (copy) types. Is it possible in the IR to signal that a register must not be copied and instead mutated in place? cc @otrho

The IR itself has no concept of registers, just values.  If we add explicit mutability to Sway we will add it to the IR in the same fashion.   And the it and the codegen can manage the copying or non-copying of registers.
The IR already has pointers, although not quite fully realised as types, but we can add that and then the IR can truly have by-reference semantics, even if we keep it implicit in Sway.  That way mutable arguments can work whether they end up being registers or memory locations.

How about we start by allowing the mut keyword only for self so that something like this works:
script;

struct A {
    a: u64,
}

impl A {
    fn f(mut self) {
        self.a = 0;
    }
}

fn main() {
    let mut a = A { a: 0 };
    a.f();
}

This is a fairly easy to change to make and would unblock Vec<T>. If we do this, then we don't have to worry (for now) about:

Dealing with mutable copy types. These require some modification to the IR to handle well (due to store and load expecting pointers etc.)
Dealing with the ref keyword
Disallowing mutable arguments in ABI methods
and maybe other stuff.

The lack of (2) bugs me a bit but we seem to be treating self as a ref anyways already.
@FuelLabs/sway-compiler thoughts?

This actually sounds quite reasonable: :+1:

Now that Vec<T> is under way, I think this is now even more important because users will probably want to pass Vec<T> to functions as an argument, and they can't actually call any of the methods that modify Vec<T> without the mut keyword.

One option here that was brought up in a discussion earlier is to allow mut arguments only for reference types and error out when the mut keyword is used with copy type arguments. That would help the Vec<T> case. The syntax here would still not require ref.

https://github.com/FuelLabs/sway/pull/2363 closes this. We probably won't ever support mut args without ref but we will see if that ever comes up in the future.


Merge the user-input parameters and the preset values in the resource definition for variables of type map and object

Is your enhancement related to a problem? Please describe.

Test Steps:

New a resource definition.
Add two matching rules using template aws-rds-mysql and alicloud-rds-mysql.
Fill in different Vpc Ids in two templates and save.
Remove the field for vpc_id from the UI schema of the resource definition.
Using the definition to create a resource.

Test Result:

The user-input parameters are not merged with the preset values in the resource definition.

Describe the solution you'd like

Merge the user-input parameters and the preset values in the resource definition for variables of type map and object.
Describe alternatives you've considered

Additional context

Environment

Walrus version: main 57e7bb83ce7aacac99c5a1c723bb488ad5ef5a4e
UI version: dev-a712ac5

Test Result: pass


Make DamageAccountValue as horizontally centered as possible

"xpos" "c-52" "wide" "100":

"xpos" "c-51" "wide" "98":

"xpos" "c-52" "wide" "100" (1080p)
"xpos" "c-51" "wide" "98" (1080p)
"xpos" "c-51" "wide" "98" (720p)

Looks good to me :) thank you. I'll check out how it looks later on 4k/other high dpi resolutions as well. One thing I also hadn't considered was using a cs value like the crosshairs, which would also allow it to be even more centered perhaps

Theoretically you should be able to be pixel accurate with cs values. However if c-51 already gets you there, there'd probably not be too much reason to switch. Even then, fonts can never be truly perfectly centered due to different glyphs. cs value would be what I'd use in this case if it was only one pixel off.
it happens with crosshairs, you have to nudge the cs values in the millionths to get a glyph truly centered: https://github.com/rbjaxter/budhud/blob/5624ea4b77075c7ea0567cefae87b40b7a7e3e81/%23customization/_enabled/bh_crosshairs.res#L23
....but you'd have to adjust it on a per-glyph basis


Deprecate kyma provision gardener

Description
Changes proposed in this pull request (what was done and why):

Deprecate kyma provision gardener

Changes refer to particular issues, PRs or documents:

https://github.com/kyma-project/kyma/issues/18674

Traceability

[x] The PR is linked to a GitHub issue.
[ ] New features have a milestone set.
[ ] New features have defined acceptance criteria in a corresponding GitHub Issue, and all criteria are satisfied with this PR.
[x] The corresponding GitHub issue has a respective area and kind label.
[x] The follow-up issues (if any) are linked in the Related Issues section.
[x] Adjusted the documentation if the change is user-facing.
[ ] The feature is unit-tested
[ ] The feature is e2e-tested

/area cli
/kind chore

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

/retest


Create Virginia Key DSPECS

TODO:

[x] Create all 6 DSPECS (12hr, 24hr, 48hr, 72hr, 96hr)
[ ] Add interpolationParameters for predicted X,Y Wind Components

method linear, 4 hour limit

[ ] Include any necessary DB migration data in comment below

DB Migration InitData
virginiakeyLocations.csv:
code,displayName,notes,latitude,longitude
VirginiaKey,Virginia Key,Virginia Key Biscayne Bay FL,25.731667,-80.161667
virginiakeyMapping.csv:
dataLocationCode,dataSourceCode,dataSourceLocationCode,priorityOrder
"VirginiaKey","NOAATANDC","8723214",0

Extra Notes

dWnSpd and dWnDir need to be added as possible ingest_series cases in NOAATANDC


raise ParserError if year, month, or date is not found in string

this attempts to resolve https://github.com/crsmithdev/arrow/issues/456
using arrow.get('31.12.2017') as an example, the input string matches the format YYYY generated by parser.parse_isoformat. As a result, only year is parsed from the input string which leads to month and date defaulting to 1 in  _build_datetime.
This attempts to resolve the attached issue by raising parser error if year, date or month is missing from the  parts parameter provided to _build_datetime

closing. bad solution -_-


Space-cli get db-schemas get schemas in wrong order

Describe the bug
Suppose I have 3 tables A.B, C where A has a foreign key on B, and B has a foreign key on C.
If a use space-cli for getting database schemas ina file using get command. It fetches schemas in any order. Let's say after running the command i get schemas in order B,C,A.
If i use space apply commad to apply the above schema file generated by get db schema i'll get error, because for creating table B, Table C should already be created.
Expected behaviour
space apply db-schema-file should work seamlessly
Actual behaviour
space apply db-schema-file doesn't work
Steps to reproduce

create tables as mentioned above with foreign keys
generate the schema file using space get db-schema command
apply the generated file

Your environment

Space Cloud version: - OS:
Kubernetes or Docker:
The client api version (if applicable):
Browser (if applicable):

If this bug restricts your use of space-cloud, give it a thumbs up reaction, so that we can determine which bugs need to be fixed immediately.
👍

Coming in v0.20.0


"Async iterators and generators" not findable on website?

Hello. I added #4105 quite a while ago, hopefully resolving #3690.
However I can't seem to find it on caniuse.com . Searching for "Async" doesn't show me the feature in the search results list. Searching for the full description "Async iterators and generators" doesn't show it.
It seems like something is wrong but I don't know what it is. I'd love to be able to see this feature on caniuse.com. Any ideas what the issue is?

Yes, the data is still pending review before it's appears to the site to ensure correctness. Apologies that it's been a while. If it's possible for you to provide a test case to check browser support that would help speed things up.


hideNavigationDrawer doesn't apply to smaller screens

https://github.com/user-attachments/assets/60d4d19b-5cad-4309-af77-ed8646e9fbc3

Anyone working on this, please include this fix as well
Hide AppBar on the login page (especially on smaller screens) #126
I missed this in my previous work #135


Disable CCache for Release(Reproducible) builds jdk-19+

For reproducible builds (currently jdk-19+), ccache must be disabled, automatically add --disable-ccache for jdk-19+ when RELEASE=true.

Will this be wrapped in "if temurin" or will it apply to all variants?
Is there compelling reason to not make this applicable to all versions right away?
@pshipton could confirm, but I think Semeru would be interested in this for all releases. I think we've talked about it in the past.

@AdamBrousseau
So I was going to make all variants, as it should apply. Basically, with ccache enabled the path substitution by CCache is not always deterministic in its use of ../../ in path substitutions.

Including Semeru sounds good.

We should include JDK17 in this too especially since Linux/JDK17 should be able to be built in a reproducible way going forward too?

We should include JDK17 in this too especially since Linux/JDK17 should be able to be built in a reproducible way going forward too?

Yes, there's a few other things for that to be done as well

related to https://github.com/adoptium/temurin-build/issues/2977


KeeWeb Connect looses connection from Safari on macOS

KeeWeb Connect looses connection from Safari to KeeWeb on macOS 10.15.7 (Intel).
Versions:

Safari: Version 14.1.1 (156<IP_ADDRESS>.6, 15611)
macOS: 10.15.7 (19H1217)
KeeWeb: KeeWeb v1.18.6 (070573e, 2021-05-19)
Environment: electron v12.0.7
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)
AppleWebKit/537.36 (KHTML, like Gecko) KeeWeb/1.18.6 Chrome/89.0.4389.128
Electron/12.0.7 Safari/537.36
KeeWeb Connect: 0.3.4 (App Store)

Expected behavior:

Navigate to login form
hit the key toolbar button
credentials selection dialog should appear

Actual behavior (scenario A):

Navigate to login form
hit the toolbar button
nothing happens (even no logs)

Actual behavior (scenario B):

Navigate to login form
right click username/password input field, choose KeeWeb -> Insert Username+password
nothing happens
the Web Inspector — KeeWeb Connect – Background Page contains error messages

[Log] -> KW – {action: "ping", data: "hI5AgQeqnVAbPw=="} (background.js, line 692)
[Log] <- KW – {data: "hI5AgQeqnVAbPw=="} (background.js, line 697)
[Log] -> KW – {action: "get-logins", message: "F+ntu1hiMA4R9Hg+SNIz7jpEd1yWuiy4+K/YkMFfwUNhNDro55…RKW3uxU2WsLci8by6hV38YqiEtJsyzksF9E5qqcQArdQAfwM=", nonce: "dquU/OnJ5NUIafLzI3C32990Cv5fJL/y", …} (background.js, line 692)
{action: "get-logins", message: "F+ntu1hiMA4R9Hg+SNIz7jpEd1yWuiy4+K/YkMFfwUNhNDro55…RKW3uxU2WsLci8by6hV38YqiEtJsyzksF9E5qqcQArdQAfwM=", nonce: "dquU/OnJ5NUIafLzI3C32990Cv5fJL/y", clientID: "vSSdRisiroifjn7XOcb4SO1sk+Vj+7jK"}Object
[Log] <- KW – {error: "Client not connected: vSSdRisiroifjn7XOcb4SO1sk+Vj+7jK", action: "get-logins", errorCode: 0} (background.js, line 697)
[Error] Unhandled Promise Rejection: Error: KeeWeb returned an error: Client not connected: vSSdRisiroifjn7XOcb4SO1sk+Vj+7jK
	(anonymous function) (background.js:1507)
	asyncFunctionResume
	(anonymous function)
	promiseReactionJobWithoutPromise
	promiseReactionJob

Hi! That's interesting, maybe KeeWeb was restarted in between? Does it happen if you close / open KeeWeb and Safari and then do the same?

Hello,
Yes, re-starting KeeWeb diminishes the connection which requires Safari re-start. I'm aware of that.
But this situation is happening suddenly after several successful login details fillings. After that KeeWeb Connect looses connection and behaves like described.
Unfortunately I'm not familiar with Safari debugging, so I don't know where to look.
Do you have any ideas what I should try or focus on?
Thanks a lot.

Let's try to check logs. Open Console.app and enter KeeWeb Connect into the filter field, then change its type to Process:

and click "Start":

After this it should log what happens in the extension.
How fast does this happen after a successful connection?

Excellent. Did that and it worked!
Now the situation happened just after the first logging attempt.
The steps:

Launch KeeWeb and Console Monitoring
Launch Safari
Navigate to https://mojeid.cz/auction/password/ (the site uses Yubikey as 2FA as well)
Fill login details by hitting Cmd+Shift+A
Hit green retry button to activate YubiKey challenge (the site has issues LOL)
I'm in.
Log-out and close tab
Open new tab
repeat from stem 3

The first successful attempt with the second failed one are logged in the log1.txt. I did another failed attempt which is in the log2.txt but with debug details enabled as well. Finally, all the attempts are recorded in the JavaScript/Extension log (console.txt).
log1.txt
log2.txt
console.txt

It would be good to see also what happens in KeeWeb, it's not clear why it loses the connection because it looks like the native host didn't crash. Why did it change the id though?

Hello, here is a log snippet from KeeWeb.
[INFO ] 2021-05-31T07:12:53.540Z [browser-extension-connector] New connection with socket 4
[DEBUG] 2021-05-31T07:12:53.540Z [browser-extension-connector] Init connection 4 {"pid":43967,"origin":"safari-keeweb-connect"}
[INFO ] 2021-05-31T07:12:53.540Z [browser-extension-connector] Socket 4 activated for app: "Safari", extension: "KeeWeb Connect", pid: 43967, ppid: undefined
[DEBUG] 2021-05-31T07:12:53.540Z [browser-extension-connector] Extension[4] -> KeeWeb {"action":"change-public-keys","publicKey":"RbGs8g09sXPbHKloAodcyEBPmnbFlF30b0S4TkksHHk=","extensionName":"KeeWeb Connect","nonce":"AnQGqssWRPvXRNPO/tXWihZC8mAuFKrJ","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","version":"0.3.4"}
[INFO ] 2021-05-31T07:12:53.545Z [browser-extension-connector] New client key created Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN 0.3.4
[DEBUG] 2021-05-31T07:12:53.546Z [browser-extension-connector] KeeWeb -> Extension[4] {"action":"change-public-keys","version":"1.18.6","publicKey":"67l6uDkUwm1EW0OmGuVuOAqAxn0ExopbeOmfpsxVr3o=","nonce":"A3QGqssWRPvXRNPO/tXWihZC8mAuFKrJ","success":"true","appName":"KeeWeb"}
[DEBUG] 2021-05-31T07:12:53.549Z [browser-extension-connector] Extension[4] -> KeeWeb {"action":"ping","data":"yUp0JhyWzYgSpw=="}
[DEBUG] 2021-05-31T07:12:53.550Z [browser-extension-connector] KeeWeb -> Extension[4] {"data":"yUp0JhyWzYgSpw=="}
[DEBUG] 2021-05-31T07:12:53.554Z [browser-extension-connector] Extension[4] -> KeeWeb {"message":"kw/csrxX6/GtORA5iPYnV8ThixeiPyVsoT9uwZcFdcnYYzN5EDOy7QYi7XzQWv0ZjIsCLDxJhs6HvnWCWzyoKreIpVwCFKeAcGabz+wUFJL9SdGYookbaYAuJGc6V028rQo=","nonce":"JVpK8ylFIVp7OTFZJyBes2TNaq08KZ+b","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[DEBUG] 2021-05-31T07:12:58.357Z [browser-extension-connector] KeeWeb -> Extension[4] {"action":"get-logins","error":"The request was denied","errorCode":"6"}
[INFO ] 2021-05-31T07:13:18.386Z [browser-extension-connector] Socket 4 closed
[INFO ] 2021-05-31T07:13:25.314Z [browser-extension-connector] New connection with socket 5
[DEBUG] 2021-05-31T07:13:25.314Z [browser-extension-connector] Init connection 5 {"pid":43974,"origin":"safari-keeweb-connect"}
[INFO ] 2021-05-31T07:13:25.314Z [browser-extension-connector] Socket 5 activated for app: "Safari", extension: "KeeWeb Connect", pid: 43974, ppid: undefined
[DEBUG] 2021-05-31T07:13:25.314Z [browser-extension-connector] Extension[5] -> KeeWeb {"action":"ping","data":"YzvYya1yX4nqdg=="}
[DEBUG] 2021-05-31T07:13:25.315Z [browser-extension-connector] KeeWeb -> Extension[5] {"data":"YzvYya1yX4nqdg=="}
[DEBUG] 2021-05-31T07:13:25.317Z [browser-extension-connector] Extension[5] -> KeeWeb {"message":"yNZDU/FlEre/x+bQHBB2KZue5pqV/TZaB4fpNwZ2OufUmMchSQ5B+3+Cn9nziFm+PTaINIE0GbKrZTmOhDeL0SJKTn9szHAd3bJSDobQt2FhhHzkRtDlBWllZ/0CetIBPxw=","nonce":"VpslZJycq7f652GQD7EBpbiLMQgvVfHw","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:13:25.317Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:13:25.317Z [browser-extension-connector] KeeWeb -> Extension[5] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[INFO ] 2021-05-31T07:13:45.818Z [browser-extension-connector] Socket 5 closed
[INFO ] 2021-05-31T07:13:53.798Z [browser-extension-connector] New connection with socket 6
[DEBUG] 2021-05-31T07:13:53.798Z [browser-extension-connector] Init connection 6 {"pid":43987,"origin":"safari-keeweb-connect"}
[INFO ] 2021-05-31T07:13:53.798Z [browser-extension-connector] Socket 6 activated for app: "Safari", extension: "KeeWeb Connect", pid: 43987, ppid: undefined
[DEBUG] 2021-05-31T07:13:53.798Z [browser-extension-connector] Extension[6] -> KeeWeb {"action":"ping","data":"1ETbx/FS36My9g=="}
[DEBUG] 2021-05-31T07:13:53.799Z [browser-extension-connector] KeeWeb -> Extension[6] {"data":"1ETbx/FS36My9g=="}
[DEBUG] 2021-05-31T07:13:53.801Z [browser-extension-connector] Extension[6] -> KeeWeb {"message":"HMSpo20HR9I8/gGhFZVmf7iemZyI4hEL9TCZfUCJPRtgFgmlh8c7D5IuWiJ+xStxLPb3kczDtmgxiM74nCa7vbTYiw==","nonce":"Z6QOUJzsMj9rCSG0UDYcqp/rfroOpVcm","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:13:53.801Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:13:53.802Z [browser-extension-connector] KeeWeb -> Extension[6] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[DEBUG] 2021-05-31T07:14:12.634Z [browser-extension-connector] Extension[6] -> KeeWeb {"action":"ping","data":"kw5EskYiw8s4eg=="}
[DEBUG] 2021-05-31T07:14:12.635Z [browser-extension-connector] KeeWeb -> Extension[6] {"data":"kw5EskYiw8s4eg=="}
[DEBUG] 2021-05-31T07:14:12.638Z [browser-extension-connector] Extension[6] -> KeeWeb {"message":"OoaOvuh5MbLsZcrr1Zx37DobXb22SBxWn+ouz07Jie5x2eodyiM4EEvsLJAyXpY0XSn2oOTAhb322ryythL7cX93Ag==","nonce":"BuMMMELFuub5nzFriF8IHKBg8zXqAGHf","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:14:12.638Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:14:12.639Z [browser-extension-connector] KeeWeb -> Extension[6] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[DEBUG] 2021-05-31T07:14:23.376Z [browser-extension-connector] Extension[6] -> KeeWeb {"action":"ping","data":"g4zUVAzuB4cBfw=="}
[DEBUG] 2021-05-31T07:14:23.377Z [browser-extension-connector] KeeWeb -> Extension[6] {"data":"g4zUVAzuB4cBfw=="}
[DEBUG] 2021-05-31T07:14:23.379Z [browser-extension-connector] Extension[6] -> KeeWeb {"message":"92twZy+tDLIWFAusN6IPlMd/GrmXN6h2L+4zijJpd1n8Sa4N2Wu4/0vFpH305pHAcw3JiCU8RXj6SRRlgHZYSYYDfw==","nonce":"x8mtsircL+r+H62F0KWNTg/dfMiJsUFR","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:14:23.379Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:14:23.380Z [browser-extension-connector] KeeWeb -> Extension[6] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[INFO ] 2021-05-31T07:14:43.869Z [browser-extension-connector] Socket 6 closed
[INFO ] 2021-05-31T07:15:02.207Z [browser-extension-connector] New connection with socket 7
[DEBUG] 2021-05-31T07:15:02.207Z [browser-extension-connector] Init connection 7 {"pid":44002,"origin":"safari-keeweb-connect"}
[INFO ] 2021-05-31T07:15:02.207Z [browser-extension-connector] Socket 7 activated for app: "Safari", extension: "KeeWeb Connect", pid: 44002, ppid: undefined
[DEBUG] 2021-05-31T07:15:02.207Z [browser-extension-connector] Extension[7] -> KeeWeb {"action":"ping","data":"RJNRWiHkwrUEOQ=="}
[DEBUG] 2021-05-31T07:15:02.208Z [browser-extension-connector] KeeWeb -> Extension[7] {"data":"RJNRWiHkwrUEOQ=="}
[DEBUG] 2021-05-31T07:15:02.210Z [browser-extension-connector] Extension[7] -> KeeWeb {"message":"kk7eQPV8LaVg1NHJRJEWZCMFWY37iWUwe3VNXCNhLfWHtosmAjg55dLOS2i8mYY4xUVLODR5HYCnnliSaBTlnE0WTw==","nonce":"7ffN/rdQ+XpHIw/7JPkIT10idDVHwYgD","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:15:02.211Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:15:02.211Z [browser-extension-connector] KeeWeb -> Extension[7] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[INFO ] 2021-05-31T07:15:22.709Z [browser-extension-connector] Socket 7 closed
[INFO ] 2021-05-31T07:17:22.202Z [browser-extension-connector] New connection with socket 8
[DEBUG] 2021-05-31T07:17:22.202Z [browser-extension-connector] Init connection 8 {"pid":44027,"origin":"safari-keeweb-connect"}
[INFO ] 2021-05-31T07:17:22.202Z [browser-extension-connector] Socket 8 activated for app: "Safari", extension: "KeeWeb Connect", pid: 44027, ppid: undefined
[DEBUG] 2021-05-31T07:17:22.202Z [browser-extension-connector] Extension[8] -> KeeWeb {"action":"ping","data":"T0LIbppINMExOg=="}
[DEBUG] 2021-05-31T07:17:22.203Z [browser-extension-connector] KeeWeb -> Extension[8] {"data":"T0LIbppINMExOg=="}
[DEBUG] 2021-05-31T07:17:22.206Z [browser-extension-connector] Extension[8] -> KeeWeb {"message":"qJTMdeNAsYIRieZIdkdcjUaWfLjYDWe3SYLA/F9vvoY80uciTk2h2lPkBpHzDbFGoL86aqSv/LLj8m4tbzG84WjpGw==","nonce":"JZCWmF9nZPf3SsuQ0KJsTToYzp4p/sjm","clientID":"Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","action":"get-logins"}
[ERROR] 2021-05-31T07:17:22.206Z [browser-extension-connector] Error in handler get-logins Error: Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN
[DEBUG] 2021-05-31T07:17:22.207Z [browser-extension-connector] KeeWeb -> Extension[8] {"action":"get-logins","error":"Client not connected: Ru7H5SoXVPPrZEc0FJGSEQPLZtRam2QN","errorCode":0}
[INFO ] 2021-05-31T07:17:42.210Z [browser-extension-connector] Socket 8 closed

I really don't know how the stuff works. I've seen in the Console something related to opendirectoryd and UID, EUID… It seems it trying to resolve my account to something. Is that something the Id you're talking about?

Strange, it's not clear why it disconnects. I need to reproduce it somehow, will try to do it this weekend.

I'm also experiencing this issue. After the first auto-fill by the extension on Safari the next attempts don't work until Safari is restarted. FWIW can't replicate this issue on Google Chrome, there sequential auto-fill/logins continue to work fine.

This should affect only Safari, yes.

it don't work any time!

I have the same problem on macOs Catalina 10.15.7 and Safari 15.5.
The first time it works, then the connection is closed. See log below.
standard	11:50:02.807298+0200	KeeWeb Connect Extension	[u ECE1884A-DEB9-4021-A44E-FB6ED6EF31DF] [<private>(<private>)] Removed sole personality.
standard	11:50:02.807647+0200	KeeWeb Connect Extension	[u ECE1884A-DEB9-4021-A44E-FB6ED6EF31DF] [<private>(<private>)] host connection from pid 34441 invalidated


Edit order

As a user of the console I want to be able to edit an order.

Have added designs here:
https://www.figma.com/file/K49Nx7XfocShhHcwwQzQkm/Console-V2?node-id=1652%3A26052
Adds an edit button to the Orders panel:

Which then launches a modal:

And then goes through the wallet confirmation process:


Multi gtfs instance

Description

This PR aims to allow for cleaning, validation, filtering, saving and visualising multiple GTFS files at once. It also refactors bbox_filter_gtfs. This function has remained however and is now depreciated.
Fixes #204
Motivation and Context

This motivation of this PR is to validate and clean multiple GTFS files in order to complete end to end pipeline runs.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Test configuration details:

OS: Windows 10
Python version: 3.9.13
Java version: N/A
Python management system: Conda

Advice for reviewer

Checklist:

[x] My code follows the intended structure of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

Additional comments

Codecov Report
Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (fda1e1b) 97.81% compared to head (1e0408e) 96.92%.

Files
Patch %
Lines

src/transport_performance/gtfs/multi_validation.py
88.33%
14 Missing :warning:

src/transport_performance/gtfs/gtfs_utils.py
95.00%
2 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #207      +/-   ##
==========================================
- Coverage   97.81%   96.92%   -0.89%
==========================================
  Files          14       15       +1
  Lines        1188     1365     +177
==========================================
+ Hits         1162     1323     +161
- Misses         26       42      +16

Flag
Coverage Δ

unittests
96.92% <91.01%> (-0.89%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

After discussing notebook update with team, we agreed that this PR should create a copy of the e2e marseilles notebook with an example implementation of how to use MultiGtfsInstance.

One final request - discussing with the team yesterday, there's a risk that empty feeds may cause problems when running the transport network. A delete_empty_feed parameter on the MultiGtfsInstance filter methods would safeguard against this. Behaviour would be something like:

delete_empty_feed=False (default), if filtering operation results in any empty feed, raise exception with path to problem GTFS.
delete_empty_feed=True, if filtering results in any empty feed, warn and delete that feed from the instance.
This should avoid empty feeds being written to file.

done

@CBROWN-ONS please see patch branch for fix to discovered bug in writing gtfs to file.
One more problem exists before we can merge. The summarise_routes and trips functions are not reporting the expected values for the minimum values:

@CBROWN-ONS please see patch branch for fix to discovered bug in writing gtfs to file.
One more problem exists before we can merge. The summarise_routes and trips functions are not reporting the expected values for the minimum values:

@CBROWN-ONS thanks for adding in the return_summary functionality. This has allowed me to inspect the above scenario more thoroughly. Can I clarify with you, we are reporting 12 bus routes minimum across all available Mondays in both GTFS feeds. I felt this should be 36 (minimum Newport + minimum Chester). You suggested that it's 12 because there are dates in the summary that only exist for one feed and not the other. I have now confirmed this to be the case.
set(chester_mondays).difference(newport_mondays)
{Timestamp('2024-03-04 00:00:00'),
 Timestamp('2024-03-11 00:00:00'),
 Timestamp('2024-03-18 00:00:00'),
 Timestamp('2024-03-25 00:00:00'),
 Timestamp('2024-04-01 00:00:00'),
 Timestamp('2024-04-08 00:00:00'),
 Timestamp('2024-04-15 00:00:00'),
 Timestamp('2024-04-22 00:00:00')}

Available dates in Chester extends to April '24 while Newport stops in March '24. I would suggest the current stat is misleading and the better thing to do would be:

Warn that the feeds have differing time ranges and...
Filter the dfs to common dates before calculating  summary stats.
This would need to work with multiple gtfs sources and considering how to handle differently overlapping date ranges that potentially have no overlap may need some consideration. Keen to get your thoughts on this issue.

@CBROWN-ONS please see patch branch for fix to discovered bug in writing gtfs to file.
One more problem exists before we can merge. The summarise_routes and trips functions are not reporting the expected values for the minimum values:

@CBROWN-ONS thanks for adding in the return_summary functionality. This has allowed me to inspect the above scenario more thoroughly. Can I clarify with you, we are reporting 12 bus routes minimum across all available Mondays in both GTFS feeds. I felt this should be 36 (minimum Newport + minimum Chester). You suggested that it's 12 because there are dates in the summary that only exist for one feed and not the other. I have now confirmed this to be the case.
set(chester_mondays).difference(newport_mondays)
{Timestamp('2024-03-04 00:00:00'),
 Timestamp('2024-03-11 00:00:00'),
 Timestamp('2024-03-18 00:00:00'),
 Timestamp('2024-03-25 00:00:00'),
 Timestamp('2024-04-01 00:00:00'),
 Timestamp('2024-04-08 00:00:00'),
 Timestamp('2024-04-15 00:00:00'),
 Timestamp('2024-04-22 00:00:00')}

Available dates in Chester extends to April '24 while Newport stops in March '24. I would suggest the current stat is misleading and the better thing to do would be:

Warn that the feeds have differing time ranges and...
Filter the dfs to common dates before calculating  summary stats.
This would need to work with multiple gtfs sources and considering how to handle differently overlapping date ranges that potentially have no overlap may need some consideration. Keen to get your thoughts on this issue.

Input from Ethan:

in each instance, calculate number of trips / routes by date and route type.
for each date, sum the trips / routes for each route type, across all gtfs. (total of trips/routes by route type per day).
Derive summary stats and aggregate.

@CBROWN-ONS please see patch branch for fix to discovered bug in writing gtfs to file.
One more problem exists before we can merge. The summarise_routes and trips functions are not reporting the expected values for the minimum values:

@CBROWN-ONS thanks for adding in the return_summary functionality. This has allowed me to inspect the above scenario more thoroughly. Can I clarify with you, we are reporting 12 bus routes minimum across all available Mondays in both GTFS feeds. I felt this should be 36 (minimum Newport + minimum Chester). You suggested that it's 12 because there are dates in the summary that only exist for one feed and not the other. I have now confirmed this to be the case.
set(chester_mondays).difference(newport_mondays)
{Timestamp('2024-03-04 00:00:00'),
 Timestamp('2024-03-11 00:00:00'),
 Timestamp('2024-03-18 00:00:00'),
 Timestamp('2024-03-25 00:00:00'),
 Timestamp('2024-04-01 00:00:00'),
 Timestamp('2024-04-08 00:00:00'),
 Timestamp('2024-04-15 00:00:00'),
 Timestamp('2024-04-22 00:00:00')}

Available dates in Chester extends to April '24 while Newport stops in March '24. I would suggest the current stat is misleading and the better thing to do would be:

Warn that the feeds have differing time ranges and...
Filter the dfs to common dates before calculating  summary stats.
This would need to work with multiple gtfs sources and considering how to handle differently overlapping date ranges that potentially have no overlap may need some consideration. Keen to get your thoughts on this issue.

Input from Ethan:

in each instance, calculate number of trips / routes by date and route type.
for each date, sum the trips / routes for each route type, across all gtfs. (total of trips/routes by route type per day).
Derive summary stats and aggregate.

@ethan-moss Are you able to clarify your suggestions here? I'm unclear.

@CBROWN-ONS please see patch branch for fix to discovered bug in writing gtfs to file.
One more problem exists before we can merge. The summarise_routes and trips functions are not reporting the expected values for the minimum values:

@CBROWN-ONS thanks for adding in the return_summary functionality. This has allowed me to inspect the above scenario more thoroughly. Can I clarify with you, we are reporting 12 bus routes minimum across all available Mondays in both GTFS feeds. I felt this should be 36 (minimum Newport + minimum Chester). You suggested that it's 12 because there are dates in the summary that only exist for one feed and not the other. I have now confirmed this to be the case.
set(chester_mondays).difference(newport_mondays)
{Timestamp('2024-03-04 00:00:00'),
 Timestamp('2024-03-11 00:00:00'),
 Timestamp('2024-03-18 00:00:00'),
 Timestamp('2024-03-25 00:00:00'),
 Timestamp('2024-04-01 00:00:00'),
 Timestamp('2024-04-08 00:00:00'),
 Timestamp('2024-04-15 00:00:00'),
 Timestamp('2024-04-22 00:00:00')}

Available dates in Chester extends to April '24 while Newport stops in March '24. I would suggest the current stat is misleading and the better thing to do would be:

Warn that the feeds have differing time ranges and...
Filter the dfs to common dates before calculating  summary stats.
This would need to work with multiple gtfs sources and considering how to handle differently overlapping date ranges that potentially have no overlap may need some consideration. Keen to get your thoughts on this issue.

Input from Ethan:

in each instance, calculate number of trips / routes by date and route type.
for each date, sum the trips / routes for each route type, across all gtfs. (total of trips/routes by route type per day).
Derive summary stats and aggregate.

@ethan-moss Are you able to clarify your suggestions here? I'm unclear.

@CBROWN-ONS - sure, no problem! When considering trips/routes for multiple GTFS, the summary descriptive statistics should reflect the case where all the GTFS data where 'one input'. That would then reflect the all services as a whole instead of mins/maxes for each GTFS file - the latter wouldn't give a useful insight into how many services run on days.
To consider all GTFS as 'one input', it can be done in 3 steps:

For each GTFS file, calculate the number of trips/routes by date and route_type.
For each date, sum trips/routes across all inputs - this is then gives the number of trips/routes by date as if all the GTFS inputs were a single input.
Can then aggregate by day of the week.


One-time passcode authentication is no longer in preview

Removed '(preview)' addition to link for "Email on-time passcode authentication" as that feature is no longer in preview.

@webash : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


SoftmaxPro ValueError

I'm new to the Allotrope Data Format so if any of this info is covered there, would love to learn more about how it's handled.
Context: the SoftmaxPro has a feature call PathCheck that allows the instrument to measure the liquid level in each well and normalize the measured Optical Density to a 1 cm path length (to compare to a cuvette method). This feature is necessary in labs that have multiple ways of collecting Optical Density measurements, such as ours. It works by measuring the absorbance peak of water in the near infrared region to calculate the optical path length.
Problem: If there is no liquid or the instrument cannot accurately measure the path of a well, the output result is: "Path?". When running softmax_pro_structure, a ValueError is thrown since the parser expects a float value.
The easy fix is importing the try_float method from allotropy.parsers.utils.values and inserting it into line 337 (well_data[well].processed_data.append(float(value)) becomes well_data[well].processed_data.append(try_float(value))) however it might be worth making a note somehow of the wavelength error - could help automation teams track down liquid pipetting errors in the future. Additionally, it might be worth including pathcheck as metadata in the results so companies can keep track of OD data by equipment and measurement type.

@corona779 it's a good point about capturing this as metadata field since many plate readers have a "path length correction" feature.
The Allotrope Ontology has a term for cell path length to capture the actual length, but not a term to capture whether a path length correction is applied.
If implemented, this part of the device control document and would likely be something like: path length correction enabled setting, as it would be analogous a term like automatic cycle threshold enabled setting that is used in the qPCR model.
@alejandro-salgado could you look at suggested edit? I don't believe any of our test data has examples where a string appears in as the absorbance value, but since it can occur this might make the adapter more robust.

@LimulusPolyphemus @alejandro-salgado if it helps I can provide y'all with an output file that contains this error.

@corona779 if you could that would be super helpful.

@LimulusPolyphemus here you go: path_check_example.txt.


{{moment-calendar ...}} not found

Using the  {{moment-calendar}} helper triggers following error:
Uncaught Error: Assertion Failed: A helper named 'moment-calendar' could not be found.
I'm using  "ember-moment": "4.1.0".

@badazz91 moment-calendar landed in 5.1.0

Sry my bad. Updated to the latest ember-moment. Now it works...
But the latest tag on this project is still 4.1.0. Isn't this odd?


Unable to run tests with latest daily

No tests are discovered in VS, and the command line runner fails with the error below. Same error occurs on the AppVeyor build server...
Env: VS 2015 RTM, build 7.0.0-beta7-13844, xunit 2.1.0-beta4-build3073 - Test project is csproj based C# class lib, targeting .NET 4.6. Suspect maybe an issue related to XUnit?
Any suggestions how to fix and get unblocked?
And what does this even mean:
Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context.

C:\Code\EntityFramework.SqlServerCompact>packages\xunit.runner.console.2.1.0-bet
a4-build3079\tools\xunit.console.x86.exe test\EntityFramework7.SqlServerCompact.
Tests\bin\Debug\EntityFramework.SqlServerCompact.Tests.dll
xUnit.net Console Runner (32-bit .NET 4.0.30319.42000)
Discovering: EntityFramework.SqlServerCompact.Tests
System.BadImageFormatException: Could not load file or assembly 'System.Runtime,
Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of it
s dependencies. Reference assemblies should not be loaded for execution.  They c
an only be loaded in the Reflection-only loader context. (Exception from HRESULT
: 0x80131058)
System.BadImageFormatException: Cannot load a reference assembly for execution.
If I remove the System.Runtime reference, I get this error, which Refers to 2 different version of System.Runtime:
C:\Code\EntityFramework.SqlServerCompact>packages\xunit.runner.console.2.1.0-bet
a4-build3079\tools\xunit.console.x86.exe test\EntityFramework7.SqlServerCompact.
Tests\bin\Debug\EntityFramework.SqlServerCompact.Tests.dll
xUnit.net Console Runner (32-bit .NET 4.0.30319.42000)
Discovering: EntityFramework.SqlServerCompact.Tests
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime
, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of
its dependencies. The system cannot find the file specified.
System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime
, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of i
ts dependencies. The system cannot find the file specified.

@bricelam Any updates, this issue sadly Blocks me from moving forward with beta 7 dailies since middle of july..

Leaving https://github.com/aspnet/EntityFramework/issues/2755 open with various issues, and closing this, as I am now unblocked on actually running the tests with variuos workarounds


Rename wallet variable to private key

We should update deposit.md too and any other docs that mention this.

Yes, I will test it against my local network and also modify the doc.


k:.Idx n doesn't make sense

My latest commit fixed some UB which materialized in the CI that this test here
ds2cps_ax_cps2ds_dependent2.thorin

failed to work sometimes on Windows.
Problem is: A literal 42:.Idx n doesn't make sense and should have been a type error (which it is now). It caused UB in some normalizers (which I have fixed).
One edge case is: 0:.Idx n. This kind of makes sense, as 0:.Idx 0 actually means 0:.Idx 2^64. So this literal is guaranteed to exist - but this is still kind of odd Oo Should we really allow that?
For now I disabled the test case above, as it makes use of this erroneous index.
See also #101, #105.
tl;dr:

We need to fix ds2cps_ax_cps2ds_dependent2.thorin
should we allow 0:.Idx n?

A cast s2s/u2u should also fix the problem is probably the correct fix.
I think we should allow 0:.Idx n as it might be helpful to have a zero sometimes and it exists from a theoretical point of view.

@NeuralCoder3: can you fix the ds2cps_ax_cps2ds_dependent2.thorin test case? Then, I can close this one here already.


Upgrade to Next.js v13

What does this PR do?

Upgrades Next.js to v13.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How should this be tested?

[ ] Existing unit tests

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code and corrected any misspellings
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

I tested this locally and got the peer dependencies error during npm install. Free to merge this PR after NextAuth officially support Next 13. Here is the related https://github.com/nextauthjs/next-auth/pull/5657

@niwsa Please take a look. Upgraded the NextAuth version and fixed the build issue.

The build runs fine locally without the xmlbuilder dependency. Not sure why its unable to resolve it in CI.

This looks good to go now


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/766 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target.git

Docs Build status updates of commit b259adf:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-GitBranchDeletedOrForcePushed] Cannot sync git repo to specified commit because branch Release_Archive_master_2021-02-02-06-31-41 has been deleted or has been force pushed

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


Getting error while building wheel file for regex while installing botbuilder-dialog library

Use this query to search for the most popular feature requests.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

botbuilder-dialog for python 3.9 depends on regex version <=2019.08.19 and these regex versions are not compatible with python 3.9. Code throwing an error while building wheel file for regex.

@MamtaKumari24,
Thank you for the report, I am Invetigating.

At the moment, we support the use of Python 3.7 or 3.8 for Bot Framework bots built with the Python SDK. I have verified that Python bots correctly build and run on the supported version.
We will keep your feature request in mind, but please note that due to the forthcoming retirement and end of support for the Python SDK in November 2023, it is unlikely that newer Python versions will be supported.

At the moment, we support the use of Python 3.7 or 3.8 for Bot Framework bots built with the Python SDK. I have verified that Python bots correctly build and run on the supported version.
We will keep your feature request in mind, but please note that due to the forthcoming retirement and end of support for the Python SDK in November 2023, it is unlikely that newer Python versions will be supported.

Is this end of support still planned?


⚠ ClamAV has degraded performance

In 0f62f72, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 52 ms

Resolved: ClamAV performance has improved in e77df4d.