System.Data.SqlClient (not Microsoft.Data) does not support DateTime2 in Batch Update/Insert anymore

Describe the bug
When utilizing the System.Data.SqlClient.SqlDataAdapter with UpdateBatchSize>1 in .Net 5 (or previous .Net Core Versions), DateTime2 Columns loose their precision (as if they were declared as DateTime Columns)
This does not happen when executing the same code with .Net 4.8
To reproduce
See attached minimal repro sample (a simple .net 5 console app)
A Sql Server is required on your system, but the DDL Code is included
bulkinsertbug.zip
Build&Run the csproj with net48 and net5.0 frameworks
Build&Run the code with  UpdateBatchSize  1 and 2
Comment/Uncomment the  CreateDbAndTablesIfNotExists  method as needed
The result (viewable in Sql Server) will look like this:

In the screenshot you can see the Framework (Enviroment.Version) as well as the UpdateBatchSize for convenience
You can clearly see the date values with lost precision in the described scenario (last 2 entries)
Expected behavior
The datetime precision is kept, even when UpdateBatchSize is >1
Further technical details
Microsoft.Data.SqlClient version: (found on the nuget or Microsoft.Data.SqlClient.dll)
.NET target: Core 5.0.8, and Framework 48 (4.0.30319.42000)
SQL Server version: SQL Server 2019, 13.0.15888.11
Operating system: Windows 20H2
Additional context
We already analyzed the problem in detail and found the culprit to lie in the discrepancy how SqlParameter.CopyTo is implemented
.Net Framework Code:
https://referencesource.microsoft.com/#System.Data/fx/src/data/System/Data/SqlClient/SqlParameter.cs
=> See SqlParameter.CloneHelper
.Net Core Code
https://github.com/dotnet/corefx/blob/release/3.1/src/System.Data.SqlClient/src/System/Data/SqlClient/SqlParameterHelper.cs#L185
This code is responsible for the bug - why is the type name / db type not copied?
For good measure the new source (Microsoft.Data.*)
https://github.com/dotnet/SqlClient/blob/ba2dbfed0e3a080caf9db9ca154ed4479a41cc53/src/Microsoft.Data.SqlClient/netcore/src/Microsoft/Data/SqlClient/SqlParameter.cs#L999
Here it seems to be fixed again (e.g. type name is correctly copied)
Now we can't upgrade to Microsoft.Data.* and instead need to use System.Data.* because we are still using EF6 with .net core

@Inspyro
System.Data.SqlClient in .NET Core (CoreFx) is no longer actively maintained and only receives important security fixes.
We only actively work on Microsoft.Data.SqlClient, which seems to be fixed for your use case, so there's nothing we can do to help you for System.Data.SqlClient. For anyone else encountering this issue, your repro and description provided would be helpful!

Thanks for the quick reply
Is there any official guidance that states that an Upgrade to EF Core is required/recommended when upgrading to .Net Core?
Until now I assumed using EF6 with .Net Core was officially supported.

I cannot provide that unfortunately, but dotnet/efcore would be the right place to ask about this.
I can only speak for SqlClient perspective, and that Microsoft.Data.SqlClient is the new officially supported SqlClient library.
https://devblogs.microsoft.com/dotnet/introducing-the-new-microsoftdatasqlclient/

Cool, thank you @Inspyro for this issue, I've wondered how to build a repro sample, but you've already done it :) .
We're having this issue while migrating from .NET Framework 4.8 to .NET 5. In .NET Framework 4.8, the code posted by @Inspyro works alright, in .NET 5, it's broken. We also cannot move to Microsoft.Data.SqlClient because we have a lot of code and dependencies that use the old APIs. We don't need new features or new fixes, we just need parity regarding correctness between .NET Framework and .NET 5. Can we get this?

@fschmied

because we have a lot of code and dependencies that use the old APIs.

Could you provide some examples of that? Why are you unable to simply switch to M.D.S. - it is just a namespace change?

we just need parity regarding correctness between .NET Framework and .NET 5.

Yes, if you use M.D.S. with .NET 5

because we have a lot of code and dependencies that use the old APIs.
Could you provide some examples of that? Why are you unable to simply switch to M.D.S. - it is just a namespace change?

Well, it's a very large project using Entity Framework 6, plus an internal netstandard 2.0 library that uses S.D.S (that we need to integrate with at the S.D.S level), plus a third-party component that doesn't have support for M.D.S. yet.

we just need parity regarding correctness between .NET Framework and .NET 5.
Yes, if you use M.D.S. with .NET 5

So is it official policy that we need to expect new bugs in S.D.S in .NET 5 (i.e., bugs not present in .NET Framework) and this is not going to be fixed? I'm a bit surprised about this as I thought the .NET Framework -> .NET 5 transition is meant to be as smooth as possible. But if that's the official policy, it's good to know, then we may need to reprioritize.

@fschmied from the EF perspective, non-Core EF6 is archived (a bit similar to System.Data.SqlClient); it's supported in the sense that security patches would be released, but that's generally it. There definitely aren't any plans to allow non-Core EF6 to work with the newer Microsoft.Data.SqlClient - you'd have to switch to EF Core if you want that.

@roji Thanks for the info. From our perspective, we just want (need) to switch from .NET Framework 4.8 to .NET 5 at the moment, but are facing this one specific compatibility issue (due to a .NET Core/5 only bug) in System.Data.SqlClient.

@cheenamalhotra From the blog post you linked to earlier:

We have no intention of dropping support for System.Data.SqlClient any time soon. It will remain as-is and we will fix important bugs and security issues as they arise. If you have a typical application that doesn’t use any of the newest SQL features, then you will still be well served by a stable and reliable System.Data.SqlClient for many years.

Isn't this an important bug, as it breaks compatibility between .NET 4.8 and .NET 5 and makes the transition more difficult?

@roji

There definitely aren't any plans to allow non-Core EF6 to work with the newer Microsoft.Data.SqlClient - you'd have to switch to EF Core if you want that.

I just did that, sorry!

Isn't this an important bug, as it breaks compatibility between .NET 4.8 and .NET 5 and makes the transition more difficult?

It truly depends on audience impact, and we hadn't heard of this need to fix in SDS until now. The decision to accept this fix is in dotnet/corefx team hands, but they would only accept if there is a very strong impact on a very large audience, and/or for a premium customer. That's why we too require a strong case to take forward with them. As @roji mentioned, "non-Core EF6 is archived (a bit similar to System.Data.SqlClient)" it implies the EF6 framework isn't in mainstream support area anymore.
Also, System.Data.SqlClient was last built and shipped with .NET Core 3.1 SDK as it was maintained as a runtime library earlier, hence it's serviced under dotnet/corefx and is tied to .NET Core 3.1 SDK release cycle. It was later removed from dotnet/runtime and has never been released with the .NET 5 SDK.
Microsoft.Data.SqlClient is the new out of the box library that's an independent and has no ties to .NET SDK release cadence.

@cheenamalhotra I.e., if I understand correctly, this is probably not the right repo to report this bug?
Would the right repo then be dotnet/corefx? Or rather dotnet/runtime?

I would say dotnet/runtime, but we will be communicating again as it would come to our queue, so I'd let it stay here.
cc @David-Engel @danmoseley for any additional insights upon this request for backporting to System.Data.SqlClient.

The bar for backporting to servicing (eg., .NET 3.1 effectively in this case) would likely be "high impact for significant number of customers without reasonable workaround". Of course risk of introducing a bug has to be balanced. Another possible consideration in this case is that there is no newer version to move to, unless you make code changes to move to M.D.SqlClient.
We would largely rely on the judgement of area owners like @cheenamalhotra to evaluate against those criteria.

Oh I see @cheenamalhotra essentially said the same thing.
I don't see mention here of whether the fix is known, and how confident we are in it and the safety of the fix?

About the fix: The System.Data.SqlClient version used by .NET Framework does not have this bug. As analyzed by @Inspyro , the fix is probably in comparing and aligning the SqlParameter.CopyTo implementation with the .NET Framework version:

.NET Core: https://github.com/dotnet/corefx/blob/release/3.1/src/System.Data.SqlClient/src/System/Data/SqlClient/SqlParameterHelper.cs#L185
https://referencesource.microsoft.com/#System.Data/fx/src/data/System/Data/SqlClient/SqlParameter.cs,1043

I should also add that blocking migration from .NET Framework is also a factor that supports a fix decision. @cheenamalhotra I think this will come down to risk/cost. Thoughts?

@danmoseley @cheenamalhotra
To add a little history here, this was originally fixed through issue #67 (which was originally filed against SDS) in the MDS 1.0 preview timeframe (mid-2019). The code change is very small and has by now been pretty widely used in the MDS releases since then (plus NetFx). This would be a low-risk change in SDS, in my opinion:

Plus the corresponding test: https://github.com/dotnet/SqlClient/blob/v1.0.19239.1/src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/ParametersTest.cs#L103
As for cost, it would be minimal for the SqlClient team to submit the PR. I don't know what the cost is on the runtime side for turning the release wheels for SDS, which hasn't happened in about a year.
I would support back porting since the transparent loss of precision is not something someone would notice immediately and could very well be happening to more customers than we are aware of.

If you want to proceed @cheenamalhotra @David-Engel please throw up a PR against https://github.com/dotnet/corefx/tree/release/3.1 with the usual template and the label servicing-consider.


alternate clock configuration for the stm32 ospi driver

For stm32 MCUs, this PR is configuring the clock source for the ospi peripheral based on the definition of the DTS.
The clock source of the Octospi is set by a bit field in one independent clock configuration register of the  RCC.
In the clock control driver, the alternate clock source of the peripheral is defined by the macro OCTOSPI_SEL() or OSPI_SEL() depending on the serie. This is valid for any stm32 device with octospi peripheral.
The DTS of the board gives the clock source to be applied like:

stm32l562e_dk
b_u585i_iot02a

Signed-off-by: Francois Ramu<EMAIL_ADDRESS>

refine with the new  STM32_DT_CLOCKS macro from  https://github.com/zephyrproject-rtos/zephyr/pull/46842

rebase after merge of https://github.com/zephyrproject-rtos/zephyr/pull/46842


AttributeError: 'ASRServerExecutor' object has no attribute 'max_len'

Describe the bug
asr_inference 服务异常
To Reproduce
Steps to reproduce the behavior:

使用 paddlespeech_server 启动 asr_inference 服务
调用 asr_inference 服务

Expected behavior
服务正常工作
Screenshots
[2022-05-31 09:14:27,188] [   ERROR] - 'ASRServerExecutor' object has no attribute 'max_len'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/paddlespeech/cli/asr/infer.py", line 365, in _check
    if audio_duration > self.max_len:
AttributeError: 'ASRServerExecutor' object has no attribute 'max_len'
[2022-05-31 09:14:27,189] [   ERROR] - can not open the audio file, please check the audio file format is 'wav'.
                  you can try to use sox to change the file format.
                  For example:
                  sample rate: 16k
                  sox input_audio.xx --rate 16k --bits 16 --channels 1 output_audio.wav
                  sample rate: 8k
                  sox input_audio.xx --rate 8k --bits 16 --channels 1 output_audio.wav

[2022-05-31 09:14:27,189] [    INFO] - file check failed!
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/paddlespeech/server/restful/asr_api.py", line 71, in asr
    asr_engine.run(audio_data)
  File "/usr/local/lib/python3.8/dist-packages/paddlespeech/server/engine/asr/paddleinference/asr_engine.py", line 215, in run
    logger.info("inference time: {}".format(infer_time))
UnboundLocalError: local variable 'infer_time' referenced before assignment

same as https://github.com/PaddlePaddle/PaddleSpeech/issues/1932
fixed in https://github.com/PaddlePaddle/PaddleSpeech/pull/1930
please git clone PaddleSpeech, cd PaddleSpeech, and pip install .  to install develop paddlespeech

This issue is solved. Thank you for a quick reply.


Test names with parentheses won't run in specs2

Seems that specs2 sanitizes () to [] due to some junit displaying limitation:
https://github.com/etorreborre/specs2/blob/5d14719c4438ea01cf0c7cc7c4a52c1d8fd6c29b/junit/shared/src/main/scala/org/specs2/reporter/JUnitDescriptions.scala#L106-L111
Test names containing parentheses should be sanitized to match specs2.

I do see that specs2 changes () to [], but I believe that's just for displaying, not for actually filtering the tests. If I have a test with parentheses (e.g. "foo should (bar)" in { ...), it is selected and executed properly when I use parentheses in the test filter (bazel test "--test_filter=com.example.Test#\QFoo.bar should::foo should (bar)\E$") and is not selected or executed when I use braces in the test filter (bazel test --test_filter=com.example.Test#\QFoo.bar should::foo should [bar]\E$"), i.e. changing () to [] results in the specs2 specification not being selected instead of being selected


[BUG]: typed parameter compatibility issues

Describe the bug
When trying to implement PHP Domain Parser
https://github.com/jeremykendall/php-domain-parser/blob/92709ef1e58ef57e24571a1129b112f58b596976/README.md
Towards the bottom, you can see an implementation using a PsrStorageFactory. This first param is typed as Psr\SimpleCache\CacheInterface
Trying to use this proxy will cause this error on a bunch of functions
Declaration of Phalcon\Proxy\Psr16\Cache::get($key, $default = null) must be compatible with Psr\SimpleCache\CacheInterface::get(string $key, mixed $default = null): mixed

To Reproduce
Steps to reproduce the behavior:
            $redisCache =  new \Phalcon\Cache\Adapter\Redis(
                $this->getShared('serializerFactory'),
                $redisConfig
            );

            $factory = new PsrStorageFactory(
                new \Phalcon\Proxy\Psr16\Cache($redisCache),
                new \GuzzleHttp\Client(),
                $requestFactory
            );

Expected behavior
For the proxy to to work with Psr\SimpleCache\CacheInterface
Details

Phalcon version: 5.3.1
PHP Version: 8.1
Operating System: Ubuntu 20
Server: Apache

Resolved in https://github.com/phalcon/proxy-psr16/pull/6
Thank you @ALameLlama . I will update composer and make a new release.


[Enhancement][recycle-bin] unpartition table need to keep in recycle bin in insert overwrite case

follow up for
Issue Number: close #xxx
if a insert overwrite happens to un-partitioned table, partition info needs to be kept inside the recycle bin. (without this tablets may leak)
once the data kept in recycle bin, it can be recovered using below step.
recover the partition to table as another partition in the table.
create another temp user table , and select the recover partition info and insert to it.
replace original user table from  temp user table.
drop user table force.

run buildall


How to send a custom Date Field via API

Hi, I have created a custom field in Date format.
How do I update this field to a contact via API? The field I'm trying to update is field [17].
Ex:
parameters = {
         "contact": {
         'phone': '999',
                 "field [17]": "2020-04-01T00: 00: 00-00: 00"
         }
         }
         print ('I pass the parameters:' + str (parameters))
         answer = ac_client.contacts.update_a_contact (idMarketing, parameters)

Is this issue still open? Looking for a similar usecase to update multiple custom fields that are not being set by the contact update api. @ingmferrer is there a fix incoming for the same?

upvote.  Looking to write and access datetime and multiselect custom fields on Accounts, Contacts, and Deals

Is this package still being updated? We're Looking for a solution as well.

Is this package still being updated? We're Looking for a solution as well.


[WIP] internal/util/projutil: standardize operator type checks

Description of the change:

internal/util/projutil/project_util.go: functions for detecting operator type
internal/util/projutil/project_util.go: remove scaffold dependency
cmd/operator-sdk: return projutil.ErrUnknownOperatorType if no valid operator type found

Motivation for the change: new functions in projutil will help type checking projects for print-deps in #1001. Code that checks operator type should return a standard error.

/retest


I18n translation

What steps will reproduce the problem?
Default labels not translated
What is the expected result?
$label = ArrayHelper::remove($toggleButton, 'label', Yii::t('yii/bootstrap5', 'Show'));

What do you get instead?
https://github.com/yiisoft/yii2-bootstrap5/blob/97a3b653b74f6dbff799d8cbf2acc716536b6d82/src/Modal.php#L239
Additional info

Q
A

Yii vesion
2.0.*

PHP version
any

Operating system
any

I'm not sure if this makes sense to implement. It will break the widgets if the translations are not configured in the application.
@bizley, @samdark What's your opinion?

It shouldn't break the app but it would require bootstrapping the new translation category. I would keep it as it is especially that user can provide own labels. Unless somewhere there is no option to do that - in that case we need to enable it.

@simialbi we can configure translation in BootstrapAsset::init(), require only English version. users translates messages in application - it's better/handy than redefine labels for every widget instance.

@simialbi if translation isn't found it falls back to the message itself which is usally English (in case of Yii2). I think it is fine to implement it.

@samdark There is no fallback:
https://github.com/yiisoft/yii2-bootstrap5/actions/runs/1657857625

@samdark @bizley
May be better to roll back?

What about adding the new category in bootstrapping process?

Good idea. But how do you plan to achieve that without Module?

Like this: https://www.yiiframework.com/doc/api/2.0/yii-base-bootstrapinterface although I have never used it before.

Yes that's possible, but where to call it? Still needs an addition in configuration...

The first approach is mainly used by extensions and is managed by the Composer installation process. You mainly need to list the bootstrapping class of your extension in the composer.json file like following...
If the extension is installed, the bootstrap information will be saved in yii\base\Application::$extensions.

@simialbi to roll back what exactly?

@samdark This commit:
https://github.com/yiisoft/yii2-bootstrap5/commit/cb48dd6e4529b3d20f99e0020db5bdcd7239aea5

@bizley seems to work: https://test.dev.tonic.ag/site/test-bootstrap
extensions.php
  //[...]
  'yiisoft/yii2-bootstrap5' =>
  array (
    'name' => 'yiisoft/yii2-bootstrap5',
    'version' => 'dev-master',
    'alias' =>
    array (
      '@yii/bootstrap5' => $vendorDir . '/yiisoft/yii2-bootstrap5/src',
    ),
    'bootstrap' => 'yii\\bootstrap5\\i18n\\TranslationBootstrap',
  ),
  //[...]

translated rendered alert (aria-label):
<div id="w12" class="alert-dark alert alert-dismissible" role="alert">

A simple dark alert—check it out!
<button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Schliessen"></button>

</div>

@simialbi still need a rollback or no?

@samdark No seems to work with bootstrapping via composer.jon


can not connect to dockershim.sock

criproxy can not connect to dockershim.sock and dockershim can not start with following errors:
Connecting to docker on unix:///var/run/docker.sock
client.go:92] Start docker client with request timeout=2m0s
cni.go:189] Unable to update cni config: No networks found in /etc/cni/net.d
docker_service.go:208] Docker cri networking managed by kubernetes.io/no-op
docker_service.go:225] Setting cgroupDriver to cgroupfs
dockershim[14093]: error: listen tcp <IP_ADDRESS>:12345: bind: cannot assign requested address
my host ip is <IP_ADDRESS>

What are the arguments that you pass to kubelet and dockershim?

kubelet:
KUBELET_EXTRA_ARGS="--experimental-cri --feature-gates=DynamicKubeletConfig=true --anonymous-auth=true --container-runtime=remote --container-runtime-endpoint=/run/criproxy.sock --image-service-endpoint=/run/criproxy.sock --enable-controller-attach-detach=false"
dockershim:
--api-servers=http://<IP_ADDRESS>:8080 --address=<IP_ADDRESS> --hostname-override=node2 --allow-privileged=true --pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest --cgroup-driver=cgroupfs --cluster-dns=<IP_ADDRESS> --cluster-domain=cluster.local.

Do you pass anything besides $KUBELET_EXTRA_ARGS to kubelet? E.g. if you have deployed your cluster with kubeadm, you also will have something like KUBELET_NETWORK_ARGS=--network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin You need to pass these args to dockershim, too.

all args of ps aux|grep kubelet are passed to dockershim; config of kubelet are as follows:

kubernetes kubelet (minion) config

The address for the info server to serve on (set to <IP_ADDRESS> or "" for all interfaces)
KUBELET_ADDRESS="--address=<IP_ADDRESS>"

The port for the info server to serve on
#KUBELET_PORT="--port=10250"

You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=node2"

location of the api-server
KUBELET_API_SERVER="--api-servers=http://<IP_ADDRESS>:8080"

pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"

Add your own!
KUBELET_ARGS="--cgroup-driver=cgroupfs --cluster-dns=<IP_ADDRESS> --cluster-domain=cluster.local."
KUBELET_EXTRA_ARGS="--experimental-cri --feature-gates=DynamicKubeletConfig=true --anonymous-auth=true --container-runtime=remote --container-runtime-endpoint=/run/criproxy.sock --image-service-endpoint=/run/criproxy.sock --enable-controller-attach-detach=false

How did you configure CNI on your nodes?

use flannel as network plugin

What's in /etc/cni/net.d ?

10-flannel.conf:
{"name":"flannel",
"type":"flannel"}
add kubelet-network-args to kubelet and dockershim, but the problem remains

what's the ip <IP_ADDRESS>:12345 mean? in my opinion dockershim should listen at localhost:12345, if so , how can i change the ip to my host ip ?

Looks like outdated issue. If it's still relevant - please create new one about that in https://github.com/Mirantis/criproxy repo.


duckling_http suddenly giving wrong year

Rasa Core version: 0.10.3
Python version: Python 3.6.6
Operating system (windows, osx, ...):
Issue:
On thursday last week duckling was working fine. Now duckling ner is always returning the year 1970.
Example:

d. 12 juli klokken 16:30
'value': '1970-07-12T16:30:00.000+00:00'
'extractor': 'ner_duckling_http'}]'

Or "today":

i dag
'value': '1970-01-18T00:00:00.000+00:00',
'extractor': 'ner_duckling_http'}]'

However it is not completely broken. Giving explicit date and the time "quarter past 8"

1/1/2018 klokken kvart over 8
'value': '2018-01-01T08:15:00.000+00:00'
'extractor': 'ner_duckling_http'}]'

If I test duckling separately it works as expected; the year is 2018. "i dag" again is "today"

curl -XPOST http://<IP_ADDRESS>:8000/parse --data 'locale=da_DK&text=I dag'

"value":"2018-08-06T00:00:00.000-07:00"

can you please open this issue in the NLU repo instead?


Refactor body of periodic and delete performance test cases of splinter_test to utilize shared functions carved out in #334

Issue #334 introduces refactoring of body splinter_test function into sub-functions. Apply those to chunks of test_splinter_periodic() and test_splinter_delete(), so that we can reduce code duplication.

HI, @ajhconway - Qs about refactoring test_splinter_periodic() - not urgent; take your time.
I was working on refactoring test_splinter_periodic() to use previously refactored / modularized functions, and see that it seems to reduce to the following sequence of perf-execution workloads:

splinter_perf_inserts()
10 repeat_round iterations to call (a newly refactored) splinter_do_perf_inserts(); i.e. just run the n-insert threads again. But, nothing seems to be obviously changing across each iteration of the insert workload in this loop (from /main):

1440 platform_status
1441 test_splinter_periodic(trunk_config    *cfg,
[...]
1581    for (uint64 repeat_round = 0; repeat_round < 10; repeat_round++) {
1582       platform_default_log("Beginning repeat round %lu\n", repeat_round);
1583       platform_error_log("Beginning repeat round %lu\n", repeat_round);

splinter_perf_lookups()
splinter_perf_range_lookups()

Other than step (2), the work load executed is identical to what's happening with --perf option; i.e. test_splinter_perf().
Unless I am making some obvious mistakes, missing to see some "subtle conditions" in the periodic perf function, there does not seem to be anything new being added with --periodic option.
Do you recall what's diff or special about this --periodic variation of --perf option? Am I missing something with this refactoring?

The idea behind test_splinter_periodic was to do a bunch of overwrites in order to test the performance. I don't recall exactly why. Perhaps @rtjohnso remembers.


Adds normalize parameter to ToTensor operation

It should be the case that the divide by 255 is an optional step that is taken when converting to a pytorch tensor.  The parameter should default to True however to preserve backwards compatibility.

Hi @xksteven, I disagree with your premise

It should be the case that the divide by 255 is an optional step

The native image type of torchvision is a torch.Tensor with dtype==torch.float. As its normal for floating point pixels the values should be from the interval [0.0, 1.0]. Thus, the division should not be optional.
One could then ask why we use values from this interval and not [0.0, 255.0]. Since all real closed intervals (ignoring some edge cases that are not important for my opinion) can be mapped to each other it is indifferent which interval is used. Thus, I think it is reasonable to use the simplest of them: [0.0, 1.0].
Since 255.0 as a upper limit is just a remnant of the max value of images stored with a color depth of 1 byte, I think its fair to ask, why you want to use this interval.

If you want to use [0.0, 255.0] as default value range you could simply create a custom transform:
class FloatToUint8ValueRange:
    def __call__(self, img):
        return img * 255.0

and bundle it after ToTensor:
from torchvision import transforms

transform = transforms.Compose([transforms.ToTensor(), FloatToUint8ValueRange()])

If we want to include this, I suggest that we at least change the name of the parameter, since normalize here could be easily confused with the z-score normalization.

Hello @pmeier,
One particular use case I was dealing with is loading semantic segmentation maps which are conveniently stored as images and can be loaded nicely as labels with this transform except for the division by 255.
I thought by making the division optional it will allow for the loading of these and other types of images that do not necessarily follow that the maximum value is that if 255. It would this allow for much easier convenience instead of forcing to add another transform.
I agree that the use of the word normalize could be confusing. I thought that the word "standardize" would be worse. Do you have an idea of what it should be called?
Thanks for your comments!

One particular use case I was dealing with is loading semantic segmentation maps which are conveniently stored as images and can be loaded nicely as labels with this transform except for the division by 255.

I'm not sure if I understood you right here, but I fail to see why the segmentation maps have to be in the interval [0.0, 255.0]. What are you doing with them that is not possible or at least significantly harder than with maps in the interval [0.0, 1.0]? Could you elaborate?

I thought by making the division optional it will allow for the loading of these and other types of images that do not necessarily follow that the maximum value is that of 255. The parameter would allow for much easier convenience instead of forcing to add another transform.

From this PR I think you load the images with PIL and pass them through the ToTensor() transform to get a torch.Tensor. Couldn't you simply use the ComposedTransform from my comment above instead? I acknowledge that in theory it is not a good idea to first divide by a value and simply multiply by that value again in the next step. Still, I can't imagine a scenario where this could lead to a bottleneck.

I agree that the use of the word normalize could be confusing. I thought that the word "standardize" would be worse. Do you have an idea of what it should be called?

If I had to, and I'm not in favor of that, I would go with something like upper_value_limit=1.0 and simply multiply by that factor once the image is in the interval [0.0, 1.0]. For your case you could then set upper_value_limit=255.0.

I'm not sure if I understood you right here, but I fail to see why the segmentation maps have to be in the interval [0.0, 255.0]. What are you doing with them that is not possible or at least significantly harder than with maps in the interval [0.0, 1.0]? Could you elaborate?

The semantic segmentation maps are themselves the labels per pixel. So each pixel value represents a class encoded by the integer value 0,1,2 ..., N.  This can be loaded and then passed directly into the loss function (the values might need to be converted into type long first in some cases).

From this PR I think you load the images with PIL and pass them through the ToTensor() transform to get a torch.Tensor. Couldn't you simply use the ComposedTransform from my comment above instead? I acknowledge that in theory it is not a good idea to first divide by a value and simply multiply by that value again in the next step. Still, I can't imagine a scenario where this could lead to a bottleneck.

Honestly I agree I think this is a very small pull request and it introduces minimal changes that I personally would like but maybe suboptimal.  It isn't a bottleneck I just thought it would be better to make what the ToTensor operation slightly more transparent.  It felt weird to me that this one function was doing several functions together not only was it converting the images into a pytorch Tensor but it was also normalizing them in the [0.0, 1.0] range.  I thought it might be useful to keep the first functionality and make the second functionality optional.
If the changes are too small and inconvenient I'm open to just closing the pull request.

Codecov Report

Merging #2060 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2060      +/-   ##
=========================================
- Coverage    0.48%   0.48%   -0.01%
=========================================
  Files          92      92
  Lines        7411    7415       +4
  Branches     1128    1130       +2
=========================================
  Hits           36      36
- Misses       7362    7366       +4
  Partials       13      13

Impacted Files
Coverage Δ

torchvision/transforms/transforms.py
0% <0%> (ø)
:arrow_up:

torchvision/transforms/functional.py
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ed2fa3...cd6e66f. Read the comment docs.

The semantic segmentation maps are themselves the labels per pixel. So each pixel value represents a class encoded by the integer value 0,1,2 ..., 255. This can be loaded and then passed directly into the loss function (the values might need to be converted into type long first in some cases).

If you need to convert them to torch.long anyway, can't you just do the conversion to [0.0, 255.0] there? If yes, you do not need a special transformation for the segmentation masks during the loading.

[...] I just thought it would be better to make what the ToTensor operation does slightly more transparent. It felt weird to me that this one function was doing several functions together. Not only was it converting the images into a pytorch Tensor but it was also normalizing them in the [0.0, 1.0] range. The main reason it felt weird because the name ToTensor doesn't seem to imply the normalization functionality you have to look at the documentation for that part to become apparent.

I think right now we aren't communicating the native image type of torchvision very well since questions about it (or a PR for that matter) keep popping up from time to time. Do you have an idea how to improve that?

If the changes are too small and inconvenient I'm open to just closing the pull request.

Don't give up yet: although I'm in favor of closing this PR, I have no say here. We'll have to wait for a member of PyTorch to drop in and settle this.

I think right now we aren't communicating the native image type of torchvision very well since questions about it (or a PR for that matter) keep popping up from time to time. Do you have an idea how to improve that?

Maybe there could be an additional parameter to resulting output tensor?  If one does
transforms.Compose([ ToTensor(), ToPILImage() ]
Information about the mode has been lost in the process and it does not return the same PIL Image that one started with. I believe it may result in an error in some cases (although I'd have to do more tests to confirm it's true about throwing an error).
It would be nice if those two operations resulted in the identity operation to be able to easily convert back from images to tensors or vice-versa.

Hi,
Thanks for the PR!
I agree with @pmeier comments, and this is something that has been come up a few times already. The underlying issue is that ToTensor has too many implicit assumptions, and can lead to a number of surprising behaviors, see https://github.com/pytorch/vision/issues/546 https://github.com/pytorch/vision/issues/856 for a few examples.
I think we should re-design this abstraction altogether, so that the contract is clear from the beginning.
I propose to split ToTensor in two:

the as_tensor part, which just converts a PIL Image into a torch.Tensor of the original dtype
the casting part, which could follow TensorFlow's convert_image_dtype.

This way, the semantics of the functions are clear, and there are no surprises for the users.
Also, when the image reading PRs are merged in torchvision, the as_tensor part will become optional, and only the convert_image_dtype will be necessary.
Thoughts?

So as_tensor would keep to original dtype, correct? For example "normal" PIL image would result in dtype=torch.uint8 while an image with mode F would be dtype==torch.float32 right away.
convert_image_dtype would take in a tensor of arbitrary dtype and return dtype==torch.float32 with values from [0.0, 1.0], correct?

So as_tensor would keep to original dtype, correct? For example "normal" PIL image would result in dtype=torch.uint8 while an image with mode F would be dtype==torch.float32 right away.

I like this approach.

convert_image_dtype would take in a tensor of arbitrary dtype and return dtype==torch.float32 with values from [0.0, 1.0], correct?

I think  the convert_image_dtype should not also do the normalization. It should only do the potential scaling from max of one dtype to another and as tensorflow does they add a parameter to called saturate if the user wishes to avoid overflow or underflow problems when converting from max float to max int for example.
I really don't like the implicit scaling of values without any warning to the user that the normalization is occurring.

I think the convert_image_dtype should not also do the normalization.

If as_tensor and convert_image_dtype should replace to_tensor, one has to implement the scaling. So if you want a conversion from dtype==torch.uint8 to dtype==torch.float32 you will get values from [0.0, 1.0] even if we split the functionality of to_tensor. If I read that correctly, tensorflow does it the same.
If you just want to change the dtype without any scaling just call .to(dtype) on your image.

If as_tensor and convert_image_dtype should replace to_tensor, one has to implement the scaling. So if you want a conversion from dtype==torch.uint8 to dtype==torch.float32 you will get values from [0.0, 1.0] even if we split the functionality of to_tensor. If I read that correctly, tensorflow does it the same.
If you just want to change the dtype without any scaling just call .to(dtype) on your image.

You're right they do the scaling.  From their documentation it was not apparent to me that they were doing the scaling because MAX was described as largest positive representable number

Image data stored in integer data types are
expected to have values in the range [0,MAX], where MAX is the largest
positive representable number for the data type.

I'll withdraw my criticism as it appears everyone else is doing the conversion and normalization, and the "as_tensor" function would handle my own use cases as well.

@pmeier correct about your first point. One thing to consider is if we want to do the transposition from HWC to CHW in as_tensor, I think we should, but adding the memory_layout information so that there is no copy.
About your second point:

convert_image_dtype would take in a tensor of arbitrary dtype and return dtype==torch.float32 with values from [0.0, 1.0], correct?

Partially, as the user can specify if he wants to convert to a float type (the default) or a different type.
I would be happy to review PRs addressing those two points. And for ToTensor, I would leave it as is, and mark it as deprecated. Maybe we should also revisit ToPILImage on the way.

I'll close the pull request and begin working on the new function :)

Thanks @xksteven !
Just a heads up that @pmeier is already working on the convert_image_dytpe in #2078


Would welcome any improvements here (docs or data generate).

I might recommend removing the keywords or throwing an error/warning if they are used until the logic has actually been implemented.

I think our (unstated) policy is to match the signature, but raise if the keyword is specified but not implemented. So a PR checking for that would also be most welcome.

It seems reasonable to me to not match the signature if we genuinely don't
support the keyword
On Mon, Jul 30, 2018 at 7:27 AM, js3711<EMAIL_ADDRESS>wrote:

Thanks Tom, I can work on that later.

Raise if the keyword is specified but not implemented
Update doc strings
Update dask-ml documentation examples that use the keywords. Things
like: https://dask-ml.readthedocs.io/en/latest/examples/predict.
html?highlight=make_classification
https://dask-ml.readthedocs.io/en/latest/examples/predict.html?highlight=make_classification

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-ml/issues/323#issuecomment-408882607, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszJeMhOSG-9Lc7FrJaA2zC-A1h6Vaks5uLxfAgaJpZM4VmS7V
.

Scikit-Learn functions often have many keywords.  Raising informative
NotImplementedErrors for all of them sounds potentially painful
On Mon, Jul 30, 2018 at 8:03 AM, Matthew Rocklin<EMAIL_ADDRESS>wrote:

It seems reasonable to me to not match the signature if we genuinely don't
support the keyword
On Mon, Jul 30, 2018 at 7:27 AM, js3711<EMAIL_ADDRESS>wrote:

Thanks Tom, I can work on that later.

Raise if the keyword is specified but not implemented
Update doc strings
Update dask-ml documentation examples that use the keywords. Things
like: https://dask-ml.readthedocs.io/en/latest/examples/predict.ht
ml?highlight=make_classification
https://dask-ml.readthedocs.io/en/latest/examples/predict.html?highlight=make_classification

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask-ml/issues/323#issuecomment-408882607, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AASszJeMhOSG-9Lc7FrJaA2zC-A1h6Vaks5uLxfAgaJpZM4VmS7V
.

@TomAugspurger @mrocklin  Just a heads up that I'm working on closing the gaps here.


Setup questions

I know in your readme you say you don't feel the need to explain this step, but I can't find an example .cfg file in your github to use as a template.
Also, on line 1 of admintray-subaccmenu.inc.js it makes the following call:
if (ccsd.util.hasAnyRole('admin','root_admin')) {
Even if we comment this if statement (and the closing curly bracket) out, as we don't have the ccds.util declared anyway that we can tell.
We have put all the files (apart from the .inc.js one) into a folder on a web server, and pointed url: at the URL, and still it gives errors.

Hi Dave, apologies for missing this for so long.
There is no cfg to setup within the script itself. There is a cfg object in the script, but the only setting is recursive, and you'll likely want that.
https://github.com/robert-carroll/ccsd-canvas/blob/master/admintray-subaccmenu/admintray-subaccmenu.js#L9
The utility methods we use are found available, https://github.com/robert-carroll/ccsd-canvas/tree/master/util
You're probably better off with this...
<EMAIL_ADDRESS>if (['admin','root_admin'].some(a => ENV.current_user_roles.includes(a))) {
    // ...
}

I'd need to know more about the errors or the server you're hosting the files on to troubleshoot. Double check that you're using HTTPS and have CORS enabled.


Support external CUPS server

It is sometimes better to have a CUPS server that is located on a different server. (Like when the CUPS server is running nearby the printer on a different floor.) There is currently no supported method of setting the IP address and port of the CUPS server for commands lp and lpstat in mars. It could be beneficial to add an additional field in .env which allows arbitrary command line arguments to be inserted in front of the -d and -W directive respectively.

#5

My original description did not fully show the importance of supporting external CUPS servers.
We have deployed a Raspberry PI next to the printer and connected the printer and RPI with a USB cable. The CUPS server is running on the RPI, so Mars needs to connect to that CUPS server. We have made a secure connection between the RPI and Mars's PHP FPM (with something similar to a VPN).
The changes had two main goals. One of them was to make our systems more resilient against attacks. A less important one was that from now on printers without networking capabilities would be also a great option for the dormitory.
Similarly to #364, the changes that were done in the deployment branch seem to allow these settings. And I also plan to make the proper changes on the upstream development branch. Unfortunately, I did not have enough time to do it yet.
Therefore, I would like to reopen this issue for a potential second round of evaluation.


WIP: [service-bus] Add in abortSignal support in createBatch, subscribe and receiveMessages

Adding in abortSignal support to:

subscribe
receiveMessages
createBatch

There were some refactors to make this more consistent (testing is still happening, so this is just a draft):

Made the openLink (equivalent of the old .open()/_init() calls in MessageSender and MessageReceiver) a common function with no LinkEntity dependency (easier for testing)
Added in intermediate aborts into that function
Made a few abort signal helpers (in utils.ts) to make it easier to register with an abort signal and properly clean up (fixing a bug I left in MessageSender where I didn't clean up my abort signal handlers).

/azp run js - servicebus - tests

/azp run js - servicebus - tests

/azp run js - servicebus - tests

/azp run js - servicebus - tests

Hey @richardpark-msft,
I have reviewed and left my first round of comments, please take a look.
If it is possible, then I'd much prefer the work being done here to be done in phases in two separate PRs. Phase 1 would involve the utilities that you added for timeout and abort signal with them being used from link creation code path, Phase 2 would involve introducing openLink to consolidate the opening of AMQP links across senders and receivers.
I havent looked at the streaming and batching receiver yet

Created PR#9138 for the abort helpers.

Resolved all my comments on the utils.ts file as they have either been addressed in #9138 or I have copied the conversation over to that PR

Closing this PR - the work has migrated into two separate PRs:
#9233 - sender
#9284 - receiver


Usage of Admin API

Even though there is currently a client for Admin API there is no option to obtain the JWT token for the Admin, which is required for the majority of API's. And when obtaining it manually there is no way to set it for the client.

@NikiforovAll Yeah I'm a bit perplexed by this too. I tried setting it up how you have it in the simple sample where you technically use the same credentials for the Admin API and AuthN/AuthZ and I'm getting a 403 Forbidden trying to create a user.

We should use a separate account in order to integrate with admin API, it should have corresponding roles/permissions. I'm going to create an example for you guys. Also, feel free to contribute if you figure this out :)

I looked through the master clients for a client with a client id/secret and couldn't find anything and as far as I'm aware, the proper way to create a user is through the master realm 🤷

I'm getting a 403 Forbidden trying to create a user.

The way de Admin API is setup right now is that it uses the same client ID and client secret from the configuration, in the Keycloak instance you need to make sure the client has the manage-users role applied, I think this is worth documenting.

an example with README.md file with instructions might do the trick. I just don't have time for it atm

friends, PRs are welcome as usual

hello,can you please provide a small exemple on how to use it ?
I keep getting 401 unauthorized even when i provide a client secret

Colleagues, confirm or correct me - a new user can only be created through the admin rest api using the Bearer authorization header with an administrator access token?

Colleagues, confirm or correct me - a new user can only be created through the admin rest api using the Bearer authorization header with an administrator access token?

I don't think there's a concept of administrator access token, to be able to create a user you need to be authenticated with something that has the manage-users client role applied (for each realm that you want to manage), be it a user or a client. In the case of this library, it uses the client ID and client secret specified in your configuration, so that client needs the role applied.

Colleagues, confirm or correct me - a new user can only be created through the admin rest api using the Bearer authorization header with an administrator access token?

I don't think there's a concept of administrator access token, to be able to create a user you need to be authenticated with something that has the manage-users client role applied (for each realm that you want to manage), be it a user or a client. In the case of this library, it uses the client ID and client secret specified in your configuration, so that client needs the role applied.

Victor, I'm currently developing a service for interacting with Keycloak. I am using the latest version of Keycloak - 21.1.1, but it seems like your library uses a different version, as the endpoints contain 'auth/', which is not present in the current version. I only interact with Keycloak through the Admin REST API, and the only available option for adding a new user is by using the POST {base url}/admin/realms/{realm}/users endpoint. To do this, we need to use Bearer authorization and add an administrator access token to the request header, otherwise the request will be unauthorized and a 401 status code will be returned. In the request body, we include a JSON object with information about the user that we want to create.
For example:
curl -X POST \   'http://{keycloak-url}/admin/realms/{realm}/users' \   -H 'Authorization: Bearer {access-token}' \   -H 'Content-Type: application/json' \   -d '{         "username": "new_user",         "email"<EMAIL_ADDRESS>        "enabled": true,         "credentials": [             {                 "type": "password",                 "value": "password",                 "temporary": false             }         ]     }'
I will also say that there are no manage-users in the current version.
Therefore, could you please explain how to create a new user using client_id and client_secret with the REST API? If possible, could you also provide an example using a curl request? I couldn't figure out the code in your library, that's why I'm asking.

@timofdev the endpoints themselves do not add the /auth prefix, that is something you can add to your keycloak URL in the config (same as the examples in the README.md) or you can just leave it out like this:
{
  "Keycloak": {
    "Realm": "realm",
    "AuthServerUrl": "http://localhost:8080/",
    "SslRequired": "external",
    "VerifyTokenAudience": true,
    "Resource": "client-id",
    "Credentials": {
      "Secret": "client-secret"
    }
  }
}

Configuration remains the same, add this to Program.cs:
builder.Services.AddKeycloakAuthentication(configuration);

For the manage-users role, it's actually a Service account role, on the Clients page, select the client, in the Service accounts roles click on Assign role, filter by clients, search for manage-users and assign,

Then, to create a user, you can inject either IKeycloakUserClient or IKeycloakClient and pass a user object to the create method, example below is the code I'm using right now with Keycloak 20 (realm configured to use email as username, so I don't include it):
public async Task<ErrorOr<string>> CreateUserAsync(string email, string password, string? fullName)
{
    var (firstName, lastName) = fullName?.SplitFullName() ?? (null, null);

    var user = new User
    {
        Email = email,
        FirstName = firstName,
        LastName = lastName,
        Enabled = true,
        EmailVerified = false,
        Credentials = new[] {new Credential {Type = "password", Value = password, Temporary = false}}
    };

    using var response = await _keycloakUserClient.CreateUser(_applicationContext.Settings.Keycloak.Realm, user);

    if (!response.IsSuccessStatusCode)
    {
        _logger.LogError(
            "Error while creating Keycloak user: {@User}; Response: {@UserCreationResponse}, Response Content: {@UserCreationResponseContent}",
            user, response, await response.Content.ReadAsStringAsync());

        return Errors.Authentication.UserCreationFailed;
    }

    return response.Headers.Location?.AbsolutePath.Split('/').LastOrDefault() ?? string.Empty;
}

@victor-borges Victor, thank you very much! I found it, figured it out, and it helped! Now I can obtain an access token using client_id and client_secret, include it in the authorization header with the bearer scheme, and create a new user. The main thing, as Victor correctly mentioned earlier, is to add the 'manage-users' role to your client for the service account


Updates for Multi-Cluster docs

This PR includes:

Clean up older variables that are not used in single-key-vault environment
adds references to required parts of the tutorials that are pre-reqs to create two clusters
aligns some vnet variables to make it easy to follow
removes and rewords versions of the template.  Bedrock does not have a release or tag for 1.0 so ended up with errors when following tutorial

cc: @NathanielRose @jred55

@jmspring @jsturtevant I went ahead and resolved the issues


Adding PWM support to Pi

So, the Pi doesn't have userspace level PWM support, but you can mmap the raw registers and configure it that way, which is what this patch is doing. Tested with Raspberry Pi 3 B. I am able to set the duty cycle, change the frequency, and set the pulse width. Signals verified with logic analyzer.
Let me know if there's anything that you think needs cleaning up.
Note that the only accessible pwm is on GPIO18, which is also used by the audio subsystem.

Looks good to me!


How to generate usdt address?

I  has done the setup of Omni layer on Debian aws instance. how can i generate USDT(Tether) address?

Usdt address is btc address

Thank you @ngocbd
can you please tell me how to transfer usdt to one address to another address.

so basically according to you command should look like this " omnicore-cli omni_send addressSender addressReciver 31 "1" "

Hello, I'm new in omnicore client but as I understand 31 property identifer is assign for Usdt (tether) but
when i execute this command "omnicore-cli "omni_getproperty" 31" i'm getting this as a result
"
{
"propertyid": 31,
"name": "tokenmagic.com",
"category": "TOKENMAGIC",
"subcategory": "TOKENMAGIC",
"data": "",
"url": "Beeswax",
"divisible": false,
"issuer": "mrAVAPxdQEZxFkunh56skB6sgJa6vrfrpo",
"creationtxid": "9588ab12ef040767b1e9fe4181e294409da6cab9792f89ccc55bc621b47e2563",
"fixedissuance": true,
"managedissuance": false,
"totaltokens": "80000"
}
"
if 31 is for tether then why 31 show tokenmagic.com instead of tether.to

Because you're using testnet. It's just PropertyID.


[BUG] Error in Poetry: No module named 'libcamera'

Please only report one bug per issue!
Describe the bug
I've successfully installed picamera2 in a Poetry venv, but when I try to run a script I get an error saying No module named 'libcamera'.
To Reproduce
Install Poetry dependencies:
Rust: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Cryptography deps: sudo apt-get install build-essential libssl-dev libffi-dev python-dev
Poetry: pipx install poetry
Create a new Poetry project.
Add to pyproject.toml:
[tool.poetry.dependencies]
picamera2 = "^0.3.7"

Then poetry lock and poetry install.
And finally run a minimal script, e.g.
from picamera2 import Picamera2
import time

picam2 = Picamera2()
picam2.start()
time.sleep(3)
picam2.stop()

Expected behaviour
Just start a preview.
Console Output, Screenshots
Traceback (most recent call last):
  File "/home/nivenlab/Github/bux/bux_recorder/picamtest.py", line 1, in <module>
    from picamera2 import Picamera2
  File "/home/nivenlab/Github/bux/.venv/lib/python3.9/site-packages/picamera2/__init__.py", line 1, in <module>
    from .configuration import CameraConfiguration, StreamConfiguration
  File "/home/nivenlab/Github/bux/.venv/lib/python3.9/site-packages/picamera2/configuration.py", line 1, in <module>
    from .controls import Controls
  File "/home/nivenlab/Github/bux/.venv/lib/python3.9/site-packages/picamera2/controls.py", line 4, in <module>
    from libcamera import ControlType, Size, Rectangle
ModuleNotFoundError: No module named 'libcamera'

Hardware :
RPi4.
Additional context
It seems that it's impossible to access the underlying libcamera dependencies from within the env. Is there som way of having those dependencies being bundled with picamera2 as extras? (e.g. picamera2[venv]/picamera2[libcamera])

Seems to be the same issue as #446. I tried the proposed idea, but it doesn't resolve the issue. Worth mentioning that my Poetry environment even runs the same Python version as the global python (currently 3.9.2), not 3.10.

When you get this failure, can you check what do you have in your Python search path, and what packages you have there? I'm afraid I know nothing about poetry, does it have a way to install/import system packages?

How do you normally check that? I can surely try! I can also create an issue on Poetry, as I can't find any information about how it interacts with system packages (and I'm sure it does to some extent, as I also use OpenCV dependencies which are system-wide).

I'm guessing it might copy packages rather than point to system folders directly, though I have no particular knowledge. So the question might be whether any of those folders contain "libcamera" (I expect not). You might even be able to copy the libcamera package over yourself... though that would be complete hackery and I'm sure there are many reasons why that would be a bad thing to do.

I'll have a look to see if there's a similar issue with Poetry already, otherwise I'll start one. Thanks for the quick help!


add tests for + stream id to get last entry (v5)

Description

Describe your pull request here

Checklist

[ ] Does npm test pass with this change (including linting)?
[ ] Is the new or changed code fully tested?
[ ] Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

I don't see what value those tests add..


RuntimeError: CUDA error: device-side assert triggered

Hi Yoshitomo,
My machine has 2 TitanV + Torch 1.7.1 + Cuda11.0 + TorchVision 0.8.2
I ran python examples/image_classification.py --config configs/sample/ilsvrc2012/single_stage/kd/alexnet_from_resnet152.yaml --log log/ilsvrc2012/kd/alexnet_from_resnet152.txt
Then got error RuntimeError: CUDA error: device-side assert triggered:
2021/02/24 13:53:53     INFO    torchdistill.common.main_util   Not using distributed mode
2021/02/24 13:53:53     INFO    main        Namespace(adjust_lr=False, config='configs/sample/ilsvrc2012/single_stage/kd/alexnet_from_resnet152.yaml', device='cuda', dist_url='env://', log='log/ilsvrc2012/kd/alexnet_from_resnet152.txt', start_epoch=0, student_only=False, sync_bn=False, test_only=False, world_size=1)
2021/02/24 13:53:53     INFO    torchdistill.datasets.util      Loading train data
2021/02/24 13:53:58     INFO    torchdistill.datasets.util      dataset_id ilsvrc2012/train: 4.215242624282837 sec
2021/02/24 13:53:58     INFO    torchdistill.datasets.util      Loading val data
2021/02/24 13:53:58     INFO    torchdistill.datasets.util      dataset_id ilsvrc2012/val: 0.18817710876464844 sec
2021/02/24 13:53:59     INFO    torchdistill.common.main_util   ckpt file is not found at ./resource/ckpt/ilsvrc2012/teacher/ilsvrc2012-resnet152.pt
2021/02/24 13:54:02     INFO    torchdistill.common.main_util   ckpt file is not found at ./resource/ckpt/ilsvrc2012/single_stage/kd/ilsvrc2012-alexnet_from_resnet152.pt
2021/02/24 13:54:02     INFO    main        Start training
2021/02/24 13:54:02     INFO    torchdistill.models.util        [teacher model]
2021/02/24 13:54:02     INFO    torchdistill.models.util        Using the original teacher model
2021/02/24 13:54:02     INFO    torchdistill.models.util        [student model]
2021/02/24 13:54:02     INFO    torchdistill.models.util        Using the original student model
2021/02/24 13:54:02     INFO    torchdistill.core.distillation  Loss = 1.0 * OrgLoss
2021/02/24 13:54:02     INFO    torchdistill.core.distillation  Freezing the whole teacher model
2021/02/24 13:54:06     INFO    torchdistill.misc.log   Epoch: [0]  [    0/40037]  eta: 1 day, 21:05:22  lr: 0.0001  img/s: 11.305412092278724  loss: 7.0715 (7.0715)  time: 4.0543  data: 1.2238  max mem: 2885
/opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [25,0,0] Assertion t >= 0 && t < n_classes failed.
Traceback (most recent call last):
File "examples/image_classification.py", line 181, in
main(argparser.parse_args())
File "examples/image_classification.py", line 163, in main
train(teacher_model, student_model, dataset_dict, ckpt_file_path, device, device_ids, distributed, config, args)
File "examples/image_classification.py", line 123, in train
train_one_epoch(training_box, device, epoch, log_freq)
File "examples/image_classification.py", line 66, in train_one_epoch
metric_logger.update(loss=loss.item(), lr=training_box.optimizer.param_groups[0]['lr'])
RuntimeError: CUDA error: device-side assert triggered

Hi @PotatoThanh
I just tried to reproduce the error, but example/image_classification.py is running well on multiple GPUs with configs/sample/ilsvrc2012/single_stage/kd/alexnet_from_resnet152.yaml so far.
Could you provide 1) OS info, 2) Python ver., and 3) torchdistill ver. as well?
Also, if you have made any change on code and/or yaml config file, please share them here too.
Thank you

Hi @yoshitomo-matsubara,
I am using Ubuntu=20.04, TorchDistill=0.1.4, NvidiaDriver=450.102.04,  Torch=1.7.1, Cuda=11.0, TorchVision=0.8.2.
I did not modify anything from your code as well as yaml files. I am trying to reproduce your results on ImageNet.
Thank you!

@PotatoThanh
And which python version are you using? Your provided environment is more or less the same with mine, so it should be fine as long as you're using Python 3.6 - 3.8 and you follow this instruction for ImageNet dataset
Besides, if you'd like to reproduce the results reported in my paper, please follow the instructions under configs/official/
As noted here, all the config files under configs/sample/ are not tuned, but mostly for debugging purpose.

Thank you @yoshitomo-matsubara,
Yes, I am using Python 3.8.5 as well as your instructions for ImageNet.

Thank you for providing the info @PotatoThanh
I'm assuming you're using the latest version in this repo (currently 627abd50af720aec7061f57066f9ae7ba4466c52) for image_classification.py.
If you still face the same error, please make sure that your ImageNet folder contains 1000 sub folders only as the following error message implies that sometimes targets contains at least one class index that is out of range 0 - 999

/opt/conda/conda-bld/pytorch_1607370172916/work/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [25,0,0] Assertion t >= 0 && t < n_classes failed.

Thank you @yoshitomo-matsubara,
I found the problem. When I preprocess the ImageNet using
# Go to the root of this repository mkdir ./resource/dataset/ilsvrc2012/{train,val} -p mv ILSVRC2012_img_train.tar ./resource/dataset/ilsvrc2012/train/ cd ./resource/dataset/ilsvrc2012/train/ tar -xvf ILSVRC2012_img_train.tar for f in *.tar; do   d=basename $f .tar   mkdir $d   (cd $d && tar xf ../$f) done rm -r *.tar
There is a folder name ILSVRC2012_img_train under ./resource/dataset/ilsvrc2012/train/. Therefore, when the code loads data, it will raise error.

@PotatoThanh
That makes sense. Probably I forgot to add a couple of commands to rename and delete files or something.
Thank you for pointing out! I'll update the instructions.

Actually, I double-checked on Ubuntu 18 that tar -xvf ILSVRC2012_img_train.tar does not produce a folder ILSVRC2012_img_train/ but found some typos in commands to process validation dataset instead.
So the initial commands should be fine for training dataset.


Discord channel can not be queued.

Code: https://ptero.co/wexewiwuqo.xml
Ran by: https://ptero.co/uzikulasak.php
Output data in horizon: https://ptero.co/aqupemahim.makefile
Stacktrace: https://ptero.co/azahiqozux.shell

Note: when not implementing ShouldQueue notification runs fine. Only when implementing this interface does it not run.

+1. I'm having the same issues.

Scratch that. I forgot to add use Queueable; in addition to implements ShouldQueue. Works as intended for me


Can Spark read data using Presto JDBC?

I'm a rookie.
At present, I use Presto to complete the processing through Java program using JDBC..
Can I use spark.read().format(" JDBC ")?
How do I define Kerberos authentication in Option?
Like this.
SparkSession spark = SparkSession
.builder()
.appName("test_spark_jdbc_presto")
.getOrCreate();
    Dataset<Row> dataset = spark.read()
            .format("jdbc")
            .option("driver", "io.prestosql.jdbc.PrestoDriver")
            .option("url", "jdbc:presto://<IP_ADDRESS>:8080/catalogs/schemas")
            .option("user", "root")
            .option("dbtable", "table_name")
            .load();

    dataset.show(10);

Thank you very much for your answer.

Yes, you can connect to Presto via JDBC format. Please try appending url connection string as jdbc:presto://<IP_ADDRESS>:8080/catalogs/schemas?KerberosRemoteServiceName=....
It's troubleshooting than issue. Could you please join https://prestosql.io/slack.html and ask in #troubleshooting channel?


[SPARK-25450][SQL] PushProjectThroughUnion rule uses the same exprId for project expressions in each Union child, causing mistakes in constant propagation

What changes were proposed in this pull request?
The problem was cause by the PushProjectThroughUnion rule, which, when creating new Project for each child of Union, uses the same exprId for expressions of the same position. This is wrong because, for each child of Union, the expressions are all independent, and it can lead to a wrong result if other rules like FoldablePropagation kicks in, taking two different expressions as the same.
This fix is to create new expressions in the new Project for each child of Union.
How was this patch tested?
Added UT.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3179/
Test PASSed.

Merged build finished. Test PASSed.

Test build #96166 has started for PR 22447 at commit 7193de3.

Test build #96166 has finished for PR 22447 at commit 7193de3.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96166/
Test FAILed.

retest this please

Test build #96221 has started for PR 22447 at commit 7193de3.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3216/
Test PASSed.

Merged build finished. Test PASSed.

Test build #96221 has finished for PR 22447 at commit 7193de3.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96221/
Test FAILed.

retest this please

Test build #96233 has started for PR 22447 at commit 7193de3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3225/
Test PASSed.

Test build #96233 has finished for PR 22447 at commit 7193de3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96233/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3252/
Test PASSed.

Test build #96272 has started for PR 22447 at commit c7756ed.

Test build #96272 has finished for PR 22447 at commit c7756ed.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96272/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3254/
Test PASSed.

Test build #96280 has started for PR 22447 at commit c7756ed.

Test build #96280 has finished for PR 22447 at commit c7756ed.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96280/
Test FAILed.

Test build #4343 has started for PR 22447 at commit c7756ed.

Test build #4343 has finished for PR 22447 at commit c7756ed.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Thanks! Merged to master/2.4/2.3


Fix matchesBasicPattern sometimes matching on incorrect domains

Fixes #685: Default domain matching sometimes matches on incorrect domains.
See src/background/__tests__/utils.test.ts for details of the new matching logic.

looking forward to this, but in my case, the domain looks the same but isn't being matched, so not sure if a UTF thing or what.

looking forward to this, but in my case, the domain looks the same but isn't being matched, so not sure if a UTF thing or what.

What's the pattern and the domain it should be matching? Is there an issue for this? If it's relevant and should indeed match, I can add a test for it in the PR (no idea when/if this is gonna get merged though).

What's the pattern and the URL it should be matching? Is there an open GitHub issue for this? If it's relevant and should indeed match, I can add a test for it in the PR (no idea when/if this is gonna get merged though).

It's an internal url to a wiki, and oddly doesn't work on only one page where the div id's match. Tried to build the patch but it failed, but then again, I have no experience with yarn, npm, and the readme's don't really provide details for the uninitiated. I cloned stlyebot-fix-685, which couldn't apply some patch. So, hence I'm looking forward to this being merged in to give it a shot.

It's an internal url to a wiki, and oddly doesn't work on only one page where the div id's match. Tried to build the patch but it failed, but then again, I have no experience with yarn, npm, and the readme's don't really provide details for the uninitiated. I cloned stlyebot-fix-685, which couldn't apply some patch. So, hence I'm looking forward to this being merged in to give it a shot.

The URL being internal won't matter, what matters is its structure. Can you post it and the pattern here? If there's sensitive info in there, you can always replace it with foo/bar etc.
Also, are you certain it's a URL matching issue rather than an element matching issue? If you add a style rule of * { outline: 3px solid red; }, does that also fail to show up on the affected page?

The URL being internal won't matter, what matters is its structure. Can you post it and the pattern here? If there's sensitive info in there, you can always replace it with foo/bar etc.

Something like: https://etwiki.sys.company.net/display/~mememe/x.json

Also, are you certain it's a URL matching issue rather than an element matching issue? If you add a style rule of * { outline: 3px solid red; }, does that also fail to show up on the affected page?

At the above url, the red outlines did not show.
If I change the name of the wiki from x.json to xjson, all worked well, red outlines appeared.
But again, that's with the current version, not yours. So if your changes keep the host matching to the first slash, it should be better.

Something like: https://etwiki.sys.company.net/display/~mememe/x.json
Pattern: etwiki.sys.company.net

That URL matches with both the old and the new logic, but judging by the extension, it's a JSON file, and CSS is for styling HTML rather than JSON. Why d'you want to style a JSON file anyway? JSON is a data interchange format, rather than a format for displaying to end users. Or is it actually just an HTML file that misleadingly ends in .json?

That URL matches with both the old and the new logic, but judging by the extension, it's a JSON file, and CSS is for styling HTML rather than JSON.

well, it's really a wiki that happens to be named "x.json", and so the browser I guess is confused, or the confluence server really did set the content type as json.
I got around it by renaming the page, but to me, it shouldn't matter what the name is.

I checked x.json wiki again, and in debugger the content type is: content-type: text/html;charset=UTF-8 for what it's worth.

well, it's really a wiki that happens to be named "x.json"

Ahh I see. This PR has no effect on how content types are determined, so you'll need to open a separate issue.


Possible bug involving turnstile operator

See PR #24 for MWE.

Ok, so the issue was in your call to commutative_monoid:
The symbol :I was treated as a pattern variable (directly escaped as I \wedge a => a). The EGraphs pattern matcher, to treat symbols as literal values, does instead use "double quoted" symbols: :I \wedge a => a . The difference is subtle but crucial. Should I change the library functions to double quote symbols automatically, or just document this fact?

@jpfairbanks
If I change lines 5-7 to
(notice the subtle change)
Context = commutative_monoid(:∧, :(:I)) ∪ @theory begin
    (A.Ob ⊢ id(A)) == (A.Ob ∧ id(A).(A→A) ⊢ id(A))
end

This errors on line 19, 20, 30, 31, 32, 33. Why?
Line 19
@test @areequal Context ((A ∧ A) ∧ B ⊢ f) (B ∧ A ⊢ f)

(A ∧ A) is not equal to A in a commutative monoid. Same as line 20 for the other way around. Think about the monoid of multiplication of integers: x*x is not x. :robot:
Same should apply to line 30 and 31
Will investigate more about the other rules.

What I suggest: i think you need a set on the LHS of turnstile, but you need it to be immutable. Note that I am using a dynamic rule:
import Base.ImmutableDict
c = @theory begin
    # using nothing here just because a dict with no values but just keys is a set and we do not have ImmutableSet in base librarry:
    a::ImmutableDict{Any, Nothing}  ⊢ whatever       |>    do_something_with(collect(keys(a)))          #(and remember to return an expr!)
end

I think that the odds that you would want the identity of a commutative monoid to not be double quoted is basically zero, so I think that the API should make my usage an error condition.

Fixed! Now you can do commutative_monoid(:∧, :I) and it is automatically double quoted.


Out of memory error when typing and searching multiple projects in Project Selector

Cloud Tools for IntelliJ version: 17.11.2-SNAPSHOT
Google Cloud SDK version: 179.0.0
OS: Linux 4.4.0-97-generic
What did you do?
Run -> Edit Configuration, open App Engine Flex deployment configuration (from gs-spring-boot sample code).
Edit project name - type something partial (not a complete project name).
Open project selector drop down to see the accounts/projects tree.
It helps if you have multiple accounts (2 in this scenario) and many available projects in selector tree.
Keep typing and deleting project name. It seems to happen faster if many project names matches the phrase (i.e. "test" in this scenario matched 10+ projects).
What did you expect to see?
Filtered list of matching projects.
What did you see instead?
Projects match the typed phrase, list expands and collapses as phrase changes for some time.
Then, UI freezes, keys are not accepted.
Memory exceptions are printed to console and continue to be printed when attempting to use the application.
Nov 14, 2017 1:38:51 PM sun.awt.X11.XToolkit processException
WARNING: Exception on Toolkit thread
java.lang.OutOfMemoryError: GC overhead limit exceeded
Exception in thread "JobScheduler FJ pool 8/12" Exception in thread "JobScheduler FJ pool 2/12" java.lang.OutOfMemoryError: Java heap space

@ivanporty do you mind looking into this to see if anything jumps out at you?
This is definitely something we need to fix for the next release (or perhaps for a patch release).


Build failed with an exception. * What went wrong: Execution failed for task ':lottie-react-native:generateCodegenSchemaFromJavaScript'.

Description
getting this error after updating the react-native version from 0.73.10 to 0.76.5
Steps to reproduce
update react-native version from 0.73.10 to 0.76.5
then rebuild the project
Snack or a link to a repository
no
Lottie React Native version
7.1.0
React Native version
0.76.5
Platforms
Android
Workflow
React Native
Architecture
Fabric (new Architecture)
Build type
None
Device
None
Acknowledgements
Yes

Ran into the same issue as well

same here

As you can see builds are working in CI: https://github.com/lottie-react-native/lottie-react-native/actions/runs/12108296272/job/33757340897
Please make sure you have cleared your gradle cache and your environment is set up correctly.

@vitor3g  @justinsoong @hareesh7777
Fixed by removing lottie things from these files :
app/build.gradle
settings.gradle


Play .ts streams with VideoJs Player

Is your feature request related to a problem? Please describe.
It is not related to a problem.
Describe the solution you'd like
I have an m3u file with some URLs ending with .ts
iptvnator's VideoJs Player does not play the channels wich URLs end with .ts
Describe alternatives you've considered
It works seamlessly in VLC, but I prefer using a build-in player over an external player
Additional context
Add any other context or screenshots about the feature request here.

VideoJs has limits that prevent it from playing some formats, that's why third party players were added.


Add support for jiff

I have added basic support for the new jiff datetime library.

On their README, they have:

My plan is to iterate on the Jiff API and make occasional breaking change releases over the next ~year.

Though considering we don't actually depend on the api, only on the type names, we should remain relatively unaffected?

Yes, it is å fairly new create, but this is only for the basic types, and their serialization is most likely quite stable. And in the unlikely event that they break, then we just need to update. Users of jiff are aware that it is unstable, so it should be OK.

One possibility is to use a jiff01 feature instead. That way, if there's a jiff 0.2, then you can add a jiff02 feature and so on. But it looks like you aren't doing this for other libraries, so I assume you're doing semver incompatible releases whenever a public dependency gets a semver incompatible release?
But yes, I do expect the serialization format to be stable. Everything conforms to the Temporal ISO 8601 spec (which combines ISO 8601, RFC 3339 and RFC 9557).

We do have multiple features for a couple of crates, at least for uuid IIRC.
I can't remember if there's ever been any incompatibilities to be honest, we've either been lucky or they're not reported.
Maybe still we should jiff01 for now just in case. What do you think @snaggen?

I confess I don't understand the focus on names here. If jiff 0.2 is published without any changes to names, it is still true that the 0.1 version of jiff::Timestamp is a distinct type from the 0.2 version of jiff::Timestamp.
To me, the question is, what will you do when chrono (for example) publishes a 0.5 release? Would you add a chrono05 feature? Publish a breaking change? Wish you had added a chrono04 feature initially?

What I mean is that we only do a simple mapping, that will annotate the type
jiff::Timestamp as a String with hint DateTime. As long as there is a struct jiff::Timestamp things will still compile, and as long as that Struct serializes to a String we are also correct. If the type is not a valid timestamp, well then the hint DateTime is not correct, but that is a minor thing.
So, that is why I say that the only way we risk breakage is if the Structs change names, since that will prevent compilation.
So, this is why I don't think it is worth to use "jiff01" since I think that there are not really much risk of breakage since we have a very basic support.

Hmmm, okay. I think I probably don't have the context for this project to make a suggestion then. If Jiff types don't appear in the public API of this crate, then I don't think my concerns apply.

Yes, the details of the jiff datatypes are not exposed or used here in any way, the only thing this patch does is to add these basic annotations for the types Timestamp, Zoned, Span, civil::Date, civil::Time. So you may change the semantics of the types, add remove methods and this will not be affected. As long as the Structs still exists and serialize to String, we are ok.

To me, the question is, what will you do when chrono (for example) publishes a 0.5 release? Would you add a chrono05 feature? Publish a breaking change? Wish you had added a chrono04 feature initially?

If the type names don't change, then maybe we could cover both 0.4 and 0.5 within the same feature name?

Hmmm, okay. I think I probably don't have the context for this project to make a suggestion then. If Jiff types don't appear in the public API of this crate, then I don't think my concerns apply.

Basically this crates has macros which go through http handler signature and parse them to generate an openapi spec.
In order to do this we have to know how a given type can be represented in openapi.
For this we have traits (ex: TypedData) and macros which we use to implement the traits, example:
macro_rules! impl_type_simple {
    ($ty:ty, $dt:expr, $df:expr) => {
        impl TypedData for $ty {
            fn data_type() -> DataType {
                $dt
            }
            fn format() -> Option<DataTypeFormat> {
                Some($df)
            }
        }
    };
}

impl_type_simple!(jiff::Timestamp, DataType::String, DataTypeFormat::DateTime);

And now if an http handler mentions the type jiff::Timestamp, then we can generate it as an openapi string data type with the DateTime format, example:
#[api_v2_operation]
    async fn get_timestamp() -> Result<web::Json<jiff::Timestamp>, actix_web::Error> {
        ...
    }

And this is why we add these features, so we can map external crates types to openapi types.
Another way of doing is can sometimes can be to use newtype pattern and implementing the traits yourself. Ofcourse this is not always possible.

Gotya okay. If the Jiff data types aren't actually appearing in the API of your crate, and this really is just type names, then I agree this is probably fine because it means Jiff isn't actually a public dependency.
There is a chance that Jiff's civil module gets renamed to plain in the future though, so it would then end up being plain::DateTime instead of civil::DateTime.

Gotya okay. If the Jiff data types aren't actually appearing in the API of your crate, and this really is just type names, then I agree this is probably fine because it means Jiff isn't actually a public dependency.
There is a chance that Jiff's civil module gets renamed to plain in the future though, so it would then end up being plain::DateTime instead of civil::DateTime.

Ok, that would cause this to break, so maybe we go with jiff01, jiff02 until the API is stable, I will push an update

Alright let's go with jiff01 for now, thank you both!


'How many?' question prompt in 'How many people are applying for work authorization at this station?' question causes hesitation/confusion

Each time I come across this very first question, I go cockeyed and have to re-read the top-line question text to realize, 'How many what...?' Perhaps it's just me, but I would suggest Matt takes a look and evaluates it.

@aidanielson Since this program is for advocates, and they've used it hundreds of times now without mentioning anything, I'm okay with leaving it as is. There are ways it can be simplified or clarified, but I suspect by this point the preparers just blow by it and get to "How many?"

I'm likewise fine closing this out now that it's been specifically called out, considered, and affirmed.


Reset enemies

After losing a game, and starting a new, the intensity of the enemies must be resetted to the original value.

This was the classic copy list reference bug.


netopeer2-server cant connect to onos/odl

Hi all I would be grateful if you can please help me with the problem.
I am using Netopeer2 and Sysrepo to connect a device to ONOS or ODL controllers, however any time I push configurations about the device (such as ip addr, port, username, password) to controllers netopeer2-server gets an error.  (please check the file for error).
I cant seem to find a solution. I have installed everything from scratch to get latest versions of code, but error is still there.
netopeer2-server_error_on_connecting _to_controller.txt
Thank you kindly for your help.
Yuliya

please use the latest version of libssh.
Regards,
Swathin

I have re-downloaded and recompiled everything  again, and error is still there. What else can I try to get rid of this problem?
Thank you.
Yuliya

Hi,
you did not have to redownload and recompile all the applications, just libssh, did you? What version are you using?
Regards,
Michal

Hi Michal,
I know I didn't have to redo everything, i did it to make sure everything was up to date.
The libssh version i am using is Libssh: 0.7.9 , Netopeer2: 0.5.24, sysrepo: 0.7.4
Thank you
Yuliya

Hi Yuliya,
I am still leaning towards libssh causing the issue (based on the output, it crashes during ssh_handle_key_exchange()). The last release was version 0.7.5 so you must be using some unofficial one. Please, install libssh 0.7.5 and try again.
Regards,
Michal

Hi  Michal,
Thank you very much for your help. You were correct it was libssh version mismatch. I used the command "git clone http://git.libssh.org/projects/libssh.git" from https://github.com/CESNET/libnetconf2 to install the library. But for some reason using this command it downloaded wrong version of libssh, when i downloaded libssh zip from the https://git.libssh.org/projects/libssh.git/ it all worked.
Thank you for your help.
Yuliya

Hi Yuliya,
glad to hear that.
Regards,
Michal

Hi @yuliya2017
Having same problem, could you tell me which versions of netopeer2, libyang, libnetconf2, sysrepo and libssh you have?
Thank you,
Ana


Update main.py and model.py to account for removal of VerificationMaterials from Sigstore

Summary

Sigstore 3.0 has removed VerificationMaterials as stated in Release v3.0.0
Instead, verifier takes in bundles as argument now.
Release Note

Bug fix
Unable to run main.py according to README.md instructions using dependencies in install/requirements_Linux.txt.
Documentation

These changes enabled me to sign artifacts using the instructions on README.md

Actually, can you make your commits be signed, please? We use DCO, see https://github.com/sigstore/model-transparency/blob/main/CONTRIBUTING.md

@spencerschrock I think we missed the regression when updating our deps because we don't have e2e tests https://github.com/sigstore/model-transparency/issues/5. This issue is another good starting one

Can you amend the last commit please to pass the DCO check? (same as https://github.com/sigstore/model-transparency/pull/212#issuecomment-2174639942)


support for className

This PR is for #6 .

@gorangajic I have modified the code to use desctructuring.

great, thanks

Thanks for merging


Way to stop the server or bind

I have been looking for a way to stop the listening port without coming out of the console app
Is there something built in that I am missing or would I have to make a function to do this.

Indeed this is hard to achieve. I should add some functionality to do that.
For now this could be a workaround:
// ...
smtpServer.BindAndListenTo(IPAddress.Loopback, 25);
// ...
// undo the binding from above
PortListener portListener;
if (smtpServer.Bindings.TryTake(out portListener))
{
    portListener.StopListen();
}
// ...


missing baseline files when using .xml file extension

Expected Behavior
./gradlew detektBaselineMain detektBaselineTest should output a separate baseline file for each variant.
allprojects { project ->
     project.afterEvaluate {
          detekt {
               def subprojectPath = project.path.drop(1).replace(":", "-")
               baseline file("$rootProject.rootDir/config/detekt/baseline/$subprojectPath/detekt-$subprojectPath-baseline.xml")
          }
     }
}

Observed Behavior
it only outputs for a single variant config/detekt/baseline/foo/detekt-foo-baseline-release.xml, but debug missing.
if I change the file extension to anything but .xml it works and outputs both -release and -debug
e.g. s/xml/xsl
              baseline file("$rootProject.rootDir/config/detekt/baseline/$subprojectPath/detekt-$subprojectPath-baseline.xml")

Context
means we have to name our baseline files with a file extension other than .xml
Your Environment

AGP 4.1.0

Version of detekt used: 1.14.1
Version of Gradle used (if applicable): 6.7
Operating System and version: macOS 10.15.7

Hey @yogurtearl,
Unfortunately I wasn't able to reproduce.
Here a sample project where I tried to reproduce.
Your configuration is applied here: https://github.com/cortinico/reproducer-detekt-3146/commit/cc2589704b5911abd49f1f3e5ce17a274fc313a6
The output is here https://github.com/cortinico/reproducer-detekt-3146/commit/5fd0e26d1e9d738fc0d4e572ee4feab6a7a74b8a

Closed because of not being able to reproduce this.


Support for pgvector data store

It would be nice to support pgvector as a data store.

using pgvector as a data store can be a good choice for projects that need to perform similarity searches on vector data, and it's a popular option for machine learning and natural language processing applications.


Plugin miscellaneous fixes

This is to unblock #26398
In the plugin config, Capabilities has been moved to
Linux.Capabilities to avoid confusion with Interface.Types[i].Capability
Support for host networking.
Do not try to contact disabled plugin.
Fix various tests
Vendor docker/go-connections#25 to provide better error messages in case unix socket path is too long.
A DeviceCreation boolean has also been added to the config under the Linux field. This could
be changed in the future to be specific to a major number.
Support for devices

@tiborvass tests fail and seems related

LGTM when green,

Please take care of the following review comments:

https://github.com/docker/docker/pull/26398#discussion_r88549759
https://github.com/docker/docker/pull/26398#discussion_r88571481
https://github.com/docker/docker/pull/26398#discussion_r88569186

LGTM when green


Try using unsafe/pointers to exactly copy the original C code?

More of an exercise than an actual necessity, but I'd be interested to see if a) it helps performance (although that's not really an issue anyway) and b) if it sorts out the remaining failing tests

Does Span<T> help us with this? See here: http://adamsitnik.com/Span/

b) if it sorts out the remaining failing tests

$ dotnet test MAB.DotIgnore.Test
Build started, please wait...
Build completed.

Test run for A:\MAB.DotIgnore\MAB.DotIgnore.Test\bin\Debug\netcoreapp2.0\MAB.DotIgnore.Test.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Total tests: 82. Passed: 82. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.7129 Seconds

Are there any failing tests still?

original C code

Incidentally, which/where is the original C code?

original C code

Incidentally, which/where is the original C code? Would that be wildmatch.c from Git?

Never mind. I found it my answer. 😄

Sorry, this issue was only really ever meant for me, so I wasn't very clear...
The original C function I'm testing against (copied from the git source) is here.
The failing tests I'm talking about are the wildmatch tests here (also copied from the git repo).
I commented some of these out as my code fails them, but they are fairly uncommonly-used patterns so I made the decision not to support them at the time.
Two years later, I've still not encountered a case where this has been a problem, so I'm inclined to leave it. However, pull requests are gratefully accepted!

I think it's possible to maintain 99% of the original C code without using pointers or Span<T> and it will help performance (especially with respect to GC and fewer allocations), as well as take care of any failing tests. I'll try and share the results of my spike. Nevertheless, one item of concern is that the Git source is published under the GPL license, which means that anything derived from it will need to be released under the same license and terms, including this library. At present, it re-publishes under the MIT License, which technically violates the terms of the GPL. 😕

I think 1 is the only viable and fair option. 3 and 4 have been done by others so wonder what would be the distinctive offering if this library were to do the same. A fifth option is to have an out-of-process service that's GPL-licensed with a non-GPL library wrapper but I think we can agree that would be a clear winner in the overkill category.

…the original wildmatch implementation was a replacement for fnmatch, and actually came from rsync…

Interesting but rsync is also distributed under the GPL license so it doesn't help the situation.

There is a BSD-licensed C implementation here, …

This could be a call but what attracted me to this project was that it was based on the Git implementation and tests.

Me earlier:

I'll try and share the results of my spike.

So I've published my spike, for whatever it's worth. 💰
See dowild in GitWildMatch. The code is 99% identical to the original source from Git (minus the whitespace changes). As the README states, I used every dirty trick offered by C# to create types that can emulate the behaviour of the original C types. All tests pass, including those that were commented out here! 🙌I have not measured the performance improvements but I reckon there are many since there is no added GC pressure during the execution of the algorithm. The only call to String.Substring is entirely adjacent.
License issues pending, I'm happy to collaborate to combine our efforts for the benefit of the community.

Thanks, I do appreciate your feedback and help. That spike looks very interesting, nice work!
Everything you said regarding the licensing options makes sense; however, I am currently looking into a different, custom implementation anyway. While I'm all for the GPL, there are going to be use cases (not least my own, in future) which it will complicate dramatically.
I agree that it's a great selling point that this passes the same tests as the git implementation, but I'm not sure many people (other than us 😉) will care about how it actually achieves those results.
Another thing which has come out of this is that the reference git tests in this project are actually quite outdated, so it's possible that the existing library doesn't quite perform in the same way as the current git version anyway.
If I can't come up with anything which matches the existing performance and results, then incorporating your code and re-licensing under the GPL would be my next step.
Anyway, I will hopefully get some time to work it out this weekend—I'll keep you posted!

but I'm not sure many people (other than us 😉) will care about how it actually achieves those results.

To be honest, I shouldn't be caring either because that's the whole point of passing tests but I found it easier going to bed using a library that was based off the original code. That aside, I felt if the code can be kept 99% identical at the source level then sync'ing up with any changes would take 5 minutes. As a side-effect, there's the performance benefit from having zero-allocations.

Another thing which has come out of this is that the reference git tests in this project are actually quite outdated, so it's possible that the existing library doesn't quite perform in the same way as the current git version anyway.

Ouch! On my spike, I'm seeing this coverage:

Module
Line
Branch
Method

WildWildMatch
90.2%
82.4%
83.1%

📎 WildWildMatchCoverageReport.zip

There are certainly some missed code paths.

then incorporating your code and re-licensing under the GPL would be my next step.

:+1: Happy to submit a PR if it helps.

I'm closing this one, because the regex-based version is working well enough and this was more of an exercise anyway.


Hide markdown syntax as cursor leaves the position?

Is it possible to have the markdown syntax hide as the cursor leaves the text? Sort of like Obsidian does in their live preview mode?
Eg writing **bold** to only show bold? (Notice that the asterisks have been "hidden")

It's technically possible by using NSGlyphGenerator to replace the glyphs for * with none (e.g. unless selected)
The more convenient way to change the behavior is to use a delegate instead of overriding the glyph generator (which doesn't even work on iOS): NSLayoutManagerDelegate.layoutManager(_:shouldGenerateGlyphs:properties:characterIndexes:font:forGlyphRange:)
Example https://stackoverflow.com/a/57697139/1460929

I tried following that stackoverflow post but couldn't get it to work - can you show a crude example of how you would?

Sorry, no, just notes.
There's a WWDC session: Dan Schimpf and Aki Inoue (2010):  Advanced Cocoa Text Tips and Tricks.
PDF version e.g. available here:
https://docs.huihoo.com/apple/wwdc/2010/session_114__advanced_cocoa_text_tips__tricks.pdf
I'd suggest trying that on macOS, first, because iOS didn't have the API back then. NSLayoutManagerDelegate then will be the way forward IIRC.

This would be great functionality. Unfortunately it is something that I am unable to get to: @DivineDominion's suggestions are good ones.


ionic.Platform.isAndroid() returns true in Chrome for Android

Not sure if this is intentional, but it surprised me when running my Ionic app in Chrome for Android. There are certain features I need to conditionally disable for the web, such as the Camera, and there's no built-in way I'm aware of to do this.
So now that Ionic supports the mobile web, it would be very helpful to pin down more specific functions for ionic.Platform, particularly regarding mobile web.
Right now, I'm looking for the platform-browser class in <body> but it's clunky and I'm not sure how reliable it is.

Hey! So, the correct way you would want to be testing for if your running on the web or on a device would be to use the ionic.Platform.isWebView(). Here is a good link to learn about that method: http://ionicframework.com/docs/api/utility/ionic.Platform/  Thanks!

Oh my. Sorry, not sure how I missed that. Thanks!!

i think you are using chrome simulator mode which is selected an ios device..try with closing simulator view

you can use window.cordova &&  ionic.Platform.isAndroid()