feat: Enumeration Management Group

https://docs.zephyrproject.org/latest/services/device_mgmt/smp_groups/smp_group_10.html

I need some help with the typing and validation. Tuple vs List and names for the elements in GroupDetailsResponse
I need something like a GroupDetails(BaseModel), right?

Take a look at TaskStatistics: https://github.com/JPHutchins/smp/blob/43ee844ddaf5bcdb8dda2b81c52a95438833aa69/smp/os_management.py#L73-L111
And ImageStates: https://github.com/JPHutchins/smp/blob/43ee844ddaf5bcdb8dda2b81c52a95438833aa69/smp/image_management.py#L42-L126
Hopefully one of them matches.  I'll prefer Tuple to List I guess, even though I used List before.

BTW, see those other groups for how to add documentation. It's cut and pasted from the Zephyr docs. And then it generates a page here, e.g.: https://jphutchins.github.io/smp/latest/image_management/#smp.image_management.ImageState

Lots of type checking for old python is solved with from __future__ import annotations.

BTW, see those other groups for how to add documentation. It's cut and pasted from the Zephyr docs. And then it generates a page here, e.g.: https://jphutchins.github.io/smp/latest/image_management/#smp.image_management.ImageState

Will also need a matching .md file here: https://github.com/JPHutchins/smp/tree/main/docs

This is in header, not sure if helpful or harmful: https://github.com/JPHutchins/smp/blob/43ee844ddaf5bcdb8dda2b81c52a95438833aa69/smp/header.py#L94C1-L94C44

ChatGPT got me a little example working: https://github.com/sgfeniex/smp/pull/1

I see the lint and test issues. I'll be able to get back to this in a few days.

Thanks! I can check with tox. I suspect that Annotations needs to come from typing_exensions to support old pythons.


Commit 6a85cac breaks working class file with  "The font size command \normalsize is not defined"

Please see my comment on commit 6a85cac here:
https://github.com/wspr/fontspec/commit/6a85cac3f95ed09c7af412742402d7b3be4f33be

Strange if that slipped through the test suite; are you loading fontspec before \documentclass by any chance?

Indeed.   In my custom document class that derives from letter. \RequirePackage{fontspec} called was coming before the \LoadClass{letter}. When I swap the order, and restore the commit, all is fine.

Thanks for the confirmation — I'll add a fix to only call \normalsize if it's needed, then. I won't be able to do this straight away, however…

Thanks. It's easy enough to work around in the meantime. Cheers!

I had the exact same problem. I was defining a class based on letter. If \RequirePackage{fontspec} goes prior \LoadClass{letter}, it triggers the error
! LaTeX Error: The font size command \normalsize is not defined:
               there is probably something wrong with the class file.

I don't know if @wspr fixed the issue, if this is a recurring bug, nor if it even needs to be fixed.
In LaTeX 2ε for class and package writers, page 13, it is stated that \normalsize, \textwidth, \textheight and \pagenumbering need to be set by the class. This is taken care of by letter but not before it is loaded, therefore one can simply set its values at an earlier stage. As suggested in the aforementioned reference:
\renewcommand{\normalsize}{\fontsize{10pt}{12pt}\selectfont}
\setlength{\textwidth}{6.5in}
\setlength{\textheight}{8in}
\pagenumbering{arabic}


Update regex to v1.6.0

Automated changes by create-pull-request GitHub action

Codecov Report

Merging #32 (f8d4d3e) into main (3fee595) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main      #32      +/-   ##
==========================================
- Coverage   86.04%   86.02%   -0.03%
==========================================
  Files          43       43
  Lines        4415     4415
==========================================
- Hits         3799     3798       -1
- Misses        616      617       +1

Impacted Files
Coverage Δ

src/macros.rs
95.42% <0.00%> (-0.66%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3fee595...f8d4d3e. Read the comment docs.


ga

GA!

ANSI:INCITS 38:2009 13!

Can you make sure you're adding docs/code to describe how you're building these zips that end up in data.openaddresses.io?

This makes me think we need a way to detect staleness in the data we are hosting ourselves.

@migurski I agree. We have the year tag which I added in the original spec to do just that. But with machine this would be far better to automate and then deprecate the year tag.


feat: add ability to use nested routers from ServerDir files

🔗 Linked issue
https://github.com/nuxt/nuxt/issues/18571
❓ Type of change

[x] 📖 Documentation (updates to the documentation or readme)
[ ] 🐞 Bug fix (a non-breaking change that fixes an issue)
[ ] 👌 Enhancement (improving an existing functionality like performance)
[x] ✨ New feature (a non-breaking change that adds functionality)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description
These changes allow routers to be added to the h3 app using the same file-based routing system present in nitro.
Typically, this makes it possible to have one router per file.
For example, a todo file with GET /, POST /, PUT /:id, DELETE /:id ...
// api/{todo}.ts
const db = ['todo 1', 'todo 2']
export default createRouter()
  .get('/', eventHandler(() => db))
  .post('/', eventHandler(async (event) => {
    const { todo } = await readBody(event)
    db.push(todo)
    return todo
  }))
  .put('/:id', …)

In this modification, I use the syntax {route-name}.ts to determine that it's a router that should be added (not a function handler). Does this make sense to you? I was hesitating with route-name.use.ts or route-name.router.ts, but the dot is used to specify HTTP request method, so it might be a bit confusing.
Resolves nuxt/nuxt#18571
📝 Checklist

[x] I have linked an issue or discussion.
[x] I have updated the documentation accordingly.

Hi, and thanks for the PR! I think it would be amazing that we support multiple HTTP methods per route for simplicity. But probably using a full router is adding too much overhead. Also it makes it harder to apply route rules, generated nested types, etc.
I am working on a new format for h3 event handlers with object syntax let's see if we can introduce a simpler and more lightweight alternative approach.

Yes, that would be very convenient!
The main goal was to allow the use of a nested router in nuxt,
but I understand about the rules/types, can't wait to see this new format!
In the meantime I'll look for another way to fix https://github.com/nuxt/nuxt/issues/18571

So is this feature supported or deprecated? I can't find it in documentation.

So is this feature supported or deprecated? I can't find it in documentation.

This feature is not implemented.
I implement something interesting in server-block-nuxt, which gives you a syntax to define multiple request handlers in a route :
const message = "Hello World!!!"
const bye = "bye!"
export const GET = defineEventHandler(() =>({ message }))
export const POST = defineEventHandler(() =>({ message: bye }))

We could implement something like this in Nitro, @pi0 what do you think ?

Currently, defining routes per request per file is very tedious.
Given that we have typed fetch, this I believe, will greatly enhance full stack code management.


Support Azure App Configuration references in Azure App Services

What about offering the opportunity to reference App Configuration values the same way we can do with Key Vault References.
Key Vault: @Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931)
App Configuration: @Microsoft.AppConfiguration(ConfigurationUri=https://my.azconfig.io/configs/mykey/mylabel)
That way, we can provision those references during provisioning phase, keeping client code agnostic of the environment in which its deployed.
This is especially interesting when using Managed Identities.

@arkiaconsulting,
It's great suggestion indeed! My understanding is that specific reference is only applicable for Azure App Services/Azure Functions. Nevertheless, we should work with these respective teams to help App Configuration feel native across Azure.

We just announced the Key Vault Reference feature in App Configuration. Please open new issues if you have any feedback.
https://github.com/Azure/AppConfiguration-Announcements/issues/1

Hi,
This is a « must have » feature ! Without references link for app services, it’s useless at this time. We really love to have it as feature.
Thanks,
Alex

Hello!
Do you guys have any news on providing this feature?
Thanks!

We've been in conversation with the App Service/Azure Functions team but looks like this was not able to be scheduled within this year. We will keep trying to get this into their future planning.
cc: @lisaguthrie @AlexandraKemperMS

@adriend-advz  I think that with the AppConfiguration nuget, binding your App Service/Function is very easy, removing the need to be able to reference App Configuration items that way. Do you have a special use case ?

@adriend-advz I think that with the AppConfiguration nuget, binding your App Service/Function is very easy, removing the need to be able to reference App Configuration items that way. Do you have a special use case ?

Hi @arkiaconsulting , from Ops side, this is a not acceptable solution that AppService get directly information from a KV or App Configuration. Usually dev team ask a grant for list all settings, and so we can't control which config/secrets especially when KV/AppConfiguration are shared by multiples teams/projects.
So, to reference in AppSetting permit to control who and what are consumed from WebApp/FunctionApp, and in more when we deploy infra with IaC (like Terraform) we can know if some configs/secrets are not use anymore and can be delete, without tracking GIT repo for code as C# for people doesn't know some dev language.
In conclusion, AppConfiguration for ops teams without AppSettings referencing is acually useless and we still use AppSettings for configuration. And about Key Vaults secrets, like referencing is possible, we deploy it everytime and everywhere when it's needed. So until we don't have this feature, AppConfiguration will never use in our projets.
Hope that this use case, who is not special, can explain the reason of this must-have feature.
Have a nice day, and remain at your disposal if need further informations
Regards
Alexandre

Hi!
I just have one more question regarding this: I was planning to however use this app configuration task but removing everything related to key vault references to avoid pushing wrong reference.
When doing the deployment, it basically wiped all the keys that were not in the properties file.
Is there a way to avoid such a behavior?
Thanks in advance!

@adriend-advz What app configuration task are you talking about ?

Hi @arkiaconsulting! My bad, I was referring to the release pipeline task "Azure App Configuration Push".

@Yiming-Jia, can you help @adriend-advz ?

@adriend-advz the issue you described is tracked in #398 (or open a new issue if it's different). Let's not mix that with the current thread.

@alex-3sr thanks for sharing the feedback.

Found App Configuration just yesterday and was a bit disappointed that it has no app service integration in this way.
OP's example - @Microsoft.AppConfiguration(ConfigurationUri=https://my.azconfig.io/configs/mykey/mylabel) - would make AppConfiguration much more useful and earier to work with.

But... I'd like to suggest an alternative:

For Key Value pairs: @Microsoft.AppConfiguration(KeyValueUri=https://my.azconfig.io/configs/mykey/mylabel)
For Feature toggles: @Microsoft.AppConfiguration(FeatureToggleUri=https://my.azconfig.io/features/kyKey/mylabel)

I'm wondering if recursive resolution might be possible for this, also? So if my App Configuration reference was to return a key vault reference, if that could then be resolved to the keyvault item value as normal:
Web App configuration reference
@Microsoft.AppConfiguration(ConfigurationUri=https://my.azconfig.io/configs/mykey/mylabel)

App Configuration item value:
@Microsoft.KeyVault(SecretUri=https://myvault.vault.azure.net/secrets/mysecret/ec96f02080254f109c51a1f14cdb1931)

This could also resolve App Configuration Key Vault references automatically, too. Although I cannot work out how to deploy these via ARM template so have just been storing the URI reference to key vault items, instead.

For anyone who is interested in this feature, it will be helpful for the App Service team to prioritize the work if you can upvote the request below. Thanks!
https://feedback.azure.com/forums/169385-web-apps/suggestions/41402335-support-app-configuration-as-reference-app-setting

For anyone who is interested in this feature, it will be helpful for the App Service team to prioritize the work if you can upvote the request below. Thanks!
https://feedback.azure.com/forums/169385-web-apps/suggestions/41402335-support-app-configuration-as-reference-app-setting

For anyone who is interested in this feature, it will be helpful for the App Service team to prioritize the work if you can upvote the request below. Thanks!
https://feedback.azure.com/forums/169385-web-apps/suggestions/41402335-support-app-configuration-as-reference-app-setting

Hi,
There was already a feedback for this : https://feedback.azure.com/forums/920545-azure-app-configuration/suggestions/39505483-app-config-reference

For anyone who is interested in this feature, it will be helpful for the App Service team to prioritize the work if you can upvote the request below. Thanks!
https://feedback.azure.com/forums/169385-web-apps/suggestions/41402335-support-app-configuration-as-reference-app-setting

Hi,
There was already a feedback for this : https://feedback.azure.com/forums/920545-azure-app-configuration/suggestions/39505483-app-config-reference

Hi all,
Sorry to bump a thread, I usually never do that .. but I'm still exciting to wait this feature ^^
Have a nice day to all
Regards
Alexandre

Thanks for the query @alex-3sr. It may take some time, but we started working on this together with the App Service team now.

This feature is critical for properly implementing 12factor app design, which mandates that configuration be provided as orthogonal environment variables.  We have multiple Azure app services with dozens of settings each and we cannot use App Configuration because those apps are configured to get all their configuration from environment variables.

What would this mean for injection of IConfigurationRefresher checking sentinels etc or the feature flags with the [FeatureGate] attributes on the controllers?

It'd be separate from that, wouldn't it?

What would this mean for injection of IConfigurationRefresher checking sentinels etc or the feature flags with the [FeatureGate] attributes on the controllers?

If you are using the App Configuration provider library, you won't need this because you can pull configuration and feature flags from App Configuration directly. This integration is useful for applications that don't know (or you don't want it to know) about App Configuration because key-values in App Configuration will be available as environment variables to your applications.

What would this mean for injection of IConfigurationRefresher checking sentinels etc or the feature flags with the [FeatureGate] attributes on the controllers?

If you are using the App Configuration provider library, you won't need this because you can pull configuration and feature flags from App Configuration directly. This integration is useful for applications that don't know (or you don't want it to know) about App Configuration because key-values in App Configuration will be available as environment variables to your applications.

Understood @zhenlan, clients who care about these features should still be using the AppConfiguration library. Thanks.

Hi @arkiaconsulting, @alex-3sr, @adriend-advz, @mattboothman, @cpdohert, @PanosKousidis, @BzSpi, @Kittoes0124, and @Azalenski,
based on your requests, we have been building the @Microsoft.AppConfiguration Reference in App Service and it is now available in Private Preview! We would love for you to try out this feature and hear your feedback. To get access to the private preview, please fill out this survey: App Config Reference Private Preview Access Survey and we will reach out to you. Note that for any Microsoft private preview, you or your company will need to sign an NDA if you/they have not already done so.

Hello @arkiaconsulting, @alex-3sr, @adriend-advz, @mattboothman, @cpdohert, @PanosKousidis, @BzSpi, @Kittoes0124, and @azalenski, this feature is now in Public Preview.
Please let us know if you have any feedback!

Hello @arkiaconsulting, @alex-3sr, @adriend-advz, @mattboothman, @cpdohert, @PanosKousidis, @BzSpi, @Kittoes0124, and @azalenski, this feature is now in Public Preview. Please let us know if you have any feedback!

Hi @avanigupta - where do you want feedback?  If here then:
I'm hitting our App Configuration Service (ACS) from an Azure Function using app configuration references.  I notice that a request appears to be made to ACS for each app setting individually, ie the config retrieval is not batched into a single call.
I can retrieve all of our app config keyvalues using azure cli eg
az appconfig kv list --key "/ian/api/dev/*" --connectionstring "Endpoint ..."

where "/ian/api/dev/" is the common prefix of all of my app config key names.
Obviously the key names could be completely different with no common prefixing.
But I wonder whether some logic could be built into the "fetch" code that checks for common prefixing and retrieves a whole batch (up to 100 entries?) in a single call.  Retrieved keyvalues could then be matched to app settings based on their full keyname.
Would eliminate some of the 429s we're seeing (even in Standard tier).
Thanks!

Thanks for the feedback @ianrandell-sh. This is definitely something we want to add in future.

When a AppService uses a Key Vault Reference and the secret changes, the App Service will be restarted. From this article

If a version is not specified in the reference, then the app will use the latest version that exists in the key vault. When newer versions become available, such as with a rotation event, the app will automatically update and begin using the latest version within 24 hours.

Do changes to AppConfig also trigger restart of an AppService with AppConfig References?  If not, is this on the roadmap?

Changes in AppConfig won't be picked up in App Service automatically today. You will have to manually trigger a restart of your App Service for it to pick up the changes. Will an auto refresh within 24 hours, like what's supported in the Key Vault reference, work for you?

@zhenlan Bouncing Yes, automatically restarting - even ~24hrs after changing config - would be useful when changing non-essential config, like log level or APM config.


Units conversion problems main branch

The units conversion for "mol" leads to wrong internal units apparently. It looks like replacing "g/mol" by "Da" and replacing 1/mol by "Da/g" remove the problems. Observed on molecular system in FF parameters and species using exastamp.

particle to SI amount unit (mol) conversion factor was wrong. it has been corrected to 1.0e-23 / 6.02214076


ICMP check with IPv6 address

Hi
We can't use an IPv6 address for ICMP checks.
│ Error: Invalid value
│
│   with module.Datacenter.module.uptime_checks.uptime_check_icmp.icmp_uptime["****"],
│   on modules/uptime_checks/main.tf line 110, in resource "uptime_check_icmp" "icmp_uptime":
│  110:   address        = each.value["endpoint"]
│
│ value must be a valid DNS hostname

The hostname validation needs to be updated.

Thank you. @hatamiarash7
I'll test it and try to release it this week.

@hatamiarash7 release was postponed a bit, but we plan to issue a release in a few days.

@hatamiarash7 Thank you for the PR; I tested it and found that DNS validation for ICMP check also requires IPv4 validations and preserves host validations simultaneously. I decided to remove validation on the provider side to avoid a mismatch with the API. So, the https://github.com/uptime-com/terraform-provider-uptime/pull/127 schema hasn't changed, but now we rely on the API validation for that field.
Thank you for your work; I'm closing this PR. Changes from #127 merged to develop. We are preparing a release.


Upgrade to Go1.18

What does this PR do?
Updates everything to Go1.18.5.
Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes
General agent QA -- at this stage, running in various clusters and looking for issues.
Reviewer's Checklist

[ ] If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
[ ] Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
[ ] A release note has been added or the changelog/no-changelog label has been applied.
[ ] Changed code has automated tests for its functionality.
[ ] Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
[ ] At least one team/.. label has been applied, indicating the team(s) that should QA this change.
[ ] If applicable, docs team has been notified or an issue has been opened on the documentation repo.
[ ] If applicable, the need-change/operator and need-change/helm labels have been applied.
[ ] If applicable, the k8s/<min-version> label, indicating the lowest Kubernetes version compatible with this feature.
[ ] If applicable, the config template has been updated.

I didn't realize I had forgotten to mark this as "draft" - sorry!

It looks like the only two non-green things here were due to GitHub workers dying, not to Go1.18.  One is CodeQL, which shouldn't depend on the Go version.  The other is the MacOS tests, which have secretly been running Go1.18 for a day or two already without trouble.  At any rate, I've re-run them both.  Now it's time for review.

Great work thanks ! Do you plan to setup a go workspace afterwards ? https://go.dev/doc/tutorial/workspaces

Great work thanks ! Do you plan to setup a go workspace afterwards ? https://go.dev/doc/tutorial/workspaces

I wasn't planning on doing so myself.  However, we've been discussing having each component bundle be a distinct module, in which case go workspaces might be very useful.  Are you interested in digging into that idea?


OperatingSystemConfigs are cleaned before the MachineDeployment being deleted

What happened:
The background is we were doing some chaos tests for our platform applications, including random killing VMs via the cloud provider API. And the platform application would provision and tear down worker groups in the Shoot cluster frequently.
Yesterday we saw that hundreds of worker machines that should have been released were still running. After some investigation, we noticed that the though those worker groups had been deleted in the shoot spec, the MachineDeployments still exists:
➜  aws kg machinedeployment
NAME                                       READY   DESIRED   UP-TO-DATE   AVAILABLE   AGE
shoot--chaos--8fd6130c-master-z1           1       1         1            1           3d5h
shoot--chaos--8fd6130c-master-z2           0       0         0            0           3d5h
shoot--chaos--8fd6130c-master-z3           0       0         0            0           3d5h
shoot--chaos--8fd6130c-app-2a-z1           0       2         2            0           24h
shoot--chaos--8fd6130c-app-2b-z1           0       2         2            0           24h
shoot--chaos--8fd6130c-app-2c-z1           0       2         2            0           24h

Meanwhile, the extension-provider-aws Pod complains that:
{"level":"error","ts":"2020-04-01T09:05:00.756Z","logger":"controller-runtime.controller","msg":"Reconciler error","controller":"worker_controller","request":"shoot--chaos--test/test","error":"Failed while waiting for all machine deployments to be ready: 'Machine shoot--chaos--test-victim-z1-6d969bf888-vvvxc failed: Machine shoot--chaos--test-victim-z1-6d969bf888-vvvxc is not healthy since 10m0s minutes. Changing status to failed. Node Conditions: [{Type:FrequentContainerdRestart Status:False LastHeartbeatTime:2020-04-01 08:50:48 +0000 UTC LastTransitionTime:2020-04-01 08:30:38 +0000 UTC Reason:NoFrequentContainerdRestart Message:containerd is functioning properly} {Type:KernelDeadlock Status:False LastHeartbeatTime:2020-04-01 08:50:48 +0000 UTC LastTransitionTime:2020-04-01 08:30:38 +0000 UTC Reason:KernelHasNoDeadlock Message:kernel has no deadlock} {Type:ReadonlyFilesystem Status:False LastHeartbeatTime:2020-04-01 08:50:48 +0000 UTC LastTransitionTime:2020-04-01

But the OperatingSystemConfigs and the actual Secrets that contains the userdata had already been cleaned (only the one of master worker group was left):
 aws kg operatingsystemconfig
NAME                                   TYPE     STATUS      PURPOSE     AGE
cloud-config-master-16ffc-downloader   ubuntu   Succeeded   provision   3d5h
cloud-config-master-16ffc-original     ubuntu   Succeeded   reconcile   3d5h

The shoot reconciliation continuously reported error on the Gardener UI, and the machine deployments were not get cleaned for hours since we noticed the problem. Finally we deleted the MachineDeployments and Shoot manually in order to release the machines.
What you expected to happen:
Machines and MachineDeployments were properly deleted after the worker groups are deleted in the shoot spec.
How to reproduce it (as minimally and precisely as possible):

Create a shoot cluster with two work groups, master and victim, each with 1 worker.
Terminate the machine of victim via the cloud provider API (in my environment, AWS)
Wait the machine appeared in .status.failedMachine field of the MachineDeployment of the victim group
Delete the victim worker group in the shoot spec before the newly created machine download the userdata.
The newly created machine will stick in Pending forever because the userdata has been deleted.

Anything else we need to know?:
I suspected that the problem is caused by a race condition of Gardenlet and WorkerController in the extension Pod. According to the code, when reconciling Shoot, Gardenlet will delete the unused OperatingSystemConfigs first and then update the Worker object. Meanwhile, the WorkerController will wait for all MachineDeployments to become ready before cleanup the unused MachineDeployments. So, if a machine is down and a new machine is provisioned, there is a chance that the OperatinSystemConfigs are deleted before the newly created machine download the userdata.
Environment:

Gardener version: v1.0.4
Kubernetes version (use kubectl version): v1.16.7
Cloud provider or hardware configuration:

AWS
Ubuntu
Instance type m5.xlarge

I've noticed that the reconciling process of Botanist has been changed in master, but ComputeOpreatingSystemConfigs is still executed before DeployWorkers. So I suspect master suffers this issue too, though I have had time to verify yet.
IMHO:

Botanist should only clean OperatingSystemConfigs that is not used by any workers, so at least the cleanup of OperatingSystemConfigs should be postponed after the workers are reconciled;
WorkerController should be able to clean unused MachineDeployments even if there are unready MachineDeployments. It is too optimistic to assume that all the MachineDeployments should be ready  and before proceeding to cleanup work.

I'd like to submit patches if these make sense. How do you think?

Hello @aylei, thanks for filing the issue. I haven't fully understood why the OperatingSystemConfig is relevant here.
Could it be more the case that you are describing the same like https://github.com/gardener/gardener-extensions/issues/623?

@rfranzke Thanks for your quick response!
IIUC, OperatingSystemConfig is the resources that generate the userdata for the worker nodes to download. I've sshed to the worker node and cannot download the usderdata from kube-apiserver.
For the issue you posted, I think they are not relevant because all the MachineDeployments I described are owned by the same Worker object.

After look at the pull request again, I ensure that  https://github.com/gardener/gardener-extensions/issues/623 does solve the problem, thanks @rfranzke !

IIUC, OperatingSystemConfig is the resources that generate the userdata for the worker nodes to download. I've sshed to the worker node and cannot download the usderdata from kube-apiserver.

Yep, correct.

For the issue you posted, I think they are not relevant because all the MachineDeployments I described are owned by the same Worker object.

Hm, but your steps for reproduction talk about two worker pools. The Worker object always contains the list of all pools. Out of this list the worker controller generates the desired MachineDeployments.
Now, in your reproduction steps you make one of these machines in the victim group fail and then you remove the worker group from the pool spec. This means to the worker controller that it doesn't generate a desired MachineDeployment for victim anymore (because it doesn't know it anymore). However, in its reconciliation loop it was waiting for all machine deployments instead of only the desired ones. This will make it stuck forever.

OK, thanks a lot. Let's keep the issue open until you came back with results. By the way, we plan to release this soon (this week, hopefully).

Thanks, I will update to that version and keep you posted

I'm closing the issue as we believe it's fixed and as the issue was inactive for a while.
Kindly reopen if you still think the bug exists.
/close


fix: immediate value check in Async:interval

What

Fix immediate argument value check in Async:interval.
Why

Resolves #288
Additional info (optional)

👍🏻👍🏻


PEs w/ .rsrc section won't work as expected

Hi! I was doing some experiments with some PEs with embedded resources: the generated payload seems to not account for them.
I have produced a minimal example to debug the issue with mingw (tweak Makefile if needed): https://github.com/phra/mingw-pe-resources-example
When the regular application is launched, the icon of the window is set up properly, while if passed through donut, the default system icon is shown instead.
I suspect the generated payload is trying to access the .rsrc section of the PE hosting its execution instead of the original one and, if confirmed, I think it can require some manual patching similar to the command line override.
Any ideas?

Can you provide more details on how you are executing the shellcode? Is this via process hollowing or injecting into a running process?
Are you wanting donut to actively change the icon of the process it is injected into one that is included in the payload (if it is included)?

I have confirmed in my own testing that native applications loaded through Donut may access their own resources in the .rsrc section. Though, I had to comment out the code that wipes the PE's headers before execution. As such, I created an issue #84 for an option that causes Donut to preserve the payload's PE headers.

// resource.cpp : This file contains the 'main' function. Program execution begins and ends there.
//

#include <iostream>
#include "resource.h"
#include <Windows.h>

EXTERN_C IMAGE_DOS_HEADER __ImageBase;
#define HMOD_THISCOMPONENT ((HMODULE)&__ImageBase)

int main()
{

    HRSRC rsc = FindResource(HMOD_THISCOMPONENT, MAKEINTRESOURCE(IDI_ICON1), RT_GROUP_ICON);

    HGLOBAL hres = LoadResource(HMOD_THISCOMPONENT, rsc);

    LPVOID ptr = LockResource(hres);

    std::cout << "Address: " << ptr << "\n";

    system("pause");
}

great @TheWover, thanks for your amazing work!

Added an option to dev that allows the user to request that PE headers of native PE modules are preserved. This option will be included in the next release version of Donut.

Hi, even with the -k 2 option, FindResource in my code is returning 0. The PE works as expected before being converted into shellcode. Any ideas?

I tested this with a modified version of my code above that had an RCDATA resource that was a string. It worked with the -k 2 setting, found the string, and could print it. For now I will close this issue and will reopen if anybody else has any problems since I cannot reproduce any bugs.

#include <iostream>
#include "resource.h"
#include <Windows.h>

EXTERN_C IMAGE_DOS_HEADER __ImageBase;
#define HMOD_THISCOMPONENT ((HMODULE)&__ImageBase)

int main()
{
    HRSRC irsc = FindResource(HMOD_THISCOMPONENT, MAKEINTRESOURCE(IDI_ICON1), RT_GROUP_ICON);
    HGLOBAL ihres = LoadResource(HMOD_THISCOMPONENT, irsc);
    LPVOID iptr = LockResource(ihres);

    HRSRC drsc = FindResource(HMOD_THISCOMPONENT, MAKEINTRESOURCE(IDR_RCDATA1), RT_RCDATA);
    HGLOBAL dhres = LoadResource(HMOD_THISCOMPONENT, drsc);
    LPVOID dptr = LockResource(dhres);

    std::cout << "Address of IDI_ICON1: "   << iptr << "\n";
    std::cout << "Address of IDR_RCDATA1: " << (LPVOID) dptr << "\n";
    printf("Data: %s\n", dptr);

    system("pause");
}


Using a custom controller

Overriding a method of the controller is easy as it's just requires to extend the ResourceController. However in the case we wish to add a new action to the controller, for example countAction, to get the API endpoint GET /collection/count; It would be possible to add our route manually in our routing file, but then our rout would not "inherit" the prefixes that it would get if it were declared directly in the parent controller ResourceController. Is there a way to that in a more elegant manner?

As done for many other bundles, defining a prefix in your custom routing (trough annotations, XML or YAML) works and doesn't conflict with the bundle own routing system (I've already done that).
A cleaner but untested way should be to use the bundle "operation" definitions to map the custom route:

Create your custom action in your controller
Register the operation like the following:

services:
    my_dummy_resource:
        class:     Dunglas\JsonLdApiBundle\JsonLd\Resource
        arguments:
          - Dunglas\JsonLdApiBundle\Tests\Behat\TestBundle\Entity\Dummy
          - []
          - []
          - []
          - ~
          - ~
          - ~ # To customize collection operations it's here
          - # And here for item operations
            - { "hydra:method" => "GET" } # We add back default operations and let them be populated by the bundle
            - { "hydra:method" => "PUT" } # Same here
            - { "hydra:method" => "DELETE" } # Same here
            - { "hydra:title": "My custom operation", "!controller": "MyController:myAction", "!route_name": "my_route_name", "!route_path": "{id}/custom", "hydra:method" => "GET" } # Here is the custom operation
        tags:
          - { name: json-ld.resource }

Let me know if it works.

Sorry for the late answer. It seems to work fine with the following schema:
services:
    my_dummy_resource:
        class:     Dunglas\JsonLdApiBundle\JsonLd\Resource
        arguments:
            - AppBundle\Entity\Dummy
            - []
            - []
            - []
            - ~
            - ~
            - # Customize collection operations
                - { "hydra:method": "GET" }
#                - { "hydra:method": "PUT" }
#                - { "hydra:method": "DELETE" }
                -
                    "hydra:title":  "Count the number of instances"
                    "!controller":  AppBundle:Dummy:count
                    "!route_name":  "json_ld_api_users_count"
                    "!route_path":  "dummy/count"
                    "hydra:method": "GET"
            - ~
        tags:      [ { name: "json-ld.resource" } ]

However if I check into the Hydra Console I'm still lacking the Hydra documentation (when I go to the operations supported on my collection, I fail to see my action.
The return value of the Nelmio doc is also wrong (the return values are incorrect, I guess there's a parameter lacking somewhere.
I had to comment the two methods PUT and DELETE as shown in the snippet above. I get the following error:
ContextErrorException in Resource.php line 367:
Notice: Undefined index: hydra:title

Also the service configuration is growing big really fast, is it planned to change the arguments into an object to which we may pass the values filters, serialize, deserialize... instead of an array in which the order matters?

I've just updated the README and created a Behat scenario to explain how to do and prove that work.

Great, thanks for the update :)


Suggested changes to improved error handling

Thanks for the new native bluetooth support in GATT, we are really looking forward to this feature.
I have been messing around with it to keep up with whats happening and ran into some issues, as identified, with older kernels.
This resulted in the code failing due to the errors being swallowed up and a nil HCI being returned. I was even more confused when I started getting EBUSY messages, rather than EINVAL which the syscall was returning.
To remedy this and try and make the API provide good feedback during operation I suggest passing these errors down the line where possible. In addition to this I have suggested some generalised errors when retrying fails socket open, or bind times out.
Note I know this is early days, just providing some suggestions which are more inline with what I see in the golang core.
cc @roylee17

I won't have a chance to review this for a couple of weeks (I am on leave), so if it looks good to you, @roylee17 , feel free to merge. If you'd rather wait for me, then @wolfeidau I apologize for the delay and I'll review it as soon as I can.

No problem thanks for the note!


Could you share the code for the image simplification application in the paper?

I am very interested in this application, thanks a lot!

We are sorry for the late reply. We don't have the source codes for the application.


Limit parallelism in Dataset.cleanup_old_versions

Running into s3 rate limits when trying to cleanup a very large dataset with dataset.cleanup_old_versions. Can't seem to control this via LANCE_IO_THREADS

This could be due to hard-coded concurrency in object_store: https://github.com/apache/arrow-rs/blob/a937869f892dc12c4730189e216bf3bd48c2561d/object_store/src/aws/mod.rs#L252
We might need to make this controllable upstream somehow.

Hm, is there any way to temporarily monkeypatch rust-level code in python?

Actually, we don't use delete_stream (mainly by chance) so we probably don't need to worry about object_store.  I suspect this is fixed in 0.17.0b9 (released yesterday) via https://github.com/lancedb/lance/pull/2773
We were previously using num_cpus::get and now are using LANCE_IO_THREADS.

Actually, we don't use delete_stream (mainly by chance) so we probably don't need to worry about object_store.

What makes you say that? I see us call remove_stream here:
https://github.com/lancedb/lance/blob/2f25fc473dd69c8bc298c4f4e171b81f87660656/rust/lance/src/dataset/cleanup.rs#L289
Which dispatches to delete_stream here:
https://github.com/lancedb/lance/blob/2f25fc473dd69c8bc298c4f4e171b81f87660656/rust/lance-io/src/object_store.rs#L614-L615

I'm now getting
OSError: LanceError(IO): Generic S3 error: Got invalid DeleteObjects response: unknown variant `Code`, expected `Deleted` or `Error`, /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/fns.rs:368:13

Maybe this is happening because a previous cleanup operation failed without marking the version as deleted so it's getting a not found? Not sure how to work around this.

What makes you say that? I see us call remove_stream here:

Ah, I was just searching for delete_stream and saw the parallelism on old_manifests and assumed that was it.  My mistake.

OSError: LanceError(IO): Generic S3 error: Got invalid DeleteObjects response: unknown variant Code, expected Deleted or Error, /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/fns.rs:368:13

That's a new one for me.  Seems almost like a malformed S3 response.


1.41 on Mojave washed out colors

Updated from 1.40.2 to 1.41 and noticed that colors are washed out. I remember that the same thing happened when i updated to 1.40 from the previous version but then it was fixed, don't remember if i edited something or not. But now this new version is causing same issues.
Also in change log there was a mention about font rendering, i don't see any improvements.
Edit: improved font rendering is for Windows and Linux only no Mac :(
1.40.2

1.41

I've changed from false to true and it's fixed. But does it have to true?
// argv.json
{
"disable-color-correct-rendering": true
}

@bpasero @deepak1556 I'm seeing this too.
1.40.2

1.41

@MariuzM yes "disable-color-correct-rendering": true is the correct configuration, @dhowden can you check if you have the right value for the "disable-color-correct-rendering"

Yes, setting it to true fixed the problem.  It was set to "active" (which I believe was recommended for one of the previous issues).

The only valid configuration is: "disable-color-correct-rendering": true

Just want to give some updates, today i have used i1Profiler to calibrate display, but today i have decided that in it's advance option instead using Matrix curve i will use Table. After calibration i have noticed that my screen has more contrast deeper blacks and I'm getting full black range wheres before with Matrix i was not. In other words lucks much nicer now.
But with VSCode and "disable-color-correct-rendering": true the UI / Scrolling lags a lot. like 5fps or something. So ok then i changed this to False and walla smooth. Also colors look much better now with it being disabled.
So this setting i feel looks into what ICC profile you have and either works with it great or not. So to utilise this setting properly you would need to have correct ICC profile (I'm assuming)


Link doesn't work

[Enter feedback here]
In 'Datasets and datastores' section, the link doesn't work in here :
"For more examples using Datasets, see the sample notebooks."
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f8e951c2-65b8-2bf2-5c67-f4b57fbab467
Version Independent ID: e369876f-650d-c9f0-9f3b-d25c92e111cf
Content: Architecture & key concepts - Azure Machine Learning service
Content Source: articles/machine-learning/service/concept-azure-machine-learning-architecture.md
Service: machine-learning
Sub-service: core
GitHub Login: @Blackmist
Microsoft Alias: larryfr

@ebrub Thanks for the feedback. We are investigating into the issue and will update you shortly.

@Blackmist The alias link that fails is https://aka.ms/dataset-tutorial since it is referenced to a wrong source. The correct link should be the following:
https://github.com/Azure/MachineLearningNotebooks/tree/master/work-with-data/datasets
which needs to be updated on traffic manager. May Hu is the current owner who can help us update the alias(https://aka.ms/dataset-tutorial)
@ebrub Please use this link to check the dataset example.


ARROW-8604: [R][CI] Update CI to use R 4.0

Among the updates and fixes here:

Change the default R version everywhere from 3.6 to 4.0 (current version as of April 24, 2020). Some jobs that were using a "release" alias were already running on 4.0, but many were not.
Adapt the apt dockerfile to find the 4.0 CRAN apt servers
Switch the Windows builds to use the new Rtools40 toolchain, which includes enough msys2 functionality that we can delete a bunch of setup
Fix a few bugs in the crossbow config that meant that we weren't actually testing on a build matrix of linux distros (environment variables needed to be exported to get picked up by docker-compose.

@github-actions crossbow submit test-r-rstudio*

@github-actions crossbow submit -g r

@github-actions crossbow submit -g r

@github-actions crossbow submit -g r


Differentiate for and while statements routes

+1 Looks great, thanks for finding and fixing this issue!
~Timothy


chore: fix typos and link broken

fix some typos and link broken

fix some typos and link broken

Thank you!


Using intermock in tests ?

Hello there and thank you for this great project!
I wanted to use intermock to generate stub data while testing. I tried to run it as:
import {mock} from 'intermock'

describe('generate mocked types', () => {
   it('should generate mocked types', () => {
     const mck = mock({files: [['./types.ts', 'MyType']]})
     console.log(mck)
  }
})

But whenever I run this, mck is empty ({}) - any idea what I'm doing wrong ? There's no error printed to the terminal, so I'm thinking that I'm doing something wrong somewhere 🤔

I've also tried running it in CLI and received:
Unsupported Primitive type 173

So, it seems that it doesn't like some of my types that look like:
interface TypeA {
    info: {
      id: number,
      name: string
    }
}

If I extract the type like this:
interface TypeA {
    info: TypeAInfo
}

interface TypeAInfo {
  id: number,
  name: string
}

It works great 🤔

If I understood the API correctly this should work:
import {mock} from 'intermock'
import fs from 'fs'

describe('generate mocked types', () => {
   it('should generate mocked types', () => {
     const mck = mock({
       files: [['./types.ts', fs.readFileSync('./types.ts')]],
       interfaces: ['MyType']
     })
     console.log(mck)
  }
})

Ah, so I need to actually read the file contents - didn't think of that, will test and get back with the result

Would anyone be willing to put together a small codesandbox example of this working for testing as proposed? It would be really helpful. Maybe even enough to include in the docs cause I'm a little lost here.

If I understood the API correctly this should work:
import {mock} from 'intermock'
import fs from 'fs'

describe('generate mocked types', () => {
   it('should generate mocked types', () => {
     const mck = mock({
       files: [['./types.ts', fs.readFileSync('./types.ts')]],
       interfaces: ['MyType']
     })
     console.log(mck)
  }
})

For anyone reading this, you need to specify the file contents as a string:
files: [['./types.ts', fs.readFileSync('./types.ts').toString()]]

This is my current itteration for reusability
just replace    const typesDir = join(process.cwd(), 'src', 'types'); with however your storing the type/interface definitions
import { readdirSync, readFileSync } from 'fs';
import { join } from 'path';

import { mock } from 'intermock';

// eslint-disable-next-line max-len
// eslint-disable-next-line @typescript-eslint/explicit-function-return-type, @typescript-eslint/explicit-module-boundary-types
export const typeFactory = (interfaces: string[], isOptionalAlwaysEnabled: boolean = true) => {
    const typesDir = join(process.cwd(), 'src', 'types');
    const files = readdirSync(typesDir, { encoding: 'utf8', recursive: true })
        .filter(file => file.endsWith('.ts'))
        .map(file => [join(typesDir, file), readFileSync(join(typesDir, file), 'utf8')] as [string, string]);

    const mockedObjects = mock({
        files,
        interfaces,
        isOptionalAlwaysEnabled
    });

    return mockedObjects;
};


Add Landing rate to discord broadcast

Hello, In the last few days I implemented this option in my VA and my pilots liked it, that is why I am sending this change for other VA.

Thank you!


[BUG] when trying to retrieve BU

Describe the bug
trying to retrieve BU, only this particular BU in SFMC instance failed to retrieve showing following error info. While when tried to retrieve other BU from same SFMC instance, It retrieved well.

remaining part of screenshot

as discussed, the issue was based on putting your project folder (git repo) inside of your one drive folder which then caused bad interaction


Any plan for code release as described in paper?

Hi, Thanks for the great work!!
Is there any plan for code release, as described in paper?

Which part are you interested in? The replacement of the embedding object has been integrated into the main AdapterHub framework.

I thought I could generate MF^{10}_{NEURAL} with mBERT embeddings using the provided code.

I'm looking for the same thing (implementation of the matrix factorization method), I wasn't able to find it in the AdapterHub framework. Could you provide a link?

For anyone still looking for the implementation of semi non-negative matrix factorization, take a look at the repository that was mentioned in the paper: https://github.com/cthurau/pymf


fix(seeding): change client_id of service accounts

Description

change client_id of service accounts in seeding data
remove service account for daps

Why
align client_id of service accounts with id of client in cx-central realm of centralidp keycloak instance
Issue
n/a
Checklist

[x] I have performed a self-review of my own code
[ ] I have successfully tested my changes locally

@Phil91 as discussed, could you please add the updating of service account to the seeder?
Would be great if we also have a todo regarding users.
We have only one user in the base seeding:
https://github.com/eclipse-tractusx/portal-iam/blob/chores-3.2-release/import/realm-config/generic/catenax-central/CX-Central-realm.json#L2154

@Phil91 as discussed, could you please add the updating of service account to the seeder? Would be great if we also have a todo regarding users. We have only one user in the base seeding: https://github.com/eclipse-tractusx/portal-iam/blob/chores-3.2-release/import/realm-config/generic/catenax-central/CX-Central-realm.json#L2154

@evegufy changes are in see [90058d9](https://github.com/eclipse-tractusx/portal-backend/pull/210/commits/90058d922c7477ef3711987b7e6420f12c601d45.
Theoretically we could have an issue if the userEntityId is not matching with the id of the keycloak user when for example requesting the api/administration/user/owncompany/users/{userId}/businessPartnerNumbers/{bpn} endpoint for the specific user.

@Phil91 as discussed, could you please add the updating of service account to the seeder? Would be great if we also have a todo regarding users. We have only one user in the base seeding: https://github.com/eclipse-tractusx/portal-iam/blob/chores-3.2-release/import/realm-config/generic/catenax-central/CX-Central-realm.json#L2154

@evegufy changes are in see [90058d9](90058d9.
Theoretically we could have an issue if the userEntityId is not matching with the id of the keycloak user when for example requesting the api/administration/user/owncompany/users/{userId}/businessPartnerNumbers/{bpn} endpoint for the specific user.

As just discussed, the user_entity_id in portal db for the one user (ac1cf001-7fbc-1f2f-817f-bce058020006) matches already the id of the user in the keycloak, so that should be fine


Replace atomic var by normal var

Because db is locked when  add/sub  bg_gc_scheduled_, so this variable is no need to be atomic

Thanks for your contribution @GlitterIsMe
/cc @yiwu-arbug @Connor1996

Codecov Report

Merging #39 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   84.12%   84.11%   -0.02%
==========================================
  Files          43       43
  Lines        2577     2575       -2
==========================================
- Hits         2168     2166       -2
  Misses        409      409


Settings object - splitting attributes

Follows #1954, specifically:

Split up Settings a bit, and don't pass the whole thing around as much (e.g. InputColumn only needs a fraction of information)

The main content is two new classes ColumnInfoSettings and TrainingSettings which each take a little bit of functionality from Settings, which now only has this info via .column_info_settings and .training_settings attributes.
Most of the rest of this PR is simply adjusting references, but there are a few other differences:

_source_dataset_column_name_is_required is now 'officially' private - external uses all now just check if source_dataset_column_name is None. Previously we had a mix of the two bits of logic - IMO this version is a bit neater
InputColumn now only gets fed a ColumnInfoSettings rather than the full Settings

To stop this PR blowing up too much some work is only partial, but can be done piecemeal:

Comparison takes a ColumnInfoSettings as well - ultimately it can just be fed the rest of the info it needs, but this is a bit meatier, as we need to ensure things stay in sync, for instance when we switch off retain_intermediate_calculation_columns and the like
expectation_maximisation.count_agreement_patterns_sql() takes just a list of Comparisons rather than a Settings, and a similar adjustment in predict.predict_from_agreement_pattern_counts_sqls - there are various other places we can limit access (passing subsets of Settings, instead of Settings, or even Linker) in follow-ups

Overall this PR may look like it is just adding complexity without a huge gain, but I think in combination with the second point we can:

in many instances limit the amount of data being passed to functions (so they don't need to know all about Settings, or Linker, but just the small bit they need)
limit the places where Settings is directly accessed, which should make future changes easier to manage (as they should only touch a relatively few places in the codebase)

I have kept the interface to Settings mostly the same (an additional arg + mostly keyword) for now, but we could instead accept e.g. column_info_settings and put the construction logic instead in SettingsCreator. Seemed initially to me more like an implementation detail so should be more 'hidden', but given that the details are actually publicly accessed I'm a bit less convinced.
There will probably be a slight conflict with #1957 so will not merge until after that.

re: test failure ☝
seems i was accidentally excluding some tests when running locally, so missed this. Looking into the fix now - don't anticipate it being a huge change.

I think it's right to split this up.  There's one thing I want to raise to make sure we have a sense of where this will end up.
At the moment EM training is messy but works like this:
An EMTrainingSession is creeated which holds:

the original linker object
a copy (called _training_linker) of the original linker so it can write to the m and u values without compromising original linker

EM works by:

The expectiation step, which is running predict() step using the current m and u probabilities
The maximisation step, which uses the results of the predict() step to compute new m and u probabilities, and write them back to the _training_linker.

But where do these m and u values end up?
Well, we need:

their full history, so we can plot how they changed through EM iterations _settings_obj_history

to store all their different values if different training runs have produced different estimates.  e.g. here.

The later are stored on the comparison_levels e.g. here and then retrieved with e.g. _trained_m_median _populate_m_u_from_trained_values
Why am I mentioning this?  Because it feels relevant to the ColumnInfoSettings and ColumnInfoSettings and decisions about how to continue splitting this up going forwards.
I haven't got any proposed solution but I think it might be good to try and come up with a high level plan about how we eventually see this panning out

Maybe this all ends up just as further continuation of splitting things out.  Rather than copying the whole linker, we could somehow copy only the parts needed.

These are good points, and things I have been thinking about a bit, particularly while I have been doing this. Apologies if this is a little stream-of-consciousness.
One of the things that I find difficult while working on some parts of the code is that it often feels like we are dealing with global state - partly this is the fact that we have an 'external service' (i.e. the database) where information lives, but the other part is the linker/settings, and the fact that lots of places in the code we are mutating this, which is not necessarily obvious at the calling site. For example this portion of code - looking at this alone it is hard to tell what gets changed (only directly the comparison level cl - but also em_training_session indirectly as this knows about cl via Settings).
What I would like in general is to strip the Linker of its power (and to a lesser extent Settings, which is perhaps more pertinent to this workstream). One of the things I am struggling with a little bit is understanding ultimately what the Linker (Splink 4 version) is for - what are its responsibilities? It seems to me something like 'orchestrating the linking' (which is still quite nebulous), or perhaps just 'a convenient user interface for some functionality'. But ultimately I think that whatever it is, probably a lot of code we have that currently involves the linker probably doesn't need it. I think that many things just need the db_api + some subset of the Settings.
For example EMTrainingSession - I would like this to not take the linker, but (roughly):

db_api (so that we can execute SQL)
'core model' settings (lambda + comparisons), which perhaps can live together, at least in this context
fixed training configuration stuff (convergence, max iterations)

other fixed info from settings we need - columns to select etc

(which I think potentially works well with what we've discussed with the jinja templating).
Then I think stuff should generally not be mutated, but we make new versions, more like
core_model_settings = maximisation_step(db_api, core_model_settings, param_records)
em_training_session.update_history(core_model_settings)

so that it's a bit clearer where values get set, and we can largely treat the objects we're dealing with as fixed.
There are probably complications to this, and there is some thought needed around the m / u values like you say, but that's the rough idea I have in mind.

Yeah - agree with all your points.    There are lots of places where linker is basically treated as global state.
Also agree it would be good to articulate the purpose of the Splink 4 linker object because I think it would help clarify a key part of the 'end state' we're working towards.
In Splink 3, it was a mix of many things:

API discovery.  Once instantiated, provides a listing of most/all the things Splink can do
Keeping track of data tables and the cache
A place to 'shove' arbitrary global state such as self._find_new_matches_mode, so as to modify SQL-generation behaviour
A place to put things when i wasn't sure where else to put them because they needed to 'know' about the linker.  This last point is probably the reason it's so messy because this is a bit of vicious cycle.

Most of this simply reflects my lack of expertise in how to design things 'properly' (i.e. my lack of awareness of alternative options), rather than a deliberate decision  e.g. where does the cache live if not in the linker.

A bit confused here -- why did the CI tests not run for this PR?
I'm getting a test failure related to this, which appears not to have been updated:
https://github.com/moj-analytical-services/splink/blob/90c0455b7c09e6b787c1b9b658fc7c92c7a43a9b/splink/edge_metrics.py#L36

A bit confused here -- why did the CI tests not run for this PR?

The tests did run, but not for the latest commit (as it was autoformatting which is run by CI) - this is an annoying feature of how our workflows are currently setup. You should be able to see the test runs in the penultimate commit

I'm getting a test failure related to this, which appears not to have been updated:
https://github.com/moj-analytical-services/splink/blob/90c0455b7c09e6b787c1b9b658fc7c92c7a43a9b/splink/edge_metrics.py#L36

Not sure quite what happened here, but this line seems have been reverted in places, possibly a merge error. Not sure exactly where it is fixed atm, but it is corrected at the very least in pending PRs, if not on the branch. It should be uid_cols = linker._settings_obj.column_info_settings._unique_id_input_columns

It's not corrected on the branch -- the link I shared was to the splink4_dev branch. I don't see any recent changes to this in the history. My best guess is that the intended diff was removed from this PR erroneously as part of a rebase (it looks like there was a force-push here).

Right, sorry, hadn't looked at the branches properly. So for reference what happened here was:

I opened this PR to splink4_dev, tests run + passed
i also had recently merged an unrelated PR to master, including the module with the line in question being added (which works for Splink 3) (#1894)
this PR was approved. There were some things I wanted to check with related work, do didn't merge straight away
this change then got brought to splink4_dev when we merged in master in #1972
I merged this PR, which then had a clash with the edge_metrics module, and I didn't think to re-run the tests with the new base context
this got fixed as part of other PRs separatetly - one of which has not been merged yet, the other is in now. As it has mostly been a couple of us working on Splink 4 so far I didn't prioritise getting the branch back to a working state when I noticed the error

I'm fairly confused by the CI setup, specifically when the tests run and when they don't. They should have failed on this PR with its diff as-is; why didn't they?

The tests are run when commits are pushed, against the nominal merge commit wrt the target branch. When I pushed the last commit (well the penultimate one, see above comment) the tests ran, and passed fine, because the edge_metrics module didn't exist in the target branch yet, so there was no problem.
Currently there is no way to get the tests to automatically re-run when the target branch receives an update. We can manually do so or merge in the target (and I do on occasion when nearby changes come in) - to be honest it didn't occur to me in this instance. Will chat with the team about what we can do to avoid this

Ah, I understand now. What you're looking for is this: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-suggestions-to-update-pull-request-branches. I didn't even remember this was optional, since I've always used it on projects with CI. This enforces (or maybe just strongly encourages) that you test your merge before performing it.

You can fully enforce it by making splink4_dev a protected branch with "strict" required status checks: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-status-checks-before-merging


chore(deps): bump deck to v1.21.0

Do we want to include some concrete (regression?) tests for https://github.com/Kong/deck/pull/918 which 1.21 includes? 🤔

Do we want to include some concrete (regression?) tests for Kong/deck#918 which 1.21 includes? 🤔

Actually, that might be a good idea. Hold on, let me try to come up with something.

@pmalek That was a fruitful journey that should allow us to make the upgrade tests more reliable as trying to trigger the regression I discovered some lacks of those:

feat: all-in-one manifests migrations job improvement #4116 (migrations job was not working even if we removed the old one)
tests: make upgrade tests more robust #4117 (we weren't waiting for the upgraded replicas to become ready and we weren't testing that we're able to change the configuration after the upgrade)

I'm glad 😅 ... That we found those now, not e.g. after a release.
🎖

TestDeployAndUpgradeAllInOnePostgres_FeatureGates passes after the bump. 👍

Two e2e don't pass

run / gke (v1.25.5, TestDeployAndUpgradeAllInOnePostgres_FeatureGates$)
run / gke (v1.25.5, TestDeployAllInOneDBLESSGateway$)

@programmer04 Ahhh, it's again the problem with GKE E2E tests running against the nightly image 🤦 It will be fixed once we merge it.


Lazyload-app-icons edits

@zeke Let me know if there are other necessary changes. 😺

Thanks, @sethlu!


Update node modules

This PR updates the node modules, as the number of security issues has grown a bit. I excluded some packages which seemed to be unused.
It seems to be the recommended way to include the package-lock.json file, c.f. this info or others like https://github.com/microsoft/TypeScript.

Added some updates for the properties, following https://docs.npmjs.com/cli/v8/configuring-npm/package-json.


Is it possible to use CSS Modules to completely replace 'css-loader'?

Hi, thank you for creating the loader!
Lightning CSS does support producing CSS Modules, but seems like it doesn't work with lightningcss-loader
This works
use: [
  {
    loader: isDev ? 'style-loader' : MiniCssExtractPlugin.loader,
    options: {
      ...(isDev ? {injectType: 'styleTag'} : {esModule: true}),
    },
  },
  {
    loader: 'css-loader',
    options: {
      esModule: isRelease,
      sourceMap: isRelease,
      importLoaders: 1,
      modules: {
        mode: 'local',
        exportGlobals: true,
        exportLocalsConvention: 'asIs',
        localIdentName: '[name]_[local]'
      },
    },
  },
  {
    loader: 'lightningcss-loader',
    options: {
      sourceMap: isRelease,
      analyzeDependencies: false,
      implementation: LightningCSS,
      drafts: {
        nesting: true,
        customMedia: true,
      },
      errorRecovery: false,
      unusedSymbols: [],
    },
  },
],

Suggested solution
But this doesn't work
use: [
  {
    loader: isDev ? 'style-loader' : MiniCssExtractPlugin.loader,
    options: {
      ...(isDev ? {injectType: 'styleTag'} : {esModule: true}),
    },
  },
  {
    loader: 'lightningcss-loader',
    options: {
      sourceMap: isRelease,
      analyzeDependencies: false,
      implementation: LightningCSS,
      cssModules: {
        pattern: '[name]_[local]',
        dashedIdents: false,
      },
      drafts: {
        nesting: true,
        customMedia: true,
      },
      errorRecovery: false,
      unusedSymbols: [],
    },
  },
],

Alternative
It feels a bit excessive to use css-loader. Do you think lightningcss-loader could replace both postcss-loader and css-loader in the pipeline?
Additional context
No response
Validations

[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

lightning-css is only a css processor and does not integrate with webpack support.
We need the css-loader to provide pure css to webpack.

Hello, I would also be interested by such feature.
css-loader uses Postcss internally, and if I use lightningcss I would be happy to not go through postcss after it.

https://github.com/vercel/next.js/pull/58712  Next.js has transformed this Repo into a usage similar to css-loader

css-loader 里要处理的东西是很多的，我不认为 lightning loader 这一个就能代替的了，除非 fork 一整份。

See, css-loader will parse the CSS with postcss anyway, unless postcss-loader is in front, in which case it's able to reuse its pre-parsed AST:
https://github.com/webpack-contrib/css-loader/blob/master/src/index.js#L159

The goal of using lightningcss to me is that we shouldn't need to parse CSS with postcss anymore. We need to get rid of css-loader to benefit from it.

@fz6m hey! I'm working with @kdy1 to extract the loader he wrote for Next.js and make it reusable for other webpack users. I'd like to make it more officially maintained as well. Would you be open to donating the npm package name to such an effort? We'd bump a major version when publishing the updated package to ensure nothing existing breaks.

Thank you for your interest in this project.
I have added you and kdy1 as npm package owners.
if want to move the repo to an organization, contact me anytime ^_^

Thanks for the quick response! I'll work on this soon and give an update here when I make progress.


Contribute: GTFS sensor optimizations

As a developer, I want to publish my GTFS component improvements so that other users can benefit from them.
Acceptance criteria

It's done when a Home Assistant PR is submitted to support for next departures.
It's done when a Home Assistant PR is submitted to document database speed-up tweaks.

References

/gtfs/README.md
https://community.home-assistant.io/t/faster-gtfs-schedule-lookups/37624

Currently creating a series of GTFS pull requests to cover the following:

Clarify SQL query
Set icon per route type
Adjust arrival date to tomorrow when bus arrives after midnight
Extend lookup to include departures crossing midnight
Search tomorrow when no more departures today (optional)
Return None instead of "0 minutes" when no more departures
Cache metadata queries
Enable departure, timer, countdown, and auto state modes
Fix default name handling to prevent entity_id changes
Support multiple departures/sensors (with internal cache)
Investigate adding missing indexes to SQLite database
Prevent race conditions when building the database
Add type hints + Code cleanup


[Typography] Add custom variants support

This PR adds support for custom variants in the Typography component as part of https://github.com/mui-org/material-ui/issues/21749

Does this PR cover the following use case (see image attached)?

From a developer perspective, this would reduce verbosity tremendously

@cristobalchao You can see an example of the API here, under "Basic example":
https://github.com/mui-org/material-ui/issues/21749

@cristobalchao
That usecase is supported, the syntax is a bit different.
const theme = createMuiTheme({
  variants: {
    MuiTypography: [
      {
        props: { variant: 's8' },
        styles: {
          fontSize: 14,
          lineHeight: '20px',
          // other styles...
        },
      },
    ],
  },
});

And than you can use:
<Typography variant="s8">Hello world</Typography>

Here is a docs example: https://next.material-ui.com/customization/components/#adding-new-component-variants that describes this API for the Button component

@mnajdova
Thanks for sharing this. I wonder for future versions, if it would make sense to consolidate everything under typography. As we can do now with non-custom variants, which would make everything more streamlined. Have you considered that approach?

@cristobalchao this is definitely something we want to explore. We are looking into different ways of using dynamic styles based on props. Once we solve that issue, this can be easily implemented.

@cristobalchao I believe it will require the same solution to the dynamic color issue #13875. While #15573 focus on allowing the support of new props, right from the theme, #13875 goes a step further. It's meant to bypass the design tokens -> style transformation step. People could leverage the design tokens of the theme directly.

Thanks for making this update! However, I'm still seeing the prop type warning in my console. I wonder if anyone else is seeing the same.

@mikeyamato can you please ensure that you are on the latest v5 alpha version? This should not happen as the proptypes is basically relaxed to accept any string. If this is still the case for you, please create an issue.

@mnajdova totally makes sense. Literally moments before you responded I noticed that this wasn't part of v4.11.0 but rather part of v5.0.0-alpha.6. Okay, going to test it out with v5. Thanks!

Quick update from my end. I tried to upgrade to v5 but that resulted in various dependency issues with my existing project. Unfortunately I won't be able to take advantage of this feature just quite yet. But looking forward to utilizing it in the near future. Till then I'll need to live with the errors 😢 .

@mikeyamato What dependency issues did you face?

@cristobalchao
That use case is supported, the syntax is a bit different.
const theme = createMuiTheme({
  variants: {
    MuiTypography: [
      {
        props: { variant: 's8' },
        styles: {
          fontSize: 14,
          lineHeight: '20px',
          // other styles...
        },
      },
    ],
  },
});

And than you can use:
<Typography variant="s8">Hello world</Typography>

Here is a docs example: https://next.material-ui.com/customization/components/#adding-new-component-variants that describes this API for the Button component.

FYI It's here now:
https://next.material-ui.com/customization/theme-components/#adding-new-component-variants

@EliRobinson For the typography variant, it's no longer needed: https://next.material-ui.com/customization/typography/#adding-amp-disabling-variants.


Allow parameters in init() of smart contracts

It would be very handy to let us pass parameters to the init() of smart contracts.
The idea would be to set those parameters using the command:
eosc set contract hello hello.wast hello.abi {params}

init() was removed.


If you add a Console logger when AddWebApiConventions is enabled, startup fails.

Object reference is equal null
at Microsoft.AspNet.Mvc.Logging.ActionDescriptorValues.<>c.<.ctor>b__0_5(KeyValuePair2 i)    at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector, IEqualityComparer1 comparer)    at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable1 source, Func2 keySelector, Func2 elementSelector)
at Microsoft.AspNet.Mvc.Logging.ActionDescriptorValues..ctor(ActionDescriptor inner)
at Microsoft.AspNet.Mvc.Core.DefaultActionDescriptorsCollectionProvider.GetCollection()
at Microsoft.AspNet.Mvc.Core.DefaultActionDescriptorsCollectionProvider.get_ActionDescriptors()
at Microsoft.AspNet.Mvc.Routing.AttributeRoute.GetInnerRoute()
at Microsoft.AspNet.Mvc.Routing.AttributeRoute..ctor(IRouter target, IActionDescriptorsCollectionProvider actionDescriptorsCollectionProvider, IInlineConstraintResolver constraintResolver, ILoggerFactory loggerFactory)
at Microsoft.AspNet.Mvc.Routing.AttributeRouting.CreateAttributeMegaRoute(IRouter target, IServiceProvider services)
at Microsoft.AspNet.Builder.BuilderExtensions.UseMvc(IApplicationBuilder app, Action`1 configureRoutes)
at Microsoft.AspNet.Builder.BuilderExtensions.UseMvc(IApplicationBuilder app)
at Mvc6Sandbox.StartupImplementation.Configure(IApplicationBuilder app, ILoggerFactory loggerFactory) in C:\Users\Darrel\Documents\Source\GitHub\darrelmiller\Mvc6Sandbox\Mvc6SandboxApi\Startup.cs:line 42
Here is my startup:
public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        services.AddLogging();
        services.AddMvc();
        services.AddWebApiConventions();
    }

    public void Configure(IApplicationBuilder app, ILoggerFactory loggerFactory)
    {
        loggerFactory.AddConsole(LogLevel.Verbose);
        app.UseMvc();
    }
}

How old is this build? ActionDescriptorValues and related logging code was removed a long while ago.

It's the beta4.  I'll see if I can repro in beta5 and reopen an issue if I can.

@darrelmiller thanks. I'll likely that we just had an undiscovered bug in the logging code that could be triggered when using the webapi conventions. Regardless, all of that logging code was removed.


Add predict_proba to elastic net wrapper

Resolves #496

You mean this resolves partially for #496, right? There's also GBM that's problematic.

@terrytangyuan Yes


fine tune

I feel like the success rate of a given objective will always be significantly less than 100%. E.g., I've been testing Tarsier to try and get it to make a reservation at a restaurant using resy.com, but it fails at the login step because it incorrectly picks the label on the "password" input to type text into (and wants to type my email in that field no less).
I wonder if there's some system that could be built that has a human in the loop correcting mistakes made by the AI, and those corrections are then used to fine tune the model for that specific task.
Anyway this is more of a "idea" than an issue, but wondering what you think! (Feel free to close)

Are you using GPT-4-Vision or the text based approach?
In the text based case, you could also add a critic loop for the LLM to validate that it made the correct choice. Another option is to first have the LLM choose candidate elements and then provide more context on the selected elements. (Such as HTML attributes)

Closing due to inactivity!


Make git mapping entry-not-found error more clear

This error may arise when some directory specified in git.add does not exist:
Error: phase build on image service1 stage setup handler failed: entry not found

fixed
│ │      size: 60.2 MiB
│ └ Building stage test/from (0.78 seconds)
└ ⛵ image test (3.38 seconds) FAILED

Running time 4.05 seconds
Error: phase build on image test stage gitArchive handler failed: error checking stage gitArchive is empty: "git.add: /not-found" in werf.yaml matches no files. git.add requires at least one file matched by it. Fix and retry


fixed issue #764

Changes Before

After changes

(removed testimonial)

Hey @Git21221 , please delete the previos branch and then make required changes. And try to commit everything at once.
In the navbar the code is changed a bit. So, better delete the other branch , pull the repo properly and then make required cahnges in new branch!


Fix GroupBy limit push down error when buffer is too small

Fixes #4739
This PR adds a check before the LimitedBufferHashGrouper is used, to ensure that there is enough buffer capacity to apply the limit push down optimization.
A test is included that merges results for a query run on two segments, one where limit push down is applied and the other where the optimization is skipped because of insufficient buffer capacity.

LGTM after the latest changes.


Creating cluster fails with Windows password validation error - every time

Running az 2.0.68 with aks-preview 0.4.5.
az aks create step fails with a validation error for the Windows admin password:
validation error: Parameter 'ManagedClusterWindowsProfile.admin_password' must conform to the following pattern: '^(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%\\^&\\*\\(\\)])[a-zA-Z\\d!@#$%\\^&\\*\\(\\)]{12,123}$'.

The doc say that if this happens, to try again with a new RG in a new region, but that doesn't work for  me. I've tried multiple RGs in East and West US and West Europe, using ever more complex passwords, and I get this error every time.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6de70aec-9ebe-5655-6657-5c171de374b1
Version Independent ID: c9eb35c8-8703-511b-987b-e2a441be2e47
Content: Preview - Create a Windows Server container on an Azure Kubernetes Service (AKS) cluster
Content Source: articles/aks/windows-container-cli.md
Service: container-service
GitHub Login: @TylerMSFT
Microsoft Alias: twhitney

Hi @sixeyed Thank you for your feedback! We will review and provide an update as appropriate.

I got the same but fixed it by changing the password to a new one that uses only allowed characters and then everything worked!

@sixeyed
Can you set a password with the below constraints and let me know.
These requirements apply for the windows virtual machines.

Password must have 3 of the following: 1 lower case character, 1 upper case character, 1 number, and 1 special character.
The value must be between 12 and 72 characters long.

@asicoe I hope your password satisfied the above requirements. Let me know

#please-close

Tried few minutes ago with different password combinations but none of them matched!
Tried even with example from Microsoft sample, "P@ssw0rd1234!".
It has one lower case character, it has one uppercase character, it has 1 number, it has 1 special character, it is longer than 12 and lower than 72.
Still not working!

@andreivirgiltudor
can you give me the command which you tried and the error which you got

First I've issued
$windowsPassword = "RES4Ever!"
$windowsUser = "WindowsMaster"
az aks create --resource-group k8s-win --name k8s-win-test --node-count 1 --enable-addons monitoring --kubernetes-version 1.14.0 --generate-ssh-keys --windows-admin-username $windowsUser --windows-admin-password $windowsPassword --enable-vmss --networ
k-plugin azure

Which had this output:
The behavior of this command has been altered by the following extension: aks-preview
Finished service principal creation[##################################]  100.0000%validation error: Parameter 'ManagedClusterWindowsProfile.admin_password' must conform to the following pattern: '^(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%\\^&\\*\\(\\)])[a-zA-Z\\d!@#$%\\^&\\*\\(\\)]{12,123}$'
```.

So I ran this command (first password did not match length requirement):

$windowsPassword = "P@ssw0rd1234"
az aks create --resource-group k8s-win --name k8s-win-test --node-count 1 --enable-addons monitoring --kubernetes-version 1.14.0 --generate-ssh-keys --windows-admin-username $windowsUser --windows-admin-password $windowsPassword --enable-vmss --networ
k-plugin azure

The behavior of this command has been altered by the following extension: aks-preview
validation error: Parameter 'ManagedClusterWindowsProfile.admin_password' must conform to the following pattern: '^(?=.[a-z])(?=.[A-Z])(?=.[!@#$%\^&\\(\)])[a-zA-Z\d!@#$%\^&\*\(\)]{12,123}$'

So I've decided to find a matching password on [REGEX101](https://regex101.com) against regular expression from error message: `^(?=.*[a-z])(?=.*[A-Z])(?=.*[!@#$%\\^&\\*\\(\\)])[a-zA-Z\\d!@#$%\\^&\\*\\(\\)]{12,123}$`
Got no match.

Few minutes ago I've isseud again:

$windowsPassword = "P@ssw0rd1234!"
az aks create --resource-group k8s-win --name k8s-win-test --node-count 1 --enable-addons monitoring --kubernetes-version 1.14.0 --generate-ssh-keys --windows-admin-username $windowsUser --windows-admin-password $windowsPassword --enable-vmss --networ
k-plugin azure
The cluster is now created!

I can't find reason why it failed before!

ok. If you face the issue again please open a new issue and then we will troubleshoot.


Strip non-breaking and zero-width spaces in collapseWhitespace helper

We noticed when migrating some assertions in our tests from an internal whitespace helper to qunit-dom's hasText that we had some failures. It was showing a mismatch due to collapseWhitespace not collapsing non-breaking spaces and zero-width spaces.

@patocallaghan see discussion in #50 😁

@CvX Oh interesting, on reflection I agree with that, we shouldn't strip non-breaking spaces from within actual content.
On the other hand, I do think we should strip non-breaking spaces from the start and end of a string as that's exactly how ES5's trim() works. That was more the use case that was affecting my tests.
Also thinking about it, removing the zero-width space doesn't seem appropriate here either, so I'll probably remove that.

I'd agree with using trim() instead of replacing space characters at the beginning and end, but then again I'm biased against matching any whitespace. 😉

@Turbo87 On consideration I think you are right with regards zero-width space and in-content html characters.
On the other hand, would you consider any sort any variant of hasText which would use .trim() to strip whitespace from the beginning and end of the string, possible as an option? For example, chai-dom offers this behaviour on .text via a flag.

On the other hand, would you consider any sort any variant of hasText which would use .trim() to strip whitespace from the beginning and end of the string, possible as an option? For example, chai-dom offers this behaviour on .text via a flag.

my goal is to keep the API as simple as possible, and I personally haven't yet ran into any situation where I would have needed that. can you give a few real-life examples of where this would be useful?

@Turbo87 sure I understand. I'll see if I can come up with real-life examples 👍

So I managed to convert all of our test cases and it wasn't as much of an issue as I initially thought. I managed to use containsText in the cases where .trim() may have been an issue. Closing this PR.


speech_contexts parameter of Sample.long_running_recognize() limited to 50 phrases?

The speech_contexts parameter to long_running_recognize method of the Sample has a description that starts with:

A list of strings (max 50)...

Meanwhile, the true API limits for this parameter are described here: https://cloud.google.com/speech/limits#content
and specifically say:

Phrases per request 	500
Total characters per request 	10,000
Characters per phrase 	100

Does the python client API impose additional limits (50 strings) then? Or is it just an error in the description of this parameter and we can use 500 strings with total 10000 characters and 100 characters per string?

OS type and version
Linux 4.10.11
Python version and virtual environment information python --version
3.6.0
google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze
Version: 0.24.0
Stacktrace if available
n/a
Steps to reproduce
n/a
Code example
n/a

@lukesneeringer Do you know the answer to this? Could you point an API owner to this question?

Since this ticket was filed, we have released a new Speech library which entails effectively a complete rewrite. Since it is very likely to solve this issue, I am closing this. However, please feel free to reopen if my closure is premature.
Note: Make sure you are using google.cloud.speech.SpeechClient, as the old Client still exists as a deprecated artifact for a little longer.


Should shared.js' createTokenCache not recreate an existing cache?

Context: I'm writing a custom rule and would like to use the forEachLine method (#134). That method uses the tokenCache created by createTokenCache.
If it just so happens that a custom rule's version of markdownlint is different from the package's, then forEachLine will crash as its version of tokenCache won't exist. The solution is to manually call shared.createTokenCache() in the rule... but that seems wasteful for the general case of everyone using the same markdownlint version.

Per #134, these functions are not public at present (partly due to challenges like this), so this scenario is not one that should be encountered.


Comparison with quantity not working

Example:
x = Quantity(jnp.array([1., 0, -1.0]), u.kpc)
x > 0.0  # returns incorrect boolean array
x > 1.0  # should fail, instead returns incorrect boolean array

The problem appears to be
@register(lax.gt_p)
def _gt_p_qv(x: AbstractQuantity, y: ArrayLike) -> ArrayLike:
     try:
         xv = ustrip(one, x)
    except UnitConversionError:
         return jnp.full(_bshape((x, y)), fill_value=False, dtype=bool)

    # re-dispatch on the value
    return qlax.gt(xv, y)

which appears in many of the comparison operators.
Partly solving this is trivial. We could just move the xv = ustrip(one, x) out of the except block. This would fix x > 1.0 comparisons.
But then x > 0 would also fail. Fully solving this is tricky since we can't concretize y for conditional logic.

Solved it! equinox.filter_jit provides the magic sauce.
     x = eqx.error_if(  # TODO: customize Exception type
        x,
        x.unit != one and jnp.logical_not(jnp.all(y == 0)),
        f"Cannot compare Q(x, {x.unit}) > y (except for y=0).",
    )
    return qlax.gt(x.value, y)  # re-dispatch on the value

We lose the UnitConversionError, which isn't ideal, but maybe we can push a PR to equinox to allow for customizing the exception class. Either way, I think that's a secondary concern compared to the existing bug.


DownloadableFileSource shouldn't use url in cache-key

See original issue on GitLab
In GitLab by [Gitlab user @gokcennurlu] on Jan 17, 2019, 18:11
Summary
I have many large and small files as remote sources. After changing URLs to mirrors, which I know provides the same file with same contents, BuildStream considers this RESOLVED instead of CACHED, mostly because this line:
https://gitlab.com/BuildStream/buildstream/blob/ce91ce5d2eb73295d1862aa8514e631b6231300e/buildstream/plugins/sources/_downloadablefilesource.py#L95
Because it builds the mirror directory via this logic:
    def _get_mirror_dir(self):
        return os.path.join(self.get_mirror_directory(),
                            utils.url_directory_name(self.original_url))

    def _get_mirror_file(self, sha=None):
        return os.path.join(self._get_mirror_dir(), sha or self.ref)

I expected BuildStream to use file's content only for such work.
Steps to reproduce
One can create a remote source, fetch it, and then change the url :.
What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

Possible fixes
Maybe we can consider changing _get_mirror_dir and get_unique_key and removing urls role there.
Other relevant information

BuildStream version affected: /milestone %BuildStream_v1.x

In GitLab by [Gitlab user @tristanvb] on Jul 1, 2020, 06:52
marked this issue as related to #569


GetVOD does not work

i have this error when using getVOD:
Error: Twitch returned status code 403
at IncomingMessage. (/sandbox/node_modules/twitch-m3u8/index.js:78:14)
at IncomingMessage.emit (events.js:412:35)
at endReadableNT (internal/streams/readable.js:1333:12)
at processTicksAndRejections (internal/process/task_queues.js:82:21)

Same issue

It seems that Twitch has disabled a number of APIs that this, and a few other solutions, use to get the HLS stream URL. I haven't found any working solution that gets around these disabled API endpoints.

what is streamlink doing?


Android 5.1 problem on "Pick A Time"

On Android 5.1 it seems the "onItemSelected" is not fired after once selected.
Have tried it with your sample app and there is the same problem.
How to reproduce on 5.1:

Select the time and select "Pick a time".
Select a time that is not a smart date time for example 13:15
You see the toast message "time selected ..."
Now tap again on the time spinner and select again "Pick a time".
Select a time different to a time of the last one
No toast message with the new time is appearing

Can you check this whats going on in "Android 5.1"!
Also a strange issue is that the 24 hour mode is not working in 5.1!
It seems it takes always the setting from the system.
On a 24 hour system if you set 12 hours you will still get 24 hours.
The method "setTimeFormat is ignored in 5.1 !?!
I haven't found any changes to this in the offical docs from 5.1!

OK this is bad, I thought that maybe once it worked, it would at least be compatible in future Android versions...
Well good you noticed it, I'll take a look at it when I get the time (though I'm quite busy these days). It might need an additional hack (first resetting the selection, then waiting and blocking that selection callback and then reselecting the temporary item)...

There is your fix, it turned out that in Android 5.1 they somehow group multiple spinner selection callbacks together, and only the last one will actually call the onItemSelected listener (which is actually a lot nicer!). That broke my temporary selection workaround because it was using a simple int counter, now it is way more robust ;).
As for your issue with the 12 hour mode not being set, I can't reproduce that on my N5. I called datePicker.setTimeFormat(new SimpleDateFormat("hh:mm a")); directly after seting the listener in the sample's onCreate() method and it worked correctly. Can you point me to how exactly you're calling that method?

Thanks for the quick fix!! Great!
Will merge this into my code.
I will recheck the problem with the 12 hour mode.

OK good, I tried to check the 12 hour bug again and now I can't seem to reproduce it...

Forget the problem with 12 hours. This was my mistake.
I thought this was with 5.1 but it was since my last update of the app.
Closing and thanks for your support.


[ZEPPELIN-1422][zeppelin-interpreter] fix InterpreterProperty equals methods, add null check

What is this PR for?
in PR https://github.com/apache/zeppelin/pull/1382,
there is a small bug that InterpreterProperty.equals do not check null.
jdk lib request the null check otherwise it has the risk causing java collections operation throw exception.
so I fix it.
What type of PR is it?
Bug Fix
Todos
N/A
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1422
How should this be tested?
existing test.
Screenshots (if appropriate)
Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

LGTM and merge if there're no more discussions.

LGTM


0.11 release blog

"Fixes #issue-number" or "Add description of the problem this PR solves"
Proposed Changes

Looks good. Thanks Dan!


HTML code of the infowindow image header is not displaying the activated fields

The HTML code of the image header infowindow option, is not showing the activated fields of the interface menu.
Steps:

Select the fields of the dataset that you want to display in the image header infowindow:

Click in the HTML button, the infowindow HTML code is not showing the activated fields or the correct code:

If we do the same operation with selecting the light infowindow template, the fields are displayed in the HTML code:

Only happening with the image type or with the colored header too?

this is working as expected, needs to be implemented the option in the fields to select the header image and the title

first field is the image
second field is the title
third field and on are the fields in the popup content

closing then


[MODFIN-358] - Add tenantId to the locations schema for restrict by Fund functionality

Purpose
https://folio-org.atlassian.net/browse/MODFIN-358
Approach
Added migration to update DB according to new schema.
Learning
Pre-Merge Checklist:
Before merging this PR, please go through the following list and take appropriate actions.

Does this PR meet or exceed the expected quality standards?

[ ] Code coverage on new code is 80% or greater
[ ] Duplications on new code is 3% or less
[ ] There are no major code smells or security issues

Does this introduce breaking changes?

[ ] Were any API paths or methods changed, added or removed?
[ ] Were there any schema changes?
[ ] Did any of the interface versions change?
[ ] Were permissions changed, added or removed?
[ ] Are there new interface dependencies?
[ ] There are no breaking changes in this PR.

If there are breaking changes, please STOP and consider the following:

What other modules will these changes impact?
Do JIRAs exist to update the impacted modules?

[ ] If not, please create them
[ ] Do they contain the appropriate level of detail?  Which endpoints/schemas changed, etc.
[ ] Do they have all the appropriate links to blocked/related issues?

Are the JIRAs under active development?

[ ] If not, contact the project's PO and make sure they're aware of the urgency.

Do PRs exist for these changes?

[ ] If so, have they been approved?

Did you modify code to call some additional endpoints?

[ ] If so, do you check that necessary module permission added in ModuleDescriptor-template.yaml file?

Ideally, all the PRs involved in breaking changes would be merged on the same day
to avoid breaking the folio-testing environment.
Communication is paramount if that is to be achieved,
especially as the number of inter-module and inter-team dependencies increase.
While it's helpful for reviewers to help identify potential problems,
ensuring that it's safe to merge is ultimately the responsibility of the PR assignee.

@imerabishvili here we introduced API breaking change, so need to increate major version of fund interface


[FR] Randomize wg internal subnet address

Is your feature request related to a problem? Please describe.
Currently pivpn sets the same internal wireguard subnet <IP_ADDRESS>/24 for each installation and does not allow in the installer flow to change that subnet address. User unaware of that makes more than one install and the result is he cannot connect two tunnels to that two locations because of subnet conflict.
Describe the solution you'd like
randomize the subnet C part during installation 10.C.0.0 and make sure that is not conflicting with any subnets on current device interfaces or subnets in route table.
Describe alternatives you've considered
As there is no easy way to change this subnet after installation that needs to be addresed during installation. Tried editing pivpn vars and wg0.conf with no luck whatsoever. Tried those with no success: https://www.reddit.com/r/pivpn/comments/fysb4y/changing_the_pivpn_wireguard_default_network_space/
Additional context
Not needed, hope above is obvious.

Hi Thanks for your Feature Request, just a note that there are some things you should have read and you did not:
https://docs.pivpn.io/#general-guidelines
https://github.com/pivpn/pivpn#feedback--support
Mainly
https://github.com/pivpn/pivpn#feature-requests
I don't think the solution you propose is optimal ...
But providing an Advanced option where more savvy users can tweak that setting sounds a little bit more sane.
@orazioedoardo what you think?
Moving this to discussions


Question: Which VAT calculation method is required by mollie?

Hi again,
separate from the VAT issues in general with mollie. In Shopware 6 it is now possible to setup which VAT calculation method to use. If it should be calculated horizontally or vertically.
I would like to know which one, I should use with mollie or does it support both?
Maybe changing this setting would actually fix the VAT calculation thing as e.g. vertical will calculate the VAT based on the last total.

Hi AndreasA, the Mollie plugin should work in both scenarios. The only VAT issues that are known to us, are because of wrong VAT calculation in Shopware for discounts. Shopware is looking into this at the moment. In the meantime, we can provide a hotfix that fixes this on the plugin side (but that's not the way this should be solved). If you want, we can provide this hotfix.

@milan-kiener No, you are correct regarding tax calculation should be fixed with SW. I just wanted to know, if one of those two methods should be used or if it does't matter. It is not just discounts but in general summation of taxable amounts that is the issue I think.
As it doesn't matter. I am fine with it.


[WIP] Add functionality to convert Cicero DOM To Pdf [WIP]

Issue #51

Changes

Made changes in markdown-pdf package. Added toPdf function in PdfTransformer Class

Flags

Coverage remained the same at 88.594% when pulling 3a26ff5820b2fabb958f54809d34cc6538fc5575 on Anubhavshakya:ciceromarkdom-To-pdf into dbaebb40a759b091e490919be720a051cff000c1 on accordproject:master.

hi @Anubhavshakya Any progress on this?

@Anubhavshakya I'm closing as stale, feel free to come back to it though!


Addition to fnv1a_xor_hashes.txt

msmpeng :<PHONE_NUMBER>225757871
aka Microsoft Security Essentials

10374841591685794123 : win64_remotex64
which i think is to do with IDA Pro ..?

640589622539783622 : xagtnotif
10063651499895178962 : mssense

Thanks! Added in #19