Encoding dependency on 1 or more generated file

The documentation on need says:

This function should not be called with wildcards (e.g. *.txt - use getDirectoryFiles to expand them),

But this "doesn't work" if the files don't exist already -- the returned list is empty.[1] For example,
let foogen = "bar" </> "*.foo"
foogen %> const (cmd [Cwd "bar"] "python3 generate.py")
phony "run-myscript" $ do
  need =<< getDirectoryFiles "bar" ["*.foo"]
  -- need trivially succeeds if the files don't exist
  cmd "myscript"

Whenever you want to generate some new files and use them, both myscript and generate.py need to be updated. In the current situation, it seems like the actual file names also need to be hard-coded in the build system (indirectly computing them is not an option because they're generated from scratch, not from some other files).
However, the build system doesn't need to care about this. All it needs to know is that generate.py will make 1 or more .foo files, which are needed by myscript (because myscript will anyways fail if it doesn't find the files it needs). Is there a way of encoding predicates in need?
You might be wondering: what's the use case here? In my case, the python script is generating some dummy files for benchmarking and the myscript is running some benchmarks on them. While myscript itself could trigger the generation if the files don't exist, I was considering separating out the logic into the build system so that files always exist when you start running the benchmark.
I suppose one way to "fake" this would be hard-code 1 file name (which we know will not change) and actually generate everything else as well (but don't track the other files). Would you suggest using that approach or something else?
[1] Yes, I realize that * encodes 0 or more, so I've put "doesn't work" in quotes.

Yes, there is also a note on getDirectoryFiles that:

You should normally only call this function returning source files

So calling it on a file that is generated is a bad plan.
My suggestion is to create a .stamp file, so have gen.stamp as the thing produced by python3 generate.py, and just touch it at the end. Then have run-myscript depend on gen.stamp. An alternative approach would be to have generate.py write to stdout where it put the file. My concern is that if generate.py and myscript are communicating via files, and the files aren't known, if you don't clean up between runs it's going to give the wrong answer.

I think I'll hard-code the file names for now, but I can switch over to the stamp solution once there are many files.

if you don't clean up between runs it's going to give the wrong answer.

That's a good point, I hadn't thought of that at all. Now my code looks like:
    phony "generateFoos" $ cmd "python3 generate.py"

    "*.foo" %> \_ -> need ["generateFoos"]

    "generate.py" %> \_ -> need ["generateFoos"]

    phony "run-myscript" $ do
      -- if either thing changes, their rules should fire
      need ["generate.py", "x.foo"]
      cmd "myscript"

I'm not sure if I've created a race condition though (because both need rules might run in parallel), or whether the two will resolve correctly and run generateFoos only once (assuming I modified generate.py and deleted x.foo).

generateFoos will be called at most once - there is no race condition. They might run in parallel, but Shake correctly starts each rule with a lock held, then has the other execution wait for it.

Ah okay. That's one less concern. On a related note, I was at first expecting that command to not re-run generateFoos in case both generate.py and x.foo are unchanged since we last ran run-myscript. However, that doesn't seem to be the case.

Is it because generateFoos is phony, so it is always counted as "dirty" (hence we must re-build generate.py and x.foo always)?
More generally, if X needs Y which is phony, then the rule for Y will always fire if you transitively have a need [X] somewhere?

Sorry for not responding earlier, I missed this notification.

Yes. Phony rules always rebuild.
Yes, Phony rules always change, so depending on a phony ensures ordering (the phony completes first), but otherwise is equivalent to alwaysRerun.

No worries, that resolves it. Thanks!


Automate adding vector extension to 'kai' db

Once the db is up and running, I manually add the following statement before loading the store
CREATE EXTENSION IF NOT EXISTS vector;
These are the steps I follow in my local:

Create db container
podman exec -it postgresql_database bash
psql -U postgres
 \c kai
 CREATE EXTENSION IF NOT EXISTS vector;
exit db and container

This should automatically occur when the tables are created on server start: https://github.com/konveyor-ecosystem/kai/blob/main/kai/data/sql/create_tables.sql#L1

@fabianvf Thank you. I did not notice that. I added that to avoid an error but that was before you got some changes in

@fabianvf - looks like we need to be super user to create that extension
Connected to the PostgreSQL server.
Error initializing PSQLIncidentStore: permission denied to create extension "vector"
HINT:  Must be superuser to create this extension.```

Hmm, fun, I'll see if I can figure out a way around that

Using make target to run postgres db works. Closing this issue


Add Tracing in SqlClient

SqlClient doesn't have any tracing in .Net Core. All the Bid Tracing API calls were removed during the port to Core.
One of the options is to use EventSource to emit traces during runtime.
Something similar was done by @stephentoub in System.Data.Common .
Ref: https://github.com/dotnet/corefx/blob/master/src/System.Data.Common/src/System/Data/Common/DataCommonEventSource.cs
cc @corivera

As recently announced in the .NET Blog, focus on new SqlClient features an improvements is moving to the new Microsoft.Data.SqlClient package. For this reason, we are moving this issue to the new repo at https://github.com/dotnet/SqlClient. We will still use https://github.com/dotnet/corefx to track issues on other providers like System.Data.Odbc and System.Data.OleDB, and general ADO.NET and .NET data access issues.

@saurabh500 @David-Engel @roji @danmosemsft @vancem
Would like to bring this topic to light again for Microsoft.Data.SqlClient.
For best user experience, what would you suggest we implement in both projects (NetFx and NetCore) that is easy to configure and integrates well with .NET ecosystem?

@cheenamalhotra when I looked a couple years back, I think I saw that SqlClient was already generating events via DiagnosticSource, can you confirm? Would this be about generating EventSource events in addition to that?
Note that unrelated to tracing, there's also the question of the new .NET Core performance counters, which are yet another thing. I did this work for the recently-released Npgsql 4.1.0, and am planning to open a discussion issue for other providers as well.

Hi @roji
'DaignosticSource` is in use for generating events in NetCore but its not as extensive as Bid Tracing in NetFx and its not applicable in NetFx too. NetFx used to have BID Tracing, but that's not possible to implement in NetCore for cross platform support.
Now in Microsoft.Data.SqlClient we are looking for a uniform solution that we can implement understanding what can be consumed for Application Insights and that can be made available from driver, irrespective of target framework.

@cheenamalhotra I'm not super familiar with this domain... However, unless I'm mistaken EventSource exists in .NET Framework too, and could be the right thing if you need the same tech to support both Core and Framework. You may want to read this useful comment about DiagnosticSource vs. EventSource.
Note also https://github.com/dotnet/corefx/issues/21160, which is a stalled issue about standardizing DiagnosticListener for ADO.NET providers.

Based on the amount of tracing information that was being emitted by NetFx Event Source is the way to go. We are not looking to capture the whole application context, but the internal execution information during different API execution in SqlClient.
The usecase for BID tracing is that we capture BID traces for postmortems, to know what happened at granular level inside the driver, when problems occur at customer's end. CSS usually asks the customer to enable the tracing and analyses the traces or sends it to the Product team to take a look and explain the failure behaviors.
The addition of DiagnosticSource to SqlClient in .Net core can help application developers look inside the success or failures of SqlClient from the perspective of the applications. However for a deep dive into things like why did the client send out an attention packet after issuing a query, Event Source like tracing mechanism is what we are looking at.
FWIW, System.Data.Common uses EventSource as a replacement for BID as well. @stephentoub had done the migration for System.Data.Common when he brought in the code to provide parity with .Net framework.
Also SqlClient NetFx should have the same DiagnosticSource capabilities that .Net Core offers for parity.

@roji @saurabh500
Thanks for details. So I believe we should be looking at "EventSource" in terms of replacement for BID tracing for High Volume traces that can apply to both .NET Framework and .NET Core, starting with replacing NetFx BID calls, porting over to NetCore. So that can be called as Deep Tracing.
"DiagnosticSource" on the other hand can be labelled as Shallow Tracing, to provide logging capabilities for Application Developers and App Insights, as currently implemented in NetCore to be ported over to NetFx.
I do see both are useful and have their own purpose and benefits, but looking at the amount of work this will bring, it seems unlikely to complete in 1.1.
Please confirm if we all agree on implementing both styles of tracing for SqlClient drivers (NetFx and NetCore), and we shall accommodate this in next release plan.

It does seem that both styles of tracing may be relevant, although it also seems that there is considerable overlap in what they provide, and I personally still don't have a 100% clear picture of which should be used for what.
Regardless, what would be really great is some sort of quick spec/writeup on which events you're planning to emit and when, before implementation. That would allow us to have a more concrete conversation about it possibly align across different ADO providers.
Note: in addition to https://github.com/dotnet/corefx/issues/21160, https://github.com/dotnet/corefx/issues/29534 is an additional issue I filed at the time. tl;dr SqlCient's current implementation of DiagnosticSource seems like it doesn't really follow best practices. I understand that the priority here is to bring over what exists from Core to Framework and not to change things, but this could also be a good moment to rethink the DiagnosticSource support itself. If you're willing to do this, I'd be happy to work together and produce a spec.

Hi @roji
It would be great if you can send us a spec/writeup for the same.
We'd also like to address this nicely with best practices.

It seems that the focus here is on app insights. I want to make sure that the Visual Studio Diagnostic Tools are not ignored as they are more helpful before the app is deployed.

Hi @jwooley
Could you please elaborate a bit about how Visual Studio Diagnostic Tools pick Diagnostic Logs from library and if "EventSource" and "DiagnosticSource" would suffice the requirement?
If not, then what is the desired logging/tracing that can be implemented in SqlClient uniformly for .NET Framework, .Net Core, and .NET Standard targeting applications?

I'm not sure what triggers items to appear in the Diagnostic tools internally, but liked the ability that we had with .Net to see the ADO requests in there. This ability seems to have been lost in Core. Assuming adding back in proper source calls would fix this issue, I'm fine with the change. The conversation was primarily around supporting App Insights and I wanted to make sure that the Diagnostic tools weren't forgotten about.

Request from @davidmikh [dotnet/runtime#32774]:
When using the WriteCommandBefore and  WriteCommandAfter DiagnosticSource events defined in this file it doesn't seem like there's a way to associate commands with the transactions they were a part of. In the WriteCommand events you're able to tell if the command was part of a transaction based on the Transaction property of the SQLCommand being not null, however it doesn't seem like there's a way to tell which transaction the command was a part of since the transaction object itself doesn't have any public properties to help identify it from other transactions using the same connection. As a result you can't tell which commands were grouped together in which transaction, you can only tell if a command was part of a transaction at all. Would it be possible to add a property to the transaction object itself or to one of the WriteCommand DiagnosticSource events so that you can associate which commands were part of the same transaction together?
It would also be ideal if there was a way to associate the WriteCommand events with the WriteTransaction events for the queries that are part of a transaction, however I'd be happy with just being able to tell which commands are part of the same transaction.

Closing issue since EventSource tracing is now added to SqlClient.


Deleting data from Hive Box when the data is stored as a list at the 0th position

Question
I have been storing paths of some files in Hive by using the below code, the code basically takes a PDF file's path from pdfsBox and puts that at 0th position of starred box if a user wants to star that file. All the paths of starred PDF's are stored at 0th location in a list (a list of paths). (The paths are stored in a similar manner in pdfsBox)
onPressed: () async {
     File file = await File(
     pdfsBox.getAt(0)[index]
      );
     final path = file.path;

    List<dynamic> files = Hive.box('starred').getAt(0);
    if (files.contains(path)) {
        Scaffold.of(context).showSnackBar(SnackBar(content: Text('Already a starred document')));
    } else {
        files.add('$path');
        Hive.box('starred').putAt(0, files);

        Scaffold.of(context).showSnackBar(SnackBar(content: Text('Added to starred documents!')));
   }
}

The problem comes when I am trying to un-star the PDF, I can't figure out how to delete the path from the Hive Box. As all the data is stored at 0th location as a list, if I use
deleteAt(0)

it deletes the complete list and naturally throws an error on the screen where I am displaying the starred documents that there is no data. Then I tried using
deleteAt(0)[index]

but this is not a valid expression.
Ideal output
I want to delete a specific element from the list stored at 0th position.
I have been looking everywhere for the solution but couldn't find one, it would be great if someone here can help me with this. Thank you so much!
Version

Platform: Windows
Flutter version: Flutter 1.22.6 • channel stable
Hive version: ^1.4.4+1
hive_flutter version: ^0.3.1

I found a solution, for people looking for the solution use the code snippet mentioned below insteade of delete or deleteAt
Hive.box("YourBoxName").getAt(0).removeAt(index);

I will be closing the issue.


Azure UI - Collapse All

I think it'd be nice to have a way to collapse all the nodes.  The ribbon might be a bit too busy to add another button though.

Done


DOC: Added qgrid logo to README and docs index

Not sure how to have them share the directives instead of copying.  Let me know if there's a preferable method.

This looks good to me.  Any objections @TimShawver?

Looks good!  I triggered a readthedocs build so the icon is there too.

Awesome, thanks @TimShawver !


Nancy.ViewEngines.ViewNotFoundException: Unable to locate view 'index'

I created a self hosted project by using the Nancy Self host with razor project template. When I hit run all i get is this error message:
Nancy.RequestExecutionException: Oh noes! ---> Nancy.ViewEngines.ViewNotFoundException: Unable to locate view 'index'
Currently available view engine extensions: sshtml,html,htm
Locations inspected: views/Index/index-en-ZA,views/Index/index,Index/index-en-ZA,Index/index,views/index-en-ZA,views/index,index-en-ZA,index
Root path: C:\Sandbox\TFS\MusicBox\MusicBox.Debug\bin\Debug
I haven't made any changes to what was created when I created the project, and I have compared to another project I Created last year with nancy and there are no differences..
So what gives, why does this project template not work?

similar issue when I set two projects, and hand in a HostConfiguration with UrlReservations.CreateAutomatically = true in both, and two different uris, localhost:3579 and localhost:3580.  The first project succeeds the second one - identical except uri, fails.
        var uri =
            new Uri("http://localhost:3579");
        var uris = new[] {uri};
        var hostConfig = new HostConfiguration();
        hostConfig.UrlReservations.CreateAutomatically = true;

        using (var host = new NancyHost(hostConfig, uris))

I am also having this problem. Has a really simple cshtml file in my app, added the NuGet Razor view engine and it says
Nancy.RequestExecutionException: Oh noes! ---> Nancy.ViewEngines.ViewNotFoundException: Unable to locate view 'razor.cshtml'
Currently available view engine extensions: sshtml,html,htm
Locations inspected: views/v1/v1/razor.cshtml-en-US,views/v1/v1/razor.cshtml,v1/v1/razor.cshtml-en-US,v1/v1/razor.cshtml,views/v1/razor.cshtml-en-US,views/v1/razor.cshtml,v1/razor.cshtml-en-US,v1/razor.cshtml,views/v1/razor.cshtml-en-US,views/v1/razor.cshtml,v1/razor.cshtml-en-US,v1/razor.cshtml,views/razor.cshtml-en-US,views/razor.cshtml,razor.cshtml-en-US,razor.cshtml
Root path: G:\Development\Axiom Pro Future Architecture Test 1\Source\Host\bin\Debug
If you were expecting raw data back, make sure you set the 'Accept'-header of the request to correct format, for example 'application/json'
   at Nancy.ViewEngines.DefaultViewFactory.GetRenderedView(String viewName, Object model, ViewLocationContext viewLocationContext)
   at Nancy.ViewEngines.DefaultViewFactory.RenderView(String viewName, Object model, ViewLocationContext viewLocationContext)
   at Nancy.Responses.Negotiation.ViewProcessor.Process(MediaRange requestedMediaRange, Object model, NancyContext context)
   at Nancy.Responses.Negotiation.DefaultResponseNegotiator.NegotiateResponse(IEnumerable`1 compatibleHeaders, NegotiationContext negotiationContext, NancyContext context)
   at Nancy.Responses.Negotiation.DefaultResponseNegotiator.CreateResponse(IList`1 compatibleHeaders, NegotiationContext negotiationContext, NancyContext context)
   at Nancy.Responses.Negotiation.DefaultResponseNegotiator.NegotiateResponse(Object routeResult, NancyContext context)
   at Nancy.Routing.DefaultRouteInvoker.<>c__DisplayClassa.b__6(Task`1 completedTask)
   --- End of inner exception stack trace ---
   at Nancy.NancyEngine.InvokeOnErrorHook(NancyContext context, ErrorPipeline pipeline, Exception ex)

I found that inside the web.config it was referring to a different version (2.0) while in the references it said version 3.0.  See here for the official answer which got me in the correct direction....
http://stackoverflow.com/questions/34493086/which-dll-version-should-i-use-razor-dll-manifest-does-not-match-0x80131040

When I copied my solution over the Internet, Windows 10 blocked some files without telling me, including the file Nancy.ViewEngines.Razor.dll. Unblocking the file through its properties context menu fixed the issue for me.


Terminal title doesn't update

Testing #199046
armhf setup
When for example running find the terminal title in the terminal panel doesn't update. It always shows qemu-arm

This behavior has been there since the beginning, we get the process name using /proc/pid/cmdline on linux refs https://github.com/microsoft/node-pty/blob/d6ce76a801f68730f7204b7b27bb21dbf8e63001/src/unix/pty.cc#L666 and when running binaries via qemu, the cmdline will always be of the format /usr/bin/qemu-<arch> ...
@Tyriar assigning it to you


Turn flow branches parametrizeble

Now, some scripts do directly validations for origin/main branch. Furthermore, some scripts do directly validations for feature/* and hotfix/*. It would be interesting if these branches are customizable.

Fixed by #18


Update plexus-sec-dispatcher dependency to fix potential vulnerabilities

The plexus-sec-dispatcher dependency (from org.sonatype.plexus) has not been maintained since May 2011 and potential vulnerabilities have been found. The new version is provided by org.codehaus.plexus (https://mvnrepository.com/artifact/org.codehaus.plexus/plexus-sec-dispatcher/2.0).
The asciidoc-confluence-publisher-maven-plugin should be updated with the new dependency.

Hi @cgreppin, the new release 0.15.1 containing the updated plexus-sec-dispatcher library is now available on Maven Central and Docker Hub. Thanks again for your contribution!

Hi @cstettler, Thank you for the update and information!


Automator: update istio/client-go@master dependency in istio/istio@master

Generated by Automator - 2024-06-28T00:28:33+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
eeb99de3b5eb532f6c198e0c71f331879bd0a695
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.


ElasticSearch not deployed in production ready state

By default (and without simple override) we deploy ElasticSearch in a non-production ready state by setting node.store.allow_mmap: false
See:  https://github.com/infrawatch/service-telemetry-operator/blob/master/roles/servicetelemetry/tasks/component_elasticsearch.yml#L19
Official documentation has a note that node.store.allow_mmap should be set to true. This needs to be tested in our OpenShift production style deployments to validate that the proper node descriptor counts are defined.
Documentation should be updated on how to check OpenShift nodes (MachineConfig?) have the proper resource descriptor sizing to allow for scheduling of the ElasticSearch pod.
https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html#k8s-deploy-elasticsearch

https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-pod-template.html#k8s-pod-template

Need to make sure we set a label on scheduled pods that matches the expected label by OpenShift. Update the elasticsearch_manifest.j2 file to provide the tuned.openshift.io/elasticsearch label.


Highlight selected model in project explorer

If I have a lot of models(imported from .dae) in the scene, I can not easily find the selected model in the project explorer. so if it can automatically highlight and scrolling to focus on selected mode in the project explorer, that will easy to use.
Version

0.9.3

Platform

Web Version [X]

Thanks a lot for the suggestion i think that would be a good addition to the editor.
I have to study the best way to do this.

It is now implemented, on selection the tree scrolls to the selected object


Add delete case endpoint definition to API

I had to remove the uuid format on the id field... according to OpenAPI, mongoDB's ids are not valid UUIDs: https://github.com/swagger-api/swagger-ui/issues/5281. I added a regex that matches MongoDB's ids instead
I also changed the server to return a 204, and not return the case object
Screencast:

 Implement dataserver API (minus Case entity)


azure: sometimes during off tx is necessary

Expected Behavior

Connecting to HTTPs services on Kubernetes work on azure everytime.
Current Behavior

Something like this will hang on Azure sometimes.
curl -k --compressed https://example.com/  -H "Authorization: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" > /dev/null

Possible Solution

This fixes it:
sudo ethtool -K eth0 tx off sg off tso off

Steps to Reproduce (for bugs)

Spin-up Kubernetes cluster on Azure using Tectonic Installer
Roll the dice because sometimes a cluster comes up working and sometimes it doesn't
Run the above curl against the Tectonic console from a Linux host
Hang....

Context

Some more potential explanation, though I don't totally understand it.
This could can do this too: https://github.com/weaveworks/weave/pull/2307/files#diff-4af03c67995550022d5fd9e1f8e72e8bR25

Here are tcpdumps of TX on and TX off: https://gist.github.com/philips/d680458e7cc2a36b16989a20df5ae0c2

This is a "normal" tcpdump from this machine to a cloud CDN for comparision: https://gist.github.com/philips/e3acde0cda317ec134900323cc5d924b#file-gistfile1-txt-L44-L55

We had to write a service to disable tx checksum offloading for Tectonic clusters on Azure https://github.com/coreos/tectonic-installer/pull/1586; it would be much nicer and more maintainable to do this in flannel configuration in a configmap than using systemd units.
Below are some iperf benchmarks we ran to compare how different MTU and tx checksum offloading combinations would perform. We found that disabling TX checksum offloading and keeping the default MTU does not result in a major performance penalty. By only disabling tx checksum offloading we can take advantage of any future changes in Azure network MTU changes without having to modify existing deployments.      Summary Results:
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-30.00  sec  2.58 GBytes   738 Mbits/sec  9948             sender
[  4]   0.00-30.00  sec  2.58 GBytes   738 Mbits/sec                  receiver
CPU Utilization: local/sender 0.8% (0.0%u/0.8%s), remote/receiver 1.4% (0.1%u/1.3%s)

@philips @squat Do you think this issue should stay open? Do you think this can or should be fixed in flannel? I could see a few options for resolving this

Adding a note to the docs about this issues and the steps to (manually) resolve it.
Always turning off tx checksum offloading
Trying to detect if flannel is running on azure and turning off tx checksum offloading
Just closing the issue as it's already been resolved in the tectonic installer :scream:

Oh, it's so much worse than that. It depends on the underlying HyperV OS; the networking driver for some versions of Windows support checksum offloading only for TCP (hence breaking vxlan), while newer ones support both TCP and UDP.
Given that it's not really a flannel problem, this can probably just be closed.


Feature/readme

Authors

Tawnya Wessar
Ben Clowe
Jackson Reber
Jeremy Hebert
Dan Ferber
Andrew Schaer
Jake Harris
Cara Locke

Description
Removed template boilerplate and added HRF logo, added API Documentation section with placeholders, and styled README
Loom Video
https://www.loom.com/share/23d20e92ecbe488986157df89f6958d3
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] This change requires a documentation update

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have removed unnecessary comments/console logs from my code
[x] I have made corresponding changes to the documentation if necessary (optional)
[x] My changes generate no new warnings
[x] I have checked my code and corrected any misspellings
[x] No duplicate code left within changed files
[x] Size of pull request kept to a minimum
[x] Pull request description clearly describes changes made & motivations for said changes

 Update BE README

Looks awesome, but is showing conflicts. Once we get those situated I think it'll be great!


Unknown arg mode in where.name.mode for type StringNullableFilter

https://github.com/SoftwareBrothers/adminjs-prisma/blob/76e07cbe3115489816282df9557086d8e5df7eab/src/utils/converters.ts#L59
Hello!
Faced with a problem when using the package with Prism and MySQL database. When using search by fields that do not have foreign keys, I get an error:
Unknown arg `mode` in where.name.mode for type StringNullableFilter. Did you mean `lte`? Available args:
type StringNullableFilter {
  equals?: String | Null
  in?: List<String> | Null
  notIn?: List<String> | Null
  lt?: String
  lte?: String
  gt?: String
  gte?: String
  contains?: String
  startsWith?: String
  endsWith?: String
  not?: String | NestedStringNullableFilter | Null
}

The convertFilter  function contains a strict mode: 'insensitive' setting, which causes an error when working with the MySQL
According to Prisma documentation:
MySQL uses case-insensitive collation by default. Therefore, filtering with Prisma Client and MySQL is case-insensitive by default.

mode: 'insensitive' property is not required and therefore not available in the generated Prisma Client API.

Propose to add some kind of exception for processing requests to MySQL database.
Environment:

prisma: 3.9.1
@prisma/client: 3.9.1

@serembon I removed "mode: 'insensitive'" for now so that each provider sticks to it's default behaviour. The fix is included in the latest release.


feat: add more unit tests for float parsing

the float parser change had removed some negative tests.
reintroduced some negative tests, and some more formatting tests.

After looking the tests you added, I'm not entirely sure your previous change is correct.
I played around with a floating point converter.
According to this, the hexadecimal value 0x11 is equal to 2.4e-44: all exponent bits are zero so the two bits in the mantissa only make a very small value. The equivalent of 11.0 in hexadecimal notation would be 0x41300000.
Of course it might be the case that the Autosar tool you mentioned in the previous pull request actually only represents the decimal part of a float value as hex, and is not using the in-memory format of an ieee754 number. In that case your change would be correct after all, but it might be good to add a comment explaining this.

I have added some comments in the code, i hope this is what you are expecting

Yes, that neatly explains what is going on here


Adding Instruction to install tf using Conda

It's a well know fact that the Installation of tf-gpu might become a difficult process .
To solve this issue we can use the method of installation through conda .
Conda provide a single line solution to install all the files required like cuda , cdunn etc.
I hope adding this step as a Installation process will help many user to
quickly start with tensorflow-gpu.

@googlebot I signed it!

Hi
I am waiting for a reply
Please review it .
If you want to reject you can do that also
Thanks

We don't own/test/support TensorFlow with conda, so it's hard for us to include this in the official docs.
Right @lamberta?

Hey Mark
I know you don't support the conda .
But still conda is one of the way  to use tensorflow .
There are lot of problem which the user face in installing the tensorflow-gpu .As you also have to install Cuda , Cudnn  etc .
Its better to use a single line command using conda . This will automatically install the most appropriate version of cuda , Cudnn etc .
It would be great if you include this installing process also in your official docs.
Thanks

We mention/link to Conda here in the pip install guide, but Conda is not officially supported (and often lags behind). I think the TensorFlow Docker images would be a better recommendation.
I'm open to a brief comment and link to the Conda section in the TF install guide, but including commands here seems out-of-scope.
cc: @gunan

Okay should I move on to the  "  TensorFlow Docker images " installation . If you are okay including that .

I agree, if we document conda here, then we open ourselves to questions about why conda packages are available much later than pip packages, bugs in conda due to conda using different build options.
TF builds with manylinux 2010 standard, and builds a whole ecosystem around using the same toolchains. Conda releases use a different toolchain, that can cause incompatibilities with pip packages, or custom ops, or other packages under tensorflow umbrella, like addons.
So, I am definitely against having full instructions in TF website.
If Anaconda has a page with TF instructions, we can point to there.
So, we definitely cannot accept this change.

Okay should I move on to the " TensorFlow Docker images " installation .

@shubham15gupta09 Yes, we should link to this section: https://www.tensorflow.org/install/docker#gpu_support

Hi @lamberta  @MarkDaoust @MarkDaoust
We can include these links to the tf installation instructions as Anaconda has instructions to install tensorflow .
I think it would be better to give these as a link .
https://docs.anaconda.com/anaconda/user-guide/tasks/tensorflow/
https://www.anaconda.com/blog/tensorflow-in-anaconda
https://anaconda.org/conda-forge/tensorflow


wrapper for lua_yield/lua_yieldk

Will be support it later?

Impossible to support at the moment.
Lua uses longjmp / setjmp to perform a lua_yieldk. It's extremely restrictive and does not work because performing a longjmp completely throws away any callstack information and any other such information. True coroutine support / generator support is just not there in C++, and even when we do get coroutine support I'd need to figure out a way to pushing results to Lua even after jumping out of some arbitrary execution block a Developer specifies with sol::yield() or whatever.
It can't happen without me taking some dependency on something like Boost.Fiber or writing my own coroutine library.

I would also be interested in this feature, if it ever becomes possible (assuming it's what allows Luabind's yield policy or luacppinterface's CreateYieldingFunction). I'd really like to stop using Luabind and Sol 2 is by far the best alternative I've come across. But I'm using the ability to yield from C++ to Lua to create a scripting system, something similar to this coroutine article but with C++ implementation of some of the coroutines.
Could you perhaps add a note or comparison point about this feature to your feature matrix? It took me some time to realize it's not even possible. Thanks for creating this!

I would have to look into how to make this safe in C++ again. Would you mind opening a new issue, so I have something to keep track of while I attempt this again?

Sure. Do you mean another issue for lua_yield support, or just for documentation? Just making sure I don't create duplicates.

An issue for lua_yield support in some sensible manner.


Refactor Server and Web App Code Smells

What needs to be completed?
Remove the code smells, duplication, and security vulnerabilities in the server and web app.
Success Criteria

[x] reduce code smells in server
[x] reduce duplication in server
[x] remove vulnerabilities in server
[x] reduce code smells in web app
[x] reduce duplication in web app
[x] remove vulnerabilities in web app

In addition to removed duplicate code, also removed duplicate API endpoint that gets less of the same data from another API endpoint.


Vulnerability roundup 84: ant-1.10.2: 1 advisory

search, files

[ ] CVE-2020-1945 CVSSv3=9.1 (nixos-20.03, nixos-unstable)

Scanned versions: nixos-20.03: 82b5f87fcc7; nixos-unstable: 0f5ce2fac0c. May contain false positives.

Bumping to 1.10.9

Updating nixos-20.03 is sufficient. Other branches have alrady unaffected versions.

20.03 is frozen and EOL.
Closing.


【function request】:Added GGUF for chatglm3

Hello, would you like to add quantitative support for chatglm3?
Below is the link address:
https://huggingface.co/mradermacher/chatglm3-6b-GGUF/tree/main

Thanks for sharing this with us


PRO-17495: ProvarDX: Added Generate License Key Command

Added new command:
-   ProvarDX: Generate License Key: Command to register the user and generate a temporary license key.

It would save you a lot of work for no reason so I think it's worthwhile.
On Thu, Jan 28, 2021 at 4:52 PM riteshnahar<EMAIL_ADDRESS>wrote:

@riteshnahar commented on this pull request.
In src/commands/generateLicenseKey.ts
https://github.com/ProvarTesting/provardx-vscode/pull/8#discussion_r566202039
:

All rights reserved.

Licensed under the BSD 3-Clause license.

For full license text, see LICENSE.md file in the repo root or https://opensource.org/licenses/BSD-3-Clause

*/

+import * as vscode from 'vscode';
+import { messages } from '../messages';
+import { HttpService } from '../utils/httpService';
+
+const DEFAULT_VALIDATE_INPUT = (value: string) => {

return value === '' ? messages.generate_license_error_empty_field : null;
+};

+const VALIDATE_PHONE_INPUT = (value: string) => {

const regex = /^\d+$/;
return !regex.test(value) && value.length

Ritesh, does this help at all?
https://github.com/catamphetamine/libphonenumber-js
@paulrichards-provar https://github.com/paulrichards-provar it's MIT
licensed
@PeteProvar https://github.com/PeteProvar we can add this npm package
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ProvarTesting/provardx-vscode/pull/8#discussion_r566202039,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQ7XWQLFP6ZWQ2ZGYYGUKHLS4GB3NANCNFSM4WVFTCZQ
.

--
Pete Haughie
provar
Web: provartesting.com
Address: Head Office: White Collar Factory, Old Street Yard, London, EC1Y
8AF
https://t.xink.io/Tracking/Index/mQsCACIoAQA_c7gA0
Provar has an average 4.5 / 5 rating on G2. Click here to read our latest
user reviews. https://t.xink.io/Tracking/Index/mQsCACIoAQA_c7gA0

It would save you a lot of work for no reason so I think it's worthwhile.
On Thu, Jan 28, 2021 at 4:52 PM riteshnahar<EMAIL_ADDRESS>wrote:

@riteshnahar commented on this pull request.
In src/commands/generateLicenseKey.ts
https://github.com/ProvarTesting/provardx-vscode/pull/8#discussion_r566202039
:

All rights reserved.

Licensed under the BSD 3-Clause license.

For full license text, see LICENSE.md file in the repo root or https://opensource.org/licenses/BSD-3-Clause

*/

+import * as vscode from 'vscode';
+import { messages } from '../messages';
+import { HttpService } from '../utils/httpService';
+
+const DEFAULT_VALIDATE_INPUT = (value: string) => {

return value === '' ? messages.generate_license_error_empty_field : null;
+};

+const VALIDATE_PHONE_INPUT = (value: string) => {

const regex = /^\d+$/;
return !regex.test(value) && value.length

Ritesh, does this help at all?
https://github.com/catamphetamine/libphonenumber-js
@paulrichards-provar https://github.com/paulrichards-provar it's MIT
licensed
@PeteProvar https://github.com/PeteProvar we can add this npm package
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ProvarTesting/provardx-vscode/pull/8#discussion_r566202039,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQ7XWQLFP6ZWQ2ZGYYGUKHLS4GB3NANCNFSM4WVFTCZQ
.

--
Pete Haughie
provar
Web: provartesting.com
Address: Head Office: White Collar Factory, Old Street Yard, London, EC1Y
8AF
https://t.xink.io/Tracking/Index/mQsCACIoAQA_c7gA0
Provar has an average 4.5 / 5 rating on G2. Click here to read our latest
user reviews. https://t.xink.io/Tracking/Index/mQsCACIoAQA_c7gA0


Add zap stanza for Paragon Casks

References #7327
The current uninstall stanzas for cask paragon-ntfs and cask paragon-extfs leave some files and folders on the system in preparation for a re-install later.  A zap stanza would be needed to eliminate these files and folders.
"~/Library/Application Support/Paragon NTFS for Mac OS X/.notfirsttime"
"~/Library/Preferences/com.paragon-software.filesystems.ntfs.plist"
"~/Library/Preferences/com.paragon-software.NTFS.prefpane.plist"
"/Library/Application Support/Paragon Software/NTFS10"
"/Library/Application Support/Paragon Software/NTFS12"
"/Library/Application Support/Paragon Software/NTFS14"

"~/Library/Application Support/Paragon ExtFS for Mac OS X/.notfirsttime"
"~/Library/Preferences/com.paragon-software.filesystems.extfs.plist"
"/Library/Application Support/Paragon Software/ExtFS9"

It is not clear to me why paragon-ntfs references two similar preference pane plists; I expect one of them is from an older version of paragon-ntfs.
Should a zap stanza remove all known instances of historical installations, or should a zap stanza only touch files that the most recent known brew cask install paragon-ntfs would create?

or should a zap stanza only touch files that the most recent known brew cask install paragon-ntfs would create?

That one. Otherwise we’d have a confusing ever-expanding list.

Care to submit the PR?

I plan to submit a PR, however I just realized that if a zap stanza only touched files from the most recent known brew cask install paragon-ntfs, then files installed by older versions have an opportunity to never be removed.
For example, the use case:

Fresh System
brew cask install paragon-ntfs
Paragon creates a new version such as changing from NTFS14 to NTFS15
Developers update paragon-ntfs.rb to install newer version as well as zapping only newer version
brew update
User still has NTFS14 installed and performs brew cask zap paragon-ntfs
User is left with stale NTFS14 file.
brew cask install paragon-ntfs

You are correct. However, that is a limitation of the system as it works currently. In the future, we’d like to have glob support (i.e. /Library/Application Support/Paragon Software/NTFS* would consider (from your examples) 10, 12, 14.
As we don’t support it yet, we have to keep only the newer versions. You can do something like this, though. It won’t work for any version currently, but will be ready for when the feature lands.

brew cask install paragon-ntfs
==> Downloading http://dl.paragon-software.com/demo/ntfsmac14_trial_e.dmg
Already downloaded: /.../Homebrew/paragon-ntfs-14.dmg
==> Verifying checksum for Cask paragon-ntfs
==> Note: running "brew update" may fix sha256 checksum errors
Error: sha256 mismatch
Expected: 9b2a2e5b0b2eec51ef85aa72048a9da7aa3eab3a07477a3e670dca31cd7a4ad9
Actual: 2bdb44c0cf5feaf3e5619a737fef849d0f171b782784f3e2315e50bf6c68ce12
File: /.../Homebrew/paragon-ntfs-14.dmg
To retry an incomplete download, remove the file above.

@haodong Next time, open up an new issue. Your error is unrelated.
Fixed in #16474

No need to keep this open anymore. Question has been answered.


Change vector size computation for ops with mixed data lengths

(#8849) This PR changes the algorithm that computes the vector sizes for
vectorizing operations with mixed data lengths. Instead of computing
the vector sizes based on the smallest element type we now use the
largest element type. This significantly reduces the code sizes and
register pressure/spilling on mixed data lengths scenarios. More fine
tuning is needed to determine the optimal vector size for each
particular code to be vectorized.

Checking performance impact of the change.

I'm surprised that no lit tests change... In this case, could you add a test?

Checking performance impact of the change.
I would expect some slowdowns but it should be feasible to get the performance back with more sophisticated heuristics.

I'm surprised that no lit tests change... In this case, could you add a test?
Yeah, I didn't see any tests from #8552 changing. Would you expect any of them to change?

matmul_x86_i8_i8_i32 should change. Let me take a look...

Well, I think we have code size vs performance situation :). Not sure how significant these regressions are. Please, let me know.
Regarding impact on existing tests, matmul is not impacted by this change because the tile sizes are fixed and do not depend on the computed min tile sizes (IIUC) (I guess there's some work to be done to make those tile sizes more dynamic and even data type dependent). This means that this PR should mostly impact generic ops.
Perhaps we can find a middle ground... For cases like i8 (VF=64) and i32 (VF=16) we could choose the size in the middle (VF=32). Not super excited about the idea but it's the best I can think of that do not require super involved heuristics (which should be the long term solution).

Checking performance with the middle-group implementation that I mentioned before. Please, do not review the code yet.

Well, I think we have code size vs performance situation :). Not sure how significant these regressions are. Please, let me know.

Mobilbebert int8 seems to have the regression here. It might be good to see why that is happening. This describes how you could use Tracy to profile the model. (https://github.com/google/iree/blob/main/docs/developers/developing_iree/profiling_with_tracy.md). That might give you insights into before and after..

Regarding impact on existing tests, matmul is not impacted by this change because the tile sizes are fixed and do not depend on the computed min tile sizes (IIUC) (I guess there's some work to be done to make those tile sizes more dynamic and even data type dependent). This means that this PR should mostly impact generic ops.
Perhaps we can find a middle ground... For cases like i8 (VF=64) and i32 (VF=16) we could choose the size in the middle (VF=32). Not super excited about the idea but it's the best I can think of that do not require super involved heuristics (which should be the long term solution).

Heuristics are always ad-hoc. So I dont really have an opinion on it. It does do a balance between  code size and performance. If this works might as well. Its mostly for base heuristics, and some regression is tolerable when fiddling with defaults as long as the regressions have been looked into to see they arent other issues highlighted by the regression.

Mobilbebert int8 seems to have the regression here. It might be good to see why that is happening. This describes how you could use Tracy to profile the model. That might give you insights into before and after..

Thanks, it will be an interesting learning activity for me. I'll also look at the ops that are being impacted by this heuristic. I have the impression that we may be able to get a better outcome if we look inside the generic ops and take more data type information into account.

Mobilbebert int8 seems to have the regression here. It might be good to see why that is happening. This describes how you could use Tracy to profile the model. That might give you insights into before and after..

Thanks, it will be an interesting learning activity for me. I'll also look at the ops that are being impacted by this heuristic. I have the impression that we may be able to get a better outcome if we look inside the generic ops and take more data type information into account.

First time setup of Tracy can be a bit of a curve. Reach out if you need help.

@dcaballe anything you want to do here or close as obsolete?

We will probably have to look into this again in the short term but let's close it for now.


Get token fails for public client on grant_type "authorization_code" (with PKCE)

Describe the bug
Spring authorization server returns error response 'invalid_request ' when requesting a token with grant type "authorization_code" (with PKCE) from a client which is configured public (no client_secret, client-authentication-method "none").
To Reproduce
Started with a clean spring boot project for a local try-out of spring boot authorization server.
Configured two clients:

a confidential client following the config from https://docs.spring.io/spring-authorization-server/reference/getting-started.html
a public client following the instructions from https://docs.spring.io/spring-authorization-server/reference/guides/how-to-pkce.html

Via Postman I tried to issue a token from each client using grant_type "authorization_code" (with PKCE). Same user credentials were used in both cases, requested same /oauth2/authorize and /oauth2/token endpoints.
Both requests get a code returned from /oauth2/authorize endpoint & try to exchange it for a token at  /oauth2/token sending grant_type (authorization_code), code, redirect_uri and code_verifier. The request for the confidential client succeeds, while the request for public client gets returned {"error": "invalid_request"}.
Error log from spring authorization server:
2024-11-11T22:47:04.407+01:00 TRACE 79294 --- [identityprovider] [nio-9000-exec-3] o.s.a.w.OAuth2ClientAuthenticationFilter : Client authentication failed: [invalid_request]

org.springframework.security.oauth2.core.OAuth2AuthenticationException: null
	at org.springframework.security.oauth2.server.authorization.web.authentication.ClientSecretBasicAuthenticationConverter.convert(ClientSecretBasicAuthenticationConverter.java:78) ~[spring-security-oauth2-authorization-server-1.3.3.jar:1.3.3]
	at org.springframework.security.oauth2.server.authorization.web.authentication.DelegatingAuthenticationConverter.convert(DelegatingAuthenticationConverter.java:60) ~[spring-security-oauth2-authorization-server-1.3.3.jar:1.3.3]
	at org.springframework.security.oauth2.server.authorization.web.OAuth2ClientAuthenticationFilter.doFilterInternal(OAuth2ClientAuthenticationFilter.java:134) ~[spring-security-oauth2-authorization-server-1.3.3.jar:1.3.3]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.14.jar:6.1.14]
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:374) ~[spring-security-web-6.3.4.jar:6.3.4]

Expected behavior
Authorization server successfully issues a token for the public client.
Sample
Please see client config and requests below or in sample project
application.yml
server:
  port: 9000

spring:
  application:
    name: spring-authorization-sample
  security:
    user:
      name: user1
      password: password
    oauth2:
      authorizationserver:
        client:
          confidential-client:
            registration:
              client-id: "confidential-client"
              client-secret: "{noop}secret"
              client-authentication-methods:
                - "client_secret_basic"
              authorization-grant-types:
                - "authorization_code"
                - "refresh_token"
                - "client_credentials"
              redirect-uris:
                - "http://<IP_ADDRESS>:8080/login/oauth2/code/confidential-client-oidc"
              scopes:
                - "openid"
                - "profile"
                - "message.read"
            require-authorization-consent: true
          public-client:
            registration:
              client-id: "public-client"
              client-authentication-methods:
                - "none"
              authorization-grant-types:
                - "authorization_code"
              redirect-uris:
                - "http://<IP_ADDRESS>:8080/login/oauth2/code/public-client-oidc"
              scopes:
                - "openid"
                - "profile"
                - "message.read"
            require-authorization-consent: true
            require-proof-key: true

Postman collection
{
	"info": {
		"_postman_id": "8cb75b26-f6b7-4ddf-82bf-c5e6e9e0600f",
		"name": "Spring Authorization Sample",
		"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json",
		"_exporter_id": "29654337"
	},
	"item": [
		{
			"name": "Confidential client token request",
			"request": {
				"auth": {
					"type": "oauth2",
					"oauth2": [
						{
							"key": "grant_type",
							"value": "authorization_code_with_pkce",
							"type": "string"
						},
						{
							"key": "password",
							"value": "",
							"type": "string"
						},
						{
							"key": "username",
							"value": "",
							"type": "string"
						},
						{
							"key": "scope",
							"value": "message.read",
							"type": "string"
						},
						{
							"key": "clientSecret",
							"value": "secret",
							"type": "string"
						},
						{
							"key": "redirect_uri",
							"value": "http://<IP_ADDRESS>:8080/login/oauth2/code/messaging-client-oidc",
							"type": "string"
						},
						{
							"key": "clientId",
							"value": "messaging-client",
							"type": "string"
						},
						{
							"key": "authUrl",
							"value": "http://localhost:9000/oauth2/authorize",
							"type": "string"
						},
						{
							"key": "accessTokenUrl",
							"value": "http://localhost:9000/oauth2/token",
							"type": "string"
						},
						{
							"key": "tokenName",
							"value": "spring auth",
							"type": "string"
						},
						{
							"key": "headerPrefix",
							"value": "Bearer ",
							"type": "string"
						},
						{
							"key": "addTokenTo",
							"value": "header",
							"type": "string"
						}
					]
				},
				"method": "GET",
				"header": [],
				"url": {
					"raw": ""
				}
			},
			"response": []
		},
		{
			"name": "Public client token request",
			"request": {
				"auth": {
					"type": "oauth2",
					"oauth2": [
						{
							"key": "grant_type",
							"value": "authorization_code_with_pkce",
							"type": "string"
						},
						{
							"key": "username",
							"value": "",
							"type": "string"
						},
						{
							"key": "password",
							"value": "",
							"type": "string"
						},
						{
							"key": "scope",
							"value": "message.read",
							"type": "string"
						},
						{
							"key": "accessTokenUrl",
							"value": "http://localhost:9000/oauth2/token",
							"type": "string"
						},
						{
							"key": "authUrl",
							"value": "http://localhost:9000/oauth2/authorize",
							"type": "string"
						},
						{
							"key": "clientId",
							"value": "public-client",
							"type": "string"
						},
						{
							"key": "code_verifier",
							"value": "",
							"type": "string"
						},
						{
							"key": "challengeAlgorithm",
							"value": "S256",
							"type": "string"
						},
						{
							"key": "clientSecret",
							"value": "",
							"type": "string"
						},
						{
							"key": "redirect_uri",
							"value": "http://<IP_ADDRESS>:8080/login/oauth2/code/public-client-oidc",
							"type": "string"
						},
						{
							"key": "tokenName",
							"value": "spring auth",
							"type": "string"
						},
						{
							"key": "headerPrefix",
							"value": "Bearer ",
							"type": "string"
						},
						{
							"key": "addTokenTo",
							"value": "header",
							"type": "string"
						}
					]
				},
				"method": "GET",
				"header": [],
				"url": {
					"raw": ""
				}
			},
			"response": []
		}
	]
}

@jkornatzki Questions are better suited to Stack Overflow. We prefer to use GitHub issues only for bugs and enhancements.
See the guide How-to: Authenticate using a Single Page Application with PKCE for example configuration and the PublicClientTests that demonstrates a working flow.


dm/cdc: refine foreign key integration test

Signed-off-by: crazycs520<EMAIL_ADDRESS>
What problem does this PR solve?

Issue Number: ref #7521
What is changed and how it works?
Since TiDB Already enable tidb_enable_foreign_key by default(https://github.com/pingcap/tidb/pull/39677), and plan to delete tidb_enable_foreign_key variable later(https://github.com/pingcap/tidb/pull/39672), so this PR remove setting tidb_enable_foreign_key in dm/cdc integration test.
Check List
Tests

No code

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
None

/run-all-tests tidb=master

/merge

/run-all-tests

/run-all-tests

/run-dm-integration-test


RasaX Install Error when execute task [backwards compatible credentials]

Rasa version: 1.7.0
Python version: 3.6
Operating system (windows, osx, ...): Ubuntu 18.04
Issue:
When I install RasaX with below command code as Docker-Compose Quick Install tutorial, when execute task [backwards compatible credentials]，I met error "mv: cannot move 'credentials' to a subdirectory of itself, '/etc/rasa/credentials/credentials'".
Error (including full traceback):

Command or request that led to error:
$ curl -sSL -o install.sh https://storage.googleapis.com/rasa-x-releases/0.25.2/install.sh
$ sudo bash ./install.sh

Content of configuration file (config.yml) (if relevant):
rasa_x_playbook.yml

I fix the problem by command the task [backwards compatible credentials], and add a new task [Create the directory for the credentials(backwards compatible credentials)] as follows:

I think this may be a potential bug, I hope this will help someone who have the same problem.

I had the same issue ... believe it is a small bug in the rasa_x_playbook.yml file
I think the first reference to credentials is supposed to be the credentials.yml file and not the credentials directory.
In the rasa_x_playbook.yml file I replaced
command: mv credentials {{RASA_HOME }} / credentials
with
command: mv credentials.yml {{RASA_HOME }} / credentials
and it seemed to work.

Thanks for the issue, @amn41 will get back to you about it soon!
You may find help in the docs and the forum, too 🤗


客户端更新至v0.40.14出错

更新软件版本后并按要求更新lipcap及lipcap-bin后全部重新配置，但是还不能使用(且DEV/TUN等内核提示更新成功但仍旧显示”查询失败“)
nohup: failed to run command ‘/etc/openclash/clash’: Exec format error
2020-10-28 11:53:30 Error: OpenClash Can Not Start, Try Use Backup Rules Start Again
nohup: failed to run command ‘/etc/openclash/clash’: Exec format error
2020-10-28 11:53:30 Error: OpenClash Can Not Start, Try Use Raw Config Restart Again
2020-10-28 11:53:30 Error: Get General 'redir-port' Option Error, OpenClash Can Not Start With Raw Config File
nohup: failed to run command ‘/etc/openclash/clash’: Exec format error
2020-10-28 11:57:25 Error: OpenClash Can Not Start, Try Use Backup Rules Start Again
nohup: failed to run command ‘/etc/openclash/clash’: Exec format error
2020-10-28 11:57:25 Error: OpenClash Can Not Start, Try Use Raw Config Restart Again

尝试更新nohup后，显示如下：
2020-10-28 12:28:55 Nameserver Option Must Be Setted, Auto Completed
nohup: failed to run command '/etc/openclash/clash': Exec format error
2020-10-28 12:28:55 Error: OpenClash Can Not Start, Try Use Backup Rules Start Again
nohup: failed to run command '/etc/openclash/clash': Exec format error
2020-10-28 12:28:55 Error: OpenClash Can Not Start, Try Use Raw Config Restart Again
2020-10-28 12:28:55 Error: Get DNS 'listen' Option Error, OpenClash Can Not Start With Raw Config File
2020-10-28 12:28:55 Error: Get DNS 'enhanced-mode' Option Error, OpenClash Can Not Start With Raw Config File
2020-10-28 12:28:55 Error: Get General 'redir-port' Option Error, OpenClash Can Not Start With Raw Config File

Exec format error means the binary downloaded does not match your platform's architecture.
In your screenshot, your OS architecture is shown as aarch64_generic, but you've selected linux-amd64 as your Clash binary arch.
Change it to linux-armv8 and re-download all the binaries, then try again.

Exec format error means the binary downloaded does not match your platform's architecture.
In your screenshot, your OS architecture is shown as aarch64_generic, but you've selected linux-amd64 as your Clash binary arch.
Change it to linux-armv8 and re-download all the binaries, then try again.

It works now, many thanks and sorry for my shortage of basic knowledge :(


PUT /packages/{id}/items/{name}

Path: /packages/{id}/items/{name}
Method: PUT
Auth: COORDINATOR
Parameters:

name={String}

Payload:
{
  name: String,
  type: String,
  description: String,
  img: String,
  price: Number // €,
  vat: Number // %
}

Response:
{
  name: String,
  type: String,
  description: String,
  img: String,
  price: Number // €,
  vat: Number // %
}

Notes:
Changes an items matching name, inside a package.
id corresponds to an _id index of mongodb.
Status codes:

Code
Rule

404
Specific object not found

400
Invalid payload

200
Otherwise

Tests:

No package with id

Should return a 404 status code.

No items matching name

Should return a 404 status code.

Invalid payload

Should return a 400 status code.

Valid new item info

Should return a 200 status code.

You should not change the items by their name, but by their id.
This should be PUT /items/{id}. The creation of an item is in a specific context. This way we can avoid the creation of items not inserted in any context. After their creation, the modification can be made directly.
The response object is missing its id.


Example landscape doesn't run

When you download SCRAPPLE and try to run the example landscape, it triggers an error because it can't find any of the maps.  You need to supply all the img files for this example.

Fixed and new installer uploaded to web site.


Fixes setting of X-Forwarded-For-Proto

The downstream request 'X-Forwarded-For' headers should be set
to whatever the proxy is serving. Also do not overwrite or limit the ability to proxy to any schema, independent of what the proxy in serving.
Given the following setup: sudo devd -p 443 --tls http://localhost:8000, devd should accept https connections and proxy to http://localhost, while setting the X-Forwarded-Proto to https. In its current implementation however the header is set to whatever schema the downstream server (a.k.a. target) has.
Also fixes a minor logging issue, that only showed up now, as request.URL includes the scheme, which was simply not set before. Using the URI instead.

Looks good. And apologies for taking a while to get to this. :)


Jitpack build does not work

I will just publish it to maven central most likely

It is now on maven central


moment instances don't have some locale methods of global moment

Seems that localeData objects of moment doesn't get the functions to get list of month names or weekday names, instead it only gets the function that formats one moment instance.
> moment.months()
[ 'January',
  'February',
  'March',
  'April',
  'May',
  'June',
  'July',
  'August',
  'September',
  'October',
  'November',
  'December' ]
> var m = moment();
> m.months()
8
> m.localeData().months()
TypeError: Cannot call method 'month' of undefined
    at Locale.extend.months
    at repl:1:17
    at REPLServer.self.eval (repl.js:110:21)
    at Interface.<anonymous> (repl.js:239:12)
    at Interface.emit (events.js:95:17)
    at Interface._onLine (readline.js:202:10)
    at Interface._line (readline.js:531:8)
    at Interface._ttyWrite (readline.js:760:14)
    at ReadStream.onkeypress (readline.js:99:10)
    at ReadStream.emit (events.js:98:17)

See https://momentjs.com/docs/#/-project-status/


Can't find 'node' binary to build React Native bundle

Environment
Environment:
OS: macOS 10.14.1
Node: 8.11.2
Yarn: Not Found
npm: 5.6.0
Watchman: 4.9.0
react-native: 0.55.4,
Xcode: Xcode 10.1 Build version 10B61
Android Studio: 3.2 AI-181.55<IP_ADDRESS>56338
Packages: (wanted => installed)
react: 16.3.1 => 16.3.1
Description
I cant build my app on iOS anymore,

❌  error: Can't find 'node' binary to build React Native bundle```

Can you help? I have tried everything I could find on the previous issues

Hey, this looks like an environment issue. 'Can't find node binary' is triggered when the Node.js installation is invalid. Try checking your path and the workaround mentioned. If you think this is an actual issue on RN side, provide us with a minimal repro and we'll have a look again.

I would recommend against symbolically linking your node binary; it can cause issues elsewhere. Instead set the binary within XCode as it suggests in the error:

For the people coming across this error that have added node args via the NODE_BINARY env variable (e.g. export NODE_BINARY='node --max_old_space_size=4096'), you should instead be setting NODE_ARGS:
export NODE_ARGS='--max_old_space_size=2048'
(as of https://github.com/facebook/react-native/pull/24156)

I actually semi-redact my previous comment. Although it works fine when there's only one person working on the project, it causes issues if more than one person wants to use the same xcode file. Also breaks things like appcenter.ms which has no knowledge of your local file structure!

@MaffooBristol sorry to disappoint you, but your proposed solution (xcode build phase - /usr/local/opt/node@12/bin/node) did not have any effect on my freshly created app, metro could not find a node binary; only creating the symbolic link proposed by @tristankenney eventually made it work


Problem on training

Hi,
I'm using apex for the first time and I'm experiencing weird behavior. I followed all instructions for the environment installation and to launch the training routine.
Once all this done I was able to launch the training and the code started running successfully. However, I stopped the training routine and I have not been able to make it run again since. The code starts and blocks 11Mb in both selected GPUs and it stays blocked in there.
Do you have any hint on what could be happening...?
Thanks in advance for the help and for sharing the great work. Have a nice day.

Hi,
According to my previous experience, it is possible that the code was blocked at the beginning of train.py, such as some import commands. For example, I was once blocked at the command from seg_opr.sync_bn import DataParallelModel, Reduce, BatchNorm2d several days ago. Commenting it solves the problem and it will not affect the other part of the code.

Hi,
Yep, it was blocked just in there. I was able to solve it. For people that might have the same issue, when the training routine starts: apparently, a block file is created in the code that it is tried to import (path was /TorchSSC/furnace/seg_opr/sync_bn/src/cpu in my case) when the routine starts and once it has already been imported. Since I interrupted the code abruptly, the code is not able to delete this file in order to unblock the folder for a next training.
Just delete this file and it should work. It did for me at least. Good luck :).


能否提供一个纵向联邦学习的例子

Thanks for your interest :)
The example of vertical federated learning is provided in federatedscope/vertical_fl , and you can refer to tutorial for more details about this example.


High CPU consumption with InfluxDB 1.7.9

Hi,
We're using InfluxDB with telegraf on a monitoring server. Only telegraf database is present in InfluxDB.
With telegraf, we use the 'tail' plugin to insert data from a file using InfluxDB format. All data is stored in measurement "loopingd"
We have a script to check data present in influxDB. This script run the following query very regularly:
select count(rtt), mean(rtt) from loopingd where server='$server' and time >= '$start_ts' and time < '$end_ts' group by time(1d)

Where:

$server is the server we want to check
$start_ts is a timestamp in nanoseconds corresponding to previous minute, rounded to the minute (exemple: if now is 15:30:34, this will be the timestamp for 15:29:00).
$end_tsis a timestamp in nanoseconds corresponding to current minute, rounded to the minute (exemple: if now is 15:30:34, this will be the timestamp for 15:30:00).

We currently have 24 097 076 points stored in this measurement.
After upgrading InfluxDB to v1.7.9 we observed an increase in CPU usage from 15% usage to 53% usage.
When doing a top command, we saw a lot of cpu consummed by the queries above.
Steps to reproduce:
I'm not sure on how to reproduce the issue, so I will try to explain how to create a setup like mine.

Install InfluxDB 1.7.8
Create a database named telegraf
Insert 50 entries in a measurement loopingd with following labes: host, path, server and a field named rtt
Perform multiple queries select count(rtt), mean(rtt) from loopingd where server='$server' and time >= '$start_ts' and time < '$end_ts' group by time(1d) and measure cpu usage on server
Upgrade to InfluxDB 1.7.9
Execute the same queries and measure CPU usage

Expected behavior:
CPU usage is the same between point 4 and 6
Actual behavior:
CPU usage is higher at point 6 than at point 4.
Environment info:

System info: Run uname -srm and copy the output here

Linux 4.4.0-1096-aws x86_64

InfluxDB version: Run influxd version and copy the output here

InfluxDB v1.7.9 (git: 1.7 23bc63d43a8dc05f53afa46e3526ebb5578f3d88)

Other relevant environment details: Container runtime, disk info, etc

N/A

Config:
Copy any non-default config values here or attach the full config as a gist or file.
We use the default configuration file.

Logs:
Include snippet of errors in log.
None provided
Performance:
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.
# Commands should be run when the bug is actively.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt

# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.

profiles.tar.gz
vars.txt
iostat.txt

Hi,
Anyone had a chance to look at this?
Let me know if more information is needed to troubleshoot the issue.
Thanks

I'm experiencing a similiar issue.
My use case:

telegraf using the exec plugin to consume a remote csv file every 15s (via curl)
telegraf using the exec plugin to consume a remote csv file every hour (via curl)
telegraf using a tail plugin that watch some folders for csv files, those files are not placed periodically, just manually.

All data is stored in one measurement. I gather information from 4 different devices, I have an influxdb database per device.
Seems that randomly Influxdb starts to consume resources (cpu, ram, and write operations). Despite  the hardware is limited with just 1GB RAM (raspberry pi 3B+), seems the swap was untouched.
I had to manually restart influxdb and all resources return to normal behaviour. Please check the next pictures. Manually restart of the process was at 10:05 aprox.

infux-report.zip at 10:45, not during the issue

I have the exact same issue described here


added hello world in java

What does the program do ?
prints "hello, world"
In what programming language it is written?
Java

Thank you for participating  in hacktoberfest ❣ if you liked working on this repo you can share it with friends and star this project for helping others


[2.x] Reduce number of long-living instances to speed up loading

After loading, instances of ScopedKey, Scope, Select or ProjectRef represent 25.6% of the total number of instances, and 27.3% of the heap size. The main idea of this PR is to reduce the number of duplicated instances using uniqueness caches.
Uniqueness cache on Scope
It reduces the number of Scope instances by 94%. On its own, it slows down the loading, because Scope.apply becomes more expensive and it is called too many times. This can be compensated by using temporary caches in Project.transform and delegates: Scope => Seq[Scope], to avoid calling Scope.apply to create too many short-lived instances.
It also reduces the number of Select by 92% and ProjectRef by 99.8%, which make them totally negligible.
Since we use a uniqueness cache we might as well pre-compute the hash code of each scope, which reduce the CPU time.
Uniqueness cache on ScopedKey (uncommited)
I think it is also worth mentioning that I tried a uniqueness cache on ScopedKey but it was not really better. Indeed, on one hand it reduced the number of ScopedKey by 59.6%, but on the other hand it was counterbalanced by the number of nodes in the cache itself.
Other changes
The first commit reduces the number of Init.GetValue by 48% for free.
Result
I am still using sttp to collect those metrics, on my 12 CPUs laptop, and JDK temurin 17 (G1GC).
It shows up a 20% speedup of the loading time compared to sbt 2.0.0-M2, and a 41% speedup compared to sbt 1.10.2. Those are rough estimations.
Compared to 2.0.0-M2

Settings
Instances
Used Heap size
Load time

2.0.0-M2
212961
20,920 K
532 MB
7949 ms

This PR
212961
16,365 K
410 MB
6328 ms

100 %
78 %
77 %
80 %

Compare to 1.10.2

Settings
Instances
Used Heap size
Load time

1.10.2
197031
23,524 K
675 MB
10629 ms

This PR
212961
16,365 K
410 MB
6328 ms

108 %
70 %
61 %
59 %

@adpi2 Nice work!


Forum link in Management console is wrong

Description:
When u click "Forum" in the sign in page of the APIM 2.5.0 RC4 pack of the Management console it takes you to a "page not found" link. Please fix this.
Suggested Labels:
2.5.0, bug
Suggested Assignees:
Chamin
Affected Product Version:
APIM 2.5.0
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:

fixed.
duplicate witn https://github.com/wso2/product-apim/issues/2859


Torque adaptor fails with timeout in develop

Since merging the last two pull requests the torque adaptor fails with timeouts. The pull requests themselves did not have this issue.
Can this be related to the fix in #378 ?

Indirectly, because of #378 recent docker version is used.
The hostname resolving/networking in docker has changed, which causes torque queues to be in 'E' state.

Debugging using docker-compose:
cd src/integrationTest/docker
docker-compose run -e BOOT_DELAY=15 -e MYUID=$UID xenon-test bash
ssh root@xenon-torque
# 'root' as pw see https://hub.docker.com/r/agaveapi/torque/
qstat -Q
tail /var/log/supervisor/pbsserver.log
04/22/2016 13:10:29;0001;PBS_Server.23;Svr;PBS_Server;LOG_ERROR::Unknown node  (15064) in process_host_name_part, host docker not found
04/22/2016 13:10:38;0001;PBS_Server.26;Svr;PBS_Server;LOG_ERROR::get_node_from_str, Node localhost is reporting on node 8f84f68ace61, which pbs_server doesn't know about

So at the moment the torque docker container is kaput.

At the start of a torque container the hostname needs to be set inside the torque config, similar to src/integrationTest/docker/xenon-gridengine/etc/my_init.d/gridengine.sh.


Clean up and separate shared components

Remove the components vs specific_resources style discrepancy
Clean up and remove all of the unused styles and assets ported from the old dashboard UI
Move shared styles to shared style repo, like aptible-sass#diesel

Fixed by #292


Databricks OAuth secret not working as expected in 1.8

Describe the bug
We are sometimes running dbt as a service principal from the command line. profiles.yml then looks something like
project:
  outputs:
    o1:
      auth_type: oauth
      client_id: [sp_id]
      client_secret: "{{ env_var('DB_CLIENT_SECRET') }}"
      type: databricks
      [...]

The value of DB_CLIENT_SECRET has been a Databricks OAuth secret as per the docs. This has been working fine with dbt-databricks>=1.7 but now when upgrading to 1.8(.5) it starts failing with
Runtime Error
  invalid_client: AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '[sp_id]'.

And if I instead supply the client secret that I set up in Microsoft Entra ID it works again.
What is the intended usage? Do I need to switch from using Databricks OAuth secrets here to using that from Microsoft Entra ID?

This is a legit bug that we aim to fix shortly, but what's surprising to me is the idea that it was introduced in 1.8...I'm reasonably sure we had this same bug in 1.7.  Which version of 1.7?

Yeah you're right, it's there in 1.7 too it seems. But using the versions I happened to have installed I'm not able to reproduce the issue:
❯ dbt --version
Core:
  - installed: 1.7.13
  - latest:    1.8.5  - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - databricks: 1.7.3 - Update available!
  - spark:      1.7.1 - Update available!

Using these I am able to reproduce it even on dbt-databricks 1.7:
❯ dbt --version
Core:
  - installed: 1.7.18
  - latest:    1.8.5  - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - databricks: 1.7.17 - Update available!
  - spark:      1.7.1  - Update available!

I tried a few different combinations of dbt-databricks and dbt-core and you probably know better than me but I think it's after your 1.7.3 that it starts to break, but not all versions of dbt-core 1.7 will work even though dbt-spark says ">=1.7.0,<1.8.0" so it's hard for me to see a pattern :)
But also: What is the intended behavior? Is it the Databricks OAuth secrets that I should be using whenever this is fixed?

When it's fixed you will have the option of either using Databricks OAuth secret, or AAD secret.  The reason it could work in old versions of 1.7.x if we didn't use to pin our version of the Databricks SDK.  We started pinning when the SDK updated and broke our tests...but it broke our tests because we were using Azure and doing it with an AAD secret, and the SDK started using Databricks OAuth secret by default.  So, on an old 1.7, you can install Databricks SDK 0.18.0 or higher, which works with Databricks OAuth secret.

So, long story short, I'm working on some stability work right now (ensuring that new versions of the Databricks runtime don't break dbt users), and the next thing I'll work on after that is getting both ways of doing OAuth on Azure to work.

Hi apologies for the long delay.  We had been waiting on a fix in the Databricks SDK, which I believe has landed.  Now I'm waiting for @eric-wang-1990 (my OAuth expert :) ) to free up to finish off the work on our side.


lib/Handler.go:79: self.XsrfFormHtml undefined (type *BaseHandler has no field or method XsrfFormHtml)

$ go version
go version go1.6 darwin/amd64
go build -ldflags "-s" app.go
sdc/lib
lib/Handler.go:79: self.XsrfFormHtml undefined (type *BaseHandler has no field or method XsrfFormHtml)

@ryderkun This problem is due to the beego is currently changing the api.


Add timeout to requests calls

Many developers will be surprised to learn that requests library calls do not include timeouts by default. This means that an attempted request could hang indefinitely if no connection is established or if no data is received from the server.
The requests documentation suggests that most calls should explicitly include a timeout parameter. This codemod adds a default timeout value in order to set an upper bound on connection times and ensure that requests connect or fail in a timely manner. This value also ensures the connection will timeout if the server does not respond with data within a reasonable amount of time.
While timeout values will be application dependent, we believe that this codemod adds a reasonable default that serves as an appropriate ceiling for most situations.
Our changes look like the following:
 import requests

- requests.get("http://example.com")
+ requests.get("http://example.com", timeout=60)

More reading

https://docs.python-requests.org/en/master/user/quickstart/#timeouts

Powered by: pixeebot (codemod ID: pixee:python/add-requests-timeouts)

FYI - This change was autogenerated from a GitHub app - called Pixeebot. A code-quality GitHub App; like Dependabot, but for source code. Feel free to check it our for more details for how you can install onto your project's repo for continued code hardening and code security recommendations.

@maxachis what do you think of this 👀 seems repeat-y? should we be more selective? ignore AI nonsense?

@josh-chamberlain This does seem like a good practice, although obviously I think we should make an issue for this rather than just making a code change without testing. I'd probably want to make it so that we're calling a singular function which makes the request with the necessary parameters rather than having to put the same argument in every call to requests.get().


Add replacement output template via &

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[ ] Checked the code with flake8

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[ ] Bug fix
[ ] Improvement
[ ] New extractor
[x] New feature

Description of your pull request and other information
This pull requests adds a new internal operator to output templates: &. This causes the value of the field to be replaced by a given static string.
The purpose of this is to build output templates that are highly flexible, e.g. for handling non-playlist and playlist videos with the same configuration.  Using the following template works fine: %(playlist_index)s - %(title)s.%(ext)s
But it has a flaw: if downloading non-playlist video (i.e. playlist_index is None), the filename is prefixed by NA - ; we don't want that. A first improvement would be to use an empty default string like so:
%(playlist_index|)s - %(title)s.%(ext)s
This at least removes the NA, but still leaves the useless -.
With this pull request, it is possible to create the following template, which removes both the NA and - if playlist_index is None (or more precisely, only adds the - if playlist_index is not None):
%(playlist_index|)s%(playlist_index& - )s%(title)s.%(ext)s
An alternative I have considered is to add an append operator, which appends a given string to the value if the field is not None:
%(playlist_index& - |)s%(title)s.%(ext)s
This makes the output template more readable, but is also less flexible (e.g. prefixes are not possible). By using the "replacement" interpretation we have the highest flexibility in creating custom output templates.
The resulting output template quickly become complex, but they are clearly a power-user feature. The idea is to write an output template once, that covers all use-cases and save it in the config file. This removes the necessity to switch config files (or pass -o parameter by other means) when downloading playlists vs. non-playlist videos.

This can already be done using --parse-metadata, but that is much more complicated that your approach
--parse-metadata "%(playlist_index|)s - :(?P<prefix>\d.+)"
-o "%(prefix)s%(title)s"

So this is a nice addition.

Should falsey values trigger this? On one hand, this removes the ability to distinguish b/w False and None. On other hand, we can do %(is_live&LIVE|VOD)s

This can already be done using --parse-metadata, but that is much more complicated that your approach
--parse-metadata "%(playlist_index|)s - :(?P<prefix>\d.+)"
-o "%(prefix)s%(title)s"

I did not know this. But as you say, the method from this PR is probably easier to discover and use.

Should falsey values trigger this? On one hand, this removes the ability to distinguish b/w False and None. On other hand, we can do %(is_live&LIVE|VOD)s

That's an interesting question. I don't feel strongly either way. How does the current default operator | work exactly? I think & and | should have the same semantic, otherwise things become confusing.
Another option would be to add an = operator: use the value of the field only if it is equal the given string. So your example would become:
%(is_live=true&LIVE|VOD)s

This is, again, the most flexible solution. However, that's out of scope for this particular PR.

You are right. It was unnecessary confusion in my part. | only looks at None. So & should do the same

I integrated all your suggestions; thanks for your feedback.

I also rebased onto current master (I hope that's ok)

generally, don't do this. But this PR is small enough that it doesn't actually matter


Top level marked.min.js out of sync with lib/marked.js

minified file edited "a year ago", while lib/marked.js was edited 2 months ago.

We see the same thing, making the release more or less useless for us. Is it possible to re-release it?

The marked.min.js file should be updated very quickly. This is a high priority issue. I'm seeing a lot of FLOSS using this file without knowing that they are vulnerable to #592. This issue is creating XSS in all of them.
Please @chjj update it!

Any news?

Fixed in latest release, I believe.


Bedrock support

Current state:
fastfetch shows the hijack distro fetch info/logo
Wanted state:
--------------------------------------   alone@arcobox
--------------------------------------   -------------
--------------------------------------   OS: Bedrock Linux
---\\\\\\\\\\\\-----------------------   Kernel: 5.17.9-zen1-1-zen
----\\\      \\\----------------------   Uptime: 42 mins
-----\\\      \\\---------------------   Packages: 1344 (pacman), 237 (dpkg)
------\\\      \\\\\\\\\\\\\\\\\------   Shell: bash 5.1.16
-------\\\                    \\\-----   Resolution: 1920x1080
--------\\\                    \\\----   WM: bspwm
---------\\\        ______      \\\---   Theme: Mountain [GTK2/3]
----------\\\                   ///---   Icons: Mountain [GTK2/3]
-----------\\\                 ///----   Terminal: kitty
------------\\\               ///-----   Terminal Font: Iosevka Nerd Font 13.0
-------------\\\////////////////------   CPU: Intel i5-4200M (4) @ 3.100GHz
--------------------------------------   GPU: NVIDIA GeForce GT 755M
--------------------------------------   GPU: Intel 4th Gen Core Processor
--------------------------------------   Memory: 2678MiB / 7859MiB (34%)

fastfetch could show bedrock logo and multiple package managers.
Why the change is sensible:
It would be very cool to have Bedrock support :wink: although you don't have to add it it's your decision

I try to do it without a VM right now, as this is definitely a feature missing. Can't promise how well this will work out though.

Should work now, can you please test it?

Sure

Logo works only pacman shows up and nix

Oh then i guess the issue my color scheme

No, i set a color which doesn't exist, so it used white, which is the default color on my system, but probably not on your. Now i explicitly set white.

oh

/bedrock/strata/<debian,ubuntu,devuan>/var/lib/dpkg/status

The location didn't change much the only thing that changed is that it's located in /bedrock/strata/<distro>/

Does it make sense to iterate over every folder in /bedrock/strata and do the package detection code there?

Well i don't think there is another way

It's the fastest way i can think of neofetch probably uses apt list --installed | wc -l but it's way too slow and it breaks the point of fastfetch

neofetch runs this code in every strata dir

neofetch checks if apt/dpkg is available if not it doesn't run it

it doesn't check every strata directory it only uses the package manager to check how many packages are installed

It overwrites tot on bedrock linux

Hm :thinking:

Finally found the time to implement package support. Can you please test it?

Should be working again

Even after a run with --recache?

Ah after -recache shows Bedrock!

Can be closed


Reduce frequency of background checks for the existence of new language server

Currently the process used to check for the latest ls, download, install is synchronous.
However this results in unnecessary delays in loading the extension when majority of the time no new ls will be available.
This drawback was discussed, however at that time, the decision was to go with the quickest and fastest solution.
Currently the above process takes over 500ms on my personal computer.
@brettcannon 	/cc

As this impacts the end user experience of the extension, i believe it warrants a label of P1.
@brettcannon /cc

@DonJayamanne can you confirm if this is still a problem? Please close it if not.

Not a problem anymore


Add httpget/2

This utility covers a subset of curl like commands.
It performs an HTTP GET request on a specified URL and can print
or save the response to a local path.
It can display headers with a verbose switch.
It will auto add http:// if this is left off the URL.
Closes #50

These look good, thanks for catching the spelling mistake. Any preference on how the suggestions are incorporated? I can squash them into the commits or apply as a follow on commit.

These look good, thanks for catching the spelling mistake. Any preference on how the suggestions are incorporated? I can squash them into the commits or apply as a follow on commit.

Manually merged in 1e86cdbf1f5bec9f4761d99dd3720ebb74d20807.

Manually merged in 1e86cdbf1f5bec9f4761d99dd3720ebb74d20807.

Thanks @fhunleth!

Thanks @fhunleth!


Add support for predefined constraints

Using the rather new (released within the last week or two as part of protovalidate-go v0.8.0 from this pr) support for predefined constraints, the openapi output looks a little odd.
In and effort to reduce boilerplate, I went from a definition of this:
message EarmarkCreateRequest {
     string event_item_ref_id = 1 [(buf.validate.field).cel = {
       id: "refid.format"
       message: "must be in refid format"
       expression: "this.matches('^[0-9a-zA-Z]{26}$')"
     }];
}

To this:
edition = "2023";

extend buf.validate.StringRules {
  bool refid = 70000001 [(buf.validate.predefined).cel = {
    id: "string.refid"
    expression: "this.matches('^[0-9a-zA-Z]{26}$')"
    message: "must be in refid format"
  }];
}

message EarmarkCreateRequest {
    // (buf.validate.field).string.(refid) is now re-usable across many other messages
    // that all have the same field type
     string event_item_ref_id = 1 [(buf.validate.field).string.(refid) = true];
}

And the openapi output changed thusly:
         eventItemRefId:
           type: string
           title: event_item_ref_id
-          description: |+
-            must be in refid format:
-            ```
-            this.matches('^[0-9a-zA-Z]{26}$')
-            ```
-
+          description: |-
+            string event_item_ref_id = 1 [(buf.validate.field).cel = {
+            id: "refid.format"
+            message: "must be in refid format"
+            expression: "this.matches('^[0-9a-zA-Z]{26}$')"
+            }];

Is this description change expected?
If using a predefined constraint, it would be nice to retain the simpler and more easily readable description, if possible (eg. drill down to show the message and expression like before).

Yeah, this tool doesn't yet support predefined constraints... but I'm curious why it's outputting that way for you
because I'm not getting what you're showing at all. This is what I get:
    		+        eventItemRefId:
    		+          type: string
    		+          title: event_item_ref_id
        	+          description: |-
        	+            (buf.validate.field).string.(refid) is now re-usable across many other messages
        	+             that all have the same field type

It looks like I need to add some code to specifically get the predefined constraints add add it to the description like the unpredefined version. I'm going to adjust your title a little bit to reflect this.

My output was different because the comment in my example was just added to show some reasoning why I was doing it for the GH issue. The actual code lacked that comment block.

Oh got it! Understood! But yeah, thanks for the issue! This should be quick but I am asking buf folks about the best way to get this information since they don't yet provide an easy interface for getting constraints defined this way.

Sounds good @sudorandom! 👍

Related: https://github.com/bufbuild/protovalidate-go/issues/147

@sudorandom looks like the upstream change made it into a protovalidate-go release.
https://github.com/bufbuild/protovalidate-go/releases/tag/v0.7.3


License

What is license for this code?
https://choosealicense.com/
Thanks

Thank you. I will add a license asap.

Done! I added the license. It is under the MIT license.
Let me ask you and not be nosy: what are you planning on doing with this code?


QUESTION/REQ: custom functions

I really appreciate this library and this language, however I would need to have more specific function for my specific scenarios.
It would be nice to add those custom functions without modifying the library.
I've found that it's currently possible to add additional functions with this code:
Expressions expr = Expressions.parse(jsonata);
final FrameEnvironment environment = expr.getEnvironment();
environment.setJsonataFunction(BasenameFunction.FUNCTION_NAME, new BasenameFunction());
environment.setJsonataFunction(DirnameFunction.FUNCTION_NAME, new DirnameFunction());
JsonNode val = expr.evaluate(jsonObj);

where BasenameFunction and DirnameFunction are classes implementing my custom functions.
Currently this is not documented and I'm scared that the methods I'm using to add custom functions, or to implement those functions may change in the future.
It would be nice if you can document and officially support a way to add custom functions.
Another possible problem could be the conflict between custom functions and future native functions, a possible solution is to reserve force custom functions to follow a name pattern (e.g. $<prefix>-<function>)
Thanks,
Franco

I would be happy to have you contribute functions if desired. Let me know if you agree to the terms in  the agreements here and I'll add you as a developer. Otherwise, I don't believe there is any reason to change the registration mechanism so I think you are safe there. However, the functions should extend FunctionBase and implement Function to ensure the appropriate methods are available. In particular, there are some getMinArgs, getMaxArgs, and getSignature used to help assess when using the Context is allowed and to determine optional arguments. The function signature is defined as a pattern as defined here: https://docs.jsonata.org/programming#function-signatures. At present there are  no functions defined starting with X so I could see reserving function names with X_ prefix as you've suggested to ensure no collisions and advertising them as extensions. This would be a local agreement for JSONata4Java as I can't speak for Andrew Coleman and the large following  at jsonata.org.
I'm glad you are finding the library useful. I'm continuing to work on covering all the capabilities of jsonata.org but it is quite an extensive library. I'll have a new release soon that adds more compatibility for arrays, and possibly the lambda character function declarations.

I agree that contributing to the open source would be the best option.
However in this case the custom functions that I need are strongly related to the product/context/scenario where I'm looking to use JSONata, so I was looking to add them in the consumer project.
I've already implemented some functions, and I was wondering how to reduce the risk to have to rewrite them due to changes in the next JSONata4Java releases.
I will use _ for the name, they should never conflict with JSONata functions that looks to use a camel case naming convention.
About the implementation, I was surprised to see that most of the function code is to implement validations, even if most can be extrapolated by the signature (I also see you have Signature class with a validate method, but currently this is not used)
Thanks,
Franco

Yes, the Signature is not ready for use so I haven't fully implemented the logic.
I  believe you should use a letter and underscore since I believe these can be considered variables and they should begin with a letter. So rather than just underscore, if you would use X_ as the prefix I think we'll be in better compliance.


[1.12] Dependency not listed on Curseforge

Apparently it requires tesla core lib, but it does not list that as a dependency.

Check the dependencies in the files


Return headers in Properties for access to Content-Type

Description
Exposes Content-Type and other header values from GetObjectMetadataAsync

This is useful. Thanks for your contribution!

Released.

https://www.nuget.org/packages/FluentStorage/5.6.0
https://www.nuget.org/packages/FluentStorage.Azure.Blobs/5.3.0
https://www.nuget.org/packages/FluentStorage.AWS/5.5.0


Missing description for ChangeSettingsAccountEnabled

Published page contains placeholder instead of description for ChangeSettingsAccountEnabled.
"
-ChangeSettingsAccountEnabled
This parameter is functional only in the cloud-based service.
 {{ Fill ChangeSettingsAccountEnabled Description }}

"
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 022da80e-29cd-f6b5-aa00-48e2d16c4be8
Version Independent ID: c3353c03-e416-f693-104f-6b6f5955b721
Content: Set-OwaMailboxPolicy (ExchangePowerShell)
Content Source: exchange/exchange-ps/exchange/Set-OwaMailboxPolicy.md
Service: exchange-powershell
GitHub Login: @chrisda
Microsoft Alias: chrisda

@jalrok, I'm tempted to resolve this as a duplicate of #10276.
Same question: are you being directed to use that parameter from other content, or are you just pointing out the fact that it's empty?


[WIP] LH2: send map size with computed locations

What remains to do:

add the missing tests
~- add the missing UI in the web page~

Codecov Report

Merging #98 (97c1848) into main (f4b3966) will decrease coverage by 0.11%.
The diff coverage is 75.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##             main      #98      +/-   ##
==========================================
- Coverage   76.32%   76.21%   -0.11%
==========================================
  Files          18       18
  Lines        1533     1564      +31
==========================================
+ Hits         1170     1192      +22
- Misses        363      372       +9

Impacted Files
Coverage Δ

dotbot/controller.py
33.73% <ø> (ø)

dotbot/tests/test_protocol.py
100.00% <ø> (ø)

dotbot/server.py
97.27% <57.14%> (-2.73%)
:arrow_down:

dotbot/lighthouse2.py
49.69% <62.50%> (+0.32%)
:arrow_up:

dotbot/protocol.py
98.47% <81.25%> (-1.53%)
:arrow_down:

dotbot/models.py
100.00% <100.00%> (ø)