Fix internal API writing to ReadOnlySpans

(follow-up from https://github.com/badcel/HidApi.Net/issues/101#issuecomment-2282892674)
Currently, the internal API of NativeMethods takes ReadOnlySpan parameters, and writes to them. This is undefined behavior and can lead to bugs/errors. Everything works as of now since you're only calling it from sites following a pattern where Spans would be equivalent, but new/different calls could lead to unpredictable behavior.
This could be fixed by changing the implementations in NativeMethods like in this example:
- public static int GetFeatureReport(DeviceSafeHandle device, ReadOnlySpan<byte> data)
+ public static int GetFeatureReport(DeviceSafeHandle device, Span<byte> data)
  {
      return GetFeatureReport(device, ref MemoryMarshal.GetReference(data), (nuint) data.Length);
  }

And the implementations in Device like this:
  public ReadOnlySpan<byte> GetFeatureReport(byte reportId, int maxLength)
  {
      if (maxLength < 1)
          throw new ArgumentOutOfRangeException(nameof(maxLength), maxLength, "Please provide a value greater than 1");
      var data = new byte[maxLength];
      data[0] = reportId;
-     ReadOnlySpan<byte> spanData = data;
+     Span<byte> spanData = data;
      var result = NativeMethods.GetFeatureReport(handle, spanData);
      if (result == -1)
          HidException.Throw(handle);
      return spanData[..result];
  }

This way the type checker will notice writes to potentially dangerous ReadOnlySpans, without affecting the current API or behavior :)
What do you think?

From my point of view the code is fine. I see no bug or potential bug.
I don’t understand what you mean with this: “ This way the type checker will notice writes to potentially dangerous ReadOnlySpans”
Some memory is allocated and stored in a variable which describes its content as a pointer with a length. The content of this variable can’t be changed via its underlying type as it just don’t have the necessary methods. I pass this variable on to native code as a pointer and a length. The native code fills the memory. In the end the variable is returned to the user.
I don’t see why allocating a Span makes any difference?
can you give a concrete sample where there would be a problem?

A problem will occur if a ReadOnlySpan is passed in that is expected to be read-only. For example, if you decided to add overloads to the other Get functions, but kept them as ReadOnlySpans, calling them would result in invalid behavior, which the compiler would warn about if Span was used instead.
Additionally (although very unlikely), this could potentially break due to new/different optimizations in the compiler. The code is creating a zero-filled array that's supposed to be read-only, and it's not calling an unsafe operation, so it could be valid to optimize such that attempting a write leads to crashes or unpredictable behavior.
Are you using ReadOnlySpan instead of Span to improve performance here? Or is there some other advantage to the current implementation? I couldn't find any information about this approach, so I'm assuming the type conversion is optimized away enough that it doesn't matter.

There is no advantage to this approach. I agree that it is a bit mysterious to pass an empty ReadOnlySpan to some method and afterwards it is filled. This code is probably misleading for contributors. When developing this code I did not see this aspect of it.
You mean the compiler would error out if I try to convert a ReadOnlySpan to a regular Span?
In regard to "Spans" in general you can treat them a little bit like an interface. They promise to provide you some valid memory address and the size of this memory block which is safe to access.
On this level it does not matter if you have the read only Span variant or not. It's just a pointer. The access regulation got lost. So the compiler can't optimize anything because behind the "Span interface" and its pointer there is some actual memory hidden.
For example a new byte[8] (on the heap) or some memory on the stack (stackalloc byte[8]). The user of the span does not know what kind of memory it is. Just how big it is and which area is allowed to be accessed.
So what should the compiler optimize? Spans do not allocate anything, they point to existing things.
This implementation avoids some CPU cycles because no new "Span" must be created as we would need two different instances.

If your only reason for this implementation is optimization, I really think you don't need it - both compile to the same assembly:
; ReadOnlySpan<byte> buffer = new byte[1024]; return buffer;
Program:GetReadOnlySpanDirect():System.ReadOnlySpan`1[ubyte] (FullOpts):
G_M34647_IG01:  ;; offset=0x0000
       push     rbp
       mov      rbp, rsp
G_M34647_IG02:  ;; offset=0x0004
       mov      rdi, 0x7D047B634180      ; ubyte[]
       mov      esi, 0x400
       call     CORINFO_HELP_NEWARR_1_VC
       add      rax, 16
       mov      edx, 0x400
G_M34647_IG03:  ;; offset=0x0021
       pop      rbp
       ret

; Span<byte> buffer = new byte[1024]; return buffer;
Program:GetReadOnlySpanIndirect():System.ReadOnlySpan`1[ubyte] (FullOpts):
G_M53968_IG01:  ;; offset=0x0000
       push     rbp
       mov      rbp, rsp
G_M53968_IG02:  ;; offset=0x0004
       mov      rdi, 0x7D047B634180      ; ubyte[]
       mov      esi, 0x400
       call     CORINFO_HELP_NEWARR_1_VC
       add      rax, 16
       mov      edx, 0x400
G_M53968_IG03:  ;; offset=0x0021
       pop      rbp
       ret

I'm certain this also holds true for JIT compiler. Since Microsoft built this API for performant operations, and they are recommending to use ReadOnlySpans instead of Spans for common operations like logging, this would probably be addressed somewhere in their usage guidelines.

Ah I did not know that it is this highly optimized. Additionally your thought that it is not only important for the reader of code but even for the compiler to use an appropriate type makes a lot of sense.
I learned something new the last days, thank you!
From a coding style point of view I would tend to something like this:
public ReadOnlySpan<byte> GetFeatureReport(byte reportId, int maxLength)
{
    if (maxLength < 1)
        throw new ArgumentOutOfRangeException(nameof(maxLength), maxLength, "Please provide a value greater than 1");

    var data = new Span<byte>(new byte[maxLength]);
    data[0] = reportId;
    var result = NativeMethods.GetFeatureReport(handle, data);

    if (result == -1)
        HidException.Throw(handle);

    return data[..result];
}

What do you think, this makes the code a little more compact. If you agree feel free to create a PR. :rocket:

I think your view on the things is absolutely right. This project is more of a side project for me, but this is the same reason I want to have it as clean and expressive as possible: To avoid bugs and unhappy users.
If you are interested in improving this code base even more, it is very welcome. I would even dare to mention that I split out part of HidApi.Net in a separate library called WcharT.Net. I thought it might be of use for other interop projects even if it is relatively small. If you are interested I would welcome any feedback on this project, too. Especially as it is not at the same quality level as HidApi.Net. It improved a lot since the original code in HidApi.Net but I'm not even sure it works in 100% of the cases.
This library is probably one of the bigger reason why HidApi is cross platform in comparison to the other competing nugets.
Let's end this comment with a big thank you for all the time you already invested into this project.

@TimonLukas Do you plan on contributing more before you would need a new release of HidApi.Net?

@badcel I won't be able to spend more time on this in the next few weeks sadly. So I'd suggest that you create the release, and potential improvements are left for the future - there shouldn't be a need to hard break with the API that I'm aware of right now :) thank you!


fix: Updated authentication check endpoint to take the auth token as …

…POST request data

@Vafilor getting these errors.


Different AssemblyName across *.props files

If we list Test.Props files, we will see some differences:
base/test.props
  <PropertyGroup>
    <AssemblyName>$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

dependencycollector/test.props
  <PropertyGroup>
    <AssemblyName>$(RootNamespace).$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

Web/Test.props
  <PropertyGroup>
    <AssemblyName>Microsoft.ApplicationInsights.$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

e2etests/test.props
  <PropertyGroup>
    <AssemblyName>$(RootNamespace).$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

Web/Test/test.props
  <PropertyGroup>
    <AssemblyName>$(RootNamespace).$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

Question: does it matter what would be the assembly name for our test projects? If no, can we add this section to Test.props inside .props (root folder) and make all reference this?
  <PropertyGroup>
    <AssemblyName>$(MSBuildProjectName)</AssemblyName>
  </PropertyGroup>

I would expect AssemblyNames to be defined in their csproj file.
If I needed to lookup a project's AssemblyName, that's the first place I would inspect.
I'm spot-checking our csprojs and most of them already define the AssemblyName.
I think we should try to remove this from the props.


[WebView] Add additionalHttpHeaders to WebContent.Url

I have a use case where I need to pass additional HTTP headers to WebView.loadUrl().
While extending WebContent.Url I noticed that state.content is reassigned in several callback methods of WebViewClient, for example in shouldOverrideUrlLoading(). While additionalHttpHeaders are used for the initial loadUrl() call, this means that the state property additionalHttpHeaders is lost on reassignment. I'm not sure if this poses an issue and if so how to handle this situation.

@bentrengrove Can you please have a look at this PR?

I think if we were to support this you would have to ensure those additional headers weren't lost when the page was loaded and the state reassigned. For instance, you might need the headers passed across multiple redirects and the way it is right now they wouldn't be.

I think if we were to support this you would have to ensure those additional headers weren't lost when the page was loaded and the state reassigned. For instance, you might need the headers passed across multiple redirects and the way it is right now they wouldn't be.

I can make sure that the headers are not lost by using the copy() operator on WebContent.Url. However whether WebView keeps using the headers passed to the initial loadUrl() call on redirects is implementation detail of the WebView and out of our control imho.

Oh, I get it know. You mean when state.content is reassigned which triggers a new loadUrl()? I just pushed a commit ensuring that additionalHttpHeaders are not lost on reassignment. Please have a look.

I don't think a mutable map is the solution here. Does webview actually return the headers it used in onPageFinished or anything like that where we update our state? We basically try to mirror the state of the webview with our state class, that is why it is reassigned all the time. If that was the case then we could store the headers again there.

I don't think a mutable map is the solution here. Does webview actually return the headers it used in onPageFinished or anything like that where we update our state? We basically try to mirror the state of the webview with our state class, that is why it is reassigned all the time. If that was the case then we could store the headers again there.

I'm just passing a mutable map because the documentation of WebView's loadData() states that

Some older WebView implementations require additionalHttpHeaders to be mutable.

I don't think we have access to the actual headers that are used by WebView. The only data we get in onPageFinished are the WebView itself and the URL. I believe WebView is more or less a black box in that matter.

Since the attribute is called additionalHttpHeaders I think it is fine that they don't match the actual headers which WebView is using because - as I assumed previously - we probably don't have access to the actual headers.
With the current implementation additionalHttpHeaders are used for the initial and all subsequent loadUrl calls. Should the user however desire that subsequent requests use different headers, I guess we have to think of another solution then.

Is there anything left to do?

Yes, we would still need tests and documentation in order to merge this in. You would also need to run ./gradlew metalavaGenerateSignature to update the API signature.

Could probably make the header map non-nullable, default it to emptyMap() and apply it to loadUrl() if it's not empty.

Could probably make the header map non-nullable, default it to emptyMap() and apply it to loadUrl() if it's not empty.

Sure, why not. I don't have a strong opinion on that one. What do you think @bentrengrove?

I updated the documentation, metadata and added a test. Hope you like it 🤞🏼😃
PS: How can I get rid of the following error when calling any Gradle command?
> Failed to apply plugin 'com.vanniktech.maven.publish'.
   > The property SONATYPE_NEXUS_USERNAME was removed. Use mavenCentralUsername instead.

I had to temporarily comment out the com.vanniktech.maven.publish plugin in all build.gradle files.

This all looks good to me and I like the emptyMap(). It just needs a rebase now as another WebView change was merged in ahead of it sorry.
As for the sonatype error, I am not actually sure! I am not seeing that here.

Branch is now rebased on main.

As for the sonatype error, I am not actually sure! I am not seeing that here.

Is there anything else to do after checking out the project via Git? Maybe I'm missing some environment variables?

Thanks for the rebase.
I can't spot anything environment variables in my setup at all, there shouldn't be anything you have to do

Hm, it seems the test is flaky?

com.google.accompanist.web.WebTest > testAdditionalHttpHeaders[test(AVD) - 9] FAILED
androidx.compose.ui.test.junit4.android.ComposeNotIdleException: Idling resource timed out: possibly due to compose being busy.

I pushed another commit, trying to work around the flaky test similar to testLoadingState().

That test is marked as flaky with an issue for it to be fixed, as in it is currently broken. Copying that test is not a valid workaround sorry. You might just be able to increase the timeout.

That test is marked as flaky with an issue for it to be fixed, as in it is currently broken. Copying that test is not a valid workaround sorry. You might just be able to increase the timeout.

I assumed both tests are flaky because they both use MockWebServer and the fix would be the same. I can remove the annotation, sure. But I'm unable to test whether the latest modifications fixed the test since I'm not allowed to run the GitHub workflows.

Happy to rerun it for you but yeah my point is that I think it will still be flaky because that test is currently marked as flaky, it's not working around anything.

Please rerun :) If it works now I will remove the annotation.

I removed the annotation and increased the timeout. However the test that was failing for the latest run is testNavigatorCanGoForward(), unrelated to this PR.

By the way, the test was / is "working on my computer" even before the latest modifications. So it's really hard for me to reproduce and fix this 😐

Unfortunately the test is still failing 😞  Do you @bentrengrove have an idea? Because I've run out of ideas 🤷🏼♂

Maybe go back to the original version of the test when I approved it, add the flaky annotation and then I will have a look and see if I can fix both tests at the same time

I reverted the test changes

Thank you!


Fix typo, add required php version, composer instructions.

To help avoid frustrations I ran into with Sami 4 not working correctly (without any error messages) on PHP 5.  Also instructions on how to use composer to install, as an alternative to the .phar.
Also, fixed a typo.

Closing as I've added the info in the README now. Thanks.


LECL Chain doesn't process multiple inputs properly.

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.

Example Code
This is my code:

prompt = """
This prompt takes two variables like this:

variable1:
{variable_1}

variable2:
{variable_2}

"""

model_choice="gpt-4-0125-preview"
temperature=0.0

# Initialize the model and prompt configuration
prompt = ChatPromptTemplate.from_template(prompt)
output_parser = StrOutputParser()
model = ChatOpenAI(model=model_choice, temperature=temperature)
setup = RunnableParallel({"variable1":RunnablePassthrough(),"variable2":RunnablePassthrough()})

llm_output = (
    setup
    | prompt
    | model
    | output_parser
)

llm_output.invoke({"variable_1":"testing var1","variable_2":"testing var2"})

When the code is executed, the langsmith trace shows runnable sequence input as
{
"variable_1": "testing var1",
"variable_2": "testing var2"
}
But the ChatOpenAI call shows the prompt sent as:
This prompt takes two variables like this:

variable1:
{'variable_1': 'testing var1', 'variable_2': 'testing var2'}

variable2:
{'variable_1': 'testing var1', 'variable_2': 'testing var2'}

What am I missing? Shouldn't it be
This prompt takes two variables like this:

variable1:
'testing var1'

variable2:
'testing var2'

Error Message and Stack Trace (if applicable)
No response
Description
I am using Langchain to take two variables in a prompt and send an output. While two variables are in correct JSON format, the prompt that is ultimately sent to OpenAI seems to be incorrect.
System Info
System Information

OS:  Darwin
OS Version:  Darwin Kernel Version 23.3.0: Wed Dec 20 21:30:59 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6030
Python Version:  3.11.7 | packaged by conda-forge | (main, Dec 23 2023, 14:38:07) [Clang 16.0.6 ]

Package Information

langchain_core: 0.1.22
langchain: 0.1.1
langchain_community: 0.0.19
langsmith: 0.0.87
langchain_openai: 0.0.5

Packages not installed (Not Necessarily a Problem)
The following packages were not found:

langgraph
langserve

Solved it by removing runnable pass through.


Removing BrowserRouter Causes Page Rerenders

My tree looks roughly like:
<Provider>
  <ConnectedRouter>
    <BrowserRouter>
      <Switch>
        <Route path="profile-page" component={ProfilePage} />
        ...
      <Switch>
    <BrowserRouter>
  <ConnectedRouter>
</Provider>

and my profile page looks something like :
props => (
     <Header/>
     <CurrentProfile />
)

and my CurrentProfile component looks something like:

props => {
  const {profiles} = useSelector(profileListSelector);
  const {location} = useSelector(locationSelector);
  let currentProfile= null;
  if(profiles[location.query.id]){
    currentProfile = profiles[location.query.id];
  }
   return (
  <h1>{currentProfile.name}</h1>
  )
}

So relatively simple, I've got a bunch of pages, but pages are just display / layout components and don't interact with redux at all. On the page is a currentprofile component that displays the current profile that it gets from redux. Therefore when I change the current profile, I do not need to rerender the page component, just the current profile component.
This works as expected with my current setup, however, I noticed that if I leave the BrowserRouter in my tree, the @@router/LOCATION_CHANGED event doesn't fire from normal react-router-dom links and the documentation says to remove it.
However, when I remove the BrowserRouter from my tree, it's causing my page component to rerender whenever I change any portion of the URL, including query params that come after that page's route.
i.e. if I call push('/profile-page/?id=14') and then  push('/profile-page/?id=21')  I don't expect the page to rerender since the page itself is only dependent on the first portion of the url, and it hasn't changed. This is how it behaves with a BrowserRouter in my tree; the Profile Page component will not rerender. However, once I remove BrowserRouter and just have ConnectedRouter, it will rerender the full page whenever the URL changes, even if it's just the query param that's changing.
How do I implement the memoizing route behaviour of BrowserRouter with ConnectedRouter?

I had the same problem. I found that in BrowserRouter there was a prop called forceRefresh, and when I set true, it forced a refresh.However, there is no similar API available in ConnectedRouter

Push action is always being called for some reason when clicking on link and it's being added to history, even if it's exactly the same route (with params). Any solution to this?


Private Endpoint Batch Account and Pool With Public IP

In my corporate environment, we only enable service and GA features.
Since batch account private link is GA, so I would like to use the feature.
But create the pool in private vnet without public ip is in preview
Does this combination make any sense, batch account FQDN is my private endpoint, but the pool is created with public IP?
Why I need a private endpoint batch account to communicate to pool node via public ip?
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ccb356b2-fb60-e833-94c0-68e7b020949e
Version Independent ID: 06b27a8c-86c4-3f99-0393-0cc74f3e3053
Content: Use private endpoints with Azure Batch accounts - Azure Batch
Content Source: articles/batch/private-connectivity.md
Service: batch
GitHub Login: @JnHs
Microsoft Alias: jenhayes

@RonnieQuan Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate. We will update you accordingly.

@JnHs Can you please check and add your comments on this doc update request as applicable

Hi @RonnieQuan, these are separate features and that scenario is supported if it meets your needs. Of course, using private endpoints with pools that don't use public IP addresses is also a supported scenario although you are correct, pools without public IP addresses are still in public preview at this time.
I will close this issue now but let us know if there are further issues related to this document!
#please-close


Disable remote buttons until vcenter is monitored

Description
When you instantiate a vm with vcenter, remote access to it fails.
To Reproduce
Instantiate a vm and try to access the console remotely. (ex: VNC)
Expected behavior
Until vcenter doesn't monitor the vm, remote access to the vm shouldn't appear.
Details

Affected Component: Sunstone
Hypervisor: [e.g. KVM]
Version: 5.12

Additional context
Add any other context about the problem here.

Progress Status

[ ] Branch created
[ ] Code committed to development branch
[ ] Testing - QA
[ ] Documentation
[ ] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

@tinova PRs to be approved:

Code
Docs


overflow checks for money

@NicolasDorier this PR fixes the bugs discussed in my previous PR #80:
Fixes

Checks for overflow in all ctors where an overflow could occurs. The following throws OverflowException now:

	[Fact]
	public void Overflow()
	{
		var money = new Money(9223372036854775808uL);
	}

Validates MoneyUnit values passed in parameters. The following is no allowed any more:

    var unitToUseStr = Request["unitSelectedByUser"];  // "47"
    var unit = int.Parse(unitToUseStr); // 47
    var money = new Money(1, (MoneyUnit)unit);

Oparations involving doubles and ulong are safe now
Operator --(Money left) checks overflows but it still modifies the internal state, making the Money class mutable. The only difference is that with this PR the internal state remains valid.

Also, I have changed the Money.ToString() method's internal implementation. It returns exactly the same strings as before but it is clearer.
There are two new UTs for the Money class (more are required).
Things that still smell bad

operator /() is error prone and a good way to lose money.

    [Fact]
    public void IntegerDivision()
    {
        var total = new Money(1.0m, MoneyUnit.BTC);
        var op = (total/300000) * 300000;
        Assert.Equal(total.Satoshi, op.Satoshi, "we've lost money bro!");
    }

Test 'NBitcoin.Tests.util_tests.IntegerDivision' failed: Assert.Equal() Failure
Expected: 100000000
Actual:   99900000

operator --() and operator ++() change the internal state and Money should be inmutable.

Some operators make no sense because they are okay for integers but no for money.
For example, this could be okay:
     money.AddSatoshis(1)

but, what does this mean?:
     money++

++ and -- make sense for integers but doesn't make any sense for money.

Thanks a lot for this review. Truth to be told I am afraid of having missed that ++ and -- modified internal state, this is a serious bug. I'm merging that and review and act on your comments.

For the division, I will remove it and provide a DivRemainder for euclidian division instead.

On commit 0ce42287165fac7a9ccad346c72f8e0ffffb55e6 I cleaned some stuff. (Add a Split method to split money in parts)
I'm tempted to remove multiply by decimal and double, which can also result in loss.

Split remainder fairly on 4f660fa5abf3a0a4fe72ffa958661df813da07e7


Parsing With statements

Is it possible to parse a query starting with 'WITH' clause?
Like:
with cross_items as ...

I believe a WITH clause like above is not standard SQL, but supported by at least Oracle and SQLServer. See:

[http://www.brighthub.com/internet/web-development/articles/91893.aspx]
[https://msdn.microsoft.com/en-us/library/ms175972.aspx]

It looks like JSQLParser does not have a problem with it (I am using version 0.9.4). The following statement parses just fine:
WITH MySubQuery AS (SELECT Col1, Col2 FROM MyTable) SELECT * FROM MySubQuery
The following also parses correctly and saves you from using non-standard SQL:
SELECT MySubQuery.* FROM (SELECT Col1, Col2 FROM MyTable) AS MySubQuery

I am having some problems with the following: http://pastebin.com/ryhU1Gc5

I see that it's a large file containing multiple statements separated by semicolons. Have you tried breaking that down into separate statements and then parsing each one individually? I'd try that first.

It's probably not the with statement but "x" operators you have which look like a cross product between queries which might be a problem. Remove those and test

On May 10, 2016, at 6:05 AM, mlkammer<EMAIL_ADDRESS>wrote:
I see that it's a large file containing multiple statements separated by semicolons. Have you tried breaking that down into separate statements and then parsing each one individually? I'd try that first.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub

**WITH ** statements are supported. As mujiqadri mentioned this 'x' operator is not supported by JSqlParser. Therefore the first thing to try is a query without it.

Could we close this issue?


[NetAppFiles] Add zones to volume-group, update min for usage-threshold volume create

Related command

az netappfiles
Description

Adds zones as a convenience argument to az netappfiles create volume-group as users had express dificuly passing zones in with complex paramter volumes
Testing Guide

History Notes

[NetAppFiles] az netappfiles volume-group create: Add zones argument to set Availability Zone for volume group volumes
[NetAppFiles] az netappfiles volume create: Update maximum value for --usage-threshold to support large volumes
[NetAppFiles] az netappfiles volume update: Update maximum value for --usage-threshold to support large volumes

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR title and description has followed the guideline in Submitting Pull Requests.

[x] I adhere to the Command Guidelines.

[x] I adhere to the Error Handling Guidelines.

NetAppFiles

aaz https://github.com/Azure/aaz/pull/453

aaz https://github.com/Azure/aaz/pull/453


[BUG] Unauthorized access

Environment
Self-Hosted (Docker)
System
No response
Version
2.1.1
Describe the problem
the "disableConfiguration" has set to "true" but the configuration is able to change via http request

Additional info
No response
Please tick the boxes

[X] You have explained the issue clearly, and included all relevant info
[X] You are using a supported version of Dashy
[X] You've checked that this issue hasn't already been raised
[X] You've checked the docs and troubleshooting guide
[X] You agree to the code of conduct

CWE-ID: CWE-284 (Improper Access Control)
Assigned CVE-2023-5916

Seems like a Duplicate of #668
Please close this.


Feature Request: Cause-Specific Hazards Models

From this open issue, it seems there isn't much support for competing risks models in lifelines. I find myself working on a competing risks problem for which I'd like to use a cause-specific hazards model. As mentioned in this source:

Cause-specific hazard models can be fit in any statistical software package that permits estimation of the conventional Cox proportional hazards model. One simply treats those subjects who experience a competing event as being censored at the time of the occurrence of the competing event.

So actually achieving this model using two instances of CoxPHFitter isn't horrendous, but it's a bit of a pain to not be able to call, eg, a single survival function. It seems like implementation could be straightforward based on that quote. Is there any interest in a contribution adding such a model to lifelines ?

Hello, I second this request, would be immensely useful


Dropdown scroll with wheel mouse is very very slow

Dropdown scroll with wheel mouse is very very slow.
When user try to scroll with wheel mouse its moving very slow. 2-3 records per scroll.
It should very very smooth like standard dropdown  with 10 records.

@shaangidwani I will check and fix it in the future release.
Thanks for letting me know it.

As per the comment from an another issue, this issue is not exist anymore.
@shaangidwani Please try again


Update setup.py for usability with pip 10

Stop using internal PipSession to read reqs since it's removed in pip 10
See fixes in elex and apfake

Damn, I should probably also do a tag and republish on PyPI.


Use RabbitMQ Queue instead of Channel

Need AMQPClient.queue_declare(rabbitmq_channel[], <queue_name>) added to the RabbitMQ initialization.
Possibly need a change to publish_to_rabbitmq

Fixed in #111


unity3d_env.py bug: done state

In unity3d_env.py, line 161:
bool((raw_data[((self.observation_space.size + self.reward_space.size))]))
will be set to True regardless of the data values.

Hi
Current workaround is using python3.
I am reorganizing the code currently.  When done, I will push to github.

or simply
np.fromstring(raw_data[self.observation_space.size + self.reward_space.size], np.bool)[0]
Anyways this repo has great stuff, keep it up


[Intel GPU] Add Intel GPU backend

In this PR, We add Intel GPU device for PaddlePaddle, which includes:

runtime
mean kernel.
UT of mean kernel
cmake module and some scripts for oneAPI enable.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hi, @qili93 , we seperated the #289 and this is the first PR with only one op for infra, would you please take some minutes to review this PR? thx

Also, codestyle check CI failure, please follow this guide to use pre-commit to fix code style :)
https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/code_contributing_path_cn.html

Also, codestyle check CI failure, please follow this guide to use pre-commit to fix code style :)
https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/code_contributing_path_cn.html

Hi @qili93 Seems In the code style check pipeline, submodule is not fetched, so some symlinks which source is in submodule are broken, could u assign a validation engineer or point out where the scritds should be modified?

Also, codestyle check CI failure, please follow this guide to use pre-commit to fix code style :)
https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/code_contributing_path_cn.html

Hi @qili93 Seems In the code style check pipeline, submodule is not fetched, so some symlinks which source is in submodule are broken, could u assign a validation engineer or point out where the scritds should be modified? I think we should fetch submodule then map the dir to docker container

codestyle CI check will skip submodule code, the CI failure is caused by these codes

So please run the following steps
# go to the root directory
cd PaddleCustomDevice

# ensure precommit and clang-format has been installed
pre-commit install

# run the command showd in CI error message above
# it will automaticlly fix the code stype error of your codes
pre-commit run --files backends/intel_gpu/CMakeLists.txt backends/intel_gpu/README.md backends/intel_gpu/cmake/dummy.c.in backends/intel_gpu/cmake/external/gflags.cmake backends/intel_gpu/cmake/external/glog.cmake backends/intel_gpu/cmake/external/gtest.cmake backends/intel_gpu/cmake/external/mkldnn.cmake backends/intel_gpu/cmake/external/onednn_gpu.cmake backends/intel_gpu/cmake/external/pybind11.cmake backends/intel_gpu/cmake/generic.cmake backends/intel_gpu/cmake/paddle.cmake backends/intel_gpu/cmake/third_party.cmake backends/intel_gpu/kernels/dnn_support.hpp backends/intel_gpu/kernels/mean_kernel.cc backends/intel_gpu/load.sh backends/intel_gpu/runtime/runtime.cc backends/intel_gpu/setup.py.in backends/intel_gpu/tests/CMakeLists.txt backends/intel_gpu/tests/test_mean_op.py

# re-commit the code after change
git add .
git commit -m "xxxx"
git push xxx

Detailed refer to the doc link showed in CI message here:https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/codestyle_check_guide_cn.html

BTW, propose to provide an official dockerfile for compile and development, for example, https://github.com/PaddlePaddle/PaddleCustomDevice/blob/develop/backends/custom_cpu/tools/dockerfile/Dockerfile.ubuntu18-x86_64-gcc82

Also, codestyle check CI failure, please follow this guide to use pre-commit to fix code style :)
https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/code_contributing_path_cn.html

Hi @qili93 Seems In the code style check pipeline, submodule is not fetched, so some symlinks which source is in submodule are broken, could u assign a validation engineer or point out where the scritds should be modified? I think we should fetch submodule then map the dir to docker container

codestyle CI check will skip submodule code, the CI failure is caused by these codes

So please run the following steps
# go to the root directory
cd PaddleCustomDevice

# ensure precommit and clang-format has been installed
pre-commit install

# run the command showd in CI error message above
# it will automaticlly fix the code stype error of your codes
pre-commit run --files backends/intel_gpu/CMakeLists.txt backends/intel_gpu/README.md backends/intel_gpu/cmake/dummy.c.in backends/intel_gpu/cmake/external/gflags.cmake backends/intel_gpu/cmake/external/glog.cmake backends/intel_gpu/cmake/external/gtest.cmake backends/intel_gpu/cmake/external/mkldnn.cmake backends/intel_gpu/cmake/external/onednn_gpu.cmake backends/intel_gpu/cmake/external/pybind11.cmake backends/intel_gpu/cmake/generic.cmake backends/intel_gpu/cmake/paddle.cmake backends/intel_gpu/cmake/third_party.cmake backends/intel_gpu/kernels/dnn_support.hpp backends/intel_gpu/kernels/mean_kernel.cc backends/intel_gpu/load.sh backends/intel_gpu/runtime/runtime.cc backends/intel_gpu/setup.py.in backends/intel_gpu/tests/CMakeLists.txt backends/intel_gpu/tests/test_mean_op.py

# re-commit the code after change
git add .
git commit -m "xxxx"
git push xxx

Detailed refer to the doc link showed in CI message here:https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/codestyle_check_guide_cn.html

Hi @qili93, as the following pic, if we don't use symlink, it's hard to resued some submodule like glog gtest gflag, we just follow npu impl here

Also, codestyle check CI failure, please follow this guide to use pre-commit to fix code style :)
https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/code_contributing_path_cn.html

Hi @qili93 Seems In the code style check pipeline, submodule is not fetched, so some symlinks which source is in submodule are broken, could u assign a validation engineer or point out where the scritds should be modified? I think we should fetch submodule then map the dir to docker container

codestyle CI check will skip submodule code, the CI failure is caused by these codes

So please run the following steps
# go to the root directory
cd PaddleCustomDevice

# ensure precommit and clang-format has been installed
pre-commit install

# run the command showd in CI error message above
# it will automaticlly fix the code stype error of your codes
pre-commit run --files backends/intel_gpu/CMakeLists.txt backends/intel_gpu/README.md backends/intel_gpu/cmake/dummy.c.in backends/intel_gpu/cmake/external/gflags.cmake backends/intel_gpu/cmake/external/glog.cmake backends/intel_gpu/cmake/external/gtest.cmake backends/intel_gpu/cmake/external/mkldnn.cmake backends/intel_gpu/cmake/external/onednn_gpu.cmake backends/intel_gpu/cmake/external/pybind11.cmake backends/intel_gpu/cmake/generic.cmake backends/intel_gpu/cmake/paddle.cmake backends/intel_gpu/cmake/third_party.cmake backends/intel_gpu/kernels/dnn_support.hpp backends/intel_gpu/kernels/mean_kernel.cc backends/intel_gpu/load.sh backends/intel_gpu/runtime/runtime.cc backends/intel_gpu/setup.py.in backends/intel_gpu/tests/CMakeLists.txt backends/intel_gpu/tests/test_mean_op.py

# re-commit the code after change
git add .
git commit -m "xxxx"
git push xxx

Detailed refer to the doc link showed in CI message here:https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/codestyle_check_guide_cn.html

Hi @qili93, as the following pic, if we don't use symlink, it's hard to resued some submodule like glog gtest gflag, we just follow npu impl here

Got it, thanks and the PR CI issue has been fixed :)
Please kindly help to resolve the review coments above.

BTW, propose to provide an official dockerfile for compile and development, for example, https://github.com/PaddlePaddle/PaddleCustomDevice/blob/develop/backends/custom_cpu/tools/dockerfile/Dockerfile.ubuntu18-x86_64-gcc82

Hi @qili93 assume there is no intel data center gpu max series hardwares on bcs now, we plan to add a docker file to compile only in other pr, this pr only for establishing

Hi, @Zhiwei35 PR merged and thanks so much for your contribution :)


correct definition of boot method

the boot method called after other service providers have been registered

Thanks for finding this confusing paragraph! 😄


Absolute layout extension

Description of Change

Added ViewInAbsoluteLayoutExtensions class. This static extension class allows the declaratively add children to AbsoluteLayout
Linked Issues

Fixes #25

PR Checklist

[x] Has a linked Issue, and the Issue has been approved(bug) or Championed (feature/proposal)
[x] Has tests (if omitted, state reason in description)
[x] Has samples (if omitted, state reason in description)
[X] Rebased on top of main at time of PR
[X] Changes adhere to coding standard
[ ] Documentation created or updated: https://github.com/MicrosoftDocs/CommunityToolkit/pulls/XYZ

Additional information
There doesn't seem to be any documentations for MCT Markup in docs repo, so I haven't included it. I have a sample page included in CommunityToolkit.Maui.Markup.Sample.Pages, however it is not a part of any ViewTree in that solution, as I wasnt able to find an appropriate place to put it. Sample page was tested on Android in a blank project

Thanks @danielftz!!! Could you also open a PR on the Docs repo?
We are still working on writing the C# Markup docs, but you can add the Absolute Layout Extension doc in the /docs/maui/markup/ folder:  https://github.com/MicrosoftDocs/CommunityToolkit/tree/main/docs/maui/markup
And you can check out the /maui/converters/ docs folder (where we have written some docs) for templates + formatting:
https://github.com/MicrosoftDocs/CommunityToolkit/tree/main/docs/maui/converters


Readme setup instructions conda

For https://github.com/Snowflake-Labs/snowpark-python-demos/tree/main/Advertising-Spend-ROI-Prediction
Setup instructions says use pip install conda then run conda ... but get following error:
conda
ERROR: The install method you used for conda--probably either `pip install conda`
or `easy_install conda`--is not compatible with using conda as an application.
If your intention is to install conda as a standalone application, currently
supported install methods include the Anaconda installer and the miniconda
installer.  You can download the miniconda installer from
https://conda.io/miniconda.html.

Therefore should update the instructions to remove pip install conda and install via miniconda instead

Thanks @samadarshad! I've updated the instructions in README.


added fps cam

I added a fps-cam matrix to my project and figured, that this library could also use it.

The camera controls as implemented by this function make several assumptions on the choice of coordinate base and control mechanism. Also it introduces a specific transformation relation between point of view and world space. In the current form I don't see it fit for inclusion.
At the very least I see the need for specifying the axis about which each parameter rotates the view. And then there's the issue that this kind of "FPS" view transform generator is hardly useful for anything but the most simple application. You can't use it in VR, because there you get the intrinsic matrices handed down from the tracker API. And for games camera control is (unfortunately) not that simplistic.

Thank you for your input. I'll see if I have time to improve it in the near future.


tighten up regexes for path items

https://spec.openapis.org/oas/v3.1.0#pathTemplating
"The value for these path parameters MUST NOT contain any unescaped “generic syntax” characters described by [[!RFC3986]]: forward slashes (/), question marks (?), or hashes (#)."

This looks good to me if @MikeRalphson confirms that it accurately reflects the specification.


Configure pre-commit to run in GitHub automatically.

There is a bot which should be configured to run in the repo.

This has been actioned via managing the access permissions for the bot.


Broaden augment target support & action/rpc input/outputs compilation

Allow SchemaNode.get to lookup

Action, Case, Choice, Notification, Output, Rpc & Ext by argument
Input & Output by their statement name

For SchemaNode.compile

Keep choice/case-nodes - for augment targets
Add empty Input/Output nodes to - for augment targets
Compile children of Input/Output

LGTM!
Would prefer more tests to show new stuff working =) Golden tests FTW =) but ain't gonna let that block.... merging!


[ISSUE #7512]  Simplify message sending at the basic layer and reduce redundant judgments

Which Issue(s) This PR Fixes

Fixes #issue_id
issue
Brief Description

How Did You Test This Change?

We're currently processing your upload.  This comment will be updated when the results are available.


whether the return value of crf in the call function needs to be the same In the stage of train and valid ？

hello, @howl-anderson :
May i ask you some questions? Details are as follows：
file path: https://github.com/howl-anderson/addons/blob/feature/crf_layer_on_stable_release/tensorflow_addons/layers/crf.py:
code: def call(self, inputs, mask=None, text=None, **kwargs):
the call func return value is always decoded_sequence in the train and valid phase， but in the keras-contrib (keras-contrib/keras_contrib/layers/crf.py): the call func is defined as "out = K.in_train_phase(train_output, test_output)", it is diffient in the train and valid phase, so i have doubts about whether the return value of crf in the call function needs to be the same In the stage of train and valid？
Looking forward to everyone's answers， thx.

Thank you @bhack for mention me. H, @colynhn thank you for your question, I will do some investigation on this (due to the outdated codebase), and feedback here when it's done.

H, @howl-anderson Is there any conclusion to this question? thank you.


Rewrite C++ FieldValue using traditional polymorphism

While working on migrating FSTFieldValue implementations to FSTDelegateValue I noticed that we make a lot of copies of FieldValue. A lot of copies.
This happens directly as we're building up arrays and maps of values both in proto parsing and in parsing user data. This also happens indirectly as arrays resize themselves and trees rebalance themselves. This means that cheap copies are actually way more important than small size.
At the same time, once we're committed to using std::shared_ptr to avoid copies, there's little reason to maintain the error-prone union mechanism that we have. With a std::shared_ptr to a base class this can end up looking much more like the other ports.
I've added a benchmark that shows some differences:

BM_FieldValueStringCopy shows that std::shared_ptr is 10-20x faster for common string sizes
BM_FieldValueStringHash shows that std::shared_ptr has little effect on operations that use the value: union vs vtable are about equivalent (though I imagine this varies by operation).
BM_FieldValueIntegerFill shows that std::shared_ptr does negatively impact creation of primitive FieldValues, but only by 2x
BM_FieldValueStringFill shows that std::shared_ptr makes string creation ~4x faster
BM_FieldValueCreation shows that std::shared_ptr makes an "average" workload ~2x faster

Note that these are benchmarks running on the host, where it's much easier to show this. The benchmarks also aren't exhaustive--I'm actually uninterested the exact metrics, but more interested in the fact that this isn't an outright bad idea. What these numbers show me more than anything is that we didn't need to go to the complexity of the union implementation, and that pushing for smaller FieldValue objects actually made things worse.
In terms of space used this is a loss. FieldValue itself will stay about the same size (int64 + pointer vs two pointers for shared_ptr). The pointed-to thing will now be larger by a vtable pointer in all cases. Additionally for the primitives that were previously allocated inline we're now allocating memory on the heap. While this is worse it's no worse than what the Objective-C version already does and the simplification and speed-up makes this worth it.
Run on (12 X 2900 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 262K (x6)
  L3 Unified 12582K (x1)
---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_FieldValueStringCopy/4             75 ns         75 ns   53249892
BM_FieldValueStringCopy/64           191 ns        191 ns   22180713
BM_FieldValueStringCopy/1024         150 ns        150 ns   28036597
BM_FieldValueStringCopy/32768        742 ns        742 ns    5801177
BM_FieldValueStringHash/4              5 ns          5 ns  847059393
BM_FieldValueStringHash/64             7 ns          7 ns  586815923
BM_FieldValueStringHash/128           14 ns         14 ns  284254917
BM_FieldValueStringHash/1024          66 ns         66 ns   62470810
BM_FieldValueIntegerFill              40 ns         40 ns   98054551
BM_FieldValueStringFill/4            352 ns        352 ns   10000000
BM_FieldValueStringFill/64           732 ns        731 ns    8676037
BM_FieldValueStringFill/128          543 ns        543 ns    8139361
BM_FieldValueStringFill/1024        2775 ns       2775 ns    2231921
BM_FieldValueCreation                210 ns        210 ns   20110859

---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_FieldValueStringCopy/4              8 ns          8 ns  499717422
BM_FieldValueStringCopy/64             8 ns          8 ns  507560844
BM_FieldValueStringCopy/1024           8 ns          8 ns  504256768
BM_FieldValueStringCopy/32768          8 ns          8 ns  512729111
BM_FieldValueStringHash/4              4 ns          4 ns<PHONE_NUMBER>
BM_FieldValueStringHash/64             6 ns          6 ns  688971857
BM_FieldValueStringHash/128           14 ns         14 ns  301020675
BM_FieldValueStringHash/1024          67 ns         67 ns   63061453
BM_FieldValueIntegerFill              82 ns         82 ns   49015088
BM_FieldValueStringFill/4             93 ns         93 ns   45357204
BM_FieldValueStringFill/64           163 ns        163 ns   25539538
BM_FieldValueStringFill/128          192 ns        192 ns   22902487
BM_FieldValueStringFill/1024         711 ns        711 ns    6349178
BM_FieldValueCreation                 89 ns         89 ns   46928633

This happens directly as we're building up arrays and maps of values both in proto parsing and in parsing user data. This also happens indirectly as arrays resize themselves and trees rebalance themselves. This means that cheap copies are actually way more important than small size.

It feels like a lot of them should be moves. Am I wrong to assume this?

This happens directly as we're building up arrays and maps of values both in proto parsing and in parsing user data. This also happens indirectly as arrays resize themselves and trees rebalance themselves. This means that cheap copies are actually way more important than small size.

It feels like a lot of them should be moves. Am I wrong to assume this?

This assumption is wrong because of the way we do immutability. In the array case, up to a small limit, we just copy the whole array for each addition. In the case of the LLRB tree implementation, we can't change any existing nodes so we copy key and value into any new nodes that are created. Each addition results in log(n) copies.
The one case where do use moves is in building up actual array values during user data and proto parsing. However, if the field values containing those arrays were copied we would still copy.
This change started because I was going to change all of our SortedMap<std::string, FieldValue> to SortedMap<std::string, std::shared_ptr<FieldValue> to make the copies cheap without a structural change, but then I realized that it was insanity to do that and then also have FieldValue also have a std::unique_ptr inside it for all of the non-primitive values. Then I looked at switching to a flat absl::variant internally, but there's a massive size difference between our largest and smallest values.
This implementation makes copies cheap, is otherwise smaller than std::shared_ptr<FieldValue> (for the old FieldValue) in all cases.


deps: weallbehave, weallcontribute, tap, update-notifier

Coverage remained the same at 100.0% when pulling 306986f1f76804ef364e22a0cd52df2e67d5b7ce on deps/update into 57cb1e0988ca5fc23957ebe8e4b924530557133e on latest.


larger images pan/zoom more smoothly than smaller ones, as long as not SCALE_TYPE_CENTER_CROP

Stumbled across a odd problem: our app uses 15 large panorama images (JPEGs). Most of these are in the 14000x5000 px range, but some are smaller: around 7500x5000px
For these smaller images, panning & zooming is always choppy. However, panning & zooming the large panorama's is buttery smooth - that is, until we set setMinimumScaleType(SubsamplingScaleImageView.SCALE_TYPE_CENTER_CROP) - then we get the same choppy behaviour as with the smaller images.

It's possible that on high res screens the smaller images are loaded in full - when I first built this library most phones had a canvas max bitmap size of 2048px but now many support far larger bitmaps and have such high density screens that even a 7500x5000px bitmap won't be tiled or subsampled. The performance of these huge bitmaps may be quite poor.
Try setDebug(true) to see the tile boundaries and sample sizes, that should give you some clues why different images perform better. One option might be to fork the code and override the canvas max bitmap size with a smaller value that works better. If that helps I'll make it a configuration option.

That was exactly it - the smaller images got divided into just two (huge) tiles. Larger images did get multiple tiles, until you set SCALE_TYPE_CENTER_CROP: then they got divided into two tiles as well. (That might be a bug? The overall size of the image hasn't changed - just the pan-limits)
Anyway, I've added a mth to set a max tile size (if set, that overrides the values returned from getMaxBitmapDimensions), and that works out just fine. I can send a pull request if you like.

I don't think the behaviour with SCALE_TYPE_CENTER_CROP is a bug, it's just a combination of screen size, density, max bitmap size, image size and settings that results in large tiles being loaded.
If you don't mind sacrificing a bit of quality, try calling setMinimumTileDpi(200) - by default the view will load tiles subsampled to at least match the density of the screen, and that's not really necessary on a 400dpi screen, especially if you're displaying photos.
A PR would be good, I think this will be useful.

yes, you're right of course - I guess that method does more than I expected: I suppose I was looking for setMinScale() to get rid of the initial letterbox-view of the panoramas.


[SPARK-46083][PYTHON] Make SparkNoSuchElementException as a canonical error API

What changes were proposed in this pull request?
This PR is a kind of a followup of https://github.com/apache/spark/pull/43927 which added SparkNoSuchElementException into errors. This PR makes SparkNoSuchElementException as a canonical error API.
Why are the changes needed?
For the end users to catch those exceptions properly via pyspark.errors.SparkNoSuchElementException with/without Spark Connect.
Does this PR introduce any user-facing change?
Yes, it adds a new API SparkNoSuchElementException class.
How was this patch tested?
Manually tested:
./bin/pyspark

./bin/pyspark --remote local

from pyspark.errors import SparkNoSuchElementException

try:
    spark.conf.get("my_key")
except SparkNoSuchElementException as e:
    print("caught properly")

Was this patch authored or co-authored using generative AI tooling?
No.

cc @itholic

Build: https://github.com/HyukjinKwon/spark/actions/runs/6975938047
Thank you @dongjoon-hyun !

All tests passed. Merged to master for Apache Spark 4.0.0.


Add copyrights to .bat files (and update the checker to report missing ones)

Fixes https://github.com/grpc/grpc/issues/5479.

It looks like there are some more bat files (Ignoring /third_party)
./tools/run_tests/build_artifact_protoc.bat
./tools/run_tests/pre_build_node.bat
./tools/run_tests/run_node.bat
./tools/run_tests/build_artifact_csharp.bat
./tools/run_tests/pre_build_csharp.bat
./tools/run_tests/build_artifact_python.bat
./tools/run_tests/post_tests_csharp.bat
./tools/run_tests/pre_build_c.bat
./tools/run_tests/build_artifact_node.bat
./tools/run_tests/build_node.bat
./vsprojects/build_vs2010.bat
./vsprojects/build_vs2013.bat
./vsprojects/build_vs2015.bat
./vsprojects/coapp/zlib/buildall.bat
./vsprojects/coapp/openssl/buildall.bat
./vsprojects/build_plugins.bat
./examples/csharp/route_guide/generate_protos.bat
./examples/csharp/helloworld/generate_protos.bat
./test/distrib/csharp/run_distrib_test.bat
./test/distrib/csharp/build_vs2015.bat
./src/csharp/build_packages.bat
./src/csharp/buildall.bat

Some of the .bat files already had the copyright headers.
To me it seems  tools/distrib/check_copyright.py  is now passing (and before it reported the list of files that needed to be fixed after I added the functionality to check .bat files)

LGTM.

Sanity is failing because I forgot to update the templates for .bat files.


⚠ Minecraft has degraded performance

In 7774f76, Minecraft (https://www.minecraft.net) experienced degraded performance:

HTTP code: 200
Response time: 1187 ms

Resolved: Minecraft performance has improved in 26d419f.


Fix merge trafficpolicy

Please provide a description of this PR:
fix #39566

type TrafficPolicy struct {
	...
	Tls *ClientTLSSettings `protobuf:"bytes,4,opt,name=tls,proto3" json:"tls,omitempty"`
	// Traffic policies specific to individual ports. Note that port level
	// settings will override the destination-level settings. Traffic
	// settings specified at the destination-level will not be inherited when
	// overridden by port-level settings, i.e. default values will be applied
	// to fields omitted in port-level traffic policies.
	PortLevelSettings []*TrafficPolicy_PortTrafficPolicy `protobuf:"bytes,5,rep,name=port_level_settings,json=portLevelSettings,proto3" json:"port_level_settings,omitempty"`

The api says we should do what this patch does

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

lint_istio
637f3d6ef32bab584771c30485cc7414b5cc8e5b
link
true
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
d0ccee3697618092d60d3399f8365f090b491dbe
link
true
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
aa097b4b482dde0680a2446e2546232976962e01
link
true
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Another thing is current documented behaviour is different from what subset level traffic policy does? I do not know if there are any specific reasons for port level policies to not inherit parent -One  reason I could thin is, if it is a different port in a service, it is considered different set of APIs exposed by same service and hence should not inherit from main policy unlike subsets.
I think we should avoid breaking users - if we have to change may be some flag to control the new behaviour may be better.

One reason I could thin is, if it is a different port in a service, it is considered different set of APIs exposed by same service and hence should not inherit from main policy unlike subsets.

I do believe this was the original design reasoning, as well as indecision on how main, subset, and port policies should interact with one another.  Based on the API, the port level policies should be getting the default values specified in MeshConfig: h2UpgradePolicy, tcpKeepalive, localityLbSetting, timeouts, etc but NOT any of the main or subset trafficPolicy settings.  The DestinationRule API was written before MeshConfig, and so I assume as more settings got added to MeshConfig that implementation detail was overlooked.

I am a bit more inclined to just change the API docs to match the real behavior. WDYT?

TBH, I donot think this is a good way. It will make users feel istio community would respect a mistake they made yesterday rather than fixing it.

TBH, I donot think this is a good way. It will make users feel istio community would respect a mistake they made yesterday rather than fixing it.

I agree. If the original design was to have different port level settings to cater to different set of APIs that @GregHanson seems to also agree with - I think we can introduce a flag to change to new behaviour (documented behaviour) and add a release note to flip it back if needed and we can remove the flag few releases down the line (Similar to how Envoy handles breaking changes with runtime configuration). May be we should need some thing similar in Istio rather than adding too many env var flags. But we can do that later

I agree with this suggestion, @howardjohn WDYT?

TBH, I donot think this is a good way. It will make users feel istio community would respect a mistake they made yesterday rather than fixing it.

Yes, this is exactly what I want. Our users have given us feedback over and over again that the most painful part of Istio is making backwards incompatible changes. Very few things are more important than preserving compatibility IMO. This doesn't seem like one of them.

Similar to how Envoy handles breaking changes with runtime configuration

Envoy's API is largely an implementation detail of products, so I wouldn't use it as a rubrik for a good way to make breaking changes, as these are often hidden by higher level APIs like Istio

@howardjohn What do you mean? Should i continue fixing in this PR or not?

I don't think we should change the behavior of our existing APIs.

// Traffic policies specific to individual ports. Note that port level
// settings will override the destination-level settings. Traffic
// settings specified at the destination-level will not be inherited when
// overridden by port-level settings, i.e. default values will be applied
// to fields omitted in port-level traffic policies.
PortLevelSettings []*TrafficPolicy_PortTrafficPolicy `protobuf:"bytes,5,rep,name=port_level_settings,json=portLevelSettings,proto3" json:"port_level_settings,omitempty"`

This is not changing the API, on the opposite, I think the current implementation is not right.

It is bad. But making breaking API changes is the absolute worst thing we can do based on user feedback research


[Feature]: The cfs-cli displays disk information of DataNode

Describe the solution you'd like
The following information needs to be displayed:

Missing dp situation (lack partitions)
dp quantity, extent situation
Space usage, load

Additional context
At present, the bad disk information will be reported and displayed through metrics, but there is no usage display of the disk dimension on the cli side.

Is this problem still exists and need to be resolved ?


Fixed an issue with no format-stream in amlogic devices

Summary: Fixed an issue with no stream-format in amlogic devices
Type: Fix
Test Plan: Unittests, Component test, Manual tests
Jira: RIALTO-545

https://jenkins.code.rdkcentral.com/job/github-triggered/51963/ started

https://jenkins.code.rdkcentral.com/job/build-component/91113/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/15985/ started

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/build-component-platform/23446/ started.

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44395/

https://jenkins.code.rdkcentral.com/job/github-triggered/52025/ started

https://jenkins.code.rdkcentral.com/job/build-component/91285/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/16016/ started

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/build-component-platform/23581/ started.

build-brcm-refboard2 Console Log uploaded to Gist URL: https://gist.github.com/rdkcmf-jenkins/8a7636cc3ad63fdc77bf0e1782aa5cab

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44546/

https://jenkins.code.rdkcentral.com/job/github-triggered/52027/ started

https://jenkins.code.rdkcentral.com/job/build-component/91294/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/16017/ started

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/build-component-platform/23590/ started.

https://jenkins.code.rdkcentral.com/job/github-triggered/52029/ started

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/16018/ started

https://jenkins.code.rdkcentral.com/job/build-component/91295/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/build-component-platform/23593/ started.

build-brcm-refboard2 Console Log uploaded to Gist URL: https://gist.github.com/rdkcmf-jenkins/495d6f62ca65c77599337b7e790a6fc6

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44550/

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44555/

https://jenkins.code.rdkcentral.com/job/github-triggered/52055/ started

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/16031/ started

https://jenkins.code.rdkcentral.com/job/build-component/91307/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/build-component-platform/23606/ started.

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44572/

https://jenkins.code.rdkcentral.com/job/github-triggered/52126/ started

https://jenkins.code.rdkcentral.com/job/build-component/91410/ started.

https://jenkinsrdkm.cmf.code.rdkcentral.com/job/github-triggered/16068/ started

build-brcm-refboard2 Console Log uploaded to Gist URL: https://gist.github.com/rdkcmf-jenkins/44929b65f64b47425e162faa24a9e499

Sanity tests for ['RPI4 Mediaclient'] added.
https://jenkinsrdkm.cmf.code.rdkcentral.com/job/test-components-flow/44714/


fix(scully): attach validator correctly

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/scullyio/scully/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Other... Please describe:

What is the current behavior?
validator attached on the wrong insertion point

Issue Number: N/A
What is the new behavior?
the validators are properly attached
Does this PR introduce a breaking change?

[] Yes
[x] No

Other information

@peterbsmith2 This is the validator fix for your issue


Run example notebooks on CI

details https://stackoverflow.com/questions/35471894/can-i-run-jupyter-notebook-cells-in-commandline
add to azure

Closed by change of workflow #360, CI runs notebooks when merged into master.


feat: Add ETC and Kotti RPC endpoints to config

What was wrong / missing?
RPC Public endpoints for ETC and Kotti Testnet
How was it fixed / added?
Add them to config file so it makes the tutorial I'm working on for Brownie easier to do.

Looks good, just need to add the new network names in tests/main/test_config.py
Would also be great to add support for atlantis and agartha as valid evm rulesets in the config file, and then translate them to byzantium/petersburg prior to launching ganache or solc.  But that's probably worthy of a separate PR.


Fuzzy Searching

Summary:
Currently it is possible to search for specific properties, like title, artist, year and so on.
We can search for an exact match, e.g. with !artist="Lorna Shore".
I think it would be good if we can also perform a fuzzy search or sub-string search, similar to how fzf works
For example, it would be nice if the artist Lorna Shore would also show up for queries like:

Lorna
LShore
shore (case-insensitive)
Sho

Why?
I think this would be especially useful if a field contains special characters/accents that I might not have on my keyboard, so searching for part of the value would be sufficient.
Also, it would allow for listing of similar items, e.g. performing a search like !genre="death" could list:

Death Metal
Deathcore
Blackened Deathcore
Melodic Death Metal

etc etc.

Should be simple to implement...just change == to Contains() when searching...

True...i'll look for a true fuzzy search implementation

True...i'll look for a true fuzzy search implementation

@fsobolev thanks!!

https://github.com/NickvisionApps/Tagger/releases/tag/2023.7.1-rc1

Please test the beta and let me know if the search covers your use cases :)


Removing an operation with mv command does not remove the operation

Describe the bug
When the operation is removed using the mv command, the operation is not getting removed from the mapper's operation list.
And the new list is not updated in the c8y cloud.
To Reproduce

Install thin-edge and connect to c8y cloud
check the supported operations list that is sent to c8y. (For example on c8y cloud the support operations like restart the device, log request, and software management must be visible)
Now remove an operation sudo mv /etc/tedge/operations/c8y/c8y_LogFileRequest /tmp
Now check if the log request operation is still supported on the c8y cloud

On the device, one can observe the mapper logs below
023-05-15T16:00:27.167652807Z  INFO Inotify: send Modified("/etc/tedge/operations/c8y/c8y_LogfileRequest")
2023-05-15T16:00:27.167866808Z  INFO CumulocityMapper: recv Some(FsWatchEvent(Modified("/etc/tedge/operations/c8y/c8y_LogfileRequest")))
2023-05-15T16:00:27.168082809Z ERROR c8y_mapper_ext::converter: Mapping error: No such file or directory (os error 2)
2023-05-15T16:00:27.16820841Z  INFO C8yMapper => Mqtt: send Message { topic: Topic { name: "tedge/errors" }, payload: "No such file or directory (os error 2)", qos: AtLeastOnce, retain: false }
2023-05-15T16:00:27.168425311Z  INFO MQTT: recv Ok(Some(Message { topic: Topic { name: "tedge/errors" }, payload: "No such file or directory (os error 2)", qos: AtLeastOnce, retain: false }))

Expected behavior
When an operation is removed, the new list has to be sent to the c8y cloud excluding the operation that is removed.
The removed operation must not be visible on the c8y cloud.
Screenshots
023-05-15T16:00:27.167652807Z  INFO Inotify: send Modified("/etc/tedge/operations/c8y/c8y_LogfileRequest")
2023-05-15T16:00:27.167866808Z  INFO CumulocityMapper: recv Some(FsWatchEvent(Modified("/etc/tedge/operations/c8y/c8y_LogfileRequest")))
2023-05-15T16:00:27.168082809Z ERROR c8y_mapper_ext::converter: Mapping error: No such file or directory (os error 2)
2023-05-15T16:00:27.16820841Z  INFO C8yMapper => Mqtt: send Message { topic: Topic { name: "tedge/errors" }, payload: "No such file or directory (os error 2)", qos: AtLeastOnce, retain: false }
2023-05-15T16:00:27.168425311Z  INFO MQTT: recv Ok(Some(Message { topic: Topic { name: "tedge/errors" }, payload: "No such file or directory (os error 2)", qos: AtLeastOnce, retain: false }))

Environment (please complete the following information):

OS [incl. version] Ubuntu
Hardware [incl. revision] X64
System-Architecture [e.g. result of "uname -a"]
thin-edge.io version [e.g. 0.1.0]: 0.10.0

Additional context

Proposed solution:
When the file modified event is generated and there is no file present, then remove that operation from the operations list and send the new operations list to the c8y cloud.

QA has thoroughly checked the bug and here are the results:

[ ] Test for ticket exists in the test suite.
[x] QA has tested the bug and it is not reproducable


Can I conduct distributed experimental training on eight GPUs on two servers?

Thanks for your great works.
I have been running through the model recently, but i found that training on a server with four GPUs is a bit slow. therefore, i would to ask
can I conduct distributed experimental training on eight GPUs on two servers?

Yes, you can do so, however you may need to follow mpi settings, here is what I am using as a job template for job submission:
  job_template:
    name: train_seem_v1_focalt_enc6_fpn_dec10_lang_bs{batch_size_train}_ep{epoch}_scw{spatial_class_weight}_sdw{spatial_dice_weight}_smw{spatial_mask_weight}_nsm{num_spatial_memories}_lr{lr}_ts{top_spatial_layers}_fb{fix_backbone}_fl{fix_lang}_fe{fix_encoder}_spa{spatial_on}_grd{grounding_on}_iterbase_pn_v2_maxi{max_iter}_qsn{spatial_query_num}_mc{max_candidate}
    sku: 4x32G8-V100
    mpi: True
    process_count_per_node: 8
    command:
    - ifconfig
    - export GLOO_SOCKET_IFNAME=eth0
    - export DETECTRON2_DATASETS=/mnt/data
    - export DATASET=/mnt/data
    - export DATASET2=/mnt/data2
    - export VLDATASET=/mnt/xxxx
    - export WANDB_KEY=xxxxx
    - export PATH=$$PATH:/mnt/output/xueyanz/coco_caption/jre1.8.0_321/bin
    - export PYTHONPATH=$$PYTHONPATH:/mnt/output/xueyanz/coco_caption
    - python entry.py train --conf_files configs/seem/focalt_unicl_lang_v1.yaml
      --overrides
      SAVE_DIR /mnt/output/xueyanz/mainzvision/seem_v1_focalt_enc6_fpn_dec10_lang_bs{batch_size_train}_ep{epoch}_scw{spatial_class_weight}_sdw{spatial_dice_weight}_smw{spatial_mask_weight}_nsm{num_spatial_memories}_lr{lr}_ts{top_spatial_layers}_fb{fix_backbone}_fl{fix_lang}_fe{fix_encoder}_spa{spatial_on}_grd{grounding_on}_iterbase_pn_v2_maxi{max_iter}_qsn{spatial_query_num}_mc{max_candidate}
      COCO.INPUT.IMAGE_SIZE 1024
      MODEL.DECODER.HIDDEN_DIM 512
      MODEL.ENCODER.CONVS_DIM 512
      MODEL.ENCODER.MASK_DIM 512
      MODEL.DECODER.NUM_OBJECT_QUERIES 101
      FP16 True
      WANDB True
      SOLVER.MAX_NUM_EPOCHS {epoch}
      SOLVER.BASE_LR {lr}
      SOLVER.FIX_PARAM.backbone {fix_backbone}
      SOLVER.FIX_PARAM.lang_encoder {fix_lang}
      SOLVER.FIX_PARAM.pixel_decoder {fix_encoder}
      MODEL.DECODER.COST_SPATIAL.CLASS_WEIGHT {spatial_class_weight}
      MODEL.DECODER.COST_SPATIAL.MASK_WEIGHT {spatial_mask_weight}
      MODEL.DECODER.COST_SPATIAL.DICE_WEIGHT {spatial_dice_weight}
      MODEL.DECODER.TOP_SPATIAL_LAYERS {top_spatial_layers}
      MODEL.DECODER.SPATIAL.ENABLED {spatial_on}
      MODEL.DECODER.GROUNDING.ENABLED {grounding_on}
      TRAIN.BATCH_SIZE_TOTAL {batch_size_train}
      TRAIN.BATCH_SIZE_PER_GPU {batch_per_gpu_train}
      TEST.BATCH_SIZE_TOTAL {batch_size_test}
      VOC.TEST.BATCH_SIZE_TOTAL {batch_size_test}
      SBD.TEST.BATCH_SIZE_TOTAL {batch_size_test}
      REF.TEST.BATCH_SIZE_TOTAL {batch_size_test}
      WEIGHT {pre_unicl}
      RESUME_FROM {resume_from}
      FIND_UNUSED_PARAMETERS True
      ATTENTION_ARCH.SPATIAL_MEMORIES {num_spatial_memories}
      MODEL.DECODER.SPATIAL.MAX_ITER {max_iter}
      ATTENTION_ARCH.QUERY_NUMBER {spatial_query_num}
      STROKE_SAMPLER.MAX_CANDIDATE {max_candidate}


Compile Error in Sveltekit

When I try to add to  SvelteKit v1.0.0-next.230, I get the below error:
svelte-pdf doesn't appear to be written in CJS, but also doesn't appear to be a valid ES module (i.e. it doesn't have "type": "module" or an .mjs extension for the entry point). Please contact the package author to fix.

Here is the implementation:
<script>
  import { onMount } from "svelte";
  let PdfViewer;

  onMount(async () => {
    const module = await import("svelte-pdf");
    PdfViewer = module.default;
  });
</script>

<svelte:component this={PdfViewer} url="YOUR-PDF-URL"/>

Please advise if there is something I can change to make it work it SvelteKit?

I had to change the following line in PdfViewer.svelte:
import pdfjsWorker from "pdfjs-dist/build/pdf.worker.entry";
to
import "pdfjs-dist/build/pdf.worker.entry";
I only have the changes in a local copy of the project, but perhaps worth a shot. My project isn't using SvelteKit, but is using Vite.

Hi, @ralphwest1 Please try with the latest svelte-pdf v.1.0.10, fixed error : it doesn't have "type": "module" or an .mjs extension for the entry point


fix(APIKeyModal): improve screen reader announcement

Closes #6253 #6248
Improved the screen reader announcement.
Issue 1 : This issue is occur in PasswordInput component from Carbon, attaching the recording of PasswordInput.
Issue 2 : The information "This is your unique API key…" was changed from body of APIKeyDownloader to helperText of PasswordInput. Had a discussion with @davidmenendez on this.
Issue 3: Working as expected
Issue 4 to 6 : The success message is currently showing in heading, in that case it won't read success message immediately if we give aria-live or alert ( Also if we follow reading heading approach, in Custom Edit variant we have both Heading change & success message at a time so it will ignore reading heading since giving more importance to success message)
Had a discussion with @davidmenendez and  Fix done : Added a successMessage below
Issue 7 :  Divya Vani Bollayyagari from Raghu's team confirmed that it is working as expected now
Required design input on issue 2 and 4-6
Carbon's PasswordInput https://github.com/user-attachments/assets/163f4d12-e6d5-478b-a254-a7df3f6f907a
What did you change?
packages/ibm-products/src/components/APIKeyModal/APIKeyDownloader.js
packages/ibm-products/src/components/APIKeyModal/APIKeyModal.stories.jsx
packages/ibm-products/src/components/APIKeyModal/APIKeyModal.test.js
packages/ibm-products/src/components/APIKeyModal/APIKeyModal.tsx
packages/ibm-products/src/components/APIKeyModal/APIKeyModal.types.ts
How did you test and verify your work? storybook, Jaws

@matthewgallo ,
Just noticed the needs design opinion. Not approving for now.
@oliviaflory,
Could you please look into this?

lets bring this up with michael gower at the next carbon access guild 👍

Hey all, thanks for reviewing!
Initially I tried to make the title readable by screen reader/jaws, in the current implementation, title of the modal header is wrapped inside a . This means that applying attributes like aria-live or aria-relevant will only affect the  itself and will not trigger the title announcement.

         {label &&
{label}
}         {title &&
{title}
}         {children}
To address this, I tried passing the title as a child of the ModalHeader and applied the necessary accessibility attributes. While in this approach, the successTitle is announced but it does not happen immediately. I believe the successMessage should be announced promptly as soon as it appears on the screen.
Also its valid that we are duplicating the successMessage by displaying it both in the title and at the bottom of the modal.
At the same time in current behaviour of API Key Modal, there are some inconsistency where only error message and editSuccess message is displayed at bottom.
So for now as I mentioned in the Engineering huddle, the alternative solution I proposed is to avoid changing the title for a success. Instead, we will update only the success message displayed below.
Regarding the helper text, it was previously being passed in the body of APIKeyDownloader, but it functions more like a helper text. Additionally, Raghu requested that the text needs to be announced when the TextInput gains focus. For this reason, I moved it as a helper text.
As we discussed in call, lets bring this up with Michael Gower for more clarity!
@oliviaflory The "more content menu" is included as part of the VoiceOver narration, I tried in Carbon TextInput too.
Proposed Solution: Please check deployment preview

based on the carbon accessibility meeting today, this solution is the preferred method. Michael Gower agrees that having the header update isn't the most intuitive and that utilizing the existing area that we are using for status updates (failure, loading) is a better place for that information.
that being said, it sounds like there's an interest in having a larger conversation regarding how status updates are handled in modals. even across the modals in C4P (export, import, create, delete) there's an inconsistency between how each handles success.
export:

also should be noted that the error state for export doesn't appear to be working
apikey:

create & delete: neither have a success, failure, loading, or error state, unlike the other modals which have specific props for displaying and handling those states.
might be something we want to look at 👍 🤔
as far as this specific issue and PR are concerned the solution Anamika has submitted is Gower approved.

Thank you @davidmenendez for discussing this with Michael Gower!
I updated the style of status message with a CheckmarkFilled icon as suggested by @oliviaflory. Now its looking similar to Export Modal success message. Additionally I have deprecated two props editSuccessTitle and generateSuccessTitle instead introduced editSuccessMessage and generateSuccessMessage.

tagging @RichKummer for any other design insight. he mentioned to me removing the info icon next to download and using the check mark, but this new update adds it next to the success message. thoughts?


Fix broken import.

It wouldn't start without this change.

@KnightHawk3 I am not sure how, but I complete missed this! This looks good.
Thank you!


how use destroy?

$(window).on('load ready resize', function () {
sticky = new Sticky('#sticker');
});

Have you tried this?
var sticky = new Sticky('.sticky');

sticky.destroy();

See: https://github.com/rgalus/sticky-js#methods


Feature Request: sub-command to retrieve director manifest

Is it possible to make a command to the effect of bosh director-manifest that returns the manifest with which the bosh director was created?
Referencing https://github.com/cloudfoundry/bosh-bootloader/issues/274

This has come up a few times over the years, but we are not excited about the idea for a couple reasons...

The only state for a create-env'd VM is the state.json. If we were to treat it as the "director database" where a manifest is normally stored, that means adding the manifest into that file. That has a couple problems though...

The manifest may contain credentials, which is not something the state.json file currently has. Since the state should be stored somewhere, we want to reduce the area where credentials may potentially be stored.
If we were able to generate the manifest to store in state.json, that means we have access to the original manifest itself. At that point though, I don't think it makes sense to try and store off a copy because it's already nearby.

If not storing locally, that means we should store the manifest on the VM. If storing on the VM, that likely means we also need to support retrieving from the VM, too. These have a couple issues, too...

There is no API call for storing a raw copy of the manifest on a VM. If the goal of a VM is to be provisioned with the minimal amount of credentials/information/data it needs to be running, this would go against that philosophy. This also isn't something director-managed VMs should support.
Eventually users would want/need a way to retrieve their original manifest off the VM. However, this now introduces a new, significant attack vector for revealing potentially sensitive configuration data off a VM. That's not a route we want to go down, either.

If not storing anywhere, that means a manifest must be generated or retrieved on demand. Typically, scripts should be used to document how bosh create-env is invoked, which then track the ops files and variable references in a self-documenting way. The director-manifest subcommand in that case then becomes interpolate (and without the --state option).

Ultimately what this request typically comes down to is: teams did not do a sufficient job of documenting their deployment strategies when first deploying a VM (and/or we did a poor job recommending it). Rather than provide a way to store off the manifest, we think it's better to encourage documentation/scripting of the deploy itself since it's easy to flag or swap out create-env to review the manifest.
So, those are some of the typical discussions we've seen around this. Does that explain things a bit more? Do you have other thoughts or perspective we should consider around this?

There is no API call for storing a raw copy of the manifest on a VM. If the goal of a VM is to be provisioned with the minimal amount of credentials/information/data it needs to be running, this would go against that philosophy. This also isn't something director-managed VMs should support.

I thought generic configs was an api for storing arbitrary YML, surely the bosh cli could use that? I realize that this is not the canonical place, but if the bosh cli uploaded the secret-free YML of the director manifest after creating the director with create-env, then it would at least be a handy place for future generations of operators to answer questions like "wait, what options did we enable on this director, anyway?"

Ultimately what this request typically comes down to is: teams did not do a sufficient job of documenting their deployment strategies when first deploying a VM (and/or we did a poor job recommending it).

If BOSH has developed a good convention for structuring director manifests, and directory trees of many of those manifests, then why is the BOSH cli not aware of what it is? Can we enshrine this convention in code? What if bosh envs and bosh env were aware of this structure, and could tell you everything about that director?

... generic configs ...

True, although keep in mind that create-env is generic and doesn't only deploy the director. People would also be interested in this functionality for concourse or other deployments.
This would also introduce a dependency for create-env to know how to authenticate to a director and assume it's a recent director which supports generic configs. I could see this as a followup step if people really really really wanted to store off their manifest as a config... but I don't think that should be tied to create-env.

... directory trees ...

Not sure what you mean. I do think it might be useful to have a few more conventions around recommended directory layouts of manifests and ops, but I'm not sure the create-env command would ever be create-env this-directory-of-ops-n-vars-n-stuff. Or, maybe with a lot of conventions, but still, I imagine a file which lists manifest, ops/vars references to be more explicit. This is kind of similar to discussions in https://github.com/cloudfoundry/bosh-cli/issues/287.
What you're mentioning kind of sounds like the Configured with ./configure --so-many --opts which shows up in some binaries' --version-type output. Even in this case though, it requires a lot of context about what those files represent (maybe my super-fast-director-ops.yml means something different than another environment).

@tjvman says "create-env creates parity and reasonable expectations with env/envs command"

I hadn't really considered in a while how the generic bosh create-env is misleading if you end up doing bosh -h and randomly discovering envs. I think there was some lengthy discussions a long time ago when this was introduced, and the confusions it implies since env/envs are director-specific commands. The later points you mention kind of stem around that, and in that sense, I agree.
Generically speaking, I'd expect bosh envs to tell me about the "envs that I created". In that sense, they wouldn't care whether they're bosh, concourse, vpn, nat, whatever. One of the pieces of information it could tell me might be how it was deployed, or which directory I was in, or other highly-specific, local information. Seems a bit weird and too detailed, but crazy possibility.
If that were a direction which would be desirable, I'd then expect the ~/.bosh/config to have many more details, not just bosh director auth/ca information. So, if starting from that point... then there would need to be a command which "binds" a generic "env" which was created to the "director" functionality that the rest of the bosh CLI is used to.  So, maybe I create-env (with a -e alias?) and then follow up with a set-director-env where I might pass a CA cert and which toggles the env as being director-friendly or something.
Really though, that sounds more like a separate bootstrap, wrapping tool. But that's more reminiscent of the dedicated, separate bosh-inti command.
Dunno, just brainstorming; hadn't really thought much about it recently. I appreciate the continued discussion.

True, although keep in mind that create-env is generic and doesn't only
deploy the director...
@tjvman https://github.com/tjvman says "create-env creates parity and
reasonable expectations with env/envs

Aha! I think we might be on to something! I don't particularly find
create-env particularly useful outside of making a director, and I don't
think even concourse-deployment is that strongly advocating using it as the
best way to make a concourse. Could we rename create-env
"deploy-without-a-director"? and begin to make "create-env" a bit more
opinionated about how state and credentials are stored? That way, targeting
and reasoning about directors could be more foolproof by relying on these
conventions?

Hi, I find this discussion very interesting.
Let me tell what we are struggling with: We have created a script and quite some vars and ops files to create our director with create-env everything is checked including the state.json reflecting the state of our director. Unfortunately if some changes config and re-executes deployment on his personal clone of our config repository state.json will be changed. If this operator now forgets to checkin state.json and another operator changes config in his personal folder he will most probably work on an outdated state.json ...
Now coming to a possible solution: As I understand create-env will always create only one vm. This vm is based on normal stemcell. So why not store all state (manifest and state.json) on this vm? bosh cli could then be pointed to that vm and operate similar as a normal director would do - at least make it possible to download manifest and state.json. Does that make sense?

So why not store all state (manifest and state.json) on this vm?

this is one of these tough bootstrapping decisions... if we encourage people to keep their state in the vm, it's very likely that at some point that vm will be damaged and there will be no easy way to get that state out. that means that "recovery" procedure (which is just re-running bosh create-env) will become fairly complex and will be no longer exercised on regular basis.


simplify activity indicator frame setting

I noticed that the frame variable was mutable this diff makes it imutable.

Cool. Thanks


[flutter_tools] check if process manager can find adb

While the Android SDK checks if the adb executable is present, this is not the same logic as what the ProcessManager class does to look up the executable. In some cases, the AndroidSDK can find the executable while the ProcessManager fails to locate it.
Pending a longer term fix of package:process resolution, don't invoke adb unless the ProcessManager can locate the executable.

adb is a bit special because its not downloaded to a known path via artifacts, so more in common with the cocoapods or xcode dependencies. We probably do need this check in more places.

adb is a bit special because its not downloaded to a known path via artifacts, so more in common with the cocoapods or xcode dependencies. We probably do need this check in more places.

g3 failure is a flake.

g3 failure is a flake.


Support for flake8 --diff mode

Coming from having used Syntastic for a short while, I have it configured to lint python files using flake8 roughly as follows: git diff -U0 | flake8 --diff, or in English, "Only show me errors in the code I've just written because I cba fixing up the whole damn file".
Obviously, this only really works when the file is opened or saved because until you write the file to disk where git can't pick up on the diff, so you don't get any output.
It's also particularly useful in conjunction with the loclist split because getting all the errors from the whole file (and there are a lot of them; this is a years long, multi-person project) is information overload, where as only getting the loclist of things I've just made a mess of is much more manageable.
Before I start hacking away and making my dodgy contributions, I wanted to know if this is something that's already supported that I've missed in looking through and searching the documentation, or whether this is a new feature.
Also, whether this is something that has any legs at all for the wider community, or whether it's just me.
Current plan would be getting the equivalent of the following command running and reporting via ALE somehow:
[pipenv run ]flake8--diff -< <(diff -U0 ${filename} ${tmpfilename})

Your best bet would be to define a linter for running that command. See :help ale#linter#Define, and look at a few existing linter files for examples.


core: cargo update

Description
cargo update to update a few vulnerable libraries for dependabot. Also removes the pin on qdrant-client.
Risk
Low
Deploy Plan

deploy core

r? @tdraier as eng runner

PMRR

LGTM


Implement the DROP evaluation

From the GPT-3 paper

Next we evaluate GPT-3 on the task of reading comprehension. We use a suite of 5 datasets including abstractive, multiple choice, and span based answer formats in both dialog and single question settings. We observe a wide spread in GPT-3’s performance across these datasets suggestive of varying capability with different answer formats. In general we observe GPT-3 is on par with initial baselines and early results trained using contextual representations on each respective dataset.
GPT-3 performs best (within 3 points of the human baseline) on CoQA [RCM19] a free-form conversational dataset and performs worst (13 F1 below an ELMo baseline) on QuAC [CHI+18] a dataset which requires modeling structured dialog acts and answer span selections of teacher-student interactions. On DROP [DWD+19], a dataset testing discrete reasoning and numeracy in the context of reading comprehension, GPT-3 in a few-shot setting outperforms the fine-tuned BERT baseline from the original paper but is still well below both human performance and state-of-the-art approaches which augment neural networks with symbolic systems [RLL+19]. On SQuAD 2.0 [RJL18], GPT-3 demonstrates its few-shot learning capabilities, improving by almost 10 F1 (to 69.8) compared to a zero-shot setting. This allows it to slightly outperform the best fine-tuned result in the original paper. On RACE [LXL+17], a multiple choice dataset of middle school and high school english examinations, GPT-3 performs relatively weakly and is only competitive with the earliest work utilizing contextual representations and is still 45% behind SOTA.

[X] Data processing code implemented
[ ] Evaluation implemented

The evaluation code should be modeled after the interface in lm_eval/base.py and the example of the BoolQ task in lm_eval/tasks/suerglue.py

Note: HuggingFace includes this in its datasets package.
https://huggingface.co/datasets/drop

added in ec4d361514ec0779e502460a705bd875cc11299c

Also, should we prepend 'Passage: ' for each passage? It seems OA did not do this, not sure if it was intentional.

My vote is “no” because I see no reason to do so.

@anishthite I don't see a line for this dataset in lm_eval/tasks/init.py. Am I missing something, or is this not quite finished?

@jon-tow How is the work on this coming?


ci: separate integrations

What does this PR do?
Installing these integration introduces cyclic dependencies.

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @carmocca @akihironitta @borda

@carmocca moreover, it addresses an issue with not being able to build docker because of these strategies...