[Bug]: AutoGen Studio - Anyscale Mixtral model -  api_type error

Describe the bug
Hi,
Lates AutoGen Studio v0.0.34a used with a non OpenAI model.
the API base: https://api.endpoints.anyscale.com/v1 and the model : mistralai/Mixtral-8x7B-Instruct-v0.1
Models,  Agents, and Workflow are configured, but got this error:
TypeError: Completions.create() got an unexpected keyword argument 'api_type'
If i use the mixtral-8x7b-instruct-v0.1.Q8_0.gguf in local AI (with Text Generation Webui  - API endpoint) - this error not appears!
So this is only when i use anyscale.com -
Please help.
Steps to reproduce
this is the Model Specification:

Expected Behavior
API Type is empty, and should be accepted as empty.
Screenshots and logs

Additional Information
Lates AutoGen Studio v0.0.34a used with a non OpenAI model.
Linux Mint 21.3 Virginia
Python 3.10.12

if i add
open_ai
in API Type in the Model Specification Mixtral_Anyscale then still get the same error.

Try setting API type to openai. Under model specify the complete model name, for example: mistralai/Mixtral-8x7B-Instruct-v0.1. See if that works? Full model names can be found under the documentation at Anyscale.

Hi Pushad.
Many thanks for the advice!
I tried openai (instead of open_ai) - unfortunately it is still the same error message.
I set the model name :

if its not set there was an error message.
So i assume, "api_type" config parameter, should be configured somehow, but i dont know how.
If you have any advise pls do it.  :)

Hi,
It appears you have the model name filled in at the wrong input.
I just tried it with anyscale and it is working. See my screenshot:

Yess!!!
You are genius!  :)
Many many thanks for the very fast and effective help!!!  :))
best regards, Robert

and it works if you eliminate - leaving empty-
"openai" in the Model Specification  "API Type"

many thanks - im closing this case.

I'm getting a different error

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 443, in handle
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/routing.py", line 292, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/routing.py", line 180, in serialize_response
    return jsonable_encoder(response_content)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/encoders.py", line 287, in jsonable_encoder
    encoded_value = jsonable_encoder(
                    ^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/encoders.py", line 223, in jsonable_encoder
    obj_dict = _model_dump(
               ^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/_compat.py", line 177, in _model_dump
    return model.model_dump(mode=mode, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/pydantic/main.py", line 308, in model_dump
    return self.__pydantic_serializer__.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'method'>

+1 same error here

I'm getting a different error

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/python3.11/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/python3.11/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 443, in handle
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/routing.py", line 292, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/routing.py", line 180, in serialize_response
    return jsonable_encoder(response_content)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/encoders.py", line 287, in jsonable_encoder
    encoded_value = jsonable_encoder(
                    ^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/encoders.py", line 223, in jsonable_encoder
    obj_dict = _model_dump(
               ^^^^^^^^^^^^
  File "/python3.11/site-packages/fastapi/_compat.py", line 177, in _model_dump
    return model.model_dump(mode=mode, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/python3.11/site-packages/pydantic/main.py", line 308, in model_dump
    return self.__pydantic_serializer__.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'method'>

+1 same error here


time_intervals ability like in alertmanager

I want to have a single chat where non-critical alerts sending only in work time, but critical always. I tried to achieve this with settings in alertmanager, but robusta should receive alerts always for history reason, just not sending them.

Thanks for reporting it @lictw
We have adding activity hours on the sinks, on our roadmap.
Would you be interested in contributing this part?

It's nice, I can't contribute big programs with python, so I'm just waiting..) Thanks!


Add integration test for changes in PR #562

What would you like to be added:
Integration tests for changes in #562
Why is this needed:
Improving test coverage

@googs1025 if you want to take a shot at this feel free, I was running into some strange issues adding an integration test for this, probably some error I was overlooking with tired eyes. If not no worries, I'll resume work on it tomorrow.

@danielvegamyhre  Sorry, I just saw it now. Thanks for your invitation.I am not familiar with how to write integration tests. You can finish it first. I will look at this PR to learn how to write integration tests.

Hello, @danielvegamyhre , I'm trying to write this integration test case, but I keep encountering an error. It seems that the jobs are not being deleted properly when performing the deletion. I'm not sure where I went wrong as I'm not familiar with integration testing myself.
ginkgo.Entry("jobset with foreground delete should delete all child jobs", &testCase{
			makeJobSet: testJobSet,
			updates: []*update{
				{
					jobSetUpdateFn: func(js *jobset.JobSet) {
						gomega.Expect(k8sClient.Delete(ctx, js, client.PropagationPolicy(metav1.DeletePropagationForeground))).To(gomega.Succeed())
					},
					checkJobSetState: func(js *jobset.JobSet) {
						expectedJobNum := 0
						gomega.Eventually(func() bool {
							var jobList batchv1.JobList
							gomega.Expect(k8sClient.List(ctx, &jobList, client.InNamespace(js.Namespace))).To(gomega.Succeed())
							return len(jobList.Items) == expectedJobNum
						}, timeout, interval).Should(gomega.Equal(true))
					},
				},
			},
		}),

@googs1025 yeah I ran into the same issue when writing the integration test. Haven't had a chance to debug it yet


refactor: clean up implementation & tests of models with hyperparameters

Summary of Changes

Add leading underscore to the names of all hyperparameters
Add some comments to mark sections of a constructor
Move all validation code to the front of the constructors

🦙 MegaLinter status: ✅ SUCCESS

Descriptor
Linter
Files
Fixed
Errors
Elapsed time

✅ PYTHON
black
30
0
0
0.82s

✅ PYTHON
mypy
30

0
1.79s

✅ PYTHON
ruff
30
2
0
0.05s

✅ REPOSITORY
git_diff
yes

no
0.02s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff
MegaLinter is graciously provided by

:tada: This PR is included in version 0.12.0 :tada:
The release is available on:

v0.12.0
GitHub release

Your semantic-release bot :package::rocket:


FIX the CI (replace with CircleCI?)

The CI seems to be broken, perhaps node version?
Perhaps we should switch to CircleCI if we're going to invest time into fixing it?
Any takers? Should be easy to setup.

@xyc could you connect CircleCI to the repo?

We should also add Chromatic on top of the storybook!
@xyc if you don't have time do do it? can we transfer this repo into an organisation so I can get administrator-right and do it for you?

Build fixed in #60


new file format, automated build, code quality

The biggest change is the support for the new træfik acme.json format. The new format does no longer include an url parameter, so I removed it.
I also implemented a multi-stage build in the Dockerfile to allow automated builds with docker hub. Because of this, I removed the push target in the Makefile (when an automated build is set up, this should no longer be needed).
The other commits should be self-explanatory.

Thank you very much for your contribution.
I'm not working with Kuberenetes ans Dopcker at the moment so I have no setup to test against.
As you basicly rewrote this small utility, we could just declare your fork to be official and I'll point to your Repo from the README. Would you be ok with that?

I am ok with that. I adopted your license and described the origin of my fork in my readme.

@sbruder thank you very much; I linked to your Fork in my README.


Devcontainer does not update remote user's UID to match host machine's UID when group already exists

VSCode Version: 1.67.2 (user setup)
Local OS Version: Windows_NT x64 10.0.22000, but running over Remote - SSH on an Ubuntu 20.04 host.
Local chip architecture: x86
Reproduces in: Remote - Containers
Name of Dev Container Definition with Issue: Update UID feature applied to any devcontainer. In this case, I was using the C++ devcontainer.

Steps to Reproduce:

On the container host, create a user with a UID of something other than the default 1000 (I used 1577 in my case) and a GID of 100.
Create a C++ VSCode devcontainer definition with no modifications and launch the container. I tried this using the ubuntu-20.04 , ubuntu-21.04 and debian-11 variants, it doesn't seem to matter.
Check whether the remote user's UID has been updated to 1577. In my case, it had not been updated, causing permission issues when writing files in the bind mount from the host system.

I did a little bit of investigation into why this is the case by looking at the Dockerfile that the UID update feature uses. It looks like what is happening is that a group with ID 100 already exists in the container so the feature skips updating both the UID and GID. However, I do not understand why this logic is in place as we would still want to update the UID in this case, and I see no harm as well in assigning the vscode user to the existing GID.

Perhaps @Chuxel can weigh in on this design decision.  Just want to make sure - is this purely based on curiosity, or is the current logic we have here causing a problem of some kind?

This sounds like a dupe of https://github.com/microsoft/vscode-remote-release/issues/2402.  This is not about the images in this repo, but rather what is possible when doing UID/GID syncing.
The core challenge is that it's a bit unclear as to what is safe to do in this scenario - low group IDs like that are often system groups, so reusing likely isn't safe to do. I'd suggest moving the conversation to this existing issue.  However, 100 is "users" so in concept that could be made an exception.
Closing as a dupe, more discussion can happen over in the other issue.


MWPW-135198 - Optimizes quiz-results DOM handling

Removed a querySelectorAll
Removed iteration through metadata divs
Added lana logging if the block has been misconfigured and doesn't implement a proper variant

Resolves: MWPW-135198
Test URLs:

Before: https://uar-integration--milo--adobecom.hlx.page/drafts/colloyd/quiz-results-block/quiz-results?quizKey=cc-quiz-result&debug=quiz-results&martech=off
After: https://mwpw-135198-quiz-results-dom-optimization--milo--colloyd.hlx.page/drafts/colloyd/quiz-results-block/quiz-results?quizKey=cc-quiz-result&debug=quiz-results&martech=off

To properly test this, hit the test url and open dev tools. Add the 'cc-quiz-result' key to Local Storage and set the value to this:
{"primaryProducts":["lr","pr"],"secondaryProducts":["ps","au"],"umbrellaProduct":"cc","basicFragments":["https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-marquee-cc","https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-card-list"],"nestedFragments":{"marquee-product":["https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-marquee-product-lr","https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-marquee-product-pr"],"commerce-card":["https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-card-lr","https://main--milo--adobecom.hlx.page/fragments/colloyd/sample-uar-fragments/uar-sample-card-pr"]}}

UT for quiz flow is failing and we will fix that in a follow-up PR. Not related to this PR.

Some of the test unit tests are failing and need to be checked out


RFC dates are too unspecific

From @strogonoff  (https://github.com/ietf-ribose/relaton-data-rfcs/issues/4)

According to Relaton model, we expect an ISO 8601 datetime string:
https://github.com/relaton/relaton-models/blob/main/models/DateTime.lutaml#L11
Now that BibXML service is doing some validation, dates in the RFCs source are failing it because they are too unspecific (for example, "2005-03").
Can we add specificity by setting them at least to the first day of the month?

According to Relaton model, we expect an ISO 8601 datetime string:

ISO 8601-2 supports "unspecified" digits:

We could use this.

Can we add specificity by setting them at least to the first day of the month?

That will break the model. We have come across Relaton dates that only have year and cannot be further specified.

I believe we need to update the grammar[0] as well? I don’t think the regex allows for “XX” values.
https://github.com/relaton/relaton-models/blob/369b88802ac768524e1463139cc4f49ef3c1e30b/grammars/biblio.rnc#L24

@ronaldtse @strogonoff could we close this issue?

This issue is not fixed. We need to represent the correct date and with the correct specificity. We have to update the schema as appropriate.

I believe we need to update the grammar[0] as well? I don’t think the regex allows for “XX” values.
https://github.com/relaton/relaton-models/blob/369b88802ac768524e1463139cc4f49ef3c1e30b/grammars/biblio.rnc#L24

Yes we should update the grammar.

@ronaldtse I just checked the regex expression in the grammar. It allows unspecific dates https://regex101.com/r/QV1TPS/1

That's great, thanks @andrew2net !

@andrew2net So regex allows for dates like 2005-03-XX? That’s great. Now seems like we only need to change the dates to match ISO 8601

Yes we will need to update the data, and also update the date parsing mechanism.

I expect date parsing in BibXML service (provided by Pydantic) should continue working for unspecific dates with XX instead of day (since this complies with ISO 8601), if not then I can provide/adjust custom validator.

@andrew2net So regex allows for dates like 2005-03-XX? That’s great. Now seems like we only need to change the actual data so that date format matches ISO 8601 4.6.3

@strogonoff the regexp matches only 2005-03. You can see it here https://regex101.com/r/QV1TPS/2
We need to ask @opoudjis to change the grammar.

@andrew2net So regex allows for dates like 2005-03-XX? That’s great. Now seems like we only need to change the actual data so that date format matches ISO 8601 4.6.3

@strogonoff the regexp matches only 2005-03. You can see it here https://regex101.com/r/QV1TPS/2 We need to ask @opoudjis to change the grammar.

The problem is that regexp is wrong, ISO 8601 doesn’t allow 2005-03

Ah! Good question that I had to search for...
ISO 8601-1:2019 supports the "yyyy-mm" pattern:

ISO 8601-2:2019 supports the "unspecified form" of "yyyy-mm-xx" pattern:

Either would be fine.

ISO 8601-1:2019 supports the "yyyy-mm" pattern:

Ah.

I’ll look into the parser on my side and confirm whether a change if needed. If YYYY-MM is allowed by ISO8601, this raises the possibility of parser being non-compliant. 🤦♂

ISO 8601-1:2019 supports the "yyyy-mm" pattern:

I wonder if there is a semantic difference between “this date references given month” or “this date references an unspecific date within given month”. But it sounds more and more like bikeshedding.

Probably beyond the scope to know whether publication procedure took a month or it was some particular day. (I should’ve read the standard more carefully, perhaps I was looking at the wrong version…)

@andrew2net is this still a problem? Are we setting the publication dates properly?

@ronaldtse Relaton stores dates as they are in a source data. It could be "yyyy", "yyyy-mm", or "yyyy-mm-dd". Relaton doensn't add X to unspecific dates. Should it add?

@ronaldtse Relaton stores dates as they are in a source data. It could be "yyyy", "yyyy-mm", or "yyyy-mm-dd". Relaton doensn't add X to unspecific dates. Should it add?

No need to add. If there's nothing left to do here can you please close it? Thanks.


Can't change alpha on CameraPreview

please complete the following information

Device: Xiaomi Mi 9T Pro and Google Pixel 7 Pro
Android version: 12 and 13
Library version: 0.1.0

Describe the bug
I'm not able to make the preview transparent.
To Reproduce
        CameraPreview(
            modifier = Modifier
                .alpha(0.1f),
            cameraState = cameraState,
            camSelector = camSelector,
        ) {
            // Camera Preview UI
        }

Expected behavior
Content should be transparent
Screenshots
Are you willing to provide a PR?

Workaround: Add a box after it and add a transparent background:
CameraPreview(
    modifier = Modifier
        .alpha(0.1f),
    cameraState = cameraState,
    camSelector = camSelector,
) {
    // Camera Preview UI
}
Box(
    modifier = Modifier.background(MaterialTheme.colorScheme.surface.copy(alpha = 0.9f))
)

Hey @Myzel394, thanks for opening the issue!
Oh... that's rather awkward. I'll investigate the cause of this issue and endeavor to respond ASAP.

Hi Myzel, I apologize for the delay!
It appears that there are some limitations on interoperability with the Modifier alpha when using AndroidView with PreviewView. However, as you mentioned, the workaround seems to be effective.
Another approach involves wrapping the content parameter from the CameraPreview composable:
CameraPreview(
    modifier = Modifier
        .alpha(0.1f),
    cameraState = cameraState,
    camSelector = camSelector,
) {
    Box(
        modifier = Modifier.background(MaterialTheme.colorScheme.surface.copy(alpha = 0.9f))
    ) {
         // Camera Preview UI
    }
}

Given this info, I won't fix this since it doesn't seem to be related to the Compose library.  I hope this approach is sufficient for you.
Thanks once again for opening the issue!


FileSchedulerService cannot process new packages

Hi !
I've tried to host a SymbolSource server, and I got some errors in the SymbolSource.Processor.Console application.
The console App crash when a new package is pushed to the server, in the FileSchedulerService, at the Process method throwing an IOException telling that the process cannot access a file already in use.
I understand that the Web Server is renaming a file to signal the Console app that a new package is ready, but sometimes it seems that the events raised by the FileSystemWatcher happens although the file handle is not really closed by the web server.
The incorrect line of code is :
var message = JsonConvert.DeserializeObject<PackageMessage>(File.ReadAllText(file));
This should be replaced by something like that :
using (var fs = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Delete))
{
    using (StreamReader sr = new StreamReader(fs))
    {
        var message = JsonConvert.DeserializeObject<PackageMessage>(sr.ReadToEnd());
    }
}

As I understand, the File.Move is moving the file and the FileSystemWatcher is getting the event before the lock is released.
That's not a problem as long as the file is read with the FileShare.Delete flag.
I tried a simple application that rename a file in a loop a read it or delete it :
FileSystemWatcher fsw = new FileSystemWatcher("./");
fsw.Renamed += (o, e) =>
{
    //using (var fs = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.Delete))
    //{
    //    using (StreamReader sr = new StreamReader(fs))
    //    {
    //        var data = sr.ReadToEnd();
    //    }
    //}
    File.Delete(e.FullPath);
};
fsw.EnableRaisingEvents = true;
while (true)
{
    File.WriteAllText("test", "kikoo");
    File.Move("test", Guid.NewGuid().ToString());
}

It never throw an exception and the files are correctly deleted/read, so I thing the FileShare.Deleted is good enough for reading.
Anyway, I tried to run my sample app on a local driver and on a network share. Runs good in both case. Not sure why jkommeren is getting an issue on the Delete :/

The docs seem to imply that FileShare.Delete works the other way round - it should allow a different process to delete a file that is being accessed by the FileStream (actual removal happens after reading finishes). It should allow reading of a file that is locked or being deleted by a different process.
If you have proof, however, that this does fix things, then I'm not going to argue. Would you care to submit a PR with this change? I would just suggest that you extract that piece of code to a method like string ReadAllTextWithFileShareDelete(string path).


[feature] Pass current user and current visit to api data pipeline

ability to reference current_user and current_visit in API Actions
Addresses: https://github.com/restarone/violet_rails/issues/457
Email action

HTTP Action

example usage

Co-authored-by: Prashant Khadka<EMAIL_ADDRESS>

@alis-khadka This looks good except for broken recaptcha v3 --  I keep getting recaptcha verification failed
:test_tube:  UAT findings

current_user and current_visit can be used in HTTP Action :heavy_check_mark:
current_user and current_visit can be used in Email Action  :hourglass_flowing_sand:
general regression pass :heavy_check_mark:
:red_circle: recaptcha v3 might be broken (recaptcha v2 works)

@donrestarone
I have attached the demo-video in this PR's description.

@alis-khadka since merging dynamic segments we have conflicts-- can you fix them when you get a chance?


Improve API key extraction

The API key is currently extracted once on Kodi startup with a service script.
This can be improved by using a property:
@property
def api_client_id(self):
  if <should_fetch_id_again_because_enough_time_passed>:
    # fetch key
    self._api_client_id = <new id>
    # write to cache

  if not self._api_client_id:
    self._api_client_id = <read from cache>

  return self._api_client_id

Advantages:

Get rid of service
Extraction of API key is more transparent, since it happens directly in the APIv2 class
The refresh interval can be configured

Was implemented in #46.


Changelogs are written to private packages

They only ever include "Patch Changes" with dependency updates, because you cannot currently write changesets for private packages anyway

https://github.com/changesets/changesets/issues/1093


Run Travis tests in 6.x/8.x and latest

By running older and newer versions of Node.js we can be more confident that the Kit will work for more users.
We know lots of Kit users download the latest stable build, while we develop on 8.x currently.
This may slow builds down since it needs to run many more tests, but they should be in parallel, so long as there isn't congestion it shouldn't be too slow.
See https://github.com/nodejs/Release for release information
Closes https://github.com/alphagov/govuk-prototype-kit/issues/542

Looks like argon (4.x) fails due to ES6 usage. Are we OK with that?

Closing this for now, we should focus on getting less flakey builds before increasing the chance of failures.


Add item template function

To further customize item creation it can be useful to have an actual function.
Regardless of whether you'd want to add this functionality the first four commits might be a nice refactoring for the structure of templater.

This is a really nice feature @boidolr! Thank you so much and sorry for taking almost two years before I merged it 😳

great, first time since two years I use it, and have to do directly an update 😂

I believe this broke .values(newValues), see #698

Further to @devgianlu's comment, I believe this regressed https://github.com/javve/list.js/issues/438

+1 ro @adriancb. This isn't gonna get fixed anytime soon :(


Chore/configure dependabot for streampipes-website

Purpose

Configure dependabot properly for streampipes-website

@dominikriemer does this make sense? 🤔
https://github.com/apache/streampipes-website/blob/e1495e3c635c4a970fd502036d76f64ce7a1e4f7/package-lock.json#L9

Regarding the version I would say it makes sense to use the same as for the StreamPipes release, also see https://github.com/apache/streampipes-website/blob/e1495e3c635c4a970fd502036d76f64ce7a1e4f7/package.json


Don't set rpath relative to $ORIGIN on Linux builds

What does this PR do?
It stops patching the rpath in the packaged binaries to use paths relative to $ORIGIN.
Motivation
A number of setups require (or are made easier) by using setcap to for example let the Agent listen on reserved ports (< 1024) without requiring it to run as root. For security, $ORIGIN is not followed by binaries that have been modified permissions wise in this fashion (same with comands such as setuid). This causes a setcap-ed Agent to not find libraries when $ORIGIN is used in the rpath.
Describe how to test/QA your changes
Tested on a Ubuntu VM:
$ datadog-agent version
Agent 7.61.0-devel - Meta: git.238.43b5fca - Commit: 43b5fca25b - Serialization version: v5.0.135 - Go version: go1.22.8
$ datadog-agent
# [ outputs help ]
$ sudo setcap CAP_NET_BIND_SERVICE=+ep /opt/datadog-agent/bin/agent/agent
$ datadog-agent
# [ outputs help normally, before this fix it would complain about not finding rtloader ]

Possible Drawbacks / Trade-offs
This throws a spanner in the works in our plan to make a location-independent Agent, but it's not urgent enough to warrant inconveniencing users and thus can be postponed.
Additional Notes

/merge

/merge


System.MissingMethodException: void SQLite.SQLiteConnection..ctor(string,bool)

Hello!
I'm trying monkeycache  <IP_ADDRESS>-beta in a sample app where I'm using sqlite-net-pcl 1.5.231 too.
Barrel.ApplicationId = "TestAppNSPrism7";
var data = Barrel.Current.Get("test");

on the second line I'm getting a:
System.MissingMethodException: void SQLite.SQLiteConnection..ctor(string,bool)
Any hint on what am I doing wrong?
Thanks!

thanks, so should I wait an upgrade of <IP_ADDRESS>-beta or check myget?

I would rely on the current dependency of 1.4.X
I plan on finishing up a new 1.0-beta tomorrow :)

Works perfectly thanks!


Add replaceRange functionality to RealArray

Make sure that the tests are passing.

@aleph7 This is ready for merger.

@aleph7  I'll have to use CollectionType because it's both a SequenceType and Indexable

@aleph7 Okay, looks like it's ready for merge.


chore: allow to run Mocha tests in parallel

This PR fixes our test suite to allow it to run in parallel with new mocha 8 feature:
npx mocha --parallel
It doesn't switch to this mode by default due to the fact that at the moment nor c8 nor nyc can collect test coverage in this mode.
Basically this PR makes the test server to bind to a random port (0) instead of fixed one and removes the use of mocha root hooks.
One test was refactored into an async function because for some reason it was failing in parallel mode otherwise.
There are no speed improvements right now from running tests in parallel, mostly due to the fact that our current test suite is the monolith of one file, but it will/may improve both with mocha improvements and out tests de-composition, so, it's nice to have:
mocha --reporter=dot
212 passing (2s)

# vs
mocha --parallel --reporter=dot
212 passing (3s)

Partially deprecates/helps for  #832 / #833

Awesome job @tinovyatkin

This only motivation to rewrite in AVA at #833 is parallel test execution. I really appreciate big work @xxczaki is doing there, but I do think that this rewrite will introduce more risks than benefits and parallel tests are already here.

I still think ava is the way to go because of its overarching simplicity and explicitly.

I didn't mean to discuss ava vs mocha here as I have not enough experience to work with ava and these changes is beneficial anyway, I just saying that the only motivation of test runner switch at original issue #832 was the ability to run tests in parallel and it's here.

Regarding simplicity, I reserve my opinion that for an average JavaScript developer mocha will be simpler just because of the fact that it is over 20x more popular on npmjs and so familiar.

20x more popular on npmjs

I'm quoting Sindre here:

Popularity is rarely a good metric for quality. Yes, it’s cool to have a popular project, but the most fun projects I’ve done are not very popular, and I couldn’t care less. Do it for you, not the stars.

The second motivation for migrating to ava was that every other repository in this org already uses it.
I also recommend reading this article which is pretty useful for mocha developers to get their head around ava.

@Richienb I have some recommendations for you too 😄
https://kentcdodds.com/blog/migrating-to-jest

You can't compare test framework needs for timeout-signal with the same for node-fetch. Every other repository in @node-fetch was created on the shoulders of node-fetch popularity and have 1/20 of if tests count (at best). So, it's node-fetch needs should dictate the requirements in this organization.

Your opinion is yours and my opinion is mine.
Since this PR allows the current tests to run 30% faster, I'm approving it.


Issue with paraxial working partially

In a simple double-couple + paraxial element test I have noticed that Paraxial elements are not being applied to all borders. While it is demanded in the arc file paraxial elements be applied to borders left, right, top, and bottom (see code snippet below), for a simple square test domain, with double couple source at the center
    <paraxial-boundary-condition>
      <surface>left</surface>
    </paraxial-boundary-condition>
    <paraxial-boundary-condition>
      <surface>top</surface>
    </paraxial-boundary-condition>
    <paraxial-boundary-condition>
      <surface>right</surface>
    </paraxial-boundary-condition>
    <paraxial-boundary-condition>
      <surface>bottom</surface>
    </paraxial-boundary-condition>

we notice that only top and bottom paraxial elements are working , seen in the animation below


ovn-controller is stuck for a long time, causing the port to be unable to come up.

There are a total of 200 to 300 compute nodes online, with approximately 20,000 to 30,000 ports (mainly distributed across two logical switches). Each compute node has about 200,000 OVS flow entries. After the ovn-controller restarts, it gets stuck for a long time (about 8 minutes). during the flow table processing period, the network on this node is unavailable.
ovn-controller logs are as follows:
2024-07-04T02:36:15.012Z|15050873|inc_proc_eng|INFO|node: logical_flow_output, handler for input SB_logical_flow took 2612ms
2024-07-04T02:44:29.806Z|15050874|inc_proc_eng|INFO|node: physical_flow_output, handler for input SB_port_binding took 494793ms
2024-07-04T02:44:30.174Z|15050875|rconn|WARN|unix:/var/run/openvswitch/br-int.mgmt: connection dropped (Broken pipe)

the pstack output is:
Thread 4 (Thread 0x7f9e321e0700 (LWP 1451710)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb2aefc0 in stopwatch_thread ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 3 (Thread 0x7f9e329e1700 (LWP 1451706)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb297b3a in ovsrcu_postpone_thread ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 2 (Thread 0x7f9e331e2700 (LWP 1451705)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb1db761 in pinctrl_handler ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 1 (Thread 0x7f9e36979a40 (LWP 1451678)):
#0  0x00005614bb2568a7 in mf_format ()
#1  0x00005614bb2656f0 in ofpacts_format ()
#2  0x00005614bb1c9308 in ovn_flow_to_string ()
#3  0x00005614bb1c9888 in ovn_flow_log ()
#4  0x00005614bb1cc6de in ofctrl_add_flow ()
#5  0x00005614bb1ece85 in consider_port_binding ()
#6  0x00005614bb1ee928 in physical_eval_port_binding ()
#7  0x00005614bb1eebb5 in physical_handle_flows_for_lport ()
#8  0x00005614bb1e50cb in pflow_output_sb_port_binding_handler ()
#9  0x00005614bb20da23 in engine_run ()
#10 0x00005614bb1b17e7 in main ()

Thread 4 (Thread 0x7f9e321e0700 (LWP 1451710)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb2aefc0 in stopwatch_thread ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 3 (Thread 0x7f9e329e1700 (LWP 1451706)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb297b3a in ovsrcu_postpone_thread ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 2 (Thread 0x7f9e331e2700 (LWP 1451705)):
#0  0x00007f9e349f5ac1 in poll () from target:/lib64/libc.so.6
#1  0x00005614bb2bacb5 in time_poll ()
#2  0x00005614bb2b009c in poll_block ()
#3  0x00005614bb1db761 in pinctrl_handler ()
#4  0x00005614bb29a293 in ovsthread_wrapper ()
#5  0x00007f9e355611ca in start_thread () from target:/lib64/libpthread.so.0
#6  0x00007f9e348fc8d3 in clone () from target:/lib64/libc.so.6
Thread 1 (Thread 0x7f9e36979a40 (LWP 1451678)):
#0  0x00007f9e348fd2fb in socket () from target:/lib64/libc.so.6
#1  0x00007f9e349faa1b in openlog_internal () from target:/lib64/libc.so.6
#2  0x00007f9e349faf0f in __vsyslog_chk () from target:/lib64/libc.so.6
#3  0x00007f9e349fb1c3 in __syslog_chk () from target:/lib64/libc.so.6
#4  0x00005614bb2c7779 in vlog_valist ()
#5  0x00005614bb2c6e43 in vlog ()
#6  0x00005614bb1c98c8 in ovn_flow_log ()
#7  0x00005614bb1cb4e5 in remove_flows_from_sb_to_flow ()
#8  0x00005614bb1cc9ed in ofctrl_remove_flows ()
#9  0x00005614bb1eeba7 in physical_handle_flows_for_lport ()
#10 0x00005614bb1e50cb in pflow_output_sb_port_binding_handler ()
#11 0x00005614bb20da23 in engine_run ()
#12 0x00005614bb1b17e7 in main ()

It seem that the time is mainly consumed in the pflow_output_sb_port_binding_handler function.
I found that after the ovn-controller restarts, all OVS flow tables are cleared, causing network issues for existing ports. I tried using the external_ids:ovn-ofctrl-wait-before-clear option. Although it can prevent the flow tables from being cleared, during this period, the ports assigned to this node cannot come up, and the mac_binding table cannot be updated properly.
In this scenario, how can it be optimized? Are there any other options I might have missed?

Hi,
this sounds like something that might be addressed by<EMAIL_ADDRESS>If you have the chance would you mind trying this commit if it helps?

Thank you very much for your response. I’m very sorry for the late reply.
I tried the patch, and the effect is very noticeable. After restarting the ovn-controller, the downtime was reduced from 8 minutes to less than 1 minute.


[Infra] Add job that builds ClientApp with source Firestore via SPM

In addition to the source FST builds in firestore.yml, this should add some additional test coverage.
#no-changelog

Re-run CI when https://github.com/grpc/grpc-ios/pull/218 is merged and released. Then, CI should be green and this can merge.

Merging now that CI passes with grpc 1.49.2


Request for Documentation: Fix Set-ADFSRelyingPartyTrust description in SAML doc to refer to correct step

Mattermost user sven.huester from https://community.mattermost.com has requested the following be documented:
We also need to change that line because it refers to a step 7 that is actually a step 4

See the original post here.
This issue was generated from Mattermost using the Doc Up plugin.

Hi @svelle! Could you share the link for this doc change - there are now two ADFS documents, one for 2012 and one for 2016, let me know if both are affected. Thanks! :)

@svelle - Can you please confirm whether the requested product documentation updates are still required?


Significantly improved efficiency of HollowHistoryKeyIndex

This PR optimizes HollowHistoryTypeKeyIndex by eliminating the need for both readStateEngine and writeStateEngine, effectively removing a round trip for each state update. This significantly enhances performance in terms of CPU usage and memory management.
In the newly proposed implementation, HollowHistoryTypeKeyIndex maintains the following associations in Java HashMaps:

Hashed ordinals mapped to assigned ordinals
Hashed fields mapped to assigned ordinals
Assigned ordinals mapping to the corresponding record's field string representations

This approach has a few key advantages:

Efficient Updates: Round trips are completely eliminated for key updates, instead, updates to mappings occur based on the latest typeState.
Immediate Storing and Hashing: Keys are stored and hashed immediately upon update and within the same function, removing the need for added indirection present previously.
Enhanced Querying Performance: Querying is now significantly faster due to the direct lookup in a hashmap instead of going through the readStateEngine.

The modified HollowHistoryTypeKeyIndex is also more concise, with about 300 lines of code as compared to the previous 500 lines, contributing to improved readability.
The performance impact of these changes is considerable. Preliminary profiling indicates that history processing is 3-4 times faster. Additionally, memory usage is likely reduced as it's no longer necessary to store objects in conjunction with their hashes and mappings.
Todos (potentially after initial CR, will discuss with Drew)

[x] Implement support for multi-field key delimiters during queries
[x] Store field values by object, rather than by string
[ ] Fix potential hash collision edge case where records are not equal but hashes are
[x] Implement memoization for fields

[x] Strings: intern() builtin
[x] Other (bytearray, float, etc): implement intern equivalent

Unit/integration tests for potential sources of bugs (or make sure they already exist):

[x] Unit tests for ObjectInternPool
[ ] Record hash collision test? inherently hard to find two records that collide though

Profiler screenshots to illustrate the performance improvements
Note the stack percentages in the bottom left
Old implementation

New implementation

The added memoization functionality (MemoizeObjectPool) works by checking if the same boxed primitive already exists in a HashMap. If so, it returns a reference to the existing one (and the garbage collector will deallocate the additional one).
In effect, this makes it so if many records have the same values (1f, 10d, "United States", etc), it will only have to store one copy of each. If we have 1000 records with equal primary keys, we only need to store the primary key values once.
The downside of this is that the storage of each object involves an additional hash. However, this should be fairly inexpensive and is well worth it for the decreased memory usage.


feat: Copy rows as html table, so we can paste it to spreadsheet

#499
Here is the demo:

Nice job, i really appreciate, thank you!

@all-contributors please add @dyaskur for code.


What is a proper wiring of 23LC1024 SPI SRAM?

Hi,
I did try virtmem in Arduino UNO and NANO and was not able to get it working.
My HW test for the board is here: https://github.com/vitasam/Basic-Nano
What is a proper wiring of 23LC1024 SPI SRAM to use it with  virtmem?

It seems that I got your code working with my board:
*vpi = 42
*vpi = 42
*vpi = 42
*vpi = 42
*vpi = 42


JavaToolInstaller - usage of public libraries to Install

Question, Bug, or Feature?
Type: Feature
Enter Task Name: JavaToolInstaller
Make the support of picking up the JDK archive from public libraries.

Will this feature being taken care of or not? In any case, a comment from the feature owner would be appreciated.

Will this feature being taken care of or not? In any case, a comment from the feature owner would be appreciated.

Six months w/o any human reaction from the dev team... at least a comment indicating any plan or progress would be appreciated. @damccorm - any chance to get feedback on this one?

@anatolybolshakov thoughts on this one? If we want to pick it up, we could do something like Actions and pick up Zulu support - https://github.com/actions/setup-java/blob/main/src/distributions/zulu/installer.ts

@damccorm yes it make sense, thanks 👍  probably we could just do something similar to this action.

If we want to pick it up, we could do something like Actions and pick up Zulu support - https://github.com/actions/setup-java/blob/main/src/distributions/zulu/installer.ts

@damccorm - Danny, this sounds reasonable to me. Just wondering if there is any plan or even progress on this one?

@damccorm - Danny, is there any hope to get this ever realized?

Hi @FranzKohl we are working on more prioritized items at the moment, but going to get back to this one as possible enhancement once be able to

@damccorm & @anatolybolshakov
Yesterday I found that the .NET Tool Installer (in Azure DevOps 2020.1) is even able to download the brand-new .NET 7 SDK... so it looks to me, that the team behind this tool installer did a good job.
In that context I'm really sad to see, that after more than two years there still isn't a similar solution available for the Java Tool Installer. This will be my last note on this thread, I'm giving up, sorry.


--extract-audio and skipping video file download

Hello,
By default --extract-audio erases the video file. That is, the next time using youtube-dl with --extract-audio again grabs the video file to convert it although it already exists as audio file.
Can you think of a way to implement a check procedure that skips to download the video file if the corresponding audio file already exists without actually keeping the corresponding video file?
Regards
Robert

rename the downloaded and converted files to have .m4a extension and youtube-dl will skip them.


remove NSException from firestore client

This is fine as an intermediate state, but really we should be putting ourselves into a failed state and erroring operations that attempt to use persistence.

Which is basically everything except transactions?
Alternatively we could fall back to memory persistence..


Compile a list with the  guardrails rules that limit/enable the workload accounts

Describe the issue
There is not a guardrails list. The closest thing is a bunch of Service Control Policies that can be found at https://github.com/aws-samples/aws-secure-environment-accelerator/tree/main/reference-artifacts/SCPs.
The goal of this ticked it to create such a list
Additional context
The guardrails are a crucial characteristic of the ECF. They are tested and poked in several aspects (CSPM, scripts that detect any change in the rules/policies). The approach for which this ticket is required is to impersonate a user, and then poke the guardrails, to see which actions are allowed and which ones are not permitted. However, before doing this "poke" we need to have the list of actions.
Definition of done
A list (excel, text document) of the actions that are allowed/denied,  probably sorted by role
For example

User can create a S3 bucket
User can not create a new billing report
User can not create a EC2 asset

Done.
Results in spreadsheet guardrailsActionsList.xlsx is saved as part of the repo (https://github.com/bcgov-c/aws-ecf-probing-scp)


Describe how to use publicIPs to access guestbook externally when not us...

...ing external load balancers

Thats true.  but There isn't much need to add it to the raw jsons (because it by nature is specific to a system you're running on).  So i think in this corner-case, its good to deviate from the actual json files.  Its easier to grok this by seeing it embedded in the json file for the example, rather than write a long sentence explaining how it can appended to the json file.

I'd suggest showing it without and how to add it, or adding a line explaining why it differs from the source file.

Fair enough :) modified @lavalamp @roberthbailey

LGTM, thanks.


Check if extensions in README.md are still important

Some extensions in the README.md are not in the environment.yml

jupyterlab-latex
jupyterlab-sidecar
jupyterlab_code_formatter

Some extensions in the environment.yml are not in the README.md

jupyterlab-unfold
jupyterlab-git

Do either of these files need to have updates?

We want to leave the extensions in our list even if they aren't included in our tutorial. That said, we should probably make sure that we distinguish between the two in the README. Maybe we should have a two different headings. One that says something like "Extensions in this repo" and "Other extensions" or something like that.
Sidenote, maybe for "Extensions in this repo" we should also put links to the notebooks that cover them, like this:

jupyterlab-lsp (notebook) (video)


⚠ Processors has degraded performance

In 965c5e7, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 162 ms

Resolved: Processors performance has improved in ab8e2ee.


ruff: 0.0.281 -> 0.0.282

Diff: https://github.com/astral-sh/ruff/compare/v0.0.281...v0.0.282
Changelog: https://github.com/astral-sh/ruff/releases/tag/v0.0.282
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 246590 run on x86_64-linux 1

41 packages built:

baserow
baserow.dist
dmarc-metrics-exporter
dmarc-metrics-exporter.dist
emacsPackages.lsp-bridge
irrd
irrd.dist
paperless-ngx
python310Packages.bentoml
python310Packages.bentoml.dist
python310Packages.dirty-equals
python310Packages.dirty-equals.dist
python310Packages.lektor
python310Packages.lektor.dist
python310Packages.pytest-examples
python310Packages.pytest-examples.dist
python310Packages.python-lsp-ruff
python310Packages.python-lsp-ruff.dist
python310Packages.unstructured-api-tools
python310Packages.unstructured-api-tools.dist
python310Packages.watchfiles
python310Packages.watchfiles.dist
python311Packages.bentoml
python311Packages.bentoml.dist
python311Packages.dirty-equals
python311Packages.dirty-equals.dist
python311Packages.lektor
python311Packages.lektor.dist
python311Packages.pytest-examples
python311Packages.pytest-examples.dist
python311Packages.python-lsp-ruff
python311Packages.python-lsp-ruff.dist
python311Packages.unstructured-api-tools
python311Packages.unstructured-api-tools.dist
python311Packages.watchfiles
python311Packages.watchfiles.dist
ruff
ruff-lsp
ruff-lsp.dist
strictdoc
strictdoc.dist


Add Type for SVG Imports in Typescript

Feature Request
https://github.com/react-native-community/react-native-svg#use-with-svg-files suggests to import svg files.
Why it is needed
For good type suggestions a new type must be added to allow this
Possible implementation
declare module '*.svg' {
  import Svg from 'react-native-svg'

  const content: Svg;
  export default content;
}

Code sample
Please show how the new code could work, if doable

Bumping this.

Bump!!! Please, this is breaking our linter 😄

Any workaround for this?

Hello,
As I see, it was already implemented if you still have any suggestions feel free to open a new issue.
Thank you.


What are the defaults values of "maxConcurrentActivityFunctions" and "maxConcurrentOrchestratorFunctions"?

What are the defaults values of maxConcurrentActivityFunctions and maxConcurrentOrchestratorFunctions?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: afd094d0-fb49-e206-2f99-c3fec27bb20f
Version Independent ID: 42fd1479-2478-3137-b43e-da13a4c10a48
Content: Performance and scale in Durable Functions - Azure
Content Source: articles/azure-functions/durable/durable-functions-perf-and-scale.md
Service: azure-functions
Sub-service: durable
GitHub Login: @cgillum
Microsoft Alias: azfuncdf

@GP4cK
Thanks for your feedback! We will investigate and update as appropriate.

@GP4cK I have got the confirmation from my team that the maximum value for the maxConcurrency parameters (by maxConcurrentActivityFunctions and maxConcurrentOrchestratorFunctions is Int32.MaxValue which is<PHONE_NUMBER>.
If you set the limit that high, in practice that just means that you want “no limit” (because concurrency will never even get near to that number).

@MayankBargali-MSFT thanks but I'm asking for the default values.
If I don't specify maxConcurrentActivityFunctions or maxConcurrentOrchestratorFunctions in my host.json and I use the fan out / fan in pattern with 100 activities for example, how many activities will run concurrently?

@GP4cK The maxConcurrentOrchestratorsDefault is 5 or 10 * Environment process count and maxConcurrentActivitiesDefault  is 10 or 10 * Environment processor count
The max number of activities that run concurrently in a function app is then (number of worker nodes) * (maxConcurrentActivitiesDefault)
So, this number depends on the scale out (because the limit is applied per worker node).
When considering the value to use for maxConcurrentActivitiesDefault, the main consideration is usually the local resources a function requires to run, like memory or CPU. Setting this limit low can help to avoid running out of memory or oversubscribing the CPU (because it prevents running too many of these functions on a single node and instead distributes them over more nodes). However, some activities consume almost no resources to speak of (e.g. an async call to an external service), so for those situations setting it to 100 or higher is no problem and can speed up your app.
We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


data: project addition by

Add a new project to the list
[GSSOC'23] Want to add a Tetris Game made using React.js
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Completed. Thanks for your contributions 🎉 ❤


Allow Function Name Customization for Firebase Build Preset

After successfully deploying Nuxt 3 to Firebase using the Nitro Firebase build preset, I noticed that the Firebase Function name generated through nuxi build is 'server' by default.
Firebase allows its users to host multiple sites and create multiple cloud functions within a Firebase project, however, each function must be uniquely named.
Because of the build output not being customizable, to support multiple Nuxt 3 sites in the same Firebase project, I am forced to add a 'hacky' build step to update the .output/server/index.mjs file (created by Nuxt) to rename the exported function from 'server' to a custom function name.
I have not been able to find a way to customize the function name in the build output, and it seems this is not yet supported by Nitro, and therefore is not exposed in the Nuxt config either.
I would like to request this as a feature, possibly something that could be added to the nitro.config.js file (and ultimately the nuxt.config.js file).
Example:
export default defineNitroConfig({
  preset: 'firebase',
  firebase: {
    serverFunctionName: 'my-custom-function-name'
  }
})

If this is already supported in any way, and I missed it, let me know.
If you have questions or need more input, please let me know.
Thanks, Nitro team,

Found a similar topic here.
https://github.com/nuxt/framework/discussions/2414

Because of this limitation, I have chosen to instead use the advanced node handler preset which seems to feature an unopinionated path to handing off requests to the Nitro server.

Can you share a code sample of your working nuxt and firebase configuration? Specifically a package.json and a nuxt.config.ts? I haven't been able to get the firebase preset working on my machine at all

As a workaround, you can use rollup replace to rename your functions, see https://github.com/unjs/nitro/issues/1369


BandagedBD keeps on giving an error

This is what I get everytime I try installing BetterDiscording using it. I've tried the regular installer and that refuses to work. Whenever I install BetterDiscord with the regular installer, it never actually opens.
Better Discord Server says there is an 'outage'.
Any help?

Redownload the installer, move it to another folder and try running it with or without administrator.
Next time, issues for BandagedBD goes to here

Thanks! Although, slightly irrelevant problem. I'm unable to access BD server for plugins because of an outage. Any help on that?

This gist provides a raw list of plugins


Use unique segment paths for Kafka indexing

#5187 was intended to fix an issue with Kafka indexing service where a task would push segments to deep storage and then fail, and then the subsequent retry task would attempt to push its segment (which contains a different set of offsets from the first task) and publish the metadata, but the push would not happen because the segment data already existed in deep storage. In this scenario, the metadata (most importantly the offset cursors) would not match what's actually in the segment and exactly-once semantics would be violated.
This was fixed by supporting file overwriting in deep storage, but it turns out that there are other cases where overwriting the existing segment is undesired behavior. As an example (which may or may not be happening in practice), there could be a task which pushes its segments and publishes metadata but then fails before it can notify the supervisor that it is done. If the supervisor then creates a retry task to attempt again, the task will eventually fail when the transactional metadata commit fails, but by this point it will have already pushed its version of the segments into deep storage, overwriting the good set of segments there. This will lead to a similar situation as the previous case, where the same segment ID could have different data on historicals and in deep storage, and where Kafka messages can be duplicated or missed.
The cleanest way to handle all these cases is to have each Kafka task write its segments to a separate location in deep storage so that the metadata published definitely matches the segment set and there's no interaction between tasks. If a task pushes and then the metadata publish fails, it will attempt to clean up the orphaned segments, but it is expected that in some unhandled exception cases, the segments may remain in deep storage and may need to be cleaned up manually.

@dclim not sure how someone would read this

the segments may remain in deep storage and may need to be cleaned up manually.

Can you please provide some guidance on how to find out those leaked segments and delete it manually?

@b-slim right now, it would involve some scripting, something like: list out the segment directory and find any partition directories that have more than one child directory, and then compare these directory names to the druid_segments table and delete any directory that isn't referenced in the loadSpec of an entry in that table. You'd probably also want to check timestamps and only process dangling segments from sometime in the past so you don't inadvertently wipe segments that are pushed but not published.

@dclim this seems pretty complicated to me, so imagine user whom just adopted Druid.
IMO we need to fix this by doing this work you have described as a cron task running at supervisor to cleanup stuff after aborted transaction. I don't have to much knowledge about the design/fix here but i think this should be something like task runner that get a transaction ID that we can use to track aborted transactions and clean the leaked segments. I don't think putting the burden on the user is fair.

@b-slim @dclim how about making one change in this patch: kill segments in the "Our segments really do exist, awaiting handoff" path.  And this can already lead to loose segments even today (if a segment is pushed but not published, it stays in deep storage forever). So it'd be just as good as what we have today.

@gianm changes made, thanks for the review guys. Good catch on killing segments at "Our segments really do exist, awaiting handoff", I meant to do that but missed it and only caught the exception case.
@b-slim that seems reasonable to me; I think it makes sense though to wait and see if anyone complains about this; I think the cases that it will leave garbage are actually quite small, and not much higher than the current implementation where tasks overwrite each other's segments.

@dclim can you please add a short section about when the leak can occur and how to find leaked segments? Thanks

@gianm since it is a Kafka indexing issue i think we can added to the new Kafka ingestion page maybe as Misc section or Operation section for me the goal is more to document this i guess even a github issue can be enough if you don't like the idea of section within documentation.

@gianm yeah, as i said it is to help ppls operating the cluster/help me to know where to dig without reading the code base and maybe good to have it as issue that someone can tack if they want to.
Thanks

@gianm @b-slim I took another look through the code, and with the change Gian suggested to remove segments if we discover someone already published it before us, I can't find any real scenario in which garbage would be left behind, other than if there are bugs in Druid's code, such as these sanity checks tripping in AppenderatorImpl.mergeAndPush():
    // Sanity checks
    for (FireHydrant hydrant : sink) {
      if (sink.isWritable()) {
        throw new ISE("WTF?! Expected sink to be no longer writable before mergeAndPush. Segment[%s].", identifier);
      }

      synchronized (hydrant) {
        if (!hydrant.hasSwapped()) {
          throw new ISE("WTF?! Expected sink to be fully persisted before mergeAndPush. Segment[%s].", identifier);
        }
      }
    }

In these cases, if an exception was thrown after some segments had already been pushed, those segments would be orphaned.
But - my point here is that this is not something related at all to the useUniquePath functionality, nor is it something related to Kafka indexing exclusively. This is something that was always in Druid indexing and happens with every type of ingestion, where a sanity check failure due to broken Druid logic that causes the task to end prematurely does not clean up already pushed segments.
After looking at it more closely, I'm leaning towards the opinion that it would actually cause more harm than good to explicitly call it out in documentation since it would scare people unnecessarily. Folks who view this as a major issue should already have been running periodic consistency checks comparing deep storage to the metadata storage.
I think it might be interesting to write a tool, sorta like a 'druid-fsck', that can validate the consistency of the deep storage and clean up orphans, but that would be for another time.

@dclim that makes sense to me, so I'll merge this.
@b-slim I raised https://github.com/druid-io/druid/issues/5716 about a potential loose segment cleanup tool.

Wait, I won't merge this, since there's a checkstyle error showing up in travis. @dclim could you please take a look?
[ERROR] /home/travis/build/druid-io/druid/extensions-core/hdfs-storage/src/main/java/io/druid/storage/hdfs/HdfsDataSegmentPusher.java:54: 'HdfsDataSegmentPusher' has incorrect indentation level 0, expected level should be 4. [Indentation]

Travis is looking good; merging.

thanks @gianm

@dclim I tagged this 0.12.1 since I brought up on the dev list the prospect of including this there, and nobody argued. Does that sound reasonable to you & are you able to do a backport?

@gianm yes that sounds good, will backport


Enable outputs doesn't have any effect

Seems to be working, may have fixed it when fixing another issue


python311Packages.qpsolvers: refactor and add solvers clarabel and cvxopt

Description of changes
update xxxBuildInputs according to https://github.com/qpsolvers/qpsolvers/blob/main/pyproject.toml

quadprog is one of the solvers so needs to be in propagatedBuildInputs not checkInputs
supported solvers clarabel and cvxopt are in nixpkgs so add them

Result of nixpkgs-review pr 286802 run on x86_64-linux 1

2 packages built:

python311Packages.qpsolvers
python311Packages.qpsolvers.dist

Result of nixpkgs-review pr 286802 run on x86_64-darwin 1

2 packages built:

python311Packages.qpsolvers
python311Packages.qpsolvers.dist

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

cc maintainer: @renesat

eh -- closing cause i don't have time for this

I went ahead and made the change in https://github.com/NixOS/nixpkgs/pull/287076.


Better handle missing chromosomes

A collaborator tried to run Hornet with python ./Hornet/mapping/find_intersecting_snps.py -p -C 1 file.bam chr_files  but chr files consisted of only the file chr1.snps.txt.gz. Hornet does not try to remove the chr from names, so this silently led to an error.
In parallel, there has been an error in the calculation of unpaired reads where we were essentially doing:
unpaired1 + unpaired2/total instead of (unpaired 1 + unpaired 2)/total.
Add in the fact that total was all reads, not just all reads on the selected chromosome, and this was very confusing.

Just looked through the code, I can't see any obvious issues and this change definitely makes sense, so I am going to go ahead and merge it now.
Thanks @petercombs!


Goofys systemd file credentials error

Hi guys!
I try to make a systemd service file for my goofys+minio mount, but it doesn't want to succed.
/lib/systemd/system/goofys.service
`[Unit]
Description=Goofys
[Service]
#EnvironmentFile=/etc/default/goofys
ExecStart=/usr/bin/goofys "--endpoint=https://xxxxxxxxxxxxx" "--profile=minio" --file-mode=0644 --dir-mode=0777 -o=allow_other "image-xxxx" "/mnt/minio-xxxx"
[Install]
WantedBy=multi-user.target`
And i received this error:
May 16 09:18:20 ip-172-31-43-251 systemd[1]: Reloading. May 16 09:18:22 ip-172-31-43-251 systemd[1]: Started Goofys. May 16 09:18:23 ip-172-31-43-251 /usr/bin/goofys[594233]: s3.ERROR code=NoCredentialProviders msg=no valid providers in chain. Deprecated.#012#011For verbose messaging see aws.Config.CredentialsChainVerboseErrors, err=<nil>#012 May 16 09:18:23 ip-172-31-43-251 /usr/bin/goofys[594233]: main.ERROR Unable to access 'image-data': NoCredentialProviders: no valid providers in chain. Deprecated.#012#011For verbose messaging see aws.Config.CredentialsChainVerboseErrors May 16 09:18:23 ip-172-31-43-251 /usr/bin/goofys[594233]: main.FATAL Mounting file system: Mount: initialization failed May 16 09:18:24 ip-172-31-43-251 goofys[594224]: 2022/05/16 09:18:24.200261 main.FATAL Unable to mount file system, see syslog for details May 16 09:18:24 ip-172-31-43-251 systemd[1]: goofys.service: Main process exited, code=exited, status=1/FAILURE May 16 09:18:24 ip-172-31-43-251 systemd[1]: goofys.service: Failed with result 'exit-code'.
If i try to run the same exact command by hand, it's works fine.
I use root user.
Have you any idea?

Hi Balla,
My best guess would be that systemd does not populate the HOME environment variable. AWS SDK for Go V1 will look for this environment variable when setting the default credential file location.
https://github.com/aws/aws-sdk-go/blob/e2d6cb448883e4f4fcc5246650f89bde349041ec/aws/credentials/shared_credentials_provider.go#L127-L132
This would explain why it works for you when running by hand under user root.
You can get around this by setting the environment variable AWS_SHARED_CREDENTIALS_FILE to point to your credentials file, which should be stored safely with the correct permissions.


Fix rounding issues in FCCtrlEdit and FCString measurement methods

This PR fixes the measurement rounding issues that are partly causing the problems described in #389.
Here are the changes made:

Rewrote GetMeasurement method from scratch in FCMString to solve the rounding problem. I also emulated the handling of overrides from FCCtrlEdit on Windows, which is more comprehensive than what FCString does (eg handling Picas overrides correctly).
Changed the rounding of minimum in GetMeasurementInteger so that if decimal values are passed, they are handled correctly instead of being discarded (if I want an integer between 12.5 and 20 and I give it 11, I should get 13 in return, not 12).
GetMeasurementEfix now returns whole EFIXes instead of decimals, rounded correctly (which is really what it should've done in the first place, just like GetMeasurementInteger does with EVPUs). My reason for doing this is that if a value like 63.75 is returned and then used on an EFIX field the decimals will be discarded, meaning that the actual value saved will be 63 instead of 64. By returning an already rounded value, the script author doesn't need to think about handling this.
Added *Measurement10000th methods for getting and setting values in 10,000ths of an EVPU, again mainly so that correct rounding can be handled automatically. This means that there are now methods for returning correct values in all 3 measurement types used in Finale (integer EVPU, integer EFIX and 10,000th EVPU)
Modified FCMCtrlEdit so that it uses the new FCMString methods

I haven't touched the Set* methods. I might be able look into it next week, but at the moment finding time at the computer for testing is a bit tight and there's only so much I can write on my phone without being able to test.
Anyway, here is a test script. Since FCMCtrlEdit just calls the FCMString methods internally, the values should be the same but I put it in there just to make sure:
local mixin = require("library.mixin")
local dialog = mixin.FCXCustomLuaWindow()
dialog:CreateStatic(0, 3):SetText("Enter a measurement:"):SetWidth(120)
dialog:CreateEdit(110, 0, "edit")
dialog:CreateMeasurementUnitPopup(190, 0)

local function create_handler(method)
    return function(self)
        local unit = self:GetParent():GetMeasurementUnit()
        local edit = self:GetParent():GetControl("edit")
        local str = mixin.FCMString()
        edit:GetText(str)

        local msg = ""
        if edit[method .. "_"] then
            msg = msg .."FCCtrlEdit: " .. edit[method .. "_"](edit, unit) .. "\n"
        end
        if str[method .. "_"] then
            msg = msg .. "FCString: " .. str[method .. "_"](str, unit) .. "\n"
        end
        if str[method] then
            msg = msg .. "FCMString: " .. str[method](str, unit) .. "\n"
        end
        if edit[method] then
            msg = msg .."FCMCtrlEdit: " .. edit[method](edit, unit) .. "\n"
        end

        finenv.UI():AlertInfo(msg, method .. " Results")
    end
end

dialog:CreateButton(0, 30):SetText("GetMeasurement"):SetWidth(90):AddHandleCommand(create_handler("GetMeasurement"))
dialog:CreateButton(95, 30):SetText("GetMeasurementInteger"):SetWidth(120):AddHandleCommand(create_handler("GetMeasurementInteger"))
dialog:CreateButton(220, 30):SetText("GetMeasurementEfix"):SetWidth(110):AddHandleCommand(create_handler("GetMeasurementEfix"))
dialog:CreateButton(335, 30):SetText("GetMeasurement10000th"):SetWidth(130):AddHandleCommand(create_handler("GetMeasurement10000th"))

dialog:ExecuteModal()

I intentionally made it regular text box, not a MeasurementEdit, so changing the unit doesn't change the display value, it just changes what unit the text box value is treated as when you click the buttons. Play around with it, including with overrides. For a list of overrides, check the code or look here: https://usermanuals.finalemusic.com/Finale2012Win/Content/Finale/OptionsOMeasurement_Units.htm There are hours of fun to be had!
@rpatters1 @cv-on-hub

This looks good to me. I would like to hear from @cv-on-hub before merging it.

LGTM. In fact LVGTM. Amazing to discover after all these years that GetMeasurement10000th is a thing.

You didn't miss anything, in that they actually don't exist in the standard classes. But I thought that they should, so I added them to the mixins.


[PROBLEM] 37.Sudoku Solver

Difficulty
Hard
Problem Description
Write a program to solve a Sudoku puzzle by filling the empty cells.
A sudoku solution must satisfy all of the following rules:
Each of the digits 1-9 must occur exactly once in each row.
Each of the digits 1-9 must occur exactly once in each column.
Each of the digits 1-9 must occur exactly once in each of the 9 3x3 sub-boxes of the grid.
The '.' character indicates empty cells.
Example 1:

Input: board = [["5","3",".",".","7",".",".",".","."],["6",".",".","1","9","5",".",".","."],[".","9","8",".",".",".",".","6","."],["8",".",".",".","6",".",".",".","3"],["4",".",".","8",".","3",".",".","1"],["7",".",".",".","2",".",".",".","6"],[".","6",".",".",".",".","2","8","."],[".",".",".","4","1","9",".",".","5"],[".",".",".",".","8",".",".","7","9"]]
Output: [["5","3","4","6","7","8","9","1","2"],["6","7","2","1","9","5","3","4","8"],["1","9","8","3","4","2","5","6","7"],["8","5","9","7","6","1","4","2","3"],["4","2","6","8","5","3","7","9","1"],["7","1","3","9","2","4","8","5","6"],["9","6","1","5","3","7","2","8","4"],["2","8","7","4","1","9","6","3","5"],["3","4","5","2","8","6","1","7","9"]]
Explanation: The input board is shown above and the only valid solution is shown below:

Constraints:
-board.length == 9
-board[i].length == 9
-board[i][j] is a digit or '.'.
-It is guaranteed  that the input board has only one solution.
Link
https://leetcode.com/problems/sudoku-solver/

Can you please assign this issue to me @godkingjay?


Add website link at the head of README.md

Adding the website link to the head of the README will help new viewers to easily navigate to the website from the repo.
Website link

Its not the website of this repo,
I will try to find a time to create one :)

Oh okay.


simplify builtins diagnostics test

Follow up on
https://github.com/nvimtools/none-ls.nvim/discussions/25

Only one small issue left: please lint the code

addressed and merged


require option

Is it possible to use the original implementation's require option?
e.g,
sass -r sass-globbing

How would you like to use it in the context of node? You might be better off by creating your own package.json file, I think...

What do you mean creating my own package.json? I can't see how I can use sass plugins with node-sass. Is this possible at the moment?

You cannot use sass plugins written in Ruby with node-sass, because we are using C/C++ library (https://github.com/sass/libsass) and Node.js, which runs in JavaScript. So you can pretty much script your own custom functions and custom importers in JavaScript, but you cannot use Ruby.
Please note that this feature is still experimental in the recent node-sass.

Ah..huh. I see. Thanks!


Add Lookback to the User Research section

Adds Lookback (a tool we find quite helpful for remote prototype testing and annotation at GigSalad) to the list of User Research tools. Is there a better section for this? I considered the following sections before landing on User Research:

Design Feedback Tools
Prototyping Tools

This pull request also shuffles the Feedback Lite and GoToMeeting entries in the User Research section to make the section alphabetical.
Checklist

[x] I made something good today 💐.
[x] I put links in the correct format: [Tool](link) — description.
[x] My description starts with the lower-case letter and ends with the full stop.
[x] I put the tool in the alphabetical order.

Optional

[x] I added the appropriate labels: free, open-source or mac.
[x] I put the tool only to one category.
[x] I followed Contribution Guidelines.

@johnbacon thanks for your contribution!


Dead Link on OCF homepage

On this page https://www.opencollective.foundation/
Please redirect this link to go to https://docs.opencollective.foundation/how-it-works/fees

thanks @SudharakaP ! :)

thanks @SudharakaP ! :)

You are welcome. 🤗


Allow plugin to run under Netbox 4.X

There are only a few minor differences between the plugin API in Netbox 3.X and 4.X. This patch should allow the plugin to run under both Netbox 3 and 4.

Do you have the possibility to test your changes and then call the /api/schema endpoint of Netbox? I get an err with the latest version of this plugin, which hinders my Ansible dynamic inventory integration. I get this error:
`<class 'AttributeError'>
'MetricValueTypeOptionsViewSet' object has no attribute 'queryset'`

@Pixelshifter We do not currently use the /api/schema endpoint for anything, so other than that I now get a YAML file that looks somewhat sane, I can't really validate.
Feel free to reach out if my patch doesn't work for you.

Hi everyone,
Thanks a lot for your effort and sorry for the slow replies on my side. Let me take in all the changes, test them with the latest version and do a new release!
Thanks,
Daan

Netbox 4 compatibility will be tracked in daan/netbox4. Initial changes and Netbox 4 migrations can be found here: https://github.com/TheDJVG/netbox-more-metrics/commit/3da31f30ec86a074d439db6a21f1c288b28db26a

Version 0.3.0 is now released with NetBox 4.0 support!


jsonschema: Switch to map based encoding

Provides a public interface for #35, and switches the internal encoding
logic to rely on maps, as propsed in #51. This makes the implementation
simpler and more straight forward.
Compared to the poposed solution 1 in #35 that would rely on passing an
io.Writer to an encoder method, this solution will more easily handle a
corner-case where a custom validator writes zero bytes to the io.Writer.
The reduction in complexity does not come for free though. This looks to
be 4x slower for very tiny schemas, and aproximately 2x slower for small
schemas. As jsonschema is used as API documentation, and can be easily
cached, this performance loss should still be quite acceptable.

@rs, @yanfali. This provides the following public interface for custom validators to implement:
+type Formatter interface {
 +	// JSONSchema should return a map containing JSON Schema Draft 4 properties that can be set based on
 +	// FieldValidator data. Application specific properties can be added as well, but should not conflict with any
 +	// legal JSON Schema keys.
 +	JSONSchema() (map[string]interface{}, error)
  }

This is, based on "Solution 2" from #35, even you preferred solution 1 there @rs. The internal logic is also switched to rely on maps.
As far as I can see, this solution has the following benefits over solution 1:

Simpler
Allows future support for users to encode JSON Schema into other formats than JSON, e.g. YAML as supported by Swagger.
Avoids a corner case for an io.Writer based solution to #35 where no bytes are written by a custom validator. See schema.go#L104

It has the following disadvantages compared to solution 1:

Slower

@rs, @yanfali. This provides the following public interface for custom validators to implement:
+type Formatter interface {
 +	// JSONSchema should return a map containing JSON Schema Draft 4 properties that can be set based on
 +	// FieldValidator data. Application specific properties can be added as well, but should not conflict with any
 +	// legal JSON Schema keys.
 +	JSONSchema() (map[string]interface{}, error)
  }

This is, based on "Solution 2" from #35, even you preferred solution 1 there @rs. The internal logic is also switched to rely on maps.
As far as I can see, this solution has the following benefits over solution 1:

Simpler
Allows future support for users to encode JSON Schema into other formats than JSON, e.g. YAML as supported by Swagger.
Avoids a corner case for an io.Writer based solution to #35 where no bytes are written by a custom validator. See schema.go#L104

It has the following disadvantages compared to solution 1:

Slower

Maybe builder then. Formatter suggests final output
On Sun, Jan 1, 2017, 12:38 Sindre Røkenes Myren<EMAIL_ADDRESS>wrote:

@smyrman commented on this pull request.
In schema/encoding/jsonschema/schema.go
https://github.com/rs/rest-layer/pull/76:

-func (fw *fieldWriter) resetPropertiesCount() {

fw.propertiesCount = 0
-}

for key, field := range fields {

if field.Required {

	required = append(required, key)

}

formatter, err := validatorFormatter(field.Validator)

I call it formatter in line with the ResponseForamtter interface
http://rest-layer.io/#custom-response-formatter-sender from rest-layer.
It's not an encoder anymore, as it doesn't encode the data, only format the
map which will be encoded later.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rs/rest-layer/pull/76, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACK5U7SVMTPyTReFLkNlx_xwkvioSDt1ks5rOA7CgaJpZM4LYt8C
.

Maybe builder then. Formatter suggests final output
On Sun, Jan 1, 2017, 12:38 Sindre Røkenes Myren<EMAIL_ADDRESS>wrote:

@smyrman commented on this pull request.
In schema/encoding/jsonschema/schema.go
https://github.com/rs/rest-layer/pull/76:

-func (fw *fieldWriter) resetPropertiesCount() {

fw.propertiesCount = 0
-}

for key, field := range fields {

if field.Required {

	required = append(required, key)

}

formatter, err := validatorFormatter(field.Validator)

I call it formatter in line with the ResponseForamtter interface
http://rest-layer.io/#custom-response-formatter-sender from rest-layer.
It's not an encoder anymore, as it doesn't encode the data, only format the
map which will be encoded later.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rs/rest-layer/pull/76, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACK5U7SVMTPyTReFLkNlx_xwkvioSDt1ks5rOA7CgaJpZM4LYt8C
.

I find setting null in json tends to be worthless. Much better to not set
the field at all and use undefined in JS or OK pattern in go.
I still think for nil it's much better to set the type for consistency at
the receiver when emitting as it's explicit. Implicit behavior tends to get
people doing different things because of interpretation. I'm arguing set
the type to object for the nil case.
On Sun, Jan 1, 2017, 12:50 Sindre Røkenes Myren<EMAIL_ADDRESS>wrote:

@smyrman commented on this pull request.
 That is what we should encode if the field.Validator is set to
nil. We could change the example, but I think it's the correct think to
write in this situation. If it's set to nil, we can't enforce it to be an
object.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rs/rest-layer/pull/76, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACK5U9QgZIL3c94Ky0kD9YRgOx4GblWWks5rOBGogaJpZM4LYt8C
.

I find setting null in json tends to be worthless. Much better to not set
the field at all and use undefined in JS or OK pattern in go.
I still think for nil it's much better to set the type for consistency at
the receiver when emitting as it's explicit. Implicit behavior tends to get
people doing different things because of interpretation. I'm arguing set
the type to object for the nil case.
On Sun, Jan 1, 2017, 12:50 Sindre Røkenes Myren<EMAIL_ADDRESS>wrote:

@smyrman commented on this pull request.
 That is what we should encode if the field.Validator is set to
nil. We could change the example, but I think it's the correct think to
write in this situation. If it's set to nil, we can't enforce it to be an
object.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rs/rest-layer/pull/76, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACK5U9QgZIL3c94Ky0kD9YRgOx4GblWWks5rOBGogaJpZM4LYt8C
.

I find setting null in json tends to be worthless. Much better to not set
the field at all and use undefined in JS or OK pattern in go.

True, but that is besides the point...

I still think for nil it's much better to set the type for consistency at
the receiver when emitting as it's explicit. Implicit behavior tends to get
people doing different things because of interpretation. I'm arguing set
the type to object for the nil case.

Maybe you misunderstand?
I meen if schema.Field.Validator is not set or equal to nil (in Go), we don't know which type the value may have. The way I read the code in the schema package, it may have any type. So it's pretty simple what the jsonschema package should do.
The behaviour as it stands is identical to what it was before this change.
The only viable alternative would be to raise an error if we don't want to support Validaor set to nil in the rest-layer/schema package, but I personally don't see why we would want that.

I find setting null in json tends to be worthless. Much better to not set
the field at all and use undefined in JS or OK pattern in go.

True, but that is besides the point...

I still think for nil it's much better to set the type for consistency at
the receiver when emitting as it's explicit. Implicit behavior tends to get
people doing different things because of interpretation. I'm arguing set
the type to object for the nil case.

Maybe you misunderstand?
I meen if schema.Field.Validator is not set or equal to nil (in Go), we don't know which type the value may have. The way I read the code in the schema package, it may have any type. So it's pretty simple what the jsonschema package should do.
The behaviour as it stands is identical to what it was before this change.
The only viable alternative would be to raise an error if we don't want to support Validaor set to nil in the rest-layer/schema package, but I personally don't see why we would want that.

On another topic, the last change addresses most of the review comments.
I will do one more commit soon where I remove the size hints. and rerun the benchmarks. I will leave it as one commit for now and rebase later, in case you want to add/runn one more benchmarks @yanfali.

On another topic, the last change addresses most of the review comments.
I will do one more commit soon where I remove the size hints. and rerun the benchmarks. I will leave it as one commit for now and rebase later, in case you want to add/runn one more benchmarks @yanfali.

Removing the size hint appears to be a good call @yanfali:
benchmark                                                    old ns/op     new ns/op     delta
BenchmarkEncoder/Schema=Student                              20355         20169         -0.91%

benchmark                                                    old allocs     new allocs     delta
BenchmarkEncoder/Schema=Student                              136            136            +0.00%

benchmark                                                    old bytes     new bytes     delta
BenchmarkEncoder/Schema=Student                              7033          6746          -4.08%
sindrero@sjokolade ../rest-layer/schema/encoding/jsonschema (git)-[remotes/smyrman/jsonschema-ma

Removing the size hint appears to be a good call @yanfali:
benchmark                                                    old ns/op     new ns/op     delta
BenchmarkEncoder/Schema=Student                              20355         20169         -0.91%

benchmark                                                    old allocs     new allocs     delta
BenchmarkEncoder/Schema=Student                              136            136            +0.00%

benchmark                                                    old bytes     new bytes     delta
BenchmarkEncoder/Schema=Student                              7033          6746          -4.08%
sindrero@sjokolade ../rest-layer/schema/encoding/jsonschema (git)-[remotes/smyrman/jsonschema-ma

The only viable alternative would be to raise an error if we don't want to support Validator set to nil in the rest-layer/schema package, but I personally don't see why we would want that.

I will check what the schema package does today. If it does indeed raise an error, we should do the same in the jsonschema package for now at least.

The only viable alternative would be to raise an error if we don't want to support Validator set to nil in the rest-layer/schema package, but I personally don't see why we would want that.

I will check what the schema package does today. If it does indeed raise an error, we should do the same in the jsonschema package for now at least.

So if a field is defined as nil in schema we are to output a JSONSchema was empty object which is a stand-in for any type? I can sort of understand that interpretation but it doesn't help the consumer of that schema understand that this field member is used for which is effectively ignore this field?
This is the one major weakness of JSON as a specification, lack of comments. If we're going to do that, I would suggest we explicitly put a "description" that describes this behavior for this field. As a consumer of such an API I would be grateful for explicit documentation explaining this rather than leaving me to guess what it means. This wouldn't violate the JSONSchema spec and would be a reasonable compromise. I use JSONSchema as a way to communicate intent to other humans, not just machines. I also feel this better conveys the intent of leaving the Validator nil.

So if a field is defined as nil in schema we are to output a JSONSchema was empty object which is a stand-in for any type? I can sort of understand that interpretation but it doesn't help the consumer of that schema understand that this field member is used for which is effectively ignore this field?

Not necessarily an empty object, just an object without the "type" property. Field properties such as "required", "description" and "readOnly" could still appear in the object, if that's what's added to schema.Field.

This is the one major weakness of JSON as a specification, lack of comments. If we're going to do that, I would suggest we explicitly put a "description" that describes this behavior for this field.

I understand it's confusing, but if we set a default description for some fields and not for others, that becomes an inconsistent and unexpected behaviour. Instead the one specifying the schema should add a fitting description to all of his fields. Hopefully it should be possible to understand that if the "type" property is not set in JSON, it means that any type can be passed in.

I will do one more commit soon where I remove the size hints. and rerun the benchmarks. I will leave it as one commit for now and rebase later, in case you want to add/run one more benchmark @yanfali.

OK, rebased to one commit as I don't see it's going to be beneficial to do any more benchmarking specifically for the sizeHint removal...

Ok,  updated the docs as well.

I like this solution better finaly. GG.
I would rename the method of the Builder interface to BuildJSONSchema. It is better to recall the name of the interface in a single method interface.

Let's merge #78 first, and I will run the benchmarks.

Benchmark results are  here!
To summarize, we see that:

A tiny schema (one field) is 5x slower when the target buffer is pre-allocated, does 3x the number of allocations and allocates more than 8x the amount of memory in total.
A small schema is more than 2x slower, has 2.5x as many allocations, and allocates 5x the amount of memory in total.
A quite complex schema is just below 2x slower, does less than 2x the amount of allocations, and allocates slightly above 2x the amount of bytes.

Details:
benchmark                                                    old ns/op     new ns/op     delta
BenchmarkEncoder/Schema={Fields:{"b":Bool{}}}                1022          5203          +409.10%
BenchmarkEncoder/Schema={Fields:{"s":String{}}}              1005          5168          +414.23%
BenchmarkEncoder/Schema={Fields:{"s":String{MaxLen:42}}}     1285          5831          +353.77%
BenchmarkEncoder/Schema=Simple                               9128          20294         +122.33%
BenchmarkEncoder/Schema=Complex1                             34508         69803         +102.28%
BenchmarkEncoder/Schema=Complex2                             87383         165865        +89.81%

benchmark                                                    old allocs     new allocs     delta
BenchmarkEncoder/Schema={Fields:{"b":Bool{}}}                12             37             +208.33%
BenchmarkEncoder/Schema={Fields:{"s":String{}}}              12             37             +208.33%
BenchmarkEncoder/Schema={Fields:{"s":String{MaxLen:42}}}     14             41             +192.86%
BenchmarkEncoder/Schema=Simple                               55             133            +141.82%
BenchmarkEncoder/Schema=Complex1                             228            430            +88.60%
BenchmarkEncoder/Schema=Complex2                             545            1004           +84.22%

benchmark                                                    old bytes     new bytes     delta
BenchmarkEncoder/Schema={Fields:{"b":Bool{}}}                240           2001          +733.75%
BenchmarkEncoder/Schema={Fields:{"s":String{}}}              224           2001          +793.30%
BenchmarkEncoder/Schema={Fields:{"s":String{MaxLen:42}}}     248           2104          +748.39%
BenchmarkEncoder/Schema=Simple                               1200          6088          +407.33%
BenchmarkEncoder/Schema=Complex1                             6161          19268         +212.74%
BenchmarkEncoder/Schema=Complex2                             14235         44641         +213.60%

Command used to produce benchmarks:
$ go test -run=NONE -bench=. -benchmem -cpu 1

Comparison done using benchmp.

@rs, @yanfali: This is ready for merge if you are happy with it.

@rs, @yanfali: This is ready for merge if you are happy with it.
Hold on -- need to update one docstring after searchreplacing JSONSchema() -> BuildJSONSchema()

OK, ready.

Those numbers are pretty good for the realistic use case. I have no objection to merging.

rebased again due to new commits to master (no changes).
@rs, are you OK with merging this?
Or do you want to consider an alternate implementation?