docs: add badges and update community description

Add dragonfly twitter badges and update community description.
Signed-off-by: Qinqi Qu<EMAIL_ADDRESS>

Codecov Report
Base: 29.59% // Head: 29.51% // Decreases project coverage by -0.09% :warning:

Coverage data is based on head (93e48c6) compared to base (e50f8e5).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #333      +/-   ##
==========================================
- Coverage   29.59%   29.51%   -0.09%
==========================================
  Files          37       37
  Lines        3720     3720
==========================================
- Hits         1101     1098       -3
- Misses       2499     2502       +3
  Partials      120      120

Impacted Files
Coverage Δ

pkg/blob/blob_manager.go
29.59% <0.00%> (-3.07%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[Bug]: linqpad hash check failures

Prerequisites

[X] I have written a descriptive issue title.
[X] I have searched all issues/PRs to ensure it has not already been reported or fixed.
[X] I have verified that I am using the latest version of Scoop and corresponding bucket.

Package Name
linqpad
Expected/Current Behaviour

Sometimes, Scoop fetches the hash of an old version of linqpad.
The url in the manifest points to the latest version of linqpad, causing frequent hash check failures.

Steps to Reproduce
PS> scoop update linqpad
linqpad: 8.0.10 -> 8.0.13
Updating one outdated app:
Updating 'linqpad' (8.0.10 -> 8.0.13)
Downloading new version
LINQPad8.zip (38.2 MB) [==========================================================================================================================================================] 100%
Checking hash of LINQPad8.zip ... ERROR Hash check failed!
App:         extras/linqpad
URL:         https://linqpad.azureedge.net/public/LINQPad8.zip
First bytes: 50 4B 03 04 14 00 00 00
Expected:    47de1fe898996a1ab8b276382836ed837bf7c7204dc24c982325bcd6954748c2
Actual:      f3465ef065b0ec51b766d8a7c05ef652de176166ec28d25d3841f9f658b48e0a

Possible Solution

The url in the LINQPad manifest should be manually updated between versions to https://linqpad.azureedge.net/public/LINQPad8.zip?cache=<IP_ADDRESS>91236 instead of https://linqpad.azureedge.net/public/LINQPad8.zip
Alternatively, autoupdate.url should use https://www.linqpad.net/GetFile.aspx?LINQPad8.zip and Scoop should autogenerate final redirection url as https://linqpad.azureedge.net/public/LINQPad8.zip?cache=<IP_ADDRESS>91236.

Scoop and Buckets Version
PS> scoop --version
Current Scoop version:
v0.3.1 - Released at 2022-11-15

'extras' bucket:
dbb4df8f8 (HEAD -> master, origin/master, origin/HEAD) twitchdownloader: Update to version 1.53.5

'games' bucket:
9c00781f (HEAD -> master, origin/master, origin/HEAD) yuzu-pineapple: Update to version EA-3987

'java' bucket:
54652ba7 (HEAD -> master, origin/master, origin/HEAD) temurin8-nightly-jre: Update to version 8.0.402-2.0.202311201855

'main' bucket:
f229c99c0 (HEAD -> master, origin/master, origin/HEAD) twitchdownloader-cli: Update to version 1.53.5

Scoop Config
PS> scoop config

last_update        : 11/22/2023 17:49:28
virustotal_api_key :
show_manifest      : True
aria2-enabled      : True
cat_style          : default
gh_token           :
scoop_repo         : https://github.com/ScoopInstaller/Scoop
scoop_branch       : master

PowerShell Version
PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.3.9
PSEdition                      Core
GitCommitId                    7.3.9
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

Additional Softwares
No response

Can confirm, had an issue for this: , see #12286
Still is with linqpad 8.0.14


META tag errors when submitting invalid logon parameters via SOAP

Hi Guilherme,
I noticed that when passing invalid logon parameters (wrong password, invalid client number, invalid username) using the soap connection, errors are returned in the form of "The META start tag on line X position XX does not match the end of tag of 'head'.." instead of returning meaningful errors e.g. access denied etc. Is it possible to make the library to return meaningful errors re. invalid parameters passed? I also noticed that the NetWeaver actually returned http access denied error to web client instead of passing a proper xml error response, but I haven't gone further on this though to be 100 pct sure. Many thanks and keep up the good work!
Kind regards,
Faizul

I'll look into it as soon as I can. Thanks for your input.


cloud_logic_array is undefined

I've tried to run this sample twice now but encountering the same error "TypeError cloud_logic_array is undefined" in the console when I try to click the List Restaurants or Insert Restaurants buttons:

Any ideas what I'm missing there?

Here's my project constants:
// AWS Mobile Hub Project Constants
const awsmobile = {
'aws_cognito_identity_pool_id': 'ap-southeast-2:043b399f-8df3-42ce-ada1-7a1d5d9cf6b5',
'aws_cognito_region': 'ap-southeast-2',
'aws_content_delivery': 'enable',
'aws_content_delivery_bucket': 'myproject-hosting-mobilehub-194268341',
'aws_content_delivery_bucket_region': 'ap-southeast-2',
'aws_content_delivery_cloudfront': 'enable',
'aws_content_delivery_cloudfront_domain': 'd2gptdv4406s6q.cloudfront.net',
'aws_dynamodb': 'enable',
'aws_dynamodb_all_tables_region': 'ap-southeast-2',
'aws_dynamodb_table_schemas': [{"tableName":"myproject-mobilehub-194268341-bbq_restaurants","attributes":[{"name":"id","type":"S"}],"indexes":[],"region":"ap-southeast-2","hashKey":"id"},{"tableName":"myproject-mobilehub-194268341-bbq_menu_item","attributes":[{"name":"restaurant_id","type":"S"},{"name":"id","type":"S"}],"indexes":[],"region":"ap-southeast-2","hashKey":"restaurant_id","rangeKey":"id"},{"tableName":"myproject-mobilehub-194268341-bbq_orders","attributes":[{"name":"id","type":"S"}],"indexes":[],"region":"ap-southeast-2","hashKey":"id"}],
'aws_mandatory_sign_in': 'enable',
'aws_project_id': '55b361cd-1aa7-11e8-8ff4-15b9bbb42332',
'aws_project_name': 'my-project-2018-02-26-14-43-15',
'aws_project_region': 'ap-southeast-2',
'aws_resource_name_prefix': 'myproject-mobilehub-194268341',
'aws_sign_in_enabled': 'enable',
'aws_user_pools': 'enable',
'aws_user_pools_id': 'ap-southeast-2_RIBV302w0',
'aws_user_pools_mfa_type': 'ON',
'aws_user_pools_web_client_id': '5joseqbfg5pj2d36epehsvqt39',
}
export default awsmobile;

In the end I worked out I had reached the maximum number of API Gateways I could create in the region. After deleting a couple it worked ok


Review description of  the CategoricalVariation.members property

The description of the CategoricalVariation.members property reads: "A non-exhaustive list of VRS variation Constraints that satisfy the constraints of this categorical variant." . . .  the first 'Constraints' in this definition seems wrong.  See here.
Shouldn't the description read "A non-exhaustive list of VRS variations that satisfy the constraints of this categorical variant" . . . or am I not understanding something?

would it be meaningful to capitalize Variation here as well?

@mbrush Thanks for catching that.  Yes, that's wrong in exactly the way you describe.  I'll fix it.

would it be meaningful to capitalize Variation here as well?

uh....   yes iff gks:core/json/IRI objects are all of the same vrs:Variation class as well, which I thikn is the case.  @ahwagner is that right, that all gkscore:IRI objects are ultimately vrs:Variations?

Yes, please change and create a PR


Add initial version of bia_ingest package

Add bia_ingest package to bia-integrator. This is current state of ingest code that is WIP. This commit has functionality to attach biosample, specimen and image acquisition to images.

My understanding was that the guideline for updating with main was with rebasing instead of merging (unless an exception), this has some of the merged changes from a different branch, which makes the PR harder to understand. Is it intentional?

My understanding was that the guideline for updating with main was with rebasing instead of merging (unless an exception), this has some of the merged changes from a different branch, which makes the PR harder to understand. Is it intentional?

No - it was not intentional. I got the workflow wrong - apologies.

A lot of the data class code feels the same or similar to what bia-integrator has. Is this observation correct? Do we need to address this somewhere?

A lot of the data class code feels the same or similar to what bia-agent has. Is this observation correct? Do we need to address this somewhere?
As a wider comment, all of the the repos are getting more reliant on each other. The repos to me only serve to generate different CLIs. I'd be pro additional merging of repos and making a mega monolithic ?

Yes - we have similar models. @sherwoodf is taking a look at this and future iterations of the code base will attempt to address this. It will be good to discuss the structure of the repos at one of the team meetings.


exp cli never finds my keystore to upload

Hi!
I'm trying a standalone android build but when I select option 2 in the command line and fill in the path to my keystore, I keep getting the message "File does not exist". I tried both relative and absolute path.
My system: Windows 10
node v8.8.1
npm v5.4.2
I appreciate any help.

Hi, can you try with the latest version of exp? There was a small bug introduced by some cleanup code, but I believe the fix has been released now.
If updating exp doesn't fix it for you, please open an issue on https://github.com/expo/expo per the issue template.


The Mixture of Depths router leads to model degeneration

I have been fighting with the Mixture of Depths module for a long time now, and not making significant progress. For some reason, when training with this router, results can be very good, for a very long time, before suddenly - deep into training - output quality starts to collapse. Degeneration typically progresses through 2 stages:

First, the model becomes "fixated" on a specific topic, such as engineering. When this happens, the model will start to produce text containing this word, or engineering-related topics extremely frequently (like, 10 times in a single paragraph).
After this stage, the model basically degenerates to single-token repetition, like "white hair white hair white hair..."

This has been tough to debug, because it happens late in training (after 12 hours, on my GPU). Thus, iteration is slow.
At this point, I suspect that ALiBi is probably the culprit. Learned positional encoding might work better here, or maybe no positional encoding would be best, for now? Putting this out there for anyone who might come along and be able to help.

The NoPE research suggests that attention heads without scaling can sometimes harm generalization, because they can cause the weights to become too diffuse. It makes me wonder if the differential attention might be the source of our router problems; perhaps the noise-cancellation behavior is harming the router in some way. We should test that.

Even after testing with NoPE and RoPE, I'm finding that Mixture of Depths still leads to model degeneration. At this point, I suspect the main culprit is with the shuffling mechanism, and how current routers are tied to each expert, and shuffled in many positions. The degeneration must come from exposing an expert to BOTH full sequences and MoD-reduced sequences; it must be sending the expert a mixed signal, and thereby teaching it that repetition is the fastest way to reduce losses (somehow). I don't fully understand it, but we probably need to adjust router assignment.

This problem was largely addressed by binding routers to experts, such that a single expert only ever sees full sequences, or shortened/routed sequences: never both. After a week of testing, the new strategy does appear to fix the problem.


Entropy Source on CC2538 hardware

Entropy source of mbedtls supports on unix and windows, but no cc2538 hardware.
We need to implement hardware specific mbedtls_hardware_poll for cc2538 example to use mbedtls.

I'd suggest extending the otPlatRandom API to provide an entropy stream, and have mbedtls_hardware_poll link to that in all cases.


TestScheduler subscriptions log - incorrect unsubscription

Bug Report
Current Behavior
Sometimes expectSubscriptions counts unsubscriptions incorrectly. I don't know what is the general rule when the incorrect behaviour occurs, please see the example.
Reproduction
https://stackblitz.com/edit/rxjs-testscheduler-unsubscribe-bug?file=index.ts
const doNotUnsubscribe = (source: Observable<any>) => {
  return new Observable<any>(subscriber => {
    const sourceSub = source.subscribe(subscriber);
    return { unsubscribe: () => { } };
  });
};

testScheduler.run(({ cold, expectSubscriptions }) => {
  const source = cold(' --b----|');
  const unsub = cold('  ----u')
  const expSourceSub = '^------!';
  source.pipe(doNotUnsubscribe, takeUntil(unsub)).subscribe();
  expectSubscriptions(source.subscriptions).toBe(expSourceSub);
});

The test fails: Actual: [{"subscribedFrame":0,"unsubscribedFrame":4}], expected: [{"subscribedFrame":0,"unsubscribedFrame":7}], which means that source is unsubscribed from at unsub, even though this should be blocked by the operator.
Expected behavior
The source is passed through doNotUnsubscribe which should prevent it from ever being unsubscribed, or at least it should prevent it from being unsubscribed from until the source finishes. So I expect that either this test passes, or a test with expSourceSub = '^-'; should pass (i.e. never unsubscribe).
Environment

RxJS version: 6.2.2

Additional context/Screenshots
Implementation of ColdObservable seems to operate with subscriptions in some strange way. I would expect it to call its own subscribe and then return a wrapper around unsubscribing somehow. I don't understand what it actually does, but it looks to me that it's a hack that breaks sometimes.

This actually doesn't have to do with the test scheduler... in the name of efficiency, for RxJS 5 and 6, we bound the Subscriber directly to the Subscriptoin, so that it basically threads one Subscriptoin through the entire chain.
In order to break that chain, you'll need to implement your doNotUnsubscribe such that it doesn't subscribe directly with the subscriber (because then it's detected and threads through).
I'm going to see what I can do to fix this for v7, because I agree it's confusing, if a bit corner-casey.
Here is a demo
const doNotUnsubscribe = () => source => new Observable(subscriber => {
  source.subscribe({
    next(x) { subscriber.next(x); },
    error(err) { subscriber.error(err); },
    complete() {
      subscriber.complete();
    }
  });
});

I hope that helps.


How to close the mysql connection before event loop is closed?

Hello,
I'm using FastAPI with async unit tests. The unit test is passing successfully, but I get every time an ignored exception.
Exception ignored in: <function Connection.__del__ at 0x11023a3a0>
Traceback (most recent call last):
  File "/Users/houmie/.pyenv/versions/v391/lib/python3.9/site-packages/aiomysql/connection.py", line 1072, in __del__
  File "/Users/houmie/.pyenv/versions/v391/lib/python3.9/site-packages/aiomysql/connection.py", line 298, in close
  File "/Users/houmie/.pyenv/versions/3.9.1/lib/python3.9/asyncio/selector_events.py", line 700, in close
  File "/Users/houmie/.pyenv/versions/3.9.1/lib/python3.9/asyncio/base_events.py", line 746, in call_soon
  File "/Users/houmie/.pyenv/versions/3.9.1/lib/python3.9/asyncio/base_events.py", line 510, in _check_closed
RuntimeError: Event loop is closed

Test:
def setup(self):
    metadata.create_all(engine)
    metadata_radius.create_all(engine)

def teardown(self):
    metadata.drop_all(engine)
    metadata_radius.drop_all(engine)
    self.r.flushall()

@pytest.mark.asyncio
    async def test_unconfirmed_account_returns_4007(self):
        async with AsyncClient(app=app, base_url="http://testserver") as ac:
            ac_id = await<EMAIL_ADDRESS>"pass1")
            await crud_device.create_device("u1", DeviceType.IPHONE, ac_id)
            response = await ac.post(
                "/login/",
                headers={},
                json={},
            )
            assert response.status_code == 400
            assert response.json()["detail"] == ErrorType.ACCOUNT_NOT_CONFIRMED

conftest.py
@pytest.yield_fixture(scope="session")
def event_loop(request):
    """Create an instance of the default event loop for each test case."""
    loop = asyncio.get_event_loop_policy().new_event_loop()
    yield loop
    loop.close()

login.py
@router_login.post("/", status_code=200)
async def login(
    request: Request,
    body: Register,
    response: Response,
    email: Optional[str] = Header(...),
    password: Optional[str] = Header(...),
):
    ac = await crud_account.get_account_by_email(email)
    if auth_service.authenticate(ac, password):
        pass
    else:
        response.status_code = status.HTTP_401_UNAUTHORIZED
        return {"detail": ErrorType.NOT_AUTHORISED}

The exception happens when this method is run, but why?
async def get_account_by_email(email: str) -> account:
    query = account.select().where(account.c.email == email)
    if not db.is_connected:
        await db.connect()
    result = await db.fetch_one(query)
    return result

It seems I have to close the connection to database before the loop is closed at the end of the unit test. But how do I do this?
One thing I have noticed is that these methods are only called when using TestClient:
@app.on_event("startup")
async def startup():
    await db.connect()

@app.on_event("shutdown")
async def shutdown():
    await db.disconnect()

But because I'm using AsyncClient from httpx, these are not called. What can I do please?

You should use a library like asgi-lifespan to trigger the events with httpx. There is a working example in the README.

Thank you. I'm confused about the part where he instantiates the whole app in the test fixture. I want to test the real app, I don't want to create a fake one. I don't understand that.
@pytest.fixture
async def app():
    async def startup():
        print("Starting up")

    async def shutdown():
        print("Shutting down")

    app = Starlette(
        routes=[Route("/", home)],
        on_startup=[startup],
        on_shutdown=[shutdown]
    )

    async with LifespanManager(app):
        print("We're in!")
        yield app

It's for the sake of the example ; in your case, app is just your FastAPI app you wrote in your code. You can just import it in your test file 😄

Ha, sorry. I'm new to fast api and asyncio. Please be patient with me.
So I added this to contest.py
from main import app

@pytest.fixture
async def app1():
    async def startup():
        print("Starting up")

    async def shutdown():
        print("Shutting down")

    async with LifespanManager(app):
        print("We're in!")
        yield app

@pytest.fixture
async def client(app1):
    async with AsyncClient(app=app1, base_url="http://testserver") as client:
        print("Client is ready")
        yield client

Then I changed my test to:
    async def test_unconfirmed_account_returns_4007(self, client):
        ac_id = await<EMAIL_ADDRESS>"pass1")
        await crud_device.create_device("u1", DeviceType.IPHONE, ac_id)
        response = await client.post( ...

Test passes, but still throws the same exception.
Only the following messages are shown:  We're in! and Client is ready.
Which means startup and shutdown are not called.

Well, it should work like this. I've made a REPL to show you on a simple example: https://repl.it/@frankie567/fastapi-issue-2819
Maybe you have code in your tests that try to query the database after the shutdown event is passed (the setup and teardown methods you show in your first post look suspicious to me).

Thank you very much for the example. I misunderstood it at first and thought that I had to override/inject the startup/shutdown in the test itself. Now I understand what you mean.
But one issue remains. This is all fine as long as the calls to the app are made through AsyncClient await client.post("/", ... .
But there are cases where I would like to test an async function that accesses the database, which isn't exposed directly to any router. So I would like to test that function and see that it behaves correctly. What do I do there? Due to lack of Startup/Shutdown event in this scenario, I won't be able to disconnect the database cleanly and it throws the same error as mentioned earlier.

What you could do to solve this is to create a yieldable fixture for the database. Something like:
@pytest.fixture
async def test_database():
  await database.connect()
  yield database
  await database.disconnect()

Brilliant. Thank you. I'll close the issue then. It seems that I have it all working now. 👍🏼

Thanks for the help here @frankie567 ! 👏  🙇
Thanks for reporting back and closing the issue @houmie  👍

Sorry for the long delay! 🙈  I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.


ceph: 17.2.5 -> 18.2.0

Description of changes
Changelog: https://docs.ceph.com/en/latest/releases/reef/#v18-2-0-reef
See commit message for details on the build changes made.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

[x] NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This works for me! @krav @johanot @adevress

(Or rather, after rebasing it, it works)

Can this please be merged? :)

If possible, I'd like to get another "seems to be working for my cluster/usage" from another NixOS Ceph user:
CC @adev @ak @johanot @krav and from past PRs @srhb @mweinelt @wkennington

I'm sorry, Things:tm: came up this week and I'm unlikely to make it in time since I need our staging cluster on 17 until at least next week. If you can wait, I am still happy to do the testing on it as soon as I can, but otherwise I won't hold this back.

So far seems to be working fine in our cluster too.

So far seems to be working fine in our cluster too.

I'd merge it on this second report, but Github is currently degrated as per status.github.com, so the merge button is not available.


api: Give quicker feedback if your credentials are wrong

When you use the api and instantiate a Client, it will succeed as long as the credentials are well-formed, even if the credentials aren't actually valid.  Of course, as soon as you try to use them, you'll get a proper error, but it might make sense to have some kind of "ping" method that you can call that literally just makes sure you can connect to the server, so that program can handle that error immediately before starting real work.
More discussion here:
https://chat.zulip.org/#narrow/stream/127-integrations/topic/python.20client/near/813065
Even though this might be a quick fix in terms of the amount of code, I recommend that you work on this only after you've worked with the API a bit and have a feel for the use cases.  A big part of this issue is figuring out the strategy that makes the most sense here.
We'll also want to improve the docs here to reflect any new methods we add or best practices we decide to recommend.

@zulipbot claim

@showell  . Am I on the correct path to interpret the issue Eg: whenever a user try to use an api for some random reason "x"  (let say to create a bot and get it integrated).
After adding the  wrong credentials lets say (incorrect email id / incorrect token) we need a method handler that checks and give the feedback that   the entered credentials are valid or not  immediately and not after the whole process.

@zulipbot claim

@zulipbot claim

@zulipbot I am currently working on this issue.

@zulipbot abandon

@zulipbot claim

I am starting to work on it now. 😄

I think #592 is attempting to address this.

#592 has been abandoned, so somebody should feel free to work on this, ideally after looking at that PR to see how much of it is headed in the right direction.

@zulipbot claim

Just been setting up my new environment in WSL, back at this! 🚀

One thing to clarify--if the API has the wrong credentials, we should probably throw an exception, and not use sys.exit(1) or similar, since it's plausible that the calling code will simply want to handle the exception and move on.
We should take care to provide useful messaging to standard output, however, since 90% of use cases are probably people who are clearly just dealing with some kind of confusing configuration situation.

@zulipbot  claim

@zulipbot claim

please assign this issue to me

@zulipbot claim

@zulipbot claim

@zulipbot  claim

@zulipbot claim

@showell can you add help wanted tag here so I can contribute to this issue , thanks


Tweak skin visibility

On LCE, the Skin Selector presumably only renders up to 7 skins at a time. This means that if a list is less than 7, one or more of the skins will disappear at the end of the cycling animation. For alignment, an even number of skins would hide the right most skin, and an odd number would hide the left most skin.
https://github.com/user-attachments/assets/17d1ae64-aa15-4cd1-9ab0-bc21912609a2
Current behavior:

If a skin is removed from a list, the skin on the side with more skins visible will become the new selected skin


feature: support search in branch name when creating a new pull request

As for the title, when creating a new pull request, we can not search branch names in the branch name select.
Because we have lots of branches, even if we don't have lots of branches it will also be helpful to have this function.

If expand the drop down and just start typing it will filter.

@alexr00 I tried, no. Can you show me a gif or video that proves it is?

Sure, here I typed "alexr00"


Setup travis

Travis files for the swift-buildpack.

[ ]  Research specifying ruby version inside travis.yml file. As a reference, see https://docs.travis-ci.com/user/languages/ruby/.

[ ] Rename DELETE to DELETE_FLAG
[ ] Rename push_and_optionally_delete to push_application
[ ] Rename KITURA_STARTER_TIME to KITURA_STARTER_PUSH_TIME
[ ] Rename SWIFT_HELLOWORLD_TIME to SWIFT_HELLOWORLD_PUSH_TIME

@rolivieri all changes have been made.  Doing final tests now

@rolivieri changes were made to the readme and the travis file for the branches and deploy sections


Install a theme

Looking to install a SharePoint theme into the tenant

I'm going to use this one for AMA Demo - please keep this reserved to myself.

Done #76 Closing.


Update 0.26.2 breaks sense2vec install

Not sure if this should be reported here, but this is the error message:
    building 'sense2vec.vectors' extension
    creating build
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/sense2vec
    gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/ubuntu/miniconda3/envs/industry-mapper/include/python2.7 -I/home/ubuntu/industry-mapper/sense2vec/include -I/home/ubuntu/miniconda3/envs/industry-mapper/include/python2.7 -I/home/ubuntu/miniconda3/envs/industry-mapper/lib/python2.7/site-packages/numpy/core/include -I/home/ubuntu/miniconda3/envs/industry-mapper/lib/python2.7/site-packages/murmurhash/headers -c sense2vec/vectors.cpp -o build/temp.linux-x86_64-2.7/sense2vec/vectors.o -O3 -Wno-unused-function -fno-stack-protector
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
    sense2vec/vectors.cpp:258:36: fatal error: murmurhash/MurmurHash3.h: No such file or directory
     #include "murmurhash/MurmurHash3.h"
                                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

Just encountered this too.
The sense2vec install is currently occurring via the repository, which takes a different path from the regular package installation (as it builds the Cython files). I haven't worked out why this breaks the headers yet.

@henningpeters , could this be caused by zip_safe=True?
I think we're pretty lax about using pkg_resources instead of __file__ to access things, e.g. the header files.

Oh yes, that's very likely.

I hope I fixed the issue. While cleaning up some stuff in murmurhash I broke the header lookup. I also changed the code to use pkg_resources and did some basic tests with eggs. Would be great to get feedback whether it's not working for you.


Using the facet filter in a solution .NET

Hi,
The solution is great. I am trying to install that for my project, but I am know about C# and .CSHTML and I do not know about JavaScript.
Where do you set the fields from the azure search index? For instance,  I need mapping 'AcademicTitle' from my azure search index with the Textbox.
Thanks,
Best Regards

@mau1133 Could you clarify what you're trying to do? It sounds like you want to know how to render document fields on your results page using JavaScript... is that right?

yes, I need to mapping the index fields with the results page using the javascript code that you have shared.
Thanks

@mau1133 the way this works for this project is that the index.cshtml file contains the JavaScript code to show the results of the Azure Search query in HTML (using JavaScript).  You will see in this file how a JQuery Post is made to /home/search whenever a search is requested.  This makes a call to the Search function in the HomeController.cs which actually executes the search and then returns the values to the client in the JavaScript code to be visualized,
Hope that helps.


Explosions now stun and throw mobs

About the PR
Title
Why / Balance
Similar to CM13, explosions should stun mobs in close vicinity, xenos shouldn't be able to dance around nades, same goes for marines. This makes nades a double edged sword where they are now relatively more impactful if used correctly but if you screw up then the ff damage can also be significant.
Technical details
We can maybe make this an optional parameter instead, so that some mobs like queen and crusher can potentially be immune to these effects.
Media
https://github.com/RMC-14/RMC-14/assets/73014819/80a2da72-ac73-47be-9ed6-d9478063e315

[x] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
N/A
Changelog
:cl:

add: Explosions will stun & toss mobs in close vicinity

maybe the overall quantity of nades can be reduced as well, but I mean that probably will happen regardless once more weapons & kit choices are added.

nothing wrong with lots of nades

as discussed in the technical details section I think having a field value for whether or not the mob is affected by the explosion might be a good idea, however, I don't think the actual adjusted force should be a configurable parameter.

I'm going to refactor the nade's effects on mobs entirely.

i'll get back to this later, unless someone beats me to it.


migration should be possible w/out running local dynamodb

The following should be teased apart: migrate should be possible against a target host that is not necessarily started by this plugin. This is currently too highly coupled and does not account for the possibility someone may want to leverage the auto migration feature against a separate local dynamodb instance that is not necessarily managed/started by this plugin.
Config should be more like this:
  serverless-dynamodb:
    stages:
      - local
      - dev
      - qa
      - stage

    migrate:
      enabled: true
      port: 8000
      host: someOtherLocalDynamodb.local

    start:
      bindPort: 8000
      bindAddress: <IP_ADDRESS>
      docker: false

For most use cases, the existing noStart option will work here. This allows users to only do the migration and seeding at a separate dynamodb instance.
However, you do rightly point out users currently can't both start a local instance AND migrate/seed a different instance, using a single command (but this is possible with two commands e.g. by swapping the port and noStart option based on a variable).
I'm concerned supporting this might introduce scope creep to being a generic dynamodb migrator / seeder, rather than a way of running dynamodb locally easily as part of the serverless framework. I was actually thinking of trying to decouple the migration and seeding parts completely (maybe keeping them in the plugin for backwards compatibility).


NE scraper fails

NE -                      -       - FAILED 2020-03-31T20:00:07+02:00  # URLs: https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Pages/Coronavirus.aspx # Errors: Missing date. Missing cases. Date and time is None. Confirmed cases is None. Deaths is None

cc @baryluk

I don't see any numbers any more on https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Pages/Coronavirus.aspx
I send them an email.
Could you look around yourself, maybe I missed some page where this data is available?
If not send them an email too please.

I see some data in https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Documents/Covid-19-Statistiques/COVID19_PublicationInternet.pdf
Scrape possible, but still non trivial.

I looked into pdftotext for this PDF, and I would say, it is essentially impossible to parse reliably, due to nature of the graph, rotated text, invisible line breaks, and other factors (like there is no guarantees of the data order, and correspondence of data and number of cases).

I checked archive.org, it seems as if they just removed the block with the numbers, that was previously in the section titled "Quelle est la situation actuelle ?"

@baryluk I think you should be able to grab the number here with "pdfgrep". See:

If you know what the last number is ;). Just always take the last number in the row. This is actually Text in the file ;).

@zdavatz That is not usable.

@metaodi Yes,
http://web.archive.org/web/20200329210634/https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Pages/Coronavirus.aspx

@zdavatz , @metaodi Send them an email of some form to complain. Thanks.

@jb3-2 Could you nudge them a bit too. Thanks.

@zdavatz , @metaodi Send them an email of some form to complain. Thanks.

Ok, will do so.

scraper didn't pick up on the number of cases from yesterday's data

I did receive response for my email 3 hours ago, claiming it is now fixed, but it is not. The webpage still doesn't have numbers.
I respond, back to check it again.
Maybe they think about the PDF, which is unusable for our purposes. Not sure.
@claudia013 Yes, it is expected scraper is still broken. We can't do anything about it, other than to complain to Neuchâtel Canton people about it.

I will disable the scraper and note that NE is updated manually on the README

Hi, I'm working on a small script to parse that PDF into a csv. Would that be of use? If so, what header should I use for it?

anyone you like, as long as you get the numbers ;)

@maekke97 Yes, if you can make it work that would be really nice. Could you make it available somewhere to test?
It appears there is a table at the end of PDF now (it wasn't there few days ago), that is in reasonable form, and can be scraped.
Data from graphs are much harder to scrape, possible, but not very reliable.
With pdftotext -layout I got this for part of the table:

...
27mars2020                       17                316                36                                            2                   7                45                                          2                     7                   8                       17              1                     12
28mars2020                       21                337                39                                            3                   8                50                                          3                     8                   6                       17              2                     14
29mars2020                        9                346                44                                           10                   6                60                                         10                     6                   0                       16              3                     17
30mars2020                       32                378                45                                            6                   6                57                                          6                     6                   3                       15              2                     19
31mars2020                       24                402                45                                            6                   8                59                                          6                     8                   3                       17              2                     21
1avr
   il2020                        18                420                52                                            6                   8                66                                          6                     8                   4                       18              2                     23
2avr
   il2020                        10                430                49                                            2                  10                61                                          2                    10                   4                       16              2                     25
3avr
   il2020                        14                444                50                                            2                   9                61                                          2                     9                   3                       14                                    25
4avr
   il2020                         5                449                49                                            1                   9                59                                          1                     9                   4                       14              1                     26
5avr
   il2020                        10                459                46                                            4                   8                58                                          4                     8                   5                       17                                    26
6avr
   il2020                                                             49                                            2                   8                59                                          2                     8                   4                       14              1                     27

Which looks decent, but still not trivial to scrape. Probably the best approach is to try to parse it into arrays, and then have checks in the scraper for the other known dates, that columns match properly.

tesseract is also doing very good job:

$ convert -density 300 'COVID19_PublicationInternet.2020-04-06.pdf[4]' covid19_ne_table.png
$ tesseract -l fra --psm 1  covid19_ne_table.png stdout

...
1 mars 2020 1 1
2 mars 2020 0 1 1 1
3 mars 2020 0 1 1 1
4 mars 2020 7 8 4 4
5 mars 2020 1 9 4 4
6 mars 2020 4 13 2 2
7 mars 2020 5 18 2 2
8 mars 2020 6 24 3 3
9 mars 2020 3 27 4 4
10 mars 2020 4 31 3 1 4 1 1
11 mars 2020 6 37 5 1 6 1 1
12 mars 2020 9 46 5 1 6 1 1
13 mars 2020 13 59 8 1 9 1 1
14 mars 2020 9 68 9 1 10 1 1
15 mars 2020 6 74 8 1 9 1 1
16 mars 2020 19 93 11 2 13 2 2
17 mars 2020 21 114 12 4 16 4 4 1 1
18 mars 2020 32 146 15 3 18 3 3 1 2
19 mars 2020 29 175 16 3 19 3 3 1 3
20 mars 2020 13 188 17 4 21 4 2 6 3
21 mars 2020 12 200 15 5 20 5 1 6 1 4
22 mars 2020 16 216 22 6 28 6 1 7 4
23 mars 2020 31 247 22 5 6 33 5 6 0 11 1 5
24 mars 2020 18 265 24 2 6 32 2 6 3 11 1 6
25 mars 2020 15 280 31 3 7 41 3 7 2 12 3 9
26 mars 2020 19 299 33 2 7 42 2 7 1 10 2 11
27 mars 2020 17 316 36 2 7 45 2 7 8 17 1 12
28 mars 2020 21 337 39 3 8 50 3 8 6 17 2 14
29 mars 2020 9 346 44 10 6 60 10 6 0 16 3 17
30 mars 2020 32 378 45 6 6 57 6 6 3 15 2 19
31 mars 2020 24 402 45 6 8 59 6 8 3 17 2 21
1 avril 2020 18 420 52 6 8 66 6 8 4 18 2 23
2 avril 2020 10 430 49 2 10 61 2 10 4 16 2 25
3 avril 2020 14 444 50 2 9 61 2 9 3 14 25
4 avril 2020 5 449 49 1 9 59 1 9 4 14 1 26
5 avril 2020 10 459 46 4 8 58 4 8 5) 17 26
6 avril 2020 49 2 8 59 2 8 4 14 1 27
$

Unfortunately, due to the holes in the table, it is still not reliable.
It is possible to extra more layout info, using tesseract -l fra --psm 1  covid19_ne_table.png stdout -c tessedit_create_hocr=1, and then do some analysis of bounding boxes:

      6
      avril
      2020
      49
      2

but that looks tedious and hard.

I took pdftotext and then went through decreasing separators to catch all the lines. Works for the current pdf, I'm not sure if it is 100% stable...
`
import pdftotext
import re
COLUMNS = [
'date',
'ncases_per_day',
'ncumul_cases',
'n_hospital_non_icu',
'n_hospital_icu',
'n_hospital_icu_non_tube',
'n_hospital_icu_tube',
'n_hospital_total',
'n_icu_covid',
'n_icu_non_tube',
'n_icu_tube',
'n_icu_non_covid',
'n_icu_total',
'n_deceased',
'ncumul_deceased'
]
def pdf_ne_to_csv(input_filename: str, output_filename: str):
with open(input_filename, 'rb') as ifh:
pdf_content = pdftotext.PDF(ifh)
pdf_content_string = '\n'.join(pdf_content)
# magic column fix (don't know if this is stable)
pdf_content_string = pdf_content_string.replace('avr\n   il', 'avril')
# pdf_content_string = pdf_content_string.replace('avr\n il', 'avril')
# find the start of the table on page 5
res = re.search('1mars2020', pdf_content_string)
data_str = pdf_content_string[res.span()[0]:]
data_str = data_str.replace('2020                      ', '2020:')
data_str = data_str.replace('                    ', ':')
data_str = data_str.replace('              ', ':')
data = [[dat.strip() for dat in line.split(':')] for line in data_str.split('\n')]
with open(output_filename, 'w', encoding='utf-8') as ofh:
ofh.write(','.join(COLUMNS) + '\n')
for line in data:
ofh.write(','.join(line) + '\n')`
COVID19_PublicationInternet.txt

@maekke97 Could you edit your comment and use a triple backtick to include the source?
Like this:

```python3
def xyz(a):
  pass
```

@baryluk Thanks for the hint.

$ ./scrape_ne.py  | ./parse_scrape_output.py
NE 2020-04-05T          459      26 OK 2020-04-06T23:01:21+02:00 # Extras: ncumul_hosp=58,ncumul_ICU=17,ncumul_vent=8 # URLs: https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Pages/Coronavirus.aspx, https://www.ne.ch/autorites/DFS/SCSP/medecin-cantonal/maladies-vaccinations/Documents/Covid-19-Statistiques/COVID19_PublicationInternet.pdf

Looks good. This is for yesterday, because a row for yesterday, doesn't have confirmed cases published yet. (it does some other numbers, like deceased reported, +1 compared to yesterday).

Fixed in #470


Creating a no-op circuit breaker policy

We are currently evaluating Failsafe for our usecase and would like to test whether it suits our needs in a non intrusive way. For that we reason a noop approach that would simply log all state transitions according to the configuration would work.
Is anything like that supported out of the box or via some trickery concerning transition thresholds? E.g by not setting a withDelay() value or similar.

You mean you'd like state transitions to occur but for CircuitBreakerOpenException to not be thrown? That's currently not possible.
You can of course use the onOpen, onHalfOpen, onClose handlers to do your logging and for the purposes of testing, wrap your executions in something that ignores any CircuitBreakerOpenExceptions that are thrown.

Hm, yes and no.
We already have logging enabled in the on* methods (which are very handy indeed). We would also like to Failsafe to change state with a given configuration but (essentially) prevent it from staying in the open state and thus preventing modifying behavior as the CircuitBreakerOpenException is never thrown.
As far as I can currently see the only option to achieve that is by omitting withDelay() from the initial configuration. In that case, regardless of whether or not the circuit breaker changes to open it should immediately/on the next request change to half-open and never throw the CircuitBreakerOpenException.
Question is … is that intended, configurable behavior or merely a side effect of something else entirely? Are there any consequences to such a configuration when the application/circuit breaker has to manage a lot of requests?

Question is … is that intended, configurable behavior or merely a side effect of something else entirely?

It's somewhat intended. I say somewhat, because a delay is pretty much always needed to indicate when the circuit is safe to test again. That's part of the premise of the pattern - that the delay gives the system under load some time to recover. Defaulting to no delay and not explicitly requiring one may be something we should change (though that's a separate issue).
I don't think there are any consequences to having a delay or not in terms of servicing lots of requests. It's a simple check to determine if the delay has been exceeded. Each state change does result in a new object being allocated but, that's about it.

As far as I can currently see the only option to achieve that is by omitting withDelay() from the initial configuration. In that case, regardless of whether or not the circuit breaker changes to open it should immediately/on the next request change to half-open and never throw the CircuitBreakerOpenException.

That is incorrect. It only allows a maximum of concurrent executions in half-open, by default 1. It will throw CircuitBreakerOpenException.

Sadly, I am no longer attached to the project in which we originally used this library so I cannot speak for the current state of things but at the time the CircuitBreaker was used in conjunction with Failsafe. I have since also worked in a project where the CircuitBreaker was used in a standalone mode.
Still, I am certain that the use case is will crop up again (if not for me, then for somebody else). So yes, generally speaking I'd still be interested in that feature albeit without an actual project attached right now.

Will leave this open for now, but am interested to see if anyone else has this and can describe their use case.


Import base layer images

If a project creates a base layer image and wants to directly import the base then this ensure the baseline file are in place.  It is a follow up to #1637 and enables containerd with ctr image import image.tar of a base layer.
@gabriel-samfira @helsaawy @TBBle
I didn't find any tests that I could run but I am happy to add something if I get a pointer in the right direction.

If I understand correctly, this is to make arbitrary tarballs importable as base layers? My initial question is: does this introduce any risks or issues if multiple imports of the same tarball actually end up with different (meta)data inside Files/ (i.e. user-visible differences in creation datestamps etc).
If support for this feature is desired, we might need a stronger ensureBaseLayer that ensures that any created files have metadata based on some immutable information in the layer, or just constant.
Also, I can't think of any issue this can cause, but this does mean that the baseLayerWriter is now always writing into a pre-created directory tree rather than creating everything as it goes during layer writing. I suspect the existing code for handling directory timestamps will fix up any divergence in the Windows/system32/config's resulting timestamps, but perhaps this should be called in baseLayerWriter's cleanup instead, so that it's effectively a no-op for the existing base layer-formatted layers, where the requisite files are already in the tarball.

I didn't find any tests that I could run but I am happy to add something if I get a pointer in the right direction.

I skipped adding tests in hcsshim, as this is heavily used in containerd, and we run the containerd integration tests as part of the hcsshim CI pipeline. Sadly, the containerd branch using this is not yet merged.

If a project creates a base layer image and wants to directly import the base then this ensure the baseline file are in place. It is a follow up to #1637 and enables containerd with ctr image import image.tar of a base layer.

To be honest, I can't think of any negative implications of being able to import and mount any arbitrary layer. They won't actually be usable to start a container on Windows, but they will be mountable at least. I see no harm in this tbh, but I am hardly an authority on this :smile:

So definitely an important use-case, and given existing HCS layer-mounting requirements, I can't see any better approach than running something like ensureBaseLayer during import. (Unless we don't use the Windows Snapshotter on Windows for WASI container images, but that smells worse to me)

I considered a different snapshotter as well but this would be quite a bit of additional overhead.  Maybe longer term it could be the right approach (and now that @gabriel-samfira has enabled Bind Filters in Windows we might be able to use the native snapshoter with some minor tweaks? I only look briefly and this is a bit out of my comfort zone)

They won't actually be usable to start a container on Windows, but they will be mountable at least. I see no harm in this tbh, but I am hardly an authority on this 😄

The other use case for this is HostProcess Containers for images like https://github.com/microsoft/windows-host-process-containers-base-image.  I user could generate an image for this locally, import it and then run it.

Edit again: Actually, perhaps WASI images can't be cross-platform like I thought in my earlier edit, since AFAIR the WCOW layer format includes the top-level Files/ directory (and potentially UtilityVM/Files), so they're never going to be compatible if HCS is involved. (Which brings back the containerd-level "is this the right snapshotter for WASI?" question)

With this change the image generated at https://github.com/containerd/runwasi/blob/main/crates/wasi-demo-app/build.rs doesn't require any changes to run on windows (since this adds those meta data files).  It does add some additional files that won't be used (like the vhd files).  so the image is technically cross platform if you squint.  Again maybe using a custom snapshotter would make sense longer term but this also opens the possibility to do this with HPC.

Also, I can't think of any issue this can cause, but this does mean that the baseLayerWriter is now always writing into a pre-created directory tree rather than creating everything as it goes during layer writing. I suspect the existing code for handling directory timestamps will fix up any divergence in the Windows/system32/config's resulting timestamps, but perhaps this should be called in baseLayerWriter's cleanup instead, so that it's effectively a no-op for the existing base layer-formatted layers, where the requisite files are already in the tarball.

I am not sure I understand all of this, but could try putting it during the cleanup?  It needs to be called before ImportLayer or it errors because those files are not present.

CI / integration-tests (windows-2022) (pull_request) failure doesn't look related to this change

The other use case for this is HostProcess Containers for images like https://github.com/microsoft/windows-host-process-containers-base-image. I user could generate an image for this locally, import it and then run it.

This is actually great. We can finally have FROM scratch with host process containers and not have to pull in a large image just to distribute a binary.

CI / integration-tests (windows-2022) (pull_request) failure doesn't look related to this change

For peace of mind, you could try out the snapshotter tests in https://github.com/containerd/containerd/pull/8043. Just replace hcsshim with your branch in go.mod and run:
sh.exe -c "EXTRA_TESTFLAGS=-test.run=TestSnapshotterClient make integration"

This is semi-related: https://github.com/opencontainers/runtime-spec/issues/1185

Also, I can't think of any issue this can cause, but this does mean that the baseLayerWriter is now always writing into a pre-created directory tree rather than creating everything as it goes during layer writing. I suspect the existing code for handling directory timestamps will fix up any divergence in the Windows/system32/config's resulting timestamps, but perhaps this should be called in baseLayerWriter's cleanup instead, so that it's effectively a no-op for the existing base layer-formatted layers, where the requisite files are already in the tarball.

I am not sure I understand all of this, but could try putting it during the cleanup? It needs to be called before ImportLayer or it errors because those files are not present.

Yeah, I'm thinking maybe put it before https://github.com/microsoft/hcsshim/blob/4e0656f62e6c27a54e65243c092210460e77c6cb/internal/wclayer/baselayerwriter.go#L161, so that the "fixup directory times" step is run after ensureBaseLayer.

sh.exe -c "EXTRA_TESTFLAGS=-test.run=TestSnapshotterClient make integration"

I am getting an error even when not using my PR.
time="2023-03-17T16:08:22.821364900-07:00" level=info msg="containerd successfully booted in 0.033075s"
time="2023-03-17T16:08:22.932857100-07:00" level=debug msg="garbage collected" d=3.5324ms
time="2023-03-17T16:08:23.278667900-07:00" level=debug msg="event published" ns=testing topic=/namespaces/update type=containerd.events.NamespaceUpdate
time="2023-03-17T16:08:23.592158800-07:00" level=debug msg="(*service).Write started" expected="sha256:3bb312dcc36fa551766a13cf2fb8e2be90726774589c96bea9198a26307bf2a2" ref="index-sha256:3bb312dcc36fa551766a13cf2fb8e2be90726774589c96bea9198a26307bf2a2" total=350

Haven't dug into it, might be related to my dev env.

I'm somewhat curious, is "tar up a filesystem and ctr image import it" a common workflow? Until this PR, I thought ctr image import only accepted Docker and OCI image/image list tarballs.

The image https://github.com/microsoft/windows-host-process-containers-base-image isn't in an oci format, but it probably should be.
As for the common, making your own super small compact image for HPC should be something that could be enabled.  As an example, provide an oci formatted tar package that has only your binary you want and I would expect directly importing to work.

I'm somewhat curious, is "tar up a filesystem and ctr image import it" a common workflow? Until this PR, I thought ctr image import only accepted Docker and OCI image/image list tarballs.

The image https://github.com/microsoft/windows-host-process-containers-base-image isn't in an oci format, but it probably should be.

Skimming https://github.com/microsoft/windows-host-process-containers-base-image/blob/main/New-HostProcessBaseImage.ps1, that does appear to be creating a Docker-formatted image tarball (i.e. with /manifest.json) with the requisite hive files in its rootfs. So it should not be affected by this PR, nor would anything using it as a base image.

As for the common, making your own super small compact image for HPC should be something that could be enabled. As an example, provide an oci formatted tar package that has only your binary you want and I would expect directly importing to work.

Yeah, I think I had a faulty assumption earlier, I thought this was for handling arbitrary tarballs (just a raw rootfs), not an OCI or Docker image tarball with a handrolled rootfs tarball (contrasting with a rootfs tarball prepared by, e.g. wclayer makebaselayer and wclayer export).
The use case I understand we're talking about now is similar to #750, although since that'd be solve in this repo, it'd be able to use wclayer to get a base layer in the currently-supported format.
I think the rest of my concerns (about being able to repeatedly import a given tarball and produce a 100% identical result including file-creation/modification metadata and ensuring this is a no-op when the root tarball does contain all of the necessary files) are applicable though.

Contrasting this to the existing approach (including those files in the exported base layer), this approach provides simpler base-layer hand-assembly, including not relying on access to empty hive templates or a hive-creation library. (Modulo that zero-byte hives failed to be processed last time I tested it, but perhaps that's changed again, since New-HostProcessBaseImage.ps1 does this and (I assume) works on Server 2022.)
However, base layers created without the required hives will only be importable with hcsshim-using systems that have taken a version of hcsshim after this pull is completed, or they'll get failures referencing hcsshim::ProcessBaseLayer coming from docker pull or ctr image pull.
Hence, I'd be concerned to start seeing such images pushed with the windows platform tag. If there's an different platform tag for WASI images, then that would be fine as there's no non-preview system that can pull WASI-platform images on Windows now, so we can ensure this feature is landed and made available to containerd before containerd starts formally supporting WASI on Windows.
So, I wouldn't use this feature to implement FROM scratch for Windows images; I'd still use hcsshim to process and export the base layer to ensure maximum compatibility, or for non-Windows image creation, use empty hive templates or a cross-platform hive-editing library to prepare a fully-compatible base layer.

Are you saying it would be preferable to generate the empty hive files like the HPC example does vs generating them on import? This won't work since there are checks to make sure those paths exist, which the don't even with the dummy files.
I think the HPC is a good example of a use case that could be updated to be a valid OCI tar format with a tool like  https://github.com/containerd/runwasi/tree/main/crates/oci-tar-builder and should be importable without having to use hcsshim to create base layers.
Or is there another way to something similiar (point at a folder with an executable and generate a base image that can be shared via tarball)?
I will have to dig deeper on your concerns about the reproducibility of imports.

Are you saying it would be preferable to generate the empty hive files like the HPC example does vs generating them on import? This won't work since there are checks to make sure those paths exist, which the don't even with the dummy files.

I don't understand what you mean by not working here. If the dummy files exist, then a check for them by path should pass.
Maybe there's some confusion on my part; I have assumed the HPC base image generated by that repo is importable into containerd today, as importing of layers hasn't been changed by the work I've been involved with.
But I haven't tried it myself, so if it's failing, knowing that (and how it fails) would be informative.
But yeah, my general preference is to produce base layers in the existing format, and avoid things in the import process that produce a Files/ directory that differ from the distributed layer tarball. (Hence my suggestion that when given a current-format base layer, this PR's effect should result in no changes to file data or metadata.)

I think the HPC is a good example of a use case that could be updated to be a valid OCI tar format with a tool like https://github.com/containerd/runwasi/tree/main/crates/oci-tar-builder and should be importable without having to use hcsshim to create base layers.

What's invalid about the HPC base-image's format now? It's in Docker format, rather than OCI format, but it looked valid as a base layer from reading the script that produces it. (I didn't run it up side-by-side with the Docker image manifest spec though.)

Or is there another way to something similiar (point at a folder with an executable and generate a base image that can be shared via tarball)?

I don't think there's any complete pipeline that can do that, apart from the containerd snapshotter integration tests, which (once the PR lands) will be using the existing makeBaseLayer functionality in hcsshim to generate the necessary hive files and other things to be mountable.
Now that hcsshim-side stuff has landed, I was considering putting a script together (like the HPC repo) using the wclayer tool here to fulfill #750, but I'm still a fair way from having time to put into building and testing that right now.

What's invalid about the HPC base-image's format now? It's in Docker format, rather than OCI format, but it looked valid as a base layer from reading the script that produces it. (I didn't run it up side-by-side with the Docker image manifest spec though.)

nothing was invalid but it wasn't importing.  I figured out why though, it doesn't have an annotations so the image wasn't identified.  If I run ctr image import --index-name=hpc .\windows-host-process-containers-base-image.tar it does import.

I don't understand what you mean by not working here. If the dummy files exist, then a check for them by path should pass.

I get the following when I put dummy files in the oci image. Now that I've figured out why the HPC image wasn't loading this does seem to work for window I'm looking deeper into why.
error="failed to extract layer sha256:cbd83a58d0840f98b9458176b4887b3413f06c61dfea6bd0277bce7d9ccbb3ba:
 Failed to safefile.OpenRelative failed in Win32: open
\\\\?\\C:\\ProgramData\\containerd\\root\\io.containerd.snapshotter.v1.windows\\snapshots\\1590\\Files\\Windows\\System32\\config\\DEFAULT: The system cannot find the path specified. (0x3) Files\\Windows\\System32\\config\\DEFAULT: unknown" key="extract-705276600-qBGq sha256:cbd83a58d0840f98b9458176b4887b3413f06c61dfea6bd0277bce7d9ccbb3ba"

Are you building the tarball in code? My best guess for your error is that entries for each directory in the chain Files\Windows\System32\config have not yet been processed so the directory for this tar entry has not been created.

For reference, the bare-minimum base-layer image looks like this:
> tar tvf t.tar
d---------  0 0      0           0 Apr 01 10:31 Files
d---------  0 0      0           0 Apr 01 10:31 Files/Windows
d---------  0 0      0           0 Apr 01 10:31 Files/Windows/System32
d---------  0 0      0           0 Apr 01 10:31 Files/Windows/System32/config
----------  0 0      0        8192 Apr 01 10:31 Files/Windows/System32/config/DEFAULT
----------  0 0      0        8192 Apr 01 10:31 Files/Windows/System32/config/SAM
----------  0 0      0        8192 Apr 01 10:31 Files/Windows/System32/config/SECURITY
----------  0 0      0        8192 Apr 01 10:31 Files/Windows/System32/config/SOFTWARE
----------  0 0      0        8192 Apr 01 10:31 Files/Windows/System32/config/SYSTEM

That's created with
mkdir t\Files
wclayer makebaselayer t
wclayer export t -o t.tar
# Test that it imports again
wclayer import -i t.tar t2
# Mount-test
wclayer create -l t2 s2
wclayer mount -l t2 s2 m2
# Clean up
wclayer unmount s2 m2

I say "bare mimimum", but those are real (emtpy) hives. I tried with empty files, and it worked, but in the past mounting has failed so this might vary by OS build.
This is an empty-file equivalent to wclayer makebaselayer t3:
mkdir t3\Files\Windows\System32\config
New-Item t3\Files\Windows\System32\config\DEFAULT
New-Item t3\Files\Windows\System32\config\SAM
New-Item t3\Files\Windows\System32\config\SECURITY
New-Item t3\Files\Windows\System32\config\SOFTWARE
New-Item t3\Files\Windows\System32\config\SYSTEM

Are you building the tarball in code? My best guess for your error is that entries for each directory in the chain Files\Windows\System32\config have not yet been processed so the directory for this tar entry has not been created.

This was it. I was generating the file via code and the tar file didn't have the appropriate header files for the directories (TIL tar -t).  I've added those entries explicitly and it is working.  This works for me now and is equivalent to the HPC version now.
Thanks for helping me work through the issue. I will close this for now given the fact that this doesn't reproduce repeatable images as you pointed out.


CPU profiling to identify potential performance improvements

Also any suggestions for tools? There's xdebug + webgrind... any other favorite workflows?

https://blackfire.io/, what else ;)
Blackfire is the PHP profiler I develop with my team. It does extended raw data gathering, callgraph based visualizations, and continuous integration for automated performance testing (thus could help #8 also).
I'd be happy to help start with it and provide a premium or enterprise plan if needed.

@nicolas-grekas haven't used it before, so that would be amazing! I have a hunch there are some very simple improvements we could make in some core areas like parseList that would have a pretty significant impact on perf, so it would be awesome to identify those hot spots.
I would also be interested in learning more about their perf test infrastructure and what the variance is like - their docs don't seem to make any references to basic statistical techniques, so I'm pretty skeptical about relying on it for anything beyond getting a general sense for a trend over time.
Also re: memory see #13 - it would be great to understand what Node types we could optimize for the greatest perf boost (e.g. we probably want to minimize empty properties or sparse arrays on common node kinds)

https://github.com/adsr/phpspy is another useful tool - it allows generating flame graphs of the time spent in each function and has low overhead.
Making sure all constant accesses and function invocations are fully qualified would be a small boost to performance - The major issues such as #283 should have already been identified, though. https://github.com/phan/phan/blob/master/.phan/plugins/NotFullyQualifiedUsagePlugin.php can be used to do that.


Automatically sync all kubeConfigPaths

Signed-off-by: Sebastian Malton<EMAIL_ADDRESS>fixes #3221

It works for different kubeconfig paths 🎉 , but migration is started only if app version number is moved to 5.0.1. Probably tests are failing for the same reason.

@aleksfront mentioned that this branch is causing app crash on startup: https://github.com/lensapp/lens/issues/3242
Please lets make sure this is fixed before merging the PR. @Nokel81


feat: use centralized constants

Since the atc-dataplatform use configuration variables as name, partitioning, format, etc. These string values are used very often. In order to ensure that it is always the same string values that are used across the atc we could introduce them as constants.
Following PEP8 Constants:
https://peps.python.org/pep-0008/#constants

Maybe we should use Enums instead. Something like this as an example for table_property:
from enum import Enum
class TABLE_PROPERTY_NAME(Enum): PATH = "path" NAME = "name"
def table_property( self, table_id: str, property_name: TABLE_PROPERTY_NAME, default_value: str = None ): property_value = self.get_all_details().get( f"{table_id}_{property_name.value}", default_value )
table_property(property_name=TABLE_PROPERTY_NAME.PATH)

Good point. I will try using StrEnum.


[WIP] Turn on jemalloc, turn off custom allocators by default

tl;dr: 20% faster, 20% less memory :)
Not 100% sure the CI configs are correct yet.
CUSTOM_ALLOCATORS=ON, JEMALLOC=OFF, TCMALLOC=OFF:
	compilation 4.34s
	decl post-processing 797ms
	elaboration 69.4s
	elaboration: tactic compilation 12.6s
	elaboration: tactic execution 29.3s
	parsing 13.9s
	saving .olean 1.09s
	type checking 520ms
...
	User time (seconds): 125.32
        Maximum resident set size (kbytes): 505388

CUSTOM_ALLOCATORS=ON, JEMALLOC=ON:
	compilation 3.69s
	decl post-processing 703ms
	elaboration 52.7s
	elaboration: tactic compilation 9.01s
	elaboration: tactic execution 21.7s
	parsing 10.6s
	saving .olean 772ms
	type checking 423ms
...
	User time (seconds): 94.83
        Maximum resident set size (kbytes): 465244

CUSTOM_ALLOCATORS=OFF, JEMALLOC=ON:
	compilation 3.81s
	decl post-processing 654ms
	elaboration 53.2s
	elaboration: tactic compilation 8.74s
	elaboration: tactic execution 22.3s
	parsing 11.1s
	saving .olean 803ms
	type checking 439ms
...
	User time (seconds): 96.11
	Maximum resident set size (kbytes): 402700

CUSTOM_ALLOCATORS=OFF, TCMALLOC=ON:
	compilation 4.32s
	decl post-processing 747ms
	elaboration 64.1s
	elaboration: tactic compilation 11.3s
	elaboration: tactic execution 27.5s
	parsing 12.8s
	saving .olean 1.02s
	type checking 492ms
...
	User time (seconds): 113.37
	Maximum resident set size (kbytes): 354648

So, some observations:

Linux: The release build fails when trying to statically link both glibc and jemalloc. This is a known issue with jemalloc that was fixed in a later version. However, we could also just stop linking glibc statically (which I believe is regarded as a bad idea anyway, as seen with our libpthread troubles).
OS X: The debug build got faster, but the release build is timing out??
Mingw: No change - presumably because the mingw-w64-jemalloc package has all functions prefixed with je_, so we're still using the default malloc/free.
MSVC: There is a vcpkg jemalloc package, but building it fails.

I can change the behavior of the STATIC flag, but it would be great if others could investigate the other platforms (and measure whether it's an actually beneficial change there).

Fixed the vcpkg jemalloc package (on vcpkg's master now) -- it looks like a patchfile got corrupted by some editor trimming a trailing space :(

@ras0219-msft Thanks, building jemalloc is working now! It looks like jemalloc.h is missing a reference to their strings.h compatibility header (which sounds like an upstream bug), but we can live without it.

However, we could also just stop linking glibc statically (which I believe is regarded as a bad idea anyway, as seen with our libpthread troubles).

@leodemoura What do you think about changing STATIC? If you agree, I will open a new PR to enable jemalloc by default just on Linux.

@leodemoura What do you think about changing STATIC? If you agree, I will open a new PR to enable jemalloc by default just on Linux.

@kha I think we can remove the STATIC flag. We originally added it because we wanted to be able to create a Linux binary that works in any distribution. For example, it would not depend on a particular version of glibc. I think this kind of feature is only relevant for closed source systems. For open source project like Lean, we can tell users to build Lean on their own non-standard Linux distributions if there isn't an official binary release for it.
Moreover, in Lean4, we will depend on LLVM, and it seems crazy to statically link it.

BTW, I will try this PR today on OSX and MSys2.

I think we can remove the STATIC flag

Fine with me.
Thanks for looking into it! Btw, do you think it makes sense to keep using Msys2 over MSVC for the release builds? I don't know which build is faster right now, though that may also depend on which of the two we get jemalloc working for.

Btw, do you think it makes sense to keep using Msys2 over MSVC for the release builds?

I think so. At least for me, Msys2 is the preferred way of building Lean on Windows.

Here is some information for the PR.
I managed to build Lean on OSX using
AppleClang <IP_ADDRESS>00039 and jemalloc 5.0.1
I installed jemalloc using brew install jemalloc
It worked with the following two configurations:

cmake -D JEMALLOC=ON -D CUSTOM_ALLOCATORS=OFF -G Ninja ../../src
cmake -D JEMALLOC=ON -G Ninja ../../src
Time to build core lib with 1) configuration is 41 secs. The default configuration takes 50.1 secs.


Remove explicit STATIC in add_library call so dynamic libraries can be built

Should default to STATIC, and if shared libraries are desired, use the  -DBUILD_SHARED_LIBS=ON cmake option

Sounds good, I'm re-running the CI as it failed on a pre-install step (sometimes happens for no reason for macOS builds..).
Will merge as soon as it turns green.


Add a targetSearchConcurrency parameter to LogMergePolicy.

This adds the same targetSearchConcurrency parameter to LogMergePolicy that #13430 is adding to TieredMergePolicy. The implementation is simpler here since LogMergePolicy is constrained to only merging adjacent segments.
From simulating merging on an index that gets appended 555 equal-size segments (like nightly benchmarks) and a targetSearchConcurrency of 8, I get the following results:

19.1 segments in the index on average, vs. 11.1 with a targetSearchConcurrency of 1.
Maximum number of segments in the index of 28, vs. 22 with a targetSearchConcurrency of 1.
Write amplification through merging of 3.6 instead of 2.9. For comparison, 3.6 is about the write amplification that you get with a merge factor of 7 otherwise.
The resulting index has 24 segments, including 11 segments on the highest tier, vs. 15 segments and 5 segments on the highest tier with a targetSearchConcurrency of 1.

Thanks for looking @stefanvodita! I added a CHANGES entry and improved the logic to not apply a threshold on the doc count for merges below the min merge size. I will merge soon.

improved the logic to not apply a threshold on the doc count for merges below the min merge size

Woops I need to revert this for now. This made sense for TieredMergePolicy but this causes some test failures with LogMergePolicy in the corner case when minMergeSize maps to a bigger merge than maxMergeDocs.


[Bug]: codeunit 5748 "Transfer Whse. Post Shipment" ModifyHeader

Describe the issue
codeunit 5748 "Transfer Whse. Post Shipment"
Integrationevents OnInitSourceDocumentHeaderOnBeforeTransferHeaderUpdatePostingDate and OnInitSourceDocumentHeaderOnBeforeTransHeaderModify where boolean "ModifyHeader" should by ByVar.
[IntegrationEvent(false, false)]
local procedure OnInitSourceDocumentHeaderOnBeforeTransferHeaderUpdatePostingDate(var TransferHeader: Record "Transfer Header"; var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; var WarehouseShipmentLine: Record "Warehouse Shipment Line"; ModifyHeader: Boolean; IsHandled: Boolean)
begin
end;
[IntegrationEvent(false, false)]
local procedure OnInitSourceDocumentHeaderOnBeforeTransHeaderModify(var TransferHeader: Record "Transfer Header"; var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; ModifyHeader: Boolean)
begin
end;

Expected behavior
[IntegrationEvent(false, false)]
local procedure OnInitSourceDocumentHeaderOnBeforeTransferHeaderUpdatePostingDate(var TransferHeader: Record "Transfer Header"; var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; var WarehouseShipmentLine: Record "Warehouse Shipment Line"; var ModifyHeader: Boolean; IsHandled: Boolean)
begin
end;

[IntegrationEvent(false, false)]
local procedure OnInitSourceDocumentHeaderOnBeforeTransHeaderModify(var TransferHeader: Record "Transfer Header"; var WarehouseShipmentHeader: Record "Warehouse Shipment Header"; var ModifyHeader: Boolean)
begin
end;

Steps to reproduce
Create transfer order --> create whse shipment --> post whse Shipment
Additional context
No response
I will provide a fix for a bug

[ ] I will provide a fix for a bug
Internal work item: AB#558501

OK

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 27498.


[Bug]: Inconsistencies in Applying Display Settings for Enabled, Motion, and Light Buttons

What happened?
Even after configuring the settings to hide buttons associated with enabled, motion, or light plugins, the buttons stubbornly remain visible in Apple HomeKit.
Device Type
Doorbell (Supported)
Plugin Version
v2.3.x (Best Effort)
HomeBridge Version
v1.7.x (Supported)
NodeJS Version
v20 (Supported)
Operating System
Ubuntu 22.04+ (Supported)
Relevant log output
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:162]        Jardin Enabled config: true
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:165]        Jardin has a enabled, so append switchEnabled characteristic to it.
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:146]  Jardin REGISTER CHARACTERISTIC Switch / On / DEVICE_Jardin_Enabled
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:152]  Jardin REGISTER CHARACTERISTIC (00000049-0000-1000-8000-0026BB765291) / (00000025-0000-1000-8000-0026BB765291)
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:162]        Jardin Motion config: true
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:165]        Jardin has a motionDetection, so append switchMotion characteristic to it.
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:146]  Jardin REGISTER CHARACTERISTIC Switch / On / DEVICE_Jardin_Motion
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:152]  Jardin REGISTER CHARACTERISTIC (00000049-0000-1000-8000-0026BB765291) / (00000025-0000-1000-8000-0026BB765291)
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:162]        Jardin Light config: true
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [CameraAccessory.ts:165]        Jardin has a light, so append lightbulbLight characteristic to it.
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:146]  Jardin REGISTER CHARACTERISTIC Lightbulb / On / DEVICE_Jardin_Light
[04/16/2024 13:35:04]   [EufySecurity-3.0.0]    DEBUG   [BaseAccessory.ts:152]  Jardin REGISTER CHARACTERISTIC (00000043-0000-1000-8000-0026BB765291) / (00000025-0000-1000-8000-0026BB765291)

config
{
  platform: 'EufySecurity',
  username: '[***]',
  password: '[***]',
  deviceName: '[***]',
  enableDetailedLogging: true,
  omitLogFiles: false,
  CameraMaxLivestreamDuration: 30,
  pollingIntervalMinutes: 10,
  hkHome: 1,
  hkAway: 0,
  hkNight: 3,
  hkOff: 63,
  ignoreStations: '[***]',
  ignoreDevices: '[***]',
  country: 'FR',
  stations: [],
  cameras: [
    {
      serialNumber: '[***]',
      rtsp: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      snapshotHandlingMethod: 2
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      snapshotHandlingMethod: 2
    },
    {
      serialNumber: '[***]',
      rtsp: false
    },
    {
      serialNumber: '[***]',
      rtsp: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      enableCamera: true,
      videoConfig: {
        maxWidth: 640,
        maxHeight: 480,
        maxFPS: 15,
        useSeparateProcesses: true
      },
      snapshotHandlingMethod: 2,
      talkback: true
    },
    {
      serialNumber: '[***]',
      rtsp: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      talkback: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      talkback: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      talkback: false
    },
    {
      serialNumber: '[***]',
      rtsp: false,
      snapshotHandlingMethod: 2,
      enable: false,
      motion: false,
      light: false
    }
  ],
  cleanCache: true,
  ignoreMultipleDevicesWarning: false,
  autoSyncStation: false,
  nodejs_security: false
}

+1 on experiencing this issue.
Let me know if you need my current config or anything else to help w/ troubleshooting.

+1 Same issue for me as well. I don’t know if I’ve ever gotten it to work. I can also supply config or log. On Apr 16, 2024, at 07:58, Ben Towne @.***> wrote:
+1 on experiencing this issue.
Let me know if you need my current config or anything else to help w/ troubleshooting.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

I am also experiencing this issue.

just release 3.0.0-0 prerelease which should fix it


support callback style queries

Issue #, if available:
#34
Description of changes:
Add support for callback style queries.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

looking forward to this. Thanks for taking the time to look into it

@jafl
Sorry for the delayed response. Thank you for digging into #34 and submitting a solution! Does this have any impact when using promises or when using pg 6.x?

@chrisradek - this PR shouldn't affect the promises API - that was already merged in a previous PR.  I haven't tested pg 6.x.  If you're using it, try it and let me know :)

Hi everyone, thanks for the continuous efforts put towards this repo.
Would it be possible to integrate this change?


Add pytorch-3dunet dependency

I created a separate PR just for the dependency, because there is an issue with the grpcio and protobuff.
So, we know for the new protobuff > 5, we need grpico >= 1.44. The tensorboard dependency of the pytorch-3dunet that can work with protobuff > 5 is the 2.18. For our system, the latest version that conda fetches is the 2.11.
I have attempted to find a solution by forcing the latest tensorboard version trying a dry run install:
$ conda install tensorboard=2.18 --dry-run

Could not solve for environment specs
The following packages are incompatible
├─ grpcio 1.44**  is installable with the potential options
│  ├─ grpcio [1.44.0|1.49.1|...|1.67.1] would require
│  │  └─ python >=3.10,<3.11.0a0 , which can be installed;
│  ├─ grpcio [1.44.0|1.49.1|...|1.62.2] would require
│  │  └─ python >=3.8,<3.9.0a0 , which can be installed;
│  ├─ grpcio 1.44.0, which can be installed;
│  └─ grpcio 1.44.0 would require
│     └─ python >=3.7,<3.8.0a0 , which can be installed;
├─ pin-1 is not installable because it requires
│  └─ python 3.9.* , which conflicts with any installable versions previously reported;
└─ tensorboard 2.18**  is installable and it requires
   └─ grpcio >=1.48.2  with the potential options
      ├─ grpcio [1.44.0|1.49.1|...|1.67.1], which can be installed (as previously explained);
      ├─ grpcio [1.49.1|1.50.1|...|1.67.1] would require
      │  └─ python >=3.11,<3.12.0a0 , which can be installed;
      ├─ grpcio [1.44.0|1.49.1|...|1.62.2], which can be installed (as previously explained);
      ├─ grpcio [1.49.1|1.50.1|...|1.67.1] conflicts with any installable versions previously reported;
      ├─ grpcio [1.57.0|1.58.1] would require
      │  └─ python >=3.12.0rc3,<3.13.0a0  with the potential options
      │     ├─ python [3.12.0|3.12.1|...|3.12.8], which can be installed;
      │     └─ python 3.12.0rc3 would require
      │        └─ _python_rc, which does not exist (perhaps a missing channel);
      ├─ grpcio [1.57.0|1.57.1|...|1.67.1] would require
      │  └─ python >=3.12,<3.13.0a0 , which can be installed;
      └─ grpcio [1.67.0|1.67.1] would require
         └─ python >=3.13,<3.14.0a0 , which can be installed.

So, we need to either bump python 3.9, or restrict the protobuff. I am not sure how to proceed with this one. For now, I just use the environment variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python to make it work.

Unfortunately there isn't grpcio=1.48.2 for python 3.9
$ conda search grpcio

grpcio                        1.48.1  py39hfaff5cf_1  conda-forge
grpcio                        1.49.1 py310h5b20944_1  conda-forge


Add safe navigation operator before concat in organisation method

Fixes https://rollbar.com/dfe/teacher-vacancies/items/3754/

Review app deployed to https://teaching-vacancies-review-pr-3149.london.cloudapps.digital


The type initializer for 'GraphQL.Net.DynamicTypeBuilder' threw an exception.”

I'm having the exact same issue 😢

that's so sad

Anyone find out the cause of the error?

For everyone experience this issue, what version of .NET are you running? I expect this is due to API changes in dynamic type creation on .NET core.

I have the same issue and run on .net core. Is this library even done for .net core? Visual Studio tells me, that it is incompatible with my current .net core project.

If you're using .NET Core it will always throw this error, we should consider targeting .NET Standard as mentioned in #34

I allready tried to compile it against .net core, but i have too few experience in this and I failed. I would love to see a .net core version of this and would like to help, if there is something i could do.

Hi guys, i just could workaround this issue by doing this change https://github.com/jvitor83/graphql-net/commit/26a5f6d267c62a468d1fd4daeb822c20dae1f7be#diff-8fa9aa5aed6d6219daacaa099553fe78R16
Don't know if it is enough to a PR fix, but i could make it work with asp.net core.


[INLONG-1677][doc] add the correct URL for architecture image

Fixes #1677
Motivation
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
Modifications
Describe the modifications you've done.
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
If a feature is not applicable for documentation, explain why?
If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

Codecov Report

Merging #1678 (7f86163) into master (b3566e1) will increase coverage by 0.02%.
The diff coverage is n/a.

:exclamation: Current head 7f86163 differs from pull request most recent head 094a91e. Consider uploading reports for the commit 094a91e to get more accurate results

@@             Coverage Diff              @@
##             master    #1678      +/-   ##
============================================
+ Coverage     12.20%   12.22%   +0.02%
  Complexity     1050     1050
============================================
  Files           392      392
  Lines         32755    32755
  Branches       5159     5159
============================================
+ Hits           3997     4004       +7
+ Misses        27993    27987       -6
+ Partials        765      764       -1

Impacted Files
Coverage Δ

.../java/org/apache/flume/sink/tubemq/TubemqSink.java
55.42% <0.00%> (+4.00%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3566e1...094a91e. Read the comment docs.


Build NativeAOT and mono

Prerequisites for adding NativeAOT and mono support (https://github.com/compiler-explorer/compiler-explorer/issues/5668).
/cc: @mattgodbolt @partouf
/cc: @DaZombieKiller for mono jit inspector

@mattgodbolt Can you please take a look?

Sorry for slowness; thanks @partouf for picking this up


[SPARK-9725] [SQL] fix serialization of UTF8String across different JVM

The BYTE_ARRAY_OFFSET could be different in JVM with different configurations (for example, different heap size, 24 if heap > 32G, otherwise 16), so offset of UTF8String is not portable, we should handler that during serialization.

Merged build triggered.

Merged build started.

Merged build triggered.

Merged build started.

Merged build finished. Test FAILed.

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #40904 has started for   PR 8210 at commit 4ba1ac0.

Merged build triggered.

Merged build started.

Test build #40913 has started for   PR 8210 at commit e677bc3.

Test build #40904 has finished for   PR 8210 at commit 4ba1ac0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public final class UTF8String implements Comparable<UTF8String>, Externalizable

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40904/
Test FAILed.

Merged build finished. Test FAILed.

Test build #1611 has started for   PR 8210 at commit e677bc3.

Test build #40913 timed out     for PR 8210 at commit e677bc3     after a configured wait of 175m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40913/
Test FAILed.

Merged build finished. Test FAILed.

Test build #1613 has started for   PR 8210 at commit e677bc3.

Test build #1611 timed out     for PR 8210 at commit e677bc3     after a configured wait of 175m.

Test build #1613 timed out     for PR 8210 at commit e677bc3     after a configured wait of 175m.


Add a Gitter chat badge to README.rst

Netflix/lemur now has a Chat Room on Gitter
@kevgliss has just created a chat room. You can visit it here: https://gitter.im/Netflix/lemur.
This pull-request adds this badge to your README.rst:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

Already added


fqdn in destination address field, is wrongly treated as an ip range.

Describe the bug
Hi guys, im trying to establish a base code configuration with a bunch of rules.
Basically, i may have to use adresses with IP Netmask and FQDN's types (and URL Categories when i have to deal with some wildcards) in the security policies.
Basically the whole terraform side "seems" ok because i can easily apply the objects, and rule groups configuration to the target device groups on panorama, and the panorama commit is successful.
The problem is when i push de configuration to the devices (which here, are the Cloud NGFW appliances on Azure).
Below is the state description of an example rule which fails :
     {
        "action" = "allow"
        "applications" = toset([
          "any",
        ])
        "audit_comment" = ""
        "categories" = toset([
          "any",
        ])
        "data_filtering" = ""
        "description" = <<-EOT
        allow runner pods to access gitlab
        Source configuration file : /DG_NGFW/security_policies/post-rulebase/201.cloudplatform/r_cloudplatform_azfw.yml
        EOT
        "destination_addresses" = toset([
          "cdn.artifacts.gitlab-static.net",
          "gitlab.com",
        ])
        "destination_devices" = toset([
          "any",
        ])
        "destination_zones" = toset([
          "Public",
        ])
        "disable_server_response_inspection" = false
        "disabled" = false
        "file_blocking" = ""
        "group" = ""
        "group_tag" = ""
        "hip_profiles" = toset([])
        "icmp_unreachable" = false
        "log_end" = true
        "log_setting" = ""
        "log_start" = true
        "name" = "aks-runners-gitlab"
        "negate_destination" = false
        "negate_source" = false
        "negate_target" = false
        "schedule" = ""
        "services" = toset([
          "tcp-443",
        ])
        "source_addresses" = toset([
          "<IP_ADDRESS>/23",
          "<IP_ADDRESS>/22",
        ])
        "source_devices" = toset([
          "any",
        ])
        "source_users" = toset([
          "any",
        ])
        "source_zones" = toset([
          "Private",
        ])
        "spyware" = ""
        "tags" = tolist([
          "azure-firewall",
          "tf-managed",
        ])
        "target" = toset([])
        "type" = "universal"
        "url_filtering" = ""
        "uuid" = "6780e6f3-170d-458f-b256-31d24d51dc42"
        "virus" = ""
        "vulnerability" = ""
        "wildfire_analysis" = ""
      },

When i push the whole configuration to the 3 appliances, i get this error :

All of this despite the configuration being apparently ok :

Expected behavior
Well, that should be pushed without any errors :blush:
Current behavior
Apply/Panorama Commit ok / Push to device fails
Possible solution
No idea
Steps to reproduce
Create an address with type fqdn (as im typing, the name = fqdn, i will try with an alt name).
Create a rule using the fqdn object
Push to Cloud NGFW Device
Screenshots
Above
Context
This is part of a migration from a managed solution to a hub & spoke architecture on Azure using the palo managed service

We can resolve this when checking this option in panorama settings :
"Share Unused Address and Service Objects with Devices"
Which is obviously a problem regarding the limits difference between the managed devices on this panorama environment (hardware appliances versus cloud ngfw).
Created a case on the CSP.


JIRA Server should required or warn about missing protocol

Right now if we do not specify the protocol of the JIRA Server url it will fail to resolve the server, building an url like https://github.com/<user>/<repo>/compare/<JIRA Server>/browse/SR-1810?expand=1
It could use a default https or at least warn the user that a protocol is required

I agree here. There seems to be some quirks with this extension and I think this would make things slightly easier.

I have added a pull request for this which adds validation for the JIRA Server field 🎉


Disable felte handlers on custom components

I have a custom component that wraps a textbox. On change, the value is transformed, then it should update felte's data store.
Because felte automatically binds the textbox's events/value, there is no way for me to transform this value.
Is there a way to disable felte's event handlers on a certain element and manually wire up setField?
Edit: a data attribute to prevent this would be awesome!

I am guessing transforming the value inside of onSubmit does not work for your use case?
Adding a data attribute such as data-felte-ignore sounds good to me, in fact I'm surprised I have not added it yet 😅
What's your specific use case?

Right. The issue is that my validation expects a boolean for a field, but I'm using a select box so felte is storing its value as a string. I'm looking to convert the value into a boolean before it hits the store so my validation succeeds.
data-felte-ignore sounds like a great solution! Thank you so much!

Just added the attribute for<EMAIL_ADDRESS>and<EMAIL_ADDRESS>It should work if you add it either to the input itself or a parent container.
If this solves your issue feel free to close this!

Works beautifully, I really appreciate it!

does this attribute works for svelte as well ? i am using felte verison 1.2.7 but unable to get it to work


xychart.addSeries hanging with larger data sets

I'm attempting to about 3 series, each with about 500k points.  The addseries method appears to be hanging.  Anything I can do to make it faster? Any known limits?

Figured it out.  The utils class uses a for loop,  however I provided a linked list which made it very slow.  Consider using and iterator in the future

would iterator make it faster?

absolutely
On Thu, Sep 6, 2018 at 6:54 AM Tim Molter<EMAIL_ADDRESS>wrote:

would iterator make it faster?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/knowm/XChart/issues/280#issuecomment-419051621, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB4kGRyL8j98FhLxR7DwQXUGUQ5eSZLNks5uYP7ZgaJpZM4WSnjd
.

The test found to be very slow
List.size() = 100000 takes 12 seconds, 5 million needs to wait a long time.
Test code:
List<Integer> list = new LinkedList<>();
for (int i = 0; i < 100000; i++) {
	list.add(i);
}

long  t1 = System.currentTimeMillis();
Utils.getDoubleArrayFromNumberList(list);
long  t2 = System.currentTimeMillis();
System.out.println(t2 - t1);

LinkedList.get(index) is starting from 0 to index. It really needs to be changed to Iterator. Iterator has a lot of speed improvement, traversing 5 million data, and it takes less than 100 milliseconds.


Using helm for k8s install

You could do a lot of what your scripts are accomplishing by using helm charts. Users would simply call
helm repo add seldom HTTPS://github.com/seldonio
helm up
helm fetch seldonio/server
helm install server

Anywyas just a suggestio. Cool project!

Thanks for the suggestion. We'll check helm out.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
/lifecycle stale


Story: Requirements for Function orchestration

Context
HL7 message processing follows the same general flow...

Receive/Debatch
Redact
Structure Validate (with NIST IGAMT)
Normalize to a Data Product

The application code for (1), (2) and (3) should be identical for all HL7v2 messages.
For the pedants:

(2) identical application code; redact config changes
(3) identical application code; IGAMT input varies

However, the normalize to data product will differ depending on the purpose of the HL7v2 message.
For example Case Notification messages vs Laboratory messages
Business requirements

We should be able to orchestrate functions into pipelines, such that the pipelines can i) change and ii) reuse Functions

Solutioning: Orchestration
Current state: Orchestrated from EventHub queues.
Potential options for "config-based" orchestration...

Azure Logic Apps: https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-overview
Durable Functions: https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp