Optimize collision component load time

Optimizes application load time, when there are many disabled entities in a scene with a collision component.
Before (200 disabled entities):

After:

Tested with See More and a local physics test bed.
I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

Will this lead to postponed CPU-intensive operations on enabling?
Basically, does this eliminates unnecessary work in general, or it shifts it from before enabled to after enabled?

Both. If the entity is never enabled, then it eliminates the unnecessary work. If it does get enabled at some point, then it processes when that happens. If it is enabled from the beginning, then no changes.

Both. If the entity is never enabled, then it eliminates the unnecessary work. If it does get enabled at some point, then it processes when that happens. If it is enabled from the beginning, then no changes.

This leads to a problem: CPU stalls when entities are enabled, similar to particles case, which enforces developers to enable and disable entities on load, to ensure they are fully initialized without further CPU stalls.

Yes, you care correct. The stall does take place when the entity gets processed. We can't really avoid it, as CPU has to generate the collision shape, unless we move it to worker, but the main thread would still be affected.
However, think of a scene with many deep branches, where only one branch is active at a time. At the moment, the stall will be quite significant on game start, specially on mobile devices. Currently a developer, have no way to mitigate it, except accepting it, because the engine desided to process the whole scene in one go. This PR at least gives us a control on when that stall should take place and how large it should be.
I do take enabling/disabling the particles system as a good practice, rather than an enforced hurdle we have to accept. Speaking of this made me even think if we should have a "preload" property for entity, which marks it to be fully loaded and initialized on app start, similar to assets. It would remove the need in enabling/disabling mechanic.

Using a template for chunks that not necessary are required, or using multiple scenes - is a good practice to reduce the complexity of hierarchy based on application logic.
Please, provide a way for developers to enforce collision initialization beforehand (on loading), or by a function call. To allow developers to create a smooth experience where they can afford loading time stall.
Also if any assets are loading, while JS is blocked - this loading continues, so it is efficient to stall the JS thread during assets loading.
Freezes during running application - is really bad for user experience, and must be avoided as much as possible.
Enabling/disabling things - feels a workaround, not a nice solution, a better solution is required, also for particles as it has been an issue in many projects.

The issue here is that a default Ammo build has about 64Mb of memory for itself. When the engine processes collision components of a complete scene, it can potentially crash, because it would run out of memory before the next step comes where it removes the disabled objects. I do agree though that a better solution is needed, instead of enabling/disabling mechanic. What are the options here? We can also mark this PR as draft, until a better alternative is found. I can simply keep a monkey patch in my project until then.

Closing this one. Whether the collision components are parsed at start (used or not) or at use time seems to be an opinionated preferring of a developer. Forcing one way or the other is equally bad. A better system would allow a user to switch it via some sort of a setting, but that would also need to consider other component types.


Lazy load defaults

@tseaver This can be split into 3 distinct PRs if you like:

Use an Environment class and an instance in _implicit_environ (attrs of instance replace class "constants")
Moving the code for set_default_dataset_id from __init__ to _implicit_environ
Add a lazy loading descriptor for setting the default dataset ID on the default instance from _implicit_environ

Still missing: a lazy loading descriptor for connection (it's pretty tangled in to the tests, so I wanted to discuss this before putting in too much work on that)
UPDATE: I think the best strategy for testing might be to just replace _implicit_environ.DEFAULT_ENVIRON with a fake instance that has Nones explicitly set in place of the lazily loaded pieces

Coverage remained the same at 100.0% when pulling 662ad4a2c93defd48b845d2ebb6c06f704707d24 on dhermes:lazy-load-defaults into 8e68a20cce2306f055557362b0c4a34909a36d1e on GoogleCloudPlatform:master.

@jgeewax This begins to cover one of the concerns in #647.
Also note that this adds a fair amount of complexity to our codebase all for the purposes of letting people forego
datastore.set_defaults()

FWIW I'd really like to be able for this to "just work" but it will involve a lot of actual work.

I'm a huge fan of your "UPDATE" note above... Wouldn't it be easier to just create an Environment... and then make the dataset field a property method, that calls get_default_dataset_id when it's needed?

@jgeewax You want the environment to be long lived because all the checks require I/O of some kind, so do it once and never again. Also, my comment was just about testing, where the assumption in most places was that these values would return None and not try to lazily load.

LGTM

@tseaver I'm hesitant to merge, would like to discuss.
I've been hacking on getting the lazy loaded connection and have realized

We don't want lazy loading in tests (unless we are testing lazy loading)
Every time a fresh DEFAULT_ENVIRON object is _Monkey-patched in a test, the property is lazy loaded
If the modules import the DEFAULT_ENVIRON object, patching it on the _implicit_environ namespace doesn't change the object that has been imported
Being super careful to only import the _implicit_environ module (instead of the object) is not future proof

I am debating a base class that explicitly sets the attribute to None:
class DatastoreBaseTest(unittest2.TestCase):

    def setUp(self):
        from gcloud.datastore import _implicit_environ
        self._replaced_default = _implicit_environ.DEFAULT_ENVIRON
        _implicit_environ.DEFAULT_ENVIRON = _implicit_environ.Environment()
        _implicit_environ.DEFAULT_ENVIRON.dataset_id = None

    def tearDown(self):
        from gcloud.datastore import _implicit_environ
        _implicit_environ.DEFAULT_ENVIRON = self._replaced_default

but this is again not future proof since it requires tests to inherit from it.

@tseaver Maybe we can put the contents of _implicit_environ.py there and most of the stuff in __init__.py as well? We can use a default Dataset object in there, i.e. dataset._DEFAULT_ENVIRON. But users will only see
get_default_dataset_id()
set_default_dataset_id()
get_default_connection()
set_default_connection()
set_defaults()
# get_defaults() would return that object, should we allow to use at their own risk?

Basic design (to add to Dataset) after #660 is in:
https://gist.github.com/dhermes/36eb807880718703cb9e
Then at the bottom of dataset.py create
_DEFAULT_DATASET = Dataset(_implicit=True)

def get_default_dataset_id():
    return _DEFAULT_DATASET.dataset_id

...

and in tests manually set
dataset._DEFAULT_DATASET.dataset_id = None
dataset._DEFAULT_DATASET.connection = None

either in setupModule or setupClass (so the lazy loading does not occur).

Wouldn't this be simpler?
with _Monkey(dataset, _DEFAULT_DATASET=Dataset('DATASET_ID', connection)):
       ...

If every test needs do that to avoid lazyily loading by mistake, it will become quite cumbersome. Though you're right that the _DEFAULT_DATASET object should be replaced wholesale.

Arg! After #660 there is really nowhere to put a Dataset object since it creates cycles everywhere that _implicit_environ.CONNECTION (e.g.) was used.

Just out of curiosity (I absolutely could be oversimplifying here) but are we abstracting this away the right way?
Looks like we have now:
_DEFAULT_DATASET = Dataset(...)

def get_default_dataset_id():
  return _DEFAULT_DATASET.dataset_id

would this be better as...
DEFAULT_DATASET_ID = None

def set_default_dataset_id(dataset_id):
  DEFAULT_DATASET_ID = dataset_id

def get_default_dataset_id():
  return DEFAULT_DATASET_ID or magic_method_to_get_my_dataset_from_appengine()

def get_default_dataset():
  return Dataset(dataset_id=get_default_dataset_id())

?
And then we never know about a special variable name, but always call the get_default_dataset() method when we need a dataset?

@jgeewax We were talking about unittests.

Sorry about that -- misread. Ignore me.

#663, #664, #665, #666, #667, and #668 step in for this, closing this PR


API for writing tests for lints

I'm working on a repo so that we (the Haskell community) can share lints for commonly used packages. I'm hoping to announce it on Reddit etc. once I've solved https://github.com/theindigamer/hlint-roller/issues/1 and https://github.com/theindigamer/hlint-roller/issues/2 .
To make sure that the lints work, I'd like to write tests (issue 1). The thing is, I don't want to duplicate a bunch of logic which you already have here, for example all the stuff needed to run the tests under data/hlint.yaml.
Would it be possible to expose a small API for writing tests for lints? If yes, I'm happy to contribute a PR if you can provide guidance on what would need to be changed.

You are meant to be able to do hlint test --hint=myhints.yaml to run the hints at the bottom of the file just hlint hlint.yaml. I haven't tested this in ages, but give it a go, and if you find any flaws we can figure out how to fix them to meet your needs.

Okay, it did work correctly! Awesome. There are a couple of minor problems:

The test subcommand requires that a .hlint.yaml file be present in the directory, which I didn't expect. However, this isn't an issue in practice.
It seems that I need to duplicate imports for every test; is that correct? This is what is being done in data/hlint.yaml but it seems a bit painful in practice because I end up writing stuff like:

# import qualified Data.Map as Map \
# import qualified Data.Set as Set \
# yes = Set.fromList (Map.keys m) -- Map.keysSet m
# import qualified Data.Map as Map \
# yes = fmap fst (Map.toList x) -- Map.keys x

I'm planning to write several tests combining stuff across modules and it seems like I'll end up writing more imports than test code :sweat_smile: ...

I fixed the issue with .hlint.yaml - nice and easy.
Yes, you need to duplicate imports, by design. The reason is that different tests will want different sets of imports - but I can see why it's a hassle for you. I guess if I'm honest I probably wouldn't do overly much testing of the rules you are adding - the rule pretty much speaks for itself - and any misunderstanding that breaks the rule is likely to be reflected in the test. I don't add new tests with new rules in hlint.yaml.

I guess I'm curious what mistake you envisage making in the hint and how the test will catch it? That might direct what features are needed.

 Surprisingly, the test didn't pass.
After about a minute or so, I realized that I got the lhs and rhs messed up. Duh.
Then I tried:
# import qualified Data.Map.Strict as MapS \
# no = (MapS.lookup k1 m, MapS.insertWithKey f k2 x m)

and it did work as expected.
I can't come with a better example off the top of my head right now but I recall writing a lint for some optparse-applicative code because I'd made a mistake with using one combinator. I thought "hmm, this is perfect, we'll never have this mistake again, awesome!". Well, I made the same mistake a week later. Then I got deja-vu and went back to check the lint, and finally ended up pushing a "Fix incorrect lint" commit.
After that, I've felt wary of writing lints without small test cases. The thing is, even if something in your application blows up, you may go and ask "hey why didn't our test suite catch this earlier" but you're probably not going to your linter and check whether there was a lint for it that may not have fired.
Anyways, sorry for the rambling story, it isn't really a big deal. The big deal is being able to run the tests easily if I want to and that does work without hiccups :D.

It feels very wrong to dissuade someone from tests... Perhaps some special syntax in an annotation should lead to that portion being copied into all subsequent annotations in that file? Perhaps \ at the end (rather than ) should be continue and accumulate? What do you suggest?

Perhaps \ at the end (rather than )

I don't quite understand. Could you give an example?
Perhaps something like the following?
# <TEST_WITH_SHARED_IMPORTS>
# import Foo
# import Bar
# yes = ... -- ...
# no = ... -- ...
# </TEST_WITH_SHARED_IMPORTS>

Yep, that works as I'd expect.

Do you want to hack up the code? If not, I'll try and get round to it.

I'll send a PR in a few days (it isn't urgent), working on other stuff ATM.


04/02/2018 17:44 : Create Pull Request for branch prTesting_2018040217 -> master

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestRef.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


My devices not detected (Account is in AU)

Hi,
I have a couple of Haier devices that use the SmartHQ app, my country is Australia.
I added the integration and it was able to connect to my account successfully, but there was only two options for the region (EU and US),  I tried using both but no devices / entities were added under the integration.
Happy to provide any logs if required
thanks for the great work

Confirmed, same issue here in Australia. I only have one device in SmartHQ, a Haier Front Load Washing Machine. Let us know what's needed to add our devices.

For what its worth I used US, (EU gives invalid auth) and my device (fridge) got pulled in.

Hmm, I only see the two options on the login site, it seems like @partsdotpdf's suggestion might be the way to make it work, please let me know if it doesn't work.  If not, can you show me the login page you're using and I might be able to figure out how to get it incorporated.

@simbaja
I used the login site mentioned in your comment and I can see my devices when I login.
However, the integration doesn't have any devices shown attached to it.

when I signed up first time in SmartHQ, I used the mobile application to do so and I chose "AU" as my country
Let me know if you need any logs
Thanks

Based on the above, I think you'll need to set the region to US to make it work.  Can you give that a shot and let me know if it works?

Thanks for your reply, not sure how to change the region though.
When I sign up, I am asked to enter my address which determines the country, do I need to use an address in the US?
I don't see specific settings for a region similar to other platforms like Tuya for example

In the integration configuration, there are two choices, EU or US, select US and see if it works.

Yes, I already tried choosing US in the HA integration and the authentication is successful but it doesn't pull and devices. Do you think I should try re-adding the devices?

Is there anything in the log stating that there's an unknown device?  Not all types are supported, especially other brands outside of GE.  I'd also suggest going out to the sdk repository and following the directions to produce a debug log which may also help understand what's happening.

Nothing in the logs other than this warning

I will try to produce a debug log

I was able to connect successfully using the "" application in your repo using the "US" as the region.
It was able to find my devices and read the status, can see an error message coming though, but not sure what it is
2023-01-24 16:39:39,667 DEBUG    Sending keepalive ping
2023-01-24 16:39:39,667 DEBUG    > TEXT '{"kind": "websocket#ping", "id": "keepalive-ping", "action": "ping"}' [68 bytes]
2023-01-24 16:39:39,888 DEBUG    < TEXT '{"kind":"websocket#pong","id":"keepalive-ping"}' [47 bytes]
2023-01-24 16:39:40,107 DEBUG    Requesting update for GeAppliance(D828C9DBAF61) (ErdApplianceType.DRYER)
2023-01-24 16:39:40,107 DEBUG    Requesting update for client D828C9DBAF61
2023-01-24 16:39:40,108 DEBUG    > TEXT '{"kind": "websocket#api", "action": "api", "hos... "D828C9DBAF61-allErd"}' [171 bytes]
2023-01-24 16:39:40,109 DEBUG    Requesting update for GeAppliance(D828C9D4635B) (ErdApplianceType.WASHER)
2023-01-24 16:39:40,109 DEBUG    Requesting update for client D828C9D4635B
2023-01-24 16:39:40,109 DEBUG    > TEXT '{"kind": "websocket#api", "action": "api", "hos... "D828C9D4635B-allErd"}' [171 bytes]
2023-01-24 16:39:40,463 DEBUG    < TEXT '{"kind":"websocket#api","id":"D828C9DBAF61-allE...1-24T04:24:01.383Z"}]}}' [3291 bytes]
2023-01-24 16:39:40,464 ERROR    Got ValueError ErdCode.WIFI_MODULE_UPDATING - 0A
2023-01-24 16:39:40,644 DEBUG    < TEXT '{"kind":"websocket#api","id":"D828C9D4635B-allE...1-24T01:26:24.786Z"}]}}' [3141 bytes]
2023-01-24 16:39:40,645 ERROR    Got ValueError ErdCode.WIFI_MODULE_UPDATING - 0A
2023-01-24 16:39:41,748 DEBUG    % sending keepalive ping
2023-01-24 16:39:41,748 DEBUG    > PING a2 ed 70 1a [binary, 4 bytes]
2023-01-24 16:39:41,937 DEBUG    < PONG a2 ed 70 1a [binary, 4 bytes]
2023-01-24 16:39:41,937 DEBUG    % received keepalive pong
2023-01-24 16:40:01,948 DEBUG    % sending keepalive ping
2023-01-24 16:40:01,949 DEBUG    > PING ' @^w' [text, 4 bytes]
2023-01-24 16:40:02,139 DEBUG    < PONG ' @^w' [text, 4 bytes]
2023-01-24 16:40:02,139 DEBUG    % received keepalive pong
2023-01-24 16:40:09,683 DEBUG    Sending keepalive ping
2023-01-24 16:40:09,683 DEBUG    > TEXT '{"kind": "websocket#ping", "id": "keepalive-ping", "action": "ping"}' [68 bytes]
2023-01-24 16:40:09,884 DEBUG    < TEXT '{"kind":"websocket#pong","id":"keepalive-ping"}' [47 bytes]
2023-01-24 16:40:22,147 DEBUG    % sending keepalive ping
2023-01-24 16:40:22,148 DEBUG    > PING 44 21 18 b1 [binary, 4 bytes]
2023-01-24 16:40:22,331 DEBUG    < PONG 44 21 18 b1 [binary, 4 bytes]
2023-01-24 16:40:22,331 DEBUG    % received keepalive pong
2023-01-24 16:40:39,687 DEBUG    Sending keepalive ping

2023-01-24 16:35:40,428 DEBUG    Setting ErdCode.MODEL_NUMBER to HWFH600AN1
2023-01-24 16:35:40,428 DEBUG    Setting ErdCode.SERIAL_NUMBER to
2023-01-24 16:35:40,428 DEBUG    Setting ErdCode.APPLIANCE_TYPE to ErdApplianceType.WASHER
2023-01-24 16:35:40,429 DEBUG    Setting ErdCode.UNIT_TYPE to ErdUnitType.TYPE_120V_MONOGRAM
2023-01-24 16:35:40,429 ERROR    Got ValueError ErdCode.WIFI_MODULE_UPDATING - 0A
2023-01-24 16:35:40,442 DEBUG    Setting ErdCode.WIFI_MODULE_UPDATING to b'\n'
2023-01-24 16:35:40,442 ERROR    Got ValueError ErdCode.WIFI_MODULE_UPDATING - 0A
2023-01-24 16:35:40,443 DEBUG    Setting ErdCode.WIFI_MODULE_SW_VERSION to <IP_ADDRESS>
2023-01-24 16:35:40,443 DEBUG    Setting ErdCode.WIFI_MODULE_SW_VERSION_AVAILABLE to <IP_ADDRESS>
2023-01-24 16:35:40,445 DEBUG    Setting ErdCode.ACM_UPDATING to False
2023-01-24 16:35:40,445 DEBUG    Setting ErdCode.APPLIANCE_SW_VERSION to <IP_ADDRESS>
2023-01-24 16:35:40,445 DEBUG    Setting ErdCode.APPLIANCE_SW_VERSION_AVAILABLE to <IP_ADDRESS>
2023-01-24 16:35:40,446 DEBUG    Setting ErdCode.APPLIANCE_UPDATING to False
2023-01-24 16:35:40,446 DEBUG    Setting ErdCode.LAUNDRY_MACHINE_STATE to Standby
2023-01-24 16:35:40,448 DEBUG    Setting ErdCode.LAUNDRY_SUB_CYCLE to ErdLaundrySubCycle.CYCLE_NONE
2023-01-24 16:35:40,448 DEBUG    Setting ErdCode.LAUNDRY_END_OF_CYCLE to False
2023-01-24 16:35:40,448 DEBUG    Setting ErdCode.LAUNDRY_TIME_REMAINING to 0:31:00
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_CYCLE to ErdLaundryCycle.COTTONS
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_DELAY_TIME_REMAINING to 0:00:00
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_DOOR to ErdLaundryDoorStatus.CLOSED
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_WASHER_DOOR_LOCK to False
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_WASHER_SOIL_LEVEL to ErdSoilLevel.EXTRA_LIGHT
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_WASHER_WASHTEMP_LEVEL to Tap Cold
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_UNKNOWN201D to b'\x00'
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_REMOTE_DELAY_CONTROL to 0:00:00
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_REMOTE_STATUS to False
2023-01-24 16:35:40,449 DEBUG    Setting ErdCode.LAUNDRY_REMOTE_POWER_CONTROL to ErdOnOff.OFF

Can you try the latest pre-release version and see if it resolves your issue?  Someone else encountered a similar issue and I added some logic that should fix it.

Which one please?

0.6.6-dev1

Good news.
I am also in Australia and used the US option when installing this Integration.
My Haier washer did not appear.  Then I tried the 0.6.6-dev-1 update and it WORKS!!!!!
Thanks so much for working on this.

Yeah, works perfectly for me so far.
@simbaja hope it will be merged to the main branch soon so that we can start using the main version and get updates.


Never unlatch using Nuki, use opener or other device instead

Hi :)
As with almost all detached homes in Germany, we have a front door and an intercom that can operate a door buzzer. If I use the Smart Lock to unlock and unlatch the door, it opens a little and stays in this position after the latch is released. If I open but no one goes through the door and closes it afterwards, the door stays open. However, this doesn't happen if the door buzzer is being used instead of the latch. Nevertheless, using the door buzzer immediately doesn't make sense; I'd rather have it "wait" for the Smart Lockto unlock the door.
So here's what I'd like:

"Locking the door" lock the door using the Smart Lock
"Unlock the door" unlocks the door using the Smart Lock
"Open the door"

if locked, instructs the Smart Lockto unlock the door and waits for the Smart Lockto complete, then triggers the Opener
if unlocked, instantly trigger the Opener

And here's the premium: It would be cool, if I didn't need to buy a bridge and an opener for that but could use any switch/button in my home as the opener (maybe though some kind of automation).
It might be that there already is a way to implement what I want though automations. Please share your ideas!

I'm afraid,  don't have much experience with buzzers.  The doorbell/intercom/buzzer installation in my apartment works fine with the Nuki Opener, and I've been using that.

if locked, instructs the Smart Lockto unlock the door and waits for the Smart Lockto complete, then triggers the Opener
if unlocked, instantly trigger the Opener
It might be that there already is a way to implement what I want though automations.

I suppose this could be automated, but might be a bit tricky.  I think it requires three automations and a flag (or dummy switch).  On "open the door", when the door is locked, you'd set the flag and unlock the door.  On the door reporting it's opened, when the flag is set, you'd activate the buzzer and clear the flag.  On "open the door", when the door is unlocked, simply activate the buzzer.

It would be cool, if I didn't need to buy a bridge and an opener for that but could use any switch/button in my home as the opener

I suppose, if you can figure out which wires activate the buzzer, you could short them using a relay.  This could be a Zigbee smart relay, with Homebridge deCONZ, or a relay controlled through a Raspberry Pi GPIO pin, with Homebridge RPi.  I suppose there's many other types of relays, integrated in HomeKit, but I have no experience with these either.
You might also consider a low-tech solution: install a mechanical door closer to keep the door closed, even when unlatched.  It seems a bit superfluous to use both a lock and a buzzer, although, I suppose, you would this for what we would call a "night lock".

Hi Erik,
I build my entire smart home using hb, esp8266s and mqtt. I am able to install something else than the opener. And I am able to install a mechanical door closer, but that is not the point I am trying to make. The point is: this setup is the absolut standard in detached homes in the DACH area.
The question is: is there anyone willing to implement something like a "detached home front door" (dhfd) mode that has a "use opener" bool parameter?
a) with "use opener" set to true, the dhfd mode creates three buttons (see post above) or a lock and a button (lock for locked and unlocked, button for buzzer). The lock (or two of the buttons) lock and unlock the smart lock. The additional button checks if the smart lock is unlocks, and if so tells the opener to open or tells the smart lock to unlock, waits for the smart lock to be done and afterwards tells the opener to open.
b) with "use opener" set to false, the dhfd mode creates the same three buttons (or lock and one button) as in a) and an additional sensor (door contact?). Everything stays the same except for the opener. Instead of triggering the opener, the sensor state is changed. One can then attach any HomeKit/Homebridge device to this sensor using an automation; no need to buy bridge and opener then.

I think Homebridge is about exposing devices that don’t (fully) support HomeKit as HomeKit accessories.  At least, that’s what my plugins do.  Creating new accessories by combining multiple devices is beyond that.
I suppose, technically, you could create a dedicated dhfd plugin for your own use.  Note that Homebridge plugins don’t have access to other HomeKit accessories.  The dhfd plugin would need to use one of the Nuki APIs to check and control the lock state.  And it would need to interact with the relay on the esp8266 (over mttq) to activate the buzzer.


Include link to Anaconda status page in README

What is the idea?
In order to more easily check the Anaconda status page, it would be great to include the link in the README

Was meaning this repo's README specifically, but it could make sense in other places
Suggested here since folks raising infra issues (with Anaconda.org) might appreciate seeing the status before raising (to see if it is a known issue)


Preserve timestamps on untransformed files

Allowed targets to specify the preserveTimestamps option, as defined in fs-extra. The option is then passed though.
As transformed files are different from their source file, I didn't add support for preserving timestamps in that path.
I wasn't able to test this yet.
Fixes #17

@oliversalzburg Thanks. I have released this as v0.6.0!

Nice. Thanks for the additions and the quick merge. Appreciate it! :)

I was also able to verify that it works as intended now. Thanks again


[bazel] Bazel-invoked fusesoc gets stuck in dependency resolution

ERROR: /usr/local/google/home/mcyoung/ot/ot/hw/BUILD:9:14: FuseSoC hw/build.verilator failed: (Exit 1): fusesoc failed: error executing command fusesoc '--cores-root=.' '--cores-root=.' '--cores-root=.' run '--flag=fileset_top' '--target=sim' --setup --build '--build-root=bazel-out/k8-fastbuild/bin/hw/build.verilator' ... (remaining 1 argument(s) skipped)
WARNING: Unknown item virtual in section Root
WARNING: Unknown item virtual in section Root
WARNING: Unknown item virtual in section Root
WARNING: Unknown item virtual in section Root
WARNING: Unknown item virtual in section Root
WARNING: Unknown item virtual in section Root
ERROR: Conflicting requirements:
Requirements: 'lowrisc_dv_chip_verilator_sim == 0.1-0' <- 'lowrisc_systems_chip_earlgrey_verilator >= 0-0' <- 'lowrisc_systems_ast >= 0-0' <- 'lowrisc_ip_rstmgr_pkg >= 0-0' <- 'lowrisc_ip_interfaces_alert_handler_reg >= 0-0'
    +lowrisc_ip_rstmgr_pkg-0.1-0 was ignored because it depends on missing packages
Requirements: 'lowrisc_dv_chip_verilator_sim == 0.1-0' <- 'lowrisc_systems_chip_earlgrey_verilator >= 0-0' <- 'lowrisc_systems_ast >= 0-0' <- 'lowrisc_ip_rstmgr_pkg >= 0-0'
    lowrisc_systems_ast-0.1-0 requires (+lowrisc_ip_rstmgr_pkg-0.1-0)
Requirements: 'lowrisc_dv_chip_verilator_sim == 0.1-0' <- 'lowrisc_systems_chip_earlgrey_verilator >= 0-0' <- 'lowrisc_systems_ast >= 0-0'
    lowrisc_systems_chip_earlgrey_verilator-0.1-0 requires (+lowrisc_systems_ast-0.1-0)
Requirements: 'lowrisc_dv_chip_verilator_sim == 0.1-0' <- 'lowrisc_systems_chip_earlgrey_verilator >= 0-0'
    lowrisc_dv_chip_verilator_sim-0.1-0 requires (+lowrisc_systems_chip_earlgrey_verilator-0.1-0)
Requirements: 'lowrisc_dv_chip_verilator_sim == 0.1-0'
    Install command rule (+lowrisc_dv_chip_verilator_sim-0.1-0)

This is running on a master sync'd this morning, and I made sure the verilator I have installed is new enough.

It's fusesoc that is failing, though, not verilator. Is your edalize / fusesoc version new enough?
Might try
pip3 install -r python-requirements.txt

I can't reproduce this (presumably because I have a newer version of fuseoc) so I'll assign it back to you @mcy

@mcy reported updating the python-requirements resolved this.


Tie-in scripts incompatible with Yosemite Messages

Looks like Messages was changed a fair amount in Yosemite. Only one handler script can be assigned, and it must be in ~/Library/Application Scripts/com.apple.iChat/. I've tried both copying and symlinking messageReceived.scpt there, but get a pretty generic error when receiving an iMessage:
Event: Message Received in Active Chat
File: messageReceived.scpt
Error: The operation couldn’t be completed. /Users/xbmc/Library/Application Scripts/com.apple.iChat/messageReceived.scpt: execution error: No error. (-1708)
I assume this is due to Apple's usual point release musical chairs, but it's possible I've done something wrong...

Yeah, i can't get it to work either.

TypeError: Cannot read property 'split' of undefined
  at iMessageAdapter.run (/Users/KingScooty/Desktop/hubot/node_modules/hubot-imessage/src/imessage.coffee:23:55, <js>:51:61)
  at Robot.run (/Users/KingScooty/Desktop/hubot/node_modules/hubot/src/robot.coffee:443:5, <js>:397:27)
  at Object.<anonymous> (/Users/KingScooty/Desktop/hubot/node_modules/hubot/bin/hubot:135:8, <js>:146:11)
  at Object.<anonymous> (/Users/KingScooty/Desktop/hubot/node_modules/hubot/bin/hubot:5:1, <js>:149:4)
  at Module._compile (module.js:460:26)

@KingScooty @tylerkerr Either of you solve this?

No, unfortunately not! This would have been for a pet project so I didn't spent too long digging into it.

I managed to get past the Cannot read property 'split' of undefined error by setting the ENV variable HUBOT_IMESSAGE_HANDLES. This is the whitelist that the hubot will listen too and takes in the email or number of the imessage.

@Vannilaknight So you got everything working? Not clear on what you mean by "setting the ENV variable" -- can you elaborate? I've added my relevant emails and phone numbers to 'HUBOT_IMESSAGE_HANDLES' via export<EMAIL_ADDRESS>


[Bug]: Duplicates not sent for MQTTv5 and QoS 1

Environment

VerneMQ Version: 2.0.1-8-g61c1938f
OS: Debian 12
Erlang/OTP version (if building from source): 26.2.5
Cluster size/standalone: standalone

Current Behavior
MQTTv5: If a subscriber does not respond to a publish message with a ack, the broker never resends the message.
No matter how long the subscriber waits with sending the pub ack. Next messages are sent to the subscriber as they come.
MQTTv3.1.1: The message is resent with the DUP flag.
Expected behaviour
The message should be resent for both versions.
Configuration, logs, error output, etc.
Connect Flags: 0xc2, User Name Flag, Password Flag, Will QoS Level: At most once delivery (Fire and Forget), Clean Session Flag
No properties

Subscription Options: 0x01, Retain Handling: Send msgs at subscription time, QoS: At least once delivery (Acknowledged deliver)
No properties

In Sub Ack: Granted QoS 1 (1)

Publish Header Flags: 0x33, Message Type: Publish Message, QoS Level: At least once delivery (Acknowledged deliver), Retain
or for not retained
Header Flags: 0x32, Message Type: Publish Message, QoS Level: At least once delivery (Acknowledged deliver)

bin/vmq-admin session show --client_id --statename --waiting_acks
+-----------+---------------------------------+--------------+
| statename | client_id                       | waiting_acks |
+-----------+---------------------------------+--------------+
| online    | mqtt-dup-test                   | 684          |
+-----------+---------------------------------+--------------+

Nothing in logs even with log.console.level = debug

Code of Conduct

[X] I agree to follow the VerneMQ's Code of Conduct

@pszafarczyk Please review the following section of the v5 spec: https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901238
The behaviour has changed in v5 with the reasoning that the TCP stack already should not loose messages, although maybe the message might be waiting in a TCP buffer somewhere.

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@ioolkos Thanks. I did not know that.
What should happen with problems at an application level? What the subscriber should do?
If it sends an ACK and is not able to store all the messages coming, the messages will be lost.
Should it disconnect after any failure and reconnect (with Clean Start set to 0) to have them being resend?

@pszafarczyk MQTT is transfer or responsibility for messages. If your application has not yet seen a message from the TCP stack, there's not much use in sending it again within the same TCP session. With a new TCP session, the situation is different, therefore the requirement for the initial exchange for outstanding acks and messages.
If a consumer cannot keep up with incoming messages, you'll have to increase performance on the consumer or use consumer side load balancing (shared subscriptions).
I'm not convinced that disconnecting when the consumer has "troubles" of whatever sort is the way to go.

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

@ioolkos Thanks a lot.
I meant troubles like a temporary database problem. Let's say the database is being rebooted. There is not much the subscriber can do. It just needs a moment to be able to precess the messages again.


Enable pipeline to configure connection pooling

I want to send more concurrent filter queries to speed up processing of large batches of events. I've been unable to find any documentation that might explain the behavior here. Can anybody clear things up for me?
-- Justin McAleer, via discuss

Since Sequel supports connection pooling by default, exposing the ability to control aspects of pooling up to the pipeline configuration should be pretty straight-forward.

+1 For the issue, it seems that several connections are opened and kept in pool, but only one of the used at any time.

Any updates please.

Any updates please?

Is the problem here that one filter is serially accessed by each worker?
We have discussed batch event processing in enhancement filters before. No conclusive approach was decided upon.
One strategy that I am interested in exploring is distinct value processing, assumes that batches have clusters of events that will lookup the same value, e.g. IP address or product id. This could also be called cache by batch (cache cleared after each batch).

@guyboertje In our particular case the problem was unexpected number of connections to database and no ability to control them.
We had 10 pipelines with several filters each and this setup sometimes gave bursts of 100 open connections which were too many for our DB.
The best solution for this case would be ability to configure shared connection pool in all pipelines in filters, but it may lead to starvation and not sure if technically possible to implement. So the bare minimum is to simply allow set number of connections for each filter's connection pool. Since now it uses 4 connections per filter as default and cannot be set to 1 connection per filter max.

@guyboertje Thank you for your response.

Is the problem here that one filter is serially accessed by each worker?

Yes, In our case we have one pipeline, with 5 workers and a batch size of 5000.
Input - JDBC plugin connecting to Oracle DB, each event (input data row) has a unique key and independent of the other.
Filters - Two JDBC Streaming filters that retrieve additional data (from same Oracle DB) using key value(s) from the event.
Output - To Elasticsearch
It is a basic setup, however, we noticed that the process freezes for records > 50. We have close to 2.5 Million rows from the input SQL query, if we don't include the JDBC streaming, then the process completes in a jiffy. We suspect that the JDBC steaming is causing some bottleneck (sequential executions??) not sure if the contention is on the Oracle connection or something else.

We have discussed batch event processing in enhancement filters before. No conclusive approach was decided upon.

Would this approach help with the JDBC streaming filter?

One strategy that I am interested in exploring is distinct value processing, assumes that batches have clusters of events that will lookup the same value, e.g. IP address or product id. This could also be called cache by batch (cache cleared after each batch).

Sorry, but I'm not too familiar with the architecture of Logstash i.e. how the workers get distributed among the batches etc.
Any help or insight into our situation would be appreciated.
I'm including a trimmed out snippet of  our config:
input {
   jdbc {
        clean_run => false
	jdbc_driver_library => "C:\logstash\ojdbc8.jar"
	jdbc_driver_class => "Java::oracle.jdbc.OracleDriver"
	jdbc_connection_string => "jdbc:oracle:thin:hostserver.abc.xyz/service"
        jdbc_user => "xxxx"
	jdbc_password => "xxxxx"
	# Connection pool configuration.
	# Validate connection before use.
	jdbc_validate_connection => true
	#How often to validate a connection (in seconds)
	jdbc_validation_timeout => 120
	#jdbc_validation_query => "SELECT 1 FROM DUAL"
	#Maximum number of times to try connecting to database
	connection_retry_attempts => 5
	# Number of seconds to sleep between connection attempts
	connection_retry_attempts_wait_time => 2
	lowercase_column_names => true

	# General/Vendor-specific Sequel configuration options.
	# https://github.com/jeremyevans/sequel/blob/master/doc/opening_databases.rdoc
	sequel_opts =>
		{
			login_timeout => "60"
			prefetch_rows => "5000"
			jdbc_properties =>
				{
					"defaultRowPrefetch" => "5000"
					"loginTimeout" => "60"
					"inactiveConnectionTimeout" => "120"
					"timeoutCheckInterval" => "120"
					"tcpKeepAlive" => "true"
					"oracle.net.READ_TIMEOUT" => "5000"
					"validationQuery" => "SELECT 1 FROM DUAL"
				}
		}

	# SQL Query
	statement => " SELECT id, abc, def, xyz FROM table "
   }
}

filter {
	jdbc_streaming {
			...
			jdbc_connection_string => "jdbc:oracle:thin:hostserver.abc.xyz/service"
			...
			target => "target_field_1"
			use_cache => "false"
			cache_expiration => "5"
			cache_size => "0"
			statement => "SELECT blah FROM TABLE_1 WHERE ID = :id"
			parameters => {"id" => "id"}
	}

	jdbc_streaming {
			...
			jdbc_connection_string => "jdbc:oracle:thin:hostserver.abc.xyz/service"
			...
			target => "target_field_2"
			use_cache => "false"
			cache_expiration => "5"
			cache_size => "0"
			statement => "SELECT blah, blah, blah FROM TABLE_2 WHERE ID = :id"
			parameters => {"id" => "id"}
	}

  #Add field first to handle if there is any null code
	mutate { add_field => { "[xyz][code]" => "" }}

	if [code] {	mutate { replace => { "[xyz][code]" => "%{code}" } } }
}

output {
	elasticsearch {
		hosts => xxxx
		index => "indexname"
		http_compression => true
	}
}

Ahh OK. So you mean global connection pooling. I don't think this can be done in Logstash as each filter is seen as an autonomous transformation engine working on a single event at a time.
I have seen similar questions before some time back now, at that time I searched for any kind of jdbc db proxy. I searched again today and found:

sqlrelay
HA-JDBC

Both are actively developed and are open source. I have not tested any of them. Please feedback any conclusions if you decide to evaluate them and/or use one in production.

@guyboertje
Ok. That leads to me another question, do (or can) the workers process filters in parallel, i.e. lets suppose I have 5 workers, each is processing an input event (so 5 in all) in parallel, and for each of those events will there be an autonomous instance of a filter be created to process the event?
Thanks for the pointers to the JDBC connection pooling, depending on the answer to my above question, we can narrow down to where the bottle neck is, i.e. if the filters can't be processing multiple events in parallel, then connection pooling becomes secondary (but a bottleneck nonetheless).

@stanuku
From @dmitrymurashenkov above...

We had 10 pipelines with several filters each and this setup sometimes gave bursts of 100 open connections which were too many for our DB.

Looking at the Logstash and Sequel code, each pipeline is autonomous and each filter plugin instance (as seen in the config) declares (from the authors) itself threadsafe or not. Threadsafe filters are reused across workers (but not pipelines). Threadsafe filters are assumed to be able to be called in parallel by each worker thread. On my Macbook, the jdbc_streaming filter uses a threaded connection pool with has 4 connections by default.
Each worker takes a batch of events from the queue (the inputs feed newly minted events into the queue) and feeds the events from the batch through each filter sequentially based on the conditional logic in the config (if any). This means that if you have two jdbc_streaming filters one after the other then only one will be executing a statement at any one time per worker thread.
Simultaneous execution of a statement by multiple workers threads is probable (up to the 4 connection limit, the default pool size) but to what degree this simultaneous statement execution occurs is determined by the how synchronised the worker loops become as each worker loop is subject to variable delays as it executes the filters and output(s).
Thinking about a worst case scenario, imagine that a jdbc_streaming filter is used to lookup user details from an userid held in an event and that a stream of 2500 events all refer to the same user id. The exact same statement will be executed 2500 times - this is clearly very wasteful but the jdbc_streaming filter is designed with volatility in mind, i.e. that the lookup source tables change often and unpredictably. The jdbc_static filter is designed with non-volatile lookup sources in mind (but a refresh schedule allows for periodic refill of the latest changes). However, jdbc_static is less useful when the lookup sources are very large. There are some mix and match options possible to mitigate downloading millions of lookup records in the jdbc_static by using a jdbc_static for the "hot" 20000 ids and a conditional jdbc_streaming filter for the "cold" rest.
To test whether a bigger pool size will improve throughput or whether a smaller pool size will put less load on the DB you can modify the jdbc_streaming filter source code (in place). Replace line 53 of the file vendor/bundle/jruby/2.3.0/gems/logstash-filter-jdbc_streaming-1.0.4/lib/logstash/plugin_mixins/jdbc_streaming.rb with these 3 lines:
    opts = {:user=> @jdbc_user, :password=>  @jdbc_password.nil? ? nil : @jdbc_password.value}
    opts[:max_connections] = 9 # number of workers plus 1
    @database = Sequel.connect(@jdbc_connection_string, opts)

If you do this test, please report your findings back here.

@guyboertje Much appreciated!, will keep you posted.


time_range for the finetuning experiment

Thanks again for this awesome repo. It helps me a lot. I've got a question regarding which time_range to use for sampling subgraphs for test. For example, in finetune_OAG_PF.py, this line is used to prepare the input to GNN:
node_feature, node_type, edge_time, edge_index, edge_type, x_ids, ylabel =  node_classification_sample(randint(), test_pairs, test_range)

where test_range is used to filter out nodes when sampling the subgraph as shown in L128 in data.py
if source_time > np.max(list(time_range.keys())) or source_id in layer_data[source_type]:
    continue

It looks that some test edges (which are not the prediction targets for current batch but might be the prediction targets for other batches) might be included in the sampled subgraph even after the masking process in Line 114 in finetune_OAG_PF.py.
    '''
        (3) Mask out the edge between the output target nodes (paper) with output source nodes (L2 field)
    '''
    masked_edge_list = []
    for i in edge_list['paper']['field']['rev_PF_in_L2']:
        if i[0] >= args.batch_size:
            masked_edge_list += [i]
    edge_list['paper']['field']['rev_PF_in_L2'] = masked_edge_list

    masked_edge_list = []
    for i in edge_list['field']['paper']['PF_in_L2']:
        if i[1] >= args.batch_size:
            masked_edge_list += [i]
    edge_list['field']['paper']['PF_in_L2'] = masked_edge_list

I'm not sure how this will impact on the evaluation. Looking forward to your feedback on this.

Hi Yihong:
Thanks for pointing that out. Yes it's indeed a issue if we didn't know the
other batches output.
The reason I add the label node is to allow some kind of label propagation,
but it definitely shouldn't allow ground-truth of other test nodes. I'll
modify the masking code to remove them. A most simple way is to remove all
the links to the label during the masking, and it should be fine.
On Mon, May 3, 2021, 13:01 Yihong Chen @.***> wrote:

Thanks again for this awesome repo. It helps me a lot. I've got a question
regarding which time_range to use for sampling subgraphs for test. For
example, in finetune_OAG_PF.py
https://github.com/acbull/GPT-GNN/blob/master/example_OAG/finetune_OAG_PF.py,
this line is used to prepare the input to GNN:
node_feature, node_type, edge_time, edge_index, edge_type, x_ids, ylabel =  node_classification_sample(randint(), test_pairs, test_range)
where test_range is used to filter out nodes when sampling the subgraph
as shown in L128 in data.py
https://github.com/acbull/GPT-GNN/blob/8ac50bf0720e5260b175f55e338c09e3a3adf729/example_OAG/GPT_GNN/data.py#L128
if source_time > np.max(list(time_range.keys())) or source_id in layer_data[source_type]:
continue
It looks that some test edges (which are not the prediction targets for
current batch but might be the prediction targets for other batches) might
be included in the sampled subgraph even after the masking process in Line
114 in finetune_OAG_PF.py
https://github.com/acbull/GPT-GNN/blob/8ac50bf0720e5260b175f55e338c09e3a3adf729/example_OAG/finetune_OAG_PF.py#L114
.
'''
    (3) Mask out the edge between the output target nodes (paper) with output source nodes (L2 field)
'''
masked_edge_list = []
for i in edge_list['paper']['field']['rev_PF_in_L2']:
    if i[0] >= args.batch_size:
        masked_edge_list += [i]
edge_list['paper']['field']['rev_PF_in_L2'] = masked_edge_list

masked_edge_list = []
for i in edge_list['field']['paper']['PF_in_L2']:
    if i[1] >= args.batch_size:
        masked_edge_list += [i]
edge_list['field']['paper']['PF_in_L2'] = masked_edge_list

I'm not sure how this will impact on the evaluation. Looking forward to
your feedback on this.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/acbull/GPT-GNN/issues/31, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHREXR4FR5HKWMU42WQHS7LTL3JGZANCNFSM44BFASZA
.

Hi @acbull, Thanks for the very quick feedback and the suggested solution :)


Add ability to trigger change listeners manually

Hi,
I've implemented some code which will fire events across browser windows. In my case, I need this so I can act upon the removal of the session cookie (i.e. logging out).
For me to do this, I need to be able to trigger the change listeners manually (the variables and function are private).

I don't think this should be part of the API. This would expose way too much implementation details.


Update publish.md

Docs Build status updates of commit 603dcc9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/deploy-and-publish/appsource/publish.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thank you @prkosh! I merged these changes with pr #1674.


Certificate Generator

Aim

What is the objective of the Script
To generate certificate

Details

What features will your script have
The certificate will be generated automatically and will get downloaded in the directory after generation.

Please assign me this issue under SWOC.

Sure Go ahead @sakshikhachane

Sure Go ahead @sakshikhachane

It has been 25 days guys. I can do this in 1 day or 2. Pls assign me this issue. @avinashkranjan sir.

Under SWoC

Sure Go Ahead @achalesh27022003

@avinashkranjan  sir this PR has been counted under SwoC na?

@avinashkranjan sir this PR has been counted under SwoC na?

Yes


Add arm64 to MSVC hosts

This enables using native arm64 toolset
The following changes are proposed:

Add arm64 as a valid host for MSVC

The purpose of this change
Visual Studio 2022 17.4 Preview introduced native build tools on arm64. This change enables the vscode-cmake-tools to detect the native tools so we can build with it instead of emulating x86/x64 to run the non-native build tools.

Ah, it currently looks like cmake errors out at this time since it only expects x86 or amd64 for the host.

It looks like if I modify the %LOCALAPPDATA%\CMakeTools\cmake-tools-kits.json and remove the instances of:
"toolset": "host=arm64"

We stop passing in -T host=arm64 which works-ish for building and debugging... but not sure how to get this working correctly 😅. From cmake documentation, it looks like only host=x86 or host=x64 are valid options, and not host=arm64.

Is this screenshot from an arm64 machine where you scan for kits?
if yes, do all 3 host compilers exists? the paths look something like this:
C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.34.31921\bin\Hostx64

Is this screenshot from an arm64 machine where you scan for kits? if yes, do all 3 host compilers exists? the paths look something like this:
C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.34.31921\bin\Hostx64

It is a screenshot from an arm64 machine where all 3 host compilers exist.
There appears to be an issue with the cmake command that gets ran when attempting to configure with an arm64 host. -T host=arm64 does not work as the only valid values are x86 or x64, so with just the changes to scan it won't work for building yet.

Can you check your hard drives to see if all 3 host compilers exist? and share the paths here?
(I can't see the paths from the screenshot)

Looking at the source code for CMake, it looks like ARM64 is all capitalized.  Can you try that instead of arm64?
https://github.com/Kitware/CMake/blob/af58b6d2ad9a000146e8db2b1c3c74b1a93456ef/Source/cmGlobalVisualStudio12Generator.cxx#L141

Looking at the source code for CMake, it looks like ARM64 is all capitalized (and case-sensitive). Can you try that instead of arm64?
https://github.com/Kitware/CMake/blob/af58b6d2ad9a000146e8db2b1c3c74b1a93456ef/Source/cmGlobalVisualStudio12Generator.cxx#L141

Thanks for digging into this @bobbrow ! I had to do another check to get it back to lower case (otherwise I was missing arm64 as both host and target), but it appears to be working and generate this as my cmake-tools-kits.json:
[
  {
    "name": "Clang 15.0.1 (GNU CLI) for MSVC 17.5.33103.201 (Visual Studio Enterprise 2022 Preview - amd64)",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x64",
    "compilers": {
      "C": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\x64\\bin\\clang.exe",
      "CXX": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\x64\\bin\\clang.exe"
    }
  },
  {
    "name": "Clang 15.0.1 (GNU CLI) for MSVC 17.5.33103.201 (Visual Studio Enterprise 2022 Preview - x86)",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x86",
    "compilers": {
      "C": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\bin\\clang.exe",
      "CXX": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\bin\\clang.exe"
    }
  },
  {
    "name": "Clang 15.0.1 (MSVC CLI) for MSVC 17.5.33103.201 (Visual Studio Enterprise 2022 Preview - amd64)",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "x64",
      "toolset": "ClangCL,host=x64"
    },
    "compilers": {
      "C": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\x64\\bin\\clang-cl.exe",
      "CXX": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\x64\\bin\\clang-cl.exe"
    }
  },
  {
    "name": "Clang 15.0.1 (MSVC CLI) for MSVC 17.5.33103.201 (Visual Studio Enterprise 2022 Preview - x86)",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x86",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "win32",
      "toolset": "ClangCL,host=x86"
    },
    "compilers": {
      "C": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\bin\\clang-cl.exe",
      "CXX": "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Tools\\Llvm\\bin\\clang-cl.exe"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - amd64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "x64",
      "toolset": "host=x64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - amd64_arm64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "arm64",
      "toolset": "host=x64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - amd64_x86",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "win32",
      "toolset": "host=x64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - arm64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "ARM64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "arm64",
      "toolset": "host=ARM64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - arm64_amd64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "ARM64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "x64",
      "toolset": "host=ARM64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - arm64_x86",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "ARM64",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "win32",
      "toolset": "host=ARM64"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - x86",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x86",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "win32",
      "toolset": "host=x86"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - x86_amd64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x86",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "x64",
      "toolset": "host=x86"
    }
  },
  {
    "name": "Visual Studio Enterprise 2022 Preview - x86_arm64",
    "visualStudio": "f64f9d0b",
    "visualStudioArchitecture": "x86",
    "preferredGenerator": {
      "name": "Visual Studio 17 2022",
      "platform": "arm64",
      "toolset": "host=x86"
    }
  }
]

Ah, the test failure appears to be because arm64 as a host was added in Visual Studio 2022 17.4 and running against 2019 where arm64 is an invalid option.

The test appears to be failing for something else, but there are a lot of errors printing out during Kit scanning that should be addressed before we take this PR.  It should probably be addressed in tryCreateNewVCEnvironment in kit.ts.  It has access to the VSInstallation info, so I think you need to do a version check there before treating ARM64 as a valid host arch.

The test appears to be failing for something else, but there are a lot of errors printing out during Kit scanning that should be addressed before we take this PR. It should probably be addressed in tryCreateNewVCEnvironment in kit.ts. It has access to the VSInstallation info, so I think you need to do a version check there before treating ARM64 as a valid host arch.

The commit you just pushed works too, though I'd like to see if we can get a more accurate version number to test against.  I believe we have APIs for that.  Give me a minute.

The commit you just pushed works too, though I'd like to see if we can get a more accurate version number to test against. I believe we have APIs for that. Give me a minute.

There is a parseVersion function in util.ts but it expects a 3-part version number.  That function could be modified to accept a 2-part version number and then you could use it to get the version number instead of parseInt which is kind of a hack that was never cleaned up.


Deprecation of CUDA linux_ppc64le support

As of CUDA 12.4.0, linux_ppc64le support is deprecated as noted in the CUDA 12.4.0 release notes. Have copied the relevant snippet below:

1.4.1. Deprecated Architectures
CUDA Toolkit 12.4 deprecates NVIDIA CUDA support for the PowerPC architecture. Support for this architecture is considered deprecated and will be removed in an upcoming release.

This is happening as part of the CUDA 12.5.0 update: https://github.com/conda-forge/cuda-feedstock/issues/29


Determinate progress-bar breaks in text-align: right

Bug, feature request, or proposal:
Bug
What is the expected behavior?
Progress bar is not affected by text-align right context.
What is the current behavior?
Determinate progress bar display fails in text-align: right; context.
What are the steps to reproduce?
Place it inside any container that has text-alignment set to  right
Plunker
http://plnkr.co/edit/rJCIiDvu4ZcQhYjLOK99?p=preview
What is the use-case or motivation for changing an existing behavior?
It's easy enough to work around, but a block-level element shouldn't break on text-alignment.
Which versions of Angular, Material, OS, browsers are affected?
Windows, Chrome, Angular2 RC 4/5/6, Material alpha.6 - alpha.7.4
Is there anything else we should know?
Nope. Thanks for the great work!

😃


Generate Normals

If mesh doesn't contain normals then they must be generated if this option is not disabled.
This is optional for users. Default behavior is generate normals if needed.

implemented with https://github.com/recp/assetkit/commit/750aed3a4bc996f3aaef316ad48fc2edf02f1319


shellsort.java

Related Issue
-#404

sorry I have not added the checklist because I am new to opensource contribution .next I will make pull requests perfectly.I hope you understand

It would have been better if you followed guidelines before contributing if you are new to open source .
Implementation of Shell Sort #329 duplicate
Java not welcomed , thanks


Set content length for requests

The project has moved to https://github.com/rakyll/hey. Closing the PRs on this legacy repo.


c_unittests failed with LeakSanitizer leak in qdr_core_subscribe

https://github.com/skupperproject/skupper-router/actions/runs/4392581183/jobs/7692426562#step:30:2791
68: ==4651==ERROR: LeakSanitizer: detected memory leaks
68:
68: Direct leak of 56 byte(s) in 1 object(s) allocated from:
68:     #0 0x7faba71766af in __interceptor_malloc (/lib64/libasan.so.8+0xba6af)
68:     #1 0x8fe9bd in qd_malloc /home/runner/work/skupper-router/skupper-router/skupper-router/include/qpid/dispatch/ctools.h:225
68:     #2 0x8fe9bd in qdr_core_subscribe /home/runner/work/skupper-router/skupper-router/skupper-router/src/router_core/route_tables.c:147
68:     #3 0x7ec280 in _vflow_init_address_watch_TH /home/runner/work/skupper-router/skupper-router/skupper-router/src/vanflow.c:1531
68:     #4 0x7dc7c2 in _vflow_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/vanflow.c:1216
68:     #5 0x7faba583912c in start_thread (/lib64/libc.so.6+0x8b12c)
68:
68: SUMMARY: AddressSanitizer: 56 byte(s) leaked in 1 allocation(s).
24/34 Test #68: c_unittests ..........................***Failed   25.77 sec

https://github.com/skupperproject/skupper-router/blob/f5078c52a52b0b1445c11b4bc5f5891309c3d1c0/src/vanflow.c#L1591
https://github.com/skupperproject/skupper-router/blob/f5078c52a52b0b1445c11b4bc5f5891309c3d1c0/src/vanflow.c#L1209-L1226
https://github.com/skupperproject/skupper-router/blob/f5078c52a52b0b1445c11b4bc5f5891309c3d1c0/src/vanflow.c#L1513-L1533
https://github.com/skupperproject/skupper-router/blob/f5078c52a52b0b1445c11b4bc5f5891309c3d1c0/src/router_core/route_tables.c#L138-L153
I am guessing this is an unfortunate shutdown leak, possibly exacerbated by whatever c_unittests is doing.

I tried investigating and I don't see how could this possibly be leaking (in a flaky manner, in this case). The subscription should be always freed, even when discarding.

https://app.travis-ci.com/github/skupperproject/skupper-router/jobs/598878852#L8177
72: Direct leak of 56 byte(s) in 1 object(s) allocated from:
817872:     #0 0x4d2668 in malloc (/home/travis/build/skupperproject/skupper-router/build/tests/c_unittests/c_unittests+0x4d2668)
817972:     #1 0x8bdff0 in qd_malloc /home/travis/build/skupperproject/skupper-router/include/qpid/dispatch/ctools.h:225:17
818072:     #2 0x8bdff0 in qdr_core_subscribe /home/travis/build/skupperproject/skupper-router/src/router_core/route_tables.c:147:31
818172:     #3 0x7efbc8 in _vflow_init_address_watch_TH /home/travis/build/skupperproject/skupper-router/src/vanflow.c:1531:39
818272:     #4 0x7ef25c in _vflow_thread /home/travis/build/skupperproject/skupper-router/src/vanflow.c:1216:17
818372:     #5 0xffff9ca31620 in start_thread (/lib/aarch64-linux-gnu/libpthread.so.0+0x7620)
818472:     #6 0xffff9bfe0498  (/lib/aarch64-linux-gnu/libc.so.6+0xd1498)

Here's it on clang https://github.com/jiridanek/skupper-router/actions/runs/4566236288/jobs/8058456155#step:32:2852

https://github.com/skupperproject/skupper-router/actions/runs/5280672516/jobs/9553224150?pr=1131#step:10:1789

This looks like the same failure,
73: =================================================================
73: ==3187==ERROR: LeakSanitizer: detected memory leaks
73:
73: Direct leak of 56 byte(s) in 1 object(s) allocated from:
73:     #0 0x7f5d728defdf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
73:     #1 0x55d73ffe0d85 in qd_malloc ../include/qpid/dispatch/ctools.h:225
73:     #2 0x55d73ffe1b49 in qdr_core_subscribe ../src/router_core/route_tables.c:147
73:     #3 0x55d73ff180c1 in _vflow_init_address_watch_TH ../src/vanflow.c:1686
73:     #4 0x55d73ff144bf in _vflow_thread_TH ../src/vanflow.c:1315
73:     #5 0x55d73fef381e in _thread_init ../src/posix/threading.c:206
73:     #6 0x7f5d70c94b42 in start_thread nptl/pthread_create.c:442
73:
73: SUMMARY: AddressSanitizer: 56 byte(s) leaked in 1 allocation(s).

https://github.com/skupperproject/skupper-router/actions/runs/6338420921/job/17215601909?pr=1241#step:10:2583
This is from one of those
Error: Process completed with exit code 143.

Ubuntu runs, so it might not be a reliable message.

Here's it again, https://github.com/skupperproject/skupper-router/actions/runs/6597463058/job/17925042549?pr=1268#step:10:4621
I will take a look soon, since this is popping up with low frequency, but it is persistent.


Set opacity for images and texts

Is there any way to set opacity for images and texts using jsPDF? I have gone through the documentation I couldn't find any settings here https://mrrio.github.io/jsPDF/doc/symbols/jsPDF.html.

Yes +1
I tried:
new Image().style.opacity = '.5';
but it does not works.

Can this be re-opened?  It is still relevant for the purpose of adding a watermark over the page.
In my case, I want to add "DRAFT" in large letters with like 25% opacity over the entire page.

~Can this be re-opened? It is still relevant for the purpose of adding a watermark over the page.~
~In my case, I want to add "DRAFT" in large letters with like 25% opacity over the entire page.~
Actually, I think I figured how to do it for what I needed. The "trick" is using setGState() to set opacity. Admittedly, I'm not quite sure how GState works in jsPDF and the docs are a little thin on this topic, but the following code is what worked for me:
for (let i = 1; i <= pageCount; i++) {
    doc.setPage(i)
    doc.saveGraphicsState();
    doc.setGState(new doc.GState({opacity: 0.2}));
    doc.text('DRAFT', x, y, {align: 'center', baseline: 'middle'})
    doc.restoreGraphicsState();
}

This comment is what told me opacity was supported using GState.

Works perfectly, thanks!

~Can this be re-opened? It is still relevant for the purpose of adding a watermark over the page.~
~In my case, I want to add "DRAFT" in large letters with like 25% opacity over the entire page.~
Actually, I think I figured how to do it for what I needed. The "trick" is using setGState() to set opacity. Admittedly, I'm not quite sure how GState works in jsPDF and the docs are a little thin on this topic, but the following code is what worked for me:
for (let i = 1; i <= pageCount; i++) {
    doc.setPage(i)
    doc.saveGraphicsState();
    doc.setGState(new doc.GState({opacity: 0.2}));
    doc.text('DRAFT', x, y, {align: 'center', baseline: 'middle'})
    doc.restoreGraphicsState();
}

This comment is what told me opacity was supported using GState.

Brilliant !!!
I has looking for it for watermark feature. It works as expected.


Double precision is a real problem

Any work around?
Tracing the issue it appears that inv(x'*x) in MathNet is off vs. Matlab's version. I tried every each way and function MathNet results come out the same way. In fact, Excel's Linest generates exactly the same result as MathNet.

What exactly are you trying to do?
Matlab, Math.NET Numerics and Excel all use double precision. In fact almost all numerical software does. I'm not aware this has prevented any of them from being used in commercial software :)

I am doing a simple linear regression y=k*x+b  with intercept true. I took the slide from the internal circulation when we discussed this issue. Some regressions come out correct, while others are not - whatever Matlab is doing makes a difference.- hence a question about a work around.

What routines are you using in Matlab and Math.NET Numerics?

(sorry, clicked the wrong button)

May I ask what routines you use, both in Matlab and Math.NET Numerics?

Thanks for a quick response.
Data: x-input, y-output - both are vectors
Matlab: x = [ones(N,1),x]; b=x/y or b=inv(x'x)(x'*y) both generate the same answer
MathNet: append a column of 1's to x call it oneX (oneX.Transpose() * oneX).Inverse() * (oneX.Transpose() * y); or use MultipleRegression.everymethod(x,y,true) - all generate the same answer that is not equal Matlab's version.

For any of those cases where there are different results, how does SimpleRegression.Fit(x,y) compare to them?

No difference in results across platforms or functions. Excel generates the exact same answer as MathNet.

In Math.NET Numerics, do you use a native provider like MKL or just the managed one?
If Excel, Math.NET MultipleRegression and SimpleRegression (which are implemented very differently) all agree with each other, the odd one seems to be Matlab. Are you absolute sure you pass exactly the same data to these routines, with the same precision?

Not sure - I used nuget and whatever came along with it.
In Excel I used a native linest function and do not have MathNet installed - I only used it to confirm I am comparing apples to apples and as a sanity check.

See http://numerics.mathdotnet.com/MKL.html. With the MKL provider enabled you get practically the same implementation that Matlab is using (on MultipleRegression.XY).

Thanks I will check it out. Any performance metrics for it for comparison?

Got an exception from Matrix: "Matrices and vectors of type 'Decimal' are not supported. Only Double, Single, Complex or Complex32 are supported at this point."
Might as well say intended for children ages 6-12 only.
I have never seen any serious math calculations working with double consistently. Not to be mean, but spent a lot of time integrating the lib that now needs to be ripped out by the looks of it..I would add Decimal asap.

Let me ignore the claim that apparently the entire scientific community and software industry around numerical methods is not doing "serious math calculations". I'd be very surprised if this is related to numerical precision at all. Even more so since Matlab is using double/extended precision as well.
Any chance you could provide a simple dataset where I could repro the issue?

This is thread is a bit confusing.
Are there supposed to be axis labels on the figure? And what is it supposed to show?
I agree with cdrnet that double is good enough for plenty of scientific and commercial applications. Of course you can devise scenarios where double precision is challenged and aberrant behavior occurs. It's a bit of a leap to go from that and claim that double is not sufficient for any calculation, or that you "might as well say intended for children ages 6-12 only".

I have never seen any serious math calculations working with double consistently. Not to be mean, but spent a lot of time integrating the lib that now needs to be ripped out by the looks of it..I would add Decimal asap.

Before you rip it out you may actually want to try and find another library which does support decimal. I don't know of any in .Net or C/C++ (or any other language for that matter). Decimals are not implemented in hardware so are very slow for doing a large number of calculations. They also have a lot smaller range.


fix import/include

dockstore/dockstore#3349
Need to add $ to fix tests for this PR https://github.com/dockstore/dockstore/pull/3794

FYI, this seems like it would insta-break other builds. So only merge after merging dockstore/dockstore#3794 and let other people know to update from develop

FYI, this seems like it would insta-break other builds. So only merge after merging dockstore/dockstore#3794 and let other people know to update from develop

I can only merge the other PR when this is merged, otherwise the tests will fail. Unless I consider $import and import valid like @coverbeck suggested, then I won't have to change repo. The other option is merging this to a new branch, and change the tests in my other PR to reference the new branch.
On another note, this cwl is invalid in general before and after adding $ to the imports.

FYI, this seems like it would insta-break other builds. So only merge after merging dockstore/dockstore#3794 and let other people know to update from develop

I can only merge the other PR when this is merged, otherwise the tests will fail. Unless I consider $import and import valid like @coverbeck suggested, then I won't have to change repo. The other option is merging this to a new branch, and change the tests in my other PR to reference the new branch.
On another note, this cwl is invalid in general before and after adding $ to the imports.

We're actually not supposed to use the master branch in tests, you should use a tagged version

Okay, I made a ticket dockstore/dockstore#3802. For now I made a new branch to merge this PR into and I'll make a tagged release off of that branch and use that in the tests. The tests that use it now all use master


Importmaps helpers conflicts with same file from other engine

Hey! I was trying to use mission_control-jobs with one of the apps I'm using, based on https://github.com/spree/spree . Sadly, Spree has the same importmaps module export as mission_control-jobs called helpers and that fact makes the one from spree not being loaded - it gets overwritten by the one from mission_control-jobs therefore my app frontend is broken.
Do you have any idea what might be done in order to deal with that?
Thanks!

Ohhh @mbajur sorry about that! What version of mission_control-jobs are you using?

It turned out updating it to most recent version fixed that for me :) Thank you!

Ahh, great! Thanks so much for confirming 🙏


Creating network with OpenStack hypervisor fails

Any attempts of running Beaker with the OpenStack hypervisor fails with the message "Unable to create OpenStack Network instance"

Fixed in #564


GlobalMarineNet.WeatherNet version 1.0

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] This PR only modifies one (1) manifest
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.4 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

Automatic Validation ended with:

You already know what I will say.

You already know what I will say.

As mentioned in other PRs, we don't have the ability to test all the various silent flags. Please perform the tests and either update the PR with the silent flags or ask that this be marked interactive only.

[Policy] Interactive-Only-Installer


Password popup, preshared key and dns option, some bug fixes

Hi,
I implemented some missing features and also fixed some bugs.
Sudo password popup - fixes https://github.com/DavidVentura/Wireguard_qml/issues/4

6dd567f7b63ea1c3759bed1b43cf1e065c8e7134
e5cda82472a04e755879b797eb205124b1e7d03a
I implemented a potential insecure way, of doing it, because the password get's saved in a qml property and could be leaked by logs for example. Also the password gets echoed to the subprocess commands and would be visible in the shell itself.
Nevertheless it works and it is the best I could come up with.

Preshared key and DNS - fixes https://github.com/DavidVentura/Wireguard_qml/issues/5

73c3457601a3f35af116544920fae5be1bc9e90d
95ba6e7af34bf6ebd0d1790a0fd36ce1baf7af71
The preshared key is straight forward, nothing special.
The DNS thing is more complicated and I am not sure, if this is the right way of doing it. Actually I thought this would be handled by the extra routes somehow, but that seems not to be the case.
I also looked at the debian manpage of wg-quick, how it is doing such things (https://manpages.debian.org/unstable/wireguard-tools/wg-quick.8.en.html), but the interface specific resolvconf command provided there, does not work out on UT.

Bug: the userspace implementation was always used, even if the kernelmodule was recognized

d2e0159f845314f881fae7b5fd18b17b26dfcd9a
8bbc34b6c49ea4569f12f24a9275c116f758ca28
Or at least, the daemon was always started. Then I saw in the code, that the daemon should not be used, if the kernel module is available.
The useUserspace QML setting, was at first never set in the PickProfilePage and second not updated, when switching back to the PickProfilePage. This is fixed now.

InterfaceName was not set

0b3e15003257a7d86fdde450026857b7840e1f66
Somehow the interfacename was not set correctly. I am not sure, if this issue was a result of my changes or only happaning with the  kernelmodule way... however, I reworked the way, how the name is set.
It depends now on the count and index of the interface-item in the listmodel. -> wg0, wg1, wg2, etc.

Other

added action to delete interfaces from the list f806e190fdc981edd24e96bc74ccd16b533aa562
fixed that the log got spammed, trying to get the inteface-status, if there was no interface yet 06a1c55f24e3c6dbf2d5cd9bf5c69b8d95303cfd
fixed text color for the traffic-text, if a dark theme was used a60dde988dd224654be5f2d0b1951a2f84878e3a
on saving a profile, qml tried to log an error, which did not occur a655d9f577db45a8d1664fa689feee18b3a44c8f

Looks very nice, I've been out of the ubuntu-phone circle for a while, I'll need to flash my PinePhone to test this.
For general routing: I am now convinced that this should use fwmark for  routing decisions, otherwise it's likely to end up with a routing loop or stale routing

Sorry for the long delay (
Thanks!


[ADD]: PACIFIC AIR BATTLE GAME (3D)

PR Description 📜

This PR is about adding the "Pacific Air Battle" Game, it's an action phased attacking game
In this game, you will take control of a fighter jet and engage in aerial combat against enemy jets.
You will need to use your skills and experience to dodge enemy fire and take down your targets.

Fixes #710
Fixes/Closes:

#710

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[x] I have added entries for my game in main README.md
[x] I have added README.md in my folder
[ ] I have added working video of the game in README.md (optional)
[x] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you @0xabdulkhalid ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

@0xabdulkhalid Plz mention * in place of a number in main README

@0xabdulkhalid Plz mention * in place of a number in main README

Done @Arun9739 !

@0xabdulkhalid Resolve merge conflicts

Conflicts has been resolved @Arun9739 !

Thank you  @0xabdulkhalid , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀

/assign


Allow passing extra build flags to podman/docker

/kind user-story
User Story
As an odo user running Mac with AppleSilicon, I want to be able to build images for different platforms (amd64) So that I can run them on my cluster is amd64
Acceptance Criteria

[x] odo deploy and odo build-images should check for ODO_IMAGE_BUILD_ARGS env variable and use it as additional arguments for podman/docker build
[x] odo run should check for ODO_CONTAINER_RUN_ARGS env variable and use it as additional arguments for podman/docker run
[ ] Tests
[ ] Documentation

ODO_IMAGE_BUILD_ARGS we should come up with a better name for this variable. As in the future, we will have in-cluster build where this variable won't make sense.
ODO_CONTAINER_RUN_ARGS come up with better name ;-)
example:
ODO_IMAGE_BUILD_ARGS="--platform linux/amd64" odo deploy should run podman build  --platform linux/amd64 ...
for discussion:
Or this could also be a cli flag for odo build-images and odo deploy
/kind user-story

This seems related to https://github.com/redhat-developer/odo/issues/6178

To better separate global options from those that are specific to a Podman/Docker sub-command, I have introduced an additional env var (ODO_CONTAINER_BACKEND_GLOBAL_ARGS), which will be passed as global flags to any Podman/Docker command executed by odo.
This will be needed for example if we want to pass global options like --root and --runroot to better isolate our integration tests on Podman.
ODO_IMAGE_BUILD_ARGS will be passed as flags specific to the podman build and docker build commands.
ODO_CONTAINER_RUN_ARGS will be passed as flags specific to the commands creating resources, like podman play kube when running odo against Podman.
So for example, running this:
$ ODO_IMAGE_BUILD_ARGS='--platform=linux/amd64;--build-arg=MY_ARG=my_value' \
  ODO_CONTAINER_BACKEND_GLOBAL_ARGS='--root=/tmp/podman/root;--storage-driver=overlay' \
  ODO_CONTAINER_RUN_ARGS='--configmap=cm-foo.yml;--quiet' \
  odo dev --platform=podman

will result in running the following commands:
podman \
  --root=/tmp/podman/root --storage-driver=overlay \
  build \
  --platform=linux/amd64 --build-arg=MY_ARG=my_value \
  ...

...
podman \
  --root=/tmp/podman/root --storage-driver=overlay \
  play kube \
  --configmap=cm-foo.yml --quiet
  ...

...
# Any other podman commands executed by odo under the hood
# will use the global args, e.g.:
podman \
  --root=/tmp/podman/root --storage-driver=overlay \
  pod list --format {{.Name}} --noheading

podman \
  --root=/tmp/podman/root --storage-driver=overlay \
  pod stop $podName


reproduce result that only uses soft prompt

Hi,
in the paper it is claimed that "We note that even though Stable Diffusion and CLIP share the same text encoder, soft prompts do not transfer well compared to all hard prompt methods in our evaluation".
How could I reproduce this soft prompt result with you code?  I guess I need to directly pass the soft prompt embedding to the Stable diffusion but not sure how to pass this to SD, as SD only support hard prompt as input. Even though the SD accepts prompt embedding as input, the format of this prompt embedding is different from the one you optimized.
Thanks in advance for any guidance.

Hi, thanks for reaching out.
You can pass the soft prompt as prompt_embeds.
For the soft prompt optimization, you can simply don't do the forward projection: https://github.com/YuxinWenRick/hard-prompts-made-easy/blob/main/optim_utils.py#L163-L164, and do projected_embeds = prompt_embeds, and at the end, you can return prompt_embeds as the final soft prompt.
Let me know if you have further questions!

Hi, I think you are right. You need to pass it to the text encoder first. That's a nice catch.
I double-checked the code I implemented for the soft prompt experiments. Actually, what we did was to train a soft prompt and then project it to the nearest neighbor as a hard prompt. The diffusers version (v0.11.0) we were using actually didn't support prompt_embeds at that moment: https://github.com/huggingface/diffusers/blob/v0.11.0/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L409-L424.
Meanwhile, we mentioned this in our paper:
, as we focus on discrete prompt optimization.
However, I am curious if the real soft prompts work. Have you tried it using the method you mentioned?

Hi, I think you are right. You need to pass it to the text encoder first. That's a nice catch.
I double-checked the code I implemented for the soft prompt experiments. Actually, what we did was to train a soft prompt and then project it to the nearest neighbor as a hard prompt. The diffusers version (v0.11.0) we were using actually didn't support prompt_embeds at that moment: https://github.com/huggingface/diffusers/blob/v0.11.0/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L409-L424.
Meanwhile, we mentioned this in our paper: , as we focus on discrete prompt optimization.
However, I am curious if the real soft prompts work. Have you tried it using the method you mentioned?

Thanks for the responsible double-check!
I did try the method, but I found the prompt embedding that the stable diffusion accept is still different from that processed by the method I mentioned, and can not find the reason yet. I leave it as a bug for now and continue with others. Anyway, thanks for the clarity!


New coreax release v0.2.2 (or v0.3)

The new version should not introduce any breaking changes, i.e. that would break existing functionality for users. Any changes that might eventually lead to breaking functionality should be marked as deprecated.

Blocked by #764

As a reminder, we have some deprecated features marked for removal at v0.3. It would be nice to release v0.3 once #504 is merged, which will hopefully be before next week. I think this would provide a significant enough change to justify v0.3 over v0.2.2.

Blocked by #779.


Add notebook tests

Description
Using papermill library to execute the notebooks and check the necessary ouputs.
Tryng to fix #122
Type of change
Please tick all options which are relevant.

[ ] Bug fix
[x] New feature
[x] Added/removed dependencies
[ ] Required documentation updates

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.05%. Comparing base (edd830f) to head (dee154f).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #142   +/-   ##
=======================================
  Coverage   82.05%   82.05%
=======================================
  Files          17       17
  Lines         613      613
  Branches       59       59
=======================================
  Hits          503      503
  Misses        110      110

Is this pull request still struggling to show a good user message if a notebook execution ends up with unexpected results or we are seeing something else?


custom newtonsoft dll collides with regular newtonsoft dll, causing hard-to-find errors

Our newtonsoft json dll is different from the regular one to work with il2cpp. If a project has the wrong dll included, the project compiles and loads fine, but has runtime errors, on il2cpp devices only.
To avoid this, we should either

change our newtonsoft json dll to have a different filename (newtonsoft.mre?) and a different namespace, which would trigger compile errors if failing, OR
fix so we don't need a custom newtonsoft json.

I personally lean towards 1, to avoid problems if other people customize their newtonsoft for other reasons

I had suggested this a while back, but I forget why we decided not to do this.  I do know that this easiest course of action would be to change the name after building the project, because the build scripts that are used to generate the various binaries is not very receptive to massaging in any fashion.  I would recommend choice 1 for sure.


Implement a YugaByteDB datastore

Creating this issue to gauge interest.

@jzelinskie Seems almost 10 person could be interested in the Yugabyte Datastore.
As a test I try to run the postgresql test against YugabyteDB but face two issue :
ALTER TABLE namespace_config ADD COLUMN id BIGSERIAL PRIMARY KEY failed because YDB do not support adding another primary key yet because the sharding is based on the primary key but i guess that could be solve with a new migration script.
Once fixing the previous issue I am getting the following error as well
?       github.com/authzed/spicedb/internal/datastore/postgres/common   [no test files]
?       github.com/authzed/spicedb/internal/datastore/postgres/migrations       [no test files]
?       github.com/authzed/spicedb/internal/datastore/postgres/version  [no test files]
--- FAIL: TestPostgresDatastoreWithoutCommitTimestamps (0.00s)
    --- FAIL: TestPostgresDatastoreWithoutCommitTimestamps/postgres-13.8 (23.89s)
        postgres.go:113:
                Error Trace:    /Users/gwenn/work/github/spicedb/internal/testserver/datastore/postgres.go:113
                                                        /Users/gwenn/work/github/spicedb/internal/datastore/postgres/postgres_test.go:200
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/datastore.go:33
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/namespace.go:38
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/datastore.go:72
                Error:          Received unexpected error:
                                unable to compute head revision: multiple or zero head revisions found: [add-rel-by-alive-resource-relation-subject add-unique-datastore-id]
                Test:           TestPostgresDatastoreWithoutCommitTimestamps/postgres-13.8
        --- FAIL: TestPostgresDatastoreWithoutCommitTimestamps/postgres-13.8/TestNamespaceNotFound (5.08s)
            testing.go:1490: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
--- FAIL: TestPostgresDatastore (0.00s)
    --- FAIL: TestPostgresDatastore/postgres-13.8-head- (26.66s)
        postgres.go:113:
                Error Trace:    /Users/gwenn/work/github/spicedb/internal/testserver/datastore/postgres.go:113
                                                        /Users/gwenn/work/github/spicedb/internal/datastore/postgres/postgres_test.go:75
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/datastore.go:33
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/namespace.go:38
                                                        /Users/gwenn/work/github/spicedb/pkg/datastore/test/datastore.go:72
                Error:          Received unexpected error:
                                unable to compute head revision: multiple or zero head revisions found: [add-rel-by-alive-resource-relation-subject add-unique-datastore-id]
                Test:           TestPostgresDatastore/postgres-13.8-head-
        --- FAIL: TestPostgresDatastore/postgres-13.8-head-/TestNamespaceNotFound (5.28s)
            testing.go:1490: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
FAIL
FAIL    github.com/authzed/spicedb/internal/datastore/postgres  27.739s
FAIL
Error: running "go ./internal/datastore/postgres/..." failed with exit code 1
exit status 1

For that on I am not sure what the issue is.
I think creating a Ydb datastore should be that hard, as the compatibility with PG is high.

@shinji62 We don't think implementing Yugabyte would be difficult, however, adding a new datastore needs to be carefully analyzed. Each datastore has its own quirks and limitations, and evolving SpiceDB with N datastores underlying it adds considerable engineering and maintenance overhead. Just deciphering the mysteries of each database query planner involves a non-trivial amount of time, making sure performance does not regress as the service on top evolves, migrations, testing with multiple versions, accounting for the specifics of each datastore client configuration, tunning...
It's not difficult, but it does not come for free.

@vroldanbet First an apology, I wasn't trying to under estimate the work, I fully agree with what you say,  if does not come for free, for the "client configuration / tunning and so on", I would be more than happy to help.
Thanks for the quick answer.

@shinji62 no need to apologize! It was a good opportunity to shed some light on what it takes to add a new datastore. We mantainers haven't done a good job at clarifying what it takes to support a new database technology in SpiceDB.
We are certainly keeping an eye on database technologies that align with SpiceDB requirements (strong consistency, global distribution, horizontal scalability and exposed MVCC semantics). I'm not familiar with how Yugabyte supports these, do you have some insight?

@vroldanbet let me respond to your question
Overall YugabyteDB is a Distributed SQL database which provide 2 API one been a Postgresql compatible (almost 100%, we use the psql codebase) and one cassandra compatible in top of a distributed storage which handle automatic sharding, distributed transaction and so on.., Ydb is a CP database with strong HA.
strong consistency:  This is one of the core, are we are CP Db we do have strong consistency and transactional support for the Postgresql API and the Cassandra one.
global distribution: Not sure what you mean by global distribution, but we do have geo replication and geo distribution, for example pinning certain data to region for compliance for example.
horizontal scalability:  Ydb is share nothing and distributed DB so scale-out is one of the core feature as well
exposed MVCC semantics: Not sure what you mean by that.

@shinji62

strong consistency: This is one of the core, are we are CP Db we do have strong consistency and transactional support for the Postgresql API and the Cassandra one.

Sounds good. Just to make sure we are talking about the same thing because those terms tend to be overloaded: I'm referring to strict serializability isolation level and external consistency. I make emphasis because SpiceDB requires this, and for example, CockroachDB, being an open-source implementation of Spanner, has some caveats on their isolation/consistency guarantees that are problematic for SpiceDB.

global distribution: Not sure what you mean by global distribution, but we do have geo replication and geo distribution, for example pinning certain data to region for compliance for example.

My apologies for using such a vague term. Geo replication and geo distribution does not necessarily describe with precision how the system behaves in light of writes distributed across the globe.
SpiceDB requires a database capable of providing the strongest levels of isolation and consistency when distributed globally. It means providing the same strong guarantees but with multiple reads and writes distributed worldwide. Single-node architectures like Postgres/MySQL are out of the equation here, and if my recollection of Cassandra is correct, reaching consensus of N nodes distributed around the world will likely be very slow, but I'm not sure that's how YugaByte works. Not that Spanner and CockroachDB are blazing fast in this regard, but they are designed with that use-case in mind.

exposed MVCC semantics: Not sure what you mean by that.

The database should be able to return query results at a given snapshot. This is fundamental to SpiceDB's bounded staleness, which is the trick to make it scale. While this is not a _hard requirement, the bookkeeping needed to layer a snapshotting system on top of the database is additional overhead.

Thanks @vroldanbet
Yugabyte support the same isolation as PG. Now for the external consistency or I guess Linearizability we do support for single-row but  for not for multi-row transactions which support three isolation levels: Serializable, Snapshot (also known as repeatable read), and Read Committed isolation.

Reaching consensus of N nodes distributed around the world will likely be very slow, but I'm not sure that's how YugaByte works. Not that Spanner and CockroachDB are blazing fast in this regard, but they are designed with that use-case in mind.

I think that quite similar to cockroach or spanner, mostly depends on the latency between the node.

Thanks for the info @shinji62 🙏🏻

Would also like to see this supported. We were looking to use Authzed with Yugabyte as the database provider.

Hi team, are we planning on adding Yugabyte support for SpiceDB? We are also looking to use Authzed with Yugabyte as the database provider

@sameer-m-dev there are no plans to add yugabyte support, and I think it wouldn't be feasible without multi-row linearizable transactions.

Hi, developer advocate for YugabyteDB here, trying to understand the consistency requirement.

"The database should be able to return query results at a given snapshot": YugabyteDB fully supports this through MVCC and SQL isolation levels

"multi-row linearizable transactions": YugabyteDB guarantees linearizability for single-row transactions. However, it doesn't provide "strong serializability" aka what Spanner calls "external consistency", for transactions that haven't exchanged messages between the concerned nodes. This is due to clock skew, where physical clock skew can occur, and logical clocks are synchronized only when exchanging messages (lamport). It is more a hardware limitation. We cannot do the same as Spanner (wait for may clock skew for each write) without atomic clocks.

The anomaly can happen when A and B run transactions without any objects in common, and A and B also communicate outside of the database to know which one committed before - this may be different from the commit order in the database. Is that a problem for SpiceDB? If it is, we have to wait that atomic clocks are more popular

Hey @FranckPachot, thanks for chiming.
Yes, external consistency is a requirement for SpiceDB, which follows the Zanzibar paper which was built on top of Spanner. Changes to unrelated parts of the graph can impact how results are computed, and thus wouldn't guarantee the system addresses what the Zanzibar paper refers to as the "new enemy problem". We've engineered the implementation of SpiceDB on top of each datastore taking into account which consistency guarantees does the database offer.
FYI CockroachDB does support external consistency, and they do it without atomic clocks, so I don't think atomic clocks are a prerequisite - they certainly help though. CockroachDB's implementation is limited though to rows in the same range, so we had to come up with some clever tricks to workaround it.

Ok, thanks. The tricks for CockroachDB should work the same for YugabyteDB. "Same range" means single-shard.

@FranckPachot does that mean that Yugabyte supports external consistency for one or more rows as long as they fall under the same shard? (Which makes sense)

@vroldanbet, I checked with colleagues to be sure. Yes, it doesn't violate external consistency if they fall into the same shard. And falling into the same shard is guaranteed with hash sharding. For example with
create table t ( a int, b int, c int, primary key ( a hash, b asc) );

all rows with the same values for a will go into the same shard.


feat: add fortio/fortio

#5564 fortio/fortio: Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats
$ aqua g -i fortio/fortio

Thanks.

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.32.0