swift4: Fixes for closure arguments

Fix destructuring

Use correct number of arguments in closures

Compatible with swift 3 & 4

@swift-ci test

I'm not sure I understand what was wrong with the previous syntax. Why did this become a problem?

There were 2 sets of errors:

The closure argument count needs to match the signature, eg  Contextual closure type '(UnsafeMutableRawPointer, Int) -> Void' expects 2 arguments, but 1 was used in closure body and Contextual type for closure argument list expects 1 argument, which cannot be implicitly ignored

Some cases for { (key, value)  needed to be replaced with { (arg) ; let (key, value) = arg to solve the following:
Closure tuple parameter '(Notification, [RunLoopMode])' does not support destructuring

Most of the fixes were from fix-its in Xcode

Just yesterday, the error message "..does not support destructuring" occurred to me in some places in one of my projects. The Xcode fix-it has helped, but I would also like to know why Swift 4 is different in this case than Swift 3.

Its due to SE-110 https://github.com/apple/swift-evolution/blob/master/proposals/0110-distingish-single-tuple-arg.md
I think it made an incorrect argument count an  error in Swift4

Is there an alternative to effectively re-declaring the argument all over the place? This can't be the pattern we really want people to use.

I sent a message to the swift evolution mailing list about this. I think we should have a further discussion before we accept this; it seems like a dramatic reduction in usability of the language in general.

Yeah thats a good idea. It does seem quite a source breaking change given how much closures are used in general

It seems like most of those are derived from problems in they way functions in the Swift Standard Library are defined, for example public func forEach(_ body: ((key: Key, value: Value)) throws -> Void) rethrows expects a function that receives a tuple, whereas if it was written as func forEach(_ body: (_ key: Key, _ value: Value) throws -> Void), it would expect a function that receives two params.

Rebased. I added an extra commit with some tidy ups of the closure arguments although of course these are subject to personal taste more than anything.
@parkera Is this ok for merging now? I noticed most of the SE-110 discussion seems to have ended and any possible changes to the parser will be post swift4 now.

I don't think that's actually the case, or at least I strongly hope it's not, so I still want to hold off on this.

Closing for now, this is not needed as the SE-110 change is going to be backed out. See https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170619/037616.html
If any individual closures still generate warnings after the change a new PR can be opened.


Add a prefix to URL patterns?

What should I do to run pyspider at http://localhost:5000/pyspider/ ?

You can use other HTTP servers to rewrite the urls to localhost:5000 such as Nginx, Apache, Caddy.

Hi,@WuKong96
Should I modify the html file after doing that?


[V2 Scheduler] Re-design and develop the runOnce feature

🚀 Feature Request
Relevant Package

 This feature request is for Scheduler
Description
In the Interval object, there is a field called runOnce, and the purpose of it is to make sure the related IntervalAction runs only once.
However, this design would cause the following issues:

Interval runOnce feature can not trigger with new action. https://github.com/edgexfoundry/edgex-go/issues/3362
After the service restarted, all the runOnce IntervalAction will be triggered again.

Describe the solution you'd like
TBD, we need more user feedback
Describe alternatives you've considered

Move runOnce field from Interval to IntervalAction, because it is hard to determine which IntervalAction should be triggered only once if it is in Interval.  Especially, Interval and IntervalAction are updated (we have Patch API for both of them).
If runOnce in an IntervalAction is true, update its adminState to LOCKED after run it once.  This could prevent the IntervalAction from being re-triggered after the service restarted.

Discussed in Core WG on 6/3/21.  Decision:  we need to fix this the right way in a future EdgeX release.  For now, we are removing runOnce (for Ireland) and will relook at a later time or as a use case manifests.


[libde265]+Build+error

Host Environment

OS: [e.g. Windows/Linux etc...]
Compiler: revision

To Reproduce
Steps to reproduce the behavior:
./vcpkg install xxxx
Failure logs

Cut and paste the appropriate build messages from the console output.
Attach every failure log file mentioned in the console output.

Additional context
Add any other context about the problem here, such as what you have already tried to resolve the issue.

Please describe your issue in detail.

We don't have enough information to solve this issue, unfortunately; if you have any more information to help us solve this, please reopen!


[Model] Adding Granite model.

Adds Granite model class to vLLM.
Model will be on Hugging Face once https://github.com/huggingface/transformers/pull/31502 is merged.

Thanks @shawntan! Could you rebase on the latest main branch, and add a test in https://github.com/vllm-project/vllm/tree/main/tests/models similar to the other model architectures?

@shawntan I have moved this to draft since it currently depends on a future version of the transformers library.
How about including the GraniteConfig class in its own file here in the meantime? Then we can clean that up later once vLLM moves to the necessary transformers version.

@shawntan to avoid duplication / maintenance overhead would it make more sense to just add the optional multipliers to llama.py?

@njhill HF PR merged.


Reference mode controls

Add a separate control panel for different modes of reference calculation and other referencing parameters.

added to master_beta


Docs Revamp: Spaces documentation

This PR migrates and updates the Spaces content so that everything is contained under the following hierarchy:

Getting Started
SDKs
Reference
Advanced Usage
Changelog

Let me know if any of the content looks like it could be filled out some more, or if pages could be split into smaller sections. (Also if there's anything that's missing!)

making it more hands-on and wrapping it in a narrative like "Create your own Hot dog/not hot dog" app could make it more interesting

Good call! I'll change Spaces setup to Creating your first Space and write it up as a short guide to making a Gradio space.


Add Distance operation implementation

Adds file distance.go:

Adds Distance functionality described by #17

Closes #17

Thank  you @sathuhebbar !


componentWillReceiveProps needed.

Hi,
i try to use your component, but you forget add componentWillReceiveProps method.
Because when i set checked state from my form, it still going with own current state. For checked props you can add;
componentWillReceiveProps(nextProps) {
        this.setState({isChecked: nextProps.checked})
}

Happy Coding.

Works, but in my case the state command set like this:
this.setState({isChecked: nextProps.isChecked})

yes, my code just a sample.

thank you ,it works for me.

Thank you soooo much.I've been stuck here for quite a long time...

Guys you can use https://github.com/DavitVosk/react-native-check-box.git fork


chore: refactor pageComponents/signature-mint, add tests

What changed? Why?

refactor pageComponents/signature-mint
add tests

Notes to reviewers
How has it been tested?
Unit tests

Btw for general context, happy to review and merge this PR, but we are going to deprecate the signature mint page soon.
The idea is that we should have just ONE page for minting, and that's it.
cc @eragon512 @arsood

Thanks for the context, I'll go ahead and close this PR since this is being deprecated 👍


Add Daily LST from METEOSWISS

Is your feature request related to a problem? Please describe.
Since we need daily LST from  CM-SAF / Meteoswiss for the VHI we need to upload them daily to GEE
*Steps

[x] Assets: Check if assets for a specific date range exist
[x] netcdf files: download data from data.geo.admin.ch location , check if file exist
[x] extract rename copy properteis for specific layer
[x] upload to gcs
[x] upload to GEE with properties

Additional context
VHI Product step1 has to log error when files does not exist

fixed with https://github.com/swisstopo/topo-satromo/tree/dev-20240517 and https://github.com/swisstopo/topo-satromo/commit/789ef4048d07d0b31814d5bc1d6f5787c8b44b22
@Tschoun : you can test it : projects/satromo-int/assets/LST_SWISS are here for the first five days of  APRIL 2024


2.0.6

Added a script variable and code for ignoring applications installed in the /Users/* hierarchy.
When App-Auto-Update finds an app that is installed in /Users/* and asks Installomator to update it, the machine generally ends up with two copies of the application. The current version installed in the /Applications folder and the downlevel version still install in the original /Users/* location. This gives the ability to simply ignore those apps.

Thanks @gilburns, closing #49 (Ignore applications within /Users/* directory)


Google travel time improvements

Description:
Big improvements to the Google Travel Time sensor.

Allow passing any options that Google supports in the options dict of your configuration.
Deprecate travel_mode.
Change name format to show the mode
Support providing departure_time and arrival_time as simple timestamps (i.e. 08:00:00 for departure times at 8am that morning)

Related issue (if applicable): #2024
Example entry for configuration.yaml (if applicable):
sensor:
  platform: google_travel_time
  api_key: API_KEY
  origin: 37.8156380,-122.2628050
  destination: 37.7968520,-122.4055500
  options:
    mode: transit
    transit_mode: rail
    transit_routing_preference: less_walking

Checklist:
If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

🐬

Looks great.
Thanks for the improvement

TODO: Need to write better docs for the website once this is merged.

@Danielhiversen I don't want to set any defaults on any of the items under options because they are subject to change at any time, and a lot of the items have multiple data formats.

Ok, but the you should not default to "driving" on line 85

@Danielhiversen I'm only setting Driving as the default for the title, because driving is the default that Google sets whenever a mode is not presented. Driving is not actually sent to Google.

Docs PR @ home-assistant/home-assistant.io#467

🐬

options are not actually optional:
16-05-13 10:56:13 homeassistant.components.sensor: Error while setting up platform google_travel_time
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/entity_component.py", line 109, in _setup_platform
    discovery_info)
  File "/usr/src/app/homeassistant/components/sensor/google_travel_time.py", line 83, in setup_platform
    if options.get('units') is None:
AttributeError: 'NoneType' object has no attribute 'get'

@nkgilley That's.... weird. I'll take a look tonight hopefully.

You should set default=dict() for CONF_OPTIONS

@nkgilley Should be fixed in latest dev: #2069


Improve FormBuilder API and added HeaderField and LiteralField

Recreate pull request from #5880 to force travis to rebuild

See https://travis-ci.org/open-sausages/silverstripe-framework/builds/152528128 for CI build.


@JuliaRegistrator register()

@JuliaRegistrator register()

Registration pull request created: JuliaRegistries/General/121612
Tip: Release Notes
Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.
@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.
Tagging
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v1.2.3 -m "<description of version>" c4d7d3282297300d668bda01092a844e3afe3974
git push origin v1.2.3


onBubbleTap not working when app goes to background

onBubbleTap: () async {
print("object");
},
this call is not working when the app is in the background

onBubbleTap: () async { print("object"); }, this call is not working when the app is in the background

await DashBubble.instance.startBubble(
onBubbleTap:
() async {
print("object");
await LaunchApp
.isAppInstalled(
androidPackageName:
'com.example.app');
},
options: BubbleOptions(
keepAliveWhenAppExit:
true));
Trying to open the same app

Hello Abdur Rahman 👋🏻
The onBubbleTap callback is tested in the foreground and background and should work on both cases, kindly try the Example Project and see if you can reproduce the same issue.
Also you can share me a minimal reproducible example for the issue. 🔁
Thanks 💙

Hello @moazelsawaf
I just tried the example project you quoted, Expected behavior works well when the app is on the screen(onBubbleTap works, prints debug statement and shows a snackbar), but when I close the app and tap on it I don't get anything on my console ie, print('Bubble Tapped'); or neither snackbar(of course!)
Tested Example App:
Ran the example app

Started the bubble
Tapped on the bubble > Snackbar appeared with message 'Bubble Tapped' and got printed on the console print('Bubble Tapped');
Closed the app(just pressed back button/cleared from the recent as well) in wither case
Now tapped on the bubble: Got nothing on the console
Again, Opened the app manually(bubble still exists, again tapped on the bubble), this time also no output on the screen(no snackbar and not even console print('Bubble Tapped');)

When you close the app by pressing back when the navigation stack is empty or clearing the app from the recent apps, it terminates the app, so it generally can not handle any action anymore, neither printing nor showing a SnackBar.
That is why the parameter keepAliveWhenAppExit exists, because when you terminate the app, the bubble will remain alive but with no ability to handle events to trigger the callbacks and execute a Dart code, so if you are planning to fire a callback, then you should set the parameter to false (the default value), in order to close the bubble when the app terminates and becomes not able to handle callbacks anymore.
Also, in the last step of your mentioned scenario, when you opened the app with an already shown bubble, the bubble and the app are not connected together because each one is related to a different instance of the app, the already shown bubble is tied to a terminated instance of the app so it can not handle events, and the new instance of the app has no shown bubble yet.
I hope it clears up the misunderstanding. And if you need any help, please do not hesitate to ask me ... 🙏🏻

Absolutely, but the expected outcome is not meeting what I wanted to achieve through the bubble (reopening the app on tap) as the documentation does not explain when the onTap action will be valid.
Anyways, can you help me Run the bubble in the background in a separate instance and handle the events as well(just like truecaller/Facebook messenger bubble to reopen the app or maybe trigger some function calls)? :)

Currently you can't perform any callback from Dart side if the app is terminated.
For opening the app while its terminated or brining it to foreground while its working in background, I have already planned to work on this feature and ship it on a future release, but sadly I have no specific time for when it would be released.
If you really need this feature soon, then I advise you to fork the plugin and add a piece of code to open the app from the Native Side while setting the parameter keepAliveWhenAppExit to be false in the Dart side.

Thanks, @moazelsawaf
I have implemented tap to open or bring to front if already running from the native side (kotlin)


Update-MarkdownHelp ignoring PositionalBinding = $false

Steps to reproduce
create an Advanced function with the following cmdletbinding attribute PositionalBinding = $false and do not specify the Position parameter attribute.
generate markdown help with New-MarkdownHelp and the parameters will have a Position value even though they should be 'Named'

function foo-bar {

    [CmdletBinding(
        ConfirmImpact = "Low",
        SupportsShouldProcess = $true,
        PositionalBinding = $false
    )]
    [OutputType(
        [System.String]
    )]

    param (
        [Parameter()]
        [String]
        $MyParam = $null
   )
}

Expected behavior
Generated markdown will not give positional value for Parameter MyParam.
Actual behavior
Parameter MyParam has positional value of 0 in the output markdown
Environment data

powershell core v6.1.3
v013.0
I started troubleshooting by opening an issue in the PowerShell repo first, but we have concluded this seems to be an issue with platyPS.  please see the following thread:
https://github.com/PowerShell/PowerShell/issues/9151

ok, I have more info on this.  This is happening when you run Update-MarkdownHelp and the parameter attributes have been changed.  so if the xml help file has already been created, and you import in your module to run Update-MarkdownHelp and the parameter Position attribute has been altered or removed, it seems to still be pulling the value from the xml, and not following the changes.  so, I had to delete my xml help file, start a clean PowerShell session, import in my module, then run Update-MarkdownHelp.  Is there a better way to handle this?

Hey, sorry for the late reply. Unfortunately I don't think there is a better way. PlatyPS consumes the Get-Help output, so that's really how PS help system works. If there is an out of date xml, we are going to use it. Perhaps platyPS can try to have a param to do some trick to avoid loading xml help (like copy all the files except xml somewhere before loading the module). But that could not be the default behavior: we want to still pull as much useful info from xml as possible.

@vors What useful info can be found in xml what is not in markdown already? I might be missing something but wouldn't it make sense to delete the xml and then regenerate it from scratch using the markdown files so the generator cannot get confused from out-of-date xml?

@pavel1269 the out-of-date xml that I'm talking about above is the xml that is shipped with the module. So in the non-developer case, it's the part of the thing that you download from the gallery. Does it makes sense?

@vors Yes it does. But then deleting the xml makes sense for me still in cases where there was no xml prior, e.g. user is an owner of said module, does this make sense too?

I think in this scenario user can figure out how to delete the xml outside of PlatyPS commands, since this seems as a developer scenario.


Forcibly clean and destroy the session on logout

Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

need backport

Ah I think I understand!
We don't necessarily have to destroy the session completely.

make L-G-T-M work

Please send backport to v1.11


This is a good suggestion. From discussion: armed_for_triggers should be awaited before the status is returned, because it is part of the arming process. The status returned is actually for when acquisition completes.

          This is a good suggestion. From discussion: `armed_for_triggers` should be awaited before the status is returned, because it is part of the arming process. The status returned is actually for when acquisition completes.

Originally posted by @callumforrester in https://github.com/bluesky/ophyd-async/pull/345#discussion_r1624132573

Suggestion from @coretl rename the Controller.arm method to something clearer e.g. arm_and_provide_complete_status


Module has trouble catching up on upgrade?

Hey folks! I was upgrading a 3.x installation to 4.x and the actual install itself went fairly smoothly. What I did notice though was that the module started iterating through all the events from midnight of the same day, and it is having trouble catching up with the AWS events firehose. Is there anyway to tell the module to start from the latest available event on upgrade?
Thanks!

FWIW the config is very simple
module "cloudtrail-to-slack" {
  source                         = "fivexl/cloudtrail-to-slack/aws"
  version                        = "4.2.1"
  default_slack_hook_url         = "https://hooks.slack.com/services/xxx/xxxx/xxxxx"
  cloudtrail_logs_s3_bucket_name = data.aws_s3_bucket.trail_bucket.id
  events_to_track                = "OpenTunnel"
  use_default_rules              = false
  function_name                  = "${var.name_prefix}-fivexl-cloudtrail-to-slack"
}

I downgraded to 3.x (via destroy) but no joy.


Added Funds with Vendors as Source Sending Emails

Describe the bug
It seems that when we add a vendor as the source of an added fund and email is being sent. This is the usual "Thank you for your contribution" email (order.thankyou.hbs). However this email is being delivered to the collective admins.
To Reproduce
Steps to reproduce the behavior:

Go to a collective you host
Add a fund (added fund)
Notice the emails being sent

Expected behavior
For vendors as source we should not send any automated email.
Screenshots
An example could be found in the freshdesk ticket; https://opencollective.freshdesk.com/a/tickets/294969

cc: @kewitz : Just tagging you as you've recently worked on the vendors feature.


.jbuilder extensions are ruby

Rails includes rails/jbuilder by default, and views with name.json.jbuilder extensions are becoming more common.
This is still just ruby though, much like erb.  https://github.com/facelessuser/ApplySyntax just delegates to the ruby bundle under sublime

Hi @wyaeld!
Is the extension always .json.jbuilder or can it be just .jbuilder?

usage is just convention I think.  Rails convention seems to be
.json.jbuilder, but the actual handler registration with sprockets is based
only on .jbuilder, so thats probably more reliable.  I've seen gists from
sinatra projects where they only have .jbuilder on the template
On Wed, Mar 4, 2015 at 10:30 AM, Paul Chaignon<EMAIL_ADDRESS>wrote:

Hi @wyaeld https://github.com/wyaeld!
Is the extension always .json.jbuilder or can it be just .jbuilder?
—
Reply to this email directly or view it on GitHub
https://github.com/github/linguist/issues/2194#issuecomment-77039281.

#2202 should fix this ;)

excellent, thankyou
On Fri, Mar 6, 2015 at 7:08 PM, Paul Chaignon<EMAIL_ADDRESS>wrote:

#2202 https://github.com/github/linguist/pull/2202 should fix this ;)
—
Reply to this email directly or view it on GitHub
https://github.com/github/linguist/issues/2194#issuecomment-77511810.


Environment file for new GPU compatibility

The environment-gpu.yml creates an environment that is not compatible with the latest GPUs.
I created a new environment file (environment-gpu-2019) that is compatible with the latest GPUs.
Read the  environment-gpu-2019-notes-warning.txt for more information and compatibility warnings.
Warning: This can be used for the projects but not for the final trained model that will be uploaded on Carla.

Unfortunately, we are not able to support versions of TensorFlow higher than 1.3.0 at this time, which matches the version we use in the classroom. We plan to upgrade this in the coming months.

I know that Carla runs TensorFlow 1.3.0 and the final project should be implemented with that version.
On the notes I have a warning about that.
There are many students that have a new PC or a new GPU and they could  use it to save time with GPU acceleration.
The users with new GPUs are also Udacity's potential customers. Someone who have a $1000 GPU is more  likely to enroll a $1600 nanodegree than someone who does not have a GPU.
Last year I lost many hours waiting for the training. I had a slow GPU on my laptop and I did not have internet all the time, I could not use AWS. Recently I upgraded to a new PC with 2080 Ti. I planed to use it for the capstone project. I lost three days experimenting with the environment and the system setup just to realize that tensorflow 1.3.0 is not compatible with the new GPUs.
I think it will be good to have this environment, even with a notice that it is unsupported.

All classroom code is implemented with TensorFlow 1.3.0. The only GPU's that can't use this are the RTX GPUs, which a substantially small portion of our student base uses. We can re-visit this at a later time, but for now, since the vast majority of students are able to use this version and have lower end GPU's than late 2018/2019, we will not make this change at this time.


Use with pipeline or generate

Hello,
Thank you very much for making this work available. I would like to ask you if this works with the transformer.pipeline or the model.generate ? Or do we have to use a TextStreamer as per the example ?

TextStreamer is a very simple class from transformers that only has one goal: printing out tokens as they are being generated - it's kind of separate from the generation process itself.
You can do generation like model.generate just like in normal transformers.
Only if you want to use multi-step generation (i.e. where the previous outputs are added as history to new prompts), then you should use a manual generation loop that calls model(input_ids, use_cache=True, past_key_values=past_key_values) and extracts the past_key_values for the next inputs and extracts the logits to determine the generated token.

Tom Aarsen

Thank you very muck !


🐛 Use absolute path to refer to operator installation from quickstart

What this PR does / why we need it:
Use absolute path to refer to operator installation from quickstart to be able to properly render it when referred to from the CAPI book.
Also, removes highlighting of Hetzner provider in the reference link (I would assume it was typo and not intentional)
Note: currently link to "full documentation" in CAPI book refers to the doc in operator repo which can't be accessed and throws not found error, and we are replacing that with absolute path to make it possible to render it properly
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

cc @alexander-demicev @Fedosin

cc @dtzar


since moving to Godot 4.x HeightMapShape3D shows a warning in the editor

Godot version
4.1
System information
Windows 10
Issue description
the HeightMapShape3D only has a few parameters and seems relatively undoced/untalked about
as it only has a few parameters, the map_height, map_depth and the data as a serial array... presumably therefore i assumed the only way to change it's actual scale and size properly was to scale the CollisionShape3D
this now provides and editor warning.... maybe the HeightMapShape3D should have extra features? like scale? or maybe the editor is making a needless warning and it is correct to scale the CollisionShape3D
the docs do not provide much information on the correct usage
Steps to reproduce
create a HeightMapShape3D, fill it with data
scaling any CollisionShape3D provides the same warning
Minimal reproduction project
N/A

Please include the actual warning

yes as i said "scaling any CollisionShape3D provides the same warning", that warning
in which case what is the correct way of using the HeightMapShape3D?

as i said in the issue i opened... it DOES work... and the only way to actually scale the heightmap is to scale the shape
please can you demonstrate how you change the heightmaps lateral (x z) scaling by interaction with any other parameter

Yes, scale it in all axes
Just adjust the hight in the map so it works when scaled

yes for the height.... but what happens with this grid (undocumented)... it's actual size is based on the depth and the height values... it gets bigger the more squares you add
if you do your approach therefore you can't change the scaling of the grid, you will have to leave the squares all the same size... normaly you can scale a grid

I mean, just change the height values?
If you want (1, 2, 3, 4) and you need to make it twice as wide and deep just make the values (0.5, 1, 1.5, 2)

sorry maybe someone else will get it... the height... okay i am fine with that yes just change height values
the actual lateral dimensions to scale these... there are no options... i tried to explain how it works to you i need to talk with someone who's used it i think sorry i can't convey it to you

I know how it works, I've used it, there aren't any options because it's a simple type, you can't use it for more advanced cases, just like with BoxShape3D

well what i have works, but it only works with this scaling
look for example at the docs:
map_depth
"Changing this will resize the map_data."
this is NOT the scaling, this and map_height is the size of the map data
so the ONLY thing you can set in this heightmap is the data... the cells, and their heights... not scale... you have to make 4x as many squares to get coverage of an area that is 2x the height width
are you SURE you used it and you managed to scale it laterally like i said? how?!? there are 3 parameters

are you SURE you used it and you managed to scale it laterally like i said?

Yes, I'm sure I used it... If you're uninterested in my feedback because you think I'm lying to you then go ahead

okay providing some screenshots to demo what i mean so people can understand:

in this image... i have generated a heightmap and got it to fit my shader (a lot of hard work, like i say this object could be more intuitive)
in the image we can see i have had to scale the CollisionShape3D.. there are no other settings that achieve this
let's say i accept then it's supposed to be a limitation, lets scale to 1x1x1 as the warning says:

now the HeightMapShape3D  could not even cover my whole mountain... because i'd need far too many squares

Just set the scale to all same values, as instructed, and adjust the height values accordingly, that's all I can tell you

i will try seeing if i can refactor to change that okay i could see if i can make it so i need a "uniform scale", fair point i did not pay propper attention to "uniform scale" i will try that
still i don't actually get any issue with the current code, just a warning... indeed the warning saying "probabally"

thanks man i will shut this down actually i was missunderstanding the error so not communicating no worries i don't see what anyone else would be able to say
i want to for myself find out if maybe static collider scaled are okay, i should do that with another method thanks


Data update fails - Telenet website has changed

To be checked

here not working either, didnt reinstall dont know the issue

wanted to fix this now, but seems to work correct again...


Feature request for centercrop

My dataset has nonsquare samples, so on training, I'm doing random crop and for validation, I'm calculating fid with center cropped copy of dataset.
Would be cool to have some option for passing my own transform, for example.

Thank you for pointing out this use case, I will add this flag in the next version of the library.

this feature is available in the current clean-fid version 0.1.17
Any transforms can be passed in through the flag custom_image_tranform
You can pass in a function that takes an image as an numpy array as input and returns the transformed image. For instance, the function below can be used to apply a crop to the images when computing the FID score.

    def fn_crop(x):
        return x[60:-60, 60:-60, :]

This option is only applied when the FID is computed between two folders.

Feel free to re-open this issue if you run into any issues with this.


apt install for compose-switch?

Any reason why compose-switch is an outlier in terms of not being able to be apt install'd?
Would be really great if we could install it just like docker-ce and docker-compose-plugin instead of having to manually download the binary, update-alternatives, etc.

I see there is an install script which I've tried out; unfortunately there are some problems with it which I've documented here: https://github.com/docker/compose-switch/issues/33
It would still be great if we could just apt install compose-switch!

It would be great to be able to install it like everything else.
I think a better name would be apt install docker-compose-switch by the way.


[Bug]: can't load ‘Performance Graph’ when workstage‘s name had any space characters

error e.g:
<workstage name="test write">
    <work name="w_1" workers="50" totalOps="5000000" driver="driver1">
        <operation type="write" ratio="100" config="cprefix=feiteng-0;containers=c(2);oprefix=w1-1kb-;objects=s(1,20000000);sizes=c(1)KiB" />
    </work>
</workstage>

result:

write e.g:
<workstage name="test-write">
    <work name="w_1" workers="50" totalOps="5000000" driver="driver1">
        <operation type="write" ratio="100" config="cprefix=feiteng-0;containers=c(2);oprefix=w1-1kb-;objects=s(1,20000000);sizes=c(1)KiB" />
    </work>
</workstage>

result:

maybe add some words in the handbook.

bug fixed.


Output embeddings

I've just started doing it. In java i got response from pretrained model (inception v1)
as matrix[570[128]. 128 are the features, but what is 570 and it always changing based on image.
And how do i check if two faces(features) match and its percentage?
Thank you.

Are you inputting 160x160 px imaged? It sounds like you are feeding something else. Sounds like compare.py does roughly what you want to do.

I've been running it on Java and yeah the input was too big. It figured it
out and everything's good now. Thank you for your reply.
On Apr 11, 2018 7:14 PM, "David Sandberg"<EMAIL_ADDRESS>wrote:

Are you inputting 160x160 px imaged? It sounds like you are feeding
something else. Sounds like compare.py does roughly what you want to do.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/davidsandberg/facenet/issues/688#issuecomment-380509775,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXEz42gLEucZi7y6V3fqmbniP7vvFc8Gks5tnivYgaJpZM4TNX_v
.


Docs disappear when adding drafts:true

Link to reproduction
/
Describe the Bug
I had a collection with products on which versions and draft were both NOT enabled. After the collection had been filled through api, i realized this and enabled drafts and versions:
  versions: {
    drafts: true,
  },

When doing this, the entire collection appears empty in the admin panel. When removing the config again, the docs re-appear.
When only adding versions: true all docs appear like normal.
To Reproduce

Create a collection without any versions or draft setting
add docs to this collection (in my case via rest api)
enable versions and drafts using

  versions: {
    drafts: true,
  },

Payload Version
2.0.13
Adapters and Plugins
@payloadcms/db-mongodb

I tried manually adding a _status field with value published before enabling drafts. I thought maybe the docs don't show because this field is missing. Unfortunately, this does not solve the problem.

We should improve the docs and DX here. Changing a collection with existing data to have drafts will cause this exact behavior.
We have a few options here, but showing a warning and maybe linking to docs and a migration file would be a good start.

Something like this would work for you, https://github.com/payloadcms/payload/discussions/434.
Saving all the docs again will add them to the versions collection, which is where draft enabled collections are queried from when draft:true (this is how the list view queries).

this happened for me in 3.0.0-beta.74 when adding versions / drafts to my Posts collection.
The documentation did not give me a warning or I overlooked it.  https://payloadcms.com/docs/beta/versions/overview.
But found this closed issue.
Here is my solution.
Server action.
'use server';

import { getPayload } from '@payload-api';

export const republishPosts = async () => {
  const payload = await getPayload();

  payload.logger.info(`Republishing posts...`);

  const allPosts = await payload.find({
    collection: 'posts',
    where: {
      published: { equals: true },
    },
    limit: 5000,
    depth: 0,
  });

  payload.logger.info(`Found ${allPosts.totalDocs} posts`);
  for (const post of allPosts.docs) {
    payload.logger.info(`Republishing post: ${post.title}`);
    await payload.update({
      collection: 'posts',
      id: post.id,
      data: {},
    });
  }
};

Add to component.
export default function RepublishButton() {
  return (
    <form action={republishPosts}>
      <Button>Republish Posts</Button>
    </form>
  );
}

And to the collection
export const Posts: CollectionConfig = {
  slug: 'posts',
  admin: {
    components: {
      beforeListTable: [RepublishButton],
    },
  },
  versions: {
    drafts: true,
  },

Ran it and all posts got republished and are now visible again.


feat(CategoryTheory/Galois): decomposition in connected components

Shows that any object in a Galois category is the sum of its connected components. Also shows that the connected components are unique up to isomorphism.

[x] depends on: #10400
[x] depends on: #10418

This will be used in the followup PR to show that the fiber of any object is represented by a Galois object, which essentially shows the pro-representability of fiber functors.

Thanks for all the reviews!

I don't know why the build keeps failing, I just merged master, maybe that helps. Could someone send this to bors again?

Let me try again
bors r+


Implement kernel trick in SVM

The svm function does not support kernelization for now.

The kersvm function was implemented in Hadrien-Montanelli@bf357c1.


Download whole album in playlist

[ x ] Using latest version as provided on the master branch
[ x ] Searched for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug
[ ] Feature Request
[ x ] Question
[ ] Other

Description

Hey,
is there any way I can download the whole albums in a playlist.
What I mean by that is that if I have "Song xy" by "Artist abc" in my playlist 3 times, because they are in 3 different albums only one gets downloaded. I can't find out how I can download them all to have the complete albums.
Thanks!
Log

You can Set custom filenames to also add  the {artist} key in filenames. Something like file-format: '{artist} - {track_name} ({album})' should work and then the script won't think about overwriting previously downloaded track as tracks from different albums will end up having different filenames.

Works like a charm! Thank you. :)


Sharing is Caring Power App Sample JennyDay

Pull Request Body:
Category

[x] New Sample

What's in this Pull Request?
Power Apps Calendar Component

Hi,
Yes, ok! I thought, with being the only one left, that it must've been rubbish!
Thanks
Jenny
Sent with Proton Mail secure email.
------- Original Message -------
On Friday, December 2nd, 2022 at 18:18, David Warner II @.***> wrote:

@.***(https://github.com/noisychicken), it looks like everything was done well here. Do you mind if we reopen and approve the PR so you get contribution credit? :)
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

@chipswithgravy not at all rubbish. We love them. Looks great!
Be sure to sign up for the Recognition Program if you have not already as you are eligible for a badge for this contribution. 💜🧡🤗
https://aka.ms/powerplatform/recognition

@chipswithgravy, actually since you changed your github username, even if you had already registered, would probably be good to re-register with your new name. :)


[HOLD] Internal Tags

refs #6165
TODO:

[x] update existing code to use 'visibility' property
[x] ensure code works

[x] client-side setup
[x] internal tags don't have /tag/-slug-/ pages
[x] internal tags don't show up in post tags
[x] {{#has}} helper works with internal tags

[x] tests

@acburdine looks good, thanks! 😄 Unfortunately I think something has been missed from our conversations (sorry it wasn't documented better!) where there shouldn't be any checkbox in the tag settings menu. Instead any new tag created that begins with a # will be internal so I think that behaviour can be moved out of the PSM and into the Tag model so it's universal. I believe that also means that the visibility attribute will be ignored when passed to the API so that the server will have the final say based on the name & labs flag. (@ErisDS c/d?)
We'll also need a label added to internal tags in the tags screen list because people migrating with old data may have hash tags which are not internal so there needs to be some way to visibly distinguish internal tags.

Holding until after 0.8.0 is released

Ok - so I think this and pr/4 are pretty much ready to merge - I have left only just minor comments on them both.
What I was working on, when I got utterly stuck, was some improvements to the original spec. I have now written those up here.
I'm working on rebasing the working changeset I had on top of this PR so I can show you where I got to  with it.

So... the contents of my stash for this are now here
If you're wondering what the fuck.. well you're right. Implementing this for tags was a doddle... but the foreach helper just wouldn't work.
The problem was something to do with the object containing the resources to loop over that I got back from the foreach helper. It can be an object or an array, but if it's an array, it's one that's been modified with a length property.
I could remove items from the object, but I could not update the length property. I've had an idea though, so I'm gonna have another go just now - and come back to you in a few mins.

@ErisDS fixed all the comments


How is the project executed?

Hi, sorry. I have seen your projects on GitHub pro which are great, you have the same line of my research and I want to complete it with a database, but I have a problem when I execute the human-vrm project with the command "npm run
dev "but the project will not be executed if a build, js file is not executed in the path file: /// C<EMAIL_ADDRESS>in the browser but which is not carried out by the human-vrm project.

if you run
node<EMAIL_ADDRESS>--profile dev

it is NOT possible that output is the same.
please run the command and post output here (not email). and not screenshot - copy & paste text.

si tu corres
nodo node_modules @ vladmandic \ b uild \ d ist \ b uild.js --profile dev

NO es posible que la salida sea la misma.
ejecute el comando y publique el resultado aquí (no por correo electrónico). y no captura de pantalla: copie y pegue texto.

C:\HI5  NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile dev
node:internal/modules/cjs/loader:936
throw err;
^
I get this
C:\HI5  NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile dev
node:internal/modules/cjs/loader:936
throw err;
^
'C:\HI5  NO<EMAIL_ADDRESS>←[90m    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)←[39m
←[90m    at Function.Module._load (node:internal/modules/cjs/loader:778:27)←[39m
←[90m    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12)←[39m
←[90m    at node:internal/main/run_main_module:17:47←[39m {
code: ←[32m'MODULE_NOT_FOUND'←[39m,
requireStack: []
}

or this?
C:\HI5  NO BORRA\human-vrm-main\node_modules@vladmandic\build>npm run build

<EMAIL_ADDRESS>build
rimraf build.log && node dist/build.js --profile production

2021-10-20 19:41:17 INFO:  @vladmandic/build version 0.6.2
2021-10-20 19:41:17 INFO:  User: LENOVO Platform: win32 Arch: x64 Node: v16.10.0
2021-10-20 19:41:17 STATE: Application log: C:\HI5  NO<EMAIL_ADDRESS>2021-10-20 19:41:17 INFO:  Application: { name: '@vladmandic/build', version: '0.6.2' }
2021-10-20 19:41:17 INFO:  Environment: { profile: 'production', config: '.build.json', package: 'package.json', tsconfig: true, eslintrc: true, git: true }
2021-10-20 19:41:17 INFO:  Toolchain: { build: '0.6.2', esbuild: '0.13.8', typescript: '4.4.4', typedoc: '0.22.6', eslint: '8.0.1' }
2021-10-20 19:41:17 INFO:  Build: { profile: 'production', steps: [ 'clean', 'compile', 'typings', 'typedoc', 'lint', 'changelog' ] }
2021-10-20 19:41:17 STATE: Clean: { locations: [ 'types/', 'typedoc/' ] }
2021-10-20 19:41:17 STATE: Compile: { name: 'build module', format: 'esm', platform: 'node', input: 'src/build.ts', output: 'dist/build.esm.js', files: 163, inputBytes: 641907, outputBytes: 596161 }
2021-10-20 19:41:20 STATE: Typings: { input: 'src/build.ts', output: 'types', files: 1 }
2021-10-20 19:41:24 STATE: TypeDoc: { input: 'src/build.ts', output: 'typedoc', objects: 4, generated: true }
C:\HI5  NO BORRA\human-vrm-main\node_modules\eslint\lib\cli-engine\file-enumerator.js:324
throw new AllFilesIgnoredError(pattern);
^
AllFilesIgnoredError: All files matched by 'src/.ts' are ignored.
at FileEnumerator.iterateFiles (C:\HI5  NO BORRA\human-vrm-main\node_modules\eslint\lib\cli-engine\file-enumerator.js:324:27)
at iterateFiles.next ()
at CLIEngine.executeOnFiles (C:\HI5  NO BORRA\human-vrm-main\node_modules\eslint\lib\cli-engine\cli-engine.js:789:48)
at ESLint.lintFiles (C:\HI5  NO BORRA\human-vrm-main\node_modules\eslint\lib\eslint\eslint.js:567:23)
at run4 (C:\HI5  NO BORRA\human-vrm-main\node_modules@vladmandic\build\dist\build.js:16669:33)
at Build.run (C:\HI5  NO BORRA\human-vrm-main\node_modules@vladmandic\build\dist\build.js:17015:17) {
messageTemplate: 'all-files-ignored',
messageData: { pattern: 'src/.ts' }
}

Can you help me how can I exercise it?

node<EMAIL_ADDRESS>--profile development

I get this but it does not run on the web page wl 3d drawing
C:\HI5  NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile dev
node:internal/modules/cjs/loader:936
throw err;
^
Error: Cannot find module 'C:\HI5  NO<EMAIL_ADDRESS>←[90m    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)←[39m
←[90m    at Function.Module._load (node:internal/modules/cjs/loader:778:27)←[39m
←[90m    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:79:12)←[39m
←[90m    at node:internal/main/run_main_module:17:47←[39m {
code: ←[32m'MODULE_NOT_FOUND'←[39m,
requireStack: []
}
C:\HI5  NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile dev
2021-10-20 19:03:21 INFO:  human-vrm version 0.2.2
2021-10-20 19:03:21 INFO:  User: LENOVO Platform: win32 Arch: x64 Node: v16.10.0
2021-10-20 19:03:21 ERROR: Profile not found: dev
C:\HI5  NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile development
2021-10-20 22:44:25 INFO:  human-vrm version 0.2.2
2021-10-20 22:44:25 INFO:  User: LENOVO Platform: win32 Arch: x64 Node: v16.10.0
2021-10-20 22:44:25 INFO:  Application: { name: 'human-vrm', version: '0.2.2' }
2021-10-20 22:44:25 INFO:  Environment: { profile: 'development', config: 'build.json', package: 'package.json', tsconfig: true, eslintrc: true, git: false }
2021-10-20 22:44:25 INFO:  Toolchain: { build: '0.6.2', esbuild: '0.13.8', typescript: '4.4.4', typedoc: '0.22.6', eslint: '8.0.1' }
2021-10-20 22:44:25 INFO:  Build: { profile: 'development', steps: [ 'serve', 'watch', 'compile', 'src/human-vem.html' ] }
2021-10-20 22:44:25 STATE: WebServer: { ssl: false, port: 10030, root: '.' }
2021-10-20 22:44:25 STATE: WebServer: { ssl: true, port: 10031, root: '.', sslKey<EMAIL_ADDRESS>sslCrt<EMAIL_ADDRESS>}
2021-10-20 22:44:25 STATE: Watch: { locations: [ 'src/**', 'README.md', 'https://localhost:10031' ] }
2021-10-20 22:44:27 STATE: Compile: { name: 'human-vrm', format: 'esm', platform: 'browser', input: 'src/human-vrm.ts', output: 'dist/human-vrm.esm.js', files: 2, inputBytes: 17279, outputBytes: 4101959 }
2021-10-20 22:44:27 WARN:  Build: unknown step src/human-vem.html
2021-10-20 22:44:27 INFO:  Listening...

Please can you answer me?

RELAX! I cannot be online for 24h!
C:\HI5 NO BORRA\human-vrm-main>node<EMAIL_ADDRESS>--profile development
2021-10-20 22:44:25 INFO: human-vrm version 0.2.2
2021-10-20 22:44:25 INFO: User: LENOVO Platform: win32 Arch: x64 Node: v16.10.0
2021-10-20 22:44:25 INFO: Application: { name: 'human-vrm', version: '0.2.2' }
2021-10-20 22:44:25 INFO: Environment: { profile: 'development', config: 'build.json', package: 'package.json', tsconfig: true, eslintrc: true, git: false }
2021-10-20 22:44:25 INFO: Toolchain: { build: '0.6.2', esbuild: '0.13.8', typescript: '4.4.4', typedoc: '0.22.6', eslint: '8.0.1' }
2021-10-20 22:44:25 INFO: Build: { profile: 'development', steps: [ 'serve', 'watch', 'compile', 'src/human-vem.html' ] }
2021-10-20 22:44:25 STATE: WebServer: { ssl: false, port: 10030, root: '.' }
2021-10-20 22:44:25 STATE: WebServer: { ssl: true, port: 10031, root: '.', sslKey<EMAIL_ADDRESS>sslCrt<EMAIL_ADDRESS>}
2021-10-20 22:44:25 STATE: Watch: { locations: [ 'src/**', 'README.md', 'https://localhost:10031' ] }
2021-10-20 22:44:27 STATE: Compile: { name: 'human-vrm', format: 'esm', platform: 'browser', input: 'src/human-vrm.ts', output: 'dist/human-vrm.esm.js', files: 2, inputBytes: 17279, outputBytes: 4101959 }
2021-10-20 22:44:27 WARN: Build: unknown step src/human-vem.html
2021-10-20 22:44:27 INFO: Listening...

So web server is started and I see no errors.

I get this but it does not run on the web page wl 3d drawing

What happens when you go to the web page?
Open browser inspector window and copy the log and paste it here.

you do not need to specify model names unless you're changing them from defaults
and default model for hand details analysis has been changed to handlandmarks-full
if you didn't change default values for model name, you wouldn't have the problem - only change config values if you need to, not just to hard-code things that should be left at defaults

Sorry, but the link that is in your project (https://vladmandic.github.io/human-vrm/src/human-vrm.html) does not work, it only comes out in 3d dubjo, but nothing works nor does the camera hear none of those.

needed to rebuild human-vrm to take effect of the new human release, done - link now works.

Thank you very much for the updated link, but when I want to download the dependencies with (npm install) I get the following error. I think you have updated your project. Could you update I will be very grateful.
npm install error:
C:\HI5  NO BORRA\human-vrm>npm install
npm notice
npm notice New patch version of npm available! 8.1.2 -> 8.1.3
npm notice Changelog: https://github.com/npm/cli/releases/tag/v8.1.3
npm notice Run npm install -g<EMAIL_ADDRESS>to update!
npm notice
npm ERR! code 1
npm ERR! git dep preparation failed
npm ERR! command C:\Program Files\nodejs\node.exe C:\Users\LENOVO\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js install --force --cache=C:\Users\LENOVO\AppData\Local\npm-cache --prefer-offline=false --prefer-online=false --offline=false --no-progress --no-save --no-audit --include=dev --include=peer --include=optional --no-package-lock-only --no-dry-run
npm ERR! npm WARN using --force Recommended protections disabled.
npm ERR! npm WARN cleanup Failed to remove some directories [
npm ERR! npm WARN cleanup   [
npm ERR! npm WARN cleanup     'C:\Users\LENOVO\AppData\Local\npm-cache\_cacache\tmp\git-clonea6LPGD\node_modules',
npm ERR! npm WARN cleanup     [Error: EPERM: operation not permitted, unlink<EMAIL_ADDRESS>{
npm ERR! npm WARN cleanup       errno: -4048,
npm ERR! npm WARN cleanup       code: 'EPERM',
npm ERR! npm WARN cleanup       syscall: 'unlink',
npm ERR! npm WARN cleanup       path<EMAIL_ADDRESS>npm ERR! npm WARN cleanup     }
npm ERR! npm WARN cleanup   ]
npm ERR! npm WARN cleanup ]
npm ERR! npm ERR! code ERR_SOCKET_TIMEOUT
npm ERR! npm ERR! network Socket timeout
npm ERR! npm ERR! network This is a problem related to network connectivity.
npm ERR! npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! npm ERR! network
npm ERR! npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! npm ERR! network 'proxy' config is set properly.  See: 'npm help config'
npm ERR!
npm ERR! npm ERR!      C:\Users\LENOVO\AppData\Local\npm-cache_logs\2021-11-04T23_26_38_882Z-debug.log
npm ERR!      C:\Users\LENOVO\AppData\Local\npm-cache_logs\2021-11-04T23_26_40_283Z-debug.log

That is an error on your side, again seems like your node_modules got corrupt permissions.

the truth before the dependencies are downloaded well but now they no longer

yes, and that is a problem with your system.

Hello, sorry, the truth all week I was seeing how to exercise and I do not understand, I ask you to be patient with me since I do not know much English and also I am a person with disabilities and being like this your project is very good to develop for a translation of deaf people for that reason is the insistence of how to understand their project of how to exercise and develop. My question is if you can advise me on how to exercise? If there is an average of a cost, just tell me!
I await your answer, engineer, thank you very much for your patience!

I helped as much as I can - all the questions were answered
But you're ignoring suggestions and just keep repeating questions, so nothing I can do
This project is almost trivial as everything complex happens within the library, you need to learn very basics of web development
And for sure, installation is as basic as it gets
(You don't even need to install the project, just use single HTML and JS file as a template)


How to get process group

I have trouble find the process group during distributed training, at first I located the api/train.py, where distributed the model, but seems like the process group is hidden. I also tried to get the default process group with pytroch _get_default_group API, but then a error occured. How can I get process group correctly?

The entrance is here
https://github.com/open-mmlab/mmdetection/blob/a1cecf63c713c53941b8dcf8a9d762baf8511f2c/tools/train.py#L120


Fix custom control min height and indicator alignment

Account for when $font-size-base has been customized from 1rem to something else

Hi @sw-double thanks for your contribution.
This change has been done on this PR #25602

Closing as duplicate of #25602.


Macro for easy sheet switching

Set overlay icon on the token.
Optionally also add loot from table, and/or grant permissions, at the same time.

LootClaims.changeSheet() to default to currently-selected tokens.  Or pass an array of tokens.

Also LootClaims.addLoot() and LootClaims.givePermission().


[Presentation] Reduce allocations when parsing number in NumericTextBox

Following #1811, remove some string allocations.

Thanks !


#161: Fixed deserialization to check for no content

And send back an exception with the response. Also added test cases for no content deserialization.

I'm OK with code changes if 204 response should be treated as UnsuccessfulResponseException.


Move VIRTUAL_COL_USES translation to col_index method in ChargeableField

Caused by https://github.com/ManageIQ/manageiq/issues/17592
Chargeback for Projects is using v_derived_cpu_total_cores_used from yaml definition but
we are using from MetricRollup#cpu_usage_rate_average to get metric value and use it in chargeback calculation.
But reason why it was not caught by specs is that we were using cpu_usage_rate_average  in rspec factories(instead of v_derived_cpu_total_cores_used as it is used in yaml) and thanks to that this case have not been covered.
Links
https://bugzilla.redhat.com/show_bug.cgi?id=1602818

@miq-bot add_label blocker, chargeback
@miq-bot assign @gtanzillo

@lpichler Can this be gaprindashvili/yes?

@simaishi  yes,
@miq_bot add_label gaprindashvili/yes

@miq-bot add-label gaprindashvili/yes

Gaprindashvili backport details:
$ git log -1
commit ced41ae549f2075210adeccf4be44a3f27ba0c4b
Author: Gregg Tanzillo<EMAIL_ADDRESS>Date:   Wed Aug 1 09:39:51 2018 -0400

    Merge pull request #17747 from lpichler/fix_translation_from_v_derived_cpu_total_cores_used_to_cpu_usage_rate_average_chargeback_for_projects

    Move VIRTUAL_COL_USES translation to col_index method in ChargeableField
    (cherry picked from commit ee216b089a981c8a09916f130979989df95bca40)

    https://bugzilla.redhat.com/show_bug.cgi?id=1613295


LLVM Headers missing

Hi,
Sorry if this is a noob question. Im trying to compile the usbcvstream program for my camera running on a nanopi. It looks like I'm missing LLVM headers (like raw_ostream) which wasn't listed in the required packages in the README. Should these have been installed with one of those or do I actually need them at all? In addition if I'm already using GCC why do I even need this headers, aren't they associated with Clang which is a different compiler altogether?
Thanks!

For reference I already did a sudo apt-get install llvm, to no avail.

Cscore depends on the wpiutul library. That can be found at GitHub com/wpilibsuite/wpiutil.

Ahh I see, it has all those headers. Looks like I had missed an upper case letter in my path to the include folder and all my libs were out of order :/, closing.
Thanks! Now everything works but the stream is completely black, though the output says I'm getting frames.
I dont want to trouble you too much to actually help me debug stuff but do you have any ideas or tools that you can recommend that would be helpful to start?
i'm getting the output:
got frame at time<PHONE_NUMBER>6223238 size [320 x 240]
got frame at time<PHONE_NUMBER>7023199 size [320 x 240]
got frame at time<PHONE_NUMBER>7823120 size [320 x 240]
got frame at time<PHONE_NUMBER>8623107 size [320 x 240]
got frame at time<PHONE_NUMBER>9423098 size [320 x 240]
got frame at time<PHONE_NUMBER>0222989 size [320 x 240]
got frame at time<PHONE_NUMBER>1022961 size [320 x 240]

To confirm, I've used the mjpg streamer tool before and have confirmed that my camera can output reasonable images with default camera settings.

 Perhaps the brightness/exposure needs to be fixed.

What kind of camera?  Also, what web browser are you using to view the stream?

It's an FA CAM-202.
http://www.friendlyarm.com/index.php?route=product/product&product_id=90
On second thought I just tried two tests,
a) Writing a solid color to the streaming output. This works so the actual streaming seems to be fine.
b) Writing out the captured image in usbcvstream.cpp to a file using cv::imwrite. I get a black image here. Perhaps the camera doesn't play nicely with the sink.
Oh! I should probably be using the device name for the camera shouldn't I? Not just "usbcam" but more like "/dev/video0"

Ah oops,  I missed this guy:
CS: ERROR: ioctl VIDIOC_G_EXT_CTRLS failed at UsbCameraProperty.cpp:33: Input/output error (UsbUtil.cpp:159)
Which occurs right at the beginning when I run the executable, any idea what this could be referring to? Thanks again!

Everything works fine if I use OpenCV's videowriter to capture the frames, so it looks like it's just the cvSink that's messing up somewhere. Ideally though I'd like everything to be using CSCore since it has much better support for managing camera settings.
Maybe I can do something like set the camera settings via CSCore, then release the camera object, then use video writer to capture the frames after the settings have been set. Though I'm sure using the sink class has some kind of benefits in terms of performance as well.

The first parameter to the constructor is just a name; the second parameter is the actual connection location; if you pass 0 it uses /dev/video0. The EXT_CTRLS error means it’s having trouble enumerating the settings.  Most likely the issue is that the camera doesn’t support the image format requested.  Can you try compiling and running  https://github.com/wpilibsuite/cscore/blob/master/examples/enum_usb/enum_usb.cpp and posting the output?

0: /dev/video0 (HC 3358+2100)
Properties:
CS: usbcam: Connect to USB camera on 1
CS: ERROR: ioctl VIDIOC_G_EXT_CTRLS failed at UsbCameraProperty.cpp:33: Input/output error (UsbUtil.cpp:159)
CS: WARNING: usbcam: failed to get property exposure_auto_priority (UsbCameraImpl.cpp:929)
CS: usbcam: set format 1 res 1600x1200
raw_brightness (int): value=0 min=-127 max=127 step=1 default=0
brightness (int): value=50 min=0 max=100 step=1 default=50
raw_contrast (int): value=64 min=0 max=127 step=1 default=64
contrast (int): value=50 min=0 max=100 step=1 default=50
raw_saturation (int): value=64 min=0 max=255 step=1 default=64
saturation (int): value=25 min=0 max=100 step=1 default=25
raw_hue (int): value=0 min=-16000 max=16000 step=1 default=0
hue (int): value=50 min=0 max=100 step=1 default=50
white_balance_temperature_auto (bool): value=1 default=1
gamma (int): value=100 min=16 max=500 step=1 default=100
power_line_frequency (enum): value=1
0: Disabled
1: 50 Hz
2: 60 Hz
white_balance_temperature (int): value=5200 min=2800 max=6500 step=1 default=5200
raw_sharpness (int): value=4 min=0 max=31 step=1 default=4
sharpness (int): value=12 min=0 max=100 step=1 default=12
exposure_auto (enum): value=3
1: Manual Mode
3: Aperture Priority Mode
raw_exposure_absolute (int): value=625 min=1 max=5000 step=1 default=625
exposure_absolute (int): value=12 min=0 max=100 step=1 default=12
exposure_auto_priority (bool): value=0 default=0
Video Modes:
PixelFormat:YUYV Width:1600 Height:1200 FPS:7
PixelFormat:YUYV Width:640 Height:480 FPS:30
PixelFormat:YUYV Width:800 Height:600 FPS:20
PixelFormat:YUYV Width:1280 Height:720 FPS:11
PixelFormat:YUYV Width:352 Height:288 FPS:30
PixelFormat:YUYV Width:320 Height:240 FPS:30
PixelFormat:YUYV Width:640 Height:480 FPS:30
Also I did already try setting video mode to kYUYV instead of kMJPEG though that did not seem to change my results.

There is definitely a bug with how the default video mode is selected for cameras that don't support any MJPEG formats, but if you tried using SetVideoMode() with kYUYV that should fix that.  Could you try a tweaked version of the usbstream.cpp example for camera=0 and kYUYV? (like the below)
int main() {
  cs::UsbCamera camera{"usbcam", 0};
  camera.SetVideoMode(cs::VideoMode::kYUYV, 320, 240, 30);
  cs::MjpegServer mjpegServer{"httpserver", 8081};
  mjpegServer.SetSource(camera);

  std::getchar();
}


Is it possible to compile with GradlePluginKotlinEnabled?

If we set <preference name="GradlePluginKotlinEnabled" value="true" />, cordova-plugin-firebasex crashes with this error (only after the 16.2.0 version  though, the 16.1.0 version still works fine):
Task :app:compileDebugKotlin FAILED
e: C:/Users/sevan/.gradle/caches/transforms-3/1ee3f7841abe9979797edc39b62bd26a/transformed/jetified-firebase-auth-22.2.0-api.jar!/META-INF/java.com.google.android.gmscore.integ.client.firebase-auth-api_firebase-auth-api.kotlin_module: Module was compiled with an inc
ompatible version of Kotlin. The binary version of its metadata is 1.9.0, expected version is 1.7.1.
e: C:/Users/sevan/.gradle/caches/transforms-3/dade25bead44bca314189166709326d8/transformed/jetified-play-services-measurement-api-21.5.0-api.jar!/META-INF/java.com.google.android.gmscore.integ.client.measurement_api_measurement_api.kotlin_module: Module was compiled
 with an incompatible version of Kotlin. The binary version of its metadata is 1.9.0, expected version is 1.7.1.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:compileDebugKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction

Unfortunately, I'm using admob-plus-cordova too, and I need to set it to true to make it work. So, is there a way to make this plugin work after the 16.2.0 version using the Kotlin extensions library?
To Reproduce
cordova plugin add admob-plus-cordova
cordova plugin cordova-plugin-firebasex
cordova build android
Task :app:compileDebugKotlin FAILED

Expected behavior
Both plugins should work without failing to compile. I guess that since "GradlePluginKotlinEnabled" is deprecated, it's admob-plus that should fix this issue, but I'm not a Kotlin expert.
Additional context
The Task :app:compileDebugKotlin FAILED only started to happen after<EMAIL_ADDRESS>(I'm still using<EMAIL_ADDRESS>and it compiles).

The error message states that the Kotlin module for the Firebase Measurement API was compiled with Gradle v1.9.0 but your project is configured to use v1.7.1 so it can't build.
To fix this, you need to configure your project to use v1.9.0 or above.
In a Cordova project this can be done in the config.xml:
<platform name="android">
    <preference name="GradlePluginKotlinEnabled" value="true" />
    <preference name="GradlePluginKotlinVersion" value="1.9.0" />
</platform>

does it build with v1.9.0 if you remove admob-plus-cordova from the project?

No, it still gives me the same error even with admob-plus-cordova removed:
> Configure project :app
Error: The 'kotlin-android-extensions' Gradle plugin is no longer supported. Please use this migration guide (https://goo.gle/kotlin-android-extensions-deprecation) to start working with View Binding (https://developer.android.com/topic/libraries/view-binding) and t
he 'kotlin-parcelize' plugin.
strippedNativeLibsDir is deprecated and no longer used. Please remove it from your build configuration.

> Task :app:compileDebugKotlin FAILED
e: The Android extensions ('kotlin-android-extensions') compiler plugin is no longer supported. Please use kotlin parcelize and view binding. More information: https://goo.gle/kotlin-android-extensions-deprecation

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:compileDebugKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details

Keep in mind that if I set GradlePluginKotlinEnabled to false, cordova-plugin-firebasex builds fine, but admob-plus-cordova won't work.

https://github.com/apache/cordova-android/issues/1642

For anyone having the same problem, I fixed it with the following steps:
On config.xml:

Then, on \platforms\android\app\build.gradle:
Delete the line 24: apply plugin: 'kotlin-android-extensions'
I've created a Cordova hook so I don't have to manually delete the line everytime I add or remove the android platform. This is just temporary solution until cordova-13 arrives (that line will be automatically removed in cordova-13).


CoffeeScript support

Closes #1556. Adds support for CoffeeScript test files, similar to the existing TypeScript support.

Hi @GeoffreyBooth! Thank you for the great job done.
Could you please take a moment to help us better understand what this feature will give us. Nobody in our team has ever worked with CoffeeScript closely, so please clarify some points:

Which features are important for you in CofeeScript that can't be found in ES6 and TypeScript?

Do you believe there will be many TestCafe users who would prefer to write E2E tests using the CoffeeScript syntax? Have you heard from any of your colleagues or acquaintances about such a desire?

Which features are important for you in CoffeeScript that can’t be found in ES6 and TypeScript?

CoffeeScript provides a cleaner syntax that many people find easier to read. Compare this file to this file, for example. Code that’s more concise is often easier to follow, which helps lead to fewer bugs.

Do you believe there will be many TestCafe users who would prefer to write E2E tests using the CoffeeScript syntax? Have you heard from any of your colleagues or acquaintances about such a desire?

I want to write tests in CoffeeScript because I have an entire app written in CoffeeScript, and I want to keep all my code in the same language. I don’t think this is that uncommon of a use case. I work on a team at a big company, and we’ve written several large apps in CoffeeScript.

@AndreyBelym Any thoughts on this? Here’s an article to help explain the state of CoffeeScript today: https://codeburst.io/why-coffeescript-is-still-alive-aeb369b91b85
Also, Cypress is written in CoffeeScript: https://github.com/cypress-io/cypress 😉

@testcafe-build-bot retest

@testcafe-build-bot retest

@testcafe-build-bot retest

@testcafe-build-bot retest

@testcafe-build-bot retest


Allow PutResult.newUpdateResult() with 0 rows updated

Resolves #453.
@nikitin-da @krpiotrek PTAL

:+1:

Flaky test again… (I'll try to fix it later).


feat: add support for Qt6-based builds

With Mumble moving to use Qt6 this container was no longer able to build recent versions successfully. This commit adds support for building Qt6-based version of mumble-server.

this build seems to have a peoblem:
2024-11-30 04:37:28.893 No functional TLS backend was found
2024-11-30 04:37:28.893 SSL: this version of Murmur is built against Qt without SSL Support. Aborting.


Memory problem while using the dataframe schema validation

Describe the bug
I need to validate a large pandas dataframe (1M rows x 300 columns).
I am using pandera 0.5.1 and pandas 1.1.5 within a Jupyter notebook.
16GB of RAM memory are not enough for the task and the memory is not freed after the validation.

[x] I have checked that this issue has not already been reported.
[x] I have confirmed this bug exists on the latest version of pandas.
[ ] (optional) I have confirmed this bug exists on the master branch of pandas.

Expected behavior
The RAM memory should be freed after the validation is completed.
A finer control of memory allocation is needed to avoid OOMs.
Desktop:
OS: Linux Ubuntu 16.04.1
Browser: Chrome

Hi @superflexo, thanks for trying pandera out! If mutation isn't an issue for, can you try using validate(..., inplace=True) and see if that solves your issue?
By default, pandera will make a copy of the dataframe to avoid mutation of the original data, e.g. when coerce=True and the data types are coerced to the ones specified in the schema.

Thank you @cosmicBboy for the reply!
I tried your suggestion but it does not fix.
I add that the validation (a lazy one) ends with several SchemaErrors (as expected) so there is no dataframe as output in this case.

kk, can you share a toy version of your validation code, and the associated error you're getting? I'd like to better understand what's going on.

Simple toy code (module in attachment).

Error Counts
------------
- schema_component_check: 4

Schema Error Summary
--------------------
                                                failure_cases  n_failure_cases
schema_context column    check
Column         N_TREE    pandas_dtype('int32')        [int64]                1
               P_TREE    not_nullable                   [nan]                1
               SEX       isin({'M', '_N', 'F'})            []                1
               SUM_ITEMS pandas_dtype('int32')        [int64]                1

Usage Tip
---------

Directly inspect all errors by catching the exception:

try:
    schema.validate(dataframe, lazy=True)
except SchemaErrors as err:
    err.schema_errors  # dataframe of schema errors
    err.data  # invalid dataframe

Hope this toy can be of help.
Many thanks!
utils.zip

Thanks @superflexo! So is the OOM error happening after validation has been executed?

Yes, in the case of 1M rows and many columns the notebook stops the execution and restart the kernel and errrors are not raised.
In case it fits in memory, the process ends just outputting the errors and leaving the memory filled (I was able to partially free the memory executing the validation through an external module and then invoking the garbage collector).

Thanks for the zipped files @superflexo!
#360 tries to address these memory issues... I've been aware of them for some time, but haven't gotten the chance to really tackle it until you reported this 🐞 !
I'll be doing some more systematic benchmarking later on, but I made this script to prove to myself things are much more memory efficient. This is the script I used:
import gc
from memory_profiler import profile

@profile
def main():

    from __toy_validation_schema import schema
    import pandas as pd
    import numpy as np
    import time

    data = {
        'TYPE': ['TEST01', 'TEST02', 'TEST03']
        , 'CODE_STRING': ['A', 'B', 'C']
        , 'FLAG': [0, 1, 0]
        , 'JOB': ['Teacher', 'Employee', '']
        , 'CLUSTER_JOB': ['CL_1', 'CL_2', 'CL_2']
        , 'MONTHS': [50.0, 650.0, 300.1]
        , 'AGE': [17.0, 102.0, 51.0]
        , 'SEX': ['M', 'F', '_N']
        , 'CODE_AREA': ['AREA 1', 'AREA 2', 'AREA 3']
        , 'N_TREE': [0, 30, 100]
        , 'P_TREE': [550.03, 1500.8, 1000.1]
        , 'SUM_ITEMS': [59, 1, 800]
    }

    df = pd.DataFrame({k: v * 1000000 for k, v in data.items()})
    print(df.shape)

    try:
        df = schema.validate(df, lazy=True)
    except Exception as exc:
        pass

    gc.collect()

main()

I multiplied the toy data by 1 mill to simulate the data size of your real-world situation.
So with pandera 0.5.1, it looks like there's about a 300% memory overhead when
calling validate 🤯.
Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
     4     38.1 MiB     38.1 MiB           1   @profile
     5                                         def main():
     6
     7     97.3 MiB     59.2 MiB           1       from __toy_validation_schema import schema
     8     97.3 MiB      0.0 MiB           1       import pandas as pd
     9     97.3 MiB      0.0 MiB           1       import numpy as np
    10     97.3 MiB      0.0 MiB           1       import time
    11
    12     97.3 MiB      0.0 MiB           1       data = {
    13     97.3 MiB      0.0 MiB           1           'TYPE': ['TEST01', 'TEST02', 'TEST03']
    14     97.3 MiB      0.0 MiB           1           , 'CODE_STRING': ['A', 'B', 'C']
    15     97.3 MiB      0.0 MiB           1           , 'FLAG': [0, 1, 0]
    16     97.3 MiB      0.0 MiB           1           , 'JOB': ['Teacher', 'Employee', '']
    17     97.3 MiB      0.0 MiB           1           , 'CLUSTER_JOB': ['CL_1', 'CL_2', 'CL_2']
    18     97.3 MiB      0.0 MiB           1           , 'MONTHS': [50.0, 650.0, 300.1]
    19     97.3 MiB      0.0 MiB           1           , 'AGE': [17.0, 120.0, 51.0]
    20     97.3 MiB      0.0 MiB           1           , 'SEX': ['M', 'F', '']
    21     97.3 MiB      0.0 MiB           1           , 'CODE_AREA': ['AREA 1', 'AREA 2', 'AREA 3']
    22     97.3 MiB      0.0 MiB           1           , 'N_TREE': [0, 30, 100]
    23     97.3 MiB      0.0 MiB           1           , 'P_TREE': [550.03, 1500.8, np.nan]
    24     97.3 MiB      0.0 MiB           1           , 'SUM_ITEMS': [59, -1, 800]
    25                                             }
    26
    27    735.6 MiB    638.3 MiB          15       df = pd.DataFrame({k: v * 1000000 for k, v in data.items()})
    28    735.6 MiB      0.0 MiB           1       print(df.shape)
    29
    30    735.6 MiB      0.0 MiB           1       try:
    31   2796.9 MiB   2061.3 MiB           1           df = schema.validate(df, lazy=True, inplace=False)
    32   2796.9 MiB      0.0 MiB           1       except Exception as exc:
    33   2783.5 MiB    -13.4 MiB           1           pass
    34
    35    621.9 MiB  -2161.6 MiB           1       gc.collect()

It doesn't seem to matter that inplace is True or False.
Now with the changes in #360, we can reduce the overhead to about 100%:
Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
    27    739.9 MiB    638.4 MiB          15       df = pd.DataFrame({k: v * 1000000 for k, v in data.items()})
    28    739.9 MiB      0.0 MiB           1       print(df.shape)
    29
    30    739.9 MiB      0.0 MiB           1       try:
    31   1432.9 MiB    693.0 MiB           1           df = schema.validate(df, lazy=True, inplace=False)
    32   1432.9 MiB      0.0 MiB           1       except Exception as exc:
    33   1358.5 MiB    -74.4 MiB           1           pass
    34
    35    475.4 MiB   -883.2 MiB           1       gc.collect()

And even more if inplace=True
Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
    27    740.0 MiB    638.4 MiB          15       df = pd.DataFrame({k: v * 1000000 for k, v in data.items()})
    28    740.0 MiB      0.0 MiB           1       print(df.shape)
    29
    30    740.0 MiB      0.0 MiB           1       try:
    31   1158.3 MiB    418.3 MiB           1           df = schema.validate(df, lazy=True, inplace=True)
    32   1158.3 MiB      0.0 MiB           1       except Exception as exc:
    33   1083.9 MiB    -74.4 MiB           1           pass
    34
    35    498.3 MiB   -585.6 MiB           1       gc.collect()

The remaining memory overhead is due to the failure cases being stored in memory, so if we modify the data so that
there are no failure cases:
data = {
    'TYPE': ['TEST01', 'TEST02', 'TEST03']
    , 'CODE_STRING': ['A', 'B', 'C']
    , 'FLAG': [0, 1, 0]
    , 'JOB': ['Teacher', 'Employee', '']
    , 'CLUSTER_JOB': ['CL_1', 'CL_2', 'CL_2']
    , 'MONTHS': [50.0, 550.0, 300.1]
    , 'AGE': [17.0, 110.0, 51.0]
    , 'SEX': ['M', 'F', '_N']
    , 'CODE_AREA': ['AREA 1', 'AREA 2', 'AREA 3']
    , 'N_TREE': [0, 30, 100]
    , 'P_TREE': [550.03, 1500.8, 100.1]
    , 'SUM_ITEMS': [59, 1, 800]
}

Then we get somehow get negative overhead (?) I'm still wrapping my head around this one.
Line #    Mem usage    Increment  Occurences   Line Contents
============================================================
    27    740.0 MiB    638.4 MiB          15       df = pd.DataFrame({k: v * 1000000 for k, v in data.items()})
    28    740.0 MiB      0.0 MiB           1       print(df.shape)
    29
    30    740.0 MiB      0.0 MiB           1       try:
    31    643.0 MiB    -97.0 MiB           1           df = schema.validate(df, lazy=True, inplace=True)
    32                                             except Exception as exc:
    33                                                 print(exc)
    34
    35    643.0 MiB      0.0 MiB           1       gc.collect()

Summary

#360 as of now minimizes dataframe/column views and copying, reducing memory overhead by about 70%.
Failure cases are responsible for the other 30% of the memory usage. I need to think about how to handle this
more elegantly, for example providing the option to write the failure cases out to disk in a flatfile or sqlite db
so that it isn't sitting in memory. Another possibility is to provide an option to summarize the failure cases in
a more compact way so as not to store every single failure cases and column/index position.
Any thoughts you have would be welcome!

I'll be cutting the 0.6.0 release with at least the changes in #360 in the next week or so. In the mean time,
one work-around would be to chunk up your data file (e.g. if you're reading the data with pandas.read_{csv,sql}
you can use the chunksize argument and you can validate the data in chunks.

Thank you @cosmicBboy for your full support! Waiting for release 0.6.0, in the meantime I'll follow your suggestion.
PS: maybe 'negative' overhead is for data type coercion which produces a more memory efficient object?

@superflexo FYI I was able to sneak memory performance gains into #360 by limiting the failure case reporting with the n_failure_cases argument in the Check definition (which already existed, but wasn't being used 🤦🏾 ), so that should achieve the negative overhead I mentioned above. Setting n_failure_cases=None will report all failure cases, which you probably don't want to do with large dataframes :)


Fast hash/equality for Model objects

As a part of performance work, I have been looking into generating hash functions for Model objects (see my WIP branch). It isn't too bad to make a basic implementation, but it is very slow, taking almost an entire second cumalitively computing hashes for each edit in the large JSON benchmark.
My first approach here is to generate "identityHash" functions which do lookups on the node object for each known property, recursively calling "identityHash" on all the nested objects, for example Interface.identityHash looks like this:
  /// Hash code for comparing instances of this extension type.
  int get identityHash => Object.hash(
        Object.hashAll((node['members'] as Map?)
                ?.cast<String, Member>()
                ?.entries
                .map((entry) =>
                    Object.hash(entry.key, entry.value.identityHash)) ??
            const []),
        (node['thisType'] as NamedTypeDesc?)?.identityHash ?? 0,
        Object.hashAll((node['metadataAnnotations'] as List?)
                ?.cast<MetadataAnnotation>()
                ?.map((entry) => entry.identityHash) ??
            const []),
        (node['properties'] as Properties?)?.identityHash ?? 0,
      );

Ultimately the result of this is that even cached macro phases take an unacceptable amount of time (multiple milliseconds), so we will need to come up with something faster and evaluate exactly what is making this so slow.

Some ideas:

It is probably faster to iterate the entries - although we would have to do a switch for the keys to cast things to the correct extension types.
We could also potentially try just using the deep hashing functionality from package:collection and work at the raw map level.
We could potentially build up a hash in the JsonBufferBuilder as we build objects, but unless we also serialize that hash on the wire it would require reading the entire payload each time when we first receive a buffer.

Hmmmm when the query finishes on the host, the Model should be the only thing in the buffer, e.g. after the initial query here:
https://github.com/dart-lang/macros/blob/178c59faae7b6d8a4c3542ff36987bb761a54ec9/pkgs/_analyzer_macros/lib/macro_implementation.dart#L110
the host could then hash the whole buffer and send the result. It'd be nice to have it as a field on Model, we can only do that today by having a Map field since those can be added to after creation.
That's assuming that hashing the buffer as raw bytes turns out to be fast, I hope so :)
Hashing raw bytes will only do what we want if the bytes are sufficiently deterministic, I don't know if that will be the case, but it's a lot easier than hashing objects so I guess it's worth a try :)

We could also potentially try just using the deep hashing functionality from package:collection and work at the raw map level.

I did give this a shot and it was much worse, fwiw (about 3x worse).

It looks like just hacking something in to grab the raw bytes and hash those is definitely faster, 150ms or so per edit for the large json example, cumulatively spent doing hash computations. I also edited my branch to save the hash instead of the Model objects for the cached responses, and compare against that, so we do about half as many total hashes.
Still a lot of time to spend hashing, and I don't love that it is dependent on specific ordering in which the buffer is built up (technically, my other approach also was though, but it would have been relatively trivial to make it not so).

I read a bit about what the google3 build does, there is a public doc about a specialized hash, PSHA2, it's based on SHA256 with tweaks to add parallelization so it can run using SIMD instructions, i.e. it uses parallelization within a single CPU. The parallel part of psha256 kicks in for message sizes of 1024 bytes so it looks like it won't be too hard to hit it.
Comparing the command line versions of md5sum, sha256sum, sha1sum and psha2sum on my machine using 1GB of input: md5 hits 0.54GB/s, sha256 is 1.08GB/s, sha1 is 1.23GB/s and psha256 is 1.56GB/s.
Using the package:crypto example script AOT-compiled the numbers are: md5 0.17GB/s, sha1 0.10GB/s, sha256 0.08GB/s
Based on these numbers, if you were using the package:crypto implementations, it looks like there would be about 9x speedup with native-performance PSHA2, 7x with native-performance SHA1, 3x with native-performance md5sum.
I wonder how much of that native-performance boost we could get with ffi? But actually these are so important, I could see an argument for directly supporting them in the platform, e.g. put them in dart:io.

Some bazel discussion https://github.com/bazelbuild/bazel/discussions/22011 mentions blake3, which looks even faster, b3sum hits 7.87GB/s, mostly because it also parallelizes across multiple cores: with 1 thread (b3sum --num-threads 1) it's 1.83GB/s which is just a bit faster than PSHA2, then it gets considerably faster as I add 1-3 more threads.

Do you really need to compute hash to compare things? Can't you just (for example) to byte-for-byte comparison of the incoming data? It seems you are just using it a proxy for equality anyway.
That being said: I puzzled as to why we are trying to solve all these problems externally to the tools which are supposed to have all information necessary to short cut the what has changed computation. CFE is supposed to have fine grained incremental recompilation of the dependencies, analyzer does not - but should have it. It seems like we are reinventing the same calculation with macros specific twists.

Do you really need to compute hash to compare things? Can't you just (for example) do a simple byte-by-byte comparison of the incoming data? It seems you are just using it a proxy for equality anyway.

If a match means there is no more work to do then it's nice to get a match based on hashes, because then you can get a match without having to store all possible matches as full data.

That being said: I puzzled as to why we are trying to solve all these problems externally to the tools which are supposed to have all information necessary to short cut the what has changed computation. CFE is supposed to have fine grained incremental recompilation of the dependencies, analyzer does not - but should eventually implement it anyway. It seems like we are reinventing the same calculation with macros specific twists.

Neither analyzer nor CFE has a data model that's immediately useful to macros, because they are private to the analyzer and CFE, which means you can't code against them--they change. So, macros have their own data model that is public and stable. (The JSON representation and corresponding binary format and extension types).
Macros describe what data they need as a query, the host (analyzer or CFE) converts its own data model to the macro model and sends it in response.
Macros usually only care about a part of the code, for example fields in classes with a particular annotation and their types, so what each macro receives is significantly cut down from the full host model. This also means that it should be very common that when a file changes the macro does not have to rerun: something changed but it wasn't what the macro cares about.
This investigation is about noticing that the data being sent to a macro is the same as last time, and so the output from last time can be reused. "The same as last time" is easy to check by keeping a hash from last time and comparing.
It's true that we could perhaps optimize further by pushing some part of the "same as last time" check before the conversion to the macro data model, so for example if the CFE could compare what changed against the macro query before it even starts to do the conversion. But this would be a lot more work to do, and it's possible than convert-then-compare gets us most of the performance, so we obviously check that first.

That being said: I puzzled as to why we are trying to solve all these problems externally to the tools which are supposed to have all information necessary to short cut the what has changed computation.

We have to implement it twice that way, which is extra work as well as a larger potential for bugs. The question of "what does this macro depend on" is fairly challenging to answer, and the query results hash model is quite trivial and reliable.
It is probably a lot more expensive, especially in terms of memory, to maintain back-links across the program to the macro applications which depend on those nodes. The current solution we are exploring just needs to store a single hash per query that the macro runs. We do trade off CPU cost for that, to recompute the queries and hash them, absolutely. We are just exploring now the actual cost of re-running those queries and hashing the results.
The hashing approach also could be more compatible with re-using macro results in modular compiler. Although we probably won't actually do this.

I hooked up the CPU profiler, from head but with https://github.com/dart-lang/macros/pull/134 applied.
Here are some noteworthy things, from a profile spanning a single incremental edit:

about 27% of the total time is still spent computing hashes. We definitely need to improve that.

I don't know exactly all this is doing, but it seems pretty expensive and if we were more deeply integrated we could possibly avoid re-doing this work when it is cached cc @scheglov .

We are spending >10% of the time just re-running the queries to see what has been changed. It looks like almost half that time is spent just creating typed maps though (almost all the created objects here are a result of queries), which doesn't even account for all the work spent building up these objects.
This indicates we need to optimize the speed for creating these objects.

This isn't too unexpected, we do expect a lot of the time to go to parsing, since every library file gets re-parsed, and all augmentation code is parsed twice I believe.

One of the previously recorded tree of performance operations in https://github.com/dart-lang/sdk/issues/55784#issuecomment-2185360485 provides details what we do in Linker._mergeMacroAugmentations.

Merge separate macro results into single code.
Parse.
Build unlinked data. Mostly hashing.

See also my previous benchmarks for hashing, Dart vs. Rust.
Or, maybe more fairly, different algorithms, some of which can be optimized for modern CPUs.

See also my previous benchmarks for hashing, Dart vs. Rust.

Fwiw, the actual hashing is not the problem in this particular case, it is the work to pull out the interesting bits of the objects that we want to hash that is expensive.
In my PR I am just using Object.hash and Object.hashAll, at least initially, and they consume combined only about 300ms, which isn't nothing to be sure but its not the primary issue by any means.

Re #147 which uses md5.
I have never tried FFI before, I figured it's about time and a good use of a Friday afternoon :)
I got a random md5 C implementation working easily enough
https://github.com/dart-lang/core/commit/89faa07d8f1b74ca79c3fc929574148b76e51309
it's 2x faster than the Dart one
> dart example/example.dart
 FFI: 71ms
9be5897a856106bf8fc9ef292366a3ac
 FFI: 71ms
9be5897a856106bf8fc9ef292366a3ac
...
...
Dart: 130ms
9be5897a856106bf8fc9ef292366a3ac
Dart: 145ms
9be5897a856106bf8fc9ef292366a3ac

it would be interesting to try one of the newer+faster hashes with a C implementations like blake3. (I notice though that the blake3 C implementation says that only the rust implementation provides multithreading).
Total FFI newbie, as I mentioned, but my newly-gained understanding is that if we allocate bytes natively (FFI "malloc") then you can treat the bytes on the Dart side as a Uint8List.
So we could write JSON directly into one, and there would be no copying to hash it. This assumes we can hash the whole buffer, of course, which is not something we can do yet, we'd have to write a new buffer just for hashing. But it should be fast.
The reason you have to allocate the bytes natively then use as a Uint8List, rather than the other way around, is that bytes allocated on the Dart side can be moved by the garbage collector, while bytes allocated natively can't.

@davidmorgan you can mark your hashing function as a leaf, that would allow you to do md5(buffer.address) even if buffer is allocated in the Dart heap because leaf functions don't allow GC to happen while they execute.

Thanks Slava! I saw a reference to "leaf" functions, but didn't know it was something I can just opt into. That makes sense.
I managed to hack a working blake3 example, too.
https://github.com/davidmorgan/core/tree/crypto-example-blake3/pkgs/crypto (build with: cmake -DBUILD_SHARED_LIBS=ON . && make)
For the example above it hits ~32ms, so another 2x speedup.
The example does lots of small (2000 byte) hashes which does not hit maximum throughput, the difference gets much bigger if we do a small number of large hashes, e.g. for one hash of a billion bytes
  FFI blake3: 291ms
570a1dd602f4c46553f0c4c211b4d08e32d8013425dda2cf18fcd18262cda24c
  Dart md5: 6182ms
66b64ed98371d30f6a48c0a9b0b8b6cd

Note that the MD5 approach in https://github.com/dart-lang/macros/pull/147 uses chunked encoding, which might not translate as well to the FFI approach, since there are many very small chunks.
But it might translate well to the approach where we build up a buffer just for hashing. Then we are also more order dependent though, and I am not sure we want to rely on that, although it might turn out that things are pretty deterministic already if the analyzer/cfe have a deterministic ordering of members.


[bug] Re-use central point as initial guess for displacements in Hessian calculation

The Hessian and frequency calculations can give incorrect results, as discussed in summer school. It needs to be run as a single point at the given geometry. Then the calculations are displaced geometries can be run in parallel, using the wave function from the central point as the initial guess for those calculations.

Wavefunction can now be passed as initial guess to distributed algorithms. See PR51 and bugfix PR52


Fix potential overflow of casting

Pull Request Checklist
Please leverage this checklist as a reminder to address commonly occurring feedback when submitting a pull request to make sure your PR can be reviewed quickly:
See the detailed list in the contributing guide.

[x] C++ Guidelines
[x] Doxygen docs
[x] Unit tests
[x] No unwanted commits/changes
[x] Descriptive title/description

[x] PR is single purpose
[x] Related issue listed

[x] Comments in source
[x] No typos
[x] Update changelog
[x] Not work-in-progress
[x] External references or docs updated
[x] Self review of PR done
[x] Any breaking changes?

Hi @ahsonkhan , I want to use uint64_t for all sizes, and int64_t for all offsets (within storage sdk, so core sdk will not be affected). The API interfaces that are public to customers will remain int64_t so our customers are not affected.
The advantage is that I no longer need to consider the comparison between int64_t and size_t. (which is nightmare because on 32bit platform, size_t will be promoted to signed 64 bit integer, while on 64bit platform, int64_t will be converted to unsigned)
This can also save us a lot of static_casts. What do you think?

/azp run cpp - storage


Fixing error in kube smoke tests

Description
Fixes the test failure in test_04_autoscale_kubernetes_cluster

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

Results
TestName: test_04_autoscale_kubernetes_cluster | Status : SUCCESS

@blueorangutan package

@blueorangutan test


godot: root scale change

See https://github.com/godotengine/godot-proposals/issues/4379 for the exact details, but resolved.


Add an uninstall step for xcpretty for travis runs.

https://github.com/google/EarlGrey/issues/302

LG

Xcpretty 0.2.4 released with fix for this, sorry about that.
Feel free to revert


Donor login

Description:
Donors are unable to log in to their accounts via the donor login portal. Upon entering valid credentials, the system either returns an error or redirects them back to the login page without any notification.
Expected Behavior:

Donors should be able to log in using their registered user and password.
Upon successful login, donors should be redirected to their donor's homepage.
Proper error messages should be displayed if login fails due to incorrect credentials.

Steps to Reproduce:

Navigate to the donor login page.
Enter valid user and password.
Click the "Login" button.

donor login and donor bank is not the same?


CFLint JAR became 22MB again

michal.paluchowski [4 hours ago]
only the runtime is needed after the antlr Maven plugin runs
michal.paluchowski [4 hours ago]
the full antlr includes com.ibm.icu:icu4j:jar:58.2 in its dependencies, which adds a whopping 12MB to the final JAR size
michal.paluchowski [4 hours ago]
the full antlr dependency was only needed in tests, hence the dependency in test scope
michal.paluchowski [3 hours ago]
plus, cfparser depends on both runtime 4 and 3, because there's both code using one or the other present

I wonder if @ryaneberly could fix it quickly and we push out a 1.2.3 (and also fix the issue with the wrong version number inside of 1.2.1)?

Solved in CFlint release 1.2.3