upgrade snowball to 34f3612e5e8c (round two)

Forever we've accumulated various performance differences to the snowball code.
These are now integrated upstream, see https://github.com/snowballstem/snowball/pull/195
Since we aren't modifying this snowball code in crazy ways, all the dictionary-based testing is removed: snowball tests this stuff itself well.
I will be following up with more changes, to try to simplify how we integrate with snowball, by sending upstream more odds-and-ends until we can get all this minimal.

@uschindler How can i exclude the org.tartarus code from this check (or really all checks) without touching it?
Forbidden method invocation: java.util.Arrays#copyOf(**) [Prefer using ArrayUtil as Arrays#copyOf fills zeros for bad bounds]
> Task :lucene:analysis:common:forbiddenApisMain FAILED
  in org.tartarus.snowball.SnowballProgram (SnowballProgram.java:337)
Scanned 653 class file(s) for forbidden API invocations (in 0.81s), 1 error(s).

@uschindler How can i exclude the org.tartarus code from this check (or really all checks) without touching it?
Forbidden method invocation: java.util.Arrays#copyOf(**) [Prefer using ArrayUtil as Arrays#copyOf fills zeros for bad bounds]
> Task :lucene:analysis:common:forbiddenApisMain FAILED
  in org.tartarus.snowball.SnowballProgram (SnowballProgram.java:337)
Scanned 653 class file(s) for forbidden API invocations (in 0.81s), 1 error(s).

Yes you can, something like that in forbiddenapis.gradle:
allprojects { prj ->
  plugins.withId("java", {
     //.....
    if (prj.path==:lucene:analysis:common') {
      forbiddenApisMain.exclude('org/tartarus/**')
    }
    //....
  }
}

thanks, i will do this. It isn't an option to call lucene ArrayUtil methods etc from snowball code.

thanks, i will do this. It isn't an option to call lucene ArrayUtil methods etc from snowball code.

And we also won't if it starts being published as jar and used as a normal dependency. It isn't our code.

I know, that seems crazy, but I think it would be the ultimate goal. Will have to find an "easy" / "dead-simple" way to achieve publishing snowball properly that avoids craziness of maven etc. They have a simple make-based build and support MANY languages, but e.g. they publish their python bindings to pypi.
For now I'm just trying to just deal with smaller changes, reduce the size of these differences and remove hacks and stuff.

Speaking for snowball, we'd likely take patches adding useful code quality checks.  If they're rather dependency heavy we could do them only in CI.


Create clusters for training and validation as well as for testing set

We should think about how to create better sets for training and validation, since it's evident that right now the optimized validation loss is not working properly for the testing set, and thus the networks do not really generalize, but overfits on the training data.
We should do this for

[ ] Peptides clusters
[ ] Alleles clusters

We can start here a discussion about it. Which are the issues on doing so right now?

Peptide clustering was scraped from the project. Allele clustering validation set could still be implemented for future experiment, but it's not planned at the moment.


Implement Mock Data provider for API Testing

Implement Mock Data provider for API Testing
Description
Currently, when testing API endpoints in the API Dash repository, users must manually enter or modify details such as email, username, and other user-specific information. This manual effort can be tedious and error-prone, especially during repeated testing. To streamline this process, we propose implementing a feature that generates random placeholder values for common fields like email, username, etc.
Feature Details
Introduce a set of predefined random data variables (e.g., {{$randomEmail}}, {{$randomUsername}}) that can be used in the API requests. These placeholders will automatically populate with randomized data when the API is called, allowing testers to quickly simulate unique user details without manual input.
Benefits

Increased Efficiency: Simplifies testing by automating data entry, reducing repetitive manual work.
Consistency: Standardizes testing by providing consistent random values across sessions.
Enhanced User Experience: Enables faster and more reliable API testing workflows.

Implementation Suggestions

Define random generators for various data types (e.g., email, username, numeric ID).
Update the API to recognize and replace variables like {{$randomEmail}} with generated values.
Ensure variables can be used across multiple endpoints, maintaining flexibility and ease of use.

This feature will greatly enhance developer productivity and provide a more seamless testing experience in the API Dash environment.

Can I work on this issue???

You can mention the below details in this thread:

Where are you planning to add this feature in the UI? How will the user know this feature is available ?
List of random data you will provide - how will you provide it? what package will you use?
We already have environment variables .. will you leverage it?

Implementation Details

UI Addition:
This feature will be accessible within the request body editor. Users can invoke it by typing {{ (double curly brackets), which will reveal the list of available random placeholders.

Random Data Types:
Initial options will include random values for email, username, names, and potentially others. The faker package in Flutter will be used to generate this data without needing to modify environment variables.

Package Used:
I will utilize the faker package for Flutter to generate these random values.

Environment Variables:
No need to leverage environment variables for this feature, as the faker package provides sufficient randomness for testing purposes.

As you are a first time contributor to this project, I recommend you to pick up an open issue first and resolve it to get familiar with the codebase, before working on adding this completely new feature.

Alright, Thanks for your response.


Remove power assert and refactor tests with jest

power assert do not show anything helpful in addition to jest expect.

Cool -- nice one.


Mastermind_Mania Game Added

PR Description 📜
New Game: Mastermind_Mania Added
Fixes #855

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for website preview in assets/images
[x] I have added entries for my game in main README.md
[x] I have added README.md in my folder
[ ] I have added working video of the game in README.md (optional)
[x] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you @Sheetal-05 ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

Thank you  @Sheetal-05 , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀

Hey @Sheetal-05,
You can't work on a closed issue 👀!
Follow the proper guideline and make a new PR again 😀.
Happy Hacking 💗


feat: allows for passing of sound name.

allows for passing trough sound_name:
toastify send  "Title" "Body message" -s "submarine"

woop thank you

Any chance we can get a new version published that includes this feature please?

there we go
https://crates.io/crates/toastify/0.5.3


Changing month closes date picker on Chrome Version 62.0.3202.89

I downloaded the latest version of Chrome, version 62.0.3202.89, and using this breaks our date picker.
When I attempt to change months by selecting the previous month or next month arrow, the date picker closes.
My colleague has just pointed out that this issue is not present on the website: http://amsul.ca/pickadate.js/date/ on the latest version of Chrome, so it may be an issue with our system and I will verify. Thanks folks.
Update 1: Seeing an issue here: https://github.com/amsul/pickadate.js/blob/master/lib/picker.js#L266.
This section of code is only triggered in the latest version of Chrome.
Update 2: target is returning the incorrect value on line https://github.com/amsul/pickadate.js/blob/master/lib/picker.js#L268 in Chrome but I am not sure why? I think again that this could be an issue with our system.
Update 3: Using var target = event.currentTarget instead of var target = event.target seems to work on the same line mentioned above. However, the date picker will not close unless a date is selected.
Update 4: I have a input field on the same screen as the datepicker that is of type password with autocomplete off. I think this is the issue. If I remove this field or change it to disabled there is no issue with the date picker.
Update 5: We believe there is an issue with autocomplete in Chrome, we have a password input field on the same screen that uses autocomplete="off" but we think we should be using autocomplete="new-password" and that this was somehow interfering with the date picker in Chrome. I will confirm this.
Final update: So autocomplete will cause an issue with Chrome in the latest version, possibly since earlier this year, if you also have a password input field on the same view with autocomplete="off".

+1

I am having the exact same problem on computers with Chrome 62. What completely threw me off was that it worked fine in incognito mode. I also converged on the cause being the password fields in the page but thanks to @adrianmann, I was able to workaround this by adding the autocomplete="off" attribute to my password fields.
No other attribute value would work for me. Setting the autocomplete attribute to off, on, current-password, would not work, only new-password on all of the password input fields in the DOM would fix this.
Still baffled by this.

I am having the exact same problem. I was struggling because my password fields are hidden... but still present in my DOM. As @bbbbburton said, you have to put autocomplete="new-password" in all the password fields from the DOM.
I still find that it's a hack, since I would still want my passwords to autofill... Any solution to this?
I don't know if it's connected but I found a bug that seems related in another datepicker and that was fixed... might help?

I am having the same issue. I do have autocomplete="off" since I am also using a typeahead plugin and Chrome's typeahead would clash with my plugin's dropdown. Unfortunately, the autocomplete="new-password" workaround won't work for me because I don't have the autocomplete set on a 'type="password"' but a 'type="text"'(for my search bar). This search bar is in the same form as my datepicker so I guess that's why I am having this issue with the calendar disappearing.
Any other solution to this problem? Has anyone else experienced this same problem where the autocomplete is deactivated for a text input and not password?

Same issue here. I found the culprit:
When changing the month, the picker gets re-rendered. (basically pickerbox.html(html_string)).
This is where the bug is. Chrome somehow thinks this re-rendering has something todo with any login forms on page and perfoms and autofill (where the inputs get marked yellow). This triggers an focusin event on the login inputs, and since the picker listens for focusin events in order to close itself if a different input has been focused, it closes.
The workaround for now, is to simply ignore any focusin events that are simulated. (The focusin event triggered by the autofill has the isSimulated flag set to true)
picker.js.fixed.zip


Notify when sparkcore tests are being skipped

I am still not a fan of an environment variable being used for tests outside of integration, but at least with this it will be more obvious to developers that the tests are indeed being skipped.

Current coverage is 63.84% (diff: 100%)

Merging #1726 into master will not change coverage

@@             master      #1726   diff @@
==========================================
  Files           353        353
  Lines         16674      16674
  Methods       16383      16383
  Messages          0          0
  Branches        146        146
==========================================
  Hits          10646      10646
  Misses         6028       6028
  Partials          0          0

Powered by Codecov. Last update 1fcc6db...0d4d17b

I think @rabbitonweb might be on vacation or something, so will assign someone else in the interest of shorting open PR time.

@jedesah LGTM, needs a merge from master. I'll merge once that's done and green.


Using new information for SyntaxError for Python 3.10

SyntaxErrors now include new arguments indicating the end of a range; the full arguments are
e.msg, e.lineno, e.offset, e.end_lineno, e.end_offset

The offset attribute can now point to a location prior to that pointed at in previous Python version - where a single position (line, column) was shown instead of a range, highlighting part of an expression causing problems.
This change makes it more useful for users of cPython ... but it messes up the analysis currently done by friendly which relies on a single "bad token" begin flagged.
I need to use this information to do the same kind of highlighting that cPython does. Additionally, I need to figure out how to use this information in a way that is compatible with the analysis I was doing before.
This is discussed in https://github.com/python/cpython/pull/25582/files, but has not made it to the What's New section yet.

Just thinking about how to go about this ... Perhaps what I can do is check if these attributes exist. If they do not exist, I can artificially add them, something like:
e.end_lineno = e.lineno (or None)
e.end_offset = e.offst + 1

This would likely make the analysis going forward much easier when new Python versions are introduced.

This has been taken care of in previous commits.


Declare gradle plugin in root project, use only in sub-projects

Description of the issue:
Is it possible to declare jib plugin dependency in a root project and then use the plugin only in sub-projects?
Given the following configuration:
//root project
plugins {
    id 'com.google.cloud.tools.jib' version '0.9.0'
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'com.google.cloud.tools.jib'
}

Command gradle :root:subproject:jib yields
Caused by: org.gradle.api.UnknownTaskException: Task with path 'classes' not found in project ':services'.Close stacktrace
at org.gradle.api.internal.tasks.DefaultTaskContainer.getByPath(DefaultTaskContainer.java:382)
at com.google.cloud.tools.jib.gradle.JibPlugin.lambda$apply$0(JibPlugin.java:61)

Expected behavior:
I expected a successful completion.
Other:
The plugin works fine if I declare it in the sub-project.
I am not certain whether this is my misuse of Gradle/Plugin or the plugin needs some fine-tuning.
Environment:
Gradle 4.8

Thanks for reporting this! I think this is a bug on the way the plugin by default depends on the classes task. We'll take a look into this and provide a fix soon.

Hi @coollog, the root project does not contain any sources and therefore does not have the java plugin declared.
I did not realise that the declaration in the plugins {} block already activated Jib plugin.
I fixed the issue by applying the apply false.
//root project
plugins {
    id 'com.google.cloud.tools.jib' version '0.9.0' apply false
}

Sorry, false flag.


race condition mounting docker-in-docker socket in build

When using docker-in-docker, there is a chance that dind hasn't started when a build is requested. If this happens, the volume mount to load /var/run/dind/docker.sock into the build container could occur before dind has created the socket, in which case the volume mount will have its default behavior of creating a directory at the mount point. If this happens, docker-in-docker will be inaccessible until /var/run/dind is cleaned out manually and the dind pod is restarted.
Two possible fixes:

postStart hook to add a node label and use nodeSelectors to schedule builds only where dind is known to be up and running. This should fix the issue robustly, but requires elevated permissions to modify the node.
mount directory rather than the file. This will still result in a failed build when the race occurs, but at least it won't mess up the socket for all subsequent builds on the node. Downside: when not using dind, the directory to mount is /var/run, which can have lots of stuff in it.

This should have been fixed by #539 but doesn't seem to have done. It's possible the same fix is needed elsewhere, or we need a kube version bump on the nodes.

Hopefully this is fixed, if not please reopen!


WFLY-18125: Upgrade HAL to 3.6.9.Final

https://issues.redhat.com/browse/WFLY-18125

retest this please

Thanks @hpehl and @darranl


Need to show descriptions of each plan

On the /configure screen, there is currently no way for the user to differentiate the different plans. We need to render a "?" icon that opens a drawer or tooltip with more information. We don't want to simply redirect the user to the cloud-pricing page, though.

Over in the cloud-cms I have opened a PR here to add a plan description field.
Thenon this branch - I've injected a drawer toggler into each line of the planSelector and added a drawer that renders the plan name + description.
TODO:

Add icon (we have 'info' as a placeholder for now)
Any further styling updates

@JessChowdhury @jacobsfletch What if instead of a drawer for each plan, we add a button 'Compare Plans' that opens up a drawer that shows the comparison chart? Would offer much less clicking I think.

@jacobsfletch draft PR here: #245.

@tylandavis I think this is probably a nice-to-have so if you can't get it working, the main goal here is to simply display the information to the user. They can still select another plan after exiting the drawer. BUT that being said, this should be pretty straightforward to implement. The drawer state needs to be entirely controlled by the checkout page, so you pass the value into the drawer, but then also lift it's changes out of the drawer back into the checkout. Something like this:
const [state, setState] = useState();

<MyDrawer
  value={state}
  onChange={setState}
/>

Closed with #245


Update Language Pack

Added 7 translations.
Change 1 translation in Magento_Customer since i think this is more proper german term.

Thank you so much!


Top bar is hiding the headings in CHROME.

I can see that top bar on the page is covering the headings after click on the PAGE CONTENTS menu on the right.:
Example: RDS -> UMN -> Introduction -> Basic Concepts
https://docs-int.otc-service.com/relational-database-service/umn/introduction/basic_concepts.html#rds-01-0015
The main title is displayed correctly after click on the LEFT MENU.

Heading of the section that was picked on the PAGE CONTENTS menu on the right is hidden.

Thanks for the issue. We are trying to fix that.


Add Travis CI build

Changed log

Add Travis CI build. The Travis CI build log is available here.
Using the class-based PHPUnit namespace to be compatible with the latest stable PHPUnit version.

Thanks!


fix(Imports): :ambulance:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.007%) to 79.267%

Totals

Change from base Build<PHONE_NUMBER>:
-0.007%

Covered Lines:
1876

Relevant Lines:
2293

💛  - Coveralls


driver doesn't inform nomad about jail status

the jail is running, but nomad alloc status shows it still 'Pending'.
zsh@userfbsd:~/testjobs nomad run basicjail.nomad
==> Monitoring evaluation "b6bd4ae3"
    Evaluation triggered by job "bj-job"
    Allocation "dc4fa081" created: node "3e5c4135", group "example"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "b6bd4ae3" finished with status "complete"
zsh@userfbsd:~/testjobs nomad alloc status dc4fa081
ID                  = dc4fa081
Eval ID             = b6bd4ae3
Name                = bj-job.example[0]
Node ID             = 3e5c4135
Node Name           = userfbsd
Job ID              = bj-job
Job Version         = 0
Client Status       = pending
Client Description  = No tasks have started
Desired Status      = run
Desired Description = <none>
Created             = 37s ago
Modified            = 37s ago

Task "bj-task" is "pending"
Task Resources
CPU      Memory   Disk     Addresses
100 MHz  300 MiB  300 MiB  http: X.X.X.X:5678

Task Events:
Started At     = N/A
Finished At    = N/A
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type                   Description
2019-07-11T10:27:53-07:00  Downloading Artifacts  Client is downloading artifacts
2019-07-11T10:27:53-07:00  Task Setup             Building Task Directory
2019-07-11T10:27:53-07:00  Received               Task received by client
zsh@userfbsd:~/testjobs jls
   JID  IP Address      Hostname                      Path
   132  X.X.X.X     bj-task-dc4fa081-8a1d-4084-80 /tmp/NomadClient026711311/dc4fa081-8a1d-4084-80cf-d2a05dc511c1/bj-task
zsh@userfbsd:~/testjobs

as we can see, the nomad status thinks it's still Pending, but jls clearly shows it running (and so does ps auxww).  So I imagine there is some status that has to be sent back to Nomad?

PR https://github.com/cneira/jail-task-driver/pull/11 should fix this.

Yup! thanks! <3


fix: Allow on demand instance type to be launched if cheaper than spot price

Fixes 4923
https://github.com/aws/karpenter/issues/4923
Description
Ensure the cheapest instance type is launched for mixed capacity type configurations, if spot is cheapest it should be launched or if on demand is cheapest it should be launched. Currently for mixed capacity type provisioners regardless of price spot instances will be launched if they are available to be launched, on demand instances would only be launched if there was no availability for spot for the selected instance type(s)
This change modifies the pricing api to return an error when the spot price for an instance type/az is more expensive than the on demand price for the same instance type.  This means these spot instance types will not be considered available to launch, but the on demand instance type can still be launched if it is the most appropriate.
How was this change tested?

[x] Added unit tests for change to pricing lookup
[x] Added unit tests to ensure correct instance type is launched when on demand is cheaper than spot pricing based on aws pricing api responses
[x] Extended existing unit tests to ensure correct instance type is launched when on demand is more expensive than spot pricing based on aws pricing api responses
[x] Manual testing

For manual testing I scaled up a deployment to force a new ec2 node to be launched by karpenter and check the correct capacity type is launched
2023-10-25T09:51:41.140Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.960000       {"commit": "895504c-dirty"}
2023-10-25T09:51:41.145Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.740500       {"commit": "895504c-dirty"}
2023-10-25T09:51:41.349Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.714000       {"commit": "895504c-dirty"}
2023-10-25T09:51:41.350Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.962500       {"commit": "895504c-dirty"}
2023-10-25T09:51:41.546Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.434500       {"commit": "895504c-dirty"}
2023-10-25T09:51:41.549Z        INFO    controller.pricing      rkirk spot m4.10xlarge - 4.245000       {"commit": "895504c-dirty"}
{"level":"info","ts":1698227502.2016387,"logger":"fallback","caller":"pricing/pricing.go:303","msg":"rkirk ondemand m4.10xlarge - 1.000000"}
2023-10-25T10:16:53.176Z        INFO    controller.provisioner  found provisionable pod(s)      {"commit": "895504c-dirty", "pods": "apps/curl-deployment-f467cbbd7-qd7qf, apps/curl-deployment-f467cbbd7-wjh95, apps/curl-deployment-f467cbbd7-97kdb, apps/curl-deployment-f467cbbd7-58lkf, apps/curl-deployment-f467cbbd7-bst74 and 13 other(s)", "duration": "183.677533ms"}
2023-10-25T10:16:53.176Z        INFO    controller.provisioner  computed new machine(s) to fit pod(s)   {"commit": "895504c-dirty", "machines": 1, "pods": 18}
2023-10-25T10:16:53.190Z        INFO    controller.provisioner  created machine {"commit": "895504c-dirty", "provisioner": "karpenter-on-demand", "machine": "karpenter-on-demand-qxr79", "requests": {"cpu":"39055m","memory":"144536680Ki","pods":"27"}, "instance-types": "c5.24xlarge, c5.metal, c5a.24xlarge, c5ad.24xlarge, c5d.24xlarge and 95 other(s)"}
2023-10-25T10:16:55.893Z        INFO    controller.machine.lifecycle    launched machine        {"commit": "895504c-dirty", "machine": "karpenter-on-demand-qxr79", "provisioner": "karpenter-on-demand", "provider-id": "aws:///us-east-1b/i-00e9d1b2657485a92", "instance-type": "m4.10xlarge", "zone": "us-east-1b", "capacity-type": "on-demand", "allocatable": {"cpu":"39830m","ephemeral-storage":"52926258176","memory":"148623Mi","pods":"234"}}

The following modification was made for the manual testing as I cannot run this in my payee account where on demand is typically cheaper than spot pricing due to discounts that are applied.
diff --git a/pkg/providers/pricing/pricing.go b/pkg/providers/pricing/pricing.go
index 77795975..78aa9bbb 100644
--- a/pkg/providers/pricing/pricing.go
+++ b/pkg/providers/pricing/pricing.go
@@ -299,7 +299,8 @@ func (p *Provider) onDemandPage(prices map[string]float64) func(output *pricing.
 					if err != nil || price == 0 {
 						continue
 					}
-					prices[pItem.Product.Attributes.InstanceType] = price
+					prices[pItem.Product.Attributes.InstanceType] = price * 0.5
+					logging.FromContext(context.Background()).Infof("rkirk ondemand %s - %f", pItem.Product.Attributes.InstanceType, prices[pItem.Product.Attributes.InstanceType])
 				}
 			}
 		}
@@ -334,7 +335,8 @@ func (p *Provider) UpdateSpotPricing(ctx context.Context) error {
 			if !ok {
 				prices[instanceType] = map[string]float64{}
 			}
-			prices[instanceType][az] = spotPrice
+			prices[instanceType][az] = spotPrice * 5.0
+			logging.FromContext(ctx).Infof("rkirk spot %s - %f", instanceType, prices[instanceType][az])
 			InstancePriceEstimate.With(prometheus.Labels{
 				InstanceTypeLabel: instanceType,
 				CapacityTypeLabel: ec2.UsageClassTypeSpot,

Does this change impact docs?

[ ] Yes, PR includes docs updates
[ ] Yes, issue opened: #
[x] No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hey @rkirk-r7, sorry for the delay here.
I understand you've tested this with your code to modify the Spot prices to be 5x more expensive, and On-demand prices 2x cheaper, essentially making every spot price more expensive than its OD counterpart, unless the spot discount is > 90%.

Yes this was just a way to be able to test and change the pricing, it's not reflection of the discounted price we pay.

Is the intent of this PR to be merged after your previous one? If that PR isn't merged, which I believe we discussed some alternatives in WG, then I don't think we should be merging this one. It adds more functionality that is an invariant with spot/OD pricing.
What are your thoughts?

The intent was this would be safe to merge regardless of the price multiplier PR and would not change the behaviour for the non-discounted pricing where spot would be cheaper than on demand.  Whatever approach we take regarding getting more accurate pricing/discount aware pricing be it the multiplier, per instancetype config, assumable IAM role, a.n.other approach then I believe this change would be required.  Whilst the price filtering handles determining the correct pricing between spot and on demand types, the launching code later does not check the price and if the filtered instancetypes have spot Offerings then it will default to launching spot regardless of price.  This change is intended to filter out the spot Offerings which are more expensive than the on demand price so we can launch spot where it is cheapest but fallback to ondemand when it is cheapest.
As an example, if this was the following was the output of filtering the instancetypes by price
{
    "instancetypes": [
        {
            "instancetype": "m6a.xl",
            "offerings": [
                {
                    "capacity-type": "spot",
                    "price": 2.0
                },
                {
                    "capacity-type": "on-demand",
                    "price": 1.0
                }
            ]
        }
    ]
}

Then the current code would launch spot because there is a spot offering for the filtered instancetype(s) without rechecking the pricing.
This change would end up with the filtered list looking like
{
    "instancetypes": [
        {
            "instancetype": "m6a.xl",
            "offerings": [
                {
                    "capacity-type": "on-demand",
                    "price": 1.0
                }
            ]
        }
    ]
}

Which would lead to on-demand type being launched.
Another example would be if the discount means spot is sometimes cheaper, but sometimes on-demand is, so this type of scenario
{
    "instancetypes": [
        {
            "instancetype": "m6a.xl",
            "offerings": [
                {
                    "capacity-type": "spot",
                    "avilability-zone": "az-1",
                    "price": 1.0
                },
                {
                    "capacity-type": "spot",
                    "avilability-zone": "az-2",
                    "price": 1.5
                },
                {
                    "capacity-type": "on-demand",
                    "price": 1.2
                }
            ]
        }
    ]
}

In this case spot would be launched but we would allow it to be launched in either az so could get the good price and could get the more expensive price. If the requirement was for launching in az-2 then the more expensive spot would be used.
This change would end up with the following, which would allow the cheap spot instance if we can launch in az-1 or would then drop back to on demand
{
    "instancetypes": [
        {
            "instancetype": "m6a.xl",
            "offerings": [
                {
                    "capacity-type": "spot",
                    "avilability-zone": "az-1",
                    "price": 1.0
                },
                {
                    "capacity-type": "on-demand",
                    "price": 1.2
                }
            ]
        }
    ]
}

@rkirk-r7 got it, I agree that this would be necessary for when custom pricing overrides are implemented, but I'm wondering if this PR should be blocked on more progress and consensus for https://github.com/aws/karpenter/pull/4697. As I understand it, even though you are paying a lower cost, Karpenter's pricing API isn't aware of this, and will continue to think On-demand is always more expensive than spot, never triggering the error that you've implemented.
I totally understand your desire to get this in as a stepping stone for https://github.com/aws/karpenter/pull/4697, but do you think this PR is valuable to get in if the other doesn't?

@rkirk-r7 got it, I agree that this would be necessary for when custom pricing overrides are implemented, but I'm wondering if this PR should be blocked on more progress and consensus for #4697. As I understand it, even though you are paying a lower cost, Karpenter's pricing API isn't aware of this, and will continue to think On-demand is always more expensive than spot, never triggering the error that you've implemented.
I totally understand your desire to get this in as a stepping stone for #4697, but do you think this PR is valuable to get in if the other doesn't?

I think this PR would work however we get more accurate pricing information, whether thats via #4697 or per instance overlay configuration etc. Kinda feel like i'm a bit stuck with this, because if I can get karpenter to be aware of more accuate pricing then it will not honour that pricing when launching, but can't fix the launching because the current price view always sees spot as the cheapest if its available. So i'm not really sure how to try and take this forward at this point.

I totally understand, and I'm sorry that you've gone through the effort for this, but haven't been able to make progress. I think this is more indicative of the much larger design space that's been teased at here within our discussion and in Working Group. One of the reasons we ask for a design when considering features is to ensure that we can align on a direction before the developers spend their time and energy on implementing.
We can definitely improve on communicating these contributing requirements and guidelines better so it's clearer when a feature has the go-ahead.

Kinda feel like i'm a bit stuck with this, because if I can get karpenter to be aware of more accuate pricing then it will not honour that pricing when launching, but can't fix the launching because the current price view always sees spot as the cheapest if its available.

I think this pretty much shows that this is a stepping stone for https://github.com/aws/karpenter-provider-aws/pull/4697. Correct me if I'm wrong, but since this isn't a bug with the current code, and we don't know how the overarching feature will be implemented, I don't think we should merge this. @rkirk-r7 thoughts?

Closing this because of the above comment. Please feel free to reach out to us on slack if you have more questions.


Handle registry changes

Not done, but I want to see what the coderabbit gets us.

Summary by CodeRabbit

New Features

Introduced a RegisterNode method for registering nodes in the Smart Contract Registry.
Added subscription management improvements in the syncWorker for better handling of node subscriptions.

Bug Fixes

Enhanced error handling during the shutdown process of the ReplicationServer.

Refactor

Simplified the get-all-nodes command by removing unnecessary parameters.
Updated logging levels across various components to improve verbosity and clarity.

Documentation

Minor adjustments to comments and formatting for better readability.

@mkysel I just changed the settings so that the bot will review draft PRs as well. That will let us get feedback from the bot before reviewers get automatically tagged.


typo

It's no typo. It indicates where the -ico comes from. Take a look at the whole nwg-dock-hyprland -h output.


Add scoring information to the README

We need to provide documentation on how the scores are evaluated for the given query vector.
k-NN score = 1/(1+distance).
Elasticsearch orders documents by default in the decreasing order of scores. For knn the lesser the distance the more relevance it is to the query vector. So to bring up the least distances to top of the scores, we take inverse of the distance.

https://github.com/opendistro-for-elasticsearch/k-NN/issues/84


metaclass issue with fastai

Minimum running example for the error:
import  tsai.models.XResNet1d
import torch
import torch.nn
import ipdb

model = tsai.models.XResNet1d.xresnet1d50(c_in=1024, c_out=512)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
with higher.innerloop_ctx(
    model,
    optimizer,
    copy_initial_weights=False,
    track_higher_grads=True) as (fnet, diffopt):
    ipdb.set_trace()

The issue:
lib/python3.9/site-packages/higher-0.2.1-py3.9.egg/higher/patch.py in _make_functional(module, params_box, params_offset, root_patched)
    217
    218     # type checking of next line disabled as mypy is iffy with dynamic types
--> 219     class MonkeyPatched(_ModuleType, _MonkeyPatchBase):  # type: ignore
    220         __metaclass__=makecls()
    221         _wrapped_name = type(module).__name__

TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

It appears that this trick is causing the issue, redefining the individual blocks (such as ResBlock) inheriting from torch.nn.Module seems to get rid of the error.

I met the same issue when defining my own metaclass. Can you explain to me how to deal with this issue?

I'm also seeing the same issue with a custom model which is a torch.nn.Module. I didn't understand the trick mentioned above. Any suggestions?

I'm also seeing the same issue with a custom model which is a torch.nn.Module. I didn't understand the trick mentioned above. Any suggestions?

It seems like we can't use higher on model which inherits custom metaclass. I solve this issue by remove the meta class of my model.


Account for new data directory for Jupyter

The directory for installation of Jupyter kernlespecs and other files have changed
see here https://github.com/jupyter/jupyter_core/pull/292/files
Already effecting users, see here https://github.com/microsoft/vscode-jupyter/issues/12279#issuecomment-1360406135

Nothing to do here, its just a new pypi package.
Paths are still the same


Stripe - Upgrade membership

Hi!
It seems there is no Stripe integration for signup to a paid account or upgrade to a pay account ? :)
How to integrate this into the Boilerplate please?

Hi @liot-io,
In the future, it should have a Stripe integration. For the time being, if you need the Stripe integration, you can check out the official documentation.
Or, if you don't want to lose too much time implementing yourself or building from scratch, you can check out https://nextjs-boilerplate.com/pro-saas-starter-kit, the project include Stripe subscription payment out of the box. And, It's totally built on top of this project (SaaS-Boilerplate).
It's also a way to support the project, I'm planning to support this SaaS Boilerplate for several years.

Hi @ixartz
How do i purchase it? And what is the price?
The 3 prices afe confusing to me? What are the difference?
søn. 2. jun. 2024 01.38 skrev Remi Wg @.***>:

Hi @liot-io https://github.com/liot-io,
In the future, it should have a Stripe integration. For the time being, if
you need the Stripe integration, you can check out the official
documentation.
Or, if you don't want to lose too much time implementing yourself or
building from scratch, you can check out
https://nextjs-boilerplate.com/pro-saas-starter-kit, the project include
Stripe subscription payment out of the box. And, It's totally built on top
of this project (SaaS-Boilerplate).
It's also a way to support the project, I'm planning to support this SaaS
Boilerplate for several years.
—
Reply to this email directly, view it on GitHub
https://github.com/ixartz/SaaS-Boilerplate/issues/21#issuecomment-2143629584,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A6AIDRYVSNEEON5C74R5T5TZFJLO3AVCNFSM6AAAAABIUOJIM6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBTGYZDSNJYGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi @liot-io
Sorry for the confusion, it's new to me. No doubt I need to improve the wording and improve the landing page.
If you are interested, you can purchase directly with the previous link. You should be able to find the buy button and it'll redirect you to a Stripe checkout page for payment.
The difference between the price is based on the number of project you want to use and the number of developer who will work on the project. Otherwise, it's the same code and same feature.


Reorganizing classes, specifically those referring to pop culture

Are there any leanings towards reorganizing the library so that references to things like Rick and Morty and Overwatch fall under a PopCulture class instead?

Yes, see #1318


Retrying on tenant startup

Targeting 1.10 to focus on the changes. Don't merge.

The scenario is that at startup there might be some SQL hiccups and the tenant would not start, and you would not know unless you read the error logs frequently. As Azure Websites  restarts the sites regularly, some sites might go down. It happens sometimes on weblogs.asp.net as we have 800 tenants.

I like this change.  This can happen for us as well.  What is the retry timeout? Exponential?

What do people think about retry delay?


no data points

I am new for zabbix and grafana, and is making some test for this, all my zabbix server， mysql and grafana is on the same server.
I'm submitting a ...

[x] Question / Support request

Software Versions

Grafana
Zabbix
Grafana-Zabbix Plugin

4.3.1
3.4.4
3.8.1

What did you do?
add zabbix data source successed
url: http://x.x.x.x/zabbix/api_jsonrpc.php
access:proxy
api details: username api
save &test get zabbix api version 3.4.4
the api user has granted the read permission of all groups.
What was the expected result?
all the dashboard shows "No data points"
and i edit one graph, from the metrics table, i can get the groups, host, application,and item by double click the blank. after i choose one item which i am sure that the zabbix server has already get the data, is shows no data points too.
What happened instead?
data
If it relates to metric data visualization would be great to get

An image or text representation of your metric query
The raw query and response for the network request (check this in chrome dev tools network tab, here you can see metric requests and other request, please include the request body and request response)

I found it seems to be work for a while ,and then turns to be "no data points"

Try to increase PHP memory.

Where is the file for this php config? i changed it on "/etc/httpd/conf.d/zabbix.conf" and restarted the apache server.
There is other file to configure this?
Ty for your time.


uninitialized constant ImportProducts::Engine::UserMailer (NameError)

I'm installing a fresh 0.70.1 spree and want to include import-products however i'm hitting the following error. I'm still new to Spree (and to some extent Ruby) so any advice you can offer would be much appreciated.
jupiter$ rails g spree:site
/gem-source/spree-import-products/lib/import_products.rb:14:in `activate': uninitialized constant ImportProducts::Engine::UserMailer (NameError)
(stack cut for brevity)

Hi @JupiterJones,
UserMailer is a part of Spree (at least, it was last time I checked). You could try removing spree-import-products from your Gemfile, running rails g spree:site, and then re-adding the gem. If this still doesn't work, then it's possible that the Spree developers have moved or renamed this mailer.
Let me know how you get on with this anyway - if it doesn't work, I'll pull down 0.70.1 and see if I can fix up the way I call UserMailer to work.


Tests failing due to a locale issue

Describe the bug
While executing mvn clean install, one of the tests (LabelEvaluatorTest) will fail due to an issue with locales using a comma instead of a point in numbers with a decimal point.
To Reproduce
Set your PC's locale to any language using a comma (,) instead of a point (.). In my case this was Belgian (Dutch).
Expected behaviour
The tests should still pass on computers using a different locale.
Screenshots

System information:

OS: macOS
Java Version: Java 15 (build 15+36-1562)
JDK Vendor: OpenJDK

I've opened a PR which should fix this issue. Telling Maven to use nl_BE.UTF-8 triggered the failure on my machine, and the PR fixes it. The rest of the tests already pass in that locale.

Closing this as the patch has been integrated. Feel free to re-open it if this locale issue recurs, or make a new issue if there are other localisation issues.


Vision dashboard backend without data characteristics tab

The vision dashboard backend without the data characteristics tab so as to make the PR smaller.
The backend functionality and additional fixups include:

display images from dataset passed into backend
compute single explanation on the fly with loading indicators for each image
image explorer view UI changes to make thumbnail size slider work
search functionality to filter by predicted or true labels
no data characteristics tab
no cohort functionality

Description

Checklist

[ ] I have added screenshots above for all UI changes.
[ ] I have added e2e tests for all UI changes.
[ ] Documentation was updated if it was needed.

Codecov Report

Merging #1668 (78e85cf) into main (611ac56) will decrease coverage by 5.31%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1668      +/-   ##
==========================================
- Coverage   89.04%   83.73%   -5.32%
==========================================
  Files         105       22      -83
  Lines        5534      418    -5116
==========================================
- Hits         4928      350    -4578
+ Misses        606       68     -538

Flag
Coverage Δ

unittests
83.73% <ø> (-5.32%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

raiwidgets/raiwidgets/responsibleai_dashboard.py

...idgets/raiwidgets/responsibleai_dashboard_input.py

responsibleai/responsibleai/_interfaces.py

...iwidgets/raiwidgets/explanation_dashboard_input.py

...bleai/responsibleai/_tools/causal/causal_result.py

...is/erroranalysis/_internal/surrogate_error_tree.py

.../databalanceanalysis/aggregate_balance_measures.py

...ponsibleai/responsibleai/rai_insights/constants.py

responsibleai/responsibleai/exceptions.py

...ai/responsibleai/rai_insights/rai_base_insights.py

... and 71 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Enable make check

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Tests pass. Merging


cm: Supporting cluster manager callback removal during iteration

Commit Message: Cluster Manager - Supporting cluster manager callback removal during iteration
Additional Description:
Allowing the cluster manager's callback handler to onClusterAddOrUpdate and onClusterRemoval to be removed while iterating over the callback handlers.
Risk Level: Low
Testing: Unit test added.
Docs Changes: None.
Release Notes: None.
Platform Specific Features: None.
Signed-off-by: Adi Suissa-Peleg<EMAIL_ADDRESS>

/cc @antoniovicente

/cc @antoniovicente

We have another path to achieve the callback removal using the current removal mechanism.
Closing this PR.

We have another path to achieve the callback removal using the current removal mechanism.
Closing this PR.


FIX-#4093: Refactor base to be smaller

What do these changes do?

Split out Resampler, Window, and Rolling classes from modin.core.pandas.base.

[ ] commit message follows format outlined here
[ ] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[ ] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[ ] signed commit with git commit -s
[ ] Resolves #4093?
[ ] tests added and passing
[ ] module layout described at docs/development/architecture.rst is up-to-date
[ ] added (Issue Number: PR title (PR Number)) and github username to release notes for next major release

@jeffreykennethli please add a release note, otherwise lgtm!

Release notes added, thanks!


[Suggestion] Tag Search For Downloads

The search is good but sometimes you just don't remember the title. A tag search for downloaded galleries would be useful.
Also for old downloads (downloads without metadata) an "Fetch Metadata" button or "Fetch Metadata For All Galleries" would be good.

Fixed in beta 2.7.8

Tried adding the tag search but it has degraded a lot the performance of the loading and the search due to the saving method of these galleries


fixed diskspd cmdline

was missing a now-required flag (changed in later release of the tool)

This is ready to publish #sign-off


ZAF V2

Hi,
Will the vsts-zendesk-app work under Zendesks new integration API ZAF V2? I got an email from Zendesk stating that the integration will be affected by the new API. More about the new API at https://developer.zendesk.com/apps/docs/developer-guide/migrating
Best regards,
Thomas

Version 0.6.0 is on ZAF V2


Using TrayIcon with ShellFile Thumbnails causes System.ExecutionEngineException

When using a TrayIcon with the thumbnails gathered from https://github.com/contre/Windows-API-Code-Pack-1.1, closing a window causes System.ExecutionEngineException. Interestingly, loading the image itself is not enough, it only occurs when the image is displayed and then closed.
I'm setting up a tray icon like this:
Icon icon = Icon.FromFile("(Path to the file)");
tray = new TrayIcon(icon, "(Name)");
tray.TrayIconLeftMouseDown += Tray_TrayIconLeftMouseDown;

And loading the icons into an image via binding and:
public BitmapImage Image { //Using(?) https://stackoverflow.com/questions/26260654/wpf-converting-bitmap-to-imagesource
            get
            {
                if (_Image == null)
                {
                    ShellFile shellFile = ShellFile.FromFilePath(Path);
                    Bitmap bitmap = shellFile.Thumbnail.SmallBitmap;

                    //bitmap = (Bitmap)Bitmap.FromFile("(path to a file)"); //If you do this, the issue no longer occurs, even if you still load the bitmap from the line above.

                    MemoryStream ms = new MemoryStream();
                    bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);

                    bitmap.Dispose();
                    shellFile.Dispose();

                    ms.Seek(0, SeekOrigin.Begin);  //Magic line I randomly guessed.

                    _Image = new BitmapImage();
                    _Image.ImageFailed += (s, e) =>
                    {
                        Debug.WriteLine(e.ErrorMessage);
                    };
                    _Image.SetSourceAsync(ms.AsRandomAccessStream()).Completed += (s, e) =>
                    {
                        ms.Dispose();
                    };
                }
                return _Image;
            }
        }

The issue only occurs if both the shell icons and the tray icon are used, there is no issue if only one is used.
I understand that this is probably not a fault of this project, but any help would be appreciated.

Update: the following code does cause a crash so I guess it has to do with saving?
public BitmapImage Image { //Using(?) https://stackoverflow.com/questions/26260654/wpf-converting-bitmap-to-imagesource
            get
            {
                if (_Image == null)
                {
                    ShellFile shellFile = ShellFile.FromFilePath(Path);
                    Bitmap bitmap = shellFile.Thumbnail.LargeBitmap;

                    MemoryStream ms = new MemoryStream();
                    bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp); //This causes the issue???

                    bitmap.Dispose();
                    ms.Dispose();

                    bitmap = (Bitmap)Bitmap.FromFile("Assets/ic_programnew.ico");

                    ms = new MemoryStream();
                    bitmap.Save(ms, System.Drawing.Imaging.ImageFormat.Bmp);

                    bitmap.Dispose();
                    shellFile.Dispose();

                    ms.Seek(0, SeekOrigin.Begin);

                    _Image = new BitmapImage();
                    _Image.ImageFailed += (s, e) =>
                    {
                        Debug.WriteLine(e.ErrorMessage);
                    };
                    _Image.SetSourceAsync(ms.AsRandomAccessStream()).Completed += (s, e) =>
                    {
                        ms.Dispose();
                    };
                }
                return _Image;
            }
        }

I am still creating the tray icon in the same method as the original comment.


Added 'LEVEL_PROFILE' to the map of levels

Q
A

Is bugfix?
yes

New feature?
no

Breaks BC?
no

Tests pass?
yes

Thanks!


Remove version: Jackett.Jackett version 0.21.1390

404 - release deleted
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/Jackett/Jackett/releases/download/v0.21.1390/Jackett.Installer.Windows.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 918)


[PORT]Support several question mark operators in Expression

Original issue: https://github.com/microsoft/botbuilder-dotnet/issues/4506
C# pr:https://github.com/microsoft/botbuilder-dotnet/pull/4689
https://github.com/microsoft/botbuilder-dotnet/pull/4686
Description
Nullish Coalescing
Nullish coalescing adds the ability to truly check nullish values instead of falsey values.
For example:
a ?? b ?? c, if a is null/undefined return b, if b is also null or undefined return c
Conditional Operator
Achieve a variable depending on a condition.
For example:
a ? b : c ,  if 'a' is true, return b, otherwise return c.
which is equals to if(a, b, c)

Pull Request Test Coverage Report for Build 168927

6 of 6 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.007%) to 83.268%

Totals

Change from base Build 168867:
0.007%

Covered Lines:
15515

Relevant Lines:
17718

💛  - Coveralls


[BUG] OutOfMemoryError: Direct buffer memory when uploading large file

Describe the bug
When I'm trying to upload file much bigger than heap to blob storage size I'm getting OutOfMemoryError
Exception or Stack Trace
Operator called default onErrorDropped
java.lang.IllegalStateException: java.lang.OutOfMemoryError: Direct buffer memory
	at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndCompleteAll(AbstractCoalescingBufferQueue.java:351)
	at io.netty.channel.AbstractCoalescingBufferQueue.releaseAndFailAll(AbstractCoalescingBufferQueue.java:207)
	at io.netty.handler.ssl.SslHandler.releaseAndFailAll(SslHandler.java:1814)
	at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1793)
	at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1757)
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:775)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:748)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:740)
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:726)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:531)
	at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125)
	at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:356)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:748)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:740)
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:726)
	at reactor.netty.channel.MonoSendMany$SendManyInner.run(MonoSendMany.java:296)
	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
	at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:758)
	at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:734)
	at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:245)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:227)
	at io.netty.buffer.PoolArena.allocate(PoolArena.java:147)
	at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:342)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
	at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
	at io.netty.handler.ssl.SslHandler$SslEngineType$1.allocateWrapBuffer(SslHandler.java:215)
	at io.netty.handler.ssl.SslHandler.allocateOutNetBuf(SslHandler.java:2145)
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:821)
	at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:792)
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:773)
	... 17 more

To Reproduce
Steps to reproduce the behavior:
Code Snippet
azure.getBlobClient(blobName).uploadFromFile(absolutePath, overwrite)

Expected behavior
Do not fail with error
Setup (please complete the following information):

OS: Windows, Linux, Mac OS X
IDE : IntelliJ
Version of the Library used: 12.6.1

Additional context
I have bunch of another reports/questions because I have very special setup: I'm developing plugin for IDE, so my files are being loaded with another classloader (which is not even URLClassloader) and also it works inside IDE so potentially strange side-effects may occure.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[X] Bug Description Added
[X] Repro Steps Added
[X] Setup information Added

Here you can find temporary link to heapdump: https://send.firefox.com/download/9d8d79c79476569b/#NV3srXLNSMtz3GhgPqoEDw

Here is minimal reproducable example:
fun main() {
    BlobServiceClientBuilder()
        .endpoint("https://pavelfinkelshteynjb.blob.core.windows.net")
        .credential(/* credentials here */)
        .buildClient()
        .getBlobContainerClient("testcontainer1")
        .getBlobClient("temp")
        .uploadFromFile(
            "/home/finkel/temp",
            ParallelTransferOptions(null, null, { println("Uploaded $it") }, null),
            null,
            null,
            null,
            null,
            null
        )
}

where /home/finkel/temp is 4-gigabyte file, created by fallocate.

Hi, @asm0dey Thanks for reporting this. A few things. The file you shared is too large to be opened by any editors I have... Any way you could trim some of it down?  (This one may already be in the heap dump) What's the size of your heap?
My hypothesis is there's too much concurrency going on in reading the file. I suspect that we're reading the file much faster than we are able to upload blocks and free that memory, which means too much of the file is being held in memory at once. If that's the case, which hopefully your heapdump can support once I can take a look, then we can potentially offer a way to limit the concurrency.
Another thing I'd be curious about is if you update your ParallelTransferOptions to shrink the block size if we might see that help a bit. With smaller blocks, uploads can complete more quickly and hopefully free up some memory to reduce pressure. This doesn't have to be a permanent solution, but it would again go towards validating this theory.

It's heap dump, not thread dump, so it can't be opened with editors, it should be opened with Eclipse MAT/whatever tool you use for heap dump analyzing. My heap is smth like 1Gb.

But if this basic example doesn't work — does this functionality works at all?

I'm quite confident uploadFromFile works. I just uploaded a 12GB from my machine and even limited the heap size to 1GB and didn't have any problems there. It could be something to do with your environment, but I'm not certain.
@alzimmermsft It looks like a lot of the memory is being held onto by Netty. Any other thoughts on why it might not be releasing memory?

I can create minimal project that will reproduce the issue if you need.

If you are able to do that, it would be very helpful for the investigation. Thank you!

export.zip
You can just change credentials and endpoint and run application. For me it fails. And for several of my colleagues too.

BTW, my java is:
openjdk version "13.0.2" 2020-01-14
OpenJDK Runtime Environment (build 13.0.2+8)
OpenJDK 64-Bit Server VM (build 13.0.2+8, mixed mode)

My OS is
Linux UNIT-1793 5.5.19-1-MANJARO #1 SMP PREEMPT Tue Apr 21 08:12:19 UTC 2020 x86_64 GNU/Linux

@asm0dey Thank you for the repro. I think I was able to confirm that I think the file is being read too quickly. I actually was able to upload the 4gb file if I set the chunk size to 100kb. That's probably not something you want to keep permanently but might be a good stop gap.
I think the actual fix is we need to give you guys a way to configure the level of parallelism on these operations. I was also able to upload successfully by limiting the number of concurrent operations happening in a patched version of the sdk, so if we allow customers to set that value, then you should also be able to experiment and find what works for your environment.

So does it reproduce for you in my example? If yes — than it's very basic scenario as I said. But I can't even imagine why did it work for you on 12gb file?
About level of parallelism — I believe there is such option, ParallelTransferOptions#numBuffers, it's lowest possible setting is 2, but setting it doesn't help in my example.

It does reproduce when I limit the size of the heap to 1GB. I was testing a 12GB file without limiting the size of the heap, so that is probably the difference. I think this hasn't shown up before because although we have done some memory profiling, we don't usually test under such memory constraints, and the combination of conditions under "normal" circumstances simply allows us to complete requests and free memory before this becomes an issue.
We only use that option when doing a buffered upload, which is a different api. In that case we have to allocate new memory to copy the source into to make it safe for retries among other reasons. We don't do that copy when uploading from files because it isn't necessary, so that parameter gets ignored, as I believe is stated in the doc for that type. Although broadly, yes, in that case the number of buffers does affect the amount of parallelism because we can't issue more requests than the number of buffers we have.
In this case, there is an overload on one of the methods we are using from the underlying async framework that we can take advantage of which actually is called maxConcurrency and explicitly limits the number of concurrent operations rather than implicitly because of resource constraints (i.e. numBuffers in the other case). I found that setting this to an arbitrary value of 5 worked for this example just as a proof of concept.

Oh, I get it. I believe that reactor should handle this under almost any memory limits — backpressure is all about that. Also, I believe that 1-2Gbs of heap is quite normal for end-users' computers.
Do I get you right that I should use another API for uploading files? If yes — that using wrong API is indeed my fault. Which one should I use?

Ideally it would handle it, but we have some custom code for reading from AsynchronousFileChannels, so it's possible backpressure isn't being handled properly at any point between reading the file all the way to getting it on the network. @anuchandy or @alzimmermsft can either of you comment on whether it sounds reasonable that we're reading from the source file faster than we're able to push data on the network and thus hitting memory problems? And if so, whether you agree that limiting the concurrency in our flatmap operation would be the right way to go?
No there should just be the one uploadFromFile method, so we'll fix this one and get it working for you. You shouldn't have to use different methods based on memory constraints :)

We are seeing a very similar error, but when downloading via BlobClient.downloadToFile(), using azure-storage-blob version 12.6.1:
Caused by: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 65536 byte(s) of direct memory (used: 52416116, max: 52428800)

We've observed that we have to set -XX:MaxDirectMemorySize to a size larger than the blob/file size, e.g. to download a 500MB file we must set something like -XX:MaxDirectMemorySize=600M.
Should I open a separate issue? Or include this as part of this issue?

@zcox Thank you for chiming in here. At this point I think we can keep them in the same thread. Without falling into too much confirmation bias, I think the reverse of what I've described above is conceivable in the download case as well. If we are being too aggressive on parallelism, we'll issue too many concurrent requests and then be holding a bunch of that data in memory before it can be written to the file. Like I said, I don't want to assume these things have the same root cause, but it is possible.

Interesting detail is that be default you have as much direct memory allowed as big your allowed heap is. So uploading 12G-file will take max 24gb available to commit on your computer :)

@rickle-msft - piggybacking off part of the comment you left

From: Rick
In this case, there is an overload on one of the methods we are using from the underlying async framework that we can take advantage of which actually is called "maxConcurrency" and explicitly limits the number of concurrent operations... "I found that setting this to an arbitrary value of 5 worked for this example just as a proof of concept".

the maxConcurrency parameter of flatMap operator (that is not configurable from Azure SDK) default to 256. One way to configure it is through the system property reactor.bufferSize.small. Here is the doc reference for this system property.

@anuchandy Thanks for sharing that. I tried using xs yesterday, and I think I still hit oom errors. Would we be opposed to allowing customers configure this maxConcurrency directly on the api level?

@rickle-msft - agree with you on allowing the user to configure concurrency (maybe via ParallelTransferOptions) on the SDK API level.
Between the XS_BUFFER_SIZE i.e. "reactor.bufferSize.x" controls the number of chunk-metadata values prefetched from the upstream file chunk-metadata publisher. In my previous comment, I meant SMALL_BUFFER_SIZE i.e. the system property "reactor.bufferSize.small" which in a way controls the number of parallel call to client.stageBlockWithResponse that actually read part of File. Just want to make sure we're on the same page w.r.t the application of these two properties.

@rickle-msft @asm0dey - I tried uploading 4GB file with -XX:MaxDirectMemorySize=512M and as expected it failed with OOM.
Then I tried setting both concurrency and prefetch to a lower value and 4 MB block-size like below (by keeping direct memory as it is i.e. 512MB):
System.setProperty("reactor.bufferSize.small", "5")
System.setProperty("reactor.bufferSize.x", "5")

// The upload code here, blocksize set to 4MB
// ParallelTransferOptions(4 * 1024 * 1024, null, { println("Uploaded $it") }, null),
//    null,
//    null,
//    null,
//    null,
//    null)

with this upload seems working fine, its past 2 GB now. (It used OOM long before with default concurrency and prefetch).
So I believe Rick's hypothesis on too much concurrency is correct, and we should allow the users to configure concurrency to fine-tune it based on their environment. I'm also checking the direct memory allocation now.
@asm0dey can you give it a try in your environment? and if that works would you be able to use it as a short term solution to unblock your development?

as an another validation - along with direct memory I also set a limit on the heap memory.
-XX:MaxDirectMemorySize=256M -Xmx256M where total memory (direct + heap) is 512 MB. The upload is functioning fine with these constraints as well, as long as we limit concurrency and prefetch.
System.setProperty("reactor.bufferSize.small", "5") // limit concurrency
System.setProperty("reactor.bufferSize.x", "5")        // limit prfetch

// The upload code here, blocksize set to 4MB
// ParallelTransferOptions(4 * 1024 * 1024, null, { println("Uploaded $it") }, null),
//    null,
//    null,
//    null,
//    null,
//    null)

I believe that default settings should be safe and conservative by default. Nothing should fail with default settings. Do you agree?

@anuchandy
There’s this open bug related to the reactor.bufferSize settings, so one should be careful what settings are applied and not:
https://github.com/reactor/reactor-core/issues/1958

@asm0dey I agree that defaults should work in most cases out of the box. When we make the fix, we'll explore what some reasonable values might be. I think the conversation at this point is just validating that this will resolve the issue before we take the change. To that end, it'd be helpful if you could try setting the configuration that Anu shared to verify if it helps in your environment.
@roneiv Thanks for sharing this! That's really helpful and we'll keep this in mind when we are implementing the fix.

@rickle-msft Changing the bufferSize settings did not have an impact on the async dataLake client at least, we still experienced the same problem with memory leak and an OOM in the end.

@rickle-msft  probably the default the 256 for conucrrency is seems high. Maybe this shouldn't be aggressive but use a default value based on the number of connections available. Number of Connections is based on the number of cores.
At the same time, we should explore allowing the user to set the concurrency (i.e. the number of blocks to be uploaded in parallel) if override is desired.

@rickle-msft Setting these properties didn't help us too.

@asm0dey if possible could you check whether the flatMap is indeed picking these settings in the plugin environment? A breakpoint here should show picked value.

@anuchandy Nice catch! No, they don't :(

@asm0dey thanks for checking. Do you see 256 or 16 in plugin env? I noticed that the reactor won't allow setting concurrency lower than 16, so any value below we provide will result in 16.

Yes, that's what I see.

@asm0dey Just to be clear, we're assuming you mean you're seeing 16. It's interesting that setting these properties doesn't fix anything as Anu and I are both seeing these system properties allow the upload to complete successfully. I'm using a 1000mb heap.
Anu also found that we can try setting the system property reactor.netty.pool.maxConnections to work on limiting the connections as well. (The defaults are shown here). Can we try reducing the number of connections and see if that makes a difference?
Also just to double check something from earlier, you said you tried playing with your ParallelTransferOptions. I was able to get the transfer working with a block size of 100kb. Did you try a value in that range at all?

Nop, didn't try 100kb yet, will try.
And about sys props — I believe it can be all about classloaders and visibility. Or can't? You can easily check that I'm doing things right in simpish plugin from comment above: https://github.com/Azure/azure-sdk-for-java/issues/10823#issuecomment-625493785

Another option here is if you're willing, you can download the branch where I limited the concurrency on this api specifically, install that, and try having your application pick up this dependency. The branch is here. It's set to version 12.7.0-beta.1.oom. From the top of the repo, you should be able to run mvn install -f pom.xml --fail-at-end -DskipTests -Dgpg.skip -pl "com.azure:azure-storage-blob" -am. I was able to successfully run the sample project you gave using this dependency is is more or less what we'd change in the actual fix (except we'd make it configurable instead of hard coded).

Thank you guys as well for being such active participants in resolving this bug. It is very helpful.

@asm0dey FYI #11478

Thanks!
On 20/05/27 12:35PM, Rick Ley wrote:

@asm0dey FYI #11478
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Azure/azure-sdk-for-java/issues/10823#issuecomment-634895518

--
Regards,
Pasha
Big Data Tools @ JetBrains

Hi, @asm0dey I have just released 12.7.0-beta.1 which should have a number of fixes for issues you've opened. Please give it a try and let us know so we can hopefully finalize before we GA those fixes next week. In the case of the OOM error you're hitting, you should be able to play with the maxConcurrency value on ParallelTransferOptions and see a difference. You might try setting it to something like 2 just to see if it works at all and then scale up from there.
Please let us know how it works for you!

Yay!
Thank you so much! I will start testing it today :)
On 20/06/04 04:43PM, Rick Ley wrote:

Hi, @asm0dey I have just released 12.7.0-beta.1 which should have a number of fixes for issues you've opened. Please give it a try and let us know so we can hopefully finalize before we GA those fixes next week. In the case of the OOM error you're hitting, you should be able to play with the maxConcurrency value on ParallelTransferOptions and see a difference. You might try setting it to something like 2 just to see if it works at all and then scale up from there.
Please let us know how it works for you!
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/Azure/azure-sdk-for-java/issues/10823#issuecomment-639174093

--
Regards,
Pasha
Big Data Tools @ JetBrains

Thanks! 2 works for us and I think we'll stick with it because you can never predict in which environment your customer runs IDE :)

Makes sense to me! Thank you for giving it a test! I am going to close this issue since I believe it has been resolved. The fix is in master and will go out with our GA release this week or next.

According to the reactor Queues class, this is the minimum:
    System.setProperty("reactor.bufferSize.small", "16"); // default 256
    System.setProperty("reactor.bufferSize.x", "8"); // default 32


Fix issue with empty string columns

My original bug fix didn't work because I tested it for empty columns with None values it causes column dtype to be object and "infer_dtype" function interprets series dtype from its values. Upload with UI sets null values to np.nan, it causes data frame column dtype to be float and "infer_dtype" function interprets float series dtype from it's dtype not values.
I additionally fixed and added checks to test_return_error_message_when_not_correct_datatypes test function.

Hi, we're looking to merge this into our next release. This looks good to me.


bedrock: accept realm invite in getRealmFromInvite()

cc @LucienHH

Currently, there are 2 ways of adding a Realm:

By entering a Realm's invite code which is this endpoint
And by being invited by the host which the player has to accept in the menus.

Handling both may mean we need to have separate functions
acceptRealmInviteFromCode - Handles Realm code invites
acceptRealmInviteFromId - Handles individual invites
What'd you think?

Do you want to do the PR?

https://github.com/PrismarineJS/prismarine-realms/pull/18 Could you check this?


initial GPU support

Pass device argument to enable GPU

I will add a WHYLABS_DISABLE_GPU env variable, and see if I can enable GPU for sentiment.


⚠ Bofrost.de has degraded performance

In 86dd00b, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1699 ms

Resolved: Bofrost.de performance has improved in 61f55bc after 20 minutes.


Write initial spec text, with eager evaluation of async subgraphs

@guybedford Could you checkout my main branch and push it to the main branch of this repo? I merged https://github.com/tc39/template-for-proposals's updates to get the proper preview of the spec, but now this PR shows a lot of extraneous commits.

Preview: https://nicolo-ribaudo.github.io/proposal-defer-import-eval/ (step-by-step evaluation examples)
Debugging tool: https://nicolo-ribaudo.github.io/es-module-evaluation/ (significant example)
engine262 implementation: https://github.com/nicolo-ribaudo/engine262/tree/defer-eval (demo with test cases)
This spec text is probably not fully correct, but it is definitely good enough for a proposal trying to move from stage 1 to stage 2.
The modules evaluation algorithm has been updated to include three new states: ~async-subgraphs-searching~, ~async-subgraphs-evaluating-async~ and ~async-subgraphs-evaluated~. They are ordered after ~link~ and before ~evaluating~, but they are only used for modules that have been deferred.
When evaluating a module A that contains import defer * as ns from "B", where B does not use top-level await, B shouldn't be evaluated but all the modules that it transitively imports that use top-level await must be.

The traversal algorithm sets B's status to ~async-subgraphs-searching~, and recursively traverses its dependencies setting their status to ~async-subgraphs-searching~ until it finds a module C that uses top-level await.
If there is no such module C, all the modules transition to ~async-subgraphs-evaluated~: all their async dependencies (i.e. none) have been evaluated, and they can be later evaluated synchronously.
If C exists, the traversal algorithm proceeds by normally evaluating C and its dependencies. When C's status transitions to ~evaluating-async~, all its deferred ancestors transition from ~async-subgraphs-searching~ to ~async-subgraphs-evaluating-async~. This is similar to how in the case of full imports all the ancestors of the async module transition to ~evaluating-async~.

If there is any error, all the ancestors transition from ~async-subgraphs-evaluating-async~ to ~evaluated~ since any further import of those modules should just relay the error.
When C's status transitions to ~evaluated~, its ancestors transition to ~async-subgraphs-evaluated~.

If at any point any module that is in one of the ~async-subgraphs-...~ states gets eagerly imported by another non-deferred module, its status will transition to ~evaluating~ and it will go through the normal evaluation process, together with its eager dependencies.

When evaluating a module A that contains import defer * as ns from "B", where B does not use top-level await, B shouldn't be evaluated but all the modules that it transitively imports that use top-level await must be.

This algorithm makes it harder to be implemented in bundlers (see https://github.com/webpack/webpack/pull/16567). I suggest making it a link error or totally ignoring the defer.

@Jack-Works I'd love to learn more about the implementation difficulties. What makes it hard to evaluate asynchronous dependencies with deferred imports, that isn't already hard with normal imports?

@Jack-Works I'd love to learn more about the implementation difficulties. What makes it hard to evaluate asynchronous dependencies with deferred imports, that isn't already hard with normal imports?

it's a bit sloppy for me to say difficulties, I'll try it in the webpack implementation first, then give more feedback.

@nicolo-ribaudo unfortunately I don't have permissions to merge on this repo either, so we would need someone else to do that.

Before merging this PR I need to copy the ModuleRequests changes you did in the import source reflection proposal, to align them.


fix: register device tokens http request

QA testing found that registering push tokens HTTP request is not happening. I was able to reproduce it and this fixed it. Retrofit was throwing an error saying it was not able to create a @Body request object for DeviceRequest. I found out that it was because none of our custom Moshi JsonAdapter classes were being added to Retrofit so our CustomAttributesJsonAdapter was not being used.
Complete each step to get your pull request merged in. Learn more about the workflow this project uses.

[ ] Assign members of your team to review the pull request.
[ ] Wait for pull request status checks to complete. If there are problems, fix them until you see that all status checks are passing.
[ ] Wait until the pull request has been reviewed and approved by a teammate
[ ] After pull request is approved, and you determine it's ready add the label "Ready to merge" to the pull request and it will automatically be merged.

Pull request title looks good 👍. If this pull request gets merged, it will cause a new release of the software.
Example: If this project's latest release version is 1.0.0. If this pull request gets merged in, the next release of this project will be 1.0.1.
To merge this pull request, add the label Ready to merge to this pull request and I'll merge it for you.

:tada: This PR is included in version 2.0.0-beta.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Todo List for each user

Hey There,
Really glad to find this repo. I was wondering if we could add a boiler plate so that when a user creates a to-do list, he/she would be the only one able to view it.
Currently, all users could see all todos. I am newbie in laravel, however, I would appreciate if I could get some help there too :)

Hey,
glad you like this repo.
That boilerplate is a good idea. I like it, too. It will show an example on how to play around with authentication and authorization. I think I will do a prototype for this once I find the time. Maybe this weekend.


Feature: make DefinePlugin works on magic comments (provides context while processing comment options)

What kind of change does this PR introduce?
Feature

Did you add tests for your changes?
Yes I have add tests.

If relevant, link to documentation update:
Not yet.

Summary

Cleaner code
DefinePlugin has no effect at magic comments now because of the different implementation between parsing codes and comments.
As result (sometimes) we need some codes like this:
getComponent:() =>
  __SERVER__ ?
  import(/* webpackMode: "eager" */ "./path/to/module") :
  import("./path/to/module")

But I would prefer to reduce it to following if I could:
getComponent:() =>
  import(/* webpackMode: __SERVER__ ? "eager" : "lazy" */ "./path/to/module")

Remove duplicated calls
it's also helpful to avoid problems at analyse of import() calls, such as react-loadable which need to record deps from each import() call for SSR. For example:
Loadable({
  loader: () =>
    __SERVER__ ? import(/*webpackMode: "eager" */ './Bar') : import('./Bar'),
});

Processed by react-loadable/babel:
Loadable({
  loader: () => __SERVER__ ? import(/* webpackMode: "eager" */ './Bar') : import('./Bar'),
+  modules: ['./Bar', './Bar'],
+  webpack: () => [require.resolveWeak('./Bar'), require.resolveWeak('./Bar')],
});

Without duplicated import() call:
Loadable({
  loader: () =>
    import(/* webpackMode: __SERVER__ ? "eager" : "lazy"*/ './Bar'),
});

Processed:
Loadable({
  loader: () => import(/* webpackMode: __SERVER__ ? "eager" : "lazy" */ './Bar'),
+  modules: ['./Bar'],
+  webpack: () => [require.resolveWeak('./Bar')],
});

Does this PR introduce a breaking change?
It may bring effect or break if someone already has some wrong or non-effect defininitions in their magic comments, considering at current behavior it wouldn't warning or throw any error yet, it's hard to known by themself (Seems it has been added by other PRs that will show warning or error while parsing comments failed).

Other information

Don't know why the bot doesn't update labels. Nevertheless, it has ready for review at your convenience @sokra .

Everything is done, just wait for review.


[BUG]: Docker does not build neither does conda env file

so many conflicts in env.yml which results in conda not creating
Can anyone update env.yml ?
Even docker gets stuck at solving env. When I add strict priority, it goes but end up with tons of conflicts

same here, is anyone listening....? Thanks.

@gaussiangit meanwhile, did you check the last developments? https://github.com/CosmiQ/solaris/blob/0.5.0/ROADMAP.md
this conda environment is working for me on Ubuntu 20.04: https://github.com/CosmiQ/solaris/blob/0.5.0/environment.yaml
I guess @rbavery will soon drop the current main branch and replace it with 0.5.0...

Hi @remtav Thank you - I had not read that either. What caught my eye was the geojson to coco work and is what drew me to solaris in the first place; will be helpful for instance segmentation work. I will try the different environment yaml.

@IAINATTWATER environment seems broken actually. Check #462

hi @remtav and others! I recently took on maintainership of this project in order to fix this and other issues with solaris and make it simpler to install, since it wasn't maintained for some time before this. That work is largely addressed in this release branch but I haven't had the time recently to push this through to merge to main: https://github.com/CosmiQ/solaris/tree/0.5.0
I think instead of moving installs to the pip section of the env file like https://github.com/CosmiQ/solaris/pull/462, we should keep them all installed by conda to benefit from a single solver, like in https://github.com/CosmiQ/solaris/blob/0.5.0/environment.yaml
I'll have more time to push this through to completion in July. I think the final step is to get a new readthedocs account set up and hooked up to pypi so that when new releases are published, the docs get updated (we don't have access to the old account, see https://github.com/CosmiQ/solaris/issues/447). For now I suggest using the 0.5.0 branch, any issues you find there and report I can help address, when I have time.

Hi @rbavery  - thank you for the update! I tried the release branch but needed to make one change on the env file. The packe could not resolve opencv-python>=4.1, so I removed the "-python" bt and it seemed to complete and installed cleanly. I tried a python shell and was able to import cv2 so believe all is well - please alert me if this is incorrect.
Thank you again for your help.
Cheers

after further investigation I'm noticing some rearrangement of the API. I was particularly looking for the function that used to reside in the sol.data.coco namespace - geojson2coco function. Looks like the data ns does not now have a coco path and it looks like there are some functions related to coco in geo.py. Therefore is there a geojson2coco funtion now?
Thank you.
Cheers

@IAINATTWATER  thanks for the update on the install! looks like that fixes it on the release branch. I'll submit a pr and merge tot he release branch.  Can you open a new issue for the api question?

a fix for this is merged here: https://github.com/CosmiQ/solaris/pull/464
I'll close this issue once 0.5.0 is released and merged to main
sidenote, as @IAINATTWATER noted in #465, 0.5.0 includes substantial changes to the solaris api. most of this is removing solaris functionality for model training and keeping it focused to preprocessing and postprocessing so that it is more maintainable and easy to plug in to actual ML frameworks (pytorch-lightning, keras, detectron2, fastai, etc.) We would appreciate feedback and discussion from solaris users on the roadmap here: https://github.com/CosmiQ/solaris/discussions/437  CC @remtav @srmsoumya


Fix FreeBSD build: link with libexecinfo, fix Runtime/Mutex.h

This pull request fixes two more build breakages on FreeBSD introduced recently:

Add FreeBSD to the list of platforms with pthread support in Runtime/Mutex.h
Link core library with libexecinfo which is required for backtrace_symbols
Link unit tests runtime with libexecinfo for the same reason

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    OS X platform               | @swift-ci Please test OS X platform
    Linux platform              | @swift-ci Please test Linux platform

Note: Only members of the Apple organization can trigger swift-ci.

LGTM.
@swift-ci Please test and merge


build(deps): update deps to fix audit issues

Description

[ ] Tests for the changes have been added (for bug fixes / features)
[x] The commit message(s) follow conventional commits
[ ] Documentation has been added / updated (for bug fixes / features)
[x] Changes follow the contributing document.

Motivation and Context

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Which merge strategy will you use?

[x] Squash
[ ] Rebase (REVIEW COMMITS)

Might need to wait @mate-from-mattr 's pr to merged first to fix the rust lint issue.

@ZedMattr @BenLiuMattr The build problem has been solved, please rebase onto master


[Bug]Use close to replace finalize method to avoid OOM

This CL mainly solve the problem that when recycle OlapTableSink
object, GC thread will not do it immediately because the class override
the finalize method, and it will cause OOM.

For #3362

@WingsGo Hi, would better choose a name could describe the finalize method work. current work is not close.

@kangkaisen what about terminate or finish?

@kangkaisen what about terminate or finish?

How about complete()? Which means the end of "handle" this object.
But I am curious why I didn't encounter this kind of OOM in our environment?
And I found that InsertStmt also has a finalize() method, can you modify it too?


Add callback transformer

Resolves issue #37

Hey all -- I may have resolved the signed commit issue (I did not realize I was missing that aspect)
I suspect, however, if I was to for instance reword the last commit and force-push the same code, it would trigger a new review requirement. But I could do that if it would help on your end. LMK if I should, otherwise I will leave as-is.

@kdebisschop Not an issue with the signed commit :) We will consider that later (for all the contributors), if really needed or not.
We merge this PR.
I would like to wait to have the full support for PHP8 before tagging the new version.


jodd-mail: SendMailSession.createAttachmentBodyPart(): Content-Disposition and Content-ID should be independent from each other

Use case:

Sending emails with embedded images that do not show up as attachments

The problem:

In general it is up to the mail client whether / how to present inline attachments
For decorative images, it is not intended to make them visible as attachments
To support the intended behaviour, it seems to be a common practice to explicitly not set the Content-Disposition but only the Content-ID in the image mail part header

Suggestions:

adjust SendMailSession.createAttachmentBodyPart() to test for attachment.isInline() and attachment.getContentId() != null independently
introduce EmailAttachmentBuilder.setContentId(String contentId) which sets contentId but does not touch inline

Hey @JensPiegsa! If I understood right, your suggestion is to separate setting inline and contentId flag. Few questions, just to be sure:

setting inline without setting contentId does not have much sense, right? I mean, embedding attachments without contentId make no sense, so I wonder if I can join these methods into one:
setContentId(String contentId, boolean inline)

(Of course, overloaded 1-argument version would be available).

The following two attachments would be different:
.embed(attachment().bytes(new File(PNG)).setContentId("jodd1.png"))
.embed(attachment().bytes(new File(PNG)).setContentId("jodd2.png").setInline(true))

I don't see any difference in Gmail web client, both inline and non-inline shows as 'embedded', i.e. attachment is not shown. But again, as you said, its just email client. Do you have a public one, just to test?

I prepared changes, but this time it might be smart if someone else check them. Would you be so kind to review the changes if I push them to a separate branch?

Thank you, I think this is all for now :)

Hi @igr !
I completely agree with your ideas - that's how it should be.
The different behaviour was observed  with Group Office and Outlook.
I'd be happy to review / test this change.
Jens

Hey @JensPiegsa
If you can find some time to review: https://github.com/oblac/jodd/tree/email-404
It would be super cool! 🥇

Hey @igr
I build and tested the branch and want to give you a short update about this issue.
I'm still puzzled here. Eliminating the spotted difference between the working and the non-working email did not lead to the expected solution. There seems to be something else causing the wanted behaviour. Maybe it's about the kind of nesting of multi-parts.
Given that this is a rather minor issue, we should propably leave it well alone, until there is a clear way to solve it with plain java-mail.
Thanks again for the effort!

PS: some unit test are not compatible with Windows:
jodd.io.FileNameUtilTest > testGetRelativePaths FAILED
org.junit.ComparisonFailure at FileNameUtilTest.java:157

org.junit.ComparisonFailure: expected:<..[/../b/]c> but was:<..[..\b]c>

jodd.io.FileNameUtilTest > testResolveHome FAILED
org.junit.ComparisonFailure at FileNameUtilTest.java:151

org.junit.ComparisonFailure: expected:<[C:\Users\piegsaj]/> but was:<[~]/>

jodd.util.PathUtilTest > testResolve FAILED
org.junit.ComparisonFailure at PathUtilTest.java:43

org.junit.ComparisonFailure: expected:<[/aaa/bbb/]ccc> but was:<[\aaa\bbb]ccc>

@JensPiegsa if you find such email you can download it to EML format (plaintext) and send it to me, so I can check the headers...
Thanx for testing!

see: index.txt

.. I xed out the addresses.

Thanx @JensPiegsa, this one has just content-ID set, no inlines.
More I think I believe that the branch change is good overall. In my tests, both inline and content-id behaves the same, in Gmail.

Also, see: https://github.com/oblac/jodd/blob/a334b475b15198e9fb9db5b8738324b97c86c60e/jodd-mail/src/test/java/jodd/mail/InlineVsContentTest.java

After reading some RFCs and discussions I can conclude the following:

inline and content-ID can be set separately.
embedded images should be set using Content-Disposition. Of course, setting content-ID is required if you want to use them
I could not find any text about having just inline and not content-ID.

So I guess, the embed() method should set both content-ID and inline for user.


Creating short answer question Fixed

Changes
Users are able to create Short Answer Question now
Context
Testing

Create a Short answer question
Publish it
Check from student view

Pre-merge checklist

[ ] Build succeeds locally and in CI
[ ] Proper documentation
[ ] Tests
[ ] Git history clean

Coverage decreased (-0.7%) to 27.143% when pulling 4cade8de07bbeb7969be73609012eeab805ce0b0 on bik-shortQnA into 42ec39b936f76c9a62c150aa63641c5231ff2e82 on master.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v5.16.10",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=7fcbed38503bb34c6e6538b6a9482d1c6bead1e8"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "7fcbed38503bb34c6e6538b6a9482d1c6bead1e8",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "scsi: qedf: Fix refcount issue when LOGO is received during TMF\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.16.10 by commit 7fcbed38503bb34c6e6538b6a9482d1c6bead1e8. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2022-1000496


Extract additional expression values with pure_eval

Another part of https://github.com/getsentry/sentry-python/issues/748

Like with https://github.com/getsentry/sentry-python/pull/749 please make this a separate integration. btw I think this is by far the most valuable feature out of everything you've brainstormed!


fixed speaking error

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Please delete this note before submitting the pull request.
For 1.12 Features: set Milestone to 1.12 and Base Branch to release-1.12
Help editing and submitting pull requests:  https://deploy-preview-9510--kubernetes-io-master-staging.netlify.com/docs/contribute/start/#submit-a-pull-request.
Help choosing which branch to use, see
https://kubernetes.io/docs/contribute/start#choose-which-git-branch-to-use.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@zhaochaobo the Chinese translation files are pulled into this repository from another repo. Please remove them from this PR, and we can merge the other two. Thanks!
/hold