Null or empty select list values take value from previous lists on the same page

I have found some behavior I think it potentially dangerous. I have several select lists on a page together. They are rendered using Html.DropDownListFor, and each has a default option with the text "Select" and an empty string for a value. When the page is rendered, and any one of them has a "real" value selected, any subsequent lists //without// a value will take on the same value as the one before them on the page. These incorrect values will cascade down the page, filling in values on subsequent select lists.
I have created a small example solution to demonstrate the issue at https://github.com/MelGrubb/SanityTest

@MelGrubb agreed there was an issue here.  We fixed the problem earlier in the RC1 milestone.  See #524.  Thanks for your involvement.

I'm seeing this on a fresh project using MVC 5.2.3 (Jan 15, 2015). Either this is a regression, or it's not the same issue as #524. Specifically, this is about the selected VALUES in a drop-down taking on values from previous drop-downs on the same page, not the list of options.

Oh, you mean rc1 of MVC 6. I see it now.

Yes, we're tracking a similar issue for MVC 5.x at https://aspnetwebstack.codeplex.com/workitem/1913.  That issue may soon be closed due to the back-compatibility concerns @yishaigalatzer mentioned there.

I hope so. This has the potential to corrupt data in production systems. Certainly in the one I'm working on, throwing out perfectly valid nulls, and substituting them with arbitrary values based on a completely different field is pretty bad. I'm going to work around it for now by giving the nulls a magic value (-1) instead, and then translating them back to nulls on the post.

@MelGrubb FYI the recommended workaround is to "create separate enumerables, containing distinct SelectListItem instances, for each property in your model."

I guess that's what I'll do for now. Unfortunately, this particular list is rather lengthy, so my viewmodel just exploded. Fortunately, it's an internal application, so bandwidth is not a problem.


Initializer is inaccessible due to 'internal' protection level

I'm not Carthage or Swift frameworks expert, but shouldn't the initializers be public?
I've installed the library via Carthage on macOS project and I got this compile error:
'Defaults.Key<T>' initializer is inaccessible due to 'internal' protection level

extension Defaults.Keys
{
    static let myLovelySetting = Defaults.Key<Double>("myLovelySetting", default: 42.0)
}

🎉
Thanks!


Getting Involved and Commenting

Hey Daniel,
I added in some comments and Docstrings for the functions within gzip_regressor.py to help people understand what is happening. (people being myself) Thought I would upload it at least. I think I will go through everything and write out comments/Docstrings at least.
I think this looks like a cool project and would love to help contribute. I am a PhD student (finishing up my first year) at the University of Oregon. I work on ML and computational methods for macrocyclic peptide design. I'm still relatively newish in the ML space, so I would love to contribute anyway that I can!
My biggest confusion with this repo is where exactly the data is being pulled from? I see that main can run without any data in the repo. I am guessing it is being pulled from some hosting site? Though I am having troubles finding it within main.
Looking forward to helping out!

hey (I am not the owner of the repo but I can answer the 2nd part of your question:
have a look at
https://deepchem.readthedocs.io/en/latest/api_reference/moleculenet.html
it is used in
https://github.com/daenuprobst/molzip/blob/6247efd8a02d84621290141944fce88126c5f91a/main.py#L137
for the benchmark tests :)
This is nice because you just have to pip install deepchem and avoid what causes 90 % of the problems in ML: Data preprocessing.

I was looking for something to try out and figured I would give Lasso Regression and add in some extra features. I didn't know if you wanted to include any other features besides just NCD. Though if you don’t then we don’t need to incorporate the onehot function at all. It was more for my curiosity.
Overall it seemed like Lasso Regression was not helpful. The model did worse for both approaches that I tried. I haven’t tried to use Lasso Regression in the past, so it is possible that I incorporated it wrong.
Two approaches that I used:

Incorporated a non-kNN Lasso Regression and a k-NN Lasso Regression. However, I only included the scores from the non-kNN Lasso Regression. Both of them were similar and not very good.

I incorporated a onehot vector to the kNN regression task. I am pretty sure this is not good practice. I still wanted to see what would happen. It actually did better than approach 1. However, it still wasn’t very good.

All my functions are included in the gzip_lasso_regressor.py. I also included a LASSO_REGRESSION.md file. This includes the two test results. I had a couple thoughts on other approaches that could be better than this. Though I would love to hear thoughts on the direction you think this project should go.
I’m looking forward to helping. Let me know if you think I should fix/change anything.


suggest to make DOCKER_TIMEOUT configurable

Hi Team,
I'm having issues on using the liberty maven pluign in devc mode.
It uses the ci.common to execut docker commands.
On doing so I always run into timeout issues as the connection from docker for windows with wsl2 enabled is not fast enough to finish the commands within the fixed 20 seconds timeout.
I'd suggest to make this configurable by a env variable or so, that I can override it for my needs.
Best regards
Chris

Thank you for your interest. You mention a 20 second timeout, what version of the plugin are you using? The current version is 3.3.5-M1 or 3.3.4 and these feature an option called dockerBuildTimeout that should do what you need. It defaults to 600 seconds but you can increase it if you are using large images or your connection is slow. Is this what you are asking about?

Closing due to inactivity. If this is still an issue, please reopen with additional info.


https://get.pow.cx instead of http://get.pow.cx

TL;DR: Why don't you serve the source code from https://get.pow.cx?

Hi!
I searched issues (and when GitHub search found nothing, looked at all issues manually) and didn't find anything relevant.
Why isn't there a https://get.pow.cx? We're supposed to curl get.pow.cx | sh and then enter our admin passwords. It means that quite literally, we're running whatever the junk the insecure server at get.pow.cx:80 (or, whatever resolves to that address in our network) might return as a shell script, as the admin user! I don't know about you, but it really gives me the chills...
Unless I'm terribly mistaken, an attacker (be it the government or a rogue sysadmin in, say, college dorm or coffee-shop, or any unlikely, but nevertheless possible attacker) can now serve a totally bogus script and own our computer.
I know, we can download the source from the website, validate that it's not harmful, save it as pow-install.sh and then do ./pow-install.sh, but I think it's silly, when we (37signals, actually) can just purchase an SSL certificate (from a CA that curl has no problem with) and serve the source securely?
So my question is this: why just not provide a more secure, https://get.pow.cx version of the source, and change the install instructions to curl https://get.pow.cx | sh?

This is unsafe, and disappointing to see that, in December 2015, you are still asking users to run code directly from a non https URL.
For reference, here's what happened to Cisco a few days ago for using the same mechanism:
http://wadofstuff.blogspot.com.au/2015/11/cve-2015-6357-firepwner-exploit-for.html

I'm not sure if it was an option in 2013, but CloudFlare offers an SSL certificate with their free plan. I'm sure @basecamp can afford an SSL cert, but the fact that a good free solution exists makes this kind of ridiculous.


PromQL expression QOL

We're updating our alerts to make use of the metrics exposed by the prometheus-exporter feature of aws-quota-checker. We have a generic expression which aims to alert whenever we've breached 70% of any limit.
The expression is quite unweildly:
round( 100 *
    label_replace({__name__=~"^awsquota_([3a-z_]+(_count|_instances))$",account=~".+"}, "resource", "$1", "__name__", "^awsquota_([3a-z_]+(_count|_instances))$")
    / on (resource)
    label_replace({__name__=~"^awsquota_([3a-z_]+(_limit))$",account=~".+"}, "resource", "$1", "__name__", "^awsquota_([3a-z_]+)(_limit)$")
) > 70

A couple suggestions which would aid in crafting PromQL expressions:

It would be great if the metrics had an additional label e.g.awsquota_rds_instances{resource="rds_instances"}
awsquota_rds_instances_limit{resource="rds_instances"}

A bigger change, but what if the metrics were exposed using the same pair of metrics, just with the additional label as above? e.g.awsquota_usage{resource="rds_instances"}
awsquota_limit{resource="rds_instances"}

Feel free to disregard if this is too niche or opinionated in a direction you'd rather not take. A solution to those facing similar grievances could be through the use of recording rules.

Hey @kedoodle, thanks for opening this issue. I understand the benefit of switching to the awsquota_usage{resource="rds_instances"} scheme. But what would be the benefit of adding a resource label to the existing metrics?

Hey @brennerm, appreciate the response!
I'm thinking of a scenario for "generic" expressions where we want to alert on any and all AWS limits reaching a certain threshold (as opposed to a singular resource).
Existing metrics:
awsquota_s3_bucket_count{account="123456789012"}
awsquota_s3_bucket_count_limit{account="123456789012"}

Existing expression (same as original issue comment):
round( 100 *
    label_replace({__name__=~"^awsquota_([3a-z_]+(_count|_instances))$",account=~".+"}, "resource", "$1", "__name__", "^awsquota_([3a-z_]+(_count|_instances))$")
    /
    label_replace({__name__=~"^awsquota_([3a-z_]+(_limit))$",account=~".+"}, "resource", "$1", "__name__", "^awsquota_([3a-z_]+)(_limit)$")
) > 70

Existing metric names with additional resource label:
awsquota_s3_bucket_count{account="123456789012",resource="s3_bucket_count"}
awsquota_s3_bucket_count_limit{account="123456789012",resource="s3_bucket_count"}

New expression with existing metric names with additional resource label:
round( 100 *
    {__name__=~"^awsquota_([3a-z_]+(_count|_instances))$",account=~".+"}
    / on (resource)
    {__name__=~"^awsquota_([3a-z_]+(_limit))$",account=~".+"}
) > 70

It could also be nice for specific alerts where you want to use the resource as part of the alert details e.g. the alert could have a description (using metric labels) that the we have reached 70% of the limit on s3_bucket_count in<PHONE_NUMBER>12. I understand that you can get the resource from the metric name - it just requires an extra label_replace for a seemingly common use-case.
round( 100 *
    {__name__="awsquota_s3_bucket_count"}
    / on (resource)
    {__name__="awsquota_s3_bucket_count_limit"}
) > 70

@kedoodle I agree with your point of view. I added a new label called quota in https://github.com/brennerm/aws-quota-checker/commit/585f1b62f5b6b3061e56fca5561f3c5e0d7f6ea7 that contains the quota name.
Could you provide feedback on that change? If it works for you I'll create a new release.
I'll probably also switch to the proposed awsquota_usage and awsquota_limit scheme at some point in time but that'll be part of a new major release as it's a breaking change.

Hey @brennerm, I've built and deployed from 585f1b6. The new label looks great!

I understand with awsquota_usage and awsquota_limit being a breaking change. Would love to see it in a future release.

That's great to hear. The change has been released with version 1.10.0.
I'll leave the ticket open until I switch to the breaking change scheme.

Thanks @brennerm!
I'm in the process of deploying 1.10.0 into a few different k8s clusters. Probably unrelated to #31, but I'm seeing some high spikes in memory (~800 MiB) usage during refreshing current values. I've increased memory limits and will let you know (in another issue?) next week if the spikes persisted over the weekend.
Container logs, after which the pod is OOMKilled:
AWS profile: default | AWS region: ap-southeast-2 | Active checks: cf_stack_count,ebs_snapshot_count,rds_instances,s3_bucket_count
07-Jan-22 04:46:33 [INFO] aws_quota.prometheus - starting /metrics endpoint on port 8080
07-Jan-22 04:46:33 [INFO] aws_quota.prometheus - collecting checks
07-Jan-22 04:46:33 [INFO] aws_quota.prometheus - collected 4 checks
07-Jan-22 04:46:33 [INFO] aws_quota.prometheus - refreshing limits
07-Jan-22 04:46:34 [INFO] aws_quota.prometheus - limits refreshed
07-Jan-22 04:46:34 [INFO] aws_quota.prometheus - refreshing current values

Hello @kedoodle
thanks for your work. Your expression doesn't work with this metric
awsquota_elb_listeners_per_clb
We are trying to find a new alert rule, we will get back to you !
Thanks !

Hello @kedoodle
thanks for your work. Your expression doesn't work with this metric
awsquota_elb_listeners_per_clb
We are trying to find a new alert rule, we will get back to you !
Thanks !

Hopefully you can adapt the expression to something that works for your use case in leiu of the proposed awsquota_usage and awsquota_limit breaking change being implemented.


bucket_test: add test for ErrValueTooLarge on insert

Adds test coverage for these lines: https://github.com/boltdb/bolt/blob/master/bucket.go#L283-L285
[zsh|matt@nerr-2]:~/git/go/src/github.com/boltdb/bolt 0 *(master) ± go test -v -run=TestBucket_Put_ValueTooLarge
seed: 46260
quick settings: count=5, items=1000, ksize=1024, vsize=1024
=== RUN TestBucket_Put_ValueTooLarge
--- PASS: TestBucket_Put_ValueTooLarge (0.40s)
PASS
ok      github.com/boltdb/bolt  0.408s

Hi @benbjohnson , any thoughts here?

This one looks good too. :dancer:

Thanks for the test coverage PRs!


Abstract lexer logic

I want to lex for import and require too, mainly to crawl the dependency graph. The lexing logic required is very similar to the current guess_js_syntax function. If this can abstracted and configured through a callback for different usecase, it would help shrink the final output.
This would also make testing easier, since we can check what's passed to the callback directly.

Done in be36864


fix: exorcised the scary head. halloween is over

Please go the the Preview tab and select the appropriate pull request template for your changes:

Code Changes
Documentation Changes

The links to select a template didn't work.

Here's an example of the new error message:


[Text] Propagate to other components

As a developer, our components that can use our newly created text component probably should be adapted to do so.
AC:

Identify components that could make use of our text component
Apply the text component
Update documentation as needed.

Other Considerations:

Should this be a timebox issue as we have no real knowledge of which components need updating, nor priority of one component over another?

This story has been changed. You can see the original below.
After starting to propagate Text as originally desired, we realized two important aspects of the work:

For a majority of cases, the fontFamily & lineHeight styles we already get from componentStyleReset are the only styles we'd be using from Text.
For the remaining cases, which are basically headings, we expose component-level theme variables for many of the style properties provided by Text. As a result, incorporating Text actually increased lines of code and complexity, not decreased.

Because of those two factors, we decided to change the story as represented above.
Original title: [Text] Propagate to other components
Original content:
As a developer, our components that can use our newly created text component probably should be adapted to do so.
AC:

Identify components that could make use of our text component
Apply the text component
Update documentation as needed.

Other Considerations:

Should this be a timebox issue as we have no real knowledge of which components need updating, nor priority of one component over another?


Cannot install the dependency: lshash3

My package depend on pqkmeans. Some time ago, it started producing errors during installation, because pqkmeans requires lshash3, and for some reason, lshash3 doesn't install anymore.
When I run pip install pqkmeans (Ubuntu, Python 3.10, reproducible in this notebook), I get the following error:
trace

Collecting pqkmeans==1.0.4
  Downloading pqkmeans-1.0.4.tar.gz (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.5/158.5 kB 3.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from pqkmeans==1.0.4) (1.23.5)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.10/dist-packages (from pqkmeans==1.0.4) (1.2.2)
Collecting pipe (from pqkmeans==1.0.4)
  Using cached pipe-2.0-py3-none-any.whl (8.8 kB)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from pqkmeans==1.0.4) (1.11.3)
Collecting typing (from pqkmeans==1.0.4)
  Downloading typing-<IP_ADDRESS>.tar.gz (78 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.6/78.6 kB 8.3 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from pqkmeans==1.0.4) (1.16.0)
Collecting texmex_python==1.0.0 (from pqkmeans==1.0.4)
  Using cached texmex_python-1.0.0-py3-none-any.whl
Collecting lshash3 (from texmex_python==1.0.0->pqkmeans==1.0.4)
  Using cached lshash3-0.0.8.tar.gz (9.4 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->pqkmeans==1.0.4) (1.3.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn->pqkmeans==1.0.4) (3.2.0)
Collecting bitarray (from lshash3->texmex_python==1.0.0->pqkmeans==1.0.4)
  Using cached bitarray-2.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (286 kB)
Building wheels for collected packages: pqkmeans, typing, lshash3
  Building wheel for pqkmeans (setup.py) ... done
  Created wheel for pqkmeans: filename=pqkmeans-1.0.4-cp310-cp310-linux_x86_64.whl size=236311 sha256=940d560d65bd2e8ecec652ff7756502f85e47a99a9101b0b500afac859cb8948
  Stored in directory: /root/.cache/pip/wheels/84/fa/fe/7cf22e3d319bb9c9652768e4761b2b219541ad0334b127f340
  Building wheel for typing (setup.py) ... done
  Created wheel for typing: filename=typing-<IP_ADDRESS>-py3-none-any.whl size=26305 sha256=a64e1e1c1f212dc750d7dffea39c73c41cb5d2e96e166917de49adeefd665c72
  Stored in directory: /root/.cache/pip/wheels/7c/d0/9e/1f26ebb66d9e1732e4098bc5a6c2d91f6c9a529838f0284890
  error: subprocess-exited-with-error
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Building wheel for lshash3 (setup.py) ... error
  ERROR: Failed building wheel for lshash3
  Running setup.py clean for lshash3
Successfully built pqkmeans typing
Failed to build lshash3
ERROR: Could not build wheels for lshash3, which is required to install pyproject.toml-based projects

Here is where we have discovered the problem: https://github.com/avidale/compress-fasttext/issues/19
Can you please fix this error, e.g. by switching to a well-maintained implementation of lshash, such as https://github.com/guofei9987/pyLSHash?

Happened to be poking around after the SO question, & so also wanted to point out:

lshash3 (not updated since 2017) is only brought in via the mysterious texmex-python package, which has no description on either PyPI or Github, and is maybe just an optional demo dataset that'd be easy to drop from requirements unless needed?
package lshashpy3 may be another suitable replacement for a texmex-python maintainer to drop-in - seems based on same original Kay Zhu code, so might be API-compatible

@avidale @gojomo
Thanks for pointing out the issue with the current installation.
I have created a PR (https://github.com/DwangoMediaVillage/pqkmeans/pull/47) to move texmex-python to an additional dependency :)
Unfortunately, however, another issue (numpy compatibility) is preventing installation with python 3.10 at the moment: please use Python 3.9 🙇

@avidale @gojomo Update: We've released recent v1.0.6, which supports python 3.10 & 3.11. Plz try pip install -U "pqkmeans>=1.0.6"


Use noinput in migrations.

Coverage remained the same at 91.415% when pulling 99afc4435d308ce2b25684b3626cbbdc38ebd5b8 on akatsoulas:update-docker-dev into 46899f2f52e417e81318cadcfce2dc548d7961a0 on mozilla:master.


shell.exec('npm pack') painfully slow

Node version (or tell us if you're using electron or some other framework):
10.9.0
ShellJS version (the most recent version/Github branch you see the bug on):
0.8.2
Operating system:
Windows 10
Description of the bug:
With the version 6 of NPM, shell.exec('npm pack') or shell.exec('npm publish') became painfully slow.
Example ShellJS command to reproduce the error:
console.time('execution-time');
if (shell.exec(`npm pack`).code !== 0) {
    process.exit(-1);
}
console.timeEnd('shell.exec(\'npm pack\')'); // shell.exec('npm pack'): 189658.260ms

With the version 6 of NPM, shell.exec('npm pack') or shell.exec('npm publish') became painfully slow.

Hmmm this wasn't slow with npm@v5? Can you provide timing values for v5 so we can compare?
Can you also give us timing values for running npm pack in the terminal?

with npm@v5, 2111.266ms for the same folder.
with npm@v6, npm pack runs almost instantly if used directly, but I noticed something strange and probably the problem isn't related with shelljs, but with npm itself.
Running Measure-Command { npm pack } took 188976,3689ms, which is basically the same time it took with shelljs.
Ok, I think you can close this issue, the problem is with npm itself (or maybe node.js). npm starts 2 node.js processes and they keep running for ages.

Closing for now, but let me know if there's anything we need to do on the shelljs-side. Thanks for further investigation!


Add startswith and endswith validators

Make sure that you've checked the boxes below before you submit PR:

[x] Tests exist or have been written that cover this particular change.

Change Details:

Add startswith and endswith validators
Provides general and useful functionality that seems to align with other validators
A very small wrapper around strings.HasPrefix and strings.HasSuffix
Enhances functionality vs contains validator
Was about to implement this in my own project, decided to make a pull request instead.

@go-playground/admins

Coverage remained the same at 100.0% when pulling 3945da16ee820591754b888b40b1361fb112ed66 on thcipriani:v9 into b199fa0642d29caca62b6a99d65cc981ba5edc3b on go-playground:v9.

ey @thcipriani going to merge this, but could you also create another PR to add these validations to the documentation 🙏

ey @thcipriani going to merge this, but could you also create another PR to add these validations to the documentation ￼

@deankarn happy to! I added some info in doc.go, which seems to have made it to the documentation site -- should I be adding documentation elsewhere as well?

No that should be it thanks!


FeatureFlagService fails to run on Mac M1 Pro

Bug Report
Which version of the demo you are using? (please provide either a specific
Latest main or
b6e75ee
Symptom
What is the expected behavior?
FeatureFlag UI loads on going to http://localhost:8081/ so we can turn on different feature flags
What is the actual behavior?
FeatureFlag UI http://localhost:8081/ doesn't load on Mac M1 Pro
Reproduce

On Mac M1 Pro , follow the getting started steps for Docker https://github.com/open-telemetry/opentelemetry-demo/blob/main/docs/docker_deployment.md , essentially, docker compose up --no-build
Wait about 5min for containers to start
Notice featureflag container shows it's running
Look at the logs of feature flag service, you will see
qemu: uncaught target signal 11 (Segmentation fault) - core dumped Segmentation fault
My understanding is qemu is the upstream component  used for running Intel (amd64) containers on M1 (arm64)
Then go to http://localhost:8081/  and notice the page doesn't load as expected

Additional Context
My best guess is the image available for featureflag service ghcr.io/open-telemetry/demo:v0.4.0-alpha-featureflagservice does not support ARM architecture. The same issue exists for other services but somehow they do seem to run successfully on M1.
refer below screenshot which shows a warning message on M1 for amd64 images.

I think we need to build multi-arch images for all services with buildx and then push them to ghcr.

If you run docker compose up --build the image will be generated on your architecture and it will work. I went through this too and it's working.

@janssenlima That's correct and I am aware of it as well. It's a good workaround until this issue is resolved by building multi arch images that get pushed to ghcr as part of build pipeline.

This is resolved with @austinlparker's PR https://github.com/open-telemetry/opentelemetry-demo/pull/536
I built v1.0.0 on amd64 and this error occurs when running the image under emulation with Docker Desktop on apple silicon
I applied the single line change (OTP v25->v23) then rebuilt on amd64 and the error does not occur on apple silicon.

thanks @joshleecreates

Yes, now on my Mac M1 Pro as well after running docker compose up --no-build i am able to see feature flag service running successfully.


A Content Information Type Specification is a mechanism used to extend the scope of the CSIP?

I disagree with this sentence:

A Content Information Type Specification is a mechanism used to extend the scope of the CSIP by defining additional requirements for specific Content Information Types. The OAIS Reference Model defines Content Information as “A set of information that is the original target of preservation, or that includes part or all of that information. It is an Information Object composed of its Content Data Object and its Representation Information”.

A CITS does not extend the scope of the CSIP, it narrows it by specifying what particular formats and information should be place where. The CITS is always a subset of the CSIP, therefore it does not extend the scope. It narrows the scope.
Do you agree?

Thats is seen both ways depending on who you ask, thats the tricky part. I'll see if I can make it clearer.


Refactor property hint names retrieval in VisualScript

Edit: closes #52434.

Adds Object::get_property_hint_name() to be used by VisualScriptEditorVariableEdit for populating property hints automatically, without having to hardcode values.
Also naturally updates newly added property hints to visual script.
This mechanism is similar to existing Variant::get_type_name().
This would also be useful for goostengine/goost#30.

Closing as this contributor can no longer update this further.


Enable selections in the CL

Allow users to select a certain part of their command using SHIFT + left/right arrows. This text will then be replaced by whatever they start typing later.

Closed in #53.


[SYSTEMML-1050] Refactor Tutorials Page After Design Update

update get-started page with new css and hero banner

LGTM. Thank you @jazares. I'll merge.


The values of mesh.area do not sum to one

This is fine. The first and last entries of mesh.area contain the area of the polar caps, and the rest of the entries of the array contain the area of a quad cell at a specific latitude.
Therefore, sum(mesh.area[2:mesh.ny-1])*mesh.nx+mesh.area[1]+mesh.area[mesh.ny] = 1.


Pass registryPullSecrets from Installation to Execution and DeployItems

How to categorize this PR?
/area deployers
/kind enhancement
/priority normal
What this PR does / why we need it:
This PR picks up the registryPullSecrets from the Installation, passes it on to Execution and DeployItem.
The Helm and Container deployer pick up the registryPullSecrets and use them to fetch Helm chart and execution container.
Which issue(s) this PR fixes:
Fixes #125
Special notes for your reviewer:
I've provided unit tests where *_test.go files where already in place.
I checked both Helm and Container DeployItems manually for e2e functionality.
Release note:
Helm charts and execution containers can be fetched from private OCI registries with the credentials provided as `Installation.spec.registryPullSecrets`

Thank you @loewenstein for your contribution. Before I can start building your PR, a member of the organization must set the required label(s) {'reviewed/ok-to-test'}. Once started, you can check the build status in the PR checks section below.

/ok-to-test

/test

/ok-to-test


Update error handling in UTXOFetcher.swift

This ticket is about to use of ZcashError. How to do it is described in #361.

Closed with #997


Wrong use of AND instead of OR

Fixes #25
If no options were ever entered, it would never claim you need to enter your Open AI key

@krlvi nothing is preventing you from doing that. Release everyday, and twice on Friday 🥳


Add string keys to expose accessibility string

Platforms Impacted

iOS
visionOS

Description of changes
Today we do not have a control for a navigation controller in swiftUI. Part of the navigation controller is providing a custom back button which will need an accessibility identifier. Until we support a NavigationStack for swiftUI, we need the same accessibility identifier. Exposing it so it can be used.
Binary change
Total increase: 20,984 bytes
Total decrease: 0 bytes

File
Before
After
Delta

Total
31,260,080 bytes
31,281,064 bytes
⚠ 20,984 bytes

 Full breakdown

File
Before
After
Delta

StringKeys.o
0 bytes
19,104 bytes
⚠ 19,104 bytes

__.SYMDEF
4,769,144 bytes
4,771,024 bytes
⚠ 1,880 bytes

Verification
Ran VoiceOver and the accessibility identifier is still read properly on the navigation bar.
Pull request checklist
This PR has considered:

[ ] Light and Dark appearances
[ ] iOS supported versions (all major versions greater than or equal current target deployment version)
[X] VoiceOver and Keyboard Accessibility
[ ] Internationalization and Right to Left layouts
[ ] Different resolutions (1x, 2x, 3x)
[ ] Size classes and window sizes (iPhone vs iPad, notched devices, multitasking, different window sizes, etc)
[ ] iPad Pointer interaction
[ ] SwiftUI consumption (validation or new demo scenarios needed)
[ ] Objective-C exposure (provide it only if needed)

Microsoft Reviewers: Open in CodeFlow

Could we also get an example of how this would be consumed by an outside client in the Demo app?

just in the PR you mean, or in the fluent demo app?

Could we also get an example of how this would be consumed by an outside client in the Demo app?

sorry read this too fast the first time. Do we really care to do that? Just because we are under the shared understanding that this shouldnt live on forever. Also, we would have to create a whole new entry in the demo app, as we dont have a navigationstack in the demo app yet, and that seems a bit overkill?


feature HAPI FHIR

This PR:

installs a HAPI FHIR service
creates a database in the gen3 postgres instance, default database name fhir
exposes the FHIR API endpoint at /fhir

Note:

there is no UI other than the swagger page
there is no data loaded to the server
no authorization is applied

Potential improvements:

read hapi-fhir-jpaserver.externalDatabase.* from gen3 values e.g. {{global.postgres.master.host}}
add native HAPI.FHIR.security interceptor

Secrets/value.yaml additions:

# A new chart that simply creates a database for use by  hapi-fhir-jpaserver
hapi-fhir-db-init:
  postgres:
    dbCreate: true
    database: fhir
    username: fhir
    password: marionberry

# configure external hapi chart see "https://hapifhir.github.io/hapi-fhir-jpaserver-starter/"
hapi-fhir-jpaserver:
  postgresql:
    enabled: false  # the hapi chart should not create a postgresql database instance
  externalDatabase: # instead, it should use a pre-existing database instance
    host: local-postgresql  # coordinate with {{global.postgres.master.host}}
    password: marionberry
    user: fhir
  ingress:
    enabled: true
    hosts:
      - host: aced-training.compbio.ohsu.edu  # coordinate with {{global.hostname}}
        paths:
          - /fhir  # coordinate with `location` in helm/revproxy/gen3.nginx.conf/hapi-fhir-jpaserver-service.conf
  # see https://github.com/hapifhir/hapi-fhir-jpaserver-starter/blob/master/src/test/resources/application.yaml
  extraConfig: |
    hapi:
      fhir:
        fhir_version: R5
        auto_create_placeholder_reference_targets: true
        delete_expunge_enabled: true
        expunge_enabled: true

@lbeckman314 Can you take a look?
@teslajoy FYI.


Nami wallet can't connect anymore to many website

Hello there,
I have issues where websites I was already connected to can't connect me anymore to my Nami wallet. JPG store for instance, since maybe a week. It loads for an unlimited amount of time and never sync with my wallet.

This is not the only website were it happens... is it only the case for me?
BR

@guilbaudl Talk to Cardano Support on the Live support page so that your issue can be looked into.


Update livingsocial_spider.py

Updated xpath for item_field 'title'

original xpath no longer works so I made the relevant update and it now works again


update Japanese file and bug fix

I updated ja.js.
The reason is written in #117 .

hi,
just to tell you that with the addition of the Japanese language the /language command no longer works.

hi,
just to tell you that with the addition of the Japanese language the /language command no longer works.

Oh, I see. As soon as I will check program of my branch.

@byorage
Thank you for your message. I fixed that bug.
Its change is in #117 .


Bump dependency version on dependent charts: mariadb 0.6.3

Method
PRs opened using my dependency version bump utility script (see https://github.com/kubernetes/helm/issues/1947), because https://github.com/kubernetes/helm/pull/2041 is not ready yet (also see https://github.com/kubernetes/helm/issues/2205 for future issue discussion).
PRs to review

[ ] drupal #1343
[ ] ghost #1344
[ ] jasterreports #1345
[ ] jooma #1346
[ ] magento #1347
[ ] mediawiki #1348
[ ] moodle #1349
[ ] opencart #1350
[ ] orangehrm #1351
[ ] osclass #1352
[ ] owncloud #1353
[ ] phabricator #1354
[ ] phpbb #1355
[ ] prestashop #1356
[ ] redmine #1357
[ ] testlink #1358
[ ] wordpress #1359

Note https://github.com/kubernetes/charts/issues/1051#issuecomment-303124526 has not been added to the script yet, so will manually bump each.

@prydonius Chart versions bumped.

Thanks @scottrigby, these are all merged now!


Add support for MSSQL

So, data can be synced between MM and an MSSQL DB.

Duplicate of #74

I was sure there was a ticket but nothing was returned when I searched for mssql :man_facepalming:


Use environment variables

Thank you for this plugin.
Is there a way to set then access env variables? Let's say I need to connect to github API, I don't want to write my github token in .graphqlconfig.yml. Is there a way to set then use env vars?

Thanks for raising this issue :)
Yes, it is possible and documented here: https://github.com/prisma/graphql-config/blob/master/specification.md#referencing-environment-variables
It is a part of the GraphQL config project.
Please feel free to raise a PR if you think that this information belongs to the README of this project as well?
If your issue is resolved, please close the issue, thanks.

Thank you for your answer @divyenduz. It solved my problem! I don't think it has to be in the readme, so it's OK I close this issue!


[Space] UI to convert any model from the HuggingFace Hub (and open a PR)

Currently in beta (notably, max concurrency is 1 and it will fail badly if multiple users use it at the same time), but sharing it here in case it's useful:
https://huggingface.co/spaces/safetensors/convert

cc @Narsil @osanseviero @gary149 @abidlabs

completed


CLOUD-311: default values changes

@biharitomi pls review

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/865/


Actions fail on python 3.7

Describe the bug
The github actions fail when using python 3.7.
The problem is in the dependency installation step.
Pip generates a lot of connection errors, and eventually times out.
Environment (please complete the following information):

python 3.7

To Reproduce
Run, or check old executions of the Development Action.
https://github.com/ing-bank/probatus/actions
Error traceback
There are errors when installing a lot of packages. Here's the last example in the traceback:
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
Collecting pandas>=1.0.0
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out. (read timeout=15)")': /packages/74/0f/118a4201f552e2b6adb63cfcde4d16c7b3ae545490d4107a9265e8462db8/pandas-1.3.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  Downloading pandas-1.3.4-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 76.2 MB/s eta 0:00:00
  Downloading pandas-1.3.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.3/11.3 MB 70.9 MB/s eta 0:00:00
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.

Expected behavior
The Action should run without errors, just like for other python versions.

Looking up the warning, I came across this page https://github.com/pypa/pip/issues/9508
It seems this is related to a connectivity issue with IPv6, but I'm not sure how we can fix it.

This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance.

This + the logs showing downloading pandas 1.3.4 and 1.3.3 suggests we might make it easier for the resolver to be stricter with the pandas requirement.. maybe try pandas>=1.3.0 or even pandas>=1.4.0 ?

Hi Tim!
I would not change the requirements to python >=3.8, because we are still using 3.6 in some projects 🥴  But we could do the same as we did for 3.6: drop testing, and don't mention that we support it.

Probatus has silently dropped support for 3.7.


Implement {}, [], and ()

{}, [], (), $ and others should be implemented.
Here is a full list: http://vim.wikia.com/wiki/Moving_around

@hos, just curious, does it work now?

{} and $ work, [] and () do not. I have atom 0.205.0 and vim-mode 0.53.0.
On Thu, May 21, 2015 at 11:12 AM Scott Bronson<EMAIL_ADDRESS>wrote:

@hos https://github.com/hos, just curious, does it work now?
—
Reply to this email directly or view it on GitHub
https://github.com/atom/vim-mode/issues/228#issuecomment-104189577.

As stated in the README, this package is no longer maintained and is deprecated. We recommend that people use the vim-mode-plus package instead. Because of this, we are archiving this repository and closing all issues and pull requests. Thanks very much for your support and contributions!


garbage and duplicates

You do not think that if you run twice the extension EricsMark there will be duplicates it would be nice to do a cleaning before this style otherwise the idea is good
func perform(with invocation: XCSourceEditorCommandInvocation, completionHandler: @escaping (Error?) -> Void ) -> Void
{
    var lineClassMark:Int?
    var linePropertieMark:Int?
    var lineIBOutletMark:Int?
    var lineSuperMethodMark:Int?
    var lineIBActionMark:Int?
    var linePrivateMethodMark:Int?
    var lineExtensionMark:Int?

    var bRemove  = true
    let array = ["//MARK: - IBOutlets","//MARK: - Properties","//MARK: - Super Methods","//MARK: - Private Methods","//MARK: - Extensions","//MARK: - IBActions"]
    while ( bRemove == true)
    {
        bRemove = false
        for i in 0..<invocation.buffer.lines.count
        {
            let line = invocation.buffer.lines[i] as! String

            if array.contains(line) == true
            {
                invocation.buffer.lines.removeObject(at: i)
                bRemove = true
                break
            }
        }
    }

....

no it 's not good
    var bRemove  = true
    let array = ["//MARK: - IBOutlets","//MARK: - Properties","//MARK: - Super Methods","//MARK: - Private Methods","//MARK: - Extensions","//MARK: - IBActions"]
    while ( bRemove == true)
    {
        bRemove = false
        for i in 0..<invocation.buffer.lines.count
        {
            let line = invocation.buffer.lines[i] as! String

            if line.contains(array[0])  || line.contains(array[1]) || line.contains(array[2]) || line.contains(array[3]) || line.contains(array[4]) || line.contains(array[5])
            {
                invocation.buffer.lines.removeObject(at: i)
                bRemove = true
                break
            }
        }
    }

Hi thierryH91200, it is a good idea. I was wondering about this, send me a pull request.


When fetching all releases handle 429 by backing off

I've started seeing errors causing cargo update to fail using the private registry.
ERROR ssh{...}:data:build_packfile:fetch_releases_by_crate: gitlab_cargo_shim: error=error decoding response body: expected value at line 1 column 1

I traced them back to 429 responses from gitlab.com when fetching all releases GET https://gitlab.com/.../package_files.
This PR adds 429 handling by backing off and retrying.
Also:

Support env var RUST_LOG log filter configuration.
Add info logs for release & metadata fetch latency.
Improve fetch error logging.
Limit release fetching to max 32 concurrent requests (reduces the backoff wait necessary), previously was unlimited.

Thanks!


Expect don't validate Dictionary arguments

The Expected function (MockExpectation > match) don't check arguments of type dictionary.

Thank-you @Dracks, it's on my list of things to do.
Pull requests welcome! :-p
I'm considering changing the accept fund in MockCallHandler protocol from:
func accept(expectationReturnValue: Any?, functionName: String, args: Any?...) -> Any?
to
func accept(expectationReturnValue: Any?, functionName: String, args: Dictionary<String, Any?>) -> Any?
so that a test can easily capture arguments by name:
var capturedArgument: String
mock.expect().call(
    mock.function3(mock.anyString()))
    .andCapture({ (args) -> Void in
        capturedArgument = args["towel"] as! String
    })

Also, you've got a problem when the parameters are custom values. You should document that, (in the sample) because it can be a problem for anyone.
Thanks for the library.

Yes, custom parameters are high in the TODO list.
I'll probably support that with a matcher protocol, which can be implemented in your test suite.
I will look into it this evening :)

@Dracks I've updated the project to add support for matching custom types. Look for MockMatcherExtension example in the README.


update dockercloud events api doc with filters

This updates dockercloud api docs with the filters for events

Deploy preview ready!
Built with commit ade815dacff22210dcc193b437affe2c1f0ca88c
https://deploy-preview-581--docker-docs.netlify.com

@fermayo for review - you should be able to see the changes at https://deploy-preview-581--docker-docs.netlify.com/apidocs/docker-cloud/
I can merge when it's been approved and you're ready.

Cool, just let me know when.

@MaximeHeckel can you make sure this is up to date?

this is up to date


fix(deps): update org.jetbrains.compose version to the latest

I had to downgrade the version of my Kotlin Multiplatform project from 1.7.0-alpha03 to
1.6.11 (to the same version as the current compose version of this QR library) to make it work on iOS otherwise it crashes.

Please have the look at the current version of the org.jetbrains.compose version since it is regularly updated.

Originally issue posted by @jvondermarck in https://github.com/Chaintech-Network/QRKitComposeMultiplatform/issues/8#issuecomment-2302171054

@ctn-mobile-innovation @Siva-chaintech  when can we expect this?

@jvondermarck @Kashif-E we usually don't extend library support to alpha version, still if you guys needed it let us know we will consider this request!

Alright ! We will wait until they release an LTS version then.

@jvondermarck @Kashif-E we usually don't extend library support to alpha version, still if you guys needed it let us know we will consider this request!

I also have this error with the following versions: kotlin = "2.0.20" and compose-plugin = "1.6.11". Is there any temporary workaround to avoid stopping development while waiting for a fix?
Uncaught Kotlin exception: kotlin.native.internal.IrLinkageError: Function 'UIKitView' can not be called: No function found for symbol 'androidx.compose.ui.interop/UIKitView|UIKitView(kotlin.Function0<0:0>;androidx.compose.ui.Modifier;kotlin.Function1<0:0,kotlin.Unit>?;androidx.compose.ui.graphics.Color;kotlin.Function1<0:0,kotlin.Unit>?;kotlin.Function2<0:0,kotlinx.cinterop.CValue<platform.CoreGraphics.CGRect>,kotlin.Unit>?;kotlin.Boolean;androidx.compose.runtime.Composer?;kotlin.Int;kotlin.Int){0§<platform.UIKit.UIView>}[0]'
    at 0   App                      0x1064dd0cb        kfun:kotlin.Throwable#<init>(kotlin.String?){} + 119
    at 1   App                      0x1064d5fdf        kfun:kotlin.Error#<init>(kotlin.String?){} + 115
    at 2   App                      0x106513b7f        kfun:kotlin.native.internal.IrLinkageError#<init>(kotlin.String?){} + 115
    at 3   App                      0x106513c43        kfun:kotlin.native.internal#ThrowIrLinkageError(kotlin.String?){}kotlin.Nothing + 175
    at 4   App                      0x107cc33c7        kfun:qrscanner#RealDeviceCamera(platform.AVFoundation.AVCaptureDevice;kotlin.Boolean;kotlin.Function1<kotlin.String,kotlin.Unit>;androidx.compose.runtime.Composer?;kotlin.Int){} + 2103
    at 5   App                      0x107cccda3        kfun:qrscanner.AuthorizedCamera#internal + 2079
    at 6   App                      0x107ccc307        kfun:qrscanner#QrCodeScanner(androidx.compose.ui.Modifier;kotlin.Boolean;kotlin.Function1<kotlin.String,kotlin.Unit>;androidx.compose.runtime.Composer?;kotlin.Int){} + 6231
    at 7   App                      0x107ccda73        kfun:qrscanner.QrCodeScanner$lambda$4#internal + 279
    at 8   App                      0x107ccdd57        kfun:qrscanner.$QrCodeScanner$lambda$4$FUNCTION_REFERENCE$1.invoke#internal + 127
    at 9   App                      0x107ccdee3        kfun:qrscanner.$QrCodeScanner$lambda$4$FUNCTION_REFERENCE$1.$<bridge-DNNNU>invoke(androidx.compose.runtime.Composer?;kotlin.Int){}#internal + 159
    at 10  App                      0x10663949b        kfun:kotlin.Function2#invoke(1:0;1:1){}1:2-trampoline + 115
    at 11  App                      0x106808adf        kfun:androidx.compose.runtime.RecomposeScopeImpl#compose(androidx.compose.runtime.Composer){} + 767
    at 12  App                      0x1067d07bb        kfun:androidx.compose.runtime.ComposerImpl.recomposeToGroupEnd#internal + 1435
    at 13  App                      0x1067d32a7        kfun:androidx.compose.runtime.ComposerImpl#skipCurrentGroup(){} + 1955
    at 14  App                      0x1067d8663        kfun:androidx.compose.runtime.ComposerImpl.doCompose#internal + 2959
    at 15  App                      0x1067d7a5f        kfun:androidx.compose.runtime.ComposerImpl#recompose(androidx.compose.runtime.collection.ScopeMap<androidx.compose.runtime.RecomposeScopeImpl,kotlin.Any>){}kotlin.Boolean + 487
    at 16  App                      0x1067ecf63        kfun:androidx.compose.runtime.CompositionImpl#recompose(){}kotlin.Boolean + 883
    at 17  App                      0x10691c0b7        kfun:androidx.compose.runtime.ControlledComposition#recompose(){}kotlin.Boolean-trampoline + 91
    at 18  App                      0x1068138cb        kfun:androidx.compose.runtime.Recomposer.performRecompose#internal + 1451
    at 19  App                      0x10681adb3        kfun:androidx.compose.runtime.Recomposer.runRecomposeAndApplyChanges$lambda$4$lambda$3#internal + 2587
    at 20  App                      0x106820b67        kfun:androidx.compose.runtime.Recomposer.$runRecomposeAndApplyChanges$lambda$4$lambda$3$FUNCTION_REFERENCE$13.invoke#internal + 131
    at 21  App                      0x106820de3        kfun:androidx.compose.runtime.Recomposer.$runRecomposeAndApplyChanges$lambda$4$lambda$3$FUNCTION_REFERENCE$13.$<bridge-DNNU>invoke(kotlin.Long){}#internal + 123
    at 22  App                      0x1066346ef        kfun:kotlin.Function1#invoke(1:0){}1:1-trampoline + 107
    at 23  App                      0x1067b94ef        kfun:androidx.compose.runtime.BroadcastFrameClock.FrameAwaiter.resume#internal + 439
    at 24  App                      0x1067b9d93        kfun:androidx.compose.runtime.BroadcastFrameClock#sendFrame(kotlin.Long){} + 575
    at 25  App                      0x1073ab833        kfun:androidx.compose.ui.scene.BaseComposeScene#render(androidx.compose.ui.graphics.Canvas;kotlin.Long){} + 1339
    at 26  App                      0x107492127        kfun:androidx.compose.ui.scene.ComposeScene#render(androidx.compose.ui.graphics.Canvas;kotlin.Long){}-trampoline + 107
    at 27  App                      0x10740b147        kfun:androidx.compose.ui.scene.RenderingUIViewDelegateImpl.onRender#internal + 515
    at 28  App                      0x106a46903        kfun:org.jetbrains.skiko.SkikoRenderDelegate#onRender(org.jetbrains.skia.Canvas;kotlin.Int;kotlin.Int;kotlin.Long){}-trampoline + 123
    at 29  App                      0x1074706c3        kfun:androidx.compose.ui.window.RenderingUIView.object-1.render#internal + 303
    at 30  App                      0x10749d7bb        kfun:androidx.compose.ui.window.MetalRedrawerCallbacks#render(org.jetbrains.skia.Canvas;kotlin.Double){}-trampoline + 107
    at 31  App                      0x1074668df        kfun:androidx.compose.ui.window.MetalRedrawer.draw#internal + 4363
    at 32  App                      0x1074693ff        kfun:androidx.compose.ui.window.MetalRedrawer.<init>$lambda$0#internal + 443

@jvondermarck @Kashif-E we usually don't extend library support to alpha version, still if you guys needed it let us know we will consider this request!

Could you please let me know when the LTS version will be available?
I really need a working version with the latest Kotlin 2.0.20, as some other libraries only work with it. I would greatly appreciate getting an alpha version during the active development of the iOS app.
More context https://github.com/Chaintech-Network/QRKitComposeMultiplatform/issues/8#issuecomment-2382737184

@zhimbura you can use this library until the support comes

@zhimbura you can use this library until the support comes

Thank you for the offer, but I would like to know when support will be available. If it's a matter of a month, I can wait, but if it's longer, I will have to consider alternatives.

@zhimbura We had started working on that last week, we might provide update by next week

@Kashif-E Library looks good, let us know if somehow we can contribute in amazing work!

@ctn-mobile-innovation that would be great. Let me know if you think something can be improved or maybe a new feature

@zhimbura We have updated version to latest, Please check and let us know

@zhimbura We have updated version to latest, Please check and let us know

Yes, now it's works on ios, thanks


"Upsert" config for write operations

Exposing "useUpsert" configuration in the base ARM template.  Setting it to true will make all insert operations to use "upsert" for SQL/Core API.

Hey @RaviTella,
Not sure if I remember correctly, but you might also need to add these in cosmos/scripts/azurecosmos-run.sh to use this parameter on YCSB side
if [ ! -z "$useUpsert" ]; then
sed -i "s/^[#]\sazurecosmos.useUpsert\ =.*/azurecosmos.useUpsert\ =\ $useUpsert/" azurecosmos.properties
fi


csv etl loading statistics tells wrong with "haltOnError": false

when using
"config": {"haltOnError": false}
actual loaded vertex number does not match with the loader's print out.

Low priority

This issue was moved to orientechnologies/orientdb#5174


Substitution doesn't seem to work with empty layers

I'm not sure exactly the root cause here, but there is a failing functional tests here: https://review.gerrithub.io/#/c/395168/

Issue resolved.


Laghos GPU addons [laghos]

This PR brings:

a couple of optimizations from @pwang234 (SmemPAMassApply3D & ElementRestriction::Mult),
few new QuadratureInterpolator tensor functions:

Values: interpolate the values of the E-vector at quadrature points,
Derivatives: interpolate the derivatives of the E-vector,
PhysDerivatives: interpolate the derivatives in physical space of the E-vector,

a way to propose small linear algebra code useful on the GPU now in linalg/blas.hpp.

Before:
plot2_MFEM_cuda_bp1_linux_gcc_N001_std.pdf
After:
plot2_MFEM_cuda_bp1_linux_gcc_N001_all.pdf

[x] NVTX cleanup
[x] move code into linalg/blas.hpp

PR
Author
Editor
Reviewers
Assignment
Approval
Merge

https://github.com/mfem/mfem/pull/1208
@camierjs
@tzanio
@artv3 + @v-dobrev
01/24/21
⌛due 02/16/20
⌛due 02/23/20

Looks good to me! Only thought I have is should we change the blas namespace to something else?

@v-dobrev, the review on this PR is due this week

Adding a few extra days for approval...

Can you please merge master and resolve the conflicts?

@camierjs, I believe the changes in 9b19b8c are correct, but may be worth double-checking in Laghos, etc.

Merged in next for testing ...

There are a number of regression test differences from the 03/08/20 runs on tux426 and tux429 with this branch.
On tux429 the following runs have round-off level errors:
ex3 -m ../data/beam-hex.mesh
mpirun -np 4 ex3p -m ../data/star.mesh -pa -d cuda -no-vis
mpirun -np 4 ex3p -m ../data/star.mesh -pa -d raja-cuda -no-vis
mpirun -np 4 ex3p -m ../data/star.mesh -pa -d raja-omp -no-vis
mpirun -np 4 ex3p -m ../data/beam-hex.mesh -pa -d cuda -no-vis

On tux426 the following runs have round-off level errors:
ex3 -m ../data/beam-hex.mesh -o 2 -pa -no-vis
ex3 -m ../data/square-disc-nurbs.mesh -no-vis
ex3 -m ../data/amr-hex.mesh -no-vis
ex10 -m ../data/beam-quad.mesh -s 3 -r 2 -o 2 -dt 3 -no-vis
ex10 -m ../data/beam-hex.mesh -s 2 -r 1 -o 2 -dt 3 -no-vis
ex10 -m ../data/beam-wedge.mesh -s 2 -r 1 -o 2 -dt 3 -no-vis
ex10 -m ../data/beam-hex.mesh -s 14 -r 1 -o 2 -dt 0.05 -vs 20 -no-vis
ex10 -m ../data/beam-quad-amr.mesh -s 3 -r 2 -o 2 -dt 3 -no-vis
ex17 -m ../data/beam-hex.mesh -no-vis
ex17 -m ../data/beam-wedge.mesh -no-vis
ex17 -m ../data/beam-quad.mesh -r 2 -o 3 -no-vis
ex17 -m ../data/beam-hex.mesh -r 2 -o 2 -no-vis
ex21 -m ../data/beam-quad.mesh -o 3 -no-vis
ex21 -m ../data/beam-hex.mesh -no-vis
ex22 -m ../data/inline-quad.mesh -o 3 -p 1 -no-vis
ex24 -m ../data/fichera-q3.mesh -no-vis
... (many more parallel runs)

Is this expected or is it likely that there is a bug in the dense matrix functions?

The PA mass kernel has changed, as well as the restriction, so it might impact at round-off level all examples/apps using it, yes.

On further investigation the issues seems to be in 4eca3f0 (the results from the previous commit 5579f0b, at least on ex3 -m ../data/beam-hex.mesh -o 2 -pa -no-vis appear to be the same).
@v-dobrev, are these due to these changes: https://github.com/mfem/mfem/commit/4eca3f0ba8ee908cdf88a5ca936c66c987ce35f6#diff-8c02460d99a43b0e608d4e0d8e1c3a40, and if so do we really expect them to affect so many examples, many of them not using PA?

Could the namespace be ::kernel_utils?

We had a long discussion about the name. What's the concern with mfem::kernels?

Could the namespace be ::kernel_utils?

We had a long discussion about the name. What's the concern with mfem::kernels?

I was thinking the functions are more like utilities for kernels, but the namespace is fine with me.

We are still open for suggestions... I was not completely happy with kernels :)
Some other things we considered: blas, linalg, small, inlined or inlines, ...
These free functions are the implementations of dense linear algebra algorithms for "small" matrices and vectors that can be directly inlined in CPU functions and/or device kernels. It will be good if somehow the namespace name reflects this :)

I still have an issue:
c++: error: unrecognized command line option ‘-march=native’
on Lassen with miniapps/performance/CMakeFiles/performance_ex1.dir/ex1.cpp.o

This is governed by the MFEM_PERF_SW variable in `miniapps/performance/makefile'. It probably has to be updated for Lassen.
See also, https://github.com/CEED/benchmarks/blob/master/machine-configs/lassen.sh

Related to #219


Write & mmap chunks in the background

Mmapping chunks is expensive and so we should avoid having this done on the hot path to avoid blocking scrapes and rule evaluations.
This change allows tsdb to keep most recent chunks in-memory and write & mmap them on a schedule by a background goroutine.
Signed-off-by: Łukasz Mierzwa<EMAIL_ADDRESS>This is an attempt to address #10377, early testing shows that it does in fact resolve most of the issue for me.
@codesome / @replay

/prombench main

FYI the main improvement here, I think, is the way writes are scheduled.
There's now one big loop that iterates all chunks, locks each one and mmaps them. So we have strict ordering which helps to avoid getting blocked and waiting long for a lock.
Before chunk appends would trigger mmap, so at some point we would have huge number of chunks competing for a lock, hence some of them would get unlucky and stuck waiting on a lock, blocking reads and appends to such chunk.

More screenshots with Prometheus metrics after deploying latest patch.
Query and rule evaluation duration did improve for p90:

And I'm no longer seeing missed evaluations:

Sample rate is more stable, before we had a dip when scrapes get blocked followed by a spike as it got unblocked:

And rule evaluation duration is now more stable, no more big spikes in duration as rules get blocked:

But there is a slight memory increase, which I need to look into and see if there's anything I can do about it:

Note that the rule until now is that TSDB only uses one core, therefore one go routine.

Note that the rule until now is that TSDB only uses one core, therefore one go routine.

chunk write queue, which was added a few releases ago and disabled recently as it caused us to notice big performance degradation in the linked issue, already runs an extra goroutine to process writes

Each background thread is 1.33 cores that every user needs to provision, as you need to presume the worst case that they're all active at the same time.

Each background thread is 1.33 cores that every user needs to provision, as you need to presume the worst case that they're all active at the same time.

As I've mentioned on https://github.com/prometheus/prometheus/issues/10377#issuecomment-1129071559 running mmap off a goroutine scheduled every N minutes is arbitrary. The goal was to move this whole process away from append() path so it doesn't block scrapes & rules. I wasn't sure what's the best moment to mmap extra chunks and so I went with a background goroutine. Happy to remove this goroutine but I'm not sure where & when mmap should be triggered (assuming the rest of the patch can stay).

Could we do it at the previous head compaction? After the very first, we should have a good idea of how many we'll need.

Do you mean here - https://github.com/prometheus/prometheus/blob/707600d84f5567354ef48d4593ddab84c4837811/tsdb/db.go#L887 ?

I wonder if mmap should also be forced in ChunkSnapshot() - https://github.com/prometheus/prometheus/blob/9558b9b54bd3d0cb1d63b9084f8cbcda6b0d72fb/tsdb/head_wal.go#L581

Deployed latest patch to another instance and it also show improvement. No longer seeing rule evaluation duration spikes leading to missed iterations.
Memory usage looks similar to vanilla v2.35.0

/prombench cancel

I see that you have already done some investigation in https://github.com/prometheus/prometheus/issues/10377. Did you have the queue enabled? Did this patch with the queue make the situation better?

I see that you have already done some investigation in #10377. Did you have the queue enabled? Did this patch with the queue make the situation better?

Enabling chunk_write_queue causes severe degradation, which is why #10377 was raised. The patch for queue locks didn't show any noticeable performance improvement for my workload.
Without queue enabled servers I run that have 20+ million time series will show a minute or two minute long performance issue, as new chunks are being mmapped. Enabling the queue turns it into 5+ minute window of poor performance.
I did dig a bit more into both the code and history of changes around mmapping chunks, and I did realize that the queue added in #10051 was aimed at fixing the exact issue I'm observing, but it seems that it was focused at fixing remote write problems, so it might have improved writes but with unfortunate side effect of degrading reads under some conditions.
I see this patch as an alternative solution to the same problem. In both cases you keep chunks in memory until you're ready to mmap them. The difference is that the queue variant moves in-memory chunks to a new single component with additional locking that might have new/different bottlenecks, while I was trying to keep everything as parallel and distributed as it is (since that works well) and do the mmapping in a more ordered and controllable way.
With the queue I hit an hour on the wall clock and all in-memory are suddenly sent to the queue where they fight for locks. With this patch there's a loop that writes one chunk at a time, minimising any potential for lock congestion to minimum.

I see that you have already done some investigation in #10377. Did you have the queue enabled? Did this patch with the queue make the situation better?

I think that the queue doesn't really help here because it calls WriteChunk() which then needs to hold a lock
https://github.com/prometheus/prometheus/blob/59727ab0eb48802690e47e2920934e1b342894cf/tsdb/chunks/head_chunks.go#L382-L383
which essentially serialised all writes. Without the queue this path is more expensive since we need to do the actual write, but it seems that this code path is expensive enough even if we just queue it, rather than do a full write.
When tsdb block needs to be written and suddenly we need to create a new chunk for every time series being scraped and rule being evaluated we hit a race window. Let's say scrapes and rules have 1 minute interval. We hit tsdb block max duration and start creating new chunks for everything. We have a rule evaluation that produces a sample or a scrape that returns a sample - we need to append it.
As we call append() tsdb cuts a new chunk.
Since cutting a new chunks is blocking (because there are locks like the one mentioned above) we are suddenly against the clock - our append is potentially blocked by appends on other chunks.
And so if we get unblocked within a minute then everything works.
But if our append gets blocked for more than a minute then we've missed scrape/rule interval.
And that's just for the write path. There are also reads to consider.
Our chunk might need to be read for queries or rule evaluations, if it's blocked for more than 2 minutes then any query using it will timeout.
So it's very important to keep any blocking under scrape/rule interval and query timeout. Otherwise we can't keep up.
The problem I'm hitting is that I have an instance with 20-26M time series. Since WriteChunk is essentially serialised then the more chunks I have in Prometheus the longer any blocking will last. This is made worse the more rules using chunks produced by other rules I have. So eventually I will hit that 1 minute (rule interval) and 2 minute (query timeout) marks and see failures.
Since the problem is with appends getting blocked, which only happens when we need to cut new chunks, and the queue doesn't seem enough to remediate this issue, I've decided that the easiest way to deal with it is to get rid of blocking on append() path. If we don't need to block and wait for mmap when appending then always have same performance level.
Now since mmapping of chunks was added to lower memory usage we still do want to mmap it all, but this patch moves that to be run around GC time as suggested by @brian-brazil.
Mmapping is done in a loop, one chunk at a time, this has the advantage that we minimise any blocking, since we mmap one chunk at a time the lock in WriteChunk is unlikely to be held, and even if it is that's gonna be a lot fewer chunks locking it then during that minute when tsdb block is written and all time series need new chunks.
Now it was also mentioned that we want to avoid having more than a single goroutine for tsdb, which this patch does have, but the queue code needs to spin additional goroutine to service writes, which seem to violate this requirement.

This approach is interesting and something we can consider. I would like us to wait until https://github.com/prometheus/prometheus/pull/11075 is merged because doing it the other way round will be more challenging.

SGTM, great job on this!

Rebased on top of v2.39.0

Hey @prymitive, just popping in here to say that I haven't forgotten about this. I hope to get back to this soon. Multiple conferences (and hence conference driven development) taking up my time at the moment :)

Hey @prymitive, just popping in here to say that I haven't forgotten about this. I hope to get back to this soon. Multiple conferences (and hence conference driven development) taking up my time at the moment :)

Thanks.
I did rebase this on top of 2.39, which was a big change and required a bit of a rewrite in some places.
I started deploying 2.39.1+this patch to my servers. So far no issues identified.

Any chance this be moved forward now?

I was reviewing the PR, but I stopped after noticing something: are we m-mapping only right before the compaction or am I missing someplace where the m-mapping is triggered as well?

Initially I've added a goroutine that mmaps on a schedule. But there were comments that TSDB is not allowed to spin additional goroutines - https://github.com/prometheus/prometheus/pull/10709#issuecomment-1129715073 - and so I removed that and instead hooked mmapping around compaction.
But frankly I don't know what is the best moment to mmap chunks.
For majority of chunks it's probably best to mmap after block range, so every 2h, right after all memSeries get a new head chunk due to reaching max allowed time.
But depending on the setup there might be a lot of jobs with short scrape interval that's not very well aligned with maxt and so they might produce more than one chunk during a single block range, and for these it would make sense to try and mmap chunks more often.
The current code effectively runs mmap every ~1 minute on my servers, with mmaping taking 0-15s depending on the number of time series and usually mmaps 300-1500 chunks.
This doesn't cause any performance issues that I observed but it's probably more often than it needs to be.
Not sure where and when it should be ideally triggered.

Oh, I see it now. We will trigger the function everytime Compact() is called, and does not depend on compaction happening.
I would suggest moving that call inside DB.run(), and inside the reloadBlocks() case. It would be good to not depend on the compaction call cycle and reliably trigger it every minute. If the frequency turns out to be too much, we can always tweak it in the future.
I will review rest of the code soon.

/prombench main
One last benchmark after some recent changes.

/prombench restart main

/prombench cancel

/prombench main
Some CI issue earlier and I messed up the CI by restarting it. Trying again.

Prombench start CI failed again with the same error:
09:21:36 gke.go:594: error while applying a resource err:error applying 'manifests/prombench/benchmark/6_loadgen.yaml' err:Request for 'applying deployment:prometheus-loadgen-querier' hasn't completed after retrying 50 times

But we still have a load of around 1m series. So will let it run for a while.

There is a sustained 5% increase in the memory RSS, I believe it is the switch to a slice of headChunks and other factors. CPU usage is also slightly up.
I would like to borrow @bboreham's expertise here. @bboreham: This basically helps with latency spikes as described in https://github.com/prometheus/prometheus/issues/10377. Apparently, the async mmap queue was not helpful. This PR triggers the mmap of in-memory chunks in the background by holding a slice of in-memory chunks per series. Does the memory/CPU increase look concerning or is there a way to optimize it?

/prombench cancel
I think we have enough data for now.

There is a sustained 5% increase in the memory RSS, I believe it is the switch to a slice of headChunks and other factors.

Slice needs extra 24 bytes if I recall. That's 24MB per 1 million time series.
Last time I checked (https://github.com/prometheus/prometheus/pull/11003#issuecomment-1181676566) memSeries was using 224 bytes, so extra 24 bytes is 10% more just for baseline memory usage.
That being said most memory in memSeries will be eaten by samples and labels. In my setup I usually see around 3.2KB per memSeries and 24 bytes is <1% memory usage bump when you compare it with that number. Real world overhead depends on how big your memSeries are really.
But the bottom is obvious here - having a slice here uses more memory.
But at the same time there was another slice added for out of order chunks, and we also have another slice for mmapped chunks. An obvious saving would be if we managed to keep all different chunks on a single slice, regardless if they are head, mmapped or OoO chunks. That would require a lot more changes but there are bits of code that I found difficult to get my head around quickly as they need to get the right chunk from passed offset (see tsdb/head_read.go changes) and having a single slice could potentially simplify some of that, which would be great.
Chunk write queue and other alternative solutions are likely to eat up some extra memory too, but those might be a bit harder to calculate exact memory usage growth due to more dynamic nature of it. Plus with queues you don't pay that extra memory usage all the time (except for the base structure to have the queue running), so from purely "bytes payed" PoV slice is more of a small but permanent cost that scales with the number of time series.
The upside is that it removes blocking behaviour from the hot path. Another solution could be to keep two pointers on memSeries: one for current HEAD chunk and one for previous one that should be mmapped. That only adds 8 bytes. The reason I went for the slice is that it felt safer - with two pointers I need to ensure "previous" chunk is mmapped before current HEAD one gets full, otherwise we need to block again - there's no such worry about slices.

Since you changed a single pointer to a slice, it should be +16 bytes per series. (When looking at rss this number is doubled by Go's garbage-collector).
Would it work as a linked list?  I have pondered making that change for mmappedChunks and oooMmappedChunks.

Would it work as a linked list?

I cannot think of any reason why it wouldn't. Reading from memSeries would become a bit more complicated, since referencing chunks via series.headChunks[3] wouldn't be possible and we would have to iterate over the list until we get to the 4th chunk. But that's the main complication I can think of. No idea how that would impact read performance.

I think in practice those indices are only ever used from an iterator stepping through the set of chunks, but the code assumes this can be effectively expressed by an integer.

Here's a linked list version of this PR - #11818

Closing this one, there's #11818 instead


Update HCR & HNSR to move NS creation to HNSR

This commit removes the hc.Spec.RequiredChildren field and the
enable-hns-reconciler flag. It also moves the creation of the owned
(previously requiredChild) namespaces from HC Reconciler (HCR) to HNS
Reconciler (HNSR). The logic in HCR is cleaned up to adapt this change.
Tested with the current and updated integration tests by "make test".
Also manually tested the 'k hns describe'. The HNS_MISSING condition was
set when I deleted the HNS instance in the owner. The owned namespace
got recreated after I deleted it, which means the new watch in HNSR is
working.
Fully described in #552
Part of #457, #479

/assign @adrianludwin
/assign @sophieliu15

/lgtm
/approve
/hold cancel


grammar

fix grammar in README.md

what grammar issue

i gave you collabortator so just fix it urself


Dashboard setup

Tasks

[ ] Add navbar
[ ] Add sidebar
[ ] Search box in the navbar

Notes

Ensure this layout can wrap the main components

@devvspaces I'll like to be assigned to this task as well

Any update on this @orunto

I have taken the liberty of opening #26 for this issue as I see it is a blocking point to continue with the frontend development progress.

Any update on this @orunto

I'm working on the styling today

@orunto styling to make a pr?


Introduced a memory arenas API

This API has two benefits:

The user doesn't have manage memory on his own anymore.
Repeated allocations and deallocations are not required since we allocate enough characters in the buffer that will fit at least minimum hash size and 16 numbers which is enough for the average case.

It's useful to me since I'll be able to save a lot of allocations in my Python bindings to this library.

While this might be a good idea, you're reallocating memory on supposed-to-be-read-only-functions (despite been called on *arenas)
Again, the "nothing-should-be-done-for-you" principle is broken here.
""" I've paid my time making sure I'll not break your boundaries - don't make me change them """

So if I want memory arenas I should implement them in my own code?

Well, your "arenas" are a simplified wrapper over the base functions of Hashids - do a wrapper library on your own.
Classically, I'd reject everything that'd realloc memory on itself.


Not working on Android

Hey guys,
I've installed the android version from the app store. When I go to load most examples, I see RED error screens, as though these apps are trying to load in debug/local mode, instead of packaged/release bundles. So the app is currently not usable?

Currently the native apps are not working with our new site. Hopefully soon we'll get a new version up based on Exponent.js.

The new Android build is published on the Play Store now. Thanks!


[Question] Handle attachment along with LUIS dialog

I am currently able to upload attachment from slack to bot, data is coming perfectly.
I written code to handle attachment in On Default Dialog .
Noticed unexpected behaviour about attachment, lusi is taking attachment name as text and calling to corresponding dialog.
For Example:
Current Output
Let's say my Attachment name is "hi.png" so it is calling to welcome dialog
Actual output should be:
whenever user upload any attachment ,it should goes into default dialog

Duplicate of "Luis intent to identify attachment".
@aakashkag since I started helping you on Stack Overflow before you posted this issue, I'm closing this issue and going to continue helping you there.

Ok thanks :)


matFromRollPitchYaw name does not match parameter order

The fn name is in the order of roll, pitch, yaw, but the parameter order is pitch, yaw, roll.
This is especially confusing for the vector versions.
Affected fns:
fn matFromRollPitchYaw(pitch: f32, yaw: f32, roll: f32) Mat;
fn matFromRollPitchYawV(angles: Vec) Mat;
fn quatFromRollPitchYaw(pitch: f32, yaw: f32, roll: f32) Quat;
fn quatFromRollPitchYawV(angles: Vec) Quat;

Since quatFromRollPitchYawV requires | pitch | yaw | roll | 0 | I think we should rename these fns.

This is by design. The name describes the order of rotations (the order in which matrices are multiplied). The order of parameters is x (pitch), y (yaw), z (roll).
This is the convention borrowed from DirectX Math. I don't think we should change it.

IMHO it makes sense that the name matches the order they are applied, but I'm not sure why it makes sense to pass them in in order of x,y,z, since that's not typically names you use anyway in this context. That is, even if I had my euler angle as a vec3, I would still first make the mental conversion of x=pitch and assume it should go as the second parameter.
Granted, there is some value to adhering to DX but I bumped into this and I haven't used the DX math library as I would assume is true of most people. But, if there was a comment about it then it would be easier to note the difference.
An alternative would be to take a struct {pitch, yaw, roll} as then it would be impossible to make the mistake, and if it was union'd with a [3]f32 you could pass that in in the (likely?) offchance that's how you're storing your euler angle.

Please remember that we also have version that takes vector as a parameter:
fn matFromRollPitchYawV(angles: Vec) Mat;
This is for speed to keep everything in CPU registers. It would be very confusing to mess with ordering here.

What if we rename these fns to matFromAngles, matFromAnglesV, quatFromAngles, quatFromAnglesV and annotate the vector versions with doc-comments to describe the layout? (for the scalar versions the argument name is sufficient documentation)


Question about model training.

I watched all your videos and followed along, it tooks about 5 days 😀, it's very fun and appreciate you!
Now I wonder how to train this model.
I also watched another video of yours “How diffusion models work - explanation and code!”.
This is also very useful and great video, thank you again!!
The video was about how to train unet(diffusion model) for latent denosing.
But we have four major models in here:
VAE-encoder, VAE-decoder, unet, and clip
If we want to train unet(diffusion mode) like "diffusion model training youtube",
does we freeze other models and train only unet?
However, the definition of learning is not well understood.
For example, if we want to create image B with a specific style of A, like A image -> styled B image
Where should I feed images A or random(input) and styled B(output), respectively?
The inference will look like this, but I don't know how to do it in training phase.
A(or random) -> VAE-encode -> [ z, clip-emb, time-emb -> unet -> z] * loop -> VAE-decode -> B
It is also questionable whether clip-embeding should just be left blank or random or specific text prompt?.
or should I input A image for clip-embeding?
I have searched on youtube for that how people train stable diffusion model then most video was using dreambooth.
It looks very hight level again like hugging face.
I would like to know exact concept and what happen under the hood.
Thanks to your video code I could understand stable diffusion ddpm model but I want to expand training concept.
Thank you for amazing works!
Happy new year!

Hi, Same question.
PS : great videos mate !

Hi, same question here too and really appreciate your video!

Hello!
Given my current priorities, I don't think I'll be coding the training script anytime soon. Feel free to contribute it yourself... that's the spirit of open source and pull requests.

I actually tried to set up a training script, but I seem run out of ram LOL. Do yall know what is an expected resources I should prepare for training this? I tried to set up the model similar to the pipeline.py (I have NO CLUE if my code works at all) but the forward looks like:
def forward(self, images, captions, tokenizer, strength=0.8):
      batch_size = len(captions)
      latents_shape = (batch_size, 4, self.LATENTS_HEIGHT, self.LATENTS_WIDTH)
      generator = torch.Generator(device=self.device)
      tokens = tokenizer(captions, padding="max_length", max_length=77, return_tensors="pt", truncation=True).input_ids.to(self.device)
      # tokens = torch.tensor(tokens, dtype=torch.long, device=self.device)
      context = self.clip(tokens)
      sampler = DDPMSampler(generator)
      sampler.set_inference_timesteps(self.n_inference_steps)
      if images is not None:
          encoder_noise = torch.randn(latents_shape, generator=generator, device=self.device)
          latents = self.encoder(images, encoder_noise)
          sampler.set_strength(strength)
          latents = sampler.add_noise(latents, sampler.timesteps[0])
      else:
          latents = torch.randn(latents_shape, generator=generator, device=self.device)

      timesteps = sampler.timesteps
      for timestep in timesteps:
          time_embedding = self.get_time_embedding(timestep).to(self.device)
          model_input = latents
          model_output = self.diffusion(model_input, context, time_embedding)
          latents = sampler.step(timestep, latents, model_output)

      return self.rescale(self.decoder(latents), (-1, 1), (0, 1), clamp=True), context

But even just feeding one image per batch [1, 3, 512, 512] to this forward, it ran out of memory.
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB. GPU 0 has a total capacity of 15.72 GiB of which 287.44 MiB is free.

did u try to resize your images to a smaller one, if u got still out of ram error, maybe u can run on Colab.

I actually tried to set up a training script, but I seem run out of ram LOL. Do yall know what is an expected resources I should prepare for training this? I tried to set up the model similar to the pipeline.py (I have NO CLUE if my code works at all) but the forward looks like:

Hello, did u work on this training script further? It would be really useful for me if you provide me the training script.

I think you may follow this course by Jermey Howard. You can skip any section that seems familiar.

I actually tried to set up a training script, but I seem run out of ram LOL. Do yall know what is an expected resources I should prepare for training this? I tried to set up the model similar to the pipeline.py (I have NO CLUE if my code works at all) but the forward looks like:
def forward(self, images, captions, tokenizer, strength=0.8):
      batch_size = len(captions)
      latents_shape = (batch_size, 4, self.LATENTS_HEIGHT, self.LATENTS_WIDTH)
      generator = torch.Generator(device=self.device)
      tokens = tokenizer(captions, padding="max_length", max_length=77, return_tensors="pt", truncation=True).input_ids.to(self.device)
      # tokens = torch.tensor(tokens, dtype=torch.long, device=self.device)
      context = self.clip(tokens)
      sampler = DDPMSampler(generator)
      sampler.set_inference_timesteps(self.n_inference_steps)
      if images is not None:
          encoder_noise = torch.randn(latents_shape, generator=generator, device=self.device)
          latents = self.encoder(images, encoder_noise)
          sampler.set_strength(strength)
          latents = sampler.add_noise(latents, sampler.timesteps[0])
      else:
          latents = torch.randn(latents_shape, generator=generator, device=self.device)

      timesteps = sampler.timesteps
      for timestep in timesteps:
          time_embedding = self.get_time_embedding(timestep).to(self.device)
          model_input = latents
          model_output = self.diffusion(model_input, context, time_embedding)
          latents = sampler.step(timestep, latents, model_output)

      return self.rescale(self.decoder(latents), (-1, 1), (0, 1), clamp=True), context

But even just feeding one image per batch [1, 3, 512, 512] to this forward, it ran out of memory. torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB. GPU 0 has a total capacity of 15.72 GiB of which 287.44 MiB is free.

Hi. I have the same problem actually. Please can you help us with your scripts so that we could try to use it for training?

I end up give up since adjusting model architecture (reducing number of parameters means I can not use the pretrain weight). I would recommend looking more into building a LoRA fine tuning of it, since that reduces the training cost by a lot.

Please can you still share what you did that was leading to out of memory. I will try to train from scratch and reduce parameters myself.


Fixes updating ui in background #5155

Fixes #5155

@oa-s why do you think this happens? Prior to #5156, the receive(to:) was caller with .main earlier on the chain, doesn't that move everything after that to be on .main? The code that called sink{} in the screenshot above is already on main right? What causes the sink's closure arg to be running in the background?

its because of order of receive(to:) we receive only  Just(address) on main thread, the rest of operators might return data on non main queue.


nats account backup Indefinite Hang

Encountered an indefinite hang while running nats account backup
Logs:
Performing backup of all streams to account-backup/

    Streams: 6
       Size: 189 MiB
  Consumers: 142

Starting backup of Stream "KV_aaa" with 0 B

Received 792 B compressed data in 2 chunks for stream "KV_aaa" in 4ms, 8.5 KiB uncompressed

Starting backup of Stream "KV_bbb" with 7.9 MiB

Received 9.7 MiB compressed data in 93 chunks for stream "KV_bbb" in 156ms, 20 MiB uncompressed

Starting backup of Stream "KV_ccc" with 181 MiB
<<<---  HANGING INDEFINITELY HERE --->>>

Consider adding a client-side timeout - if backup data stops flowing after N seconds, then exit with an error so that the user knows to retry the backup.

Should now timeout after 5 seconds of no data - here I ^C mid backup:
$ nats --context nats_development s backup X /tmp/backup --no-progress
Starting backup of Stream "X" with 947 MiB
13:04:28 Disconnected due to: EOF, will attempt reconnect

nats: error: snapshot failed: backup timed out after receiving no data for a long period


When I return an item to merchant to get a refund, this error pops up

Interface/AddOns/RPGLootFeed/LootDisplay.lua:180: bad argument #1 to 'GetCoinTextureString' (Usage: local result = C_CurrencyInfo.GetCoinTextureString(amount [, fontHeight]))
Count: 1

Call Stack:
[string "=(tail call)"]: ?
[string "=[C]"]: in function `GetCoinTextureString'
[string "@Interface/AddOns/RPGLootFeed/LootDisplay.lua"]:180: in function `ShowMoney'
[string "@Interface/AddOns/RPGLootFeed/Features/Money.lua"]:37: in function `OnMoneyLooted'
[string "@Interface/AddOns/RPGLootFeed/RPGLootFeed.lua"]:65: in function `?'
[string "@Interface/AddOns/andromeda/libraries/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:114: in function <...ibraries/CallbackHandler-1.0/CallbackHandler-1.0.lua:114>
[string "=[C]"]: ?
[string "@Interface/AddOns/andromeda/libraries/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:24: in function <...ibraries/CallbackHandler-1.0/CallbackHandler-1.0.lua:20>
[string "@Interface/AddOns/andromeda/libraries/CallbackHandler-1.0/CallbackHandler-1.0.lua"]:59: in function `Fire'
[string "@Interface/AddOns/andromeda/libraries/AceEvent-3.0/AceEvent-3.0.lua"]:120: in function <...ns/andromeda/libraries/AceEvent-3.0/AceEvent-3.0.lua:119>

to fix your trouble try download this fix, i see it in another issue,
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/6mkbh6xhau31g
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

@ffainy I'm pretty sure this occurs when a negative number is passed in. I wasn't able to replicate the issue, but I'm pretty confident the code I added will fix the issue. Let me know if you still see it after updating to v1.4.3


Datapipeline processing service

User story

As a software developer
I want to send a csv file to the datapipeline
So that it can be processed and I receive the result

Acceptance criteria

[x] Backend service that give a csv file to datapipeline
[ ] Backend receive the result of the datapipeline processing

DoD general criteria

Feature has been fully implemented
Feature has been tested
Feature has been merged into the main branch
Important parts have been documented in the technology wiki
All acceptance criteria were met
Product owner approved features
Developers agreed to release

still need testing

Datapipeline not receiving files dynamically. Right now which file to process is hardcoded into the code

Have datapipeline output, need to connect with backend endpoint


all: Add/fix reference server health check and integration test.

PR Checklist
PR Structure

[ ] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[ ] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[ ] This PR's title starts with name of package that is most changed in the PR, ex.
paymentservice.stellar, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[ ] This PR adds tests for the most critical parts of the new functionality or fixes.

What
Add/fix reference server health check and integration test.
Why
N/A
Known limitations
Only KafkaListener is checked. We should add SQS checks.

Reference Server Preview is available here:https://anchor-ref-pr277.previews.kube001.services.stellar-ops.com/SEP Server Preview is available here:https://anchor-sep-pr277.previews.kube001.services.stellar-ops.com/

Reference Server Preview is available here:https://anchor-ref-pr277.previews.kube001.services.stellar-ops.com/SEP Server Preview is available here:https://anchor-sep-pr277.previews.kube001.services.stellar-ops.com/

Reference Server Preview is available here:https://anchor-ref-pr277.previews.kube001.services.stellar-ops.com/SEP Server Preview is available here:https://anchor-sep-pr277.previews.kube001.services.stellar-ops.com/


Delete MMX name mangler case

The type was removed in https://github.com/llvm/llvm-project/pull/98505

@nhaehnle @dstutt @Flakebi


stage all service on enable

@ibuildthecloud  This requires my changes to rancher-compose. I'll leave the changes to Godeps here as a representative of the changes I made to rancher-compose

Holding off on the fix for this. Will reopen when relevant


Add db uri support

Can now specify a database as a URI / connection string!
Additional options can be passed as query parameters.
Examples:
database: 'mysql://username:password@<IP_ADDRESS>:3306/database_name?synchronize=true',

database: 'postgres://username:password@<IP_ADDRESS>:5432/database_name?synchronize=true',

database: 'mongodb://username:password@<IP_ADDRESS>:27017/database_name?synchronize=true',

This is not a breaking change as passing an object still works, this just makes database configuration much easier for most use cases.
This matches the behaviour in the documentation - I wanted to make it this simple from the outset (which is is how it's described in the documentation), but took a little while to get to this point.
@TODO Update documentation to include these examples (and update the example site to use this approach too).

Published as<EMAIL_ADDRESS>

This already worked? I was connecting to Postgres with a string. Or did you break it 😂

LOL really? (!) It was supposed to work that way, but I wasn't able to get it to work.
I think it was maybe working by accident! 😂
I'd be interested to know what your config looked like!

It was a few versions ago I don't remember which one exactly. I started using SQLite for quick testing. It was the same as above but without synchronize=true and it worked perfectly lol.

That's also why I added it to the docs.

It is actually supposed to be a feature of TypeORM according to some docs, as are using environment variables, but I could get them to work either. 🤷♂
It's possible I was looking at old docs though - ran into that quite a bit with TypeORM, especially when it came to adding MongoDB support.


Knex Query Wrapper

Hey all,
I need to decamelize my columns on all my queries and was thinking it would be best to do this at the lowest level possible. Does anyone have any good suggestions for doing this as a wrapper to knex?
Example:
insert into table (colOne, colTwo)  values (1, 2)
would become
insert into table (col_one, col_two)  values (1, 2)
It would also be useful to camelize on the way out as well.
Thanks!

It looks like, based on this line:
https://github.com/tgriesser/knex/blob/7cc827f6cf495faa128b0b881d2ec4d6aa454fd9/src/query/compiler.js#L20
We may be able to pass in a formatter here?

@bgphipps I too have gone done this route and would love a solution like what you're talking about. I have sort of accomplished this by decorating the query object and intercepting the various call (e.g. where, update, insert, etc). However, that solution stinks, because it's fragile and it requires a lot of repetition. @sheeley I like the idea of doing something with the formatter. I might play around with that and see if I come up with anything.

I personally don't see how this belongs in knex's code base, but I don't have a strong opinion against it either.
For adding something like this I would recommend adding it to the formatter's columnize function as that should be the best place in the code base.
I'm just springing ideas now but to make it as dynamic as possible, maybe something like .transformColumns() could be nice.
knex('table').insert(data).transformColumns((column) => { return decamelize(column); });

(For lack of a better name)

@wubzz I agree. I'm thinking of something that wraps knex.

@wubzz totally agree - don't need transformation to be part of knex, but hooks to allow it would be sweet.
i'm trying to abstract things between our frontend & api devs (like @bgphipps) and our DBAs, so if there were a way to do something like this and have hooks into where, join, etc that'd be awesome. Right now, we're manually wrapping everything, and it works but isn't a joy.
const readableToColumn = {
  objectName: `obj_nm`,
  objectId: `obj_id`
}

const columnToReadable = someReverse(columnNames)

knex.registerTransformIn((row) => {
  const out = {}
  for (const key of Object.keys(row)) {
    out[readableToColumn[key] || key] = row[key]
  }
})

knex.registerTransformOut((row) => {
  const out = {}
  for (const key of Object.keys(row)) {
    out[columnToReadable[key] || key] = row[key]
  }
})

const objectId = [1, 2, 3]
knex.select([`objectName`, `objectId`]).where({objectId})
// => [{objectName: `object 1`, objectId: 1}, ...]

I have created a "debug helper" for knex/lib/formatter.js, which prints out console messages about the return value of each function: https://gist.github.com/kripod/127d03e598d3840e57d9755d65283ab3
Basically, it seems that the wrap and _wrapString functions are responsible for processing fields which should be queried. By the way, I am currently working on a Knex query wrapper for my soon-to-be-released project called knex-orm.

I have just completed my little query wrapper for my ORM today, see the following code parts for my solution:
https://github.com/kripod/knex-orm/blob/f852fcb09b041ffeb79a1df16eb8fd76e5bc34ca/src/index.js#L32
https://github.com/kripod/knex-orm/blob/f852fcb09b041ffeb79a1df16eb8fd76e5bc34ca/src/query-builder.js#L42

Thanks, that looks super useful!

Translating INTO the db:
export const db = knex(config)

// Override knex formatter to translate columns from appId to app_id, etc.
const formatterPrototype = db.client.Formatter.prototype
const originalWrapString = formatterPrototype._wrapString
formatterPrototype._wrapString = function _wrapString(value) {
  return originalWrapString.call(this, readableToColumn[value] || value)
}

In addition, I need to camelize queries results. I do like this:
const originQuery = .knex.client.query;
  knex.client.query = function query(connection, obj) {
    const promise = originQuery.call(this, connection, obj);
    promise.then(function(obj) {
      obj.response[0] =  camelizeKeys(obj.response[0]);
      return obj;
    });
    return promise;
  }

It run normally, do you think it's a good solution?

@wubzz @SamDuvall  I don't think defining a specific function for translating columns is a good idea.
A suggested alternative is having a format / parse interface at the QueryBuilder level (or even at the root level: makeKnex), where a user could register a function for modifying the query (format) and the result (parse):
let db = (table) => knex('table')
  .format((q) => { q.sql = snakeify(q.sql); return q })
  .parse((res) => camelify(res))

db('testTable')
  .column('colOne', 'colTwo')
  .select()
  .then(console.log)

This would be a more generic approach which would allow other kinds of transformations while keeping the interface composable.

@razvanz This cannot be done: snakeify(q.sql) you would have to implement pretty much complete SQL parser to know what parts of string must be transformed.
So transformation must be done at some point when we still know that certain string is identifier etc. so hook would be probably something like formatIdentifier(colName => snakeify(colName)).

@elhigu How about the parsing part? I came to this issue interested in having parse hook, commonly registered for then / stream / asCallback / etc.

I would be surprised if there isn't already an opened issue for that feature, but sure create a new one if it doesn't exist yet.

It looks like the new wrapIdentifier () config option will be perfect for this:
https://github.com/tgriesser/knex/blob/9658c78d4fba55a129acc3843b5150de93d553c4/src/client.js#L175
const knex = Knex({
  wrapIdentifier(value, wrapIdentifier) {
    return wrapIdentifier(underscore(value))
  }
})

This will only be available once 0.14.0 is out...