ggd-run.sh returned non-zero exit status 255

I'm trying to install bcbio (27th Dec 2021), but am encountering this issue when installing the genome. Seek your help in this matter please. Thank you.
My command:
bcbio_nextgen.py upgrade -u skip --genomes GRCh37 --aligners bwa --aligners star
Writing to .uacRYK
Merging 221 temporary files
[E::bgzf_flush] File write failed (wrong size)
[merge_blocks] Error: cannot write to dbsnp-154.vcf.gz
Cleaning
Traceback (most recent call last):
File "/bcbio/v129/anaconda/bin/bcbio_nextgen.py", line 228, in
install.upgrade_bcbio(kwargs["args"])
File "/bcbio/v129/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 109, in upgrade_bcbio
upgrade_bcbio_data(args, REMOTES)
File "/bcbio/v129/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 361, in upgrade_bcbio_data
args.cores, ["ggd", "s3", "raw"])
File "/home/centos/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
_prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
File "/home/centos/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
retrieve_fn(env, manager, gid, idx)
File "/home/centos/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
File "/home/centos/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
recipe["recipe"]["full"]["recipe_type"], system_install)
File "/home/centos/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
subprocess.check_output(["bash", run_file])
File "/bcbio/v129/anaconda/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/bcbio/v129/anaconda/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/bcbio/v129/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 255.

Apologies, I think it's because the disk ran out of space.

Hi,
Turns out it's not due to disk space; hence re-opening the issue.
I ran the installation of genomes on a 11TB disk, and the error still shows up
Merging 221 temporary files
[E::bgzf_flush] File write failed (wrong size)
[merge_blocks] Error: cannot write to dbsnp-154.vcf.gz
Cleaning
Traceback (most recent call last):
File "/bcbio/v129/anaconda/bin/bcbio_nextgen.py", line 228, in
install.upgrade_bcbio(kwargs["args"])
File "/bcbio/v129/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 109, in upgrade_bcbio
upgrade_bcbio_data(args, REMOTES)
File "/bcbio/v129/anaconda/lib/python3.7/site-packages/bcbio/install.py", line 361, in upgrade_bcbio_data
args.cores, ["ggd", "s3", "raw"])
File "/projects/install.bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 354, in install_data_local
_prep_genomes(env, genomes, genome_indexes, ready_approaches, data_filedir)
File "/projects/install.bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 480, in _prep_genomes
retrieve_fn(env, manager, gid, idx)
File "/projects/install.bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/genomes.py", line 875, in _install_with_ggd
ggd.install_recipe(os.getcwd(), env.system_install, recipe_file, gid)
File "/projects/install.bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 30, in install_recipe
recipe["recipe"]["full"]["recipe_type"], system_install)
File "/projects/install.bcbio/tmpbcbio-install/cloudbiolinux/cloudbio/biodata/ggd.py", line 62, in _run_recipe
subprocess.check_output(["bash", run_file])
File "/bcbio/v129/anaconda/lib/python3.7/subprocess.py", line 411, in check_output
**kwargs).stdout
File "/bcbio/v129/anaconda/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bash', '/bcbio/v129/genomes/Hsapiens/GRCh37/txtmp/ggd-run.sh']' returned non-zero exit status 255.

Hi @gis-nlsim!
How much RAM do you have in the installation session?
SN

This is an R5A.4xLarge AWS instance with 128GB (not a cluster)

Unfortunately, I can't reproduce this issue, I've successfully installed GRCh37 reference with
bash-4.2$ which python3
/n/app/bcbio/1.2.9/anaconda/bin/python3
bash-4.2$ bcbio_nextgen.py upgrade -u skip --genomes GRCh37 --aligners bwa
Upgrading bcbio
Upgrading bcbio-nextgen data files
--2022-01-08 23:50:17--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... <IP_ADDRESS>

dbsnp message:
Resolving ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)... <IP_ADDRESS>, <IP_ADDRESS>, 2607:f220:41e:250::13, ...
Connecting to ftp.ncbi.nih.gov (ftp.ncbi.nih.gov)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2925007 (2.8M) [application/x-gzip]
Saving to: ‘dbsnp-154-orig.vcf.gz.tbi’

dbsnp-154-orig.vcf.gz.tbi                                   100%[=========================================================================================================================================>]   2.79M  --.-KB/s    in 0.1s

2022-01-09 02:15:15 (26.0 MB/s) - ‘dbsnp-154-orig.vcf.gz.tbi’ saved [2925007/2925007]

Writing to .vL8Hgy
Merging 221 temporary files
Cleaning
Done
Running GGD recipe: GRCh37 hapmap 3.3

and it continued till the finish:
  inflating: data/GRCh37.75/sequence.HSCHR6_MHC_SSTO.bin
  inflating: data/GRCh37.75/sequence.X.bin
  inflating: data/GRCh37.75/sequence.Y.bin
  inflating: data/GRCh37.75/sequence.bin
  inflating: data/GRCh37.75/snpEffectPredictor.bin
 extracting: data/GRCh37.75/cytoBand.txt.gz
bcbio-nextgen data upgrade complete.
Upgrade completed successfully.

The install took 5h (including STAR), session with 20G RAM, dbsnp processing step took 100GB of intermediate files, total install size 79G.
In your place, I would try to look at AWS freeze/timouts/runnig as nohup.


Coverage always fails with Error: Expected key

Versions

cairo-coverage: 0.1.0
snforge: 0.30.0
scarb: 0.28

Describe the bug
I created a brand new snforge project and tried to get coverage both by running snforge test --coverage and by using cairo-coverage directly on the json files and it always fails with message:
Error: Expected key: github.com/software-mansion/cairo-coverage but was missing

To Reproduce

Steps to reproduce the behavior:

Create a new starknet foundry project by doing snforge init my_project
Run snforge test --coverage

Expected behavior
Coverage report should be generated successfully.

As stated in the docs, you need to add following section to your Scarb.toml manifest first.
[cairo]
unstable-add-statements-code-locations-debug-info = true
unstable-add-statements-functions-debug-info = true
inlining-strategy = "avoid"

You can use

Hi @ggonzalez94,
Did that help you, or are you still encountering this issue? Also, please remember that currently, coverage doesn't work on contracts.

Hey @maciektr @ksew1 thanks this solved it. I'm closing the issue.

coverage doesn't work on contracts.
I guess this applies to components as well?


[tools] Update web.py to repair the bug of clicking button

In some cases, the related event will not be executed when clicking button by the method perform(), so replace it with js code.

Please update prefix to "[ATIP]", specific for atip PRs

LGTM


Exception when running integration tests with minikube upgrade available

Saw this while running integration tests:
build/mvn -B integration-test -Pkubernetes -Pkubernetes-integration-tests -pl resource-managers/kubernetes/integration-tests -am -Dtest=none -DwildcardSuites=org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite:
Exception encountered when invoking run on a nested suite - Unknown status There is a newer version of minikube available (v0.16.0).  Download it here: *** ABORTED ***
  java.lang.IllegalStateException: Unknown status There is a newer version of minikube available (v0.16.0).  Download it here:
  at org.apache.spark.deploy.kubernetes.integrationtest.minikube.Minikube$$anonfun$getMinikubeStatus$2.apply(Minikube.scala:69)
  at org.apache.spark.deploy.kubernetes.integrationtest.minikube.Minikube$$anonfun$getMinikubeStatus$2.apply(Minikube.scala:69)
  at scala.Option.getOrElse(Option.scala:121)
  at org.apache.spark.deploy.kubernetes.integrationtest.minikube.Minikube$.getMinikubeStatus(Minikube.scala:69)
  at org.apache.spark.deploy.kubernetes.integrationtest.minikube.Minikube$.startMinikube(Minikube.scala:51)
  at org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:82)
  at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.scala:187)
  at org.apache.spark.SparkFunSuite.beforeAll(SparkFunSuite.scala:31)
  at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:253)
  at org.apache.spark.deploy.kubernetes.integrationtest.KubernetesSuite.org$scalatest$BeforeAndAfter$$super$run(KubernetesSuite.scala:46)

Should either set minikube config set WantUpdateNotification false (https://github.com/kubernetes/minikube/blob/master/pkg/minikube/notify/notify.go#L68) or parse the results more rigorously

I'm starting to wonder if this kind of automation is a good idea, although it is convenient at times. We're relying quite a bit on minikube's messages and not any real API. Perhaps we should let our integration tests run on any kubernetes cluster that is specified, rather than relying on Minikube. In the future, as minikube changes in other ways, we may run into more issues. Thoughts on separating out integration tests from minikube?

I'd support being able to run against any kubernetes cluster rather than just the embedded minikube ones.  We shouldn't rely on anything minikube-specific, so if there are any assumptions there moving to alternate k8s clusters would ferret them out.
I'll file another issue for supporting running tests against any specified k8s apiserver.  Maybe should read apiserver config from kubectl config too.


Resolve Build Warnings on DNN.Platform Solution

When building the DNN Solution in Visual Studio a number of build warnings appear for code-quality related elements.  This clustters the build logs and makes the identification of actual issues complicated.
Description of bug
Building DNN Platform results in 676 Build Warnings, making it impossible to identify if there are any "actual" issues.
Steps to reproduce

Open the solution
Build
View Output Window and notice 676 Warnings

Expected result
Zero build warnings

Please keep it Stalebot :)

Hello,
Regarding this issue, I have set up and built the dnn.platform successfully, but only 62 warnings appear.
please note that I am using Visual studio 2019 community

@eng-aomar That is about right.  I'm going to go through this again and fix everything soon.

I want to contribute to this issue if it is assigned to nobody.

@eng-aomar We would welcome contributions for this.
Please do include detail comments for any changes made to help with review

Just a note @eng-aomar if you do this soon and without breaking changes you could target the release/9.4.x branch for a faster merge. If they are not all solved in one go, any partial fixes are also a welcome step in the right direction :)

@valadas  I will do my best.
Thank you very much, your video titled [Setting up the Tools for DNN Development] on youtube was of great help (https://www.youtube.com/watch?v=zLB6uzdqGF8)

Yeah, some things on this playlist are now outdated, but the good news is that it is now easier, we merged the AdminExperience repository into this one, so that part is now easier to test/build. Also you can now just run in powershell ./builds.ps1 to generate all assets into the Artifacts directory. Just ping us here if you run into any issue building.

According to the warnings with the code CS0618 such as for example this one "Upgrade.AddPortal(XmlNode, bool, int)' is obsolete: 'Deprecated in DNN 9.3.0, will be removed in 11.0.0. Use the overloaded method with the 'superUser' "
when tracing doing the warning in the install.aspx.cs file
I used the overloaded  with superUser passed = null
Upgrade.Upgrade.AddPortal(node, true, 0,null); and the warning is gone
Or should it be passed to some value?

I have worked on solving the warning and I succeeded to decrease the number of warnings to 2, as illustrated in the attached picture.
The first warning is concerned with newtonsoft.jason library, Dnn.AuthServices.Jwt\bin\Newtonsoft.Json.dll  Version=<IP_ADDRESS>while the HintPath=....\packages\Newtonsoft.Json.10.0.3\lib\net45\Newtonsoft.Json.dll, I tried to update the newtonsoft.jason using NuGet package manager, but the warning did not disappear. Besides, DNN.EditBar.Library uses newtonsoft.json  v =<IP_ADDRESS>.
The second is yarn.MSBuild.1.160.0
I want to ask if there is a test mechanism used in the case of those warnings or it is just enough to have a free-error solution? and if there is any suggestion about those two warnings to be solved.
Thank you

Oh, there are way more warnings than those 2. Many of the projects throw 10-20 warnings each for multiple mixed reasons.
As for the two you see here, the first issue is that two projects depend on different versions of that same assembly, we need to find which and reference the same as the higher version.
For the yarn.MSBuild one, it must be imported twice in the build scripts on different files, on this second one, make sure you get the latest updates, this may have been fixed already in the last couple of days...

@valadas  Thanks for responding. Actually, There were more than 2 warnings, but I have fixed them all, as the solution was showing 62 when first I build it.
I will work on the remaining and try to fix them soon, but I have to know if there is a test mechanism used to verify the functionality rather than having an error-free solution?

Oh awesome. There are some automated tests that will run with the Pull Request automatically but they do not cover everything. We will also review the pull request and if we see any risks from there we will both let you know and test the affected areas. If you want reviews early before it is all final, you can also submit a "work in progress" pull request


GH Action for processing Blazor issues

@adegeo (cc: @Rick-Anderson @tdykstra @wadepickett) ... I'll give this a shot and see what happens ...

Moving the Blazor issue processing over to a new workflow.
The new workflow will apply the aspnet-core/svc and blazor/subsvc labels, along with the usual job tasks that I do for Blazor issues: place a 'hello' comment, label it with Blazor, assign me to the issue, and remove the Not Triaged label (if present ... if Repoman ran and applied it).
I'll try to run this workflow on an opened issue with a path of aspnetcore/blazor/**. I'm not sure about the on conditional checks because GH docs don't show an example of this combination working ... but we'll see soon enough! 😆

If this works, my recommendation will be to remove the initial labels that Repoman is applying and apply all them via a workflow.
I'll ping everyone in a moment from a test issue, and we can see if this actually works 🤞🍀.
If paths: - 'aspnetcore/blazor/**' doesn't trigger the job because paths only works for file push actions, then I'll try a conditional-based approach at the job level to see if I can get it to recognize the path text aspnet/core/blazor in the opening comment text.

Internal previews

📄 File
🔗 Preview link

.github/workflows/blazor-issue-processing.yml
.github/workflows/blazor-issue-processing

.github/workflows/issue-processing.yml
.github/workflows/issue-processing

I haven't tested yet; but the more I think about this, the less likely I think it will work. The path wouldn't make sense for issues ... only file changes on the repo.
I'll move to trying an conditional job after this (probably) fails. I still have a chance to make a check on the opening issue comment text.


Change predicate pattern to match paths better

Not much to it, but had me scratching my head for a while, hopefully of use....
gateway-root/my-app/my-endpoint
was resolving to
my-app-root/my-app
now it resolves to
my-app-root/my-endpoint

@admoriarty I think this may be obsolete now. Will you verify?

Yup @spencergibb its obsolete...closing


[Question] Can dolphinscheduler send alert information by http request? How to use it?

My dolphinscheduler version is 1.3.2,it is deployed in the interval network,the engineer only provided the http interface,How do i use it?
我的ds的版本是1.3.2,目前是被部署在内网,要发企业微信告警,目前运维只提供了http请求的接口,我要如何使用它呢?

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

OK,Thank you!

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

OK,Thank you!

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

OK,Thank you!

You can follow this ISSUE (#3049). I will close #3049 when finished

You can wait a little, we will probably release alert-spi in the next version, which can meet your needs, and you can check the documentation to finish it then.

OK,Thank you!

You can follow this ISSUE (#3049). I will close #3049 when finished


[BUG] When using (*JsonParseNode).GetXValue methods it panics

When using (*JsonParseNode).GetXValue methods it panics due to the value being assert as specific type when it's not.
module version: 1.0.7
golang version: 1.22.5
OS: Window 11
minimum reproducible code:
package main

import (
	"log"

	jsonserialization "github.com/microsoft/kiota-serialization-json-go"
)

func main() {
	data := []byte(`"I am not a float"`)
	node, _ := jsonserialization.NewJsonParseNode(data)
	float, err := node.GetFloat64Value()
	if err != nil {
		log.Print(err)
		return
	}
	log.Print(float)
	return
}

console output:
panic: interface conversion: interface {} is *string, not *float64

goroutine 1 [running]:
github.com/microsoft/kiota-serialization-json-go.(*JsonParseNode).GetFloat64Value(0xc000068620)
	.../go/pkg/mod/github.com/microsoft/kiota-serialization-json-go@v1.0.7/json_parse_node.go:463 +0xb2
main.main()
	.../min.go:12 +0x105

would probably be easiest to just use something like this:
func GetValue[T any](in interface{}) (*T, error) {
	var zero *T

	if in == nil {
		return nil, nil
	}

	val, ok := in.(*T)
	if !ok {
		return nil, fmt.Errorf("Expected type: %T, got %T", zero, in)
	}

	return val, nil
}

in each method

Thanks for raising this @michaeldcanady
I believe the suggestion here is valid. The code seems to assume the type is surely known ahead of time but such a failure should indeed be propagated through the error return.
Any chance this is something you'd be willing to submit a PR for?

@andrueastman

Do we want anything that can be of type X to be converted (ex: the value is 1, which is a number, but the user calls GetStringValue, should it error or return "1".)?

@andrueastman I believe if it's possible for the conversion to occur, we should return the value.
As this is a Json library, the parsed json would ideally have string surrounded by double quotes. So, calling GetFloat64Value on a json value surrounded by double quotes should fail(its a string). Similarly, if the json value is a number, it won't have doulbe quotes, so calling GetStringValue value would also fail.
This is however different from types derived from strings like Guid where you can still get a string value from it or get the Guid value. Let us know if you have any more questions.

One more, hopefully, the following test from .\intersection_type_wrapper_test.go is failing - assuming it was passing before - it seems to be failing since the new parsing won't handle the int as string.
func TestItParsesIntersectionTypeComplexProperty2(t *testing.T) {
	source := "{\"displayName\":\"McGill\",\"officeLocation\":\"Montreal\", \"id\": 10}"
	sourceArray := []byte(source)
	parseNode, err := NewJsonParseNode(sourceArray)

	if err != nil {
		t.Error(err)
	}
	result, err := parseNode.GetObjectValue(internal.CreateIntersectionTypeMockFromDiscriminator)
	if err != nil {
		t.Error(err) // value '10' is not compatible with type string
	}
	assert.NotNil(t, result) // Expected value not to be nil.
	cast, ok := result.(internal.IntersectionTypeMockable)
	assert.True(t, ok) // panic: runtime error: invalid memory address or nil pointer dereference
	assert.NotNil(t, cast.GetComposedType1())
	assert.NotNil(t, cast.GetComposedType2())
	assert.Nil(t, cast.GetStringValue())
	assert.Nil(t, cast.GetComposedType3())
	assert.Nil(t, cast.GetComposedType1().GetId())
	assert.Nil(t, cast.GetComposedType2().GetId()) // it's expected to be null since we have conflicting properties here and the parser will only try one to avoid having to brute its way through
	assert.Equal(t, "McGill", *cast.GetComposedType2().GetDisplayName())
}

As this is a Json library, the parsed json would ideally have string surrounded by double quotes. So, calling GetFloat64Value on a json value surrounded by double quotes should fail(its a string). Similarly, if the json value is a number, it won't have doulbe quotes, so calling GetStringValue value would also fail.

Hello @saviorand! Thank you for your question, Assuming you’re referenced JSON is like the following:
{
"Key":"1"
}

Then yes.
the goal is to parse what the underlying type is. So in the above example even though it’s a stringified integer, the value is a string. I included the discussion point where this was decided.

@saviorand I unfortunately do not.


COMP: Replace CoordRepType with CoordinateType

Addressed the warnings at https://open.cdash.org/viewBuildError.php?type=1&buildid=10063093 reported by Dženan Zukić (@dzenanz) at https://github.com/InsightSoftwareConsortium/ITK/pull/5006#issuecomment-2515093995
Following ITK pull request https://github.com/InsightSoftwareConsortium/ITK/pull/4997 commit https://github.com/InsightSoftwareConsortium/ITK/commit/e667fa39a6914d666ae5dd0b47202ebfb2acfd1f

I now realize that your recent refactoring needs to make its way into a Python release before we can use it in CI. Let's wait with this PR until then.


Added support to load from memory

I added support to load a shapefile from memory. This way if you store the shp/dbf files using go-bindata then you can load it using byte arrays.
    shpFileData, err := ImageData.Asset("ne_50m_admin_0_countries.shp")
    if err != nil {
        panic(err)
    }
    dbfFileData, err := ImageData.Asset("ne_50m_admin_0_countries.dbf")
    if err != nil {
        panic(err)
    }
    shape, err := shp.OpenFromMemory(shpFileData, dbfFileData)
    if err != nil {
        return err, nil
    }
    defer shape.Close()

Codecov Report

Merging #36 into master will decrease coverage by 1.1%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #36      +/-   ##
==========================================
- Coverage   60.04%   58.94%   -1.11%
==========================================
  Files           7        7
  Lines         801      816      +15
==========================================
  Hits          481      481
- Misses        264      279      +15
  Partials       56       56

Impacted Files
Coverage Δ

reader.go
54.79% <0%> (-6.28%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c721ed4...14fa0b3. Read the comment docs.


Get details off but sometimes click on icon its shows and not close then

Get details off but sometimes click on icon its shows and not close then

icon pe click kr k drag karne pe hota hai ye issue

remove popover design
new design flow process  updated.


Bower ?

Hi,
This issue is just an idea for future releases to make the starter kit more flexible.
Bootstrap is "pre-packaged" in your app, so when we clone your repo we will have a specific version of bootstrap you choose.
Why not using Bower to install it after cloning, and write an Elixir recipe to handle it from where it's installed ?
source : http://www.codeheaps.com/php-programming/creating-website-using-laravel-5-elixir-assets/
Thanks for your work :)

Good idea, I like it. I will leave it here for some others to add their thoughts.

This morning I worked on it.
So now, not only Bootstrap but even Angular and its modules are managed with bower.
Steps to reproduce :
create a bower.json
{
  "name": "app",
  "version": "0.0.0",
  "homepage": "http://www.app.com",
  "license": "MIT",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "angular": "1.4.2",
    "angular-route": "1.4.2",
    "angular-resource": "1.4.2",
    "ngstorage": "~0.3.7",
    "angular-bootstrap-file-field": "~0.1.3",
    "bootstrap": "~3.3.5"
  }
}

Then
bower install

Change your gulpfile.js to :
var elixir = require('laravel-elixir');
// require('laravel-elixir-livereload');
/*
 |--------------------------------------------------------------------------
 | Elixir Asset Management
 |--------------------------------------------------------------------------
 |
 | Elixir provides a clean, fluent API for defining some basic Gulp tasks
 | for your Laravel application. By default, we are compiling the Less
 | file for our application, as well as publishing vendor resources.
 |
 */

elixir(function (mix) {
  mix
    .less(
      'app.less',
      'variables.less' // to customize bootstrap.
    )
    .styles([
      'style.css'
    ])
    .scripts([
      '../../../bower_components/jquery/dist/jquery.js',
      '../../../bower_components/angular/angular.js',
      '../../../bower_components/ngstorage/ngStorage.js',
      '../../../bower_components/angular-route/angular-route.js',
      '../../../bower_components/angular-resource/angular-resource.js',
      '../../../bower_components/bootstrap/dist/js/bootstrap.js',
      'app.js',
      'appRoutes.js',
      'controllers/**/*.js',
      'services/**/*.js',
      'directives/**/*.js'
    ])
    .version([
      'css/all.css',
      'js/all.js'
    ])
    .copy(
      'public/js/all.js.map', 'public/build/js/all.js.map'
    )
    .copy(
      'public/css/all.css.map', 'public/build/css/all.css.map'
    );
    //.livereload();
});

Run
gulp && gulp watch

et voilà, everything is versioned.
You can safely delete (no longer used) :

resources/assets/less/bootstrap*
resources/assets/js/libs*

You can add a resources/assets/less/variables.less to customize bootstrap styles.
Now, each time you need a new angular module or other front end stuff, just run
bower install yourStuff --save

It will be installed in bower_components. Don't forget to add it to your gulp file.
NB : to complete this, install livereload on chrome, laravel-elixir-livereload and uncomment lines in my gulpfile.

Excelent!

Great work, @pierrerigal!
I thought of making this part of the repo by default. Though I think this repo should be as close as possible to what Laravel and AngularJS are like out of the box in order to focus on the Laravel/AngularJS implementation and not adding any other "complexities".
What I have in my mind is a collection of pull requests of prototypes that demonstrate some certain scenarios, possibilities and expansion capabilities of this fundamental repo.
Could you do a PR for the Bower integration. I would appreciate it very much. Thank you!

Done :)


send context(instance) for all function calls

for now it is impossible to use methods when defining an element like this:
$loaderControl.infinitescroll({
            nextSelector:$('[data-lazy-loading-btn="true"] a:first',that.$el),
            navSelector:$('[data-lazy-loading-btn="true"]',that.$el),
            appendCallback:false,
            path:this.lazyPath,
            // debug:true,
            prefill:true,
            itemSelector:'.vc_grid-item',
            loading: {
                msg: $loader,
                selector: that.$loaderControl,
                start: function(opts){ (INSTANCE).beginAjax(opts); // this is impossible yet. },
                finished: function(opts) { $loader.css('opacity',0); }
            }
        }

this is because you send only wrapper and options but not sending context.
opts.loading.start.call($(opts.contentSelector)[0],opts);
better would be to send
opts.loading.start.call($(opts.contentSelector)[0],opts,this);
so it will allow to use infinitescroll methods (like _prefill, beginAjax and etc).

Infinite Scroll v3 is out!. It's an all new re-write, so take a look at how to upgrade from v2.
Infinite Scroll v3 comes with vanilla JS support. Additionally, you can access the instance with [.data('infiniteScroll')](https://infinite-scroll.com/api.html#jquery-fn-data-infinitescroll).
The loading.start and loading.finished option methods have been replaced with events: request and append or load.
I recommend upgrading to v3. Please open a new issue if you run into trouble with Infinite Scroll v3. 🌈🐻


[C-1883] Re-direct app.crowd.dev to app.crowd.dev/auth/signup (if user is not logged in)

Problem
When visiting https://app.crowd.dev/, users can re-directed to https://app.crowd.dev/auth/signin if they are not logged in
Solution
Users should be re-directed to https://app.crowd.dev/auth/signup if they are not logged in to increase conversion
From SyncLinear.com | C-1883

@joanagmaia is this still a valid issue? If so, please provide more context as I cannot reproduce this

I think it's resolved.

@peoray @a0m0rajab, I think the issue still holds up. To perform a quick test, I would do the following steps:

Launch an incognito browser window
Navigate to https://app.crowd.dev/
We're still being redirected to https://app.crowd.dev/auth/signin

However, the improvement suggested here aims to redirect users who aren't signed in to https://app.crowd.dev/auth/signup instead.

@yeganathan18 with your steps if works as expected, since we are not logged in the incognito mode

Exactly, so we should be changing the redirection from /signin to /signup.

@a0m0rajab wanna pick it up?

Yeah, I was thinking to pick it up, mostly the code needed to change is in this lines:
https://github.com/CrowdDotDev/crowd.dev/blob/e91b729162b96eb4707bf0ab91cfbc5fe02830ef/frontend/src/middleware/auth/auth-guard.js#L15-L40
@peoray if you like you we can collaborate on it? (yes, it's simple but yeah)

@peoray Thanks for that! If you feel to pick it up, it's okay for me too!


Fix manual release/dispatch workflows

What

manual_build_release now includes reusable_build_web_demo/reusable_upload_web_demo
manual_release now uploads the the commit from rerun-demo/commit/<commit> to /version/latest
bandaid for breaking manual_dispatch: changed BUILD_WEB input in manual_dispatch to accept a string instead of a bool (example values: app, demo, app,demo), and removed BUILD_WEB_DEMO

without this the number of inputs exceeds 10 and the workflow is invalid and cannot be used

bandaid to ensure adhoc_web will not overwrite the latest commit build: added UPLOAD_COMMIT input to upload_web and upload_web_demo which is on by default, and set it to false in manual_adhoc_web

Checklist

[x] I have read and agree to Contributor Guide and the Code of Conduct
[ ] I've included a screenshot or gif (if applicable)

PR Build Summary: https://build.rerun.io/pr/2230

manual_release now uploads the the commit from rerun-demo/commit/<commit> to /version/latest

We already use latest as a tag for the latest release, e.g. https://github.com/rerun-io/rerun/tree/latest so I think we should use a different word for "the most recent manual release", but I'm not sure what that would be


This program is not suitable for development scale 👎

describe issue:
Using import like the code below will collect garbage, this can cause RAM leaks.
Target issue:
https://github.com/xyzencode/ZaydenBot/blob/287374fde4d1db61cbcb5ca5bd340e39ce2c722a/main.js#L212

OK, thanks, I'll update later


fix(HACBS-2471): create tag based on full git sha

Create tag based on full git sha

Shouldn't this be 0.10, not 1.0?

fixed


Example code going beyond textwidth

The example code provided in the following screenshot for the blood-type problem, seems to go beyond the specified textwidth limit of the website.

Looks like it can be fixed by changing the white-space style from white-space: pre to white-space: pre-wrap
I also found the issue on other pages occur when the window size is decreased, like the Colourful Resistors problem


Improvements based on improvement proposals

PR based on improvements specified in https://github.com/hyperledger/composer/issues/3130
Changes made
General

Changed vehicle-lifecycle naming to vehicle-manufacture

Installer

Updated references to vehicle-lifecycle-network business network to be vehicle-manufacture-network
Updated flows.json to remove previous Node-RED implementation and replaced it with a solution that listens for an UpdateOrderStatusEvent and sends and email if status is updated to delivery, the user paul has an email set and the user of the demo has configured the send email node.
Added install for using fabric v1.1 which installs against composer @next
Install now listens for an environment variable of DEMO_FABRIC_VERSION if unset it defaults to fabric 1.0 and composer @latest, if set to hlfv11 it uses fabric 1.1 and composer @next

Car builder

Updated config to include reference to the REST server details
Updated app.js to return the JSON config unless the environment variable REST_SERVER_CONFIG is set with the REST server details.
Updated building screen to show tick when selecting trim and extra options
Updated building screen to prevent submission without filling in trim, colour and interior details for the car
Updated building screen to submit PlaceOrder transaction to REST server
Updated PlaceOrder request format to reflect changes in the business network
Updated status screen to listen to websockets coming from the REST server

Manufacturing

Updated config to include reference to the REST server details
Added app.set to server/app.js that stores value from config file or if set the environment variable REST_SERVER_CONFIG
Updated websocket in server/app.js to prevent app death on disconnection from remote and set it to use the config set so that it listens to the REST server for events
Removed vehicles and transactions endpoints.
Added orders endpoint to retrieve PlaceOrder transactions using the REST server and combine with retrieved UpdateOrderStatus transactions to handle that order assets no longer store an array of the updates placed on them
Added updateOrderStatus endpoint to submit UpdateOrderStatus transactions via the REST server
Updated dashboard controller loading of statusUpdates to handle the way they are returned by the orders endpoint
Updated dashboard controller to use a single websocket for all events
Updated updating of order status' in dashboard controller to use updateOrderStatus endpoint

VDA

Updated config to include reference to the REST server details
Added app.set to server/app.js that stores value from config file or if set the environment variable REST_SERVER_CONFIG
Updated websocket in server/app.js to prevent app death on disconnection from remote and set it to use the config set so that it listens to the REST server for events
Removed suspicious vehicles page
Removed suspicious vehicles number from header
Updated dashboard controller to work with v0.17.0 and remove composer events from VDA list e.g. AddUser.
Updated daashboard controller to display the identities involved in the transaction correctly by name rather than using hard coded defaults
Updated to use a single websocket to listen for events coming from the REST server

Relies upon

[ ] vehicle-manufacture-network being created as a sample in http://github.com/hyperledger/composer-sample-networks
[ ] install will only function if docker image released using new vehicle-manufacture naming
[ ] hlfv11 installer's node red flow relies on updated composer node red modules available @next

PLEASE DO NOT MERGE UNTIL RELIED UPON TASKS ARE COMPLETE

Sample network appears on npm but version does not match 0.1.1 only at version 0.2.0

Hmm think they were a hangover from when I had a play with multiuser mode I'll work at taking them out. Also found that the VDA completely crashes if there are no transactions so I will work to fix that in the same commit


Update information in/structure of CONTACTS.yaml

Stab in the dark at better structure for contact information for foundation/council roles.
Objective of this file is to avoid having contact information manually sprinkled everywhere. The website will use this information to populate relevant sections. I hope to replace website email list reference with a form (perhaps).

LGTM! Short and to the point.


WIP youtube-dl stitching fixes

How the hell did the python 3.5 test pass here?
https://travis-ci.org/internetarchive/brozzler/builds/429929711
That was the commit that had the test, but not the fix.
Hmm looking at the travis-ci output I guess it grabs the latest from the pull request branch. I guess I pushed the fix too quickly.
$ git fetch origin +refs/pull/119/merge:
$ git checkout -qf FETCH_HEAD


Add category/platformType to emulators

@devoncarew @jonahwilliams does this seem like a reasonable change? I started showing emulators in my "device picker" but noticed that for a macOS project it would list emulators. For now I've made it assume that emulators are only useful if supported platforms contain ios or android, but it'd be nicer to not have assumptions like that (and if in future we ever have emulators for non-mobile platforms, it'd fall down without this).
Description
Adds category/platformTypes to emulator results from the device daemon.
Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
[x] I signed the [CLA].
[x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Does your PR require Flutter developers to manually update their apps to accommodate your change?

[ ] Yes, this is a breaking change (Please read [Handling breaking changes]). Replace this with a link to the e-mail where you asked for input on this proposed change.
[x] No, this is not a breaking change.

Looks like Cirrus has failed to update the status for the Windows bot (it's in-progress here, but has completed and is green at https://cirrus-ci.com/build/5159499699060736) so merging this.


Minecraft crash

When I send the command on a minecraft server, it crashes bruh

Crashes what exactly? The MC server or client? We had issues where it was crashing the client too. Haven't looked more into this but I think it happens because it doesn't get a valid response from the server.

the client

first port is python's http server and second port is nlvp ncat server

Do you get anything like this:

Hi,
I ran into some similar issues while we were creating this POC. Your issue is most one of 2 things:

1 could be that your Minecraft server cannot reach your LDAP server.
2 if your LDAP server is being reached, then it possibly isn't returning anything valid. Is there any output on the LDAP server logs?

As a side not I want to ask that in order to make both of our lifes a little bit easier, could you please add a little bit more detail into your issues. It would be helpful for me a few more details including, but not limited entire logs of both your LDAP and Minecraft server, as well as what os and java version you are running.

the minecraft server isn't running on my pc but yes I can give you poc's logs

HTTP server

POC

NETCAT

I also opened these ports

Hi,
Sorry, what do you mean about the server not running on your machine. Do you have access to pentest the minecraft server? If you do can you please send us some sort of proof of that before we can help you on. We do not want to aid or endorse our programs being used for any criminal activities.

Okay, I see it gets the Exploit.class from the HTTP server, so it is working there so you should get a shell. My assumption is that its because you are port forwarding from your PUBLIC IP (which you probably shouldn't do) or its the server you are testing it on, thats why we asked for logs... You should probably set up a server to use instead of port forwarding..
As for what @svmorris  , please only test if you have permissions to do so.

it's the server of a friend, I wan't to troll him but I don't have logs (I'll not use my public ip on another server lmao)
I'm going to try with a self hosted server

I'm getting this error

Understandable bcz the lib is not present on the server :/


Fixes issue 668 - fail to read OSPFv2 LSA Type 2

Please review this PR for issue https://github.com/google/gopacket/issues/668#issue-459392815 (OSPFv2 Type 2 LSAs not being decoded).
This PR adds a new NetworkLSAV2 struct following RFC 2328 A.4.3, and a new test case for it.

Thanks for the contribution


Amend NEP-3

@neo-project/core @Tommo-L

💯

It's good for me, I think that it will improve regular executions

It's also good for me


add .items() methods to support dict dump

Added methods to support dict(config.items()) (#55)
Didn't quite get how to make tests for this case since there is no tests/test_config.py, only tests/test_autoconfig.py.

@henriquebastos can you have a look?

For the tests I think you could simply add a method to test_env.py and test_in.py

I've added tests but there was some issues on the way.

RespositoryIni didn't preserve case sensitivity for keys and values of config. So I've added self.parser.optionxform = str in initialization.
In Python3 {}.items() returns class <dict_items> and in Python2 same returns list. I've omitted type checking for RepositoryEnv in tests. Another options is to cast explicitly list(self.parser.items()) for RepositoryEnv.items method, if we want RepositoryEnv.items and RepositoryIni.items return same type.

@henriquebastos

@henriquebastos can you have a look?

Closing the PR until a concrete use case makes sense.


Left column navigation on /learn pages.

The expected behavior of the accordion nave pages in the learn section is:
When you click on a parent heading, the sub items should expand. If I then click on it again they should accordion closed again.
What happens. The only way to close a set of sub headings is to click another parent heading.
This wasn't an issue when the document set was light, but with so many docs pages sometimes the accordion'd subsections require a scroll to get to another parent level section.

@mattgreg under each item ( even if it has children items inside ) we have an article, so clicking on it will cause open this article. This could be confusing. My suggestion is to make this chevron icon clickable which will make the action we need ( collapse/expand ) WDYT ?

yes. that chevron icon action is perfect.

@mattgreg merged to develop (#327)


Update guidance for XFS - nobarrier option removed

As of kernel version 4.19, the nobarrier and barrier options for XFS have been removed. Please provide updated guidance on how to optimize disk performance when using XFS on distributions are using newer kernel versions. This affects VMs running Ubuntu 18.04 LTS and likely others.
For example, on Ubuntu 18.04, the command mount -t xfs -o defaults,nobarrier /device/path /mnt/path for example results in the error:
mount: /mnt/path: wrong fs type, bad option, bad superblock on /device/path, missing codepage or helper program, or other error.

Note these are dummy device and mount paths. But the same exact command on older distros like RHEL 7 works.
Sources:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1c02d502c20809a2a5f71ec16a930a61ed779b81
https://kernelnewbies.org/Linux_4.19#File_systems

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f7792d5a-de76-c48a-0c8d-46b3e918a503
Version Independent ID: 23b37b12-4a16-d108-2705-2d96ceb484c8
Content: Optimize your Linux VM on Azure - Azure Linux Virtual Machines
Content Source: articles/virtual-machines/linux/optimization.md
Service: virtual-machines-linux
Sub-service: disks
GitHub Login: @RicksterCDN
Microsoft Alias: rclaus

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

This has been opened since May, any feedback from the author ?

We are currently cleaning up old issues and closing out items that are greater than 90 days old. If an issue is still present, please open a new feedback item on the document so we can prioritize correctly.
#please-close


ARXIVNG-556 Audit tool for missing records

Added the script audit.py in the root of the project, implemented with click. This takes a list of paper IDs (including version affix), and outputs a list of missing paper IDs. A full scan may take one to three hours, depending on your internet connection.
From the docstring:
Check the index for missing papers.

    Uses HEAD requests to the document endpoint using each paper ID in the
    provided list. Uses multiprocessing with a configurable number of worker
    processes to speed things up.

    Parameters
    ----------
    id_list : str
        Should be a path to a file with paper IDs. There should be one paper ID
        per line. Paper IDs should include version affixes.
    batch_size : int
        Number of records per batch. Each batch is divided into equal chunks
        and distributed across workers. Smaller batches mean more frequent
        updates/checkpoints. Must be divisible by the number of workers.
        Default: 1,600.
    n_workers : int
        Number of worker processes to run for each batch. Default: 8.
    output : str
        Path to a file into which to deposit paper IDs not found in the index.

For example:
ELASTICSEARCH_HOST=<IP_ADDRESS> pipenv run python audit.py -l list_of_papers.txt -o missing.txt

Coverage decreased (-1.4%) to 82.872% when pulling 81702f5b18193106dc4bbb1dfd6961bb5b6a1b97 on bug/ARXIVNG-556 into fd07541fab05e856a0e761f14cd8155ddff77555 on develop.

Fixes #139


SASL Authentication for Clients

Here's a peek at where this code is at, still needs to handle too many authentication attempts, but is otherwise fairly complete. PLAIN mechanism has been moved to a rudimentary module since the last time you've seen it. I wasn't sure if there was a more proper way to add the SASL mechanisms to the CAP LS output, so I just hacked it in for now. Any suggestions are welcome. I will be further testing module behavior for more complex mechanism types shortly.

Any new on this?

Rebased and addressed comments in https://github.com/znc/znc/pull/1873


Upgrade cron parser

It fixes some bugs cron-parser had in previous versions.
Added some cron jobs tests.

BTW, instead of running real timers in the tests I've used sinon faketimers. It would be interesting using fake timers for the rest of the tests so they run faster. At the moment it takes over a minute running them


[1.19.2 / SUGGESTION] Add config option to disable tooltip added by mod.

Add config option to disable tooltip added by mod.

Added as of 9a837f20a9eb4c532d37fbd1df5e482dbb0c3a66 and 27df22addb58344a3c089e924a66b28be418975e


Removed extra Travis jobs

Summary

This PR removes redundant Travis jobs to reduce build times.

Closes #96
Deployment Link

https://terra-graphs-deployed-pr-#.herokuapp.com/

Thank you for contributing to Terra.
@cerner/terra
@cerner/carbon

Just curious. It looks like the only graphs-spec.js wdio test file in both carbon-graphs and terra-graphs are commented out. The travis build currently doesn't have any wdio tests to run even for the remaining default theme. Does it make sense to also remove the jobs for the default theme to save even more time until when there are wdio tests written?

@benbcai That's a good point. However, we're currently working on wdio tests so we don't really get any substantial benefit from doing a change that'll get undone in the coming weeks.

@benbcai That's a good point. However, we're currently working on wdio tests so we don't really get any substantial benefit from doing a change that'll get undone in the coming weeks.

Oh I see. I wasn't aware wdio tests are coming very soon so ignore my previous comment.


Support uniform arrays in impeller shader configuration

uniform Foo {
  float values[8];
} foo;

Currently this only generates a single float and 7 floats worth of padding. If this uses only a single slot might be worth it, but it if this takes up 8 then we should use different types.

actually, the generated metal shaders look like they correctly define an array of floating point values - its our generated headers/metadata that seem incorrect. Still investigating


Update storage-service-encryption.md

I'm guessing this FAQ should be updated since the encryption settings for storage accounts in the Azure Portal states:
"Please note that after enabling Storage Service Encryption, only new data will be encrypted, and any existing files in this storage account will retroactively get encrypted by a background encryption process."

@AWahlqvist : Thanks for your contribution! The author, @lakasa, has been notified to review your proposed change.

@AWahlqvist - Thank you for making that update!
Approved - #sign-off

@tamram: I'm sorry - only the author of this article, @lakasa, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


Draft: Performance mega boost - queue per app

Issues related
https://github.com/kubeflow/spark-operator/issues/1138
https://github.com/kubeflow/spark-operator/issues/1574
https://github.com/kubeflow/spark-operator/issues/783
https://github.com/kubeflow/spark-operator/issues/1489
Purpose of this PR
First of all - this PR is mainly a Draft which we think should be discussed, and that's why we are submitting it even though we haven't added documentation not unit tests.
The original design of spark-operator posses one queue which is used for all the applications.
This design is causing a huge latency issue when trying to deal with hundreds / thousands of applications concurrently.
In benchmarks me and my fellows from Mobileye preformed, we showed clearly a linear latency increase depend on the amount of apps handled.
When getting to more than 500 applications, the avg time from creating a spark application object until pod creation is ~130 seconds for app to start. When going up to 1200 apps concurrently it can go Up to 20 minutes on average for each spark application to be created.
Scale up vertically would not be helpful as the CPU is doing nothing, most of the time is spent on the queue mutex.
The change we are presenting here is to  create a queue for each app.
It required a big change all around the code, but it is not changing the main flow in any way.
Our benchmarks showed that even with 1000 apps concurrently, Avg time for application creation is ~7 seconds
We also added a nice feature of using memoryLimit for driver / executor which is larger than the request by using the admission webhook.
Proposed changes:

create a queue for each app
Add a parameter to controll qps and burst for k8s api
adding memoryLimit option for executors and drivers

Change Category

[ ] Feature (non-breaking change which adds functionality)

What are we still missing:

fixing broken unit test
documentation
Peer review

I would point out that this code currently runs on our production environment with massive scale without any issues.

/assign @ChenYi015
/assign @yuchaoran2011
Please review the changes when you get a chance

Impressive work! A few questions:

What do we need K8s burst and QPS for?
What's the impact on memory and CPU consumption of setting a higher or lower frequency of queue cleaning?
Will there be edge cases where a queue's mutex is already deleted due to not receiving any updates for a while but the application still exists?

Hi @yuchaoran2011  thanks for reviewing!

Qps:  we first did all the rest of the code and did not increase qps and Burst and the outcome was pretty good, but several issues still bothered us:

CPU usage was very high at high scale
Even though app submit was fast, application status updates were very slow and could take 20 minutes at scale, which was really bad for us
At some point app were just stuck forever without status, even though the app was finished, and when spark operator restarted - it submitted the apps again as they had no status and no resources related
After profiling we discovered most of the time was spent on kiss API exponential back off that was caused from low qps and Burst
(We also tested just raising the qps and Burst without the rest of our changes and it was not preforming well)

As for memory and CPU consumption -  we mostly tried to avoid running the queue clean often as it's locking a map when it cleans it. We haven't noticed high memory nor CPU but I can provide some CPU and memory consumption graphs to show the affects of amount of apps etc.

about the edge case, yes of course it can happen, especially of you

set low values of queue cleanup intervals, but it does not cause any harm as every access to an app queue is always by the get or create method and the cleanup is not cleaning queues that aren't empty

Thanks for this patch!
We also ran into similar issues at scale. What we observed was a race condition between the SparkOperator reconciliation and k8s Pod GC. We have autoscaler that terminates nodes when no Pods is running so completed Pods are deleted from API server when the node is terminated. If the SparkOperator could not update the status in time, it won't find the driver Pod and thus make the app as failed due to driver pod not found.

@bnetzi Thanks for the clarification. That makes sense. The default is 50, if I remember correctly. Doubling the Spark operator QPS strikes a balance between allowing for more requests from operator to be processed and not overloading k8s API server. Overall the idea looks really solid. I second fleshing out this PR to include tests

@bnetzi Thanks for the clarification. That makes sense. The default is 50, if I remember correctly. Doubling the Spark operator QPS strikes a balance between allowing for more requests from operator to be processed and not overloading k8s API server. Overall the idea looks really solid. I second fleshing out this PR to include tests

The default in golang is actually 5 qps and 10 burst which is very low.
Maybe we should put default value which is lower than 100 and 200 by default for small k8s clusters or such with mix usage

Please help me to understand your statement in the description more

Scale up vertically would not be helpful as the CPU is doing nothing, most of the time is spent on the queue mutex.

For the original runWorker()->processNext(), processNext()'s queue.Get() get a spark-app to process. Multiple workers calling runWorker() each will pull spark-app items from queue sequentially because of queue mutex, but the items will be processed concurrently inside each worker's processNext(). Is 20 minutes on average incurred by processing the item, but not by blocking in queue.Get()?

@bnetzi I think @vincentye38 brought up a valid point above. What he's proposing (i.e. having more workers concurrently process items) complements the solution proposed in your PR. Have you tried bumping up the number of workers to see what effect that has on performance?

@vincentye38 @yuchaoran2011 Thanks for commenting
First I would mention that we obviously increased the amount of controller threads while raising the cpu, and it did not help at all. the cpu was 5% on a 32 vCPU instance.
I think that in order to understand it we need to understand some of the mechanism and parameters involved here.
In my understanding (and this behavior is what we saw in the logs as well) each of the go routines is waiting in line to preform a q.Get() but the next item in queue be processed by another go routine only when q.Done() is called which is after we are doing the sync. It actually makes a lot of sense as if we have an event that triggers app creation, and then some app update - we want the creation to be done and the app state to be synced before we are preforming the next operation.
But this design makes all other go routines to wait for this task to be done. So the controller threads not making a significant impact on performance.
Also - the operator is using a a RateLimitingInterface workqueue, and this queue is also adding waits when the rate limit is reached, so it makes everything even worse. We tried to increase the limits and it helped a little but did not help enough, and that's why we decided on a queue per app approach.
Another point to mention here, (and in my opinion it is what  causing us to get to finally 20 minutes after running a long time in a large scale env) is that we have a vast amount of events per app. It is not just app creation, driver pod start and finish, but also each executor pod state change is an event. This is causing much longer waiting time as there are much more

@bnetzi Thank you for explaining. It's interesting that the cpu utilization is so low.

each of the go routines is waiting in line to preform a q.Get() but the next item in queue be processed by another go routine only when q.Done() is called which is after we are doing the sync.

My understanding from the source code of q.Get() and q.Done() is that go routines can pickup other items from the queue. the queue deduplicates item which has identical one in the queue or in process when the item is inserted. The new identical item will be set aside into dirty set util the one in process is marked by q.done(item), then the new item will be moved from dirty to queue. WorkQueue is a dedup-queue. so q.done() doesn't block other go routine to q.Get() other item.
I guess the problem that cause low cpu utilization, low throughput and blocking items being dequeued is RateLimitingInterface. Could you mind to test out using WorkQueue directory without wrapping it with RateLimitingInterface? Thank you.

@bnetzi Thank you for explaining. It's interesting that the cpu utilization is so low.

each of the go routines is waiting in line to preform a q.Get() but the next item in queue be processed by another go routine only when q.Done() is called which is after we are doing the sync.

My understanding from the source code of q.Get() and q.Done() is that go routines can pickup other items from the queue. the queue deduplicates item which has identical one in the queue or in process when the item is inserted. The new identical item will be set aside into dirty set util the one in process is marked by q.done(item), then the new item will be moved from dirty to queue. WorkQueue is a dedup-queue. so q.done() doesn't block other go routine to q.Get() other item.
I guess the problem that cause low cpu utilization, low throughput and blocking items being dequeued is RateLimitingInterface. Could you mind to test out using WorkQueue directory without wrapping it with RateLimitingInterface? Thank you.

Although from the source code it looks like it - I don't think the way you describe it is the way it actually works. We are missing something that would explain exactly why it doesn't work in parallel.
The things I know for sure are - in reality it did not ran the important code segments in parallel, profiling with pprof showed most of time the code is stuck on mutex, and increasing the rate limit even to absurd numbers (100,000) did not change this behavior.
In any case - in my opinion the solution we came up with is a better design overall and it actually works for us in large scale production and after extensive tests. I don't have the capacity nor the resources to preform more tests at large scale just in order to understand the bug here, especially as we have a working version . However, If you can try to create a version without rate limit and test it yourself I think it would be beneficial

Adding a google doc I've made for illustrating the issue and our solution
https://docs.google.com/document/u/0/d/14x76jK7naRjxeFCkqUe7MgoDzZjYETixXXa828o0kSM/mobilebasic

I wanted to share that we've been running a build where we replaced the RateLimitingInterface with the default workqueue. We've seen a significant reduction in reconciliation delay, even without increasing the client QPS as suggested in this PR. Occasionally, during peak hours, we still experience up to a 5-minute delay before the process starts, but this might not be entirely due to the operator reconciliation delay.
We haven't noticed any negative impacts from removing the rate limiter. This makes me wonder why the spark-operator uses it. What specific scenario is it designed to protect against?

I wanted to share that we've been running a build where we replaced the RateLimitingInterface with the default workqueue. We've seen a significant reduction in reconciliation delay, even without increasing the client QPS as suggested in this PR. Occasionally, during peak hours, we still experience up to a 5-minute delay before the process starts, but this might not be entirely due to the operator reconciliation delay.
We haven't noticed any negative impacts from removing the rate limiter. This makes me wonder why the spark-operator uses it. What specific scenario is it designed to protect against?

ratelimit is necessary to control the pressure of apiserver and limit the repeated updates of the spark application

We are doing load test on spark operator and we also run into this issue. The test is run on an EKS cluster with 50 m5.4xlarge EC2 instance. We keep increasing the input traffic load by creating many spark application from client, we noticed spark job submitted from operator can keep up with the speed of the input rate at the begining, but once the input rate goes between 250 or 300 jobs per minute the spark-submit rate become stalled and cannot further increase. The start job latency once increased to more than 800 seconds. We are running operator with 30 worker and tried increase further but it does not really help. Looking at the metrics, we noticed the same behavior that CPU utilization of the operator is always below 10 vcores, but at the same time, the workqueue depth keep increasing to even more than 1000, where we have to stop the test.
We suspect we are mostly hit by not able to increase qps and burst as we see lots of such errors in operator log
I0926 11:50:49.403544      10 request.go:682] Waited for 39.99761059s due to client-side throttling, not priority and fairness, request: PUT:https://<IP_ADDRESS>:443/apis/sparkoperator.k8s.io/v1beta2/namespaces/spark-job4/sparkapplications/spark-pi-e6a3693e-6512-4e30-ad7e-69b4f33bdb66/status
I0926 11:50:59.403555      10 request.go:682] Waited for 39.997706246s due to client-side throttling, not priority and fairness, request: PUT:https://<IP_ADDRESS>:443/apis/sparkoperator.k8s.io/v1beta2/namespaces/spark-job3/sparkapplications/spark-pi-3fa65941-2ca1-44d4-8898-d1071ee0a2ee/status

But we may also be impacted by the single workqueue mutex given the high traffic.
It will be really appreciated if this issue can be fixed in further spark operator versions. Thanks.

For all the followers if this MR, per the massive change in spark operator design, I've waited with this MR to see if it is still relevant after the changes done.
I haven't had time to test V2 at scale yet, nor to accommodate my changes to V2.
I would be happy to know if anyone already tested V2 at scale and have some conclusions

My company actually went with a different implementation (customizing the workqueue ratelimiter) to handle high load clusters
The change from manually creating the controller (historical implemtation) to using controller-runtime (V2 changes) make it still relevant
I've actually open a PR to backport it here -> https://github.com/kubeflow/spark-operator/pull/2186
To add context we implemented this to fix a customer issue because their airflow would submit 25K SparkApplication CR at the same time
Obviously this is not a silver bullet, you also need to increase cpu/memory and the number of controller threads

For all the followers if this MR, per the massive change in spark operator design, I've waited with this MR to see if it is still relevant after the changes done.
I haven't had time to test V2 at scale yet, nor to accommodate my changes to V2. I would be happy to know if anyone already tested V2 at scale and have some conclusions

I actually tested V2 operator on the cluster with same size (50 ec2 m5.4xlarge) today see similar behavior as V1. The job start latency is 2-4 seconds when the input rate is around 50-100 jobs per minutes. After increasing it to 150, start latency goes to 12s, which is stills table. Further increased to 200 job per minutes causes the start latency increased continuously, at one time it reached 150s. It is also interesting that after the operator run with 200 job per min for some time (maybe after 30 minutes), the start latency and running job number all dropped to 0 (and at same time SUBMITTED/COMPLETED/SUCCEEDING job number all keep increasing), if I randomly sample job from operator log, the latency still increases to more than 4 minutes.

@xyzyng from what @ImpSy wrote it sounds like the main issue with V2 is just some tweaks with the controller runtime configurations (which make sense, since it used for many high volume operators)
@vara-bonthu - referring you to the discussion here, I think we just need to provide in general more options to configure the controller runtime, and that my PR is irrelevant

@bnetzi , @vara-bonthu , regarding the point 'referring you to the discussion here, I think we just need to provide in general more options to configure the controller runtime, and that my PR is irrelevant',

Does it mean ' one queue per app and one goroutine per app' is not a solution for the performance issue faced? Is https://github.com/kubeflow/spark-operator/pull/2186 solution for the same
Do we see performance opportunity improvement with the approach that we have tried? - https://github.com/kubeflow/spark-operator/issues/1574#issuecomment-1699668815

Summary of changes :
     - port spark-submit to golang
     - this removes JVM invocation, hence, performance-wise faster
     - no dependency on apache spark(as the frequency and quantity of changes to driver pod going to be minimal in future releases of apache spark)

@bnetzi , @vara-bonthu , regarding the point 'referring you to the discussion here, I think we just need to provide in general more options to configure the controller runtime, and that my PR is irrelevant',

Does it mean ' one queue per app and one goroutine per app' is not a solution for the performance issue faced? Is FEATURE: add cli argument to modify controller workqueue ratelimiter #2186 solution for the same
Do we see performance opportunity improvement with the approach that we have tried? - Spark Driver Pod Getting Stuck in init state due to no driver configmap found #1574 (comment)

Summary of changes :
     - port spark-submit to golang
     - this removes JVM invocation, hence, performance-wise faster
     - no dependency on apache spark(as the frequency and quantity of changes to driver pod going to be minimal in future releases of apache spark)

Thanks @gangahiremath ! Please add your feedback to this issue https://github.com/kubeflow/spark-operator/issues/2193

Hey @bnetzi -

we at Salesforce are running performance tests for Spark Operator using your PR branch.. From the recent conversation in this thread, it seems like V2 operator (without this PR) still suffers from performance issues as mentioned by @xyzyng . Can you please let us know the following:

Do you intend to pursue this MR and do you still see the one mutex per queue change as the major reason for low performance in V2?

Can you update your PR branch with the latest (or V2) changes and we can try to run performance tests on that branch. That’ll help us test your change+V2 for performance.

Do you or any of the other folks on this thread, suspect JVM spin-up times as a possible reason for spark job submission latency?

Thank you.

@bnetzi , @vara-bonthu , regarding the point 'referring you to the discussion here, I think we just need to provide in general more options to configure the controller runtime, and that my PR is irrelevant',

Does it mean ' one queue per app and one goroutine per app' is not a solution for the performance issue faced? Is FEATURE: add cli argument to modify controller workqueue ratelimiter #2186 solution for the same
Do we see performance opportunity improvement with the approach that we have tried? - Spark Driver Pod Getting Stuck in init state due to no driver configmap found #1574 (comment)

Summary of changes :
     - port spark-submit to golang
     - this removes JVM invocation, hence, performance-wise faster
     - no dependency on apache spark(as the frequency and quantity of changes to driver pod going to be minimal in future releases of apache spark)

I'm not sure that MR 2186 is the solution here, as there are also qps and burst configurations for other k8s API users (such as CRD Listers etc.)
Unfortunately I don't have the time nor resources to test V2 at scale and detect the performance issues there. I would probably be able to get to it only on mid November.
Would be great if someone else can test it.
As for the JVM overhead. Although it has some performance impact, moving to golang submit would require an operator version for every change in spark submit and would probably tend to break / have missing functions more often on new spark versions.
I believe we can get to a good enough solution without adding it.

Hey @bnetzi -

we at Salesforce are running performance tests for Spark Operator using your PR branch.. From the recent conversation in this thread, it seems like V2 operator (without this PR) still suffers from performance issues as mentioned by @xyzyng . Can you please let us know the following:

Do you intend to pursue this MR and do you still see the one mutex per queue change as the major reason for low performance in V2?
Can you update your PR branch with the latest (or V2) changes and we can try to run performance tests on that branch. That’ll help us test your change+V2 for performance.
Do you or any of the other folks on this thread, suspect JVM spin-up times as a possible reason for spark job submission latency?

Thank you.

I can't promise when, but if someone else wouldn't in the meantime - I will make sure V2 won't have scale performance issues. It might take a while but our platform is heavily based on spark operator and we do no intend to be stuck with V1.
The thing is in V2 the whole code was changed and many infra issues were fixed by moving to be based on controller runtime.
So the way I see it -  work queue per app might not longer be the solution. As I suggested in other comments in this MR, tweaking the k8s config (qps + burst) for the controller + the work queue might be a simpler more effective way here. But only testing would find out.

Another solution could be to move the submission phase to a dedicated pod That way app reporting will not have side effect on the most important part of the operator (launching spark applications)

I think in the first place the reason we have a queue is to make sure all actions on the pod are preformed by order. If you would separate it you might change this behavior.
For example - what would happen if while the creation pod is trying to create the pod, there is an API request to delete the app and app is deleted before pod creation? Probably what would happen as the app is the owner ref, the pod would be created but the k8s garbage collector will terminate it eventually, but when? It might cause very weird bugs.
But in general I do think the spark operator pod is doing way too much work, especially executors related stuff and should be separated to different containers at least

I think in the first place the reason we have a queue is to make sure all actions on the pod are preformed by order. If you would separate it you might change this behavior.
I only made this proposition because that's the implementation that the apache/spark-kubernetes-operator decided to make
But I feel like 2 components running the reconciliation on the resources (Application CR in that case) would not break the order processing since the Submitted State would only be set after a successful submission by the submission worker and the controller would pick the lifecycle of the app afterwards

@bnetzi  - Just wanted to share some feedback and our findings from testing on this PR. We (Salesforce) applied your PR and tested your changes at scale within our clusters. We compared your version, which  is essentially a "queueless" version (since there's only one job in each queue) against the version @gangahiremath had authored, which skips spark-submit thus avoiding all of the JVM latency. From our testing we found @gangahiremath version outperform significantly than the queueless version as well as the current master branch.
Also, we found that this PR runs into OOM issues as soon as the cluster is stressed. Specifically, we tested for one job submission per second for 15 minutes, i.e. 900 jobs in total and the SO with this PR experienced OOM. I would guess that arises from the fact that all the controller threads are just spark-submitting the jobs as soon as they are enqueued in their respective queues and cause OOM.
Other than that, your approach is correct in the sense that having a single queue does affect SO performance whenever the rate of enqueueing becomes greater than the rate of dequeueing within the queue causing the queue length to grow. The performance is further affected because of the queue being guarded by a mutex. One approach could also be to use a concurrent lock-free single queue, which wouldn't require having a mutex. This PR does outperform the vanilla version of spark operator under light/normal load.
Hope this is helpful. Thanks.

@c-h-afzal - thanks for the info!
This is really interesting, as we (Mobileye) are running this in production for months now in high load without any OOM issues, but it does require us to use 31 CPUs pod.
Also - we did notice that our k8s itself has issues of handling too many pods in a single namespace, so we stop submitting at ~3500 pods (drivers+ executors)
We also tried @gangahiremath version and under heavy load it didn't performed well for us.
So my Guess is there is a difference in the way we are testing that creates the difference is results.
The difference might be:

the qps config we used
the controller threads (which in our code defines the amount of queues maps) which is also very high in our case
The size of k8s cluster we are using
The amount of overall spark apps running concurrently

I think it can be
P.s. I would not say though that this PR is queue less, as each app operations are in a queue, and it prevents concurrent actions on the same app, which is the point of the current queue.
Anyway - we started to work on a version that is compatible with spark operator V2, so more updates on that soon

Gotcha! Thanks for response @bnetzi
If it helps, we ran perf tests with Spark Operator running on a 4 vpcu and 16 GB memory node. We made all other variables e.g. controller threads, qps, the k8s cluster-size, number of spark jobs - constant across all the tests, so they shouldn’t have an impact on the results. Also, increasing the controller threads can only be as useful as the number of vCPUs/CPUs on the node. A number higher than available cores may not necessarily help.
Given Ganga’s fix is working so well for us, we have gone ahead and proposed to make the spark app launch mechanism pluggable in spark operator. The default will always be spark-submit. The change would be a no-op for folks continuing to use spark-submit. The plugin mechanism would allow us to insert Ganga’s fix and circumvent spark-submit JVM penalty. If we find enough interest from other folks, we’ll also open-source Ganga’s fix in a separate repo.
And acknowledge your comment about queuless - the intent was to describe queless from the perspective of multiple spark apps but ops for a single app still need to queue.
Hoping to see your changes and future improvements in spark operator soon :)

@c-h-afzal I think the specs are the main diff.
We ran our tests with 32 cores and ~100GB memory and got to 100% CPU usage at peaks (although memory was only 50% at peaks).
As for controller threads - in my code the controller threads are controlling the amount queue maps, so it is not coupled with amount of vCPUs, it only has more impact on memory. We have used 10,000 in our tests just to avoid map mutexes as much as possible.
I think it is great to have the submit with go as an opt-in option. I'll comment in the issue you opened.


Parameters of custom gates not recognized

I created a custom gate cu1mol(theta) a, b in a file containing definitions of quantum gates.
`
OPENQASM 2.0;
include "qelib1.inc";
gate cxmol a, b {
h b;
cz a, b;
h b;
}
gate cu1mol(theta) a, b {
u1(theta/2) a;
cxmol a, b;
u1(-theta/2) b;
cxmol a, b;
u1(theta/2) b;
}
If I try to define in a qasm file a gate likecu1mol(pi/2) q[1], q[0];nuqasm2 does not recognizepi/2, thus having on the output file u1(theta/2) q[1];instead ofu1(pi/2) q[1];`

@giovanniamedeocirillo thanks, looking at this.

Yes, you are right. The source and output of nuqasm2 -a -c -d -o my_cu1mol.out my_cu1mol.qasm is attched.
my_cu1mol.zip

@giovanniamedeocirillo ... try the issue1_parmexpand bug fix branch and tell me if that works for you.

@jwoehr now it seems ok for cu1mol(pi/3) it returns
u1(1.0471975511965976/2) q[1]; h q[0]; cz q[1],q[0]; h q[0]; u1(-1.0471975511965976/2) q[0]; h q[0]; cz q[1],q[0]; h q[0]; u1(1.0471975511965976/2) q[0];
So now the parameter is properly interpreted, thank you so much!
I don't know if it is hard, but if it was also possible to evalutate the division by 2 (more in general the expression in which the parameter is employed) it would be the best, IMHO.
In execution something like ******bound_param theta param 1.0471975511965976 b_list [] ******the_op u1 the_reg_list [Qubit(QuantumRegister(3, 'q'), 1)] the_param_list ['1.0471975511965976/2'] ******the_op cxmol the_reg_list [Qubit(QuantumRegister(3, 'q'), 1), Qubit(QuantumRegister(3, 'q'), 0)] the_param_list None  is printed on screen. Is it OK (maybe for debugging)?

@jwoehr now it seems ok for cu1mol(pi/3) it returns
u1(1.0471975511965976/2) q[1]; h q[0]; cz q[1],q[0]; h q[0]; u1(-1.0471975511965976/2) q[0]; h q[0]; cz q[1],q[0]; h q[0]; u1(1.0471975511965976/2) q[0];
So now the parameter is properly interpreted, thank you so much!

Great, I'll clean up the code and merge the branch.

I don't know if it is hard, but if it was also possible to evalutate the division by 2 (more in general the expression in which the parameter is employed) it would be the best, IMHO.

Yes, that would be better. If you open an issue on that as a feature request, I'll take a look at it.

In execution something like ******bound_param theta param 1.0471975511965976 b_list [] ******the_op u1 the_reg_list [Qubit(QuantumRegister(3, 'q'), 1)] the_param_list ['1.0471975511965976/2'] ******the_op cxmol the_reg_list [Qubit(QuantumRegister(3, 'q'), 1), Qubit(QuantumRegister(3, 'q'), 0)] the_param_list None  is printed on screen. Is it OK (maybe for debugging)?

Ha! Yes, I forgot and left some debug print() statements in. Thank you!

I see I introduced a bug in my bug fix ... hold on while I fix that :)

All my test cases now work, how are things for you, @giovanniamedeocirillo ?

Default parametric gates as u1(pi/4) and rx(pi) provide an error TypeError: 'NoneType' object is not subscriptable   when called inside custom gates.

@giovanniamedeocirillo , You are correct. I did not explain fully before, it was late ;)

Delete the issue1_parmexpand branch
Check out master again

I did introduce a bug in my patch branch, but it is fixed in the master branch.
Try that and let me know.

@jwoehr it works! Thank you so much!

Thanks for testing.
I will close this issue and cut a new release.


Cannot read the reports in Staging

Select the "Read report" button from anywhere in the program generates a "There was an error processing this request.  Please contact an administrator." message for every report selected.  The report is not displayed.  This started happening after the new version was added to staging on 16 August.

This issue can be closed.  I had not updated the staging database with the latest updates via "db migrate" command.


Update packages

Pull Request Test Coverage Report for Build 11

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 10:
0.0%

Covered Lines:
29

Relevant Lines:
29

💛  - Coveralls

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


New Unifi component does not expose the ap_mac attribute after 0.97.1

Home Assistant release with the issue:

0.97.1
Last working Home Assistant release (if known):
0.96.5
Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io in docker on debian
Component/platform:

Unifi
Description of problem:
From 0.97.0 to 0.97.1 the attribute ap_mac was removed. I use that attribute to do automations based on which Unifi access point the phones and watches of our household are near to, e.g. enabling alarm when my wife and I have been on first or second floor for a period of time in the evening.
Removing the attribute from Unifi effectively ruins that functionality. Please make it either configurable or include the attribute again. Thanks!
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

Traceback (if applicable):

Additional information:

Blame #25787.

Hi!
We can't keep them as part of the default attributes of client trackers. See this issue for details: #25782
We can't allow configuration of extra attributes due to this architectural decision: https://github.com/home-assistant/architecture/blob/master/adr/0003-monitor-condition-and-data-selectors.md
Feel free to suggest another solution that can make fit within these limits

Seriously, that's a bit of a regression IMO.
I can only see two other options then:

don't upgrade past 0.96.5/0.97.0
revert the #25787

Do you see any other options?

I agree with Lars here. Removing ap_mac seriously limits the functionality of the unifi-integration. Being able to know which AP a device is connected to is an easy way to know where a person is in the house. I’ve got a similar situation as Lars where I have automations triggered based on what AP i’m Connected to.

@Kane610 Could ap_mac be restored? Does that really change that frequently?

@andrewsayre we probably could. It was brought along with other more frequently changing attributes.

Yes please give the ap_mac back, i’m also using it in automations.

Ive had a talk with balloob about ways to solve this. He will add gui support for config entry options, after that I can implement enabling this.

Great to hear, @Kane610
Do you have any ETA?

I am also using this functionality for presence detection. @Kane610 would it be possible to re-enable ap_mac while that solution is implemented? This breaks most of my automations. Do people really have so many APs at home that makes this attribute so unstable?

Lets look go forward and make stuff better
@larserhardsen no, but Balloob had a first refactoring related to this up for review just earlier today.

Happy that I found this issue here after noticing that ap_mac was gone and great that it's fixed again in 97.3 that's coming out soon (I hope, it's dark in here!).
It's a key functionality for my presence detection between floors and other than some motion detection there's no other cool way to fix this.


Run KfP v2 pipeline on cluster

Validate that our KfP v2 pipelines run on a new KfP v2 cluster and fix issues we encounter along the way.
### Tasks
- [ ] #418
- [ ] https://github.com/ml6team/fondant/issues/486
- [ ] https://github.com/ml6team/fondant/issues/421
- [ ] https://github.com/ml6team/fondant/issues/505
- [ ] https://github.com/ml6team/fondant/issues/506

Done in #477


"Int16 out of bounds" messages in log from NodeMarkup when saving a game

Which IMT are you using?  Stable
What computer OS are you using?  Windows 10
Describe the bug
When I save a game, I can see a lot of messages like following in log:
[NodeMarkup] Start save map data
[NodeMarkup] Map data saved in 10ms; Size = 5822 bytes
Int16 out of bounds: 44542
   at System.Environment.get_StackTrace()
   at ColossalFramework.IO.DataSerializer.WriteInt16(Int32 value)
   at BuildingManager+Data.Serialize(ColossalFramework.IO.DataSerializer s)
   at ColossalFramework.IO.DataSerializer.Serialize(System.IO.Stream stream, Mode mode, UInt32 version, IDataContainer data)
   at LoadingManager+<SaveSimulationData>c__IteratorC.MoveNext()
   at AsyncTask.Execute()
   at SimulationManager.SimulationStep()
   at SimulationManager.SimulationThread()  [Serialization - Internal]

To Reproduce
Steps to reproduce the behavior:

Load save game https://steamcommunity.com/sharedfiles/filedetails/?id=2779122444 (though I use around 50 mods and a lot of assets)
Save the game and check the output log

Expected behavior
So far I don't see any harm done in the save game itself - but I would like to be sure that it's really harmless. Some times it generates quite a huge amount of these error messages.
Additional information
I did attached output_log and compatibility report.
output_log.txt
CompatibilityReport.txt

Aha, great to know. Then I’m closing this one.


Warning with opcache preloading

I'm running the following configuration :

Apache server running on Alpine
PHP 8.2
Symfony 6.4
SncRedisBundle 4.7.0
opcache preloading set to opcache.preload=/var/www/app/config/preload.php
APP_ENV=PROD
composer install with the --no-dev flag

I'm getting the following message on server startup :
Warning:  Can't preload unlinked class Snc\RedisBundle\DependencyInjection\Configuration\RedisDsn@anonymous: Unknown parent Snc\RedisBundle\DependencyInjection\Configuration\RedisDsn in /var/www/app/vendor/snc/redis-bundle/src/Factory/PhpredisClientFactory.php on line 151

The issue doesn't occurs in dev environnement.
I'm using Redis for the session with Redis Sentinel, it works in production even with the warning.
I don't know if this a symfony or bundle issue.

Hello!
I'm having the same issue with the following configuration:

Nginx server running on Alpine
PHP-FPM 8.2
Symfony 6.2
SncRedisbundle 4.7.0
Opcache preloading set to opcache.preload=/var/www/config/preload.php
APP_ENV=prod
composer install without flags

There error message is the following:
Warning:  Can't preload unlinked class Snc\RedisBundle\DependencyInjection\Configuration\RedisDsn@anonymous: Unknown parent Snc\RedisBundle\DependencyInjection\Configuration\RedisDsn in /var/www/vendor/snc/redis-bundle/src/Factory/PhpredisClientFactory.php on line 151

It's also running inside of Docker. I'm using regular Redis in a different container.

Apparently, the fix would be to add the following line to PhpredisClientFactory.php, before the class declaration.
Similar like in Twig
class_exists(RedisDsn::class);


Tests for identify/people.set

I added in tests for #70.
@poteto: I've been using Ember for a while now, but this is my first contribution ever to any open source project. Please let me know if this is good or if I should do anything differently.

@poteto I'm not exactly sure why the Travis build failed. Can you rerun it?

@homu r+

:pushpin: Commit ded8f72 has been approved by poteto

Thanks @mpeven and @Jordan4jc!

:zap: Test exempted - status


supaclock: 0.02

Adjusted main font size not to overflow on certain times of the day, minor tweaks.

Cool, thanks


fix: fixing CI file name

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@b58cb1b). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #3   +/-   ##
=======================================
  Coverage        ?   34.09%
=======================================
  Files           ?       12
  Lines           ?       44
  Branches        ?        0
=======================================
  Hits            ?       15
  Misses          ?       29
  Partials        ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b58cb1b...652784c. Read the comment docs.


setting registry.consul.tagprefix doesn't seem to work

I'm trying to use the registry.consul.tagprefix config value to change the default urlprefix- to fabio-. I have services in consul passing health checks and registering service tags of the form fabio-... but when I run a Fabio instance with the tag prefix set to fabio- it doesn't find any services.

@far-blue does it work if you use the default? I'm successfully using this feature in all my deployments, even with the latest stable release (v1.6.0). Are there any logs in fabio related to the tags or the config?

no, no logs. Everything appears to be working correctly but fabio simply doesn't "see" any services to route.

@far-blue we're using this feature in production as well (overriding registry.consul.tagprefix for some fabios).  Can you help come up with a minimal environment that demonstrates this issue?  can you try setting the log level to TRACE ?  That might shed some light.


prefer HEAD to master in update tasks

When we fetch book and manpage data from other repositories, we hard-code master as the branch name to use. Let's switch instead to the default-branch HEAD, which will do the right thing if those repositories ever change their default branch (e.g., as discussed in #1501).
This isn't quite just s/master/HEAD/ because of the way GitHub's API works. See 002990557f0412f8b9ea30e87711d3b5e059bcd0 for details. On the plus side, that change helped me identify and fix a possible race condition in the update code (fixed in cee3bf1198eef10634256619fabaf288ed075951).

Hi @peff you might want to amend the commit message for commit 0029905, it contains a small spelling error:

Note that we can't ust swap out "HEAD" here.

I think you meant Note that we can't just swap out "HEAD' here.

Thanks, fixed! I had run this locally when I wrote it yesterday, but I just double-checked by running the tasks manually on the production site, too. Everything seems good.

I just double-checked by running the tasks manually on the production site, too. Everything seems good.

...and this wasn't quite true. It did build everything correctly, but it broke the commit-sha caching in a way that would blow up next time it tried to run. I pushed the fix straight to master (f945dfa1214567a293db25d3fd590c2c665baaa4) and am re-running the book task manually.


LW-11271 Utils for K6 load tests

Context
TBD
Proposed Solution
TBD
Important Changes Introduced
TBD

GHA changes are not propagated in the github actions page.
I am trying to close and reopen this PR to work around it


Keep getting sporadic errors (may be related to packer.nvim?)

Every now and then I get the error below. I've tried to find a way to reproduce it but it's very difficult. I think it only occurs after I run source <afile> | PackerCompile after saving my init.lua, and it appears to trigger randomly every now and then when I type in insert mode and a snippet appears as the hovered selection in the popup menu.
Without a way to reproduce it's not much help but I still wanted to post this issue to hear if there's an easy fix or if anyone else is having the same issue.
Error executing vim.schedule lua callback: ...e/pack/packer/start/cmp_luasnip/lua/cmp_luasnip/init.lua:106: attempt to index a nil value
stack traceback:
	...e/pack/packer/start/cmp_luasnip/lua/cmp_luasnip/init.lua:106: in function 'resolve'
	.../nvim/site/pack/packer/start/nvim-cmp/lua/cmp/source.lua:348: in function 'resolve'
	...e/nvim/site/pack/packer/start/nvim-cmp/lua/cmp/entry.lua:454: in function 'resolve'
	...re/nvim/site/pack/packer/start/nvim-cmp/lua/cmp/view.lua:225: in function 'fn'
	.../site/pack/packer/start/nvim-cmp/lua/cmp/utils/async.lua:41: in function <.../site/pack/packer/start/nvim-cmp/lua/cmp/utils/async.lua:39>

I don't know much about packer's internals, does it change anything about the loaded plugins when PackerCompile is called?
Your error shouldn't occur during normal usage I think.

Try find for get_documentation function and add:
local function get_documentation(snip, data)
	if snip == nil then
		return "undocumented"
	end
	local header = (snip.name or "") .. " _ `[" .. data.filetype .. "]`\n"
        ...

@melkster Could you make sure you're adding snippets via ls.add_snippets(ft, snippets)?
I didn't consider that not everybody is using that already (it is deprecated, but not yet unsupported).

What is deprecated? get_documentation?

@wuilliam321 adding snippets by directly accessing the snippet table

Could you make sure you're adding snippets via ls.add_snippets(ft, snippets)?

This might actually be it! I wasn't using ls.add_snippets() and after changing to it I haven't experienced the issue!

Great! was the same for me!


Module Federation Plugin for Vite

Clear and concise description of the problem
With the advent of @module-federation/esbuild, can we expect an official plugin to work with Vite? I say this because Vite has a large adoption. This would be wonderful for the module federation ecosystem.
Suggested solution
Today, for Vite, we have some options, but none integrated with the core of module federation, as the @module-federation/vite plugin still depends on @softarc/native-federation-esbuild.
Alternative
No response
Additional context
No response
Validations

[X] Read the Contributing Guidelines.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Happily this is something we can expect to see as an "officially" supported.  @yyx990803 has put this on the roadmap for Rolldown which is going to be the basis for the next version of Vite.

@ScriptedAlchemy @zmzlois would love to dig into this.

how should I handle the first parameter of init and the shared of init

https://github.com/MadaraUchiha-314/rollup-plugin-module-federation/blob/main/packages/rollup-plugin-module-federation/src/index.ts#L585
This is how you use the shareScope that's passed into init ^

how should I handle the first parameter of init and the shared of init

https://github.com/MadaraUchiha-314/rollup-plugin-module-federation/blob/main/packages/rollup-plugin-module-federation/src/index.ts#L585
This is how you use the shareScope that's passed into init ^

Thanks, this is very useful
I already support dev and build modes of vite, and have all the main features, but I haven't done other enhancements of 2.0, such as manifest.json and runtime plugins, etc. Can I PR here and then we can improve it together?
https://github.com/module-federation/vite/issues/4

@zhangHongEn would love to check your PR and work on it together too

@zhangHongEn would love to check your PR and work on it together too

pr is here, it can speed up some progress

https://github.com/module-federation/vite


It is not full screen when giving notification

I downloaded stretchly-0.19.1-win.zip and run from .exe.
Everything is fine. But it is not going fullscreen.
I have dual monitor on windows. But without monitor, also it is not going fullscreen.
I am attaching screenshot

This is settings

This is specs

I looked for issues but did not see same issue. So please forgive if there is similar and solved issues.

Hi, this is a known issue in electron: https://github.com/electron/electron/issues/16907
I would recommend not using "breaks are fullscreen". We will have to wait for electron fix ;/


due date issue is solved

in this if user didn't specify any due date to a task the error message prompts as "Please specify a due date!!!" .

I am working on the issue now i will get to you shortly.

I would approve the pr request but "!!!!" are bugging me......... it could just be me but great work.

Thanks for your valuable contribution.
Don't forget to star the repo.

Code peak'23