Add CRD Selectable Field Support

What this PR Does
Add support for specifying arbitrary selectable fields in a CUE kind, using new selectableFields string array in versions[version]. Generated CRDs will now include the provided selectable fields.
Added support for exposing custom selectable fields from resource.Schema, for future use (primarily API servers and the like) which will need this information. Codegen automatically adds these to the generated resource.Schema when applicable.
Testing
Unfortunately, I was unable to get k3d to work with the kubernetes CustomResourceFieldSelectors feature gate, so this was tested on a kind cluster, and k3d support is not built into the local setup.
Setup files
kind:
package kinds

foo: {
	kind: "Foo"
	group: "sftest"
	apiResource: {
		scope: "Namespaced"
	}
	current: "v1"
	versions: {
		"v1": {
			version: "v1"
			schema: {
				spec: {
					firstField: string
					secondField: int64
				}
			}
			selectableFields: ["spec.firstField"]
		}
	}
}

Cluster info for ctlptl:
apiVersion: ctlptl.dev/v1alpha1
kind: Cluster
product: kind
registry: ctlptl-registry
kindV1Alpha4Cluster:
  nodes:
    - role: control-plane
      image: kindest/node:v1.30.0
  featureGates:
    CustomResourceFieldSelectors: true

example.yaml:
apiVersion: sftest.ext.grafana.com/v1
kind: Foo
metadata:
  name: example-1
  namespace: default
spec:
  firstField: foo
  secondField: 10
---
apiVersion: sftest.ext.grafana.com/v1
kind: Foo
metadata:
  name: example-2
  namespace: default
spec:
  firstField: bar
  secondField: 20

Test output
%> ctlptl apply -f local/ctlptl.yaml
Creating registry "ctlptl-registry"...
No kind clusters found.
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.30.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜
 ✓ Starting control-plane 🕹
 ✓ Installing CNI 🔌
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
   Connecting kind to registry ctlptl-registry
Switched to context "kind-kind".
 🔌 Connected cluster kind-kind to registry ctlptl-registry at localhost:62808
 👐 Push images to the cluster like 'docker push localhost:62808/alpine'
cluster.ctlptl.dev/kind-kind created
%> kubectl apply -f definitions/foo.sftest.ext.grafana.com.yaml
customresourcedefinition.apiextensions.k8s.io/foos.sftest.ext.grafana.com created
%> kubectl apply -f example.yaml
foo.sftest.ext.grafana.com/example-1 created
foo.sftest.ext.grafana.com/example-2 created
%> kubectl get foos
NAME        AGE
example-1   18s
example-2   18s
%> kubectl get foos --field-selector spec.firstField=bar
NAME        AGE
example-2   25s

Resolves https://github.com/grafana/grafana-app-sdk/issues/355

Works for me using kind
jq '.spec.versions[0].selectableFields' definitions/relation.gamma.ext.grafana.com.json
[
  {
    "jsonPath": ".spec.relationType"
  },
  {
    "jsonPath": ".spec.fromRef.apiVersion"
  },
  {
    "jsonPath": ".spec.fromRef.kind"
  },
  {
    "jsonPath": ".spec.fromRef.name"
  },
  {
    "jsonPath": ".spec.toRef.apiVersion"
  },
  {
    "jsonPath": ".spec.toRef.kind"
  },
  {
    "jsonPath": ".spec.toRef.name"
  }
]

kubectl get relations --field-selector spec.relation=dependency --field-selector spec.fromRef.name=service-a

NAME                             AGE
service-a-depends-on-service-b   3m5s

@radiohead
 === RUN   TestCustomCacheInformer_Run_CacheState
    informer_customcache_test.go:231:
        	Error Trace:	/home/runner/work/grafana-app-sdk/grafana-app-sdk/operator/informer_customcache_test.go:231
        	Error:      	Should be true
        	Test:       	TestCustomCacheInformer_Run_CacheState
        	Messages:   	timed out waiting for event

Looks like it, the timed out waiting for event is due to the channel write being blocked the the mutex lock stopping the listener from starting the channel read to unblock it. Let me re-run the tests and see if it passes, as it's a race condition that seems to depend on the resourcing of the test runner (as it comes down to order of execution in goroutines prior to the fix).


New package: Dubins3D v0.1.0

Registering package: Dubins3D
Repository: https://github.com/comrob/Dubins3D.jl
Created by: @petvana
Version: v0.1.0
Commit: 350ac86a2ae83a740e07de30b8e4d3feb0e42af8
Git reference: master

I will fix the version

[noblock] I would like to use Dubins3D instead of Dubins3d because 3D is a well-known abbreviation.


⚠ Bofrost.lu has degraded performance

In 6fa2c30, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 801 ms

Resolved: Bofrost.lu performance has improved in 67e2393 after 6 minutes.


Check the Mac while making the acrhive

While making the Archive the Macintosh can prompt you for a password in order to codesign something, so if its taking a really long time, check the Mac.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb327a64-010d-dba8-5ebc-546892d739af
Version Independent ID: 48a24297-7b8f-b1b2-7b25-44f40fe33eac
Content: Publishing Xamarin.iOS apps to the App Store - Xamarin
Content Source: docs/ios/deploy-test/app-distribution/app-store-distribution/publishing-to-the-app-store.md
Product: xamarin
Technology: xamarin-ios
GitHub Login: @davidortinau
Microsoft Alias: daortin

Thanks for submitting an issue and we apologize for the delayed response. The requested updates haven't been made since the creation of the issue, and the timeline for resolution may vary based on resourcing, so the issue is being moved to an internal repo to better track it.


Write a Python program to print multiplication table

Description
Write a Python program to print multiplication table
Input  : 2

2 x 1 = 2
2 x 2 = 4
2 x 3 = 6
2 x 4 = 8
2 x 5 = 10
2 x 6 = 12
2 x 7 = 14
2 x 8 = 16
2 x 9 = 18
2 x 10 = 20

How to contribute

Save the solution in print-multiplication-table.py file
Add print-multiplication-table.py file in print-multiplication-table folder

!assign


Outdated files in dev-1.18-ko.7

This is a Bug Report
Problem
Outdated files in the upstream/dev-1.18-ko.7 branch.
59 files to be modified

[ ] content/en/community/_index.html 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/_index.md 1 file changed, 2 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/cluster-administration/cloud-providers.md 1 file changed, 2 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/configuration/configmap.md 1 file changed, 6 insertions(+), 6 deletions(-)
[ ] content/en/docs/concepts/configuration/overview.md 1 file changed, 2 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/configuration/resource-bin-packing.md 1 file changed, 9 insertions(+), 9 deletions(-)
[ ] content/en/docs/concepts/containers/images.md 1 file changed, 93 insertions(+), 166 deletions(-)
[ ] content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/extend-kubernetes/extend-cluster.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/overview/kubernetes-api.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/overview/what-is-kubernetes.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/overview/working-with-objects/namespaces.md 1 file changed, 7 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/overview/working-with-objects/object-management.md 1 file changed, 6 deletions(-)
[ ] content/en/docs/concepts/scheduling-eviction/taint-and-toleration.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/security/overview.md 1 file changed, 75 insertions(+), 86 deletions(-)
[ ] content/en/docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases.md 1 file changed, 30 insertions(+), 29 deletions(-)
[ ] content/en/docs/concepts/storage/volume-snapshot-classes.md 1 file changed, 25 insertions(+), 12 deletions(-)
[ ] content/en/docs/concepts/storage/volumes.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/cron-jobs.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/concepts/workloads/controllers/deployment.md 1 file changed, 12 insertions(+), 2 deletions(-)
[ ] content/en/docs/concepts/workloads/controllers/garbage-collection.md 1 file changed, 1 insertion(+), 9 deletions(-)
[ ] content/en/docs/concepts/workloads/controllers/ttlafterfinished.md 1 file changed, 4 insertions(+), 4 deletions(-)
[ ] content/en/docs/concepts/workloads/pods/pod-lifecycle.md 1 file changed, 2 insertions(+), 3 deletions(-)
[ ] content/en/docs/contribute/advanced.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/contribute/new-content/open-a-pr.md 1 file changed, 19 insertions(+), 3 deletions(-)
[ ] content/en/docs/home/_index.md 1 file changed, 2 insertions(+)
[ ] content/en/docs/reference/glossary/cronjob.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/reference/glossary/job.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/reference/kubectl/cheatsheet.md 1 file changed, 3 insertions(+), 3 deletions(-)
[ ] content/en/docs/reference/kubectl/overview.md 1 file changed, 43 insertions(+), 32 deletions(-)
[ ] content/en/docs/setup/best-practices/cluster-large.md 1 file changed, 3 deletions(-)
[ ] content/en/docs/setup/best-practices/node-conformance.md 1 file changed, 1 deletion(-)
[ ] content/en/docs/setup/production-environment/container-runtimes.md 1 file changed, 15 insertions(+), 6 deletions(-)
[ ] content/en/docs/tasks/_index.md 1 file changed, 65 deletions(-)
[ ] content/en/docs/tasks/access-application-cluster/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/administer-cluster/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tasks/configure-pod-container/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/debug-application-cluster/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/extend-kubectl/kubectl-plugins.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tasks/inject-data-application/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/manage-daemon/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/manage-daemon/rollback-daemon-set.md 1 file changed, 33 insertions(+), 44 deletions(-)
[ ] content/en/docs/tasks/manage-gpus/scheduling-gpus.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/manage-hugepages/scheduling-hugepages.md 1 file changed, 2 insertions(+), 12 deletions(-)
[ ] content/en/docs/tasks/manage-kubernetes-objects/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/network/_index.md 1 file changed, 2 insertions(+), 1 deletion(-)
[ ] content/en/docs/tasks/run-application/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/run-application/horizontal-pod-autoscale.md 1 file changed, 2 insertions(+), 1 deletion(-)
[ ] content/en/docs/tasks/tls/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/tools/_index.md 1 file changed, 1 insertion(+)
[ ] content/en/docs/tasks/tools/install-kubectl.md 1 file changed, 121 insertions(+), 120 deletions(-)
[ ] content/en/docs/tutorials/_index.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tutorials/configuration/configure-redis-using-configmap.md 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tutorials/kubernetes-basics/scale/scale-intro.html 1 file changed, 1 insertion(+), 1 deletion(-)
[ ] content/en/docs/tutorials/stateful-application/basic-stateful-set.md 1 file changed, 407 insertions(+), 254 deletions(-)
[ ] content/en/docs/tutorials/stateful-application/mysql-wordpress-persistent-volume.md 1 file changed, 3 insertions(+), 3 deletions(-)
[ ] content/en/partners/_index.html 1 file changed, 67 insertions(+), 56 deletions(-)

1 files to be renamed

[ ] R097 content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md -> content/en/docs/concepts/workloads/controllers/job.md

0 files to be deleted
Proposed Solution
Use git diff to check what is changed in the upstream. And apply the upstream changes manually
to the content/ko of upstream/dev-1.18-ko.7 branch.
For example:
# checkout `upstream/dev-1.18-ko.7`
...
# check what is updated in the upstream
git diff upstream/dev-1.18-ko.6 upstream/dev-1.18-ko.7 -- content/en/community/_index.html
# apply changes to content/ko
vi content/ko/community/_index.html
...
# commit and push
...
# make PR to `upstream/dev-1.18-ko.7`

Pages to Update
/language ko

/assign @pjhwa

content/ko/community/_index.html
content/ko/docs/concepts/_index.md
content/ko/docs/concepts/cluster-administration/cloud-providers.md
content/ko/docs/concepts/configuration/configmap.md
content/ko/docs/concepts/configuration/overview.md
content/ko/docs/concepts/configuration/resource-bin-packing.md
content/ko/docs/concepts/containers/images.md
content/ko/docs/concepts/extend-kubernetes/api-extension/custom-resources.md
content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md
content/ko/docs/concepts/extend-kubernetes/extend-cluster.md
content/ko/docs/concepts/overview/kubernetes-api.md
content/ko/docs/concepts/overview/what-is-kubernetes.md
content/ko/docs/concepts/overview/working-with-objects/namespaces.md
content/ko/docs/concepts/overview/working-with-objects/object-management.md
content/ko/docs/concepts/scheduling-eviction/taint-and-toleration.md
content/ko/docs/concepts/security/overview.md
content/ko/docs/concepts/services-networking/add-entries-to-pod-etc-hosts-with-host-aliases.md
content/ko/docs/tasks/configure-pod-container/assign-memory-resource.md

/close


Website

Added the website for the app.
Do let me know if any changes are required.
Issue solved : #7
Attached below is a pic of the home page .
All the code for the website can be found in the Website directory inside q-stream folder.

Thanks for your contribution! ❄
Enjoy the Winter ☃

Thanks for your contribution! ❄
Enjoy the Winter ☃

Thank you very much for this opportunity.
Due to this project I began learning React again.

Thank you very much for this opportunity.
Due to this project I began learning React again.

Glad to hear!!
This thing made my day!!

Glad to hear!!
This thing made my day!!


[Proposal] Incrase language coverage

Is your feature request related to a problem? Please describe.
Not of a problem, more of an enhancement
Describe the solution you'd like
Add other languages from other repos, assuming that they use the Unicode codepoint + n-grams model.
Describe alternatives you've considered

https://github.com/wooorm/franc/tree/master/packages/franc-all (JS, 401 languages)

Codepoints https://github.com/wooorm/franc/blob/master/packages/franc-all/expressions.js
Ngrams https://github.com/wooorm/franc/blob/master/packages/franc-all/data.json

https://github.com/cloudmark/language-detect (Python, 271 languages)
https://github.com/cloudmark/language-detect/tree/master/data/udhr
https://github.com/kapsteur/franco (Golang, 175 languages)

Codepoints https://github.com/kapsteur/franco/blob/master/expression_data.go
Ngrams https://github.com/kapsteur/franco/blob/master/script_data.go

https://github.com/patrickschur/language-detection (PHP, 110 languages)

https://github.com/patrickschur/language-detection/tree/master/resources

https://github.com/richtr/guessLanguage.js (JS, 100 languages)

Codepoints https://github.com/richtr/guessLanguage.js/blob/master/lib/guessLanguage.js
Ngrams https://github.com/richtr/guessLanguage.js/blob/master/lib/_languageData.js

https://github.com/saffsd/langid.py (Python, 97 languages)

Alternate https://github.com/saffsd/langid.c
Alternate https://github.com/saffsd/langid.js
Alternate https://github.com/carrotsearch/langid-java

https://github.com/feedbackmine/language_detector (Ruby, 96 languages)

https://github.com/feedbackmine/language_detector/tree/master/lib/training_data

https://github.com/jonathansp/guess-language (Golang, 94 languages)

Codepoints

https://github.com/jonathansp/guess-language/blob/master/data/blocks.go
https://github.com/jonathansp/guess-language/blob/master/data/languages.go

Ngrams

https://github.com/jonathansp/guess-language/blob/master/data/trigrams.go

https://github.com/abadojack/whatlanggo (Golang, 84 languages)

Codepoints

https://github.com/abadojack/whatlanggo/blob/master/script.go
https://github.com/abadojack/whatlanggo/blob/master/detect.go

Ngrams https://github.com/abadojack/whatlanggo/blob/master/lang.go

https://github.com/chattylabs/language-detector (JS, 73 language)

https://github.com/chattylabs/language-detector/tree/master/data/resources

https://github.com/optimaize/language-detector (Java, 71 languages)
https://github.com/endeveit/guesslanguage (Golang, 67 languages)

https://github.com/endeveit/guesslanguage/tree/master/models

https://github.com/dsc/guess-language (Python, 64 languages)

https://github.com/dsc/guess-language/tree/master/guess_language/trigrams
Co-reference https://github.com/kent37/guess-language

https://github.com/decultured/Python-Language-Detector (Python, 58 languages)

https://github.com/decultured/Python-Language-Detector/tree/master/trigrams

https://github.com/Mimino666/langdetect (Python, 55 languages)

Codepoints

https://github.com/Mimino666/langdetect/blob/master/langdetect/utils/unicode_block.py
https://github.com/Mimino666/langdetect/blob/master/langdetect/utils/messages.properties
https://github.com/Mimino666/langdetect/blob/master/langdetect/utils/ngram.py

Ngrams https://github.com/Mimino666/langdetect/tree/master/langdetect/profiles

https://github.com/pemistahl/lingua (Kotlin, 55 languages)

Codepoints https://github.com/pemistahl/lingua/blob/master/src/main/kotlin/com/github/pemistahl/lingua/internal/Alphabet.kt
Ngrams https://github.com/pemistahl/lingua/tree/master/src/main/resources/language-models

https://github.com/landrok/language-detector (PHP, 54 language)

https://github.com/landrok/language-detector/tree/master/src/LanguageDetector/subsets

https://github.com/shuyo/language-detection (Java, 53 languages)
https://github.com/newmsz/node-language-detection (JS, 53 languages)

Codepoints https://github.com/newmsz/node-language-detection/blob/master/index.js
Ngrams https://github.com/newmsz/node-language-detection/tree/master/profiles

https://github.com/pdonald/language-detection (C#, 53 languages)

https://github.com/pdonald/language-detection/tree/master/LanguageDetection/Profiles

https://github.com/malcolmgreaves/language-detection (Java, 53 languages)
https://github.com/FGRibreau/node-language-detect (JS, 52 languages)

Codepoints https://github.com/FGRibreau/node-language-detect/blob/master/data/unicode_blocks.json
Ngram https://github.com/FGRibreau/node-language-detect/blob/master/data/lang.json

https://github.com/webmil/text-language-detect (PHP, 52 languages)

Codepoints https://github.com/webmil/text-language-detect/blob/master/lib/data/unicode_blocks.dat
Ngram https://github.com/webmil/text-language-detect/blob/master/lib/data/lang.dat

https://github.com/pear/Text_LanguageDetect (PHP, 52 languages)

https://github.com/pear/Text_LanguageDetect/tree/master/data

https://github.com/Imaginatio/langdetect (Java, 50 languages)

https://github.com/Imaginatio/langdetect/tree/master/src/main/resources/profiles

https://github.com/dachev/node-cld (C++, 160 languages)

co-reference https://github.com/jtoy/cld
co-reference https://github.com/mzsanford/cld
co-reference https://github.com/jaukia/cld-js
co-reference https://github.com/vhyza/language_detection
Co-referecne https://github.com/ambs/Lingua-Identify-CLD
Co-reference https://github.com/jaukia/cld-js

https://github.com/CLD2Owners/cld2 (C++, 83 languages)

Co-reference https://github.com/rainycape/cld2
Co-reference https://github.com/dachev/node-cld
Co-reference https://github.com/ropensci/cld2
Co-reference https://github.com/fntlnz/cld2-php-ext

https://github.com/commoncrawl/language-detection-cld2 (Java)
https://github.com/lstrojny/php-cld (PHP)

https://github.com/Mimino666/langdetect/issues/67

Nice ! Will look into it soon enough. Thank.

@Ousret
1.There is a major problem with franc's trigram based data: they all used the UDHR as the base dataset, which is very weak in nature https://github.com/wooorm/franc/issues/78
2. There seems to be a repeating pattern with Google's CLD and CLD2, that it is the most commonly cited. The reason I am avoiding CLD3 and similar is because they overused machine learning.

Okay so here is something unique https://github.com/pemistahl/lingua#4--how-good-is-it-top-

Extra thing to note: Fasttext has their own dataset back at https://fasttext.cc/docs/en/dataset.html used by https://github.com/iamaziz/language-detection-fastText (Python) and https://github.com/rse/fasttext-lid (JS)

CLD3 uses machine learning instead of simpler techniques for language detection

https://github.com/akihikodaki/cld3-ruby (Ruby)
https://github.com/bsolomon1124/pycld3 (Python)
https://github.com/kwonoj/cld3-asm (Wasm)
https://github.com/ntedgi/cld3-kotlin (Kotlin)
https://github.com/ropensci/cld3 (C++)

@Ousret when you have free time, should we start reading these two dozen repos, one by one, analyse why some of them claims that they are the best (https://github.com/pemistahl/lingua I am looking at you) and attempt to find the best dataset and model for achieving the best results?

I have already started. Will be back.

@Ousret apologies but I updated the list last week (which has been finalized) to make sure it has most of the tools covering multiple programming language and techniques, if you don't mind. Hope you can use the current list as reference.

I Will. Thank you. 🙏

Hi, how are you? Hope you are doing well.
I am planning to list all the languages that are supported by most of these libraries into a spreadsheet with alphabet information and language type for ease of comparison https://ethercalc.org/charset_normalizer

There are also other datasets like

http://www.cs.cmu.edu/~ralf/langid.html with ~1000 languages
Language dataset for Langid https://github.com/saffsd/langid.py
Language dataset for Langdetect https://dumps.wikimedia.org/backup-index.html
Language dataset for CLD2 https://github.com/CLD2Owners/cld2/blob/master/internal/test_shuffle_1000_48_666.utf8.gz
Language dataset for Lingua https://wortschatz.uni-leipzig.de/de

Hi,
I tried many ways to increase the language detection coverage, but it is costly one way or the other. Most of the time, performance-wise.
It is unlikely this package will change the main method of language detection sometime soon.
All the research you have done was very helpful, thanks.


Entity as another entity's field

Fixes #43

It adds support to use an entity as a field of another entity without error
Proposed Changes

Added a parameter in the functions entityField2gql and entityFieldType2gql, so that it can identify when the function is creating a type and when it is an input type

Readiness Checklist
Author/Contributor

[x] If documentation is needed for this change, has that been included in this pull request
[x] Remember to check if code coverage decrease, if so, please implement the necessary tests

Reviewing Maintainer

[ ] Label as breaking if this is a large fundamental change
[ ] Label as either automation, bug, documentation, enhancement, infrastructure, or performance

@pamellaas merge master into your branch and then delete your package.lock file.
After deleting the file, run npm install, add the changer files and make a new push, please.

I don´t get it. This PR is suppose to handle entities as type for other entities. However, from what I could read from the code it is a implementation of 'Input' types. How does this fix the problem?

I don´t get it. This PR is suppose to handle entities as type for other entities. However, from what I could read from the code it is a implementation of 'Input' types. How does this fix the problem?

In the issue we imported the "B" entity in the wrong way, we should have done this way :
const { B } = require('../entities')
When we import in that way, it trow this error :

This is the problem that we solved, by adding a new parameter when calling the function entityField2gql and creating a new if statement in the entityFieldType2gql function.

:tada: This PR is included in version 2.1.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

@pamellaas, @Vx-Nico it seems this version is not working properly.
Could you please fix this asap? https://github.com/herbsjs/todolist-on-herbs/issues/106


[QuickStart.md] Error when running deploy-infra pipeline due to changed folder structure

[QUICKSTART.md]
When running the "tf-ado-deply-infra.yml" pipeline I receive this error:

The reason might be due to changed folder structure in row 7 and 9 (tf-ado-deploy-infra.yml) :
- template: ../../config-infra-prod.yml
- template: ../../config-infra-dev.yml

Here we might need go one hierarchy level higher.

Reviewing the error and trying to reproduce

@setuc This is going to be because of the sparse checkout, or lack thereof. It should be working fine if the sparse checkout is run first.

The effects of the following lines from sparse_checkout.sh seem to be making it hard to keep the relative paths correct in Azure/mlops-project-template:
mv infrastructure/$infrastructure_version $infrastructure_version
rm -rf infrastructure
mv $infrastructure_version infrastructure

For instance, there's been back and forth here: https://github.com/Azure/mlops-project-template/commits/main/infrastructure/terraform/pipelines/tf-ado-deploy-infra.yml
And the current relative path in mlops-project-template doesn't work in an ADO pipeline after running sparse_checkout.sh


add videos to extension pages

@Usha I'm assuming the Youtube videos aren't displaying in the preview because it's a preview?


Fixes for having multiple starter kit apps installed

Resolve https://github.com/blefebvre/aem-phonegap-starter-kit/issues/14 by changing "Shell" to "App"
Resolve https://github.com/blefebvre/aem-phonegap-starter-kit/issues/12 by setting app's component group to "Brand name" "App name"
Resolve https://github.com/blefebvre/aem-phonegap-starter-kit/issues/17 by setting allowed path for templates to just the app directory rather than all phonegap apps

+1 - thanks, @savs!


WMAgent pypi packaging

These are the changes I have made to be able to build the WMAgent pypi package.
Documentation updates regarding pypi packaging will be made on this wiki page:
https://github.com/dmwm/WMCore/wiki/DMWM-Packaging-with-PyPi
There are some caveats regarding setup.py that I will explain.
The original setup.py (used to build the wmagent rpm) has been renamed to setup_wmcore.py
A new file setup_wmagent.py was created that is used to build the wmagent pypi package.
To build the pypi package, I create a symlink setup.py -> setup_wmagent.py
To build the rpm, the symlink can be changed setup.py -> setup_wmcore.py
Regarding versioning, pypi packages need to follow PEP 440: https://www.python.org/dev/peps/pep-0440/
WMCore uses versioning like "1.1.15.pre6", which will get normalized during the build process.
UserWarning: Normalizing '1.1.15.pre8' to '1.1.15rc8'

@goughes Erik, apparently there is a conflict with this PR.
Now that Yuyi merged the DBS PRs, can you please run another wmcore/wmagent test with 1.2.1 and make sure everything works fine? Thanks

test this please

test this please

@goughes I just noticed we ended up with 3 requirements txt files now. Sorry if I missed it, but couldn't we use requirements.txt in place of requirements.wmagent.txt? AFAICS they have the same list.
In addition to that, maybe the wmcore one should have all the wmagent dependencies plus whatever we have in central services spec. What do you think?

I thought we were going to keep requirements.txt as a full list of requirements for any package that can be built from WMCore (wmagent, reqmgr2, etc.) mainly for documentation purposes. The list of dependencies in requirements.txt will grow once we add the requirements for the other packages and diverge from the wmagent one. The more I think about it, I suppose we could even rename requirements.txt to something else or potentially remove it altogether.
For the wmcore requirements file, including all dependencies creates problems. For instance, cx-Oracle requires ORACLE_HOME to be set and MySQL-Python requires mysql_config to be present. So those really should just be in the wmagent requirements file.

I believe the GH alerts are sent based on the requirements.txt, so if we rename or remove it, we might miss the vulnerabilities-wise automatic alerts sent by GH. Are you saying then that:

requirements.txt: could be a list with all WMCore python dependencies
requirements_wmagent.txt: list of WMAgent related dependencies
requirements_wmcore.txt: no requirements at all
?


Add watch option using WatchExec

Closes #4
The only working watch tool that i could get to work is Watchexec.
This PR leverages this file watcher to trigger a new compilation on every file that is created/modified.
TODO:

[ ] Add documentation

I am not sure about this path to add a watch mode. What do you think about looking at the implemenentation they did on castor ? Their implementation looks simpler, and I think it's better to sync with the community with think that already has been done

Castor uses a custom golang tool and compile it for mac, windows and linux. Since they agree that we use their binary so we might as well just ship these 3 binaries to make things easier. Thanks @WebMamba for suggesting this and asking for permission :)

I haven't tried this yet, but should we merge it and give it some real-world experience?

Yes, this feature shouldn't have too much impact on the normal behavior so let's put it to the test :)


Add new seccomp action

Updating documentation with a new seccomp action made available on libseccomp v2.4.0.
PR to implement this on runC:
https://github.com/opencontainers/runc/pull/1951
Libseccomp release:
https://github.com/seccomp/libseccomp/releases/tag/v2.4.0
Signed-off-by: Paulo Gomes<EMAIL_ADDRESS>

CC @mrunalp @vbatts

LGTM.

LGTM

just need another +1, PTAL? @caniszczyk @dqminh @hqhq @mrunalp @philips @tianon @vbatts @vishh

LGTM

On Oct 5, 2019, at 12:48 PM, Paulo Gomes<EMAIL_ADDRESS>wrote:
just need another +1, PTAL? @caniszczyk @dqminh @hqhq @mrunalp @philips @tianon @vbatts @vishh
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Module build failed: Module not found: "buffer"

Full Output:
 WARN  Compiled with 1 warnings                                                                                5:20:20 PM

Module build failed: Module not found:
"./node_modules/js-yaml/lib/js-yaml/type/binary.js" contains a reference to the file "buffer".
This file can not be found, please check it for typos or update it if the file got moved.

webpack compiled with 1 warning
Done in 114.26s.

System:
yarn -v && npm -v && node -v

yarn: 1.22.15
npm: 8.1.2
node: v16.13.1

Usage:
Using Webpack to bundle my assets for a website. This library is a dependency of a library used in the website.

As mentioned on the pull request (https://github.com/nodeca/js-yaml/pull/666), this is not to do with this library, but the environment to which it's used.
In my case, I recently upgraded from Webpack 4 to Webpack 5. In Webpack 5, polyfills for NodeJS APIs, are no longer supported (probably not wholly true but at least in part).
Here's an article describing how to polyfill buffer within a Browser-based application.


Add -Xmx256m to JLM test processes, decrease thread limit in server load

resolves https://github.com/AdoptOpenJDK/openjdk-systemtest/issues/249
igned-off-by<EMAIL_ADDRESS><EMAIL_ADDRESS>

Background: We need to re-calibrate the jlm tests and add -xmx256m to the sub-processes. We also need to change the number of threads the server is using from 200 to 30 - so that less number of threads are used. (details: https://github.com/AdoptOpenJDK/openjdk-systemtest/issues/249)

(Grinder link shared with reviewer on slack)


docs: update tldr page

Hi, I am a maintainer at tldr pages and we recently made some changes/fixes to the typeinc page. This PR updates the page to align with the upstream one.
Note: I removed the mnemonic to instead use the short option/long option syntax so that the short options don't get skipped when the page is translated.

One ques: 2 examples mentioned in contributing.md, shows the use of -s/--score to be of [s]core format, yet now you are changing this to the format mentioned in your PR. I had previously added a command (called cheatshh) which Sebestiaanspeck(one of maintainer for tldr ig) changed to the [s]core format.
Next time I am adding a command, which format shuold i go for?

Also Thanks for adding the PR, ill merge it soon. I noticed you guys put a spanish translation, for which i express my gratitude.
I hope you all liked Typeinc.

One ques: 2 examples mentioned in contributing.md, shows the use of -s/--score to be of [s]core format, yet now you are changing this to the format mentioned in your PR. I had previously added a command (called cheatshh) which Sebestiaanspeck(one of maintainer for tldr ig) changed to the [s]core format.
Next time I am adding a command, which format shuold i go for?

Yeah, previously we only had the mnemonic format which was difficult to integrate into translations. So when we are updating pages we are switching it into this new both options syntax if the example isn't too long.
Mnemonics are still used for contextual places (to highlight words in descriptions that isn't an option i.e. some Powershell commands) and when examples cross the 250 character limit when the new syntax is used.

Ok sure. Got it. Ill see the tldr update for which format to use. Thanks! Ill merge the PR.


Tuples nested in namedtuples

Looks like https://github.com/scipy/scipy/pull/16082 fixed the multiline thing, but not the minmax=(0, 9) part and the doctest now passes with vanilla DoctestChecker. I.e. will fail if numbers are truncated.
The crux of the issue is the regex in try_convert_namedtuple

OK, was a user fluke not a bug: the DTChecker was missing a fix from scipy/scipy#16082. One more argument for having a this all centralized.


Duplicated event sent from dev environment.

When forcing a warning event to Sentry I noticed that two events are sent when running in dev mode, while it is sent just once when running on prod mode.
Events from dev:
https://sentry.io/organizations/dev-curumas/issues/3767108540/?project=5505764&query=is%3Aunresolved&referrer=issue-stream
Note that the 1st event is marked as handled:false while the second event is marked as handled:true.
Event from prod:
https://sentry.io/organizations/dev-curumas/issues/3775484061/?project=5505764&query=is%3Aunresolved&referrer=issue-stream
This event is marked as unhandled, the same way as the 1st event from dev.
Environment
Symfony: 5.4.14
Sentry Symfony SDK: v4.4.0 (not tested in the new v4.5.0).
PHP: 7.4.33
Steps to Reproduce
Repro app:
sentryTest.zip

add a sentry DSN to config/packages/sentry.yaml and .env
Choose one of the environment to run first (prod/dev)
run the server with symfony server:start
go to localhost/l to trigger the warning event.
Repeat 2-4 with the second environment.

The events are sent from the LuckyController class in src/Controller/LuckyController.php
Expected Result
just one event sent from both environments.

You have monolog enabled but not configured to use sentry. I suspect the reason it happens in dev and not prod is that the settings are different, the error is being logged by monolog in a sentry-detectable way in dev but not in prod, and you're getting sentry entries either way.
If you follow the instructions to integrate with monolog does the issue persist?

I was never able to reproduce this. Given it's bee a while, I'm closing the issue.


fix: avoid reset of security.json if get fails (#659)

@smoldenhauer-ish your PR is from an organizational fork, so I'm unable to push a changelog entry, but can you merge main into your branch, then add the following changelog entry to helm/solr-operator/chart.yaml?
    - kind: fixed
      description: Avoid reset of security.json if get request fails
      links:
        - name: Github Issue
          url: https://github.com/apache/solr-operator/issues/659
        - name: Github PR
          url: https://github.com/apache/solr-operator/pull/660

I'll be good to merge this after you do that!

Added the changes entry - The 'edit by maintainers' is only available to private owned forks?


deaktivate 1 ticket maximum

dose anyone know a option that deaktivate the 1 ticket maximum?
Cann i only delete this If?

I have found a solution

Comment this solutions


Bug: Executors cant install pip packages assigned to the functions

When I try to run the workflow I get error
failed to fetch logs: diagnostic payload not found
stderr:
 Traceback (most recent call last):
  File
"/usr/local/lib/python3.10/site-packages/indexify/executor/function_worker.py",
line 142, in _run_function
    _load_function(namespace, graph_name, fn_name, code_path, version)
  File
"/usr/local/lib/python3.10/site-packages/indexify/executor/function_worker.py",
line 54, in _load_function
    function_wrapper = IndexifyFunctionWrapper(
  File
"/usr/local/lib/python3.10/site-packages/indexify/functions_sdk/indexify_functio
ns.py", line 173, in __init__
    ] = indexify_function()
  File "/home/ubuntu/python-api/workflows/pdf4.py", line 53, in __init__
ModuleNotFoundError: No module named 'inkwell'

To reproduce the issue follow the steps
I am using fastapi to manage the deploy and invocation
here is the main.py
from fastapi import FastAPI
from workflows import pdf4

app = FastAPI()

app.include_router(pdf4.router, prefix="/pdf")

@app.get("/")
async def root():
    return {"message": "Hello World"}

here is pdf4.py
from fastapi import HTTPException,APIRouter,File

router = APIRouter()

from indexify.functions_sdk.image import Image
from pydantic import BaseModel
from psycopg2.extras import Json
from typing import List, Optional, Dict
from inkwell import Page, Document as InkwellDocument
from indexify.functions_sdk.data_objects import File as IndexifyFile
from indexify.functions_sdk.graph import Graph
from indexify.functions_sdk.indexify_functions import IndexifyFunction, indexify_function

image1 = (
    Image()
    .name("tensorlake/indexify-executor-default")
    .base_image("ubuntu:22.04")
    .run("apt update")
    .run("apt install -y libgl1-mesa-glx git g++")
    .run("pip install torch")
    .run("pip install numpy")
    .run("pip install langchain")
    .run("pip install langchain_text_splitters")
    .run("pip install<EMAIL_ADDRESS>    .run("apt install -y tesseract-ocr")
    .run("apt install -y libtesseract-dev")
    .run("pip install py-inkwell")
)

class DeployInput(BaseModel):
    namespace: str
    indexifyUrl: str
    agentName: str

class PdfOutput(BaseModel):
    file: IndexifyFile

class PDFParser(IndexifyFunction):
    name = "pdf-parse"
    image = image1
    description = "Parser class that captures a PDF file"

    def __init__(self):
        super().__init__()
        from inkwell import Pipeline
        self._pipeline = Pipeline()

    def run(self, input: PdfOutput) -> PdfOutput:
        return input

@indexify_function(image=image1)
def upload_pdf(pdf: PdfOutput) -> PdfOutput:
    return pdf

@indexify_function(image=image1)
def example(pdf: PdfOutput) -> bool:
    return True

@router.post("/deploy")
async def deploy_workflow(input: DeployInput):

    g = Graph(
        input.agentName,
        start_node=upload_pdf,
    )

    g.add_edge(upload_pdf, PDFParser)
    g.add_edge(PDFParser, example)

    from indexify import RemoteGraph
    import sys
    RemoteGraph.deploy(g, server_url=input.indexifyUrl,additional_modules=[sys.modules[__name__]])

    return {"message":"Deployed pdf_graph succesfully"}

from fastapi import  UploadFile, File  ,Form

@router.post("/run")
async def run_workflow(
    agentName: str = Form(...),
    indexifyUrl: str = Form(...),
    file: UploadFile = File(...),

):
        # Read file contents
    fileData = await file.read()

    # Create an Indexify File object for the PDF
    pdffile = IndexifyFile(data=fileData, mime_type=file.content_type)

    # Create a PdfOutput object
    pdf_output = PdfOutput(
        file=pdffile,
    )

    # Proceed with the graph execution
    from indexify import RemoteGraph
    print("before graph by name")
    graph = RemoteGraph.by_name(name=agentName, server_url=indexifyUrl)

    # Run the graph
    print("before graph run")
    invocation_id = graph.run(pdf=pdf_output,block_until_done=True)

    result = graph.output(invocation_id,"upload_pdf")

    print("result is",result)
    # Return successful response
    return {"message": f"PDF processing initiated {invocation_id}"}

RUN the server via
uvicorn main:app --host <IP_ADDRESS> --port 8000 --reload

Installed the indexify server via helm chart using local.yaml pointing to aws s3
use executor version 3.10
executors:
  - name: indexify-executor
    image: tensorlake/indexify-executor-default:3.10

The environment where this python server is running has
indexify version: 0.2.16
python version: 3.10.15 (same as executor version)

Closing this as images for executors have to be built for open source installation. We build images automatically in tensorlake.


packit-prod: get PR ended with 404

[2019-09-11 09:21:50,907: INFO/ForkPoolWorker-1] Github issue 330 comment event.
[2019-09-11 09:21:50,908: INFO/ForkPoolWorker-1] Target repo: rpm-software-management/mock.
[2019-09-11 09:21:53,633: ERROR/ForkPoolWorker-1] Task task.steve_jobs.process_message[41ece6ab-1551-4e36-9fcb-ceb45cbcb514] raised unexpected: UnknownObjectException(404, {
'message': 'Not Found', 'documentation_url': 'https://developer.github.com/v3/pulls/#get-a-single-pull-request'})
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/celery/app/trace.py", line 641, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/packit_service/worker/tasks.py", line 42, in process_message
    return SteveJobs().process_message(event=event, topic=topic)
  File "/usr/local/lib/python3.7/site-packages/packit_service/worker/jobs.py", line 236, in process_message
    jobs_results = self.process_jobs(event_object)
  File "/usr/local/lib/python3.7/site-packages/packit_service/worker/jobs.py", line 86, in process_jobs
    package_config = event.get_package_config()
  File "/usr/local/lib/python3.7/site-packages/packit_service/service/events.py", line 282, in get_package_config
    self.base_ref = self.get_project().get_pr_info(self.issue_id).source_branch
  File "/usr/local/lib/python3.7/site-packages/ogr/services/github.py", line 478, in get_pr_info
    pr = self.github_repo.get_pull(number=pr_id)
  File "/usr/lib/python3.7/site-packages/github/Repository.py", line 2253, in get_pull
    self.url + "/pulls/" + str(number)
  File "/usr/lib/python3.7/site-packages/github/Requester.py", line 276, in requestJsonAndCheck
    return self.__check(*self.requestJson(verb, url, parameters, headers, input, self.__customConnection(url)))
  File "/usr/lib/python3.7/site-packages/github/Requester.py", line 287, in __check
    raise self.__createException(status, responseHeaders, output)
github.GithubException.UnknownObjectException: 404 {'message': 'Not Found', 'documentation_url': 'https://developer.github.com/v3/pulls/#get-a-single-pull-request'}

I don't know, is this an issue or a race?

This line looks bad to me:
self.get_project().get_pr_info(self.issue_id).source_branch

It is IssueCommentEvent so there is no branch we can work with.
(Reason for the traceback: We are trying to get pull-quest with the number equal to the issue number.)
I am not sure, but we can fix that by simply removing lines 281-282?


Roll engine e625e174c5df..0816c040fbba (6 commits)

https://github.com/flutter/engine/compare/e625e174c5df...0816c040fbba
git log e625e174c5df..0816c040fbba --first-parent --oneline
2020-01-31<EMAIL_ADDRESS>Deprecated FlutterView v1 points to FlutterView v2 (flutter/engine#16304)
2020-01-31<EMAIL_ADDRESS>Roll src/third_party/skia 0ee16f689843..4513cdde4912 (13 commits) (flutter/engine#16307)
2020-01-31<EMAIL_ADDRESS>Roll fuchsia/sdk/core/linux-amd64 from 93K0d... to VMcGV... (flutter/engine#16267)
2020-01-31<EMAIL_ADDRESS>Isolate and move common portable ELF loading from fixtures into //flutter/testing. (flutter/engine#16305)
2020-01-31<EMAIL_ADDRESS>Ensure that Android embedding initialization always uses an application context (flutter/engine#16204)
2020-01-31<EMAIL_ADDRESS>Fix runtime_unittest in AOT mode by loading AOT symbols from ELF loader. (flutter/engine#16283)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md

Trybots failed. These were the failed builds: https://github.com/flutter/flutter/runs/420479766

Commit queue failed; closing this roll.


The app interferes with the navigation bar?

When I activate your app with the quick setting, everything works fine except when I use an app with a light background, my navigation bar becomes white and the buttons disappear weirdly. Do you have any idea what's the issue?
Device: OnePlus 6
ROM: LOS 16 unofficial
Kernel: Blu Spark KERNEL

Screenshot:

This is explained in the terms you were supposed to read.
https://github.com/zacharee/NachoNotch/blob/master/app/src/main/assets/Terms.md

@zacharee is there anything that can be done to mitigate this? The terms don't really explain why this behavior happens and I don't know enough about android to investigate - is it a limitation of the overlay api or something?  It's the only bug/behavior that really frustrates me with this app


a small piece of advice

When users create a quest, they have to additem(or learn spell).There are also some items (e.g.49426) that are not placed in bag.
Therefore, I suggest increasing the method of input id.

Fixed with #2. Thank you!


When two channels are on different nodes - hold in call manager doesn't work

Hold plays music to one end whilst silencing the other. When a mix is created which has a joining leg when unholding (stopping the playback and reenabling media on other)  doesn't get the media flowing again and music continues to play.

We have removed support for channels through multiple nodes - this function has moved to CallManager which will re-negotiate end point


⚠ SceneTime has degraded performance

In 1621750, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1197 ms

Resolved: SceneTime performance has improved in 3f8e294 after 8 minutes.


Missing function org-link-make-string

I'm trying to test out this package, but I am getting the following error:
Symbol’s function definition is void: org-link-make-string
I can't seem to find out where that function is from, any help is appreciated!
I'm on Emacs 26.3 and I installed org-clubhouse via quelpa.

hmm, org-link-make-string is part of org-mode - what version of org-mode do you have installed?

Ah, that was the issue, thanks! I was on 9.1.9 and once I upgraded to the latest version it works!


Documentation bug - Config file location listed incorrectly.

The install documentation 2.6.1 list the location of the configuration files as "ontask/settings" (i.e. <app_base>/ontask/settings given the context of this usage elsewhere).   This appears to be actually in  "/settings".

Oops! It is one of the changes in the latest version. Thought we had all these fixed in the documentation, but clearly not... Thanks!

On second thought, I just saw the version you mentioned. 2.6.1. The config file needs to be in <app_base>/ontask/settings. In version 6.0 it needs to be in <app_base>/settings (assuming that app_base is the folder you get once you cloned the repo...

@abelardopardo sorry that was section 2.6.1 of the documentation, not the version number.  The Ontask version is 6.0.1

Ok, good catch.

Fixed in release 6.1


replace extendio.i18n with li18ngo

at this point, with the changes in config to be in line with other projects like traverse now make ci build fail in the same way as it does locally due to an issue in avfs which is an indirect dependency via extendio's reliance on avfs:
ginkgo run -r -json-report ginkgo.report -coverpkg=./... -coverprofile=coverage.out
Failed to compile cfg:

# github.com/snivilised/pixa/src/app/cfg.test
github.com/avfs/avfs.(*Utils[go.shape.*uint8]).cleanGlobPathWindows: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.NewPathIterator[go.shape.*uint8]: relocation target path/filepath.volumeNameLen not defined

Failed to compile command:

# github.com/snivilised/pixa/src/app/command.test
github.com/avfs/avfs.(*Utils[go.shape.*uint8]).SplitAbs: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.(*Utils[go.shape.*uint8]).cleanGlobPathWindows: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.NewPathIterator[go.shape.*uint8]: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.(*Utils[*github.com/avfs/avfs/vfs/memfs.MemFS]).VolumeNameLen: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs/vfs/memfs.(*MemFS).VolumeNameLen: relocation target path/filepath.volumeNameLen not defined

Failed to compile proxy:

# github.com/snivilised/pixa/src/app/proxy.test
github.com/avfs/avfs.(*Utils[go.shape.*uint8]).SplitAbs: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.(*Utils[go.shape.*uint8]).cleanGlobPathWindows: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.NewPathIterator[go.shape.*uint8]: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs.(*Utils[*github.com/avfs/avfs/vfs/memfs.MemFS]).VolumeNameLen: relocation target path/filepath.volumeNameLen not defined
github.com/avfs/avfs/vfs/memfs.(*MemFS).VolumeNameLen: relocation target path/filepath.volumeNameLen not defined

no composite coverage computed: all suites included programatically focused specs

Ginkgo ran 5 suites in 2m8.318621s

There were failures detected in the following suites:
      cfg ./src/app/cfg [Compilation failure]
  command ./src/app/command [Compilation failure]
    proxy ./src/app/proxy [Compilation failure]

Test Suite Failed

It is at thie point now, we have to remove extendio entirely to get rid of this issue.


[Proposal] Pack version for bundled / system packs (core, linux, etc.) should match StackStorm version

Right now we don't ever update version attribute in pack.yaml for packs which are bundled with StackStorm packages (core, linux, etc.). which means it's hard to check if pack content on disk is pristine / has been updated.
See https://forum.stackstorm.com/t/upgraded-to-st2-2-10-1-from-2-7-and-dont-have-core-winrm-cmd/467/6 for an example issue.
I propose updating version attribute for all the bundled packs to match StackStorm version. This should work fine since StackStorm version is also a valid semver string.
This will require us to update our release workflows to automate it.

+1
Should the st2 pack version match as well?

st2 pack is a bit unique and more similar to other StackStorm-Exchange packs since it's not bundled with StackStorm installation.
So until we bundle it, there is no pressure to have the pack version match StackStorm version.

Agreed - I think automating this will be helpful

Could the release automation also include notes in the changelog to say what has changed in the pack since the last version (even if the only thing that is changing is the version)?
eg:
https://github.com/StackStorm/st2/blob/master/contrib/chatops/CHANGES.md
https://github.com/StackStorm/st2/blob/master/contrib/core/CHANGES.md
I was curious to see a complete list of packs included with st2. So here's a list.
Packs bundled and installed with st2:

[ ] chatops
[ ] core
[ ] default
[ ] linux
[ ] packs

Would the packs that are not installed by default with st2 also be versioned with st2?

[ ] examples
[ ] hello_st2 (<- is this for a tutorial somewhere?)

Then there's the st2 pack that is not bundled and would not be versioned with st2 (That makes sense as it works across multiple versions of StackStorm):
https://github.com/StackStorm-Exchange/stackstorm-st2

Initial PR is here - https://github.com/StackStorm/st2cd/pull/369.
In the future, we also need to decide how to handle st2 pack. Perhaps we should move it back to the core (we've done this back and forth before)?
As far as the changelog entry goes - it's a bit harder to do, but possible.

Confirmed that this release process change is now working end to end - https://github.com/StackStorm/st2/commit/9a1351418c12de73734f41b51b02007644ee408d.
I will close this issue and a new one can be opened for improvements for non-bundled system packs such as st2.
As far as changelog goes - I agree we should also keep that up to date, but this will be manual work which will need to happen when someone updates the bundled pack.


Improve performance & lower memory usage

Although we have made several improvements and clear memory leaks, everything can always be improved.
In order for the webtorrent-webrtc-based projects to reach more users easily, the trackers need to be able to scale with greater performance.
At the time of writing, 1K connected users go for about 500MB of RAM and each connected user is around 0.45MB (BTorrent's tracker)

@DiegoRBaquero Agreed, this is important.
From your testing, do you know which objects are using the most memory?

@feross Took a heapdump:  https://webseed.btorrent.xyz/heapdump-1587638310.868854.heapsnapshot
Sintel in the tracker memory:

Other ways to approach this problem:

REST tracker over HTTP2
FIFO peers stack

@feross
Tested uWS and it was a HUGE improvement. The only thing lost is the real IP address (which is never used, actually protects peers' privacy) and the full headers of the request (Again, protects peers' privacy from trackers' operators but would not allow filtering or blocking).
@alxhotel @jakefb You might want to check this out.

That looks interesting! Nice work! Have you looked at the heap when using µWS?
Do you thing we could make significant memory optimizations in our codebase?

@yciabaud I didn't even bother, but I could get a heapshot if you'd like to see if there's anything else we can improve.
We are only left with clearing 1-dead-peer-only zombie infoHashes (https://github.com/feross/bittorrent-tracker/issues/164)
By the way, guys, could I get comments on<EMAIL_ADDRESS>Thanks

@DiegoRBaquero interesting. Did you try to disable permessage-deflate in both ws and uws? I know that the overhead for ws is pretty big. Also to reduce memory usage a little you can try null the upgradeReq object when the 'connection' event is emitted, uws does this under the hood.
wss.on('connection', (ws) => {
  ws.upgradeReq = null;
});

I'm actually not sure if this works as expected.
It might also be interesting to see if installing ws from master makes any difference. There are some improvements including breaking changes that have not been released yet.
npm i websockets/ws bufferutil

I also see that the clientTracking option for ws is set to true (default value). It seems that this lib uses its own way to track clients so you may want to set it to false.

@lpinca Had not try either the options you describe, had I seen them in the docs por better performance I would have tried them before.
I'll be monitoring uWS performance for the weekend and PR the change on monday if everything is cool.

That looks interesting! Nice work! Have you looked at the heap when using µWS?
Do you thing we could make significant memory optimizations in our codebase?

µWS doesn't use the JS memory at all (except for the small JS wrapper that is) so that heapdumper won't show real memory usage.

I also see that the clientTracking option for ws is set to true (default value). It seems that this lib uses its own way to track clients so you may want to set it to false.

µWS has built in "clientTracking" that cannot be turned off.

Did you try to disable permessage-deflate in both ws and uws? I know that the overhead for ws is pretty big. Also to reduce memory usage a little you can try to null the upgradeReq object when the 'connection' event is emitted, uws does this under the hood.

µWS has permessage-deflate enabled by default, just like ws. Removing upgradeReq from ws does not improve memory usage, it actually slightly increases it (whatever V8 is doing, it is not doing the correct thing). The reason why µWS stresses so hard on removing upgradeReq is because it holds the net.Socket object which is massive. µWS does not use the net.Socket at all, so removing upgradeReq means a 600 mb memory increase for 1 million connections. Again: this does not apply to ws.
TLDR<EMAIL_ADDRESS>is 100% legit. Nice post dude, but I would recommend disabling permessage-deflate for a lot better throughput. I do agree permessage-deflate should not be enabled by default.

I'm actually not sure if this works as expected.

Well, you certainly seem to know a lot about the upgradeReq since you have been insinuating all over that ws could easily scale to µWS levels if upgradeReq was removed. Obviously you have never actually tried this, which you also show here.
Bravo :clap: :clap:

@lpinca Maybe you should listen to me when I tell you what is the best usage of µWS?  I have told you since version 0.4.0 that you have to remove the upgradeReq in the µWS transformer - not only because it massively hurts memory usage to leave it in, but also the fact that it holds references to deleted memory that should not be used by the app developer.
Did you listen? Doesn't seem like it.

@alexhultman

I'm actually not sure if this works as expected.

I'm not sure because there might be other references that prevent the upgradeReq from being GC'd.

Well, you certainly seem to know a lot about the upgradeReq since you have been insinuating all over that ws could easily scale to µWS levels if upgradeReq was removed.

I've never said that ws could scale at the same levels of uws, if I did show me where please. I only said that your numbers are wrong and they still are. permessage-deflate seem to have serious issues in ws, see https://github.com/websockets/ws/issues/804.

 I have told you ages ago that you have to remove the upgradeReq in the µWS transformer

This make me sad... Did you actually read why we can't do that? I guess not.
Want to discuss about Primus? I'm all for it but it's not relevant to this issue. Let's do it somewhere else.
I've only suggested how to improve ws performance here.

@DiegoRBaquero can we move forward on this? Do you still think we should switch to uWS? Did you test disabling permessage-deflate?

@alxhotel @jakefb Could you guys update to latest and test memory consumption with per message deflate and client tracking off in ws?

@DiegoRBaquero Done! Results from the last 12 hours are aprox: 5x less memory consumption and 3x more network bandwidth.

@alxhotel Were there any restarts due to errors?
I think I caused a restart when loading /stats, which evicts old peers.

If you want a sweetspot between low memory usage and low network usage you can spend some time optimizing the WebTorrent protocol. I assume it is based on JSON right now and that is very ineffective vs something binary. I bet you can improve it a ton in size.

@alexhultman We could try switching back to raw bencoding since that's what the other tracker implementations (http and udp) use. But we would need to consider a transition period where we supported both bencoding/json for several months until clients are updated.

@DiegoRBaquero Yep. See issue #205

I seeded a 4G file, it had token almost 8G memory.


add README.md

I don't know the license that will be used, you may provide it and update it or tell me which one and I update it.

use the MIT license


add HLS to dev shell. create direnv environment

Why no HLS?

Thanks for your contribution @Josemarialanda! I wanted to make a couple of changes so I've done that in #10.
It's better practice not to commit .envrc so that each person can put their own customisation in there.


Automated cherry pick of #2123: fix: get keypair info fail after rebuild-root

Cherry pick of #2123 on release/2.10.0.
#2123: fix: get keypair info fail after rebuild-root

/approve
/lgtm


Fetch config from the API

Closes #28

@TechHummel I've finally decided to include the config files from the original app, as the newest configs are available in the public API, so are hardly a secret.
I'm not using them to convert the BLE messages, but rather to test if configuration parsing is working correctly for different types of appliances.

I really like this approach! Allows us to have well tested and generic code without any hassle for users!

I really like using the already available configuration files for testing 😄 Is using the API required now or is this optional, e.g. when parsing is not working correctly?

It is required
Meanwhile, I've added the translations, which are also using the API


[geometry-factory-param] Add support for parametrization of JTS Geometry...

Add support for GoemetryFactory parametrization.

I believe that @purpleP's work that I've just merged addresses this, so I'm closing now. Thanks in any case! Alex


Added resources for Ruby, Ruby on Rails and Linux.

@saranshkataria Please take a look.

Looks good. Thanks!


Revert "HACK: Update ImagePullPolicy to Always"

This reverts commit 7ac43761848e616d2e5eda00a314badb7c2ffadc.
I'm documenting another way to set this for a dev workflow.

@mdbooth @MaysaMacedo when time permits.
I explain how do solve this in our new document.


high load factor

bytell_hash_set's rehash is not expected load factor. My flat emhash can set load factor to 1.0
I have update dynamic_rehash_effect.cc.
static void high_load()
{
    size_t maxSize = 1U << 28;
    size_t numReps = 100;

    std::random_device rd;
    auto dis = std::uniform_int_distribution<uint32_t>{0, (1U << 31) - 1};

    for (size_t rep = 0; rep < numReps; ++rep) {
        {
            auto rng = std::mt19937(rep);
           //https://github.com/ktprime/emhash/blob/master/hash_set4.hpp
            emhash9::HashSet<uint32_t> set;
            set.max_load_factor(0.999);

            auto t1 = getus();
            while (set.size() < maxSize) {
                auto key = dis(rng);
                size_t prevCap = set.bucket_count();
                set.insert(key);
                if (set.bucket_count() > prevCap) {
                    size_t prevSize = set.size() - 1;
                    auto lf = static_cast<double>(prevSize) / prevCap;
                    std::cout << prevCap << " " << prevSize << " " << lf << "\n";
                }
            }
            std::cout << "emhash loop " << rep << " time use " << (getus() - t1) / 1000000.000 << " sec\n";
        }

        {
            auto rng = std::mt19937(rep);
            ska::bytell_hash_set<uint32_t> set;
            set.max_load_factor(0.999);

            auto t1 = getus();
            while (set.size() < maxSize) {
                auto key = dis(rng);
                size_t prevCap = set.bucket_count();
                set.insert(key);
                if (set.bucket_count() > prevCap) {
                    size_t prevSize = set.size();
                    auto lf = static_cast<double>(prevSize) / prevCap;
                    std::cout << prevCap << " " << prevSize << " " << lf << "\n";
                }
            }
            std::cout << "skaset loop " << rep << " time use " << (getus() - t1) / 1000000.000 << "sec\n\n";
        }
    }
}

//the result.
8 8 1
16 16 1
32 32 1
64 64 1
128 128 1
256 256 1
512 512 1
1024 1023 0.999023
2048 2046 0.999023
4096 4092 0.999023
8192 8184 0.999023
16384 16368 0.999023
32768 32736 0.999023
65536 65471 0.999008
131072 130941 0.999001
262144 261882 0.999001
524288 523764 0.999001
1048576 1047528 0.999001
2097152 2095055 0.999
4194304 4190110 0.999
8388608 8380220 0.999
16777216 16760439 0.999
33554432 33520878 0.999
67108864 67041756 0.999
134217728 134083510 0.999
268435456 268167021 0.999
emhash loop 0 time use 59.2812 sec
0 1 inf
16 16 1
32 32 1
64 64 1
128 128 1
256 255 0.996094
512 509 0.994141
1024 1014 0.990234
2048 1973 0.963379
4096 3929 0.959229
8192 7745 0.945435
16384 15540 0.948486
32768 30946 0.944397
65536 61592 0.939819
131072 121771 0.929039
262144 244278 0.931847
524288 485419 0.925863
1048576 963137 0.918519
2097152 1916894 0.914046
4194304 3790756 0.903787
8388608 7541070 0.898966
16777216 14901313 0.888187
33554432 29881340 0.890533
67108864 57400366 0.855332
134217728 115062312 0.857281
268435456 230845682 0.859967
skaset loop 0 time use 49.4685sec

Sorry, I am not sure what you are asking for in the issue?  Is there something you want to change or some other behavior you would prefer we use?

Sorry, I am not sure what you are asking for in the issue? Is there something you want to change or some other behavior you would prefer we use?

sorry for no any issuse, I just want to say bytell_hash_set high load factor(0.999) is not accurate during rehash in test.

I believe @ktprime is saying that a proper implementation of unordered set can rehash on insert only if load_factor() would otherwise become greater than max_load_factor(). In other words, the following test MUST pass for any Set that models std::unordered_set:
template <class Set>
void Test(Set& set, const typename Set::value_type & value) {
  size_t bucket_count = set.bucket_count();
  set.insert(value);
  if (bucket_count == set.bucket_count()) return;
  double would_be_load_factor = 1. * set.size() / bucket_count;
  assert(would_be_load_factor > set.max_load_factor());
}

(Let's ignore potential issue due to floating point precision.)
I don't think this behavior is required by the standard.  Here's what it says about max_load_factor():

Returns: A positive number that the container attempts to keep the load factor less than or equal to. The container automatically increases the number of buckets as necessary to keep the load factor below this number.

If my reading is correct, this sentence is saying that load_factor() <= max_load_factor() MUST be true. It does not, however, prohibit rehashing when it's not strictly required. This can be a useful property though: it would give users more control over load factor and would allow one to tell in advance whether insert() will rehash or not. Implementations of classes that model std::unordered_set are free to document whether they provide the extra guarantee. In the absence of explicit documentation on this front we should assume only what the standard says.

Most flat hash map can't set high load factor for performance issue, but my emhash7 can set load factor to 0.999.
The following benchmark shows that emhash has amazing performance even insert & erase with a high load factor.
#include "hash_table7.hpp"
static void TestHighLoadFactor()
{
    std::random_device rd;
    const auto rand_key = rd();
    std::mt19937_64 srngi(rand_key), srnge(rand_key);

    const auto max_lf   = 0.9990f; //<= 0.9999f
    const auto vsize    = 1u << (20 + rand_key % 6);//must be power of 2
    emhash7::HashMap<int64_t, int> myhash(vsize, max_lf);
    assert(myhash.bucket_count() == vsize); //no rehash

    auto nowus = getus();
    for (size_t i = 0; i < size_t(vsize * max_lf); i++)
        myhash.emplace(srngi(), i);
    assert(myhash.bucket_count() == vsize); //no rehash

    //while (myhash.load_factor() < max_lf - 1e-3) myhash.emplace(srngi(), 0);
    const auto insert_time = getus() - nowus; nowus = getus();
    //erase & insert at a fixed load factor
    for (size_t i = 0; i < vsize; i++) {
        myhash.erase(srnge()); //erase a exist old key
        myhash[srngi()] = 1;
    }
    const auto erase_time = getus() - nowus;
    printf("vsize = %d, load factor = %.5f, insert/erase = %ld/%ld ms\n",
        vsize, myhash.load_factor(), insert_time / 1000, erase_time / 1000);
    assert(myhash.bucket_count() == vsize); //no rehash
    assert(myhash.load_factor() >= max_lf - 0.001);
}

  vsize = 1048576, load factor = 0.9990, insert/erase = 25/76 ms
  vsize = 2097152, load factor = 0.9990, insert/erase = 52/222 ms
  vsize = 4194304, load factor = 0.9990, insert/erase = 117/450 ms
  vsize = 8388608, load factor = 0.9990, insert/erase = 251/1009 ms


refactor: update collateral token to relay chain native token

I found that we were referring to a relay chain's native tokens like DOT or KSM as collateral token. I think it's not correct. In the past, the relay chain's native token was the only collateral but now we are supporting multi-collaterals including KINT and USDC and more will be added. So I think we should update names accordingly.
This PR does not include any logical changes but just renaming. So I'd say it's safe to merge this PR without having a look at every single line of change in detail.

Morning @crypto-engineer - this all sounds like it makes sense, but going to hold off approving until the KINT support PR is merged (not expecting any issues as that moves the vault collateral tokens to their own config, but don't want to risk any delay as that feature is already quite behind schedule!).

Looks good to me 👍 edit: just saw @tomjeatt's comment, so dismissed the review :)

Hi Peter, sorry could have been clearer :) no problem approving, just wanted to hold off merging!

@tomjeatt
Do you think it's ready for being merged?


Move test-support helpers to addon-test-support

Similar to: https://github.com/ciena-blueplanet/ember-test-utils/issues/40.
Problem
An exported addon test file that is using the drag-drop helpers cannot be imported into my app. The specific problem I was running into:
@my/addon
  - addon-test-support
     - addon-test-file.js
       import "dummy/tests/helpers/drag-drop" <-- works because my test app is named "dummy"

@my/app
  - node_modules/@my/addon
  - tests/app-test-file.js
    import "@my/addon/test-support/addon-test-file" <-- fails because my app has a name different from "dummy"

Solution
Move test-support/helpers/* (except ember-drag-drop.js, which is deprecated) to addon-test-support/helpers/* to allow for scoped ember-drag-drop/test-support/* imports.
Re-export the addon-test-support/helpers files from test-support/helpers for backwards compatibility.
Add a test that imports the old and new paths and asserts the exported values are equal.

Interesting, I wasn't aware of the change to the paths for exporting test helpers. Thanks for the PR, I'll merge it soon.


Tangentially related issue

I've tried getting RServices installed and set up on my organization's SQL Server 2016 Standard instance several times now.  Each time I follow the set up/installation instructions published on Microsoft's website and it has yet to succeed.
I have the Launchpad service started and all of the MSSQLSERVER users are created as is the SQLRUserGroup logon.  I've granted permissions to execute external scripts and am one of the sys admins on the server itself.  However, no matter what I do I always get the same uninformative error message:
exec sp_configure 'external scripts enabled', 1
reconfigure with override;

Exec sp_configure  'external scripts enabled';

exec sp_execute_external_script  @language =N'R',
@script=N'OutputDataSet<-InputDataSet',
@input_data_1 =N'select 1 as hello'
with result sets (([hello] int not null));
go

Msg 39011, Level 16, State 1, Line 1
SQL Server was unable to communicate with the LaunchPad service. Please verify the configuration of the service.
Msg 11536, Level 16, State 1, Line 1
EXECUTE statement failed because its WITH RESULT SETS clause specified 1 result set(s), but the statement only sent 0 result set(s) at run time.

Is it asking too much to get installation instructions that will work as expected or for more complete installation/troubleshooting documentation to be published?  For example, some users have mentioned something about a db_rrerole database role, but this isn't created when I work through the installation process and since there isn't any mention of this role in the official documentation I have no way of knowing if this was something only needed during the community testing release or if it is still required in the current release.

Although this is an old issue, please note that this repo is for samples only. If you have issues with RServices, please use our forums.


MSBuild should have a directory import in Microsoft.Common.Targets

My request is to be able to put enterprise level logic in a msbuild target at a solution level rather than modifying every single .*proj file. This way I can modify one file in the solution directory and all projects which implement Microsoft.Common.Targets can set things like $(OutDir), Reference Paths, Analyzers, Nuget restores/updates or even Build server logic (Strongname signing, digital signuates, assembly versions). It would be priceless to enforce logic on a solution without relying on developers to install the custom target file into the right directory. Currently I do this by modifying all the projects to import a solution wide target file.
There are more details on the uservoice page including other suggestions to change but it would essentially be adding <Import Project="$(SolutionDir)Custom.$(MSBuildThisFile)" Condition="$(SolutionDir)Custom.$(MSBuildThisFile)')"/> or some variation of this to Microsoft.Common.CurrentVersion.targets just under $(MSBuildProjectFullPath).user.
I created a user voice post previously but when I saw the news about MSBuild being open source I decided to raise it directly with you here. https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/9674760-add-a-custom-solution-import-in-microsoft-common-t
I apologise for my persistence with this request however it would make maintaining an enterprise build system much easier.
I am happy to answer any queries you may have, especially if it strengthens the possibility of this becoming a reality! Thanks

This is an interesting idea, and there is definitely room for improvement in large-scale configurability.  However, I have a number of concerns about this approach.
The main issue is that, since a solution isn't a core MSBuild concept, using this extensibility point would be confusing for anyone who lives in a world where they don't only build through the solution.
That would be a problem for many CI build servers that are configured to build through a traversal project rather than a solution (which is quite common).
It would also be confusing in the fairly common case of having multiple "subsolutions" for different teams or roles within a codebase.  One would have to have the canonical configuration, but how would changes be coordinated between solutions?
And tying a project to a solution would also dramatically change single-project builds from the command line, since MSBuild would have no way to know what solution to find configuration in.
In general, I favor the explicit approach of adding an import to all projects, as you currently do.
Thoughts?

Thanks for taking the time to write a reply. Sorry if this is a long winded reply but you asked for my thoughts. :)
I can see your concern with solutions being out of scope of MSBuild. Perhaps I jumped the gun in suggesting a solution rather than giving my user story of the pain in managing our enterprise solution. Explicitly adding imports to every project does work because we currently do it, however we have been caught out time after time when one of the developers decide to add a new project and are unaware of the need to import a custom target, or import it in the wrong place. In our organisation we have been split between developers using a solution and build servers using .proj files, we would really just love to target the solution in both worlds. I would love the day when solutions are restructured to natively be msbuild projects but I can't see that happening in the near future. At the end of the day I am struggling to have all my developers and build servers use the same build tools and processes in their own environments.
I do not feel adding an import to the solution directory would be any more confusing than some of the existing 23 imports in Microsoft.Common.CurrentVersion.targets such as the visual studio specific <Import Project="$(MSBuildProjectFullPath).user" Condition="Exists('$(MSBuildProjectFullPath).user')"/> line, or the presence of the SolutionDir property in the file already. I can agree that just because those exist doesn't mean more hacks should be added :) The truth is the extensibility point that answers all my needs already exists in the file: the property $(CustomBeforeMicrosoftCommonTargets) but setting it from msbuild targeting a solution needs hacks and from visual studio is impossible (To my knowledge).
There currently is an extensibility point in solutions using after.MySolution.sln.targets (Setting Set MSBuildEmitSolution=1, running msbuild.exe /v:n MySolution.sln and then looking at the MySolution.sln.metaproj that is generated shows the import) but this calls MSBuild on each project without letting me set the properties, and visual studio completely ignores the import (Which is understandable). Through using some creative hacks (See post) I am able to inject that property in using msbuild.exe but I still have no way of tricking visual studio.
To be honest, I had to look up traversal projects and while I understand the concept I have never had to use them in my career. Perhaps I am lucky enough to be working on small enough solutions or they are broken up in a way that allows me to build 30 projects at a time and chain the next set as dependencies from a build server. I personally don't think it would create much confusion using traversal project or single projects, I would just suggest they do not use the file/import. :)
I am fine if it is not in msbuild's best interest to cater for solutions any more than it currently does, but while visual studio needs developers to have them I will try make our build servers use those files.
Thanks again

Can you put targets files in the MSBuildExtensionsPath or MSBuildUserExtensionsPath folder?  Microsoft.Common.CurrentVersion.targets will import any targets files in the following folders:
  <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.targets\ImportAfter\*" Condition="'$(ImportByWildcardAfterMicrosoftCommonTargets)' == 'true' and exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.targets\ImportAfter')"/>
  <Import Project="$(MSBuildUserExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.targets\ImportAfter\*" Condition="'$(ImportUserLocationsByWildcardAfterMicrosoftCommonTargets)' == 'true' and exists('$(MSBuildUserExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.targets\ImportAfter')"/>

Now you probably don't want to put your enterprise level logic there, but you can put a targets file that will import the right targets file from a directory above the project directory.  Something like this:
<Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildProjectDirectory), enterpriselogic.targets))\enterpriselogic.targets"
  Condition="Exists('$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildProjectDirectory), enterpriselogic.targets))')"/>

Does this help?  It's not explicitly scoped to the solution, but rather to whatever level in the directory structure you put an enterpriselogic.targets file in.

I am aware of the Import directories, in my use case i have done something very similar except i use SolutionDir and it works perfectly. The problem is ensuring everyone who builds the application also has it installed. My ideal situation is for any developer to load up a solution with visual studio installed and they will get the same experience as anyone else. Nuget has provided a huge leap in this area with including msbuild targets but is lacking at the solution level.
I could add an  conditioned to the absence of that file to tell them to ensure they put the file there and I can think of many hacks such as having the first project add the file into the directory. It may seem to be sensible to say 'everyone who builds this must first run this msi' but you'd be surprised how often you'd find someone missed a step in the setup process.
If you think this problem is currently an edge case and the existing methods are good enough I am fine with it. I was just using this as an opportunity to get feedback to see if I am the odd one out. :)

What if you didn't need to put anything in the Import directories, you just had to put a targets file with the right name in one of the ancestor directories of your project?  This sounds similar to what you're asking for, except instead of using $(SolutionDir), which isn't always defined, it would walk up the directory tree looking for the file to import.

So basically like the import you said earlier but baked into msbuild? That would meet all my needs. :)
I'm no expert on GetDirectoryNameOfFileAbove but I can't imagine it would be cheap to add to every single project? I would hate to have everyone else suffer a performance hit for my needs.

@Serivy Yes, I think we'd need to investigate the performance impact of adding an import with GetDirectoryNameOfFileAbove in it.  Personally I feel like this would be broadly useful (ie you're not the only one with these type of needs).

+1

My only objection to the proposal is discoverability (not like MSBuild is super well documented as it is) and the law of unintended consequences:
e.g.

+1 to the "walk up the tree" approach. Would you walk all the way up to the root gathering all the common targets or stop at the first one found?

I guess just the first one is probably safest. If you want it to continue walking you could include the same import code in that one :) Although i'm not fussed either way!
I must say I am quite excited about this change should it happen!

I also agree with the basic need expressed by the OP.  Thanks to @Serivy for taking the time to coherently capture the need.  Rest assured:  from my perspective, at least, you are _not _the "odd one out".
Having a capability along these lines would be valuable (i.e., an ability to apply "enterprise build logic" across all projects in a solution without requiring something specific be authored into each individual .csproj and without requiring something special be installed on a build machine).
Another way of stating the desire might be:  the "enterprise build logic" should be something that can be reasonably and naturally stored and retrieved from the same repository structure in which you retrieve the actual solution source itself.  Once retrieved, one would ideally not have to engage in "special steps" to get the associated enterprise logic incorporated into your machine's msbuild system.

Thanks @aheusser
Modifying all the proj files is something i've been doing for years and I thought there had to be better way, especially with features slipping in such as before.SOLUTIONNAME.sln.targets which helped it but didn't solve. That is a good way to look at it, I want to get to the point where you can git clone a solution, nuget restore and off we go on anyone dev/build machine.
The landscape will be changing so much in the near future with project.json files, Msbuild going cross platform, and just how node/npm has changed builds with solution level before/after targets. I would be interested if some day MSBuild adopts a similar 'gulp watch' sort of system for keeping an eye on file system changes.
Exciting times.

The team has discussed this and we want to get some feedback on our design.
We don’t believe a solution-based approach is a good idea for MSBuild.  MSBuild itself doesn’t directly support solutions since they’re really just a Visual Studio concept.  MSBuild can build solutions by transforming it to a project and then performing a build.  Also, when building a single project via MSBuild, there is also no mapping from a project to a solution so there would be no way for MSBuild to know which solution-based project to import.  There is also a case where you could have multiple solutions in your tree and would need to replicate your build logic for each one.
We think a more generic approach would be to have MSBuild traverse up the directory tree for a project to import similar to how it will do a wildcard import of projects in a specific location.  Our design would be to use the GetDirectoryNameOfFileAbove() property function to look for a global.build.props to import.  This global.build.props could then give users all of the flexibility they need to import a project based on the solution or have a global.build.props at any level of the tree.  There could be a slight added overhead because whenever you build, it would be walking up the directory structure looking for a file but we believe since it’s only checking for file existence it will add very little overhead.
Another thing we’re considering is when to stop looking in parent directories.  When it is determined that the search is looking at the root of a Git repository for instance, it should probably stop traversing up.  For example, there could be unintended results if you place a global.build.props in the root of your drive because projects could import it.  If we have logic to stop traversing up the tree, we could avoid this situation.  The real issue is the list of known stopping points could get hard to manage.  We think adding a check for a .git folder would probably suffice.
So to summarize, we’re thinking of adding the following logic to Microsoft.Common.props and something similar to Microsoft.Common.targets:
<PropertyGroup>
    <ImportGlobalBuildProps Condition="'$(ImportGlobalBuildProps)' == ''">true</ImportGlobalBuildProps>
    <GlobalBuildProps Condition="'$(ImportGlobalBuildProps)' == 'true' and '$(GlobalBuildProps)' == ''">$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildProjectDirectory), global.build.props))\global.build.props</GlobalBuildProps>
</PropertyGroup>

<Import Project="$(GlobalBuildProps)" Condition="'$(ImportGlobalBuildProps)' == 'true' and exists('$(GlobalBuildProps)')"/>

Please let us know what you think.

While I was pretty keen on a solution based approach at first I agree now it isn't right. I really do wish .sln files were to become MSBuild files, rather than requiring MSBuild to generate a metaproj file from its data. Any plans on this, especially considering .Net Cores full adoption of MSBuild projects? :) Traversing up the directory tree sounds like a perfect solution to my needs. Feel free to rename this issue to something more suited. Even if I wanted to (which I don't) I could even do solution property checks in there. I am very happy to hear of the low overhead of it on the build.
Knowing when to stop sounds like a difficult problem. My commercial projects don't use git so wouldn't really impact me but I am not too worried about rogue target files in the root of my drive. I have to check MSBuild logs daily so its not too hard to see which file the target is running from.  Realistically my project folders are getting filled up with a lot of these kind of configuration files, some traverse all the way to the root until they get a match (nuget.config, tslint.js, Settings.StyleCop) and some dont (.npmrc).
I will copy the proposed code to my copy of common.props just to see how it goes, if I find anything I think is important ill reply to this issue.
I know I may be sticking my nose in where its not wanted but if i could have a few moments of your time to consider something and tell me if I am barking up the wrong tree? I don't speak for the Nuget team but I follow it closely and I cant help but notice the technique we're looking at here could aid in one of their goals (http://blog.nuget.org/20141010/nuget-is-broken.html) to "Leave Project Files Alone". When you install a nuget package with a target file it will modify the project and inject an import for each nuget package along with an error task if its not found. Something like you're doing here could have a profound effect if they can make nuget just dump the .target files into a known directory structure. There may be too many issues like requiring the latest MSBuild, missing error messages if the nuget package wasn't restored, or something beyond my comprehension. Either way I thought i'd just bring it to your attention in case it could be useful.
Thanks :)

@Serivy Regarding the NuGet stuff: we're working with the NuGet team and the .NET CLI folks on evolution of how packages get referenced. I don't think there's really anything to show yet, though.
But there is a form of NuGet that is a little cleaner today, which is how UWP apps use project.json files to refer to packages. That process doesn't require changing the project file--they have .targets files that include things from the referenced packages that get automagically included.
We expect that the future direction of NuGet would be similar to that (but again, it's in flux).

My initial commit is out for a pull request.
Yes, as @rainersigwald said, the project.json currently addresses this by doing the following:

They install a file under %ProgramFiles(x86)%\MSBuild\14.0\Microsoft.Common.Targets\ImportAfter so they can inject targets.
nuget restore generates an MSBuild project for importing any .props or .targets files in the NuGet package in the folder where the project being built exists.
%ProgramFiles(x86)%\MSBuild\Microsoft\NuGet\Microsoft.NuGet.props and %ProgramFiles(x86)%\MSBuild\Microsoft\NuGet\Microsoft.NuGet.targets then import the dynamically generated projects.

This means that the original CSPROJ is not modified and the imports happen magically.  This is true for the assembly references as well.  This only works for MSBuild 14.0 and above because of where the files are installed to.
That said, the future of project.json is up in the air and we're working closely with the NuGet team.

Thanks! Looking forward to it.

This functionality has been merged.

Sorry for the spam, when will this feature be released? Thanks

@wangzq This will be in the next full release of MSBuild, which should come with Visual Studio "15". Thanks for asking, because in the course of double-checking my "it's in Preview 4" answer, I discovered that it's actually not, because of #979! In Preview 4, you can use the .targets form, but .props is broken. We'll get it fixed before the next preview release.

For others who find this issue but can't figure out why the global.build.props and global.build.targets files discussed above don't get loaded by MSBuild 15+: the file names were changed to Directory.Build.props and Directory.Build.targets (as hinted at by the second Feb 13 reference title).

I have been using them in production and loving them. Thanks for the change!

Directory.Build.props is now documented at https://docs.microsoft.com/en-us/visualstudio/msbuild/customize-your-build. Feedback is welcome!

I have had a read through and it is good. I have already forwarded it on to a few colleagues. Well done and thanks!

@rainersigwald That seems not to work for old MSBuild project files (not sdk style). Should it be supported for them, too?

Provided it imports Common at the right tools level it will read in the Directory.Build.targets file.
If it does not import common or the tool versions are too old, such as WIX projects then i add this to the .wixproj project.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <_ImportDirTarget>$([MSBuild]::GetDirectoryNameOfFileAbove('$(MSBuildThisFileDirectory)', 'Directory.Build.targets'))\Directory.Build.targets</_ImportDirTarget>
  </PropertyGroup>
  <Import Condition="'$(MSBuildToolsVersion)' < '15' and Exists('$(_ImportDirTarget)')" Project="$(_ImportDirTarget)"/>
  ...
 </Project>

@Serivy  thanks for clarification. I was not aware of that the common targets need to be imported for that feature.


Added folder ImageLikeUI

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

[ ] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[ ] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[ ] The title of my pull request is a short description of the requested changes.

Screenshots

Note to reviewers

Hi @1san1  , thank you for your contribution. I was looking deploy preview of your PR. I can't see the like button. Am I missing sth?

you are talking about implementation on Gallery.jsx file am i right?

you are talking about implementation on Gallery.jsx file am i right?

yes


Making any change to the form layout makes the data filled in form preview fields to disappear

Describe the bug
When we have provided a few values to the fields in the form preview and try to add new form elements to the form definition we could see that the previously filled in values disappear and needs to be filled in again
Steps to reproduce

Create a form with multiple form elements such as Text field,number and so
In the form preview enter values for the text field and number
Now in the form definition when we try to add new form elements we could see that the previously filled in values in preview disappears

https://user-images.githubusercontent.com/85225325/229804667-0892f9f7-fdaa-41d2-8ae1-d5fe95e5dce1.mp4
Expected behavior
Retain the previously filled in values in the form preview for the user not to re-enter again while changing the form definition view
Environment

OS: [Windows 10]
Camunda Modeler Version: [5.10.0]
Execution Platform:  Camunda Cloud]

Additional context
No response

This is normal, editing anything about the form definition resets the data state. We could look into being more persistent, but this would be quite a task because then we have to consider a whole suite of state management concerns.

This has always been the default behavior of the playground. However, I agree that it's confusing.

It doesn't block the release so I'll leave it up to HTO team to decide on this

I think, we'll leave this for now, because it's not a bug and we don't really have this as a feature request either. At best it's something we may want to have a look at some day, @christian-konrad do you think we want to make some changes to this, or is it covered by future epics? In which case I would advise closing the issue.


Cpython Code not Reading from Correct Location

I am currently trying to setup the pygslib library with my computer (Mac OSX Catalina 10.15.1) but I am receiving the following traceback:
`---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
 in
----> 1 import pygslib
~/opt/anaconda3/envs/deepmine/lib/python3.6/site-packages/pygslib/init.py in
24
25 # import pygslib modules (compatible with python 3x)
---> 26 from . import drillhole, blockmodel, vtktools, nonlinear, sandbox, gslib, plothtml, charttable, progress
27
28 from . import version
~/opt/anaconda3/envs/deepmine/lib/python3.6/site-packages/pygslib/gslib/init.py in
40
41 #gslib fortran code
---> 42 from . import __gslib__kt3d
43 from . import __gslib__postik
44 from . import __read_gslib
ImportError<EMAIL_ADDRESS>2): Library not loaded: /usr/local/gfortran/lib/libgfortran.3.dylib
Referenced from<EMAIL_ADDRESS>Reason: image not found`
Not sure why __gslib__kt3d.cpython-36m-darwin.so is reading from my /usr/local directory.
In addition, when trying from pysgslib.drillhole import Drillhole I end up with the following error statement:
AttributeError: type object 'pygslib.drillhole.Drillhole' has no attribute '__reduce_cython__'
Any help on this matter would be greatly appreciated!

Following the instructions in Issue #12 I am now able to get the drillhole module to work. However, I am having some troubled importing the kt3d file when calling import pygslib. Continuing to look into this today.

@djcurill , did you have any luck with the kt3d? I'm facing the same problem here.
File "C:\ProgramData\Anaconda3\lib\site-packages\pygslib\gslib_init_.py", line 1233, in kt3d
assert parameters['bhid'] is not None, 'Error: BHID required if nbhid > 0'
KeyError: 'bhid'
Please let me know if you got a breakthrough.
Thanks

Good day guys.
Have anyone found a solution to the kt3d problem?
I still get the following error.
TypeError                                 Traceback (most recent call last)
~/opt/anaconda3/lib/python3.6/site-packages/pygslib/gslib/init.py in kt3d(parameters)
1327         output['outlagr'],
-> 1328         output['outwmean']) =__gslib__kt3d.pykt3d(**parameters)
1329
TypeError: 'bhidint' is an invalid keyword argument for this function
During handling of the above exception, another exception occurred:
NameError                                 Traceback (most recent call last)
 in
354
355 # estimating in one block
--> 356 estimate, debug, summary = pygslib.gslib.kt3d(kt3d_Parameters)
~/opt/anaconda3/lib/python3.6/site-packages/pygslib/gslib/init.py in kt3d(parameters)
1330     except:
1331
-> 1332         raise NameError('unexpected error in FORTRAN kt3d code')
1333
1334
NameError: unexpected error in FORTRAN kt3d code
Please if anyone found a fix, I would greatly appreciate.
Thanks!

I uploaded the conda package of pygslib v <IP_ADDRESS> for mac, linux and windows for Python 3.8. Please try and let me know if it works. This conda package has libgfortran as a dependency.


NP-8839 Only Show Sections with Titles

For some reason, the sections array has an extra entry at the end that is 'visible' but it is just the data for the entity and not actually a section. The way we distinguish whether to show a section is if it has a title, so I'm adding that to the filtering.
This fixes: https://nanopay.atlassian.net/browse/NP-8839

So this is in addition to permissions?
Or is the issue that the section doesn't have a title?

Why is there a blank section? Could we just fix things so that it isn't there?

Agree with @kgrgreer we shouldn't even have the section with empty title, should investigate why that exists