[Enhancement request] SpringWolf to be compatible with Reactor Kafka

Hello team,
Came into this project from this talk:
https://www.youtube.com/watch?v=DylvTW_ia4Y
If nothing else, this project seems to be very cool, very interesting.
We are using Kafka, and want to integrate with SpringWolf.
However, we are not using a regular Spring Kafka, but using reactor kafka https://github.com/reactor/reactor-kafka
Same question as in the talk around minute 33, can this project support reactor kafka?
Thank you

Hi @KafkaProServerless, thanks a lot for your interest in Springwolf and watching the video.
Springwolf heavily relies on Spring Framework and therefore we do not plan nor have the capacity to support other frameworks.

Hello @sam0r040, thank you for the answer.
But reactor kafka is also relying on Spring Framework.
It is not like reactor kafka is something that has nothing to do with spring kafka, it is just the reactor version of it.

Hi @KafkaProServerless,
if you have Spring ecosystem available, then you can use Springwolf. We can imagine that the following could work for you:
Given the example consumer from ractor-kafka, you can use Springwolfs generic annotations like this:
    @AsyncListener(
            operation = @AsyncOperation(
                    channelName = "kafka-topic-name"
            )
    )
    private void receiveRecord(CountDownLatch latch, @Payload ReceiverRecord<Integer, PayloadDto> record) {
    }

    record PayloadDto(String myProperty) {
    }

We extracted the lambda into a method so that you can make use of the @Payload annotation so that Springwolf can automatically detect payload type.
Don not forget to configure extractable classes for ReceiverRecord to that the payload is extracted correctly from the second generic type:
springwolf.payload.extractable-classes.reactor.kafka.receiver.ReceiverRecord=1
See https://www.springwolf.dev/docs/configuration/documenting-consumers#asynclistener for more details.

Let me try that!

Hello!
I am glad to have found this post, as I am trying to accomplish the same thing.
Actually, I would like to help the community by adding an example, for the integration between reactor kafkaesque and Springwolf!
However, I am having difficulties making this work as well.
I prepared a minimal reproducible example, something which can be pasted to the example folder later.
This minimal example, with just 5 files, is what I am targeting to achieve.
https://github.com/patpatpat123/springwolfreactorkafka/tree/main
The first file is the pom:

As you can see in the pom, there is the dependency to reactor Kafka. Not spring kafkaesque, not spring cloud stream, but just springbok and reactor Kafka!
As mentioned, there won't be any kafkalistener annotation, or any spring cloud stream annotation, yet, it is a plain spring based project.
Also, in the pom, there are the dependencies springwolf-core springwolf-kafka springwolf-asyncapi to all at version 1.6.0
Are those dependencies enough? Too much?
Now, looking at the code: there are just three java files.
1 - Is the "main" file, the SpringBootApplication file, which has nothing interesting. -> SpringWolfReactorKafkaApplication.java
2 - is the configuration file, where you can see the "reactive configuration" KafkaReceiver.create(receiverOptions.subscription(Collections.singleton("the_topic"))) Here, it differs from the traditional Spring Kafka. By the way, please feel free to replace the kafkaesque broker, and the topic name with your information, this projects should work "out of the box" -> SpringWolfReactorKafkaConsumerConfiguration
3 - this most important file, the actual reactive processing:
https://github.com/patpatpat123/springwolfreactorkafka/blob/main/src/main/java/org/example/SpringWolfReactorKafkaService.java#L21
    @Override
    @AsyncListener(operation = @AsyncOperation(channelName = "the_topic"))
    public void run(final String... args) {
        kafkaReceiver.receiveAutoAck().concatMap(message -> message).flatMap(this::processMessages).subscribe();
    }

You can see here the AsyncListener suggested above. The code is really just a reactive consumer which consumes from a topic reactively.
Finally, a properties files: https://github.com/patpatpat123/springwolfreactorkafka/blob/main/src/main/resources/application.properties#L1
With this example, I was topic to also see the UI in the Youtube presentation, as well as some of the features presented there.
Could you guys please help on pointing out what is missing?
This would be a nice addition to the example folders
Thank you!


MINOR: Modify unnecessary access specifiers

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@chia7712
May I ask for a review of this PR?

Ping
@chia7712
May I ask for a review of this PR?

@chia7712
Thank you for the comment Code was modified

@chia7712
Thank you for the comment Code was modified


Update k8s version for kind to 1.25.3

0f2f044df10ab6eb513a0a75d6b76577d80415d6 sets the defaultMinimumVersion to 1.24.0:
https://github.com/knative-sandbox/eventing-kafka-broker/blob/0f2f044df10ab6eb513a0a75d6b76577d80415d6/vendor/knative.dev/pkg/version/version.go#L36
This leads to a CrashLoopBackof of the kafka-broker-controller, when the kind cluster was created with the ./hack/create-kind-cluster.sh skript, as this is based on 1.23.3 currently.
...
{"level":"fatal","ts":"2022-11-17T11:45:37.451Z","logger":"kafka-broker-controller","caller":"sharedmain/main.go:368","msg":"Version check failed","commit":"440801d","knative.dev/pod":"kafka-controller-7b566dd9dd-4jzjl","error":"kubernetes version \"1.23.3\" is not compatible, need at least \"1.24.0-0\" (this can be overridden with the env var \"KUBERNETES_MIN_VERSION\")","stacktrace":"knative.dev/pkg/injection/sharedmain.CheckK8sClientMinimumVersionOrDie\n\tknative.dev/pkg@v0.0.0-20221117013641-f5c1a03ab4f1/injection/sharedmain/main.go:368\nknative.dev/pkg/injection/sharedmain.MainWithConfig\n\tknative.dev/pkg@v0.0.0-20221117013641-f5c1a03ab4f1/injection/sharedmain/main.go:256\nknative.dev/pkg/injection/sharedmain.MainNamed\n\tknative.dev/pkg@v0.0.0-20221117013641-f5c1a03ab4f1/injection/sharedmain/main.go:161\nmain.main\n\tknative.dev/eventing-kafka-broker/control-plane/cmd/kafka-controller/main.go:61\nruntime.main\n\truntime/proc.go:250"}

https://github.com/knative-sandbox/eventing-kafka-broker/blob/7574def47021a166bb39553e81eb5eec75d89b4c/hack/create-kind-cluster.sh#L23-L24
Proposed Changes

:bug: Bump the k8s version for kind to 1.25.3
:broom: Bump kind version for CI to v0.17.0

Release Note

/test reconciler-tests-namespaced-broker
/test integration-tests
(seems like a maven central connection problem: Failed to execute goal com.mycila:license-maven-plugin:4.1:check (check-license) on project data-plane: Execution check-license of goal com.mycila:license-maven-plugin:4.1:check failed: Plugin com.mycila:license-maven-plugin:4.1 or one of its dependencies could not be resolved: Could not transfer artifact javax.enterprise:cdi-api:jar:1.0 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar: Connection reset -> [Help 1])


Fix failing action

Description
Fixes failing action by removing the dependency on the only job in the pr_merged workflow.

Its because this is the only job in the workflow, which makes no sense why it would have other jobs listed as a dependency. You can see the failure here: https://github.com/newrelic/gatsby-theme-newrelic/actions/runs/175992749

The workflow is not valid. The workflow must contain at least one job with no dependencies.


group by count

Hi,
I was looking for a solution, but the only thing I found was #856
A very similar problem to mine.
I would like to count the groups, but the counter is not in the structure.
type table struct {
Colname string
}

# select colname, count(*) FROM "table" GROUP BY colname;
   colname   | count
-------------------+-------
a   |    17
b   |     1
c   |     1

I tried something like that:
type table struct {
Colname string
Count int `xorm:"-"`
}

var tables []table
engine.GroupBy("colname").Select("colname, count(*) AS count").Find(&tables)

However, the count field has not been filled with values.
And I tried something like that:
type table2 struct {
table
Count int
}

var tables []table2
engine.GroupBy("colname").Select("colname, count(*) AS count").Find(&tables)

panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
And:
count, err := engine.Table("table").GroupBy("colname").Select("colname, count(*)").Count()

SELECT colname, count(*) FROM "table" GROUP BY colname
[ERROR] [sql: expected 2 destination arguments in Scan, not 1]
Ideally, it would be something like that:
mycounts := make(map[string]int)
mycounts, err = engine.Table("table").CountByGroup("colname")
//SQL: select colname, count(*) FROM "table" GROUP BY colname;

type table2 struct {
table `xorm:"extends"`
Count int
}

Thank you @lunny, your solution works like a charm.
In the meantime, I did something like this:
func countGroups(engine *xorm.Engine, tableName, groupByColumn string) (map[string]int, error) {
	rows, err := engine.
		Table(tableName).
		GroupBy(groupByColumn).
		Select(groupByColumn + ", count(*) AS count").
		Query()

	if err != nil {
		return nil, err
	}

	res := make(map[string]int, len(rows))
	for _, row := range rows {
		i, err := strconv.Atoi(string(row["count"]))
		if err != nil {
			return nil, err
		}
		res[string(row[groupByColumn])] = i
	}
	return res, nil
}

It would be great to have such mapping in xorm session.
map[string]int := engine.
Table(tableName).
GroupBy(groupByColumn).
CountGroups()

Or something similar.


Set default S3Region when not provided

After Fix:
- s3profilename: s3profile-remote-cluster-ocs-storagecluster
  s3bucket: odrbucket-9af550a2d48c
  s3compatibleendpoint: https://s3-openshift-storage.apps.odf-dade9868-20fb-471e-bcd0-3fefc21db2bf.devcluster.openshift.com
  s3region: nooba
  s3secretref:
    name: 2152428ad05a47e326a4b6ae6adfcaa21f2896c
    namespace: openshift-dr-system

Signed-off-by: Gowtham Shanmugasundaram<EMAIL_ADDRESS>

Change commit title MCO operator is setting "nooba" as default s3region to Set default S3Region when not provided.

done

Change commit title MCO operator is setting "nooba" as default s3region to Set default S3Region when not provided.

done

edit PR title also?

Change commit title MCO operator is setting "nooba" as default s3region to Set default S3Region when not provided.

done

edit PR title also?

done

/cherry-pick release-4.10

/retest


Add bash auto completion support

Adds auto completion support that's generated via:
$ kompose complete

This file is added to /etc/bash_completion.d/kompose.sh

Closes https://github.com/kubernetes-incubator/kompose/issues/37

Coverage remained the same at 47.007% when pulling 15a1946fd7becabf2ec8f31a64b2cc10a53ee63b on cdrage:add-autocompletion into 2ba88312e169634f71d4aedf910c8dba888ea003 on kubernetes-incubator:master.

@ngtuna It should. Doesn't look like you're running as sudo / have user permission to /etc/bash_completion.d/

Ah see it now... It doesn't work on Mac. Could we add this support to darwin ?

works on fedora for me :+1:

Can we do it like kubectl or oc? Print to stdout instead to file? (https://kubernetes.io/docs/user-guide/kubectl/kubectl_completion/) or at least have option for it.
Second thing is that I don't think that it is no nice write to /etc by default without any warning.

And one more. Some people are using ZSH (like me for example :wink: )

@kadel Alright :) I'll do these three things:

Output to stdout
Add the ZSH option (btw @kadel zsh > bash. #zshmasterrace)
Add support for Darwin

@ngtuna @kadel
Ready for another review!
It now:

Outputs to stdout :+1:
Added bash support :+1:
For the users who use a laptop with a magical touchbar that removes the ESC key, you can simple source it with source <(kompose completion bash). :+1:

@kadel
Tests should be all green now (validate error with some unreachable code).
I've also updated the README with the appropriate instructions on how to use this!
Merge when you're ready :+1:

LGTM @ngtuna can you confirm?

Yes. Confirmed


Ignore ENV.update calls in Rails/SaveBang

We can't detect Hash#update false positives in the general case, but when the receiver is ENV we know it's not an Active Record instance.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[x] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[x] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

Make sense 🌟

Wow that was fast 😄 thanks for the review as always @koic 🙇


Bug while feeding input image argument from web

This block should be sorted: https://github.com/serengil/retinaface/blob/master/retinaface/commons/preprocess.py#L29
With similar approach of sorting this bug: https://github.com/serengil/deepface/issues/1023

Closed with PR - https://github.com/serengil/retinaface/pull/83


chore(cli): extract vite config as a submodule to the library

What?
This PR extracts the duplicated vite plugins as @storyblok/field-plugin/vite.
Why?
JIRA: EXT-1875

How to test? (optional)

@johannes-lindgren something we talked about. pinging you for visibility and in case you have another feedback


Add more cancellation info

What's this PR do?
It fills in some info on recurring donation cancellations that the membership team have asked for, namely...

adds a cancel date to the RDO object (no matter what the cause)
adds a cancel date to the Contact object (when cancelled directly by a member)
adds a cancel source (either "Member Portal" when triggered directly by member or "Staff" otherwise)
messages the #bot-cancellations channel when a donation is cancelled

Why are we doing this? How does it help us?
This will give more knowledge and awareness around recurring donation cancellation patterns and habits.
How should this be manually tested?

On staging, add a new donation here using the email you most often log into the portal with
On staging, add another new donation using a different email that you don't often use to log into the portal

Cancelling your own membership:

Log into the portal using the same email from step 1 and click memberships
You should see the donation you made earlier listed, but if not, give it some time and refresh after a bit
Cancel the donation you made earlier (no need to fill out the reason form)
Bounce over to salesforce and find the recurring donation you just cancelled (quickest way is to search by the email you used and a table of recurring donations tied to that email should be the first thing you see)
When you're on that record's page, you should see two fields...

"Cancellation Date" should have today's date
"Cancellation Method" should say "Member Portal"

Over in slack land, a message should have populated on the #test-tech channel about the cancellation

Cancelling another member's donation:

Still in the membership portal, use the tool that allows you to view as another member and enter the email you used in step 2 (above)
From here largely follow the same steps for this donation as you did the last one with one caveat...

"Cancellation Method" should say "Staff"

How should this change be communicated to end users?
We'll let the membership team know when it's done.
Are there any smells or added technical debt to note?
There's probably more discussion to be had around the date field on Contact... if a member cancels their donation and a week later gives a new recurring donation, do we need to wipe that field? How about if there are two rdos on one customer record (not likely but can happen)?
What are the relevant tickets?
https://airtable.com/appyo1zuQd8f4hBVx/tbloNZu8GkM52NKFR/viwS1XPty68eK4Ett/recDfgKWcBEkyfuZH?blocks=hide
Have you done the following, if applicable:
(optional: add explanation between parentheses)

[ ] Added automated tests? ( )
[ ] Tested manually on mobile? ( )
[ ] Checked BrowserStack? ( )
[ ] Checked for performance implications? ( )
[ ] Checked accessibility? ( )
[ ] Checked for security implications? ( )
[ ] Updated the documentation/wiki? ( )

TODOs / next steps:

[ ] your TODO here

Test steps all WFM! I left some notes, but nothing seems particularly blocking.
From your PR note here...

There's probably more discussion to be had around the date field on Contact... if a member cancels their donation and a week later gives a new recurring donation, do we need to wipe that field? How about if there are two rdos on one customer record (not likely but can happen)?

... I feel like when I donated as the same account, they all got counted as separate RDOs and then as I cancelled each, through portal, staff, and stripe, they were all handled separately with the right values. That seems like exactly what we want in my head, but I could also be testing for the wrong scenario.
Speaking of which, cancelling through Stripe is one where I noticed nothing came through for the "method." Doesn't seem like the end of the world, but if it's easy enough to add some better label there that might be nice.
All in all, the observability here is a great improvement!

Oh, that great news. Thanks for the review!


Added dot notation map access to index func

Resolves #3289

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Michael Black seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

@blackmichael this looks great, I will merge once the tests go green. And again, sorry for the delay.


Fix typos

Issue #, if available:
Description of changes:
Fix typos in Makefile.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve
/woof


AddedChlidren

I added Children because alot of children that I know want to go into tech use this website too!

[ ] I have read freeCodeCamp's contribution guidelines.
[ ] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the master branch of freeCodeCamp.
[ ] None of my changes are plagiarized from another source without proper attribution.
[ ] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @marcus-campbell-codes, thank you for opening this pull request.
This is a standard message notifying you that we’ve reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you and happy coding.


module not found: org.scala-sbt#sbt;1.1.5

I installed the sbt today and it successfully installed. However, when I tried to run sbt, I got following issue.
:: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
unresolved dependency: org.scala-sbt#sbt;1.1.5: not found
Error during sbt execution: Error retrieving required libraries

sbt is on Maven Central, so something must be wrong with your connection to Maven Central, perhaps?
https://repo1.maven.org/maven2/org/scala-sbt/sbt/1.1.5/


trackpoint: fix typo in trackpoint speed description

Please git amend the actual commit title as well, as that's what will end up in the history.

okay, done
On 10 Jan 18:32, Tobias Geerinckx-Rice wrote:

Please git amend the actual commit title as well, as that's what will end up in the history.

Reply to this email directly or view it on GitHub:
https://github.com/NixOS/nixpkgs/pull/12315#issuecomment-170418650


An alternative solution to #219

See #219. Mutually exclusive

cannot-parse.zip

Cherry-picked to #219


"Github" theme - some character colors are nearly indistinguishable from background

Hi,
Perhaps its just on my system, or my eyes are different from others', but I really struggle with distinguishing three of the character colors from the background, in the "Github" theme:

The offending colors are

#F8EEC7
#FFFFFF

Perhaps some other colors should be selected for the Github theme?

yea, Accessibility is a big problem in various themes.
https://gogh-co.github.io/Gogh/wcag/
I'm going to close this issue because we already have this:
https://github.com/Gogh-Co/Gogh/issues/352

I'm not quite sure how to solve this matter either, especially when it comes to themes that come from a third party, like the majority of contributions on Gogh.


MAINT: check for negative weights in Rotation.align_vectors

Negative weights should not be permitted in Rotation.align_vectors.
Here I have added a check for this.

Thanks @tupui


Release 1.0.0

Successfully ran acceptance test for beaker sets:

centos-72-x64
centos-66-x64
fedora-25-x64
fedora-24-x64

Additionally successfully tested with centos/6 and centos/7
distribution upstream vagrant boxes and the same beaker test.
Closes #184
Included commits since 0.8.0 release not directly in this PR:
510c303 (GH-148) Add spec tests for class parameters
de69bf8 Use correct type in create_resource with permissive parameter
b9041d2 (GH-148) Make use of class selinux parameters
a25d44d (GH-119) Don't accept udp6 and tcp6 as protocol name with selinux::port
5744329 Fix for Security/YAMLLoad rubocop warning
211de87 Adapt required rubocop config change
0467ea8 modulesync 0.19.0
0bf10cc Fix broken link to puppet strings documentation
699e6b2 Generated puppet strings docs
0f83a6f Provide resource ordering UML diagram
d12a836 Convert inline docs to puppet-strings format
c0f1168 Add redcarpet gem needed for puppet strings
65ecdde Generate puppet strings in docs/ folder
7fd977a Document known problems / limitations
c34f99c [blacksmith] Bump version to 0.8.1-rc0
3442761 modulesync 0.18.0
a304af6 bump required puppet version to 4.6.1
4f99a3d (GH-147) Test default resource ordering in acceptance test
b306f49 (GH-147) Add spec tests for the ordering of resources
8116a4f (GH-147) Add ordering of resources

rebased to latests master

Seems fine to me at least. Let's get this out of the way and start breaking things. 😛

Do we want to support puppet3 with this release? The CHANGELOG.md for the last release says that that one was the last with puppet3 support.

we dropped all the puppet3 stuff on master with the last modulesync. if 1.0.0 should support puppet3 we have to sync again IMO.

I would be happy if 1.0.0 supports only puppet4

I read the changelog as saying it will start to introduce some actually-breaking changes soon but it hasn't yet, even though it's puppet 4 only from a testing standpoint.

yes 1.0.0 should support puppet3, breaking changes will be in 2.0.0 as discussed in #184 .
but it seems the CHANGELOG needs clarification.

will not be release as 1.0.0. will be some 0.9.0 release.


Missing ssh-agent in 3.5

which ssh-agent just returns ssh-agent not found and it is not in /usr/local/bin/ssh-agent

you need to install the openssh-client package: apk add --no-cache openssh-client


mambabuild: support for anaconda tokens

Similar to https://github.com/mamba-org/mamba/pull/370
Seems like a simple change to port, but I'm not sure about token leaks elsewhere in the code.

I think that would be fine, yes! If you want to open a PR that would be great!

I tried using the get_index from mamba.utils but it seems like the solver still can't find the packages. The solver cache does seem to have the packages there, so it looks like this change is a little bit harder than I initially thought.


IMAGING-238: Return copied byte arrays in PNG Chunk and ICCP PNG Chunk

Returns copied byte arrays, adds javadocs, and also adds a unit test for the ICCP chunk.

Coverage increased (+0.1%) to 74.743% when pulling f92618c71bf5160f8ec4c796983e76548efd150c on kinow:fix-todos-pngchunks into f2aed4ca6d795c03ed241a83b176467706fda9e2 on apache:master.


hide race/ethnicity/indigenous sections for IRONMAN patients

address this story: https://www.pivotaltracker.com/story/show/144417951
remove race/ethnicity/indigenous status sections from profile for IRONMAN patients

Paul, I see the shortcoming of referencing the organization (in this case IRONMAN) by ID, since that can potentially change.  I fixed it.  Let me know if you see other issues with my fixes.

closing this for now,  will incorporate call to API to do this check


Unable to Install Plugin on Logseq marketplace

Error message displayed:
"[Error] :release-channel-issue "request to https://api.github.com/repos/briansunter/logseq-plugin-gpt3-openai/releases/latest failed, reason: getaddrinfo ENOTFOUND api.github.com"]"

I tried again today and it works. I was able to install the plug-in.


Web show blank white screen when deploy new code

Amplify CLI Version
1
Question
I have vue3 project and use amplify to deploy, I had a problem that every time I deploy new code and when I access website again, it often show blank screen page, I need clear cache storage or reload 3 times to get content of my sites.
This is my config
import { defineConfig } from 'vite'
import vue from '@vitejs/plugin-vue'
import { VitePWA } from 'vite-plugin-pwa'
import { resolve } from 'path'

export default defineConfig({
  plugins: [
    vue({
      template: {
        compilerOptions: {
          isCustomElement: (tag) => ['ion-icon'].includes(tag),
        }
      }
    }),
    VitePWA({
      base: '/',
      includeAssets: ['img/**/*.*', 'fonts/**/*.*'],
      useCredentials: true,
      manifest: {
        name: 'My App',
        short_name: 'MyApp',
        start_url: '/',
        display: 'standalone',
        background_color: '#fff',
        theme_color: '#fff'
      }
    })
  ],
  server: {
    port: 5000,
    host: '<IP_ADDRESS>',
    hmr: {
      host: 'localhost',
      port: 5000
    }
  },
  build: {
    chunkSizeWarningLimit: 100,
    manifest: true
  },
  resolve: {
    alias: {
      '~': resolve(__dirname, 'src')
    }
  }
})

Amplify
version: 1
applications:
  - appRoot: client
    frontend:
      phases:
        preBuild:
          commands:
            - npm install
            - npm ci
        build:
          commands:
            - npm run build
      artifacts:
        baseDirectory: dist
        files:
          - '**/*'
      cache:
        paths:
          - node_modules/**/*

It seem it does not cache my content ? everytime it fetch all file ?

Hey,👋 thanks for raising this! I'm going to transfer this over to our backend repository for better assistance on Amplify hosting 🙂


nth root function

How about extending the library with a nthroot function... Currently I'm extending mine with:
P['nthrt'] = P['nthroot'] = function (n)
{
    var x = this, negate, approx;

    // Convert n to a Big
    n = new Big(n);

    // Checking to see if root is negative
    negate = n.mod(2) == 1 && x['s'] == -1;

    if (negate)
        x = x.times(-1);

    // Initial nth root result (nth root = x to the power of 1 / n)
    approx = x.pow(Big(1).div(n));

    // Validating result & return
    n = approx.pow(n);
    if (x.minus(n).abs(x - n) < 1 && (x > 0 == n > 0))
        return negate ? approx.times(-1) : approx;
};

Calculation is based on the contents of an article at http://cwestblog.com/2011/05/06/cube-root-an-beyond/

@raybiez You must wonder why you're getting an update on a 3 year old post, but this saved my butt. I'm converting a bunch of stuff in my code from JavaScript numbers to using Big instances, and I realized there was one module where we were calculating a cubic root. The precision didn't need to be high, but the input could certainly be beyond JS's safe maximum number. Very relieved to have come across this, works for me!

big.js does not support .pow(1/3)
however, it's cousin lib ... decimal.js-light ... does
To get around this painful aspect of big.js I've used the 2 libs in tandem as follows:
const Big = require('big.js')
const Decimal = require('decimal.js-light')

const nthRoot = (bigNumber, intRoot) => {
  const strBigNumber = bigNumber.toFixed()
  const decimal = Decimal(strBigNumber)
  const root = decimal.pow(1 / intRoot)
  return Big(root.toFixed())
}

module.exports = nthRoot


SX-Prop: fontSize with suffix not working since v5.11.13

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Link to live example:
not working from v5.11.13 until latest (v5.11.14): https://codesandbox.io/s/elated-violet-32fbb1?file=/demo.tsx
working example until v5.11.12: https://codesandbox.io/s/shy-dust-znclfb?file=/demo.tsx
Current behavior 😯
sx={{ fontSize: "h1.fontSize" }} not working anymore, while sx={{ fontSize: "h1" }} works.
Expected behavior 🤔
sx={{ fontSize: "h1.fontSize" }} should work like sx={{ fontSize: "h1" }} until a breaking change was intended.
Context 🔦
I noticed that the behaviour of the SX-Prop "fontSize" has changed. I think this was not intended by @mui, like the previous prefix-bug, but now with the "fontSize"-suffix.

  Output from `npx @mui/envinfo` goes here.

System:
OS: macOS 13.2.1
Binaries:
Node: 18.14.2 - /usr/local/bin/node
Yarn: 1.22.19 - /usr/local/bin/yarn
npm: 9.5.0 - /usr/local/bin/npm
Browsers:
Chrome: 111.0.5563.64
Edge: Not Found
Firefox: Not Found
Safari: 16.3
npmPackages:
@emotion/react: 11.10.6 => 11.10.6
@emotion/styled: ^11.10.6 => 11.10.6
@mui/base:  5.0.0-alpha.122
@mui/core-downloads-tracker:  5.11.14
@mui/icons-material: ^5.11.11 => 5.11.11
@mui/material: ^5.11.14 => 5.11.14
@mui/private-theming:  5.11.13
@mui/styled-engine:  5.11.11
@mui/system: ^5.11.14 => 5.11.14
@mui/types:  7.2.3
@mui/utils:  5.11.13
@mui/x-date-pickers: ^6.0.2 => 6.0.2
@types/react: ^18.0.28 => 18.0.28
react: ^18.2.0 => 18.2.0
react-dom: 18.2.0 => 18.2.0
typescript: ^4.9.5 => 4.9.5

Thanks for the report, some time ago we disabled the regressions tests for the system assuming the unit tests are enough, which is clearly not true. I will enable the tests and revert some of the changes we've introduced in the system's typography.


Better windows support + s3:// asset URL support

This pull request fixes a bug for windows users and adds a feature in a related area. I can split this down into multiple pull requests if you just want the bug fix.
On windows, assetS3Url and templateS3Url output with backslashes for the S3 URLs because the code was using path.join which is OS specific. This has been changed to use url.resolve as the output for these helpers is always a URL and should always be a forward slash.
The related feature is also a change to assetS3Url. We are using condensation to template some shell scripts for EMR bootstrapping. In one of those steps we need to reference other assets but using an s3 url rather than http. s3://bucketname/path. To support this we have added an option to assetS3Url: {{{assetS3Url 'setup_drill.rb' protocol="s3"}}}. If protocol==s3 it returns an s3:// url, otherwise the existing behavior is used.

Thank-you for the url patch and I like the implementation for the s3 protocol.
Would you be comfortable creating a test for the s3 protocol implementation?

Sure, i'll add that later today

Tests added and pushed. Had to do these as unit tests rather than end to end tests because I needed to simulate a windows and linux path separator. If i did an end to end test it would only test the os that the test runner is executing on.
I noticed that the project isn't setup to run tests recursively because your fixtures are in the test directory so i added a unitTests.js file in the root of tests for now. I think the right thing to do eventually is to move the fixtures so you can enable recursive in mocha.

@techiedarren please take a look at pull request #15.  Instead of creating multiple strings to track the https/s3 url I switched to using a url object.  Tests are updated and passing.  Want to make sure this is working for you before creating the next release.
Thanks again for the feature and the tests!


What Are the Training Time and Space Complexities for Mamba vs. Traditional Transformers?

Hi!
I've been exploring the Mamba architecture with great interest, especially its computational efficiencies compared to traditional Transformer models. The selective state space approach and the potential for handling long sequences efficiently are particularly intriguing.
However, I would appreciate some clarification on the exact time and space complexities associated with different operations within Mamba compared to Transformers. Specifically, I'm interested in understanding the complexities related to training, autoregressive inference, and the memory requirements for maintaining the model's state.
From the available documentation and discussions, I've gathered that:

Operation Type
Mamba Complexity
Transformer Complexity

Training (per iteration)
O(L)
O(L^2)

Autoregressive Inference
O(T) per step
O(L) per step

Memory Requirements
O(C)
O(L)

Where:

L stands for the sequence length.
T denotes a fixed constant that accounts for compression and selection time in Mamba's autoregressive inference.
C reflects the fixed size of the SSM (State Space Model) latent state in Mamba.

Are this correct? Could you provide more detailed insights into how these complexities are derived and any underlying assumptions?
Thank you for your pioneering work on Mamba, and I look forward to your insights and further clarifications on these questions.
Best regards

Yes, these are correct. Please check out the paper or the many blogs/videos/tutorials on the topic for more details.


refactor: Enable paste lnurl

Addresses issue #316
Send Bitcoin screen was able to handle lnurl from scanned QR code but was not able to handle user pasting lnurl directly.

added ability to paste lnurl from send bitcoin screen
added clipboard icon to bottom of QR code, if user has copied it to clipboard and taps the clipboard it will paste and transition to send bitcoin screen to confirm, similar to Wallet of Satoshi and Strik

https://user-images.githubusercontent.com/17483038/151304524-8c7487ab-3a25-4be0-8e32-1bac5f83c890.mov

Hey @danwebb1 .  I think this PR looks good.  I haven't had time to check it out to test yet unfortunately.  My only question would be does the clipboard modal pickup that a lnurl is in the clipboard and prompt the user?  Code is here https://github.com/GaloyMoney/galoy-mobile/blob/b626cd4e757e9da7fb45699b98a2444f67f3095b/app/utils/clipboard.ts#L19.

Hey @danwebb1 . I think this PR looks good. I haven't had time to check it out to test yet unfortunately. My only question would be does the clipboard modal pickup that a lnurl is in the clipboard and prompt the user? Code is here
https://github.com/GaloyMoney/galoy-mobile/blob/b626cd4e757e9da7fb45699b98a2444f67f3095b/app/utils/clipboard.ts#L19
.

Yes, it does pick up and alert the user. Additionally, I added slightly more specific error message to Alert on QR screen. If a lnurl-withdraw/auth/channel request is pasted instead of lnurl-pay it says that the its not currently supported instead of sayings its not a valid address.


The "s.platform" in podspec

I found the "s.platform" in podspec was 6.1, what about other platform?  Such as 6.0, 5.x and so on.
Any idea?

This is because SVProgressHUD, if you want to use in older version such a 5.x just download source and include without cocoapods, or download older version in pods

Many thanks!!!   I don't use SVProgressHUD, however, in order to deploy my project to IOS platform 6.0, can I use the newest version in pods, like "  pod 'MZFormSheetController', '~> 3.0.2'  " ???

No you can't but version 3.0 introduced new way presenting popup which is supported only by ios8, nothing changed

To support IOS6.0, I use " if ..... else...... " to solve problems between ios8 and other version, running normally! May I use MZFormSheetController like this below?

I just only use MZFormSheetPresentationController on ios8, and other version such as 6.0, 6.1 and so, I use MZFormSheetController

Yes you can use it in that way, dependency is 6.1 because pod didn't allow me to set lower because SVProgressHUD, but it will work on iOS 5.x

I see, many many many thanks! By the way, if spliting this pod into two pods, one is "MZFormSheetController", and the other one is "MZFormSheetController+SVProgressHUD", the first one is to be set lower ......


[Feature] Metis - Pin 📌 and archive 📂 posts as tutor

Checklist

[x] I tested all changes and all related features with different users (student, tutor, editor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[x] Server: I followed the coding and design guidelines.
[x] Server: I added multiple integration tests (Spring) related to the features (with a high test coverage)
[x] Server: I added @PreAuthorize and check the course groups for all new REST Calls (security)
[x] Server: I implemented the changes with a good performance and prevented too many database calls
[x] Server: I documented the Java code using JavaDoc style.
[x] Client: I followed the coding and design guidelines.
[x] Client: I added multiple integration tests (Jest) related to the features (with a high test coverage)
[x] Client: I added authorities to all new routes and check the course groups for displaying navigation elements (links, buttons)
[x] Client: I documented the TypeScript code using JSDoc style.
[x] Client: I added multiple screenshots/screencasts of my UI changes
[x] Client: I translated all newly inserted strings into English and German.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.

Motivation and Context

Within the discussion section on an exercise or lecture page there might be a bunch of posts listed in a row. Some of those discussions might be super important for everyone, e.g. "Are we supposed to use Mockito as mocking framework? -> Tutor answer: Yes you are." and some might be not relevant anymore, e.g. "There is a mistake in the UML in class XY." (and this problem was already solved).
Following needs were identified:

Tutors should have the ability to pin posts by clicking the dedicates emoji button (push-pin)-> this post is listed at the top of the discussion section
Tutors should have the ability to "archive" posts by clicking the dedicates emoji button (folder)-> this post is listed at the end of the discussion section

Description

I used dedicated emojis that serve as buttons for tutors, the push-pin and the folder. Clicking these buttons will invoke some method calls that update the post and flip the pinned or archived property of a post object.
The property state (true or false) is used for sorting the posts in the discussion section.
The server-side service methods are implemented in a way, that the post is either pinned or archived.
Tutors will see both of the emoji-buttons, if one is activated (by any tutor), the button is highlighted.
Students will only see the respective emoji in the list of emoji reaction, if the post is pinned/archived.
Tooltips with inform both user groups about the button/emoji state and its implications.
This PR includes a migration as the Post properties pinned and archived are new and have to be added as boolean columns to the respective database table.
Steps for Testing

The testing steps should be conducted for an exercise and a lecture, but you could also decide to test one or the other, as I provide separate checkboxes in the review section.
You could use my  test-course on TS1 for testing -> manual testing then consists of only a few clicks 😉
instructor

Log in to Artemis as tutor or instructor (e.g. user 15 in my course)
Navigate to Course with posting enabled
Navigate to an exercise page with existing posts, e.g. one of my exercise on TS1 or to a lecture page, e.g. one of my lectures on TS1.
Check the tooltips that appear to you as tutor when hovering over the emoji buttons to pin and archive a post -> do they sound correct, does it make sense to you?
Pin a post by pressing the push pin emoji -> check if this post is put at the top of the list (note: if there is another pinned post with more + emoji reaction, this one comes first; if there is another pinned post that was created first, this one comes first)
Archive a post by pressing the folder emoji -> check if this post is put at the end of the list (note: if there is another archived post with that was created later, this comes last)

student
It makes sense that you stick to the exercise/posts that you previously pinned/archived when you tests the PR as tutor/instructor.

Log in to Artemis as student (e.g. user 1-3 in my course)
Navigate to Course with posting enabled
Navigate to an exercise page with existing posts, e.g. one of my exercise on TS1, where at least one of them is pinned and one of them is archived.
Check the order of posts in the discussion section, are the pinned posts first and the archived posts last?
Check the tooltips that appear to you as student when hovering over the emojis for the pinned and archive post -> do they sound correct, does it make sense to you?
Check if posts that were not pinned/archived before do not have the pin/archive-emoji shown within the reaction bar

Review Progress

Code Review

[x] Review 1
[x] Review 2

Manual Tests

[x] Test 1 -> as student and instructor for an exercise
[x] Test 2 -> as student and instructor for an exercise
[x] Test 3 -> as student and instructor for a lecture
[x] Test 4 -> as student and instructor for a lecture

Test Coverage

postings-reactions-bar.component.ts: 85%
post-reactions-bar: 100%
post.service.ts: 91%
metis.service.ts: 91%
Screenshots

This is how a tutor would use this feature (as described in the test steps)
https://user-images.githubusercontent.com/41366522/129748132-587655b3-278c-4ae8-9c5b-556185a13b0e.mov
This is how a student would get see pinned/archived posts. In the video, the student reacts with a plus (actual emoji reaction button) and hovers over the pin emoji, which is not clickable.
The archived post is at the bottom, as expected, whereas the pinned post is at the top (even above the post with several + reactions).
https://user-images.githubusercontent.com/41366522/129748326-40401dec-8ea9-4b45-8222-1a19ed0b7373.mov

@imstreetspirit I'll address it in a follow up, added it to the list of to-dos :) thanks for testing!

@MaisiKoleni @jpbernius I implemented the suggested changes in a1c55d2, which also involves the change of the migration file. I guess we need to clear the checksums on the test servers then again.

@jpbernius @MaisiKoleni I like the suggested change of modeling pin/archived as enum! I will re-work the PR accordingly and let people retest the functionality.

The only thing I am not sure is intended is that the "archived"-icon for students is the greyed-out version of the icon, whilst when pinning it's a red pin. This doesn't hurt the functionality in any way but I just wanted to make sure you are aware of it ^^'

Thanks al lot for testing! it's the inherent color of the icons... as a pin should raise and archive lower awareness, I think it's suitable. Let's see how the users react to it :)


Allow non-sRGB views of internal camera textures

Objective

Compute shaders can be a more performant way to implement full-screen post-processing passes, but they cannot use sRGB textures as output storage textures, so a non-sRGB view has to be created first.

Solution

Allow non-sRGB views for the camera internal textures.

Hmm, seems like not all WebGPU implementations have fully implemented this part yet. I've also noticed that a bit more work is required to make compute shaders actually work (the base texture itself has to be non-sRGB), so I'll close this for now.

Ah, it's WebGL that doesn't support this. It does support compute shaders though (apparently!), so not sure what you're meant to do there.
For now I can easily get by with a fragment shader, but GPU vendors tend to recommend using compute shaders for fullscreen passes as they're potentially faster, so they would be nice to support in bevy for this use case.

Ah, it's WebGL that doesn't support this. It does support compute shaders though (apparently!)

WebGL(2) doesn't support compute shaders. There was an attempt to add them to it at one point (iirc by intel), but it was never completed and afaik no browser ever implemented support for it.
WebGPU does support compute shaders.


Container ip override

Thank you very much for the contribution,
unfortunately it does not seem to work that way out of two reasons:

If the override logic resides in org.testcontainers.dockerclient.DockerClientConfigUtils.getDockerHostIpAddress(DockerClientConfig) then org.testcontainers.dockerclient.EnvironmentAndSystemPropertyConfigurationStrategy.provideConfiguration() is still going to use a DockerClientConfig that does not know about the override.

I possible solution would be to move the override code out of DockerClientConfigUtils into EnvironmentAndSystemPropertyConfigurationStrategy like this:
`public class EnvironmentAndSystemPropertyConfigurationStrategy implements DockerConfigurationStrategy {
private final DockerClientConfig config;

public EnvironmentAndSystemPropertyConfigurationStrategy() {
	DockerClientConfigBuilder configBuilder = DockerClientConfig.createDefaultConfigBuilder();
	Optional<String> containerHostUriFromPropsOrEnv = Optional
			.of(System.getProperty("testcontainers.container.host", System.getenv("TESTCONTAINERS_CONTAINER_HOST")));
	if (containerHostUriFromPropsOrEnv.isPresent()) {
		configBuilder.withServerAddress(containerHostUriFromPropsOrEnv.get());
	}
	config = configBuilder.build();
}

`

Even if the code would work that way, the approach still is not sufficient. As it turns out the call to client.pingCmd().exec(); in EnvironmentAndSystemPropertyConfigurationStrategy will only succeed if not only the container host (URI) is set, but also a working cert path (sslConfig).
That means:

-Dtestcontainers.container.host=<IP_ADDRESS>:2376 -> does not work since there is no scheme
-Dtestcontainers.container.host=https://<IP_ADDRESS>:2376 -> does not work unless you do also set a valid cert path
-Dtestcontainers.container.host=tcp://<IP_ADDRESS>:2376 -> does not work since scheme tcp does not seem to be supported by Docker-Java
The unsupported tcp scheme is especially annoying since the docker-machine tool does setup export DOCKER_HOST="tcp://<IP_ADDRESS>:2376" on my machine.
So I guess we are back at square one. I think for the moment it is fair to say that the docker in docker workaround with sockets is not supported and can be dealt with by setting up all DOCKER_* environment variables properly in the docker container that runs testcontainers.

I'll park this for now if it doesn't solve the problem for you - happy to pick up again if there's a simple solution out there. As you mentioned, for docker-in-docker usage the best approach remains to set the DOCKER_ environment variables appropriately.


Statistical analysis of characteristics of census tracts included in one measure (any of the above) not included in ours. What are the differences between the census tracts in our tool but not in others, and vice versa?

Problem statement/question
Based on https://github.com/usds/justice40-tool/issues/245 and https://github.com/usds/justice40-tool/issues/135, it would be interesting to run a statistical analysis of the difference between census tracts included as priority communities in a comparison metric but not in the current CEJST score. E.g., "Census tracts that are included in CalEnviroScreen disadvantaged communities but are not included in the current CEJST priority communities have 20% higher incomes and 34% less linguistic isolation on average than CalEnviroScreen census tracts that are included in the current CEJST priority communities."

Just adding this as a note. In CalEnviroScreen methodology (and verbatim):
For each census tract, the data was analyzed to estimate the number of households with household incomes less than
80% of the county median and renter or homeowner costs that exceed 50% of household income. The percent of the
total households in each tract that are both low-income and housing-burdened was then calculated.

Unlike the US Census, CHAS data are ACS estimates which
come from a sample of the population and may be unreliable
if they are based on a small sample or population size. The
standard error (SE) and relative standard error (RSE) were
used to evaluate the reliability of each estimate.

The SE was calculated for each census tract using the
formula for approximating the SE of proportions provided by
the ACS (American Community Survey Office, 2013, pg. 13,
equation 4). When this approximation could not be used, the
formula for approximating the SE of ratios (equation 3) was
used instead.

The RSE is calculated by dividing a tract’s SE by its estimate
of the percentage of housing burdened low income
households, and taking the absolute value of the result.

Census tract estimates that met either of the following
criteria were considered reliable and included in the analysis:

RSE less than 50 (meaning the SE was less than half
of the estimate) OR
SE was less than the mean SE of all California census
tract estimates for housing burdened low income
households.

Census tracts with unreliable estimates receive no score for
the indicator (null). The indicator is not factored into that
tract’s overall CalEnviroScreen score.

Census tracts that met the inclusion criteria were ordered by
percent housing burdened low-income households. The
census tracts were assigned percentiles based on the
distribution across all tracts.

This ticket is already completed, or at least a basic version is: see census_tracts_score_comparisons in the comparison tool.
I'd propose we close this for now – please re-open if it feels premature.
@saran-ahluwalia let me know if I'm misunderstanding, but I'm not sure the comments you've pasted above are relevant to this topic? They're definitely relevant to conversations about handling null values, etc, but not this specific ticket.

@lucasmbrown-usds Yes, that was just a note for myself and posterity to outline how the CalEnvironScreen's methods for assigning tracts with null values. That was all. This would be a reference for future issues.


seperate ansible scripts

Signed-off-by: anandrkskd<EMAIL_ADDRESS>
What type of PR is this:

What does this PR do / why we need it:
Which issue(s) this PR fixes:

Fixes #6455
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

How to test changes / Special notes to the reviewer:

The README file at the root (scripts/ansible/README.md) is empty. Could you please document how to run the script manually (if one wants to deploy the infra into its own account to test it).

Error: ENOENT: no such file or directory, open 'scripts/ansible/requirements.yml'

The github action seems to fail

/override Kubernetes-Integration-Tests/Kubernetes-Docs-Integration-Tests
/override NoCluster-Integration-tests/NoCluster-Integration-tests
/override OpenShift-Integration-tests/OpenShift-Integration-tests
/override OpenShift-Integration-tests/OpenShift-Unauth-Integration-tests
/override windows-integration-test/Windows-test
Not related tests

/override odo-infra-stage-test / kubernetes-infra-stage-test
Failing on checking NFS server (to be debugged)

/override odo-infra-stage-test


Increment version for metricsadvisor releases

Increment package version after release of Azure.AI.MetricsAdvisor

/check-enforcer override


Cloudant Query support

In order to support CQ we need two new end points: /DB/_index and /DB/_find.
A POST to _index will need to support a few variants:
{
    "index": {
        "fields": ["foo"]
    },
    "name" : "foo-index",
    "type" : "json"
}

and
{
"fields": [
      {"name": "married", "type": "boolean"},
      {"name": "lastname", "type": "string"},
      {"name": "year-of-birth", "type": "number"}
    ],
    "name" : "foo-index",
    "type" : "json"
}

We need to inject the com.cloudant.meta.auth.users field (how I wish we'd called it something else...) either as a string, or as an object appropriately.
On the _find endpoint we need to encapsulate whatever selector the user issued into an "$and". Given
 "selector": {
    "$and": [
      {
        "_id": { "$gt": null }
      },
      {
        "year": {
          "$in": [2014, 2015]
        }
      }
    ]
  }

we'd want that transformed to something like
 "selector": {
  "$and": [
     "$and": [
       {
         "_id": { "$gt": null }
       },
       {
         "year": {
           "$in": [2014, 2015]
         }
       }
     ],
     "com.cloudant.meta.auth.users": { "$in": [ "USER" ] }
 ]
 }

had a first go at this: https://github.com/glynnbird/envoy/tree/cquery.
Indexing com.cloudant.meta.auth.users proved problematic. Switching to com_cloudant_meta.auth.users worked. Also switch "$in" for "$elemMatch".

I've tidied this up and added some tests: https://github.com/glynnbird/envoy/tree/cquery.
It broadly works ok with one proviso. Here's the rather long winded explanation:
Let's assume we have documents like this:
{"i":1}
So we define our index
ccurl -X POST -d '{"fields":[i], "type":"json"}' /db/_index
Envoy intercepts this request and instructs Cloudant to create and index like this:
{"fields":[com_cloudant_meta.auth.users,i], "type":"json"}
But when we query the index we do
 ccurl -X POST -d '{"selector":{ "i":1}}' /db/_find
which gets translated by Envoy into:
 {"selector":{"$and":[{"com_cloudant_meta.auth.users":{"$elemMatch":{"$eq":"MYUSER"}}},{"i":1}]}}
Although the query works, Cloudant/Envoy returns a warning:
{ warning: 'no matching index found, create an index to optimize query time'...

Because it doesn't think it has an index that can help it. When the index type is "text" instead of "json" then everything's fine.

My initial reaction to this is that we shouldn't support _index at all through envoy. Developers / administrators of an application can add indexes directly in Cloudant - we just need to prevent information leaking when users query them.
Some gotchas that we'd need to catch if users can add their own indexes:

default field support needs to be disabled
array length support needs to be disabled
an index defined by user A could break queries for user B (because index selection is naive)


Prometheus Metric-Labels format

Hi,
recently i discovered prometheus complaining about metrics and could track it down to this:
... msg="append failed" err="expected equal, got \"INVALID\""

~> echo 'go_gc_duration_seconds{app="metallb",component="speaker",controller-revision-hash="646fb6b695",instance="<IP_ADDRESS>:7472",job="kubernetes-pods",kubernetes_namespace="sys-k8-metallb",kubernetes_pod_label_name="speaker-7hg85",kubernetes_pod_node_name="node03",pod-template-generation="1",quantile="0.5",team="sys",monitor="k8-intern"} 0.000117204<PHONE_NUMBER>955'|promtool check metrics
error while linting: text format parsing error in line 1: expected '=' after label name, found '-'

prometheus probably has trouble with the "-" not being an underscore..

sorry not an metallb issue..


Added the translate module for MT

Removed the un-used import in align/gdfa.py
Added the translate module with model.py to load pre-calculated phrase table and language models and
added stack_decoder.py to perform stack_decoding for Statistical Machine Translation (SMT)

Sorry for the delay here, @alvations. I'm hoping for progress re #800.

@stevenbird , no problem about the delay.

Things are moving forward with the language modelling package now (please see the model branch). You might like to try it out?

Questions/Suggestions about the model package

Is that in the model branch?
What is teardown_module() doing? Is it used to dump the corpus in buffer after training the language model? Wouldn't that do so globally?
Also a NgramModel dump to arpa textfile format would be helpful. Although I would prefer to save it in pickle binaries. Or maybe allow both textfile and pickle dump.
Could the padding be set to default <s> and <\s> like most language model instead of ''? It's because '' == True is False and it's hard to do checks when other packages uses the NgramModel._lpad. But the risk is that <s> might be retokenized to ['<', 's', '>'] if users/devs dumps NgramModel.train to strings and then re-read and retokenize it. But the chances of people retokenizing NgramModel.train should be quite rare.
A dictionary of number of unique ngrams of different orders should be kept so that users can access it to perform renormalization or other functions that needs len(no. of unigrams),  len(no. of bigrams), etc.
Unbounded vs bounded method for entropy and perplexity, see below.
Should the nltk.translate.util.read_translation_model() go to nltk.model? Should I create a different model abstract class for a phrase translation model? How about tree-based translation model, different or same abstract class?
Last important question, pardon my "noobness" in git development, should I fork a different branch from model and then work on the translate package and dump my current branch. Or should I continue on the develop branch?

Perplexity and entropy be an unbound method where the user can do:
x = NgramModel(xtext)
y = NgramModel(ytext)
model.perplexity(x, y)

currently, i think one has to do:
x = NgramModel(xtext)
y = NgramModel(xtext)
x.perplexity(y.train)

Maybe we should allow both. It's sort of like the wn.path_similarity(x,y) vs x.path_similarity(y)

nltk.translate TODOs

port nltk.translate.util.read_lang_model() to nltk.model.util.read_arpa()
change :return: of read_lang_model() to nltk.model.NgramModel

About the git question, i've created a translate branch in my fork and i'll merge back with my develop branch once the model merges with develop.

@stevenbird , @Copper-Head, the model NgramModel is breaking due to the smoothing:
[in]:
from nltk.model import NgramModel
from nltk.corpus import brown
from nltk.probability import LidstoneProbDist
est = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)
lm = NgramModel(3, brown.words(categories='news'), estimator=est)

[out]:
Traceback (most recent call last):
  File "/home/alvas/git/nltk/test_model.py", line 15, in <module>
    lm = NgramModel(3, brown.words(categories='news'), estimator=est)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 121, in __init__
    self._probdist = estimator(cfd, *estimator_args, **estimator_kwargs)
TypeError: <lambda>() takes exactly 2 arguments (1 given)

Even without estimator the default estimator is not working. Also the docstring in _estimator writes GoodTuring but it's calling lidstone =(
@stevenbird , @Copper-Head, the model NgramModel is breaking due to the smoothing:
[in]:
from nltk.model import NgramModel
from nltk.corpus import brown
from nltk.probability import LidstoneProbDist
est = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)
lm = NgramModel(3, brown.words(categories='news'), estimator=est)

[out]:
Traceback (most recent call last):
  File "/home/alvas/git/nltk/test_model.py", line 15, in <module>
    lm = NgramModel(3, brown.words(categories='news'), estimator=est)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 121, in __init__
    self._probdist = estimator(cfd, *estimator_args, **estimator_kwargs)
TypeError: <lambda>() takes exactly 2 arguments (1 given)

Even without estimator the default estimator is not working. Also the docstring in _estimator writes GoodTuring but it's calling lidstone =(

@Copper-Head are you able to investigate this problem please?

@alvations can you see what happens if you set _estimator to GoodTuring?

@stevenbird, @Copper-Head, the _estimator runs with GoodTuring only for py2.x and while it breaks in py3.x because of the ConditionalFreqDist.iterkeys().
I think ConditionalFreqDist would need something like six.iterkeys() to override the dictionary.iterkeys() so that it's supported both in py2.x and py3.x.
from nltk.model import NgramModel
from nltk.corpus import brown
from nltk.probability import LidstoneProbDist, SimpleGoodTuringProbDist

#est = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)
est2 = lambda fdist: SimpleGoodTuringProbDist(fdist)

corpus = brown.words(categories='news')[:100]
lm = NgramModel(3, corpus, estimator=est2)

# Testing prob
print (corpus[8], corpus[9], corpus[12] )
print (lm.prob(corpus[12], [corpus[8], corpus[9]]) )
print

print (corpus[8], corpus[9], corpus[22] )
print (lm.prob(corpus[22], [corpus[8], corpus[9]]) )
print

print (corpus[8], corpus[9], corpus[10] )
print (lm.prob(corpus[10], [corpus[8], corpus[9]]) )
print

# Testing logprob
print (corpus[8], corpus[9], corpus[12] )
print (lm.logprob(corpus[12], [corpus[8], corpus[9]]) )
print

print (corpus[8], corpus[9], corpus[22] )
print (lm.logprob(corpus[22], [corpus[8], corpus[9]]) )
print

print (corpus[8], corpus[9], corpus[10] )
print (lm.logprob(corpus[10], [corpus[8], corpus[9]]) )
print

[out]:
alvas@ubi:~/git/nltk$ python test_model.py
(u'investigation', u'of', u'primary')
0.0124909679922

(u'investigation', u'of', u'took')
0.00400419641542

(u'investigation', u'of', u"Atlanta's")
0.0

alvas@ubi:~/git/nltk$ python3 test_model.py
Traceback (most recent call last):
  File "test_model.py", line 20, in <module>
    lm = NgramModel(3, corpus, estimator=est2)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 129, in __init__
    **estimator_kwargs)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 141, in __init__
    for word in self._words_following(ctxt, cfd):
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 162, in _words_following
    for ctxt, word in cond_freq_dist.iterkeys():
AttributeError: 'ConditionalFreqDist' object has no attribute 'iterkeys'

Also, it seems like a bug the correct ngram is getting probability of 0.0 instead of 1.0. when retrieving LanguageModel.prob(word, context). When getting logprob it goes into math domain error because of Log 0.
Although with any smoothing there should never be a log 0, but i think LanguageModel.logprob needs to handle log 0 .

I was able to reproduce @alvations's observations. Maybe it is a good idea to track that issue separately.

Hi everyone!
Sorry I've been out of touch. The downside of doing this in one's free time...
I've gone over my fix for the original bug in the NgramModel code with some friends and we've come to the conclusion that my original solution was incorrect. We know what went wrong and how to rectify it, so I'll be pushing those changes soon.
That being said, I'll look over this thread as well and let you know if I think the problem is related to the stuff I've been doing or if it's a separate issue.
Thanks,
Ilia.

Just like to check the status of the merge pull. Should this be killed and we'll try again after nltk.model is fixed? Or should we just give it up?

This PR was inadvertently closed when I recreated the develop branch.

I'm still alive and will give this a shot this week.

Thanks @Copper-Head, it would be great to resolve this.

@stevenbird I just want to make sure our logistics are clear.
I'm going to switch to my fork's model branch, revert the problematic commits, make some adjustments to docstrings etc. Then I recreate my repo's remote model branch and push to it. In my PR, do use the test branch as comparison, since you configured Jenkins for it?

@alvations I'm reading your list of suggestions/questions and wondering about the padding idea. Just to be clear I understand, if we adopt the <s>, </s> as default, the first trigam of a text ['He', 'opened', 'a', 'bottle', ...] will look like ("<s>", "<s>", "He"), correct?

@Copper-Head ,  The padding adds the <s> and <\s> to the head and end of the sentence, e.g. ['He', 'opened', 'a', 'bottle', ...] -> ['<s>', 'He', 'opened', 'a', 'bottle', ..., '<\s>']. So the first trigram would be ('<s', 'He', 'opened') and the last trigram.

Hmm, wouldn't we have to change the implementation of nltk.util.ngrams for what you suggest to work best?

@stevenbird the changes necessary for fixing the NgramModel (for real this time, I'm pretty sure) are sitting in a model branch on my fork. Let me know what your preferred logistics are for merging!

Please submit a pull request to the model branch, thanks. It would be great to include tests that demonstrate correctness.

@Copper-Head, I think there's still something wrong with the Ngram object:
from nltk.model import NgramModel
from nltk.corpus import brown
from nltk.probability import LidstoneProbDist, SimpleGoodTuringProbDist

est = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)

corpus = brown.words(categories='news')[:100]
lm = NgramModel(5, corpus, estimator=est)

Traceback:
Traceback (most recent call last):
  File "/home/alvas/git/nltk/test_model.py", line 19, in <module>
    lm = NgramModel(5, corpus, estimator=est)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 126, in __init__
    **estimator_kwargs)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 126, in __init__
    **estimator_kwargs)
  File "/home/alvas/git/nltk/nltk/model/ngram.py", line 151, in __init__
    assert (0 < backoff_total_pr < 1), backoff_total_pr
AssertionError: 1.0

After some checks it seems like total_observed_pr at https://github.com/Copper-Head/nltk/blob/model/nltk/model/ngram.py#L146 always equals to 1.

Hey, @alvations I'm afraid I haven't merged in my changes just yet. I'm working on creating a doctest file for the ngram module today and will be pushing those changes in the evening. Then I'll open a PR and ping you guys.

Thanks

@Copper-Head, @alvations – would you please update me on where things stand here?

After looking at the NgramModel and talking to @Copper-Head , for now, I'm going to:

Move nltk.translate.api.LanguageModel into nltk.model as a sibling class of nltk.model.NgramModel
Move nltk.translate.util.read_lang_model into nltk.model.util.read_lang_model (also i'll improve how i'm reading the APRA file, it's not very efficient as it is now)
Change the stack decoder code according to access the LanguageModel like that of an NgramModel.

I'll be making the above changes soon once I get some free time.
Should I create a lm.py in nltk.model or should the following be another class in nltk.model.ngram?
from nltk.model.api import ModelI
class LanguageModel(ModelI):
  pass

I'll also like to suggest to add pad_left_symbol and pad_right_symbol to the NgramModel.__init__() such that it's more consistent with existing Ngram models that uses <s> and </s> instead of null string.


ServiceInspectors assembly has dependencies on individual service binaries

Core assemblies should not have dependencies on service, platform or feature assemblies.
The new service inspectors assembly directly references

services.boundary
services.input
services,scenesystem
services.spatialawarenss
services.teleport

Per offline design discussion, giving to @Railboy to address.
Thanks, sir!

fixed.


🤖 Investigate Alternative Certificate Management Tools

User Need
As a (type of user)
I want (some action)
so that (some result)
Value
Functional Requirements (What):

[ ]
[ ]

Non-Functional Requirements (How):

[ ]
[ ]

Acceptance Criteria:

[ ]
[ ]

Notes:

From a brief conversation, some ideas that might be worth exploring:

Certificates in code
Gandi Terraform providers
OctoDNS for Gandi

Removing for now - certificates may be moving out of Ops Eng.


[docs][ftp] add ftp docs

Purpose of this pull request
Add ftp docs

Already fixed , please review again @FlechazoW


Fix #44

Implement native Debian packaging (control files are part of the source repo) with Makefile to simplify the release procedure. This commit also makes w1thermsensor and spidev "soft" dependencies which aren't required by the Debian package (because they're not packaged for Raspbian, only PyPI).
Please note that as this stands, Ben will have to build and upload the Raspbian packages (because he's listed as the maintainer in debian/changelog); if you want me to handle this do let me know but I'll have to specify myself as the deb package maintainer (debuild uses the changelog's maintainer to locate the GPG key used to sign the package - hence only the listed maintainer can build the package).

Serge, who maintains Foundation Raspbian now, has offered to help with the packaging work. I've fired off emails to the maintainers of w1thermsensor and spidev asking if they mind us packaging them. I'll wait and see if Serge wants to use this PR or do it his own way.

Frankly if these get packaged you don't want this PR anyway as it makes them "soft" dependencies, so I'll close this and work up an alternate PR which assumes those packages are available in Raspbian and keeps them as "hard" dependencies.

Yep - looks like we'll be getting them both packaged. Serge is working on it now - I already have .debs to test!


Missing documentation on documentation

I went through a few macros and almost every time, I found the lack of examples really problematic (like here for example).

Having the same problem :cry:

While I like the crate, the docs are just incredibly ineffective. Rust macros don't exactly produce nice error message and the docs are little help. Missing in particular:

A nice example in the README (or at least the root docs), showing a simple, complete program (how am I supposed to get my parsed data out of the parser)?
An example u8 vs &str
For every macro there should be one, better two examples illustrating how it works.

My apologies if this seems a bit crass, but usually, I find myself working on the parser for 5 minutes out of 60 and spending the remaining fighting nom, trying to find an example from somewhere to get it working.
I'd love to contribute a few examples, but I'm not sure I can even write proper ones at this point. =/

@GuillaumeGomez @dashed @mbr could you provide a bit more info on what you would need from the documentation? Adding examples for each macro is a clear enough target, but the rest is not so clear.
Did you find the articles in the wiki? If not, why? Were they helpful?
What is wrong with the examples in the readme? There's another one in the online documentation, is it alright? There are other complete examples in the tests, should those be used instead?
What do you mean with an example of "how to get data out of the parser"? You usually pattern match on IResult, which is specified here. Do you need something else?
I would really like to improve the documentation, but I have limited time to do it, so I would really appreciate precise feedback here, otherwise the docs won't improve much.

Just me two cents from using nom:

The docs are scattered in a couple different places. I didn't even know about that there was documentation on the wiki until I saw this issue.
The iso8601 tutorial (https://fnordig.de/2015/07/16/omnomnom-parsing-iso8601-dates-using-nom/) was super helpful for seeing how everything fits together. Something like this would be great for official docs (it'd be nice to parse something a little more substantial though).
There are variants for named! that I couldn't find documentation for (for example named!(blah<i32>, ...)

FWIW, I'm not 100% sure this is an issue with nom itself. Rust's macro error messages just tend to become utterly unreadable (at least today) compared to "regular code".

@GuillaumeGomez so, is the documentation documented now? :)

https://docs.rs/nom/4.0.0/nom/macro.eof.html -> "TODO: example" 😛
Please add a #![deny(missing_docs)] and then I guess I'll consider it as done. Thanks a lot for your work, it's truly amazing!

closing this in favor of #914


Fix cannot debug language server with BAL_DEBUG_OPTS

Purpose

Fix cannot debug language server with BAL_DEBUG_OPTS

Fixes #20429
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #20549 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #20549   +/-   ##
=======================================
  Coverage   15.21%   15.21%
=======================================
  Files          50       50
  Lines        1341     1341
  Branches      208      208
=======================================
  Hits          204      204
  Misses       1121     1121
  Partials       16       16

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c917ba1...ce5c57c. Read the comment docs.


Feature/dropdown add to body

Fixes #1030
Look for dropdown-append-to-body attribute on the dropdown directive.  If it's there, append the dropdown-menu to the body and show/hide/position it when the dropdown toggles.

I'm attempting to remove the dropdown menu from the body when the main dropdown element fires the $destroy event.  However, on the demo page the $destroy event never gets fired.  I've been able to see the event get fired in my own application using this code, though.
In the tests, I do in fact see that the $destroy event gets fired if I call element.remove() and the code successfully calls dropdownMenu.remove(). However, in the test browser window the element never gets removed from the page.  The remove call seems to do nothing.
Any ideas why?

+1!!!

This needs a test to fix the garbage collection.

@wesleycho do you mean it needs a fix to test the garbage collection?
I had a test that looked like this:
it('removes the menu when the dropdown is removed', function() {
  element.remove();
  $rootScope.$digest();
  //Here, I am able to verify that line 80 (self.dropdownMenu.remove();) is in fact called but it has no effect in the test environment.
});

What I was referring to in my comment was that I was unable to verify the outcome of self.dropdownMenu.remove() because it remained in the DOM in the test runner's browser, even though I could verify that self.dropdownMenu was defined and referred to the correct element.  I'll give it another shot, but I was hoping there might be some testing quirk that I didn't know about.

@rysilva if you're able to do so, I'm willing to review this again and merge when ready.
Perhaps element.remove() did not properly garbage collection the $scope and emit the $destroy event?

@rysilva, @wesleycho : looking forward to this fix, thanks to both of you.

@rysilva and @wesleycho, I believe I understand what is happening: the element is not being cleaned up after each test. In other words, element.remove() needs to be called at the end of his test. This is what triggers the removal of the dropdown from the body. I would actually recommend calling element.remove() in an afterEach block at the top of this whole spec. A lot of tests are doing it in a one-off way anyway, and I see no reason to keep elements around after each test.
I have issued a pull request to @rysilva that not only fixes all this, but also resolves the conflicts with the current master preventing automatic merge. @wesleycho, assuming @rysilva merges my PR to his branch, please review again.

Thanks, I'll take a look
On Apr 1, 2015, at 3:22 AM, Andy Perlitch<EMAIL_ADDRESS>wrote:
@rysilva https://github.com/rysilva and @wesleycho
https://github.com/wesleycho, I believe I understand what is happening:
the element is not being cleaned up after each test. In other words,
element.remove() needs to be called at the end of his test. This is what
triggers the removal of the dropdown from the body. I would actually
recommend calling element.remove() in an afterEach block at the top of this
whole spec. A lot of tests are doing it in a one-off way anyway, and I see
no reason to keep elements around after each test.
I have issued a pull request https://github.com/rysilva/bootstrap/pull/1
to @rysilva https://github.com/rysilva that not only fixes all this, but
also resolves the conflicts with the current master preventing automatic
merge. @wesleycho https://github.com/wesleycho, assuming @rysilva
https://github.com/rysilva merges my PR to his branch, please review
again.
—
Reply to this email directly or view it on GitHub
https://github.com/angular-ui/bootstrap/pull/3411#issuecomment-88376956.

I'm not sure why adding an afterEach would solve the issue, because you're already calling element.remove() in the test anyway.  So for that test the afterEach should have no effect, right?
I was going to run locally again but I'm currently getting an error running grunt:
Running "karma:continuous" (karma) task
[2015-04-01 08:07:20.330] [ERROR] config - Invalid config file!
 [ReferenceError: module is not defined]

You may need to run npm update.
Also the history needs to be cleaned up - the merge commit should not be there, and all of the commits should be squashed into one.

Squashed all the commits.

Cannot wait for this feature to be merged! It fixes all my problems.

Merged - thanks for the good work @rysilva and @andyperlitch !

all @rysilva... thanks @wesleycho!

@andyperlitch , @rysilva , @wesleycho : thanks a lot!!!!!!!!!!!!!!!!!!!!!


Prysm phase 1 feedback

Quick update regarding phase 1 work with Prysm. I’ve implemented most of what’s in phase 1 spec today except for custody, light client and fraud proof logic. With the proper validator shard duties in place, without networking, we can begin to simulate shard transition and crosslink in a local enviroment. This work is currently getting done here and it will get migrated to the official Prysm repo post beacon chain mainnet. Here’s some preliminary feedback from our end after implemented shard transition logic:

Add Shard to AttestationData, always having to call get_shard is not elegant. Being able to quickly validate Shard correctness for any incoming attestation is valuable
Add Shard to ShardBlock, same reason as above. Able to quickly validate Shard correctness for a shard block is valuable
Refactor shard proposer reward portion to its own method inside process_crosslink_for_shard. Would love to see process_crosslink_for_shard get split to be more granular.
Rename AttestationData’s head_shard_root to shard_head_root, it resonates better with shard_transition_root. Both prefix shard.
Rename is_shard_attestation to is_ontime_shard_attestation since attestation needs to be on time as well
Rename verify_shard_transition_false_positives to verify_empty_shard_transition or something better 😅
Put unpack_compact_validator inside light client spec since it’s not used within the scope in phase1 beacon chain
Question: is there significant amount of save on hashing to put online_countdown as it’s on field in BeaconState rather than having each Validator track own online countdown

Add Shard to AttestationData, always having to call get_shard is not elegant. Being able to quickly validate Shard correctness for any incoming attestation is valuable

Agreed with it’s not elegant, but a bit hesitant since we have a lot of Attestation traffic 😬

Add Shard to ShardBlock, same reason as above. Able to quickly validate Shard correctness for a shard block is valuable
Refactor shard proposer reward portion to its own method inside process_crosslink_for_shard. Would love to see process_crosslink_for_shard get split to be more granular.
Rename AttestationData’s head_shard_root to shard_head_root, it resonates better with shard_transition_root. Both prefix shard.
Rename is_shard_attestation to is_ontime_shard_attestation since attestation needs to be on time as well
Rename verify_shard_transition_false_positives to verify_empty_shard_transition or something better 😅

All sound good to me too 👍

Put unpack_compact_validator inside light client spec since it’s not used within the scope in phase1 beacon chain

(I don’t feel strongly about it)
Vitalik suggested that unpack_compact_validator should be next to pack_compact_validator.

Question: is there significant amount of save on hashing to put online_countdown as it’s on field in BeaconState rather than having each Validator track own online countdown

Correct!

Update: the remaining items:

[ ] Add Shard to AttestationData, always having to call get_shard is not elegant. Being able to quickly validate Shard correctness for any incoming attestation is valuable
[ ] Refactor shard proposer reward portion to its own method inside process_crosslink_for_shard. Would love to see process_crosslink_for_shard get split to be more granular.
[ ] Put unpack_compact_validator inside light client spec since it’s not used within the scope in phase1 beacon chain

Refactor shard proposer reward portion to its own method inside process_crosslink_for_shard. Would love to see process_crosslink_for_shard get split to be more granular.

This is what I have implemented for Prysm: https://github.com/terencechain/prysm-phase1/blob/phase1/beacon-chain/core/helpers/shard.go#L278-L334
They are helpers to group attestations by 1.) on-time and committee_id 2.) shard_transition_root.
Another helper is to calculate whether a given list of attestations with corresponded committee can crosslink. I used these helpers for beacon-chain's process_crosslink_for_shard and validator's get_shard_winning_roots
These helpers are similar to get_matching_attestations for epoch processing. With that said, I understand that specs are optimized for readability. We can skip them if we think the helpers bring lesser value in that regard. I don't feel strongly about enforcing them into the spec : )


MIddle click to paste on Linux no longer works.

Since there's no real selection it doesn't know to paste it.

Could try adding selection using the textarea and addRange https://developer.mozilla.org/en-US/docs/Web/API/Selection/addRange
Just need to be careful with:

Performance, the addRange calls should probably be debounces at a slower rate than the dragScroll call.
Shifting focus to and from the textarea with the selection causing glitchy focus behavior, related: https://github.com/sourcelair/xterm.js/issues/681
Any other issues that could arise with messing with the textarea.


A great amount of time for the win64_mingw73 arch

The win64_mingw73 arch takes a lot of time to be installed. As you can see from here, it takes more than 17 minutes. Before it took only 1 minute, as you can see from here.
If I use the win64_msvc2017_64 instead, the problem doesn't occur.
Thanks in advance for your help! :)

Thank @Luni-4 for report.
I found a reason why performance decreased.

switch default extractor from external 7zip command to python library.
both extractors can archive similar performance.
when calling external command, launch it and return asap, but now wait library extractor to finish.

I've compared the library and external command
library execution (pure python)
$ time py7zr x 5.12.6-0-201911111120qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z

real	0m35.086s
user	0m29.616s
sys	0m6.972s

7zip command (C++)
$ time 7zr x 5.12.6-0-201911111120qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=ja_JP.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Core(TM) i3-6100U CPU @ 2.30GHz (406E3),ASM,AES-NI)

Scanning the drive for archives:
1 file, 214532582 bytes (205 MiB)

Extracting archive: 5.12.6-0-201911111120qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z
--
Path = 5.12.6-0-201911111120qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z
Type = 7z
Physical Size = 214532582
Headers Size = 47567
Method = LZMA2:24 BCJ
Solid = +
Blocks = 2

Everything is Ok

Folders: 279
Files: 3281
Size:<PHONE_NUMBER>
Compressed: 214532582

real	0m30.473s
user	0m24.312s
sys	0m5.445s

Penalty is 5 sec / 30sec, 15%.
You can add '-E <path to 7zip command>' explicitly to use external command.

Thanks for your explanation @miurahr, but I can't understand why that doesn't happen with the win64_msvc2017_64 arch. I mean, if the extractors are the cause of the problem, they should affect the win64_msvc2017_64 arch too, or am I misunderstanding something?
Anyway, this is the performance of the  win64_msvc2017_64 arch, as you can see it took only 52s to be completed.

Now you can see detailed log at https://dev.azure.com/miurahr/892bc5d7-3681-4359-bd29-fe00ca91ef10/_apis/build/builds/1581/logs/364 where a branch try to improve performance with threading for downloading and with multi-process for extraction, and adding detailed logs.
Here is a part of logs
2020-02-10T01:33:02.7074789Z ##[section]Starting: Run Aqt (No Base URL Set)

<snip>
2020-02-10T01:33:03.0182117Z [command]C:\hostedtoolcache\windows\Python\3.7.6\x64\python.exe D:\a\1\s\bin\aqt install --outputdir D:\a\1\b/Qt 5.13.2 windows desktop win64_mingw7
2020-02-10T01:33:05.3607477Z 2020-02-10 01:33:05,352 - aqt - INFO - Downloading https://download.qt.io/online/qtsdkrepository/windows_x86/desktop/qt5_5132/qt.qt5.5132.win64_mingw73/5.13.2-0-201910281254qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z...

<snip>

2020-02-10T01:33:16.8026820Z 2020-02-10 01:33:16,457 - aqt - INFO - Extracting qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z..

<snip>

2020-02-10T01:33:19.5726840Z 2020-02-10 01:33:19,494 - aqt - INFO - Downloads are Completed.

<snip>

2020-02-10T01:57:47.1813899Z 2020-02-10 01:57:47,180 - aqt - INFO - Extraction qtbase-Windows-Windows_10-Mingw73-Windows-Windows_10-X86_64.7z done
2020-02-10T01:57:47.1832914Z 2020-02-10 01:57:47,180 - aqt - INFO - Finished installatio

During extracting qtbase package, 12 packages are completed to download and start extracting,  and 17 packages have been extracted.
It seems that qtbase extraction in my note-pc takes just 30sec but it consumes 24 min on Azure Pipelines. Other processes  mostly work fine but extraction of qtbase(200MB) and qt3d(100MB) takes loooong time.
Is it a luck of CPU time on cloud service?

Thanks for your tests @miurahr!
Hmm, that's pretty strange indeed. Could you please make a new release on test.pypi of #87 such that I could test your changes using the GitHub Actions? Let's see if the problems persists there also.
In the meantime, I've installed and extracted qtbase and qt3d using the Github Actions running curl -LO $LINK as first command and then 7z x $ZIP. They took respectively 30s and 17s, as you can see from here

You can use '[-E | --external <7zip command>] ' option for aptinstall. If it works for you, it is a problem on extraction library or related code. When specify the option, aqtinstall launch '7z x' command by 'subprocess.run()' python feature.

A root cause is 'dead lock'. When extracting mingw packages, all python process going to 'Sleep' state.
I found a description in python manual as follows:
https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor

Calling Executor or Future methods from a callable submitted to a ProcessPoolExecutor will result in deadlock.

Because py7zr extraction library uses 'concurrent.futures' to accelerate extraction in multi-thread when archive file is compressed with supporting concurrency,  that is happen on large archives, aqt run extractor in the context of concurrent.futures that call py7zr which call futures.
To solve a problem, we need to rethink a  concurrency design.

v0.7.4 released and I believe it reduce cases to slow down.

Great! it takes only 2 minutes now! Thanks a lot @miurahr ! :)
Out of curiosity, would it be possible to reduce this time to 31s as the msvc build?

Out of curiosity, would it be possible to reduce this time to 31s as the msvc build?

Unfortunately msvc downloads are total 90MB but mingw downloads are total 209MB in Qt 5.15.0,
and >300MB in Qt 5.12.
This means you need twice or 3rd times longer  for mingw download and extraction than msvc packages.
https://download.qt.io/online/qtsdkrepository/windows_x86/desktop/qt5_5150/qt.qt5.5150.win64_msvc2017_64/
https://download.qt.io/online/qtsdkrepository/windows_x86/desktop/qt5_5127/qt.qt5.5127.win64_mingw73/

Issue here solved.

Out of curiosity, would it be possible to reduce this time to 31s as the msvc build?

Unfortunately msvc downloads are total 90MB but mingw downloads are total 209MB in Qt 5.15.0,
and >300MB in Qt 5.12.
This means you need twice or 3rd times longer for mingw download and extraction than msvc packages.
https://download.qt.io/online/qtsdkrepository/windows_x86/desktop/qt5_5150/qt.qt5.5150.win64_msvc2017_64/
https://download.qt.io/online/qtsdkrepository/windows_x86/desktop/qt5_5127/qt.qt5.5127.win64_mingw73/

Oh, I see. Thanks for your explanation! :)


Support 17 digit Discord IDs

Some Discord IDs can be 17 digits long rather than the regular 18 digits...I should know, I have one!
Tiny fix to accommodate older Discord accounts.

Hey, thanks for taking the time to write a fix for this! Until now I didn't realize Discord IDs could be 17 chars. Right now the working build for the next version is on branch v2.2.0; I switched the pull request's target branch, but there was a merge conflict. It looks like the conflict is a new ChatMessageData class added that was on v2.2.0, when you are able feel free to resolve that conflict and commit the changes and I'll approve the request.

Merged v2.2.0, fixed the conflict in the process. 👍


Release candidate #1 pre-release PR for Fall24 meta-release

What type of PR is this?

subproject management

What this PR does / why we need it:
First release candidate for v0.4.0 pre-release PR for Fall24 meta-release
Which issue(s) this PR fixes:
Related to https://github.com/camaraproject/HomeDevicesQoD/issues/63
Special notes for reviewers:
N/A
Changelog input
N/A
Additional documentation
N/A

Review of home_devices_qod.yaml:
In the Info Object: remove the fields:
termsOfService: http://swagger.io/terms/
contact:
the externalDocs url in most API files points to the actual repository, so you could do that as well.

review of README.md:
some rewordings:

first bullet: Pre-release 0.4.0-rc.1 is available here --> Pre-release r1.1 with API version 0.4.0-rc.1 is available here.
second bullet: I would rephrase "latest public release" to "the previous release"

Review of CHANGELOG.md: all good

Review of home_devices_qod.yaml:
In the Info Object: remove the fields: termsOfService: http://swagger.io/terms/ contact:
the externalDocs url in most API files points to the actual repository, so you could do that as well.

Fixed in https://github.com/camaraproject/HomeDevicesQoD/pull/68/commits/a53d0255f547685a01552df6a31d9c38678bf7bb

the documentation uses "Telco speak":

Service Enabling Network Function API may not be understood by developers
DSCP, Service Class etc. is quite technical/network focused.
The section on API functionality would be a good starting section.

In fact, the API was originally designed to only use DSCP as a parameter of the request, and the API spec was adapted to be more developer friendly by incorporating the service class concept of RFC4594. More context in:  https://github.com/camaraproject/HomeDevicesQoD/issues/23 && https://github.com/camaraproject/HomeDevicesQoD/pull/25

The documentation does not cover explanation about the error cases, or the concepts of error cases: maximum nr of devices, home network occupancy, etc. this is important for the developer

Errors are documented as Examples following the  Commonalities definitions. This was previously addapted for this meta-release in https://github.com/camaraproject/HomeDevicesQoD/pull/64

Review of test file:
scenario line 33

it seems that it lists "serviceClass" as one of the possible example values. Is that the intention or is it meant to be a header of the list ?
does not list "standard" as one of the possible example options.

Review of test file: scenario line 33

it seems that it lists "serviceClass" as one of the possible example values. Is that the intention or is it meant to be a header of the list ?

"serviceClass" is the header of the Examples table. It follows the Gherkin language syntax for "Scenario Outline": https://cucumber.io/docs/gherkin/reference/#scenario-outline

does not list "standard" as one of the possible example options.

The test @setQos_01_set_service_class_success is intended to test all possible values of "serviceClass" for the IP traffic corresponding to a specific user home device (except "standard", which is the default value). Then the next test case @setQos_02_restore_default_service_class_success tests to restore the standard value from all possible values of "serviceClass". Exactly the other way around.

Review of API readiness checklist: all OK
On the error documentation, to my understanding it is mandated by the https://github.com/camaraproject/Commonalities/blob/main/documentation/API-DocumentationTemplate.md.
We can leave it for the M4 - if you could create an issue to not forget that would be perfect..

LGTM from ReleaseManagement view.

Review of API readiness checklist: all OK
On the error documentation, to my understanding it is mandated by the https://github.com/camaraproject/Commonalities/blob/main/documentation/API-DocumentationTemplate.md. We can leave it for the M4 - if you could create an issue to not forget that would be perfect..

@tanjadegroot Could you point me to some other APIs that do what you expect here in terms of errors? As I understand it, errors are documented as part of each endpoint definition and they are documented as examples with descriptions:
e.g.
examples:
            Conflict:
              value:
                status: 409
                code: CONFLICT
                message: Conflict with the current state of the target resource
            TooManyDevices:
              value:
                status: 409
                code: HOME_DEVICES_QOD.TOO_MANY_DEVICES
                message: Exceeded the maximum quantity of devices with non-default QoS behaviour
            RssiBelowThreshold:
              value:
                status: 409
                code: HOME_DEVICES_QOD.RSSI_BELOW_THRESHOLD
                message: RSSI from device is below allowed threshold
            QosTooHigh:
              value:
                status: 409
                code: HOME_DEVICES_QOD.QOS_TOO_HIGH
                message: Requested QoS is higher than the maximum QoS allowed
            OccupancyAboveThreshold:
              value:
                status: 409
                code: HOME_DEVICES_QOD.OCCUPANCY_ABOVE_THRESHOLD
                message: The occupancy is above the allowed threshold
            NotConnectedToRequiredInterface:
              value:
                status: 409
                code: HOME_DEVICES_QOD.NOT_CONNECTED_TO_REQUIRED_INTERFACE
                message: Device is not connected to the required interface (e.g. WiFi 5GHz interface)
            NotSupportedRequiredInterface:
              value:
                status: 409
                code: HOME_DEVICES_QOD.NOT_SUPPORTED_REQUIRED_INTERFACE
                message: Device does not support required interface (e.g. WiFi 5GHz interface)
            QosAlreadySetToDefault:
              value:
                status: 409
                code: HOME_DEVICES_QOD.QOS_ALREADY_SET_TO_DEFAULT
                message: Device QoS is already set to default value

Other APIs like QualityOnDemand do the same thing... e.g. https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/camaraproject/QualityOnDemand/release-0.10.1/code/API_definitions/qod-api.yaml&nocors
The same way that the following part of the documentation is documented and detailed on each endpoint definition, not in info.description:

e.g:

@tanjadegroot Please take a look at https://github.com/camaraproject/HomeDevicesQoD/pull/68#issuecomment-2257645191. I think errors are documented as expected and there would be no missing inline doc. https://github.com/camaraproject/Commonalities/blob/main/documentation/API-DocumentationTemplate.md is a bit outdated IMHO. Thanks a lot in advance!
Just to double-check before I merge the PR :)

@tanjadegroot @fernandopradocabrillo https://github.com/camaraproject/HomeDevicesQoD/pull/68/commits/a828df35df443d5d64ca35f4cc0e5573a30e22c4 is needed to address last changes in info.description template in ICM r0.2.0-rc.2 to fix link references. Further info in: https://github.com/camaraproject/IdentityAndConsentManagement/issues/185

@tanjadegroot I just approved the change done in https://github.com/camaraproject/HomeDevicesQoD/commit/a828df35df443d5d64ca35f4cc0e5573a30e22c4. The final saying from Release Management is still on you.

@tanjadegroot May I merge the PR then?

@tanjadegroot May I merge the PR then?

As @tanjadegroot is now on vacation I took over the release review issue. Looks good from release management perspective.
Minor comment, which you can ignore: #62 and #67 are not really needed within the CHANGELOG.md, as they are about subproject management and not relevant for API users.