TestAgentAPIEndPoints test is broken

What did you do?
Executed go test -short -failfast -count 10 ./web/ on revision 9fae120f2a6daa420530e44559c67ce27cb6e7eb.
What did you expect to see?
Test to pass.
What did you see instead? Under which circumstances?
$ go test -short -failfast -count 10 ./web/
http://localhost:9090/targets
--- FAIL: TestAgentAPIEndPoints (0.00s)
    web_test.go:602:
                Error Trace:    web_test.go:602
                Error:          Not equal:
                                expected: 200
                                actual  : 500
                Test:           TestAgentAPIEndPoints
FAIL
FAIL    github.com/prometheus/prometheus/web    15.060s
FAIL

Running built prometheus binary in agent mode and curl-ing gives the following:
$ curl -v localhost:9090/status; echo
*   Trying <IP_ADDRESS>:9090...
* Connected to localhost (<IP_ADDRESS>) port 9090 (#0)
> GET /status HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 500 Internal Server Error
< Date: Wed, 10 Nov 2021 11:30:40 GMT
< Content-Length: 94
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host localhost left intact
Error opening React index.html: open web/ui/static/react/index.html: no such file or directory

Same when adding some verbosity to tests:
$ go test -short -failfast -count 10 ./web/
--- FAIL: TestAgentAPIEndPoints (0.01s)
    --- FAIL: TestAgentAPIEndPoints//status (0.00s)
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/status"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:624:
                Error Trace:    web_test.go:624
                Error:          Should be true
                Test:           TestAgentAPIEndPoints//status
    --- FAIL: TestAgentAPIEndPoints//targets (0.00s)
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:613: for url: "http://localhost:39825/targets"
        web_test.go:621: Error opening React index.html: open ui/static/react/index.html: no such file or directory
        web_test.go:624:
                Error Trace:    web_test.go:624
                Error:          Should be true
                Test:           TestAgentAPIEndPoints//targets
FAIL
FAIL    github.com/prometheus/prometheus/web    15.094s
FAIL

yes, the tests require you to run make build-assets first.

Hm, that's counter-intuitive. Perhaps assets could be mocked in the tests to remove this requirement? And actual test should be an integration test or something?

Does go make a difference between integration and acceptance tests?

Well, I meant, this test could be behind condition with t.Skip or behind a build tag, to keep plain go test ./... working, as one would expect.

Wait, the test is just wrong.

http://localhost:9090/targets should be http://localhost:9090/api/v1/targets
It should not require index.html, it's a bug.

same for http://localhost:9090/query, it should be http://localhost:9090/api/v1/query etc etc

With a fresh clone, we still have an issue:
gitpod /workspace/prometheus $ git worktree add empty
Preparing worktree (new branch 'empty')
HEAD is now at 4cae78868 build(deps): bump github.com/docker/docker (#9817)
gitpod /workspace/prometheus/empty $ go test -short -failfast -count 10 -test.v  -run TestAgentAPIEndPoints ./web/
=== RUN   TestAgentAPIEndPoints
=== PAUSE TestAgentAPIEndPoints
=== CONT  TestAgentAPIEndPoints
    web_test.go:605:
                Error Trace:    web_test.go:605
                Error:          Not equal:
                                expected: 200
                                actual  : 500
                Test:           TestAgentAPIEndPoints
--- FAIL: TestAgentAPIEndPoints (0.00s)
FAIL
FAIL    github.com/prometheus/prometheus/web    0.028s
FAIL


Making a container to test SaltStack, is this pattern reasonable?

Hi! I'm have been working on a testcontainer for SaltStack, which is a OS configuration tool.
My motivation is that configuration tooling also needs to test its modules, and the normal system, salt-kitchen makes some assumptions that breaks with what i would call common structuring of the system :)
So I've made a workable solution extending the regular DockerImage with extra functionality to generate the correct configuration and mount of the roots we need, but what I'm a bit stuck at is what the running command should be. As this is a setup where the actual command (apply state) runs once and exits. From what i see/understand, by exiting the container I'm not able to start the next step of testing where i actually validate the state with help of for example testinfra
So right now I've opted for a two stage approach: Where the DockerFile stays alive with CMD [ "tail", "-f", "/dev/null" ] and the user invokes the state apply command after container.start()
    def get_salt_call_args(self, command: str = "state.apply") -> list[str]:
        return [
            "salt-call",
            "--local",
            f"--config-dir={self.config_dir}",
            f"--id={self.config_file['id']}",
            f"{command}",
        ]

    def exec_salt_call(self, command: str = "state.apply") -> tuple[int, bytes]:
        return self.exec(self.get_salt_call_args(command))

How wrong of a pattern is this for a testcontainer? From what i see all others are mainly focusing on starting services to support, and not directly being used.
Would it make sense to run a init container that runs the apply, then a new instance that i can run the state inspection tests on afterwards? Or maybe run the apply as a part of the Image building?
PS: I know it does not make a lot of sense to use a Config management tool inside a container, the point here is simply to quickly test modules that will later be used on proper operating systems :)

And here is a published variant just to show off the base idea:
https://github.com/N0K0/testcontainers-salt-poc


[Feature] Direct database access

Today, the SQL database design (table names & encodings) is considered part of the private API - the only supported way to access this data is via the ORM within event handler functions. To unlock more flexible use cases, we'd like to add the database design to the public API.
Checklist:

[ ] Stable/predictable table names, so downstream applications can reference tables reliably
[ ] Transparent encodings for all data types (#334)
[ ] Documentation. We'll need to add a new page/section with answers to questions like:

[ ] Accessing the database (connection URL, SQLite file path)
[ ] How to find/predict table names
[ ] Mapping of schema types -> database column types by engine
[ ] Potential footguns

Background
Each Ponder project repo contains one logical “app”. An app has two primary components: 1) a schema, and 2) a set of event handlers, which contain the business logic.
An app can span multiple EVM networks. For example, the app might listen to Registration events on mainnet and GameAction events on optimism. The Registration event might create an Account entity, and GameAction events might update a field on the same Account entity (eg Account.balance). When I say “multi-chain app”, I’m referring to an app that works this way.
It’s also common for apps to be deployed to multiple environments. In the single chain case (vast majority of apps today), people use EVM chains as environments. A common example would be:
Local -> Anvil
“Staging”/Testnet -> Goerli
Prod -> Mainnet
Most people don’t think about it this way - they refer to it as local/testnet/mainnet. Environments and networks are interchangeable in their worldview.
In the multichain case, this looks like:
Local -> Anvil 1 + Anvil 2
Staging/testnet -> Goerli + Optimism Goerli
Prod -> Mainnet + Optimism mainnet
While it’s quite rare for folks to build multi-chain apps today, I have a hunch that it’s going to become more common, so I’d like Ponder to support it from day one. It actually already does - this table namespacing issue is one of the last sticking points.
IMO the challenge is finding a solution that satisfies the above requirements without confusing folks who think entirely in a single-chain app world. There’s already a ton of confusion for people who use the term “multichain” to mean “I have my app deployed on both goerli and mainnet”.
Proposal

Add a required “name” field to ponder.config.ts. This represents the top-level namespace for the app. An app has a single schema and a single set of event handlers.
Adopt a pattern where an app has one config file per environment. For the vast majority of single-chain apps today, this would look like:

/my-ponder-app
  ponder.config.ts  <- local
  ponder.goerli.config.ts
  ponder.mainnet.config.ts

A multichain app might look like:
/my-ponder-app
  ponder.config.ts  <- local
  ponder.testnet.config.ts OR ponder.staging.config.ts
  ponder.mainnet.config.ts OR ponder.prod.config.ts

Use {app_name}-{environment_name}-{table_name} as a stable name for database views that are backed by the latest version of the table. Zero-downtime deployments happen at the {app_name}-{environment_name} level. During a ZDD, the "new" instance updates the views to point at its tables (just like we're doing in #335).

I like this, and I think it's 90% correct. One potential annoyance is that users will need to repeat some information in their config files - notably, the new "name" field, as well as most of the contracts/filters/factories stuff. Also, I really don't know how the Foundry integration will actually look, and if it'll be intuitive alongside this pattern. There's also the broader annoyance of ABI/address management, but that's kind of a separate issue.

Just some thoughts on the database stuff:

Table Names: We're dealing with camel case in ORM, right? Kinda annoying in Postgres since you gotta use double quotes all the time. Snake case would be way smoother.

Schema Stuff: How about letting us set up a specific schema for Ponder? Right now, everything goes into the public schema unless you're doing the extra step of creating a Ponder DB user. Would be neater to have it all in its own space.

Views on Event Tables: Would be super cool if we could create views on the tables that have event data and get to them through the API. Would make things a lot easier to handle.

Came here to +1 this as someone who is interested in the indexing data and blending it with other information to present a unified API

As is right now, pretty impossible to deal with at the database level


Modified default value of react_app_path to {{ react_checkout_path }}/app

Modified default value of react_app_path to {{ react_checkout_path }}/app

We decided to leave this as is because not all applications have the app directory. Advised to, instead, override in inventories. See this https://github.com/onaio/ansible-react/pull/22#discussion_r724999810

We decided to leave this as is because not all applications have the app directory. Advised to, instead, override in inventories. See this #22 (comment)

Ah I see, okay. 👌


Remove a couple of unused constants from src/shared/util.js

These constants were added "speculatively" in PR #10820, almost four years ago, but have never actually been used. We already have issue #10982 that tracks potentially extending support for the affected annotation-format, however until that happens I really don't think that we should keep shipping completely unused code in the PDF.js library.
For the MOZCENTRAL build-target, i.e. the Firefox PDF Viewer, this reduces the total bundle size by 1.1 kilo-byte.

/botio test

I agree; they can easily be put back once that functionality is implemented. Thanks!


Remove Secure Score from MS.AAD.7.2v1

🗣 Description
Update MS.AAD.7.2v1 to remove the changes that had been made to check the Identity Secure Score value for "RoleOverlap" which checks the tenant for the score for implementation of fine grained administrative roles. Although the checks worked on the G3, G5, and E5 tenants it was not working on the GCC High tenant. The decision was made to investigate use of Secure Score or an alternative approach in the next release.

🧪 Testing

Run the tool against AAD to verify that MS.AAD.7.2v1 is restored to a state of Result: N/A and Criticality: Shall/Not Implemented. Run the unit tests to verify all MS.AAD.7.x tests are passing.

✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] All future TODOs are captured in issues, which are referenced
in code comments.
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[x] All relevant repo and/or project documentation has been updated
to reflect the changes in this PR.
[x] Tests have been added and/or modified to cover the changes in this PR.
[x] All new and existing tests pass.

✅ Pre-merge checklist

[ ] Revert dependencies to default branches.
[ ] Finalize version.

✅ Post-merge checklist

[ ] Create a release.

@nanda-katikaneni PR is ready to merge into emerald.


[SPARK-31756][WEBUI][3.0] Add real headless browser support for UI test

What changes were proposed in this pull request?

This PR backports the change of #28627 .
Some fixes for UI like #28690 should be merged to branch-3.0 including testcases so #28627 is need to be backported.
This PR mainly adds two things.

Real headless browser support for UI test
A test suite using headless Chrome as one instance of  those browsers.

Also, for environment where Chrome and Chrome driver is not installed, ChromeUITest tag is added to filter out the test suite.
By default, test suites with ChromeUITest is disabled.
Why are the changes needed?

In the branch-3.0, there are two problems for UI test.

Lots of tests especially JavaScript related ones are done manually.
Appearance is better to be confirmed by our eyes but logic should be tested by test cases ideally.

Compared to the real web browsers, HtmlUnit doesn't seem to support JavaScript enough.
I added a JavaScript related test before for SPARK-31534 using HtmlUnit which is simple library based headless browser for test.
The test I added works somehow but some JavaScript related error is shown in unit-tests.log.

======= EXCEPTION START ========
Exception class=[net.sourceforge.htmlunit.corejs.javascript.JavaScriptException]
com.gargoylesoftware.htmlunit.ScriptException: Error: TOOLTIP: Option "sanitizeFn" provided type "window" but expected type "(null|function)". (http://<IP_ADDRESS>:60724/static/jquery-3.4.1.min.js#2)
        at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:904)
        at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:628)
        at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:515)
        at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:835)
        at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:807)
        at com.gargoylesoftware.htmlunit.InteractivePage.executeJavaScriptFunctionIfPossible(InteractivePage.java:216)
        at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptFunctionJob.runJavaScript(JavaScriptFunctionJob.java:52)
        at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptExecutionJob.run(JavaScriptExecutionJob.java:102)
        at com.gargoylesoftware.htmlunit.javascript.background.JavaScriptJobManagerImpl.runSingleJob(JavaScriptJobManagerImpl.java:426)
        at com.gargoylesoftware.htmlunit.javascript.background.DefaultJavaScriptExecutor.run(DefaultJavaScriptExecutor.java:157)
        at java.lang.Thread.run(Thread.java:748)
Caused by: net.sourceforge.htmlunit.corejs.javascript.JavaScriptException: Error: TOOLTIP: Option "sanitizeFn" provided type "window" but expected type "(null|function)". (http://<IP_ADDRESS>:60724/static/jquery-3.4.1.min.js#2)
        at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpretLoop(Interpreter.java:1009)
        at net.sourceforge.htmlunit.corejs.javascript.Interpreter.interpret(Interpreter.java:800)
        at net.sourceforge.htmlunit.corejs.javascript.InterpretedFunction.call(InterpretedFunction.java:105)
        at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.doTopCall(ContextFactory.java:413)
        at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.doTopCall(HtmlUnitContextFactory.java:252)
        at net.sourceforge.htmlunit.corejs.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3264)
        at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$4.doRun(JavaScriptEngine.java:828)
        at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:889)
        ... 10 more
JavaScriptException value = Error: TOOLTIP: Option "sanitizeFn" provided type "window" but expected type "(null|function)".
== CALLING JAVASCRIPT ==
  function () {
      throw e;
  }
======= EXCEPTION END ========

I tried to upgrade HtmlUnit to 2.40.0 but what is worse, the test become not working even though it works on real browsers like Chrome, Safari and Firefox without error.
[info] UISeleniumSuite:
[info] - SPARK-31534: text for tooltip should be escaped *** FAILED *** (17 seconds, 745 milliseconds)
[info]   The code passed to eventually never returned normally. Attempted 2 times over 12.910785232 seconds. Last failure message: com.gargoylesoftware.htmlunit.ScriptException: ReferenceError: Assignment to undefined "regeneratorRuntime" in strict mode (http://<IP_ADDRESS>:62132/static/vis-timeline-graph2d.min.js#52(Function)#1)

To resolve those problems, it's better to support headless browser for UI test.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

I tested with following patterns. Both Chrome and Chrome driver should be installed to test.

sbt / with default excluded tags (ChromeUISeleniumSuite is expected to be skipped and SQLQueryTestSuite is expected to succeed)
build/sbt -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver "testOnly org.apache.spark.ui.ChromeUISeleniumSuite org.apache.spark.sql.SQLQueryTestSuite"

sbt / overwrite default excluded tags as empty string (Both suites are expected to succeed)
build/sbt -Dtest.default.exclude.tags= -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver "testOnly org.apache.spark.ui.ChromeUISeleniumSuite org.apache.spark.sql.SQLQueryTestSuite"

sbt / set test.exclude.tags to org.apache.spark.tags.ExtendedSQLTest (Both suites are expected to be skipped)
build/sbt -Dtest.exclude.tags=org.apache.spark.tags.ExtendedSQLTest -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver "testOnly org.apache.spark.ui.ChromeUISeleniumSuite org.apache.spark.sql.SQLQueryTestSuite"

Maven / with default excluded tags (ChromeUISeleniumSuite is expected to be skipped and SQLQueryTestSuite is expected to succeed)
build/mvn -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver -Dtest=none -DwildcardSuites=org.apache.spark.ui.ChromeUISeleniumSuite,org.apache.spark.sql.SQLQueryTestSuite test

Maven / overwrite default excluded tags as empty string (Both suites are expected to succeed)
build/mvn -Dtest.default.exclude.tags= -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver -Dtest=none -DwildcardSuites=org.apache.spark.ui.ChromeUISeleniumSuite,org.apache.spark.sql.SQLQueryTestSuite test

Maven / set test.exclude.tags to org.apache.spark.tags.ExtendedSQLTest (Both suites are expected to be skipped)
build/mvn -Dtest.exclude.tags=org.apache.spark.tags.ExtendedSQLTest  -Dspark.test.webdriver.chrome.driver=/path/to/chromedriver -Dtest=none -DwildcardSuites=org.apache.spark.ui.ChromeUISeleniumSuite,org.apache.spark.sql.SQLQueryTestSuite test

CC: @gengliangwang @dongjoon-hyun
I've open this PR in response to the discussion.

Test build #123425 has started for PR 28702 at commit b0b43dc.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/28042/
Test PASSed.

Test build #123425 has finished for PR 28702 at commit b0b43dc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123425/
Test PASSed.

Thanks, merging to 3.0


Action delegates missing from 1.1

There were Action delegates in LinqBridge 1.0 and there are no now.
Is that intentional?

Original issue reported on code.google.com by<EMAIL_ADDRESS>on 28 Jan 2010 at 10:01

Yes, the Action delegates were removed because they were there for the ForEach
operator and which was also removed in 1.1 release. For rationale, see issue #9.

Original comment by azizatif on 28 Jan 2010 at 6:42

Changed state: WontFix

The Action delegates will be added back. The removal was an oversight due to
misremembering of the project scope, which specifically mentions them.

Original comment by azizatif on 4 Feb 2010 at 11:06

Changed title: Action delegates missing from 1.1
Changed state: Accepted
Added labels: Priority-High
Removed labels: Priority-Medium

Fixes in r238.

Original comment by azizatif on 4 Feb 2010 at 11:11

Changed state: Fixed

Original comment by azizatif on 16 Nov 2010 at 10:11

Added labels: Milestone-Release1.2


Time normalization picking up artifacts

When reading documents, I sometimes get artifactual timex extractions like from the sentence
This document updates the IPC projections made in December 2015 , available here :
http://www.ipcinfo.org/ipcinfo-detail-forms/ipcinfo-map-detail/en/c/379480/

in the document IPC_SouthSudan_Acute_FI-Situation_April_2016.txt we get
49063       }, {
49064         "@type" : "TimeExpression",
49065         "@id" : "_:TimeExpression_34",
49066         "startOffset" : 14189,
49067         "endOffset" : 14195,
49068         "text" : "379480",
49069         "intervals" : [ {
49070           "@type" : "TimeInterval",
49071           "@id" : "_:TimeInterval_34",
49072           "start" : "+379480-01-01T00:00",
49073           "end" : "+379481-01-01T00:00",
49074           "duration" : 31622400
49075         } ]

If this is something that could be improved or at least worth looking into on the reader side, I can send a longer list of examples, otherwise, feel free to close the issue!

That's a keen eye you have.  It looks like "text" : "379480", should be "text" : "December 2015",.

This seems to be a mistake from the temporal parser that identifies "379480" as a year. There is not much we can do for the moment. We are working on an improved version of the model that, hopefully, will be integrated into eidos soon.

Yes, they could help us to perform error analysis. Thanks.

Okay, I emailed you a log with all the invalid entries from the evaluation corpus, I'll close the issue here.


Email cubject length for personalisation

When attempting to send a subject with a length > 78, I receive a response indicating the subject must be less than 78 chars.
Is this intended?  If so, perhaps there should be something in the docs to indicate this.

@birdy247,
This issue was fixed on the API side a few weeks ago. We have removed this limit.
I just tried to reproduce this error and I could not.
Could you please give me your step by step to help reproduce?


HDDS-5247. EC: Create ECReplicationConfig on client side based on input string

What changes were proposed in this pull request?
This patch introduces a new string contsructor for ECReplicationConfig to make it possible to define EC replication scheme for any new key from CLI.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-5247
How was this patch tested?
New unit test is added.

The change looks good, minor comment - I think we need to throw exception for an unsupported scheme E.g. 10+10, and add a test case for that.
In the future we could make the list of supported policies pluggable/configurable.

Thanks for the question/suggestion @arp7 and @umamaheswararao. I also considered throwing an exception instead of printing out a warning, but finally, I decided to put only a warning here. Let me share my view here:

First of all it's a client-side change. Any serious validation should be placed on the server/OM side. (from this point of view even warning can be removed)

It's hard to predict all the possible use cases. I agree that we should introduce a configuration option to make it possible for vendors to limit the available ReplicationConfig/ECReplicationConfig set. But the Apache Ozone code should be as much flexible as possible (IMHO). Example: earlier we hard-coded support of ONE/THREE replication scheme for closed containers, but it turned out that there is a demand to use other values (like TWO)

There or other meaningful combinations: for example if dozens of nodes are available and the key sizes are bigger, it may be reasonable to use 6-4 instead of 3-2. The write amplification is the same, but 6-4 can have better durability (in exchange for more padding + memory usage).

I can imagine even 10-10 as a useful option with very specific/smart placement policies (one may require more parity to place them to other racks). But I admit it would be a very rare case.

My suggestion:
Remove the warning from here and put a server-side validation based on the configuration value. I am fine to restrict it to 3-2,6-3,10-4 by default, but power users can modify the defaults if they require it.
Will create this follow-up jira if no objection.


Issue: Zero value returned ... unless the brightness is changed with SetBrightnessLevel

I can modify the brightness level and then get its new right value. But if I don't change it, a value equal to zero is returned by BrighnessLevel property !
In addition, an error comes up when getting (only read) the system brightness level even with the SystemManagement capability.
It seems that these issues didn't exist before the last version of Windows (1803).

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb00e107-4d15-4d52-44c5-21a4751a1824
Version Independent ID: e5cff65f-4c9b-93ae-8ff6-815580e7e505
Content: BrightnessOverride.BrightnessLevel Property (Windows.Graphics.Display) - UWP app developer
Content Source: https://cpubwin.visualstudio.com/DefaultCollection/windows-uwp/_git/winrt-api-build?path=%2fwinrt-api-build%2fwindows.graphics.display.brightnessoverride.yml&version=GBlive&_a=contents
Product: windows
Git Login: wdg-dev-content
Microsoft Alias: wdg-dev-content

By chance, I have another tablet with the version 1709 of Windows. I made the same test with it and I do get the right value of the brightness level after using GetForCurrentView. With the 1803 version, the returned value is equal to zero as mentionned in the initial post.
In both cases (both versions), getting the value after using GetDefaultForSystem return an error.

Hi @loulouMS, thanks for your question.
BrightnessOverride is intended only for setting and then temporarily overriding the brightness level, not for querying or retrieving the current brightness level.  The BrightnessLevel property just returns the level specifically set on that particular BrightnessOverride instance (by a previous call to SetBrightnessLevel).
The behavior you were seeing prior to version 1803 was unintentional and has been updated.
Call GetLevelForScenario to determine the brightness level for specific user scenarios.

Hi @mtoepke, thanks for the information.
But the problem is ... : how to know which scenario is currently active?
And what is the purpose of GetDefaultForSystem method? I remain that it always returns an error even if we add the SystemManagement capability.

Hi @loulouMS , thanks for your questions.
These functions are for setting the brightness level, not querying it.  According to the developer, there are no UWP APIs that return the current screen brightness.
You do not know which scenario is currently active unless you're app has overridden the brightness itself.  I mentioned GetLevelForScenario as a way to determine the brightness level to set, in your own app, based on your app's current scenario.
GetDefaultForSystem does not return the system's brightness.  It returns a BrightnessOverride instance that still requires you to set the brightness on the BrightnessOverride before using the BrightnessLevel property to check the value that you have set.  GetDefaultForSystem is the same as GetForCurrentView, except it is used for background IoT apps that don't have a CoreWindow.
I'm also seeing a failure return code (E_FAIL) calling GetDefaultForSystem.  It turns out that call also requires enabling Embedded Mode, which is only supported in IoT.  UWP apps have a CoreWindow, so should use GetForCurrentView instead.  We'll add that to the docs.
If you're working on an IoT app, see https://docs.microsoft.com/en-us/windows/iot-core/develop-your-app/embeddedmode for info on embedded mode.
Thanks,
-Mike

Hi
We have following Use case:
We are developing an Tablet Kiosk App (Assigned access). The user needs the ability to change the brightness, therefor we add an Slider control to change the brightness. How can I set the slider  initial value , to the current brightness. (It worked proper bevor 1803)


add clearer mention of BN-254 to docs

Our stark2snark translator is operating over BN-254; we should make that clearer from the docs.
Landing the security model docs will address this to some degree, but it probably merits a mention elsewhere as well. e.g., probably on the proof system overview page and maybe some of the terminology entries

happy to revisit this but at the moment i think the security model doc is sufficient here.


False positive reports for safe Project properties

Not all allprojects accesses are evil. If you look at how CrossProjectModelAccess creates ProblemReportingProject instances, they actually list all the properties that are unsafe. These can be indentified by looking at the onAccess() calls.
Here are a few examples which are safe:

Project.toString
Project.name
Project.path
Project.rootDir
Project.displayName
Project.childProjects
Project.findProject
Project.parent
Project.rootProject

So essentially it's safe to navigate the hierarchy of projects, as long as we don't access anything that requires potential evaluation of the project.
I know this might be a bit of a feature, but even a best-effort trivial implementation of some kind of data flow analysis would be nice.
Example safe use case (based on the above restrictions):
project.tasks.register<Task>("allDependencies") {
	val projects = project.rootProject.allprojects.sortedBy { it.path }
	doFirst {
		println(projects.joinToString(prefix = "Printing dependencies for modules:\n", separator = "\n") { " * ${it}" })
	}
	dependsOn(projects.map { "${it.path}:dependencies" })
}

Current report (above code resides in gradle/plugins/src/main/kotlin/root.gradle.kts:
root_gradle$2#invoke(Ljava.lang.Object;)Ljava.lang.Object; ->
  root_gradle$2#invoke(Lorg.gradle.api.Task;)V ->
  org.gradle.api.Project#getAllprojects()Ljava.util.Set;

Thanks for the issue. This isn't something I'd be willing to spend time working on. I personally feel that it is too subtle to have high value for the vast majority of projects. However, PRs would be welcome.


[C++] Possible data race when reading metadata of a parquet file

Describe the bug, including details regarding any error messages, version, and platform.
The first and the last line of this block of code access the same metadata variable but only one of them does so holding a lock.
I assume this means the other one should too.
There are some other places in this file that access metadata in tricky ways (e.g. it is not clear from a first glance at a method whether nullptr is allowed or not). They could also race.
https://github.com/apache/arrow/blob/0dbbd43ca9133912d1809394727784560cc5e797/cpp/src/arrow/dataset/file_parquet.cc#L607-L618
Component(s)
C++, Parquet

The first and the last line of this block of code access the same metadata variable but only one of them does so holding a lock.

Which metadata variable?
Which lock are you talking about?

https://github.com/apache/arrow/blob/0dbbd43ca9133912d1809394727784560cc5e797/cpp/src/arrow/dataset/file_parquet.h#L221
https://github.com/apache/arrow/blob/0dbbd43ca9133912d1809394727784560cc5e797/cpp/src/arrow/dataset/file_parquet.cc#L783

Well, I'm not sure why it would be a problem in the snippet above. metadata() in line 607 returns before EnsureCompleteMetadata() in line 618 executes.

In this call, sure. But by the same reasoning the locking is not needed at all.
I assume (and I've observed it in TSAN reports) that this method can be called from multiple threads, with the intention of caching metadata by the first thread that arrives to read it.

Hmm, then please post a corresponding TSAN report, this would help understand the issue.

Thanks! So, trying to sum it up:

main thread has an Acero source node whose StartProducing calls FragmentsToBatches calls ParquetFileFormat::ScanBatchesAsync

    #2 arrow::dataset::ParquetFileFormat::ScanBatchesAsync(std::shared_ptr<arrow::dataset::ScanOptions> const&, std::shared_ptr<arrow::dataset::FileFragment> const&) const $ARROW_ROOT/cpp/src/arrow/dataset/file_parquet.cc:607:36 (arrow_reproduce+0x126e20b)
    #3 arrow::dataset::FileFragment::ScanBatchesAsync(std::shared_ptr<arrow::dataset::ScanOptions> const&) $ARROW_ROOT/cpp/src/arrow/dataset/file_base.cc:188:19 (arrow_reproduce+0x1156715)
    #4 arrow::dataset::(anonymous namespace)::FragmentToBatches(arrow::Enumerated<std::shared_ptr<arrow::dataset::Fragment>> const&, std::shared_ptr<arrow::dataset::ScanOptions> const&) $ARROW_ROOT/cpp/src/arrow/dataset/scanner.cc:305:3 (arrow_reproduce+0x11c4fa1)
    #5 arrow::dataset::(anonymous namespace)::FragmentsToBatches(std::function<arrow::Future<std::shared_ptr<arrow::dataset::Fragment>> ()>, std::shared_ptr<arrow::dataset::ScanOptions> const&)::$_0::operator()(arrow::Enumerated<std::shared_ptr<arrow::dataset::Fragment>> const&) const $ARROW_ROOT/cpp/src/arrow/dataset/scanner.cc:333:36 (arrow_reproduce+0x11c4fa1)

Arrow worker thread executes a Future callback from ParquetFileReader::OpenAsync's own callback, that in turn calls ParquetFileFormat::ScanBatchesAsync

    #4 arrow::dataset::ParquetFileFragment::SetMetadata(std::shared_ptr<parquet::FileMetaData>, std::shared_ptr<parquet::arrow::SchemaManifest>) $ARROW_ROOT/cpp/src/arrow/dataset/file_parquet.cc:819:13 (arrow_reproduce+0x12764c2)
    #5 arrow::dataset::ParquetFileFragment::EnsureCompleteMetadata(parquet::arrow::FileReader*) $ARROW_ROOT/cpp/src/arrow/dataset/file_parquet.cc:811:10 (arrow_reproduce+0x1275b25)
    #6 arrow::dataset::ParquetFileFormat::ScanBatchesAsync(std::shared_ptr<arrow::dataset::ScanOptions> const&, std::shared_ptr<arrow::dataset::FileFragment> const&) const::$_0::operator()(std::shared_ptr<parquet::arrow::FileReader> const&) $ARROW_ROOT/cpp/src/arrow/dataset/file_parquet.cc:618:5 (arrow_reproduce+0x1283f68)

What seems weird is that two ScanBatchesAsync calls would happen on the same fragment.

ScanBatchesAsync is only called from FragmentToBatches, which in turn can only be called from a "scan" node.
One possible explanation would be if Dataset::GetFragments returns the same fragment instance twice. How did you create your dataset?

In any case, it would probably be good for metadata() to take the lock before reading its result value.

Dataset creation itself looks quite vanilla to me
  std::vector<std::string> paths = FillPaths();
  arrow::dataset::FileSystemFactoryOptions options;
  options.partitioning = arrow::dataset::HivePartitioning::MakeFactory();
  options.partition_base_dir = root_path;
  options.exclude_invalid_files = true;
  return arrow::dataset::FileSystemDatasetFactory::Make(
             std::make_shared<arrow::fs::LocalFileSystem>(),
             paths,
             std::make_shared<arrow::dataset::ParquetFileFormat>(),
             options
  )
      .ValueOrDie()
      ->Finish()
      .ValueOrDie();

However, when reading it I do this (for filtering columns and such)
  arrow::dataset::internal::Initialize();
  std::vector<std::shared_ptr<arrow::Field>> fields;
  for (const auto& i : dataset->schema()->fields()) {
    if (options.fields_.empty() || contains(options.fields_, i->name())) {
      fields.push_back(i);
    }
  }
  dataset = dataset->ReplaceSchema(std::make_shared<arrow::Schema>(fields, dataset->schema()->metadata())).ValueOrDie();

I don't know if this is idiomatic enough

That looks reasonable to me.

I assume (and I've observed it in TSAN reports) that this method can be called from multiple threads, with the intention of caching metadata by the first thread that arrives to read it.

  Parquet datasets will cache parquet metadata the first time it is loaded.  So to trigger this you would need to:

Create a dataset
Simultaneously scan the dataset multiple times

So it is possible that we have one thread testing if a shared_ptr is null at the same time another thread is assigning to it.  Assigning to a shared_ptr is maybe one of those things that is probably atomic but not necessarily documented to be atomic.
Either way, suppressing TSAN reports is a valid enough reason to fix this.  The ParquetFileFragment::metadata method should grab the lock (which means it is no longer a const method).


Engine bump for next semver major

I have some ideas for things to improve in this module but it's kind of a pain to have them all work and tested with the ancient versions of node referenced in the engines block (>=0.6). For reference the last release of 0.6 was July 2012 and it predates any formal long term maintenance policy. The oldest LTS release was 0.10 and that reached end of life Oct 2016.
Any thoughts / objections to having the next semver major of this module support only current LTS versions? As of writing that would be 8+. Note that I'm not suggesting a rolling LTS-only policy, only that the next major version could leverage 8+ code and APIs.

So the question would be if it's actually impossible to support in older versions or just harder. Bumping it is doable, but doing so would also end up by extension bumping the Node.js required in all the modules that depend on this module.
The typical thought is that it needs to be impossible to actually accomplish something in the older versions. If you just need help getting it to work in current and old I can always lend a hand as well.

Likely more in the "just harder" category. One annoying thing I ran into is that even shims like buffer-from don't support 0.6 as internally it calls Buffer.isEncoding(...) which is >=0.9.
Figured I'd see the overall policy on bumping majors before digging into a a more expansive workaround.


INFRA-24226: Add a default minimum page count from .asf.yaml if <= 0 specified by CLI args

The other places, like the kickbuild cronjob will need to be adjusted, yes. That just can't happen in this PR.

Naturally - this PR is independent of the other places though, as they will continue to override this feature until they are tweaked.


Fix typo in MongoDB memory docs

Description: Fix typo in MongoDB memory docs
Tag maintainer: @eyurtsev

thank you @sijunhe


fix: Sidebar is not closing when clicking outside #1183

PR Fixes:

1  Sidebar now closes automatically when clicked outside.

Screencast from 2024-09-11 11-42-57.webm
Resolves #1183
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I assure there is no similar/duplicate pull request regarding same issue

Thank you sir for your contribution

Thank you sir for your contribution

Thank you sir for merging first pr of my life.


iOS/Apple, Surface Pro - not able to receive the stream (the root cause provided)

Your environment.

Browser: Safari 13.1.2 with Chrome on Windows
Other Information - see below

What did you do?

Opened Chrome on Windows and started the stream.
Opened Safari or Chrome on macOS and couldn't receive the stream.

What did you expect?
See the stream.
What happened?
I couldn't see it.
After https://github.com/pion/ion-sdk-js/pull/28 fix, I was able to experiment a bit more with Safari.
It turns out, that in the SDP Offer Safari has:

(...)
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 125 104
(...)
a=rtpmap:96 H264/90000
(...)
but in the SDP answer we have:
m=video 9 UDP/TLS/RTP/SAVPF 96
(...)
a=rtpmap:96 VP8/90000
For Safari, to get VP8/90000, the SAVPF should be set to 100.
96 works flawlessly with Chrome (on other OS than Mac/iOS), as with Chrome VP8/90000, indeed, it is 96.
To fix that, for maximum compatibility in the future, we probably need to look-up VP8 in the SDP Offer and return the same value to the browser.
Potential solution/fix/help here: https://github.com/pion/webrtc/issues/716#issuecomment-502609027

Also, this is the reason for the change: https://groups.google.com/forum/#!topic/discuss-webrtc/Z5YOWMC8TwU
It looks like Safari is using WebRTC before M57. Hence the reason why 100, but not 96.
Key quote from that thread: "External apps need to update their code to not rely on hardcoded values. This is all in accordance with the spec, payload types in the range 96-127 are dynamic and are not tied to any specific codec."
Most likely types will change again in the future.

@greko6 this issue exists iOS platform mobile and iPad in general. Since even chrome in iphone and ipad uses WebKit rendering engine not V8 just the chrome UI. So any app built on ion-sdk-js is not working on iphone and ipad irrespective of the browser

@tarrencev are we updating the docker image anytime now to include this fix?

I did compile it and created a new docker for my own SFU for testing purposes - after the patch was applied.
Unfortunately, SFU is still not matching it. Here is what I can see:
Offer:

Answer:

The answer should be:
m=video (...)
(...)
a=rtmpmap: 100 VP8/9000
With 96 Safari is still interpreting the video track as h264.

Surface Pro tablet is also affected by the same bug. As it's not fixed, can you reopen it, please?

@tarrencev could you re-open this issue, please? That way people that are using sfu will have much better visibility it's not working.

Hey @greko6 if you go to https://github.com/orgs/pion you should have an invite to join the org!
If you aren't able to join I will re-open, but hard for me to find bandwidth to help with Ion day-to-day :(

@Sean-Der : I think one of the GitHub org limitations/features is that other org members cannot re-open issues. Only the person that closed it can.
Source: https://stackoverflow.com/questions/21333654/how-to-re-open-an-issue-in-github
"You cannot re-open your own issues if a repo collaborator closed them"

@Sean-Der : This issue, once solved, will allow pion to drastically accelerate as currently, people that are testing it with MacOSes/iPhones/iPads are not getting the streams at all. More recently, I've learned that also Microsoft Surface Pro is affected because of the same simple mismatch reasons.

Update based on the Slack thread.
TL;DR most likely we'll need to wait for v3
From Jeremia:
"The main issue here is that in the current version of pion/webrtc the PayloadTypes are defined in the webrtc.MediaEngine and are usually either populated from SDP or defined before any tracks are added. Populating from SDP does not scale well if you need to dynamically add tracks after the first negotiation.
The easiest solution for this is to generate the offers on the server-side and then the browsers will use server-defined PayloadTypes in their answers.
I think there are plans for v3 to fix this behavior."

this is working now but only on ion-sfu not with ion. you can verify it by running ion-sfu with the pub-sub-in-browser example. you will need a reverse proxy for ssl though since safari doesn't like ws

@tarrencev : Nice! Did you receive a stream on Safari? Which diff fixed it?


Add integration test for geohex_grid

Description
Included integration test for geohex_grid.
Signed-off-by: Vijayan Balasubramanian<EMAIL_ADDRESS>Issues Resolved
#98
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #176 (9be675c) into feature/h3-grid (22285e3) will increase coverage by 0.25%.
The diff coverage is n/a.

@@                  Coverage Diff                  @@
##             feature/h3-grid     #176      +/-   ##
=====================================================
+ Coverage              85.10%   85.36%   +0.25%
- Complexity               363      364       +1
=====================================================
  Files                     52       52
  Lines                   1155     1155
  Branches                  87       87
=====================================================
+ Hits                     983      986       +3
+ Misses                   135      133       -2
+ Partials                  37       36       -1

Impacted Files
Coverage Δ

...earch/geospatial/index/mapper/xypoint/XYPoint.java
74.13% <0.00%> (+5.17%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


⚠ Jerry Savelle Ministries International Australia has degraded performance

In d908273, Jerry Savelle Ministries International Australia (https://jsmiaustralia.org) experienced degraded performance:

HTTP code: 200
Response time: 8533 ms

Resolved: Jerry Savelle Ministries International Australia performance has improved in e14ca4e after 19 minutes.


ensure tsmWriter#Write returns ErrMaxBlocksExceeded

Required for all non-trivial PRs

[x] Rebased/mergable
[x] Tests pass
[ ] CHANGELOG.md updated

@jwilder I updated the test so that the first file is 1 block short and the second file writes two blocks.


[BUG] Unable to recover a volume running on single-node cluster

Describe the bug
Longhorn seems to be unable to recover a volume/mount it and even restart doesn't help.
To Reproduce
I'm not at all sure, as this happens often with single node cluster, semi-frequently
Expected behavior
It would work without issues
Support bundle for troubleshooting
Available here for full year from now: https://z.skylab.fi/i/supportbundle_40bc13da-8de5-476e-80a3-e2b7324c47ad_2024-04-08T02-19-48Z.zip
Environment

Longhorn version:  v1.6.1
Impacted volume (PV): pvc-ff762777-14be-4f03-bac0-6847cecabdad
Installation method (e.g. Rancher Catalog App/Helm/Kubectl): Helm
Kubernetes distro (e.g. RKE/K3s/EKS/OpenShift) and version: K3s v1.28.2+k3s1

Number of control plane nodes in the cluster: 1
Number of worker nodes in the cluster: 1 (same node with control plane)

Node config

OS type and version: Ubuntu Server 22.04.4 LTS
Kernel version: 5.15.0-101-generic
CPU per node: 4C/8T
Memory per node: 32G
Disk type (e.g. SSD/NVMe/HDD): SATA SSD

Underlying Infrastructure (e.g. on AWS/GCE, EKS/GKE, VMWare/KVM, Baremetal): Baremetal
Number of Longhorn volumes in the cluster: 5

Additional context
It seems very often that longhorn has some sort of mounting issues, and it also seems to eat all file descriptors.

It seems that it managed to somewhat recover itself after I scaled down all workloads using longhorn.
However, there's still at least one pv/volume that's not wanting to behave aka mount.


Lightkurve 2 API - normalize data not in flux column

Problem description
With Lightkurve2, if one wants to normalize data that is not LightCurve.flux, one has to jump through a few hoops, comparing to with Lightkurve 1.
Example
To normalize SAP_FLUX, in Lightkurve1, one can do it in 1 line
    lc_normalized = lc.SAP_FLUX.normalize(unit='percent')

In Lightkurve2, one has to do it in a few steps (unless he/she uses the deprecated .SAP_FLUX)
    lc_sap = lc.copy()
    lc_sap['flux] = lc_sap['sap_flux']
    lc_sap['sap_flux_err'] = lc_sap['sap_flux_err']
    lc_normalized = lc_sap.normalize(unit='percent')

Suggestion
Make normalize  takes an optional column attribute:
    lc_normalized = lc.normalize(column='sap_flux', unit='percent')

The method gives a warning if the {column}_err does not exist.

Agreed!  lc.normalize() could take an optional column argument.
Note: if a user wants to work with SAP_FLUX, my recommendation is to use
lc = lk.read(filename, flux_column='sap_flux')`

or
lk.search(...).download(flux_column='sap_flux')

This will automatically populate flux and flux_err with sap_flux and sap_flux_err.

hmm, the having flux_column in read / download makes it less necessary to have column in lc.normalize()
The flux_column in read / download is not yet (publicly) documented, however.

Right!  I think adding a column parameter to lc.normalize could be a useful enhancement, e.g. we also added lc.plot(column=...), but I don't think it is necessary to go ahead with v2.0.
Absolutely correct that the flux_column option is not well documented as this time.  Lots of work left to do re docs!


Examples: async mode ("spawn") for Rust examples

It doesn't bring much in the official examples (except objectron in run-forever mode :upside_down_face:) as everything loads pretty much instantly, but still it's important to show how one would go about doing it.
The only gotcha is that the user code should be the one running on another thread, as egui might need the main thread for itself depending on the platform.

we have spawn!


[NEW ICON] SFML

Skill Name
SFML
Why?
It's popular among game & computer graphics programmers
Reference Image

Skill Name
SFML
Why?
It's popular among game & computer graphics programmers
Reference Image

Hi [User] 👋,
Thanks a bunch for reaching out! We've noticed the hiccup with updates in the repository. But no worries, we've got you covered! 😊 As an alternative, we've moved the icons to a new repository where updates are happening smoothly. You can find all the latest icons right here: Icons 🏝.
And guess what? Your requested icon, SFML, is already there! 🎉 You can find it nestled among the others, ready to shine. ✨
Feel free to swing by, take a look, and even contribute if you'd like! It's an open invitation to join the fun. 🚀
Oh, and one more thing – if you find the icons useful, we'd be over the moon if you could give the repository a star ⭐. It helps others discover our little corner of the internet and means the world to us!
Additionally, if you're in a position to support the continued development and maintenance of this project, we'd greatly appreciate your sponsorship 💖. Every contribution helps us keep the lights on and the icons flowing!
Catch you later! 👋
Warm regards,
islandboymv


Getting Started with Polynomial regression in Python

Article checklist
Pre-submission checks

Please ensure that you have only one open topic suggestion (issue) + in-review article (linked pull request) at a time. This will ensure that we complete the review process in a timely manner from inception to publishing

If you have not already, please go over our Resources Page for more tips and tricks on how to improve your overall technical writing so reviews are swifter, increase the chance of a payout and provide more value to our readers.

For first-time contributors and for more details on our submission guidelines, see our Contributing Guidelines.

Formatting and structure checks

[x] Have you used the correct folder and file structure? - see Contribution Guidelines

[x] Is your article properly formatted in Markdown? - see Github Markdown guide

[x] Have you used our preferred heading structure? - H3 (###) and up

[x] Does your article follow any of the suggested structure formats? - see example formats

[x] Is your article over 750 words? - Tool: Wordcounter.net

[x] Is the article you are submitting an in-depth and unique article? Does it go beyond what is in the official docs and what is covered in other blog sites. See these articles as examples.

[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.

[x] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.

[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images

Grammar & plagiarism checks

[x] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid - Grammarly
[x] Please place your article through a 3rd party plagiarism checker. We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[x] Have you checked your article for readability? - Tool: Hemmingway
[x] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide
[x] Have you checked your code runs correctly and you've highlighted all necessary dependencies for installation?
[x] Are the software programs and packages you're highlighting in your article up to date, using current versions and not deprecated?

Topic suggestion this closes

Remove the backticks and add the issue number below to link and close your Topic Suggestion (issue) when your article has been published (PR has been merged). See this video for more details.

This closes #4419

Finally, delete the article checklist notes in blockquotes and submit your PR. We look forward to reviewing your article.

Upon running your article through our 3rd party plagiarism checker it seemed to raise a few flags and the % was higher than we typically accept. Please see attached PDF - and revisit the article to ensure we are contributing wholly unique and original content. Be sure to see our resources page to see more info on plagiarism and what is considered as such.
Getting Started with Polynomial regression in Python #4694 Copyleaks Report.pdf

Have the necessary changes been made? Ping us once done.

@Faith034 Any updates?

@lalith1403 am working on the plagiarism part right now. I'll be notifying you shortly.

@lalith1403
Am done with the requested changes.

This article is ready for final review. Can you add the author details (headshot pic and bio) in the authors folder? Thank you.

Also, a picture is required.

@lalith1403
Everything is in place now!

@lalith1403
I've enclosed equations within dollar sign and are now okay.

jumping on this one


rpc: add --notls to disable TLS for RPC

Change Description
This adds a --notls config option that disables TLS encryption for both gRPC and REST connections.
This may be useful for when RPC communication happens only over localhost or a LAN. It may also be used in conjunction with a reverse proxy to enable TLS.
This PR expands on a previous change to add --noresttls which disables TLS encryption for REST connections only. That PR originally disabled TLS altogether, but was changed to REST only after a comment was raised about compatibility with lncli, see: https://github.com/lightningnetwork/lnd/pull/4648#discussion_r494798090
This PR makes the accompanying changes to lncli to support unencrypted requests.
Steps to Test
Run lnd from this branch using --notls. Observe that lncli commands with --notls work as expected and that without --notls they fail. Observe that REST connections (using curl) succeed with http but fail with https.

I don't think this is a good idea. gRPC is a variant of HTTP/2 that only works through normal HTTP/2 infrastructure (e.g. reverse proxies, load balancers) because its content is encrypted. If you remove the encryption layer, those components start to behave weirdly. And you need a special h2c compliant clients.

I don't know enough about grpc to comment on this, however grpc is supported on nginx and kubernetes ingress so I figure they must have worked their way around the concerns you raise here?

I don't know enough about grpc to comment on this, however grpc is supported on nginx and kubernetes ingress so I figure they must have worked their way around the concerns you raise here?

I'm pretty sure h2c (gRPC without TLS) is considered to be non-standard. So it's quite possible that even components that have basic gRPC support won't work with it.

@guggero See https://github.com/lightningnetwork/lnd/pull/4648#issuecomment-1109317765 - it's not an uncommon pattern to centralize TLS/PKI to other software. So the major use-case here is not actually disabling TLS but to let TLS-termination be taken care of by other software than LND itself.

it's not an uncommon pattern to centralize TLS/PKI to other software

I understand that, and I do see the need for this feature. I'm just giving you technical reasons why I think it's a bad idea and will lead to a lot of issues with proxy setups. We ourselves tried this a few times in a k8s based environment and ran into a couple of problems.
So just providing user support for this flag and its consequences is something I'm not looking forward to. That's why I think this should at least be accompanied by a doc that explains what h2c is, what works and what doesn't (e.g. lncli, macaroons and so on).

@sangaman, remember to re-request review from reviewers when ready

@guggero As per the links provided, looks like both nginx and k8s ingress support proxying h2c (the latter requiring a TLS cert to be provided for itself to make it gRPC proper).
Similarly for other popular reverse proxies like Traefik, envoy, haproxy.
What are macaroon-related gotchas, assuming that any applications (e.g. lncli) connect to lnd through a reverse proxy that perforrms the TLS termination?

Sorry this fell off my radar, I'll take a shot at it in the next day or two. If not I'll let you know. I appreciate it.

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@guggero author seems MIA - what's missing from you perspective to get this into shape?

I'd say mostly docs around the flag (maybe even point to a doc from within the description of the CLI flag). See comment above (https://github.com/lightningnetwork/lnd/pull/6479#pullrequestreview-967631511).

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

@sangaman, remember to re-request review from reviewers when ready

Closing due to inactivity

Closing due to inactivity

Closing due to inactivity

Closing due to inactivity

Closing due to inactivity

Closing due to inactivity

Closing due to inactivity


[SPARK-41634][BUILD] Upgrade minimatch to 3.1.2

What changes were proposed in this pull request?
Upgrade minimatch to 3.1.2
$ npm -v
9.1.2
$ npm install
added 118 packages, and audited 119 packages in 2s
15 packages are looking for funding
run npm fund for details
found 0 vulnerabilities
Why are the changes needed?
CVE-2022-3517
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GA

cc @sarutak FYI

Thank you all. Merged to master.


Partial permutation/nextPermutation in std.algorithm

wyatt.epp reported this on 2016-07-14T19:27:12Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=16282
CC List

greeenify

Description
It'd be nice to have a way to set k for n!/(n - k)!

std.algorithm.sorting.nextPermutation gives unique permutations (which is nice), but it modifies in-place and returns a boolean. (Also, why is it in sorting?)

std.algorithm.iteration.permutations actually returns the permutations, but doesn't look for uniqueness AND has no other settings.

greeenify commented on 2016-07-20T00:01:21Z
> It looks about how I had envisioned it, but I'm not really
jazzed about the idea of installing an entire mathematics package for a one-off
script I was expecting to take fifteen minutes.

Do you know about the new inline dub header? At least that's the tradeoff I am taking at the moment.

#!/usr/bin/env dub
/+ dub.sdl:
name "foo"
dependency "mir" version="0.15.3"
+/

In any case I recommend making a bit of fuzz at the NG (forum.dlang.org) as this thread isn't monitored by many people. D developers are actually quite responsive to constructive feedback :)

> As for the reasoning, I just figured this was obvious?

Yes, when I needed it I was shocked too and thus wrote the small combinatorics module :/

greeenify commented on 2016-07-18T17:41:40Z
Does maybe mir.combinatorics help you?

http://docs.mir.dlang.io/latest/mir_combinatorics.html

I once tried to push for that to be included into Phobos, but there was too much resistance. I hope eventually we can move combinatorial back to Phobos.
For more details:

https://github.com/dlang/phobos/pull/4026

My argument at that time was that e.g. Python's itertools.product has 160K uses on github.

wyatt.epp commented on 2016-07-19T22:03:44Z
I guess it might?  It looks about how I had envisioned it, but I'm not really jazzed about the idea of installing an entire mathematics package for a one-off script I was expecting to take fifteen minutes.

As for the reasoning, I just figured this was obvious? I mean, there probably isn't an introductory text on permutation that doesn't cover the constrained case.


docs(bindings/java): for BlockingOperator

.. alongside with introduction of spotless plugin.

This refers to https://github.com/apache/incubator-opendal/issues/2339


return new Date in c

Apologies if I'm missing the answer to this.  I'm only 2 days into writing duktape/c functions.
Is there a better/faster way to return a new Date?
So far this is the only thing I've come up with that works:
      duk_push_string(ctx, "(function(x){return new Date(x*1000);})");
      duk_eval(ctx);
      duk_push_int(ctx,0);
      duk_push_int(ctx,(duk_int_t)date_in_seconds);
      duk_call_method(ctx,1);
I am returning rows from a database, so it is possible that I'll be eval()ing that code many times.
Thanks in advance.

I figured it out:
(void) duk_get_global_string(ctx, "Date");
duk_push_number(ctx,1000.0*(duk_double_t)date_in_seconds);
duk_new(ctx,1);

Hi @aflin, that's the best you can do at present. There's no direct C API to push a Date instance.


Edit

👷 Deploy Preview for adoring-leavitt-e31a11 processing.

Name
Link

🔨 Latest commit
3376999073ae28ed7fe329fa21e5a5eb067239fb

🔍 Latest deploy log
https://app.netlify.com/sites/adoring-leavitt-e31a11/deploys/62a1dacb5ff9a40009c51f89


chore!: check newChunk size

Closes https://github.com/celestiaorg/rsmt2d/issues/170
Related https://github.com/celestiaorg/rsmt2d/issues/127
Breaking b/c modifies the function signature of a public function: SetCell

I just noticed https://github.com/celestiaorg/rsmt2d/issues/127 so tagging @Wondertan for review

Based on https://github.com/celestiaorg/rsmt2d/issues/127#issuecomment-1261079482 we may not want to return an error here b/c that's not ergonomic for celestia-node. Open to alternatives

@rootulp, returning an error is ok. This PR closes the #127
I forgot to use permalinks in the issue🤦🏻


Update to Laravel 5.5 and PHP7

Also updates local angular cli version and adds a syntax fix for new angular material version.

Thanks! Your pr has been merged.


Add a reference implementation of port forwarding

The specification includes port attributes, but the dev container CLI doesn't include port forwarding support.

👍On this one. While you can use appPort, this is currently only wired into the single container setup and is port publishing rather than full forwarding (e.g., if something only listens to localhost it won't work).  While implementors can communicate with a server via text stream for devcontainer exec, it's pretty common to need to communicate on a port instead.

I was thinking of dynamic port forwarding where we could start something along the lines of devcontainer port-forward --workspace-folder my_project and that would forward the ports configured in the devcontainer.json (plus any that we might list in command line arguments).
E.g., we have implemented that in the Dev Containers extension for VS Code by locally listening on ports and forward any connections using docker exec to run a little JavaScript snippet inside the container to connect to the target port there. The problem (and the reason we haven't included it in the dev container CLI) with this implementation is that it requires NodeJS inside the container. You could use socat inside the container to achieve the same, but that would then require socat to be available.

@chrmarti your example was a helpful confirmation, I had to reverse engineer VS Code to get that inside.
I added 'static' forwarding via devontainer.json in this branch (https://github.com/MunsMan/devcontainercli/tree/port_forwarding), there it implemented via the Docker CLI and the publish option. Currently, socat is not part of the default image to my knowledge, right? VS Code mounts it own node runtimes and server extensions, right?
For automatic forwarding, should it be a feature then? When we demand code in the container, it should be a feature. I was thinking if forwarding is possible by mounting sockets via volumes into the container, but I will build a prototype for myself first.

I built a prototype for automatic forwarding ports via a UNIX Socket (I had the assumption that WSL should support that), but even macOS didn't support sharing UNIX Sockets via a volume. The macOS support will not come in the near future (https://github.com/docker/for-mac/issues/483).
I build the Prototype in Rust and build images for the corresponding container, which then can directly be mounted. For Linux to Container or Container to Container, I had a Response Time which was 50 Times faster than the one of VS Code.
Due to the missing support for Unix Sockets on the other platforms, I'm considering using one static Port and Multiplexing all the other ports through it. A small program needs to run in the Container, scan for new ports and forward all traffic though the port. I would build the program in Rust, and then the CLI would need to pull the container corresponding binary and mount it into the container. The execution can be started via the postCreateCommand. On the Host, an Application would Demultiplex the Traffic and manage the listening ports.
Any Feedback on that approach?

I finished my prototype and when configured it works well for me. I published it in my autoForward repo. Currently, I have the problem, that I can't start a detached process on the host running, which just using initializeCommand, but maybe someone can give me a hind there. Furthermore, I have never packaged a feature, but I will hopefully figure that out.
It is pretty lightweight, especially compared to using VS Code for just auto-forwarding. Feedback is highly appreciated. Cheers

@MunsMan Thank you for your efforts to actually implement this feature. This is the single feature that is preventing me from moving to devcontainers as a standard. Without this, using devcontainers without any specific tooling is simply impossible.
If I have some free time, I will try out your code.

Any progress on this? It would help us a lot! 🤩

For anyone interested in this topic, I have implemented the desired functionality using a Python script called devcontainer-cli-port-forwarder. This solution is based on @chrmarti's suggestion to utilize socat for port forwarding. To make it work, simply add the following two lines to your devcontainer.json file, and you should be all set.
"onCreateCommand": "sudo apt update && sudo apt install -y socat",
"initializeCommand": "python3 .devcontainer/devcontainer-cli-port-forwarder/forwarder.py &"

I was thinking of dynamic port forwarding where we could start something along the lines of devcontainer port-forward --workspace-folder my_project and that would forward the ports configured in the devcontainer.json (plus any that we might list in command line arguments).
E.g., we have implemented that in the Dev Containers extension for VS Code by locally listening on ports and forward any connections using docker exec to run a little JavaScript snippet inside the container to connect to the target port there. The problem (and the reason we haven't included it in the dev container CLI) with this implementation is that it requires NodeJS inside the container. You could use socat inside the container to achieve the same, but that would then require socat to be available.

Until forwardPorts support is complete, how about treating forwardPorts as an alias for addPort?
It would require some workaround (e.g., add server process option --host <IP_ADDRESS>), but I think we can achieve the connection to the container.
I think this behavior is effective when using an already existing devcontainer.json.

@chrmarti Can you please share ETA for this issue?


MAID-2881: test connection drop

Adds tests to see how Crust reacts to remote peer termination.

Thanks for the pull request, and welcome! The MaidSafe team is excited to review your changes, and you should hear from @canndrew (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see CONTRIBUTOR.md for more information.


[Mandarin workshop]: Seurat standard analysis of single cell RNA-seq data

Abstract
Single cell RNA-sequencing (scRNA-seq) technology has been widely used in biomedical research over the last few years. One of the most popular platforms is the droplet-based protocol developed by 10X genomics. Expression profiles of thousands of cells can be obtained from a single run, which makes it available to investigate tissue heterogeneity at single cell level. Many tools have been developed for analysing the scRNA-seq data including the popular Seurat R package.
In this workshop, we will demonstrate how to analyse scRNA-seq data using the Seurat standard analysis pipeline. One sample from a single cell RNA atlas of human breast tissue (Pal et al. 2021. EMBO J.) will be analysed as an example. This sample was generated on the 10X platform. The cell feature counts were obtained by the cellranger pipeline. This workshop starts from the cell feature counts.
Language used in the workshop
Mandarin
Convenient day for your Long workshop.
03 November 2021
Comment field
Please contact Jinming Cheng<EMAIL_ADDRESS>and Yunshun Chen<EMAIL_ADDRESS>if you have any questions
Checklist

[X] This workshop has a Bioconductor focus or interest to Bioconductor users/developers.
[X] I checked and confirmed the BioC Asia 2021 Code of Conduct.
[X] I understood that it would be desirable for the workshop to be based on this template project.

This abstract was withdrawn after consultation.


Add Mnemonic changes

New changes were added:

Add a new deck type called public_mnemonic
Remove the mnemonic table and instead use the history table (still need to actually drop the table, but no longer in use)
New UI where users directly pick which mnemonic is better. This only appears when the user answers the card correctly
Other minor UI + logic fixes

The UI for picking mnemonics is great! One thing that would be great is adding keyboard shortcuts for picking one or the other (maybe [ for left and ] for right, to keep shortcuts all on the right hand). The fact that skip and equal preference have keyboard shortcuts makes me more tempted to skip it


No Android Studio needed!

You don't need Android Studio to create this app. The only thing I additionally needed is a USB driver from the smartphone manufacturer to connect the smartphone to the PC/Unity.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a093486f-cd54-d060-ccca-3f6067fbd7f5
Version Independent ID: 2e3beef3-2788-a1c7-803b-beadea7f0b20
Content: Quickstart - Create a Unity Android app with Azure Spatial Anchors
Content Source: articles/spatial-anchors/quickstarts/get-started-unity-android.md
Service: azure-spatial-anchors
GitHub Login: @craigktreasure
Microsoft Alias: crtreasu

Hmmm...interesting. There was certainly a time where this quickstart required Android Studio, but it is possible that it might not be required anymore. What worries me is asking everyone to go find a driver from their device manufacturer. Can you tell me more about that? Does Android Studio install the drivers for you or are you aware of devices that always require an additional driver?

Android Studio offers the possibility to install the "Google USB drivers" relatively easy. But it doesn't do the job in all cases.
I'd simply tell the developers that they need a mobile USB driver to properly connect their smartphone to Unity, to use the Build & Run option or to debug with the Monitor.exe ( logcat ). They can get it here: https://developer.android.com/studio/run/win-usb
And if this doesn't work they need a custom USB mobile driver from their smartphone manufacturer like when they have a LG smartphone ( https://www.lg.com/us/support/software-firmware-drivers -> "LG Universal Mobile Drivers" ).

@Alberto-Vega-MSFT FYI

Awesome! Thanks. We'll add a note indicating the driver route as an alternative option and that for some phones it may even be required.

@YutongTie-MSFT and @Alberto-Vega-MSFT I have checked in a change to address this issue.

@craigktreasure Hi Craig, thank you for let me know. Do you want to keep this opening or I am safe to close it?

@YutongTie-MSFT It can be closed.

Closing. Thank you.
@TiNovTec
We will now proceed to close this thread. If there are further questions regarding this matter, please respond here and @YutongTie-MSFT and we will gladly continue the discussion.


[minimal working example] Sample integration of submitit

Sample integration of submitit into the repo.
There are many open todo's (testing, refactoring) that you/we might want to do before merging though...

So I've debugged this a bit as I use it in real-life scenarios for a while now -- I'll add a PR revision tomorrow with the updates.

ping @bjuncek


Fixed rule about tense features (closes #67)

Hi @johenglisch !I'm sure this is great. Can we merge? Do you want me to review?


Input Reusability

What is the established pattern for re-using a defined input? I followed the readme heading "Creating a reusable form field input"
I have an input with the form control being passed in. What I am now wondering is how to access the results of that control in the parent component such that I can disable a button and so forth.
Any help is appreciated!

for context the child component looks like this:
import { Component, For, Show, createRenderEffect, mergeProps } from 'solid-js'
import { createFormControl, IFormControl } from 'solid-forms'
import * as Joi from 'joi'

const EmailInput: Component<{
  control?: IFormControl<string>
  name?: string
  type?: string
}> = (_props) => {
    const props = mergeProps({ control: createFormControl(''), type: 'email' }, _props)
    const emailControl = () => props.control
    const schema = Joi.string().required().email({minDomainSegments: 2, tlds: { allow: false }}).messages({
        'string.empty': 'Required',
        'string.email': 'Must be an email'
    })
    createRenderEffect(() => {
        const email = emailControl().value
        const errorMessage = schema.validate(email).error?.message
        if (errorMessage) emailControl().setErrors({ invalid: errorMessage })
        if (!errorMessage) emailControl().setErrors(null)
    })
    return (

            <input
                class="input-bordered input mb-2 w-full"
                classList={{
                    'input-error': emailControl().isDirty && !!emailControl().errors,
                    'input-primary': !emailControl().isDirty || !emailControl().errors,
                }}
                name={props.name}
                type={props.type}
<EMAIL_ADDRESS>                value={emailControl().value}
                onInput={(e) => {
                    emailControl().markDirty(true)
                    emailControl().setValue(e.currentTarget.value)
                }}
                required={emailControl().isRequired}
            />
            <Show when={emailControl().isDirty && !emailControl().isValid}>
                <For each={Object.values(emailControl().errors || {})}>
                    {(errorMsg) => {errorMsg}}
                </For>
            </Show>

    )
}

export default EmailInput

and parent component looks like this:
import { Component } from 'solid-js'
import { createFormControl} from 'solid-forms'
import Background from '../components/Background'
import EmailInput from '../components/Inputs/EmailInput'

const Login: Component = () => {
    const emailControl = createFormControl('')

    return (

            <Background />

Login

                        Email
                        <EmailInput control={emailControl} name="login-email" />

                            <button class="btn-primary btn w-full">Login</button>

    )
}

export default Login

Essentially I would like the parent to be able to monitor the passed in emailControl for changes such that if the input has errors I can disable the login button

Essentially I would like the parent to be able to monitor the passed in emailControl for changes such that if the input has errors I can disable the login button

Hi @chlorophant, maybe I'm misunderstanding the question, but it looks like you've already accomplished this with your login component. If you uncomment emailControl.isDisabled, is the "btn-disabled" class not being properly applied?
                            classList={{
                                'btn-disabled': true // emailControl.isDisabled,
                            }}>

I swear I tried this and had issues. I just left my home and won’t be back for about a week. When I get home I will follow your instructions and report back. Thank you for the help!

Works great. Thanks. I must have been tired the first time around. Thanks for the great library!

closing

Oh I see now. The issue im seeing is that the createEffect is not triggering each time the value changes. Is there a way to have it trigger each time the form state changes?

if the create effect looks like this:
    createEffect(() => {
        console.log(
            emailControl.isDirty,
            emailControl.isDisabled,
            emailControl.isPending,
            emailControl.isReadonly,
            emailControl.isRequired,
            emailControl.isSubmitted,
            emailControl.isTouched
        )
        // console.log(emailControl.value)
    })
    ```
    it wont trigger as the form changes, however if instead I just log the value, it seems to trigger every time the input changes:

    ```
        createEffect(() => {
        // console.log(
        //     emailControl.isDirty,
        //     emailControl.isDisabled,
        //     emailControl.isPending,
        //     emailControl.isReadonly,
        //     emailControl.isRequired,
        //     emailControl.isSubmitted,
        //     emailControl.isTouched
        // )
        console.log(emailControl.value)
    })
    ```

it wont trigger as the form changes, however if instead I just log the value, it seems to trigger every time the input changes:

Given the example you provided
    createEffect(() => {
        console.log(
            emailControl.isDirty,
            emailControl.isDisabled,
            emailControl.isPending,
            emailControl.isReadonly,
            emailControl.isRequired,
            emailControl.isSubmitted,
            emailControl.isTouched
        )
    })

The expected behavior is that you'll see a console log every time the isDirty, isDisabled, isPending, isReadonly, isRequired, isSubmitted, or isTouched property of emailControl changes. Typically, value updates to the control will not result in any of those properties changing so it's not necessarily surprising that you're not seeing the effect run again (the effect will only run when accessed properties change). isTouched might change if the user clicks on the input AND if you manually configured the input to update the control's touched state on focus or maybe on blur. But if you didn't do this then it won't change either.
Is this the behavior you are seeing? The reason why you're seeing values logged when you include emailControl.value is presumably because the value is changing as the user types into an input.

Ok, I do think its working correctly. I'm new to solid so its been a bit bumpy. Thanks for creating this awesome tool! have a good one.

I'm new to solid so its been a bit bumpy

FYI, under the hood, solid-forms controls are just Solidjs Store objects.


Design Metrics (HTTP) server for serving metrics data for Grafana dashboard

The Grafana dashboard, currently being developed to display metrics data for A/B/n, needs a way to obtain metrics data.
Consider creating a separate service that is dedicated to serving metrics for Grafana dashboard.
Under the covers, this server should invoke the GetMetrics() function (see here).
Part of epic https://github.com/iter8-tools/iter8/issues/1501

Grafana requires the following from  HTTP service:

get all applications. Expect that appliciation can be specified in dashboard URL. If not, need this.  future requirement
get all versions for a given application - returns list of versions. future requirement
get all metrics data for a given app, list of versions - expects result from latest signatures - returns 2 lists for each requested version: all metrics over all users and all metrics over transactions. Internally, this calls storage.GetMetrics(app, version); if list of versions is empty, return for all versions
get summary metrics for a given app, list of versions - expects result from latest signatures - returns 2 lists for each requested version: all metrics over all users and all metrics over transactions. Internally, this calls storage.GetSummaryMetrics(app, version); if list of versions is empty, return for all versions

Notes/Questions

storage.GetMetrics() api defined in https://github.com/iter8-tools/iter8/pull/1511
storage.GetSummaryMetrics() api defined in https://github.com/iter8-tools/iter8/blob/6a9bde9dbc700a4a58d15cb2301f7aef4def19dc/controllers/storageclient/badgerdb/simple.go#L183
Should storage.GetSummaryMetrics() continue to compute the summary metrics or move to http service?   leave it for now
Result of rest calls should mirror result of internal implementation as closely as possible.

Next Steps

define REST APIs
learn how to define HTTP service

There are a number of http multiplexers compatible with `net/http'. We have a very small number of APIs. Per https://www.alexedwards.net/blog/which-go-router-should-i-use, plan to use default multiplexer.

Here is some sample output for /summarymetrics. Data generated from sample A/B application.
% curl localhost:8080/summarymetrics -d '{"application": "default/backend"}'
{
   "0": {
      "NumUsers": 17,
      "MetricSummaries": {
         "sample_metric": {
            "SummaryOverTransactions": {
               "Count": 203,
               "Mean": 47.82011996059115,
               "StdDev": 28.835028215240904,
               "Min": 0.051382,
               "Max": 99.827381
            },
            "SummaryOverUsers": {
               "Count": 17,
               "Mean":<PHONE_NUMBER>941178,
               "StdDev": 1456.1383944589172,
               "Min": 15.18434,
               "Max": 6314.74141
            }
         }
      }
   },
   "1": {
      "NumUsers": 24,
      "MetricSummaries": {
         "sample_metric": {
            "SummaryOverTransactions": {
               "Count": 86,
               "Mean": 51.59040327906978,
               "StdDev": 29.819517493656722,
               "Min": 1.282591,
               "Max": 99.915949
            },
            "SummaryOverUsers": {
               "Count": 24,
               "Mean":<PHONE_NUMBER>5000006,
               "StdDev":<PHONE_NUMBER>5552966,
               "Min": 23.409473,
               "Max": 433.398395
            }
         }
      }
   }
}

Grafana dashboard requests data from Iter8 (HTTP) Metrics Service.
Data source is Iter8 Metrics service endpoint.
APIs
1. GET /metrics?application=app-name
Result (Content-Type: application/json):
{
  "metric1": {
    "histogramsOverTransactions": [
      {"version": v, "label": "a-b", "value": v},
      ...,
    ],
    "histogramsOverUsers": [
      {"version": v, "label": "a-b", "value": v},
      ...,
    ],
    "summaryOverTransactions": [
      {"version": v, "count": n, "mean" mean, "stddev"": stddev, "min": min, "max": max},
      ...
    ],
    "summaryOverUsers": [
      {"version": v, "count": n, "mean" mean, "stddev"": stddev, "min": min, "max": max},
      ...,
    ],
  },
  "metric2": {},
  ...
}

Response codes:

200 - ok
400 - request does not indicate application
404 - no such application
405 - method not allowed
500 - error processing request

Implementation:
find routemap from application name
initialize result
for each version in routemap
  call storage.GetMetrics(application, version, version.Signature)
  calculateHistogramMetrics(rawMetrics)
  insert in result
  calculateSummaryMetrics(rawMetrics)
  insert in result
return result

Comments

Implemented in https://github.com/iter8-tools/iter8/pull/1493


task_id_str can throw an exception if upstream=None for a task

luigi version: 2.3.3
Issue: If I create a task with no upstream the task_id_str method in task.py will throw an exception on lines 84 and 85.
.0 = <generator object <genexpr> at 0x10c8dc5a0>

    param_summary = '_'.join(p[:TASK_ID_TRUNCATE_PARAMS]
>                            for p in (params[p] for p in sorted(params)[:TASK_ID_INCLUDE_PARAMS]))
E   TypeError: 'NoneType' object has no attribute '__getitem__'

(Pdb) p p
None

(Pdb) u
> .../lib/python2.7/site-packages/luigi/task.py(85)task_id_str()
-> for p in (params[p] for p in sorted(params)[:TASK_ID_INCLUDE_PARAMS]))
(Pdb) params
{'history': 'e56783578', 'config': '281550889', 'upstream': None}

Solution One possible solution is to str() the arguments prior to attempting p[:TASK_ID_TRUNCATE_PARAMS], ensure that Parameter.serialize handles None although that handling might be intensional based on the warning, or we can just skip None values with an if check
Would you accept a PR for one of these solutions?

I don't understand this. What's an upstream?