[TACHYON-665]: Support for multihomed networks

This is the initial commit for supporting multihomed networks. It includes following tasks:
A. Handle user defined hostname
B. Method for auto discovering hostname
C. Hostname generation and usage refactoring
D. Documentation for expected behavior under various network setups
Current host binding strategy can be configured by:

Environment variables via tachyon-env.sh or from OS settings
Default properties via tachyon-default.properties file
Automatically discovered local host name via NetworkAddressUtils.getLocalHostName()

https://tachyon.atlassian.net/browse/TACHYON-665

Jenkins, test this please.

@pfxuan Thanks for the patch, I left a few comments, will take a closer look after these issues are resolved.

@calvinjia I just submitted 2nd commit. It should be able to address the most of your concern and suggestion. Thanks!

@pfxuan Thanks for the update, what issues are keeping this PR in [WIP]? Just the test failures?

Failures are not directly related this commit. The failed tests can go through my Mac machine as well as production cluster. I need to do some enhancement on Tachyon mini cluster.

Jenkins, see if this works.

@calvinjia Do you have any feedback for the current design? Thanks!

@pfxuan Could you add unit tests and documentation for the new features? Also the numerous connect and bind configurations are pretty verbose and there are a bunch of new methods in the network utils, is it possible to simplify them?

@pfxuan Thanks for the update! I think this looks good overall. Could you address the outstanding comments and add unit tests/documentation?

Yes! The unit tests have already implemented. I'm working on the integration tests for multihomed networks. I'll commit them soon. Thanks for your assistance. It works very well ~

@pfxuan Thanks for this patch! It definitely cleans the code base. I left some comments

@calvinjia Do we need to run integration tests for all those five services? Or we can just select two services for testing. Since other three have a similar logic. I don't know if it's necessary to repeat the test.

@pfxuan I would prefer to have the additional tests if its not too difficult to add. Also, could you add the new properties to the docs/Configuration-Settings.md file? Otherwise it looks good to me.

@calvinjia Okay! I'll add the tests for each of services, and also update docs.

Jenkins, test this please.

Jenkins, test this please.

@calvinjia  It looks like Jenkins server (amp-jenkins-worker-06) has a problem to fetch this commit. Got the following error message:
"stderr: error: RPC failed; result=18, HTTP code = 200"
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4151/console

Jenkins, test this please.

Jenkins, test this please.

@pfxuan Seems like the integration tests still have a few issues?

@calvinjia The master web test is supposed to get a HTTP 200 response code, but it shows 500 HTTP code on Jenkins server. This works on my own test environment. It looks like Tachyon unit tests load incorrect jar dependency from Jenkins server.

@calvinjia The problem is caused by incorrect Jetty version and unmatched dependency. I opened a JIRA case for solving this issue:
https://tachyon.atlassian.net/browse/TACHYON-780

@pfxuan Is it possible to resolve this problem without changing the jetty version? You mentioned the web service tests work even though they are similar?

Yes. The worker web service has no issue, but the master can not pass unit test.

Jenkins, test this please.

@pfxuan , I think this current PR is growing too big to effectively review. Is it possible to make this PR as the first step, and split additional changes from this PR to followup PRs?

@apc999 Good point! I'll see if I can detail this topic and materialize it in followup PRs. Do you have any other specific concern and requirement regarding the changes in current PR?

@pfxuan ,here is my suggestion: can we start by limiting the scope of this PR to only the newly added util methods (with necessary data structures / types you need in these util methods), and with some unit tests of these util methods? In that way, we can see clearly what's new and also merging them to the current code base will break nothing if no one is using these new util functions.

@apc999 So I should break up this patch to several PR phases, such as 1) new methods/types on util and unit tests; 2) plug in new util into each of Tachyon services; 3) Integration test; 4) configuration and documentation. Is this what you want to see?

@pfxuan , this looks reasonable to me.
Thanks

@calvinjia @apc999 I'm going to cherry-pick the current patch, and will start followup PRs in new pages. See you there :)

@pfxuan
Sorry, I re-read through the comments from @calvinjia  here, it seems actually this PR is very close to be merged in from his aspect. Thus I think it could be more convenient for both me and @calvinjia to continue working on this PR, rather than asking him to work on the other PR again. Also this PR here has a lot of useful discussion and links, discarding them might be a bit wasteful.
So how about still working here? Feel really sorry for the confusion and causing you extra work to open the new PRs.

@apc999 @calvinjia I'll keep all [TACHYON-665] tasks on this PR and refine it based-on your feedbacks. Thanks for both of your help through this PR!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4248/
Test PASSed.

@calvinjia and @apc999 , Please help check and see if there is anything that needs to be improved.

@pfxuan Currently if you just run the code, will the hostnames all default to empty (and then default to local hostname)? If so, we probably should at least require the master connect name to be specified for the cluster to work.

@calvinjia Yes. By using default setting, user is only required to provide the master connect name (TACHYON_MASTER_ADDRESS or "tachyon.master.hostname"). All other services will automatically bind to local hostname.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4309/
Test PASSed.

@pfxuan Could you resolve the merge conflicts with master?

@calvinjia Sure ~

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4325/Build result: FAILURE[...truncated 2321 lines...][JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.8.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-s3/0.8.0-SNAPSHOT/tachyon-underfs-s3-0.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.8.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-s3/0.8.0-SNAPSHOT/tachyon-underfs-s3-0.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.8.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-s3/0.8.0-SNAPSHOT/tachyon-underfs-s3-0.8.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/pom.xml to org.tachyonproject/tachyon-clients/0.8.0-SNAPSHOT/tachyon-clients-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/pom.xml to org.tachyonproject/tachyon-client-unshaded/0.8.0-SNAPSHOT/tachyon-client-unshaded-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.8.0-SNAPSHOT.jar to org.tachyonproject/tachyon-client-unshaded/0.8.0-SNAPSHOT/tachyon-client-unshaded-0.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.8.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-client-unshaded/0.8.0-SNAPSHOT/tachyon-client-unshaded-0.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.8.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-client-unshaded/0.8.0-SNAPSHOT/tachyon-client-unshaded-0.8.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/servers/pom.xml to org.tachyonproject/tachyon-servers/0.8.0-SNAPSHOT/tachyon-servers-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/pom.xml to org.tachyonproject/tachyon-underfs-hdfs/0.8.0-SNAPSHOT/tachyon-underfs-hdfs-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.8.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-hdfs/0.8.0-SNAPSHOT/tachyon-underfs-hdfs-0.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.8.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-hdfs/0.8.0-SNAPSHOT/tachyon-underfs-hdfs-0.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.8.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-hdfs/0.8.0-SNAPSHOT/tachyon-underfs-hdfs-0.8.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/pom.xml to org.tachyonproject/tachyon-parent/0.8.0-SNAPSHOT/tachyon-parent-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/dependency-reduced-pom.xml to org.tachyonproject/tachyon-client/0.8.0-SNAPSHOT/tachyon-client-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.8.0-SNAPSHOT.jar to org.tachyonproject/tachyon-client/0.8.0-SNAPSHOT/tachyon-client-0.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.8.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-client/0.8.0-SNAPSHOT/tachyon-client-0.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.8.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-client/0.8.0-SNAPSHOT/tachyon-client-0.8.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/shell/pom.xml to org.tachyonproject/tachyon-shell/0.8.0-SNAPSHOT/tachyon-shell-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/pom.xml to org.tachyonproject/tachyon-underfs-glusterfs/0.8.0-SNAPSHOT/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.8.0-SNAPSHOT/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.8.0-SNAPSHOT/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.8.0-SNAPSHOT/tachyon-underfs-glusterfs-0.8.0-SNAPSHOT-javadoc.jarchannel stopped
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4329/
Test PASSed.

@pfxuan This looks good to me, could you resolve the merge conflicts please?

@calvinjia  No problem!

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4342/Build result: ABORTED[...truncated 3098 lines...]Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.324 secRunning tachyon.underfs.UnderStorageSystemInterfaceIntegrationTestTests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,013.263 secRunning tachyon.hadoop.TFSRenameIntegrationTestTests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.592 secRunning tachyon.hadoop.TFSBlockLocationIntegrationTestTests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.428 secRunning tachyon.hadoop.TFSStatisticsTestTests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.328 secRunning tachyon.hadoop.HdfsFileInputStreamIntegrationTestTests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.4 secRunning tachyon.hadoop.fs.DFSIOIntegrationTestTests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.404 secRunning tachyon.client.RemoteBlockInStreamIntegrationTestBuild timed out (after 60 minutes). Marking the build as aborted.Results :Tests run: 164, Failures: 0, Errors: 0, Skipped: 0[ERROR] There was a timeout or other error in the forkBuild was abortedchannel stopped[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: Invalid object ID 10 iota=27
Test FAILed.

Jenkins, test this please.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4345/Build result: ABORTED[...truncated 3446 lines...][INFO] [INFO] --- license-maven-plugin:2.9:check (default) @ tachyon-assemblies ---[INFO] Checking licenses...[WARNING] Failed to getClass for org.apache.maven.plugin.source.SourceJarMojo[INFO] [INFO] <<< maven-source-plugin:2.3:jar (attach-sources) @ tachyon-assemblies <<<[INFO] [INFO] --- maven-source-plugin:2.3:jar (attach-sources) @ tachyon-assemblies ---[INFO] No sources in project. Archive not created.[INFO] [INFO] --- maven-javadoc-plugin:2.9:jar (attach-javadoc) @ tachyon-assemblies ---[INFO] [INFO] --- maven-install-plugin:2.4:install (default-install) @ tachyon-assemblies ---[INFO] Installing /home/jenkins/workspace/Tachyon-Pull-Request-Builder/assembly/target/tachyon-assemblies-0.8.0-SNAPSHOT.jar to /home/jenkins/.m2/repository/org/tachyonproject/tachyon-assemblies/0.8.0-SNAPSHOT/tachyon-assemblies-0.8.0-SNAPSHOT.jar[INFO] Installing /home/jenkins/workspace/Tachyon-Pull-Request-Builder/assembly/pom.xml to /home/jenkins/.m2/repository/org/tachyonproject/tachyon-assemblies/0.8.0-SNAPSHOT/tachyon-assemblies-0.8.0-SNAPSHOT.pomBuild timed out (after 60 minutes). Marking the build as aborted.[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: java.lang.InterruptedExceptionBuild was abortedchannel stopped[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: Invalid object ID 11 iota=27[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: hudson.remoting.ChannelClosedException: channel is already closed[INFO] ------------------------------------------------------------------------[INFO] Reactor Summary:[INFO]
Test FAILed.

@pfxuan Could you resolve the failures?
Running tachyon.master.JournalIntegrationTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,014.617 sec

e.g. this test definitely takes much much longer time than usual.

@apc999 The whole unit tests only takes about 5 mins on my laptop and similar amount of time on CentOS machine. It's hard to say what's the problem without touching Jenkins test machine. Is there any way to see the log file of Jenkins machine?

@pfxuan one way is to look at your commits and see which one caused the tests to start taking much longer. I suspect it has to do with the host name resolution.

@calvinjia Okay. I'm going to look at the problem. The followings are those tests with much longer time:
Running tachyon.master.JournalIntegrationTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 509.436 sec
Running tachyon.worker.BlockServiceHandlerIntegrationTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 254.989 se
Running tachyon.underfs.UnderStorageSystemInterfaceIntegrationTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,013.263 sec
Running tachyon.worker.block.meta.TieredStoreIntegrationTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 506.209 sec
Running tachyon.worker.DataServerIntegrationTest
Tests run: 60, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 522.666 sec

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4350/
Test PASSed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4355/Build result: ABORTED[...truncated 3120 lines...]Running tachyon.underfs.UnderStorageSystemInterfaceIntegrationTestBuild timed out (after 60 minutes). Marking the build as aborted.Results :Tests run: 225, Failures: 0, Errors: 0, Skipped: 0[ERROR] There was a timeout or other error in the fork[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: Invalid object ID 10 iota=27[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: Invalid object ID 10 iota=27[INFO] [INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ tachyon-integration-tests ---[WARNING] JAR will be empty - no content was marked for inclusion![INFO] Building jar: /home/jenkins/workspace/Tachyon-Pull-Request-Builder/integration-tests/target/tachyon-integration-tests-0.8.0-SNAPSHOT.jar[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: Invalid object ID 10 iota=27[INFO] [INFO] >>> maven-source-plugin:2.3:jar (attach-sources) @ tachyon-integration-tests >>>[INFO] [INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ tachyon-integration-tests ---Build was aborted[WARNING] Failed to notify spy hudson.maven.Maven3Builder$JenkinsEventSpy: hudson.remoting.Channel$OrderlyShutdownchannel stopped[INFO] [INFO] --- maven-checkstyle-plugin:2.13:check (checkstyle) @ tachyon-integration-tests ---
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4356/Failed Tests: 1org.tachyonproject:tachyon-integration-tests: 1tachyon.master.ServiceSocketBindIntegrationTest.connectDifferentAddressTest
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4358/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4360/
Test PASSed.

@calvinjia and @apc999 , the long run problem has been fixed. Integration test was accidentally using IPv6 address. A tiny bug can produce a very subtle behavior, which propagates all over the place. However, a good thing is that this behavior is fully controllable, and was converged into one method NetworkAddressUtils.isValidedAddress(). I want to emphasize this method was added after your reminders, which makes the thing more meaningful!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4362/
Test PASSed.

@pfxuan Glad the issue is resolved, unfortunately it looks like more merge conflicts were introduced, could you please resolve them?

@calvinjia Sure!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4386/
Test PASSed.

+1 LGTM

thank for working on the PR patiently! It is very important and cleans up the code!

@apc999 @calvinjia Thank for all your help and support. I'll try my best to track and improve follow-up	works!

@pfxuan Unfortunately another conflict was introduced, could you resolve it?

@calvinjia I'm working on it and will send it back to you soon

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/4392/
Test PASSed.

@pfxuan
Thanks for implementing this important feature!

@calvinjia and @apc999 ,  It's my pleasure to work with you. Thanks for code review!


Add water to water tiles.

About the PR

You can get water from water tiles just like if you were drawing from a sink. Also, they recharge.
Why / Balance

I was thirsty. Salv can now fill their water bottles in a new place that seems to make sense.
Technical details

Adds code from sink to water tiles to allow them to hold, dispense, and regenerate water.
Media

[X ] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes

Changelog

🆑 Froffy

add:  Water can now be scooped from water tiles, such as planetary rivers.

could also add trace amounts of things like iron and copper cause real life water streams aren't pure

no thats just confusing game wise

This should really just be infinite water or at least handle the solution's removal rather than an arbitrary number.

This should really just be infinite water or at least handle the solution's removal rather than an arbitrary number.

yeah but what about the manually mapped water sources like fland pool

just dont have them be drainable
if you cant pour water back in a pro can just steal the pool and run with it with no counter

The copy-paste is a little awkward. The solution shouldn't be named "tank", and it definitely doesn't need a SolutionContainerVisualizer.

the .yml appears to break when i use .inf as suggested online. i'll make it 9,999,999 water. is keeping the solution regeneration a lag concern with so many of these tiles at once..?

just make it so using a container on the tile fills it with water directly so no regeneration needed

just make it so using a container on the tile fills it with water directly so no regeneration needed
ohh. how do i do that

could also add trace amounts of things like iron and copper cause real life water streams aren't pure

this could add a potential "gold panning" action hehe. ignore water, get a chance to get minerals.

redefining the salvage meta into the california gold rush


Target Step rewrite EPIC

EPIC to rewrite the gene step in the new ETL pipeline
Details TBC.

Overview of Target re-write. There are two broad things that need to be done:

Finish Target/Target Safety and API
Update all related steps

Target
Outstanding:

bug with duplicate results see opentargets/platform#1537
API
target safety

Target Safety
This was previously part of target-DP. It will be it's own data set in the ETL, with it's own index and
API entries.
See branch split-target-safety
Update downstream steps

Step
Estimated effort

Associations
No change

AssociationsOTF
Difficult

Cancer Biomarkers
Difficult

Drug
Easy

Evidence
Medium

Known Drugs
Easy

Interactions
Medium

Mouse Phenotypes
Difficult

Reactome
No change

Associations
Does not appear to actually have any dependency on target.
Running egrep -rn 'targets|configuration' Association.scala results in:
1:package io.opentargets.etl.backend
7:import io.opentargets.etl.backend.spark.Helpers.mkRandomPrefix
8:import io.opentargets.etl.backend.spark.{Helpers => H}
9:import io.opentargets.etl.backend.spark.IoHelpers.IOResources
10:import io.opentargets.etl.backend.spark.{IOResource, IoHelpers}
97:         by configuration file application.conf when the known datasource is specified
118:        * join weight per datasource from configuration section `otc`
119:        * @param otc from ETL configuration section
146:        val associationsSec = context.configuration.associations
169:    val associationsSec = context.configuration.associations
172:      "evidences" -> context.configuration.associations.inputs.evidences,
173:      "diseases" -> context.configuration.associations.inputs.diseases
208:    val outputs = context.configuration.associations.outputs
226:    val outputs = context.configuration.associations.outputs
246:    val associationsSec = context.configuration.associations
278:    val associationsSec = context.configuration.associations

It looks like the input dataframe is never actually read.
AssociationsOTF

This needs to be updated. Columns used:

    val targetColumns = Seq(
      "id as target_id",
      "concat(id, ' ', approvedName, ' ', approvedSymbol) as target_data",
      "proteinAnnotations.classes as facet_classes",
      "reactome",
      "tractability"
    )

reactome is not a field on Target-ETL.
proteinAnnotations has changed structure.
possible changes in tractability: need to dig into the RDF more.

Cancer Biomarkers
Drug

called geneDf in Drug step (targetDF refers to ChEMBL target).
uses fields:

id
proteinAnnotations.id

To update:

use fields id and proteinIds.id

Evidence

Used in resolveTargets and excludeByBiotype methods

resolveTargets:

used to create lut (ln 440) created by calling generateTargetsLUT
Uses fields:

id
proteinAnnotations.id
proteinAnnotations.accessions
approvedSymbol
symbolSynonyms
targetSymbol << no idea there this comes from
targetName << ditto

To fix need to use fields approvedSymbol, proteinIds, synonyms, id

exlcudebybiotype uses only biotype and id (see ln: 376)

Known Drugs

Used fields:

id
approvedSymbol
approvedName
proteinAnnotations.classes.l1.label

To update

proteinAnnotations is now proteinIds and is now 'targetClass'. Filter by level and then take label.

Interactions

used in generateMapping (ln 67).
fields used:

proteinAnnotations.accessions
id
hgncId

to update:

hgncId is now is dbXrefs
proteinAnnotations is now in proteinIds

Mouse Phenotypes
This is generated as part of the target step in the data pipeline.
Using the output of the DP, we extract the ENSG and MP into a new json document using:
cat 20.04_gene-data.json | jq -r '{"id":.id,"phenotypes": [.mouse_phenotypes[]?] }|@json' > mousephenotype.json

This has the output format
root
 |-- id: string (nullable = true)
 |-- phenotypes: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- mouse_gene_id: string (nullable = true)
 |    |    |-- mouse_gene_symbol: string (nullable = true)
 |    |    |-- phenotypes: array (nullable = true)
 |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |-- category_mp_identifier: string (nullable = true)
 |    |    |    |    |-- category_mp_label: string (nullable = true)
 |    |    |    |    |-- genotype_phenotype: array (nullable = true)
 |    |    |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |    |    |-- mp_identifier: string (nullable = true)
 |    |    |    |    |    |    |-- mp_label: string (nullable = true)
 |    |    |    |    |    |    |-- pmid: string (nullable = true)
 |    |    |    |    |    |    |-- subject_allelic_composition: string (nullable = true)
 |    |    |    |    |    |    |-- subject_background: string (nullable = true)

The ETL reads in this json file, and writes out the result without further manipulation.
This needs to be completed implemented in the ETL. The DP is processing RDF files for some of it, so those transformations might be better in PIS since we already have that set up for dealing with ontology files.
Reactome
Reactome-ETL has no dependencies.
When the Target-DP is run, Reactome-DP is a requirement.

@JarrodBaker you can close this now :)


DRAFT: TES-248 Added the cloudwatch agent to the GDB AMI

Download and install the cloudwatch agent depending on AMI architecture
create the necessary configuration to run the agent
the agent won't be started by default

Description

Related Issues

Changes

Screenshots (if applicable)

Checklist

[ ] I have tested these changes thoroughly.
[ ] My code follows the project's coding style.
[ ] I have added appropriate comments to my code, especially in complex areas.
[ ] All new and existing tests passed locally.

Please update the CHANGELOG and README

It would be good to add something in the README.md about the cloudwatch agent and metrics gathered.

Wrong task in the title of the MR


[Ember Migration] Fix Error Trying to Access document in Fastboot

Issue Description
Error occurs when trying to access document in fastboot.
Parent Issue: #785
Screenshots

Severity/Priority

[ ] Critical
[ ] High
[x] Medium
[ ] Low

closed via #789


An idea for responsive display style equations

One of the difficulties with designing responsive math is that it is a priori unclear how to resize or break a user's equation if it is too wide to display on mobile devices.
I propose to handle this by allowing use of the TeX hyphenation macro \-. It would work as follows.

A user may use the \- command once per line in a display style equation.
If the equation is too large, everything after \-, is moved to the next line and right-aligned.

I'd be curious to hear thoughts. This won't cover every situation, but I think would cover most, and is better than having no responsive support at all. It could be added as a contrib module.

I'd rather do something fully LaTeX compatible.  According to this, there seem to be two main packages for automatic line breaking of equations:

breqn
autobreak
The autobreak package sounds similar to your solution, in that you need to add explicit candidate line breaks.  But it does so with \n, a bit of a weird choice...  Maybe worth checking out breqn in more detail.

I'm not familiar with either of those packages, but autobreak seems like it would work well. Its behavior is very much like the idea above, except that it treats newlines as break candidates, and requires breakable math to be wrapped in \begin{autobreak}. This seems like it would be a good addition in my view.

I looked more closely at both and agree with your assessment. autobreak seems relatively clean in design (though I'm still a little confused how it interacts with arrays) and hopefully not too hard to implement. I'm a little surprised that there isn't a way to automatically wrap all display math in \begin{autobreak}, but this does give more control (especially as relates to arrays).


Upgrade firebase to 10.7.1

What changes did you make?
Upgrades firebase from 9.9.2 to 10.7.1
Why did you make the changes?
As part of larger epic to improve bloom performance. This upgrade relates to a wider goal of reducing unused javascript - firebase 10.x uses modular api design so we can import only the functions we need, instead of the entire e.g. auth module.
Overall bundle reduced by 60kb but will also add performance gains to individual pages using firebase functions
See before and after bundle analysis:

woah - how did you get this bundle analytics


Set default external host/client

Today, the application supports only host defined in code, to docker sock default, but exists others application for execution environments for docker, for example: colima..
Is good, in the lazydocker app it possible custom set to path custom to host.. I for execute the app in local, i modify the variable in code and copile a new code.

sudo ln -f -s ~/.colima/default/docker.sock /var/run/docker.sock
I resolved with command above


"Corrections to the seeding covariance matrix"

Corrected errors in missing rotation of the Covariance matrix after the seeding plus missing conversion from curvature to 1/pT for element c[13]

Did it solve also convergence problem, or some further changes are needed? do you have some test?
Thanks

https://alice.its.cern.ch/jira/plugins/servlet/mobile#issue/PWGPP-613

Based on the QA plot https://alice.its.cern.ch/jira/plugins/servlet/mobile#issue/PWGPP-613 i merge


more readable pod and svc names

replaced:
kubectl run hostnames
for a more understandable version in the context o dealing with dns services names and hostnames
for:
kubectl run mypod

/assign @jimangel

/check-cla

Hey there! @juanitomint, looks like you haven't signed the CLA yet. Could I please have you do that? https://github.com/kubernetes/community/blob/master/CLA.md

@jimangel I've signed but for some reason my github account doesnt seems to be linked

Hi @juanitomint! Thanks for signing. I'm sorry the CLA process isn't working as intended. Does your Github e-mail address match the address used to sign CLA? If so, please take a look at the troubleshooting section. Thanks!

@juanitomint Please resolve merge conflicts in order to continue.
@thockin @bowei 👋 Thanks in advance for your review.

/close
@juanitomint When you get a chance can you please re-open and fix merge conflicts! We'd love your contribution!

Sure!


Community maintainers required

From https://github.com/jgrassler/mkdocs-pandoc/pull/30#issuecomment-437356368:

I'm not using it anymore, so I no longer have much of an inclination to maintain it.
Any of you folks want to take over maintaining it?

I've mentioned I can help out over in https://github.com/jgrassler/mkdocs-pandoc/pull/30#issuecomment-494493915.
Anyone else motivated to help this thing keep up with the mkdocs ecosystem?

Ah, wait a second, there seems to be another effort moving along just fine:

https://github.com/twardoch/mkdocs-combine

We should probably help over there instead (as there is a plugin integration planned). That project is a fork of this one. Seems like the right direction to move ... Sorry for the noise :)

For what it's worth it's got my blessing - looks a lot more active at any rate. I'll put a link to this fork into the project description...


Increment version for labservices releases

Increment package version after release of Azure.ResourceManager.LabServices

API change check
API changes are not detected in this pull request.


Tests: Add failing WitScript serialization test

thanks for reporting this. This is very strange, looking into it.

This was a huge bug. It was serializing the script as string instead as a hex string.
I just fixed it, but this basically mean this is a breaking change. Anybody who stored WitScript as json in database will have problem now... Hopefully this should be rare.

@NicolasDorier  Thanks for fixing it. What versioning scheme does NBitcoin follow? Seems odd that this was only an increment in the subminor release version. I guess there is no promise of any ABI/API stability at all?

@ysangkok I follow semver as much as I can. But when there is something that is almost unused which I break, I don't bump for this.


Issues with style imports of vanilla-extract for route

I'm trying to import a style that was created using vanilla-extract for a page but it doesn't seem to be working. The component style import works properly which leads me to believe there's some edge case for pages that need to be solved?
Minimal repro: https://codesandbox.io/p/github/silverAndroid/remix-esbuild-override/test?file=%2Fexamples%2Fvanilla-extract%2Fapp%2Froutes%2Findex.tsx&selection=[{"endColumn"%3A7%2C"endLineNumber"%3A9%2C"startColumn"%3A7%2C"startLineNumber"%3A9}]
Stacktrace:
Error: Styles were unable to be assigned to a file. This is generally caused by one of the following:

- You may have created styles outside of a '.css.ts' context
- You may have incorrect configuration. See https://vanilla-extract.style/documentation/setup
    at Object.getFileScope (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/@vanilla-extract/css/fileScope/dist/vanilla-extract-css-fileScope.cjs.dev.js:33:11)
    at generateIdentifier (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/@vanilla-extract/css/dist/vanilla-extract-css.cjs.dev.js:176:49)
    at composedStyle (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/@vanilla-extract/css/dist/vanilla-extract-css.cjs.dev.js:349:19)
    at style (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/@vanilla-extract/css/dist/vanilla-extract-css.cjs.dev.js:396:12)
    at Object.<anonymous> (/project/home/silverandroid/workspace/examples/vanilla-extract/app/routes/index.css.ts:5:26)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1028:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at /project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/@remix-run/serve/index.js:43:17
    at Layer.handle [as handle_request] (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/express/lib/router/layer.js:95:5)
    at next (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/express/lib/router/route.js:137:13)
    at next (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/express/lib/router/route.js:131:14)
    at next (/project/home/silverandroid/workspace/examples/vanilla-extract/node_modules/express/lib/router/route.js:131:14)

Sorry, vanilla-extract is no longer supported by this repository.


OCPBUGS-3176: Add config knob for enabling ip forwarding

With https://github.com/ovn-org/ovn-kubernetes/pull/3374 and https://github.com/openshift/machine-config-operator/pull/3676 we stopped enabling IP forwarding globally in the OCP host. It is now only enabled on interfaces that OVNK manages. Additionally iptables rules are put in place to limit what traffic is forwarded. This was to resolve an issue where our node by default acts as a router: https://issues.redhat.com/browse/OCPBUGS-3176
With these changes, by default in OCP we will no longer forward IP traffic. Most users should welcome this change as it means the OCP node will not be forwarding traffic unintentionally on their network. However, in case there is some corner case where a user expects traffic to be forwarded we will expose this new config knob to allow them to go back to the old behavior.

/approve
/lgtm


[SP-5389][BACKLOG-32961] Fix pan and kitchen timers to measure only j…

…ob or trans execution time, not setup time

:white_check_mark: Build finished in 27m 6s
Build command:
mvn clean verify -B -e -Daudit -amd -pl engine

:ok_hand: All tests passed!
Tests run: 3545, Failures: 0, Skipped: 7    Test Results

:information_source:  This is an automatic message


Migrate storage to use Multipart in Core

see https://github.com/Azure/azure-sdk-for-net/pull/14275

Hi @christothes, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.


Init script for running on Centos 7

This script sets up all of the dependencies required for running Cloudera Labs Ansible playbooks from a Centos 7 environment.

Closing this PR as stale, and because the script is present in cloudera-deploy and I don't think we need to duplicate it.


[license] Could you please add a license file?

Good catch!

In the mean time, the copyright file https://github.com/silx-kit/silx/blob/master/copyright describes the licenses on a per file basis.

We have to add a LICENSE file specifying that:

Most of silx is MIT
Modules using other licenses are accepted provided they do not impose anything to the end user. That means LGPL is accepted but GPL or BSD with citation clause are not.
That if the end user is using PyQt4 or PyQt5, (s)he will be under GPL unless he owns a commercial license. The end user has the alternative of using PySide in order to be free since PySide is LGPL.
That more details can be found in the copyright file.


Merging master into dma branch

Please review for pulling dma-merge into dma
The dam-merge branch includes a merge from master into dma and a few fixes in so doing as well as a compilation issue in templated method: Point image(const Point p) const and Rect preimage(const Point p) const.

Merged.
On Tuesday, July 28, 2015, streichler<EMAIL_ADDRESS>wrote:

Assigned #21 https://github.com/StanfordLegion/legion/pull/21 to
@jiazhihao https://github.com/jiazhihao.
—
Reply to this email directly or view it on GitHub
https://github.com/StanfordLegion/legion/pull/21#event-367846103.

--
Sincerely,
Zhihao


HTTP/2 RST_STREAM Regression f990f99

Motivation:
Commit f990f99 introduced a bug into the RST_STREAM processing that would prevent a RST_STREAM from being sent when it should have been. The promise would be marked as successful even though the RST_STREAM frame would never be sent.
Modifications:

Fix conditional in Http2ConnectionHandler.resetStream to allow reset streams to be sent in all stream states besides IDLE.

Result:
RST_STREAM frames are now sent when they are supposed to be sent
Fixes https://github.com/netty/netty/issues/4856

@ejona86 - FYI

SonarQube analysis reported 6 issues:

 6 major

Watch the comments in this conversation to review them.
Note: the following issues could not be reported as comments because they are located on lines that are not displayed in this pull request:

 Remove this unused method parameter "cause".
 Remove this instantiation of "ChannelFutureListener".
 Remove this instantiation of "ChannelFutureListener".
 Remove this instantiation of "ChannelFutureListener".
 Useless qualified this usage in the same class.
 Useless qualified this usage in the same class.

@Scottmitch ship it... or could you even write a unit test ?

@normanmaurer - +1. added unit test.

SonarQube analysis reported 4 issues:

 4 major

Watch the comments in this conversation to review them.
Note: the following issues could not be reported as comments because they are located on lines that are not displayed in this pull request:

 Remove this unused method parameter "cause".
 Remove this instantiation of "ChannelFutureListener".
 Remove this instantiation of "ChannelFutureListener".

SonarQube analysis reported 4 issues:

 4 major

Watch the comments in this conversation to review them.
Note: the following issues could not be reported as comments because they are located on lines that are not displayed in this pull request:

 Remove this unused method parameter "cause".
 Remove this instantiation of "ChannelFutureListener".
 Remove this instantiation of "ChannelFutureListener".

SonarQube analysis reported 4 issues:

 4 major

Watch the comments in this conversation to review them.
Note: the following issues could not be reported as comments because they are located on lines that are not displayed in this pull request:

 Remove this unused method parameter "cause".
 Remove this instantiation of "ChannelFutureListener".
 Remove this instantiation of "ChannelFutureListener".

LGTM

Thanks for review @nmittler !
Cherry-picked 4.1 (56e6e07)


getInitialProps is called twice

Hi.
As title above, I created codesandbox. Please take a look.
https://codesandbox.io/s/l2838zpq1q

same issue here 😢

sadly I have the same issue

So, it seems like this is not related to any Redux HOC, although it appears using the Redux HOC is causing the getInitialProps on the _app.js component to get swallowed or otherwise not run.
I'm still seeing two Index::getInitialProps logs even after removing the withRedux HOC.
When submitting a bug report, it's very helpful to try and narrow it down to the simplest possible reproducible example.
I'll look into this further now. Thanks for your patience everyone.

So - getInitialProps is being called twice because you're calling it yourself. You don't need to execute the if (Component.getInitialProps) clause - we're already doing that internally within next-i18next.
I believe this was just a case of misunderstanding and copy/pasting from docs. Let me know if this is clear, and if anyone thinks something should be changed.

@isaachinman  thank you for your reply.
If I remove getInitialProps of _app.js, I'm still seeing two Index::getInitialProps logs.
It means if I don't override getInitialProps of _app.js.  This problem still happens.
Here: https://codesandbox.io/s/l2838zpq1q

@isaachinman
I think the problem is not related to withRedux anymore, but only appWithTranslation.
I removed all redux related things, and still seeing two Index::getInitialProps logs
https://codesandbox.io/s/l2838zpq1q

So: both Component and WrappedComponent have their getInitialProps function called within our appWithTranslation HOC, and weirdly it seems that both are being called onto the index.js level instead of once on _app.js and once on index.js.
If I explicitly do declare a getInitialProps on the _app.js level that doesn't contain another getInitialProps call, the problem does go away. Check out this sandbox.
My guess is that NextJs is doing some bubbling/traversal from _app.js to the page level and hoisting statics if getInitialProps does not exist. I suppose we'll need some way of doing an equality check against WrappedComponent and Component.

I created a branch called do-not-hoist-getInitialProps which I think fixes this problem.
However, I'm worried that it's also a regression and will break #11. Can a Redux user please test that branch with your compose / connect HOC usage and let me know if everything works?


New command: get office 365 tenant id from domain name

Get tenant id from domain name
command: tenant -d|--domain xyz.com
Logic :

call https://login.windows.net/xyz.com/.well-known/openid-configuration
parse token_endpoint.Split('/')[3]

Note: we donot need to run graph login for this endpoint. If this is a valid suggestion, I can take this.

Nice one! Let's get it in! To make it match the naming convention, let's rename it to tenant id get. If the endpoint returns other interesting information, we could do tenant get and return ID along with other bits

Sure, thank you @waldekmastykarz.


fix tutorials so they render properly in html

misc changes here and there. verified by inspection. if anyone has time i would recommend walking through and clicking on all the links, etc.

Links seem fine to me


make menuconfig: CONFIG_LWIP_MAX_SOCKETS - 16 or 32?

Probably a small typo ... but was looking at the help for CONFIG_LWIP_MAX_SOCKETS and came across:

I think we can only have 16 sockets but the range property would indicate we can have 32?

@nkolban Sorry, it's a mistake. We will fix it ASAP.


Refactor some names

We've come up with some clearer/better names for types the more we use them. Now we just need to swap them out.

Validator in e.g. PackageValidator -> State

While you can validate using these, the struct itself really represents the state of the validator.

DynHash/Hash<D> -> AnyHash/Hash<D>

The "static vs. dynamic" naming here is accurate in some sense, but is confusing especially with the use of dyn as a keyword for things. The real distinction is that AnyHash can hold a hash of any allowed algorithm type whereas Hash<D> identifies a singular algorithm in its type.

Second half of this issue is fixed with #126.


Unable to install

I keep getting error installing the app through Microsoft store and Winget. And Winget returned the following info:
Install failed: AppxManifest.xml(164,10): error 0x80070005: Cannot register the 19282JackieLiu.Notepads-Beta_<IP_ADDRESS>_x64__echhpq9pdbte8 package because the following error was encountered while registering the windows.fileTypeAssociation extension: Access is denied.
.  Try again and contact the package publisher if the problem persists.
0x80070005 : Access is denied.

I know that it may be the problem of my OS or store. But is there a workaround or a manually installing method?

Sounds like you're using restricted Windows user Account or something is configured bad.

Probably, cause I tweaked Windows settings recently. However the problem persists when I use different administer accounts. I am so tired of fighting with Windows permissions 🤕🤕🤕

You can reset Windows then.
Anyway it's definitely not a Notepads bug.

The problem is solved


Validate module parameters, verify that kernel modules are loaded

It was relatively easy to break ZFSBootMenu to the point that it was almost unrecoverable by passing an invalid spl.spl_hostid parameter. When an invalid parameter is set on the kernel command line it can't be overridden except by directly calling insmod with the full path to the kernel module and the fixed argument to the parameter.
Additionally, if spl.spl_hostid was passed into ZBM, incorrect default-hostid logic was applied because spl_hostid wasn't detected on the kernel command line, nor is /etc/hostid part of the release builds. Both forms of passing the hostid are now accepted and the argument is normalized when possible to an 8 character hex value. If it can't be normalized, spl_hostid is unset and the global default of 00bab10c is used.
In zfsbootmenu-init.sh,  find the on-disk path for spl.ko and manually load it with insmod and spl_hostid=0. This accomplishes two things - it bypasses bad spl.spl_hostid values set on the kernel command line, and it immediately switches the module to use /etc/hostid as the hostid source, which we already do in write_hostid.
Loading the zfs kernel module has also been moved to zfsbootmenu-init.sh so that all of the helper functions are available. If modprobe zfs fails, the error output is logged. A recovery shell is invoked so that users can look at logs to determine what happened.

Instead of trying to modprobe spl and then extract a path to the file on failure, modinfo is used to find the on-disk filename for spl and then load it with a forced spl_hostid=0 module parameter.


fix: do not inject empty queries

When hasInternalMutations is true but internalMutations is empty, this will generate unexpected properties on Mutations, this will cause the error code internalClient.mutations.mutations. So leave it empty.

Checklist

[ ] run make test
[ ] PR title must follow conventional-commit-standard
[ ] tests and/or benchmarks are included
[ ] documentation is changed or added
[ ] commit message and code follows the Code of conduct

@fiam if you've tested this locally, please merge.

@jensneuse  Tested locally, works as expected


Cluster of 10 PCs, multiple container in overlaynetwork, some containers can ping each other, others can't

Hi,
Description of Setup:
10 hosts, one of which plays the role of the swarm manager
Discovery service consul on the host of the swarm manager
using an overlay network
the manager starts 9 containers with spread, one on each of the free nodes, some of them are able to ping each other, other's aren't.
They all appear in the docker -H <IP_ADDRESS>:2375 info and   docker -H <IP_ADDRESS>:2375 ps, can be entered with  docker -H <IP_ADDRESS>:2375 exec -it ID bash.
Some of the containers are working as expected, we are able to ping other containers, hosts and for e.g. google.
Other containers can ping hosts, google, but fail to ping any other container or to be pinged by them.
Which containers on which hosts have this problem changes everytime we start a new setup and seems to be completely random.
We tried out swarm:latest, :1.1.0-rc2 and 1.0.0
The same setup worked a few weeks ago. We are using ansible scripts and, shame on us, only pulled the :latest tag. We didn't write down the actual version. That's why we don't know which swarm version we used when it worked.
docker info:
Containers: 0
Images: 34
Server Version: 1.9.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 34
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.19.0-42-generic
Operating System: Ubuntu 15.04
CPUs: 2
Total Memory: 3.795 GiB
Name: ssds110
ID: N2W6:HP7F:KIAO:BHNR:LKE2:YNDM:BFO5:QAFB:CZFK:ZNKO:LX35:W4KL
WARNING: No swap limit support

We happy to provide further information and logs, please tell us which, as there are so many to choose from.
Edit:
docker -H <IP_ADDRESS>:2375 info
Containers: 9
Images: 36
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 9
 ssds111: IP.111:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds112: IP.112:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-47-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds113: IP.113:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds114: IP.114:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds115: IP.115:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds116: IP.116:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds117: IP.117:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds118: IP.118:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
 ssds119: IP.119:2375
  └ Status: Healthy
  └ Containers: 1
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 3.984 GiB
  └ Labels: executiondriver=native-0.2, kernelversion=3.19.0-42-generic, operatingsystem=Ubuntu 15.04, storagedriver=aufs
CPUs: 18
Total Memory: 35.86 GiB
Name: 5c43445107cd

Thank you,
Manuel and Carlo

@carnuel did you update your engines ? are are they still in 1.9.1 ?

We are still using Ubuntu 15.04 and just realized that docker 1.10 is not supported and will upgrade to ubuntu 15.10 soon.

@vieux Is there a special reason why docker info on a swarm is not including docker and swarm version? Should I open a new ticket for this?

@schmunk42 you mean Server Version: in docker info ? it was added in 1.10 apparently, I'm going to open a PR asap

@schmunk42 see https://github.com/docker/swarm/pull/1789

@carnuel is the problem resolved after upgrading to Engine 1.10?

@amitshukla sorry for the late response.
It works on ubuntu 15.10 with Engine 1.10.
Though I am pretty sure we faced the same problem once during testing, i.e. we couldn't ping between some containers as mentioned above. But we weren't able to reproduce the error the day after.
Altogether, it seems to work. We setup a cluster of containers on several hosts, tested a few things, took them down again, and repeated that multiple times.. It worked.

I think I'm having a related situation here.
This morning, one container was unable to ping some other containers in the cluster. Have a look at the commands below:
Try to ping containerB from containerA: KO.
# docker -H  <SWARM_MANAGER>:4000 exec -i -t containerA bash -l -c 'ping containerB'
PING containerB (<IP_ADDRESS>) 56(84) bytes of data.
^C
--- db_balancer ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

Try to ping containerA from containerB: OK
# docker -H  <SWARM_MANAGER>:4000 exec -i -t containerB  bash -l -c 'ping containerA'
PING containerA (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=64 time=0.559 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.306 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.301 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=0.387 ms
64 bytes from <IP_ADDRESS>: icmp_seq=4 ttl=64 time=0.276 ms
^C--- web2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.276/0.366/0.559/0.104 ms

And the weirdest part, just after that, try again to ping containerB from containerA: OK
# docker -H  <SWARM_MANAGER>:4000 exec -i -t containerA bash -l -c 'ping containerB'
PING containerB (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from containerB.myOverlayNet (<IP_ADDRESS>): icmp_seq=1 ttl=64 time=0.342 ms
64 bytes from containerB.myOverlayNet (<IP_ADDRESS>): icmp_seq=2 ttl=64 time=0.226 ms
64 bytes from containerB.myOverlayNet (<IP_ADDRESS>): icmp_seq=3 ttl=64 time=0.247 ms
^C
--- db_balancer ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.226/0.271/0.342/0.053 ms


Test: JS: Skip view() test for now, because it fails on Winodws

The test itself succeeds, but fails with the following error when deleting the index file created by save() in afterEach().
error: "EBUSY: resource busy or locked, unlink 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\usearch.test.index'"

Since it is only in Winodws that it fails, we will skip it on Winodws for now.
We will continue to investigate the solution.

Maybe the file opened with view() is not closed, but I couldn't find a solution right away, so I skipped it for now.
Related: #503


Your connection is not private

SSL Certificate is broken and CapRover can't load one-click apps.

@xJoeyv Thanks for reporting, it should be fixed now, can you try again and report back as i cant test it becuase i dont have caprover anymore

Works again!


DropdownButton Menu Does Not Close or Reposition on Screen Resize in Flutter Web

Steps to reproduce
1- Run the provided code in a Flutter web project.
2- Open the DropdownButton menu.
3- Resize the browser window.
Expected results

The DropdownButton menu should automatically close when the screen size changes.
The dropdown menu should reposition itself directly under the dropdown button.

https://github.com/user-attachments/assets/c5d86ff7-9ccb-4ba3-84c5-e1e1c7a8160e
Actual results

The dropdown menu remains open after resizing the window and does not reposition itself under the button, leading to usability issues.

Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  String selectedValue = 'Option 1';

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: Center(
          child: Column(
            children: [
              DropdownButton<String>(
                value: selectedValue,
                items: ['Option 1', 'Option 2', 'Option 3'].map((String value) {
                  return DropdownMenuItem<String>(
                    value: value,
                    child: Text(value),
                  );
                }).toList(),
                onChanged: (newValue) {
                  setState(() {
                    selectedValue = newValue!;
                  });
                },
              ),
            ],
          )),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
No response
Flutter Doctor output
[√] Flutter (Channel stable, 3.24.3, on Microsoft Windows [Version 10.0.19045.5011], locale ar-EG)
    • Flutter version 3.24.3 on channel stable at C:\src\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2663184aa7 (5 weeks ago), 2024-09-11 16:27:48 -0500
    • Engine revision 36335019a8
    • Dart version 3.5.3
    • DevTools version 2.37.3

[√] Windows Version (Installed version of Windows is version 10 or higher)

[!] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
    • Android SDK at C:\Users\HP\AppData\Local\Android\sdk
    X cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    X Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/to/windows-android-setup for more details.

[√] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[X] Visual Studio - develop Windows apps
    X Visual Studio not installed; this is necessary to develop Windows apps.
      Download at https://visualstudio.microsoft.com/downloads/.
      Please install the "Desktop development with C++" workload, including all of its default components

[√] Android Studio (version 2024.1)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0--11852314)

[√] VS Code (version 1.93.1)
    • VS Code at C:\Users\HP\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.98.0

[√] Connected device (4 available)
    • CPH2203 (mobile)  • NBRSTOS4BMIB8DCE • android-arm64  • Android 13 (API 33)
    • Windows (desktop) • windows          • windows-x64    • Microsoft Windows [Version 10.0.19045.5011]
    • Chrome (web)      • chrome           • web-javascript • Google Chrome 129.0.6668.100
    • Edge (web)        • edge             • web-javascript • Microsoft Edge 129.0.2792.52

[√] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

The Flutter CLI developer tool uses Google Analytics to report usage and diagnostic
data along with package dependencies, and crash reporting to send basic crash
reports. This data is used to help improve the Dart platform, Flutter framework,
and related tools.
Telemetry is not sent on the very first run. To disable reporting of telemetry,
run this terminal command:
flutter --disable-analytics

If you opt out of telemetry, an opt-out event will be sent, and then no further
information will be sent. This data is collected in accordance with the Google
Privacy Policy (https://policies.google.com/privacy).
Please note that analytics reporting was already disabled, and will continue to be disabled.

Thanks for the report.
This is replicable on desktop as well, so the reported behavior is not specific to web.

I verified using DropdownMenu widget as well and it closes the menu while trying to resize the window.
Stable: 3.24.3
Master: 3.27.0-1.0.pre.88


[AERIE-2187] Documentation: activity types and configuration

Tickets addressed: AERIE-2187
Review: By commit
Merge strategy: Merge (no squash)

Description
This PR updates documentation for activity parameters and mission model configuration.
Changes worth calling out:

A new section titled Advanced Topic: Parameters was added to generically cover parameters as used within activity types and mission model configurations.

The content is mostly the same as our wiki's Declaring-Parameters.md documentation.

Verification
N/A
Documentation
N/A
Future work
Finish the rest of the MM user guide sections.

@Mythicaeda Thanks a lot for taking the time to help with the broken links! I didn't respond to every comment but they should all be fixed now :)


Fix indentation

Attempt to fix indentation issues that could be causing Sphinx build failures.

Thanks @cgarciae @IvyZX !


Release Google.Cloud.Dialogflow.Cx.V3 version 1.12.0

Changes in this release:
Documentation improvements

Minor wording update (commit d4281ec)

Created release for Google.Cloud.Dialogflow.Cx.V3-1.12.0


ng add @ng-toolkit/universal - Wrong path app.component.ts

I'm submitting a...

[ X] Bug report
[ ] Feature request
[ ] Documentation issue or request

In package

[ ] @ng-toolkit/init
[ ] @ng-toolkit/serverless
[X ] @ng-toolkit/universal

Current behavior

I have a project with angular 6.
I have try to execute instruction of https://coursetro.com/posts/code/155/Angular-6-SEO-Tutorial-from-Scratch---It's-Super-Simple-Now!
But on "ng add @ng-toolkit/universal" i have this error.
It search on "./src/app/app/app.component.ts" but  is locate at "./src/app/app.component.ts"

I don't know if it have a impact but i have this in my tsconfig.json
"compilerOptions": {
"baseUrl": "./src/"
Any idea ?
Environment
For Tooling issues:

Npm : 5.5.1

Node version:v 8.9.1
Platform: Windows

@maciejtreder I have a project based on angular 6
ng -v

The error


Cannot add lifecycle rule to expire delete markers

Found a bug? Maybe our Slack Community can help.

Describe the Bug
Attempting to add a second rule to expire delete markers. But get The given value is not suitable for module.s3_bucket.var.lifecycle_configuration_rules declared at .terraform/modules/s3_bucket/variables.tf:159,1-41: cannot find a │ common base type for all elements.
Lifecycle rules:
locals {
  lifecycle_configuration_rules = [
    {
      enabled = true
      id      = "cleanup"
      abort_incomplete_multipart_upload_days = 1
      filter_and = {
        prefix = "prefix/"
      }
      expiration = {
        days = var.s3_expiration
      }
      noncurrent_version_expiration = {
        newer_noncurrent_versions = var.s3_expiration_noncurrent_versions
        noncurrent_days           = var.s3_expiration_noncurrent_days
      }
      transition                    = null
      noncurrent_version_transition = null
    }
    ,
    {
      enabled = true
      id      = "cleanup-delete-markers"
      expiration = {
        expired_object_delete_marker = true
      }
      abort_incomplete_multipart_upload_days = 1
      filter_and                             = null
      transition                             = null
      noncurrent_version_expiration          = null
      noncurrent_version_transition          = null
    }
  ]
}

The first rule works fine by itself. Error happens when second rule is added
The second rule is one I use all the time. Just first time using this module to create it
Expected Behavior
Both lifecycle rules defined
Steps to Reproduce

Add the lifecycle rules to a S3 bucket definition
Run terraform plan
Receive error
╷ │ Error: Invalid value for input variable │ │   on main.tf line 128, in module "s3_bucket": │  128:   lifecycle_configuration_rules = local.lifecycle_configuration_rules │ │ The given value is not suitable for module.s3_bucket.var.lifecycle_configuration_rules declared at .terraform/modules/s3_bucket/variables.tf:159,1-41: cannot find a │ common base type for all elements.

Environment (please complete the following information):

Terraform v1.3.7
AWS provider v4.53.0
Cloudposse s3_bucket module 3.0.0
macOS 13.2

This is not a bug, this is how Terraform works. All elements of the list must be of the same type. When they are objects, that means they must have all the same fields.
You need to add missing fields. This should work with Terraform 1.0.0 or later:
locals {
  lifecycle_configuration_rules = [
    {
      enabled = true
      id      = "cleanup"
      abort_incomplete_multipart_upload_days = 1
      filter_and = {
        prefix = "prefix/"
      }
      expiration = {
        days = var.s3_expiration
        expired_object_delete_marker = null
      }
      noncurrent_version_expiration = {
        newer_noncurrent_versions = var.s3_expiration_noncurrent_versions
        noncurrent_days           = var.s3_expiration_noncurrent_days
      }
      transition                    = null
      noncurrent_version_transition = null
    }
    ,
    {
      enabled = true
      id      = "cleanup-delete-markers"
      expiration = {
        days = null
        expired_object_delete_marker = true
      }
      abort_incomplete_multipart_upload_days = 1
      filter_and                             = null
      transition                             = null
      noncurrent_version_expiration          = null
      noncurrent_version_transition          = null
    }
  ]
}

That isn't true. Terraform has allowed optional fields for the last couple versions.
While they still need to be of same type, they do not have to have the same fields.
I've been using optional fields heavily lately.
But this is more of an improvement (feature request) than a bug, just due to it was probably written before terraform 1.3

The lifecycle_configuration_rules is not declared to have any optional fields. Therefore, the input is incorrect, as the error message stated, and the solution is to fix the input. That means it is not a bug.
I will leave this open as a feature request.


Add Kyrgyz Republic scraper

There are download links for all results and single processes, but I can't figure out an API from a quick inspection: http://zakupki.gov.kg/popp/view/order/list.xhtml
There seem to be only lists of releases (no packages).

Dmitry confirmed that the API is not yet public.

No expected date for becoming public. Can re-open if so.


Fix the table name when reflect is True in sqla.CopyToTable (#2604)

When the schema is specified, MetaData is already parametrized with the right
name. No need to call "meta.reflect" with the name of the schema in this case.

I launched
tox -e core test/contrib/sqla_test.py but 0 tests was ran.

Sqla is in the contrib tox attrib group
so:
tox -e contrib test/contrib/sqla_test.py

@sd2k I believe you added this bit of code. Could you review this small patch? Thanks!

Sure, I just tested this locally and it looks good (the old code failed), and it makes sense looking at the SQLAlchemy docs.
I don't think there's an easy way to add a test for it unfortunately since as far as I know SQLite (used in the tests) doesn't support schemas.

Thanks @sd2k for the review and for testing this locally.

as far as I know SQLite (used in the tests) doesn't support schemas.

I think you're right.

Thank you for the review. Do you think this can be merged in the future release? Thanks.

Thanks!! :slightly_smiling_face:


BlockchainSummary is missing staking_epoch_ledger_hash

This blocks #629.
Add staking_epoch_ledger_hash to BlockchainSummary

Does this truly block https://github.com/Granola-Team/mina-block-explorer/issues/629? I think we can remove the ledger hash on the front end without having to have it in the API.


[Bug] Overflow on judge login text input

What Happened?

Steps to Reproduce

Go to judge login

What device(s) are you seeing this issue on?
No response
What browser are you seeing this issue on?
No response
Possible Solutions
No response
Additional Info
No response

Hello, can you assign it for me, I have already work on this


update and delete iis artifacts in v5.7 does not seem to work.

I want to be able to undeploy my MSI and let the BTDF delete the virtual dir. Though the setting UndeployAction is set to Delete, undeploying does NOT remove the virtual dir (only the folder /solution_1.2.3_ on disk)
Also, if I then install a newer version of the same solution, I would expect that the still existing virtual dir gets mapped to the new version folder /solution_1.2.4_). But instead the virtual dir is mapped on the old (removed) folder.
These are my settings ....
<!-- IIS AppPools-->
  <ItemGroup>
    <IISAppPool Include="BiztalkAppPool">
      <DeployAction>Create</DeployAction>
      <DotNetFrameworkVersion>v4.0</DotNetFrameworkVersion>
      <Enable32Bit>false</Enable32Bit>
      <IdentityType>ApplicationPoolIdentity</IdentityType>
      <PipelineMode>Integrated</PipelineMode>
      <UndeployAction>None</UndeployAction>
    </IISAppPool>
  </ItemGroup>

  <ItemGroup>
    <IISApp Include="XXXX.YYYYY">
      <AppPoolName>BiztalkAppPool</AppPoolName>
<Exclusions>*.csproj;Web.Debug.config;Web.Release.config;WcfServiceDescription.xml;obj;Properties</Exclusions>
      <DeployAction>CreateOrUpdate</DeployAction>
      <PhysicalPath>..\XXXX.YYY.PATH</PhysicalPath>
      <SiteName>Default Web Site</SiteName>
      <UndeployAction>Delete</UndeployAction>
      <VirtualPath>/XXXX.YYYYYYYY</VirtualPath>
    </IISApp>
  </ItemGroup>

"undeploy my MSI" doesn't make sense.  You must execute the undeploy, then uninstall the MSI, then install the new MSI, then execute the deploy.  If you are just uninstalling the MSI, then all of the artifacts will remain in BizTalk and IIS.  Are you executing the undeploy process before uninstalling the MSI?


typo: reqest -> request

Summarize the change in the pull request title
Describe your change, specifically why you think it's needed.
Fixes #Issue_Number (if necessary)

@nschonni : Thanks for your contribution! The author, @aczechowski, has been notified to review your proposed change.

Thanks @nschonni for catching this
#sign-off


Sending different init frames to a slave...

What should this issue tell anyone? @Gubelat

Sorry, i was not able to delete this headline. and it was late Yesterday.
The Problem is that my Heatmeter is running on batteries. So it restrict the amount of readouts in standard telegram mode.
If i run the serial meter readout tool, i am only can poll data every 69 minutes for standard telegrams. If i run it for short telegrams i can run the readout every 5 minutes without restrictions.
So i need to send a special init Frame to the meter to get this short telegrams.
The sequence looks like this:
Switching to short telegram mode: 1 response telegram:
68 04 04 68 73 PB 50 50 CS   or
68 04 04 68 73 PB 53 50 CS
"50” means sending instantaneous values (for controls)

PB = Primary Bus Adress
CS = Checksum

The question is now how to edit the source code for sending this init Frame to the Meter?
There is no command line option to allow sending other init frames.
German Original Specification:
Mbus telegramm spec sensonic 3_08.09.pdf
Englisch Translation (Gooogle)
Mbus telegramm spec sensonic 3_08.09.de.en.pdf


Assertion failure when planning multiple create view statements

From @sangli00 on Gitter
I find a big bug.
if i used pgadmin3 login pipelineDB and execute sql script.
like this :
create continuoux view x1 as select xxxxxx from stream;
create continuous view x2 as select xxxxxx from stream;
...
create continuous view x10 as select xxxxxx from stream;
I need created more cv.
now ,I execute all sql script.
pipelineDB is break,is down ....
error log:
TRAP: FailedAssertion("!(list_length(parsetree_list) == 1)", File: "cont_plan.c", Line: 417, PID: 35223, Query: (null))
TRAP: FailedAssertion("!(list_length(parsetree_list) == 1)", File: "cont_plan.c", Line: 434, PID: 35222, Query: (null))
parsetree_list = pg_parse_query(view->query); we will need parse this query.
if run this sql script , a view query is so much subquery. what the fuck.....
Assert(list_length(parsetree_list) == 1); this is false . list_length(parsetree_list) = 10.
I don't know ,maybe is pgadmin3 problem,maybe pipelineDB used flex & bison create object is wrong.

@usmanm  thinks .
sometimes ,I used pgadmini3 run sql script, pipelineDB is down.
I need modify source , modify ContinuousQueryWorkerMain &ContinuousQueryCombinerMain
add   "sleep(60);"
restart pipelinedb.  quick login pipeline and drop this continuous view.
I used \i /path/xx.sql created is OK.
I copy create script in command ,is OK.

Hi @sangli00, if possible can you paste a .sql file here that reproduces this error on the master branch? Thanks!

I haven't been able to reproduce this on master, closing for now.


How do I set the URL of my XMPP server which is in the cloud vs XMPP-FTW at localhost?

I understand that XMPP-FTW is a node.js server sitting between my client code and my XMPP server. I have a Prosody XMPP server running in the cloud, and I have my client and my XMPP-FTW server code on my localhost machine.
I am following along the book Practical XMPP but it assumes everything is installed on localhost.
Where is it that I set the address of the XMPP server, so that when I send a message such as 'xmpp.login' from my client to the XMPP-FTW server, the latter will route this over to my XMPP server in the cloud? Do I need to have any code on my XMPP server? I thought that the XMPP-FTW code would send XML to my XMPP server, and receive XML from that server, translate it to JSON and then pass this along to my client. Is this the case?

The server is determined from the domain part of your jid, e.g<EMAIL_ADDRESS>would do a look up for the XMPP server on github.com. If you are running the XMPP server on a specific host and don't have the DNS set up correctly then you can pass a host parameter during login.
No special code required on your XMPP server, XMPP-FTW is essentially an XMPP client itself.

I thought that the XMPP-FTW code would send XML to my XMPP server, and receive XML from that server, translate it to JSON and then pass this along to my client. Is this the case?

:+1:

I am specifically following Chapter 4 of Practical XMPP.
Quick recap of what this means:
I have an index.js file which sets up a local http server, sets up a Primus instance using the server and websockets transformer, I use the primus-emitter plugin on my primus instance, and generate the primus client-side code in a scripts folder. My served index.html creates a new Primus socket, and based on the generated client-side code it will connect to my server. At this point there is not XMPP-FTW yet.
When the client does establish the socket connection to my http server, then we get the XMPP-FTW instance created.
My client also loaded a script called xmpp.js which defines some callbacks for specific standard XMPP-FTW events on the socket, such as xmpp.connection.
Now to my still present issue:
I can input a login (full JID from my own XMPP Prosody XMPP server in the cloud) and password. I can see in my Prosody log that it does log me in when I click login. However, the xmpp.connection event is not emitted on my client-side code.
I can open an Adium XMPP client, and log in to my XMPP server as well. From my own client, I can type a message in a textarea and send to this Adium client, and the latter does receive the message. I can then reply to the message, from the Adium client to my own client. I have this code in my xmpp.js:
socket.on('xmpp.chat.message', function(incoming) {
  console.log('Incoming');
})

My callback does not get called. However, in my terminal, I have the environment variable DEBUG=* and I see that
xmpp:connection receive: <message id='purplee496fb37' type='chat'<EMAIL_ADDRESS><EMAIL_ADDRESS>xmlns='http://jabber.org/protocol/chatstates'/><body>Hi there!</body></message>

So my own client does indeed receive the message, but why aren't the XMPP-FTW events being fired?

Ok, if you open up developer tools and monitor the websocket connection, are you seeing the data arrive at the browser (if not the problem will be at the xmpp-ftw server).
If the xmpp.connection event isn't firing then none of the callbacks will be registered in order to receive your messages actually. So we'll need to look at that.
does your set up work here: https://xmpp-ftw.herokuapp.com/demo/ ?

Do you mean if I can successfully communicate with my XMPP server using the demo? Yes I can.
Ok, I'm inspected the websocket frames, and I am indeed getting an 'xmpp.connection' message.
In my xmpp.js I had:
socket.on('xmpp.connection', connected);
var connected = function(details) {
  console.log('connected');
  $('p.connection-status').html('Online');
};

I switched the order of those statements and it worked, i.e., defined connected before passing it as a callback. I switched the order for the other event handlers as well, so now I am successfully logging in and sending messages.
One further question: in the documentation, there is a section entitled Server-side with the text If you want to run xmpp-ftw server side (e.g. to write a bot) then this should be a good starting point:.
What is this code for exactly? There is no explanation accompanying that code.

\o/
Basically if you want to work with XMPP server side but not learn how to use XMPP if you already know XMPP-FTW then you can use that code to set up XMPP-FTW server side.

So in the case of having a browser client, an XMPP-FTW node.js server, and a separate XMPP server, when you say 'server-side' you're referring to just having an extra server that would be able to handle incoming stanzas using


Upload annotations for coronal of raju data

This should be done for easy visualization of the raju brains
See https://github.com/neurodata/ndstore/issues/325

@kkutten1 When you open an issue for yourself remember to assign. It makes it simple for me to keep track of issues and who is responsible for them. :)

Sure but I can't assign issues here because I don't have the right permissions.  You may need to give me write access.  I promise I won't break anything :-)

No worries. I will curate the issues in that case. Thanks :)


Turning the 'enable overlay' variable to false breaks the hand landmarks.

Hello! I am nearing my project's end using this awesome plugin, but I'm facing a slight issue wrapping things up.
I don't want the Mediapipe window showing the raw camera feed to pop up on the final packaged project.
Turning the 'enable overlay' variable to false seems to do that, but it greatly reduces the performance of the hand landmark tracking.
My intended outcome is simply to prevent the extra window showing the webcam feed from popping up.
Am I doing this the wrong way?
Thank you.

disabling overlay shouldn't affect the performance, can you reproduce it on plugin examples?

Yes, even on the plugin examples, turning off 'enable overlay' severely decreases the performance.
How I tested this is by packaging the plugin examples and testing it on other computers as well.
When the build was packaged without 'enable overlay' left on, hand tracking became very jagged and inaccurate.

check log file for error/warning spam messages
also update to https://github.com/wongfei/ue4-mediapipe-plugin/commit/80ceef798105c5215c98b2da1e9e189f87102260
it adds console command "stat MediaPipe" that will show number of processed packets per second (for hand tracking should be around 30)

I did not exactly solve the issue, but I found a good workaround.
Turning on 'EnableVideoTexture' seems to have the same effect. I just remove the mesh when I don't want it on the level.
I still don't understand what caused the issue but thank you for your time.


Prepare for new release of Confluence

Workflow links: https://github.com/atlassian-labs/atlassian-slack-integration-server/actions/runs/5010468596, https://github.com/atlassian-labs/atlassian-slack-integration-server/actions/runs/5010470301, https://github.com/atlassian-labs/atlassian-slack-integration-server/actions/runs/5010472475.

Release check passed for Confluence 8.3.0


Memory leak with lazy callbacks

I was running valgrind on the tests and it found various leaks within CSIPaddLazyCallback:
==8578== 29 bytes in 1 blocks are still reachable in loss record 1 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x6098989: strdup (strdup.c:42)
==8578==    by 0x573C7D1: addMemlistEntry (memory.c:159)
==8578==    by 0x573CCC3: BMSallocMemory_call (memory.c:379)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
[...]
==8578== 40 bytes in 1 blocks are still reachable in loss record 5 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x573CC76: BMSallocMemory_call (memory.c:370)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
==8578==
==8578== 40 bytes in 1 blocks are still reachable in loss record 6 of 12
==8578==    at 0x4C28C20: malloc (vg_replace_malloc.c:296)
==8578==    by 0x573C784: addMemlistEntry (memory.c:154)
==8578==    by 0x573CCC3: BMSallocMemory_call (memory.c:379)
==8578==    by 0x405160: CSIPaddLazyCallback (in /home/rs/src/CSIP/test/test)
==8578==    by 0x4024B2: test_lazy (in /home/rs/src/CSIP/test/test)
==8578==    by 0x40380F: all_tests (in /home/rs/src/CSIP/test/test)
==8578==    by 0x403984: main (in /home/rs/src/CSIP/test/test)
==8578==
[...]

At first I thought that we had to SCIPfreeMemory all the CONSHDLRDATA*. But when I tried that, I got a "double free" error. Do we need to free something that is stored within this callback data?

there is a TODO in the code:
/* TODO: we should implement the free callback to release conshdlrdata */
did you try that?

I did not see that TODO. No I tried doing the freeMemory at the wrong  time, it seems.


fixing most recent fork as default and making sure it fires when gbh …

…is called on its own

Type of Change

[x] Bug fix (non-breaking change, fixes #4841 )
[ ] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] Nunit test is included
[ ] Documentation
[ ] Build system

Purpose
Ensures that the recovery fork check is called every time GBH is called on it's own (ie not by another command)
Sets default recoveryfork to the latest as that seems to be what people are expecting
Approach
Added a check for being called in a scriptblock which is the the other outcome from Get-PsCallstack
Set $RecoveryForkID to the latest recoveryforkid

This is great, thank you for all of your work on your fabulous commands 🙇


moved contents from scripts/disvis to disvis/disvis.py to streamline …

…with powerfit routine

Gydo, this is linked to my previous email, we found a workaround that kind of simplify the whole code and make it work nicely in the same way that powerfit works ;)

OK, but I prefer it better if the script of disvis is not in disvis.py. It
a bit weird with the naming then, but else merging this with the devel
version with ambiguous restraints in the near future is going to be
annoying. So for now I will move it to a script.py or main.py file in the
disvis package.
2016-11-03 9:30 GMT-04:00 Mikael Trellet<EMAIL_ADDRESS>
Gydo, this is linked to my previous email, we found a workaround that kind
of simplify the whole code and make it work nicely in the same way that
powerfit works ;)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/haddocking/disvis/pull/5#issuecomment-258142284, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFNIFlI367jsGU3f-7028n6X_hA2LBiFks5q6eH_gaJpZM4KoV6z
.

sure as long as we can use from disvis import something which would give access to the main routine from within python with something.main()  that would be fine.

It is done. The script has been moved to main.py.
2016-11-03 12:26 GMT-04:00 Joerg Schaarschmidt<EMAIL_ADDRESS>
sure as long as we can use from disvis import something which would give
access to the main routine from within python with something.main() that
would be fine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/haddocking/disvis/pull/5#issuecomment-258195187, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFNIFjtXLQ8PG1i3CPAi8JtqpJ_935eHks5q6gsxgaJpZM4KoV6z
.


test if permissions are too much

test if permissions are too much

I think they are just about right :)


Can I know how to load this Lora in the script? Thanks

Describe the issue

Loading LoRA script is here!
https://github.com/haotian-liu/LLaVA/blob/80540fb4bf4dad118d87d42bd2fb55e6f3b96f16/llava/model/builder.py#L49-L82


dd: allow skipping and seeking in FIFOs

First, this pull request creates a Source enum and simplifies the Input struct in dd, mirroring pull request #4134 that introduced the Dest enum and a simplified Output struct. Second, this pull request adds support seeking in FIFOs given as either input or output (or both). For example, dd seek=1 of=fifo and dd skip=1 if=fifo will now work.
This should cause GNU test suite file tests/dd/no-allocate.sh to change from ERROR to FAIL status.
Fixes #3321.

Well, there's a few platform-compatibility issues I'll need to work out, but this is good news:

Congrats! The gnu test tests/dd/no-allocate is no longer ERROR!

Looks like something is not working right on macos:
2022-11-20T20:44:52.4512090Z test test_dd::test_seek_output_fifo has been running for over 60 seconds
2022-11-20T20:44:53.4303380Z test test_dd::test_skip_input_fifo has been running for over 60 seconds

What do you think about just skipping those two tests on macos and opening a new issue requesting an implementation that works on macos? I don't have a Mac computer so it's hard for me to debug these issues.

I tried getting the macos build to succeed using @tertsdiepraam's suggestion, but it doesn't seem to work.

I think the dd_out should also just become a free function dd, but that's not high priority.

Great minds think alike; I have such a branch on my local machine but it depends on this branch, so I was waiting until this is merged.

I skipped the tests for this feature on macos and freebsd because I wasn't sure how to make the implementation work.


Scope doesn't return the data asked from scope on flutter

Discussed in https://github.com/pocketbase/pocketbase/discussions/5279

Originally posted by CreeperFarm July 30, 2024
Hi,
I'm having a huge trouble while I'm trying to get the user's birthday and his gender with google.
So I use the .authWithOAuth2() method with this code :
pb.collection('users').authWithOAuth2('google',` (url) async {
        await launchUrl(url);
      },
      scopes: [
        'email',
        'profile',
        'https://www.googleapis.com/auth/user.gender.read',
        'https://www.googleapis.com/auth/user.birthday.read'
      ],
      createData: {
        "role": "user",
        "emailVisibility": true,
      });

and what it return is :
{"token":"eyJhbGciOiJ..........45d5KuP0kxY","record":{"id":"jtsqafxtxkrzcho","created":"2024-07-30 18:15:05.283Z","updated":"2024-07-30 18:15:05.283Z","collectionId":"_pb_users_auth_","collectionName":"users","expand":{},"avatar":"","birthday":"","email":"usermail@gmail.com","emailVisibility":true,"gender":"other","role":"user","username":"users44988","verified":true},"meta":{"id":"107543407980256723726","name":"CreeperFarm","username":"","email":"usermail@gmail.com","avatarUrl":"https://lh3.googleusercontent.com/a/ACg8ocKqvZOBNq2nkrc4j_WYSMN_cruqDY0hLvLkAVF2q-RfFjdy8lCB=s96-c","accessToken":"ya29.a0A................sFXzyHIrF<…>

So, I don't get the answer of the scope. How can I get it so I can get the users birthday and his gender.
Thanks in advance for your answer and sorry my English is bad.

Please don't post the same thing multiple places.
In any case, PocketBase has nothing to do with the scopes and this is not a PocketBase issue. These scopes are only used to generate the url in launchUrl (you can print the launched url and confirm that the scopes are added as space separated list under the scope query parameter).
It depends on the OAuth2 provider and its OAuth2 app configuration whether the requested scopes will be returned or not. If there are returned, additional fields are returned under the meta.rawUser prop.
More than this I don't know how to help further.


unprocessed 'init' attribute

Steps to reproduce the issue
Consider the following code:
module inivalue(i_clk, i_val, o_val);
	input	wire	i_clk;
	input	wire	[2:0]	i_val;
	output	reg	[1:0]	o_val;

	reg	[2:0]	r_val;

	initial	r_val = 0;
	always @(posedge i_clk)
		r_val <= r_val + i_val;

	always @(*)
		o_val = r_val[1:0];

endmodule

Notice the unused bits of r_val.
If you process this code with either yosys -p 'synth_ice40 -top inivalue' inivalue.v or yosys -p 'synth_ecp5 -top inivalue' inivalue.v, yosys will return the warning:
Warning: Wire inivalue.r_val has an unprocessed 'init' attribute.

This warning suggests that an initial attribute has not been applied, when in reality what's taking place appears to be an appropriate optimization.
Expected behavior
I expect either 1) no warnings if the synthesized design matches the incoming warning, or 2) a more specific warning indicating that the unprocessed init attribute is associated with an unused bit.
Actual behavior
See the warning above.

If it helps at all, I'm seeing messages like this in the yosys-bench CI (running synth_ice40) too: https://cirrus-ci.com/task/6520543497945088

I think #966 fixes this. @eddiehung can you confirm that this also fixes the warnings you are seeing in yosys-bench?

This seems to fix the issue for the MCVE listed above, but not in general.  I'm still getting several other errors of this same type.
Let me see if I can put together another MCVE to help chase down the other examples I have of this bug.

I have three remaining examples of this, and each of the three examples I have crosses logic between large modules, so generating the MCVE takes some amount of work.  That said, I managed to create a second one, attached below.  Strangely, this error only occurs with synth_ecp5, and not with synth_ice40
sixremain.zip

Pushed another commit to #966. Please let me know if that fixes things. @ZipCPU

This fixes all of my problems with these initial warnings.  Thank you.  Feel free to close this issue when you merge #966 .

Negatory from me. I'm still seeing:
yosys -qp synth_ice40 sddac.v
Warning: Wire sddac.quant_in has an unprocessed 'init' attribute.
Warning: Wire sddac.quantizer has an unprocessed 'init' attribute.

when applied to sddac from here.

Warning: Wire sddac.quant_in has an unprocessed 'init' attribute.
Warning: Wire sddac.quantizer has an unprocessed 'init' attribute.

@eddiehung These warnings are both completely justified afaict. Both of those regs are initialized to 0 but are then driven by an always @* block.
    reg signed [20:0] quant_in = 0;     // Q(2,19)
...
    reg               quantizer = 0;
...
    always @(*)
    begin
...
        quant_in  <= state2 + $signed(lfsr_fb ? -21'h4000 : 21'h4000);
...
        quantizer <= quant_in[20];
    end

And the wires are being used, so no no-warnings-exception for unused regs.
CC @trcwm (Also, using non-blocking assignments (<=) in an always @* can lead to race conditions in simulation. http://www.sunburst-design.com/papers/CummingsSNUG2000SJ_NBA_rev1_2.pdf It's not as bad as using blocking assignments incorrectly in a clocked always block, but still super hard to troubleshoot if you get unlucky enough to encounter such a race condition in the wild.)

Init and non-blocking assignment issues in SDDAC should be fixed now.

Thanks @trcwm !
@eddiehung Any other suspicious warnings of this kind that I should have a look at?

All good from what I can tell.
For the record, these files no longer produce an init warning:
https://raw.githubusercontent.com/alexforencich/verilog-ethernet/fb4abb6b39fe1536162dfdc41d3cf64beb8fa834/rtl/ip_eth_tx_64.v
https://raw.githubusercontent.com/alexforencich/verilog-ethernet/8b8cfd96fdc95913c87ba42188c5045f340df6b0/lib/axis/rtl/axis_fifo.v


Possible overflow

CodeQL has picked up this issue in a number of places in our extracted code: https://github.com/microsoft/CCF/security/code-scanning/1?query=ref%3Arefs%2Fheads%2Fmaster
In case the link doesn't work:

We do not use the Frodo KEM, and it's not clear to me from context if this really can overflow if that's impossible because of some constraint somewhere, but it seemed worth reporting just in case. We're on 983baba407d2c91134ee270656a3d2c35683e1ff.

From a quick look, according to the definition of that function, there is a precondition that makes sure this never overflows.
https://github.com/project-everest/hacl-star/blob/983baba407d2c91134ee270656a3d2c35683e1ff/code/frodo/Hacl.Impl.Frodo.Sample.fst#L115
A second pair of eyes would be nice... : )

This is a computation of the form E * sizeof T.
E is statically guaranteed in F* to not exceed 2^32 - 1. So the warning is irrelevant, I think. (Amaury do you buy this?) Unless of course you're at the C/F* boundary, and you're calling an F* function from C, and the n1 parameter is provided by the caller, in which case we expect the caller to be aware of those restrictions and not violate them, otherwise all bets are off.
The outer multiplication E * sizeof T does not overflow here because since sizeof uint8 = 1, we're good to go.
In the general case where sizeof T > 1, KRML_CHECK_SIZE guarantees that there will be no overflow using SIZE_MAX.
Happy to look into this into more detail if this still seems suspicious. I'm not foolish enough to pretend I can't be bitten by C's integer promotion rules.

@msprotz that sounds entirely reasonable to me, I thought there would be a static guarantee but I wasn't sure where to look for it. So long as 2 * n1 * n2 -> E is guaranteed not to overflow, I believe there is no problem. The warning is against the promotion to unsigned long that happens because of the multiplication with sizeof, which may hide an overflow, but cannot here. So I will tag this as a false positive.
@msprotz @beurdouche thank you for taking the time to look into this!


Start rolling out the SierraRecordNumber type

Follows #2444, spun out of #2432.

Look good to me and much better than before. Only thing for me is: can we split the single test in SierraRecordNumberTests into multiple tests with individual titles, please?
Otherwise good to merge for me