Congenital intrinsic factor deficiency  MONDO:0009852

Looking to get clarification on the term Congenital intrinsic factor deficiency,  MONDO:0009852. and how to map it to UMLS's concepts/CUIs.
The OMIM # (261000), GARD 3024, Orpha332 and MESH C563242 all support a match to the UMLS concept for 'intrinsic factor deficiency" (C1394891)
However SCTID 34925000  maps to "Megaloblastic anemia due to inborn errors of metabolism" which seems to be a near match, but not the same concept. I suggest you remove this X-ref.
Would SCTID 60504009 "Megaloblastic anemia due to congenital deficiency of intrinsic factor" be the better fit for this concept? Or is this more of a clinical presentation of the "intrinsic factor deficiency" concept? Neither SCTID 60504009  nor the UMLS term that it maps to C0271969 have any indication of age of onset, so "hereditary JUVENILE megaloblastic anemia..." seems to make the distinction that this MONDO concept accounts for a childhood age of onset, but the SCTID 60504009 is "congenital" which might be an infantile/neonatal onset?  Is that biologically relevant, Is there an adult form, too? I have no idea. NORD suggests there are different subtypes of pernicious anemia, including an adult form, but this rabbit trail is getting a bit complicated (https://rarediseases.org/rare-diseases/anemia-pernicious/)
Further complicating things... OrphaNet:332 additionally maps to C0340957  "Congenital deficiency of intrinsic factor" (which UMLS equates with SCTID 234361004) as well as the C1394891 term mentioned above.
For our purposes at MedGen/GTR, I'm going to map this MONDO term to C1394891, since that has the same OMIM reference. But if there is any support for getting UMLS to merge some redundant concepts based on discussion here, I will follow up with UMLS. (Maybe merge C1394891 and C0340957   ??)

@nicolevasilevsky, perhaps this is a topic for additional discussion if we can fit it in for the workshop? Or maybe someone from GARD or OrphaNet can weigh in here?

@alyea-gard
@ana-rath-orphanet
@Orphanet

This one will be reviewed in GARD (as you know we are in a review process)  I think that the name "congenital intrinsic factor deficiency" or "hereditary intrinsic factor deficiency"  is most adequate than the current name "intrinsic factor deficiency" because the intrinsic factor deficiency have several different causes (including the  GIF gene mutation that results in the congenital form that presents from childhood and other that are acquired and may present in adults).  I agree with the suggestion to remove the SCTID 34925000 which maps to a parent term "Megaloblastic anemia due to inborn errors of metabolism".  Also, as suggested, the SCTID 60504009 "Megaloblastic anemia due to congenital deficiency of intrinsic factor" is the better fit for this concept.   Hope this is helpful, Thanks, Gioconda

Hello @kanems
I'm Sabrina, a new curator who recently joined the Mondo team.
We will address your issue as soon as possible. Thank you for your patience.
If this is a high priority, please let me know and we can prioritize it. Thank you!


Fix CreatePatreonTokenFromOAuthToken

.token.token should be just .token

I'll visit this tomorrow to validate the mistake and your fix, then I'll merge. Thanks for catching this.

I'll visit this tomorrow to validate the mistake and your fix, then I'll merge. Thanks for catching this.

@North101 Did you encounter an issue with this? I'm finding that there is a .token.token result.

@North101 Did you encounter an issue with this? I'm finding that there is a .token.token result.

@North101 Did you encounter an issue with this? I'm finding that there is a .token.token result.

Heres a minimal example:
import { AuthorizationCode, ModuleOptions } from "simple-oauth2";
import { CreatePatreonTokenFromOAuthToken } from "patreon-ts/dist/types";

(async function() {
  const code = '';

  const scope = ["identity", "identity.memberships", "campaigns"];

  const PATREON_CREDENTIALS: ModuleOptions<"patreon"> = {
    client: {
      id: <clientId>,
      secret: <clientSecret>,
    },
    auth: {
      tokenHost: <host>,
      tokenPath: <tokenPath>,
      authorizePath: <authorizePath>,
    },
  };
  const client = new AuthorizationCode<"patreon">(PATREON_CREDENTIALS);
  const accessToken = await client.getToken({
    redirect_uri: <redirectUrl>,
    scope: <scope>,
    code,
  });
  const patreonToken = CreatePatreonTokenFromOAuthToken(accessToken);
  console.log(patreonToken);
})();

Error:
TypeError: Cannot read property 'access_token' of undefined
    at Object.CreatePatreonTokenFromOAuthToken (/Users/alexwilde/git/dissonant_voices/node_modules/patreon-ts/src/types/token.ts:64:51)
    at /Users/alexwilde/git/dissonant_voices/src/test.ts:28:24
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
error Command failed with exit code 1.

@North101 Did you encounter an issue with this? I'm finding that there is a .token.token result.

Heres a minimal example:
import { AuthorizationCode, ModuleOptions } from "simple-oauth2";
import { CreatePatreonTokenFromOAuthToken } from "patreon-ts/dist/types";

(async function() {
  const code = '';

  const scope = ["identity", "identity.memberships", "campaigns"];

  const PATREON_CREDENTIALS: ModuleOptions<"patreon"> = {
    client: {
      id: <clientId>,
      secret: <clientSecret>,
    },
    auth: {
      tokenHost: <host>,
      tokenPath: <tokenPath>,
      authorizePath: <authorizePath>,
    },
  };
  const client = new AuthorizationCode<"patreon">(PATREON_CREDENTIALS);
  const accessToken = await client.getToken({
    redirect_uri: <redirectUrl>,
    scope: <scope>,
    code,
  });
  const patreonToken = CreatePatreonTokenFromOAuthToken(accessToken);
  console.log(patreonToken);
})();

Error:
TypeError: Cannot read property 'access_token' of undefined
    at Object.CreatePatreonTokenFromOAuthToken (/Users/alexwilde/git/dissonant_voices/node_modules/patreon-ts/src/types/token.ts:64:51)
    at /Users/alexwilde/git/dissonant_voices/src/test.ts:28:24
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
error Command failed with exit code 1.

Ah, they rewrote a lot of simple_oauth between 4.1.0 and 4.2.0.

Ah, they rewrote a lot of simple_oauth between 4.1.0 and 4.2.0.

I am completely unable to reproduce your issue. All of my tests, including yours, return:

If I apply your fix, all of my examples crash / fail to populate the PatreonToken.

I am completely unable to reproduce your issue. All of my tests, including yours, return:

If I apply your fix, all of my examples crash / fail to populate the PatreonToken.

Could you please try backing up to SimpleOauth 4.1.0 and see if it works?

Could you please try backing up to SimpleOauth 4.1.0 and see if it works?

Just deleted node_modules, I'm using<EMAIL_ADDRESS>and i've pinned<EMAIL_ADDRESS>I've reinstalled them all and I still get the same issue. It also happens when I deploy it to my server.
From my yarn.lock:
<EMAIL_ADDRESS>  version "4.1.0"
  resolved "https://registry.yarnpkg.com/simple-oauth2/-/simple-oauth2-4.1.0.tgz#b531e2c94e7d9ba165ad98cdfca7549e8557bdf7"
  integrity sha512-YoidkclxevXIAE7FU28k+8h4RJV5Q4mGYCQE/CKWexAT5rQsgrztKkkYAe08xyJ0KXTu2pg2yJeYV4hJjzLVCw==
  dependencies:
    "@hapi/hoek" "^9.0.4"
    "@hapi/joi" "^17.1.1"
    "@hapi/wreck" "^17.0.0"
    debug "^4.1.1"

<EMAIL_ADDRESS>  version "0.4.0-2"
  resolved "https://registry.yarnpkg.com/patreon-ts/-/patreon-ts-0.4.0-2.tgz#f835b9cdbe46c026537fa5a0ea1a9fbd55aae303"
  integrity sha512-pgqpnmioaNKmb20TAdlXVyNFlbN7xnhSAOX6jvPusFoVXx1vcy7hFtYvhwwuZJdQr3m4otZKDkQtV8PBujnpow==
  dependencies:
    dotenv "^8.2.0"
    express "^4.17.1"
    form-urlencoded "^4.2.1"
    pg "^8.4.2"
    request "^2.88.2"
    request-promise-native "^1.0.9"
    simple-oauth2 "^4.1.0"
    typescript-string-operations "^1.4.0"

node: v15.5.0
npm: 7.3.0
tsc: Version 4.1.3
yarn: 1.22.10

Just deleted node_modules, I'm using<EMAIL_ADDRESS>and i've pinned<EMAIL_ADDRESS>I've reinstalled them all and I still get the same issue. It also happens when I deploy it to my server.
From my yarn.lock:
<EMAIL_ADDRESS>  version "4.1.0"
  resolved "https://registry.yarnpkg.com/simple-oauth2/-/simple-oauth2-4.1.0.tgz#b531e2c94e7d9ba165ad98cdfca7549e8557bdf7"
  integrity sha512-YoidkclxevXIAE7FU28k+8h4RJV5Q4mGYCQE/CKWexAT5rQsgrztKkkYAe08xyJ0KXTu2pg2yJeYV4hJjzLVCw==
  dependencies:
    "@hapi/hoek" "^9.0.4"
    "@hapi/joi" "^17.1.1"
    "@hapi/wreck" "^17.0.0"
    debug "^4.1.1"

<EMAIL_ADDRESS>  version "0.4.0-2"
  resolved "https://registry.yarnpkg.com/patreon-ts/-/patreon-ts-0.4.0-2.tgz#f835b9cdbe46c026537fa5a0ea1a9fbd55aae303"
  integrity sha512-pgqpnmioaNKmb20TAdlXVyNFlbN7xnhSAOX6jvPusFoVXx1vcy7hFtYvhwwuZJdQr3m4otZKDkQtV8PBujnpow==
  dependencies:
    dotenv "^8.2.0"
    express "^4.17.1"
    form-urlencoded "^4.2.1"
    pg "^8.4.2"
    request "^2.88.2"
    request-promise-native "^1.0.9"
    simple-oauth2 "^4.1.0"
    typescript-string-operations "^1.4.0"

node: v15.5.0
npm: 7.3.0
tsc: Version 4.1.3
yarn: 1.22.10

Looks like patreon-ts is using<EMAIL_ADDRESS>even if I've pinned<EMAIL_ADDRESS>

Looks like patreon-ts is using<EMAIL_ADDRESS>even if I've pinned<EMAIL_ADDRESS>

Found my mistake, thank you for the extra information.

Found my mistake, thank you for the extra information.

@North101 https://www.npmjs.com/package/patreon-ts/v/0.4.1
Please let me know if you find anything else!

@North101 https://www.npmjs.com/package/patreon-ts/v/0.4.1
Please let me know if you find anything else!

No prob. Thanks!

No prob. Thanks!


Test binder OutputDestination only catch one message after a Spring Integration split

Hello,
Describe the issue
An IntegrationFlows does a split after a transform. OutputDestination only catches the last message (reply).
To Reproduce
Reproducible with the following test:
@SpringBootTest
class TestbinderApplicationTests {
    @Autowired
    private InputDestination input;

    @Autowired
    private OutputDestination output;

    @Test
    public void testEmptyConfiguration() {
        this.input.send(new GenericMessage<byte[]>("hello".getBytes()));
        Assertions.assertThat(output.receive().getPayload()).isEqualTo("HELLO".getBytes());
        Assertions.assertThat(output.receive().getPayload()).isEqualTo("HELLO".getBytes());
        Assertions.assertThat(output.receive().getPayload()).isEqualTo("HELLO".getBytes());
    }
    @SpringBootApplication
    @Import(TestChannelBinderConfiguration.class)
    public static class SampleConfiguration {
        @Bean
        public IntegrationFlow uppercaseFlow() {
            return IntegrationFlows.from(MessageFunction.class, spec -> spec.beanName("uppsercase"))
                    .<String, List<String>>transform(source -> List.of(source.toUpperCase(), source.toUpperCase(), source.toUpperCase()))
                    .split()
                    .get();
        }
        public interface MessageFunction extends Function<Message<String>, Message<String>> {
        }
    }
}

The test failed, because only one message is catched
Version of the framework
Spring cloud stream 3.2.1
Thank you.

The reactive approach works
        @Bean
        public Function<Flux<String>, Flux<String>> reactiveUpperCase() {
            return flux -> flux.flatMap(source -> Flux.fromIterable(List.of(source.toUpperCase(), source.toUpperCase(), source.toUpperCase())));
        }

So are you saying that your use case is where you send one message to a function but output 2+.
And it works with reactive function but not with imperative.
Correct?

Hello @olegz,
Correct

An assumption for big picture is not correct.
Look what you do with that IntegrationFlow.
You start it with the IntegrationFlows.from(MessageFunction.class, spec -> spec.beanName("uppsercase")) which essentially a messaging gateway contract.
The contract is like this: Function<Message<String>, Message<String>>. So, when end-user calls this contract, the result is going to be a single message with string payload.
Now how it works.
Spring Integration takes this interface and expose a proxy implementation called gateway. The gateway creates a Message based on the provided argument as a payload and adds a replyChannel header to wait for whatever could be as a return for that contract method call.
This  replyChannel  could be used as a fallback downstream in the flow when we don't specify an outputChannel explicitly on the last endpoint in the flow.
Your last endpoint is a split() and its goal to produce many messages to the output channel. But in our case it does that to the mentioned replyChannel header, which in turn is a return barrier for the gateway in the beginning. It is that TemporaryReplyChannel with the logic like:
	public boolean send(Message<?> message, long timeout) {
			this.replyMessage = message;
			boolean alreadyReceivedReply = this.hasReceived;
			this.replyLatch.countDown();

			String errorDescription = null;
			if (this.hasTimedOut) {
				errorDescription = "Reply message received but the receiving thread has exited due to a timeout";
			}
			else if (alreadyReceivedReply) {
				errorDescription = "Reply message received but the receiving thread has already received a reply";
			}
			else if (this.hasSendFailed) {
				errorDescription = "Reply message received but the receiving thread has exited due to " +
						"an exception while sending the request message";
			}

			if (errorDescription != null) {
				if (logger.isWarnEnabled()) {
					logger.warn(errorDescription + ": " + message);
				}
				if (this.throwExceptionOnLateReply) {
					throw new MessageDeliveryException(message, errorDescription);
				}
			}

			return true;
		}

So, whenever we send a message to this channel it becomes that replyMessage and replyLatch is fulfilled, so a gateway call is good to comeback to the return upstream.
From here we should understand that the rest messages from the splitter are just discarded. See the logic around that alreadyReceivedReply.
Therefore this is exactly what you see with that OutputDestination. And that's an expected behavior just because gateway expectations and splitter output cannot be correlated. There is just not one-to-many request-reply contract in the messaging.
See what we are doing with the splitter in the Stream Applications: https://github.com/spring-cloud/stream-applications/blob/main/functions/function/splitter-function/src/main/java/org/springframework/cloud/fn/splitter/SplitterFunctionConfiguration.java

I agree with Artem.
Also, since you mentioned that it works as expected with reactive function, the imperative variant of such function would be
Function<String, Iterable<String>>

. . . where I do see a bit of an issue


fix(curriculum): less strict calorie count step 87

Checklist:

[X] I have read and followed the contribution guidelines.
[X] I have read and followed the how to open a pull request guide.
[X] My pull request targets the main branch of freeCodeCamp.
[X] I have tested these changes either locally on my machine, or GitPod.

Closes #57147
Related to #57107

This checks the output element for the information that is needed and also removes the after-user-code.

I don't mind. It's really easy for me to forget to remove console.log.


Fix AVG computation in overview tables

Fixes #232
Averages of averages are not equal to overall averages.

please @iriberri CR this

:+1:


How to implement mosaic augmentation along with multi-scale training?

As the title mentioned, I want to achieve mosaic augmentation along with multi-scale training, but the how_to.md doesn't include. Besides, if I want to set the prob of mosaic augmentation to 0.5, will the RandomAffine operation affect normal training?

For multi-scale training, you can add Resize pipeline after Mosaic to resize the image into different scales. Besides,  RandomAffine also can randomly scale the image.
The prob of Mosaic have little influence on the normalization. You can keep the original setting.


Adding missing WIFCONTINUED function on FreeBSD systems

This adds the WIFCONTINUED function to any FreeBSD-like system. Addresses #505

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+
Thanks!

:pushpin: Commit 1b06018 has been approved by alexcrichton

:hourglass: Testing commit 1b06018 with merge f0898c3...

:sunny: Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing f0898c396c6aaa825bb5a549e23436d9b2fcc180 to master...


No command to install via Composer

I have been wanting to install Wolff via composer with no avail.
I tried the following in command line:
composer require Wolff
The command to install should be in the Wiki pages, I suggest.
Find below is a screenshot explaining the issue in details:

Done, thanks!
Wolff - Installation :)


Fix crash in Block.

This assert was overzealous. You can update a RawGestureDetector any
time before the semantics phase, it doesn't have to only be during your
child's layout.
This is good because Block has additional PointerListeners between its
GestureDetector and its Viewport.
Fixes https://github.com/flutter/flutter/issues/2232

LGTM


hotfix/issue-715

This change is

Reviewed 8 of 8 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Make it clear in abstract that public presentations shouldn't be heavily relied on

We should make it clear that public VPs should be rare and not relied on too heavily. We should link to the other protocols.

Can't assign myself 🙁

@brianorwhatever is there anything you'd like to add to the existing state of the spec concerning this issue?


Fix NVTX ranges

Description
The boundaries for the NVTX ranges for WENO and the Riemann solver were broken during the process of adding MI200 support.
Fixes #501
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

Scope

[x] This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

[x] Profile the 3D performance test for 10 time steps, yield the same profile as the one attached to this PR.
fixed_prof.zip

Test Configuration:

What computers and compilers did you use to test this:
Rogue's Gallery UWing Nodes (GH200) -- NVHPC 24.1

Checklist

[x] I have added comments for the new code
[x] I ran ./mfc.sh format before committing my code
[ ] New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
[x] This PR does not introduce any repeated code (it follows the DRY principle)
[x] I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)
To make sure the code is performing as expected on GPU devices, I have:

[x] Checked that the code compiles using NVHPC compilers
[x] Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
[x] Enclosed the new feature via nvtx ranges so that they can be identified in profiles
[x] Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR

This fixes it! Looking at the new ranges, I think it would be useful to wrap everything within s_compute_rhs in an overarching nvtx range called Compute_RHS (below the time_step scale, but above RHS-WENO, for example). you can just put it at the beginning and end of the subroutine.


Expose numerical Window ID

Is your feature request related to a problem? Please describe.
Each OS has their own way of describing a Window ID, and that's managed in Tao's WindowId struct with the different implementations depending on the OS being compiled for. After doing some digging, some of the platform-specific WindowId structs expose their numerical ID (like macOS), while some don't (like Windows). While writing a cross-platform wrapper, and trying to hook into the event loop, I am unable to access the underlying numeric ID, despite them all fitting into a 64bit type:

Win: isize, max 64bits,
Mac: usize, max 64bits,
Linux: u32, max 32 bits,
iOS: *mut Object, which looks like a pointer, so max 64bits? (Please correct me if I'm wrong!)
Android: doesn't have a numeric ID, but I guess the pointer to the WindowID struct is enough, therefore max 64bits?

Describe the solution you'd like
Ideally, I'd like the be able to access the numeric ID to have a reference to it in JS.
Describe alternatives you've considered
An alternative is to use a Hasher because all the implementations derive(Hash). The problem with this, is that I need to send a window ID with quite a few events, and therefore hashing the same thing every time (while it might get cached by the processor) is actually pretty intensive to calculate.
Would you want to assign yourself to implement this feature?

[x] Yes

I would give each struct a way of providing their numeric ID, probably through a getter, and have the Tao API call upon those. I don't know if there's a safer way to do it, though.

[ ] No

We are currently in the middle of trying to minimize the differences between tao and winit hoping to someday switch wry/tauri to use winit directly and to me this feels like an unnecessary change that could potentially get reject upstream so I'd rather not implement it this now.
I think if you are building a wrapper then you already have a Window store and you need to generate ids in order to store the window and use it later; in this case I'd recommend generating your ids, take a look at this Counter struct in druid repo that could be used to generate unique ids.

That's a shame! Is there a roadmap or Project that I can track for the conversion? I'm curious now, and I'd expect it might change the way that my wrapper might integrate...
Thanks for the alternative solution though!

For now, we only have this issue https://github.com/tauri-apps/tao/issues/470 where we keep track of the differences which we try to upstream as soon as possible.


[ENHANCEMENT] Bump KubeVirt to 1.1.0-1505<IP_ADDRESS>

Is your enhancement related to a problem? Please describe.
SLE 15 SP4 general ends on 31 Dec 2023. KubeVirt base image will change. We have to update to a new version.
The supported version for SLE 15 SP5 will be 1.0.0: registry.suse.com/suse/sles/15.5/virt-operator:1.0.0-1505<IP_ADDRESS>
Additional context
https://www.suse.com/lifecycle/#suse-linux-enterprise-server-15

Test Plan

Check can upgrade from v1.2.1 to v1.3.0 can work as expected
Check on fresh install and upgrade path to v1.3.0 Harvester cluster contains the bumped KubeVirt image to

registry.suse.com/suse/sles/15.5/virt-operator:1.1.0-1505<IP_ADDRESS>

Check the basic VM operation can work
Check the basic Volume operation can work

Verifying on v1.3.0-rc (24/01/25).
Result
$\color{Grey}{\textsf{Pending}}$ Check can  **upgrade** from v1.2.1 to v1.2-head until v1.3.0-rc2 $~~$

Wait until v1.3.0-rc2 released

$\color{Grey}{\textsf{Pending}}$ KubeVirt bumped to 1.1.0-1505<IP_ADDRESS> after **upgrade** from v1.2.1 to v1.2-head until v1.3.0-rc2 $~~$

Wait until v1.3.0-rc2 released

$\color{green}{\textsf{PASS}}$ KubeVirt bumped to 1.1.0-1505<IP_ADDRESS> after **fresh install** Harvester cluster $~~$

$\color{Blue}{\textsf{Ongoing}}$ Basic VM operation test plans $~~$

Backend automation
Manual test plans

$\color{Blue}{\textsf{Ongoing}}$ Basic Volume operation test plans $~~$

Backend automation
Manual test plans

Test Information

Test Environment: 3 nodes harvester on baremetal machines
Harvester version: v1.3.0-rc (24/01/25)

Verify Steps
Check can upgrade from v1.2.1 to v1.2-head until v1.3.0-rc2

Check the KubeVirt can be bumped to 1.1.0-1505<IP_ADDRESS> after upgrade complete or fresh installation

Harvester installation path

 Basic vm operation test

 Basic volume operation test

@khushboo-rancher we need to ensure verifying ready-for-test issues is always priority first, so that we will be able to capture issues earlier.
In this issue, the PR had been merged last December, but we are just testing it right now. There is an issue already found by engineering at https://github.com/harvester/harvester/issues/5076#issuecomment-1927301846 before QA.
This is a general rule, so let's make this apply to the team. Thanks.
cc @harvester/qa

Issue about update machine type: https://github.com/harvester/harvester/issues/5005
This issue moved to ready for testing in Feb 1, I think it is the mainly reason QA didn't notice it.

QA usually will not test upgrade before RC release, and our automation upgrade testing didn't contains VM migration while upgrading, that's why we didn't catch the bug which RD filed.
In v1.3.0, upgrade testing is deferred to RC2 (QA also delay upgrade testing in v1.3.0-rc1) because we are bumping Rancher's version^1, and will not support v1.2.1 upgrade to v1.3.0.

@lanfon72 Good clarification. Updating the state of issues should be improved, so we can let QA get involved earlier.

For the issue verification on v1.3.0-rc1, we executed backend automation test to validate basic function of virtual machine and volume. And also run some manual test plans on volume function.
Most of the automation test result for basic vm and volume function looks good.
During the manual testing of volume function, we encounter the known issue #5020 of volume would display not ready state when we create a new source volume without binding to a VM.
And we also have the known issue #5076 for vm can't migrate during upgrade from v1.2.1 to v1.3.0-rc1.
Thanks and according to above comment, we deferred the upgrade test until v1.3.0-rc2 release since we have to bump the Rancher release.
And follow the upgrade path from v1.2.1 to v1.2-head until v1.3.0-rc2 for this issue.
Next we will start testing the pending upgrade test item on v1.3.0-rc2 and rerun all the automation and manual test plans in the following second run verification

Verifying on v1.3.0-rc2 (24/02/7).
Result
$\color{blue}{\textsf{Ongoing}}$ Check can  **upgrade** from v1.2.1 to v1.2-head until v1.3.0-rc2 $~~$

Wait until v1.3.0-rc2 released

$\color{blue}{\textsf{Ongoing}}$ KubeVirt bumped to 1.1.0-1505<IP_ADDRESS> after **upgrade** from v1.2.1 to v1.2-head until v1.3.0-rc2 $~~$

$\color{green}{\textsf{PASS}}$ KubeVirt bumped to 1.1.0-1505<IP_ADDRESS> after **fresh install** Harvester cluster $~~$
harvester-kmzrw:~ # ctr -n k8s.io image ls | grep virt-operator
registry.suse.com/suse/sles/15.5/virt-operator:1.1.0-1505<IP_ADDRESS>                                      application/vnd.docker.distribution.manifest.v2+json sha256:6a1b32dd2e1732c587c15f3673ea97317a6210735425d50072f1f146eed6a398 181.1 MiB linux/amd64 io.cri-containerd.image=managed

$\color{blue}{\textsf{Ongoing}}$ Basic Volume operation test plans $~~$

$\color{green}{\textsf{PASS}}$ the following volumes backend automation test plans  $~~$

[x]    - test_1_volumes.py::test_create_volume_backing_image[json]
[x]    - test_1_volumes.py::test_create_volume_backing_image[yaml]
[x]    - test_1_volumes.py::TestVolumeWithVM::test_delete_volume_bounded_on_existing_vm
[x]    - test_1_volumes.py::TestVolumeWithVM::test_delete_volume_bounded_on_deleted_vm

$\color{blue}{\textsf{Ongoing}}$ the following volumes backend automation test plans  $~~$

[ ]   Add/remove disk to Host config
[ ]    Create Volume root disk blank Form with label

Failed with known issue: https://github.com/harvester/harvester/issues/5020

[ ]  Delete volume that is not attached to a VM (e2e_be_fe)
[ ]   Delete volume that was attached to VM but now is not (e2e_be_fe)
[ ]   Detach volume from virtual machine (e2e_fe)
[ ]  Edit volume increase size via YAML (e2e_be)
[ ]  Edit volume increase size via form (e2e_fe)
[ ] Negative delete Volume that is in use (e2e_be)

$\color{blue}{\textsf{Ongoing}}$ Basic VM operation test plans $~~$

$\color{green}{\textsf{PASs}}$ the following VM backend automation test plans  $~~$

[x]    - test_3_vm_functions.py::test_minimal_vm
[x]    - test_3_vm_functions.py::TestVMOperations::test_pause
[x]    - test_3_vm_functions.py::TestVMOperations::test_unpause
[x]    - test_3_vm_functions.py::TestVMOperations::test_stop
[x]    - test_3_vm_functions.py::test_minimal_vm
[x]    - test_3_vm_functions.py::TestVMOperations::test_start
[x]    - test_3_vm_functions.py::TestVMOperations::test_restart
[x]    - test_3_vm_functions.py::TestVMOperations::test_softreboot
[x]    - test_3_vm_functions.py::test_minimal_vm
[x]    - test_3_vm_functions.py::TestVMOperations::test_migrate
[x]    - test_3_vm_functions.py::TestVMOperations::test_delete
[x]    - test_3_vm_functions.py::test_create_stopped_vm
[x]    - test_3_vm_functions.py::TestVMResource::test_create_schedule_on_maximum[cpu]
[x]    - test_3_vm_functions.py::TestVMResource::test_create_schedule_on_maximum[memory]
[x]    - test_3_vm_functions.py::TestVMResource::test_update_schedule_on_maximum[cpu]
[x]    - test_3_vm_functions.py::TestVMResource::test_update_schedule_on_maximum[memory]
[x]    - test_3_vm_functions.py::test_minimal_vm
[x]    - test_3_vm_functions.py::TestVMResource::test_update_enable_usb
[x]    - test_3_vm_functions.py::TestVMResource::test_update_enable_user_data
[x]    - test_3_vm_functions.py::test_create_vm_no_available_resources[cpu]
[x]    - test_3_vm_functions.py::test_create_vm_no_available_resources[mem]
[x]    - test_3_vm_functions.py::test_create_vm_no_available_resources[disk]
[x]    - test_3_vm_functions.py::test_create_vm_no_available_resources[mem-and-cpu]
[x]    - test_3_vm_functions.py::test_create_vm_no_available_resources[mem-cpu-and-disk]
[x]    - test_3_vm_functions.py::test_vm_with_bogus_vlan
[x]     - test_3_vm.py::test_multiple_migrations
[x]     - test_3_vm.py::test_migrate_vm_with_user_data
[x]     - test_3_vm.py::test_migrate_vm_with_multiple_volumes

$\color{blue}{\textsf{Ongoing}}$ the following volumes backend automation test plans  $~~$

Test Information

Test Environment: 4 nodes harvester on baremetal machines
Harvester version: v1.3.0-rc2 (24/02/07)

Verify Steps
Check can upgrade from v1.2.1 to v1.2-head until v1.3.0-rc2

Check the KubeVirt can be bumped to 1.1.0-1505<IP_ADDRESS> after upgrade complete or fresh installation

 Basic volume operation test

Run backend automation tests test_volume.py on internal E2E Jenkins jobs
Run manual test plan to validate basic user operations

 Basic vm operation test

Run backend automation tests test_vm.py and test_vm_function.py on internal E2E Jenkins jobs
Run manual test plan to validate basic user operations

During the manual testing of volume function, we encounter the known issue #5020 of volume would display not ready state when we create a new source volume without binding to a VM.

From https://github.com/harvester/harvester/issues/5020#issuecomment-1911535650, #5020 is frontend issue and it's related to LH. I think we can skip it. Thank you.

Update the upgrade test status, we follow the upgrade path for this issue:
v1.2.1 -> v1.2-head -> v1.3-head
Test environment and Prerequisite

Test Environment: Three nodes harvester on remote kvm machines
Harvester version: v1.2.1 -> v1.2-head -> v1.3-head
Rancher version: v2.8.1
One RKE2 guest cluster v1.25.16 and VM is running

[PASS] Upgrade from v1.2.1 to latest v1.2-head (24/02/19)

Before upgrade, we already have one RKE2 guest cluster VM running on Harvester

We can successfully upgrade Harvester from v1.2.1 to v1.2-head  (v1.2-d542563e-head)

During the upgrading system service stage, it can upgrade KubeVirt to 1.1.0-1505<IP_ADDRESS>

And also can upgrade the longhorn-manager

During the upgrade we would encounter issue #5076 which cause VM failed to migrate to another node

The current solution so far is the upgrade process would wait until timeout and shutdown the RKE2 guest cluster VM

After upgraded to v1.2-head, we could find the VM already been shutdowned

[Ongoing] Upgrade from v1.2-head to v1.3-head
Next we would proceed to upgrade to latest v1.3-head.

During the upgrade path from v1.2.1 to v1.2-head until v1.3-head
According to the previous test result

With RKE2 guest cluster running on Harvester, we can upgrade from v1.2.1 to v1.2-head.
After upgraded to v1.2-head, the VM would be automatically shutdown
We start the guest cluster VM again and proceed the upgrade to v1.3-head
The upgrade process to v1.3-head would stuck in pre-draining which would be blocked by issue #5076

I would start over another upgrade without VM only confirm we can upgrade in a empty environment and especially the KubeVirt have been bumped to the expected version.

[PASS] Upgrade path v1.2.1 > v1.2-head > v1.3-head (without VM)

We can successfully upgrade to v1.3-head (v1.3-50a00ad1-head)

After upgrade to v1.3-head, the KubeVirt bumped to 1.1.0-1505<IP_ADDRESS>

node1:~ # ctr -n k8s.io image ls | grep virt-operator
...
registry.suse.com/suse/sles/15.5/virt-operator:1.1.0-1505<IP_ADDRESS>                                       application/vnd.docker.distribution.manifest.v2+json sha256:6a1b32dd2e1732c587c15f3673ea97317a6210735425d50072f1f146eed6a398 181.1 MiB linux/amd64 io.cri-containerd.image=managed

While running the virtual machine manual test plan after upgraded to v1.3.0-rc3.
We encounter the known issue #5005 for not able to create pc machine type vm.
Double check the vm machine type in the following test scenarios, update the test result and impact

After upgraded to v1.3.0-rc3:

Can't create vm with the pc machine type, prompted the machine type not supported

Can create vm with the qc35 machine type

Can't cover the vm already have qc35 machine type to pc type, prompted the same message

Already have one pc type and one qc35 type vm prior to upgrade

Can't start the pc type vm already exists in v1.2.1

Can start the qc35 type vm already exists in v1.2.1

Can't convert qc35 type vm to pc type.

Complete the verification on v1.3.0-rc3 and v1.3-head
According to the test result, we can summarize the following result:

After shutdown all vms (for #5076),
We can successfully upgrade from v1.2.1 to v1.2-head until v1.3.0-rc3
After upgrade to v1.2-head and v1.3.0-rc3, the KubeVirt can bumped to 1.1.0-1505<IP_ADDRESS>
In a newly installed v1.3.0-rc3 cluster, the KubeVirt can bumped to 1.1.0-1505<IP_ADDRESS>
Pass all volume backend automation test plans
Pass all volume manual test plans
Pass most of the virtual machine backend automation test, except the pc type releted test plans (for #5005)
Found some known and new issues while running the virtual machine manual test plans as the following:

#5005 (Known issue)
#5272
#5266
#5264
#5276

@bk201, @FrankYang0529
May I confirm whether above issues are related to Kubevirt functionality or inside the scope.
Since most of the basic features of virtual machine and volume are work as expected.
If they are not related, I am thinking to close this enhancement ticket and track these issues separately. Thank you.

@TachunLin we can close this one and track those remaining with the issues you file.

Close this issue since we achieved most of the expected test result for bumping the Kubevirt and checked the basic volume and virtual machine functionalities.
We would separately track the remaining issues afterward.


containers are not terminated on overtime

Let's fix

Job gh:yegor256/threecopies#4 assigned to @yegor256, please go ahead (policy).

+15 points just awarded to @yegor256, total is +3045.

working fine now

Job gh:yegor256/threecopies#4 is now out of scope.

+30 points just awarded to @yegor256, total is +3120.


prettierPath only works via CLI option, not config

🐛 Bug Report
When specifying the prettierPath option in a config file or JSON object, it is ignored. Jest prints the warning:
    Jest: Inline Snapshots requires Prettier.
    Please ensure "prettier" is installed in your project.

By putting a console.log in the module at the specified prettierPath, it is apparent that Jest never even attempts to import the supplied path. The value that eventually makes it to getPrettier() in the jest-jasmine2 package is null.
Using the same value with --prettierPath via the CLI works.
To Reproduce
Specify the prettierPath option in a config file or --config JSON object.
Expected behavior
The provided path will be used for Prettier.
Link to repl or repo (highly encouraged)
Please provide either a repl.it demo or a minimal repository on GitHub.
Issues without a reproduction link are likely to stall.
Run npx envinfo --preset jest
Paste the results here:
npx: installed 1 in 2.496s

  System:
    OS: macOS 10.14.1
    CPU: (4) x64 Intel(R) Core(TM) i5-4278U CPU @ 2.60GHz
  Binaries:
    Node: 8.9.4 - ~/.nvm/versions/node/v8.9.4/bin/node
    Yarn: 1.12.3 - /usr/local/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v8.9.4/bin/npm
  npmPackages:
    jest: ^23.6.0 => 23.6.0

It seems that argv has a default value for prettierPath and it overrides config file's prettierPath.  I could do a PR, unless I am missing something and the default value is intentional.

I refuse to believe this is intentional (if it is I'll argue against it), so PR very much welcome 🙂

Definately not intentional


Confirm license

We believe we should use Apache 2.0, but would be good to double check

This can be closed. Has been agreed as apache 2.0.


Blind consumer on credentials/tcp lost

Hi,
I tried to connect to one of my rabbitmq, it allow only one user with credentials.
I tried to define wrong credentials and the consumer process is still working, I clearly see in rabbitmq logs:
=INFO REPORT==== 26-Apr-2017::09:44:02 ===
accepting AMQP connection <0.1532.0> (<IP_ADDRESS>:37890 -> <IP_ADDRESS>:5672)

=ERROR REPORT==== 26-Apr-2017::09:44:02 ===
Error on AMQP connection <0.1532.0> (<IP_ADDRESS>:37890 -> <IP_ADDRESS>:5672, state: starting):
PLAIN login refused: user 'myusername' - invalid credentials

If I fix credentials but I don't update the vhost:
=INFO REPORT==== 26-Apr-2017::09:50:06 ===
accepting AMQP connection <0.1345.0> (<IP_ADDRESS>:37892 -> <IP_ADDRESS>:5672)

=ERROR REPORT==== 26-Apr-2017::09:50:06 ===
Error on AMQP connection <0.1345.0> (<IP_ADDRESS>:37892 -> <IP_ADDRESS>:5672, user: 'iadvize', state: opening):
access to vhost '/' refused for user 'myusername'

Is the consumer process blind to credentials failing?
Another problem, if I kill my rabbitmq, the consumer is still running.
Let me know if you want more details.

Hi,
This should be fixed in #42

great

I just tried with recently merged PR but it's not fixed.
cc @Keruspe @Geal

Here's a tracelog:
➜  lapintest master ✗ RUST_LOG=lapin=trace cargo run --example connect
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/examples/connect`
TRACE:lapin_futures::transport: sink start send
TRACE:lapin_futures::transport: will encode and write frame: ProtocolHeader
TRACE:lapin_futures::transport: serialized frame: 8 bytes
TRACE:lapin_futures::transport: sink poll_complete
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
TRACE:lapin_futures::transport: pre-poll
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentProtocolHeader)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
TRACE:lapin_futures::transport: post-poll
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentProtocolHeader)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: decoded frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@c831ff2a0c9f"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.6")}, mechanisms: "PLAIN AMQPLAIN", locales: "en_US" })))
TRACE:lapin_futures::transport: upstream poll gave frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@c831ff2a0c9f"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.6")}, mechanisms: "PLAIN AMQPLAIN", locales: "en_US" })))
TRACE:lapin_async::connection: will handle frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@c831ff2a0c9f"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.6")}, mechanisms: "PLAIN AMQPLAIN", locales: "en_US" })))
TRACE:lapin_async::connection: Server sent Connection::Start: Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@c831ff2a0c9f"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.6")}, mechanisms: "PLAIN AMQPLAIN", locales: "en_US" }
DEBUG:lapin_async::connection: client sending Connection::StartOk: Connection(StartOk(StartOk { client_properties: {"product": LongString("lapin")}, mechanism: "PLAIN", response: "\u{0} myusername\u{0}guest", locale: "en_US" }))
TRACE:lapin_futures::transport: sink start send
TRACE:lapin_futures::transport: will encode and write frame: Method(0, Connection(StartOk(StartOk { client_properties: {"product": LongString("lapin")}, mechanism: "PLAIN", response: "\u{0} myusername\u{0}guest", locale: "en_US" })))
TRACE:lapin_futures::transport: serialized frame: 64 bytes
TRACE:lapin_futures::transport: sink poll_complete
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave Ready(None)
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave Ready(None)
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: Sending heartbeat
TRACE:lapin_futures::transport: sink start send
TRACE:lapin_futures::transport: will encode and write frame: Heartbeat(0)
TRACE:lapin_futures::transport: serialized frame: 8 bytes
TRACE:lapin_futures::transport: sink poll_complete
ERROR:lapin_futures::transport: upstream poll gave Ready(None)
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave Ready(None)

and Rabbitmq Log:
=INFO REPORT==== 26-Apr-2017::18:12:45 ===
accepting AMQP connection <0.5507.0> (<IP_ADDRESS>:52018 -> <IP_ADDRESS>:5672)

=ERROR REPORT==== 26-Apr-2017::18:12:45 ===
Error on AMQP connection <0.5507.0> (<IP_ADDRESS>:52018 -> <IP_ADDRESS>:5672, state: starting):
PLAIN login refused: user 'myusername' - invalid credentials

That's weird, with the current master (with the PR merged) I get that when forcing wrong credentials in the client example:
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
DEBUG:tokio_core::reactor: consuming notification queue
DEBUG:tokio_core::reactor: scheduling direction for: 0
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:tokio_core::reactor: loop poll - Duration { secs: 0, nanos: 4325 }
DEBUG:tokio_core::reactor: loop time - Instant { tv_sec: 31267, tv_nsec: 955408760 }
DEBUG:tokio_core::reactor: loop process - 0 events, Duration { secs: 0, nanos: 9205 }
DEBUG:tokio_core::reactor: loop poll - Duration { secs: 3, nanos: 1143610 }
DEBUG:tokio_core::reactor: loop time - Instant { tv_sec: 31270, tv_nsec: 956565064 }
TRACE:tokio_core::reactor: event Ready {Readable | Writable | Error | Hup} Token(2)
DEBUG:tokio_core::reactor: notifying a task handle
DEBUG:tokio_core::reactor: loop process - 1 events, Duration { secs: 0, nanos: 65915 }
DEBUG:tokio_core::reactor: loop poll - Duration { secs: 0, nanos: 10413 }
DEBUG:tokio_core::reactor: loop time - Instant { tv_sec: 31270, tv_nsec: 956646642 }
TRACE:tokio_core::reactor: event Ready {Readable} Token(1)
DEBUG:tokio_core::reactor: loop process - 1 events, Duration { secs: 0, nanos: 16311 }
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave error: Error { repr: Os { code: 104, message: "Connection reset by peer" } }
DEBUG:tokio_core::reactor: consuming notification queue
DEBUG:tokio_core::reactor: dropping I/O source: 0
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 104, message: "Connection reset by peer" } }', src/libcore/result.rs:859
note: Run with `RUST_BACKTRACE=1` for a backtrace.

With that in rabbitmq too
=INFO REPORT==== 26-Apr-2017::20:00:48 ===
accepting AMQP connection <0.532.0> (<IP_ADDRESS>:40646 -> <IP_ADDRESS>:5672)

=ERROR REPORT==== 26-Apr-2017::20:00:48 ===
Error on AMQP connection <0.532.0> (<IP_ADDRESS>:40646 -> <IP_ADDRESS>:5672, state: starting):
PLAIN login refused: user 'guest' - invalid credentials

how I reproduce the bug:
$ docker run -d --hostname my-rabbit  -p 5672:5672 --name some-rabbit rabbitmq:3

Updating examples/client.rs in futures with:
  let opts = ConnectionOptions {
    password:"wrong".into(),
    ..ConnectionOptions::default()
  };

➜  lapin/futures master ✗ RUST_LOG=lapin=trace cargo run --example client
   Compiling lapin-futures v0.8.0 (file:///Users/Simon/work/perso/lapin/futures)
    Finished dev [unoptimized + debuginfo] target(s) in 5.14 secs
     Running `/Users/Simon/work/perso/lapin/target/debug/examples/client`
TRACE:lapin_futures::transport: sink start send
TRACE:lapin_futures::transport: will encode and write frame: ProtocolHeader
TRACE:lapin_futures::transport: serialized frame: 8 bytes
TRACE:lapin_futures::transport: sink poll_complete
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
TRACE:lapin_futures::transport: pre-poll
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentProtocolHeader)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
TRACE:lapin_futures::transport: post-poll
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentProtocolHeader)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: decoded frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@my-rabbit"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.9")}, mechanisms: "AMQPLAIN PLAIN", locales: "en_US" })))
TRACE:lapin_futures::transport: upstream poll gave frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@my-rabbit"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.9")}, mechanisms: "AMQPLAIN PLAIN", locales: "en_US" })))
TRACE:lapin_async::connection: will handle frame: Method(0, Connection(Start(Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@my-rabbit"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.9")}, mechanisms: "AMQPLAIN PLAIN", locales: "en_US" })))
TRACE:lapin_async::connection: Server sent Connection::Start: Start { version_major: 0, version_minor: 9, server_properties: {"capabilities": FieldTable({"authentication_failure_close": Boolean(true), "basic.nack": Boolean(true), "connection.blocked": Boolean(true), "consumer_cancel_notify": Boolean(true), "consumer_priorities": Boolean(true), "direct_reply_to": Boolean(true), "exchange_exchange_bindings": Boolean(true), "per_consumer_qos": Boolean(true), "publisher_confirms": Boolean(true)}), "cluster_name": LongString("rabbit@my-rabbit"), "copyright": LongString("Copyright (C) 2007-2016 Pivotal Software, Inc."), "information": LongString("Licensed under the MPL.  See http://www.rabbitmq.com/"), "platform": LongString("Erlang/OTP"), "product": LongString("RabbitMQ"), "version": LongString("3.6.9")}, mechanisms: "AMQPLAIN PLAIN", locales: "en_US" }
DEBUG:lapin_async::connection: client sending Connection::StartOk: Connection(StartOk(StartOk { client_properties: {"product": LongString("lapin")}, mechanism: "PLAIN", response: "\u{0}guest\u{0}wrong", locale: "en_US" }))
TRACE:lapin_futures::transport: sink start send
TRACE:lapin_futures::transport: will encode and write frame: Method(0, Connection(StartOk(StartOk { client_properties: {"product": LongString("lapin")}, mechanism: "PLAIN", response: "\u{0}guest\u{0}wrong", locale: "en_US" })))
TRACE:lapin_futures::transport: serialized frame: 62 bytes
TRACE:lapin_futures::transport: sink poll_complete
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
TRACE:lapin_futures::transport: upstream poll gave NotReady
DEBUG:lapin_futures::transport: AMQPTransportConnector poll transport is none? false
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave Ready(None)
DEBUG:lapin_futures::transport: conn state: Connecting(SentStartOk)
TRACE:lapin_futures::transport: waiting before poll
TRACE:lapin_futures::transport: stream poll
ERROR:lapin_futures::transport: upstream poll gave Ready(None)


Fix correct fe collector type for jvm_gc

Proposed changes
Issue Number: close #DORIS-9067

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

the problem is indroduced by #27408


Add P2P Transactions

Proposed Changes

Add P2P Transactions capability library implementation (didn't implement CLI for it)
Add json error to return error message received from paymail server (as shown below)

Closes issue #4


routing: fix mission control deadlock

Fixes the following potential deadlock situation:

Pathfinding holds a database lock and tries to obtain a mission control lock via GetProbability
ReportPaymentSuccess/ReportPaymentFail holds a mission control lock and tries to obtain a database lock to store the payment result.

Fixes https://github.com/lightningnetwork/lnd/issues/5104

I assume you mean maxRecords? I missed that because I didn't notice that MissionControlStore is not thread-safe.

Indeed that's it.

Happy to tack something on to this PR if you don't want master to get into an unsafe state. Just let me know how you prefer to have it tackled.

That would be great, I think adding a lock to the store is a good approach? Not used too widely so should be a small change.

Thinking about this a bit more, I am not so sure anymore what the best solution is.
I also noticed MissionControl.ResetHistory which does the same thing: mc lock, then db rw lock. Fixing that in the same way as AddResult introduces the risk of the memory and disk states getting out of sync. Probably not very harmful, but it doesn't feel right either.

IMO a lock-less getProbability that can be called from inside the db transaction might be preferable

I am not sure if this works. If GetProbability is lock-less, it may read internal mission control state while it is being updated via ReportPaymentSuccess?
The root of the problem here is that a bbolt transaction has no scope. Two totally independent parts of the database interfere with each other through those transactions (... no, don't think about sql now ...).
Pathfinding is particularly problematic because it does so many things in that long-living transaction (... dreaming about in-memory graphs again ...)
How about removing the global transaction from  dbRoutingTx and let dbRoutingTx.forEachNodeChannel create a new transaction for every call? Then GetProbability won't be called within a db context anymore, solving the deadlock. Also writes to the database don't need to wait anymore for an ongoing pathfinding operation to fully complete (which can take a while on slow machines).
I don't think we need to worry about consistency of the graph. It is updated in pieces anyway when gossip data comes in.

@cfromknecht @carlaKC any thoughts on the comment above?

How about removing the global transaction from dbRoutingTx and let dbRoutingTx.forEachNodeChannel create a new transaction for every call? Then GetProbability won't be called within a db context anymore, solving the deadlock.

This sounds like a good idea to me, just wondering about the tradeoff of a new tx for each channel? IIUC this would scale pathfinding up from a single db tx for all our graph lookups to one per channel we consider?

I don't think we need to worry about consistency of the graph. It is updated in pieces anyway when gossip data comes in.

Agreed that this shouldn't be much of an issue.

(... no, don't think about sql now ...)

😅

I also noticed MissionControl.ResetHistory which does the same thing: mc lock, then db rw lock. Fixing that in the same way as AddResult introduces the risk of the memory and disk states getting out of sync. Probably not very harmful, but it doesn't feel right either.

If we decide against removing the long-lived db tx, I think that this isn't the worst intermediate fix. But it doesn't really feel right either.

IIUC this would scale pathfinding up from a single db tx for all our graph lookups to one per channel we consider?

It would be one transaction per node. We fetch all channels of a node in one operation.

@cfromknecht what's your thought on this? I can update the PR, but would like to agree on the approach first.

To move this forward, I think it's worth making the store thread-safe and moving the mutex for ResetHistory as well to prevent the chances of deadlock there. I think that we can live with the possibility of brief inconsistency when we reset our state in the short term.
The long-lived tx was added due to performance issues in pathfinding, so removing it would likely be a step backwards. When we have in-memory pathfinding, we can come back to this and move the mutexes back?

The long-lived tx was added due to performance issues in pathfinding, so removing it would likely be a step backwards. When we have in-memory pathfinding, we can come back to this and move the mutexes back?

Are you saying that not having that long-lived read transaction makes pathfinding slower?

Are you saying that not having that long-lived read transaction makes pathfinding slower?

As I understand it, yes. It's before my time, but chatting to @Roasbeef that seems to be the issue with removing it.

I just ran a test on the testnet graph and saw absolutely no difference.
Pathfinding perf metrics: nodes=1097, edges=3886, time=108.588715ms
https://github.com/joostjager/lnd/tree/test

@Roasbeef can you comment on my finding here?

@joostjager did you test the worst case when all edges need to be explored? I'm not sure testnet is the best benchmark given that mainnet is now over 10x its size in terms of edge count. Also an actual test would need to evaluate the impact on the system as a whole during normal operations. For example needing possibly 10k DB transactions for a path finding attempt vs a single one means that it's possible for the path finding attempt to continually sit blocked in the face of interleaved write transactions.
We consolidated database transactions back when we did k-shortest paths, and it was routine to have to explore the entire graph depending on the set of edges passed in to ignore. I agree with you that we don't have to worry about any of this once we finally implement in-memory path finding. What we have here seems to resolve a critical issue (the deadlock) so I don't see why we wouldn't move forward with it. The possible edge case above re mission control seems minor, as its correctness isn't critical to operation.

The test case that I used explored 1097 nodes and 3886 edges in separate transactions per node. I saw no difference between this and using a single transaction for all queries. It seems unlikely to me that running it on mainnet gives different result. I just want to understand where the mentioned slow down originates from. The theory that it would be different under heavy write load makes sense, but I am curious to know how you found out back then. Was there an e2e load test?
But yes, of course it is easier to just patch resetmc and move on, then revisit when the time for in-memory pathfinding has come.

I saw no difference between this and using a single transaction for all queries. It seems unlikely to me that running it on mainnet gives different result.

I guess you can re-run your test that if you'd like, it would be surprising if an order of magnitude increase in the dataset size had a negligible impact on the runtime. Also as I mentioned above, it would need to be benchmarked in a setting that also had other concurrent access, as only then would one see the read transaction be blocked on other write transactions. As given the way mutexes work in Go, work if a read mutex is held, then a write mutex is attempted to be held, and another read mutex attempt after that, the last reader will block until the writer has finished (no reentrancy).
Therefore it follows that creating an extra read transactions can slow down a routine. An isolated experiment would just be reading a large set of keys in a single database transaction vs creating a new database transaction to read each key. It can also lead to more contention given it's possible for path finding to be blocked all together due to a long running write transaction as I mentioned above.
Re the history, back then there wasn't a benchmarking harness so the decision was based off of the long running time of the routine in practice as well as within unit tests (back when QueryRoutes would take multiple seconds to return, or just even timeout). You're welcome to spelunk back in the commit history to find this point in the repo's history if you're feeling archaeological.
I think it makes sense to move forward with this -3/+9 patch as it solves an immediate issue, if we get a benchmark that simulates real-world concurrent use and shows no degradation (with actual path finding), then we can consider removing the attempt level transaction. However, I'd wager efforts are better spent on moving to in-memory path finding.


An error occurred when transferring XCM USDT (Parallel --> Statemint)

Describe the task
An error occurred when transferring XCM USDT (Parallel --> Statemint)

Error: createType(Call):: Call: failed decoding xTokens.transfer:: Struct: failed on args: {"currency_id":"u32","amount":"u128","dest":"{"_enum":{"__Unused0":"Null","V2":"XcmV2MultiLocation","__Unused2":"Null","V3":"XcmV3MultiLocation"}}","dest_weight_limit":"{"_enum":{"Unlimited":"Null","Limited":"SpWeightsWeightV2Weight"}}"}:: Struct: failed on dest: {"_enum":{"__Unused0":"Null","V2":"XcmV2MultiLocation","__Unused2":"Null","V3":"XcmV3MultiLocation"}}:: Cannot map Enum JSON, unable to find 'V1' in __unused0, v2, __unused2, v3
at Le (461.js:2:376064)
at wa.createTypeUnsafe (461.js:2:426568)
at o (461.js:2:600296)
at Object.r [as transfer] (7941.js:2:581710)
at background.js:1:104447
at W (background.js:1:104869)
at async we.makeCrossChainTransfer (background.js:1:130599)
at async we.handle (background.js:1:172664)
(anonymous) @ background.js:1

Version test: http://<IP_ADDRESS>:8900/s/CJGck8bbq4WSmnc
Validation passed.


Vscode extension

Describe the feature in detail (code, mocks, or screenshots encouraged)
possibly it will be a good idea to have a vscode extension that will have auto complete and stuff,
What type of pull request would this be?
None
Provide relevant links or additional information.
No response

Not planned at this time.


debug: Replaced phantomJS by chrome

Added the script in package.json for running Test-suite
Signed-off-by: ossdev07<EMAIL_ADDRESS>

Coverage increased (+0.9%) to 84.0% when pulling cf6fecd9f302a25280fe1f365776e6da88be5404 on ossdev07:debug_chrome into 4490cd95bfb952e1ed756914ac225ddc987b2ba3 on visionmedia:master.

@ossdev07 just curious - is PhantomJS deprecated or something? Is there a functional benefit by using headless Chrome?

Good point. Sold!


Update Gradle Enterprise plugin version to 3.7.2

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


Rockets and My profile styling

Hello fellow coding parter, I did the following tasks:
I Added Style for Rockets page.
I Added Style for My Profile Page.

Good job so far👏
STATUS: APPROVED 💪👏
Congratulations!
Your project is complete! There is nothing else to say other than... it's time to merge it :ship:
Highlights

Good PR title ✅
Descriptive PR description ✅
Implement the features ✅
Professional README template ✅
Linters are green ✅

suggestions
Cheers and Happy coding!👏👏👏


Clippy false positive on Deserializer lifetimes

The false positive in https://github.com/Manishearth/rust-clippy/issues/292 is going to hit users really hard in the next release.
fn f<'de, D: Deserializer<'de>>(deserializer: D) -> Result<F, D::Error> {
    unimplemented!()
}

warning: explicit lifetimes given in parameter types where they could be elided
 --> src/main.rs:7:1
  |
7 |   fn f<'de, D: Deserializer<'de>>(deserializer: D) -> Result<F, D::Error> {
  |  _^ starting here...
8 | |     unimplemented!()
9 | | }
  | |_^ ...ending here
  |
  = note: #[warn(needless_lifetimes)] on by default

@oli-obk any sense of how difficult this would be to fix?

No, but I'll make it a priority

Should be fixed now in clippy 0.0.123.

Thank you so much!


alternative to filter

@iproctor

definitely more confusing in the code, but the consumers life is more consistent as you mentioned

yup


Training set too large for GPU mem, how to train in batches?

A dataset that I'm trying to fit my model on results in a too large memory allocation for the GPU, so I'm trying to resolve this by fitting on batches of the dataset.
What is the difference between using
model.fit(self, batch_X, batch_y, batch_size=128, nb_epoch=1, verbose=1, callbacks=[], validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None)
and
model.train_on_batch(self, batch_X, batch_y, class_weight=None, sample_weight=None)
for training a single step? The latter lacks a lot of features such as callbacks and validation split. So am I missing something, or is the former always better to use?
Can I just use model.fit with a section of the training set as if I were training normally, and just chain this on the remaining training set sections?

If i understand correctly, train_on_batch will train with all the data you provided in one batch.
Instead, try with a smaller batch size like :
model.fit(self, batch_X, batch_y, batch_size=2, nb_epoch=1, verbose=1, callbacks=[], validation_split=0.0, validation_data=None, shuffle=True, class_weight=None, sample_weight=None)
batch_size=2 will be very slow but if it work, you will have to increase to find a balance between speed and memory usage.


llvmPackages_13.libcxx: fix darwin build

Motivation for this change
This succeeds #146517. It fixes the Darwin build of libcxx by applying the following fixes:

Use a newer version of Clang to avoid compilation errors
Break the circular dependency between libcxx and libcxxabi. libcxxabi needs libcxx headers to build, but libcxx has a runtime dependency on libcxxabi. To fix this, this PR creates a header-only variant of the libcxx derivation for building libcxxabi.

The Linux build seems to have avoided the circular dependency problem by dropping the libcxx dependency on libcxxabi. This approach appears not to be possible on Darwin.
llvmPackages_git probably needs the same fix too, but I haven't looked into it closely yet.
ZHF: #144627
Things done
Like in #146517, I've tested on the master branched and rebased against staging to avoid the overwhelming amount of local builds.

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[X] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

I'm marking this as draft because I found out that building llvmPackages_13 with llvmPackages_12 fixes libcxx, but breaks llvm.
building
'/nix/store/ba45qlpblgyabh8kl00d14fxzb8b1307-llvm-13.0.0/lib' -> '/private/tmp/nix-build-llvm-13.0.0.drv-1/source/llvm/build/lib'
build flags: -j4 -l4 SHELL=/nix/store/r42x7q316gznkm5y9b0cl4564g174zyl-bash-5.1-p8/bin/bash
[  0%] Generating VCSRevision.h
[  0%] Creating export file for LLVMHello
[  0%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/AsmMatcherEmitter.cpp.o
[  1%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/Demangle.cpp.o
CMake Warning at /tmp/nix-build-llvm-13.0.0.drv-1/source/llvm/cmake/modules/VersionFromVCS.cmake:49 (message):
  Git not found.  Version cannot be determined.
Call Stack (most recent call first):
  /tmp/nix-build-llvm-13.0.0.drv-1/source/llvm/cmake/modules/GenerateVersionFromVCS.cmake:23 (get_source_info)
  /tmp/nix-build-llvm-13.0.0.drv-1/source/llvm/cmake/modules/GenerateVersionFromVCS.cmake:45 (append_info)

[  1%] Built target LLVMHello_exports
[  1%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/ItaniumDemangle.cpp.o
[  1%] Built target llvm_vcsrevision_h
[  1%] Building CXX object lib/Demangle/CMakeFiles/LLVMDemangle.dir/MicrosoftDemangle.cpp.o
[  1%] Building CXX object utils/PerfectShuffle/CMakeFiles/llvm-PerfectShuffle.dir/PerfectShuffle.cpp.o
[  2%] Linking CXX executable ../../bin/llvm-PerfectShuffle
[  2%] Building C object utils/count/CMakeFiles/count.dir/count.c.o
Undefined symbols for architecture x86_64:
  "operator new(unsigned long)", referenced from:
      std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > std::__1::__pad_and_output<char, std::__1::char_traits<char> >(std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >, char const*, char const*, char const*, std::__1::ios_base&, char) in PerfectShuffle.cpp.o
      Operator::Operator(unsigned short, char const*, unsigned int, unsigned int) in PerfectShuffle.cpp.o
  "operator delete(void*)", referenced from:
      std::__1::vector<Operator*, std::__1::allocator<Operator*> >::~vector() in PerfectShuffle.cpp.o
      std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > std::__1::__pad_and_output<char, std::__1::char_traits<char> >(std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >, char const*, char const*, char const*, std::__1::ios_base&, char) in PerfectShuffle.cpp.o
      Operator::Operator(unsigned short, char const*, unsigned int, unsigned int) in PerfectShuffle.cpp.o
ld: symbol(s) not found for architecture x86_64
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [utils/PerfectShuffle/CMakeFiles/llvm-PerfectShuffle.dir/build.make:97: bin/llvm-PerfectShuffle] Error 1
make[1]: *** [CMakeFiles/Makefile2:25003: utils/PerfectShuffle/CMakeFiles/llvm-PerfectShuffle.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

This is now ready for review. LLVM, Clang, and libcxx all built successfully after I replaced llvmPackages_12.stdenv with llvmPackages_11.stdenv.

@toonn


Provide Compliance Data in JSON

Feature Benefit
Benefit: Users can download each of the following groupings for analysis in CSV format (Travis 9/28)
Allowance-Based Compliance (For each trading program)
NOx Averaging Compliance (ARP)
*need to determine if these groupings are appropriate
Context
(User journeys, design mockups, architectural diagrams, business rules, data element definitions)

Invalid, duplicate of #322


Possible error using Filter and pageOptions

Hi,
The filter functionality works well. But it seems to be a bug when I want to use pagination together with my filter results.
When going to page 2, I also get the results from page 1. Is this the expected behaviour?
EXAMPLE:

client.delivery.content.filter(body, {page:1, pageSize:1})
// SUCCESS, items array has 1 item

client.delivery.content.filter(body, {page:2, pageSize:1})
// FAILS, items array contains 2 items
// I also get the results from 'page 1'

client.delivery.content.filter(body, {page:3, pageSize:1})
// FAILS, items array contains 3 items
// I also get the results from 'page 1' & 'page 2'

Hi @robinisaksson thanks for reporting. We have confirmed that we an issue with the paging of these results. So we are looking at making a fix for this.
I will update this thread as the fix gets rolled out.

Oh, and just a note that the issue lies within Umbraco and not the client library.
Just in case future me is in doubt :smile:


Query tabs have unexpected behavior

Proposal:
In cloud 2.0 UI, if you create two queries in two tabs, I would have expected the behavior to be that if I clicked 'submit' while the first tab is active, the first query would be executed and its results shown in the data explorer, and if I clicked 'submit' while the second tab is active, the second query would be executed and its results shown in the data explorer.
Current behavior:
Currently, the results of all queries are displayed simultaneously in the data explorer.
Desired behavior:
Queries can be run independently.
Alternatives considered:
Remove the tabs if we can only support running all queries in parallel, or change the UI in some way to make it clearer that all the queries are executing (I don't have any good ideas for what this UI might look like).
Use case:
Why is this important (helps with prioritizing requests)?  User confusion, as it's not consistent with other tabbed experiences.

I like the query tabs concept. It facilitates per query display configuration (such as multiple axes) and separating queries whose results need to be overlaid but otherwise are very different in implementation or datasources. A global namespace shared would be one way to reuse query snippets among tabs.  As a dashboard user, I want to quickly combine query snippets that I know work and click a handful of buttons to get a viz that's 90% of what I want in a few handful of minutes. But, yeah, well, this is just like my opinion.

Related: https://github.com/influxdata/influxdb/issues/19058


Include all_attributes when starting syncs.

This comes in handly when needing a little more context information without having to make a separate request to the API.

:+1:


Fix async for julekonsert del 2

https://mediabanken.brunstad.tv/item/VX-473940/?index=41&search_id=176852&parentPage=search
Both embedded and reaper audio are slightly asynced. This is taken from channel in youplay.

All languages should now be correct to a reasonable degree.

Is it published? Where could I find the updated audio tracks?

They are updated in MB
On December 15, 2023 5:16:34 AM GMT+01:00, Priscilla Mulunda @.***> wrote:

Is it published? Where could I find the updated audio tracks?
--
Reply to this email directly or view it on GitHub:
https://github.com/bcc-code/bcc-media-platform/issues/727#issuecomment-1857249860
You are receiving this because you were assigned.
Message ID: @.***>

The files were replaced directly.


ViewTest regular expression is wrong

What steps will reproduce the problem?

ViewTest regular expression is right
What do you get instead?
Additional info

Q
A

Yii version
2.0.35

PHP version

Operating system

I made a mistake, please ignore


Shard state/data local storage

I think we want to use sparse merkle trees
https://ethresear.ch/t/data-availability-proof-friendly-state-tree-transitions/1453/6
https://github.com/ethereum/research/blob/master/trie_research/bintrie2/new_bintrie.py

https://gitter.im/ethereum/sharding?at=5ad49182102fac7b7b19334c

@jamesray1 so the db should store collation roots (defined in src/collation/header.rs and src/collation/collation.rs) for just up to 100 separate shard collation roots?

And implemented with a sparse merkle tree data structure, yea?

Sorry @tcsiwula, I only just saw this, I tend to see Gitter comments faster.
We may use one shard p2p network as per #34.
But yes, it should store collation bodies and headers, but the current R&D direction is to use a sparse merkle binary tree to store this data.


(PR created) Allow attachments to be bound to exception more easily

Why I need this (very much!)
My code is like:
void main_function() {
  try {
    inner_function(); // indeed, functions calling functions calling functions... so the calling stack is deep
  } catch (e,s) {
    Sentry.captureException(e, s);
  }
}

void inner_function() {
  var my_data = ...;
  try {
    do_some_work_that_can_throw_exception(my_data);
  } catch (e,s) {
     Sentry.pleaseBindAttachmentToException(e, my_data); // <-- !!
     rethrow;
  }
}

Notice the Sentry.pleaseBindAttachmentToException, where I want to bind attachment to exception.
However, as we know, we cannot do it. I can create some code to bind extra to exception, because extra can be modified in EventProcessors (so I can manually create a static map holding some extras and bind to exception in my own EventProcessor). However, EventProcessors cannot modify or add attachments. So I have no way to do it :(
Proposed solution
Say, add a field to SentryOptions, e.g. void Function(Scope)? withScopeWhenCapture. Then, in Hub.captureException and similar places, call that callback. This is very simple and I will implement it if you like.
related: https://github.com/getsentry/sentry-dart/issues/646
Hi @marandaneto could you please have a look (since this thread is strongly related to #646)? Thanks! And @ueman could you please also have a look? Thanks!

Hi @marandaneto @ueman could you please suggest whether this is acceptable? If so I will make a PR. This is a blocking issue on our app...

If I'm not mistaken you can already do something like this:
Sentry.captureException(exception: ex, withScope:(scope) {
    scope.addAttachment(attachment);
});

I'm not sure if that already works, but you might try that. If not @marandaneto probably appreciates a PR. I'm not a maintainer of this SDK anymore.
Currently is also Christmas season, so most people are out of office.

@ueman No I cannot do that...Notice that I captureException in a outer function, while I addAttachment in a inner function. For example, outer_function() calls func1() which calls func2() which calls func3(), ..., which calls func100() which calls inner_function().

Oh I see. In that case the solution would probably be similar to .NETs local scopes. Dart hasn't gotten support for that yet but that would probably the solution for that.

@ueman Well that is a bit the inverse. In my case I want to bind attachment in inner function and capture exception in outer. With the proposed modification, that can be implemented trivially.

Yeah, with mine, too. The code would be something like this:
void main_function() {
Sentry.withScope((scope) {
// within this scope, the attachment is added to the capture exception
  try {
    inner_function(); // indeed, functions calling functions calling functions... so the calling stack is deep
  } catch (e,s) {
    Sentry.captureException(e, s);
  }
}
// here the attachment isn't added to the exception anymore
Sentry.captureException(ex); // this exception has no attachment
}

void inner_function() {
  var my_data = ...;
  try {
    do_some_work_that_can_throw_exception(my_data);
  } catch (e,s) {
     Sentry.addAttachment(my_data);
     rethrow;
  }
}

Agree. Since that is a bit complex (read: can introduce bugs), I will firstly implement a simpler solution

PR made: https://github.com/getsentry/sentry-dart/pull/683

I'm on vacay, maybe @bruno-garcia can help

@marandaneto No problem, I can wait for it

I'll discuss that internally and get back to you, but I agree with @ueman , the solution could be using local scopes, it'd be confusing to have a beforeSend callback without Scope, and a beforeCaptureWithScope, our public APIs are aligned and documented https://develop.sentry.dev/sdk/unified-api/ so we'd need a bit of discussion before going forward.

@marandaneto Sure. I have used it in my app in production and my implementation seems quite good.

@bruno-garcia Sounds interesting, thanks

I'll discuss that internally and get back to you

Hi, is there any updates?

Thanks for the update! As long as the goal mentioned in the first post of the issue is solved it, any method is great

@marandaneto Hi any updates?

You could solve this today by throwing a new exception that holds the reference to your attachment and the inner exception.
pseudocode:
void main_function() {
  try {
    inner_function(); // indeed, functions calling functions calling functions... so the calling stack is deep
  } catch (e,s) {
    if (e is AttachmentException) {
      attachEx - (AttachmentException)e;
      Sentry.captureException(attachEx.Exception, attachEx.Stacktrace, scope: (s) => s.addAttachment(attachEx.Attachment));
    }
  }
}

void inner_function() {
  var my_data = ...;
  try {
    do_some_work_that_can_throw_exception(my_data);
  } catch (e,s) {
     throw new AttachmentException(e,s,my_data);

Btw this has a big drawback: the exception type is changed. e.g. if outer function only knows to catch MyException, it will not look at AttachmentException at all.

@fzyzcjy yes, we are experimenting with a new API for hints on the Java repo and this will likely be the solution here as well if it works well. https://github.com/getsentry/sentry-java/pull/2045
Via hints you'll be able to add and remove attachments for specific events.

Looking forward to it!

@marandaneto Hi is there any updates? Thanks

Yeah the approach taken on sentry-java worked well, now it's a matter of doing it on Dart/Flutter, not planning on this Quarter tho.

#1136 is going to unblock this via hints

It's now possible via hints.
We should introduce a new property in the Hint class that takes attachments, so you can add/remove them on the fly either when capturing events or thru beforeSend in the hints param.


More flexible pinning of Jupyter optional dependencies

See https://github.com/kedro-org/kedro/issues/2276 and https://github.com/kedro-org/kedro/pull/3288/
Motivation and Context

How has this been tested?

Checklist

[ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
[ ] Assigned myself to the PR
[ ] Added tests to cover my changes

pip doing some heavy backtracking on kedro-datasets:
WARNING: kedro-datasets 1.8.0 does not provide the extra 'hdfs-base'
WARNING: kedro-datasets 1.8.0 does not provide the extra 's3fs-base'
WARNING: kedro-datasets 1.8.0 does not provide the extra 'spark-base'

revert a few commits, changed made in a separate PR #201.

The issue was fixed separate in #201


Replaced version(unittest) blocks with version(StdUnittest)

Should result in faster compiles for unittest builds in some cases.

Fixed


Fix relative symlinks when using --follow-symlinks

fixes #836
Changes:

Add a relative symlink to FileIteratorTests

This causes test failures if the following fixes are not applied

Fix relative symlinks when using --follow-symlinks
Eliminate duplicated files with standardizedFileURL when comparing them in FileIterator.next()

It seemed that testFollowSymlinks failed in macOS. I'll investigate it.
https://ci.swift.org/job/swift-syntax-PR-macOS/4601/console
Test Case '-[SwiftFormatTests.FileIteratorTests testFollowSymlinks]' started.
/Users/ec2-user/jenkins/workspace/swift-syntax-PR-macOS/branch-main/swift-format/Tests/SwiftFormatTests/Utilities/FileIteratorTests.swift:37: error: -[SwiftFormatTests.FileIteratorTests testFollowSymlinks] : XCTAssertEqual failed: ("4") is not equal to ("3")
Test Case '-[SwiftFormatTests.FileIteratorTests testFollowSymlinks]' failed (0.699 seconds).

The above error did not occur on macOS Sequoia. So I will prepare macOS Ventura. However, it will take time.

I have successfully reproduced the issue on macOS 13.6.9 on EC2.
When the test failed, the local variable seen looked like this:
[
    "/private/var/folders/5r/59x37fyj13l2hjhq53g287_80000gn/T/TemporaryItems/NSIRD_xctest_Jjw9up/project/real2.swift",
    "/private/var/folders/5r/59x37fyj13l2hjhq53g287_80000gn/T/TemporaryItems/NSIRD_xctest_Jjw9up/project/.hidden.swift",
    "/var/folders/5r/59x37fyj13l2hjhq53g287_80000gn/T/TemporaryItems/NSIRD_xctest_Jjw9up/project/.hidden.swift",
    "/private/var/folders/5r/59x37fyj13l2hjhq53g287_80000gn/T/TemporaryItems/NSIRD_xctest_Jjw9up/project/real1.swift"
]

So it seems that this test failure occurs if a relative symbolic link is linked to a file that its path contains symbolic links.

Fixed: ef73806f30f24736156221665543e73bff8afff8


Problem with training transformer translation model with quant-noise-pq

What is your question?
I'm trying to train translation model IWSLT'14 German to English (Transformer) from examples,
https://github.com/pytorch/fairseq/tree/master/examples/translation
with quant-noise-pq.
But it seems not to train at all.
During training loss drops from 9.9 to 9.7
and every hypothesis that model reurns looks like that:

example hypothesis: ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
or
example hypothesis: so and and and and and and and and and and and and and and and and and and and and and and and and and and and and and and and and and and

I run this training like that:
CUDA_VISIBLE_DEVICES=0 fairseq-train \
    data-bin/iwslt14.tokenized.de-en \
    --arch transformer_iwslt_de_en --share-decoder-input-output-embed \
    --optimizer adam --adam-betas '(0.9, 0.98)' --clip-norm 0.0 \
    --lr 5e-4 --lr-scheduler inverse_sqrt --warmup-updates 4000 \
    --dropout 0.3 --weight-decay 0.0001 \
    --criterion label_smoothed_cross_entropy --label-smoothing 0.1 \
    --max-tokens 4096 \
    --eval-bleu \
    --eval-bleu-args '{"beam": 5, "max_len_a": 1.2, "max_len_b": 10}' \
    --eval-bleu-detok moses \
    --eval-bleu-remove-bpe \
    --eval-bleu-print-samples \
    --best-checkpoint-metric bleu --maximize-best-checkpoint-metric \
    --quant-noise-pq 0.1 --quant-noise-pq-block-size 8

What have you tried?
I've tried --quant-noise-pq-block-size 8, 4, 2, 1
--quant-noise-pq 0.01
and still the same problem
What's your environment?

fairseq Version (0.9.0 master from 25 May):
PyTorch Version  1.5.0
OS (e.g., Linux): Ubuntu 16.04.4 LTS
How you installed fairseq:

git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable ./

Python version: 3.6.9
CUDA/cuDNN version: Cuda compilation tools, release 8.0, V8.0.61
GPU models and configuration: GeForce GTX 1080

Thanks for raising this issue, we are looking into it

I am facing a related problem. Unlike @jahutwb however I was able to train a working model with okaish BLEU while using the noise. My dataset is different with a size of 1.7M sentences but I kept the parameters same for the transformer. I used a lower --quant-noise-pq 0.05 and the model seems to work. I have still not managed to quantize it after training with the noise and facing issue with the load_state_dict. I would try what @jahutwb's hack and see if I can make progress.

To quantize trained model i run additional training with an argument
--quantization-config-path examples/quant_noise/transformer_quantization_config.yaml
I had to modify this config file a little bit.
I've changed regular expressions in layers_to_quantize, because it has problems with finding those layers.
I've replaced "\\" with "\",
changed "decoder\.embed_tokens\.embeddings\.[012]\.[01]" to "decoder.embed_tokens" and add similar expressions to quantize encoder layers.
It's important to add --max_epoch argument during this quantization training. Max epoch has to be a multiple of layers_to_quantize.

Traceback (most recent call last):
File "/home/tarun/anaconda3/bin/fairseq-train", line 11, in
load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
File "/home/tarun/fairseq/fairseq_cli/train.py", line 370, in cli_main
nprocs=args.distributed_world_size,
File "/home/tarun/anaconda3/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 171, in spawn
while not spawn_context.join():
File "/home/tarun/anaconda3/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 118, in join
raise Exception(msg)
Exception:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/home/tarun/fairseq/fairseq/trainer.py", line 238, in load_checkpoint
state["model"], strict=True, args=self.args
File "/home/tarun/fairseq/fairseq/models/fairseq_model.py", line 93, in load_state_dict
return super().load_state_dict(new_state_dict, strict)
File "/home/tarun/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 830, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for TransformerModel:
Unexpected key(s) in state_dict: "encoder.quant_noise.weight", "decoder.quant_noise.weight".
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tarun/anaconda3/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
fn(i, *args)
File "/home/tarun/fairseq/fairseq_cli/train.py", line 338, in distributed_main
main(args, init_distributed=True)
File "/home/tarun/fairseq/fairseq_cli/train.py", line 104, in main
extra_state, epoch_itr = checkpoint_utils.load_checkpoint(args, trainer)
File "/home/tarun/fairseq/fairseq/checkpoint_utils.py", line 137, in load_checkpoint
reset_meters=args.reset_meters,
File "/home/tarun/fairseq/fairseq/trainer.py", line 249, in load_checkpoint
"please ensure that the architectures match.".format(filename)
Exception: Cannot load model parameters from checkpoint checkpoints/base-quant-model/checkpoint_best.pt; please ensure that the architectures match.

I have tried adding the original yaml file and it doesn't seem to work. Any hints or hacks? @jahutwb

I didn't have exactly same error, because I did'n manage to successfully run training with quant-noise, but I see what is the problem.
It seems that it changes model sate dict keys during training with quant noise.
I had similar problem because running training with quantization-config also changes these keys.
During lading checkpoint it previously built new model task.build_model(args) and then it loads state_dict, but that freshly built model have different keys then trained with quant noise (in your case) or quantized (in my).
I handled this, as I described above, by quantize that freshly build model and then load my quantized state dict.
I suppose you could try something similar, and do with this freshly build model what train does during training with quant noise.
But I'm not an expert and I don't know if my advice is good.
Could you tell me how you run your trainning with quant noise?
What model do you train?
Does your model output sensible hypotheses?

Thanks for your response. I am going to try but since I am not an expert either I might require a bit more handholding. For training with quant noise I used 0.05 for noise and 8 for block size. I trained on my own dataset. All of my other parameters are the same as yours including the architecture. My dataset is 1.7M sentence pairs and I trained for around 30 epochs but I got the model outputting sensible hypothesis around the 10th epoch itself. My training command other than the data folder is the same as yours.

I handled this, as I described above, by quantize that freshly build model and then load my quantized state dict.

I am not exactly clear by what you mean by this. Do you write your own training loop for doing the quantization or you make some changes to the trainer.py or checkpoint_utils.py. Would be nice if you can share the snippet as a gist.
Thanks!

@reachtarunhere are you able to share any details or training procedure for training transformer without loss in metrics?  My attempts seem to work but the results are much weaker than the original model.

@jahutwb Excuse me, where did you add this code?  Thanks.

Thanks for your response. I am going to try but since I am not an expert either I might require a bit more handholding. For training with quant noise I used 0.05 for noise and 8 for block size. I trained on my own dataset. All of my other parameters are the same as yours including the architecture. My dataset is 1.7M sentence pairs and I trained for around 30 epochs but I got the model outputting sensible hypothesis around the 10th epoch itself. My training command other than the data folder is the same as yours.

I handled this, as I described above, by quantize that freshly build model and then load my quantized state dict.

I am not exactly clear by what you mean by this. Do you write your own training loop for doing the quantization or you make some changes to the trainer.py or checkpoint_utils.py. Would be nice if you can share the snippet as a gist.
My email is on my profile if it would help we can try debugging the issues together over Zoom?
Thanks!
Update: I have managed to train with quant noise as well as quantize that model without severe loss in metrics. Will share the results and how I managed to do it shortly.

I also tried, but the results of the quant noise model were 4 or 5 points worse than those of the unused model, and the quantized model lost 2 points.  Can you share your experience? Thanks.


WebUSB sample not recognized correctly in Windows 10

Describe the bug
Automatic driver installation does not work for the WebUSB sample. It just gets assigned the null GUID and is shown with a warning triangle. Zadig also shows that the device did not get the correct WICD. On Linux, the WebUSB stuff seems to work fine, so I presume this is mostly related to the MS OS 2.0 descriptors / BOS entry. The MS OS 1.0 stuff seems to be irrelevant for anything newer than Windows 7.
To Reproduce

Build the WebUSB sample with an nRF52 or nRF53 DK. (I don't think this is hardware-related.)
Plug into a Windows 10 machine or VM

Expected behavior
The WinUSB driver should be installed automatically, the device should not be should with a warning symbol in the device manager. The GUID should be reported correctly.
Impact
This is a showstopper for me since it prevents me from implementing my current project: A Zephyr implementation of CMSIS-DAP v2. The automatic driver installation part is mandatory to conform to the spec.
Environment (please complete the following information):

Windows 10
Zephyr 3.3.0

This is a showstopper for me since it prevents me from implementing my current project: A Zephyr implementation of CMSIS-DAP v2. The automatic driver installation part is mandatory to conform to the spec.

Have you seen this? https://github.com/zephyrproject-rtos/zephyr/pull/53798

Yes, I'm trying to finish that one for Johann. :)

The problem is definitely in MS OS 2.0 descriptor. One of the issue is that
https://github.com/zephyrproject-rtos/zephyr/blob/c6c93e394d6a52c4541d39135681fc81b8df7dd5/samples/subsys/usb/webusb/src/main.c#L40
hardcodes the first interface index to 0x02 while the sample when built for nrf52840dk_nrf52840 has only one interface with number 0.
Changing the index itself doesn't seem to be enough though.


Suggestion: Wariness Threshold for Nature's Bounty (AutoGig)

For the "Use Nature's Bounty" when doing AutoGig: to include an optional threshold to stop using it after a certain % of the wariness bar is filled, like 50%, 75% full, etc. Sometimes a fish will come late enough to use up the buff, but not leave enough time to have another reasonable use.

ill try and and see what i can do about that, AutoGig is a very experimental feature and tbh im surprised the option works at all LOL

Nature's Bounty will now only be used when the target fish is detected on the screen, its not a perfect solution but it works most of the time.