Cognito: unable to login with email if user attempts to update email address

Do you want to request a feature or report a bug?
reporting a bug
What is the current behavior?
Our cognito pools are set up so that users can log in with their email. We also utilize attribute verification so that once a user signs up, they will need to verify their email address. Where we get stuck is when a user needs to update their email address. We call updateAttributes and pass their email, which will then send the user a verification email with the confirmation code. There are however states where the user is unauthenticated and needs to verify email with the confirmation code. If they are in an unauthenticated state, they need to authenticate. However, cognito doesn't allow users to log back in with neither their old or new email address. Instead, they need to sign up for an account again. This is a huge issue for us since it leaves users in this limbo state.
What is the expected behavior?
The expected behavior would be for users to log in with the email address that has been verified in prior to attempting to change the email, which would then allow the user to verify the new email address.
Which versions of Amplify, and which browser / OS are affected by this issue? Did this work in previous versions?
This has never worked

I entered the same issue against the old amazon-cognito-identity-js SDK:
https://github.com/amazon-archives/amazon-cognito-identity-js/issues/515
Even though it's still open (and starred by 904 people) I think it got forgotten when the old SDK was retired. I can see how this happened since most of the issues were probably related to the old SDK, not the Cognito back-end. But it's really a Cognito back-end issue. Since there isn't a GitHub project for Cognito itself, the front-end is the only place to enter issues.

@clintfoster thanks for confirming I'm not crazy! Was really losing my sanity over this.
@yuntuowang what is the best way to communicate this to the backend team? I understand the security reasonings behind this, but definitely feels like a flaw in the overall design. I know it may take a while to get this fixed, but do you think it's worth documenting here for now: https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-attributes.html?

@danielgeri I'm having the same issue with the exception of one of your problems. I can change a user's email address, log out, and can now only log in with the new unverified email. The user is now unable to log in with the old email.
So where this personally has problems is say a user mistakenly enters the wrong email during the change email process, they are now locked out of their account if they sign out and don't remember their mis-entered email. Given that Cognito has no baked in account recovery or forgot username tools, the user is forced to contact a system admin to fix the issue.
It seems to me that Cognito shouldn't be updating the user's email/phone until after the verification process has happened. I would seem to solve both of our issues, provide a more secure system, and not be all that confusing for a user to figure out.

@sloansparger sorry to hear :( Your scenario seems super odd since that is a major security flaw. Regardless of the mechanics of the issue I would recommend the following:

handle email verification outside of cognito
use preferred username, etc as the email address that you can update and login with when email is verified (untested fyi - just a theory)
switch to an auth provider that is more proven? e.g. auth0

@danielgeri Interesting idea, I'm going to look into option 2 until we hear some feedback from the Cognito team. Thanks 👍

@danielgeri Unfortunately this doesn't seem to work as I'm getting this error: { code: "InvalidParameterException", name: "InvalidParameterException", message: "Invalid email address format." }.
After a little digging, this page in the docs seem to explain why:
If email is selected as an alias, a username cannot match a valid email format. Similarly, if phone number is selected as an alias, a username that matches a valid phone number pattern is not accepted by the service for that user pool.

Also experiencing this issue. Cognito, please advise.

Experiencing the same issue. Is there any solution for this case?

I ended up leaving this issue for my application. However, I supposed you could build your own email verification for the change email (don't change email when users submits to change, instead pass it to your own system). Send them a custom made verification link to the new email and let them verify. Once they click the link your API will need manually update their email using the CognitoIdentityServiceProvider.adminUpdateUserAttributes method in the aws-sdk package. Obviously a load of added work to get around this, but pretty sure this would work.

Is there any update on that issue?

Has anyone opened up this issue with the Cognito service center support?

@daneprime8 @lazharichir We ended up building our own email verification process.
@jordanranz I have not. I've had other core cognito issues (not related to aws amplify) and don't know where to go to reach out to the core cognito team. Mind advising on how to approach this (i.e. where is the cognito service support center)?

@danielgeri I think you'd start on the Cognito forum page.
or submit a claim by opening a case in the AWS Support Center

Also experiencing this, please respond Cognito.

I dug into this a bit more and here's what I decided to do in case it's helpful for anyone else. My Cognito is set up so that email is considered the username.
Currently, a user can sign up for a new account which requires them to confirm their email. This sets their account status to CONFIRMED and email verified to true. However, the flaw is that afterwards the user can then change their email but then not verify that email and technically still be considered an authenticated user and sign in freely. Their account status is still CONFIRMED but email verified is false.
What if the email they changed to was inputted incorrectly? Also, for security reasons I don't want any accounts where users haven't verified their email since my app deals with a lot of personal data.
Ideally, Cognito wouldn't change a user's actual email until it's been verified.
This is what I ended up doing:

I added an additional "confirm email" input to reduce chances of a user inputting an incorrect email.
Assuming they verify the new email then and there, they're logged out so that when they log back in the user's email shown as the new one using Auth.currentAuthenticatedUser(). Otherwise, it'll show the old email. Also, I wasn't able to figure out based on the Auth API how to refresh the user other than signing them out and back in.
On login, I now check a user's email_verified attributes. If it's set to false, I put them into an email confirmation flow similar to sign up and prevent them from logging in. Previously, even if email_verified was false Cognito would still sign you in.

It's not 100% perfect, but working within the constraints of Cognito it solves some of the issues. Would love to hear what others think.

Any news?  I am experiencing the same problem.  Users can change their email address without confirming it, and then log in using the new email address without it ever having been verified.

Kind of ridiculous this issue has been open for almost a year and no response from Cognito/Amplify team...I am also curious about this.

After using Cognito for a couple of years now and finding it to be reliable and well-designed overall, I would say this is one of the biggest problems with it, bordering on rendering the email alias feature unusable for production. Although it's a serious issue that is certain to leave some percentage of users locked out of their accounts, unless you've dealt with it first-hand, it's hard to describe in a way that really brings home what's happening. I did my best in my description here almost 2 years ago (now archived). At that time it was acknowledged as a "service issue" (meaning it's happening in the cloud, not the client SDK) and marked as a "feature request". Meanwhile, the GitHub issue got archived when the Cognito JS SDK was replaced with Amplify. So the SDK team has apparently washed their hands of it, and it's unclear how to get the service team to look at it.
I assume Amazon isn't "dog-fooding" Cognito since they already had a user management facility before it existed. If they were, this problem would have been addressed by now since dealing with angry users who have been locked out of their accounts has a way of making you look more carefully at an issue that might be hard to describe, but is clearly capable of happening to real users.

@Susan123456789 @cameronb23 @clintfoster the Amplify team responded but it might have not been clear that @jordanranz is a member of the team. As he and others stated this is a request for the Cognito service team and something that Amplify client libraries or tooling are unable to resolve.
For clarity Amplify is an Open Source project for development of mobile and web applications and not run by the Cognito team or any specific AWS service team. We have raised this to their attention internally, however we do recommend you post feedback directly to that team via official channels, either using your support account or the Cognito forums.

I'm also experiencing the issue of having blocked users when they update their email and logout before validating the new email with the verification code.
In addition to that, I'm facing another dangerous behavior with email updates. A user is able to change his email to an email that is already linked to another account without cognito returning an error. I don't know what is the exact state of the cognito database resulting to this action but it seems that the latter account is replaced by the one "stealing" its email address.
The expected behavior would be cognito returning an error such as "email already in use" when a user try to change its email address to an email address of an existing account.

@sChupin There's a discussion about the email capturing behavior here (unrelated to this issue).

Experiencing the same issue, as an already confirmed account email address can be updated via user attributes and even though the verification code is sent via email, no verification is required prior to start using the new email address.

not stale

not stale

Per the following response to my original report against the old Cognito SDK, this is a known service-side issue. If someone at AWS is reading this would it be possible to check on that issue...?
https://github.com/amazon-archives/amazon-cognito-identity-js/issues/515#issuecomment-350145613
If you follow my steps 1, 2, 3 in the original report you can reproduce the problem. It's hard to get the gist of it just reading the descriptions.

Finally solved this issue with a sneaky workaround and only few lines of code !! 🎉🎉
The goal is keeping the attributes unchanged until they are verified. In my case there is a sign in with phone number only and users can update their phone number and logout without verifying.
If anyone is interested here is how I did it STEP BY STEP --->
STEP 0: Custom attribute and lambda trigger for Custom Message
I introduce another attribute (called custom:validated_phone) and a cognito custom message lambda trigger for CustomMessage_UpdateUserAttribute
STEP 1: Updating user attributes with new phone number
When user wants to update their phone number client sends the current (already validated
) phone number in custom:validated_phone in updateUserAttributes function.
const result = await Auth.updateUserAttributes(cognitoUser, {
    phone_number: NEW_PHONE_NUMBER,
    'custom:validated_phone': CURRENTLY_VALIDATED_OLD_NUMBER,
  });

So cognito changes the phone_number attribute with the new one. This is something I dont want, because it isn't validated yet
Here comes the sneaky trick. Before cognito sends user a verification code it triggers a lambda to customize the message. In this lambda I look at the attribute custom:validated_phone and as an admin I update phone_number attribute back to original one (validated one). So until there is verified phone number the old one will be the still only one the in the system
if (event.triggerSource === 'CustomMessage_UpdateUserAttribute') {
  const validated_number = event.request.userAttributes['custom:validated_phone'];
  const params: AdminUpdateUserAttributesRequest = {
    UserAttributes: [
      {
        Name: 'phone_number_verified',
        Value: 'true',
      },
      {
        Name: 'phone_number',
        Value: validated_number,
      },
    ],
    UserPoolId: event.userPoolId,
    Username: event.userName,
  };
  const result = await cognitoIdServiceProvider.adminUpdateUserAttributes(params).promise();
  if (validated_number === event.request.userAttributes.phone_number) {
    throw new Error('failed to prevent sending unnecessary verification code');
  }
}

STEP 2: Verifying new phone number
After STEP 2 user still gets a verification code to his new number. The client then verifies the code. If the verification is successful then this time I again call updateUserAttributes with new number as custom:validated_phone. Because its now verified
const result = await Auth.verifyUserAttributeSubmit(cognitoUser, 'phone_number', code);
if (result === 'SUCCESS') {
  const result2 = await Auth.updateUserAttributes(cognitoUser, {
    phone_number: NEW_PHONE_NUMBER,
    'custom:validated_phone': NEW_PHONE_NUMBER,
  });
}

Now again my lambda trigger but this time both attributes are equal to each other, therefore after updating this real phone_number attribute I fail lambda intentionally so another verification code isn't sent again
  if (validated_number === event.request.userAttributes.phone_number) {
    throw new Error('failed to prevent sending unnecessary verification code');
  }

Conclusion:
I know its tricky but much better than setting up your own verification method with DynamoDB etc... Hope it helps.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

This isn't stale. I would like to mark amplify-js as stale since there are few issues most people suffer for literally years and still no improvement.

Could someone on the Cognito team try reproducing this with the three steps I outlined in the same issue against the original Cognito JS SDK almost two years ago...?
https://github.com/amazon-archives/amazon-cognito-identity-js/issues/515
You have to wait until the code expires to see it (step 2). But this is a scenario that commonly happens with real users. They get distracted and they don't enter the code right away. Then they are locked out because they cannot sign in with their old email alias, nor can they sign in with the new one.

@timwhunt
Imagine a user entered a wrong email address and realize it after not receiving the verification code on his own email, currently there's no way for him to update the email address with the same username on the sign up page. This  basic functionality has been missing for years and and please fix it for god's sake.
There are multiple threads about this and what takes your team years to notice and respond? so many developers are blocked and in pain here.

Finally solved this issue with a sneaky workaround and only few lines of code !! 🎉🎉
The goal is keeping the attributes unchanged until they are verified. In my case there is a sign in with phone number only and users can update their phone number and logout without verifying.
If anyone is interested here is how I did it STEP BY STEP --->
STEP 0: Custom attribute and lambda trigger for Custom Message
I introduce another attribute (called custom:validated_phone) and a cognito custom message lambda trigger for CustomMessage_UpdateUserAttribute
STEP 1: Updating user attributes with new phone number
When user wants to update their phone number client sends the current (already validated
) phone number in custom:validated_phone in updateUserAttributes function.
const result = await Auth.updateUserAttributes(cognitoUser, {
    phone_number: NEW_PHONE_NUMBER,
    'custom:validated_phone': CURRENTLY_VALIDATED_OLD_NUMBER,
  });

So cognito changes the phone_number attribute with the new one. This is something I dont want, because it isn't validated yet
Here comes the sneaky trick. Before cognito sends user a verification code it triggers a lambda to customize the message. In this lambda I look at the attribute custom:validated_phone and as an admin I update phone_number attribute back to original one (validated one). So until there is verified phone number the old one will be the still only one the in the system
if (event.triggerSource === 'CustomMessage_UpdateUserAttribute') {
  const validated_number = event.request.userAttributes['custom:validated_phone'];
  const params: AdminUpdateUserAttributesRequest = {
    UserAttributes: [
      {
        Name: 'phone_number_verified',
        Value: 'true',
      },
      {
        Name: 'phone_number',
        Value: validated_number,
      },
    ],
    UserPoolId: event.userPoolId,
    Username: event.userName,
  };
  const result = await cognitoIdServiceProvider.adminUpdateUserAttributes(params).promise();
  if (validated_number === event.request.userAttributes.phone_number) {
    throw new Error('failed to prevent sending unnecessary verification code');
  }
}

STEP 2: Verifying new phone number
After STEP 2 user still gets a verification code to his new number. The client then verifies the code. If the verification is successful then this time I again call updateUserAttributes with new number as custom:validated_phone. Because its now verified
const result = await Auth.verifyUserAttributeSubmit(cognitoUser, 'phone_number', code);
if (result === 'SUCCESS') {
  const result2 = await Auth.updateUserAttributes(cognitoUser, {
    phone_number: NEW_PHONE_NUMBER,
    'custom:validated_phone': NEW_PHONE_NUMBER,
  });
}

Now again my lambda trigger but this time both attributes are equal to each other, therefore after updating this real phone_number attribute I fail lambda intentionally so another verification code isn't sent again
  if (validated_number === event.request.userAttributes.phone_number) {
    throw new Error('failed to prevent sending unnecessary verification code');
  }

Conclusion:
I know its tricky but much better than setting up your own verification method with DynamoDB etc... Hope it helps.

Thanks a lot for this trick. It's really helpful. We need to verify both email and phone number. I am not sure if we can do this trick for both email and phone number simultaneously as in custom lambda message trigger, we won't know if email is being updated or phone number

Our solution was to capture email address changes as they happen and then delete and recreate the user with the new email address, therefore forcing them to verify their new email address.  It seems like a silly workaround, but it was probably the most straightforward.
On another note, it seems to me that this issue shouldn't even be an issue long past its discovery (in this case 2 years+) and should be a fundamental part of the flow.

It's absolutely crazy to me that a company as large as Amazon is unable to find the resources to fix what seems to be a pretty large flaw in their auth 'solution'... 😕

not stale

Cognito team or I rather say zombie team, any updates on this? i think it will be very easy to fix.
Also i think you should add a paragraph to your cognito getting started guide and name it 1000 reasons why cognito will not suit your business use case and maybe a warning: We are usually not fixing issues, use it at your own risk.
cc @timwhunt

We ended up with our own implementation.

Store the new email in a table
Keep the old email as the login email
Update email only after user confirmed the new email
The user is able to use the new email to login

@niraj-bpsoftware
In your solution, what's to stop a user from scripting a call themselves like so:
const result = await Auth.updateUserAttributes(cognitoUser, {
    phone_number: GARBAGE_PHONE_NUMBER,
    'custom:validated_phone': NEW_PHONE_NUMBER_I_WANT_BUT_CANT_VERIFY,
  });

If I'm reading correctly, your custom message handler will drop that NEW_PHONE_NUMBER_I_WANT_BUT_CANT_VERIFY in the phone number field, and mark it verified. Does this allow users to get around verification?

I think you're correct @mfogel, the solution @Can-Sahin proposed opens a way for a user to set any phone number as verified without actually verifying it. However, this is the only working work-around that I've found so maybe we just have to accept this security hole to be able to offer users a consistent way to verify a new phone / email.
Really hoping AWS to change the behavior so that changing phone/email attribute is postponed until it's really verified 🙏 🙏

How about we don't call this one stale until we hear some real answers?

Not stale

Just started using Cognito and ran into this problem. It's crazy that this problem has gone unfixed for nearly 2 years. I ended up having to prevent users from changing their emails to avoid security holes.

not stale

For those who don't yet have their user pool in production this comment from Cognito team member @timwhunt last fall may be helpful: https://github.com/aws-amplify/amplify-js/issues/987#issuecomment-531025897
Unfortunately we have a user pool that was created the "old way" with email aliases, rather than the new technique where the email address is literally the username. I would be curious to hear how this is working out for anyone who has tried it. I'd also love to hear any ideas for migrating an existing pool.

/// Looks like there is a condition YET to brake the loop :(
while(true) {
	notStale();
}

New to using Congnito / Amplify and ran into this issue as well.
Nothing new on my part to add to the discussion, just wanted to show the Cognito team that we're many facing this issue.

This can be done by using two steps (I've used lambda functions). You can send a code and then set the email back to the original email until the code is confirmed.
I've put some example code here:
https://jordizle.com/lab/010be7e1-a34a-4a3c-821e-36f70f3290dc/change-email-with-aws-cognito-and-amplify/

@jordizle Good write up, but the problem isn't that this issue can't be worked around.
(In my opinion) plenty of the people who have commented here will be able to work around it without much help... fundamentally the problem is that the service shouldn't have such a glaring issue it in that has been open for 2 years now with no solid resolution.
Any number of work arounds exist to fix the problem as well, the problem with doing so is that that in a larger system (or larger cooperate company) these workarounds become hidden knowledge to other developers and hidden knowledge of a problem is inevitably prone to create issues down the road for implementations or be lost with staff turnover and then no one can maintain it.
It is unbelievable that something like this can be left unaddressed for such a long time in such a core product - additionally without it being documented, closed or fixed officially.

Zombie Team,
Here is a simple solution to this issue, introduce UpdateAfterVerification parameter to update_user_attributes(...) with "False" as default value for backward compatibility.
And when set to "True" you should just send the email with verification code. and during verify_user_attribute(...) you can update the attribute if verification succeeded.
response = client.update_user_attributes(
    UserAttributes=[
        {
            'Name': 'email',
            'Value'<EMAIL_ADDRESS>        },
    ],
    UpdateAfterVerification='True',
    AccessToken='token==',
)

Hi team. Do you have any idea of solute this problem?
Changing the email address is important to us as it is a basic user management system. Is there another way to safely change my email address?

If you have the ability to do so, I would go with user pool Option 2, as described here in this same issue by Cognito team member @timwhunt last fall.
As Tim described, the Cognito team solved this problem (sort of) by allowing the username to literally be the email address (instead of having a separate username with an email "alias"). Per the Cognito docs, when you set up a new user pool there are now two choices:

Option 1 (original way): Email as sign-in alias: With this option the username cannot be an email address. But the email address can be used as an alias for sign-in, once verified. As most people have found after they go to production (sadly), this option will cause some users to be locked out when they change their email address and do not verify it in time. It has no straightforward solution (though many hacks have been discussed in this thread over the years).
Option 2 (new way): Email as username When signing up, the user is literally allowed to specify their email address as their username. If you did not choose this option when creating your user pool, you're stuck with the hacks for Option 1 unless you have the luxury of re-creating your user pool from scratch and choosing Option 2.

Having said that, as many here have pointed out, there is a straightforward fix for Option 1. Tim alluded to this as well:

We also recognize that, as has also been called out, it would be an improvement to not change the email address until after it is verified. We do have that on a list of issues to address, but I don't have any timing I can share.

AWS should implement this fix if Option 1 is still supported. They should also consider those of us who have deployed large user pools with active customers who cannot easily migrate to Option 2 without requiring all users to create new accounts. The number of people still chiming in here after almost 3 years is surely sufficient evidence that this is a significant problem.
It seems doubtful that anyone at AWS is looking at this problem, but It's hard to know since the service component of Cognito does not have a GitHub project where discussion can take place. Everyone is talking about it here because it's the only option.
For reference, here is the old issue that was archived when the old Cognito JS SDK was deprecated.

@clintfoster
Thank you for your advice. I'll consider to use user name instead of email address.

@clintfoster
It depends on the use case and what you want the user to sign in with.
I still prefer user pools with aliases and i'm not willing to use user pool with usernames even even for fresh user pool because the users can sign in with their verified email and verified phone number and preferred username.
We can't have this luxury when using user pools with usernames.
I think AWS should fix this as soon as possible, Cognito is a good service, i just don't know why AWS does not fix Cognito issues.
They just say we are aware of the issue and is in our roadmap, three years later the issue still not fixed.
Urgh what roadmap you are referring to? you are doing no effort to fix issues. this is really frustrating.

@kodless Good point. Modern apps should allow signing in with either phone number or email, which you can only do with Option 1. For example, I heard the author of whatsapp say one of the keys to its viral growth was allowing people to sign up & sign in by phone number.

If you have the ability to do so, I would go with user pool Option 2, as described here in this same issue by Cognito team member @timwhunt last fall.
As Tim described, the Cognito team solved this problem (sort of) by allowing the username to literally be the email address (instead of having a separate username with an email "alias"). Per the Cognito docs, when you set up a new user pool there are now two choices:

Option 1 (original way): Email as sign-in alias: With this option the username cannot be an email address. But the email address can be used as an alias for sign-in, once verified. As many people have found after they go to production (sadly), this option will cause some users to be locked out when they change their email address and do not verify it in time. It has no straightforward solution (though many hacks have been discussed in this thread over the years).
Option 2 (new way): Email as username When signing up, the user is literally allowed to specify their email address as their username. If you did not choose this option when creating your user pool, you're stuck with the hacks for Option 1 unless you have the luxury of re-creating your user pool from scratch and choosing Option 2.

Having said that, as many here have pointed out, there is a straightforward service-side fix for Option 1. Tim alluded to this as well:

We also recognize that, as has also been called out, it would be an improvement to not change the email address until after it is verified. We do have that on a list of issues to address, but I don't have any timing I can share.

AWS should implement this fix if Option 1 is still supported. They should also consider those of us who have deployed large user pools with active customers who cannot easily migrate to Option 2 without requiring all users to create new accounts. The number of people still chiming in here after almost 3 years is surely sufficient evidence that this is a significant problem.
It seems doubtful that AWS is working on this, but It's hard to know since the service component of Cognito does not have a GitHub project where discussion can take place. Everyone is talking about it here because it's the only option.
For reference, here is the old issue that was archived when the old Cognito JS SDK was deprecated.

Is the below what Option 2 looks like?  If so, we're still haing the same issue:

@jiwabob If you're still having the issue with Option 2, I wonder if it's a different problem?
Consider this Option 1 example:

Username: NedFlanders (or could be something like a generated UUID)
Email alias<EMAIL_ADDRESS>
To reproduce the specific problem described here, you would change the email address and fail to confirm it in time. Now you can't sign in with either the old or new email alias. However, you can still sign in with the username, assuming you know it.
With Option 2 the username actually is the email address. I haven't tried it since we're stuck with Option 1, but are you saying with Option 2 if you change your username/email and don't confirm it, neither the old or new email can be used for sign-in?

@clintfoster What we've seen is that if the user changes their email address (which IS the username as you say), but doesn't confirm it, their email is updated, and they are able to log in with the new email address regardless of the new email's confirmation status.  This becomes an issue if the user entered the wrong email address, as they would no longer be able to log in.   What I believe should happen is that until the user verifies the new email address, the email address is NOT updated.

@jiwabob If I'm understanding correctly there seems to be a pattern here with both Option 1 (email is alias for username) and Option 2 (email is username): Allowing an email address change to proceed in any way on the service side before the user has confirmed it is a bad idea. This harkens back to my earlier quote from @timwhunt:

We also recognize that, as has also been called out, it would be an improvement to not change the email address until after it is verified. We do have that on a list of issues to address, but I don't have any timing I can share.

Thinking like a programmer, I'm guessing the original service implementation never accounted for an intermediate state before the email or phone is confirmed. This would be extra work since it would require an additional field in the DB to store both the old and new values. I don't see this issue being properly solved until the service team undertakes the necessary foundational work.
It must be frustrating for the client SDK team to see all the appeals here since there apparently isn't much they can do to get the service team to look at long-standing problems with large volumes of ongoing activity from many different customers spanning all SDKs.

Same issue as @jiwabob.

Using Email address  as the username  for sign up and sign in.
Email verification is required in the user pool.
The user requests an Email change, but doesn't verify the new email with the code.
The new email is already updated in the users attributes in the user pool, even though it wasn't verified.
If the user logs out they can only log in with their new - not verified email.
The new not verified email is already taken in the user pool, which blocks any users who might have that email from using your website.
The old email the user used is now available

This would be very confusing for any user to experience.
The solution as in the quote posted from @clintfoster would be to have the email attribute updated only after it was verified.
Can we get any update on the status

not stale

Not stale...
Allowing a user to change their email address, .esp when their email address is used for sign in, without first confirming the address, seems to me to be a major security flaw.  I could be wrong.
At present, a user, if logged on, can change their email address, using Auth.updateUserAttributes, if given the opportunity in the client, after which, there is no control as to whether or not that email address has been confirmed.
The user can log out then in, with the new email, when the Cognito account in the email_verified === false state.
While this can be programmed around, client-side, I would rather rely on Cognito to prevent this behavior in the first place, as it is inherently insecure.
Prove me wrong, so that we can get a good solution here, please.

LoL 656 issues are open right now and stale bot is a nice way to get away from the things they ignore

not stale!

NOT STALE

Not Stale!

@AdamiHamza thanks for mentioning the UpdateAfterVerification trick.
I am struggling to understand how that parameter can be passed to Amplify's
Auth.updateUserAttributes()

(Python dev here)
Any hints? Thanks and apologies in advance for the dumb question!

@marcelloromani I think that it was just an example to show how to resolve the problem to the Cognito team, I don't think that UpdateAfterVerification  exists.
Still waiting for the fix :(

Following up with the Cognito service team. Thank you everyone for your patience!

Thanks for the feedback on this everyone. As @harrysolovay has mentioned above, we are raising this to the Amazon Cognito team internally. As others have called out above, we, Amplify are an Open Source project for development of mobile and web applications and not run by the Amazon Cognito team or any specific AWS service team. Can anyone provide the AWS Forums post for this issue? In the meantime, we will provide an update on this once we have one from the service team. Thanks!

Is there an update on this issue?
Wondering if you reached out to the Cognito team and what their response was? Is it a fixable issue that the Cognito team is looking at, is it just the way it's going to be, can you give us the summary of the status.
This is a real pain point for anyone who uses email as username and provides a change email functionality, because it's very easy for users to get locked out of their account if they do something wrong.

Is there an update on this issue?
Wondering if you reached out to the Cognito team and what their response was? Is it a fixable issue that the Cognito team is looking at, is it just the way it's going to be, can you give us the summary of the status.
This is a real pain point for anyone who uses email as username and provides a change email functionality, because it's very easy for users to get locked out of their account if they do something wrong.

I think the trick described earlier in this thread to use lambda triggers to circumvent this issue could be improved from a security standpoint by creating the custom attribute in the post confirmation lambda rather than creating it from the client. In the user pool client settings, the read/write for that custom attribute can be disabled.
cc @Can-Sahin  @mfogel

I think the trick described earlier in this thread to use lambda triggers to circumvent this issue could be improved from a security standpoint by creating the custom attribute in the post confirmation lambda rather than creating it from the client. In the user pool client settings, the read/write for that custom attribute can be disabled.
cc @Can-Sahin  @mfogel

If you have the ability to do so, I would go with user pool Option 2, as described here in this same issue by Cognito team member @timwhunt last fall.
As Tim described, the Cognito team solved this problem (sort of) by allowing the username to literally be the email address (instead of having a separate username with an email "alias"). Per the Cognito docs, when you set up a new user pool there are now two choices:

Option 1 (original way): Email as sign-in alias: With this option the username cannot be an email address. But the email address can be used as an alias for sign-in, once verified. As many people have found after they go to production (sadly), this option will cause some users to be locked out when they change their email address and do not verify it in time. It has no straightforward solution (though many hacks have been discussed in this thread over the years).
Option 2 (new way): Email as username When signing up, the user is literally allowed to specify their email address as their username. If you did not choose this option when creating your user pool, you're stuck with the hacks for Option 1 unless you have the luxury of re-creating your user pool from scratch and choosing Option 2.

Having said that, as many here have pointed out, there is a straightforward service-side fix for Option 1. Tim alluded to this as well:

We also recognize that, as has also been called out, it would be an improvement to not change the email address until after it is verified. We do have that on a list of issues to address, but I don't have any timing I can share.

AWS should implement this fix if Option 1 is still supported. They should also consider those of us who have deployed large user pools with active customers who cannot easily migrate to Option 2 without requiring all users to create new accounts. The number of people still chiming in here after almost 3 years is surely sufficient evidence that this is a significant problem.
It seems doubtful that AWS is working on this, but It's hard to know since the service component of Cognito does not have a GitHub project where discussion can take place. Everyone is talking about it here because it's the only option.
For reference, here is the old issue that was archived when the old Cognito JS SDK was deprecated.

Is the below what Option 2 looks like? If so, we're still haing the same issue:

hello,
sorry but I do not undersand how option 2 solves the issue.
if you signup and signin with email as username then it is impossible to change Username in cognito as per the aws doc.
plus I dont see how it solves the issue for verifying the email before modifying the email attribute ....
the only hack here is the one with lambda custom message trigger to be triggered twice

because of this bug/vulnerability I had to put our migration to cognito on hold.
is there any update on this?

hello,
I solved it by following what has been said in previous posts : using cognito lambda trigger custom message and using the following option in Cognito "attributes" settings : "Email address or phone number - Users can use an email address or phone number as their "username" to sign up and sign in. "
I had to create from scratch another user pool since you cannot switch from "Username" to "Email address or phone number".
cheers

Do we have a regex for valid email id? What is aws cognito using at backend to validate it?

2.5+ years and still no solution. Shame

2.5+ years and still no solution. Shame

I've observed the following as the current behavior/usage.

Use updateUserAttributes to change email
Use verifyCurrentUserAttributeSubmit to submit the verification code email to the user (this is sent automatically when updateUserAttributes email is called with email)
3.* if user needs, use verifyCurrentUserAttribute to resend verification codes

After calling the initial updateUserAttributes with the new email show the verification UI right away. Ideally, the user will enter the code and the new email will be verified. However, if the user signs out before performing that step:
A. Log in with the old email will fail
B. Log in with new email succeeds but directed to verification step to enter verification code

Do we have a regex for valid email id? What is aws cognito using at backend to validate it?

As always, SO is your friend :-)
https://stackoverflow.com/questions/201323/how-to-validate-an-email-address-using-a-regular-expression

@marcelloromani This board is meant for amplify library and my question is specific to what they are using in backend, so I can implement same thing in frontend as well. I don't want a generic regex from SO.

I'm officially asking: Do we have a fix?

Up

@swapnil0545 What I was trying to say is that you can only validate an email address to an extent just by looking at the string. (first question of your sentence :) )
But you want to replicate their validation on the FE. OK, only they can reply :P

This problem is still available. I'm currently working on a ticket for my company using Amplify and Cognito where user can change their email in cognito. We have a gap where I will turn off my computer between GetUserAttributeVerificationCode and VerifyUserAttribute. New email will be saved to Cognito and will be marked as unverified. However I have users walk through phone_number_verification process before that. So now the user can signIn with the new email without even verify that email belong to them :) nice. Also Cognito Attributes can't be changed. So now the only way the user can sign-in into my company application is through email and that email could potentially not even belong to them. They can still sign-in since they have 1/2 fields (email_verified, phone_number_verified) mark as true. 👎

For anyone who is looking for a workaround. Please follow this link https://github.com/aws-amplify/amplify-js/issues/987#issuecomment-508059168. This is the most fined solution and the only so far. The email attribute has nothing to do with the verifcationCode after you have sent your verificationCode to this newEmail. So you can freely switch back to the old verified email by trigger another apis.

Up on this issue! Very crucial for our user experience to allow them to update their phone number

Up

Based on my personal discussions with the amazon support team I can get in contact with, since the defect and security issue tickets I file reporting defects and securoty issues in cognito are all getting ignored, I  assume the cognito team is actively refusing to make security and other fixes. and and am now working to use a different service/provider.

Hi @duaneking and to all curious about the update on this issue. I am from Amplify team and I am checking with our Cognito partner on this issue. Thank you for your patience and sorry for the wait.
As many pointed out, Amplify as a frontend team, relies on service team (Cognito) for backend API fix/update. What I can share now is this issue is currently in Cognito's team radar. While Amplify is also waiting for the update from the service team, I wish to share some general concerns for a backend fix like this with you. Hopefully, it can help shed some light on why the fix is challenging and time consuming, and help to get your understanding on the wait :pray:
Following the common best practices, AWS cloud service is designed to be scalable and reliable for our customers. As mentioned in the above discussion, one major challenge for the fix is that it needs to somehow introduce a new "state" in the backend (so as to hold the temporary unverified email). Introducing a new state to an existing service always raises concerns on how to make it scalable (e.g. where and how to effectively and securely store the new state? when to release the resource for storing that temporary state?) and reliable/resilient (e.g. how do we re-populate the state if the service somehow needs a reboot?). Last but not the least, it also needs to care about backward compatibility and consistency. For customers who is good with the current implementation, it needs a mechanism to opt in/out the fix. This then raises the question on how to integrate the fix/feature into the "control plane"(e.g. user pool configuration), to avoid introducing confusion/unexpected behaviors to both existing and new customers.
AWS does care about the customers, and that is why the service team needs to take so many different aspects into account before launching a robust fix, which makes the process slower than expected :bow:
Again I appreciate your patience on it, and will keep update here once I hear more from the service team.

I'm a developer so when I see someone like you start with the public relations stuff and get hand wavy, I automatically assume you're just trying to lie to us.￼ But OK, let's give you the benefit of the doubt￼. Somebody has to be the first person to do that, after all. Let's start the healing.
But honestly what you said makes no sense at all so now I'm confused, because an instance of Cognito doesn't allow us to upgrade it or export or import users or data into/from it; ￼￼￼￼and a lot of the defects that I have reported in multiple tickets are not even things that would create backwards compatibility issues. How many cases these issues that I have reported actively stop users from being able to create instances of Cognito.
From my perspective as a principal level engineer, ￼This specific defect only exists because that team didn't do their due diligence or care about the user flows enough.
￼
So I have to ask what you mean here, because this seems really handwavy and that team has a consistently bad reputation for failing customers or creating issues that lead to failed customer expectations; I even had to force them to update their documentation once by pushing really hard on a bug that's still an active issue and all they would do is update a couple lines in the public documentation but refused to fix the actual issue.￼
It was not a backwards compatibility issue, not unless their architect sucks.
￼
So you can see, I've got good intentions here and I would honestly be totally open to putting myself out there, but as you can see, my hearts been hurt before.￼￼￼

@duaneking I am sorry to hear that, and I am not trying to do public relations. I am also a developer, like many of us, waiting on the fix of this issue.
My intention is just to share what I learn from the Cognito service team. As a frontend developer, I was also curious why it took such a long time for fixing it in backend. Those concerns mentioned above are what I learned (and what I can share), and I find them helpful for me to understand the challenges they are facing. So I think it might be a good idea to share them (while we are waiting together). Other than that, I could not speak too much for the Cognito service team until I get new update from them.

@hkjpotato That's appreciated.  The biggest frustration I have is that no matter how many people I managed to get on the phone after paying for the support I ended up never actually getting, nobody from the Cognito team could ever be contacted or forced to get on a call with me as a paying customer.  They refuse to interact or communicate with customers, and actively use others on various support teams that support them as cats paws/scapegoat's/fall guys/etc, throwing every other team they interact with under the bus for issues they created by their incompetency and lack of customer focus.

Hi, I am Amit from the Cognito team. Appreciate you all taking the time and providing valuable feedback. I wanted to let you know that we heard you and know we are continuously adjusting our roadmap balancing our customers’ operational needs against new features requests and enhancements to existing functionality.
That said, we are evaluating a change to address the feedback here (Can’t comment on specific timeline).
Thanks!

@AWSAmitjh
Yes, we all agree that this framework is fundamentally broken and unreliable.  But when will it get fixed? Marketing messages like the one above don't have the impact of actual code fixes and issues resolved.  We don't want to hear "good news" we want to see the fix working and have our code work as expected without giving us fear.
The Amplify team is not following the Amazon leadership principles here, at all.  Not a single one. And this sort of marketing and messaging only tells us that we cant depend on amazon or aws because its all we ever hear when we get ghosted eventually after reporting issues so you sending us this marketing messaging is kind of a bad thing as it signals that this is a low priority and no fixes will actually happen as its clear that the energy amazon want to put into this is to calm the devs down instead of actually fix the issue.  This is systemic at amazon, and as a customer who has had to deal with this constantly I'm seriously thinking about non-amazon hosting for my next deployment because all we ever hear is marketing messages and other BS and never actually have our issues resolved; And when that happens Cognito will be the reason AWS misses out on the next unicorn.

I'm excited about a potential resolution here!

Omg, 3 years and nothing!!!

Hi, I am Amit from the Cognito team. Appreciate you all taking the time and providing valuable feedback. I wanted to let you know that we heard you and know we are continuously adjusting our roadmap balancing our customers’ operational needs against new features requests and enhancements to existing functionality.
That said, we are evaluating a change to address the feedback here (Can’t comment on specific timeline).
Thanks!

As developers, we all know what "evaluating" means. Effort vs ROI. As much as I want to be hopeful, after 3 years, and now it's in evaluation.

Looking forward to fixes related to Cognito / Amplify.

Hi, I am Amit from the Cognito team. Appreciate you all taking the time and providing valuable feedback. I wanted to let you know that we heard you and know we are continuously adjusting our roadmap balancing our customers’ operational needs against new features requests and enhancements to existing functionality.
That said, we are evaluating a change to address the feedback here (Can’t comment on specific timeline).
Thanks!

As developers, we all know what "evaluating" means. Effort vs ROI. As much as I want to be hopeful, after 3 years, and now it's in evaluation.

Hey, if that's the case (likely), I've been recommending auth0 whenever I have the chance after my experiences with cognito.

Auth0 is a cancer that I believe is actively malicious to the success of their own customers by arbitrary raising the price per user for startups to a level that is unsustainable, and  it really sucks that Amazon as a company is now lacking so much customer focus that other, worse solutions are now being considered by its customers in their desperation due to amazons utter failure to keep to its commitments here, MHO.
I spoke with a MS PM the other day on a Twitter Spaces in a public chat and they mentioned they saw Cognito as something they could easily compete and win against due to the fact that AWS has so severely dropped the ball, so maybe its just a  matter of waiting for the other big cloud providers to provide this solution since AWS is so unfocused and uncaring of the success of its customers that this cognito team has been allowed to drop the ball so badly?
It is absolutely insane that the cognito team is being allowed to fail and hurt customers so badly.

It's disappointing this issue is still unresolved; nonetheless, I'm more intrigued about how is the recommended way on Cognito to prevent an unauthorized change in email while AFK? The email verification restriction before signing in again after an email change is great, but why Cognito doesn't enforce giving the current user's password to allow the change in the first place?

We also encountered this issue in our company, very disappointing..

I'm getting really tired of all the lies and be from Amazon about this.
We have waited years for a fix to a very basic thing, something that is expected of us to fully support as part of simple common standard regulatory and legal compliance; Amazon's abject failure to maintain secure systems or maintain credibility or compliance with the regulatory standards that they are now falsely claiming to be compliant with is an absolute failure that has put an untold millions of users at risk and harmed developers efforts to use the platform.￼￼￼
Somebody on the Amazon leadership team who is high enough to be able to dictate a policy and fix this needs to show Leadership and do so; The Cognito team is holding Amazon Web services back as a company￼.

This can be done by using two steps (I've used lambda functions). You can send a code and then set the email back to the original email until the code is confirmed.
I've put some example code here:
https://jordizle.com/lab/010be7e1-a34a-4a3c-821e-36f70f3290dc/change-email-with-aws-cognito-and-amplify/

This solution perfectly worked for me and it looks clean also

hey this is ridiculous, when is this going to be fixed?

I think it fair to say after many years of watching this issue, that Amazon has proven multiple times that the very jr. engineers that are placed in high priority roles for this critical service are simply too incompetent to design and working and functional system that is secure, because leadership simply does not care about customers.
Every single one of the now 16 amazon leadership principles is being violated by this incompetent, not at all customer focused team.   Cognito is the one service that is holding the rest of AWS back, and Amazon needs to look into its Cognito Problem.

I don't work at Amazon, but the Amazon recruiters wont leave me alone.  I am so sick of this issue, so I have asked the L8+ that last tried to hire me to consider re-org-ing the Cognito team, and also pointed them to this thread, citing it as an example of amazons inability to be the kind of place a good engineer wants to work.
I now will reply to every Amazon Recruiters request to connect or that I interview with a link to this thread, from now on, citing the incompetence displayed here as a reason I wont join amazon.
The Cognito team is absolutely failing its customers, every day.  They should be ashamed of themselves.  So I'm going to intentionally turn up the heat. Eventually, the recruiters will contact their executive and these people will hopefully display some leadership and raise the bar so Cognito will be reorged.

This solution has stopped working. We first noticed today but we had users report problems over the weekend. The problem seems to be that 'verifyUserAtttributeSubmit' now changes the attribute to the new one (in our case the new email). So by the time the second updateUserAttributes call is made, the lambda is not triggered sine the email is technically not being changed, therefore we cannot complete our full update because we had some extra logic in the final if statement.
Note: we implemented this hack for email change. Worked the same as the phone number example.
AWS cognito has been a pain point in our system. Lack of support, fixes, or improvements has been extremely frustrating. And in this case a change in underlying behavior that I have not found documented anywhere is just a cherry on top.
We need to figure out another hack to fix this original hack. I will report back with a fix as soon as we find one.

 Lack of documentation, no response on critical issues and features such as this, forcing developers to come up with workarounds, then the audacity to randomly, SECRETLY pushing behavior changes to prod, BREAKING live clients with no announcements, no nothing.. just wow. This cognito team, product, what a complete joke. 🤡

I was contacted today by people internal at AWS, and I sent them this bug link exactly as I said I would in my earlier post.

I was just contacted again by AWS recruiters, and I gave them this link as my reply exactly as I said I would.

I have once again presented this defect to yet another hiring manager at amazon/aws.

I hope they do something about it and improve Cognito.

Hello everyone. We've worked with the Amazon Cognito team on this issue and they have taken the feedback to address the request for changes in the service.
The service changes have been released and are available today in Cognito via the Console or CloudFormation. We will be incorporating these changes into the Amplify CLI and Studio in upcoming releases. Please see the information below for configuring the new functionality in Cognito:

A new section titled Verifying attribute changes has been added to the Amazon Cognito console. With the feature active, you can choose to keep the original value of an email address or phone number attribute value active when an update is pending. With the feature inactive, Amazon Cognito continues to follow legacy behavior when a user changes their email address or phone number: they can’t sign in or receive messages with the original or new value, until they verify the new value.
You can configure the feature to apply to email address, phone number, or both.
When you turn on this feature for a user with a pending attribute change, that user will continue to receive messages and sign in with the original attribute value until they verify they new one. After they enter a code or follow a link to verify their new information, they can receive messages and sign in only with the new attribute value. Your users will benefit from this new feature because your users won’t lose access to your app when they have a new, unverified email address or phone number.
You can change user attributes in the Amazon Cognito console or with the API operations UpdateUserAttributes and AdminUpdateUserAttributes. When you configure your user pool to retain the original attribute value pending verification, your user’s original attribute value remains active until you make a VerifyUserAttribute request.
Amazon Cognito has introduced the new settings group UserAttributeUpdateSettings, and the new parameter RequireAttributesVerifiedBeforeUpdate. To activate this feature with the Amazon Cognito API, set the value of RequireAttributesVerifiedBeforeUpdate in a CreateUserPool or UpdateUserPool API request.
For more information about these API operations and fields, see the Amazon Cognito API reference (https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/Welcome.html).


Error after install

Hello,
After installing this module (using composer), I get an error :

Best regards,

Any news about it ? Is the module still maintained ?

Hello @PubliAlex,
I'm sorry to let you waiting, I updated the plugin and fixed some bugs related to the package dependency.
Please check and confirm if it works as expected.
Thanks for reporting.
Regards

Hello,
Still have the error, when I try to install the plugin and run php artisan october:migrate
I get this error

Best regards,

Freeze lib dependency iconify/json to version 1
A recent change in the iconify/json iconset format cause an issue, will be addressed in v2 of the Plugin


Implement defence feedback 144714671

What does this PR do?
Implement feedback received from the defense session.
Description of Task to be completed?

##How should this be manually tested?
NA
Any background context you want to provide?
NA
What are the relevant pivotal tracker stories?
#144714671
Screenshots (if appropriate)
NA
Questions:
NA

Coverage remained the same at 87.692% when pulling 4e45dc9a87c0aeb5ade7ea2b21b0b5ae5ae72095 on implement-defence-feedback-144714671 into 93c7a97826ccc31c7cb0109c3776c2a6ed755dfd on develop.


Hodgkin-Huxley model parameter estimation for observed experimental traces.

Hey guys,
first of all, it's a pleasure to experiment with the sbi package. Great work on having state of the art simulator based inference algorithms in one package.
Since I have gotten only unsatisfactory results (with multiple packages) for quite some time now, I wanted to get your opinion on strategies to efficiently 1.--  create 1 compartment models and 2.-- run the inference for actually observed experimental traces. Feel free to comment on every step that I employ to infer parameters for mouse motor cortical neuron responses I see to current clamp (Patch-seq) experimental protocols.
I check the neuron cell type: Pvalb (fast spiking) e.g. and 2 channels (Na+ / K+) should in principle be suffient (I also use 'sufficient 1 compartment models from Pospischil et al.); Sst and perhaps I need an adapting K+ current (g_M in your tutorial); a bursting Vip or Sst neuron and I'd add one or two Ca++ channels (also explained in Pospischil et al.). First of all, the models in Pospischil et al. were done for ferret cortical neurons, not for mice, but I nevertheless expect these 1 compartment models to be rather sufficient to describe also mouse cortical neurons.
Then I'd check my experimentally observed membrane time constant and input resistance and derive a membrane area from it. Concretely: if I assume a specific membrane resistance c_m of 1 microF/cm2 , I'd adapt the specific membrane resistance r_m  (kOhm * cm2) so that c_m * r_m gives me the experimentally observed tau (ms). With that derived r_m, the observed input resistance R_i should equal r_m * area, so I adapt my 1 compartment area so that it is satisfied.
`
def syn_current(duration=800, dt=0.04, t_on = 100, curr_level = 1e-4, exp_input_res=270, exp_tau=10, seed=None):
# exp_input_res in MOhms!
# exp_tau in ms!
duration = duration
t_off = duration - t_on
t = np.arange(0, duration+dt, dt)
# external current
r_m = exp_tau*1e3            # specific membrane resistance (Ohm*cm**2) (so that r_m*c_m= exp_tau,
                             # c_m assumed to be 1 microF/cm**2)

A_soma = r_m/(exp_input_res*1e6)  # cm2
#A_soma=np.pi*((70.*1e-4)**2)
I = np.zeros_like(t)
I[int(np.round(t_on/dt)):int(np.round(t_off/dt))] = curr_level/A_soma # muA/cm2
return I, t_on, t_off, dt, t, A_soma`

What about the temperature of the experiment? Optimally one should adapt the rate constant equations with the Q10 factor so that the rate of change of open probabilities of ion channels is temperature scaled. Models in Pospischil et al. are optimised for T1 = 36 degree Celcius, my experiments are either done at T2 = 25 or 34 degree Celcius. I adapt, therefore, dx/dt = scale * ... with the scale being Q10**(T2-T1). Here x represents m, h, p, ... I hope I added this scale at the right place in your exponential Euler method (see code below).
`def HHsimulator(V0, params, dt, t, I, seed=None):
"""Simulates the Hodgkin-Huxley model for a specified time duration and current
    Parameters
    ----------
    V0 : float
        Voltage at first time step
    params : np.array, 1d of length dim_param
        Parameter vector
    dt : float
        Timestep
    t : array
        Numpy array with the time steps
    I : array
        Numpy array with the input current
    seed : int
    """

gbar_Na = params[0, 0]  # mS/cm2
gbar_Na.astype(float)
gbar_K = params[0, 1]  # mS/cm2
gbar_K.astype(float)
g_leak = params[0,2] # mS/cm2
g_leak.astype(float)
gbar_M = params[0,3] # mS/cm2
gbar_M.astype(float)
tau_max = params[0,3] # ms
tau_max.astype(float)
#tau_max=800
#gbar_M=0.0 # mS/cm2

# fixed parameters
#g_leak = 0.1  # mS/cm2
#gbar_M = 0.07  # mS/cm2
#tau_max = 6e2  # ms
Vt = -60.0  # mV
nois_fact = 0.1  # uA/cm2
E_leak = np.mean(voltage_obs[0:2500, curr_index])  # mV
C = 1  # uF/cm2
E_Na = 53  # mV
E_K = -107  # mV
Q10=2.5

T_1 = 36 # °C, from paper MartinPospischil et al.
T_2 = 34 # °C, experiment was actually done at 34 °C
t_adj_factor = Q10**((T_2-T_1)/10) # temperature coeff., https://en.wikipedia.org/wiki/Q10_(temperature_coefficient)
tstep = float(dt)

if seed is not None:
    rng = np.random.RandomState(seed=seed)
else:
    rng = np.random.RandomState()

####################################
# kinetics
def efun(z):
    if np.abs(z) < 1e-4:
        return 1 - z / 2
    else:
        return z / (np.exp(z) - 1)

def alpha_m(x):
    v1 = x - Vt - 13.0
    return 0.32 * efun(-0.25 * v1) / 0.25

def beta_m(x):
    v1 = x - Vt - 40
    return 0.28 * efun(0.2 * v1) / 0.2

def alpha_h(x):
    v1 = x - Vt - 17.0
    return 0.128 * np.exp(-v1 / 18.0)

def beta_h(x):
    v1 = x - Vt - 40.0
    return 4.0 / (1 + np.exp(-0.2 * v1))

def alpha_n(x):
    v1 = x - Vt - 15.0
    return 0.032 * efun(-0.2 * v1) / 0.2

def beta_n(x):
    v1 = x - Vt - 10.0
    return 0.5 * np.exp(-v1 / 40)

# steady-states and time constants
def tau_n(x):
    return 1 / (alpha_n(x) + beta_n(x))

def n_inf(x):
    return alpha_n(x) / (alpha_n(x) + beta_n(x))

def tau_m(x):
    return 1 / (alpha_m(x) + beta_m(x))

def m_inf(x):
    return alpha_m(x) / (alpha_m(x) + beta_m(x))

def tau_h(x):
    return 1 / (alpha_h(x) + beta_h(x))

def h_inf(x):
    return alpha_h(x) / (alpha_h(x) + beta_h(x))

# slow non-inactivating K+
def p_inf(x):
    v1 = x + 35.0
    return 1.0 / (1.0 + np.exp(-0.1 * v1))

def tau_p(x):
    v1 = x + 35.0
    return tau_max / (3.3 * np.exp(0.05 * v1) + np.exp(-0.05 * v1))

####################################
# simulation from initial point
V = np.zeros_like(t)  # voltage
n = np.zeros_like(t)
m = np.zeros_like(t)
h = np.zeros_like(t)
p = np.zeros_like(t)

V[0] = float(V0)
n[0] = n_inf(V[0])
m[0] = m_inf(V[0])
h[0] = h_inf(V[0])
p[0] = p_inf(V[0])

for i in range(1, t.shape[0]):
    tau_V_inv = (
        (m[i - 1] ** 3) * gbar_Na * h[i - 1]
        + (n[i - 1] ** 4) * gbar_K
        + g_leak
        + gbar_M * p[i - 1]
    ) / C
    V_inf = (
        (m[i - 1] ** 3) * gbar_Na * h[i - 1] * E_Na
        + (n[i - 1] ** 4) * gbar_K * E_K
        + g_leak * E_leak
        + gbar_M * p[i - 1] * E_K
        + I[i - 1]
        + nois_fact * rng.randn() / (tstep ** 0.5)
    ) / (tau_V_inv * C)
    V[i] = V_inf + (V[i - 1] - V_inf) * np.exp(-tstep * tau_V_inv)
    n[i] = n_inf(V[i]) + (n[i - 1] - n_inf(V[i])) * np.exp((-tstep*t_adj_factor / tau_n(V[i])))
    m[i] = m_inf(V[i]) + (m[i - 1] - m_inf(V[i])) * np.exp((-tstep*t_adj_factor / tau_m(V[i])))
    h[i] = h_inf(V[i]) + (h[i - 1] - h_inf(V[i])) * np.exp((-tstep*t_adj_factor / tau_h(V[i])))
    p[i] = p_inf(V[i]) + (p[i - 1] - p_inf(V[i])) * np.exp((-tstep*t_adj_factor / tau_p(V[i])))

return np.array(V).reshape(-1, 1)`

And then inference right? Well, no matter what I seem to opt for (BDNs, MAFs, SNLE or SNPE) I seem to either, run into posteriors that have no prior support (i.e. the summary statistics I observe lead to parameter values outside of the prior support), or I do get some posterior that gives me traces rather 'not so close' to what I experimentally observe, in terms of action potential (AP) shape, or even firing rate.
Here's an example of an Sst neuron responding to a 100 pA positive current injection.

What I seem to struggle with is that I don't have simulations arriving at the same 'height' meaning that the derived membrane area might be wrong but even when I try to scale the simulation input current (which is the same as if one wants to scale the area), or even try to infer the scale, I get no sensible results. What about the Nernst potential for E_Na+ and E_K+? Perhaps they are dissimilar to ferret conditions. I tried to use different values for them too, however, or infer them, but to no avail.
So I'm reaching a bit of a dead end. Now, a HH-like model is highly nonlinear and a true challenge for these inference algorithms. I think it is fair to say that we've not completely solved this open problem. In Fig. 4 of "Training deep neural density estimators to 2 identify mechanistic models of neural dynamics", you've fitted Allen Institute data and, although, in the right direction, AP shapes and adapting firing patterns can be very different between high posterior samples and experimentally observed traces. I was also surprised to see that you've used simpler BDNs for experimentally observed traces and more flexible MAFs for simulated data.
Any thoughts, comments, ideas are highly appreciated!

I barely believe it but at the time of writing I finally got a posterior sample which came close for the first time to an experimentally observed trace:

But, most posterior samples still did have no APs at all (I used the BDN framework, inspired by your paper, 1 round and 10000 trained on samples). Perhaps I'm slowly moving in better directions.
Anyway, thoughts and ideas still highly appreciated.

Hi Yves,
Thanks for the words of encouragement and the questions. I will here only reply to the sbi-related issues.

So I'm reaching a bit of a dead end. Now, a HH-like model is highly nonlinear and a true challenge for these inference algorithms. I think it is fair to say that we've not completely solved this open problem. In Fig. 4 of "Training deep neural density estimators to 2 identify mechanistic models of neural dynamics", you've fitted Allen Institute data and, although, in the right direction, AP shapes and adapting firing patterns can be very different between high posterior samples and experimentally observed traces.

In https://www.biorxiv.org/content/10.1101/838383v3, you are right that the high-probability (under the posterior) samples are not always in very close agreement with the experimental data from Allen Institute. In principle, this could be an indication that (1) the posterior is not well inferred or/and (2) the model is not flexible enough to fit the experimental data. At this point, we cannot tell with full certainty what is the main cause. However, we have run another algorithm (SMC-ABC) on the same cells (and with much higher number of simulations than with SNPE), and the posteriors were quite similar to the ones obtained with SNPE, suggesting that the limitations on this problem are more on the modelling side than on the inference side.

I was also surprised to see that you've used simpler BDNs for experimentally observed traces and more flexible MAFs for simulated data.

There is another important distinction between simulated data and experimental data: in the simulated data case, we ran one round SNPE, whereas we ran 2 rounds SNPE for the experimental data. Indeed, for the experimental data, we empirically found that multi-round SNPE improved substantially the quality of the posteriors. However, currently, multi-round SNPE with MAFs has some "leakage" issues (see issue https://github.com/mackelab/sbi/issues/287). Therefore, for our problem, we used multi-round SNPE with MDNs.
Note that MDNs are planned to be implemented in sbi in the next few weeks.

But, most posterior samples still did have no APs at all (I used the BDN framework, inspired by your paper, 1 round and trained on 10000 samples). Perhaps I'm slowly moving in better directions.

Do you really mean 10k samples? If so, I would suggest you to increase the number of simulations. In our case, for each experimental trace, we ran 2 rounds SNPE, each round with 125000 simulations.

Hi Pedro,
thanks for the fast reply and encouraging suggested adaptations!

suggesting that the limitations on this problem are more on the modelling side than on the inference side
This might indeed suggest that, for some neurons, we need an additional compartment modelling e.g. the axon initial segment, or simply current diffustion to dendrites and axons.

However, currently, multi-round SNPE with MAFs has some "leakage" issues (see issue #287).
Does using MAFS with SNLE have the same issue?

we ran 2 rounds SNPE, each round with 125000 simulations.
I will try 2 rounds of SNPE with much more samples and MDNs.

Note that MDNs are planned to be implemented in sbi in the next few weeks.
To me it seems implemented already? Are you sure it's not in there yet?

Hi Yves,

This might indeed suggest that, for some neurons, we need an additional compartment modelling e.g. the axon initial segment, or simply current diffustion to dendrites and axons.

Yes, absolutely. An additional compartment would perhaps be a good idea. But even before that, do increase the number of simulations for training SNPE. This might lead to substantially better results.

Does using MAFS with SNLE have the same issue?

It will not have the same issue, but it might have other issues. We have never tried SNLE nor SNRE in this problem.

To me it seems implemented already? Are you sure it's not in there yet?

Sorry, I was not very clear. You are right that MDNs are implemented.
Let me be more specific. At the moment, SNPE is implemented with atomic proposals (see https://arxiv.org/abs/1905.07488), which leads to the "leakage" issues I mentioned (independently of using MDNs or MAFs). In the next few weeks, the non-atomic version of SNPE (https://arxiv.org/abs/1905.07488) will be implemented: this one does not have the serious "leakage" issues but only works for MDNs (not MAFs).

Sorry, I was not very clear. You are right that MDNs are implemented.
Let me be more specific. At the moment, SNPE is implemented with atomic proposals (see https://arxiv.org/abs/1905.07488), which leads to the "leakage" issues I mentioned (independently of using MDNs or MAFs). In the next few weeks, the non-atomic version of SNPE (https://arxiv.org/abs/1905.07488) will be implemented: this one does not have the serious "leakage" issues but only works for MDNs (not MAFs).

I see what you mean. 2 rounds with 100000 simulations each and these 'leakage' issues seem to occur.

But even before that, do increase the number of simulations for training SNPE. This might lead to substantially better results.

I'll keep it to one round of SNPE MDN 100000 simulations for now then :). Happy to hear you guys are working on the non-atomic version. Could it be that you posted the same reference twice? I read that reference, but perhaps I'm less aware of a reference referring to non-atomic SNPE.
Cheers!

I'll keep it to one round of SNPE MDN 100000 simulations for now then :). Happy to hear you guys are working on the non-atomic version. Could it be that you posted the same reference twice? I read that reference, but perhaps I'm less aware of a reference referring to non-atomic SNPE.

I posted the same reference twice, and it was on purpose. ;) The paper has both types of proposals (see sections 3.1 and 3.2 of the paper).

I see. A closer look is required then.
Thanks again Pedro!
I hope to post a good posterior sample here soon, to partially resolve this issue.

Hi Yves, cool that you're working with sbi! :)
I've started implementing the non-atomic version of APT. You can track progress here. If everything goes well, it will be done end of next week.
All the best and see you soon in Tübingen
Michael

Hey Michael,
Sounds great!
I'll track you closely then ;)
See you in Tübingen indeed!

https://github.com/mackelab/sbi/pull/301 got merged, I'll close this issue for now. Feel free to reopen with further questions


Tim Smith | Memory Game

I added my svelte-based memory game.

Please add live project link or screenshots,
Thank you.


Drop reflection from ExtraFieldUtils

As of today to make [compress] run on native-image (GraalVM) you must reister all classes in IMPLEMENTATIONS map. With this PR this is no more needed and [compress] zip support comes OOTB.

Codecov Report
Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (1887b3d) 80.76% compared to head (f70746a) 80.69%.
Report is 22 commits behind head on master.

Files
Patch %
Lines

...ommons/compress/archivers/zip/ExtraFieldUtils.java
78.12%
7 Missing :warning:

.../commons/compress/archivers/sevenz/SevenZFile.java
88.23%
2 Missing and 2 partials :warning:

...compress/archivers/zip/X0014_X509Certificates.java
50.00%
1 Missing :warning:

...ress/archivers/zip/X0015_CertificateIdForFile.java
50.00%
1 Missing :warning:

...rs/zip/X0016_CertificateIdForCentralDirectory.java
50.00%
1 Missing :warning:

...ss/archivers/zip/X0017_StrongEncryptionHeader.java
50.00%
1 Missing :warning:

.../zip/X0019_EncryptionRecipientCertificateList.java
50.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #480      +/-   ##
============================================
- Coverage     80.76%   80.69%   -0.08%
+ Complexity     6801     6799       -2
============================================
  Files           348      348
  Lines         24945    24978      +33
  Branches       4029     4030       +1
============================================
+ Hits          20148    20157       +9
- Misses         3269     3293      +24
  Partials       1528     1528

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@garydgregory very good catch! fixed now normally. On the builder point I had the same thought and I think it is maybe worth to not expose all constructors (package scope?) and just use a single one for next release, the builder will make the api more verbose with a very low gain since most constructors are just about calling setters - and extra fields are exactly there for example.

@garydgregory very good catch! fixed now normally. On the builder point I had the same thought and I think it is maybe worth to not expose all constructors (package scope?) and just use a single one for next release, the builder will make the api more verbose with a very low gain since most constructors are just about calling setters - and extra fields are exactly there for example.

Hi @rmannibucau

I think that for this PR, the constructor changes should be as minimal and visible as you can make them, so yes, package-private or private is the way to go.
Once this PR is done, I can experiment with a Builder.

@rmannibucau
Merged!
Thank you for your work 😄


UseAngularCliServer adds --port {portnumber} to script by default!

Describe the bug
UseAngularCliServer adds --port {portnumber} to script by default!
this should be an option not an obligation, i want to run my npm script without adding --port at the end of my script

Thanks for contacting us, @Ahmed-Habbachi.
Can you please share more details regarding where do you specify the port yourself?
The logic in the Angular middleware automatically finds an available port and uses that for starting the Angular CLI behind the scene.

i use it in an electron + angular + asp.net core project for UI stuff and i have a ready to use script that allow me to start angular and electron and this UseAngularCliServer i use it as a work around to run my script. as i didn't find any way to do it differently.
my script is:
"start": "npm run postinstall:electron && npm-run-all -p ng:serve electron:serve"
so at the end of the command UseAngularCliServer adds --port {portnumber} wich is unneeded for me.

i found a work around to use the passed --port to my built script and use it to start my electron project

Glad you found a way to solve the problem you are facing. Should this issue be closed then?

yes sure it can be closed, but it is nice i think to make it optional to add or not, maybe someone want to use the default 4200 and it is hard coded in his settings.

My usecase for a specific port for UseAngularCliServer:
I have one multi-tenant Angular-app which is served on different base-urls, so each tenant has its own url like http://localhost/tenant-1 and http://localhost/tenant-2.
Since the index.html is generated by ng I have to read it from the ng-server to serve it with a MVC controller on the different routes. I can't use the from a ng build pre-generated ClientApp/dist/tenant/index.html because on ng serve there are only script tags like runtime.js instead of runtime-es5.js and runtime-es2015.js.
The index.html contains no base href, it is generated at runtime by the angular app with { provide: APP_BASE_HREF, useValue: "/" + (window.location.pathname.split("/")[1] || "") }
The correct urls for the script-tags are ensured with a "deployUrl": "/tenant/" in angular.json and there's also the port-setting in serve:options.
routes.MapRoute(
   name: "tenantapp",
   template: "{urlprefix}/{*urlargs}",
   defaults: new { controller = "View", action = "Index" });

...

private IHostingEnvironment _env;

public async Task<IActionResult> Index(string urlprefix, string urlargs)
{
  if (_env.IsDevelopment())
  {
    var proxy = new System.Net.Http.HttpClient();
    var index = await proxy.GetAsync("http://localhost:4302/tenant/index.html");
    var indexStream = await index.Content.ReadAsStreamAsync();
    return File(indexStream, "text/html");
  }
  return PhysicalFile(System.IO.Path.Combine(_env.ContentRootPath, "ClientApp", "dist", "tenant", "index.html"), "text/html");
}

Currently this only works with UseProxyToSpaDevelopmentServer where I can define the specific port.
Additionally there are more Angular apps in this project, which are all served on there own ports.
At least disabling the use of a dynamic port would help.


How can I implement a music list ?

Like Spotify does.

Not a dumb question, but maybe a bit broad! :)
You should be able to use a Scrollable with many instances of Row to implement such a list. The todos example showcases something similar.
If you need more help, I recommend you to share more details about your use case here or join our Zulip server!

As Héctor said the question is a little bit broad. ;)
Accordingly, I would recommend that when you are faced with a big problem, you divide it into smaller problems.
Smaller problems can usually be solved much faster.
Let's assume we only want to display the title and the duration (in seconds) of a song. We could use a simple tuple for this: (String, u32). Since we are working with more than one song in a playlist we would need a collection like a Vector.
With this, our data would be as follows:
struct MusicPlayer {
    playlist: Vec<(String, u32)>,
}

As mentioned earlyer we could use a Row to show the title and duration grouped in one line like this:
Row::new()
    .push(Text::new(title))
    .push(Text::new(format_duration(duration)))

Since we are storing the duration in seconds, we would need a way to format it to mm:ss. This is up to you! :)
Now that we have a way to display one song, we would need a way to display all songs from the playlist. We could use a Column for this. To add all songs from the Vector to a new Column we could use the fold(...) function of the Iterator. It could look like this:
playlist.iter_mut()
    .fold(Column::new(), | column, (title, duration) | {
        column.push(
            // Row ...
        )
    })

After solving the small problems we can now face the big one and combine everything together:
use iced::{ Element, Sandbox, Settings, Text, Container, Column, Row, Scrollable, scrollable, Length };

struct MusicPlayer {
    playlist: Vec<(String, u32)>,
    scroll: scrollable::State,
}

impl Sandbox for MusicPlayer {
    type Message = ();

    fn new() -> Self {
        MusicPlayer {
            playlist: vec!(
                (String::from("Foo"), 123),
                (String::from("Bar"), 456),
                (String::from("Baz"), 789),
            ),
            scroll: scrollable::State::new(),
        }
    }

    fn title(&self) -> String {
        String::from("Music Player")
    }

    fn update(&mut self, _: Self::Message) {

    }

    fn view(&mut self) -> Element<Self::Message> {
        let playlist: Element<_> = self.playlist.iter_mut()
            .enumerate()
            .fold(Column::new().spacing(10), |column, (i, (title, duration))| {
                column.push(
                    Row::new()
                        .spacing(20)
                        .push(Text::new(format!("{}.", i+1)))
                        .push(Text::new(title.to_owned()))
                        .push(Text::new(format_duration(duration)))
                )
            })
            .into();

        let scrollable: Element<_> = Scrollable::new(&mut self.scroll)
            .push(playlist)
            .into();

        Container::new(scrollable)
            .width(Length::Fill)
            .height(Length::Fill)
            .center_x()
            .center_y()
            .into()
    }
}

fn format_duration(seconds: &u32) -> String {
    // up to you ;)
}

fn main() {
    MusicPlayer::run(Settings::default())
}

This will give us this nice little list:

Of course this is only the foundation and there would be a lot more to do and maybe you would like to replace the tuple by a struct, but I hope I could help you out. If you have any questions, don't hesitate! :)


issue retrieving custom translations

Discussed in https://github.com/Intevel/nuxt-directus/discussions/221

Originally posted by lumos94 November 23, 2023
Hello all,
I am trying to retrieve some custom translations from Directus.  I am getting an error back saying that You don't have permission to access this.
Permissions are checked and I indeed have permission to access it (it works ok on another project I have which uses their SDK). Other collections are retrieved as expected.
const fetchFormTranslationss = async () => {
  try {
    const items = await getItems<FormTranslations>({
      collection: "directus_translations",
    });
  } catch (e) {
    console.error("Error fetching translations:", e);
  }
}
</div>

Using Postman I call the https://my-directus.com/translations endpoint and I retrieve the translations ok. The module calls the https://my-directus.com/items/translations(using theuseDirectusItems` composable) that is why I am getting the above mentioned error. Is there a way that I can call the translations endpoint directly?

@Intevel  any updates on this?

@Intevel  any updates on this?

We are currently testing a rewrite of the nuxt-directus module that is based on the Directus SDK.
You can find more information here.
If you have the time could you test that one out?


lang/elm: first attempt

Hi @hlissner
Would you be interested in user contributions? I made this tiny elm module, it was very easy thanks to your organization!
I'm not an e-lisper, so my code might be clunky...

Looks good, thanks! And yes, I welcome any user contributions.


[NEW]Please support restore command

Migration tools like redisshake usually use restore command to realize migration from redis to kvrocks。
Please support restore command。

Thanks, I MUST say that it is hard for us to implement redis RESOTRE command, since the restored value is the serialization of redis key value i.e. RDB format , so we need to support parsing all redis types, moreover, the encoding of key value serialization also would change in the future.
I think we shouldn't do this in kvrocks core code. Maybe there is a tool which can change RESTORE value to redis command, or we use other tools?

Twemproxy and Codis also have the same issue, and RedisShake supports parsing the restore command to AOF: https://github.com/alibaba/RedisShake/wiki/FAQ#q-does-redisshake-supports-codis-and-twemproxy

restore was added in #1684, will be released in the next version (or try it in the unsable branch)


[SPARK-38270][SQL] Spark SQL CLI's AM should keep same exit code with client side

What changes were proposed in this pull request?
Current Spark SQL CLI alway use  shutdown hook to stop SparkSQLEnv
 // Clean up after we exit
    ShutdownHookManager.addShutdownHook { () => SparkSQLEnv.stop() }

but use process ret to close client side jvm
while (line != null) {
  if (!line.startsWith("--")) {
    if (prefix.nonEmpty) {
      prefix += '\n'
     }
    if (line.trim().endsWith(";") && !line.trim().endsWith("\\;")) {
      line = prefix + line
      ret = cli.processLine(line, true)
      prefix = ""
      currentPrompt = promptWithCurrentDB
    } else {
      prefix = prefix + line
      currentPrompt = continuedPromptWithDBSpaces
    }
  }
  line = reader.readLine(currentPrompt + "> ")
}

sessionState.close()

System.exit(ret)
}

    if (sessionState.execString != null) {
      exitCode = cli.processLine(sessionState.execString)
      System.exit(exitCode)
    }

    try {
      if (sessionState.fileName != null) {
        exitCode = cli.processFile(sessionState.fileName)
        System.exit(exitCode)
      }
    } catch {
      case e: FileNotFoundException =>
        logError(s"Could not open input file for reading. (${e.getMessage})")
        exitCode = 3
        System.exit(exitCode)
    }

This always cause client side exit code not consistent with AM.
IN this pr I prefer to pass the exit code to SparkContext.stop() method to pass a clear client side status to AM side in client mode.
In this pr, I add a new stop method in SchedulerBackend
def stop(exitCode: Int): Unit = stop()

So we don't need to implement it for all kinds of scheduler backend.
In this pr, we only handle the case me meet for YarnClientSchedulerBackend, then we can only implement stop(exitCode: Int) for this class, then for yarn client mode, it can work now.
I think this can benefit many similar case in future.
Why are the changes needed?
Keep client side status consistent  with AM side
Does this PR introduce any user-facing change?
With this pr, client side status will be same with am side
How was this patch tested?
MT

IN this pr I prefer to pass the exit code to SparkContext.stop() method to pass a clear cliet side status to AM side in client mode. And I think this can benefit many similar case in future.

cliet -> client

IN this pr I prefer to pass the exit code to SparkContext.stop() method to pass a clear cliet side status to AM side in client mode. And I think this can benefit many similar case in future.

cliet -> client

updated

gentle ping @cloud-fan @tgravescs @mridulm

cc @bogdanghit

gentle ping @cloud-fan @bogdanghit

@AngersZhuuuu does this address the CliSuite flakiness I reported a while back?

@AngersZhuuuu does this address the CliSuite flakiness I reported a while back?

Can you provide the link？  Literally, it should be irrelevant

gentle ping @bogdanghit

@AngersZhuuuu can you rebase? We should merge this PR.

+1, and sorry for not merging it after my approval

@AngersZhuuuu can you rebase? We should merge this PR.

Done

ping @cloud-fan @yaooqinn @LuciferYang

thanks, merging to master!