Erroneous guessing of unit names

Iris guesses the name of a cubes unit for use in quickplotting (and possibly) elsewhere. However, this is sometimes incorrect e.g. units of m2 s-2 can be Grays, but equally can be Energy Transfer. There should be a way of manually overwriting the unit name in a cube if the default is wrong.

... 6 years later...
This has just come up for me.  I was also not expecting Grays when I squared my wind speeds!  I can replace my unit on the cube with 'm**2 / s**2', but it will still come up as 'Gy' on a quickplot.  I note that the Unit string method uses self.origin or self.symbol.  Perhaps quickplot should defer to str(cube.units).  Then users would see something consistent when they print and plot the cube, and be able to "fix" the unit with a string of their choice, i.e:
cube.units = 'units wot I wanted'

@rcomer Do you have a code except that highlights this issue? It would be nice to point to a concrete example... you never know, we might then actually fix it :blush:

OK, suppose I have a cube of wind speeds:
import iris
import iris.quickplot as qplt
import matplotlib.pyplot as plt
import cf_units

for module in [iris, cf_units]:
    print(module.__name__, module.__version__)

cube = iris.cube.Cube(range(5), standard_name='wind_speed', units='m/s')
coord = iris.coords.DimCoord(range(5), standard_name='time',
                             units='days since 2019-07-07')

cube.add_dim_coord(coord, 0)

print(cube)
qplt.plot(cube)

Everything looks sensible:
iris 2.2.0
cf_units 2.0.1
wind_speed / (m/s)                  (time: 5)
     Dimension coordinates:
          time                           x

Let's look at the square:
new_cube = cube ** 2

print(new_cube)
qplt.plot(new_cube)

Now we get:
unknown / (Gy)                      (time: 5)
     Dimension coordinates:
          time

Metadata looks wrong, but we can fix that, right?
new_cube.rename('wind_speed_squared')
new_cube.convert_units('m2/s2')

print(new_cube)
qplt.plot(new_cube)

Now the print out of my cube looks how I expect, but the plot still shows Grays on the y-axis:
wind_speed_squared / (m2/s2)        (time: 5)
     Dimension coordinates:
          time                           x

My "real" example involves eddy diagnostics u'**2 + v'**2 and u'**2 - v'**2, where ' indicates band pass filtering.  I don't think we want to go there...
Admittedly this isn't a high priority issue, as I can always change the plot labels myself rather than relying on quickplot.

Is this still an issue in iris 3.0.1? For now, I'm closing this and we should reopen a new one if it's still a problem.

I can still reproduce the issue under Iris 3.0.1 but neither @niallrobinson nor myself ever felt strongly enough about it to open a PR, so it's probably not important.  I'm happy to leave it closed.

Thanks for the feedback @rcomer, it's pretty straightforward to reopen this issue again if we find we have a strong reason to!


Overflow on userinfo with long name

Describe the bug
↓
Steps to reproduce
https://primal.net/e/note1yhg2rygp2936rx82gugnr4qtnpqm6f6ky0ugnyjgcy6vzmucs5dqkn0f7j

Expected behavior
No overflow
Environment
web app on chrome
Additional context
N/A

Thank you for reporting this. It should be fixed now.


Copy-Item to Remote session from a write protected medium throws exception

Steps to reproduce
# Mount a CD ISO or insert a CD to drive.
# Go to the drives folder:
cd G:
# Establish a remote session (In my case a running Nano Server vm):
$ip=[ip address]
Set-Item WSMan:\localhost\Client\TrustedHosts $ip
$s = New-PSSession -ComputerName $ip -Credential ~\Administrator
#Copy over the file to the remote system:
Copy-Item -ToSession $s -Path .\Filename.example c:\

Expected behavior
No Error messages/Exceptions are shown in case there is no valid reason.
Actual behavior
In case the file is within a mounted ISO, I get the following error message:
Copy-Item : The parameter is incorrect
At line:1 char:1
+ Copy-Item -ToSession $s -Path .\Filename.example c:\
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Copy-Item], Win32Exception
    + FullyQualifiedErrorId : System.ComponentModel.Win32Exception,Microsoft.PowerShell.Commands.CopyItemCommand

The file is still successfully copied to the remote system.
Doing the same from a normal hard drive does not show the exception.
Environment data

Reproducible on Official installed Release:
Name                           Value
----                           -----
PSVersion                      5.1.14393.953
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.953
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

And latest Github Release:
Name                           Value
----                           -----
PSVersion                      6.0.0-alpha
CLRVersion
PSRemotingProtocolVersion      2.3
PSEdition                      Core
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0
BuildVersion                   <IP_ADDRESS>
GitCommitId                    v6.0.0-alpha.18

@PaulHigin Could you please comment the Issue?

I don't know what is causing this error, but I agree that it shouldn't be reported if the copy succeeds.  It needs to be investigated.

@PaulHigin Thanks for your comment. It seemed to me that we had the same Issue.

I have exactly the same issue, same error reported but the file has been successfully copied

Can you repo if remote computer is localhost?

I tried to copy a file from my local computer to a remote. the file has been copied successully to the remote computer but still have the mentioned error.

I have same issue.

I also have the same issues when copy file to remote server via PSSession variable. File .zip was transferred successfully to remote server but still showing error

(See #16310 for possible test with Mount-DiskImage -Access ReadOnly)

Same error, I try to copy file from network directory (VMware Share) to remote server :
PS> Get-FileHash -Algorithm MD5 -Path "\\vmware-host\Shared Folders\ShareVm\psscript\INSTALL2\Certs\hello.txt"

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
MD5             5D41402ABC4B2A76B9719D911017C592                                       \\vmware-host\Shared Folders\ShareVm\psscript\INSTALL2\Certs\hello.txt

PS > Copy-Item "\\vmware-host\Shared Folders\ShareVm\psscript\INSTALL2\Certs\hello.txt" "c:\ded.dd" -ToSession $ss -Force
Copy-Item : The parameter is incorrect
At line:1 char:1
+ Copy-Item "\\vmware-host\Shared Folders\ShareVm\psscript\INSTALL2\Cer ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Copy-Item], Win32Exception
    + FullyQualifiedErrorId : System.ComponentModel.Win32Exception,Microsoft.PowerShell.Commands.CopyItemCommand

PS> $Error[0].Exception | select *
NativeErrorCode : 87 (0x57)
ErrorCode       : -2147467259 (0x80004005)
Message         : The parameter is incorrect
Data            : {}
InnerException  :
TargetSite      : System.Collections.Generic.List`1[System.Management.Automation.Internal.AlternateStreamData] GetStreams(System.String)
StackTrace      :    at System.Management.Automation.Internal.AlternateDataStreamUtilities.GetStreams(String path)
                     at Microsoft.PowerShell.Commands.FileSystemProvider.PerformCopyFileToRemoteSession(FileInfo file, String destinationPath, PowerShell ps)
                     at Microsoft.PowerShell.Commands.FileSystemProvider.CopyFileInfoItem(FileInfo file, String destinationPath, Boolean force, PowerShell ps)
                     at Microsoft.PowerShell.Commands.FileSystemProvider.CopyItemLocalOrToSession(String path, String destinationPath, Boolean recurse, Boolean
                  Force, PowerShell ps)
                     at Microsoft.PowerShell.Commands.FileSystemProvider.CopyItem(String path, String destinationPath, Boolean recurse)
                     at System.Management.Automation.SessionStateInternal.CopyItem(CmdletProvider providerInstance, String path, String copyPath, Boolean recurse,
                  CmdletProviderContext context)
HelpLink        :
Source          : System.Management.Automation
HResult         : -2147467259

However, on my remote machine the file is well copied :
PS C:\> Get-FileHash -Algorithm MD5 .\ded.dd

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
MD5             5D41402ABC4B2A76B9719D911017C592                                       C:\ded.dd

Local:
$PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.17763.592
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.592
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

WindowsProductName                      WindowsVersion OsHardwareAbstractionLayer
------------------                      -------------- --------------------------
Windows Server 2019 Standard Evaluation 1809           10.0.17763.737

Remote:
$PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.17763.592
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.592
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

WindowsProductName                      WindowsVersion OsHardwareAbstractionLayer
------------------                      -------------- --------------------------
Windows Server 2019 Standard Evaluation 1809           10.0.17763.737


Data returns 0x0000... in contracts that use libraries that are created with .new

Is there anything special to know about using MyContract.new(...) for contracts that use libraries? I'm having an issue where the deployed contract is working but a manual new does not properly link (specifically, data stored via the library, such as with the using ... for syntax, is coming back as 0x0000...).
This works:
// getData is a getter that retrieves data stored via a library
truffle(default)> MyContract.deployed().getData() // prints correct data

This returns 0x0000... for all values stored using the library:
// this should print the exact same data (different contract instance, but initialized the same way and same getter)
truffle(default)> MyContract.new().then(contract => contract.getData().then(console.log)) // prints 0x0000...

Unless I am missing something, this seems to be a bug. I can provide more details but I wanted to check first if I was missing something obvious.
Issue
Undeployed contracts are not linked properly.
Steps to Reproduce

Deploy a contract that uses a library to store data
Call a getter to see that data returns 0x0000...

(I can provide more details if necessary)
Expected Behavior
Expect data to be stored and retrieved successfully.
Actual Results
Data stored with library returns 0x0000... Data stored using the main contract without any library functions works fine.
Environment

Operating System: OSX El Capitan
Truffle version: 3.0.0-2
Ethereum client: TestRPC 3.0.0
node version: 6.1.0
npm version: 3.8.6

Hey @raineorshine, my gut reaction is that you need to call deployer.link() or deployer.autolink() in your migrations. Truffle won't link these for you automatically unless you deploy a contract through deployer.deploy(). In order to ensure the correct linkage, you have to write that into your migration.
Say I have MyContract which relies on MyLibrary, and say I plan to have users create an instance of MyContract as they use my dapp (rather than deploy MyContract as a singleton). The initial migration, then, would look something like this:
1_initial_migration.js
module.exports = function(deployer) {
  deployer.deploy(Migrations);
  deployer.link(MyLibrary, MyContract);
};

That last line will edit the build artifacts to ensure MyContract is linked to MyLibrary properly.

Thank you, I am trying that out right now!
I am getting confused though as it does not seem to be happening anymore, and now I am getting Invalid JSON RPC response: undefined. It occurs with MyContract.new(...). Other methods like web3.eth.accounts and MyContract.at(...) work fine. It appears that the contract is deployed successfully in testrpc.
Any ideas?

Very interesting. undefined suggests a fairly big, uncaught issue. How are you using the TestRPC? Is it as a server or a library?

server

Possibly related to https://github.com/ConsenSys/ether-pudding/issues/39?

The intermediary function in MyContract.sol.js is called once with no error and no address set, and then it crashes somewhere inside web3.

Is there some kind of compatibility issue with web3 v0.17.0-beta?

There could be. I haven't tried that version yet, so it sounds plausible.

Downgraded to v0.16.0. Works now.
Sorry this issue wandered from its original stated problem.
How should we proceed with documenting/identifying the 0.17.0-beta incompatibility? If it's a web3 bug, just skipping 0.17.0-beta might be smart.

I think it's because Truffle/EtherPudding uses 0.16.x. Is it automatically updating to the -beta version, or did you do that intentionally?
No problem, by the way. This is good information to know.

npm will not install 0.17.0-beta automatically if the specified version range is ^0.16.0, since the -beta is there.
truffle#beta specifies web3 ^0.16.0.
truffle#beta specifies ether-pudding ^3.2.0.
ether-pudding#0.16.0 specifies web3 ^0.15.3.
That's all fine. I think the problem is that I am explicitly using web3 0.17.0-beta in my project. I guess there is a peer dependency to use the same version of web3 that is used by truffle.


How to generate coverage properly

Hello,
I am having a typescript - angular - systemjs project and I am using it with karma-systemjs.
I have managed for it to work in loading the files needed properly when running karma tests, but I just cannot get it to work for karma-coverage.
karma.config.js:
module.exports = function (config) {
    config.set({
        basePath: './',
        files: [
            './Scripts/UnitTests/Tests/Controllers/**.js',
        ],
        exclude: [],
        frameworks: [
            "systemjs",
            "jasmine"
        ],
        logLevel: config.LOG_DEBUG,
        plugins: [
            'karma-coverage',
            'karma-systemjs',
            'karma-phantomjs-launcher',
            'karma-chrome-launcher',
            'karma-jasmine'
        ],
        coverageReporter: {
            type: 'json',
            subdir: '.',
            file: 'coverage-final.json'
        },
        reporters: ['progress', 'coverage'],
        preprocessors: {
            'app/**/!(*.spec).js': 'coverage'
        },
        browsers: ['Chrome'],
        autoWatch: true,
        singleRun: false,
        systemjs: {
            includeFiles: [
                './Scripts/Libs/angular/angular.min.js',
                './Scripts/Libs/angular-mocks/angular-mocks.js',
                './Scripts/Libs/lodash/dist/lodash.min.js'
            ],
            serveFiles: [
                'Scripts/Controllers/*.js',
            ],
            configFile: './system.config.js',
            config: {
                packages: {
                    'app': {
                        defaultExtension: 'js'
                    }
                }
            },
        }
    });
};

system.config.js:
System.config({
    baseURL: './',
    defaultJSExtensions: true,
    map: {
        'systemjs': 'node_modules/systemjs/dist/system.js',
        'system-polyfills': 'node_modules/systemjs/dist/system-polyfills.js',
        'typescript': "node_modules/typescript/lib/typescript.js",
        'es6-module-loader': 'node_modules/es6-module-loader/dist/es6-module-loader.js'
    },
    meta: {
        'app/*': { format: 'register' }
    },
    transpiler: "typescript"
});

if i change the preprocessors with the correct folder, for example
'Scripts/Controllers/*.js': 'coverage'
I get the error DashboardController.js interpreted as global module format, but called System.register.
Error loading DashboardControllerTester.js
Any suggestions?

I personally haven't had any success with Typescript and code coverage.
Part of the problem being that Istanbul (the tool which instruments the code to track it's coverage) needs to parse the code. By default it uses Esprima which can do ES5 (though it claims to handle ES6 too now).
You can configure it to use a different parser (eg. Isparta used to be the best option for ES6), but I'm not aware of one that will work with Typescript.
So the only solution I know of is to pre-compile your Typescript to Javascript before testing, but that completely defeats the point of having SystemJS call the transpiler for you.
I'm open to any bright ideas though.

Hey, I understand what you are saying. Perhaps if we define as default extension the ts files and use karma-typescript-preprocessor but don't have time to test it at the moment. Thanks anyway for the fast response! :)

I have been playing with the preprocessors but didn't manage to do it :/


⚠ Samsung has degraded performance

In 20debb0, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 3514 ms

Resolved: Samsung performance has improved in 28fdefe.


Install SSH Key not working on MacOs

Description
Hello
I need to push manually with shell on mac OS VM on Azure git, but it failed :

The script use can be found in repro step, the real problem is that script work fine on ubuntu-latest
I have some test with ssh and if i disable the strict host key check it's work fine.
Platforms affected

[X] Azure DevOps
[ ] GitHub Actions - Standard Runners
[ ] GitHub Actions - Larger Runners

Runner images affected

[ ] Ubuntu 20.04
[ ] Ubuntu 22.04
[ ] macOS 11
[X] macOS 12
[ ] macOS 13
[ ] Windows Server 2019
[ ] Windows Server 2022

Image version and build link
Runner Image
Image: macos-12
Version: 20230709.1
Is it regression?
No
Expected behavior
This ssh key and the know host was well added on MacOs
Actual behavior
[libelles 8229bb2] test
 1 file changed, 1 insertion(+)
 create mode 100644 test.txt
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
##[error]Bash exited with code '128'.
Finishing: Bash

Repro steps

Change the git url in second bash script
Try change vmImage by ubuntu-latest and it's work fine

trigger:
  - master
variables:
  - group: Pipelines-ssh

jobs:
  - job: Build
    pool:
      vmImage: 'macOS-latest'
    steps:
      - task: InstallSSHKey@0
        inputs:
          knownHostsEntry: '$(host)'
          sshPublicKey: $(pubKey)
          sshKeySecureFile: id_rsa
      # Change git information
      - task: Bash@3
        inputs:
          targetType: inline
          script: |
            git checkout -b libelles;
            git branch -m libelles;

            echo "Email initial : $(Build.RequestedForEmail)";
            email="$(Build.RequestedForEmail)";

            echo "Email final : $email";
            git config --global user.email "$email";

            echo "Nom initial : $(Build.RequestedFor)";
            name="$(Build.RequestedFor)";

            echo "Nom final : $name";
            git config --global user.name "$name";
      # Create file and push it
      # Change git url
      - task: Bash@3
        inputs:
          targetType: inline
          script: |
            echo "toto" > test.txt;
            git add *;
            git commit -m "test";
            git remote set-url origin git@ssh.dev.azure.com:v3/*****/******/**************;
            git push --set-upstream origin libelles

@florentm35 , while I'm checking the issue, small nitpick - did you try to skip installing ssh.dev.azure.com host keys ?
we add those keys already.
but anyway, installing them should not be harmful.

If i skip the ssh key install it's not working :

Seem legit, it's SSL key use for the git authentifcation

"publickey" looks ambiguous


Add menu item 'Search handshape' under 'Signs'

...and put it below "Search morpheme"
(I can add the menu text myself, but I cannot link it to https://signbank.science.ru.nl/dictionary/handshapes/, hence the request)

Done. Happy, @ocrasborn ?

Could you still hide the menu item for anonymous users, @Woseseltops ?

Of course, done.


test(CreateClientUseCase): add tests

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+2.5%) to 11.532%

Totals

Change from base Build<PHONE_NUMBER>:
2.5%

Covered Lines:
187

Relevant Lines:
1750

💛  - Coveralls


Bump oleaut32-sys

Please push a current version of oleaut32-sys to crates.io. Currently on crates.io there's only a placeholder.

oleaut32-sys = "0.2.0"


Assignee menu text is overwritten when typing name

Gogs version (or commit ref): last stable <IP_ADDRESS>9
Git version:
Operating system: OpenSuSE
Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] SQLite

Can you reproduce the bug at https://try.gogs.io:

[x] Yes
[ ] No
[ ] Not relevant

Log gist:

Description
When creating a new issue, and assigning somebody to it, the text "Assignee" is overwritten by the entry, when typing his name (or at least the first letter of his name).
Then the assignee is written twice in that place. Once where the text "Assignee" stood before, and the assignee himself below.
When clearing the assignee entry again, still the "Assignee" text before with the gear symbol next to it is gone. Either the formerly assigned assignee is still written there, or the text "clear assignee"
Especially when a lot of people are working on a project this list can be quite long, so typing is the faster way than searching in the list manually.

The same behaviour can also be seen with Labels and Milestone Menus.

the text "Assignee" is overwritten by the entry
What entry?

Hi, can you provide some screenshot because I'm pretty much confused.

Here's a gif:

Basically what you try to select by typing overwrites the controller name (in this case "Labels").

@markainick thanks, is it on demo site?

Yes, for both labels and assignees.

What browser are you using? I can't reproduce this on demo site using Chrome 56 on Mac OS X.

I'm using mainly Firefox on Linux, but just tried it in Chrome and it happens there too.
Just cross checked, it's the same behavior with Firefox and Internet Explorer on Windows.
Maybe our explanation wasn't accurate enough, sorry about that. I'll try again:
When clicking on the gear symbol to open the list of labels, milestones or assignees, you have two possibilities to select entries: selecting with mouse, or starting to type the first characters of an entry on your keyboard.
When using the second possibility, as soon as you start typing the text of the item, or only the first letter of it, the entry in the list is "highlighted" with bold font (but neither selected nor checked) and the text next to the gear symbol is overwritten by the text of this entry. The gear symbol vanishes, too.

I see, I think this is caused by default behavior of Semantic UI, should handle that.

Was looking into this and it happens on the menu items on the upper right of the page that has the list of issues. Currently looking into a fix for it.

@spectre013 can you take a screenshot?

@Unknwon fixed it and did a pull request. to see the bug go to try.gogs.io click on sort and type "o" and you will see the label item change. It only happens when you type.

@Unknwon if you select "clear labels" the server gives 500 error.
The request:
Request URL:https://try.gogs.io/testador/test/issues/1/label
Request Method:POST
Status Code:500
Remote Address:<IP_ADDRESS>:443
Referrer Policy:no-referrer
The post content:
_csrf:xS9qWwf3zh8qQp-RUn8RISJlCg46MTQ5MDIwMjIxNTI4MjMzMDEwNg==
action:clear
id:

@ogarcia I was able to reproduce, but after an update for another issue on demo site, it's impossible to 500 anymore.


adapt to v4 namespace

Signed-off-by: Maximilian Meister<EMAIL_ADDRESS>

You missed the patch file in the spec file:
https://github.com/kubic-project/caasp-container-manifests/compare/use_caasp_v4_images?expand=1


color.createScheme() method not working

createScheme method is an empty function.
createScheme should return a new scheme using the color as main color for the new scheme.
createScheme should accept a scheme configuration object or a name of onw of the configuration objects in the preset.

fixed


Focus indication for Tabs in Chrome

Until we implement #790, we're relying on default styles provided by browsers for indicating focus.
Currently, the focus styles in chrome are partially hidden, which could make it difficult for sighted keyboard users to figure out which tab has focus.

It looks like this issue is related to this overflow-x property

@jgiardino Just to clarify - is the desired behavior that the blue focus ring is shown completely circling the tab? Or just to make sure that both left and right are shown?

I'm not sure what our options are at this point. If it's super easy to get just left and right highlights, but a major rework to get all around, then maybe we go with the super easy for now, and bring this up with the css and visual design teams to figure out what we want moving forward.
Do you have an idea yet how much of a change it would require to get all around? Or even just showing sides and top?

No :-/ It's complicated with the scrolling and overflow and the vertical height with/without the scrollbar. Before I went too deep, I wanted to know for sure where to head.

Copying @mceledonia on this to loop him in. I'm curious what his thoughts are on implementing a custom focus style for this.
Custom focus styles might be something we can consider depending on what our options are for enabling the default focus styles to display.

I am definitely interested in an overall approach of utilizing default styles when we can, and using our custom style as a backup for when we can't.
If we can use a default focus style here for tabs we should go for it.

:tada: This issue has been resolved in version 2.4.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


⚠ Bofrost.lu has degraded performance

In 8937c15, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1905 ms

Resolved: Bofrost.lu performance has improved in bd68609 after 29 minutes.


Build a token contract

https://docs.substrate.io/tutorials/smart-contracts/build-a-token-contract/

Contract at https://github.com/evilrobot-01/substrate-contracts/tree/erc20


Add Acast Access pattern

Hi! It's not clear to me whether these are Regex's or Regular Expressions and what flavor they need to take - I am assuming whatever engines are being used have PCRE-style support for lookaheads.

Thanks Chris!

Hi! It's not clear to me whether these are Regex's or Regular Expressions and what flavor they need to take - I am assuming whatever engines are being used have PCRE-style support for lookaheads.

Yes, PCRE is what is intended.


Handling exception thrown by handlers

I am evaluating the library for an internal use at my company.
One issue I have currently is that if any handler throws an exception, it will prevent other handlers from receiving the same event. While that is currently the same behavior of normal .NET events, it can cause issues.
Would you consider an overloaded method for  WeakEventSource<TEventArgs>.Raise that takes a delegate that can handle any exception and decide whether event notification should continue or not.
Something like:
public void Raise(object? sender, TEventArgs args, Func<Exception, bool> exceptionHandler)
{
    var validHandlers = GetValidHandlers(_handlers);
    foreach (var handler in validHandlers)
    {
        try
        {
            handler.Invoke(sender, args);
        }
        catch (Exception ex)
        {
            if (exceptionHandler(ex))
                continue;
            throw;
        }
    }
}

I was also thinking of a more advanced handler that returns a state so that bad subscriber can be automatically unsubscribed:
[Flags]
public enum ErrorHandlingFlags
{
    None = 0,
    NotThrow = 1,
    Unsubscribe = 2,
}

public void Raise(object? sender, TEventArgs args, Func<Exception, ErrorHandlingFlags> exceptionHandler)
{
    var validHandlers = GetValidHandlers(_handlers);
    foreach (var handler in validHandlers)
    {
        try
        {
            handler.Invoke(sender, args);
        }
        catch (Exception ex)
        {
            var flag = exceptionHandler(ex);
            if (flag.HasFlag(ErrorHandlingFlags.Unsubscribe))
            {
                // find a way to unsubscribe
            }

            if (flag.HasFlag(ErrorHandlingFlags.NotThrow))
                continue;

            throw;
        }
    }
}

But I'm not sure how the unsubscription can work without the original EventHandler<TEventArgs>.

Hi @Kryptos-FR,
Thanks for your interest!
It's an interesting idea. One of the design goals of this project was to mimic the behavior of normal .NET events as closely as possible, so I never really considered this. But there's a major difference: unlike normal .NET events, you can't access the list of individual handlers of a WeakEventSource, so there's currently no way to catch errors thrown by individual handlers. So I think your idea would be a useful addition.

I was also thinking of a more advanced handler that returns a state so that bad subscriber can be automatically unsubscribed:
...
But I'm not sure how the unsubscription can work without the original EventHandler<TEventArgs>.

Yes, it would be an interesting option. However I can't think of a good way to do that... I'll look into it.

Oh, by the way, how do you see this working for async events? Would the exceptionHandler be asynchronous as well?

Oh, by the way, how do you see this working for async events? Would the exceptionHandler be asynchronous as well?

I would probably not suggest having the exceptions handler be asynchronous. I don't know the exact impact but async in catch block probably adds quite a lot of codegen, and is probably not cheap. That aside, what would be the case for the handler to be async? I believe a good handler should return as quickly as possible.
I was not considering that scenario since I don't need it for my case, so I might be missing something.

I don't know the exact impact but async in catch block probably adds quite a lot of codegen, and is probably not cheap.

Minimal impact. There's no codegen involved. Only drawback is that I can't use an exception filter in the async case, but that's not a major issue.

That aside, what would be the case for the handler to be async?

Maybe you need to log the error to a database, or something like that. I expect most exception handlers to return synchronously, but it's annoying when you're in a situation where you need to do something asynchronous, and you can't because you must return synchronously.
I could make it return a ValueTask to reduce the cost when the exception handler returns synchronously.

I believe a good handler should return as quickly as possible.

That's a good point.
On one hand, making the exception handler synchronous would probably limit the cases where this feature is usable... On the other hand, there's an easy workaround if the handler is synchronous:
List<Exception> errors = null;
await _weakEventSource.RaiseAsync(sender, args, ex =>
{
    (errors ??= new List<Exception>()).Add(ex);
    return ExceptionHandlingFlags.Handled | ExceptionHandlingFlags.Unsubscribe;
});
if (errors != null)
{
    // Asynchronously process the errors
}

I need to think more about it...

Anyway, the good news is that I found a way to unsubscribe automatically (still need to test, though). It involves significant internal changes, but I think they're good ones.

@thomaslevesque Do you have an ETA?
I can always build my own version, but I'd preferred to keep an "official" package.
Take your time though 😃. I just need to make a decision on that by the end of next week. So if you already know that it won't be possible for you, I will go ahead a make that private version.
Thanks.

Hi @Kryptos-FR,
I have most of the code sitting on my laptop, but I haven't had time to come back to it since last week. I'll try to finish it in the next few days.

@Kryptos-FR I just published a beta (4.1.0-beta.1). Could you please try it and see if it works for you?
The changes are quite extensive (mostly because of the "unsubscribe" feature), so I think a beta phase is needed to ensure I didn't break anything.
I ended up adding 2 overloads for AsyncWeakEventSource: one with a sync exception handler, and one with an async one.

I'm having second thoughts about the unsubscribe feature... It can cause incorrect behavior in some scenarios.
For instance, when subscribing with a multicast delegate:

you subscribe with a multicast delegate ABC (made up of handlers A, B, and C
the event is raised, and handler B throws. The exception handler returns Handled | Unsubscribe, so B is removed from the handlers.
now the handler list contains only AC
you try to unsubscribe ABC: but the handler list no longer contains ABC, it just contains AC, so nothing happens (because of the change implemented in #13). So you can no longer unsubscribe ABC. You'd have to unsubscribe A and C separately, but you can't do that, because you don't know that ABC is no longer in the list.
So basically, the "unsubscribe on throw" feature means that multicast delegates can be "broken" if one of their individual handlers throw.

Another issue is that the event publisher shouldn't remove subscribers unilaterally, because the subscriber wouldn't know it's been unsubscribed.
So I'm going to remove the "unsubscribe" part of this feature. It's not released yet (except in a beta), so there's no harm done. The exception handler will just return true to indicate that it handled the exception.

@thomaslevesque Awesome. I don't have access to the code outside of the office, but I will test it first thing next Monday.

Beta-1 with unsubscription feature better fit my needs. So I will stick with it for the time being. Thanks for your hard work!

Hi @Kryptos-FR,
Do what you like, but keep in mind that I won't maintain the unsubscribe feature, so if there's something wrong with it, you're on your own :wink:

About your "second thought" comment. Could that be explained in the documentation instead?

What kind of explanation would you expect? It's the same as with normal events, really; a subscriber typically doesn't expect to be removed by the publisher just because it threw an exception. The subscriber should catch its own exception, and unsubscribe itself if appropriate.

If one doesn't use multicast delegate, the ability to unsubscribe is nice to have.

I know, multicast is a bit of a corner case, and probably not many people use it. But I don't want to introduce a feature that I know won't work in some cases...


build errors with android studio 1.5.1

With a fresh  android studio install on archlinux:

android studio 1.5.1
gradle 2.8
OpenJDK java version "1.7.0_95"

1 - The following error occurs when making the project:
Error:Execution failed for task ':TextToSpeechFacade:processDebugAndroidTestManifest'.
> java.lang.RuntimeException: Manifest merger failed : uses-sdk:minSdkVersion 3 cannot be smaller than version 4 declared in library [ScriptingLayerForAndroid:Common:unspecified] /home/xavier/src/sl4a/android/TextToSpeechFacade/build/intermediates/exploded-aar/ScriptingLayerForAndroid/Common/unspecified/AndroidManifest.xml
Suggestion: use tools:overrideLibrary="com.googlecode.android_scripting.common" to force usage

This can be fixed by updating all subprojects build.gradle files with 'minSdkVersion 4'
Another way is to move the subprojects 'android' script block in the root build.gradle file so that minSdkVersion is only defined once. The build.gradle file below (sorry, could not find a way to attach it to the issue) does that and all the subprojects build.gradle files only contain in this case the 'apply' statement and the 'dependencies' script block.
2 - Once this is fixed, another error occurs when building the APK:
/home/xavier/src/sl4a/android/ScriptingLayerForAndroid/res/drawable/atari_small_notice.txt
Error:Error: The file name must end with .xml or .png
:mergeArmDebugResources FAILED

3 - Once this file has been removed, another error occurs when building the APK:
Error:Execution failed for task ':transformClassesWithDexForArmDebug'.
> com.android.build.api.transform.TransformException: com.android.ide.common.process.ProcessException: org.gradle.process.internal.ExecException: Process 'command '/usr/lib/jvm/java-7-openjdk/bin/java'' finished with non-zero exit value 2

But this must be some bug in the ide or gradle since after selecting [Clean Project], [Rebuild Project] or [Build APK] not sure what is the right sequence, then the APK files are built and this error does not appear anymore.
ScriptingLayerForAndroid/build.gradle
buildscript {
    repositories {
        mavenCentral()
    }

    dependencies {
        classpath 'com.android.tools.build:gradle:1.5.0'
    }
}

apply plugin: 'com.android.application'

dependencies {
    compile project(':Common')
    compile project(':Utils')
    compile project(':QuickAction')
    compile project(':BluetoothFacade')
    compile project(':SignalStrengthFacade')
    compile project(':TextToSpeechFacade')
    compile project(':WebCamFacade')
    compile project(':USBHostSerialFacade')
    compile project(':ScriptingLayer')
    compile project(':InterpreterForAndroid')
    compile('com.android.support:support-v4:21.0.3') {
        force = true
    }
    compile files('libs/locale_platform.jar')
}

/* rootProject properties (see gradle ExtraPropertiesExtension). */
ext {
    build_tools_version = "21.1.2"
    compile_sdk_version = 21
    min_sdk_version = 4
    target_sdk_version = 4
    com_android_tools_build_gradle = 'com.android.tools.build:gradle:1.5.0'
}

android {
    buildToolsVersion project.build_tools_version
    compileSdkVersion project.compile_sdk_version

    sourceSets {
        main {
            manifest.srcFile 'AndroidManifest.xml'
            java.srcDirs = ['src']
            resources.srcDirs = ['src']
            aidl.srcDirs = ['src']
            renderscript.srcDirs = ['src']
            res.srcDirs = ['res']
            assets.srcDirs = ['assets']
            jni.srcDirs = []   // Please build NDK modules outside gradle.
            jniLibs.srcDirs = ['libs']
        }

        instrumentTest.setRoot('tests')
    }
    defaultConfig {
        minSdkVersion project.min_sdk_version
        targetSdkVersion project.target_sdk_version
        versionCode 611
        versionName '6.1.1'
    }
    productFlavors {
        x86 {
            ndk {
                abiFilter "x86"
            }
        }
        mips {
            ndk {
                abiFilter "mips"
            }
        }
        armv7 {
            ndk {
                abiFilter "armeabi-v7a"
            }
        }
        arm {
            ndk {
                abiFilters "armeabi"
            }
        }
    }
    productFlavors {
    }
    dexOptions {
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_6
        targetCompatibility JavaVersion.VERSION_1_7
    }

    applicationVariants.all { variant ->
        variant.outputs.each { output ->
            def file = output.outputFile
            if (file != null && file.name.endsWith('.apk')) {
                def newName = file.name.replace(
                    "ScriptingLayerForAndroid",
                    "sl4a-r" + defaultConfig.versionName)
                output.outputFile = new File(file.parent, newName)
            }
        }
    }
}

subprojects { subproject ->
    buildscript {
        repositories {
            mavenCentral()
        }

        dependencies {
            classpath rootProject.com_android_tools_build_gradle
        }
    }

    afterEvaluate{
        if((subproject.plugins.hasPlugin('android') ||
                subproject.plugins.hasPlugin('android-library'))) {

            android {
                buildToolsVersion rootProject.build_tools_version
                compileSdkVersion rootProject.compile_sdk_version

                defaultConfig {
                    minSdkVersion rootProject.min_sdk_version
                    targetSdkVersion rootProject.target_sdk_version
                }

                sourceSets {
                    main {
                        manifest.srcFile 'AndroidManifest.xml'
                        java.srcDirs = ['src']
                        resources.srcDirs = ['src']
                        aidl.srcDirs = ['src']
                        renderscript.srcDirs = ['src']
                        res.srcDirs = ['res']
                        assets.srcDirs = ['assets']
                    }
                    instrumentTest.setRoot('tests')
                }
            }

        }
    }
}

hmm, I suggest you need to use oracle jdk instead of open-jdk.
Can you try it?

Right, there is a warning that pops up in android studio about using open-jdk that may cause instability in the UI.
Following your suggestion, I switched to oracle jdk:
java version "1.8.0_74"
Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.74-b02, mixed mode)

The same errors occur with oracle jdk.

I have entered issue #22 to log the third item in the list of the three errors reported here.

Did you use the updatealternatives for java?
Have you built with success the template project? it seems like your environment problem...

Did you use the updatealternatives for java?

I used archlinux-java which is the corresponding tool on archlinux.

Have you built with success the template project?

I built a new module with application template Blank Activity and did not have any problem with building the APK.

I found that the root cause of the problem is the incompatibility of the android gradle plugin version specified in the sl4a root build script (1.1.0) with the gradle version (2.8) installed by the gradle wrapper run by android studio.
When first starting android studio on sl4a, the following error message is printed:
Error:Gradle 2.8 requires Android Gradle plugin 1.2.0 (or newer)  but project is using version 1.1.0.
Please use Android Gradle plugin 1.2.0 or newer.
Fix plugin version and sync project

Clicking on Fix plugin version and sync project causes the android gradle plugin version in ScriptingLayerForAndroid/build.gradle which is 1.1.0 to be overwritten with 1.5.0 (the latest stable version). This is what I did and I was running 1.5.0 without really noticing it.
To confirm that point, I tried forcing the build with android gradle plugin version 1.1.0, and building the APK was successful without any change to the project.  I managed to do it this way:

start android studio and import the sl4A project so that the gradle wrapper is created, but do not click on Fix plugin version and sync project.
run gradle 2.11 (a standalone installation of gradle, not the wrapper) on the root project, and the following error message is printed:

   > Gradle version 2.2 is required. Current version is 2.11. If using the gradle wrapper, try editing the distribution
   Url in /home/xavier/src/sl4a/android/ScriptingLayerForAndroid/gradle/wrapper/gradle-wrapper.properties to gradle-2.2-all.zip

update gradle-wrapper.properties according to the previous message and run:

$ bash gradlew assembleDebug

this downloads gradle 2.2 and builds sl4a with gradle 2.2, the build is successful with no errors

So you may be able to reproduce the three errors reported here by just changing the android plugin version specified in the sl4a root build script to 1.5.0 (you may have to switch to a newer version of gradle using gradlew as described above).

I have come up with a more concise way than proposed in my initial post. This fixes all the errors and it also works with android plugin 1.1.0.
The diff for this fix is shown on a clone of sl4a on bitbucket at https://bitbucket.org/xdegaye/sl4a/commits/cb474d519d715f4fceafc9b3d27913a687fea046

A note in Gradle Wrapper says:
Note: You should submit all of these files to your source control system.

So this note becomes clear (to me now). This ensures that the android plugin version set in the build script matches the gradle version set in the gradle wrapper properties file.

I checked your diff and realize to fix guava library,
so I will modify the settings around the guava lib.
But will we need to append gradle wrapper script for gradle 1.5?
please tell me actual 'these files' information.

my gradle.property is empty
my gradlew seems to download gradle 2.4

I have not use gradlew script long time and
I'm not sure about gradle that is so complex tool...
I'm sorry to mislead about JDK at last response, thank you.

The files mentionned in [Gradle Wrapper] (http://developer.android.com/tools/building/plugin-for-gradle.html#gradleWrapper) are the files in the ScriptingLayerForAndroid directory:

gradlew
gradlew.bat
gradle/wrapper/gradle-wrapper.jar
gradle/wrapper/gradle-wrapper.properties

The distributionUrl in gradle/wrapper/gradle-wrapper.properties specifies which gradle version, gradlew will be using, you can change it to use any another gradle version compatible with the android plugin. I have distributionUrl=https\://services.gradle.org/distributions/gradle-2.8-all.zip and I am using android plugin 1.5. So adding and committing all those four files to the git repository will ensure that whenever the repository is cloned by someone, the build will use the specified gradle version that is compatible with the android plugin version specified in buidl.gradle.
This does work only when using the gradle wrapper, not when using a standalone gradle. But using the gradle wrapper is the recommended way anyway, by google, android studio and gradle.

Thank you so much, I will try it myself and append them later.


fix(resolver): Allow users to add/remove related terms for children glossary terms

Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[x] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
[ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

Defect:
Users with MANAGE_GLOSSARY_CHILDREN privilege were not able to add/remove Related Terms in a Glossary Term.

Hi, Is there any plans to release this fix in the next DataHub release? I just checked the latest DataHub release docs and this fix is not included.

Hi, Is there any plans to release this fix in the next DataHub release? I just checked the latest DataHub release docs and this fix is not included.

Its released in https://github.com/datahub-project/datahub/releases/tag/v0.13.1


Table 'XXX' is already defined for this MetaData instance

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the SQLModel documentation, with the integrated search.
[X] I already searched in Google "How to X in SQLModel" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to SQLModel but to Pydantic.
[X] I already checked if it is not related to SQLModel but to SQLAlchemy.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
# using code from - https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/?h=hero#the-herocreate-data-model

from typing import List, Optional

from fastapi import FastAPI
from sqlmodel import Field, Session, SQLModel, create_engine, select

class HeroBase(SQLModel):
    name: str = Field(index=True)
    secret_name: str
    age: Optional[int] = Field(default=None, index=True)

class Hero(HeroBase, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)

class HeroCreate(HeroBase):
    pass

class HeroRead(HeroBase):
    id: int

engine = create_engine(
    "postgresql+psycopg2://postgres:postgres@localhost/testing", echo=True
)

def create_db_and_tables():
    SQLModel.metadata.create_all(engine)

app = FastAPI()

@app.on_event("startup")
def on_startup():
    create_db_and_tables()

@app.post("/heroes/", response_model=HeroRead)
def create_hero(hero: HeroCreate):
    with Session(engine) as session:
        db_hero = Hero.from_orm(hero)
        session.add(db_hero)
        session.commit()
        session.refresh(db_hero)
        return db_hero

@app.get("/heroes/", response_model=List[HeroRead])
def read_heroes():
    with Session(engine) as session:
        heroes = session.exec(select(Hero)).all()
        return heroes

Description
First attempt fails with.
sqlalchemy.exc.InvalidRequestError: Table 'hero' is already defined for this MetaData instance.  Specify 'extend_existing=True' to redefine options and columns on an existing Table object.

Adding this fixes this problem, however reading various sites this is not recommended?
class Hero(HeroBase, table=True):
    __table_args__ = {"extend_existing": True} # < new
    id: Optional[int] = Field(default=None, primary_key=True)

Second attempt fails with.
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DuplicateTable) relation "ix_hero_name" already exists

[SQL: CREATE INDEX ix_hero_name ON hero (name)]

Moving the create_db_and_tables() out of the functions startup event fixes this problem and everything works as expected
create_db_and_tables()
@app.on_event("startup")
def on_startup():
    print("startup")
    # create_db_and_tables()

Am I missing something that is causing this behavior?
Operating System
Linux
Operating System Details
No response
SQLModel Version
0.0.6
Python Version
3.10.4
Additional Context
No response

Similar problem when trying to set a timing decorator. Must be something general in the context of wrapper functions or decorators respectively. This is apperently not limited to SQLModel, since I'm not using SQLModel but declarative_base from SQLAlchemy.

I cannot reproduce this error. Works fine for me.
What version of SQLAlchemy is installed in the environment where you experienced this error?
Or has this issue been resolved?

@mybigman Has it been resolved? Or should I look into it?

I had the same problem.
Surprisingly it was caused by missing the package 'psycopg2' (I'm using a Postgresql db).

I have the same problem with even less code:
`from typing import Optional
import uvicorn as uvicorn
from fastapi import FastAPI
from sqlmodel import Field, SQLModel
class Hero(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
name: str = Field(index=True)
secret_name: str
age: Optional[int] = Field(default=None, index=True)
app = FastAPI()
if name == "main":
uvicorn.run("main:app", host="<IP_ADDRESS>", port=8001)`
The same ERROR pops when I run the program with
 python main.py
But everything looks OK if I run it with
uvicorn main:app
Here is pipenv graph, if it helps:
`aiosqlite==0.17.0

typing-extensions [required: >=3.7.2, installed: 4.4.0]
fastapi==0.87.0
pydantic [required: >=1.6.2,<2.0.0,!=1.8.1,!=1.8,!=1.7.3,!=1.7.2,!=1.7.1,!=1.7, installed: 1.10.2]

typing-extensions [required: >=4.1.0, installed: 4.4.0]

starlette [required: ==0.21.0, installed: 0.21.0]

anyio [required: >=3.4.0,<5, installed: 3.6.2]

idna [required: >=2.8, installed: 3.4]
sniffio [required: >=1.1, installed: 1.3.0]
sqlmodel==0.0.8

pydantic [required: >=1.8.2,<2.0.0, installed: 1.10.2]

typing-extensions [required: >=4.1.0, installed: 4.4.0]

SQLAlchemy [required: >=1.4.17,<=1.4.41, installed: 1.4.41]

greenlet [required: !=0.4.17, installed: 2.0.1]

sqlalchemy2-stubs [required: Any, installed: 0.0.2a29]

typing-extensions [required: >=3.7.4, installed: 4.4.0]
uvicorn==0.20.0

click [required: >=7.0, installed: 8.1.3]

colorama [required: Any, installed: 0.4.6]

h11 [required: >=0.8, installed: 0.14.0]
`

The problem is because uvicorn is loading the file again, you can see this by adding print:
from sqlmodel import Field, SQLModel

print("here")

class Hero(SQLModel, table=True):

You can solve this by separating the code into two files:
main.py:
import uvicorn

if __name__ == "__main__":
    uvicorn.run("app:app", host="<IP_ADDRESS>", port=8001)

app.py:
from typing import Optional

import uvicorn as uvicorn
from fastapi import FastAPI
from sqlmodel import Field, SQLModel

class Hero(SQLModel, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)
    name: str = Field(index=True)
    secret_name: str
    age: Optional[int] = Field(default=None, index=True)

app = FastAPI()

meirdev thanks issue resolved

Thanks

I'm also getting this error when I import the *.py file with model definitions in my unittests. I'm not even importing the class that has table=True. Running the app with uvicorn actually works fine.

Similar problem with streamlit
when file changed and streamlit app hotreload, error "Table 'XXX' is already defined for this MetaData instance" raised
anyone have idea?

this work for me. try adding this.
doc
from sqlmodel import SQLModel

SQLModel.__table_args__ = {'extend_existing': True}

When the import path is incorrect, I encountered the following error:
# occurred error
from foo.bar import Something

# worked out well
from src.foo.bar import Something

I'm having the problem even without FastAPI:
main.py
from typing import Optional
import os

from sqlmodel import Field, Relationship, Session, SQLModel, create_engine, TIMESTAMP, select

from job import Job

class Parent(SQLModel, table=True):
    id: int = Field(primary_key=True)
    name: str
    deleted_at: str | None = Field(sa_type=TIMESTAMP, default=None)
    parent_id: int | None = Field(
        foreign_key='parent.id',  # notice the lowercase "n" to refer to the database table name
        default=None,
        nullable=True
    )
    parent: Optional["Parent"] = Relationship(
        back_populates='children',
        sa_relationship_kwargs=dict(
            remote_side='Parent.id'  # notice the uppercase "N" to refer to this table class
        )
    )
    children: list["Parent"] = Relationship(back_populates='parent')
    jobs: list["Job"] = Relationship(back_populates="parent")

    @classmethod
    def get_name(cls, parent):
        return parent.name

sqlite_file_name = "database.db"
try:
    os.remove(sqlite_file_name)
except FileNotFoundError:
    pass
sqlite_url = f"sqlite:///{sqlite_file_name}"

engine = create_engine(sqlite_url)

SQLModel.metadata.create_all(engine)

parent1 = Parent(id=1, name="John")
parent2 = Parent(id=2, name="David", parent_id=1)
job1 = Job(name="Police", parent_id=1)
job2 = Job(name="Nurse", parent_id=1)

with Session(engine) as s:
    s.add(parent1)
    s.add(parent2)
    s.add(job1)
    s.add(job2)
    s.commit()
    s.refresh(parent1)
    s.refresh(parent2)
    s.refresh(job1)

    print(f"parent1: {parent1}, Job is: {','.join([job.name for job in parent1.jobs])}")
    print(f"parent2: {parent2}, Job is: {parent2.jobs[0].name if parent2.jobs else None}")
    print(f"grandparent: {parent2.parent.name} is parent of {parent1.children[0].name}")
    print(job1.get_parent_name())

job.py
from sqlmodel import Field, Relationship, SQLModel

class Job(SQLModel, table=True):
    id: int = Field(primary_key=True)
    name: str
    parent_id: str = Field(foreign_key="parent.id")
    parent: "Parent" = Relationship(back_populates="jobs")

    def get_parent_name(self):
        from main import Parent
        p = Parent(name = self.parent.name)
        return p.name

By running python main.py I get the same error

Works with this edit to the get_parent_name method:
`class Job(SQLModel, table=True):
id: int = Field(primary_key=True)
name: str
parent_id: str = Field(foreign_key="parent.id")
parent: "Parent" = Relationship(back_populates="jobs")
def get_parent_name(self):
    p = self.parent
    return p.name`

If you encounter such error, it might be because the file with models gets executed twice.
Add a debug print on top of the file to check it.
print("----")

As mentioned above the cause of such behavior might be uvicorn.
In my case it was relative imports.
Will suggest to use only absolute ones.

this work for me. try adding this. doc
from sqlmodel import SQLModel

SQLModel.__table_args__ = {'extend_existing': True}

Thanks, that worked for me!

The problem is because uvicorn is loading the file again, you can see this by adding print:
from sqlmodel import Field, SQLModel

print("here")

class Hero(SQLModel, table=True):

You can solve this by calling app directly:
app = FastAPI()

if __name__ == "__main__":
    uvicorn.run(app, host="<IP_ADDRESS>", port=8001)

Or, by separating the code into two files:
main.py:
import uvicorn

if __name__ == "__main__":
    uvicorn.run("app:app", host="<IP_ADDRESS>", port=8001)

app.py:
from typing import Optional

import uvicorn as uvicorn
from fastapi import FastAPI
from sqlmodel import Field, SQLModel

class Hero(SQLModel, table=True):
    id: Optional[int] = Field(default=None, primary_key=True)
    name: str = Field(index=True)
    secret_name: str
    age: Optional[int] = Field(default=None, index=True)

app = FastAPI()

this inspired me to solve my problem, I was importing the models bad, I also added the traceback to see the imported bad models like this:
import sys
import traceback

print(f"app.models loaded as: {__name__}, id: {id(sys.modules[__name__])}")
traceback.print_stack()
```


Figure out a way to run e2e tests against Rocket

Rocket folks already have e2e run on AWS. We need to have similar setup to run on GCE. cc/ @yifan-gu Please cc this one to Brian (I dont have his github account)

/cc @bakins

Slight clarification: we are currently working on full e2e test runs on CoreOS with Docker on AWS. Close to completing this.  Should be straightforward to replicate this on GCE.
The goal is to run e2e tests on CoreOS on multiple cloud providers using both rkt and docker.

It would be great to get the e2e test runs automated and the results collected into a common dashboard. See #6605. @justinsb

The extremely early CoreOS additions for e2e: https://github.com/bakins/kubernetes/tree/coreos/cluster/coreos-aws
No where near ready to open a PR.  Tests do not pass as some tests check for the "aws" provider and include/exclude tests based on that.  I know a few people are thinking about "hierarchal" providers and how to integrate with the tests.

It's possible there will be some E2E tests (ie, the one specifically for docker entrypoint control) that don't apply to rocket.  We will need some way to differentiate them.

cc/ @yifan-gu @jonboulle
We should categorize existing tests into 3 categories:

Test should pass with rkt and it does
Test should pass with rkt, but not yet
Test is not applicable for rkt

Then later we can add forth one including tests specifically for rkt too.

@dchen1107 sgtm. I am going to take the work.

Awesome @yifan-gu

I agree that this is stale.

Coreos is running e2e on rkt.

:tada: :tada: :tada: @euank


Add controlled mutability to section definitions

Allow section definitions to be programmatically created and populated. Deletion and random insertion are not supported, because section indices are expected to remain stable and are referenced from other sections.

Thanks for sending this one!
You are going in the direction of being able to populate and write Wasm Modules in Java and I really like it 👍 Are you planning on using this module in qbicc or is it still a stretch?

After reviewing Chicory pretty thoroughly, my tentative plan is to donate most of my WASM-related things to this project (with various changes to match up with the design of Chicory - some small and some significant) and drop all of it from qbicc, just consuming Chicory instead.
I can go into more detail on this week's call if you like.

I feel a bit uncomfortable having methods and constructors that are not used nor tested in any other part of the codebase. I see a few ways how we can proceed, and I'm open to alternative options:

add a simple test to cover, at least, the basic functionality and invoking those new methods

I think I can probably come up with something. Maybe it won't be 100% (yet) but at least the basics.

spike an integration test that will demonstrate how you envision those things working together at the end of the process(e.g. create a module and write it), it can be a bit hacky at first and we can refine with iterations

What I'd like to do eventually (at a minimum) is what I did in qbicc, which is to load a model from a file, then write the model back out to another file, and then load it in again and make sure it still works and/or compare to the original. Maybe across the whole test suite? I'm not 100% sure how to approach this yet though.

show, in a test, how those things can be used all together for some practical purpose, but I struggle to come up with something "simple enough"

Right, it's all leading up to being able to write WASM modules to disk, but there are a lot of pieces involved in doing that. Locally I'm maybe 8 commits ahead and I'm still not quite there, though I'm getting close. I'm trying to do this as incrementally as possible to avoid review whiplash and to make sure each change is in line with the overall philosophy of the project, making changes as needed. So far things have been relatively simple...

After reviewing Chicory pretty thoroughly, my tentative plan is to donate most of my WASM-related things to this project (with various changes to match up with the design of Chicory - some small and some significant) and drop all of it from qbicc, just consuming Chicory instead.

Awesome ❤ we already discussed this and are open to going with you in this direction!
On my side, I'll try to be responsive and make this happen.

I can go into more detail on this week's call if you like.

This would be much appreciated 🙏  having a better context/picture will make things easier.

though I'm getting close

👍 let me encourage you to open a PR, in draft, to show the progress, it would be really helpful to connect all the dots.

I'm trying to do this as incrementally as possible to avoid review whiplash and to make sure each change is in line with the overall philosophy of the project, making changes as needed. So far things have been relatively simple...

This is great, thanks a lot for sticking with those principles!

👍 let me encourage you to open a PR, in draft, to show the progress, it would be really helpful to connect all the dots.

I've opened #208 to show my current direction (and what's more or less working).

Pushed up a slight change; now, the new mutable section constructors are used during parsing, and the array-based constructors are dropped. All of the existing parsing tests now test this functionality.

I'll put comments on that PR about the specific implementation, but this is my impression.
This model doesn't really have a life of its own. You can't do anything with it, other than create a runtime model from it (or, eventually, write it to disk). And the API is already more or less a builder. By adding another layer of builder, now you have a builder to build a builder. This adds a bunch of copying cost and API overhead, but it doesn't really give any specific benefit in return that I can see.
That said, at the end of the day it's not my decision and if you think that there is a benefit to adding the builder layer that I don't realize, then I would say that the choice is yours and I can adapt either way.

Just to make it clear, I don't think that the sections are "Builders", also they are evolving in a direction that allows mutability which is undesirable for the runtime.
But I might be biased as well, let's hear more opinions cc. @lburgazzoli @bhelx @danielperano @enebo

I don't see why the runtime would want to refer directly to these sections though. Especially once there is a nontrivial component model, the information in the model structures would certainly be copied into new runtime structures. For example, rather than a type and code section plus function names section plus function imports (and eventually debug section), you'd only have a list of functions which have a type and name on them. Rather than the data and data count sections, you'd want to have initialized memories and passive segments (active and declarative segments would be dropped after loading).
If you require the model in order to manipulate the run time programmatically, you will be forced to construct a model as if you were going to save it to disk, which is likely not the best abstraction for talking to a runtime which already has capabilities that the model does not have (for example, host functions). I just think it's a mismatch.

If #211 is based on this PR, could we merge this PR and then consider #211 separately?

This PR effectively makes the Sections mutable, which I'm worried about merging without, at least, a second pair of eyes.
With #211 on top, I think that this aspect is under control.
Let me try to get another reviewer around quickly.

If no one else will chip in, just agreed to sort this out by tomorrow with @lburgazzoli 's help.

@dmlloyd I wholeheartedly appreciate the work you are doing in this repo and the direction you are bringing it to, I'm really grateful!
In this case we have a slightly different view and I don't feel 100% comfortable in deciding the direction by myself without seeking additional advice, for the greater good of the project.
I think that both this and/or #211 are going in a direction that enables you to move on.
I'll do my best to sort this out tomorrow.

sections were already mutable

I believe this is a loose end of the current implementation(e.g. not done completely intentionally), since we are at it I'd love to bring the public API to a better place.

Thanks again @dmlloyd ! And sorry for dragging this.


Store Eatery Images on Github

Pulling them async will prevent UI blocks from main thread and make them editable.

Closed with daniel/menu-redesign


Fix not running test codes

What does this PR do?

Fixes #13085
Does your PR introduce any breaking changes? If yes, please list them.
None
Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

@carmocca Could you check the milestone that this PR is added to when you get a chance? Its issue (https://github.com/PyTorchLightning/pytorch-lightning/issues/13085) is part of 1.7 but this PR is of 1.6.x. 👀

CI has been busy all the time. Is there anything I should do? Why don't we restart it?

@ar90n There's no further action required, and we're aware of some issues with our CI. Thank you for checking.


fix typo

Docs Build status updates of commit 48f7171:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

AdaptiveCards/resources/partners.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 15722d5:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

AdaptiveCards/resources/media.md
:white_check_mark:Succeeded

AdaptiveCards/resources/partners.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Enabled Ninja build

It didn't help
CMake Error:
  Running

   'C:/Users/vzlobin/r/ninja.exe' '-C' 'C:/Users/vzlobin/r/g/build/CMakeFiles/CMakeScratch/TryCompile-aqlo4u' '-t' 'recompact'

  failed with:

   ninja: error: build.ninja:53: bad $-escape (literal $ must be written as $$)

CMake Error at C:/Program Files/CMake/share/cmake-3.26/Modules/CMakeDetermineCompilerABI.cmake:57 (try_compile):
  Failed to generate test project build system.
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:26 (CMAKE_DETERMINE_COMPILER_ABI)
  build/_deps/sentencepiece-src/CMakeLists.txt:23 (project)

Should we explicitly set Ninja as generator in some builds on GHA?

yes, will add in this pr


Tensor.nonzero fails on GPU for tensors containing more than INT_MAX elements

I've been gradually increasing the size of the input images I'm using with Segment Anything, and it seems as though once you have an image where the number of detections x height * width > 2^31 you run into an unresolved torch bug:
https://github.com/pytorch/pytorch/issues/51871
I've managed to create this using an image that's 4096 x 4096 pixels and with setting that detect > 128 objects.
I get the error message:
Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/runpod/serverless/modules/rp_job.py", line 113, in run_job
result = handler(job)
File "/make_crypts.py", line 97, in runpod_mask_save
thisMaskSet = mask_generator.generate(patch_img)
File "/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/venv/lib/python3.10/site-packages/segment_anything/automatic_mask_generator.py", line 163, in generate
mask_data = self._generate_masks(image)
File "/venv/lib/python3.10/site-packages/segment_anything/automatic_mask_generator.py", line 206, in _generate_masks
crop_data = self._process_crop(image, crop_box, layer_idx, orig_size)
File "/venv/lib/python3.10/site-packages/segment_anything/automatic_mask_generator.py", line 245, in _process_crop
batch_data = self._process_batch(points, cropped_im_size, crop_box, orig_size)
File "/venv/lib/python3.10/site-packages/segment_anything/automatic_mask_generator.py", line 318, in _process_batch
data["rles"] = mask_to_rle_pytorch(data["masks"])
File "/venv/lib/python3.10/site-packages/segment_anything/utils/amg.py", line 118, in mask_to_rle_pytorch
change_indices = diff.nonzero()
RuntimeError: nonzero is not supported for tensors with more than INT_MAX elements, file a support request

One resolution would be to anticipate this image and process only a number of masks at a time that would avoid hitting INT_MAX
Another would be to use 64 bit integers - though I don't know if that's possible in Torch.
Please let me know if you need any further information or example code.

I've come across this error. Is there any development or a resolution on your end?

I've come across this error. Is there any development or a resolution on your end?

The error is mostly caused in an external library, but I've created a workaround - until it's merged you can use my git fork, though it may be better security practice to fork your own to make similar changes


Allow strong access to each loaded resource

I wrote the following code generation tool using your library:
@tool
extends EditorScript

func _run() -> void:
	var items = preload("res://data/items.tres").load_all()
	var items_str = "\n".join(items.map(func(item: Item):
		return 'static var {id}: Item = load("{path}")'.format({"id": item.id, "path": item.resource_path})))
	var all_str = ", ".join(items.map(func(item: Item): return item.id))

	var script_content = \
"""
# This file was auto-generated
class_name Items
extends RefCounted

{items}

static var all: Array[Item] = [{all}]

""".format({"items": items_str, "all": all_str})

	print(script_content)

	var script = GDScript.new()
	script.source_code = script_content
	ResourceSaver.save(script, 'res://scripts/data/items.gd')

where res://data/items.tres is a ResourceGroup. It's been very helpful for referencing specific resources by (in this case) their id field. I was wondering if you might incorporate this into your library, perhaps have each ResourceGroup export at least a file path, which, if set, would make the resource group generate a file similar to the one above, where the details (class_name, the field that becomes the var names, etc..) could be additional exports.

The generated file looks like this:
# This file was auto-generated
class_name Items
extends RefCounted

static var apple: Item = load("res://data/items/apple.tres")
static var axe: Item = load("res://data/items/axe.tres")
static var carrot: Item = load("res://data/items/carrot.tres")
static var carrot_seed: Item = load("res://data/items/carrot_seed.tres")
static var chair: Item = load("res://data/items/chair.tres")
static var clock: Item = load("res://data/items/clock.tres")
static var hoe: Item = load("res://data/items/hoe.tres")
static var orange: Item = load("res://data/items/orange.tres")
static var peach: Item = load("res://data/items/peach.tres")
static var pear: Item = load("res://data/items/pear.tres")
static var stick: Item = load("res://data/items/stick.tres")
static var stone: Item = load("res://data/items/stone.tres")
static var table: Item = load("res://data/items/table.tres")
static var watering_can: Item = load("res://data/items/watering_can.tres")
static var wood: Item = load("res://data/items/wood.tres")

static var all: Array[Item] = [apple, axe, carrot, carrot_seed, chair, clock, hoe, orange, peach, pear, stick, stone, table, watering_can, wood]

where I can easily access each item from anywhere using e.g. Items.axe or Items.all.

Actually, when I started making this library, this was the original thing I wanted to build - a code generator. However, I was not able to come up with an output that would have been universally useful. Some projects would benefit from a list of variables like you have suggested, others would probably benefit more from a database-like structure, where you can ask things like "give me a random fruit" or "give me all tools sorted by price",  yet others would like to have a dictionary of enemies grouped by element type or cards grouped into decks or upgrade levels, etc, etc.
So whatever output I decided for, would work only for some games and at the same time would not work for a substantial amount of other games. Another problem is that this introduces a good amount of edge cases where variable or function names clash with other data or Godot's built-in functions (e.g. if for some reasony you name an item all in your example, this will break the generation). This is ok if you build the generator for your own game and can live with some limitations but a library should be robust enough to handle this kind of stuff and more. So in the end I scrapped the code generator idea and rather went for a simple but universally useful interface that would allow game developers to build their own system on top of the resource groups - like you did with your implementation. Right now I still think that this was the right call.
But if you have some ideas on how to generate a universally useful output and handle the other problems I mentioned, I'd very much like to know about it!

I think individual references to each resource is almost universally useful (and in the rare cases it isn't you can just not opt in). Regarding the all var (see below paragraph for naming conflict solution), this is merely in keeping with your existing provided use case of load_all().
Amending my earlier suggestion, export an optional string name for the all var, set the class name to the resource name of the resource group (properly cased), set the var names to the resource names of the resources. As before, allow the path to the script file to be set, where if left empty then no script is generated.
I'm guessing this would cover 90+% of the use cases for your tool, which I understand to be not having to preload each resources by hand, which I'm presently still needing to do in order to reference them individually. If people want different/more functionality they can create their own tool script (or maybe have your generator only write its own part of the script between two comment lines and let people add additional code (if you do this then maybe drop the all var, it would be easy to add manually).

Thanks for taking the time to write this up. I think if its optional and configurable to some extent it could indeed be a  good 80/20 solution. I'll have a deeper look at it, but it will be a few days as I'm currently fully engaged in another thing that needs doing :-).


Add support for a DISABLE_WEAK_CIPHER_SUITES environment variable

Our default NGiNX config provides a good balance between security and compatibility with older clients. This patch adds support for the environment variable DISABLE_WEAK_CIPHER_SUITES which, when set, disables SSLv3, the RC4 cipher, and MD5 for HMACs while adding support for TLS_RSA_WITH_3DES_EDE_CBC_SHA. The latter is used under TLS 1.0 to support IE 8 on Win XP and Java 6. The resulting configuration is also compatible with IE 6 under Win XP, but only if the client toggles a flag in their IE config to enable TLS 1.0.
The DISABLE_WEAK_CIPHER_SUITES config earns an A+ on the Qualys SSL Server Test while supporting all clients they test except for the default installation of IE 6 on Win XP.
Ideally, we could continue to provide maximum compatibility in the default configuration without sacrificing practical vulnerability to any known attacks while guaranteeing that DISABLE_WEAK_CIPHER_SUITES will earn an A+ on the Qualys SSL Server Test but might sacrifice some compatibility in the process.
/cc @fancyremarker for review

This is great. Thanks for all the research into compatibility and Qualys Flair. Feel free to merge at will. Re: this...

Ideally, we could continue to provide maximum compatibility in the default configuration [...]

Do you think we would eventually want to disable weak cipher suites (and SSLv3) by default, and allow SSLv3 only if an environment variable like ENABLE_WEAK_CIPHER_SUITES was set?

I'm happy to reverse the DISABLE/ENABLE flag it if you want. I went back and forth on that, but I'd rather err on the side of increased compatibility in our default config and allow those who need to err on the side of security for audits or other reasons to turn the flag on. The default config isn't bad at all, it isn't vulnerable to any known practical attacks, it just isn't quite as aggressive as the A+ config.

Sounds good. I too am happy to keep the flag as is.


Demo store

We decided to introduce new default layout for Saleor. It's based on bootstrap4.
UX Mockup: https://www.figma.com/file/cjGxsI8wyWHbjtLNJzRZB3mI/Saleor-Demo
Design: https://www.figma.com/file/HQ5PoE3bqrE9ojZh81vwpvAU/Saleor-Demo-design
Progress is tracked in a Github project https://github.com/mirumee/saleor/projects/1

Current coverage is 63.69% (diff: 62.50%)

Merging #668 into master will increase coverage by 0.05%

@@             master       #668   diff @@
==========================================
  Files            91         92     +1
  Lines          4296       4305     +9
  Methods           0          0
  Messages          0          0
  Branches        462        463     +1
==========================================
+ Hits           2734       2742     +8
- Misses         1468       1470     +2
+ Partials         94         93     -1

Powered by Codecov. Last update a3c9199...b8e76dd

Moving all demo related work from the fork. I'll reopen new pull request from a fresh branch.


Class Based API vnext

Taking advantage of the reactive controllers branch and it's very likely major version bump I also wanted to follow up #36 where it would be nice to expose the class based API to have better bind parity to lit.dev and also to allow newer users who may come from JS themselvs (or even other backgrounds) to be easier to just follow lit's docs and be able to do it on F# as well.
One of the big issues is that we can't emit typescript-like decorators yet and we shouldn't because the HTML spec differs from typescript's implementation (I'm not sure up to what detail but I know it does)
that being said I'm working on a computation expression which kind of matches the customElement decorator from typescript
we still need to track property definition vs properties inside the class but in my mind it should not be too much of a hassle because it's the same behavior lit.dev has in JavaScript
The proposed API would look like this
// classic class component definition for lit components
type UserProfile() =
    inherit LitElement()
    // manual declaration of internal properties
    let mutable name = ""
    let mutable age = 0

    override _.render() =
        html $"<p>{name} - {age}</p>"

// this auto registers the element using the same
// behavior as the decorator would do
registerElement "user-profile" jsConstructor<UserProfile> {
    property "name" {
        use_attribute
        use_type StringCtor
    }
    property "age" {
        use_attribute
        use_type JS.Constructors.Number
    }
}

It's usage would be as usual
<user-profile name="Peter" age="25"></user-profile

What it brings to the table is the ability to build the properties in a kind of type safe way with some auto-completition vs building the properties manually and try to remember each possibility
{| name = {| ``type`` = StringCtor; attribute = true |}
   age = {| ``type`` = JS.Constructors.Number; attribute = true |} |}

For cases where one may want to provide a component library for other F# users I also thought to have a way to let users cherry pick their own elements
[<AutoOpen>]
module ElementDefinitions =
  let userProfileDef =
    delayedElement "user-profile" jsConstructor<UserProfile> {
      property "name" {
        use_attribute
        use_type StringCtor
      }
      property "age" {
        use_attribute
        use_type JS.Constructors.Number
      }
    } // ElementDefinition rather than auto registration

// let the user choose what they want to actually use
module CherryPick =
  let userProfile() =
    // defineElement will also be provided by us
    // (likely the same function used in the registerElement CE's Run method)
    defineElement userProfileDef

current progress on this area is on #41


Update: Mozilla.Firefox.DeveloperEdition version 129.0 (129.0b5)

Created by 🥟 Dumplings in workflow run #6663.

Log
Updated: 129.0b4 → 129.0b5
Submitting WinGet manifests
Found existing pull requests:
Update: Mozilla.Firefox.DeveloperEdition version 129.0 (129.0b4) - https://github.com/microsoft/winget-pkgs/pull/163094
New version: Mozilla.Firefox.DeveloperEdition version 129.0 (129.0b3) - https://github.com/microsoft/winget-pkgs/pull/162725
New version: Mozilla.Firefox.DeveloperEdition version 129.0 (129.0b2) - https://github.com/microsoft/winget-pkgs/pull/162345
The existing pull requests are ignored as the version is updated while the real version isn't
Creating manifests
Uploading manifests and making commits
Creating a pull request

Microsoft Reviewers: Open in CodeFlow

/azp run


Supplying context to eval

Hello
I would like to have a boolean expression but need additional context for the eval method to evaluate the expression. For example,
(5 AND 9) OR 3
And then pass in a Collection to eval() which would invoke contains() on the collection to evaluate the result of the literal. Is this possible?
Thank you
Evan

I have a hard time understanding what you want to do, something like this maybe?
Expression expression = new Expression("(x and y) or z")
        .with("x", "1")
        .with("y", "0")
        .with("z", "1");

expression.eval() == 1

Oh, no. As far as I can see the literal value wins over a set variable when parsing, so it never tries to replace them with the variable.
You could "pre-process" the expression with something like this:
"(5 and 7) or 9".replaceAll("([0-9]+)", "x$1") == "(x5 and x7) or x9"

And these you can set as variables on the expression.

I don't think that adding such a feature would be feasible or easily possible. So I'm closing this.


StackMetadata causes CloudFormation to hang on creation & deletion, v1.18.4

Hi all,
We have been trialling Serverless Stack with a lot of excitement but today we hit a stumbling block.
We were on v1.15.16, and then when I tried to use the bind method, I realised we were too many versions behind, so we bumped the version all the way to latest (v1.18.4).
Then when we tried to do npx sst start to continue development, the deployment hung indefinitely (~ 30min) on a small stack.
Eventually I tried to delete the stack from CloudFormation and start again from scratch. But when I tried this, the deletion failed with the following message:

The following resource(s) failed to delete: [StackMetadata].

I then noticed that v1.18.4 had an update with the message "Fix throttled error on updating stack metadata", but I don't know if the two are related. I tried to find the corresponding AWS resource but wasn't sure where to look, I had the impression it was somehow related to Stack Manager but I couldn't find anything obvious with a name like localdev-myapp-MyStack-.....

https://github.com/serverless-stack/sst/commit/8e4ffda9dd7f97df470a614c9e2c5882a50c78ae - the relevant(?) commit

Anyway, I ended up able to delete the stack again eventually, somehow the stack metadata was able to be deleted the second time. I've rolled back the app version to v1.18.2 and I am able to deploy again (taking ~3min).
Thank you for any assistance and information on what might have gone wrong!

Confounding factors:

We were trying out EventBus rules at the time; at one point one of the EventBus rules reported to CloudFormation that it needed manual resource creation, which I thought was unusual, but it was eventually created.
I deleted one old stack which was removed from the app, and renamed the remaining stack
I changed two resource names (1x EventBus and 1x Api construct)
It's Black Friday, maybe AWS was just having a rough day

For what it's worth, the stack consisted of the following:
export function MainStack({ stack }: StackContext) {
  const bus = new EventBus(stack, "MainBus");

  const queue = new Queue(stack, "TestQueue", {
    consumer: "functions/debuglogger/lambda.handler",
  });

  const api = new Api(stack, "MainApi", {
    defaults: {
      function: {
        environment: {
          MAIN_BUS_NAME: bus.eventBusName,
        },
      },
    },
  });

  bus.addRules(stack, {
    testRule: {
      pattern: { source: ["some.anonymised.event.source"] },
      targets: {
        queue: queue,
        debuglogger: "functions/debuglogger/lambda.handler",
      },
    },
  });

  api.addRoutes(stack, {
    "POST /adaptwest/reportbridge": {
      function: {
        handler: "functions/oneofourclients/reportbridge/lambda.handler",
      },
    },
  });

  stack.addOutputs({
    ApiEndpoint: api.url,
    EventBusName: bus.eventBusName,
  });

  return { bus, api };
}

@ceigey lemme give it a try.

@ceigey how many stacks r there in the app? And have you created any secrets ie. using sst secrets, if so how many?

@ceigey how many stacks r there in the app? And have you created any secrets ie. using sst secrets, if so how many?

Hi @fwang apologies for the late reply! Regarding stacks, we had 3x overall:

first we had "TestStack"
then we added "MyStack" to test having 2x stacks
then we renamed "MyStack" to "[Business Name]Stack"*
then we deleted "TestStack"

So I think in effect, from a CloudFormation POV, we basically orphaned 2x stacks and created a copy of one with the exact same resources but a new stack name based on the stack function being renamed.
With sst secrets, unfortunately we didn't commit this part, but we seem to have added either Config.Parameter or the equivalent from Secrets. I think it was just Config.Parameter because I can see we did commit the code for using the Node helper Config from one of the lambdas.
We definitely didn't use the secrets cli at all though.
*we're rebranding next year so I want to keep the name a secret 😅

Thanks for the details @ceigey.
I'm not able to reproduce the error following these steps. I just created a new project with v1.15.16, deployed, updated to v1.18.4, deployed again.
Can you try deploying again, and if metadata gets stuck again:

go into your AWS CloudWatch console
search for the log group prefix with /aws/lambda/{STACK_NAME}_ CustomResourceHandler
check the most recent log streams in the log group, you should see an invocation that had the long duration and caused the timeout
Could u share the logs from it?

Thank you for the instructions – I haven't redeployed with v1.18.4 yet – but I can now look at the existing logs.
(I had redeployed with v1.18.2 without issues)

So first back to the CloudFormation Event logs,  I found this for localdev-[business]-[Business]Stack:
At 2022-11-25 15:43:30 UTC+1030, StackMetadata = CREATE_IN_PROGRESS
At 2022-11-25 16:00:23 UTC+1030,  localdev-[business]-[Business]Stack = DELETE_IN_PROGRESS - so actually we must have deleted after 15min, and I got the timeline confused!
At 2022-11-25 16:00:24 UTC+1030, StackMetadata = CREATE_FAILED.
At 2022-11-25 17:00:59 UTC+1030, StackMetadata = DELETE_FAILED with the following message:

CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [7736c9f8-ae0e-4767-b9ca-8ae3e7391ca0]

I've downloaded all the logs for the last 4 weeks with the keyword CustomResourceHandler and I can't find this requestID, so I'm not sure which log group CloudFormation wants me to check :-/

Now back to CloudWatch logs for localdev-[business]-[Business]Stack-CustomResourceHandler[....]
At 2022-11-25T15:43:31.887+10:30
& 2022-11-25T15:43:32.106+10:30:
2022-11-25T05:13:31.887Z	undefined	ERROR	Uncaught Exception 	{
    "errorType": "Runtime.HandlerNotFound",
    "errorMessage": "index.handler is undefined or not exported",
    "stack": [
        "Runtime.HandlerNotFound: index.handler is undefined or not exported",
        "    at Object.UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1038:15)",
        "    at async start (file:///var/runtime/index.mjs:1200:23)",
        "    at async file:///var/runtime/index.mjs:1206:1"
    ]
}

At 2022-11-25T15:43:32.140+10:30
START RequestId: d150008f-d018-4f1b-bdca-cb11a27f3248 Version: $LATEST
Unknown application error occurred
Runtime.HandlerNotFound
END RequestId: d150008f-d018-4f1b-bdca-cb11a27f3248

These errors repeat in a cycle about 4 to 5 times until 2022-11-25T15:46:32.945+10:30

Then, under a different log stream for the same log group
From 2022-11-25T16:00:44.470+10:30
Until 2022-11-25T16:03:34.564+10:30
The same cycle reoccurs another 4 to 5 times, same kind of errors etc.

Likewise, CloudWatch logs for localdev-[business]-DemoStack-CustomResourceHandler[....]
It looks like we renamed TestStack to `DemoStack at some point before deleting it, we also had given it a custom domain at this point and that's perhaps where things started getting weird...
From 2022-11-25T15:12:45.030+10:30
Until 2022-11-25T15:15:40.074+10:30
There's an identical cycle of 4 to 5 uncaught exceptions & failed requests.
But this has no corresponding StackMetadata issue in CloudFormation, so I don't know what the connection could be.

My own thought looking at these logs and our git history is maybe we tried too many "risky" things at once and put the CustomResourceHandler into an inconsistent state. E.g., renaming stacks, adding custom domain URLs, deleting stacks, etc.
And then maybe it just so happened that CloudFormation was waiting for a response from a CustomResourceHandler for StackMetadata that either no longer existed or no longer responding.
In which case then I wonder if it's an underlying CloudFormation issue with the Custom Resources?

Appreciate the logs!!

index.handler is undefined or not exported
Let me test something on my end.

Oh no, déjà vu... Started a new app with create-sst on v1.18.4, minimal go starter template.
Unable to finish creating StackMetadata, started creating it at 1:20pm, still hasn't finished by 2:12pm, so 52 minutes and still counting 😕
Alternatively, it could also be apiRouteGETIntegrationGETPermission* or something like that.
There is an entry for CustomResourceHandlerE8FB56BA with CREATE_COMPLETE, so that part seems OK.
Screenshot from 2:12pm.

I might've found the cause. Try emptying the S3 bucket that starts with cdk-hnb659fds-assets-, and then deploy again (after current deployment finishes).

Thanks @fwang ! Reattempting the deployment at 3:41pm (UTC+1030), looks like StackMetadata works now. Deployment completed in 3 minutes (3:44pm).
For future reference, are there any negative side-effects to deleting that bucket, especially if I have other CDK apps or stacks running?

I can also confirm with my terminal logs that this StackMetadata creation will time-out by the 1hr mark and return an error.
Then, on CloudFormation console, it will automatically try a rollback and again it will time-out after 1hr of StackMetadata being stuck on DELETE_IN_PROGRESS.
Terminal logs at 2:20pm:
Checking deploy status...
Checking deploy status...
Checking deploy status...
Checking deploy status...
Checking deploy status...
christianm-roberto-MainStack | CREATE_FAILED | Custom::StackMetadata | StackMetadata | CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [d9a850cb-9adf-46ab-88f1-052983be4b67]. If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the updated version.
christianm-roberto-MainStack | ROLLBACK_IN_PROGRESS | AWS::CloudFormation::Stack | christianm-roberto-MainStack | The following resource(s) failed to create: [StackMetadata]. Rollback requested by user.
christianm-roberto-MainStack | DELETE_IN_PROGRESS | AWS::SSM::Parameter | apiParameterurl1D4D90E2

Screenshot taken at 3:38pm, rollback failing after another hour.

From the terminal logs:
christianm-roberto-MainStack | DELETE_COMPLETE | AWS::IAM::Role | CustomResourceHandlerServiceRole41AEC181
christianm-roberto-MainStack | ROLLBACK_FAILED | AWS::CloudFormation::Stack | christianm-roberto-MainStack | The following resource(s) failed to delete: [StackMetadata].

 ❌  christianm-roberto-MainStack failed: CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [d9a850cb-9adf-46ab-88f1-052983be4b67]. If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the updated version.

After emptying the bucket, a positive sign! StackMetadata = CREATE_COMPLETE!

+1 Having the same issue, or at least the same result: running npm run dev with v1.18.4 and 1.18.3. It seems to hang on the database stack while setting up the migration script(see image).

I have had the same issue with the upgrade to 1.18.4 from 1.15.13. Our app has ~30 stacks in it but I haven't been able to identify any correlation to the types of constructs being created and this issue.
I haven't tried emptying the S3 bucket as suggested yet. Like @ceigey, I'm curious if there are side effects to emptying that bucket? The SST app encountering this issue is not the only SST app running in that account so I'd be deleting files associated with other apps as well.

@ceigey quick update on the issue. We finally found the root cause. It's a CDK bug where the Lambda zip gets corrupted when uploading multiple concurrently. We've put in a fix in SST v2. Will cut a release candidate soon.
@michaelleeallen I'm not sure if it's caused by the upload issue. Can you go to the AWS Lambda console, find the db migration handler, and export the code? It'd help clarifying if the upload is corrupted.
@CodyDunlap r u having the issue w/ metadata or db migration? Btw, I gave it some test, emptying the CDK assets bucket does not affect deployed apps. So safe to deploy.

This is fixed in v2 but wondering if we need to backport the fix?

If you compare the diff between 1.18.2 and 1.18.4 here
It seems that the culprit for the hang is the maxRetries: 1000:
  maxRetries: 1000,
  retryDelayOptions: {
    customBackoff: () => 3000,
  },

at least partially anyway.
As mentioned earlier in this thread - we rolled back to version 1.18.2 and the problem has disappeared.
I used the CLI command sst update 1.18.2 to get the correct dependencies. Pretty sure I nuked  the node_modules directory (rm -rf node_modules) and the package-lock.json file first though.

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble