Fix issues map use revision filename

the key of map<file, issue> generate from sonar-report.json has the format like <component path>/src/main/...
sometimes, when the project is big, the revision filename from girret is like <project path>/<component path>/src/main/....
in this condition, we can't filter issues exactly match the key with the revision filename.
this patch fix the bug.  JENKINS-37765

Please open a new PR without conflict if you think your patch is still needed.


guide/russian/agile/index.md translation improves

freeCodeCamp/freeCodeCamp/guide/russian/agile/index.md translation improves
I've done it in more readable way and translated missing parts.
Also, I've translated the title as a 2 word combination, because the translation makes sense only with "Agile methodologies".

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

LGTM! Congrats on your first contribution!

@greggubarev Thank you!
Can you also take a look on my similar pull requests like this, this, this and this one? All of them ended up with "Failure: The Travis CI build failed" and I'm worried about them, because I don't know what does this failure mean.
Also, similar pull requests above may be as good as this one.


The problem of mcsema-lift（Windows）

When I want to translate .cfg to LLVM bitcode using mcsema-lift, the output is:
E1227 17:12:40.319221 25752 CFG.cpp:220] Data cross reference at 804b0f4 targeting gmon_start at 804bf58 does not match any known externals and is not contained within any data segments.
E1227 17:12:40.320245 25752 CFG.cpp:220] Data cross reference at 804b14c targeting gmon_start at 804bf58 does not match any known externals and is not contained within any data segments.
E1227 17:12:40.320245 25752 CFG.cpp:220] Data cross reference at 804b17c targeting _Jv_RegisterClasses at 804bea0 does not match any known externals and is not contained within any data segments.
E1227 17:12:40.321254 25752 CFG.cpp:220] Data cross reference at 8049a2a targeting gmon_start at 804bf58 does not match any known externals and is not contained within any data segments.
E1227 17:12:40.324261 25752 Segment.cpp:304] Marking _ZN12GZipAllFiles7class$$E as non-constant to support lazy initialization of reference to _ZN4java4lang9Throwable6class$E from 804a9a0
E1227 17:12:40.325264 25752 Segment.cpp:304] Marking seg_804c2e0_abs as non-constant to support lazy initialization of reference to _ZN4java4lang6System3outE from 804c2e0
F1227 17:12:40.326241 25752 Lifter.cpp:471] Check failed: word_size == arg_size Expected integer argument to be machine word size (32 bits) but is is 64 instead in instruction at 804a004
*** Check failure stack trace: ***
@   00007FF73D32DBFB  (unknown)
@   00007FF73D3591EF  (unknown)
@   00007FF73D35A642  (unknown)
@   00007FF73D35612C  (unknown)
@   00007FF73D06A611  (unknown)
@   00007FF73D067988  (unknown)
@   00007FF73D0645AF  (unknown)
@   00007FF73D06268D  (unknown)
@   00007FF73D0738E6  (unknown)
@   00007FF73D075917  (unknown)
@   00007FF73D78E941  (unknown)
@   00007FF836A92784  BaseThreadInitThunk
@   00007FF8383B0C31  RtlUserThreadStart
What can cause the problem?

This is a problem caused in the CFG recovery; CFG recovery is identifying what it is in that first error.
Can you share the binary and/or the command you used to recover the CFG?

@KyleMiles
The command I used is ：
E://tools//IDAPRO//idaq64.exe -S"C:\mcsema\Lib\site-packages\mcsema_disass-2.0-py2.7.egg\mcsema_disass\ida\get_cfg.py --output C:\mcsema\binary4.cfg --log_file C:\mcsema\binary4.log --arch x86 --os windows --entrypoint main" E://ConGzip
And the binary is：
ConGzip.zip
And I have another question：The binary which is compiled on Linux，can I use mcsema to analysis it on windows？

I have the same problem.


Clean up default flags + (implicit_transitive_deps false) experiments

Supersedes https://github.com/ocurrent/opam-health-check/pull/16
cc @MisterDA

That seems perfect, thanks!


Add CLI to scripts/upload_object.py

This script has become important as a timeout feature in the main backup scripts csd3-side/scripts/lsst_backup.py means users may have to upload a small number of files "manually". Therefore it needs better usability.

Done.


Small code coverage improvement

When I wrote that if statement yesterday, I was thinking if (!($CodeCoverageStop) { would match both $false and $null. I'm really not sure why I didn't just make the comparison explicit.

In this case $null does the same thing as $true, but only up to the end of the task, where we choose whether to abort the build based on code coverage. $null works as a 'reportonly' and does not stop the build, with $true stopping it if the required code coverage percentage isn't met.

The only reason I used $true/$false/$null was that it was three state, rather than defined strings like 'Enabled'/'Disabled'/'ReportOnly', maybe that would be a better option?
In the case of $SignScripts, either the existing name, or $ScriptSigningEnabled. It might be good to get a nice theme running through the variable names for the settings to maintain a consistency that gives users a bit more of an idea about options they haven't used before based on experience of other settings they have used.

might be good to get a nice theme running through the variable names for the settings to maintain a consistency

Yeah I was thinking the same.  Go ahead and merge this PR and let's think on this some more.


Register error logs

Create an option to view error logs while browsing the app

cloese in https://github.com/LucasMatheusDev/analytics_app/commit/c1af85db29bbff522e06fe0f2f7badcb031c9fcc


Allow jQuery 3.x (there is an XSS vulnerability in previous versions)

As for now cytoscape.js-qtip requires "jQuery ^2.0 || ^1.10.0, as qTip requires it".
But qTip2 requires "jquery": ">=1.6.0", i.e. it allows 3.x versions too.
It should be possible to use jQuery 3.x, and it seems cytoscape.js-qtip won't break form this change too.
Motivation: npm audit shows "Cross-Site Scripting (XSS)" vulnerability for jQuery 2.x.
I'd just specify "jquery": ">=1.6.0" in peerDependencies as in qTip2.
Opening this issue instead of a pull request to make sure if jQuery 3.x was not disallowed intentionally.

I'm open to any workaround that doesn't break the extension for the versions of jquery that are officially supported by qtip.

Oops, wrong button...


Question: How to do Unit Testing on Sources and Sinks?

What would be the current best practice for unit testing apps that rely on cloud stream rabbit? Is there an easy way to disable all the cloud stream autoconfig, rabbit autoconfig, etc and mock the Sources/Sinks?
Changing the dependencies in test or depending on a local Rabbit server for unit testing doesn't seem like the best idea.

Hi @theonlyguills an example on how to unit test is here: https://github.com/spring-cloud/spring-cloud-stream-modules/tree/master/ftp-source
It is using SCS' test support binder: https://github.com/spring-cloud/spring-cloud-stream/tree/master/spring-cloud-stream-test-support

Awesome thanks for a quick reply.


Update firewall section

update firewall setting to be under network
there are some parts that look like quotes but aren't linked anywhere, when I tried to find them it just seems to link to other copies of this guide so I just removed the > so it doesn't show up as a quote anymore
remove security growler as the GitHub repository is archived and the last commit was 7 years ago
remove some little snitch links that go to archived apple documentation. No longer relevant since Little Snitch uses system extensions instead of kernel extensions now
remove ice floor as it no longer works and appears to be abandonware
remove link to instructions on a very outdated version of macOS

I think the firewall stuff was originally quoted from Apple docs, some of which still matches up to https://support.apple.com/guide/mac-help/change-firewall-settings-on-mac-mh11783/mac
Thanks for another great PR.


Allow co-recommenders to invite reviewers, issue decisions, and edit submission information

 Allow co-recommenders to invite reviewers, issue decisions, and edit submission information (in the Edit Report, and Edit Report Survey screens)


REST api - Creating an entity on a non existing namespace works

Tried to create a service on a namespace 'staging' which doesn't exist on my k8s.
Creation succeeded which is odd enough :)
Then navigating to the entity also works:
GET on
/api/v1beta3/namespaces/staging/services/redis-service
Navigating to the namespace itself produces 404 'namespaces "staging" not found':
/api/v1beta3/namespaces/staging/
Getting all namespaces also doesn't list that namespace which sounds reasonable given the above.
I'd expect one of the two:

most standard approach: a clear error that this namespace doesn't exist when attempting to create an entity on it.
less preferred imo, but better than nothing: creating silently behind the scenes that namespace

cc @derekwaynecarr

Sounds like an error configuring your environment.  Is this using a standard provider environment or did you install your own?
The kube-Apiserver needs the following flags set for admission_control=NamespaceLifecycle,NamespaceAutoProvision,LimitRanger,ResourceQuota
In your case NamespaceAutoProvision must not be running.
Sent from my iPhone

On Apr 10, 2015, at 3:24 AM, abonas<EMAIL_ADDRESS>wrote:
Tried to create a service on a namespace 'staging' which doesn't exist on my k8s.
Creation succeeded which is odd enough :)
Then navigating to the entity also works:
GET on
/api/v1beta3/namespaces/staging/services/redis-service
Navigating to the namespace itself produces 404 'namespaces "staging" not found':
/api/v1beta3/namespaces/staging/
Getting all namespaces also doesn't list that namespace which sounds reasonable given the above.
I'd expect one of the two:

most standard approach: a clear error that this namespace doesn't exist when attempting to create an entity on it.
less preferred imo, but better than nothing: creating silently behind the scenes that namespace

cc @derekwaynecarr
—
Reply to this email directly or view it on GitHub.

Admission control option
NamespaceExists does your goal for 1.  It is what OpenShift uses.
NamespaceAutoProvision does 2.  It is used upstream right now, but I think upstream should switch to NamespaceExists and require explicit creation, but that is harder to enable for things like e2e until more work is done.
Sent from my iPhone

On Apr 10, 2015, at 3:24 AM, abonas<EMAIL_ADDRESS>wrote:
Tried to create a service on a namespace 'staging' which doesn't exist on my k8s.
Creation succeeded which is odd enough :)
Then navigating to the entity also works:
GET on
/api/v1beta3/namespaces/staging/services/redis-service
Navigating to the namespace itself produces 404 'namespaces "staging" not found':
/api/v1beta3/namespaces/staging/
Getting all namespaces also doesn't list that namespace which sounds reasonable given the above.
I'd expect one of the two:

most standard approach: a clear error that this namespace doesn't exist when attempting to create an entity on it.
less preferred imo, but better than nothing: creating silently behind the scenes that namespace

cc @derekwaynecarr
—
Reply to this email directly or view it on GitHub.

Sounds like an error configuring your environment. Is this using a standard provider environment or did you install your own? The kube-Apiserver needs the following flags set for admission_control=NamespaceLifecycle,NamespaceAutoProvision,LimitRanger,ResourceQuota In your case NamespaceAutoProvision must not be running.

@derekwaynecarr
this is the hack/local-up-cluster.sh
I didn't find the admission_control set there at all,
perhaps it's a bug in that particular setup?


Error: cloud-frontier-frontend-dev already exists.

It seems the bucket I need is created twice using CloudFormation which gives me an error.
Serverless: Packaging service...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...
CloudFormation - CREATE_IN_PROGRESS - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_COMPLETE - AWS::S3::Bucket - ServerlessDeploymentBucket
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
CloudFormation - CREATE_COMPLETE - AWS::S3::BucketPolicy - ServerlessDeploymentBucketPolicy
CloudFormation - CREATE_COMPLETE - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
Serverless: Stack create finished...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::S3::Bucket - S3BucketFrontend
CloudFormation - CREATE_FAILED - AWS::S3::Bucket - S3BucketFrontend
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
CloudFormation - DELETE_COMPLETE - AWS::S3::Bucket - S3BucketFrontend
CloudFormation - UPDATE_ROLLBACK_COMPLETE - AWS::CloudFormation::Stack - cloud-frontier-frontend-dev
Serverless: Operation failed!
Serverless: View the full error output: https://eu-west-3.console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stack/detail?stackId=arn%3Aaws%3Acloudformation%3Aeu-west-3%3A990417434658%3Astack%2Fcloud-frontier-frontend-dev%2F0eeb5d40-d9a1-11eb-b25f-0a6b74c6c6e4

  Serverless Error ---------------------------------------

  An error occurred: S3BucketFrontend - cloud-frontier-frontend-dev already exists.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.17.1
     Framework Version:         1.83.3 (local)
     Plugin Version:            3.8.4
     SDK Version:               2.3.2
     Components Version:        2.34.9

cat: can't open 'frontend/temp-frontend-output': No such file or directory
rm: can't remove 'frontend/temp-frontend-output': No such file or directory
Front-end deplyed!

Deployment is complete! Visit https:// to get started.

I think the actual problem is that the S3 bucket names are the same for everyone who deploys the stack since they don't contain something unique (such as the AWS account ID to which the stack is being deployed).
Adding your AWS account ID in the S3 bucket names to make them unique should work.
-   S3BucketFrontend: cloud-frontier-frontend-${opt:stage, 'dev'}
+   S3BucketFrontend: cloud-frontier-frontend-#{AWS::AccountId}-${opt:stage, 'dev'}

Hi, thanks for the quick pull request. However, I get the error InvalidBucketName: The specified bucket is not valid.now.
Stack Outputs
ServerlessDeploymentBucketName: cloud-frontier-frontend-serverlessdeploymentbuck-15ox64bwphbj6
FrontendDomain: d3kovxk8rjbpp.cloudfront.net

S3 Sync: Syncing directories and S3 prefixes...

  Invalid Bucket Name ------------------------------------

  InvalidBucketName: The specified bucket is not valid.
      at Request.extractError (/cloud-frontier/node_modules/aws-sdk/lib/services/s3.js:714:35)
      at Request.callListeners (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
      at Request.emit (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/cloud-frontier/node_modules/aws-sdk/lib/request.js:688:14)
      at Request.transition (/cloud-frontier/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/cloud-frontier/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /cloud-frontier/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/cloud-frontier/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/cloud-frontier/node_modules/aws-sdk/lib/request.js:690:12)
      at Request.callListeners (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at Request.emit (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
      at Request.emit (/cloud-frontier/node_modules/aws-sdk/lib/request.js:688:14)
      at Request.transition (/cloud-frontier/node_modules/aws-sdk/lib/request.js:22:10)
      at AcceptorStateMachine.runTo (/cloud-frontier/node_modules/aws-sdk/lib/state_machine.js:14:12)
      at /cloud-frontier/node_modules/aws-sdk/lib/state_machine.js:26:10
      at Request.<anonymous> (/cloud-frontier/node_modules/aws-sdk/lib/request.js:38:9)
      at Request.<anonymous> (/cloud-frontier/node_modules/aws-sdk/lib/request.js:690:12)
      at Request.callListeners (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
      at callNextListener (/cloud-frontier/node_modules/aws-sdk/lib/sequential_executor.js:96:12)
      at IncomingMessage.onEnd (/cloud-frontier/node_modules/aws-sdk/lib/event_listeners.js:336:13)
      at IncomingMessage.emit (events.js:387:35)
      at IncomingMessage.emit (domain.js:470:12)
      at endReadableNT (internal/streams/readable.js:1317:12)
      at processTicksAndRejections (internal/process/task_queues.js:82:21)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Have you installed all the dev dependencies mentioned in package.json (using npm i -D)?
If yes, and the error still persists, you can replace the #{AWS::AccountId} part in the bucket name with any string of your choice. That should definitely work.

I'm using the Docker container, so I assume everything is installed as it should be. I'll try the other approach

Seems it's working now, however I can't reach the url. Any idea how to reach it ?
AccessDeniedAccess DeniedT69DSX4Y3GYNGQCE4ETEXRccxi8uQ/vFZ3VioBhmt7KTGoJbxm/ASYVZOuC+cQN+S/KDbvz9xeLwuqBedqWNDoLwvZY=


System Default Date fields not getting updated for Custom WIT types

I have a azure devops org in which I have inherited Agile process template where we have added custom WITs with states New,Active,Resolved , Closed.
The custom WITs are not updated with system default date fields "Activated Date" and "Resolved Date" when the state changes to either "Active" or "Resolved". The same works for the existing work items of the template like Bug/User Story or Task.
Can anyone help me on how we can get this issue fixed for custom work items as well.

Hi @Prateek1584990,
I had the same issue and just created rules for custom WITs:
"When a work item state changes to Active then copy value from Microsoft.VSTS.Common.StateChangeDate to Microsoft.VSTS.Common.ActivatedDate"

You can actually use the field names (like Microsoft.VSTS.Common.ActivatedDate) in the drop downs of the rule editor even if they are not listed. I also added rules to clear the activated or closed date of the customs WITs if the work item is moved back to a previous state.

Hi @Gizmo0815,
When I try making a post call on creating rule it gives me below response,
Rule: When state changes to "Active", copy value from Microsoft.VSTS.Common.StateChangeDate to Microsoft.VSTS.Common.ActivatedDate"
Response:
{
"$id": "1",
"innerException": null,
"message": "VS402809: System.ChangedDate is an invalid value for field type DateTime. Change the field value and try again.",
"typeName": "Microsoft.Azure.Boards.WebApi.Common.FieldRuleModelValidationException, Microsoft.Azure.Boards.WebApi.Common",
"typeKey": "FieldRuleModelValidationException",
"errorCode": 0,
"eventId": 3000
}

I configured the rules in the UI not over the API. Make sure all WITs have the closed and activated fields added to their layout and that they are of the date/time type. Are you using Azure DevOps Service or Server?

Hi @Gizmo0815
We want to try it with API calls and we want our tool to update rules irrespective of Azure Devops Services (or) Azure Devops Server.

Hi @Gizmo0815
We tried from UI and using postman api calls as well, (in the Azure Devops Services)  it did not work for custom WITs to changed the system default date fields for them.
Request your help on the same

Hi All,
Can anyone help me by providing resolution
Thanks

Hi @Prateek1584990,
I set up a new organization in Azure DevOps and tried to rebuild what is working in my other organization and ran into the same issue like you. I suppose MSFT removed these fields from the rules and didn't want to break existing ones. I better don't touch these rules anymore. The only thing I can image that you could do is an Azure Logic App that triggers on state changes and sets the dates in the corresponsing fields. However each execution of the logic app will cost you a small amount.

@Prateek1584990: The latest release should fix your problem: https://docs.microsoft.com/en-us/azure/devops/release-notes/2020/sprint-172-update#improved-rules-for-activated-and-resolved-fields

H @Gizmo0815 , Thanks for the response , i did a quick check by creating a copy of Agile template and adding custom WIT type "test" with Active and Resolved state changes . but i dont see it is reflected.PFB the snip

@Prateek1584990: Check if your tenant already has the Sprint 172 Update (lower left corner on your organization project page), they seem to roll out in different waves. Sprint 172 Update is from July 9th. One of our customers tenants is always some weeks ahead of our own tenant.

Hi @Gizmo0815
I see that Sprint 171 Update is there on the left corner on my org page.
So, do you have any idea what would be the maximum time to roll out to all the orgs?

We will release an update addressing previous problems. For new issues, create a ticket, and we'll resolve it promptly. Thank you .


Other activation function other than tanh?

Just wondering have you try other activation function other than tanh? Do you think it is worth exploring other activation functions? Is there any motivation for choosing tanh?

tanh is everywhere in RNN-based NMT so I gave it a try and it worked. I haven't tried other activation functions so I can't say.


Find a solution for linking schemes with abstract classes.

Description
The original requirement describes:
Associating each schema file and concept scheme with the appropriate abstract class(es)

Currently, the user just specifies the URL for both the abstract classes for a Configuration Profile as well as for the scheme file. So there's no way of knowing which abstract class to show the user in a list to pick from and link to the scheme.
Note
We need to do this linking step imperatively before activating the Configuration Profile, because the user needs to use it for mappings, and the schemes will not be linked to any domain if we activate the CP without doing the linking. So the domains will be all without a spine.
Proposals

Parsing/Processing the file right when the user specifies the URL for it (Abstract Classes). This implies saving the abstract class to the DB before the Configuration Profile is "active". So we should avoid repeating this process then.
Pros: Makes the final process of activating a little bit faster.
Cons: What if the user changes the URL (if we allow it on edit) of the abstract classes being the Configuration Profile still "inactive" or "incomplete"? Should we remove the previously created one, create a new one (with all its domains, remember it will not be a single entity in DB, its multiple) and link the new one?

Process the content of the file to get the concepts (domains) and show it to the user in a list (maybe saving it into an additional JSON attribute of the configuration profile) and listing from there.
Pros: Does not change too much from the current implementation.
Cons: Same as previous but simpler IMHO.

More Notes

Do we need wire-frames? No. Just go with Proposal N.2
Does linking a scheme with a domain at CP creation stage means creating a spine for that domain?

Thoughts @stuartasutton @jeannekitchens ?

Sounds like the second option makes the most sense.

Technical proposal

Add an additional attribute to the configuration profile to save the abstract classes as JSON.
Prevent to modify if the Configuration Profile is already active.
If changed, remove the schema files existing associations in step 4.
Show the abstract classes in a list to associate to a schema file in step 4.

@stuartasutton @philbarker @jeannekitchens  comments?

@richard0000, would #2 "modify" after activation include prohibiting the addition of another abstract class?  Otherwise, proposal looks good to me.

LGTM, assuming point 2 is taken in conjunction with the modifications to the Config Profile discussed in #148

@stuartasutton since the current implementation permits adding abstract classes by specifying a URL, this always adds a whole set of abstract classes, removing the previous one. So adding a single abstract class would mean, changing the original file, make those changes available online and referencing it again in the form, so this way it would mean a complete change for the platform, even if the file looks the same.
The solution would be to change the reference by URL creation of abstract classes and let the user specify the abstract classes by typing (label, URI for each) and not by importing it by URL nor upload. Internally, we can generate the skos structure. This way adding single abstract classes would always be permitted, and we can easily block removal if the CP is active. If this looks reasonable, we can do the same for mapping predicates, even for concept schemes, since all are skos files. @philbarker FYI
Thoughts?

I think we will likely have to hash this out when we in a meeting since I'm uncertain why it is a problem to make a change in an already uploaded skos file and subsequently before the configuration is finished to edit that file and reload. I just need help in understanding why that is a problem.
I know we had a brief discussion about enabling the creation of the needed skos files for abstract concepts and mapping predicates through manual entry of the concepts and their URIs during configuration. In the end, I think that being able to do this manual entry should be one of three means for entering the data:

Upload of a skos file from a local directory;
Access remotely over the internet; and
Enter the concepts directly/manually during configuration with the DESM generating the skos file.

I do think we need to consider what kinds of reuse and interoperability problems we have if every time someone creates a new configuration--even where using the same concepts--get the same concepts reentered with different URLs etc.

Putting this on hold as a new feature request. The current project does not contain any use cases or mockups for how these files should be used after uploading, so this needs to be scoped and quoted separately.


341 Add TCIA loading tutorial

Fixes #341 .
Description
This PR added the tutorial to load TCIA CSV file and execute the following processing.
Status
Ready
Checks

[ ] Notebook runs automatically ./runner [-p <regex_pattern>]

Hi @wyli ,
Thanks for your review.
I updated the tutorial based on your comments.
Could you please help review it again?
Thanks.

thanks, it looks nice to me. @kirbyju could you please help review this notebook? https://github.com/Project-MONAI/tutorials/blob/82e1e623c2cfaad3b3dd94db537bb743dce523a6/modules/tcia_csv_processing.ipynb
(for the tag issue, you can use https://gist.github.com/edrohler/7a0c2ed0db15db768271, maybe the root cause is that your fork doesn't have all the tags from the core codebase)

Hi @wyli ,
I have updated the PR based on online comments, could you please help review it again?
Thanks in advance.

Might want to make a copy of the CSV data to our shared drive? https://drive.google.com/drive/u/2/folders/1ZOqwF44newvq-jOSgnbXRDWRD9DRq8hH

Might want to make a copy of the CSV data to our shared drive? https://drive.google.com/drive/u/2/folders/1ZOqwF44newvq-jOSgnbXRDWRD9DRq8hH

Hi @wyli ,
Thanks for your suggestion.
I think this CSV file only works for this first step as a temp file(hand-made and only the first lines is useful), we can upload to MONAI google drive later when we have a finalized version for real-world training tutorial.
What do you think?
Thanks.


AzPainter 2.1.1 released

https://github.com/Symbian9/azpainter/releases/tag/v2.1.1

released  https://github.com/abcang/azpainter-mac/releases/tag/2.1.1

Thanks! ;-)


is the SRC-20 API working?

Hi, can someone tell me if this is working? endpoints give an internal server error. Any help or update would be appreciated!

Details on the src-20 API is  now located at https://stampchain.io/docs


Openapi node update fix

PR Description
My guess is obsolete rimraf behavior is changed with node 16 and it doesn't fall silently anymore on this with node 16 (like it did in 14), so this check should fix it
Fixed Issue(s)

Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

Maybe we can also update lockfile version to 3 as per last PR which was having issues: https://github.com/Consensys/teku/commit/8eb737df45d9b22649f02cdc4d26387cdf30dbce

lockfile was not updated, we don't need to change it probably


[FIX] trakt_list „Method Not Allowed“ #2165

Motivation for changes:
Unable to use
Detailed changes:

fixed API-URL

Addressed issues:

Fixes #2165

True trakt_add is not working for me either, will have a look into it during the week


vscode + luasnip snippets

My init looks like this:
local ls = require("luasnip")

local javascript = require('snippets.javascript')
local javascriptreact = require('snippets.javascriptreact')

ls.snippets = {
    javascript = javascript,
    javascriptreact = javascriptreact,
}

ls.filetype_extend("typescript", { "javascript" })
ls.filetype_extend("javascriptreact", { "javascript" })
ls.filetype_extend("typescriptreact", { "javascriptreact", "typescript", "javascript" })

require("luasnip/loaders/from_vscode").lazy_load()

this makes a lot of duplicates because it seems the luasnip extend is additive with the package.json extend.
Is there a workaround for this?

Well, the easiest way out would be to change the package.json to not include javascriptreact,... in javascript.

ok, im sry it seems i didnt mention i use friendly-snippets. so yes i can probably fix this by changing the package json so no extends exist there, but then ill have to do it again every time friendly-snippets updates.

IMO the nicest solution is to not add the snippets in the first place.
The best case would be that each source contains exactly the snippets it advertises and the snippet engine can take care of providing snippets from multiple sources if so desired.

Basically no extends in friendly-snippets and use luasnip to extend? if that is so i think i agree. I have already opened an issue in friendly snippets, i'll mention this


Execute a stored procedure

Hello,
Without digging through the code, can this package execute an anonymous pl/sql procedure or a stored one?
begin
mystoredprocedure;
end;
/
or
begin
if .....
end if;
yadda-yadda
end;
/
Thanks,
Spencer

Yes, it is possible on the server side only. Here is the working sample:
  var sql = "begin dbms_random.seed(:seed); :ret := dbms_random.random; end;";
  var params = {
    seed: 1234,
    ret: {
        dir: NpmModuleOracledb.BIND_OUT,
        type: NpmModuleOracledb.STRING,
        maxSize: 50
    }
  };

  var db = OracleDB.getDatabase(Oracle._defaultOracleOptions);
  var result = db.executeCommand(sql, params);

  console.log(result.outBinds.ret);

The OracleDB class is exposed by the meteor-oracle package and you can refer to node-oracledb npm package through NpmModuleOracledb.


Add GoSec workflow

Closes: #2007
Here we introduce the GoSec scan workflow to the Github Actions pipeline and deal with all discovered problems.
Summary of #nosec exceptions and their motivation:

https://github.com/keep-network/keep-core/blob/4bc88c6185b9df3b904733270f6ac091af45fc89/config/config.go#L18
Rule G101: Look for hardcoded credentials
Motivation: This line doesn't contain any credentials. It's just the name of the environment variable.

https://github.com/keep-network/keep-core/blob/4bc88c6185b9df3b904733270f6ac091af45fc89/pkg/chain/local/local.go#L508
Rule G404: Insecure random number source (rand)
Motivation: Local chain implementation doesn't require secure randomness.

https://github.com/keep-network/keep-core/blob/4bc88c6185b9df3b904733270f6ac091af45fc89/pkg/net/local/identity.go#L26
Rule G404: Insecure random number source (rand)
Motivation: Local network identity doesn't require secure randomness.

@pdyraga all comments addressed.

Some additional explanation about On* function changes in the chain interface:
We decided to update signatures of all On* functions to do not return an error and log the possible error from Watch* inside ethereum.go. On* and Watch* never return an error. Watch*, in case of an error, tries to resubscribe until the subscription is canceled. Refactoring Watch* function signature is out of the scope of this PR and should be done when working on https://github.com/keep-network/keep-core/issues/491 but we could at least do some cleanup and not let the incorrect interface leak out of ethereum.go.


Add 2-component vector constructor

Implement RFC: 2-component vector constructor. This includes 2-component overload for vector.create and associated fastcall function, and its type definition. These features are controlled by a new feature flag LuauVector2Constructor. Additionally constant folding now supports two components when LuauVector2Constants feature flag is set.
Note: this work does not include changes to CodeGen. Thus calls to vector.create with only two arguments are not natively compiled currently. This is left for future work.

Thanks for the comments. Pushed a revision that addresses all the points mentioned.
Also added a new micro benchmark for vector library.  Test results:
LuauVector2Constructor false:

Test
Min
Average
StdDev%
Driver

vector: create
71.122ms
72.708ms
1.727%
luau.exe

LuauVector2Constructor true:

Test
Min
Average
StdDev%
Driver

vector: create
71.517ms
73.315ms
1.447%
luau.exe

Command line used:
python bench.py --vm d:\luau\cmake\RelWithDebInfo\luau_new.exe --folder micro_tests --run-test test_vector_lib --extra-loops 50
Tested on: Windows 10, MSVC 2022, Intel i5-3570K
The difference is a bit less than 1% in favor of the old version. I don't think this will make a big difference in practice though.
I also ran the test with only two arguments (the new LuauVector2Constructor path):

Test
Min
Average
StdDev%
Driver

vector: create2
51.412ms
52.479ms
1.487%
luau.exe

Creating vectors from two components is now much faster as expected.
An idea for future work: we could consider adding a new built-in for the 2D case (luauF_vector2). This would get rid of the extra branch in luauF_vector and get back that 1% and make luauF_vector2 even faster. This would require detecting the number of arguments in the compiler and choosing the correct builtin. Not sure if it makes sense to add complexity for such a small gain though.


Add safe directory settings to global git config

Fixing issue where git tool does not recognizes current repo as a git repo / safe directory
https://stackoverflow.com/questions/71849415/i-cannot-add-the-parent-directory-to-safe-directory-in-git
Signed-off-by: Dennis Meister<EMAIL_ADDRESS>

This PR needs to be merged to create a new release tag (1.2.1) and update it than in the github workflows


[DOC] Alerting - Support Composite Monitors

What do you want to do?

[ ] Request a change to existing documentation
[x] Add new documentation
[ ] Report a technical problem with the documentation
[ ] Other

Tell us about your request. Provide a summary of the request and all versions that are affected.
A composite monitor is a workflow that requires user to define a sequence of bucket/doc/query level monitors. When a composite monitor is executed it executes a sequence of monitors. User may choose to use the findings of one monitor as a filter input to a following monitor in the same sequence. All the monitors defined in a composite monitor sequence are called delegate monitors. Composite monitors allow users to execute delegate monitors in sequential fashion hence providing more control over the data seen by composite monitors which will enable creation of more granular findings and chained alerts.
What other resources are available? Provide links to related issues, POCs, steps for testing, etc.
https://github.com/opensearch-project/alerting/issues/697
https://github.com/opensearch-project/alerting/issues/524

@eirsep Thanks for creating an issue for this. I've begin looking at the resources for this 2.8 enhancement. Do expect to have a test environment you could share at some point so I can experiment and begin to understand how the monitor creation and alerting chaining works for this new composite monitor type? Thanks.

This will be pushed to 2.9.


Introduce a GlobalSlamResultCallback

RFC: https://github.com/googlecartographer/rfcs/blob/master/text/0020-global-slam-callback.md

What a coincidence. I've just started working on this like 5 minutes ago
:-)

(Please link the RFC in the first comment/commit message.)

@cschuet what if this was the last item in the work queue, and constraint_builder_.WhenDone will not fire again (e.g. we are shutting down)? WaitForAllComputations in the destructor will wait indefinitely until work_queue_ gets reset. I am getting this kind of deadlock, where I get stuck with an empty (not nullptr!) work_queue_.

@ojura Interesting, what do you mean by 'this was the last work item'? I don't think this commit changed the way 'WhenDone()' is called or the work items on the queue. So is this a general observation unrelated to the introduction of the global SLAM optimization callback?

Sorry, that was purely me, I messed up multithreading. In the main thread, before destroying the map builder, I locked a mutex which I also lock in the global SLAM callback.
WaitForAllComputations (which got invoked in the main thread, in the pose graph destructor) got stuck because the other Cartographer thread was trying to lock the mutex in the global SLAM callback. This appeared as an infinite Optimizing.... message stream from Cartographer, so my first reaction was to bug you. Sorry about that :-)

@ojura No problem at all. Glad you were able to pinpoint the root so quickly and of course that I did not cause it :)


updated for 1.0

@conda-forge-admin, please rerender
not tested on Windows

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Is it WSL related? ^Space && ESC-Space don't work in Nano

When I try to navigate between words in Nano in WSL Ubuntu 14.04 with ^Space || ESC-Space, nothing happens.
Is there an alternative key or something that needs to be activated for this navigational functionality? Thank you,

Works for me in 14393.222.

@Benosika What Windows version are you running?

Works for me in 14936.1000

Sorry for not mentioning. Its Home 14393.222

That's curious. Esc-Spc doesn't work? I'm aware that C-Spc doesn't work correctly in WSL (C-@ doesn't  either) but Esc-@ and Esc-Spc should be fine. Alt-@ and Alt-Spc should also work, and both the Alt- and Esc- chords are mapped to the M- key chord.

I am using an English-Hebrew keyboard by Logitech, I bought years ago.

@zadjii-msft I tried both English (United States) and Russian keyboard layouts when I first checked. Both worked like a charm.

Works here on norwegian keyboard. Could it be limited to hebrew keyboards?

@sundhaug92 nope, just checkeed, typing Esc-Spc works in Hebrew as well.

I'm opening up an internal bug to track this, #9194817.
@Benosika If you're still running into this, can you show me a screenshot of what Ctrl+v, Alt+Spc outputs from a bash prompt? Also, try opening emacs, pressing Ctrl-H, K, Alt-Spc and Ctrl-H, K, Esc-Spc and see what gets output there. It should be something like this:
M-SPC runs the command just-one-space, which is an interactive compiled Lisp function.

The output of ctrl+v, alt+spc in a fresh bash prompt is this: ^[.
Tried to do Ctrl-H, K, Alt-Spc and then Ctrl-H, K, Esc-Spc but niether Emacs opened nor any output was printed besdies ^[ again.

Okay, I can't repro this locally, so i'm going to need you to help me pin down if it's a nano problem or an actual console/WSL problem.
Can you please try the following steps?

tmux to launch tmux (you may need to apt install tmux)
C-b, : To open up the tmux command line
bind-key -n M-Space display Meta-Space
bind-key -n C-Space display Ctrl-Space
Try pressing Alt-Space and Ctrl-Space, the bottom line should have show the corresponding messages briefly.

Sadly I don't have much time to dive in to this right now. After I type tmux there appears a green status bar in the end of CMD, but no such command c-b is recognized..

Sorry, my mistake. I'm using emacs syntax there. I've updated the instructions.

C-Space neither works for me in emacs. I'm running: Microsoft Windows [Version 10.0.14393]
And did do-release-upgrade a week ago (when I've installed WSL)

@gczuczy It's a known issue that C-Space doesn't work in the initial release of WSL, which is windows 14393. As on Windows Insiders build (I actually don't know which number it is), C-Space should be re-enabled.
Esc-Space should have always worked, so I'm still not exactly sure what is causing @Benosika's bug.


Give ChangeDetectorAgent access to last_property

This allows user to detect a new lowest value or a new high by comparing a new value with the existing value.

Perhaps we need new Liquid filters such as greater/lesser to make numeric comarison less messy...

Yea, that might make sense. This looks good though.

@cantino Thanks, I'll file another PR when I think of good syntax for numeric comparison.


build: add secondary entries

close #700
PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/NG-ZORRO/ng-zorro-antd/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[x] Build related changes: support both `NgZorroAntdModule` and secondary modules
[ ] CI related changes
[ ] Documentation content changes
[ ] Application (the showcase website) / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: #700
Users could only import NgZorroAntdModule to their modules, thus bundles are not optimized enough for projects that only use several the components.
What is the new behavior?
Secondary modules and entries are available now. Say a user only wants to render a NzButton on the screen, he can import NzButtonModule instead of `NgZorroAntdModule. Building is faster and the bundle's size is reduced from 432KB to 242KB.

Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

Things that need extra attention:
Circular imports:
These modules have circular imports:

menu & dropdown
tree & tree-select (resolved)

@vthinkxie stackblitz 能改成按需引入吗？这样可能会提高加载速度，也许还能暴露一些 bug，可以先发 next 测试一下

@vthinkxie stackblitz 能改成按需引入吗？这样可能会提高加载速度，也许还能暴露一些 bug，可以先发 next 测试一下

可以的，你们先review完，我这边还有一些工作，合并完之后可以先发next


Document On Chain Dataset Storage

[ ] Explain needed properties for dataset storage in the smart contract
[ ] Explain design descisions for modeling nft ownership (EIP1155?, FA2) (visualize?)

Document in wiki section

Potentially Useful:

Refungible Token Standard. Partially documented for Ethereum (article, EIP draft). Differences possible, since use case is not fully identical.
https://github.com/ethereum/EIPs/issues/864

What exactly we will store in the NFT is not yet clear, as we have not completed the implementation. As far as I can tell, we want to comply with the FA2 standard (tzip-12), which requires (MUST) the following functions: transfer, balance_of, update_operators. Beyond that we will have other functions like mint or buy etc. As of now, I see in the interface https://gitlab.com/tezos/tzip/-/blob/master/proposals/tzip-12/fa2_interface.mligo that the token_id field is assumed. In addition, there is a field token_metadata_storage that conforms to the metadata standard and provides a bigmap with the tuple (token_id, token_metadata), where token_metadata is another tuple from (token_id, (string, bytes) map). According to tzip-16, the (string, bytes) map type provides the variable where a token-specific metadata uri is stored. In our case this would be the IPFS link. This way of storing seems a bit unusual to me at first, but certainly has its reasons, which we will clarify. The following example helped me a lot to understand the structure of the metadata attribute in the smart contract: https://github.com/octo-technology/Tezos-Land-FA2-NFT-Boilerplate/blob/4a5d259836e219e376fff453daef17be20629934/src/contracts/migrations/1_deploy_tezos_land.js#L55-L60 (please don't get confused by the contract_metadata, which can also be specified optionally and describes metadata of the respective smart contract (NOT that of the token)). A good example of the JSON blob stored in IPFS can be found here: https://gitlab.com/tezos/tzip/-/blob/master/proposals/tzip-21/examples/example-020-digital-collectible.json. It corresponds to tzip-21, which extends the Base Token Metadata standard from tzip-16. Long story short, so far we need the token_id and token_metadata attributes (the IPFS uri) in the NFT.

Datasets uploaded by the provider should be represented as Non Fungible Tokens:

Dataset is unique
Owner is the respective provider
Token Metadata is used to store Off-chain data

Problem
Consumers have the option to buy datasets and thus become owners of the dataset NFT themselves. However, the NFT standard specifies that only one account can be the owner of the NFT. This owner has special privileges regarding token management (burning, transfer, token metadata updates...) that only the provider should have.
Possible Solutions

Introduce a new attribute (e.g. buyers or consumers) to indicate who bought the NFT without changing the owner attribute.
Use Fungible Tokens (FT) to manage purchasing of dataset NFTs. A new FT is created when a provider mints a new datset NFT. Consumers can only buy these Fungible Token.


Clean up landing page + instructions

More work on #55

[x] add announcement banner (link to user survey)
[x] move Tutorial Setup from 'Fundamentals/Introduction' to front page
[x] clean up overview notebook
[x] use jupyterbook cell metadata tags for scrolling outputs, etc. on website

@dcherian I simplified the Overview notebook a bit, mainly reducing the length of # Titles so that navigating the notebook with the right hand browser is easier. Also removed the creation of a dask cluster as I think that can get kind of confusing (processes vs threads), so in the interest of an overview I thought the simplest default configuration (just specifying chunks) would be good. Could add a link to https://docs.dask.org/en/stable/scheduling.html and mention the single-machine scheduler is used and go into more detail into intermediate or advanced sections

Going to merge this in now that CI is passing!


Update gitignore

Is your feature request related to a problem? Please describe.
gitignore does not cover all the files generated by running tests, so it incorrectly shows files in git status.
Describe the solution you'd like
update the .gitinore
Describe alternatives you've considered
Fix weird git submodule issues, or complain.

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[x] 1. Is it tagged with a type: defect or task?
[x] 2. Is it tagged with a priority: critical, normal or minor?
[x] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[x] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[x] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[x] 1. If the issue is a defect, is the defect fixed?
[x] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[x] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[x] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
[x] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

checklist is good.


Pass page context variables into static queries

Summary
If I understand well, at this moment, static query did not accept any input variable at all. In some cases in not possible to get data with a fixed query, for example when website is localized. Need to access page context variables to retrieve the correct data in the current context.
Basic example
Creating two different page that use the same component but with a different locale:
//gridsome.server.js

module.exports = function (api) {
  api.createPages(({ createPage }) => {
     // Create an english page
     createPage({
        path: `/en/page/`,
        component: './src/templates/Page.vue',
        context: {
          locale: 'en-gb'
        }
      })

     // Create an italian page
     createPage({
        path: `/it/page/`,
        component: './src/templates/Page.vue',
        context: {
          locale: 'it-it'
        }
      })
  })
}

Inside Page.vue is included a component ./src/components/Sidebar.vue that has the static query:
<static-query>
query($locale: String) {
   SidebarItem(id: "sidebar", language: $locale) {
      content {
        _uid
        name
        subtitle
      }
   }
}
</static-query>

the id parameter can be hard-coded but the locale not. Is too awkward to clone each component for each language (./src/components/SidebarEN.vue, ./src/components/SidebarIT.vue, ..) and include it dynamically (or using v-if based on locale).
Motivation
Some datas cannot be retrieved with only hard-coded parameters but need to depends from page context.

@daaru00 Could you not query that data in Page.vue with <page-query>, and pass as a prop to the sidebar component?

@thetre97 nope, I can't merge it together. Sidebar has too many fields so I need to retrieve it separately.

Bump

I'm not sure that this will happen, as the point of it being called a static query is because it doesn't receive variables.
A couple of workarounds is to either query the data in a page-query and pass down via props, or fetch all variations and use a computed property to filter and get the right one for the current locale.

The first one is the natural way for me, just makes the query quite long. The second option would increase the site size as the data is preshiped and the lubricated right?

Von: Travis Reynolds<EMAIL_ADDRESS>Gesendet: Monday, February 8, 2021 8:37:26 PM
An: gridsome/gridsome<EMAIL_ADDRESS>Cc: Sak<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Betreff: Re: [gridsome/gridsome] Pass page context variables into static queries (#1107)
I'm not sure that this will happen, as the point of it being called a static query is because it doesn't receive variables.
A couple of workarounds is to either query the data in a page-query and pass down via props, or fetch all variations and use a computed property to filter and get the right one for the current locale.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/gridsome/gridsome/issues/1107#issuecomment-775392221, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABETAUSTBFA23V3UQLHYRXLS6A4PNANCNFSM4MLWII6Q.

Correct, it would - although it shouldn't be too much. This query for example
query($locale: String) {
   SidebarItem(id: "sidebar", language: $locale) {
      content {
        _uid
        name
        subtitle
      }
   }
}

Wouldn't generate a huge amount of JSON.
Hopefully fragments & importing graphql files will come soon...

Another use case for this: I'm using @gridsome/vue-remark. I want to create a "related posts" component that editors can embed in their Markdown somewhere to advertise related posts somewhere within the content. They should be able to pass the component some value, for example <related-posts :tag="tags[0]" /> to find other posts with the same tag. But then the component needs to do the query because the person writing the page (the very generic "MarkdownPost" page template) has no idea what components the Markdown author will write.
Any ideas on how to accomplish something like this?

also interested in this. it seems most static site generators can't do this, but it would be a killer feature, because it lets you move the data sourcing complexity into the components responsible for that data.


Document subctl benchmark verbose flag

Related to: https://github.com/submariner-io/submariner-operator/issues/748
Signed-off-by: Maayan Friedman<EMAIL_ADDRESS>

@skitt @nyechiel  We probably want to hold off on merging this until closer to the release. Perhaps change it to a Draft.

@skitt @nyechiel We probably want to hold off on merging this until closer to the release. Perhaps change it to a Draft.

Yeah, I agree that we should wait with the docs part. Maybe it's better to propose a new label, something like "do-not-merge"?

@skitt @nyechiel We probably want to hold off on merging this until closer to the release. Perhaps change it to a Draft.

Yeah, I agree that we should wait with the docs part. Maybe it's better to propose a new label, something like "do-not-merge"?

I converted it to a Draft - I think that should suffice.

@skitt @nyechiel We probably want to hold off on merging this until closer to the release. Perhaps change it to a Draft.

Yeah, I agree that we should wait with the docs part. Maybe it's better to propose a new label, something like "do-not-merge"?

I converted it to a Draft - I think that should suffice.

Yup, I don’t think we need a new label for this. I’ve added it to the 0.8.0 project and milestone so we don’t miss it when we wrap up the release.

Sorry, now it's conflicting, can you have a look @maayanf24  ?

Duplicate of https://github.com/submariner-io/submariner-website/pull/363


Update Upsert docs

We need to add more information to our upserts page https://www.prisma.io/docs/reference/api-reference/prisma-client-reference#upsert
The first section would add information about handling Unique Key Constraint errors. This can be added at any time.
A rough section to add would be something like this:
If there are many concurrent Prisma upserts, this can lead to an upsert operation throwing a unique key constraint error.
A Prisma Upsert works by first performing a read operation using the where clause in the Upsert operation to see if that Row or document exists in the database. If it does not then it will create that row. If it does exist it will update it. When two or more upsert operations are performed at the same time, there could be a race condition where both operations do not see the row and try and create the row. One of the operations will succeed and create the new row and the other operation will fail and return a Unique Key Constraint error. In most cases, the application code can receive this error and then try the upsert operation again and it should update the row.

We will also be adding support for Native Upsert support for Postgres, sqlite and CockroachDBhttps://github.com/prisma/prisma-engines/pull/3257 in the next release. This is via theON CONFLICT .. UPDATE` sql command https://www.prisma.io/dataguide/postgresql/inserting-and-modifying-data/insert-on-conflict
We need to add documentation around this. Something like:
For specific upserts, Prisma will optimise the query to use the sql command INSERT .. ON CONFLICT (constraint) Update. This will work in the following conditions:

The data connector supports it - Sqlite, Postgres and CockroachDB

The create and update arguments do not have any nested queries. It only works when the upsert is updating one Model/table

There is only 1 unique field in the where clause. If a Model has many unique fields defined for it, it will only work if the where clause contains only 1 of those unique fields. For example with a schema of:

      model TestModel {
              id Int @id
              num1 Int
              field String @unique
              email String

              @@unique([id, num2])
          }

If the upsert query is like this:
    prisma.testModel.upsert({
      where: {
        field: "a-unique-value
      },
      create: {
       id: 1,
       num1: 1,
       field: "a-unique-value",
       email<EMAIL_ADDRESS>       },
      update: {
         email<EMAIL_ADDRESS>       }

     });

Prisma will use the native upsert. But if we have a query with multiple unique in the where clause:
prisma.testModel.upsert({
       where: {
         field: "a-unique-value,
         num: 1,
         id: 1
       },
       create: {
        id: 1,
        num1: 1,
        field: "a-unique-value",
        email<EMAIL_ADDRESS>        },
       update: {
          email<EMAIL_ADDRESS>        }

      });

This it will use Prisma's upsert instead of the database's upsert functionality.

The unique field defined in where clause has the same value as defined in the create arguments

For the same schema as above, if the unique field defined in the where clause has a different value to the same unique defined in the create clause then it will not use the database's upsert functionality. For example this query:
     prisma.testModel.upsert({
       where: {
         field: "a-unique-value
       },
       create: {
        id: 1,
        num1: 1,
        field: "a-unique-value",
        email<EMAIL_ADDRESS>        },
       update: {
          email<EMAIL_ADDRESS>        }

      });

Will work because the field in the where clause is "a-unique-value" and the field in the create is also "a-unique-value". But this query would use Prisma's upsert instead of the database:
prisma.testModel.upsert({
       where: {
         field: "a-unique-value
       },
       create: {
        id: 1,
        num1: 1,
        field: "a-different-value",
        email<EMAIL_ADDRESS>        },
       update: {
          email<EMAIL_ADDRESS>        }

      });

because the field in the where clause is "a-unique-value" and the field in the create is "a-different-value"

Thanks Garren - I'll work on this next week.

👋 we're cleaning up our backlog of issues and are closing everything older than two years. If you still feel that this issue is relevant, please feel free to re-open and our team will take a look. Thank you!


Worker processes not being cleanup properly on heroku

I'm having problems with workers not being cleaned up properly on heroku. Since heroku uses different dynos per worker, its not possible to use the built in worker cleanup inside node-resque (workers will be 1 PID per dyno).
Here is the worker code I am using:
NR = require("node-resque")

worker = new NR.worker {connection: config.resque, queues: queues, timeout: 1000}, jobs, ->
  debug "Worker running on DYNO: #{process.env.DYNO}"
  worker.start()

worker.on 'error', (queue, job, error) ->
  debug("Worker error: " + queue + " " + JSON.stringify(job) + " >> " + error)

worker.on "end", ->
  debug "Worker has ended"

process.once "SIGTERM", ->
  debug "Worker issued SIGTERM, shutting down"
  debug "Worker is currently working" if worker.working is true
  worker.end ->
    debug "Worker has been untracked, quitting process"
    process.exit()

debug "Instance of worker loaded."

With heroku, they issue a SIGTERM to all processes when a dyno shuts down. This in our case will call worker.end and then process.exit(). If the process doesn't end within 10 seconds, heroku will issue a SIGKILL - we are not seeing it get this far in our logs.
Sometimes when a dyno is terminated, we see this in the logs (a healthy shutdown):
Note also that heroku logs are not deterministic in their ordering.
Aug 13 14:21:44 app/worker.2:  Wed, 13 Aug 2014 21:21:44 GMT Worker has ended
Aug 13 14:21:44 app/worker.2:  Wed, 13 Aug 2014 21:21:44 GMT Worker issued SIGTERM, shutting down
Aug 13 14:21:45 app/worker.2:  Wed, 13 Aug 2014 21:21:44 GMT Worker has been untracked, quitting process
Aug 13 14:21:45 heroku/worker.2:  Stopping all processes with SIGTERM

Note the untracked log in the snip above, this in my mind is a healthy shutdown.
However, sometimes it is terminated and we see this (resque thinks this worker is still working):
Aug 14 09:03:21 heroku/worker.4:  State changed from up to down
Aug 14 09:03:25 app/worker.4:  Thu, 14 Aug 2014 16:03:25 GMT Worker issued SIGTERM, shutting down
Aug 14 09:03:25 app/worker.4:  Thu, 14 Aug 2014 16:03:25 GMT Worker has ended
Aug 14 09:03:25 app/worker.4:  Error waiting for process to terminate: No child processes
Aug 14 09:03:26 heroku/worker.4:  Stopping all processes with SIGTERM
Aug 14 09:03:26 heroku/worker.4:  Process exited with status 22

I have also been following up with heroku support on this issue in case it's a problem with how they shut down instances, but it appears as though the problem is from within node-resque. They mentioned that sometimes node doesn't completely flush its logs before the process exits, this might be the case for why the logs are different above, but it should still untrack the worker in redis.
Happy to provide any more information and tests to help track this down, appreciate anyone who can help look into it.

Just wanted to add my +1 to this issue, I'm seeing the same problem with AWS instances running actionhero. I have 4 processes per EC2 instance, managed by forever. I also buffed the actionhero status page to show what is currently in the queue and what is being worked on. I think my issue is that when I do a deploy, the worker nodes are disconnected from Redis (similar to your heroku dynos) and this causes state confusion in redis.
I think even if workers are correctly handling SIGTERM, we have to assume that nodes in the cloud could just vanish with no warning, and our distributed job processing systems should be able to handle this. I wonder if there is some kind of internal health check that node-resque workers could use to verify the worker statuses in redis?

In the latest versions of actionhero, SEGTERM should gracefully allow the workers to shut down (if this isn't what you are seeing, please open an issue over here https://github.com/evantahler/actionhero).  We talked about this a little over here: https://groups.google.com/forum/#!topic/actionhero-js/BlLlf1JKk6s
To bring this thread up to date, here's the main issue:
Say you have a CPU-intensive job that you are working (computer the 100th Fibonacci number).  The node process will be 100% consumed by this task, so it will be unable to send any heartbeats during this time.  Because of this, node-resque has to assume that workers are unable to to ping-back with "status" as a job is working
The good news is that the start time of every worker is stored in redis for the current job.   I would love to see a PR for a new method like `queue.cleanOldWorkers(since, queue, callback).  You can pass millisecond delay and then loop through the workers, and delete any that are too load for your application (and yes, this would be per-application).  You also have access to the payload the worker was working on, so you can re-enqueue it so no jobs are lost.
Send in that PR!

Glad to see this issue getting a bit of love, I have since in the meantime been running a script on our resque web server which just cleans up jobs and re-queues them if they are old. It doesn't figure out how to unregister workers which are gone, but it at least makes sure jobs get run which die in the middle of running. I imagine this cleanOldWorkers method would do something similar.
Resque.workers.zip(Resque.workers.map(&:job)).each do |w, job|
  next unless job["run_at"]
  run_at = Time.parse(job["run_at"])
  minutes_ago = (Time.now.to_i - run_at.to_i) / 60
  if minutes_ago > 10 # This is application specific, if jobs take longer than 10min this won't work
    Resque.enqueue_to job["queue"], job["payload"]["class"], job["payload"]["args"]
    w.done_working
    w.unregister_worker
  end
end

For us, the jobs that get stuck are interfacing with another (much slower API) and making a lot of calls over HTTP. These calls are pretty inconsistent in their response time (so we can't really estimate the runtime), and sometime the API returns us HTML error pages instead of JSON (causing all kinds of exceptions) or JSON in a bad format (causing yet more exceptions in random places). The good news here is that when a worker is waiting for the HTTP request to come back, node-resque should be able to ping it and see that it is still working and not crashed. Updating our version of actionhero/node-resque should fix this for us.
I think adding a "kill all jobs that run longer than x seconds" config might be a bit dangerous, but probably pretty useful and I can certainly see the case for considering long running jobs to be failures of some kind.

Yeh, its really dangerous. In fact we have a whitelist now of jobs that we think will take longer than 10min and don't restart those. We also make sure all our jobs are retryable. Definitely not ideal, but without having control over the heroku scale down process I'm not seeing any options.

Here's a proposed solution: https://github.com/taskrabbit/node-resque/pull/70

Bump:  Any thoughts on https://github.com/taskrabbit/node-resque/pull/70?
I personally don't really like it, but other than migrating to better servers, it might required...

+1 on this PR!
We had several Issues with the combination of actionhero too where the Tasks are stuck when the actionhero is shutdown unexpectly

Ok, I'll merge #70 and publish up a new version.
I think that the combination of #70 and #74 provide the suite of tools needed to manage failed workers and failed jobs.
Finally closing this issue!

Oh, but @heroku's worker-kill behavior probably still needs work :D

I've run into some situations like this, too, but sadly I can't come up with a reproducible test. We're not on Heroku. Hopefully #70 helps.

I see this for app on heroku...we spawn child-processes and when our server shuts down our server issues SIGKILL to all of them, so I am guessing that my server is killing the workers before heroku tries

@ORESoftware it's important to let the workers 1) finish what they are working on and then 2) be allowed to shut down gracefully (which would include removing any artifacts of their running from redis).  I think Heroku only gives you 5 seconds for this... if your jobs last longer than this, you'll see problems.  also, if you are forking, only the parent will get the signal. how are you propagating the signal to the children... and again, how long does it take?

I'm locking this issue for now.
We can talk about this kind of thing in the chat room and discuss any options.


Api contract regions

Adding API Contract Test Case for regions endpoint.

Which Traffic Control components are affected by this PR?

Traffic Ops Test

What is the best way to verify this PR?
Step 1 : Install requirements
pip install -r requirements.txt
Step 2 : To execute divisions API Contract Testcase , navigate to traffic_ops/testing/api_contract/v4
pytest --to-user Username --to-password Password --to-url URL --config Config Path --request-template Request template --response-template Response template test_regions.py
Note: Config, Request template and response template paths are optional.
PR submission checklist

[x] This PR has tests
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY (see the Apache Software Foundation's security guidelines for details)

@gokulakrishnansvm - can you make the PR title more descriptive. maybe something like "Added Api contract tests for regions"


Delete Jekyll and move gridsome to the root

This PR deletes the Hub's Jekyll setup and keeps the Gridsome setup.

I think the netlify build failure is because this originally targeted main. Can't get it to rerun so I'm gonna see if merging gridsome-main back into this branch does anything

@alexmarple @aaronsteers This PR is ready to go and I did my best to keep the change scope as focused as possible.
I suggest if you want to review, look at the commits individually rather than the whole set. I've tried to keep each commit focused on a single step.

[ ] The netlify build failures will need to be resolved just prior to/just after merging because the gridsome preview site (https://meltano-hub-gridsome.netlify.app/) has it's build config set up via UI:

[ ] A follow up PR will be immediately created that deletes the hub-2.0/ui/src/assets/logos directory (and only that). This PR already switches us over to the correct directory, I just didn't want to balloon the change set any more than I already did.

@rwfeather re:

@alexmarple @aaronsteers This PR is ready to go and I did my best to keep the change scope as focused as possible.

Fantastic! Thanks for the quick work on this.

I suggest if you want to review, look at the commits individually rather than the whole set. I've tried to keep each commit focused on a single step.

👍

The netlify build failures will need to be resolved just prior to/just after merging because the gridsome preview site (meltano-hub-gridsome.netlify.app) has its build config set up via UI:

Understood 👍

A follow up PR will be immediately created that deletes the hub-2.0/ui/src/assets/logos directory (and only that). This PR already switches us over to the correct directory, I just didn't want to balloon the change set any more than I already did.

Sounds good!

@rwfeather - I see the latest merge has created conflicts. Once those are resolved, I'd be supportive of merging even if checks are not 100% passing. We can continue to debug those over in the gridsome-main branch, and merging sooner than later will help minimize extra work from future conflicts.


db error: ERROR: invalid input syntax for type uuid: "$2"

Bug description
I have the following query statement which I can run it with no problem on pgAdmin v4
SELECT 	(gitTree.obj ->> 'path'),
		(gitTree.obj ->> 'content_url'),
		gitTree.pos as  pos,
		bgr."backgroundJobId" as bgj_id
FROM "BackgroundJobResult" bgr
LEFT JOIN LATERAL jsonb_array_elements(bgr."data")
					WITH ORDINALITY AS gitTree(obj, pos) ON true
WHERE (gitTree.obj ->> 'path') ='utils/generateAllMetricsDocumentation.py' AND "backgroundJobId" = '6d988338-5791-42ff-9c37-e1fcd6f9a6b5';

However when I try to run this query statement with $queryRaw I get the following error
 error PrismaClientKnownRequestError:
 Invalid `prisma.$queryRaw()` invocation:

 Raw query failed. Code: `22P02`. Message: `db error: ERROR: invalid input syntax for type uuid: "$2"`
     at RequestHandler.handleRequestError (/home/xxxxx/xxxxx/xxxxxxx/app1/node_modules/@prisma/client/runtime/index.js:29909:13)
     at RequestHandler.request (/home/xxxxx/xxxxx/xxxxxxx/app1/node_modules/@prisma/client/runtime/index.js:29892:12)
     at async Proxy._request (/home/xxxxx/xxxxx/xxxxxxx/app1/node_modules/@prisma/client/runtime/index.js:30864:16) {
   code: 'P2010',
   clientVersion: '4.3.1',
   meta: {
     code: '22P02',
     message: 'db error: ERROR: invalid input syntax for type uuid: "$2"'
   }

How to reproduce

Expected behavior
[
    {
        "path": "utils/generateAllMetricsDocumentation.py",
        "content_url": "https://api.github.com/repos/xxxxxxx/xxxxxxx/git/blobs/88d9ae91c71ca30feeba4339db942e91a04dbab5
    }
]

Prisma information

generator client {
    provider        = "prisma-client-js"
    previewFeatures = ["filterJson"]
}

datasource db {
    provider = "postgresql"
    url      = env("DATABASE_URL")
}

model BackgroundJob {
    id           Int                    @id @default(autoincrement())
    jobName      String
    jobId        String                 @unique @db.Uuid
    jobPayload   BackgroundJobPayload?
    jobStatus    String
    jobProgress  BackgroundJobProgress?
    jobResult    BackgroundJobResult?
    jobError     String?
    jobCreatedAt DateTime               @default(now())
    jobUpdatedAt DateTime               @updatedAt
}

model BackgroundJobPayload {
    id              Int           @id @default(autoincrement())
    data            Json
    backgroundJob   BackgroundJob @relation(fields: [backgroundJobId], references: [id])
    backgroundJobId Int           @unique
    createdAt       DateTime      @default(now())
    updatedAt       DateTime      @updatedAt
}

model BackgroundJobResult {
    id              Int           @id @default(autoincrement())
    data            Json          @db.JsonB
    backgroundJob   BackgroundJob @relation(fields: [backgroundJobId], references: [jobId])
    backgroundJobId String        @unique @db.Uuid
    createdAt       DateTime      @default(now())
    updatedAt       DateTime      @updatedAt
}

model BackgroundJobProgress {
    id              Int           @id @default(autoincrement())
    status          String        @default("processing")
    currentStep     Int
    totalSteps      Int
    message         String
    backgroundJob   BackgroundJob @relation(fields: [backgroundJobId], references: [jobId])
    backgroundJobId String        @unique @db.Uuid
    createdAt       DateTime      @default(now())
    updatedAt       DateTime      @updatedAt
}

export async function openFileFromGithubRepo(filePath: string, backgroundJobId: string,  cb: (file: File) => void): Promise<void> {
    const queryStatement = Prisma.sql`SELECT (gitTree.obj ->> 'path') as path, (gitTree.obj ->> 'content_url') as content_url FROM "BackgroundJobResult" bgr LEFT JOIN LATERAL jsonb_array_elements(bgr."data") WITH ORDINALITY AS gitTree(obj, pos) ON true WHERE (gitTree.obj ->> 'path') = '${filePath}' AND "backgroundJobId" = '${backgroundJobId}';`;

    const queryResult = await prisma.$queryRaw<Array<{ path: string, content_url: string }>>(queryStatement);
}

Environment & setup

OS: WSL2 on Windows 10
Database: PostgreSQL
Node.js version: v16.16.0

Prisma Version
Environment variables loaded from .env
prisma                  : 4.3.1
@prisma/client          : 4.3.1
Current platform        : debian-openssl-1.1.x
Query Engine (Node-API) : libquery-engine c875e43600dfe042452e0b868f7a48b817b9640b (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli c875e43600dfe042452e0b868f7a48b817b9640b (at<EMAIL_ADDRESS>Introspection Engine    : introspection-core c875e43600dfe042452e0b868f7a48b817b9640b (at<EMAIL_ADDRESS>Format Binary           : prisma-fmt c875e43600dfe042452e0b868f7a48b817b9640b (at<EMAIL_ADDRESS>Format Wasm             : @prisma/prisma-fmt-wasm 4.3.0-32.c875e43600dfe042452e0b868f7a48b817b9640b
Default Engines Hash    : c875e43600dfe042452e0b868f7a48b817b9640b
Studio                  : 0.473.0
Preview Features        : filterJson

might be related to this #5083

$queryRawUnsafe works as expected! Is there something I am missing about $queryRaw?

@ahmetkca
Prisma.sql as well as $queryRaw do not create a string but rather prepared statement - object, where text and values are separate. This a safe way to pass raw query to the driver while avoiding potential injections. So, for example, Prisma.sql`SELECT ${foo}`  will produce { text: 'SELECT $1', values: ['foo']} object. Note that it does not have quotes around substitution. And that's a mistake in your raw query — you have quotes around fileName and backgroundJobId substitutions. Correct code would be:
  const queryStatement = Prisma.sql`SELECT (gitTree.obj ->> 'path') as path, (gitTree.obj ->> 'content_url') as content_url FROM "BackgroundJobResult" bgr LEFT JOIN LATERAL jsonb_array_elements(bgr."data") WITH ORDINALITY AS gitTree(obj, pos) ON true WHERE (gitTree.obj ->> 'path') = ${filePath} AND "backgroundJobId" = ${backgroundJobId}::UUID;`

I've also added typcast around backgroundJobId — this is necessary since it is now a string variable and not a literal.

just do prisma.$queryRaw´select * from table where id = ${uuid}::UUID´


gn_ws pretrained resnext model download failure

Hello, when i tried to download pretrained gn_ws resnext models, it raised error like this:
Connecting to s3.ap-northeast-2.amazonaws.com (s3.ap-northeast-2.amazonaws.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-05-21 23:49:50 ERROR 404: Not Found.
but download pretrained gn models successfully meantime.
Is there anyone else meeting the same question? Or the file name provided is false?
'jhu/resnext50_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext50_32x4d_gn_ws-ad7a50a4.pth',  # noqa: E501
'jhu/resnext101_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext101_32x4d_gn_ws-6adc888e.pth',  # noqa: E501
Thank you for replying!

Could you update your mmcv and then have a try again?

you could try thess urls:
'jhu/resnext50_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext50_32x4d_gn_ws-0d87ac85.pth',  # noqa: E501
'jhu/resnext101_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext101_32x4d_gn_ws-34ac1a9e.pth',  # noqa: E501

Could you update your mmcv and then have a try again?

Thank u very much, solved! ^-^

you could try these urls:
'jhu/resnext50_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext50_32x4d_gn_ws-0d87ac85.pth',  # noqa: E501
'jhu/resnext101_32x4d_gn_ws': 'https://s3.ap-northeast-2.amazonaws.com/open-mmlab/pretrain/third_party/resnext101_32x4d_gn_ws-34ac1a9e.pth',  # noqa: E501

Thank u very much, solved! ^-^

What's more, did you ever add GN to ResNext50-FPN/ResNext101-FPN?
I found the mAP decreased a lot?

@HarveyGuo960817  You can find related results here.

@HarveyGuo960817  You can find related results here.
Thank you,  but we cannot find the comparison between BN and GN, especially in ResNext. It seems that GN will decrease the mAP when it comes to more complicated network?
(As the paper of Weight Standardization reported)

Models in the model zoo are with BN.

Ok, thank you!

why comparison between BN and GN in ResNext, GN get a great drop in map?  Can someone analyze the reason?

Same question.I improved on RESnet101 by about 1% with GN_WS, but decreased by 0.4% on resnext101_32x4D.Please explain?Look at the results of the experiment. It should be improved a little bit.

Same question.I improved on RESnet101 by about 1% with GN_WS, but decreased by 0.4% on resnext101_32x4D.Please explain?Look at the results of the experiment. It should be improved a little bit.


Perilous pretraining, poorly named outputs causing key errors? 'pretrain' issue.

On commit 33775e7c96adbe2924cadb13f0061fd648a74c46
Using the Regressor network I tried to pretrain using first 'layerwise' then 'pretrain'. There were failures in both.
First off for 'layerwise' it would get to the last hidden layer and then not resolve the output. Perhaps this is to do with the caching feature? Or perhaps layerwise makes assumptions about the output layer's name?
  File "/opt/hindle1/src/theanets/theanets/losses.py", line 102, in diff
    output = outputs[self.output_name]
KeyError: 'lwout:out'

Perhaps fe17ba38c4daf5fa0c3986205c99abd10418b1b2 fixes this.
I 2015-08-12 23:49:11 downhill.base:226 RMSProp 21 loss=0.145915 err=0.145915
I 2015-08-12 23:52:26 downhill.base:226 RMSProp 22 loss=0.146107 err=0.146107
I 2015-08-12 23:55:31 downhill.base:226 RMSProp 23 loss=0.147220 err=0.147220
I 2015-08-12 23:55:32 theanets.graph:574 current_pre_brain.pkl: saved model
I 2015-08-12 23:58:25 downhill.base:226 RMSProp 24 loss=0.146543 err=0.146543
I 2015-08-13 00:01:04 downhill.base:226 RMSProp 25 loss=0.146085 err=0.146085
I 2015-08-13 00:04:51 downhill.base:226 RMSProp 26 loss=0.145989 err=0.145989
I 2015-08-13 00:07:48 downhill.base:226 RMSProp 27 loss=0.146689 err=0.146689
I 2015-08-13 00:07:48 theanets.graph:574 current_pre_brain.pkl: saved model
I 2015-08-13 00:10:45 downhill.base:226 RMSProp 28 loss=0.146003 err=0.146003
I 2015-08-13 00:14:29 downhill.base:226 RMSProp 29 loss=0.146112 err=0.146112
I 2015-08-13 00:18:12 downhill.base:226 RMSProp 30 loss=0.146124 err=0.146124
I 2015-08-13 00:19:09 downhill.base:226 validation 3 loss=0.146024 err=0.146024
I 2015-08-13 00:19:09 downhill.base:402 patience elapsed!
I 2015-08-13 00:19:09 theanets.trainer:241 layerwise: training in -> hid1 -> hid2 -> hid3 -> out
Traceback (most recent call last):
  File "stft-theanet.py", line 58, in <module>
    momentum=0.9)
  File "/opt/hindle1/src/theanets/theanets/graph.py", line 354, in train
    for monitors in self.itertrain(*args, **kwargs):
  File "/opt/hindle1/src/theanets/theanets/graph.py", line 330, in itertrain
    for i, monitors in enumerate(algo.itertrain(train, valid, **kwargs)):
  File "/opt/hindle1/src/theanets/theanets/trainer.py", line 243, in itertrain
    for monitors in trainer.itertrain(train, valid, **kwargs):
  File "/opt/hindle1/src/theanets/theanets/trainer.py", line 60, in itertrain
    loss=self.network.regularized_loss(**kwargs),
  File "/opt/hindle1/src/theanets/theanets/graph.py", line 634, in regularized_loss
    return self.loss(outputs) + sum(
  File "/opt/hindle1/src/theanets/theanets/losses.py", line 163, in __call__
    err = self.diff(outputs)
  File "/opt/hindle1/src/theanets/theanets/losses.py", line 102, in diff
    output = outputs[self.output_name]
KeyError: 'lwout:out'

Then I tried to use the pretrainer and I received the following error "TypeError: Tried to provide value for implicit input: hid1.w" I do not believe the latest fix actually addresses this.
I 2015-08-13 09:12:48 downhill.dataset:144 valid: 229 of 229 mini-batches of (100, 4096); (100, 2050)
I 2015-08-13 09:12:48 downhill.dataset:144 train: 100 of 229 mini-batches of (100, 4096); (100, 2050)
I 2015-08-13 09:12:48 theanets.layers.base:296 layer Tied "tied-hid3": (out)2050 -> 2050, relu, 2050 parameters
I 2015-08-13 09:12:48 theanets.layers.base:296 layer Tied "tied-hid2": (out)2050 -> 2050, relu, 2050 parameters
I 2015-08-13 09:12:48 theanets.layers.base:296 layer Tied "tied-hid1": (out)2050 -> 4096, linear, 4096 parameters
I 2015-08-13 09:12:48 theanets.trainer:304 creating shadow network
I 2015-08-13 09:12:48 theanets.graph:119 network has 16816146 total parameters
I 2015-08-13 09:12:48 theanets.trainer:241 layerwise: training in -> hid1 -> tied-hid1
I 2015-08-13 09:12:48 downhill.base:378 -- patience = 1
I 2015-08-13 09:12:48 downhill.base:379 -- validate_every = 10
I 2015-08-13 09:12:48 downhill.base:380 -- min_improvement = 0.1
I 2015-08-13 09:12:48 downhill.base:381 -- max_gradient_norm = 0
I 2015-08-13 09:12:48 downhill.base:382 -- max_gradient_elem = 0
I 2015-08-13 09:12:48 downhill.base:383 -- learning_rate = 0.001
I 2015-08-13 09:12:48 downhill.base:384 -- momentum = 0.9
I 2015-08-13 09:12:48 downhill.base:385 -- nesterov = False
I 2015-08-13 09:12:48 downhill.adaptive:220 -- rms_halflife = 14
I 2015-08-13 09:12:48 downhill.adaptive:221 -- rms_regularizer = 1e-08
I 2015-08-13 09:12:48 downhill.base:112 compiling evaluation function
I 2015-08-13 09:12:49 downhill.base:118 compiling RMSProp function

Traceback (most recent call last):
  File "stft-theanet.py", line 58, in <module>
    momentum=0.9)
  File "/opt/hindle1/src/theanets/theanets/graph.py", line 354, in train
    for monitors in self.itertrain(*args, **kwargs):
  File "/opt/hindle1/src/theanets/theanets/graph.py", line 330, in itertrain
    for i, monitors in enumerate(algo.itertrain(train, valid, **kwargs)):
  File "/opt/hindle1/src/theanets/theanets/trainer.py", line 310, in itertrain
    for monitors in pre.itertrain(train, valid, **kwargs):
  File "/opt/hindle1/src/theanets/theanets/trainer.py", line 243, in itertrain
    for monitors in trainer.itertrain(train, valid, **kwargs):
  File "/opt/hindle1/src/theanets/theanets/trainer.py", line 66, in itertrain
    ).iterate(train, valid=valid, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/downhill/base.py", line 397, in iterate
    validation = self.evaluate(valid)
  File "/usr/local/lib/python2.7/dist-packages/downhill/base.py", line 243, in evaluate
    values = [self.f_eval(*x) for x in dataset]
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 590, in __call__
    self.inv_finder[c]))
TypeError: Tried to provide value for implicit input: hid1.w

Perhaps in the recent commits layerwise was fixed, but I'm not sure about pretrain. It'll take some time before I can confirm again.

The commit  fe17ba3 does address the issue with the name of the output layer.
I haven't run across the second issue you've described. If you can update your code to the current git master and try again, please update the report here with what happens!

Ok I updated to the latest greatest theanet and tried to use pretrain again.
Here's 1.2gb of test data https://archive.org/details/FFT2-to-STFT-with-data-using-theanets
Here's what doesn't get pretrained ( http://softwareprocess.es/2015/example.py )  :
import theanets
import pickle
import numpy as np
import climate
import logging
import os

climate.enable_default_logging()

# input 64*64 grayscale bitmap
# output samples 22050/30
# fft windows of 1024
# cut down to real values
# cut down again
inputs = 4096
win_size = 2048
swin_size = win_size / 2 + 1
output_size = swin_size * 2
hidlayersize = output_size #win_size
exp = theanets.Experiment(theanets.Regressor,layers=[
    4096
    ,dict(size=hidlayersize,std=0.001,mean=0.0)
    ,dict(size=hidlayersize,std=0.001,mean=0.0)
    ,dict(size=hidlayersize,std=0.001,mean=0.0)
    ,output_size])
net = exp.network

logging.info("Read frames.pkl")
frames = pickle.load(file('fft-frames.pkl'))
logging.info("Read stft.pkl")
audio  = pickle.load(file('stft.pkl'))
train = frames
outputs = audio
train = train.astype(np.float32)
outputs = outputs.astype(np.float32)[0:train.shape[0]]
shuffleids = np.arange(train.shape[0])
np.random.shuffle(shuffleids)
train = train[shuffleids]
outputs = outputs[shuffleids]
i = 0

logging.info("Pretraining")
net.train([train, outputs],
          save_progress="current_pre_brain.pkl",
          save_every=25,
          batch_size=4096,
          train_batches=1024,
          patience = 1,
          min_improvement = 0.1,
          algo='pretrain',
          momentum=0.9)

i = 0
for traint, validt in net.itertrain([train, outputs],
          algo='nag',
          learning_rate=1e-3,
          save_progress="current_brain.pkl",
          save_every=25,
          batch_size=4096,
          momentum=0.9):
    print('i ',str(i))
    print('training loss:', traint['loss'])
    print('most recent validation loss:', validt['loss'])
    print('training err:', traint['err'])
    print('most recent validation err:', validt['err'])
    i += 1

net.save('stft-theanet.py.net.pkl')

Here's the output (similar command, same output)
hindle1@piggy:/media/hindle1/MyMedia/deep-learning/osborne-combined-stft-both-fft2$ python stft-theanet.py
Using gpu device 0: GeForce GTX 970
I 2015-09-03 22:36:51 theanets.layers.base:462 layer Input "in": 4096 inputs
I 2015-09-03 22:36:51 theanets.layers.base:303 layer Feedforward "hid1": (in:out)4096 -> 2050, relu, 8398850 parameters
I 2015-09-03 22:36:51 theanets.layers.base:303 layer Feedforward "hid2": (hid1:out)2050 -> 2050, relu, 4204550 parameters
I 2015-09-03 22:36:51 theanets.layers.base:303 layer Feedforward "hid3": (hid2:out)2050 -> 2050, relu, 4204550 parameters
I 2015-09-03 22:36:51 theanets.layers.base:303 layer Feedforward "out": (hid3:out)2050 -> 2050, linear, 4204550 parameters
I 2015-09-03 22:36:51 theanets.graph:116 network has 21012500 total parameters
I 2015-09-03 22:36:51 root:38 Read frames.pkl
I 2015-09-03 22:37:03 root:40 Read stft.pkl
I 2015-09-03 22:37:11 root:51 Pretraining
I 2015-09-03 22:37:11 downhill.dataset:144 valid: 6 of 6 mini-batches of (4096, 4096); (4096, 2050)
I 2015-09-03 22:37:11 downhill.dataset:144 train: 1024 of 6 mini-batches of (4096, 4096); (4096, 2050)
I 2015-09-03 22:37:11 theanets.layers.base:303 layer Tied "tied-hid3": (out)2050 -> 2050, relu, 2050 parameters
I 2015-09-03 22:37:11 theanets.layers.base:303 layer Tied "tied-hid2": (out)2050 -> 2050, relu, 2050 parameters
I 2015-09-03 22:37:11 theanets.layers.base:303 layer Tied "tied-hid1": (out)2050 -> 4096, linear, 4096 parameters
I 2015-09-03 22:37:11 theanets.trainer:314 creating shadow network
I 2015-09-03 22:37:11 theanets.graph:116 network has 16816146 total parameters
I 2015-09-03 22:37:11 theanets.trainer:250 layerwise: training in -> hid1 -> tied-hid1
I 2015-09-03 22:37:11 downhill.base:378 -- patience = 1
I 2015-09-03 22:37:11 downhill.base:379 -- validate_every = 10
I 2015-09-03 22:37:11 downhill.base:380 -- min_improvement = 0.1
I 2015-09-03 22:37:11 downhill.base:381 -- max_gradient_norm = 0
I 2015-09-03 22:37:11 downhill.base:382 -- max_gradient_elem = 0
I 2015-09-03 22:37:11 downhill.base:383 -- learning_rate = 0.0001
I 2015-09-03 22:37:11 downhill.base:384 -- momentum = 0.9
I 2015-09-03 22:37:11 downhill.base:385 -- nesterov = False
I 2015-09-03 22:37:11 downhill.adaptive:220 -- rms_halflife = 14
I 2015-09-03 22:37:11 downhill.adaptive:221 -- rms_regularizer = 1e-08
I 2015-09-03 22:37:11 downhill.base:112 compiling evaluation function
I 2015-09-03 22:37:14 downhill.base:118 compiling RMSProp function
Traceback (most recent call last):
  File "stft-theanet.py", line 89, in <module>
    momentum=0.9)
  File "build/bdist.linux-x86_64/egg/theanets/graph.py", line 400, in train
  File "build/bdist.linux-x86_64/egg/theanets/graph.py", line 376, in itertrain
  File "build/bdist.linux-x86_64/egg/theanets/trainer.py", line 320, in itertrain
  File "build/bdist.linux-x86_64/egg/theanets/trainer.py", line 253, in itertrain
  File "build/bdist.linux-x86_64/egg/theanets/trainer.py", line 66, in itertrain
  File "/usr/local/lib/python2.7/dist-packages/downhill/base.py", line 397, in iterate
    validation = self.evaluate(valid)
  File "/usr/local/lib/python2.7/dist-packages/downhill/base.py", line 243, in evaluate
    values = [self.f_eval(*x) for x in dataset]
  File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 590, in __call__
    self.inv_finder[c]))
TypeError: Tried to provide value for implicit input: hid1.w

So similar behaviour on GPU and CPU.

Hm, one thing I see here is that the 'pretrain' trainer requires an unlabeled dataset as input! You could try changing either 'pretrain' to 'layerwise', or change [train, outputs] to [train] (in your call to net.train).

Alright, you're correct. If I experience the same issue with an autoencoder I'll reopen the issue. Thanks for your help.


crypto/tls: TLS 1.0 is not min version by default in HTTP server

What version of Go are you using (go version)?

$ go version
go version go1.12.9 darwin/amd64

Does this issue reproduce with the latest release?
Yes. 1.12.9 is the latest release now.
What did you do? -  Test code
Here is very simple HTTPS server code. You can choose &tls.Config{} or &tls.Config{MinVersion: tls.VersionTLS10} by the comment.
// main.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net/http"
)

func main() {
	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		_, _ = fmt.Fprintln(w, "
hello, world
")
	})

	// Start HTTP server
	fmt.Println("Listening...")

	// TLS config
	// NOTE: Switch by comment
	tlsConfig := &tls.Config{}
	//tlsConfig := &tls.Config{MinVersion: tls.VersionTLS10}

	server := &http.Server{Addr: ":8443", Handler: handler, TLSConfig: tlsConfig}
	// Start HTTPS server
	if err := server.ListenAndServeTLS("./ssl_certs/server.crt", "./ssl_certs/server.key"); err != nil {
		log.Fatal(err.Error())
	}
}

Run
go run main.go

What did you expect to see? & What did you see instead?
Testing
I use the following command to confirm whether SSLv3 is supported or not.
openssl s_client -connect localhost:8443 -ssl3

When using &tls.Config{}, SSLv3 is NOT rejected.
When using &tls.Config{MinVersion: tls.VersionTLS10}, SSLv3 is rejected and the server emits "http: TLS handshake error from <IP_ADDRESS>:65528: tls: client offered only unsupported versions: [300]", which is my expectation.
I expect both &tls.Config{} and  &tls.Config{MinVersion: tls.VersionTLS10} cases reject SSLv3, but that was not true. The official document says "If zero, then TLS 1.0 is taken as the minimum" like the following.
Document
// MinVersion contains the minimum SSL/TLS version that is acceptable.
// If zero, then TLS 1.0 is taken as the minimum.
MinVersion uint16

from: tls - The Go Programming Language
In the document, MinVersion should be TLS 1.0, so I think it rejects SSLv3. However, the server allows SSLv3.

Is this vulnerable? In default, HTTP server in Go 1.12 can be attached by downgrade attack?

@FiloSottile Thank you very much for your explanation and fix!

@FiloSottile  I'm looking forward to Go 1.13 release!


--join flag update

Updating cockroach start docs to clarify that multiple join addresses can be set as a comma-separated list following a single --join flag, using multiple --join flags, or a combination of these approaches.
Fixes #481

This change is

LGTM


Refactor kube-state-metrics Helm charts into component

How to categorize this PR?

/area dev-productivity quality robustness
/kind technical-debt test enhancement
/topology seed
/platform all
/exp intermediate
/merge squash
What this PR does / why we need it:
This PR refactors the deployment for the kube-state-metrics components. It adopts the component concept. Consequently, it eliminates the Helm chart and adds unit tests, instead. This improves several aspects and the overall code quality.
Which issue(s) this PR fixes:
Part of #2754
Release note:

NONE

/assign

/cc @wyb1 @istvanballok

/test pull-gardener-e2e-kind

/approve


Replace the Github user names with the names on Canvas in README.md

Fixed with PR https://github.com/miniatureseal/dsci-310-group-10-GCC/pull/64


Is it possible to skip/ignore irrelevant fields in a model?

I would like to define a model that only partially parses/validates XML according to fields that I care about, ignoring the rest.
If the irrelevant fields come later on in the model, I can use extra="ignore"to accomplish this. However, I have not found a way to  skip some fields at the beginning of the model, or intermediate fields (i.e. read field1 and field3, ignoring the fact that field2 comes inbetween).
Minimal example:
from pydantic_xml import BaseXmlModel, element

class MyModel1(BaseXmlModel, tag="model", extra="ignore"):
    field1: str = element()  # this works, because field2 just gets ignored
    # field2: str = element()

class MyModel2(BaseXmlModel, tag="model", extra="ignore"):
    # field1: str = element()  # this breaks, because the model expects field2 as the first field to parse!
    field2: str = element()

class MyModel3(BaseXmlModel, tag="model", extra="ignore"):
    field1: str = element()
    # field2: str = element()  # this also breaks
    field3: str = element()

xml = """<?xml version="1.0" encoding="UTF-8"?>
<model>
    <field1>1</field1>
    <field2>2</field2>
    <field3>3</field3>
</model>"""

m = MyModel1.from_xml(xml)
print(m)

m = MyModel2.from_xml(xml)
# pydantic_core._pydantic_core.ValidationError: 1 validation error for MyModel2
# field2
#   [line -1]: Field required [type=missing, input_value={}, input_type=dict]
print(m)

m = MyModel3.from_xml(xml)
# pydantic_core._pydantic_core.ValidationError: 1 validation error for MyModel3
# field3
#   [line -1]: Field required [type=missing, input_value={'field1': '1'}, input_type=dict]
print(m)

´´´

Solved using https://pydantic-xml.readthedocs.io/en/latest/pages/data-binding/elements.html#elements-search-mode
e.g.
from pydantic_xml import BaseXmlModel, element
from pydantic_xml.element import SearchMode

mode = SearchMode.UNORDERED  # SearchMode.ORDERED also happens to work here

class MyModel1(BaseXmlModel, tag="model", extra="ignore", search_mode=mode):
    field1: str = element()
    # field2: str = element()

class MyModel2(BaseXmlModel, tag="model", extra="ignore", search_mode=mode):
    # field1: str = element()
    field2: str = element()

class MyModel3(BaseXmlModel, tag="model", extra="ignore", search_mode=mode):
    field1: str = element()
    # field2: str = element()
    field3: str = element()

xml = """<?xml version="1.0" encoding="UTF-8"?>
<model>
    <field1>1</field1>
    <field2>2</field2>
    <field3>3</field3>
</model>"""

m = MyModel1.from_xml(xml)
print(m)

m = MyModel2.from_xml(xml)
print(m)

m = MyModel3.from_xml(xml)
print(m)