Issue with UsageCategory

Hi All!
I tested UsageCategory on December Update.
When I build and publish full my app. Get the crash error:

If I restart the server and open NAV again, I don't see any new page and report on Menu Suite.

After that, I test as bellow:

If I only add this property for 1 -> 19 Objects (10 Pages, 1..9 Reports) then They are added to MenuSuite as picture:

If I add this property for 20 or 20+ then I get above crash error.

Note:
I also tested on NAV 2018. I got the same result.
If you need the full package for debugging, Please provide me a email. I will send you full package.

For the first picture, have you pressed F5 or Ctrl F5. Is it Nav 2018?
If it is Nav 2018, have and you have pressed F5, have you disabled the NetFx40_LegacySecurityPolicy on the *server.exe.config
The UsageCategory will just make it possible that your page or report appears in the search on the web client.

If the Service crashed, there might be some useful exception stuff in the event log, please check.

Hi kalberes,
My Purpose is that I want to add all my new pages and reports to Search on December Update.
I always use F5 (on Visual Studio Code) to build and publish my App.
But after publishing done, both win client and web client can't access, so I can't see result.
I check eventviewer then log is recorded as bellow:
//////////////////////
Type: Microsoft.Dynamics.Nav.Client.NavClientClosingException
Fatal: False
ShowError: True
Message: The server "NAV" was unable to process the request. The application will close.
StackTrace:
at Microsoft.Dynamics.Nav.Client.WinClient.ExceptionHandler.DoExecute(Func`1 execute)
at Microsoft.Dynamics.Nav.Client.WinClient.StartWinFormsClient.RunCore()
Source: Microsoft.Dynamics.Nav.Client.WinClient
HResult: -2146233088
Type: Microsoft.Dynamics.Nav.Types.Exceptions.NavServerSessionException
Message: The server "NAV" was unable to process the request. The application will close.
StackTrace:
at Microsoft.Dynamics.Nav.Client.ConnectionStateManager.CallServer[T](BeginCallServerMethod beginCallServerMethod, EndCallServerMethod1 endCallServerMethod, ServerCallOptions options)      at Microsoft.Dynamics.Nav.Client.ServiceConnection.CallServer[T](BeginCallServerMethod beginCallServerMethod, EndCallServerMethod1 endCallServerMethod)
at Microsoft.Dynamics.Nav.Client.MetadataHandler.GetNavigationFrame(Boolean applyPersonalization, Boolean getNavigationFrameFromServer)
at Microsoft.Dynamics.Nav.Client.FormBuilder.BuilderSession.RefreshNavigationForm()
at Microsoft.Dynamics.Nav.Client.FormBuilder.BuilderSession.get_NavigationForm()
at Microsoft.Dynamics.Nav.Client.FormBuilder.BuilderSession.CreateLogicalForm(String name)
at Microsoft.Dynamics.Framework.UI.UISession.CreateLogicalForm(String name, Boolean reuseActiveForm)
at Microsoft.Dynamics.Framework.UI.Windows.ClientSessionBase.ShowLogicalForm(String formName)
at Microsoft.Dynamics.Framework.UI.Windows.ClientSessionBase.RunCore()
at Microsoft.Dynamics.Nav.Client.WinClient.NavWinFormsClientSession.RunCore()
at Microsoft.Dynamics.Nav.Client.WinClient.NavClientErrorHandler.ExecuteAndCatchExceptions(Func`1 execute)
Source: Microsoft.Dynamics.Nav.Client.ServiceConnection
HResult: -2146233088
Type: Microsoft.Dynamics.Nav.Types.Exceptions.NavServerSessionException
Message: The server "NAV" was unable to process the request. The application will close.
HResult: -2146233088
Type: System.InvalidOperationException
Message: There is an error in XML document (27, 24).
StackTrace:
at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
at Microsoft.Dynamics.Nav.XmlMetadata.NavigationDataAggregator.CreateNavigationData(String content)
at Microsoft.Dynamics.Nav.XmlMetadata.NavigationDataAggregator.CreateNavigationData(PackageId id)
at Microsoft.Dynamics.Nav.XmlMetadata.NavigationDataAggregator.UpdateNavigationDataCache(NavAppGroup appGroup)
at Microsoft.Dynamics.Nav.XmlMetadata.NavigationDataAggregator.GetNavigationData(NavAppGroup appGroup)
at Microsoft.Dynamics.Nav.XmlMetadata.NavigationDataAggregator.AppendExtensionNavigationData(NavigationDefinition existingDefinitions, NavAppGroup appGroup)
at Microsoft.Dynamics.Nav.XmlMetadata.MetadataProvider.GetNavigationDepartments(NavAppGroup appGroup)
at Microsoft.Dynamics.Nav.XmlMetadata.MetadataProvider.GetNavigationFrame(Int32 homepageId, Boolean applyPersonalization)
at Microsoft.Dynamics.Nav.XmlMetadata.MetadataProvider.GetNavigationFrame(Boolean applyPersonalization)
at SyncInvokeGetNavigationFrame(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at Microsoft.Dynamics.Nav.Service.ServiceOperationInvoker.RunInTransactionCombinator(ServiceOperation innerOperation, NSServiceBase serviceInstance, MethodBase syncMethod, Object[] inputs, Object[]& outputs)
at Microsoft.Dynamics.Nav.Service.ServiceOperationInvoker.<>c__DisplayClass25_0.b__1(NSServiceBase serviceInstance, Object[] inputs, Object[]& outputs)
at Microsoft.Dynamics.Nav.Service.ServiceOperationInvoker.TransientErrorRetryCombinator(ServiceOperation innerOperation, NSServiceBase serviceInstance, MethodBase syncMethod, Object[] inputs, Object[]& outputs)
at Microsoft.Dynamics.Nav.Service.ServiceOperationInvoker.<>c__DisplayClass25_0.b__1(NSServiceBase serviceInstance, Object[] inputs, Object[]& outputs)
at Microsoft.Dynamics.Nav.Service.ServiceOperationInvoker.ErrorMappingCombinator(ServiceOperation innerOperation, NSServiceBase serviceInstance, MethodBase syncMethod, Object[] inputs, Object[]& outputs)
Source: System.Xml
HResult: -2146233079
Type: System.Xml.XmlException
LineNumber: 27
LinePosition: 24
Message: '.', hexadecimal value 0x00, is an invalid character. Line 27, position 24.
StackTrace:
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Char[] data, Int32 length, Int32 invCharPos)
at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlReader.ReadEndElement()
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderNavigationDefinition.Read97_NavigationDefinition(Boolean isNullable, Boolean checkType)
at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderNavigationDefinition.Read98_NavigationDefinition()
Source: System.Xml
HResult: -2146232000
///////////////////
I must restart NAV Service again so that i can open NAV again. However, Don't have any new objects are added.

As you suggest, I tested on NAV 2018. I disabled the NetFx40_LegacySecurityPolicy on the *server.exe.config and test again. But the result is same, event logs is recorded the same.

Let me try to understand your issue. You have  added Usage category to 19 pages/reports in your extension package. If you try adding the 20th usage category and publish (FYI: Ctrl F5 is what publishes, F5 starts a debugging session) you get a crash on the web client?

Yes kalberes,
If I add Usage category to 19 pages/reports in my extension, and press F5 to build and publish App, It work well.
But, If I try adding the 20th object, and press F5 to rebuild and publish app again. I get above error.

Ok,  we will investigate it.
Thanks
Kalman

We have encountered this as well. The number of pages we wish to make searchable is at 20 at this time. This caused the same type of crash with the same event log. Please let me know if more detail is needed.

I have tried reproducing this by creating 22 simple pages assigning various usage categories, then the same usage category I have no repro either way. So any data @strophanthus , @hgnamvn  that would repro it would be helpful.

I took a look and I would guess it is because we still have CaptionML set on all UI.

Hi kalberes,
Please give me your email. I will send you my Package, You can test and debug on it.

Hi kalberes,
I send you the package, please check for me. Thank you so much.

Hi.
I have logged a bug and have attached your repro. I did not have time to look at it. Someone in the team will pick it up and have a look at it.

Thank Kalberes so much.


Hide children for closed meta tickets

When adding new encoding tickets to a exiting project, with existing meta tickets – from which are some already closed ("Frühstück", "Mittagessen", etc.) – these encoding tickets (e.g. subformats) should not be added to the closed ones.

Encoding tickets are always for all meta tickets in ordinary projects. We could discuss about putting tickets in a special state. But as other encoding tickets for closed meta tickets stay in material needed, I don't see any reason to do something special here.
Why are these tickets a problem?

I this weekend's conference we initially had only master mp4 encoding tickets, so I manually deleted the subtickets and got a quite nice representation similar to this:

When I later added the other encoding profile this view was reset, but I actually liked the old view – so I created this ticket. Feel free to close it, if you have another opinion.

so I manually deleted the subtickets

You shouldn't do that and you are only allowed to do that, because you are admin. Be careful! 🚧

but I actually liked the old view – so I created this ticket.

Ok, completely changed requirements. We could maybe update the default view.

Tried with CSS but selectors are not powerful enough.. the ticket li elements should have something like data-parent-state, then it's actually easy.

Tried with CSS but selectors are not powerful enough

I don't think filtering with CSS is a good idea, as these views are already heavy on database queries. Reducing the number of tickets loaded from the database can quite speed up these views.

Closed in 8c1c814.


新增功能

完善项目

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

123


[Security Issue][BUG]Missing rate limiting

**Describe the Security issue **
HTTP request handlers should not perform expensive operations such as accessing the file system, executing an operating system command or interacting with a database without limiting the rate at which requests are accepted. Otherwise, the application becomes vulnerable to denial-of-service attacks where an attacker can cause the application to crash or become unresponsive by issuing a large number of requests at the same time.
Location of the issue
Path :trainingportal/server.js
File  :server.js
Line :204
Part of the code issue
res.send('Unable to login');
});

app.get("/public/badge/:code",async(req,res) => {

Recommendation
A rate-limiting middleware should be used to prevent such attacks.
example to this issue
var express = require('express');
var app = express();

app.get('/:path', function(req, res) {
  let path = req.params.path;
  if (isValidPath(path))
    res.sendFile(path);
});

example to solve this issue
var express = require('express');
var app = express();

// set up rate limiter: maximum of five requests per minute
var RateLimit = require('express-rate-limit');
var limiter = new RateLimit({
  windowMs: 1*60*1000, // 1 minute
  max: 5
});

// apply rate limiter to all requests
app.use(limiter);

app.get('/:path', function(req, res) {
  let path = req.params.path;
  if (isValidPath(path))
    res.sendFile(path);
});

References
OWASP: Denial of Service Cheat Sheet.
Wikipedia: Denial-of-service attack.
NPM: express-rate-limit.
Common Weakness Enumeration: CWE-770.
Common Weakness Enumeration: CWE-307.
Common Weakness Enumeration: CWE-400.

The solution is provided by @omarhany0  BUT for testing purposes, we won't fix this so we can try to test our app dynamically as we can.


Moved the lyrics to the lyrics.js file

fix #32

lyrics.js file contains all the lyrics of the songs as an array of objects.
all the script files for the song pages were updated to dynamically generate the p tags for songs lyrics.

Thanks, will go through this later in the day/

Had a look at the code.... I don't want the index.js in any lyric folder. I just want to call out all the lyrics in the lyrics.js file and then call out the lyrics.js file to all the lyrics directory, without index.js being there. Idk if this is possible, coz im not good at JS but that's what I wanna do.

Hey @aliraza944. I was mistaken. I see you've called out the lyrics and in index.html called out the lyrics.js file. Although, I was trying to

Format better
Only have one js file instead of JS files in all folders

So, with this PR, it does do the job I need done by 1. So, I'll merge this and maybe create a script tag in the html too later instead of the index.js in every directory.
Thanks for your help

You are welcome. I am glad that it helped.


Added to PlayerStoreHouse, Created GUITools

Added JLabel images to PlayerStoreHouse
Moved my helper Graphics helper methods to a new class GUITools so they could be accessed by all classes. These methods simplify creating default java graphics and currently include JLabels, JButtons, and JRadioButtons

This code will not compile because the file with the class GUITools is called guiTools, but this can be easily fixed.


Next steps button "Terraform on Azure" points to Ansible

The hyperlink point to https://docs.microsoft.com/en-us/azure/ansible/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 83bb7059-c6aa-522c-474a-f516e453fe4f
Version Independent ID: 3efbdbd2-2076-5302-cc3f-0ce765bf9e71
Content: Tutorial - Create a Linux VM with a managed identity from the Azure Marketplace image using Terraform
Content Source: articles/terraform/terraform-vm-managed-identities-for-azure-resources.md
Service: terraform
GitHub Login: @TomArcherMsft
Microsoft Alias: tarcher

@theguate Good catch. I'll fix this afternoon.

@TomArcherMsft Thanks for the quick response. 👍

Issues addressed: https://github.com/MicrosoftDocs/azure-docs-pr/pull/95271
#please-close


Inplace operators behaviour for dask arrays inconsistent with numpy

Inplace operators modify numpy arrays inplace, but end up making a copy of dask arrays.
# Using dask v2.2.0, numpy 1.17.0
import numpy as np
import dask.array as da
x_np = np.zeros(5)
y_np = x_np
y_np += 1

x_da = da.zeros(5)
y_da = x_da
y_da += 1

print(x_np == y_np)
# [ True  True  True  True  True]
print(id(x_np) == id(y_np))
# True
print(x_da.compute() == y_da.compute())
# [False False False False False]
print(id(x_da) == id(y_da))
# False

I'm not sure if you'd qualify this as a bug, since the behaviour between the two is known to not line up completely. I have also seen related issues have been brought up, like #2588 and #2000.
I think a warning for this behavior could be useful. Alternatively, I think this could just be an inplace operation. Here's are some proof of concept implementations. This should probably be done with ufuncs if implemented.
import dask.array as da
import numpy as np

# Something like this
def dask_iadd(x, y):
    np.add(x, y, out=x)
    return x
da.core.Array.__iadd__ = dask_iadd

a = da.ones((10, 10))
b = a
b += 1
print(id(a) == id(b))
# True
print(np.all(a.compute() == b.compute()))
# True

 Previous crude implementation
import dask.array as da

def _replace(x, y):
    x._meta = y._meta
    x.dask = y.dask
    x.name = y.name
    x._chunks = y.chunks
    return x

da.core.Array.__iadd__ = lambda x, y: _replace(x, x + y)
# etc

z = da.zeros((2, 2))
z += 1
z.compute()
#  array([[1., 1.],
#         [1., 1.]])

Truly in-place operations would not be appropriate, since Dask tasks are functional, and tokenized, you do not want inputs changing. Consider, how would you make this work when you have workers in different processes/machines, you might modify the in-memory version, but that would have no effect on copies of the data held elsewhere, or if the worker went down.

Does the current implementation of __setitem__ avoid these issues?

Yes, exactly
In [14]: dict(x.dask)
Out[14]:
{('ones-c4a83f4b990021618d55e0fa61a351d6',
  0): (functools.partial(<function ones at 0x10871a2f0>, dtype=dtype('float64')), (5,)),
 ('ones-c4a83f4b990021618d55e0fa61a351d6',
  1): (functools.partial(<function ones at 0x10871a2f0>, dtype=dtype('float64')), (5,))}

In [15]: x[x] =5

In [16]: dict(x.dask)
Out[16]:
{('where-fc316acce29c6d2b5cef2f2593835bd6', 0): (subgraph_callable,
  ('ones-c4a83f4b990021618d55e0fa61a351d6', 0),
  5,
  ('ones-c4a83f4b990021618d55e0fa61a351d6', 0)),
 ('where-fc316acce29c6d2b5cef2f2593835bd6', 1): (subgraph_callable,
  ('ones-c4a83f4b990021618d55e0fa61a351d6', 1),
  5,
  ('ones-c4a83f4b990021618d55e0fa61a351d6', 1)),
 ('ones-c4a83f4b990021618d55e0fa61a351d6',
  0): (functools.partial(<function ones at 0x10871a2f0>, dtype=dtype('float64')), (5,)),
 ('ones-c4a83f4b990021618d55e0fa61a351d6',
  1): (functools.partial(<function ones at 0x10871a2f0>, dtype=dtype('float64')), (5,))}

Note how the "ones" remain in the graph.
Of, with concrete values, see how below the dask array's values change, but the original array is not
In [19]: inarr = np.ones(10)
In [20]: x = da.from_array(inarr, chunks=5)
In [22]: x[x] = 5
In [23]: x.compute()
Out[23]: array([5., 5., 5., 5., 5., 5., 5., 5., 5., 5.])

In [24]: inarr
Out[24]: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

OK to close?

I'm not sure I understand your point. Aren't in-place operators equivalent to something like this?
a = da.ones(5)
b = a
b[da.ones(5, dtype=bool)] = b + 1

I think I may not have been clear in my meaning of "in-place operators". I don't think that the actual computation has to happen in place, that is totally up to the execution engine. I meant the semantics of augmented assignment operators as used by numpy and, more broadly, mutable objects in python aren't being followed by dask arrays, though I think they could be.
In the issues I referenced, it sounded like this was a goal which dask was pursuing. I would be happy to make a PR implementing these.

I was looking at this and trying to figure out what the intention of those linked PRs was. My read is that they were more focused on allowing easy assignment patterns and less concerned with modifying dask collections in-place. That being said, I don't see a good reason why the behavior that you are talking about here and that you sketched out in your original post couldn't be supported.


Completions don't work with Weave.jl's .texw files

I tried to play around with the settings, but I didn't get very far.
Is there a way to enable completions in code chunks of Weave files?
See also https://github.com/mpastell/language-weave/issues/13

Can you give me a small snippet of how your code looks like? In theory you should be able to add another scope selector in the latex-completions settings, but which one (and if that would work at all) depends on the latex syntax highlighter you use.


breaks with compiler optimizations on

Nevermind, didn't read the README lmfao..
sorry...
Compiler Optimization /02 might cause wrong decryption, so use /01 or disable it


Random posts

Everytime I start the server it makes random posts

Yep, it's mock posts. It can be turned off in .env file SEED_DB variable.


Relax the column adapter api

lgtm

It's too bad Func1 isn't just a java api or something. It would mean things like Wire's ProtoAdapters would work out of the box

There's j.u.f.Function in Java 8. So in 3 years we'll be able to use it!


[Trivial] Standardised functions

A simple non-functionally-differing PR which standardises most top-level MPW functions in a format that IDEs can properly follow, and plus it's just easier to understand them by hand, too.
For example: VSCode, for me, would be unable to see many wallet.js functions while trying to apply them in other modules and files, this fixes those cases now, making development a tid bit faster, as I'm lazy.
Note:
The only 'real' functional change is that these functions can now be overridden, but this is actually a benefit for debugging, and I would hope users aren't dumb enough to paste random commands into a debug console... thinking about it, we could make a PR to display a huge warning message in the console, anyway. :shrug:

Could you add "use strict"; on top of our scripts?

Added into all scripts that make sense currently; before bitTrx.js, I'd like to refactor that whole file to get rid of the weird function-import style, but should be safe for another PR to reduce complexity and conflicts.

Corrected both issues, will need a re-ACK due to branch protection dismissal.


Add option to validate the schema

Validates the schema against whatever spec is referenced in $schema (see issue #353).
This PR is based on #362; that one needs merging first.

Is using $schema alone to determine the spec sufficient, or would a user-override via $checkMode to specifically set draft-03 / draft-04 etc. also be useful?

👍 for the idea. I wish I had had this back when I was getting started; would have saved loads of development time and confusion.
I would check $schema to start, then default to draft 4.

@shmax Done - it now defaults to draft-04, rather than the previous behavior of throwing a RuntimeException if $schema was undefined.

I haven't tried out the code yet, but what does the error output look like? Is it clear that any errors on the schema itself are related to the schema, and not the input data?

@shmax It currently brings across all schema validation errors using the standard format, and then adds another error to the stack stating that schema validation has failed. Do you think that this is sufficient, or would some kind of additional disambiguation on the errors that result from schema validation be worth having?

I'm not sure, I guess. I'll grab your code tonight and try it out and see how it feels...

@shmax Sounds good. I have no particular preference either way, but adding some disambiguation context seems like a good idea, so I've done this (prefixes the error messages with Schema validation: . The old behavior was to simply use the error message verbatim.

That might be good enough, but I'm wondering if there should be an even stronger separation. Does the validation proceed if schema errors are encountered, or does it stop dead?

It currently proceeds - I figured that as the current behavior seems to be to proceed despite errors, I should follow suit with the schema validation stuff as well (plus CHECK_MODE_EXCEPTIONS is a thing, so folks who want it to bail at the first hurdle can easily obtain that behavior).

What kind of stronger separation did you have in mind?

I don't know, I don't have the code in front of me and I can't even really remember offhand what normal error output looks like, but I'd like to see what approach AJV takes before I send you off on any wild goose chases.

Have just fixed a URL validation bug (non-compliance with RFC-3986 relative references) that showed up when I applied schema validation to the draft unit tests. Is that fix (b415918) OK as part of this PR, or should I split it out into a separate PR of its own?

If it's unrelated it's probably better to branch it off to a new PR, so we can give each issue our full attention without getting all mixed-up.

@shmax What do you consider unrelated? I'm not sure whether it should be considered related or not. It's not part of the feature, but some of the draft unit tests will fail when using schema validation until the bug is fixed.
Splitting them will mean rebasing this on the bugfix PR until such a time as it's merged. Quite happy to do this if you're wanting them separate.

I have no strong opinion, it just sounded like something that would branch its own discussion. Makes no real difference to me.

I'll split it then; if you think people will want to have a big discussion about it, then it makes sense to keep them separate :-).

OK, split done - this PR is now based on PR #358.

Sorry for the delay. I think I'm up to speed, now. validate returns an array of errors; I was hoping there would be an object with an errors property, but no such luck. I also read up on ajv, and analyzed out how it handles this kind of thing. The short version is that it doesn't mix schema validation errors with regular errors (which I kind of suspected). I've prepared a jsfiddle that you can experiment with that demonstrates the general idea:
https://jsfiddle.net/hahmvsfj/2/
I don't want to force any ideas down your throat or anything just yet, but I thought it might give you some ideas. Let me know what you think.

...would you be keen for me to overhaul the way errors work across the whole library? I don't particularly like the way they are done currently, and would love to tackle the issue, provided people are open to the idea - I don't want to spend ages working on something if nobody wants it.

Well, I for one am a big fan of your tinkerings and would love to see what you come up with. Have anything in particular in mind right off the bat?

Oh, and if you think this is going to be a really big project and you want to get some buy-in ahead of time, you might want to start with an issue and see if you can get some thumbs (up) on it.

I don't have any really solid ideas yet (although I have a few inklings), but I do know that I dislike the current model and want to change it. I might have a bit of a play, then open a PR with 'these are my ideas' code so others can comment, before I sink large amounts of time into it. Then once we have agreement on an approach, I can polish it up. I think the defaults thing / checkmode refactor benefited greatly from input other than mine, particularly yours, and ended up much better as a result.

@mathroc - do you have an opinion on whether or not this schema-validation PR should be a thing? Saw your comment on #353.

Thoughts:

Spec validation is valuable
Spec validation and constraint validation make sense as separate operations
Overhauling errors has happened before and probably didn't go far enough; however, it would be best to handle in a dedicated PR. See #239 for more on this topic.

With another review I will merge #358 so you can rebase this work.

#358 is merged. Please rebase.

@bighappyface Rebased.

Works for me :smille:

@erayd thx, I didn't noticed the associated PR. I don't mind having this or not in the lib: I'm already validating the schemas as part of my tests (the same way I told in the comment in the issue) so I don't want to validate them again when the code is running on production.
however I have a few questions about the implementation (code review comming ☺)

@mathroc Yeah, the PR won't validate them by default - you'll have to explicitly turn it on if you want that behavior.
Implementation is fluid at present - feedback is welcomed, and I'm quite happy to change stuff.

Rebased.

I was just wondering why you decided to use the same instance when validating the schema. Memory reasons? Also, it looks like you unset some of the checkmode options, but do you ever restore them?

Not just memory - also in case people have dont custom subclassing or whatever to modify behavior... this way, everything is consistent.
Yes, checkmode is restored - see the call to setCheckMode() lower down.

What are your thoughts on this PR now?

Looking good, but I would still feel better about it if you could find a way to physically separate the schema errors from the regular errors. You go so far as to make a distinction with your exception type, which is good, but people who prefer not to use exceptions--as far as I can tell--don't have any way to programmatically act on schema errors.

@shmax No problem - what about now? Errors now have a new 'context' field, which defaults to 'document'. If the error occurs during schema validation, then this field will be 'schema'. This will allow anyone post-processing errors to easily determine where each one comes from.
Factory::getContext() and Factory::setContext() can be used to get or change the current context.

Hmm. It's a start, and I can live with it, but it still feels a little wishy-washy to me, because if someone wants to know if there were any schema errors he has to walk over all the errors and peek into each one. That would work, but it feels a little passive, you know?
Just to brainstorm a few ideas (and some of these will probably be more appropriate for 6.0):

What about adding a separate getSchemaErrors method? A user that has deliberately turned on the schema validation feature would know to check this before checking getErrors.

I notice that validate doesn't return anything. What if it returned an error code? 0 for clean, 1 for schema errors, 2 for regular errors, 3 for both, etc. Then calling getErrors becomes more like an optional forensic task. A user could check the return value, then call one of getErrors or getSchemaErrors accordingly. That would achieve rough parity with something like PHP's json_decode  (where you can call json_last_error after the fact), so there's some precedent for that kind of thing.

Well, maybe that's enough for now. I'll see if I can think of any more in the morning.

@shmax Aah, I think I misunderstood you then - I thought you were wanting to address a case when a user was already walking the errors and wanted to filter in / out schema ones.

Would adding a context argument to getErrors be sufficient here (and it only returns errors in the given context, if a context is provided, otherwise return them all)? Or would you prefer that the schema errors actually be catalogued separately? I am happy with either - if you let me know your preference, I'll implement it.
I really like that idea. Having validate return an error bitmask with 0 as 'all OK' allows for a whole lot of useful scenarios. Will implement it.

Yeah, I think passing an argument to getErrors is better, as you say. It could even be another bitmask type of thing (eg. getErrors($type = Validator::ERRORS_ALL)). As for the context thing, hmm. I guess it wouldn't be necessary to expose that to the user once he has the ability to fish one or the other (or both) out of the validator with getErrors, so the question becomes how do you want to do it internally? No strong opinion is dawning on me, so I'll leave it up to you to mull while you're doing the other stuff.

@shmax How's that?
I want to go over this PR again at some point and look for messy code too. I've made lots of changes, and I need to make sure there's no remnants of previous approaches to stuff hanging around. Please let me know if you spot anything you think needs to go.

Looking sharp, but I'll have to get back to you tomorrow evening after work. Nice work.

Thanks :-). No rush.

LGTM very nice

@shmax Thanks :-). Time to revisit #362 now we've sat on it for a while? This PR (and one other) still depends on it.

Junk commits squashed, ready to merge after #362 (URI translation & local spec bundling).

Rebased.

Rebased.

Rebased.


ntdll.so Path Missing

First thanks for this image. I really appreciate it. When I use the 3.0.1 image everything compiles just fine, but any version newer than that gives me this error:
wine: cannot get path to ntdll.so
Any ideas as to why this happens? This does not happen on 3.0.1. Using the pyinstaller-windows image.

Hello, thanks for the feedback. As I understand it, since version 3.0.2 you get an error. If we compare the dockerfiles for pyinstaller-windows, the differences are minimal.
github.com/batonogov/docker-pyinstaller/compare/3.0.1...3.0.2
FROM ubuntu:20.04
FROM ubuntu:22.04

LABEL<EMAIL_ADDRESS>
ENV DEBIAN_FRONTEND noninteractive

ARG WINE_VERSION=winehq-stable
ARG PYTHON_VERSION=3.11.1
ARG PYINSTALLER_VERSION=5.7.0
ARG PYTHON_VERSION=3.11.2
ARG PYINSTALLER_VERSION=5.8.0

# we need wine for this all to work, so we'll use the PPA
RUN set -x \

I will look for a solution to the problem. Thanks.

Can't get an error. In which scenario are you getting the error?
➜  ~ docker run --platform linux/amd64 --rm -it batonogov/pyinstaller-windows:3.0.1 "wine --version && python --version"
wine --version && python --version
wine-7.0.1
0098:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
0098:err:winediag:nodrv_CreateWindow Make sure that your X server is running and that $DISPLAY is set correctly.
0098:err:systray:initialize_systray Could not create tray window
Python 3.11.1
➜  ~ docker run --platform linux/amd64 --rm -it batonogov/pyinstaller-windows:3.1.4 "wine --version && python --version"
wine --version && python --version
wine-8.0.1
00d4:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
00d4:err:winediag:nodrv_CreateWindow L"The explorer process failed to start."
00d4:err:systray:initialize_systray Could not create tray window
Python 3.11.4


feat(api): Limited support for programmatically running PAPIv≥2.14

Overview /  Changelog
These interfaces now support executing PAPIv≥2.14 protocols:

The opentrons_execute CLI
opentrons.execute.execute()
opentrons.execute.get_protocol_api()

This works towards, but does not close, RSS-268.
Known limitations:

Simulating PAPIv≥2.14 protocols will still raise a NotImplementedError.
Executing or simulating JSONv6+ protocols will still raise a NotImplementedError.
It will not print the run log. (Ticketed as RSS-283.)

Test Plan
Regression testing

[x] opentrons_execute with a PAPIv≤2.13 file

[x] Only standard labware
[x] Custom labware with -L
[x] Custom labware in the current working directory

[x] opentrons.execute.execute() with a PAPIv≤2.13 file

[x] Only standard labware
[x] Custom labware with custom_labware_paths=...
[x] Custom labware in the Jupyter Notebook directory

[x] opentrons.execute.get_protocol_api() with PAPIv≤2.13

[x] Only standard labware
[x] Custom labware with extra_labware=...
[x] Custom labware in the Jupyter Notebook directory

New feature testing

[x] opentrons_execute with a PAPIv≥2.14 file

[x] Only standard labware
[x] Custom labware with -L
[x] Custom labware in the current working directory

[x] opentrons.execute.execute() with a PAPIv≥2.14 file

[x] Only standard labware
[x] Custom labware with custom_labware_paths=...
[x] Custom labware in the Jupyter Notebook directory

[x] opentrons.execute.get_protocol_api() with PAPIv≥2.14

[x] Only standard labware
[x] Custom labware with extra_labware=...
[x] Custom labware in the Jupyter Notebook directory

[x] Do at least one of these on an OT-3

Review requests
I hate several things about this.

The _LIVE_PROTOCOL_ENGINE_CONTEXTS global.
_adapt_protocol_source().
create_protocol_engine_in_thread(). This is technically not new—it was added in #12567—but now it's being used in production code for the first time.
Letting opentrons.execute drift further apart from opentrons.simulate.

Risk assessment
Medium. This is messy, highly-duplicated, difficult to automatically test, and public-facing.
Much of this code is sensitive to differences between None and {} for bundled_labware, extra_labware, and bundled_data.
Next steps
We should try executing some Flex hardware testing scripts. From @ryanthecoder:

python3 -m hardware_testing.gravimetric --photometric --pipette 1000 --channels 96 --tip 200 --trials 5 this is how we call our scripts. the scripts themselves load a python protocol to fetch which type and location of labware but then just executes python directly talking to the protocol context

After that, we should continue working on RSS-268 until these interfaces are good enough to launch the Flex with.

Note to self: Try to add tests for PAPIv2.14 in test_execute.py. Something is better than nothing.

I'm seeing tests hang in a strange way at the closure of pytest on Windows, with Python 3.10, without opentrons-hardware. Investigating whether this is from the changes in this PR, or unrelated flakiness.

I have a verbal 👍 from @TamarZanzouri on the above change to fix the shutdown hanging.


Parse to custom class from unordered_json breaks on G++11.2.0 with C++20

What is the issue you have?
Parsing a custom class with a std::string member in it using nlohmann::ordered_json with C++20 starting with v3.10.4 results in a compilation error.
Using v3.10.3 compiles fine
Probably related issue: https://github.com/nlohmann/json/issues/3207
Please describe the steps to reproduce the issue.
Godbolt link with example showing that g++ 11.2.0 fails to compile when C++20 is enabled
https://godbolt.org/z/MsYe45qP9

Add custom class containing a std::string (custom class with just an int works)
Add custom from_json function for custom class
convert from ordered_json to class (normal unordered json works)
get compilation error

Example code, which I'd like to add to unittests, but don't know where. (for the error message I added it to the bottom of unit-deserialization.cpp)
struct MyClass {
    std::string name;
};

void from_json(const nlohmann::json &j, MyClass &obj)
{
    j.at("name").get_to(obj.name);
}

TEST_CASE("custom class parsing")
{
    SECTION("parse class directly")
    {
        nlohmann::ordered_json j = nlohmann::ordered_json::parse("{\"name\": \"class\"}");
        MyClass obj;
        j.get_to(obj);
        CHECK(obj.name == "class");
    }
}

Can you provide a small but working code example?
see above
What is the expected behavior?
Compile without error
And what is the actual behavior instead?
Compillation error:
In file included from /home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:32:
/home/nero/repos/external/nlohmann-json/single_include/nlohmann/json.hpp: In instantiation of 'void nlohmann::detail::from_json(const BasicJsonType&, ConstructibleStringType&) [with BasicJsonType = nlohmann::basic_json<nlohmann::ordered_map>; ConstructibleStringType = MyClass; typename std::enable_if<(nlohmann::detail::is_constructible_string_type<BasicJsonType, ConstructibleStringType>::value && (! std::is_same<typename BasicJsonType::string_t, ConstructibleStringType>::value)), int>::type <anonymous> = 0]':
/home/nero/repos/external/nlohmann-json/single_include/nlohmann/json.hpp:4273:25:   required from 'decltype (nlohmann::detail::from_json(j, forward<T>(val))) nlohmann::detail::from_json_fn::operator()(const BasicJsonType&, T&&) const [with BasicJsonType = nlohmann::basic_json<nlohmann::ordered_map>; T = MyClass&; decltype (nlohmann::detail::from_json(j, forward<T>(val))) = void]'
/home/nero/repos/external/nlohmann-json/single_include/nlohmann/json.hpp:4934:30:   required from 'static decltype ((nlohmann::{anonymous}::from_json(forward<BasicJsonType>(j), val), void())) nlohmann::adl_serializer<T, SFINAE>::from_json(BasicJsonType&&, TargetType&) [with BasicJsonType = const nlohmann::basic_json<nlohmann::ordered_map>&; TargetType = MyClass; ValueType = MyClass; <template-parameter-1-2> = void; decltype ((nlohmann::{anonymous}::from_json(forward<BasicJsonType>(j), val), void())) = void]'
/home/nero/repos/external/nlohmann-json/single_include/nlohmann/json.hpp:18955:45:   required from 'ValueType& nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>::get_to(ValueType&) const [with ValueType = MyClass; typename std::enable_if<((! nlohmann::detail::is_basic_json<BasicJsonType>::value) && nlohmann::detail::has_from_json<nlohmann::basic_json<ObjectType, ArrayType, StringType, BooleanType, NumberIntegerType, NumberUnsignedType, NumberFloatType, AllocatorType, JSONSerializer, BinaryType>, ValueType>::value), int>::type <anonymous> = 0; ObjectType = nlohmann::ordered_map; ArrayType = std::vector; StringType = std::__cxx11::basic_string<char>; BooleanType = bool; NumberIntegerType = long int; NumberUnsignedType = long unsigned int; NumberFloatType = double; AllocatorType = std::allocator; JSONSerializer = nlohmann::adl_serializer; BinaryType = std::vector<unsigned char>]'
/home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:1143:17:   required from here
/home/nero/repos/external/nlohmann-json/single_include/nlohmann/json.hpp:3914:7: error: no match for 'operator=' (operand types are 'MyClass' and 'const string_t' {aka 'const std::__cxx11::basic_string<char>'})
 3914 |     s = *j.template get_ptr<const typename BasicJsonType::string_t*>();
      |     ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:1127:8: note: candidate: 'MyClass& MyClass::operator=(const MyClass&)'
 1127 | struct MyClass {
      |        ^~~~~~~
/home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:1127:8: note:   no known conversion for argument 1 from 'const string_t' {aka 'const std::__cxx11::basic_string<char>'} to 'const MyClass&'
/home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:1127:8: note: candidate: 'MyClass& MyClass::operator=(MyClass&&)'
/home/nero/repos/external/nlohmann-json/test/src/unit-deserialization.cpp:1127:8: note:   no known conversion for argument 1 from 'const string_t' {aka 'const std::__cxx11::basic_string<char>'} to 'MyClass&&'

Error message when compiling with Visual Studio 16 2019 in C++20 mode:
Error    C2679    binary '=': no operator found which takes a right-hand operand of type 'const std::basic_string<char,std::char_traits<char>,std::allocator<char>>' (or there is no acceptable conversion)    test    C:\Software\vcpkg\installed\x64-windows\include\nlohmann\detail\conversions\from_json.hpp    121

Which compiler and operating system are you using?

Compiler: g++ 11.2.0 (only when setting C++20 mode)
Operating system: Ubuntu 21.10

Also on Visual Studio 16 2019 in C++20 mode
Which version of the library did you use?

[x] latest release version 3.10.5
[x] other release: 3.10.4
[x] other release: 3.10.3 (not affected by bug, working)
[x] the develop branch

If you experience a compilation error: can you compile and run the unit tests?

[ ] yes
[x] no - please copy/paste the error message below

"test-comparison" start time: Feb 01 15:05 UTC
Output:
----------------------------------------------------------
[doctest] doctest version is "2.4.6"
[doctest] run with "--help" for options
===============================================================================
/home/nero/repos/external/nlohmann-json/test/src/unit-comparison.cpp:46:
TEST CASE:  lexicographical comparison operators
  values
  comparison: less

/home/nero/repos/external/nlohmann-json/test/src/unit-comparison.cpp:213: ERROR: CHECK( (j_values[i] < j_values[j]) == expected[i][j] ) is NOT correct!
  values: CHECK( false == 1 )
  logged: i := 12
          j := 13
          j_values[i] := [1,2,3]
          j_values[j] := ["one","two","three"]

/home/nero/repos/external/nlohmann-json/test/src/unit-comparison.cpp:213: ERROR: CHECK( (j_values[i] < j_values[j]) == expected[i][j] ) is NOT correct!
  values: CHECK( true == 0 )
  logged: i := 13
          j := 12
          j_values[i] := ["one","two","three"]
          j_values[j] := [1,2,3]

===============================================================================
[doctest] test cases:    1 |    0 passed | 1 failed | 0 skipped
[doctest] assertions: 2220 | 2218 passed | 2 failed |
[doctest] Status: FAILURE!

edit: the unittest test-comparison fails in C++20 mode, as reported in issue: https://github.com/nlohmann/json/issues/3207

Can you try this:
template<typename BasicJsonType>
void from_json(const BasicJsonType &j, MyClass &obj)
{
    j.at("name").get_to(obj.name);
}

@gregmarr the workaround works! Although I don't want to use the templated function, as I'd have to move all my from_json() implementations from the cpp file into the hpp header files. But thanks for taking the time and providing a workaround :bow:

Are you currently putting extern void from_json(const nlohmann::json &j, MyClass &obj); in your header files?  If not, and it's currently working with nlohmann::json, then you don't need to move the implementations to get the templated version to work with nlohmann::unordered_json.
If you are doing that, you can just add a second version just like the first:
void from_json(const nlohmann::unordered_json &j, MyClass &obj)
{
    j.at("name").get_to(obj.name);
}

If you don't want to duplicate the whole body, you can do something like this:
template<typename BasicJsonType>
void from_json_internal(const BasicJsonType &j, MyClass &obj)
{
    j.at("name").get_to(obj.name);
}

void from_json(const nlohmann::unordered_json &j, MyClass &obj)
{
    from_json_internal(j, obj);
}

void from_json(const nlohmann::unordered_json &j, MyClass &obj)
{
    from_json_internal(j, obj);
}

@nlohmann Do we need to update the guides to tell people to use the templatized version if they're going to be using the other json variants?

@nlohmann Do we need to update the guides to tell people to use the templatized version if they're going to be using the other json variants?

Yes, I think this was forgotten when ordered_json was introduced.

so it was just a bug, that I was able to use it like that until now? (before C++20)

@NeroBurner Not sure why it worked, there may have been some temporaries being created that probably aren't desired.
@nlohmann It's not just ordered_json, it's any variation in the template parameters.

that's unfortunate, but you're probably right. Thanks for the workarounds. Should I close issue, or do you want to keep it open until documentation is updated for people to find this open issue?

Awesome!! Thank you so much for fixing this. I thought it won't be fixed because it was unintentional behaviour and creates temporary copies of the json object if the function doesn't match. But now it is fixed! :tada:
When possible I'll do the right fix with the templated from_json functions, but it is great to consciously keep the inefficient source code and still be able to update nlohmann_json
Thank you! :bow:


Lumen?

Hello,
Does this work with Lumen?

Hi, I did not test with lumen, you can add the right-hand one yourself.

Sadly enough, it doesn't work

Can you add support?

I'll try


Mangle the generated parameters

Is your feature request related to a problem? Please describe.
I found the generated bridge_generated.rs's function include a port parameter,when user function also includes a port parameter, the problem happened.
Describe the solution you'd like
Mangle the parameters used internally
Describe alternatives you've considered
a workaround is to change my port parameter to new_port or something else
Additional context
the generated snippet
#[no_mangle]
pub extern "C" fn wire_set_client(port: i64, port_: *mut wire_uint_8_list) {
//...
}
```

Agree!


Minimum Lovable Product

Our MVP should have:

Writing: WYSIWYG Markdown editor.
References: build a reference list with S2 APIs.
AI interactions: send content to a foundation model and get a response.

These 3 components should talk to each other:
1 -> 2: references know where they are cited.
1 -> 3: editor content can be sent to the AI.
2 -> 1: editor can cite references.
2 -> 3: reference content available to the AI.
3 -> 1: AI can make changes to the editor.
3 -> 2: AI can edit reference list (e.g. suggest a reference).

@hammer Given point 2 above, is it fair to assume the fully local use case (no internet) is out of scope for v1?
I think that scope decision has implications on backend design in terms of the LLM we use, any database or vector storage, etc.

@gjreda I wouldn't take these issues to be written in stone. I just wrote something up to start discussion. I would like to keep local mode high on the priority list for this 3 month work cycle, which may go beyond an MVP. Upon further thought it's probably easier to bring your own (BYO) PDF than to talk to S2 APIs, so perhaps point 2 should be BYO PDF instead of use of S2 APIs, which involves a second step of fetching the full text.

Thank you @danvk for the title improvement from Viable to Lovable 😑

@hammer just came across this one. I'm going to close it feels like we've finished the pieces mentioned here, but feel free to reopen if you want to keep it around


Added Vue.js Tutorial

This PR adds the Vue.js tutorial.
It changes the folder structure to have 2 new directories: docs/VanillaJS and docs/VueJS. The docs/tutorial.md is now used to select the preferred tutorial technology (VanillaJS or VueJS).
Looking forward to further improvements.

Nice.
Here are my first remarks (I need to try to do the full tutorial from scratch to have more):

There is conflict because I've upgraded the text in your previous PR.
The new tutorial introduction text to choose between Vue.js and VanillaJS should be improve: not everyone now what is VanillaJS or Vue.JS, you should mention that Vue.js tutorial is for people familiar with Vue.js concept and in case of doubt it's better to choose the VanillaJS version. May be you could say also that the result will be the same at end.
You should change the screen capture for File Structure in step 1 for Vue.js because new directories are now there (components)
When you talk about Sugar components in step 1 for Vue.js, it's better to talk about Sugar components for Vue.js
In step 2, you're explaining the internal of Sugar activity works in Understanding the working paragraph, I'm not sure it's useful
In step 3 for Vue.js your code mention a splitbar but it's not visible in the screen capture, I think it's better to remove it
In step 4 for Vue.js, it's a bad things that the user can't test itself if it's a new instance or not. I wonder if it's not interesting to send a journal-data-loaded event in the case of a new instance too. Specifically, the console.log message should not came from the component but from the activity itself.
In Step6, I wonder if you really need to test if (this.SugarPresence  is it possible that the SugarPresence object is null/undefined?

After latest commits;

Resolved the conflict.
Made the changes in the new tutorial introduction text, please suggest more if needed.
Added the new screen capture to step 1 in Vue.js.
Removed splitbar from step 3.

When you talk about Sugar components in step 1 for Vue.js, it's better to talk about Sugar components for Vue.js

Not sure what you mean here. Should I mention about all components?

In step 2, you're explaining the internal of Sugar activity works in Understanding the working paragraph, I'm not sure it's useful

I added this mainly to explain about define and require like in the VanillaJS tutorial. I have removed it as it seems odd to only explain working for one component and not other.

I wonder if it's not interesting to send a journal-data-loaded event in the case of a new instance too.

I kept this check in the component as it will always be required and the developer will have to add this every time in the the event handler. Do we really need to bring it out?

I wonder if you really need to test if (this.SugarPresence

It was there to prevent any console errors which might arise due to improper inclusion of the component. Well if the tutorial is followed properly this shouldn't be an issue. I think we can remove it.

The tutorial introduction is better. BTW I think it's better to talk about Vanilla JS before talking about Vue.JS.
Regarding Step 1, the idea is just to replace the sentence "contains all Sugar components which..." by "contains all Sugar components for Vue.js which...".
Regarding Journal, the problem is that very often developers need to do a different initialization when activity is loaded from the Journal and when activity is created from scratch. If we don't provide an event to differentiate both it require for developers to initialize from scratch then redo initialization when the journal-data-loaded event will be raise. So if suggest to raise an event for new instance too (may be another event if you think it's not pertinent to use the same). I'm also concern about the error since no event is raised when there is an error.
One more thing: it will be better to put files in docs/tutorial/VanillaJS and docs/tutorial/VueJS directories instead of docs/VanillaJS and docs/VueJS directories.

After latest commit:

Updated the folder structure to docs/tutorial/{language}.
Made the suggested changes to the tutorial introduction and step 1.
Added tutorial for new events in SugarJournal added in #806.

@dhruvmisra , the image paths and some links are broken ( after you Updated the folder structure to
docs/tutorial/{language}), like ![](../images/tutorial_step1_1VueJS.png) should be ![](../../images/tutorial_step1_1VueJS.png), etc. :)

Ah, must have missed these. I'll add a commit to fix it. Thanks! :)

very good tutorial :),
just few comments -

In step 2, in requirejs.config,  activity: "../../js" should be activity: "../../js"
Image paths for background images in css extracts should be url(../icons/add.svg); instead of url(../../icons/add.svg);
locale.ini is already included in vue template ,so i think we can remove the step showing integration of locale.ini in vuejs tutorial.
In step 5, this.$refs.SugarActivity.setup(); is giving error , it should be removed,
In step 6, we should have an instruction  to add this.SugarPresence = this.$refs.SugarPresence; in mounted, just like we have for this.SugarL10n  in step 5.
In step 7, css for adding insert-button should be #main-toolbar #insert-button {   ... } instead of #main-toolbar #picture-button {   ... }
In step 8 , define([...,"text!pawnpalette.html"] should be define([..., "text!activity/palettes/pawnpalette.html"]
In step 9, I think we should have a line saying that all the files like bootstrap-tour,...etc are already included in vuejs template
In step 9, <script src="lib/jquery-3.4.1.min.js"></script> should be <script src="lib/jquery.min.js"></script> as we have jquery.min.js in our template.
In step 9, we should have download links for go-left.svg, .... etc.

Hey @prakashujjwal1010, thanks for this in-depth review of the tutorial!
I'm sure these relative paths are going to be the end of me. xD I have fixed the paths at places you mentioned. Found some more in step 9, fixed them too.

locale.ini is already included in vue template

I think it was left out while stripping down the Pawn activity to make the template. The goal is to keep the files as similar to the Vanilla JavaScript version as possible. Should we keep the locale.ini @llaske?

In step 9, we should have download links for go-left.svg, .... etc

Hmm, now that I think of it, Vue components should be usable out of the box. That's why we were providing the dependencies (Bootstrap tour, jQuery, etc) already with the template. As the styling for tutorial is already present in css/activity.css, I think we should also include the icons go-left.svg, ... etc. with the template too. What do you think @llaske?
I agree with all other points and have made those changes. Waiting for more suggestions and fixes. :)

Good job @prakashujjwal1010 thanks.
Regarding presence or absence of some files, because we've decided that the VanillaJS and Vue.js are different, there is no reason to force the user to download files even if it's need for the VanillaJS tutorial. Clearly the Vue.js tutorial is easier than the VanillaJS tutorial so it's normal that some files related to Sugar Vue.js components are already there.
So we could keep an empty locale.ini file in the template, just change the text to mention that file is already there. It make sense too to provide icons for the tutorial since jQuery and bootstrap are already include.

Sure. I have updated Step 5 mentioning that locale.ini already exists. I will add the icons in the next PR along with LZString compression soon.

I've tested the full Vue.js tutorial from scratch. I took me some time but I must confess it's not the first activity I've wrote :-)
My remarks:

Remove the splitbar in index.html in the template else the result don't not match the screen capture
Step 2, you should detail each time a source code should be updated/modified which file should be change (confusion between index.html and js/activity.js) - except if the same file has just be changed before
Same remarks in step 3, step 4, step 6 and step 7
In Step 3, You should mention the full code to create a Pawn component, you don't mention where the mounted should be located. Sure it's obvious for a Vue.js user but it's good to mention it thought.
In Step 3, for the same reason mention also the location of components property in activity.js file
In Step 3, you should mention that the pawn.js file should be include before the activity.js file (else it will not work)
In Step 4, when you integrate the SugarJournal component you should mention that 2 lines are added in the index.html file  (may be you could use comments like in step 6)
In Step 5, the locale.ini is already integrated in index.html, you should remove the part where you describe how to integrate it
In Step 5, when you integrate the SugarL10n component you should mention that 2 lines are added in the index.html file  (may be you could use comments like in step 6)
In Step 5, the Played message is not localized (try with Sugarizer in French). I've tried with your version of Pawn.activity and it doesn't work neither. I think something goes wrong in your SugarL10n component
In Step 8, all CSS items related to the palette are already in the activity.css file of the template, it should not, it's purely related to Pawn activity

I've explored the issue in SugarL10n component. It seems related to a second localized event received after the one from Sugarizer language change. I suggest you to update the mounted method like that:

	mounted: function () {
		var vm = this;
		if (vm.l10n == null) {
			requirejs(["sugar-web/env", "webL10n"], function (env, webL10n) {
				env.getEnvironment(function (err, environment) {
					vm.l10n = webL10n;
					var defaultLanguage = (typeof chrome != 'undefined' && chrome.app && chrome.app.runtime) ? chrome.i18n.getUILanguage() : navigator.language;
					var language = environment.user ? environment.user.language : defaultLanguage;
					webL10n.language.code = language;
					window.addEventListener("localized", function () {
						if (!vm.eventReceived) {
							vm.code = language;
							vm.dictionary = vm.l10n.dictionary;
							vm.$emit("localized");
							vm.eventReceived = true;
						} else {
							if (webL10n.language.code != language) {
								webL10n.language.code = language;
							}
						}
					});
				});
			});
		}
	},

The else part was added, it force the language to the Sugarizer language. So it could happen again.

Made all the required changes to the tutorial text. I'll commit changes to the Activity Template in the LZString PR soon.

It seems related to a second localized event received after the one from Sugarizer language change.

I was exploring the second localized event as well. Thanks I'll update the SugarL10n component. Any reason why you think the second event was being fired with en-US (I think default) language code?

Thanks.
Regarding the localized event, it's raised one time for the browser (the default as you said) and each time a new language is load. So when the language is change like we're doing at start.


USDC: @celo/connect + web3js demo

Using demo from the @celo/connect README.
Related

https://github.com/celo-org/developer-tooling/issues/128

copy/paste insight from @aaronmgdr

ok so here is the issue.
in order to sign a cip64 tx a Wallet that is a subclass of WalletBase must be used as that is where the actual code for rlp encode and sign is. in the setup you have its using the wallet on web3. which only knows eip1559
this line https://github.com/celo-org/developer-tooling/blob/679ef0c607500a69f2d1d684740632202ab79ad2/packages/sdk/connect/src/connection.ts#L72
if it were not commented out would default the wallet on connection to a LocalWallet.  instead of a ReadOnly Wallet. we should either uncomment that or make it so calling connection.addAccount creates an LocalWallet instance (which would make more sense to me)

I'm wondering what purpose the ReadOnlyWallet interface serves.
export interface ReadOnlyWallet {
    getAccounts: () => Address[];
    removeAccount: (address: Address) => void;
    hasAccount: (address?: Address) => boolean;
    signTransaction: (txParams: CeloTx) => Promise<EncodedTransaction>;
    signTypedData: (address: Address, typedData: EIP712TypedData) => Promise<string>;
    signPersonalMessage: (address: Address, data: string) => Promise<string>;
    decrypt: (address: Address, ciphertext: Buffer) => Promise<Buffer>;
    computeSharedSecret: (address: Address, publicKey: string) => Promise<Buffer>;
}

Source: connect > wallet.ts
Why is it called "ReadOnly" if you can sign a transaction with a class that implements it?

Wondering if LocalWallet() is a web3.js specific concept.

Local wallets are an in-memory wallet that can hold multiple accounts. Wallets are a convenient way to sign and send transactions in web3.js

Source: https://docs.web3js.org/guides/wallet/local_wallet/
It's not obvious where web3.js stops and Celo-specific constructs start (e.g. ReadOnlyWallet)

Wondering if LocalWallet() is a web3.js specific concept.

Local wallets are an in-memory wallet that can hold multiple accounts. Wallets are a convenient way to sign and send transactions in web3.js

Source: https://docs.web3js.org/guides/wallet/local_wallet/
It's not obvious where web3.js stops and Celo-specific constructs start (e.g. ReadOnlyWallet).

while web3 has wallets. you must use the celo wallets to get the signing happening -- see the wallets directory in the sdks folder of developertooling repo.

Also curious how _wallet and LocalWallet() are defined.

_wallet comes from the params above
LocalWallet would be imported from @celo/wallet-local

Why is it called "ReadOnly" if you can sign a transaction with a class that implements it?

yeah i cant answer that one. seems like it should be have a better name

Also curious how _wallet and LocalWallet() are defined.

_wallet i think comes from the params above (which is now just wallet)
LocalWallet would be imported from @celo/wallet-local

Thanks, this helped a lot. I used LocalWallet in https://github.com/celo-org/feecurrency/pull/3/commits/c9b9e3f727a3b96fd561e64d3c032d298030ad34.
At the moment, the script is stuck and doesn't timeout or complete. I'll debug that next

(for future reference)
Also noting that type: 'cip64' in the Connection.sendTransaction arguments did not accept a string
Cannot convert string to Uint8Array. only supports 0x-prefixed hex strings and this string was given: cip64
const transactionReceipt = await connection
        .sendTransaction({
            from: sender,
            to: CONTRACT_ADDRESS,
            gas: 51925, // TODO: implement gas estimation
            maxPriorityFeePerGas: web3.utils.toWei("10", "gwei"),
            maxFeePerGas: web3.utils.toWei("10", "gwei"),
            feeCurrency: "0x874069Fa1Eb16D44d622F2e0Ca25eeA172369bC1", // cUSD fee currency
            data: transactionObject.encodeABI(),
+            type: 'cip64'
        })

Error:
$ yarn ts-node web3.ts
yarn run v1.22.19
$ /Users/arthur/Documents/celo-org/feecurrency/node_modules/.bin/ts-node web3.ts
Initiating fee currency transaction...
/Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth-accounts/src/common/utils.ts:344
                        throw new Error(
         ^
Error: Cannot convert string to Uint8Array. only supports 0x-prefixed hex strings and this string was given: cip64
    at toUint8Array (/Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth-accounts/src/common/utils.ts:344:10)
    at Function.typeToInt (/Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth-accounts/src/tx/transactionFactory.ts:43:48)
    at Function.fromTxData (/Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth-accounts/src/tx/transactionFactory.ts:68:37)
    at /Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth/src/utils/prepare_transaction_for_signing.ts:145:28
    at Generator.next (<anonymous>)
    at fulfilled (/Users/arthur/Documents/celo-org/feecurrency/node_modules/web3-eth/lib/commonjs/utils/prepare_transaction_for_signing.js:21:58)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Might be a different reason, but keeping track of this here in case.

Hint: Learnt that prepending a Node.js command with DEBUG=* executed the Node.js script with verbose output in the console.
For example:
$ DEBUG=* yarn ts-node web3.ts

With the updated script, @celo/connect + web3 (incl. @celo/wallet-local) successfully makes a CIP64 transaction.
For example: https://alfajores.celoscan.io/tx/0x0cbac87c455a0cf9a1cb6bd340f1332d45eb767a570e901d72b386fb58790f79


Document cache (1000) section

tl;dr: proper summary documentation is actually necessary for the 1000 section, per #51.
There was discussion on #51 about how (much) to document disabling caching etc. in section 1000, and some explanatory text discussed there was removed (seemingly on the basis of "Everyone will know what Cache is and does, and whether they want it enabled or not.")
Problem is, although the user of ghacks-user.js might (and probably does) know plenty of standard stuff about browser caching, what they don't know, is if there's some disk cache-based privacy (or other) attack/risk they haven't read about.
I know in my case, when I came to this section, my immediate thought was "Wait, why is cache being disabled? Is this just an "on-disk browsing info available locally" problem? Is it a tracking issue (ETags?) Is there some other attack vector I don't know about they're preventing here?"
So seeing the mostly undocumented inclusion of cache-disabling items, they now don't know if they can safely override them.
Compare signon.autofillForms. If the user keeps their system locally secure, they might think that feature is harmless, whereas in that case, there is both a 3rd-party and 1st-party tracking issue with leaving it enabled.
Is there a similar problem with cache? Without 1000 docs, the user doesn't know.

I've almost closed this a dozen times, and the few times I have started to try and integrate the info, it all just gets too messy @smithfred is the first person (AFAIK) in approx 3 years, since I started this thing, to ask about cache

I know in my case, when I came to this section, my immediate thought was "Wait, why is cache being disabled? Is this just an "on-disk browsing info available locally" problem? Is it a tracking issue (ETags?) Is there some other attack vector I don't know about they're preventing here?"

It's EXACTLY what you thought it was ("on-disk browsing info available locally" - no need to explain it, you got it in one. It's THAT obvious.
IF a pref is there for a specific threat, we explain it, but for generalized threats, there is no need.

I'm neither dumb nor uninformed

Didn't mean to imply you were. You raised a valid point see fresh eyes comment below

Presumably you're confident that

No, I'm not confident. I have blind spots due to the fact that I have been living in this shell for 3 years and am not immune to assumptions - that's why a fresh set of eyes is always welcome. The other side of the coin is that a line has to be drawn somewhere regards verbosity, assumed level of knowledge, etc.
600+ prefs - complex intertwined hot-mess of constantly changing tech ... kinda going to go out on a limb here and say its well beyond the average user - not that I'd don't TRY to make it easier
As said elsewhere, the answer to this is a relaxed branch etc where 30-40 prefs are relaxed for basically zero breakage - then the less knowledgeable can feel more confident in using it, and slowly build up

I listed 3 different possibilities

Your two possibilities ("something else" does not qualify) were both correct. The first was the main reason for disk cache. ETAGs are also controlled by disk cache but require memory cache also turned off. So you already had the answers.

@smithfred  if you still want to enable memory cache, then follow https://github.com/ghacksuserjs/ghacks-user.js/wiki/4.2.4-Header-Editor to get rid of ETags.

Your two possibilities ("something else" does not qualify)

Local/unlocked history access; tracking; attack (vulnerability exploit) vector. They all qualify as the point is to enumerate the possible classes of problem to demonstrate the ambiguity of undocumented pref settings, not to know them exactly. The fact that the last one is vague only strengthens the point. New vectors crop up so frequently that someone having certainty that they know all of them (and can therefore determine with certainty without documentation the purpose of a pref set) is extremely difficult.

So you already had the answers.

In the same way that someone who doesn't know the correct answer to a multiple-choice question already "has all the answers". I.e. only in a literal, and completely useless sense :J


(Transmit) React router not passing QueryParams to Transmit?

I hope Im overlooking something simple I feel like this should be straight forward. I have some react router routes
import React from "react";
import {Route, DefaultRoute} from "react-router";
import Main from "views/Main";
import Connect from "views/Connect";

export default (
  <Route path="/">
    <DefaultRoute handler={Main} />
    <Route path="/connect/:id" handler={Connect}/>
  </Route>
);

I then have Connect handler
import React from "react";
import Transmit from "react-transmit";
import { Navbar,Button } from "react-bootstrap";
import Promise from "bluebird";

const request = require('superagent-promise')(require('superagent'), Promise);

class Connect extends React.Component {

  render (){
    return (
      <div>
        <Navbar></Navbar>
          <form>
            <input type="text" />
          </form>
      </div>
    )
  }

}

export default Transmit.createContainer(Connect, {
  queries: {
      test (queryParams){
        console.log("wtf queryParams*****", queryParams, this);
      }
  }
});

queryParams is always empty in the test function. I was hoping to pull :id from the react router route to pass in to one of the Transmit queries

@RickWong any thoughts or guidance on this?
Seems similar to https://github.com/RickWong/react-transmit/issues/10
Which dosen't look like it was properly resolved

Hi @kevzettler I would recommend writing a new ConnectHandler component that handles the route, and renders <Connect queryParams={{id}} />. It functions as a bridge between React Router and Transmit.
Although it's one more component to write, it does separate routing from the Connect component. Doing it like this, enables you to re-use the Connect component somewhere else with another id.

Hi @RickWong
Sorry to bring up this old, bug but your workaround doesn't seem to work for me. Am I doing something wrong here:

Hi @RickWong,
Maybe I'm missing something, but your workaround doesn't seem to work for me. Here is what I did
// Routes
<Route name="user" path="/user/:userId" handler={IdProxy} />

// in IdProxy
render() {
  return <User userId={this.props.params.userId} />
}

// in User (ES6 syntax)
class User extends React.Component {
  render() {
    const user = this.props.user;
    console.log('p: ', this.props);
    return <div/>;
  }
}

export default Transmit.createContainer(User, {
  queries: {
    user(queryParams) {
      console.log('qp: ', queryParams);
      return new Promise((resolve, reject) => {
        resolve(1);
      });
    }
  }
});

This still ends up with the props containing the userId but the queryParams being blank. If I need to load a user in the queries, I cannot without the userId.

You should render the User like this: <User queryParams={{userId: this.props.params.userId}} />. See this DOCS section.

Thanks for clarifying :)


Add support for ghc 7.10

@sestrella @jsl @jsantos17 guys could you take a look please?

@juanpaucar LGTM great job!


[gklib] Add new port

The upstream of metis is gone, the new source metis need gklib as dependency, add new port gklib.

Closed this PR, due to PR #26020 will fix the build error of metis.


There is a difference in the maximum data size value of resource limit.

The value of Max data size (GB) is different from the actual portal screen. Please correct.
There are several such as Gen 4 1 core.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5676aae7-73bc-c084-174b-50ba4fd7be77
Version Independent ID: 0998f821-0f03-a833-b4b3-0d6ebd9468e5
Content: Azure SQL Database vCore-based resource limits - single database
Content Source: articles/sql-database/sql-database-vcore-resource-limits-single-databases.md
Service: sql-database
GitHub Login: @CarlRabeler
Microsoft Alias: carlrab

For exemple in the documentation BC_Gen5_12 claims for 1536GB Max data size (GB) but in the portal for a BC_Gen5_12 you can attach up to 3TB.

I will investigate
From: philimag<EMAIL_ADDRESS>Sent: Tuesday, February 5, 2019 8:33 AM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: Carl Rabeler<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] There is a difference in the maximum data size value of resource limit. (#23879)
For exemple in the documentation BC_Gen5_12 claims for 1536GB Max data size (GB) but in the portal for a BC_Gen5_12 you can attach up to 3TB.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F23879%23issuecomment-460706532&data=02|01|Carl.Rabeler%40microsoft.com|d884bffb641342a547ac08d68b87a8ed|72f988bf86f141af91ab2d7cd011db47|1|0|636849812111952408&sdata=RFWIBwFLSC3vEivxHHo9TKdLJV7RzCOfh4y8V1M7%2FqE%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMCBnw7lqIw85MpMEh5D6cxUUJMxIpBjks5vKbJYgaJpZM4abqfR&data=02|01|Carl.Rabeler%40microsoft.com|d884bffb641342a547ac08d68b87a8ed|72f988bf86f141af91ab2d7cd011db47|1|0|636849812111952408&sdata=v8QUTAkdrqqdfjBBtc0d3TegEI7A4j%2Bg4McVYu6m%2FBc%3D&reserved=0.

Preliminary feedback is that this is an error in the Azure portal.

stay tuned - update to this thread coming soon

@philimag   Fixed  https://github.com/MicrosoftDocs/azure-docs-pr/pull/68453 live later today
@angoyal-msft  @Mike-Ubezzi-MSFT please close

We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.

Business Critical Gen4 max data sizes appear to be wrong in the doc. The doc has it at 650 GB for 10 vCores and below, but in the portal you can go up to 1 TB for any number of vCores.
The BC_Gen 5 sizes appear to be correct.

@stevestein for your awareness and action

Reopening this issue to address additional inconsistencies not addressed by previous PRs or because additional changes have been made since then.

@v-makta, @tonyjthomas - Thanks for the feedback. Doc has been updated to match portal values.
Thanks!
#please-close


Feature: automated semver releases

We use a webhook to create stable docker images when a release test run successfully completes. Similarly we should have a webhook, such that when prod cron job completes successfully, we use semver specification to automatically create a new tag and release. On a high-level this is what the webhook needs to do:

We'll create and maintain a Scorecard GitHub app whose token/keys will be used for automated signing of tags/commits.
Use the latestTag of Scorecard repo and the commitSHA of the successful run to get all commits in-between.
Depending on the commit message headings (:sparkles:, :bug:, :book: etc.) decide whether to increase the PATCH version or the MINOR version. MAJOR version changes will be done manually and need not be handled through the webhook.
Create a new tag in compliance with semver semantic and push this new tag to Scorecard repo. This should trigger a new release and our work is done.

https://github.com/go-git/go-git might come in handy for doing the Git operation in Golang.

how do we handle automated release notes?

There was a discussion about automated release note generation, can't seem to find the issue/thread for it. Gist was that Kubernetes does something like this already, so we could do it too. @naveensrinivasan may know more about this.
But, to your point, yes we should also consider adding support for automated release note generation also. Let's create a separate issue for automated release note generation? Scope of the issue can be:

come up with a release template
automate the generation of this template
make this part of the release workflow

My main concern/question is how we automate release notes that are human-consumable. A list of merged PRs is not very eligible for human. If we could do that it'd be nice, unless we think it's overkill for minor releases.

Sorry it was not Kubernetes, but sigstore/cosign - https://github.com/sigstore/cosign/releases, released by sigstore-bot. Like you said, the release notes should be human readable not just changelog. And it is possible to automate it that way.

There was a discussion about automated release note generation, can't seem to find the issue/thread for it. Gist was that Kubernetes does something like this already, so we could do it too. @naveensrinivasan may know more about this.

My team manages this tool: https://github.com/kubernetes/release/tree/master/cmd/release-notes
Here are some examples of output from the tool:

https://github.com/kubernetes/release/releases/tag/v0.12.0
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md

While I haven't personally used it, the tool also supports supplying your own go template to enable further formatting.
One thing to note is that it expects a code-fenced block in the PR description (with release-note as the code type):
A really great human-readable release note

The idea here being that while we can't always expect the commit history to be perfect (maybe lacking convention or from a new contributor with multiple commits that need squashing), as maintainers we have access to edit the PR descriptions to ensure the release-note is does a reasonable job describing the change.
Some examples: https://github.com/kubernetes/release/pull/2370, https://github.com/kubernetes/release/pull/2330, https://github.com/kubernetes/release/pull/2326
Similar to how PR titles are validated here, in several Kubernetes repos we automatically block PRs without either:

A release-block that is populated
A release-block that has NONE as the text

As for releasing:

https://github.com/goreleaser/goreleaser
https://github.com/goreleaser/goreleaser-action
GoReleaser + cosign: https://github.com/goreleaser/goreleaser/issues/2659

@justaugustus would the work that you are doing on #1677 make it feasible to generate automated release notes?


[UPDATE] Changing URL of PRs and fixing conflict errors

Pass html_url to PR, not url attribute

Fixing conflict errors, because of request being send at the same time


Recovery middleware not working with Timeout middleware

I am using Timeout middleware with negroni . Sample code can be found here: https://gist.github.com/Vikash082/c3af4fb89728f816beb8204b4737d56d
But when there is panic occurs in my http handlers, negroni Recovery middleware doesn't work which means programs crashes. If I disable my Timeout middleware, program recovers as expected. Is there I need to change in my Timeout middleware to work with Recovery middleware.

Hi @Vikash082,
Unfortunately the Recovery middleware will not catch panics occurring in a separate goroutine (see #190 for more information).
I'm not super familiar with the context timeout pattern myself, but it seems like you should be able to do something like:
        ch := make(chan struct{})
        go func() {
                select {
                case <-ch:
                        return
                case <-ctx.Done():
                        w.WriteHeader(http.StatusRequestTimeout)
                        w.Write([]byte("Request Timed Out."))
                        return
                }
        }()

        next(w, r.WithContext(ctx))
        ch <- struct{}{}

Instead, to move the code likely to throw panics into the main request goroutine. You could also leave your code as-is, explicitly catch panics within the go func() {}, and reraise them outside.
Hope this helps!

@jszwedko Thanks for reply. Till time I was recovering from panic by using defer in below goroutine. So, which way (preferred) is better and I should follow ?
	go func() {
   		defer recovery(ch, w)
   		next(w, r.WithContext(ctx))
   		ch <- struct{}{}
   	}()
   	select {
   	case <-ch:
   		return
   	case <-ctx.Done():
   		msg := map[string]string{"err": "Request Timed Out"}
   		resp := utils.JsonResp(http.StatusRequestTimeout, msg)
   		resp.WriteHTTPResponse(w)
   		return
	   }

Hi @Vikash082,
I would move next() out rather than adding an additional defer recovery, but that is just my preference. The only advantage I see to that is to just rely on the recovery middleware to catch the panics.

Hi @jszwedko
One issue which I see is, during Timeout. goroutine writes response, even the backend function after few second try to write response . In server we get print like this:
http: multiple response.WriteHeader calls
and client hangs indefinitely.

Closing since this is a known behavior per https://github.com/urfave/negroni/issues/190


Unable to list files from s3 bucket

Bug Report
Issue
I am using the Node SDK 0.10.0 to try list the files of a bucket on AWS.
import { api, bucket } from "@nitric/sdk";

const helloApi = api("main");
const profiles = bucket("profiles").for("reading", "writing", "deleting");

helloApi.get("/files", async (ctx) => {
  try {
    const files = await profiles.files();

    console.log(files);

    return ctx.res.json({
      files,
    });
  } catch (e) {
    return ctx.res.json({
      error: e,
    });
  }
});

Steps
Steps to reproduce the behavior:

use the typescript example template
add the above code
deploy to AWS
attempt to retrieve the files via the /files GET endpoint

Expected
should return an empty array like it does on GCP
Output of nitric info -o yaml:
{
  "os": "windows",
  "arch": "amd64",
  "goVersion": "go1.18.10",
  "cliVersion": "1.15.0",
  "fabricVersion": "v0.22.0",
  "containerRuntime": "docker",
  "containerRuntimeVersion": "platform:\n  name: Docker Desktop 4.11.1 (84025)\ncomponents:\n- name: Engine\n  version: 20.10.17\n  details:\n    ApiVersion: \"1.41\"\n    Arch: amd64\n    BuildTime: \"2022-06-06T23:01:23.000000000+00:00\"\n    Experimental: \"false\"\n    GitCommit: a89b842\n    GoVersion: go1.17.11\n    KernelVersion: 5.4.72-microsoft-standard-WSL2\n    MinAPIVersion: \"1.12\"\n    Os: linux\n- name: containerd\n  version: 1.6.6\n  details:\n    GitCommit: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1\n- name: runc\n  version: 1.1.2\n  details:\n    GitCommit: v1.1.2-0-ga916309\n- name: docker-init\n  version: 0.19.0\n  details:\n    GitCommit: de40ad0\nversion: 20.10.17\napiversion: \"1.41\"\nminapiversion: \"1.12\"\ngitcommit: a89b842\ngoversion: go1.17.11\nos: linux\narch: amd64\nkernelversion: 5.4.72-microsoft-standard-WSL2\nexperimental: false\nbuildtime: \"2022-06-06T23:01:23.000000000+00:00\"\n",
  "detectedErrors": []
}

Other info
In the Node SDK, data is undefined from the this.storage.StorageServiceClient.listFiles call

Resolved permission issue in: https://github.com/nitrictech/nitric/pull/397

Release in dev providers (needs prod release of CLI & providers to be available).

released to nitric aws provider v0.23.0


TypeScript conversion groundwork

Adds groundwork to split LiteGraph classes out into TS modules.

Exports classes
Removes IIFE wrapper

Blocked until #161, #162

@webfiltered Invited you as collaborator so you can directly create branch in the repo. Let me know if you want me to resolve the merge conflict for you.


Panic in rancher when upgrading k8s version of cluster with drain enabled

What kind of request is this (question/bug/enhancement/feature request): bug
Steps to reproduce (least amount of steps as possible):

(this was an upgraded rancher HA setup from 2.4.8 to 2.5.0-rc7)
Upgrade k8s version of the cluster from 1.18.8 to 1.18.9 (custom rke cluster - 5 nodes, 1 etcd, 1 control and 3 worker nodes)
Panic in rancher server and rancher is seen restarting
Rancher logs:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2994c5d]

goroutine 10154 [running]:
github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.getFlags(0xc001c458c0, 0xc008763b00, 0x4690790, 0x0)
	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:243 +0x1cd
github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drain.func1(0x44474e0, 0xc003515b00, 0x44c8a6c, 0x7)
	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:161 +0x13d
github.com/rancher/norman/condition.Cond.doInternal(0x44c8a6c, 0x7, 0x4e25301, 0x4e253e0, 0xc003515b00, 0xc002becf38, 0x6, 0x3906a00, 0xc003499db0, 0x198)
	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:212 +0x97
github.com/rancher/norman/condition.Cond.do2(0x44c8a6c, 0x7, 0x44c8a01, 0x4e253e0, 0xc003515b00, 0xc002becf38, 0x4065ea, 0xc0047997a0, 0x0, 0xc008702000, ...)
	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:181 +0x1d8
github.com/rancher/norman/condition.Cond.do(...)
	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:170
github.com/rancher/norman/condition.Cond.DoUntilTrue(0x44c8a6c, 0x7, 0x4e253e0, 0xc003515b00, 0xc002becf38, 0x0, 0x0, 0x0, 0xc0072dc520)
	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:136 +0xb3
github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drain(0xc004c8d040, 0x4e88f60, 0xc008702080, 0xc003515b00, 0xc005d05bc0, 0xf, 0xc0086ed770)
	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:148 +0x1a5
created by github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drainNode
	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:99 +0x335

Expected Result:
No panic should be seen when drain is enabled
Other details that may be helpful:

API request sent contains:

drain: "true"
maxUnavailableControlplane: "1"
maxUnavailableUnit: "percentage"
maxUnavailableWorker: "10%"
nodeDrainInput: {force: false, deleteLocalData: false}
deleteLocalData: false
force: false

Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.5.0-rc7
Installation option (single install/HA): HA

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom RKE
Kubernetes version (use kubectl version):

1.18.9

This bug is reproduced

run Rancher v2.4.8 in HA mode in an RKE cluster v1.18.6
add a custom cluster 1.18.8 (custom rke cluster - 5 nodes, 1 etcd, 1 control and 3 worker nodes)
upgrade rancher to 2.5.0-rc8
upgrade the custom cluster with the option drain node YES

Results:

Rancher pods start crashing with the following kind of errors

[rancher-bbc54ccff-fbpxz] panic: runtime error: invalid memory address or nil pointer dereference
[rancher-bbc54ccff-fbpxz] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x2994c5d]
[rancher-bbc54ccff-fbpxz]
[rancher-bbc54ccff-fbpxz] goroutine 18385 [running]:
[rancher-bbc54ccff-fbpxz] github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.getFlags(0xc00a3173b0, 0xc009b29b00, 0x4690788, 0x0)
[rancher-bbc54ccff-fbpxz] 	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:243 +0x1cd
[rancher-bbc54ccff-fbpxz] github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drain.func1(0x44474e0, 0xc0086a6d80, 0x44c8a6c, 0x7)
[rancher-bbc54ccff-fbpxz] 	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:161 +0x13d
[rancher-bbc54ccff-fbpxz] github.com/rancher/norman/condition.Cond.doInternal(0x44c8a6c, 0x7, 0x4e25301, 0x4e253e0, 0xc0086a6d80, 0xc0098b6f38, 0x6, 0x3906a00, 0xc00d385db0, 0x198)
[rancher-bbc54ccff-fbpxz] 	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:212 +0x97
[rancher-bbc54ccff-fbpxz] github.com/rancher/norman/condition.Cond.do2(0x44c8a6c, 0x7, 0x44c8a01, 0x4e253e0, 0xc0086a6d80, 0xc0098b6f38, 0x4065ea, 0xc009155b60, 0x0, 0xc00670b000, ...)
[rancher-bbc54ccff-fbpxz] 	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:181 +0x1d8
[rancher-bbc54ccff-fbpxz] github.com/rancher/norman/condition.Cond.do(...)
[rancher-bbc54ccff-fbpxz] 	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:170
[rancher-bbc54ccff-fbpxz] github.com/rancher/norman/condition.Cond.DoUntilTrue(0x44c8a6c, 0x7, 0x4e253e0, 0xc0086a6d80, 0xc0098b6f38, 0x0, 0x203001, 0x203001, 0x390a4c0)
[rancher-bbc54ccff-fbpxz] 	/go/pkg/mod/github.com/rancher/norman@v0.0.0-20200930000340-693d65aaffe3/condition/condition.go:136 +0xb3
[rancher-bbc54ccff-fbpxz] github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drain(0xc0051d25a0, 0x4e88f60, 0xc00670b080, 0xc0086a6d80, 0xc00a4403f0, 0xe, 0xc00b67c640)
[rancher-bbc54ccff-fbpxz] 	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:148 +0x1a5
[rancher-bbc54ccff-fbpxz] created by github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer.(*nodeDrain).drainNode
[rancher-bbc54ccff-fbpxz] 	/go/src/github.com/rancher/rancher/pkg/controllers/managementuser/nodesyncer/cordonfieldssyncer.go:99 +0x335

UI is not accessible anymore with 503 Service Temporarily Unavailable

verified on an upgrade from 2.4.8 to 2.5-head - commit id: 65f3525cd

On 2.4.8 - deploy HA rke cluster and a custom RKE cluster (through API) k8s - 1.18.8
upgrade rancher to 2.5-head.
Upgrade k8s version of the cluster to 1.18.9
API request sent ocntains:

upgradeStrategy: {maxUnavailableControlplane: "1", maxUnavailableWorker: "10%", drain: "true",…}
drain: "true"
maxUnavailableControlplane: "1"
maxUnavailableUnit: "percentage"
maxUnavailableWorker: "10%"
nodeDrainInput: {force: false, deleteLocalData: false}
deleteLocalData: false
force: false

Cluster upgrades successfully and no panic/restart is seen in Rancher.

Verified on an upgrade form 2.4.8 to master-head commit id: 6bc32da

On 2.4.8 - deploy HA rke cluster and a custom RKE cluster (through API) k8s - 1.18.8
upgrade rancher to 2.5-head.
Upgrade k8s version of the cluster to 1.18.9
API request sent ocntains:

upgradeStrategy: {maxUnavailableControlplane: "1", maxUnavailableWorker: "10%", drain: "true",…}
drain: "true"
maxUnavailableControlplane: "1"
maxUnavailableUnit: "percentage"
maxUnavailableWorker: "10%"
nodeDrainInput: {force: false, deleteLocalData: false}
deleteLocalData: false
force: false

Cluster upgrades successfully and no panic/restart is seen in Rancher.

verified on an upgrade from 2.4.8 to 2.5-head 65f3525cd

in v2.4.8 deploy Rancher HA in a rke cluster
add several custom RKE clusters (through API) k8s - 1.18.8
upgrade rancher to 2.5-head
Upgrade k8s version of the cluster to 1.18.9

All clusters are upgraded, and no restart/panic for rancher nodes


The call hierarchy UI should have the ability to filter calls, or apply name filter patterns

Note: This came from VS Code Java extension, referencing the issue here: https://github.com/redhat-developer/vscode-java/issues/2179
When I use Show Call Hierarchy on a property in a Java class object that is frequently used in both the main source code, as well as in the unit tests, the Call Hierarchy list would display a bunch of results. Because, the results include both the main source code and unit tests references, there isn't a way to automatically exclude certain folder directories or paths out from unwanted invocations.
I would like to request for a UI improvement in Visual Studio Code, to allow users apply any one of the following options:

Filter test codes based on folder directory. (e.g., Filtering Java unit test codes or Java source code documents residing under src/test/java when specified)
Apply name filter patterns. (e.g., Filtering reference names that matches the given filter patterns in the reference class names.)
Limit the call depth of a given number. (e.g., Only goes down in depth up to the given number, inclusive.)

This feature is similar to Eclipse IDE's "Filter Calls" dialog:

This is accessed by invoking "Show Call Hierarchy" on a class member, then opening "Filters" in the "More Options" menu in Eclipse IDE.
Currently, we don't have anything similar to this in Visual Studio Code in general (regardless of what language support), where the call hierarchy UI allows the user to filter out what references to look for / ignore.
Environment

Operating System: Ubuntu 20.04.2 LTS
JDK version: AdoptOpenJDK 11  (jdk-11.0.11+9)
Visual Studio Code version:  1.61.2

Current Result
The "Show Call Hierarchy" looks something like this, whenever we attempt to invoke call hierarchy. I'm using Java language support as an example.

Expected Result
Call hierarchy UI with filter options would automatically filter out either all test codes, or applying a name filter pattern to exclude any references containing "test". A mock example should look something like this immediately upon invoking the call hierarchy:

This would be very helpful.  IntelliJ has a similar feature allowing you to filter to just "production" code.  The include and exclude capability that is already within vscode Search would do the trick:

It would also be nice to be able to filter based on a regex for function signatures

Bump on this. Is this being tracked in a newer issue or is it just stale? This feature seems to be widely requested on stack overflow and other help sites. @jrieken

@NeonWizard If you could help give this feature at the top with a thumbs up, it will allow them to prioritize this feature better. The more thumbs up, the more awareness this feature request will get, the more prioritization.

What surprises me is that apparently only 13 people (current count of upvotes) use the Show Hierarchy feature? Because the minute I discovered and used it for the first time today with my Go project, I thought that it'd be great to filter tests out. Hence here I am, after searching to find if someone had already made the suggestion.
Given that it was immediately apparent to me, as first time user, I definitely expected that it would be for other people too, and this was a more popular feature request. Well I guess not :)

@j1elo I'm pretty sure a lot of people uses the Call Hierarchy feature. They probably just don't realize that filtering the method invocations and classes would help them more.

We have very long lists of calls and about 90% of the results are log calls, which make it difficult to find other calls.
So it would be good if there was also the option to exclude results.
I would go one step further and implement sorting in addition to filtering.
There could be the following sorting options:

By call sequence (default)
By name
By type/file extension

When sorting by name or type, the results found would be grouped.

I was looking for this feature today, it makes debugging harder when it shows all the tests in the hierarchy

@VSCodeTriageBot We need this issue to be reassigned, Johannes seems to have gone AWOL

@NeonWizard Johannes isn't technically AWOL. He's still active as a of last week. It's just that this issue is staying in the backlog, and it hasn't been prioritized as much as the rest of the features that are being put into the upcoming monthly milestone goals.

Just mentioning closed version of this so there's a breadcrumb for others who found the other dead issue: https://github.com/microsoft/vscode/issues/205411


Tree Data Selection in [Feature Request] Table

Existing Component
Yes
Component Name
table 中的树形数据
Description
table 中的树形数据如何选中子节点父节点也被选中

I think we need to listen on select-all, which is bad！